Transfer media between Evolphin and another AWS account efficiently

By /

Problem

I want to transfer large media files between Amazon Simple Storage Service (Amazon S3) bucket in a different AWS account than Evolphin. Also, I want to be sure that my account doesn’t get charged for these transfers. How can I do that?

Solution

  1. As long as the S3 buckets in your AWS account and Evolphin are in the same AWS region, transfer will not incur any egress charges. Please confirm with your AWS admin that the S3 buckets are created in the correct region.
  2. Ensure S3 Bucket Policy is setup on your account to allow Evolphin AWS IAM API user account to programtically read or write your bucket. This avoids downloading the file into a temporary EBS storage at Evolphin end and upload from EBS disk to the destination bucket.

Sending files to your AWS account from Evolphin

In Evolphin Account

Get the Amazon Resource Name (ARN) of IAM User in Evolphin Account A that would be granted permissions to write into your AWS Account B’s bucket. For e.g “arn:aws:iam::N..:user/EvolphinYourCompanyExternalS3Writer”. If you are hosting with Evolphin, you can ask Evolphin Managed Admin to send a user ARN.

In your AWS Account

Attach an S3 bucket policy to the destination bucket in your AWS Account B that grants write access to Evolphin IAM user such as the ARN below:

{
    "Version": "2012-10-17",
    "Id": "Policy1591816311368",
    "Statement": [
        {
            "Sid": "Stmt1591816290560",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::1111111111:user/EvolphinYourCompanyExternalS3Writer"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::test-deliver-to",
                "arn:aws:s3:::test-deliver-to/*"
            ]
        }
    ]
}

As shown above, make sure:

  • replace “arn:aws:iam::1111111111:user/EvolphinYourCompanyExternalS3Writer” with the IAM user that Evolphin AWS S3 archive hub is using
  • replace the array under Resource with your S3 bucket while ensuring you put two entries:
    • bucket
    • bucket/*

That’s it, now Evolphin should be able to copy large media files from Evolphin S3 account to your AWS account efficiently!

Ingesting or Uploading files from your AWS account into Evolphin

In Evolphin Account

Get the Amazon Resource Name (ARN) of IAM User in Evolphin Account A that would be granted permissions to read from your AWS Account B’s bucket. For e.g “arn:aws:iam::N..:user/EvolphinYourCompanyExternalS3Reader”. If you are hosting with Evolphin, you can ask Evolphin Managed Admin to send a user ARN.

In your AWS Account

Attach an S3 bucket policy to the source bucket in your AWS Account B that grants read & write access to Evolphin IAM user such as the ARN below:

{
    "Version": "2012-10-17",
    "Id": "Policy1591816311368",
    "Statement": [
        {
            "Sid": "Stmt1591816290560",
            "Effect": "Allow",
            "Principal": {
                "AWS": "arn:aws:iam::1111111111:user/EvolphinYourCompanyExternalS3Reader"
            },
            "Action": "s3:*",
            "Resource": [
                "arn:aws:s3:::test-ingest-from",
                "arn:aws:s3:::test-ingest-fromo/*"
            ]
        }
    ]
}

Please narrow the access to just read actions to restrict Evolphin’s user ability to only read files.

Tips for creating S3 Bucket Policy

While you can modify the above policy JSON file in a text editor you can also use the AWS S3 bucket policy editor to do this without any syntax error:

After you click Add Statement, check the S3 ARNs are correct:

Then simply click on generate policy and stick the policy JSON into your S3 Bucket Policy UI: