S3

Moving a bunch of files using cli

We want to move all the files under …/ugc/photos to be under …/ugc.old/photos/:

aws s3 mv s3://bucketname/cache/ugc/photos/ s3://bucketname/cache/ugc.old/photos/ --recursive

Access control

• How S3 evaluates access control

• Guidelines for Using the Available Access Policy Options

“The only recommended use case for the bucket ACL is to grant write permission to the Amazon S3 Log Delivery group”…

“In general, you can use either a user policy or a bucket policy to manage permissions.”

Here’s a bucket policy to grant some IAM user complete access to a bucket:

{
    "Statement": [
        {
          "Sid":"PublicReadForGetBucketObjects",
          "Effect":"Allow",
          "Principal": {
                "AWS": "*"
             },
          "Action":["s3:GetObject"],
          "Resource":["arn:aws:s3:::BUCKET-NAME/*"
          ]
        },
        {
            "Action": "s3:*",
            "Effect": "Allow",
            "Resource": [
                "arn:aws:s3:::BUCKET-NAME",
                "arn:aws:s3:::BUCKET-NAME/*"
            ],
            "Principal": {
                "AWS": [
                    "USER-ARN"
                ]
            }
        }
    ]
}

What about read-only access? Let’s see…

seems like s3auth.com used this example:

{
  "Statement": [
    {
      "Effect": "Allow",
      "Action": ["s3:GetObject", "s3:GetBucketWebsite"],
      "Resource": [
        "arn:aws:s3:::bucket-1.example.com/*",
        "arn:aws:s3:::bucket-2.example.com/*"
      ]
    }
  ]
}

Updating metadata to improve response headers for caching

Install s3cmd, then do it like this:

s3cmd --recursive modify \
    --add-header="Expires: Thu, 31 Dec 2099 20:00:00 GMT" \
    --add-header="Cache-Control: max-age=94608000" \
    s3://caktus-website-production-2015/media/community_logos

You can use s3cmd ls to get a list of the buckets you can access.