Amazon S3

Basically Amazon S3 files behave like local files. There is just on important difference. While thumbnails for local files are generated on demand, we need to calculate the thumbnails for images stored in an Amazon S3 bucked ahead. For this, we use two buckets. One bucket contains the original files. The folder and file structure is the same like for local files. No nesting is supported. Place a folder in the bucket and upload your images in there. Example: event1/myimage.jpg. The second bucket is used to store the generated thumbnails. Browsers will have access there to load the images. You just need to create the bucket, everything else is handled by Event Gallery. Make sure you’re using the same region for both buckets.

Once you uploaded your images to Amazon S3 and configured your buckets in the components options you need to let the database know about your new files. Use the Sync Database button and sync your folders to start the sync process. Once the new files appeared hit the Sync Files button to read in the necessary data. Check the Sync Database process documentation on how to generate thumbnails. You can upload files using Event Gallery, too.

Always keep in mind that reading and writing to an Amazon S3 bucket is not free. You have to pay for each request. So don’t perform a sync or a thumbnail creation if it is not necessary.

[Note]Note

If you encounter any errors, increase your error reporting level. The thumbnail generator will output error messages which might help with invalid Amazon S3 credentials.

Technical Details

The table #__eventgallery_file has two new fields: s3_etag and s3_etag_thumbnails. The field s3_etag contains the s3 hash of the orignal file. It is used to detect image changes. The field s3_etag_thumbnails contains a JSON object. It stores the thumbnail sizes and the hash values of each thumbnail. While creating the thumbnails this data is used to detect thumbnail changes. Once a change of the original file is detected, the field s3_etag_thumbnails is cleared. The next thumbnail creation run will find those thumbnails as missing and will recreate them. Make sure you uncheck the Refresh thumbnail hashes checkbox. Otherwise the s3_etag_thumbnails field will be populated with the hash values of existing thumbnails.

Figure 3.20. Thumbnail Creator

s3 thumbnail creator

Quick Steps to set up AWS S3 for Event Gallery

  • Create two buckets in the region you like. One will contain the original files, the other the resized thumbnails.
  • Change permissions for the bucket with the resized images to public.
  • Create a new user in IAM. This will help to restrict access to the buckets only
  • Create a new policy. You can use the JSON example below. Just replace the names of the buckets.
  • Assign the policy to the user
  • Configure Event Gallery with the user data, region and bucket information.
  • Upload your images in a simple folder structure. No nesting! folder/image.jpg
  • Use the Sync Database process to detect the new files
  • Run the thumbnail creator to generate the thumbnails

Access Policy

You need to specify credentials so Event Gallery can access the S3 buckets you created. The following policy allows to read and write from/in those buckets. You just need to change the name of those buckets to the ones you created.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:ListBucket"
            ],
            "Resource": [
                "arn:aws:s3:::eventgallery-images-test-original",
                "arn:aws:s3:::eventgallery-images-test-resized"
            ]
        },
        {
            "Effect": "Allow",
            "Action": [
                "s3:PutObject",
                "s3:PutObjectAcl",
                "s3:GetObject",
                "s3:DeleteObject"
            ],
            "Resource": [
                "arn:aws:s3:::eventgallery-images-test-original/*",
                "arn:aws:s3:::eventgallery-images-test-resized/*"
            ]
        }
    ]
}

Extended steps to set up Event Gallery for AWS S3

Log into the AWS console and switch to the S3 overview page. There, you need to create two new buckets.

Figure 3.21. Create the bucket for the original files

01 create original bucket

Figure 3.22. Create the bucket for the calculated thumbnails

02 create resized bucket

Figure 3.23. Two new buckets are created

03 buckets created

Once you have the two buckets, you need to change the permissions on the thumbnail/resized bucket. It needs to be public accessible.

Allow ACL for the both buckets.  By default, items in this bucket are not accessible. When Event Gallery creates thumbnails or uploads files, it sets the ACL to make the files/thumbnails public. The buckets need to support this. Allowing the usage of an ACL helps here.

03 buckets allow acl

Figure 3.24. Change bucket permission for the thumbnail bucket

04 bucket permission change

Figure 3.25. Allow pulic access to the thumbnails bucket

05 bucket permission change

Figure 3.26. Confirm permission change

06 bucket permission change

The buckets are ready. To upload images and calculate thumbnails, you need to create a new user. Don’t use your AWS root credentials! For every Event Gallery installation, I recommend creating a new IAM user. This user has only access to the S3 buckets and to nothing else. The needed permissions are granted by an access policy. Both we’ll create in the next steps.

Figure 3.27. Create a new IAM user

07 IAM

Figure 3.28. Allow programmatic access

08 IAM new user

Figure 3.29. Create a new access policy

09 IAM create policy

Figure 3.30. Copy and modify the access policy JSON from above

10 IAM policy json

Figure 3.31. Add a name and description for the new policy

11 IAM policy review

Figure 3.32. Assign the policy to the user

12 IAM user creation policy selection

Figure 3.33. Copy the access credentials for the user

13 IAM user created

Once you have the access credentials, bucket names and the bucket region, you can configure Event Gallery and start using AWS S3 there.

All prices include VAT. The gross price will vary depending on the selected shipping country.