Request Strapi's REST API behind a Content Delivery Network (CDN)

This tutorial will focus on the problem of network latency when requesting large numbers of media assets from a REST API and, in particular, how a Content Delivery Network (CDN) can help to solve this issue.

Note: You can also deploy your websites or applications to Strapi Cloud to get a CDN, database and more out of the box.

Prerequisites

To follow this tutorial, you will need the following:

• Package manager (yarn or npm)
• Node.js (v16 or v18).
• Code editor (Visual Studio code, Sublime)
• Basic knowledge of Javascript
• Basic knowledge of Strapi
• AWS account
• Postman

Introduction

When building applications that store lots of media assets, such as images, videos, and audio files, we need to be mindful of data retrieval speeds as our applications grow.

Imagine, for instance, an e-commerce store that's hosted on a single server. This server handles all of the web traffic with a Rest API, which includes serving product images and user data. At first, the performance was good as it had a relatively small number of users. Fast forward a year, and the application now has 100,000 users. So, the single server starts experiencing a high load, leading to slower response times across the board. Depending on where the server is hosted and where the users are requesting content, there will also be a geographical latency.

Content Delivery Network (CDN)

One solution to this problem is a Content Delivery Network (CDN). This would enable the application to cache images, videos, and other static assets at multiple locations worldwide. This just means that the primary server will handle fewer requests for static content, and there's no geographical latency, seeing as static assets are cached nearer to the user's location. Thus, the server can now focus on other processing.

Ultimately, this would allow the website to handle higher traffic volumes without compromising the user experience.

Understanding Strapi and Media Asset Management

Strapi or Strapi CMS is an open-source headless content management system allowing us to create APIs quickly. The headless architecture separates the content management and content delivery layers. This feature enables us to consume content via API and use whatever front-end technology we prefer.

How does Strapi store media assets?

Strapi stores any media assets you upload to the file system of the server where Strapi is running. When you upload a file through the Strapi admin panel, it will be saved to a specified directory on the server; this would mean that on a server reset, any stored assets would be lost.

However, Strapi also supports integrations with cloud storage providers such as Cloudinary, Google Cloud, Azure, and AWS.

We will focus on uploading images to AWS S3, a widely used cloud storage service where we can upload various types of files. In addition to being performant and scalable, it is also very secure and reliable.

Strapi and AWS S3

Our infrastructure so far is a single Strapi REST API connected to an AWS S3 bucket. When we upload images to our server, they are uploaded to the S3 bucket, and the URL stored in our database for that image is the public URL provided by S3 for accessing the asset.

So now, when the front-end requests the asset, Strapi will send the S3 bucket URL as part of the response, meaning the weight of the processing will be put on AWS S3, which is pretty fast.

The problem will come when we have more users globally. S3 buckets are in a specific location (depending on where you set them up). Let's say you set one up in Europe, but someone requests an asset, and they are located in Asia. This is where the geographical latency will come in as they are further away from the source, and as such, it takes longer to serve the assets.

Cloudfront

This is where Cloudfront comes in, a CDN available on AWS. It caches static content in multiple locations around the world, meaning users can download assets from a server close to their location, reducing latency. CDNs are also designed and optimized to serve static content and handle large volumes of traffic.

Cloudfront features

• Global Edge Network: CloudFront has a network of edge locations worldwide, ensuring low-latency delivery to users.
• Cache-Control: Developers can fine-tune caching behavior using cache policies, origin request policies, and control over Time-to-Live (TTL) settings for different content types.
• Real-Time Monitoring: CloudFront provides detailed metrics and logs via Amazon CloudWatch, allowing users to monitor performance, track usage, and troubleshoot issues.
• Origin Servers: It supports multiple origin types, including S3 buckets, HTTP servers, and other AWS services.

Set Up AWS S3 and CloudFront

Now that we understand the problem and know how to solve it, let's examine how to implement it practically.

Creating an IAM user in AWS

First, we must create an IAM user for our AWS account. Using your root account credentials is generally considered bad practice, so we will create another user with just the right amount of permissions we need to access the S3 bucket and nothing else.

Navigate to AWS in your browser and log in to your account. Once logged in, type IAM into the search bar at the top of the page and select IAM from the dropdown.

This will navigate you to the dashboard. Now select users from the left-hand side menu, which will take you to the user dashboard, and select Create user on the right-hand side.

Give your user a name and click next.

Now, we will want to add our user to a group that specifies each user's permissions. That way, if our team grows, we can add users to this group quickly to give them the same permissions. So click on Add user to group and then Create group.

Give the group a name, check AmazonS3FullAccess, which will give the users in this group full permission to update and delete objects from the S3 bucket, and then click Create user group.

Once created, check the group you just created to add the user to it, and then click next; the summary should look like this.

We have a user called strapi-iam-user, which is part of the group strapi-s3, giving full access permission to the S3 service; now, create the user.

Now click on the user and navigate to Security credentials.

Scroll down and create an access key for this user. Note the secret and access key; we will use this later to enable the AWS SDK. Also,  click on your user from the table and note the IAM ARN number, which we will use later.

Create an S3 Bucket

Now that we have an IAM user, we will need to create an S3 bucket. To do so, type s3 into the search bar at the top of the page and select it from the dropdown menu.

This will bring you to the Amazon S3 dashboard.

Click Create bucket and give your bucket a name.

Under Object Ownership, tick the box to enable ACLs, which will allow our plugin from the Strapi backend to work.

Let's make the bucket public by unticking Block all public access and keeping the bottom two options ticked.

And tick the box to acknowledge that you are making your bucket public.

Now scroll down and click on Create bucket. This will take you back to the dashboard, and you will see your newly created bucket in the table.

Create a Cloudfront Distribution

Now, let's navigate to the Cloudfront console by typing cloudfront into the search bar at the top and clicking it from the dropdown menu.

Click on Create a CloudFront distribution.

Next, choose the origin domain, the S3 bucket you just created.

As our bucket's access is public, we can keep the origin access public. Under Web Application Firewall, select Do not enable security protections. Scroll down and enter index.html for the Default root object.

All other options can be left as they are. Now scroll down and click Create distribution.

This will bring you back to the dashboard, where you will be able to see your CDN in the state of Deploying. Wait a few minutes for that to finish.

Now, let's test that our CDN is working correctly. First, copy the Distribution domain name from the dashboard, and then navigate back to our S3 bucket and upload an image to it.

Click on Upload from the table.

On the next page, click Add files and add a random image if you have one. If not, hit up Shutterstock or something similar.

Scroll down and click the dropdown for permissions; for now, let's make this public just to demonstrate the CDN working in our browser; later, we won't need to do this as images will be uploaded and accessed with our credentials.

Then click Upload; if you copy your CloudFront URL and the file name you just uploaded, you should get the image displayed. For example, your URL should look something like this: https://d2urqgzsm6mm3p.cloudfront.net/polynesia.jpg

That's it now: CloudFront is set up to cache and serve files from our S3 bucket.

Integrating S3 and CloudFront with Strapi

We will be using a plugin called @strapi/provider-upload-aws-s3 to connect AWS S3 and Cloudfront with our Strapi backend.

Create a Strapi Project

First, let's create a new Strapi instance and run it locally. Run the command below in your terminal.

npx create-strapi-app@latest strapi-cdn-tutorial

That will automatically build and run your project for you. When it navigates to the register-admin screen, enter your details to create an admin user. This will land you on the Strapi admin dashboard.

Install AWS S3 Plugin

In your terminal, navigate to the root of your Strapi project and install the AWS S3 plugin from the Strapi market by running the below command

yarn add @strapi/provider-upload-aws-s3

In your project's .env file, add the following environment variables underneath JWT_SECRET.

1AWS_KEY_ID=<Access Key>
2AWS_SECRET=<Secret key>
3AWS_REGION=<the region you deployed bucket>
4AWS_BUCKET=<your bucket name>
5CDN_URL=<your CloudFront url>

The AWS_KEY_ID and AWS_SECRET are the credentials you noted down when creating the IAM user earlier.

Now open your project and paste the code below into your config>plugins.js file.

1module.exports = ({ env }) => ({
2  // ...
3  upload: {
4    config: {
5      provider: 'aws-s3',
6      providerOptions: {
7        baseUrl: env('CDN_URL'),
8        rootPath: "",
9        s3Options: {
10          credentials: {
11            accessKeyId: env('AWS_KEY_ID'),
12            secretAccessKey: env('AWS_SECRET'),
13          },
14          region: env('AWS_REGION'),
15          params: {
16            Bucket: env('AWS_BUCKET'),
17          },
18        },
19      },
20      actionOptions: {
21        upload: {},
22        uploadStream: {},
23        delete: {},
24      },
25    },
26  },
27  // ...
28});

This code targets Strapi's default upload feature. We are using the provider we installed and passing our AWS credentials, CDN URL, Bucket details, etc., through to that provider.

Create a Collection Type

Let's test that out to make sure everything is running smoothly. Restart the project so that the environment variables will be included.

In the Strapi dashboard, create a new collection type called photo.

Give it a text field called title.

Then, give it a media field called image.

Now click finish and save in the top right-hand corner to restart the server.

Create Entries

Let's create some entries under that collection type, navigate to the content manager, create an entry by entering a name, and then click to upload an asset.

Here, click "add more assets" and pick from your desktop.

Click "upload 1 asset to the library" then click "finish" on the next page.

If you navigate back to your S3 bucket, you should be able to see your newly uploaded image asset.

Now, to check that it's requesting the asset through our CDN, save and publish it, then select it from the table.

If you click the link to your image and paste it into your browser, you will see that it requests the asset through our Cloudfront CDN.

Solve S3 Thumbnail Issue with Strapi

You may have noticed that the thumbnails in the Strapi backend aren't showing the images we're uploading.

We must add the S3 and CDN URLs to the authorized sources in our strapi:security middleware config.

So under config > middlewares.js add the following code:

1module.exports = ({ env }) => [
2  "strapi::errors",
3  {
4    name: "strapi::security",
5    config: {
6      contentSecurityPolicy: {
7        useDefaults: true,
8        directives: {
9          "connect-src": ["'self'", "https:"],
10          "img-src": [
11            "'self'",
12            "data:",
13            "blob:",
14            "dl.airtable.com",
15            `https://${env("AWS_BUCKET")}.s3.${env(
16              "AWS_REGION"
17            )}.amazonaws.com/`,
18            env("CDN_URL"),
19          ],
20          "media-src": [
21            "'self'",
22            "data:",
23            "blob:",
24            "dl.airtable.com",
25            `https://${env("AWS_BUCKET")}.s3.${env(
26              "AWS_REGION"
27            )}.amazonaws.com/`,
28            env("CDN_URL"),
29          ],
30          upgradeInsecureRequests: null,
31        },
32      },
33    },
34  },
35  "strapi::cors",
36  "strapi::poweredBy",
37  "strapi::logger",
38  "strapi::query",
39  "strapi::body",
40  "strapi::session",
41  "strapi::favicon",
42  "strapi::public",
43];

Restart the project, and now, when you visit the admin dashboard, the media assets should be showing a preview like so:

Create a Custom AWS S3 Strapi Upload Provider

You may want more control over the implementation when uploading to AWS. For instance, this feature may be essential to you, and if AWS updates its SDK, this plugin may cease to work. Let's look at coding our local provider so you have complete control over the implementation and can make changes to it quickly if required.

First, create a providers folder in the root of the application, and then create a directory for our provider called strapi-provider-upload-cdn. Change into this directory and run the below command to initialize:

yarn init

Give it the name strapi-provider-upload-cdn and make sure the main entry point is index.js.

Now install the AWS SDK by running the below command:

yarn add @aws-sdk/client-s3

Create the index.js file and paste the following code inside:

1const {
2  S3Client,
3  PutObjectCommand,
4  DeleteObjectCommand,
5} = require("@aws-sdk/client-s3");
6
7module.exports = {
8  init: (config) => {
9    const S3 = new S3Client(config);
10
11    const upload = (file, customParams = {}) => {
12      const path = file.path ? `${file.path}/` : "";
13      const Key = `${path}${file.hash}${file.ext}`;
14
15      const uploadCommand = new PutObjectCommand({
16        Bucket: config.bucket,
17        Key,
18        Body: file.stream || Buffer.from(file.buffer, "binary"),
19        ACL: "public-read",
20        ContentType: file.mime,
21        ...customParams,
22      });
23
24      return new Promise((resolve, reject) => {
25        S3.send(uploadCommand)
26          .then(() => {
27            if (config.cdn) {
28              file.url = `${config.cdn}/${Key}`;
29            } else {
30              file.url = `https://${config.bucket}.s3.${config.region}.amazonaws.com/${Key}`;
31            }
32
33            resolve(file);
34          })
35          .catch((err) => reject(err));
36      });
37    };
38
39    return {
40      uploadStream(file, customParams = {}) {
41        return upload(file, customParams);
42      },
43      upload(file, customParams = {}) {
44        return upload(file, customParams);
45      },
46      delete(file, customParams = {}) {
47        return new Promise((resolve, reject) => {
48          const path = file.path ? `${file.path}/` : "";
49
50          S3.send(
51            new DeleteObjectCommand({
52              Key: `${path}${file.hash}${file.ext}`,
53              Bucket: config.bucket,
54              ...customParams,
55            })
56          )
57            .then((data) => resolve(data))
58            .catch((err) => reject(err));
59        });
60      },
61    };
62  },
63};

Here, we are just defining a module for Strapi to interact with AWS S3. It provides functions to upload and delete files from an S3 bucket. We pass the details of our CDN and bucket through with the config argument.

Update config > plugins.js to use our custom provider by copying the following code there:

1module.exports = ({ env }) => ({
2  upload: {
3    config: {
4      provider: "strapi-provider-upload-cdn",
5      providerOptions: {
6        region: env("AWS_REGION"),
7        bucket: env("AWS_BUCKET"),
8        cdn: env("CDN_URL"),
9        credentials: {
10          accessKeyId: env("AWS_KEY_ID"),
11          secretAccessKey: env("AWS_SECRET"),
12        },
13      },
14      actionOptions: {
15        upload: {},
16        uploadStream: {},
17        delete: {},
18      },
19    },
20  },
21});

In the root of your application, uninstall the official S3 plugin by running the following command.

yarn remove @strapi/provider-upload-aws-s3

And add the following line to the package.json file under dependencies:

1"strapi-provider-upload-cdn": "file:providers/strapi-provider-upload-cdn",

This is linking our local code as a dependency.

If you run the command below in the terminal, it will install our custom provider as a dependency, which can then be used from the plugins.js file.

yarn install

You can now check that the custom provider is working by uploading some images from the dashboard and then checking the S3 bucket to ensure they were uploaded.

You can check if deletion works correctly by accessing the Strapi dashboard media assets, clicking the assets you want to delete, and pressing the delete button.

When accessing your S3 bucket, the object should have been deleted, as shown below.

Requesting Assets from the Strapi REST API

Now that everything is running let's request an API to retrieve the asset from Strapi's REST API.

First, ensure you have uploaded some photos so we can access them.

By default, Strapi requires authentication to query our API and receive information, but that is outside the scope of this tutorial. Instead, we will make our API publicly accessible. We can find more about authentication and REST API in this blog post: Guide on authenticating requests with the REST API.

Set API Permissions

From the left sidebar, click on Settings. Again, on the left panel under USERS & PERMISSIONS PLUGIN, click on Roles, then click on Public from the table on the right. Now scroll down, click on Photo, tick Select all, and save at the top right to allow the user to access information without authentication.

Depending on your version of the application, open Postman either in your browser or on your desktop.

Create a new workspace, a new collection, and finally, create a new request:

Now enter the following URL in the input at the top: http://localhost:1337/api/photos?populate=* This is hitting the GET request that Strapi automatically generated for us, and we are populating the media asset (in our case, the image).

Click send to get a 200 success response and the JSON body. If you scroll down inside the image, the data should be populated, and you will be able to see the Cloudfront URL for the asset.

Conclusion

Throughout this tutorial, we have learned about CDN, Strapi media asset management, CloudFront, how to set up AWS S3 and CloudFront, integrate S3 and CloudFront with Strapi, create a custom AWS S3 Strapi upload provider plugin, and request assets from the Strapi REST API.

As you can see, the decision to integrate S3 and Cloudfront with Strapi offers significant benefits, such as enhanced performance, efficient content delivery, scalability, and improved security, reinforcing your confidence in this approach.

By setting up these services with AWS, we ensure fast, reliable access to our static assets and media, with Strapi reducing latency for our end users.

For future improvements, consider implementing advanced caching strategies, monitoring tools, and scaling techniques such as load balancing to accommodate more extensive applications.

For a Fully-managed Cloud Hosting for your Strapi Project, try out Strapi Cloud.

Additional Resources

• Github link to the complete code.
• Strapi's REST API documentation