Create Multiple Static Sites In A Single AWS S3 Bucket Served By AWS CloudFront
Tags: devops, aws, webdev, tutorial
Published: 2022-09-07 01:34:55

In this blog post, I'll share how to host multiple static websites in a single S3 bucket, where each site is stored in a single directory (path).

CloudFront will serve the static website, and the trick to fetch the relevant site from the origin, S3 bucket, in this case, relies on a Lambda@Edge function.

## Objectives

- Serve safely behind a [CDN](https://aws.amazon.com/cloudfront/) via [HTTPS](https://aws.amazon.com/certificate-manager/) - [CloudFront + ACM](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/cnames-and-https-requirements.html). 
- Accessing the S3 bucket should be forbidden to anyone except for the serving CloudFront distribution; we'll use [CloudFront OAI](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html) for that
- Store in durable storage, [S3 Standard Durability 99.999999999% (11 9's)](https://aws.amazon.com/s3/faqs/#:~:text=What%20is%20S3%20Standard?), and highly available servers, [S3 Standard Availability 99.99%](https://aws.amazon.com/s3/faqs/#:~:text=What%20is%20S3%20Standard?)

By the end of this blog post, I expect to have two live websites, which are stored on a single S3 bucket, and I want proof.

And of course, I'm too excited to keep to myself, right now I'm inspired by the below song ... So ... Are you re-e-e-e-ady let's go!

[![are-you-ready-lets-go](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jvuoeokqflkqsr03wpz9.png)](https://www.youtube.com/watch?v=xxIsmbVZuSI&ab_channel=EscapetheFate)

> [Escape the Fate - "One For The Money"](https://www.youtube.com/watch?v=xxIsmbVZuSI&ab_channel=EscapetheFate)

## (1/6) HTTPS And DNS Validation

This step is optional if you already own an eligible ACM Certificate in the AWS region **N.Virginia (us-east-1)**.

I own the domain [meirg.co.il](https://meirg.co.il), so to avoid breaking my site, I'm adding another subdomain, and I'll name it `multiple-sites`, which makes my [FQDN](https://en.wikipedia.org/wiki/Fully_qualified_domain_name)

```
multiple-sites.meirg.co.il
```

Additionally, I want to support accessing the site with `www` or any other subdomain (maybe `api`), so I'll add another FQDN

```
*.multiple-sites.meirg.co.il
```

### Create An AWS ACM Certificate

![create-acm-certificate](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/67lpt520spvvbo7fhrvt.png)

### Create DNS Validation Records In DNS 

Now I need to validate the certificate in my DNS management provider; in my case, it's [Cloudflare](https://developers.cloudflare.com/dns/manage-dns-records/how-to/create-dns-records/) (flare, not Front 😉)

Then, I copied the CNAME+Value that was generated in [ACM](https://docs.aws.amazon.com/acm/latest/userguide/dns-validation.html)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qj38m4lwuclvh83fnr62.png)

And then, I created two new DNS records in my DNS management provider ([how to do it in AWS Route53](https://docs.aws.amazon.com/Route53/latest/DeveloperGuide/migrate-dns-domain-in-use.html)). Here's how it looks like in Cloudflare:


![cloudflare-created-validation-records](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/n3b4xsb3aseyi09owbke.png)


**NOTE**: [Cloudflare provides a native CDN](https://www.cloudflare.com/cdn/) solution per DNS address, which is fantastic. As you can see, I turned it off (DNS Only) since it's irrelevant for the DNS validation.


And then ... I wonder if all those "and then" reminded you [Dobby from the Harry Potter movie](https://youtu.be/geUYoyuETBI?t=308) 🤭

### Cloudflare And AWS CAA Issue

I got into big trouble; my certificate failed to validate
![acm-failed-certificates](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/gp52puz4b60tb2htyhjy.png)

Even though this part is not directly related to this blog post, I'm putting it here for future me (or the currently frustrated you who found it). Bottom line, I fixed it by reading [How to Configure a CAA record for AWS](https://docs.aws.amazon.com/acm/latest/userguide/setup-caa.html) and created the DNS records accordingly in Cloudflare.

![cloudflare-caa-records](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lm6t5tv3k8akzphuedcy.png)

### Validated ACM 

Finally, my ACM certificate is eligible, and all DNS records were validated successfully, thanks to the CAA records created above 🙇🏻‍♂️

![acm-validated-certificates](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/zrsq4zpkrugqs2clgsie.png)

## (2/6) Create An S3 Bucket

This part is relatively easy if you're familiar with AWS S3, though I do want to emphasize something -

An AWS S3 bucket is an [object storage](https://cloud.google.com/learn/what-is-object-storage); hence it cannot be served to a browser unless set to [AWS S3 Static Website](https://docs.aws.amazon.com/AmazonS3/latest/userguide/WebsiteHosting.html).

**NOTE**: S3 is an object storage, so [directories (paths) aren't really "storage locations"](https://docs.aws.amazon.com/AmazonS3/latest/userguide/using-folders.html), but more like pointers that state where the data is stored in an organized way, like filesystem storage.

An S3 static website is only available via HTTP. Serving the site via HTTPS requires a CDN, such as AWS CloudFront or Cloudflare. This information is available at [How do I use CloudFront to serve a static website hosted on Amazon S3?
](https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-serve-static-website/)

So why am I double-posting something that is already out there? Because I found it very confusing to realize that the term "static website" and the term "S3 static website option" are different.

By its name, I would expect to enable [AWS S3 Static Website Option](https://docs.aws.amazon.com/AmazonS3/latest/userguide/HostingWebsiteOnS3Setup.html) to make this solution work. It should be **disabled**.

Here's the thing, AWS S3 Static Website doesn't support [CloudFront Origin Access Identity (OAI)](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html). As mentioned in the *Objectives*, the site should be accessed from CloudFront only, which means I need the [OAI](https://aws.amazon.com/premiumsupport/knowledge-center/cloudfront-access-to-amazon-s3/) functionality.

### Recap - AWS Static Website Option

AWS Static Website Option should be **disabled** when served behind a [CloudFront distribution](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-working-with.html).

### CloudFront Invalidation

A [CloudFront invalidation](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Invalidation.html) is the part where you purge the cache after pushing static content HTML/CSS/JS/etc., to the S3 bucket.

Later on, when I push new content, I will invalidate the cache to get immediate results. It's either that or setting the [CloudFront's distribution TTL](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Expiration.html) to a lower number than the default `86400 seconds` (24 hours).

### Create a bucket

The name doesn't matter as long as it's [a valid S3 bucket name](https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html) and that [name is globally available across ALL AWS accounts](https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingBucket.html).

![create-s3-bucket-1-name](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d7k74wlrafypsr41jnbz.png)

As you can see, I chose a logical name for my bucket (luckily, it was available)

```
multiple-sites.meirg.co.il
```

**IMPORTANT**: The S3 bucket name doesn't have to be the same as your DNS records. If the name is taken, add a suffix like `-site` or any other logical suffix that would make sense when you search for the S3 bucket by name. In this solution, The S3 bucket is served behind a CDN (CloudFront), so its name doesn't matter.

Scroll down, keep the **Block all public access** ticked, and set Versioning to **Enabled**.

![create-s3-bucket-2-versioning](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cmbrrl7nvem39jelgnsp.png)

## (3/6) Push Website Content To S3

By "push", I mean upload; I use the verb "push" because I want to keep in mind that an automated CI/CD process should "push" the content instead of "uploading" it manually via S3 like I'm about to do now.

Following the bucket creation, I've created two directories (folders); as promised in the beginning, each directory will serve a different website.

![s3-bucket-push-content-1-directories](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/t3wvr8ldusv84s5aighr.png)

And then (Dobby again), I've created this dummy `index.html` file and made two copies, so I can differentiate between the sites when I check them in my browser.


`site1/index.html`
```HTML
<h1>Welcome to site<u>1</u>!</h1>
```

`site2/index.html`
```HTML
<h1>Welcome to site<u>2</u>!</h1>
```

Moving on to the tricky parts - [Creating a CloudFront distribution](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/distribution-web-creating-console.html) and [Lambda@Edge](https://aws.amazon.com/lambda/edge/)!

## (4/6) Create A CloudFront Distribution

As part of this step, we'll also create (automatically) an Origin Access Identity, enabling access for CloudFront to the S3 bucket; Currently, none is allowed to view the websites.

Once you choose the CloudFront service, you're redirected to the "Global" region, don't let it fool you; it's considered as N.Virginia (EU-east-1), creating the ACM certificate in N.Virginia is essential, as it's the same region where CloudFront is hosted.

The story continues; I've created a CloudFront distribution and searched for my bucket `multiple-sites.meirg.co.il` as the Origin. The name of the origin was automatically changed in the GUI to `multiple-sites.meirg.co.il.s3.eu-west-1.amazonaws.com`.

![cloudfront-create-distribution-1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1a5qm72zslsz9z2o9jqm.png)

To create an OAI, I've clicked the **Create control setting** button, and after reading [Advanced settings for origin access control](Advanced settings for origin access control
), I chose **Always sign origin requests (recommended setting)** since my bucket is **private** (future to be accessible only via CloudFront).

**NOTE**: I also realized that OAI had become legacy, and I should start using the Access Control Setting feature instead.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v1pejwuaccgc5a44x2vd.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k0mbfnxfu2z9388sned7.png)

For cache key and origin requests, I've selected [CORS-S3Origin](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/using-managed-origin-request-policies.html), which is a must; otherwise, you'll get a 502 error due to failure of signing requests (try it).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ut2r8g3jftzmoe6fs5is.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/oste75q4xnayq473af62.png)

I was surprised by the excellent UX (not kidding); After creating the CloudFront distribution, I got the following pop-up.

![cloudfront-create-distribution-4-after-creation](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/d3ftlu8u2iac7iurt9h9.png)

As instructed, I clicked **Copy policy** and the highlighted link **Go to S3 bucket permissions to update policy**, scrolled down a bit to **Bucket Policy**, and clicked **Edit**.

In this screen, I pasted the copied policy, and it got a bit messed up (no idea why), so I made sure the curly braces at the top `{` and in the end `}` are not indented at all, and then I was able to click **Save Changes** (scroll to bottom) without an issue.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/txhb45nfah1lrctzcpxy.png)

Here's what it looks like after clicking the **Save Changes** button.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7rn8mafonfnzbf5e8eus.png)

## Recap So Far

- Created an ACM certificate in N.Virginia (us-east-1) with eligible DNS records
  - `multiple-sites.meirg.co.il`
  - `*.multiple-sites.meirg.co.il`
- Created an S3 bucket in Ireland (EU-west-1) and named it `multiple-sites.meirg.co.il` 
- Pushed two websites to the S3 bucket
  - `site1/index.html`
  - `site2/index.html`
- Created a CloudFront distribution and updated the S3 bucket with the generated Access Control Bucket Policy

## (5/6) Map DNS record to CloudFront

For testing purposes, I've created another `index.html` file; Remember that I set a [Default Root Object](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/DefaultRootObject.html) during the CloudFront distribution creation? That is it.

```
<h1>Welcome to root site!</h1>
```

The new `index.html` file is pushed to the root directory `/` of the bucket; Before moving on to the "site per directory solution with Lambda@Edge" I want to make sure that everything is in place and that [https://multiple-sites.meirg.co.il](https://multiple-sites.meirg.co.il) works as expected.

So now my S3 bucket looks like this:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kexz8d3tdxj1sdnbwhkj.png)

- FQDN Served by CloudFront (Not Reached) - [https://multiple-sites.meirg.co.il](https://multiple-sites.meirg.co.il)
- S3 Direct Access Link (Forbidden) - [https://s3.eu-west-1.amazonaws.com/multiple-sites.meirg.co.il/index.html](https://s3.eu-west-1.amazonaws.com/multiple-sites.meirg.co.il/index.html)

The FQDN is still unavailable as I haven't added the DNS record in my DNS Management Provider (Cloudflare); As of now, none knows that my website `multiple-sites.meirg.co.il` should point to the created CloudFront distribution.

Copied the CNAME of the CloudFront distribution

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6n0o54fpq487wa0ml4ny.png)

And created a new DNS record for that CloudFront distribution CNAME; Since I'm already using CloudFront as a CDN, I prefer to disable Cloudflare's proxy feature, as It'll be CDN to CDN, which sounds like a bad idea.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/202tep0uts9lhbr2109y.png)

Testing the root site [https://multiple-sites.meirg.co.il](multiple-sites.meirg.co.il) ...

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9yvytssfo03x27dd54vw.png)

It worked 🥳 but that's not why we're here for! Moving on to the multiple directories magic. Why magic, you ask? The nested paths `/site1`,`/site2` are restricted/forbidden; here's how accessing `/site1` looks like at the time of writing this blog post:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l62u8yo7r6kebsrf4zqj.png)

## (6/6) Create A Lambda@Edge Function

Writing code is excellent; copying from someone else and understanding what it does is even better. After Googling a bit, I stumbled on [Implementing Default Directory Indexes in Amazon S3-backed Amazon CloudFront Origins Using Lambda@Edge
](https://aws.amazon.com/blogs/compute/implementing-default-directory-indexes-in-amazon-s3-backed-amazon-cloudfront-origins-using-lambdaedge/) contains the exact code that I need.

I copied the snippet from the mentioned site and removed the `console.log`; I'll add them back if I need to debug.

```javascript
'use strict';
exports.handler = (event, context, callback) => {
    
    // Extract the request from the CloudFront event that is sent to Lambda@Edge 
    var request = event.Records[0].cf.request;

    // Extract the URI from the request
    var olduri = request.URI;

    // Match any '/' that occurs at the end of a URI. Replace it with a default index
    var newuri = older.replace(/\/$/, '\/index.html');

    // Replace the received URI with the URI that includes the index page
    request.URI = newuri;
    
    // Return to CloudFront
    return callback(null, request);

};
```

### How Does The Lambda@Edge Work?

By default, if an origin request URI doesn't contain file path, like navigating `https://multiple-sites.meirg.co.il/site1/`; An error will be raised since there's no default setting for sub-directories to use the hosted `site1/index.html`.

So why does it work when I navigate the root path of the bucket `https://multiple-sites.meirg.co.il/`? Because I've set the **Default Root Object** to `index.html` in the CloudFront distribution. That "fixes" the issue for a single website hosted at the root of an S3 bucket.

When it comes to multiple websites stored in the same S3 bucket, we need a mediator/proxy that will convert the origin request URI from `/` to `/index.html`, which is precisely what the above-copied code snippet does.

### Create The Lambda@Edge Function

Firstly, in which region should I create the Lambda@Edge Function? .... Can you guess? ......... N.Virginia (us-east-1)! Reminding you that all CloudFront-related resources need to be located in N.Virginia (us-east-1).

Go ahead and create a Lambda Function in N.Virginia, and keep the default settings "Author from scratch > Node 16.x", reminding you that the Lambda Function's code snippet was in JavaScript.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/26m2qfbeulxfc05ywfky.png)


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/loi01am6entejjvmjz6y.png)


#### Publish new version

This step is mandatory as [Lambda@Edge functions are referenced with versions](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-edit-function.html), unlike normal Lambda Functions can be referenced using the `$LATEST` alias.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x0ifs09kqvstvpumpovc.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yzki3k11zfn66vmj8naf.png)

#### Deploy Lambda@Edge To The CloudFront Distribution

Navigate back to the function and
Deploy to Lambda@Edge!

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/v7q7p9a54haxe04u42fd.png)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/dtyrynhldynknnfwqs57.png)

I clicked **Deploy** (with excitement), and then I got this pop-up message:

> Correct the errors above and try again.
Your function's execution role must be assumable by the edgelambda.amazonaws.com service principal.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hm6ltihq1j0l1gxoiscy.png)

Hold on a sec; I need to add a Trust Relationship, so CloudFront is allowed to execute the function. Googling a bit got me here - [Setting IAM permissions and roles for Lambda@Edge](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/lambda-edge-permissions.html); the copying motive continues as I copied the IAM policy from that page

```json
{
   "Version": "2012-10-17",
   "Statement": [
      {
         "Effect": "Allow",
         "Principal": {
            "Service": [
               "lambda.amazonaws.com",
               "edgelambda.amazonaws.com"
            ]
         },
         "Action": "sts:AssumeRole"
      }
   ]
}
```

After that, I edited the Lambda Function's IAM role Trust Relationship.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ah2hoc7oemigd3w9777d.png)

Let's try to deploy the Lambda Function again ... And of course, another error ...

> Cannot read properties of undefined (reading 'startsWith')

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y8nih23yibszveaj8gyi.png)

Oh well, I refreshed the Lambda Function's page ... I Tried to deploy again ... Finally, deployed! 

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/loi4z6ob9wb2g1x0pj1l.png)


## Results

I'm not sure if I did it or if it happened automatically. Still, I think it's important to document it - initially, I got to the 403 Forbidden page when trying to access [https://multiple-sites.meirg.co.il/site1/](https://multiple-sites.meirg.co.il/site1/), so I checked the CloudFront distribution and found out that the S3 origin is set to **Public**, and I have no idea how or why that happened. So I changed it back to `Origin access control settings (recommended)`, and everything works as expected.

[![site-1](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2mgl38g8mhsafnzzc3tm.png)](https://multiple-sites.meirg.co.il/site1/)

[![site-2](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1o5om517aigelkzg6n8x.png)](https://multiple-sites.meirg.co.il/site2/)

## Final Words

Initially, I wanted to use an Origin Access Identity (OAI) to restrict to CloudFront only. Then I discovered that OAI is considered legacy, so I proceeded with Access Control, and I'll probably [migrate all my OAI resources to Access Control](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html#migrate-from-oai-to-oac).

I left out (for now) the "S3 redirect from www" part because this blog post is already overkilled.