Serverless AWS With Monorepos
Tags: monorepo, serverless, aws, cicd
Published: 2021-11-30 14:00:49

Learn how to build and deploy a face-recognition application using serverless functions in AWS. Along the way, we’ll discover the Serverless Application Model, how to test Lambdas, and optimize monorepo CI/CDs.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ii6vchutyovl13gc7j61.png)

## Deploying a serverless application

We'll be deploying a face-indexing application built with a serverless backend that consumes AWS services. The components we need are:

-   **AWS Lambda**: runs serverless functions that process and index faces.
-   **API Gateway**: presents a single endpoint for the Lambda functions.
-   **Rekognition**: does all the face recognition.
-   **DynamoDB**: stores face metadata.
-   **Cloudfront**: is a global, low-latency CDN to speed up website load times.
-   **S3**: stores image files and hosts the frontend code.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0wnql2j21z5kpo2q00d0.png)

How does the application work? After populating the database with faces processed with Rekognition, a user may submit an image to look up its name. If the face has already been scanned, his or her name is returned. 

The complete code for the project is located here:

-   [aws-samples/serverless-webapp-mono-repo-ci-cd-java](https://github.com/aws-samples/serverless-webapp-mono-repo-ci-cd-java)

The demo is a [monorepo](https://semaphoreci.com/blog/what-is-monorepo), a repository holding several projects. Monorepos increase transparency across the team and help us keep code consistent. They can, however, present some scalability issues but they shouldn’t affect us as long as we use Semaphore [monorepo workflows](https://docs.semaphoreci.com/essentials/building-monorepo-projects/).

Shall we start?

## Preparations

To complete the tutorial, you will need:

-   An AWS account and IAM user with administrative privileges.
-   The [AWS CLI](https://aws.amazon.com/cli/) and [SAM CLI](https://aws.amazon.com/serverless/sam/) installed.
-   A free [Semaphore](https://semaphoreci.com) account.
-   [Node](https://nodejs.org/en/), and either [Python 3](https://www.python.org/), or Java and [Maven](https://maven.apache.org/).

First, ensure you’ve run `aws configure` to connect your machine with the AWS account.

``` bash
$ aws configure

AWS Access Key ID: <Type Your Access Key>
AWS Secret Access Key: <Type Your Secret Key>
```

While you’re setting things up, also head to Semaphore and create a [secret](https://docs.semaphoreci.com/essentials/using-secrets/) with the AWS access key. Semaphore will need this information to deploy on your behalf. Be sure to check the [guided tour](https://docs.semaphoreci.com/guided-tour/getting-started/) if this is your first time using Semaphore.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g1ioa1s5oecjkrj0sn7q.png)

Finally, pick a region for the deployment. All deployed functions and services should run in the same region when possible. In this tutorial, we’re using `eu-west-1` (Ireland).

## Building and deploying serverless Lambdas

AWS popularized serverless functions with their [Lambda](https://aws.amazon.com/lambda/) service back in 2014. Serverless keep us focused solely on the code without worrying about servers, infrastructure, or containers. There is no maintenance and we enjoy reduced runtime costs (at least at the beginning). 

The flip side is that we’re limited to the [supported languages](https://aws.amazon.com/lambda/faqs/) and must work within the [runtime limits](https://docs.aws.amazon.com/lambda/latest/dg/gettingstarted-limits.html) imposed by the platform. Also, their event-driven nature force us to rethink how applications are built. 

Amazon published [SAM](https://aws.amazon.com/serverless/sam/) (Serverless Application Model) as a way of standardizing and easing Lambda development. SAM is an open-source framework that provides shorthand expressions for interacting with the AWS ecosystem. Not only can it deploy serverless functions, but it can also test them locally, debug them while running, access remote logs, and even help us perform [canary deployments](https://semaphoreci.com/blog/what-is-canary-deployment).

Let’s see how it works.

## Deploying the backend

We’ll start with the backend. The demo includes two alternatives for this; we have serverless functions in Java and Python. Both implement the same entrypoints:

-   **Upload**: adds a new face to the database.
-   **Recognize**: scans a face and looks it up on the database.

We’ll work with the Java version. I’ll leave the other for you to figure out.

**Build the backend**. Run `sam build` inside the backend folder in the monorepo.

``` bash
$ cd java-app-backend/
$ sam build
```

The build artifacts are stored in the `.aws-sam` folder, so you may want to gitignore it.

The [template](https://github.com/aws-samples/serverless-webapp-mono-repo-ci-cd-java/blob/master/java-app-backend/template.yaml) file describes everything needed to run the functions in AWS: the API paths to expose, the permissions required, and what services it depends on.

**Test serverless functions**. SAM uses a Docker-based testing environment for rapid development. Try running `sam local start-api` to start the development environment. SAM creates a local HTTP server that hosts all your functions.

Alternatively, you can run `sam local start-lambda` coupled with `sam local invoke` to invoke a specific serverless function. In most cases, you’ll need to provide an event payload during the invocation, which you can generate with `sam local generate-event`.

For examples of how it works, check out the [developer guide](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/serverless-getting-started-hello-world.html#serverless-getting-started-hello-world-test-locally).

**Deploy the backend**. SAM packages and uploads the code to S3. Then, it creates the AWS Lambda and API Gateway definitions so the functions can run on demand. Internally, SAM generates [CloudFormation templates](https://aws.amazon.com/cloudformation/) to provision everything.

To begin deployment, run the following command.

``` bash
$ sam deploy --guided \
    --stack-name serverless-web-application-java-backend \
    --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAM
```

The `--guided` option will walk you through all the steps. With `--capabilities` you’re granting extra permissions to CloudFormation, specifically to change IAM permissions and expand macros in the template.

When asked *“X may not have authorization defined, Is this okay?* type `Y`. This is SAM’s roundabout way of warning you that the function will be publicly-accessible.

In some cases, SAM may fail to create the S3 bucket (the error is that it can’t find bucket X). In that case, you’ll need to manually create it:

``` bash
$ aws s3api create-bucket --bucket BUCKET_NAME --region eu-west-1
```

If you run into any errors, You can restart the deployment process from scratch by deleting the `samconfig.toml` file and rerunning `sam deploy`.

The deployment will take a few minutes. Once done, run the following command and check the `id` property the function has been created. Take note of it as we'll need it later on.

``` bash
$ aws apigateway get-rest-apis
{
    "items": [
        {
            "id": "iaadyd31lk", // <-- this is the unique id of the API
            "name": "serverless-web-application-java-backend",
            ...
```

## Backend build stage

Commit all changes made so far into the repository as we’ll set up [continuous deployment](https://semaphoreci.com/cicd) with Semaphore next.

Before continuing, add your repository to Semaphore and complete the basic project setup. Once that's done, **create a block with a single job** in the [build stage](https://semaphoreci.com/blog/build-stage) for the Java backend.

Take a look at these commands:

``` bash
sem-version java 8
wget https://github.com/aws/aws-sam-cli/releases/latest/download/aws-sam-cli-linux-x86_64.zip
unzip aws-sam-cli-linux-x86_64.zip -d sam-installation
sudo ./sam-installation/install
checkout
cd java-app-backend
sam build
cache store build-java-app-backend-$SEMAPHORE_GIT_BRANCH .aws-sam
```

The build job begins by switching the active Java version with [sem-version](https://docs.semaphoreci.com/ci-cd-environment/sem-version-managing-language-versions-on-linux/) and installing SAM in the [CI machine](https://docs.semaphoreci.com/ci-cd-environment/machine-types/). After doing [checkout](https://docs.semaphoreci.com/reference/toolbox-reference/#checkout) to clone the repository, we build the artifact, located in the `.aws-sam` folder, and save it in the [cache](https://docs.semaphoreci.com/reference/toolbox-reference/#cache).


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pac10940dy0mk25wa1eg.png)

Next, scroll down the right pane until you find the **Skip/Run conditions** section. Change the condition to “Run this block when conditions are met” and type the following in the When field:

``` bash
change_in('/java-app-backend')
```

[change_in](https://docs.semaphoreci.com/reference/conditions-reference/#change_in) is at the center of gravity of monorepo workflows because it allows us to tie blocks to folders and files in the repository. The function scans the Git history and figures out what parts of the code had recently changed and run or skip CI/CD jobs, cutting down build times and costs.

📙 You can learn all about continuous integration for monorepos with our **free ebook**: [CI/CD for Monorepos](https://semaphoreci.com/resources/monorepo-cicd)

## Backend continuous deployment

Then, click on **Add First Promotion** and enable automatic promotions. Type this condition in the When? field, which trigger the deployment when all tests pass on the master branch and the backend code has been modified.

``` bash
branch = 'master' AND result = 'passed' AND change_in('/java-app-backend')
```


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/6e1x9t0uqb0rmpjkrsa2.png)

Create a deployment job in the new pipeline. By now, most commands should sound familiar. The only change is that we’re using `--no-confirm-changeset`, so the process is non-interactive.

``` bash
sem-version java 8
wget https://github.com/aws/aws-sam-cli/releases/latest/download/aws-sam-cli-linux-x86_64.zip
unzip aws-sam-cli-linux-x86_64.zip -d sam-installation
sudo ./sam-installation/install
checkout
cd java-app-backend
cache restore build-java-app-backend-$SEMAPHORE_GIT_BRANCH
sam deploy --no-confirm-changeset --stack-name serverless-web-application-java-backend --capabilities CAPABILITY_AUTO_EXPAND CAPABILITY_IAM
```


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fbra1on1gojfb04a50sm.png)

Finish setting up the CI/CD block by enabling the AWS secret you created at the beginning. Finally, save the changes into the master branch by clicking on **Run the workflow** \> **Start**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/a7uzr6zl6g56gurzekez.png)

## Build and deploy the frontend

The frontend is a React SPA (Single Page Application) that interacts with the Python and Java backends. We’ll run it directly from an S3 Bucket.

In the file `frontend/src/GlobalConstants.ts`, you’ll find the URLs for both backends. Remember the `id` property for the serverless functions from earlier? You need it now. Update the URLs so they point to the Lambda functions.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/cgqz4ujwlu0at3qogyn4.png)

Now build the application with:

``` bash
$ cd frontend/
$ yarn install
$ yarn build
```

**Create the bucket** either with the S3 console or with the CLI. Pay attention to the bucket name; it should be unique, so you may need to experiment until you find one that works for you:

``` bash
$ aws s3api create-bucket --bucket myfrontend-aws-monorepo --region eu-west-1 --create-bucket-configuration LocationConstraint=eu-west-1
```

**Copy the files** and then set the bucket to host the website with the following commands:

``` bash
$ aws s3 cp build s3://myfrontend-aws-monorepo --recursive --acl public-read
$ aws s3 website s3://myfrontend-aws-monorepo --index-document index.html
```

The website can be visited with this URL (change the region as needed):

``` bash
http://BUCKET_NAME.s3-website.eu-west-1.amazonaws.com
```

To complete the deployment, provision Cloudfront and point it to the S3 website. The CDN will help you deliver the application fast to end-users all over the world. Replace the URL of the website as needed:

    $ aws cloudfront create-distribution --origin-domain-name myfrontend-aws-monorepo.s3-website.eu-west-1.amazonaws.com --default-root-object index.html
    
    {
        "Location": "https://cloudfront.amazonaws.com/2020-05-31/distribution/E26Q8XKJFH6K4G",
        "ETag": "E1E0WE2I8CVJPH",
        "Distribution": {
            "Id": "E26Q8XKJFH6K4G",
            "ARN": "arn:aws:cloudfront::890702391356:distribution/E26Q8XKJFH6K4G",
            "Status": "InProgress",
            "LastModifiedTime": "2021-11-09T20:32:02.716000+00:00",
            "InProgressInvalidationBatches": 0,
            "DomainName": "dk4ef7pcvfhnv.cloudfront.net",  // <--- Cloudfront enabled URL
    
            . . .

Open a browser at the CloudFront-enabled site (use the returned `DomainName` value). In the upper right corner of your application, you can switch between the Java and Python backends.

Try registering some faces to see if everything is working as expected.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/lhh7w7ng4bon73r58hdu.png)

You may check the Lambda remote execution logs with `sam logs -n LAMBDA_FUNCTION_NAME --stack STACK_NAME`.

## Continuous deployment for the frontend

We’ll update our CI/CD pipeline to test and deploy the frontend every time it changes.

Open the workflow editor, **add a new block** to build the frontend and cache the build files.

``` bash
sem-version node 16.13.0
checkout
cd frontend
cache restore
yarn install
cache store
yarn build
cache store build-frontend-$SEMAPHORE_GIT_BRANCH build
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7mrwfiiuxmqdwejritoc.png)

**Create a test block** to run the React unit tests.

``` bash
sem-version node 16.13.0
checkout
cd frontend
cache restore
yarn test
```


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/rymj0p3nsbtos3xpo2d0.png)

Then, set the **Skip/Run conditions** on both new blocks to the following:

``` bash
change_in('/frontend')
```

Now, **create a promotion and a deploy job**. The condition for deployment is:

``` text
branch = 'master' AND result = 'passed' AND change_in('/frontend')
```

Finally, create a deployment job to copy the build files into the S3 bucket.

``` bash
checkout
cd frontend
cache restore build-frontend-$SEMAPHORE_GIT_BRANCH
aws s3 cp build s3://${BUCKET_NAME} --recursive --acl public-read
```

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g4m6uu443jhnfzgtjeos.png)

Enable the AWS secret, then click on **Run the workflow** \> **Start**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xxzzjafb28uukbr5sanm.png)

Try the backend and frontend deployments once more to check everything is in order.

## Conclusion

In this post, we’ve learned a few tools to help us work with monorepos and AWS. We’ve seen how SAM can help us quickly develop and test serverless functions and how to individually deploy services in a monorepo with Semaphore.

**Read Next**:

- [New Book: CI/CD for Monorepos](https://semaphoreci.com/blog/new-book-cicd-for-monorepos)
- [JavaScript Monorepos with Lerna](https://semaphoreci.com/blog/javascript-monorepos-lerna)
- [TypeScript Monorepos with Yarn](https://semaphoreci.com/blog/typescript-monorepos-with-yarn)
- [Podcast] [Monorepo and Micro-Frontends with Jonathan Creamer](https://semaphoreci.com/blog/monorepo-micro-frontends-jonathan-creamer)