Workdrop — Setting Up Serverless
Tags: twiliohackathon, serverless, aws, mongodb
Published: 2020-04-27 16:46:42

*This continues my entry for #twiliohackathon!*

[GitHub Repository](https://github.com/Sergix/workdrop) (MIT Licensed)
[Stackshare](https://stackshare.io/Sergix/workdrop)

<hr />

Before we get going with the frontend, I'll describe a summary of some of the serverless setup with **MongoDB Stitch**, **AWS**, and **Twilio SendGrid**.

## MongoDB

I already discussed setting up a MongoDB cluster in the second entry in this series, but now it's time to get cooking with *serverless*.

### Functions

MongoDB Stitch, of course, has serverless functions built-in, and makes Atlas database interactions a breeze.

#### Requests

The architecture is pretty simple: users enter their assignment name, student emails, a personalized message, and their email. Then, it sends all that data to a MongoDB function.

There are two collections in the database: `requests` and `submissions`.

Before the user's data is put in the `requests` collection, two UUID tokens are generated by a separate function:

```javascript
// generateToken()

const uuid = require("uuid");
  
const token = uuid.v4('workdrop.app', uuid.v5.DNS);
const accessor = uuid.v4('workdrop.app', uuid.v5.DNS);

return { token, accessor };
```

The first token will be sent out to the requester and each student. The second `accessor` token will only be sent out to the requester, and will allow them to access a page where they can download the submissions.

Those tokens are inserted along with the user data in a new document.

```javascript
// createRequest
const { token, accessor } = context.functions.execute("generateToken");
  
const requestsCollection = context.services.get("my-cluster").db("workdrop").collection("requests");

const newRequest = {
  token, accessor, name, email, request_emails, message
};

requestsCollection.insertOne(newRequest)
  .then(result => console.log(`Successfully inserted request with _id: ${result.insertedId}`))
  .catch(err => console.error(`Failed to insert request: ${err}`));
```

After this document is inserted, a trigger calls another function which uses the **Twilio SendGrid API** to send out the emails.

#### Submissions

For submissions, the client first connects to MongoDB and then, through the MongoDB API, connects to the AWS S3 bucket *(details below)*. The submitted file is uploaded to AWS, and the returned URL is sent to MongoDB through a function virtually identical to the request function.

### AWS

Setting up AWS can be pretty complicated sometimes, especially involving authentication. This was my first time setting up an S3 bucket and permissions.

First, you need to [create your account](https://portal.aws.amazon.com/gp/aws/developer/registration). This will create a *root user*, which has access to *everything* in AWS. You should quickly enable multi-factor authentication and have a strong and secure password for your root user.

Your serverless app *should never* (and I'm not even sure if it can) authenticate with AWS via the root user. Instead, you create a special *IAM user* where you specify your exact permissions. API keys and secrets are then generated specifically for this user.

For myself, I simply created a new policy called `workdrop-stitch` and added the S3 `GetObject` and `PutObject` permissions.

After setting up AWS via the management console, I went back over to MongoDB and setup a new **Third-Party Service**, selecting **AWS**.

![MongoDB Stitch Service](https://dev-to-uploads.s3.amazonaws.com/i/eef5fpere7w15a7l83ix.png)

Then, you specify your API key and you're off to go!

## Twilio SendGrid

Here's the somewhat critical part of my entry: **Twilio API integration**. For my project, the Twilio service I'm using is [**Twilio SendGrid**](https://www.sendgrid.com).

![Twilio SendGrid](https://github.com/sendgrid/sendgrid-python/raw/master/twilio_sendgrid_logo.png)

Even if using a Twilio API wasn't a part of the hackathon requirements, I'm certain I would be still using SendGrid because of how fantastic its API is.

![Twilio SendGrid Dashboard](https://dev-to-uploads.s3.amazonaws.com/i/hkbqzz5tq8017s0ac11r.png)

### Sender authentication

In order to properly ensure that emails sent out with SendGrid are verified with all email clients and services, you must set up some DNS records to validate your domain with SendGrid. Then, you can have the emails sent out from your app look like `xxxxxx@workdrop.app` instead of using SendGrid's default `xxxxxx@sendgrid.net`.

Setup is straightforward and simple, especially if you're using [CloudFlare DNS](https://www.cloudflare.com/dns/) as a proxy or [Netlify's DNS](https://docs.netlify.com/domains-https/netlify-dns). All that's required is creating a few specific CNAME records that point to `sendgrid.net`. Instructions specific to your user are in your SendGrid dashboard.

### A note on SendGrid with SSL

If you *have* to have SSL security on all traffic through your domain, keep in mind that if you're using [Click Tracking with SendGrid](https://sendgrid.com/docs/ui/account-and-settings/tracking/) (which I do have disabled for my app at the moment), [you'll have to direct your traffic through a seperate proxy secured with a certificate for your domain](https://sendgrid.com/docs/ui/analytics-and-reporting/click-tracking-ssl/). This is because the CNAME record used to point the traffic from your domain to SendGrid's tracking servers is not HTTPS secured, so most browsers will simply refuse to connect to the link if HTTPS is enforced on your domain. And keep in mind, `.app` and `.dev` domains are SSL-enforced.

### Using the API

For JavaScript, you can either use the [SendGrid Node.JS API](https://github.com/sendgrid/sendgrid-nodejs/tree/master/packages/mail) or [manually call the API via REST](https://sendgrid.com/docs/API_Reference/Web_API_v3/index.html).

Here's an example of my app's REST API call using JavaScript and MongoDB Stitch:

```javascript
// sendRequestEmails()

// context.http.post is provided by MongoDB Stitch functions

context.http.post({ 
	"url": "https://api.sendgrid.com/v3/mail/send",
	"headers": { Authorization: [`Bearer [API_KEY]`] },
	"body": {
		"personalizations": [{
		  "to": [{
				"email": "example@example.com"
      }],
    }],
		"from": {
		  "email": "no-reply@workdrop.app"
		}
	},
	"encodeBodyAsJSON": "true"
)
  .then(res => {
	  console.log(`EMAIL SUCCESS [${res.statusCode}]`);
  })
  .catch(res => {
	  console.log(`EMAIL ERROR [${res.statusCode}]`);
  });
```

## AWS S3 Policy Setup

Here's the part of the project that was the most frustrating for me.

AWS policies can be very difficult to set up, especially for someone like me who never has done it before. And any information on specific policies can be difficult to find on forums and other sites.

### User management

First, [after you create your AWS root account](https://portal.aws.amazon.com/billing/signup#/start), you must create an IAM user that your app will use to connect to your services.

![AWS IAM User](https://dev-to-uploads.s3.amazonaws.com/i/kn5gvj46dikplphlbgqc.png)

When creating your IAM user, you must enable *programmatic access* since this will only be accessed through an API and not the AWS console.

### S3 Policy

You can attach an arbitrary number of policies to any IAM user to enable it to perform specific actions across AWS services.

For S3, there are two primary policies that we need to enable: `GetObject` and `PutObject`.

![AWS S3 Policy](https://dev-to-uploads.s3.amazonaws.com/i/93pzyvohu6o08c1cczj7.png)

In addition, I found that I had to enable `GetObjectAcl` and `PutObjectAcl`.

After enabling these policies, you simply select the bucket resource from your account in the *Resources* field pictured above.

### Bucket setup

After creating these policies, you have to manage a couple permissions for your S3 bucket.

In the **Permissions** tab of your bucket, you need to enable all *block public accecss* settings *except* `Block public access to buckets and objects granted through new access control lists (ACLs)`. This involves the `GetObjectAcl` and `PutObjectAcl` permissions we had to set up earlier so authenticated APIs can access our bucket properly.

![Alt Text](https://dev-to-uploads.s3.amazonaws.com/i/9q2yg7iopwpxxhrcm0v6.png)

Next, in the **Bucket Policy** tab, you need to insert an ARN policy to control access from users. Anything within brackets `[]` below will need to be replaced with your information. [AWS has a policy generator that can help.](https://awspolicygen.s3.amazonaws.com/policygen.html)

```json
{
    "Version": "2012-10-17",
    "Id": "[policy ID]",
    "Statement": [
        {
            "Sid": "[Sid]",
            "Effect": "Allow",
            "Principal": {
                "AWS": "[Your AWS user ARN]"
            },
            "Action": [
                "s3:GetObject",
                "s3:GetObjectAcl",
                "s3:PutObject",
                "s3:PutObjectAcl"
            ],
            "Resource": "[Your S3 bucket resource ID]"
        }
    ]
}
```

Lastly, in the **CORS configuration** tab, you need to put the following XML. This allows any cross-origin request to be authenticated with the S3 API.

```xml
<?xml version="1.0" encoding="UTF-8"?>
<CORSConfiguration xmlns="http://s3.amazonaws.com/doc/2006-03-01/">
<CORSRule>
    <AllowedOrigin>*</AllowedOrigin>
    <AllowedMethod>HEAD</AllowedMethod>
    <AllowedMethod>GET</AllowedMethod>
    <AllowedMethod>PUT</AllowedMethod>
    <AllowedMethod>POST</AllowedMethod>
    <AllowedHeader>*</AllowedHeader>
</CORSRule>
</CORSConfiguration>
```
<br>
<hr>
## Now onto the frontend. Thank you for reading! God bless!