Build a Serverless URL shortener with Go
Tags: serverless, tutorial, go, database
Published: 2022-08-02 04:13:00

*Using AWS Lambda, DynamoDB and API Gateway*

This blog post covers how to build a Serverless URL shortener application using [Go](https://go.dev/). It leverages [AWS Lambda](https://docs.aws.amazon.com/lambda/latest/dg/welcome.html) for business logic, [DynamoDB](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Introduction.html) for persistence and [API Gateway](https://docs.aws.amazon.com/apigateway/latest/developerguide/welcome.html) to provide the HTTP endpoints to access and use the application. The sample application presented in this blog is a trimmed down version of `bit.ly` or other solutions you may have used or encountered.

It's structured as follows:
- I will start off with a quick introduction and dive into how to deploy try the solution.
- After that, I will focus on the code itself. This will cover:
  - The part which is used to write the infrastructure (using Go bindings for AWS CDK)
  - And also the core business logic which contains the Lambda function (using Lambda Go support) as well as the DynamoDB operations (using the DynamoDB Go SDK)

In this blog, you will learn:
- How to use the [DynamoDB Go SDK](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/dynamodb) (v2) to execute CRUD operations such as `PutItem`, `GetItem`, `UpdateItem` and `DeleteItem`
- How to use [AWS CDK Go](https://docs.aws.amazon.com/cdk/v2/guide/work-with-cdk-go.html) bindings to deploy a Serverless application to create and manage a `DynamoDB` table, Lambda functions, API Gateway and other components as well.

Once you deploy the application, you will be able to create short codes for URLs using the endpoint exposed by the API Gateway and also access them.

```bash
# create short code for a URL (e.g. https://abhirockzz.github.io/)
curl -i -X POST -d 'https://abhirockzz.github.io/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL

# access URL via short code
curl -i $URL_SHORTENER_APP_URL/<short-code>
```

## Lets get started - Deploy the Serverless application

Before you proceed, make sure you have the [Go programming language](https://go.dev/dl/) (**v1.16** or higher) and [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html#getting_started_install) installed.

Clone the project and change to the right directory:

```bash
git clone https://github.com/abhirockzz/serverless-url-shortener-golang
cd cdk
```

**To start the deployment...**

.. all you will do is run a single command (`cdk deploy`), and wait for a bit. You will see a (long) list of resources that will be created and will need to provide your confirmation to proceed.

Don't worry, in the next section I will explain what's happening.


```bash
cdk deploy

# output

Bundling asset ServerlessURLShortenerStack1/create-url-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/access-url-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/update-url-status-function/Code/Stage...
Bundling asset ServerlessURLShortenerStack1/delete-url-function/Code/Stage...

✨  Synthesis time: 10.28s

This deployment will make potentially sensitive changes according to your current security approval level (--require-approval broadening).
Please confirm you intend to make the following modifications:
.......

Do you wish to deploy these changes (y/n)?
```

This will start creating the AWS resources required for our application.

> If you want to see the AWS CloudFormation template which will be used behind the scenes, run `cdk synth` and check the `cdk.out` folder

You can keep track of the progress in the terminal or navigate to AWS console: `CloudFormation > Stacks > ServerlessURLShortenerStack`


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/g99x3zy4xcd6f2e7otii.png)

Once all the resources are created, you can try out the application. You should have:

- Four Lambda functions (and related resources)
- A DynamoDB table
- An API Gateway (as well as routes, integrations)
- along with a few others (like IAM roles etc.)

Before you proceed, get the API Gateway endpoint that you will need to use. It's available in the stack output (in the terminal or the **Outputs** tab in the AWS CloudFormation console for your Stack):


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f6p1allp3e7hw108na5k.png)

## Shorten some URLs!

**Start by generating short codes for a few URLs**

```bash
# export the API Gateway endpoint
export URL_SHORTENER_APP_URL=<replace with apigw endpoint above>

# for example:
export URL_SHORTENER_APP_URL=https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/

# invoke the endpoint to create short codes
curl -i -X POST -d 'https://abhirockzz.github.io/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL

curl -i -X POST -d 'https://dzone.com/users/456870/abhirockzz.html' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL

curl -i -X POST -d 'https://abhishek1987.medium.com/' -H 'Content-Type: text/plain' $URL_SHORTENER_APP_URL
```

To generate a short code, you need to pass the original URL in the payload body as part of a HTTP POST request (for e.g. `https://abhishek1987.medium.com/`)

> 'Content-Type: text/plain' is important, otherwise API Gateway will do `base64` encoding of your payload

If all goes well, you should get a `HTTP 201` along with the short code in the HTTP response (as a JSON payload).

```bash
HTTP/2 201 
date: Fri, 15 Jul 2022 13:03:20 GMT
content-type: text/plain; charset=utf-8
content-length: 25
apigw-requestid: VTzPsgmSoAMESdA=

{"short_code":"1ee3ad1b"}
```

To confirm, check the DynamoDB table.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/1jj2gwt22jrxawptu3nz.png)

> Notice an **active** attribute there? More on this soon

**Access the URL using the short code**

With services like bit.ly etc. you typically create short links for your URLs and share them with the world. We will do something similar. Now that you have the short code generated, you can share the link (it's not really a short link like bit.ly but that's ok for now!) with others and once they access it, they would see the original URL.

The access link will have the following format - `<URL_SHORTENER_APP_URL>/<generated short code>` for e.g. https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b

If you navigate to the link using a browser, you will be automatically redirected to the original URL that you had specified. To see what's going on, try the same with `curl`:

```bash
curl -i $URL_SHORTENER_APP_URL/<short code>

# example
curl -i https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/0e1785b1
```

This is simply an `HTTP GET` request. If all goes well, you should get an `HTTP 302` response (`StatusFound`) and the URL re-direction happens due to the the **Location** HTTP header which contains the original URL.

```bash
HTTP/2 302 
date: Fri, 15 Jul 2022 13:08:54 GMT
content-length: 0
location: https://abhirockzz.github.io/
apigw-requestid: VT0D1hNLIAMES8w=
```

> How about using a short code that does not exist?

**Set the status**

You can enable and disable the short codes. The original URL will only be accessible if the association is in **active** state.

To disable a short code:

```bash
curl -i -X PUT -d '{"active": false}'  -H 'Content-Type: application/json' $URL_SHORTENER_APP_URL/<short code>

# example
curl -i -X PUT -d '{"active": false}'  -H 'Content-Type: application/json' https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b
```

This is an `HTTP PUT` request with a JSON payload that specifies the status (`false` in this case refers to disable) along with the short code which is a path parameter to the API Gateway endpoint. If all works well, you should see a HTTP 204 (No Content) response:

```bash
HTTP/2 204 
date: Fri, 15 Jul 2022 13:15:41 GMT
apigw-requestid: VT1Digy8IAMEVHw=
```

Check the DynamoDB record - the **active** attribute must have switched to `false`.

> As an exercise, try the following:
> 
> 1. access the URL via the same short code now and check the response.
> 2. access an invalid short code i.e. that does not exist
> 3. enable a disabled URL (use `{"active": true}`)

Ok, so far we have covered all operations, except delete. Lets try that and wrap up the `CRUD`!

**Delete**

```bash
curl -i -X DELETE $URL_SHORTENER_APP_URL/<short code>

# example
curl -i -X DELETE https://b3it0tltzk.execute-api.us-east-1.amazonaws.com/1ee3ad1b
```

Nothing too surprising. We use a `HTTP DELETE` along with the short code. Just like in case of update, you should get a `HTTP 204` response:

```bash
HTTP/2 204 
date: Fri, 15 Jul 2022 13:23:36 GMT
apigw-requestid: VT2NzgjnIAMEVKA=
```

But this time of course, the DynamoDB record should have been deleted - confirm the same.

> What happens when you try to delete a short code that does not exist?

### Don't forget to clean up!

Once you're done, to delete all the services, simply use:

```bash
cdk destroy
```

Alright, now that you've actually seen "what" the application does, let's move on to the "how".
We will start with the AWS CDK code and explore how it does all the heavy lifting behind to setup the infrastructure for our Serverless URL shortener service.

## With AWS CDK, Infrastructure-IS-code!

You can check out the code in [this GitHub repo](https://github.com/abhirockzz/serverless-url-shortener-golang/tree/master/cdk). I will walk you through the keys parts of the `NewServerlessURLShortenerStack` function which defines the workhorse of our CDK application.

> I have omitted some of the code for brevity

We start by creating a `DynamoDB` table. A primary key is all that's required in order to do that - in this case `shortcode` (we don't have [range/sort key](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.CoreComponents.html#HowItWorks.CoreComponents.PrimaryKey) in this example)

```go
	dynamoDBTable := awsdynamodb.NewTable(stack, jsii.String("url-shortener-dynamodb-table"),
		&awsdynamodb.TableProps{
			PartitionKey: &awsdynamodb.Attribute{
				Name: jsii.String(shortCodeDynamoDBAttributeName),
				Type: awsdynamodb.AttributeType_STRING}})
```

Then, we create an [API Gateway (HTTP API)](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api.html) with just one line of code!

```go
urlShortenerAPI := awscdkapigatewayv2alpha.NewHttpApi(stack, jsii.String("url-shortner-http-api"), nil)
```

We move on to the first Lambda function that creates short codes for URLs. Notice that we use an experimental module [awscdklambdagoalpha](https://pkg.go.dev/github.com/aws/aws-cdk-go/awscdklambdagoalpha/v2) ([here is the stable version](https://pkg.go.dev/github.com/aws/aws-cdk-go/awscdk/v2/awslambda) at the time of writing). If your Go project is structured in a specific way ([details here](https://pkg.go.dev/github.com/aws/aws-cdk-go/awscdklambdagoalpha/v2#readme-go-function)) and you specify its path using `Entry`, it will automatically take care of building, packaging and deploying your Lambda function! Not bad at all! 

> In addition to [Local bundling](https://pkg.go.dev/github.com/aws/aws-cdk-go/awscdklambdagoalpha/v2#readme-local-bundling) (as used in this example), [Docker based builds](https://pkg.go.dev/github.com/aws/aws-cdk-go/awscdklambdagoalpha/v2#readme-docker) are also supported.


```go
	createURLFunction := awscdklambdagoalpha.NewGoFunction(stack, jsii.String("create-url-function"),
		&awscdklambdagoalpha.GoFunctionProps{
			Runtime:     awslambda.Runtime_GO_1_X(),
			Environment: funcEnvVar,
			Entry:       jsii.String(createShortURLFunctionDirectory)})

	dynamoDBTable.GrantWriteData(createURLFunction)
```

Finally, we add the last bit of plumbing by creating a [Lambda-HTTP API integration](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-integrations-lambda.html) (notice how the Lambda function variable `createURLFunction` is referenced) and [adding a route to the HTTP API](https://docs.aws.amazon.com/apigateway/latest/developerguide/http-api-develop-routes.html) we had created - this in turn refers to the Lambda integration.

```go
	createFunctionIntg := awscdkapigatewayv2integrationsalpha.NewHttpLambdaIntegration(jsii.String("create-function-integration"), createURLFunction, nil)

	urlShortenerAPI.AddRoutes(&awscdkapigatewayv2alpha.AddRoutesOptions{
		Path:        jsii.String("/"),
		Methods:     &[]awscdkapigatewayv2alpha.HttpMethod{awscdkapigatewayv2alpha.HttpMethod_POST},
		Integration: createFunctionIntg})
```

This was just for one function - we have three more remaining. The good part is that the template for all these are similar i.e. 

1. create the function
2. grant permission for DynamoDB
3. wire it up with API Gateway (with the correct HTTP method i.e. `POST`, `PUT`, `DELETE`)

So I will not repeat it over here. Feel free to grok through the rest of the code.

Now that you understand the magic behind the "one-click" infrastructure setup, let's move on to the core logic of the application.

## URL shortener Lambda function and DynamoDB logic

There are four different functions, all of which are in their respective folders and all of them have a few things in common in the way they operate:

1. They do dome initial processing - process the payload, or extract the path parameter from the URL etc.
2. Invoke a [common database layer](https://github.com/abhirockzz/serverless-url-shortener-golang/blob/master/db/db.go) - to execute the CRUD functionality (more on this soon)
3. Handle errors as appropriate and return response

With that knowledge, it should be easy to follow along the code.

> As before, some parts of the code have been omitted for brevity

**Create function**

```go
func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {
	url := req.Body

	shortCode, err := db.SaveURL(url)
	if err != nil {//..handle error}

	response := Response{ShortCode: shortCode}
	respBytes, err := json.Marshal(response)
	if err != nil {//..handle error}

	return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusCreated, Body: string(respBytes)}, nil
}
```

This function starts by reading the payload of the `HTTP` request body - this is a `string` which has the URL for which the short code is being created. It invokes the database layer to try and save this record to DynamoDB and handles errors. Finally, it returns a JSON response with the short code.

Here is the function that actually interacts with DynamoDB to get the job done.

```go
func SaveURL(longurl string) (string, error) {
	shortCode := uuid.New().String()[:8]

	item := make(map[string]types.AttributeValue)

	item[longURLDynamoDBAttributeName] = &types.AttributeValueMemberS{Value: longurl}
	item[shortCodeDynamoDBAttributeName] = &types.AttributeValueMemberS{Value: shortCode}
	item[activeDynamoDBAttributeName] = &types.AttributeValueMemberBOOL{Value: true}

	_, err := client.PutItem(context.Background(), &dynamodb.PutItemInput{
		TableName: aws.String(table),
		Item:      item})

	if err != nil {//..handle error}

	return shortCode, nil
}
```

For the purposes of this sample app, the short code is created by generating a `UUID` and trimming out the last 8 digits. It's easy to replace this with another technique - all that matters is that you generate a unique string that can work as a short code. Then, it all about calling the [PutItem API](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/dynamodb#Client.PutItem) with the required data.

**Access the URL**

```go
func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {

	shortCode := req.PathParameters[pathParameterName]
	longurl, err := db.GetLongURL(shortCode)

	if err != nil {//..handle error}

	return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusFound, Headers: map[string]string{locationHeader: longurl}}, nil
}
```

When someone accesses the short link (as demonstrated in the earlier section), the short code is passed in as a path parameter e.g. `http://<api gw url>/<short code>`. the database layer is invoked to get the corresponding URL from `DynamoDB` table (errors are handled as needed). Finally, the response is returned to the user wherein the status code is `302` and the URL is passed in the **Location** header. This is what re-directs you to the original URL when you enter the short code (in the browser)

Here is the DynamoDB call:

```go
func GetLongURL(shortCode string) (string, error) {

	op, err := client.GetItem(context.Background(), &dynamodb.GetItemInput{
		TableName: aws.String(table),
		Key: map[string]types.AttributeValue{
			shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}}})

	if err != nil {//..handle error}

	if op.Item == nil {
		return "", ErrUrlNotFound
	}

	activeAV := op.Item[activeDynamoDBAttributeName]
	active := activeAV.(*types.AttributeValueMemberBOOL).Value

	if !active {
		return "", ErrUrlNotActive
	}

	longurlAV := op.Item[longURLDynamoDBAttributeName]
	longurl := longurlAV.(*types.AttributeValueMemberS).Value

	return longurl, nil
}
```

The first step is to use [GetItem API](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/dynamodb#Client.GetItem) to get the `DynamoDB` record containing URL and status corresponding to the short code. If the `item` object in the response is `nil`, we can be sure that a record with that short code does *not* exist - we return a custom error which can be helpful for our function which can then return an appropriate response to the caller of the API (e.g. a `HTTP 404`). We also check the status (active or not) and return an error if **active** is set to `false`. If all is well, the URL is returned to the caller.

**Update status**

```go
func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {

	var payload Payload
	reqBody := req.Body

	err := json.Unmarshal([]byte(reqBody), &payload)
	if err != nil {//..handle error}

	shortCode := req.PathParameters[pathParameterName]

	err = db.Update(shortCode, payload.Active)
	if err != nil {//..handle error}

	return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusNoContent}, nil
}
```

The first step is to marshal the HTTP request payload which is a JSON e.g. `{"active": false}` and then get the short code from the path parameter. The database layer is invoked to update the status and handle errors.

```go
func Update(shortCode string, status bool) error {

	update := expression.Set(expression.Name(activeDynamoDBAttributeName), expression.Value(status))
	updateExpression, _ := expression.NewBuilder().WithUpdate(update).Build()

	condition := expression.AttributeExists(expression.Name(shortCodeDynamoDBAttributeName))
	conditionExpression, _ := expression.NewBuilder().WithCondition(condition).Build()

	_, err := client.UpdateItem(context.Background(), &dynamodb.UpdateItemInput{
		TableName: aws.String(table),
		Key: map[string]types.AttributeValue{
			shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}},
		UpdateExpression:          updateExpression.Update(),
		ExpressionAttributeNames:  updateExpression.Names(),
		ExpressionAttributeValues: updateExpression.Values(),
		ConditionExpression:       conditionExpression.Condition(),
	})

	if err != nil && strings.Contains(err.Error(), "ConditionalCheckFailedException") {
		return ErrUrlNotFound
	}

	return err
}
```

The [UpdateItem API](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/dynamodb#Client.UpdateItem) call takes care of changing the status. It's fairly simple except for the all these `expressions` that you need - especially if you're new to the concept. The first one (mandatory) is the update expression where you specify the attribute you need to set (**active** in this case) and its value. The second one makes sure that you are updating the status for a short code that actually exists in the table. This is important since, otherwise the `UpdateItem` API call will insert a **new** item (we don't want that!). Instead of rolling out the expressions by hand, we use the [expressions](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/feature/dynamodb/expression) package.

**Delete short code**

```go
func handler(ctx context.Context, req events.APIGatewayV2HTTPRequest) (events.APIGatewayV2HTTPResponse, error) {

	shortCode := req.PathParameters[pathParameterName]

	err := db.Delete(shortCode)
	if err != nil {//..handle error}
	return events.APIGatewayV2HTTPResponse{StatusCode: http.StatusNoContent}, nil
}
```

The delete handler is no different. After the short code to be deleted is extracted from the path parameter, the database layer is invoked to remove it from the DynamoDB table. The result returned to the user is either an `HTTP 204` (on success) or the error.

```go
func Delete(shortCode string) error {

	condition := expression.AttributeExists(expression.Name(shortCodeDynamoDBAttributeName))
	conditionExpression, _ := expression.NewBuilder().WithCondition(condition).Build()

	_, err := client.DeleteItem(context.Background(), &dynamodb.DeleteItemInput{
		TableName: aws.String(table),
		Key: map[string]types.AttributeValue{
			shortCodeDynamoDBAttributeName: &types.AttributeValueMemberS{Value: shortCode}},
		ConditionExpression:       conditionExpression.Condition(),
		ExpressionAttributeNames:  conditionExpression.Names(),
		ExpressionAttributeValues: conditionExpression.Values()})

	if err != nil && strings.Contains(err.Error(), "ConditionalCheckFailedException") {
		return ErrUrlNotFound
	}

	return err
}
```

Just like `UpdateItem` API, the [DeleteItem](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/dynamodb#Client.DeleteItem) API also takes in a condition expression. If there is no record in the DynamoDB table with the given short code, an error is returned. Otherwise, the record is deleted.

That completes the code walk through!

## Wrap up

In this blog post you learnt how to use DynamoDB Go SDK using a URL Shortener sample application. You also integrated it with AWS Lambda and API Gateway to build a Serverless solution whose infrastructure was also defined using *actual* code (as opposed to `yaml`, JSON etc.), thanks to the Go support in AWS CDK.