Retrieve the CDN link of a file sent to S3 by the Lambda function on AWS
Tags: go, aws, cloud, beginners
Published: 2023-10-08 01:40:36

**TL;DR**
Here we'll retrieve a link from the AWS CloudFront, after sending a file to an AWS API Gateway that will trigger a Lambda function written in Golang to upload a file to S3, with the objective to know a bit more in practice about these AWS services. 

![Demo of the system, making a POST request to a route configured on AWS API gateway to trigger a lambda function to upload a file to S3](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/mfnofqaeqwg16rq19k46.png)

## Setup S3 and CloudFront
Let's start creating our bucket on S3, where we'll store the files sent from our lambda function later. Go to the [AWS S3](https://s3.console.aws.amazon.com/s3/) service and create a new bucket with the default configuration.

Now we'll configure our [CloudFront](https://us-east-1.console.aws.amazon.com/cloudfront), a way to share our files with a low latency. You can see more about this service [here](https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/Introduction.html). After clicking on "Create distribution", select the bucket created previously on S3 and create the control setting with the default parameters.

![Default control settings](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/p0gkdzcayfpy23wh3n7s.png)

After finishing the creation of the distribution, it'll appear a warning message to update our bucket policies. Click to copy policy go to the link highlighted and paste the copied policy created on the session "Bucket policy".

![Add policy on bucket](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/pglvi4q7jyw7pric2s3k.png)

To set our environment variables from AWS, we can use the command `aws configure` from [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)

## Setup the environment locally
First, we'll set the environment to be able to test the lambda function that will be created locally, without uploading the code on every change on AWS. This configuration is important to make it possible to test all changes that can be made in the future in a faster way, to be sure our code is correct and, if we find some issue after uploading the code, we'll know the problem is on the AWS services configurations or permissions.

For that, we'll need to have the [SAM CLI](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html) to simulate a serverless environment, and we'll create a `template.yml` to being used (needs [docker](https://docs.docker.com/engine/install/))

```yml
AWSTemplateFormatVersion: '2010-09-09'
Transform: 'AWS::Serverless-2016-10-31'
Description: An example AWS Serverless Application.
Resources:
  MyApi:
    Type: 'AWS::Serverless::Api'
    Properties:
      StageName: Prod
      BinaryMediaTypes: # allow to send binary data on requests
        - '*/*'

  MyFunction:
    Type: 'AWS::Serverless::Function'
    Properties:
      Handler: yuta # Set your handler here (the binary)
      Runtime: go1.x # Specify the Go runtime version
      CodeUri: . # Path to your Go code
      MemorySize: 128 # Set the desired memory size
      Timeout: 10 # Set the function timeout
      Events:
        ApiEvent:
          Type: Api # This connects the Lambda to an API Gateway
          Properties:
            Path: /myresource # Route triggering the function
            Method: POST # Define the HTTP method (e.g., GET, POST)

Outputs:
  ApiEndpoint:
    Description: "API Gateway endpoint URL"
    Value:
      Fn::Sub: "https://${MyApi}.execute-api.${AWS::Region}.amazonaws.com/Prod/myresource"
```

After that, let's create the code of the lambda function. Let's initialize the module on the same level as the `template.yml` created:

`go mod init github.com/your-github-user/your-repository-name`

To start, we'll create the main file. I created inside the folder `src`:
```go
package main

import (
	"context"
	"fmt"

	"github.com/aws/aws-lambda-go/events"
	"github.com/aws/aws-lambda-go/lambda"
)

func main() {
	lambda.Start(handler) // Every time this lambda function is triggered (each request), it'll call the handler function, sending the event of the request.
}

func handler(ctx context.Context, event events.APIGatewayProxyRequest) (events.APIGatewayProxyResponse, error) {
	fmt.Println("🚀 Received event! Running...")

	awsRegion := "your-aws-region" // Set your AWS region here
	bucketName := "your-bucket-name" // Set the name of the bucket created on S3 here

	fileHeaders, headersError := GetHeaders(event, 10485760)

	if headersError != nil {
		return BadRequestFeedback("Error reading headers: " + headersError.Error()), nil
	}

	fileName, uploadError := UploadFile(fileHeaders[0], awsRegion, bucketName)

	if uploadError != nil {
		return BadRequestFeedback("Error uploading file to S3: " + uploadError.Error()), nil
	}

	cloudFrontURL := fmt.Sprintf("%s/%s", "https://your-cdn.cloudfront.net", fileName) // Set the distribution name created on CloudFront here

	return OkFeedback("✅ File uploaded successfully to CDN: " + cloudFrontURL), nil
}
```

Here on `GetHeaders` helper function, we'll validate some headers sent to the gateway and get the file information.

```go
package main

import (
	"encoding/base64"
	"fmt"
	"mime"
	"mime/multipart"
	"net/http"
	"strings"

	"github.com/aws/aws-lambda-go/events"
)

func GetHeaders(lambdaReq events.APIGatewayProxyRequest, maxFileSizeBytes int64) ([]*multipart.FileHeader, error) {
	headers := http.Header{}

	for header, values := range lambdaReq.Headers {
		headers.Add(header, values)
	}

	contentType := headers.Get("Content-Type")
	if contentType == "" {
		return nil, fmt.Errorf("Content-Type header not found")
	}

	_, params, err := mime.ParseMediaType(contentType)
	if err != nil {
		return nil, err
	}

	boundary := params["boundary"]
	if boundary == "" {
		return nil, fmt.Errorf("boundary not found in Content-Type header")
	}

	stringReader := strings.NewReader(lambdaReq.Body)
	b64Reader := base64.NewDecoder(base64.StdEncoding, stringReader)
	multipartReader := multipart.NewReader(b64Reader, boundary)

	form, err := multipartReader.ReadForm(maxFileSizeBytes) // Receiving the form data
	if err != nil {
		return nil, err
	}

	var files []*multipart.FileHeader

	for _, currentFileHeaders := range form.File {
		files = append(files, currentFileHeaders...)
	}

	return files, nil
}
```

On the `UploadFile` helper function, we'll create a session on AWS and send the file to the AWS

```go
package main

import (
	"bytes"
	"mime/multipart"

	"github.com/aws/aws-sdk-go/aws"
	"github.com/aws/aws-sdk-go/aws/session"
	"github.com/aws/aws-sdk-go/service/s3/s3manager"
)

type UploadRes struct {
	S3Path string
	S3URL  string
}

func UploadFile(fileHeader *multipart.FileHeader, region string, bucket string) (string, error) {
	file, openError := fileHeader.Open() 

	if openError != nil {
		return "", openError
	}

	var fileContents bytes.Buffer
	_, readError := fileContents.ReadFrom(file)
	if readError != nil {
		return "", readError
	}

	awsSession, sessionError := session.NewSession(&aws.Config{ // Creating a session
		Region: aws.String(region),
	})

	if sessionError != nil {
		return "", sessionError
	}

	uploader := s3manager.NewUploader(awsSession)

	fileName := fileHeader.Filename

	_, uploadError := uploader.Upload(&s3manager.UploadInput{ // Uploading the file to AWS S3
		Bucket: aws.String(bucket),
		Key:    aws.String(fileName),
		Body:   bytes.NewReader(fileContents.Bytes()),
	})

	if uploadError != nil {
		return "", uploadError
	}

	return fileName, nil
}
```

For giving the error and success feedback in a JSON format, we can create a helper function, receiving the message by parameters.

```go
package main

import "github.com/aws/aws-lambda-go/events"

const (
	StatusOK         = 200
	StatusBadRequest = 400
)

func OkFeedback(message string) events.APIGatewayProxyResponse {
	return events.APIGatewayProxyResponse{
		StatusCode: StatusOK,
		Headers: map[string]string{
			"Content-Type": "application/json",
		},
		Body: `{"message": "` + message + `"}`,
	}
}

func BadRequestFeedback(message string) events.APIGatewayProxyResponse {
	return events.APIGatewayProxyResponse{
		StatusCode: StatusBadRequest,
		Headers: map[string]string{
			"Content-Type": "application/json",
		},
		Body: `{"message": "` + message + `"}`,
	}
}
```

Since we're using the Go AWS SDK, run `go mod tidy` to install them. The folder structure created looks like this:

![Folders structure](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/h9j14a292id6435fs7fq.png)

## Running locally
Now, we can run our application, building the binary of the code and starting the application locally with `go build -o yuta ./src/* && sam local start-api`. Since I have said my handler file would be "yuta" on `template.yml`, I used the command to build the binary with the name "yuta". 
Now the server should be running on the terminal, and we're able to send the request to the lambda function now, with a POST request to the route, as a `multipart/form-data`

![SAM running, awaiting the serverless event trigger](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qib9mof6eypirsgobjd1.png)

![Request running successfully after a POST request to the route provided from SAM](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bc5d4je5ckpun3q9bikm.png)

## Uploading the Lambda function to AWS
Now we need to create a `zip` to send our created code to AWS Lambda. Zip the binary code created with `go build -o yuta ./src/*` and let's upload it on Lambda!
On the [Lambda service](https://sa-east-1.console.aws.amazon.com/lambda/home?region=sa-east-1#/), create a function using the Go runtime. On the code source, click on **upload a zip file** and upload the zip of the binary code generated before. Change the **Handler** on **Runtime settings** to the name of the zip file - in this case, it would be "yuta" since it's the name of my zip file.

Now we need a way to trigger this serverless function. In this case, we'll create an [API Gateway](https://aws.amazon.com/api-gateway/?nc1=h_ls) to send the data to the Lambda function and, when a request is sent to this route mapped on API Gateway, it'll activate our function. Click to **add trigger**, select the **API Gateway**, and **create an HTTP API with security open**.

In this way, you will receive an error 403 (access denied), because Lambda doesn't have access to upload files to your S3. Go to **configuration**, **permissions**, and click on the **role name**, to be redirected to the policy created on IAM. There, we'll add a new **permission** as an **inline policy**, selecting the S3 service to give the `PutObject` access.

After that, using the link of our API Gateway, we'll be able to send the file, upload it on S3, and receive the link of the uploaded file with the CloudFront prefix.

![Success on AWS](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yn7rer3zjlxt72ahv9v5.png)

## Next steps
- After each change to our code we need to build, zip, and upload manually again on S3, what about creating a CI/CD pipeline to automatize this for us, like [AWS CodeBuild](https://aws.amazon.com/codebuild/?nc1=h_ls), or [GitHub Actions](https://github.com/features/actions)
- We could configure [CloudWatch](https://aws.amazon.com/cloudwatch/?nc1=h_ls) service to be able to see logs with more details of what's causing errors. On this project, I had some problems after uploading the file, and with the CloudWatch logs, it was very easy to fix them.
- What about dockerizing the application?
- We could use environment variables to make the main information more "dynamic"
- We could add more file validations, like validating the file size, and upload more files at the same time using [goroutine](https://go.dev/tour/concurrency/1) in a folder and return the link of this folder

## Links
- [Code on GitHub](https://github.com/RodrigoRVSN/yuta/commits/chore/code-backup)
- [AWS](https://aws.amazon.com/?nc1=h_ls)

## Conclusion
And that's it! We went through some services of AWS to know a bit more about this platform in practice in a simple way, using services like Lambda function, API Gateway, S3, and CloudFront. 
C u soon! 🎈

![Yuta okkutsu](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/k13hhahssadsziv3h9fh.gif)