Deploy lambda using serverless
Tags: go, programming, aws, serverless
Published: 2022-04-05 15:08:34

*In case it helped :)*
<a href="https://www.buymeacoffee.com/aseemwangoo" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Pass Me A Coffee!!" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>

<!-- wp:paragraph -->
<p>We will cover briefly:</p>
<!-- /wp:paragraph -->

<!-- wp:list {"ordered":true} -->
<ol><li>Options for deploying lambda</li><li>Setup serverless</li><li>Create lambda (upload image to s3 bucket) using serverless</li><li>Deploy lambda using serverless</li></ol>
<!-- /wp:list -->

{% youtube I1T3V9291yo %}

<!-- wp:heading {"level":3} -->
<h3>Options for deploying lambda</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>As of writing this article, there are <strong>two ways</strong> of deploying lambdas onto AWS</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center"} -->
<div class="wp-block-image"><figure class="aligncenter"><img src="https://cdn-images-1.medium.com/max/1600/1*WvQBPtKe5MJ0FGKsvseYIQ.png" alt="Deploy lambda using serverless"/><figcaption>Deploy lambda using serverless</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:list -->
<ul><li><a href="https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/what-is-sam.html" rel="noreferrer noopener" target="_blank"><strong>Using AWS SAM</strong></a></li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>The AWS Serverless Application Model (AWS SAM) is an open-source framework that you can use to build <a href="https://aws.amazon.com/serverless/" rel="noreferrer noopener" target="_blank">serverless applications</a> on AWS</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>One of the key benefits of <strong>SAM </strong>is <strong>Local debugging and testing</strong></p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>It lets you locally build, test, and debug serverless applications that are defined by AWS SAM templates. which helps you catch issues upfront by providing parity with the actual Lambda execution environment</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li><a href="https://www.serverless.com/" rel="noreferrer noopener" target="_blank"><strong>Using serverless</strong></a></li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p>Develop, deploy, troubleshoot, and secure your serverless applications with radically less overhead and cost by using the Serverless Framework. The Serverless Framework consists of an open-source CLI.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p>Serverless allows us to focus on the code, while it takes care of the setting up policies, and provisioning of required infrastructure onto AWS</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Setup serverless</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>To install <code>serverless</code>, follow <a href="https://www.serverless.com/framework/docs/getting-started" rel="noreferrer noopener" target="_blank">here</a>. Personally, I configured using the below</p>
<!-- /wp:paragraph -->

```yaml
npm install -g serverless
```

<!-- wp:list -->
<ul><li>Once installed you can access it by typing <code>sls</code> on your terminal</li><li>We now have access to multiple templates by this command</li></ul>
<!-- /wp:list -->

```yaml
sls create --help
```

<!-- wp:paragraph -->
<p>We will focus on the Go specific one, which is <code>aws-go-dep</code> Let's use this template to create a simple lambda. The final command for that is</p>
<!-- /wp:paragraph -->

```yaml
sls create --template aws-go-dep --path upload-s3
```

<!-- wp:paragraph -->
<p><code>--template</code> is used to specify the template name</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>--path</code> <strong>(Optional)</strong> is used to specify the directory in which your code is to be placed</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>We should see a folder structure like this</li></ul>
<!-- /wp:list -->

<!-- wp:image {"align":"center"} -->
<div class="wp-block-image"><figure class="aligncenter"><img src="https://cdn-images-1.medium.com/max/1600/1*XlUZ6DCr_GLUY4MLBBjs3w.png" alt="Golang serverless folder structure"/><figcaption>Golang serverless folder structure</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:list -->
<ul><li>By default, the <code>aws-go-dep</code> a template gives us two <code>http (GET)</code>based endpoints, which can be seen in the <code>serverless.yml</code></li></ul>
<!-- /wp:list -->

```yaml
functions:
  hello:
    handler: bin/hello
    events:
      - httpApi:
          path: /hello
          method: get
  world:
    handler: bin/world
    events:
      - httpApi:
          path: /world
          method: get
```

<!-- wp:paragraph -->
<p>The equivalent files correspond to <code>hello/main.go</code> and <code>world/main.go</code>&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>In case you see errors, try to run the following commands</li></ul>
<!-- /wp:list -->

```go
go mod init <SOME NAME HERE>
go mod tidy
```

<!-- wp:paragraph -->
<p>This will help in setting the <code>go.mod</code> which is required for any go program</p>
<!-- /wp:paragraph -->

<!-- wp:heading {"level":3} -->
<h3>Create lambda (upload image to s3 bucket) using serverless</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>We will modify our lambda to upload an image to an S3 bucket.</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>We delete the unnecessary endpoints from the <code>serverless.yml</code> and unwanted folders like <code>hello/main.go</code> and <code>world/main.go</code>&nbsp;</li><li>Create a new folder called <code>upload</code> and inside it creates a file <code>main.go</code>&nbsp;</li></ul>
<!-- /wp:list -->

<!-- wp:image {"align":"center"} -->
<div class="wp-block-image"><figure class="aligncenter"><img src="https://cdn-images-1.medium.com/max/1600/1*IcqK4hy3lI9FcxU4beWXiw.png" alt="Our final upload lambda structure"/><figcaption>Our final upload lambda structure</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:list -->
<ul><li>We will define a request body struct that will accept <code>filename</code> and the <code>body</code> as input. (Note: the body would be the base64 encoder of an image)</li></ul>
<!-- /wp:list -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>For getting the base64 of any image, <a href="https://base64.guru/converter/encode/image" rel="noreferrer noopener" target="_blank">check here</a></p></blockquote>
<!-- /wp:quote -->

```go
type ImageRequestBody struct {
 FileName string `json:"filename"`
 Body     string `json:"body"`
}
```

<!-- wp:paragraph -->
<p>We will parse the body from the input and decode it using <code>DecodeString</code> which returns the bytes represented by the base64 string</p>
<!-- /wp:paragraph -->

```go
decoded, err := base64.StdEncoding.DecodeString(bodyRequest.Body)
```

<!-- wp:paragraph -->
<p>All our image-related functions are inside <a href="https://github.com/AseemWangoo/golang-programs/blob/master/upload-s3/upload/structs.go" target="_blank" rel="noreferrer noopener">img_utils.go</a>. After decoding the image into bytes from the previous step, we now take those bytes and form a temporary file</p>
<!-- /wp:paragraph -->

```go
func imageUpload(data []byte) {
  tmpFileName := fmt.Sprintf(`/tmp/%s`, bodyRequest.FileName)
  fileErr := ioutil.WriteFile(tmpFileName, []byte(data), 0644)
  
  // CALL THE UPLOAD FUNCTION
  UploadImage(tmpFileName)
}
```

<!-- wp:paragraph -->
<p>In the first line, we are specifying the file at a location <code>tmp</code>&nbsp;. This is an important step and as per AWS documentation</p>
<!-- /wp:paragraph -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p>The Lambda execution environment provides a file system for your code to use at /<strong>tmp</strong>. This space has a fixed size of 512 MB. Each time a new execution environment is created, this area is deleted.</p></blockquote>
<!-- /wp:quote -->

<!-- wp:paragraph -->
<p>Next, we write the file with the bytes using <code>WriteFile</code>&nbsp;</p>
<!-- /wp:paragraph -->

<!-- wp:quote -->
<blockquote class="wp-block-quote"><p><strong>WriteFile</strong> writes data to a file named by filename. If the file does not exist, WriteFile creates it with permissions perm (before umask); otherwise WriteFile truncates it before writing, without changing permissions.</p></blockquote>
<!-- /wp:quote -->

<!-- wp:paragraph -->
<p>So now we have our temporary file created!</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Now we will upload the file, onto the AWS S3 bucket. For that, we will be using the <a href="https://github.com/aws/aws-sdk-go" rel="noreferrer noopener" target="_blank">AWS GO SDK</a> and in case you don’t have it installed follow this</li></ul>
<!-- /wp:list -->

```go
go get github.com/aws/aws-sdk-go
```

<!-- wp:paragraph -->
<p>We will pass our file to the <a href="https://github.com/AseemWangoo/golang-programs/blob/master/upload-s3/upload/upload_s3.go" rel="noreferrer noopener" target="_blank">upload function</a></p>
<!-- /wp:paragraph -->

```go
func uploadToS3Bucket(file io.Reader, fileName string) {
 bucketName := os.Getenv("bucket_name")
 region := "us-east-2"
 conf := aws.Config{Region: &region}
 sess, _ := session.NewSession(&conf)
 uploader := s3manager.NewUploader(sess)
 upParams := &s3manager.UploadInput{
   Bucket: &bucketName,
   Key:    &fileName,
   Body:   file,
 }
 result, err := uploader.Upload(upParams)
}
```

<!-- wp:list -->
<ul><li>We need to specify the region using <code>aws.Config</code>&nbsp;. In our case it was us-east-2.</li></ul>
<!-- /wp:list -->

<!-- wp:list -->
<ul><li>We need to specify the region using <code>aws.Config</code>&nbsp;. In our case it was us-east-2.</li></ul>
<!-- /wp:list -->

<!-- wp:paragraph -->
<p><code>session.Newsession</code> returns a new Session created from SDK defaults, config files, environment, and user-provided config files.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>aws-sdk-go</code> gives us <code>s3Manager</code> which helps in uploading the file. It takes in</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>Bucket</code>&nbsp;: Your AWS bucket name, in my case I get from <code>serverless.yml </code>environment variable</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>Key</code>&nbsp;: Object key for which the PUT action was initiated. In our case, it's the filename.</p>
<!-- /wp:paragraph -->

<!-- wp:paragraph -->
<p><code>Body</code>&nbsp;: What is the body payload to send to S3. In our case, it's the file itself</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Finally, using the above parameters, we call the <code>Uploader.Upload</code>&nbsp;</li><li>We return the response from the <code>main.go</code> back to the caller</li></ul>
<!-- /wp:list -->

<!-- wp:heading {"level":3} -->
<h3>Deploy lambda using serverless</h3>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>Till now we were creating our lambda, but now it's time to deploy it onto AWS.</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Let’s revisit our <code>serverless.yml</code> and make some changes</li></ul>
<!-- /wp:list -->

```go
functions:
  upload:
    handler: bin/upload
    events:
      - httpApi:
          path: /uploadImage
          method: post
```

<!-- wp:paragraph -->
<p>We add in the post endpoint <code>uploadImage</code> for our lambda</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Next, since we are using the S3 bucket from inside lambda, we will add some IAMRoleStatements</li></ul>
<!-- /wp:list -->

```yaml
custom:
  s3Bucket: test-bucket--aseem
  
iamRoleStatements:
    - Effect: Allow
      Action:
        - s3:*
      Resource: "arn:aws:s3:::${self:custom.s3Bucket}/*"
```

<!-- wp:paragraph -->
<p>This specifies the bucket name using <code>s3Bucket</code> and allows executing the operations on this bucket</p>
<!-- /wp:paragraph -->

<!-- wp:list -->
<ul><li>Finally, for specifying the environment variables, we make use of</li></ul>
<!-- /wp:list -->

```yaml
environment:
    bucket_name: test-bucket--aseem
```

<!-- wp:paragraph -->
<p>We will create a <code><a href="https://github.com/AseemWangoo/golang-programs/blob/master/upload-s3/Makefile" target="_blank" rel="noreferrer noopener">Makefile</a></code> and put the following</p>
<!-- /wp:paragraph -->

```yaml
deploy: clean build
 sls deploy --verbose
remove:
 sls remove
```

<!-- wp:paragraph -->
<p>This will help in easing the lambda deployment, by simply calling</p>
<!-- /wp:paragraph -->

```yaml
make deploy
```

<!-- wp:paragraph -->
<p>If everything was successful, you should see something like this</p>
<!-- /wp:paragraph -->

<!-- wp:image {"align":"center"} -->
<div class="wp-block-image"><figure class="aligncenter"><img src="https://cdn-images-1.medium.com/max/1600/1*2A91zXbPpcGTtomCRq4hnA.png" alt="Deploy lambda using serverless"/><figcaption>Deploy lambda using serverless</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p>For removing the lambda, we can call</p>
<!-- /wp:paragraph -->

```yaml
make remove
```

<!-- wp:paragraph -->
<p>For testing the lambda, use the</p>
<!-- /wp:paragraph -->

```bash
curl -X POST -H "Content-Type: application/json" \
-d '{"filename": "xyz.png", "body": "BASE_64_ENCODING_OF_AN_IMAGE"}' \
<YOUR_LAMBDA_URL>
```

<!-- wp:image {"align":"center"} -->
<div class="wp-block-image"><figure class="aligncenter"><img src="https://cdn-images-1.medium.com/max/1600/1*PuQQSVHG8Dd-GJTPivTSPw.png" alt="Image uploaded to S3"/><figcaption>Image uploaded to&nbsp;S3</figcaption></figure></div>
<!-- /wp:image -->

<!-- wp:paragraph -->
<p><a rel="noreferrer noopener" href="https://github.com/AseemWangoo/golang-programs" target="_blank"><em>Source code.</em></a>

{% youtube I1T3V9291yo %}

*In case it helped :)*
<a href="https://www.buymeacoffee.com/aseemwangoo" target="_blank"><img src="https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png" alt="Pass Me A Coffee!!" style="height: 41px !important;width: 174px !important;box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;-webkit-box-shadow: 0px 3px 2px 0px rgba(190, 190, 190, 0.5) !important;" ></a>