A Lesson Learned In Going Serverless
Tags: serverless, aws, lambda, apigateway
Published: 2018-12-04 14:44:42

---
title: A Lesson Learned In Going Serverless
published: true
description: Demonstrates how to handle multipart form-data POST requests with the API Gateway and AWS Lambda
tags: serverless,aws,lambda,api gateway
cover_image: https://thepracticaldev.s3.amazonaws.com/i/xvcgeaxibhtn0x9yjeb3.png
canonical_url: https://dev.to/mhausenblas/2018-lambda-gateway-post-binary-form-data
---

When I was working on the [serverless variant of imgn](https://github.com/mhausenblas/imgn/tree/master/functions), a simple image sharing demo app, I ran into an issue around handling multipart form-data `POST` requests with the API Gateway and AWS Lambda. This post shows how I solved it.

The `imgn` app is really very simple: it allows you to upload image files and then you can view them in a public gallery. In the background, it extracts some image metadata (just the dimensions, for now) and displays it along with the images: 

![imgn app](https://raw.githubusercontent.com/mhausenblas/imgn/master/img/monolith_gallery-with-images.png)

During the development of the serverless variant of `imgn` I ran into an issue in the context of uploading the image to S3 using a Lambda function via the API Gateway. You can read up on the issue description in greater detail on [StackOverflow](https://stackoverflow.com/questions/53537939/posting-image-using-form-data-via-api-gateway-to-lambda-function-results-in-inva), but the essence is: the payload that the API Gateway hands over to the Lambda function gets somehow butchered, so the images end up being corrupted when they land in the respective S3 bucket. 

To narrow down where this SNAFU happens, I first replaced my initial code that would create a `http.request` and use the [ParseForm](https://golang.org/pkg/net/http/#Request.ParseForm) method to get to the image data with my own implementation of [parsing the multipart form-data](https://github.com/mhausenblas/imgn/blob/42912b2334400f969d08481b3d5341042211c31a/functions/app/uploadimg/main.go#L31). Still, the corruption was the same.

Next, in  order to exclude the "upload to the S3 bucket" part as the culprit for the data corruption, I returned the parsed image data I got from the API Gateway as a result of the Lambda call. This allowed me to use `hexdump` to compare the resulting file with the original, but still no bueno. The problem persisted.

![CloudWatch logs of the UploadImageFunction](https://i.stack.imgur.com/Pdfjl.png)

The only thing left in the request path was the handover of the (binary) image data from the API Gateway to the Lambda function. 

After trawling StackOverflow and reading many seemingly relevant posts in AWS fora, I decided it was time for a different approach. I read up on [pre-signed URLs](https://docs.aws.amazon.com/sdk-for-go/v1/developer-guide/s3-example-presigned-urls.html) and decided to give this a try. These pre-signed URLs enable you to allow people to manipulate objects in S3 buckets and you can set restrictions in terms of what operation, on which object, and for how long you allow it to happen.

So, the idea would be that when a user issues a multipart form-data `POST` request via the UI, the `UploadImageFunction` would not directly upload the image to S3 but create a [pre-signed URL](https://github.com/mhausenblas/imgn/blob/master/functions/app/uploadimg/main.go#L118-L126) for a `PUT` request into said S3 bucket, valid for the file referenced in the previous form-data `POST` request and restricted to 5 minutes:

```go
req, _ := s3c.PutObjectRequest(&s3.PutObjectInput{
		Bucket: aws.String(gallerybucket),
		Key:    aws.String(imgname),
})
presurl, err := req.Presign(5 * time.Minute)
```

… and the [JavaScript code](https://github.com/mhausenblas/imgn/blob/42912b2334400f969d08481b3d5341042211c31a/functions/ui/upload.js#L31) in the UI would then issue a second `PUT` request, using the pre-signed URL to directly upload the image into the S3 bucket. And that works like a charm!

I'm still unsure if I should qualify this strategy as a hack or a good practice and would certainly appreciate to hear from other serverless practitioners what they think.