Build a Serverless Application for Image Label Detection
Tags: serverless, machinelearning, tutorial, go
Published: 2023-06-17 12:26:52

In this blog post, you will learn how to build a Serverless solution to process images using [Amazon Rekognition](https://docs.aws.amazon.com/rekognition/latest/dg/what-is.html), [AWS Lambda](https://aws.amazon.com/lambda/) and the [Go](https://go.dev/) programming language. Images uploaded to Amazon Simple Storage Service (S3) will trigger a Lambda function which will detect labels ([using the AWS Go SDK](https://pkg.go.dev/github.com/aws/aws-sdk-go-v2/service/rekognition)) and persist the image label data to an Amazon DynamoDB table.

You will be using the [Go](https://go.dev/) programming language for the business logic (thanks to [aws-lambda-go](https://github.com/aws/aws-lambda-go) library) as well as the infrastructure component ([Go bindings for AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/work-with-cdk-go.html)) to deploy the solution.

> The code is available on [GitHub](https://github.com/abhirockzz/ai-ml-golang-rekognition-label-detection)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/iz0c7pb53ycdsxs64128.jpg)

**What's covered?**

- [Introduction](#introduction)
- [Pre-requisites](#pre-requisites)
- [Use AWS CDK to deploy the solution](#use-aws-cdk-to-deploy-the-solution)
- [Extract labels from image](#extract-labels-from-image)
	- [Don't forget to clean up](#dont-forget-to-clean-up)
- [Code walk through](#code-walk-through)
- [Conclusion and next steps](#conclusion-and-next-steps)

## Introduction

Amazon Rekognition is a service that lets you analyze images and videos in your applications. You can identify objects, people, text, scenes, and activities, and detect inappropriate content. You can also do facial analysis, face comparison, and face search for various use cases like user verification and public safety. Amazon Rekognition is built on deep learning technology that doesn't require machine learning expertise to use. It has an easy-to-use API that can analyze any image or video file in Amazon S3.

Common use cases for using Amazon Rekognition include:

- Making images and videos searchable - Discover objects and scenes that appear within them.
- Face-based user verification - Confirm user identities by comparing their live image with a reference image.
- Sentiment and demographic analysis - Interpret emotional expressions such as happy, sad, or surprise, and demographic information.
- Facial search - Search images, stored videos, and streaming videos for faces that match those stored in a container known as a face collection.
- Unsafe content detection - Detect adult and violent content in images and in stored videos and use the returned metadata to filter inappropriate content based on business needs.
Text detection - Used for visual search, cataloging, and identifying vehicles based on license plate numbers

Let's learn Amazon Rekognition with a hands-on tutorial.

## Pre-requisites

Before you proceed, make sure you have the following installed: 

- [Go programming language](https://go.dev/dl/) (**v1.18** or higher)
- [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html#getting_started_install)
- [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)

Clone the project and change to the right directory:

```bash
git clone https://github.com/abhirockzz/ai-ml-golang-rekognition-label-detection

cd ai-ml-golang-rekognition-label-detection
```

## Use AWS CDK to deploy the solution

The AWS Cloud Development Kit (AWS CDK) is a framework that lets you define your cloud infrastructure as code in one of its supported programming and provision it through [AWS CloudFormation](https://aws.amazon.com/cloudformation/).

To start the deployment, simply invoke `cdk deploy` and wait for a bit. You will see a list of resources that will be created and will need to provide your confirmation to proceed.

```bash
cd cdk

cdk deploy

# output

Bundling asset RekognitionLabelDetectionGolangStack/rekognition-function/Code/Stage...

✨  Synthesis time: 5.44

//.... omitted

Do you wish to deploy these changes (y/n)? y
```

Enter `y` to start creating the AWS resources required for the application.

> If you want to see the AWS CloudFormation template which will be used behind the scenes, run `cdk synth` and check the `cdk.out` folder

You can keep track of the stack creation progress in the terminal or navigate to AWS console: `CloudFormation > Stacks > RekognitionLabelDetectionGolangStack`. 

Once the stack creation is complete, you should have:

- An S3 bucket - Source bucket to upload images.
- A Lambda function to extract image labels using Amazon Rekognition.
- A `DyanmoDB` table to store the label data for each image.
- .... along with a few other components (like IAM roles etc.)

You will also see the following output in the terminal (resource names will differ in your case). In this case, these are the names of the S3 buckets created by CDK:

```bash
 ✅  RekognitionLabelDetectionGolangStack

✨  Deployment time: 119.56s

Outputs:
RekognitionLabelDetectionGolangStack.abeldetectionoutputtablename = rekognitionlabeldetectio-labeldetectioninputbucke-v3vn9o06q3kb_labels_output
RekognitionLabelDetectionGolangStack.labeldetectioninputbucketname = rekognitionlabeldetectio-labeldetectioninputbucke-v3vn9o06q3kb
.....
```

You can now try out the end to end solution!

## Extract labels from image

To try the solution, you can either use an image of your own or use the sample files provided in the [GitHub repository](https://github.com/abhirockzz/ai-ml-golang-rekognition-label-detection). I will be used the S3 CLI to upload the file, but you can [use the AWS console](https://docs.aws.amazon.com/AmazonS3/latest/userguide/upload-objects.html) as well.

```bash
export SOURCE_BUCKET=<enter source S3 bucket name - check the CDK output>

aws s3 cp ./car.png s3://$SOURCE_BUCKET

# verify that the file was uploaded
aws s3 ls s3://$SOURCE_BUCKET
```

This Lambda function will extract labels from the image and store them in a DynamoDB table. 

Upload another file:

```bash
export SOURCE_BUCKET=<enter source S3 bucket name - check the CDK output>

aws s3 cp ./skate.png s3://$SOURCE_BUCKET
```

Check the DynamoDB table in the AWS console - you should see results of the label detection for both the images.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/hx6qyc162ketjzhb6yki.jpg)

`DynamoDB` table is designed with source file name as the partition key and (detected) label name as the sort key. This allows for a couple of query patterns:

- You can get all the labels for a given image.
- You can query for the metadata (category and confidence) for a specific source image and it's label.

You can also use the CLI to scan the table:

```bash
aws dynamodb scan --table-name <enter table name - check the CDK output>
```

### Don't forget to clean up

Once you're done, to delete all the services, simply use:

```bash
cdk destroy

#output prompt (choose 'y' to continue)

Are you sure you want to delete: RekognitionLabelDetectionGolangStack (y/n)?
```

You were able to setup and try the complete solution. Before we wrap up, let's quickly walk through some of important parts of the code to get a better understanding of what's going on behind the scenes.

## Code walk through

We will only focus on the important parts - some of the code has been omitted for brevity.

**CDK**

> You can refer to the complete [CDK code here](https://github.com/abhirockzz/ai-ml-golang-rekognition-label-detection/blob/master/cdk/cdk.go)

```go
bucket := awss3.NewBucket(stack, jsii.String("label-detection-input-bucket"), &awss3.BucketProps{
		BlockPublicAccess: awss3.BlockPublicAccess_BLOCK_ALL(),
		RemovalPolicy:     awscdk.RemovalPolicy_DESTROY,
		AutoDeleteObjects: jsii.Bool(true),
	})
```

We start by creating the source S3 bucket.

```go
	table := awsdynamodb.NewTable(stack, jsii.String("label-detection-output-table"),
		&awsdynamodb.TableProps{
			PartitionKey: &awsdynamodb.Attribute{
				Name: jsii.String("source_file"),
				Type: awsdynamodb.AttributeType_STRING},
			SortKey: &awsdynamodb.Attribute{
				Name: jsii.String("label_name"),
				Type: awsdynamodb.AttributeType_STRING},
			TableName: jsii.String(*bucket.BucketName() + "_labels_output"),
		})
```

Then, we create a DynamoDB table to store the label data for each image.

```go
function := awscdklambdagoalpha.NewGoFunction(stack, jsii.String("rekognition-function"),
		&awscdklambdagoalpha.GoFunctionProps{
			Runtime:     awslambda.Runtime_GO_1_X(),
			Environment: &map[string]*string{"TABLE_NAME": table.TableName()},
			Entry:       jsii.String(functionDir),
		})

	table.GrantWriteData(function)
	function.Role().AddManagedPolicy(awsiam.ManagedPolicy_FromAwsManagedPolicyName(jsii.String("AmazonRekognitionReadOnlyAccess")))
	bucket.GrantRead(function, "*")
```

Next, we create the Lambda function, passing the `DynamoDB` table name as an environment variable to the function. We also grant the function access to the `DynamoDB` table and the S3 bucket. We also grant the function access to the `AmazonRekognitionReadOnlyAccess` managed policy.


```go
function.AddEventSource(awslambdaeventsources.NewS3EventSource(sourceBucket, &awslambdaeventsources.S3EventSourceProps{
		Events: &[]awss3.EventType{awss3.EventType_OBJECT_CREATED},
	}))
```

We add an event source to the Lambda function to trigger it when a new file is uploaded to the source bucket.

```go
	awscdk.NewCfnOutput(stack, jsii.String("label-detection-input-bucket-name"),
		&awscdk.CfnOutputProps{
			ExportName: jsii.String("label-detection-input-bucket-name"),
			Value:      bucket.BucketName()})

	awscdk.NewCfnOutput(stack, jsii.String("label-detection-output-table-name"),
		&awscdk.CfnOutputProps{
			ExportName: jsii.String("label-detection-output-table-name"),
			Value:      table.TableName()})
```

Finally, we export the bucket and DynamoDB table names as CloudFormation output. 

**Lambda function**

> You can refer to the complete [Lambda Function code here](https://github.com/ai-ml-golang-rekognition-label-detection/blob/master/function/main.go)

```go
func handler(ctx context.Context, s3Event events.S3Event) {
	for _, record := range s3Event.Records {

		sourceBucketName := record.S3.Bucket.Name
		fileName := record.S3.Object.Key

		err := labelDetection(sourceBucketName, fileName)
	}
}
```

The Lambda function is triggered when a new image is uploaded to the source bucket. The function iterates through the list of files and calls the `labelDetection` function for each image.

Let's go through it.

```go
func labelDetection(sourceBucketName, fileName string) error {

	resp, err := rekognitionClient.DetectLabels(context.Background(), &rekognition.DetectLabelsInput{
		Image: &types.Image{
			S3Object: &types.S3Object{
				Bucket: aws.String(sourceBucketName),
				Name:   aws.String(fileName),
			},
		},
	})

	for _, label := range resp.Labels {
		item := make(map[string]ddbTypes.AttributeValue)

		item["source_file"] = &ddbTypes.AttributeValueMemberS{Value: fileName}
		item["label_name"] = &ddbTypes.AttributeValueMemberS{Value: *label.Name}
		item["label_category"] = &ddbTypes.AttributeValueMemberS{Value: *label.Categories[0].Name}
		item["label_confidence"] = &ddbTypes.AttributeValueMemberN{Value: fmt.Sprintf("%v", aws.ToFloat32(label.Confidence))}

		_, err := dynamodbClient.PutItem(context.Background(), &dynamodb.PutItemInput{
			TableName: aws.String(table),
			Item:      item,
		})
	}

	return nil
}
```

- The `labelDetection` function uses the `DetectLabels` API to detect labels in the image. 
- The API returns a list of labels, each with a confidence score. 
- The function iterates through the list of labels and stores the label name, category and confidence score in the DynamoDB table.

## Conclusion and next steps

In this post, you saw how to create a serverless solution that detect image labels using Amazon Rekognition. The entire infrastructure life-cycle was automated using AWS CDK. All this was done using the Go programming language, which is well supported in [AWS Lambda](https://pkg.go.dev/github.com/aws/aws-lambda-go) and [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/work-with-cdk-go.html).

Here are a few things you can try out to extend this solution: 

- Build a solution to [analyze videos](https://docs.aws.amazon.com/rekognition/latest/dg/video.html) stored in a S3 bucket.
- Even better, try [processing streaming video](https://docs.aws.amazon.com/rekognition/latest/dg/streaming-video.html) to detect faces, objects etc.

Happy building!