Build a Serverless GenAI solution with Lambda, DynamoDB, LangChain and Amazon Bedrock
Tags: machinelearning, serverless, cloud, go
Published: 2024-01-18 12:00:35

*DynamoDB is used as the chat history backend along with AWS Lambda Web adapter for response streaming*

In a [previous blog](https://community.aws/content/2aq9ju6xvYtywGVbuPoWFTk5oK4/build-a-streamlit-app-using-langchain-amazon-bedrock-and-redis), I demonstrated how to Redis (Elasticache Serverless as an example) as a chat history backend for a Streamlit app using LangChain. It was deployed to EKS and also make use of EKS Pod Identity to manage application `Pod` permissions for invoking Amazon Bedrock. 

This use-case here is a similar one - a chat application. I will switch back to implementing things in Go using [langchaingo](https://github.com/tmc/langchaingo) (I used Python for the previous one) and continue to use Amazon Bedrock. But there are few unique things you can explore in this blog post:

- The chat application is deployed as an AWS Lambda function along with a [Function URL](https://docs.aws.amazon.com/lambda/latest/dg/lambda-urls.html).
- It uses DynamoDB as the chat history store (aka [Memory](https://python.langchain.com/docs/modules/memory/)) for each conversation - [I extended langchaingo](https://github.com/abhirockzz/langchaingo-dynamodb-chat-history) to include this feature.
- Thanks to the [AWS Lambda Web Adapter](https://github.com/awslabs/aws-lambda-web-adapter), the application built as a (good old) REST/HTTP API using a familiar library (in this case, [Gin](https://gin-gonic.com/). 
- And the other nice add-on was to be able to combine [Lambda Web Adapter streaming response](https://aws.amazon.com/blogs/compute/introducing-aws-lambda-response-streaming/) feature with [Amazon Bedrock streaming inference API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_runtime_InvokeModelWithResponseStream.html).

As always, a diagram always helps.....

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/osbq47jmjbrzffw5idkv.png)

## Deploy using SAM CLI (Serverless Application Model)

> Make sure you have [Amazon Bedrock prerequisites](https://community.aws/concepts/amazon-bedrock-golang-getting-started#before-you-begin) taken care of and the [SAM CLI installed](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html)

```shell
git clone https://github.com/abhirockzz/chatbot-bedrock-dynamodb-lambda-langchain
cd chatbot-bedrock-dynamodb-lambda-langchain
```

Run the following commands to build the function and deploy the entire app infrastructure (including the Lambda Function, DynamoDB, etc.)

```shell
sam build 
sam deploy -g
```

Once deployed, you should see the Lambda Function URL in your terminal. Open it in a web browser and start conversing with the chatbot!

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/s1y3f0zpndx17xpq0o6k.png)

Inspect the DynamoDB table to verify that the conversations are being stored (each conversation will end up being a new item in the table with a unique `chat_id`):

```
aws dynamodb scan --table-name langchain_chat_history
```

> `Scan` operation is used for demonstration purposes. Using `Scan` in production is not recommended.

## Quick peek at the good stuff....

- **Using DynamoDB as the backend store history:** [Refer to the GitHub repository](https://github.com/abhirockzz/langchaingo-dynamodb-chat-history) if you are interested in the implementation. To summarize, I implemented the required functions of the [schema.ChatMessageHistory](https://pkg.go.dev/github.com/tmc/langchaingo/schema#ChatMessageHistory).
- **Lambda Web Adapter Streaming response + LangChain Streaming:** I used the [chains.WithStreamingFunc](https://pkg.go.dev/github.com/tmc/langchaingo/chains#WithStreamingFunc) option with [chains.Call](https://pkg.go.dev/github.com/tmc/langchaingo/chains#Call) call and then let Gin [Stream](https://pkg.go.dev/github.com/gin-gonic/gin#Context.Stream) do the heavy-lifting of handling the streaming response.

Here is a sneak peek of the implementation ([refer to the complete code here](https://github.com/abhirockzz/chatbot-bedrock-dynamodb-lambda-langchain/blob/master/main.go#L98)):

```go
		_, err = chains.Call(c.Request.Context(), chain, map[string]any{"human_input": message}, chains.WithMaxTokens(8191), chains.WithStreamingFunc(func(ctx context.Context, chunk []byte) error {

			c.Stream(func(w io.Writer) bool {
				fmt.Fprintf(w, (string(chunk)))
				return false
			})

			return nil
		}))
```

## Closing thoughts...

I really like the extensibility of LangChain. While I understand that `langchaingo` may not be as popular as the original python version (I hope it will reach there in due time 🤞), but it's nice to be able to use it as a foundation and build extensions as required. Previously, I had written about how to use the [AWS Lambda Go Proxy API](https://github.com/awslabs/aws-lambda-go-api-proxy) to [run existing Go applications on AWS Lambda](https://community.aws/tutorials/golang-and-aws-lambda/01-golang-apis-on-aws-lambda). The AWS Lambda Web Adapter offers similar functionality but it has lots of other benefits, including response streaming and the fact that it is language agnostic.

Oh, and one more thing - I also tried a different approach to building this solution using the [API Gateway WebSocket](https://docs.aws.amazon.com/apigateway/latest/developerguide/apigateway-websocket-api.html). Let me know if you're interested, and I would be happy to write it up!

If you want to explore how to use GO for Generative AI solutions, you can read up on some of my earlier blogs:

- [Building LangChain applications with Amazon Bedrock and Go - An introduction](https://community.aws/posts/amazon-bedrock-langchaingo-llm)
- [Serverless Image Generation Application Using Generative AI on AWS](https://community.aws/tutorials/amazon-bedrock-lambda-image-gen-golang)
- [Generative AI Apps With Amazon Bedrock: Getting Started for Go Developers](https://community.aws/concepts/amazon-bedrock-golang-getting-started)
- [Use Amazon Bedrock and LangChain to build an application to chat with web pages](https://community.aws/posts/amazon-bedrock-apprunner-chatterdox-webapp)

Happy building!