Generative AI With Amazon Bedrock and Unity3D
Tags: aws, bedrock, unity3d, ai
Published: 2023-10-06 23:12:18

A few days ago, Amazon made a big announcement: the service **Bedrock** was finally released.

If you missed the news, here it is: Amazon decided to enter massively into the generative AI field and invested in the company [Anthropic for $4 billion](https://www.cnbc.com/2023/09/25/amazon-to-invest-up-to-4-billion-in-anthropic-a-rival-to-chatgpt-developer-openai.html) in order to compete with popular AI services like ChatGPT, Dall-E, MidJourney, etc. The first proposal of Amazon is Bedrock, a fully managed AI service that offers multiple foundational models such as Claude, Jurassic, and Stable Diffusion XL, among others.

In my previous articles, I created Unity3D applications communicating with [ChatGPT](https://betterprogramming.pub/i-made-chatgpt-talk-using-unity3d-and-aws-b0e83e6cce66) and [Dall-E](https://medium.com/better-programming/generating-ai-images-with-dall-e-aws-and-unity3d-33e9b77a2b5d). Let's do the same with Bedrock!

Do you prefer something more interactive instead of reading? Check my video!

{% embed https://youtu.be/kzKxuGhdeZM %}

## General Architecture

Here is the general architecture of our application:

![Architecture](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nr16hen5igebbofsmf0k.png)

We use 3 Amazon services:
- API Gateway to create, expose, and consume endpoints.
- Lambda to receive the client's requests and communicate with Bedrock.
- Bedrock to generate answers based on the client's prompts.

If you want to get further adding a login mechanism with Cognito, please refer to my previous article: [Connecting Unity3D, AWS, and ChatGPT Together](https://betterprogramming.pub/i-made-chatgpt-talk-using-unity3d-and-aws-b0e83e6cce66).

## Bedrock

First, we have to enable the foundational models we want to use. By default, all the models are disabled. Go to the Bedrock console and enable the models you will use in the model access section. If you have an updated payment method, activating the models only takes a few minutes.

![Model access section in the Bedrock console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/qyfnvgqwhn4oj3gf18ep.png)

If you don't enable the models in the Bedrock console, you will receive this error in Lambda:

```
An error occurred (AccessDeniedException) when calling the InvokeModel operation: Your account is not authorized to invoke this API operation.
```

## Lambda, Boto3 and Bedrock

If you follow my work, you know I'm a Python guy, so I will use the Python SDK for AWS, [Boto3](url). Good news: [Bedrock is available](https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/bedrock.html) on Boto3; bad news: Lambda runs an old version of Boto3 in which Bedrock does not exist yet.

💡 **Here is a workaround:** creating a new Lambda layer with an up-to-date version of Boto3. This is how to do it:

1) In PyCharm, create a new project and install boto3 in a new virtual environment. Boto3 and all the dependencies will be installed.

![PyCharm project](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ny9ettalckocrk7qetf8.png)

2) Go to the venv folder of your project and copy the lib folder into a new folder called python. Then, zip it.

![venv folder](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/du3tbjsujt2wougzjsz0.png)

3) Go to the Layers section of the Lambda console and create a new layer. Upload the zip file we have created before and choose the same Python version as your local Python virtual environment for the runtime.

![Creating a new Layer](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/svddls15zr8damnwkkxh.png)

## The Lambda Functions

We create 2 Lambda functions, one for text generation and the other one for image generation, both with the same version as previously for the runtime (In my case, Python 3.9), and we add the layer we have created before.

![Adding a new layer to the Lambda function](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/bqmc3wjnj2i8gm6k21s0.png)

### The text generation function

For the text generation, I use the Claude model and implement it as described in the [Amazon documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/api-methods-run-inference.html).

{% embed https://gist.github.com/alexandre-bruffa/c876fe69e00b70b9830863e8c7f06eb2 %}

### The image generation function

For the image generation, I use the Stable Diffusion XL model and implement it, as described in the [Amazon documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/api-methods-run-inference.html).

{% embed https://gist.github.com/alexandre-bruffa/537e19a9ab05748ac77b4a10370cc20b %}

Note that the model IDs can be found in the [Bedrock documentation](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids-arns.html).

### The Lambda Policy

In order to allow Lambda to invoke a Bedrock model, we create a new managed policy and add it to the Lambda functions roles:

![Policy](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/kcmkzruiqjigl2p1szk1.png)

## API Gateway

In the API Gateway console, we create a new REST API with 2 resources: `image` and `text`. Each resource has a post method integrating with the Lambda functions we have created before.

![API resources](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0q5wtwcaz167plptac14.png)

When deploying the API, you will be asked to create a new stage. After the creation of the stage, the final endpoint will be shown. You will have to use this URL in the Unity client to perform the request.

![API stages](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e4mpmk791xaylqkxcxjp.png)

## The Unity Client

The Unity client allows the user to send a message to generate a text or an image as an answer:

![Unity client](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/e6dhwukq9il4vddybdsv.png)

### Handling the request

First, we need to create serializable classes to embed the sending and receiving data:

{% embed https://gist.github.com/alexandre-bruffa/44e8cdebd4b8e7f3c301e983b4c31c9d %}

Sending a request in Unity is an asynchronous process. Therefore, we create a coroutine that sends the request to the server:

{% embed https://gist.github.com/alexandre-bruffa/7d7221c26186168f98d8b9fb8730b896 %}

### Dealing with the generated image

If we ask for an image generation, the result sent by Bedrock is a `base64` image. To show it on the Unity UI, we must convert it into bytes and create a `Texture2D`. Then, we assign the texture to the `RawImage` component of the UI.

{% embed https://gist.github.com/alexandre-bruffa/32e5153ad0af85fc2520ca4b3da86d6b %}

## Result

Here is the result with text and image generations:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/virl8upuljl5zdqv5jjb.gif)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/4yxit9v75yz00hvpzv4k.gif)

## Costs

Let's check with the [AWS Calculator](https://calculator.aws/#/) and the [Bedrock pricing sheet](https://aws.amazon.com/bedrock/pricing/) what our system cost would be for a very pessimistic scenario: you love the app, and you perform **2,000 requests a month**. 1,000 for text generation with the Claude model and 1,000 for image generation with Stable Diffusion XL.

- **API Gateway**: With 2,000 requests to our REST API, the monthly cost is **0,01 USD**
- **Lambda**: With 2,000 requests with an average time of 5 seconds and 512 MB of memory allocation, the monthly cost is **0.08 USD**.
- **Bedrock**: 1,000 text generations by month with a maximum of 300 output tokens and 20 input tokens per request would result, in the worst case, in **$0.2204 + $9.804**. 1,000 image generations would give us a bill of **$18**.

**Total**: The total bill for our system would be approximately **28 USD** monthly. It's quite affordable!

## Final Thoughts

Using Bedrock is very intuitive, and integrating it with other Amazon services is absolutely straightforward. The fact that Bedrock is an in-house product makes the integration easier: there is no need for API keys or external requests to make it work, resulting in greater performance and security.

Every code of this article has been tested using Unity 2021.3.3 and Visual Studio Community 2022 for Mac. The mobile device I used to run the Unity app is a Galaxy Tab A7 Lite with Android 11.

Thanks for reading until the end!

## Resources

Here is the resource mentioned in the article:

- The [Unity package](https://drive.google.com/file/d/1QWbbHsmMrHi9gKolXfRmwclMdK1Sg4Qm/view) of the application
- The [text generation function](https://gist.github.com/alexandre-bruffa/c876fe69e00b70b9830863e8c7f06eb2) in Python
- The [image generation function](https://gist.github.com/alexandre-bruffa/537e19a9ab05748ac77b4a10370cc20b) in Python