Build Serverless Generative AI API Service with AWS Lambda and Amazon Bedrock
Tags: aws, serverless, generativeai, node
Published: 2023-10-02 10:38:03

In this article, we'll guide you through the process of constructing a Generative AI API Service using AWS Serverless Application Model (SAM). We'll then demonstrate how to deploy it to AWS Lambda, leveraging the robust capabilities of Amazon Bedrock for a powerful and scalable solution.

We will create an API endpoint like this
```sh
https://hwnfnespxzpw7pkz2ydjuwskpq0fokxv.lambda-url.ap-southeast-1.on.aws/
```
and then send POST request to that endpoint with our prompt question `explain to me about AWS Lambda?` as payload
```sh
curl -k \
> -X POST \
> -H 'Content-Type: application/json' \
> -d '{"prompt":"explain to me about AWS Lambda"}' \
> https://hwnfnespxzpw7pkz2ydjuwskpq0fokxv.lambda-url.ap-southeast-1.on.aws/
```

Then get a `response` like this generated by Amazon Bedrock
```json
{"message":"\nAWS Lambda is a serverless computing service provided by Amazon Web Services (AWS). It allows users to run code in response to events, without the need for them to provision or manage servers. With Lambda, you can create and run code without thinking about servers.\n\nLambda is designed to run code in response to events, such as a file in an Amazon Simple Storage Service (Amazon S3) bucket being modified, or a table being updated in an Amazon DynamoDB table. Lambda automatically manages the computing resources required to execute the functions, and scales them as needed.\n\nTo use Lambda, you first create a function by writing code in Node.js, Java, Python, or Ruby. You can then configure triggers that invoke the function when an event occurs. Lambda also allows you to configure how much memory and time the function should use when it is invoked.\n\nLambda functions can run for a maximum of 15 minutes, and are charged based on a pricing model that calculates the amount of time and resources consumed by the function when it is invoked. This means that you only pay for the actual compute time used by your function, and not for idle time.\n\nOverall, Lambda is a powerful tool that allows you to run code in the cloud without having to worry about the underlying infrastructure."}
```

## Follow me for more
{% embed https://linktr.ee/fullstacksaiyan %}

## Amazon Bedrock 

Amazon Bedrock is a fully managed service from Amazon Web Services that provides access to foundation models from Amazon and select third-party providers through a simple API. It allows customers to easily choose from a wide range of foundation models to find the one best suited for their use case. With Amazon Bedrock, customers can quickly get started with these models, privately customize them using their own data, and integrate them into applications. The service handles all infrastructure management so customers can focus on using the models without having to worry about deployment or maintenance. Amazon Bedrock also integrates with other Amazon Web Services like Amazon SageMaker for capabilities such as model testing and lifecycle management.

## Table of Contents
1. [Prerequisites](#prerequisites)
2. [Enable Amazon Bedrock Model Access](#step-1)
3. [Prepare AWS SAM Project](#step-2)
4. [Prepare Lambda function code](#step-3)
5. [Deploy your SAM project](#step-4)
6. [Cleanup](#step-5)


## Prerequisites Tools <a name="prerequisites"></a>
- [Install AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html)
- [Configure AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-configure.html)
- [Install AWS SAM](https://docs.aws.amazon.com/serverless-application-model/latest/developerguide/install-sam-cli.html)


## Enable Amazon Bedrock Model Access <a name="step-1"></a>

In order to utilize the Amazon Bedrock foundation model, you must activate model access, which is currently exclusively accessible in the `us-east-1 (N. Virginia)` region through the AWS Management Console.


![amazon bedrock console](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/09fpbkswqsl91bavvl5t.png)

Check list the model that you want to use and then click `Save changes`
![edit model access](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/0nje8y05p8w92zh19wzt.png)



## Prepare AWS SAM Project <a name="step-2"></a>

To prepare your AWS SAM project, start by creating a `lambda-bedrock` folder. Inside `lambda-bedrock` folder then create your AWS SAM template `template.yaml`.

```sh
mkdir lambda-bedrock
cd lambda-bedrock
touch template.yaml
```
These commands create a new directory called lambda-bedrock, navigate into that directory, and create a new file called `template.yaml`. The `mkdir` command creates the directory, the `cd` command navigates into that directory, and the `touch` command creates the new file. This is a common set of commands used to create a new project directory and file.

Then define your resources inside `template.yaml`
```yaml
AWSTemplateFormatVersion: "2010-09-09"
Transform: AWS::Serverless-2016-10-31
Description: >
  lambda-bedrock

  Sample SAM Template for lambda-bedrock


Resources:
  BedrockFunction:
    Type: AWS::Serverless::Function
    Region: us-east-1
    Properties:
      Timeout: 30
      CodeUri: src/
      Handler: app.handler
      Runtime: nodejs18.x
      FunctionUrlConfig:
        AuthType: NONE
        InvokeMode: BUFFERED
      Policies:
        - Statement:
            - Effect: Allow
              Action: bedrock:InvokeModel
              Resource: "arn:aws:bedrock:us-east-1::foundation-model/ai21.j2-ultra-v1"

Outputs:
  FunctionUrlEndpoint:
    Description: "Lambda Bedrock URL Endpoint"
    Value:
      Fn::GetAtt: BedrockFunctionUrl.FunctionUrl

```
This is a CloudFormation template that defines an AWS Lambda function using the `AWS::Serverless::Function` resource type. The BedrockFunction resource is defined with the specified properties.

The Type field specifies the resource type, which is `AWS::Serverless::Function`. This resource type is used to define a serverless function that can be executed in response to an event.

The Region field specifies the AWS region where the function will be created. In this case, it is set to `us-east-1`.

The Properties field contains the properties that define the function. The `Timeout` property specifies the maximum amount of time that the function can run before it times out. In this case, it is set to 30 seconds.

The `CodeUri` property specifies the location of the function's code. In this case, it is set to `src/`, which indicates that the code is located in the src directory.

The `Handler` property specifies the name of the function's handler method. In this case, it is set to `app.handler`, which indicates that the handler method is located in the `app.js` file.

The `Runtime` property specifies the runtime environment that the function will use. In this case, it is set to `nodejs18.x`, which indicates that the function will run on Node.js version 18.x.

The `FunctionUrlConfig` property specifies the configuration for the function's URL. The `AuthType` property specifies the authentication type that will be used to access the function. In this case, it is set to `NONE`, which indicates that no authentication is required. The `InvokeMode` property specifies the mode in which the function will be invoked. In this case, it is set to `BUFFERED`, which indicates that the function will be invoked in buffered mode.

The `BedrockInvokeModelPolicy` resource defines an IAM policy that allows invoking a machine learning model. The policy is attached to an existing IAM role and applies to the specified machine learning model `arn:aws:bedrock:us-east-1::foundation-model/ai21.j2-ultra-v1`.

The Outputs field contains a list of output values that are exported by the CloudFormation stack. In this case, there is only one output value. The `FunctionUrlEndpoint` output value retrieves the URL of the `BedrockFunctionUrl` resource.

## Prepare Lambda function code <a name="step-3"></a>
```sh
mkdir src
touch src/app.js
cd src
npm init -y
npm install aws-sdk
```
These commands create a new directory called `src`, navigate into that directory, create a new file called `index.js`, initialize a new Node.js project with default settings using `npm init -y`, and install the `aws-sdk` package using `npm install aws-sdk`.

The `mkdir` command creates the src directory, the `touch` command creates the `app.js` file inside the src directory, the `cd` command navigates into the src directory, the `npm init -y` command initializes a new Node.js project with default settings, and the `npm install aws-sdk` command installs the `aws-sdk` package as a dependency for the project.

Inside app.js
```javascript
const AWS = require("aws-sdk");

const bedrockRuntime = new AWS.BedrockRuntime({
  apiVersion: "2023-09-30",
  region: "us-east-1",
});

async function invokeModel(prompt) {
  try {
    const response = await new Promise((resolve, reject) => {
      bedrockRuntime.invokeModel(
        {
          modelId: "ai21.j2-ultra-v1",
          contentType: "application/json",
          accept: "*/*",
          body: JSON.stringify({
            prompt: prompt,
            maxTokens: 1000,
            temperature: 0.9,
          }),
        },
        function (err, data) {
          if (err) {
            console.log(err, err.stack);
            reject(err);
          } else {
            const response = JSON.parse(data.body);
            resolve(response.completions[0].data.text);
          }
        }
      );
    });

    return response;
  } catch (err) {
    console.log(err);
  }
}

exports.handler = async (event, context) => {
  const { prompt } = JSON.parse(event.body);

  const response = await invokeModel(prompt);

  try {
    return {
      statusCode: 200,
      body: JSON.stringify({
        message: response,
      }),
    };
  } catch (err) {
    console.log(err);
  }
};
```
This is a Node.js module that defines an AWS Lambda function that invokes a machine learning model using the BedrockRuntime class from the `aws-sdk` package. The handler function is exported as the entry point for the Lambda function.

The `require` function is used to import the `aws-sdk` package, which provides a JavaScript interface for interacting with AWS services. The BedrockRuntime class is used to invoke the machine learning model.

The `invokeModel` function is defined to invoke the machine learning model with the specified prompt. The function sends a request to the BedrockRuntime service with the specified parameters, including the model ID, content type, and request body. The function returns the response from the service, which includes the completed text generated by the machine learning model.

The handler function is defined as the entry point for the Lambda function. The function receives an event object and a context object as parameters. The event object contains the request data, including the prompt. The context object contains information about the execution environment.

The handler function parses the prompt from the request data and passes it to the invokeModel function. The function then returns a response object with a status code of 200 and a message containing the completed text generated by the machine learning model.


## Deploy your SAM project <a name="step-4"></a>

```sh
sam build
sam deploy --guided --region us-east-1
```
These commands use the AWS Serverless Application Model (SAM) to build and deploy the Lambda function defined in the app.js file to the us-east-1 region.

The `sam build` command builds the application and prepares it for deployment. This command uses the `template.yaml` file to determine the build configuration.

The `sam deploy` command deploys the application to AWS. The `--guided` option prompts the user for configuration settings, such as the stack name and deployment bucket (you only need this for the first time deployment, for subsequent deploy just run `sam deploy`). The `--region` option specifies the AWS region where the application will be deployed.

You will be prompt with this input
```sh
        Setting default arguments for 'sam deploy'
        =========================================
        Stack Name [lambda-bedrock]:
        AWS Region [us-east-1]:
        #Shows you resources changes to be deployed and require a 'Y' to initiate deploy
        Confirm changes before deploy [Y/n]:
        #SAM needs permission to be able to create roles to connect to the resources in your template
        Allow SAM CLI IAM role creation [Y/n]:
        #Preserves the state of previously provisioned resources when an operation fails
        Disable rollback [y/N]:
        BedrockFunction Function Url has no authentication. Is this okay? [y/N]: y
        Save arguments to configuration file [Y/n]:
        SAM configuration file [samconfig.toml]:
        SAM configuration environment [default]: dev
```

Then followed by this Initiating deployment confirmation
```sh
Initiating deployment
=====================

        Uploading to lambda-bedrock/5dd89a16d669f1bc7ef501c1a740d0fb.template  1185 / 1185  (100.00%)


Waiting for changeset to be created..

CloudFormation stack changeset
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------    
Operation                                                 LogicalResourceId                                         ResourceType                                              Replacement
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------    
+ Add                                                     BedrockFunctionRole                                       AWS::IAM::Role                                            N/A
+ Add                                                     BedrockFunctionUrlPublicPermissions                       AWS::Lambda::Permission                                   N/A
+ Add                                                     BedrockFunctionUrl                                        AWS::Lambda::Url                                          N/A
+ Add                                                     BedrockFunction                                           AWS::Lambda::Function                                     N/A
+ Add                                                     BedrockInvokeModelPolicy                                  AWS::IAM::Policy                                          N/A
-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------    


Changeset created successfully. arn:aws:cloudformation:ap-southeast-1:498624870591:changeSet/samcli-deploy1696242138/3e3a06d0-33c5-46cb-ad69-56edea2d1321


Previewing CloudFormation changeset before deployment
======================================================
Deploy this changeset? [y/N]:
```

After your deployment completed, you will get output like this
```sh
CloudFormation outputs from deployed stack
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------   
Outputs
--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------   
Key                 FunctionUrlEndpoint
Description         Lambda Bedrock URL Endpoint
Value               https://hwnfnespxzpw7pkz2ydjuwskpq0fokxv.lambda-url.ap-southeast-1.on.aws/
```

You can then take that Lambda endpoint and test sending your prompt like this
```sh
curl -k \
> -X POST \
> -H 'Content-Type: application/json' \
> -d '{"prompt":"explain to me about AWS Lambda"}' \
> https://hwnfnespxzpw7pkz2ydjuwskpq0fokxv.lambda-url.ap-southeast-1.on.aws/
```

Then you will get `response` from your Lambda like this
```json
{"message":"\nAWS Lambda is a serverless computing service provided by Amazon Web Services (AWS). It allows users to run code in response to events, without the need for them to provision or manage servers. With Lambda, you can create and run code without thinking about servers.\n\nLambda is designed to run code in response to events, such as a file in an Amazon Simple Storage Service (Amazon S3) bucket being modified, or a table being updated in an Amazon DynamoDB table. Lambda automatically manages the computing resources required to execute the functions, and scales them as needed.\n\nTo use Lambda, you first create a function by writing code in Node.js, Java, Python, or Ruby. You can then configure triggers that invoke the function when an event occurs. Lambda also allows you to configure how much memory and time the function should use when it is invoked.\n\nLambda functions can run for a maximum of 15 minutes, and are charged based on a pricing model that calculates the amount of time and resources consumed by the function when it is invoked. This means that you only pay for the actual compute time used by your function, and not for idle time.\n\nOverall, Lambda is a powerful tool that allows you to run code in the cloud without having to worry about the underlying infrastructure."}
```
This response is generated by Amazon Bedrock from the prompt you send to your Lambda endpoint.

## Cleanup <a name="step-5"></a>
To delete all your resources that created using SAM Template, you can run this command
```sh
sam delete --region us-east-1
        Are you sure you want to delete the stack lambda-bedrock in the region us-east-1 ? [y/N]: y
        Do you want to delete the template file fbc4997de651419e130644091ddcceb5.template in S3? [y/N]: y
        - Deleting S3 object with key 3ba0b8c3b1f20b100cab919b0cf2291a
        - Deleting S3 object with key fbc4997de651419e130644091ddcceb5.template
        - Deleting Cloudformation stack lambda-bedrock
```
This command will cleanup all the resources for you.

## Check out my previous post

{% embed https://dev.to/aws-builders/build-serverless-aws-announcements-email-service-with-aws-application-composer-and-aws-sam-2e74 %}