Use AWS Generative AI CDK constructs to speed up app development
Tags: python, infrastructureascode, machinelearning, tutorial
Published: 2024-06-26 11:13:54

> Assemble and deploy the infrastructure for a RAG solution using AWS CDK for Python

In this blog, we will use the AWS Generative AI Constructs Library to deploy a complete RAG application composed of the following components:

- [Knowledge Bases for Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base.html): This is the foundation for the RAG solution.
- [OpenSearch Serverless](https://docs.aws.amazon.com/opensearch-service/latest/developerguide/serverless.html) collection: It supports the vector search collection type that provides similarity search capability.
- An S3 bucket: This will act as [the data source](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-ds.html) for the Knowledge Base.
- **AWS Lambda function** (written in Python) along with an **API Gateway** that uses the [RetrieveAndGenerate API](https://docs.aws.amazon.com/bedrock/latest/APIReference/API_agent-runtime_RetrieveAndGenerate.html) to query the knowledge base and generate responses from the information it retrieves.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7ex2pmue5yzihyvqf0fi.jpg)

## Introduction

[AWS Cloud Development Kit (AWS CDK)](https://docs.aws.amazon.com/cdk/v2/guide/home.html) is an open-source software development framework for defining cloud infrastructure in code and provisioning it through AWS CloudFormation. You can use any of these supported programming languages (TypeScript, JavaScript, Python, Java, C#/.Net, and Go) to define reusable cloud components known as *constructs*, that represent one or more AWS CloudFormation resources and their configuration.

Constructs from the AWS Construct Library are [categorized into three levels](https://docs.aws.amazon.com/cdk/v2/guide/constructs.html#constructs_lib) with each level offers an increasing level of abstraction.
- **L1 constructs** are the lowest-level construct and offer no abstraction. Each L1 construct maps directly to a single AWS CloudFormation resource.
- **L2 constructs** provide a higher-level abstraction along with helper methods for most resources that make it simpler to define resources.
- **L3 constructs**, also known as patterns, are the highest-level of abstraction, and are used to create entire AWS architectures for particular use cases in your application.

[Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/what-is-bedrock.html) is a fully managed service that makes high-performing foundation models (FMs) from leading AI startups and Amazon available for your use through a unified API. As far as AWS CDK is concerned, [Bedrock does not support L2 constructs](https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_bedrock-readme.html) yet. 

[AWS Generative AI Constructs Library](https://github.com/awslabs/generative-ai-cdk-constructs)  is an open-source extension of the AWS Cloud Development Kit (AWS CDK) and fills the gap by providing L2 and L3 constructs to help developers build generative AI solutions using pattern-based definitions for their architecture.

Let's get started!

## Prerequisites

Make sure to have the following setup and configured: [AWS CDK](https://docs.aws.amazon.com/cdk/v2/guide/getting_started.html), Python, Docker. 

Also, [configure and set up Amazon Bedrock](https://docs.aws.amazon.com/bedrock/latest/userguide/setting-up.html), including requesting access to the Foundation Model(s). The application uses [Amazon Titan Text Embeddings model](https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html) along with the [Anthropic Claude 3 Sonnet](https://docs.aws.amazon.com/bedrock/latest/userguide/model-ids.html) as the LLM.

## Deploy the CDK stack

Start by cloning the GitHub repo:

```bash
git clone https://github.com/abhirockzz/aws-cdk-generative-ai-rag
cd aws-cdk-generative-ai-rag
```

Make sure to use the **docs** folder to add the documents (PDFs etc.) that you want to be used as a source for QnA. These will be automatically uploaded to a S3 bucket that will serve as the data source for Knowledge Base. Here is the [list of document types](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-ds.html) supported by Knowledge Base.

Activate virtual env and install dependencies:

```
python3 -m venv venv

source venv/bin/activate
pip install -r requirements.txt
```

Deploy the stack:

```bash
cdk deploy
```

It might take at least five minutes for the deployment to complete. Once done, verify the CloudFormation stack in the AWS Console.


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3kjkl0vu2t1h9dpe4527.png)

Navigate to the Knowledge Base and [sync the data source](https://docs.aws.amazon.com/bedrock/latest/userguide/knowledge-base-ingest.html) with the OpenSearch Serverless vector index.

## Start asking questions

For QnA, you can use the API Gateway endpoint. Send queries via a POST request - make sure you add `/query` at the end of the API Gateway URL. I had uploaded the [Amazon 2022 shareholder document](https://s2.q4cdn.com/299287126/files/doc_financials/2023/ar/2022-Shareholder-Letter.pdf) and asked the following question: 
*"What is Amazon doing in the field of generative AI?"*

```bash
export APIGW_URL=<enter URL>

curl -X POST -d 'What is Amazon doing in the field of generative AI?' $APIGW_URL/query
```

Below is the result returned by the Lambda function based on response from `RetrieveAndGenerate` API:

```json
{
  "question": "what is amazon doing in the field of generative AI?",
  "response": "Amazon is investing heavily in Large Language Models (LLMs) and Generative AI. The company believes that Generative AI will transform and improve virtually every customer experience across its businesses. Amazon has been working on developing its own LLMs for a while now. Amazon Web Services (AWS) is democratizing Generative AI technology by offering price-performant machine learning chips like Trainium and Inferentia, so that companies of all sizes can afford to train and run their LLMs in production. AWS also enables companies to choose from various LLMs and build applications with AWS's security, privacy and other features. One example is AWS's CodeWhisperer, which uses Generative AI to generate code suggestions in real time, boosting developer productivity."
}
```

## Delete the stack

Once you are done, don't forget to delete the stack, which in turn will delete all the components:

```bash
cdk destroy
```

## CDK stack code walk through

Here is a quick walkthrough of the different components in the stack - [refer to the complete code here](https://github.com/abhirockzz/aws-cdk-generative-ai-rag/blob/master/rag/rag_stack.py).

The Knowledge Base along with the S3 bucket, uploaded document(s), and the data source configuration:

```python
        kb = bedrock.KnowledgeBase(self, 'DocKnowledgeBase', 
            embeddings_model= bedrock.BedrockFoundationModel.TITAN_EMBED_TEXT_V1,                  
        )

        documentBucket = s3.Bucket(self, 'DocumentBucket')

        deployment = s3deploy.BucketDeployment(self, "DeployDocuments",
            sources=[s3deploy.Source.asset("docs")],
            destination_bucket=documentBucket
        )

        bedrock.S3DataSource(self, 'KBS3DataSource',
            bucket= deployment.deployed_bucket,
            knowledge_base=kb,
            data_source_name='documents',
            chunking_strategy= bedrock.ChunkingStrategy.FIXED_SIZE,
            max_tokens=500,
            overlap_percentage=20   
        )
```

The Lambda function (along with necessary IAM policies) that invokes the RetrieveAndGenerate API - [refer to the code here](https://github.com/abhirockzz/aws-cdk-generative-ai-rag/blob/master/lambda/app.py).


```python
        kbQueryLambdaFunction = _lambda.Function(
            self, 'KBQueryFunction',
            runtime=_lambda.Runtime.PYTHON_3_12,
            code=_lambda.Code.from_asset('lambda'),
            handler='app.handler',
            environment={
                'KB_ID': kb.knowledge_base_id,
                'KB_MODEL_ARN': 'arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0',
            },
            timeout=Duration.seconds(15)
        )

        kbArn = f'arn:aws:bedrock:{Stack.of(self).region}:{Stack.of(self).account}:knowledge-base/{kb.knowledge_base_id}'

        # Create an IAM policy statement
        policy_statement = iam.PolicyStatement(
            actions=[
                "bedrock:Retrieve",
                "bedrock:RetrieveAndGenerate",
                "bedrock:InvokeModel"
            ],
            resources=[
                "arn:aws:bedrock:us-east-1::foundation-model/anthropic.claude-3-sonnet-20240229-v1:0",
                kbArn
            ]
        )

        kbQueryLambdaFunction.add_to_role_policy(policy_statement)
```

Finally, the API Gateway with a `query` endpoint:

```python
        api = apigateway.LambdaRestApi(
            self, 'KBQueryApiGW',
            handler=kbQueryLambdaFunction,
            proxy=False
        )

        kb_query = api.root.add_resource('query')
        kb_query.add_method('POST')
```


## Conclusion

The patterns defined in AWS Generative AI CDK Constructs are high level, multi-service abstractions of AWS CDK constructs that provide well-architected patterns for quickly defining solutions in code to create predictable and repeatable infrastructure. I showed you a Python example, but there is [TypeScript support](https://github.com/awslabs/generative-ai-cdk-constructs?tab=readme-ov-file#getting-started) as well.

I also encourage you to [check out the samples repository](https://github.com/aws-samples/generative-ai-cdk-constructs-samples) that includes a collection of functional use case implementations to demonstrate the usage of AWS Generative AI CDK Constructs.

Happy Building!