Do You need an AI ASSISTANT? Let's build it using Amazon Q! (Part 1)
Tags: ai, genai, aws, data
Published: 2024-01-23 00:30:28

Generative AI is a branch of artificial intelligence that can create new content or data from scratch, such as text, images, audio, or video. It is powered by deep learning models that learn from large amounts of data and generate novel outputs based on a given input or context. **One of the most exciting applications of generative AI is building AI assistants** that can interact with customers or employees in natural language, provide personalized information or recommendations, and automate tasks or processes.

We are already seeing in the market the launch of **AI assistants, using a chat bot experience**, that are built to answer questions using natural language using as source of information our own personal or company documents, extending the possibility of use the internet as the information source. In this way, we can have **specialized/customized answers** for our questions applying a **personal/business context** allowing us to get more **precise and specific insights**.

To implement an **AI Assistant** there are key functional and technical steps that we need to follow:

**Define the use case and the target audience:** The first step is to identify the specific problem or opportunity that the AI Assistant will address, and the characteristics and needs of the customers or employees who will use it. Let’s define as **sample use case** that our need is to have an AI Assistant that will be able to provide us with information about **two domains**: FSI industry trends and AWS services.

**Collect and prepare the data:** The second step is to gather and process the data that will be used to **enrich the answers** and evaluate the generative AI Assistant. The data should be relevant, diverse, and high-quality, and should reflect the domains and the context of our use case. For our sample use case, let’s assume that the requirement is to include information from the **internet, document repositories and to be able to add ad-hoc relevant documents**.

**Choose and deploy the generative AI platform:** The third step is to select and implement the generative AI platform that will enable the creation and management of the AI Assistant. The platform should be scalable, secure, and easy to use, and should provide tools and features for data ingestion, testing, and deployment. **For our use case, we selected Amazon Q for Business as the platform to build our AI Assistant**.

**Design, build and test the AI Assistant:** The fourth step is to design, build and test the AI assistant, ensuring that it meets the functional and non-functional requirements, such as performance, accuracy, reliability, usability, and ethics. The AI assistant should be evaluated and validated by real users, and feedback should be collected and incorporated.

Let’s see now the **process and lessons learned** during the creation of an **AI Assistant** using **Amazon Q for Business**.

**First, what is Amazon Q for Business?**

Amazon Q for Business is a fully managed, **generative AI-powered enterprise chat assistant** created by Amazon. It allows organizations to deploy an AI agent within their company to enhance employee productivity. Amazon Q for Business is tailored specifically for organizational use by allowing administrators to connect internal systems and limit access based on user permissions and groups. This ensures employees only get information relevant to their roles from trusted sources within the company.

Let’s start then with the creation of our **AI Assistant**!

I will show the AI Assistant creation process using the **AWS Console**, and that assumes that you have an **AWS account** and an **IAM user with administration privileges** required for the provision of the resources along the process. Alternatively, it is also possible to execute this process using the **AWS CLI**.

To create our **AI Assistant**, we need to create first an **Amazon Q Application**.

Let’s sign into the **AWS Management Console** and open the [Amazon Q console](https://console.aws.amazon.com/amazonq/) using the administrative user previously created.

We need first to configure the **Amazon Q Application** initial **settings**:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/ukrh4huw4obrn6jy1znt.jpg)

Note in the image above:

- **Service Access Role:** IAM role for Amazon Q to allow it to access the AWS resources it needs to create our application. We can choose to use an existing role or create a new role. The policy associated with this role will also allow **Amazon Q** to publish information to **CloudWatch Logs** that will allow us to monitor the data ingestion processes.

- **KMS Encryption Key:** Amazon Q encrypts our data by default using **AWS managed KMS keys**, option selected for this use case. Alternatively, you can use a **customer-managed key**.

- Not shown above but requested during the creation process, it is **highly recommended** to include **Application Tags** to identify **all the resources linked to our Amazon Q application**. As an example, in our case, we used the combination **“Created_By | bbold-amazonq-app”** for all the resources created.

Next, we need to create and select a **Retriever** for our Amazon Q application. An **Amazon Q retriever** determines where an Amazon Q conversational agent will search for answers to users' questions. When creating an Amazon Q application, a retriever must be selected. The retriever connects the application to external data sources containing information that can be used to respond to questions. There are two different types of retrievers that can be chosen:

- **Native retriever:** Allows connecting directly to data sources like knowledge bases, documentation repositories, or databases using **Amazon Q data connectors**.

- **Amazon Kendra retriever:** Connects to an existing **Amazon Kendra index** to query its data. **Amazon Kendra** is an intelligent search service powered by machine learning. Amazon Kendra allows organizations to index documents from multiple sources and provide a unified search experience for internal information.

Let’s see the key **Retriever settings**:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/l28giyzm64gywnzw7o2e.jpg)

Note in the image above:

- **Retriever:** We chose **"Use native retriever"** to build an **Amazon Q retriever** for our Amazon Q application. When we select the native retriever for an Amazon Q application, **Amazon Q will create an index** to connect to and organize the data sources configured for the application. While the **native retriever index is not an Amazon Kendra index**, both serve a similar purpose of housing and organizing content for retrieval. The main difference is the **native retriever index** is managed internally by **Amazon Q**, whereas a Kendra index would be a separate service integration.

- **Index provisioning:** When creating an Amazon Q application, the **index provisioning number of units** refers to the **storage capacity allocated for the application's index**. Each unit in the index corresponds to **20,000 documents** that can be stored. Each storage unit includes 100 hours of connector usage per month. The first storage unit is available at no charge for the lesser of 750 hours or 31 days. We selected 1 storage unit to **try out Amazon Q without incurring charges during the initial evaluation period**.

- Similarly, we included **Retriever Tags** and **Index Tags**, using the same combination of the **Application tags**.

Now, we are ready to connect with our **data sources**! **Amazon Q application data sources** are repositories of information that can be connected to an Amazon Q application to power the conversational agent's responses. When a user asks a question through the **Amazon Q chat interface**, the system will search across connected data sources to **find relevant answers and responses**.

Using the **Amazon Q retriever**, we can select several **cloud and on-premises data sources**, as we can see in the following images:


![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2y5qnuusvsv74x2iiavp.jpg)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/nxeypj10r95xztrenf7i.jpg)

From my point of view, this a **VERY RELEVANT CAPABILITY** of **Amazon Q for Business**, as the power of our AI Assistant will depend on the **variety and quality of the information sources** that we will connect, and for that it is very critical that we allow the organizations to **leverage their information with security where that information is currently available**. This will also contribute a lot to **eliminating adoption barriers** and **reduce implementation time avoiding non-priority information migration efforts**.

When we are talking, for example, about business, technical and other types of **business documents**, we will find them frequently stored in repositories like **S3, SharePoint, OneDrive, or Google Drive**. If we talk about **technical/code repositories**, they will typically be stored in a **GitHub repository**. **The GOOD NEWS is that ALL these data sources are already covered by Amazon Q!**

Remembering the requirements for our use case: **_“…include information from the internet, document repositories and to be able to add ad-hoc relevant documents.”_**, I was able to support this demand using the **“Most Popular” data sources** shown above. Let’s look at each one of them:

**REQ #1: Including information from the Internet**

To include information from the web, we will add a data source of type **WEB CRAWLER**. An Amazon Q Web Crawler connector crawls and indexes either public facing websites or internal company websites that use HTTPS. When selecting websites to index, we must adhere to [Amazon Acceptable Use Policy](https://aws.amazon.com/aup/?icmpid=docs_console_unmapped) and crawl only our own web pages or web pages we have the authorization to index.

Now let’s follow the **typical process to add a data source**: We start by specifying the **name** of our data source and the **source of the URLs that we want Amazon Q to index**. As you can see in the image below there are multiple options, from **specifying an URLs list** directly in the console to specify **Sitemaps** (Sitemaps contain lists of URLs that are available for crawling on a site to help crawlers comprehensively retrieve and index content).

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/yomb652bo44yjc7q8d3n.jpg)

For our use case, I decided to use the option to provide a **Source URLs file** stored in an **S3 Bucket**, as we will use the same bucket also as **complementary data source** later. The Source URLs file is just a text file that includes one URL per line. We can include up to 100 starting point URLs in the file. As example, we included in the file a list of URLs from our own website:

- https://www.bbold.com.br/
- https://www.bbold.com.br/en/
- https://www.bbold.com.br/single-post/
- https://www.bbold.com.br/en/single-post/

After we define the list of URLs, we need to configure the **security aspects** of our connection to them. For that Amazon Q offers **several authentication alternatives**. For this example, we will use **“No Authentication”** as we are crawling a **public website**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/306mfu80wsnxsyx46p4w.jpg)

In case you need to index internal websites, it also offers the option to use a **Web Proxy** and configure the **authentication credentials** using **AWS Secret Manager**.

In terms of security also, for selected data source types, Amazon Q provides optionally the alternative to **configure a VPC and security group** that will be used by Amazon Q data source connector to access your information source (Examples: access an S3 bucker or a database through a specific VPC). Since our data source is accessible from the **public internet**, we didn't need to enable the Amazon VPC feature.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/w991fzukkw2s3mmsp7kw.jpg)

Next, we need to configure the **IAM Role for Amazon Q** to access our data source repository credentials and application content. Being the first time that we are creating this data source, the recommendation is to create a **new service Role** and provide a name for it:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2oxzvy2c53m3zdvijsti.jpg)

In the background, it will create an **IAM role and a Customer Managed Policy** to allow Amazon Q to access the S3 bucket where the source URLs file is stored, access authentication credentials stored in **AWS Secret Manager** (if applicable), manage the processing of the documents to be ingested and store information related to groups and users access to those documents.

Moving on, we need to define the **Synchronization Scope** for the documents to be ingested. We can define a **sync domain range** as seen in the following image:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/xinzn6jj9057sp9giy94.jpg)

For our use case, we selected the option **“Sync domains with subdomains only”** to prevent the scenario of indexing other third-party websites potentially linked from our company website.

Additionally, Amazon Q also offers very interesting additional options regarding the scope of documents to be indexed **(“Additional Configuration”)**. I would like to highlight some aspects of them:

- **“Scope settings | Crawl depth”**: The number of levels from the seed URL that Amazon Q should crawl. This parameter is important to ensure that all the webpages that we need to be indexed are effectively included. Recommendation here is to review the structure of the websites that you are planning to index and see how many levels you need to include. For example, a **crawl depth of “3”** means that the search engine will crawl up to 3 levels deep from the seed URL. For example: Seed URL (level 1), Pages directly linked from the seed URL (level 2) and Pages directly linked from level 2 pages (level 3).

- **“Scope settings | Maximum file size”**: Where you can define the maximum file size of a webpage or attachment to crawl. You need to calibrate this parameter based on your knowledge of the documents sizes that are in your knowledge base.

- **“Include files that web pages link to”**: When this option is selected in Amazon Q, it means that the crawler will index not only the content of the web pages specified in the seed URLs, but also any files that are linked from those web pages. This allows the full content being referenced from the web pages to be searchable (Some examples of files that may be linked from web pages include documents like PDFs, images, videos, audio files, and other attachments).

- **“Crawl URL Patterns” and “URL Pattern to Index”**: Both parameters will help you to **filter the scope of information to crawl and then to index**. Amazon Q will index the URLs that it crawled based on the crawl configuration. The **“crawl URL patterns”** specify which URLs should be crawled to the specified crawl depth starting from the seed URLs. The **“URL patterns to index”** configuration can further target which of the crawled URLs should be indexed and searchable.

Once we are done with defining the **sync scope and filters**, we need to work with the configurations related to the **synchronization jobs execution**. For that we need to configure **how and when** we need those processes to run:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/y9oct518rxqnog406frm.jpg)

As you can see above, for the how to sync **(Sync Mode)** we can choose from make a **full synchronization** or **only sync the updates**. We selected the second option for our use case. In terms of the schedule **(Sync run schedule)**, you select the frequency for the synchronization, from hourly to monthly, a custom time or On Demand. For our testing, we selected this last option.

Similarly to what we did for the **Amazon Q Application, Retriever and Index**, we can also specify **Tags** for the **Data Source**. Again, it’s **highly recommended that you include tags** to identify all resources that are related to your Amazon Q application for **cost management purposes**.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jdlodjw8uvxme3m47m01.jpg)

Finally, Amazon Q shows the **Fields Mapping** section. Amazon Q crawls data source document **attributes or metadata** and maps them to **fields in your Amazon Q index**. Amazon Q has reserved fields that it uses when querying your application. It shows the list of **default attributes** mapped for both **web pages** and **attachments** (they can be customized after data source creation):

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/40823voivurzb8iir524.jpg)

With this, you can finish the configuration of the new **WEBCRAWLER** data source with the **“Add Data Source”** option. Having done that, we need to execute the **first synchronization job** that will run in a **Full sync mode** independently of your configuration. After you complete this process you can see the **Details, Sync History, Settings, and Tags** of your **Amazon Q Data Source** in the console:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/jvws36qz4amu788amda9.jpg)

In the image above, would like to highlight the **Sync run history** section where you can see the **synchronization jobs results** in terms of total items scanned, added/modified, deleted, and failed where you have quantitative information to evaluate if the crawling/indexing process have processed all what you expected based on your configuration. At this point, Amazon Q also offers the possibility to **retrieve log information** from **CloudWatch Logs** using a link in the **“Details”** column:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fuoggigbjqpsbj2vulg8.jpg)

As you can see above, Amazon Q creates a **Log Group** related to the Amazon Q application using the **Application ID (“99fd2bd8-bc98-4d34-8153-54a2a3b189b3”)** as identifier and creates a **Log Stream** for each execution using the **Data source ID (“69374b88-5f35-4b3b-a0cc-0bfb2742c23a”)** as identifier.

Amazon Q already creates the **Query** that we can execute using **Log Insights**, and running it, you will get the logs related to the synchronization job where you will be able to see **details about the URLs processed and eventually errors that must be fixed**.

**REQ #2: Including information from document repositories**

Having completed the configuration of a WEBCRAWLER data source, we can move forward to our second requirement where we will leverage **S3 buckets** with documentation to be used by our AI Assistant. We need to configure an **S3 data source** for **each bucket** we plan to index.

The configuration of this data source includes very similar steps to the ones we already saw for the WEBCRAWLER, including data source name, configuration of utilization of a VPC or not, IAM role creation, Sync scope, mode and run schedule, tags and finally field mapping.

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/fpt84eojbefrzdyal7lj.jpg)

Same as in the case of the **WEBCRAWLER data source**, in this case you can also use **Log Insights** to get the logs related to the synchronization job where you will be able to see **details about the documents processed and eventually errors that must be fixed**.

**REQ #3: Including information from add ad-hoc relevant documents**

While I recommend **storing and organizing your documentation in an S3 bucket for durability, protection, and security purposes**, we may need also to upload specific ad-hoc files to expand the knowledge base of our AI Assistant. For those cases we can use the **FILE UPLOADER** data source:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/f8g84g71afdoqv81mz1z.jpg)

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/3njt1f3pdjcf9uz917qf.jpg)

As you can see in the image above, you will be able to see in the console the **list of uploaded files** that will be indexed and be part of the knowledge base of our AI Assistant.

Once we have created all our required data sources, completed the initial synchronization jobs, and verified that they have finished successfully and indexed our documentation, **we will be ready to TEST our AI assistant!** For that AWS has already built a **WEB EXPERIENCE** which is a web interface of our **AI Assistant / CHAT BOT application** that we can use to start interacting with it.

We can access the **WEB EXPERIENCE**, through the console:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/x4rl9z35as2xezfjpzyv.jpg)

And open our **AI Assistant / CHAT BOT application** interface:

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/2ukp61tysfcv4915aw55.jpg)

As you can see above, we can configure some attributes like the **Title** (Name of the AI Assistant), **Subtitle** (Objective of the AI Assistant) and **Welcome Message**. Our suggestion is to include in the **Welcome Message** what are the **domains or subjects** that the **AI Assistant** will be **prepared to answer questions about** using the information that you provided through **Data Sources**. This is very important to **manage expectations from the potential users of the solution**.

Now, let’s see some examples of our **AI Assistant** in action!

**SAMPLE PROMPT # 1 – Asking a question about industry trends (looking to use documents in the FILE UPLOADER):** _What are the key trends in 2024 for Corporate Banking?_

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/7i9m73shbv5x4gvosnr8.jpg)

Note that the answer includes the **reference** to the information source used by the Assistant to prepare the response and allows you to **provide feedback regarding the quality of the response**.

**SAMPLE PROMPT # 2 – Asking a question about AWS services (looking to use documents in the S3 BUCKET):** _“Please prepare a summary of how AWS is supporting Banking Clients to improve their customers experience, including reference to customer examples and which AWS services are being used”_

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/14eqlovnv4mwhbihchev.jpg)

**SAMPLE PROMPT # 3 – Asking a question about DEVOPS (looking to use documents in a WEBPAGE):** _“Please prepare a summary about how we can help our customers to accelerate the implementation of DEVOPS culture in their organizations”_

![Image description](https://dev-to-uploads.s3.amazonaws.com/uploads/articles/9bjysl8xdbf6i82e8ayf.jpg)

As you can see, using **Amazon Q for Business** we have been able to implement an **AI Assistant** and as from here we can continue **enriching the KNOWLEDGE BASE** adding more documents to the data sources and/or adding more data sources as well as preparing our **PROMPTS LIBRARY** with **templates** that we can reuse to **improve our productivity and maximize the quality of the responses** obtained from the AI Assistant.

Important to highlight that those will be **ON-GOING activities** as we want to have our Assistant always **UP TO DATE** with our **latest and verified documentation** and we want also our team **improving their productivity**, knowing when and how to use the AI Assistant.

As with any other **Cloud technology adoption process**, it’s very critical to include a proper **Organizational Change Management initiative** to make sure that the team is properly **engaged, communicated, and trained** on this technology making sure that they understand that is a **valuable tool** at their disposal to gain productivity and efficiency that **DOES NOT ELIMINATE** the need for the **“human in the loop” evaluation of the quality and applicability of the responses** before their utilization to fulfill internal or customer demands.

This is a critical success factor to **eliminate adoption barriers** and very valuable **feedback for the IT Team** to be used for **data source contents refinement and enrichment**.

Up to this moment, we have our **AI Assistant** already operating for **internal use**. We need now to take the next step to **DEPLOY the AI Assistant to our organization** so we can have **multiple users** leveraging its capabilities with the **required security measures in place**.

**Let's meet again in our next post and please feel free to share your feedback and comments!**