# Retrieval\-augmented generation #

You can use foundation models in IBM watsonx\.ai to generate factually accurate output that is grounded in information in a knowledge base by applying the retrieval\-augmented generation pattern\.

This video provides a visual method to learn the concepts and tasks in this documentation\.

**Video chapters**  
\[  0:08 \] Scenario description  
\[  0:27 \] Overview of pattern  
\[  1:03 \] Knowledge base  
\[  1:22 \] Search component  
\[  1:41 \] Prompt augmented with context  
\[  2:13 \] Generating output  
\[  2:31 \] Full solution  
\[  2:55 \] Considerations for search  
\[  3:58 \] Considerations for prompt text  
\[  5:01 \] Considerations for explainability  

## Providing context in your prompt improves accuracy ##

Foundation models can generate output that is factually inaccurate for various reasons\. One way to improve the accuracy of generated output is to provide the needed facts as context in your prompt text\.

### Example ###

The following prompt includes context to establish some facts:

    Aisha recently painted the kitchen yellow, which is her favorite color.
    
    Aisha's favorite color is

Unless Aisha is a famous person whose favorite color was mentioned in many online articles that are included in common pretraining data sets, without the context at the beginning of the prompt, no foundation model could reliably generate the correct completion of the sentence at the end of the prompt\.

If you prompt a model with text that includes fact\-filled context, then the output the model generates is more likely to be accurate\. For more details, see [Generating factually accurate output](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-factual-accuracy.html)\.

## The retrieval\-augmented generation pattern ##

You can scale out the technique of including context in your prompts by using information in a knowledge base\.

The following diagram illustrates the retrieval\-augmented generation pattern\. Although the diagram shows a question\-answering example, the same workflow supports other use cases\.

![Diagram that shows adding search results to the input for retrieval\-augmented generation](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/images/fm-rag.png)

The retrieval\-augmented generation pattern involves the following steps:

<!-- <ol> -->

1.  Search in your knowledge base for content that is related to the user's input\.
2.  Pull the most relevant search results into your prompt as context and add an instruction, such as “Answer the following question by using only information from the following passages\.”
3.  *Only if the foundation model that you're using is not instruction\-tuned*: Add a few examples that demonstrate the expected input and output format\.
4.  Send the combined prompt text to the model to generate output\.

<!-- </ol> -->

### The origin of retrieval\-augmented generation ###

The term *retrieval\-augmented generation* (RAG) was introduced in this paper: [Retrieval\-augmented generation for knowledge\-intensive NLP tasks](https://arxiv.org/abs/2005.11401)\.

> We build RAG models where the parametric memory is a pre\-trained seq2seq transformer, and the non\-parametric memory is a dense vector index of Wikipedia, accessed with a pre\-trained neural retriever\.

In that paper, the term "RAG models" refers to a specific implementation of a *retriever* (a specific query encoder and vector\-based document search index) and a *generator* (a specific pre\-trained, generative language model)\. However, the basic search\-and\-generate approach can be generalized to use different retriever components and foundation models\.

### Knowledge base ###

The knowledge base can be any collection of information\-containing artifacts, such as:

<!-- <ul> -->

 *  Process information in internal company wiki pages
 *  Files in GitHub (in any format: Markdown, plain text, JSON, code)
 *  Messages in a collaboration tool
 *  Topics in product documentation
 *  Text passages in a database like Db2
 *  A collection of legal contracts in PDF files
 *  Customer support tickets in a content management system

<!-- </ul> -->

### Retriever ###

The retriever can be any combination of search and content tools that reliably returns relevant content from the knowledge base:

<!-- <ul> -->

 *  Search tools like IBM Watson Discovery
 *  Search and content APIs (GitHub has APIs like this, for example)
 *  Vector databases (such as chromadb)

<!-- </ul> -->

### Generator ###

The generator component can use any model in watsonx\.ai, whichever one suits your use case, prompt format, and content you are pulling in for context\.

## Examples ##

The following examples demonstrate how to apply the retrieval\-augmented generation pattern\.

<!-- <table> -->

Retrieval\-augmented generation examples

| Example                                                     | Description                                                                                                                                                                                                                       | Link                                                                                                                                                                           |
| ----------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| Simple introduction                                         | Uses a small knowledge base and a simple search component to demonstrate the basic pattern\.                                                                                                                                      | [Introduction to retrieval\-augmented generation](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/fed7cf6b-1c48-4d71-8c04-0fce0e000d43)                                                                             |
| Introduction to RAG with Discovery                          | Contains the steps and code to demonstrate the retrieval\-augmented generation pattern in IBM watsonx\.ai by using IBM Watson Discovery as the search component\.                                                                 | [Simple introduction to retrieval\-augmented generation with watsonx\.ai and Discovery](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/ba4a9e35-2091-49d3-9364-a1284afab7ec) |
| Real\-world example                                         | The watsonx\.ai documentation has a search\-and\-answer feature that can answer basic what\-is questions by using the topics in the documentation as a knowledge base\.                                                           | [Answering watsonx\.ai questions using a foundation model](https://ibm.biz/watsonx-llm-search)                                                           |
| Example with LangChain                                      | Contains the steps and code to demonstrate support of retrieval\-augumented generation with LangChain in watsonx\.ai\. It introduces commands for data retrieval, knowledge base building and querying, and model testing\.       | [Use watsonx and LangChain to answer questions by using RAG](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/d3a5f957-a93b-46cd-82c1-c8d37d4f62c6)                                                                                                                 |
| Example with LangChain and an Elasticsearch vector database | Demonstrates how to use LangChain to apply an embedding model to documents in an Elasticsearch vector database\. The notebook then indexes and uses the data store to generate answers to incoming questions\.                    | [Use watsonx, Elasticsearch, and LangChain to answer questions (RAG)](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/ebeb9fc0-9844-4838-aff8-1fa1997d0c13?context=wx&audience=wdp)                                      |
| Example with the Elasticsearch Python SDK                   | Demonstrates how to use the Elasticsearch Python SDK to apply an embedding model to documents in an Elasticsearch vector database\. The notebook then indexes and uses the data store to generate answers to incoming questions\. | [Use watsonx, and Elasticsearch Python SDK to answer questions (RAG)](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/bdbc8ad4-9c1f-460f-99ee-5c3a1f374fa7?context=wx&audience=wdp)                                      |
| Example with LangChain and a SingleStore database           | Shows you how to apply retrieval\-augmented generation to large language models in watsonx by using the SingleStore database\.                                                                                                    | [RAG with SingleStore and watsonx](https://dataplatform.cloud.ibm.com/exchange/public/entry/view/daf645b2-281d-4969-9292-5012f3b18215)                                                                                                                                           |

<!-- </table ""> -->

## Learn more ##

Try these tutorials:

<!-- <ul> -->

 *  [Prompt a foundation model by using Prompt Lab](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/get-started-prompt-lab.html)
 *  [Prompt a foundation model with the retrieval\-augmented generation pattern](https://dataplatform.cloud.ibm.com/docs/content/wsj/getting-started/get-started-fm-notebook.html)

<!-- </ul> -->

**Parent topic:**[Foundation models](https://dataplatform.cloud.ibm.com/docs/content/wsj/analyze-data/fm-overview.html)

<!-- </article "role="article" "> -->