Components Chat models Google AI
ChatGoogleGenerativeAI
This docs will help you get started with Google AI chat models. For detailed documentation
of all ChatGoogleGenerativeAI features and configurations head to the API reference.
Google AI offers a number of different chat models. For information on the latest models,
their features, context windows, etc. head to the Google AI docs.
GOOGLE AI VS GOOGLE CLOUD VERTEX AI
Google's Gemini models are accessible through Google AI and through Google Cloud
Vertex AI. Using Google AI just requires a Google account and an API key. Using Google
Cloud Vertex AI requires a Google Cloud account (with term agreements and billing) but
offers enterprise features like customer encription key, virtual private cloud, and more.
To learn more about the key features of the two APIs see the Google docs.
Overview
Integration details
Class Package Local Serializable
JS
support
Package
downloads
ChatGoogleGenerativeAI
langchain
-google-
genai
❌ beta ✅ 429k/month
Model features
9/29/24, 6:21 PM ChatGoogleGenerativeAI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/chat/google_generative_ai/ 1/6
Tool
calling
Structured
output
JSON
mode
Imag
e
input
Audi
o
input
Vide
o
input
Token-
level
streaming
Nativ
e
async
Token
usage
✅ ✅ ❌ ✅ ✅ ✅ ✅ ✅ ✅
Setup
To access Google AI models you'll need to create a Google Acount account, get a Google AI
API key, and install the langchain-google-genai integration package.
Credentials
Head to https://ai.google.dev/gemini-api/docs/api-key to generate a Google AI API key. Once
you've done this set the GOOGLE_API_KEY environment variable:
If you want to get automated tracing of your model calls you can also set your LangSmith
API key by uncommenting below:
Installation
The LangChain Google AI integration lives in the langchain-google-genai package:
import getpass
import os
if "GOOGLE_API_KEY" not in os.environ:
os.environ["GOOGLE_API_KEY"] = getpass.getpass("Enter your Google AI API 
key: ")
# os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API 
key: ")
# os.environ["LANGSMITH_TRACING"] = "true"
9/29/24, 6:21 PM ChatGoogleGenerativeAI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/chat/google_generative_ai/ 2/6
Instantiation
Now we can instantiate our model object and generate chat completions:
Invocation
%pip install -qU langchain-google-genai
from langchain_google_genai import ChatGoogleGenerativeAI
llm = ChatGoogleGenerativeAI(
model="gemini-1.5-pro",
temperature=0,
max_tokens=None,
timeout=None,
max_retries=2,
# other params...
)
messages = [
(
"system",
"You are a helpful assistant that translates English to French. 
Translate the user sentence.",
),
("human", "I love programming."),
]
ai_msg = llm.invoke(messages)
ai_msg
AIMessage(content="J'adore programmer. \n", response_metadata=
{'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 
'finish_reason': 'STOP', 'safety_ratings': [{'category': 
'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability': 'NEGLIGIBLE', 'blocked': 
False}, {'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability': 'NEGLIGIBLE', 
'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability': 
'NEGLIGIBLE', 'blocked': False}, {'category': 
'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability': 'NEGLIGIBLE', 'blocked': 
9/29/24, 6:21 PM ChatGoogleGenerativeAI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/chat/google_generative_ai/ 3/6
Chaining
We can chain our model with a prompt template like so:
API Reference: ChatPromptTemplate
False}]}, id='run-eef5b138-1da6-4226-9cfe-ab9073ddd77e-0', usage_metadata=
{'input_tokens': 21, 'output_tokens': 5, 'total_tokens': 26})
print(ai_msg.content)
J'adore programmer.
from langchain_core.prompts import ChatPromptTemplate
prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"You are a helpful assistant that translates {input_language} to 
{output_language}.",
),
("human", "{input}"),
]
)
chain = prompt | llm
chain.invoke(
{
"input_language": "English",
"output_language": "German",
"input": "I love programming.",
}
)
AIMessage(content='Ich liebe das Programmieren. \n', response_metadata=
{'prompt_feedback': {'block_reason': 0, 'safety_ratings': []}, 
'finish_reason': 'STOP', 'safety_ratings': [{'category': 
'HARM_CATEGORY_SEXUALLY_EXPLICIT', 'probability': 'NEGLIGIBLE', 'blocked': 
9/29/24, 6:21 PM ChatGoogleGenerativeAI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/chat/google_generative_ai/ 4/6
Safety Seings
Gemini models have default safety settings that can be overridden. If you are receiving lots
of "Safety Warnings" from your models, you can try tweaking the safety_settings attribute
of the model. For example, to turn off safety blocking for dangerous content, you can
construct your LLM as follows:
For an enumeration of the categories and thresholds available, see Google's safety setting
types.
API reference
For detailed documentation of all ChatGoogleGenerativeAI features and configurations head
to the API reference:
https://python.langchain.com/api_reference/google_genai/chat_models/langchain_google_
genai.chat_models.ChatGoogleGenerativeAI.html
False}, {'category': 'HARM_CATEGORY_HATE_SPEECH', 'probability': 'NEGLIGIBLE', 
'blocked': False}, {'category': 'HARM_CATEGORY_HARASSMENT', 'probability': 
'NEGLIGIBLE', 'blocked': False}, {'category': 
'HARM_CATEGORY_DANGEROUS_CONTENT', 'probability': 'NEGLIGIBLE', 'blocked': 
False}]}, id='run-fbb35f30-4937-4a81-ae68-f7cb35721a0c-0', usage_metadata=
{'input_tokens': 16, 'output_tokens': 7, 'total_tokens': 23})
from langchain_google_genai import (
ChatGoogleGenerativeAI,
HarmBlockThreshold,
HarmCategory,
)
llm = ChatGoogleGenerativeAI(
model="gemini-1.5-pro",
safety_settings={
HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: 
HarmBlockThreshold.BLOCK_NONE,
},
)
9/29/24, 6:21 PM ChatGoogleGenerativeAI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/chat/google_generative_ai/ 5/6
Related
Chat model conceptual guide
Chat model how-to guides
Edit this page
Was this page helpful?
You can also leave detailed feedback on GitHub.
9/29/24, 6:21 PM ChatGoogleGenerativeAI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/chat/google_generative_ai/ 6/6
Components LLMs Google AI
Google AI
CAUTION
You are currently on a page documenting the use of Google models as text completion
models. Many popular Google models are chat completion models.
You may be looking for this page instead.
A guide on using Google Generative AI models with Langchain. Note: It's separate from
Google Cloud Vertex AI integration.
Seing up
To use Google Generative AI you must install the langchain-google-genai Python package
and generate an API key. Read more details.
%pip install --upgrade --quiet langchain-google-genai
from langchain_google_genai import GoogleGenerativeAI
from getpass import getpass
api_key = getpass()
llm = GoogleGenerativeAI(model="models/text-bison-001", 
google_api_key=api_key)
print(
llm.invoke(
"What are some of the pros and cons of Python as a programming 
language?"
9/29/24, 6:22 PM Google AI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/llms/google_ai/ 1/6
)
)
**Pros of Python:**
* **Easy to learn:** Python is a very easy-to-learn programming language, even 
for beginners. Its syntax is simple and straightforward, and there are a lot 
of resources available to help you get started.
* **Versatile:** Python can be used for a wide variety of tasks, including web 
development, data science, and machine learning. It's also a good choice for 
beginners because it can be used for a variety of projects, so you can learn 
the basics and then move on to more complex tasks.
* **High-level:** Python is a high-level programming language, which means 
that it's closer to human language than other programming languages. This 
makes it easier to read and understand, which can be a big advantage for 
beginners.
* **Open-source:** Python is an open-source programming language, which means 
that it's free to use and there are a lot of resources available to help you 
learn it.
* **Community:** Python has a large and active community of developers, which 
means that there are a lot of people who can help you if you get stuck.
**Cons of Python:**
* **Slow:** Python is a relatively slow programming language compared to some 
other languages, such as C++. This can be a disadvantage if you're working on 
computationally intensive tasks.
* **Not as performant:** Python is not as performant as some other programming 
languages, such as C++ or Java. This can be a disadvantage if you're working 
on projects that require high performance.
* **Dynamic typing:** Python is a dynamically typed programming language, 
which means that the type of a variable can change during runtime. This can be 
a disadvantage if you need to ensure that your code is type-safe.
* **Unmanaged memory:** Python uses a garbage collection system to manage 
memory. This can be a disadvantage if you need to have more control over 
memory management.
Overall, Python is a very good programming language for beginners. It's easy 
to learn, versatile, and has a large community of developers. However, it's 
important to be aware of its limitations, such as its slow performance and 
lack of performance.
llm = GoogleGenerativeAI(model="gemini-pro", google_api_key=api_key)
print(
9/29/24, 6:22 PM Google AI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/llms/google_ai/ 2/6
llm.invoke(
"What are some of the pros and cons of Python as a programming 
language?"
)
)
**Pros:**
* **Simplicity and Readability:** Python is known for its simple and easy-to-
read syntax, which makes it accessible to beginners and reduces the chance of 
errors. It uses indentation to define blocks of code, making the code 
structure clear and visually appealing.
* **Versatility:** Python is a general-purpose language, meaning it can be 
used for a wide range of tasks, including web development, data science, 
machine learning, and desktop applications. This versatility makes it a 
popular choice for various projects and industries.
* **Large Community:** Python has a vast and active community of developers, 
which contributes to its growth and popularity. This community provides 
extensive documentation, tutorials, and open-source libraries, making it easy 
for Python developers to find support and resources.
* **Extensive Libraries:** Python offers a rich collection of libraries and 
frameworks for various tasks, such as data analysis (NumPy, Pandas), web 
development (Django, Flask), machine learning (Scikit-learn, TensorFlow), and 
many more. These libraries provide pre-built functions and modules, allowing 
developers to quickly and efficiently solve common problems.
* **Cross-Platform Support:** Python is cross-platform, meaning it can run on 
various operating systems, including Windows, macOS, and Linux. This allows 
developers to write code that can be easily shared and used across different 
platforms.
**Cons:**
* **Speed and Performance:** Python is generally slower than compiled 
languages like C++ or Java due to its interpreted nature. This can be a 
disadvantage for performance-intensive tasks, such as real-time systems or 
heavy numerical computations.
* **Memory Usage:** Python programs tend to consume more memory compared to 
compiled languages. This is because Python uses a dynamic memory allocation 
system, which can lead to memory fragmentation and higher memory usage.
* **Lack of Static Typing:** Python is a dynamically typed language, which 
9/29/24, 6:22 PM Google AI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/llms/google_ai/ 3/6
Using in a chain
API Reference: PromptTemplate
Streaming calls
means that data types are not explicitly defined for variables. This can make 
it challenging to detect type errors during development, which can lead to 
unexpected behavior or errors at runtime.
* **GIL (Global Interpreter Lock):** Python uses a global interpreter lock 
(GIL) to ensure that only one thread can execute Python bytecode at a time. 
This can limit the scalability and parallelism of Python programs, especially 
in multi-threaded or multiprocessing scenarios.
* **Package Management:** While Python has a vast ecosystem of libraries and 
packages, managing dependencies and package versions can be challenging. The 
Python Package Index (PyPI) is the official repository for Python packages, 
but it can be difficult to ensure compatibility and avoid conflicts between 
different versions of packages.
from langchain_core.prompts import PromptTemplate
template = """Question: {question}
Answer: Let's think step by step."""
prompt = PromptTemplate.from_template(template)
chain = prompt | llm
question = "How much is 2+2?"
print(chain.invoke({"question": question}))
4
import sys
9/29/24, 6:22 PM Google AI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/llms/google_ai/ 4/6
Safety Seings
Gemini models have default safety settings that can be overridden. If you are receiving lots
of "Safety Warnings" from your models, you can try tweaking the safety_settings attribute
of the model. For example, to turn off safety blocking for dangerous content, you can
construct your LLM as follows:
for chunk in llm.stream("Tell me a short poem about snow"):
sys.stdout.write(chunk)
sys.stdout.flush()
In winter's embrace, a silent ballet,
Snowflakes descend, a celestial display.
Whispering secrets, they softly fall,
A blanket of white, covering all.
With gentle grace, they paint the land,
Transforming the world into a winter wonderland.
Trees stand adorned in icy splendor,
A glistening spectacle, a sight to render.
Snowflakes twirl, like dancers on a stage,
Creating a symphony, a winter montage.
Their silent whispers, a sweet serenade,
As they dance and twirl, a snowy cascade.
In the hush of dawn, a frosty morn,
Snow sparkles bright, like diamonds reborn.
Each flake unique, in its own design,
A masterpiece crafted by the divine.
So let us revel in this wintry bliss,
As snowflakes fall, with a gentle kiss.
For in their embrace, we find a peace profound,
A frozen world, with magic all around.
from langchain_google_genai import GoogleGenerativeAI, HarmBlockThreshold, 
HarmCategory
llm = GoogleGenerativeAI(
model="gemini-pro",
google_api_key=api_key,
9/29/24, 6:22 PM Google AI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/llms/google_ai/ 5/6
For an enumeration of the categories and thresholds available, see Google's safety setting
types.
Related
LLM conceptual guide
LLM how-to guides
Edit this page
Was this page helpful?
You can also leave detailed feedback on GitHub.
safety_settings={
HarmCategory.HARM_CATEGORY_DANGEROUS_CONTENT: 
HarmBlockThreshold.BLOCK_NONE,
},
)
9/29/24, 6:22 PM Google AI | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/llms/google_ai/ 6/6
Components Embedding models Google Generative AI Embeddings
Google Generative AI
Embeddings
Connect to Google's generative AI embeddings service using the
GoogleGenerativeAIEmbeddings class, found in the langchain-google-genai package.
Installation
Credentials
Usage
%pip install --upgrade --quiet langchain-google-genai
import getpass
import os
if "GOOGLE_API_KEY" not in os.environ:
os.environ["GOOGLE_API_KEY"] = getpass("Provide your Google API key 
here")
from langchain_google_genai import GoogleGenerativeAIEmbeddings
embeddings = GoogleGenerativeAIEmbeddings(model="models/embedding-001")
vector = embeddings.embed_query("hello, world!")
vector[:5]
[0.05636945, 0.0048285457, -0.0762591, -0.023642512, 0.05329321]
9/29/24, 6:23 PM Google Generative AI Embeddings | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/text_embedding/google_generative_ai/ 1/4
Batch
You can also embed multiple strings at once for a processing speedup:
Task type
GoogleGenerativeAIEmbeddings optionally support a task_type , which currently must be
one of:
task_type_unspecified
retrieval_query
retrieval_document
semantic_similarity
classification
clustering
By default, we use retrieval_document in the embed_documents method and
retrieval_query in the embed_query method. If you provide a task type, we will use that for
all methods.
vectors = embeddings.embed_documents(
[
"Today is Monday",
"Today is Tuesday",
"Today is April Fools day",
]
)
len(vectors), len(vectors[0])
(3, 768)
%pip install --upgrade --quiet matplotlib scikit-learn
9/29/24, 6:23 PM Google Generative AI Embeddings | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/text_embedding/google_generative_ai/ 2/4
All of these will be embedded with the 'retrieval_query' task set
All of these will be embedded with the 'retrieval_document' task set
In retrieval, relative distance matters. In the image above, you can see the difference in
similarity scores between the "relevant doc" and "simil stronger delta between the similar
query and relevant doc on the latter case.
Additional Configuration
You can pass the following parameters to ChatGoogleGenerativeAI in order to customize the
SDK's behavior:
client_options : Client Options to pass to the Google API Client, such as a custom
client_options["api_endpoint"]
transport : The transport method to use, such as rest , grpc , or grpc_asyncio .
Related
Note: you may need to restart the kernel to use updated packages.
query_embeddings = GoogleGenerativeAIEmbeddings(
model="models/embedding-001", task_type="retrieval_query"
)
doc_embeddings = GoogleGenerativeAIEmbeddings(
model="models/embedding-001", task_type="retrieval_document"
)
query_vecs = [query_embeddings.embed_query(q) for q in [query, query_2, 
answer_1]]
doc_vecs = [doc_embeddings.embed_query(q) for q in [query, query_2, 
answer_1]]
9/29/24, 6:23 PM Google Generative AI Embeddings | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/text_embedding/google_generative_ai/ 3/4
Embedding model conceptual guide
Embedding model how-to guides
Edit this page
Was this page helpful?
You can also leave detailed feedback on GitHub.
9/29/24, 6:23 PM Google Generative AI Embeddings | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/text_embedding/google_generative_ai/ 4/4
Providers More Qdrant
Qdrant
Qdrant (read: quadrant) is a vector similarity search engine. It provides a production-
ready service with a convenient API to store, search, and manage points - vectors with an
additional payload. Qdrant is tailored to extended filtering support.
Installation and Setup
Install the Python partner package:
Embedding models
FastEmbedSparse
SparseEmbeddings
Vector Store
There exists a wrapper around Qdrant indexes, allowing you to use it as a vectorstore,
whether for semantic search or example selection.
To import this vectorstore:
pip install langchain-qdrant
from langchain_qdrant import FastEmbedSparse
from langchain_qdrant import SparseEmbeddings
9/29/24, 6:24 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/providers/qdrant/ 1/2
For a more detailed walkthrough of the Qdrant wrapper, see this notebook
Edit this page
Was this page helpful?
You can also leave detailed feedback on GitHub.
from langchain_qdrant import QdrantVectorStore
9/29/24, 6:24 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/providers/qdrant/ 2/2
Components Vector stores Qdrant
Qdrant
Qdrant (read: quadrant ) is a vector similarity search engine. It provides a production-
ready service with a convenient API to store, search, and manage vectors with additional
payload and extended filtering support. It makes it useful for all sorts of neural network
or semantic-based matching, faceted search, and other applications.
This documentation demonstrates how to use Qdrant with Langchain for dense/sparse and
hybrid retrieval.
This page documents the QdrantVectorStore class that supports multiple retrieval
modes via Qdrant's new Query API. It requires you to run Qdrant v1.10.0 or above.
Setup
There are various modes of how to run Qdrant , and depending on the chosen one, there will
be some subtle differences. The options include:
Local mode, no server required
Docker deployments
Qdrant Cloud
See the installation instructions.
Credentials
There are no credentials needed to run the code in this notebook.
%pip install -qU langchain-qdrant
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 1/14
If you want to get best in-class automated tracing of your model calls you can also set your
LangSmith API key by uncommenting below:
Initialization
Local mode
Python client allows you to run the same code in local mode without running the Qdrant
server. That's great for testing things out and debugging or storing just a small amount of
vectors. The embeddings might be fully kept in memory or persisted on disk.
In-memory
For some testing scenarios and quick experiments, you may prefer to keep all the data in
memory only, so it gets lost when the client is destroyed - usually at the end of your
script/notebook.
OpenAI HuggingFace Fake Embedding
# os.environ["LANGSMITH_API_KEY"] = getpass.getpass("Enter your LangSmith API 
key: ")
# os.environ["LANGSMITH_TRACING"] = "true"
pip install -qU langchain-openai
import getpass
os.environ["OPENAI_API_KEY"] = getpass.getpass()
from langchain_openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings(model="text-embedding-3-large")
from langchain_qdrant import QdrantVectorStore
from qdrant_client import QdrantClient
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 2/14
On-disk storage
Local mode, without using the Qdrant server, may also store your vectors on disk so they
persist between runs.
On-premise server deployment
No matter if you choose to launch Qdrant locally with a Docker container, or select a
Kubernetes deployment with the official Helm chart, the way you're going to connect to such
an instance will be identical. You'll need to provide a URL pointing to the service.
from qdrant_client.http.models import Distance, VectorParams
client = QdrantClient(":memory:")
client.create_collection(
collection_name="demo_collection",
vectors_config=VectorParams(size=3072, distance=Distance.COSINE),
)
vector_store = QdrantVectorStore(
client=client,
collection_name="demo_collection",
embedding=embeddings,
)
client = QdrantClient(path="/tmp/langchain_qdrant")
client.create_collection(
collection_name="demo_collection",
vectors_config=VectorParams(size=3072, distance=Distance.COSINE),
)
vector_store = QdrantVectorStore(
client=client,
collection_name="demo_collection",
embedding=embeddings,
)
url = "<---qdrant url here --->"
docs = [] # put docs here
qdrant = QdrantVectorStore.from_documents(
docs,
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 3/14
Qdrant Cloud
If you prefer not to keep yourself busy with managing the infrastructure, you can choose to
set up a fully-managed Qdrant cluster on Qdrant Cloud. There is a free forever 1GB cluster
included for trying out. The main difference with using a managed version of Qdrant is that
you'll need to provide an API key to secure your deployment from being accessed publicly.
The value can also be set in a QDRANT_API_KEY environment variable.
Using an existing collection
To get an instance of langchain_qdrant.Qdrant without loading any new documents or
texts, you can use the Qdrant.from_existing_collection() method.
embeddings,
url=url,
prefer_grpc=True,
collection_name="my_documents",
)
url = "<---qdrant cloud cluster url here --->"
api_key = "<---api key here--->"
qdrant = QdrantVectorStore.from_documents(
docs,
embeddings,
url=url,
prefer_grpc=True,
api_key=api_key,
collection_name="my_documents",
)
qdrant = QdrantVectorStore.from_existing_collection(
embedding=embeddings,
collection_name="my_documents",
url="http://localhost:6333",
)
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 4/14
Manage vector store
Once you have created your vector store, we can interact with it by adding and deleting
different items.
Add items to vector store
We can add items to our vector store by using the add_documents function.
from uuid import uuid4
from langchain_core.documents import Document
document_1 = Document(
page_content="I had chocalate chip pancakes and scrambled eggs for 
breakfast this morning.",
metadata={"source": "tweet"},
)
document_2 = Document(
page_content="The weather forecast for tomorrow is cloudy and overcast, 
with a high of 62 degrees.",
metadata={"source": "news"},
)
document_3 = Document(
page_content="Building an exciting new project with LangChain - come 
check it out!",
metadata={"source": "tweet"},
)
document_4 = Document(
page_content="Robbers broke into the city bank and stole $1 million in 
cash.",
metadata={"source": "news"},
)
document_5 = Document(
page_content="Wow! That was an amazing movie. I can't wait to see it 
again.",
metadata={"source": "tweet"},
)
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 5/14
API Reference: Document
document_6 = Document(
page_content="Is the new iPhone worth the price? Read this review to find 
out.",
metadata={"source": "website"},
)
document_7 = Document(
page_content="The top 10 soccer players in the world right now.",
metadata={"source": "website"},
)
document_8 = Document(
page_content="LangGraph is the best framework for building stateful, 
agentic applications!",
metadata={"source": "tweet"},
)
document_9 = Document(
page_content="The stock market is down 500 points today due to fears of a 
recession.",
metadata={"source": "news"},
)
document_10 = Document(
page_content="I have a bad feeling I am going to get deleted :(",
metadata={"source": "tweet"},
)
documents = [
document_1,
document_2,
document_3,
document_4,
document_5,
document_6,
document_7,
document_8,
document_9,
document_10,
]
uuids = [str(uuid4()) for _ in range(len(documents))]
vector_store.add_documents(documents=documents, ids=uuids)
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 6/14
Delete items from vector store
Query vector store
Once your vector store has been created and the relevant documents have been added you
will most likely wish to query it during the running of your chain or agent.
Query directly
The simplest scenario for using Qdrant vector store is to perform a similarity search. Under
the hood, our query will be encoded into vector embeddings and used to find similar
documents in Qdrant collection.
['c04134c3-273d-4766-949a-eee46052ad32',
'9e6ba50c-794f-4b88-94e5-411f15052a02',
'd3202666-6f2b-4186-ac43-e35389de8166',
'50d8d6ee-69bf-4173-a6a2-b254e9928965',
'bd2eae02-74b5-43ec-9fcf-09e9d9db6fd3',
'6dae6b37-826d-4f14-8376-da4603b35de3',
'b0964ab5-5a14-47b4-a983-37fa5c5bd154',
'91ed6c56-fe53-49e2-8199-c3bb3c33c3eb',
'42a580cb-7469-4324-9927-0febab57ce92',
'ff774e5c-f158-4d12-94e2-0a0162b22f27']
vector_store.delete(ids=[uuids[-1]])
True
results = vector_store.similarity_search(
"LangChain provides abstractions to make working with LLMs easy", k=2
)
for res in results:
print(f"* {res.page_content} [{res.metadata}]")
* Building an exciting new project with LangChain - come check it out! 
[{'source': 'tweet', '_id': 'd3202666-6f2b-4186-ac43-e35389de8166', 
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 7/14
QdrantVectorStore supports 3 modes for similarity searches. They can be configured using
the retrieval_mode parameter when setting up the class.
Dense Vector Search(Default)
Sparse Vector Search
Hybrid Search
Dense Vector Search
To search with only dense vectors,
The retrieval_mode parameter should be set to RetrievalMode.DENSE (default).
A dense embeddings value should be provided to the embedding parameter.
Sparse Vector Search
To search with only sparse vectors,
The retrieval_mode parameter should be set to RetrievalMode.SPARSE .
'_collection_name': 'demo_collection'}]
* LangGraph is the best framework for building stateful, agentic applications! 
[{'source': 'tweet', '_id': '91ed6c56-fe53-49e2-8199-c3bb3c33c3eb', 
'_collection_name': 'demo_collection'}]
from langchain_qdrant import RetrievalMode
qdrant = QdrantVectorStore.from_documents(
docs,
embedding=embeddings,
location=":memory:",
collection_name="my_documents",
retrieval_mode=RetrievalMode.DENSE,
)
query = "What did the president say about Ketanji Brown Jackson"
found_docs = qdrant.similarity_search(query)
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 8/14
An implementation of the SparseEmbeddings interface using any sparse embeddings
provider has to be provided as value to the sparse_embedding parameter.
The langchain-qdrant package provides a FastEmbed based implementation out of the box.
To use it, install the FastEmbed package.
Hybrid Vector Search
To perform a hybrid search using dense and sparse vectors with score fusion,
The retrieval_mode parameter should be set to RetrievalMode.HYBRID .
A dense embeddings value should be provided to the embedding parameter.
An implementation of the SparseEmbeddings interface using any sparse embeddings
provider has to be provided as value to the sparse_embedding parameter.
Note that if you've added documents with the HYBRID mode, you can switch to any retrieval
mode when searching. Since both the dense and sparse vectors are available in the
collection.
%pip install fastembed
from langchain_qdrant import FastEmbedSparse, RetrievalMode
sparse_embeddings = FastEmbedSparse(model_name="Qdrant/bm25")
qdrant = QdrantVectorStore.from_documents(
docs,
sparse_embedding=sparse_embeddings,
location=":memory:",
collection_name="my_documents",
retrieval_mode=RetrievalMode.SPARSE,
)
query = "What did the president say about Ketanji Brown Jackson"
found_docs = qdrant.similarity_search(query)
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 9/14
If you want to execute a similarity search and receive the corresponding scores you can run:
For a full list of all the search functions available for a QdrantVectorStore , read the API
reference
Metadata filtering
Qdrant has an extensive filtering system with rich type support. It is also possible to use the
filters in Langchain, by passing an additional param to both the
similarity_search_with_score and similarity_search methods.
from langchain_qdrant import FastEmbedSparse, RetrievalMode
sparse_embeddings = FastEmbedSparse(model_name="Qdrant/bm25")
qdrant = QdrantVectorStore.from_documents(
docs,
embedding=embeddings,
sparse_embedding=sparse_embeddings,
location=":memory:",
collection_name="my_documents",
retrieval_mode=RetrievalMode.HYBRID,
)
query = "What did the president say about Ketanji Brown Jackson"
found_docs = qdrant.similarity_search(query)
results = vector_store.similarity_search_with_score(
query="Will it be hot tomorrow", k=1
)
for doc, score in results:
print(f"* [SIM={score:3f}] {doc.page_content} [{doc.metadata}]")
* [SIM=0.531834] The weather forecast for tomorrow is cloudy and overcast, 
with a high of 62 degrees. [{'source': 'news', '_id': '9e6ba50c-794f-4b88-
94e5-411f15052a02', '_collection_name': 'demo_collection'}]
from qdrant_client.http import models
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 10/14
Query by turning into retriever
You can also transform the vector store into a retriever for easier usage in your chains.
Usage for retrieval-augmented generation
For guides on how to use this vector store for retrieval-augmented generation (RAG), see the
following sections:
results = vector_store.similarity_search(
query="Who are the best soccer players in the world?",
k=1,
filter=models.Filter(
should=[
models.FieldCondition(
key="page_content",
match=models.MatchValue(
value="The top 10 soccer players in the world right now."
),
),
]
),
)
for doc in results:
print(f"* {doc.page_content} [{doc.metadata}]")
* The top 10 soccer players in the world right now. [{'source': 'website', 
'_id': 'b0964ab5-5a14-47b4-a983-37fa5c5bd154', '_collection_name': 
'demo_collection'}]
retriever = vector_store.as_retriever(search_type="mmr", search_kwargs={"k": 
1})
retriever.invoke("Stealing from the bank is a crime")
[Document(metadata={'source': 'news', '_id': '50d8d6ee-69bf-4173-a6a2-
b254e9928965', '_collection_name': 'demo_collection'}, page_content='Robbers 
broke into the city bank and stole $1 million in cash.')]
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 11/14
Tutorials: working with external knowledge
How-to: Question and answer with RAG
Retrieval conceptual docs
Customizing Qdrant
There are options to use an existing Qdrant collection within your Langchain application. In
such cases, you may need to define how to map Qdrant point into the Langchain Document .
Named vectors
Qdrant supports multiple vectors per point by named vectors. If you work with a collection
created externally or want to have the differently named vector used, you can configure it by
providing its name.
Metadata
Qdrant stores your vector embeddings along with the optional JSON-like payload. Payloads
are optional, but since LangChain assumes the embeddings are generated from the
documents, we keep the context data, so you can extract the original texts as well.
By default, your document is going to be stored in the following payload structure:
from langchain_qdrant import RetrievalMode
QdrantVectorStore.from_documents(
docs,
embedding=embeddings,
sparse_embedding=sparse_embeddings,
location=":memory:",
collection_name="my_documents_2",
retrieval_mode=RetrievalMode.HYBRID,
vector_name="custom_vector",
sparse_vector_name="custom_sparse_vector",
)
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 12/14
You can, however, decide to use different keys for the page content and metadata. That's
useful if you already have a collection that you'd like to reuse.
API reference
For detailed documentation of all QdrantVectorStore features and configurations head to
the API reference:
https://python.langchain.com/api_reference/qdrant/qdrant/langchain_qdrant.qdrant.QdrantV
ectorStore.html
Related
Vector store conceptual guide
Vector store how-to guides
Edit this page
{
"page_content": "Lorem ipsum dolor sit amet",
"metadata": {
"foo": "bar"
}
}
QdrantVectorStore.from_documents(
docs,
embeddings,
location=":memory:",
collection_name="my_documents_2",
content_payload_key="my_page_content_key",
metadata_payload_key="my_meta",
)
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 13/14
Was this page helpful?
You can also leave detailed feedback on GitHub.
9/29/24, 6:25 PM Qdrant | 🦜 🔗 LangChain
https://python.langchain.com/docs/integrations/vectorstores/qdrant/ 14/14
Agentic RAG
Retrieval Agents are useful when we want to make decisions about whether to
retrieve from an index.
To implement a retrieval agent, we simply need to give an LLM access to a retriever
tool.
We can incorporate this into LangGraph.
Setup
First, let's download the required packages and set our API keys:
Sign up for LangSmith to quickly spot issues and improve the performance of your
LangGraph projects. LangSmith lets you use trace data to debug, test, and monitor
your LLM apps built with LangGraph ? read more about how to get started here.
%%capture --no-stderr
%pip install -U --quiet langchain-community tiktoken langchain-
openai langchainhub chromadb langchain langgraph langchain-text-
splitters
import getpass
import os
def _set_env(key: str):
if key not in os.environ:
os.environ[key] = getpass.getpass(f"{key}:")
_set_env("OPENAI_API_KEY")
Set up LangSmith for LangGraph development
10/3/24, 10:36 AM Agentic RAG
https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/ 1/12
Retriever
First, we index 3 blog posts.
Then we create a retriever tool.
from langchain_community.document_loaders import WebBaseLoader
from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings
from langchain_text_splitters import
RecursiveCharacterTextSplitter
urls = [
"https://lilianweng.github.io/posts/2023-06-23-agent/",
"https://lilianweng.github.io/posts/2023-03-15-prompt-
engineering/",
"https://lilianweng.github.io/posts/2023-10-25-adv-attack-
llm/",
]
docs = [WebBaseLoader(url).load() for url in urls]
docs_list = [item for sublist in docs for item in sublist]
text_splitter =
RecursiveCharacterTextSplitter.from_tiktoken_encoder(
chunk_size=100, chunk_overlap=50
)
doc_splits = text_splitter.split_documents(docs_list)
# Add to vectorDB
vectorstore = Chroma.from_documents(
documents=doc_splits,
collection_name="rag-chroma",
embedding=OpenAIEmbeddings(),
)
retriever = vectorstore.as_retriever()
from langchain.tools.retriever import create_retriever_tool
retriever_tool = create_retriever_tool(
retriever,
"retrieve_blog_posts",
"Search and return information about Lilian Weng blog posts
on LLM agents, prompt engineering, and adversarial attacks on
LLMs.",
)
10/3/24, 10:36 AM Agentic RAG
https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/ 2/12
Agent State
We will de?ne a graph.
A state object that it passes around to each node.
Our state will be a list of messages .
Each node in our graph will append to it.
Nodes and Edges
We can lay out an agentic RAG graph like this:
The state is a set of messages
Each node will update (append to) state
Conditional edges decide which node to visit next
tools = [retriever_tool]
from typing import Annotated, Sequence
from typing_extensions import TypedDict
from langchain_core.messages import BaseMessage
from langgraph.graph.message import add_messages
class AgentState(TypedDict):
# The add_messages function defines how an update should be
processed
# Default is to replace. add_messages says "append"
messages: Annotated[Sequence[BaseMessage], add_messages]
10/3/24, 10:36 AM Agentic RAG
https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/ 3/12
This notebook uses Pydantic v2 BaseModel , which requires langchain-core >= 0.3 .
Using langchain-core < 0.3 will result in errors due to mixing of Pydantic v1 and v2
BaseModels .
Using Pydantic with LangChain
from typing import Annotated, Literal, Sequence
from typing_extensions import TypedDict
from langchain import hub
from langchain_core.messages import BaseMessage, HumanMessage
from langchain_core.output_parsers import StrOutputParser
from langchain_core.prompts import PromptTemplate
from langchain_openai import ChatOpenAI
from pydantic import BaseModel, Field
from langgraph.prebuilt import tools_condition
### Edges
def grade_documents(state) -> Literal["generate", "rewrite"]:
"""
Determines whether the retrieved documents are relevant to
the question.
Args:
state (messages): The current state
Returns:
str: A decision for whether the documents are relevant
or not
"""
10/3/24, 10:36 AM Agentic RAG
https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/ 4/12
print("---CHECK RELEVANCE---")
# Data model
class grade(BaseModel):
"""Binary score for relevance check."""
binary_score: str = Field(description="Relevance score
'yes' or 'no'")
# LLM
model = ChatOpenAI(temperature=0, model="gpt-4-0125-
preview", streaming=True)
# LLM with tool and validation
llm_with_tool = model.with_structured_output(grade)
# Prompt
prompt = PromptTemplate(
template="""You are a grader assessing relevance of a
retrieved document to a user question. \n
Here is the retrieved document: \n\n {context} \n\n
Here is the user question: {question} \n
If the document contains keyword(s) or semantic meaning
related to the user question, grade it as relevant. \n
Give a binary score 'yes' or 'no' score to indicate
whether the document is relevant to the question.""",
input_variables=["context", "question"],
)
# Chain
chain = prompt | llm_with_tool
messages = state["messages"]
last_message = messages[-1]
question = messages[0].content
docs = last_message.content
scored_result = chain.invoke({"question": question,
"context": docs})
score = scored_result.binary_score
if score == "yes":
print("---DECISION: DOCS RELEVANT---")
return "generate"
else:
print("---DECISION: DOCS NOT RELEVANT---")
print(score)
10/3/24, 10:36 AM Agentic RAG
https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/ 5/12
return "rewrite"
### Nodes
def agent(state):
"""
Invokes the agent model to generate a response based on the
current state. Given
the question, it will decide to retrieve using the retriever
tool, or simply end.
Args:
state (messages): The current state
Returns:
dict: The updated state with the agent response appended
to messages
"""
print("---CALL AGENT---")
messages = state["messages"]
model = ChatOpenAI(temperature=0, streaming=True,
model="gpt-4-turbo")
model = model.bind_tools(tools)
response = model.invoke(messages)
# We return a list, because this will get added to the
existing list
return {"messages": [response]}
def rewrite(state):
"""
Transform the query to produce a better question.
Args:
state (messages): The current state
Returns:
dict: The updated state with re-phrased question
"""
print("---TRANSFORM QUERY---")
messages = state["messages"]
question = messages[0].content
msg = [
HumanMessage(
content=f""" \n
Look at the input and try to reason about the underlying
semantic intent / meaning. \n
10/3/24, 10:36 AM Agentic RAG
https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/ 6/12
Here is the initial question:
\n ------- \n
{question}
\n ------- \n
Formulate an improved question: """,
)
]
# Grader
model = ChatOpenAI(temperature=0, model="gpt-4-0125-
preview", streaming=True)
response = model.invoke(msg)
return {"messages": [response]}
def generate(state):
"""
Generate answer
Args:
state (messages): The current state
Returns:
dict: The updated state with re-phrased question
"""
print("---GENERATE---")
messages = state["messages"]
question = messages[0].content
last_message = messages[-1]
docs = last_message.content
# Prompt
prompt = hub.pull("rlm/rag-prompt")
# LLM
llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0,
streaming=True)
# Post-processing
def format_docs(docs):
return "\n\n".join(doc.page_content for doc in docs)
# Chain
rag_chain = prompt | llm | StrOutputParser()
# Run
response = rag_chain.invoke({"context": docs, "question":
question})
return {"messages": [response]}
10/3/24, 10:36 AM Agentic RAG
https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/ 7/12
Graph
Start with an agent, call_model
Agent make a decision to call a function
If so, then action to call tool (retriever)
Then call agent with the tool output added to messages ( state )
print("*" * 20 + "Prompt[rlm/rag-prompt]" + "*" * 20)
prompt = hub.pull("rlm/rag-prompt").pretty_print() # Show what
the prompt looks like
********************Prompt[rlm/rag-prompt]********************
================================ Human Message ===================
==============
You are an assistant for question-answering tasks. Use the followi
ng pieces of retrieved context to answer the question. If you do
n't know the answer, just say that you don't know. Use three sente
nces maximum and keep the answer concise.
Question: {question}
Context: {context}
Answer:
from langgraph.graph import END, StateGraph, START
from langgraph.prebuilt import ToolNode
# Define a new graph
workflow = StateGraph(AgentState)
# Define the nodes we will cycle between
workflow.add_node("agent", agent) # agent
retrieve = ToolNode([retriever_tool])
workflow.add_node("retrieve", retrieve) # retrieval
workflow.add_node("rewrite", rewrite) # Re-writing the question
workflow.add_node(
"generate", generate
) # Generating a response after we know the documents are
relevant
# Call agent node to decide to retrieve or not
workflow.add_edge(START, "agent")
# Decide whether to retrieve
workflow.add_conditional_edges(
"agent",
# Assess agent decision
10/3/24, 10:36 AM Agentic RAG
https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/ 8/12
tools_condition,
{
# Translate the condition outputs to nodes in our graph
"tools": "retrieve",
END: END,
},
)
# Edges taken after the `action` node is called.
workflow.add_conditional_edges(
"retrieve",
# Assess agent decision
grade_documents,
)
workflow.add_edge("generate", END)
workflow.add_edge("rewrite", "agent")
# Compile
graph = workflow.compile()
from IPython.display import Image, display
try:
display(Image(graph.get_graph(xray=True).draw_mermaid_png()))
except Exception:
# This requires some extra dependencies and is optional
pass
10/3/24, 10:36 AM Agentic RAG
https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/ 9/12
import pprint
inputs = {
"messages": [
("user", "What does Lilian Weng say about the types of
agent memory?"),
]
}
for output in graph.stream(inputs):
for key, value in output.items():
pprint.pprint(f"Output from node '{key}':")
pprint.pprint("---")
pprint.pprint(value, indent=2, width=80, depth=None)
pprint.pprint("\n---\n")
---CALL AGENT---
"Output from node 'agent':"
'---'
{ 'messages': [ AIMessage(content='', additional_kwargs={'tool_cal
ls': [{'index': 0, 'id': 'call_z36oPZN8l1UC6raxrebqc1bH', 'functio
n': {'arguments': '{"query":"types of agent memory"}', 'name': 're
trieve_blog_posts'}, 'type': 'function'}]}, response_metadata={'fi
nish_reason': 'tool_calls'}, id='run-2bad2518-8187-4d8f-8e23-2b950
1becb6f-0', tool_calls=[{'name': 'retrieve_blog_posts', 'args':
10/3/24, 10:36 AM Agentic RAG
https://langchain-ai.github.io/langgraph/tutorials/rag/langgraph_agentic_rag/ 10/12
Quick Start
In this comprehensive quick start, we will build a support chatbot in LangGraph that
can:
Answer common questions by searching the web
Maintain conversation state across calls
Route complex queries to a human for review
Use custom state to control its behavior
Rewind and explore alternative conversation paths
We'll start with a basic chatbot and progressively add more sophisticated
capabilities, introducing key LangGraph concepts along the way.
Setup
First, install the required packages:
Next, set your API keys:
%%capture --no-stderr
%pip install -U langgraph langsmith langchain_anthropic
import getpass
import os
def _set_env(var: str):
if not os.environ.get(var):
os.environ[var] = getpass.getpass(f"{var}: ")
_set_env("ANTHROPIC_API_KEY")
ANTHROPIC_API_KEY: ········
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 1/61
Sign up for LangSmith to quickly spot issues and improve the performance of your
LangGraph projects. LangSmith lets you use trace data to debug, test, and monitor
your LLM apps built with LangGraph ? read more about how to get started here.
Part 1: Build a Basic Chatbot
We'll ?rst create a simple chatbot using LangGraph. This chatbot will respond
directly to user messages. Though simple, it will illustrate the core concepts of
building with LangGraph. By the end of this section, you will have a built rudimentary
chatbot.
Start by creating a StateGraph . A StateGraph object de?nes the structure of our
chatbot as a "state machine". We'll add nodes to represent the llm and functions our
chatbot can call and edges to specify how the bot should transition between these
functions.
Set up LangSmith for LangGraph development
from typing import Annotated
from typing_extensions import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
class State(TypedDict):
# Messages have the type "list". The `add_messages` function
# in the annotation defines how this state key should be
updated
# (in this case, it appends messages to the list, rather than
overwriting them)
messages: Annotated[list, add_messages]
graph_builder = StateGraph(State)
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 2/61
The ?rst thing you do when you de?ne a graph is de?ne the State of the graph. The
State consists of the schema of the graph as well as reducer functions which specify
how to apply updates to the state. In our example State is a TypedDict with a single
key: messages . The messages key is annotated with the add_messages reducer
function, which tells LangGraph to append new messages to the existing list, rather
than overwriting it. State keys without an annotation will be overwritten by each
update, storing the most recent value. Check out this conceptual guide to learn more
about state, reducers and other low-level concepts.
So now our graph knows two things:
?? Every node we de?ne will receive the current State as input and return a value
that updates that state.
?? messages will be appended to the current list, rather than directly overwritten.
This is communicated via the prebuilt add_messages function in the Annotated
syntax.
Next, add a " chatbot " node. Nodes represent units of work. They are typically
regular python functions.
Notice how the chatbot node function takes the current State as input and returns
a dictionary containing an updated messages list under the key "messages". This is
the basic pattern for all LangGraph node functions.
The add_messages function in our State will append the llm's response messages to
whatever messages are already in the state.
Note
from langchain_anthropic import ChatAnthropic
llm = ChatAnthropic(model="claude-3-5-sonnet-20240620")
def chatbot(state: State):
return {"messages": [llm.invoke(state["messages"])]}
# The first argument is the unique node name
# The second argument is the function or object that will be
called whenever
# the node is used.
graph_builder.add_node("chatbot", chatbot)
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 3/61
Next, add an entry point. This tells our graph where to start its work each time we
run it.
Similarly, set a finish point. This instructs the graph "any time this node is run, you
can exit."
Finally, we'll want to be able to run our graph. To do so, call " compile() " on the graph
builder. This creates a " CompiledGraph " we can use invoke on our state.
You can visualize the graph using the get_graph method and one of the "draw"
methods, like draw_ascii or draw_png . The draw methods each require additional
dependencies.
Now let's run the chatbot!
Tip: You can exit the chat loop at any time by typing "quit", "exit", or "q".
graph_builder.add_edge(START, "chatbot")
graph_builder.add_edge("chatbot", END)
graph = graph_builder.compile()
from IPython.display import Image, display
try:
display(Image(graph.get_graph().draw_mermaid_png()))
except Exception:
# This requires some extra dependencies and is optional
pass
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 4/61
Congratulations! You've built your ?rst chatbot using LangGraph. This bot can
engage in basic conversation by taking user input and generating responses using an
LLM. You can inspect a LangSmith Trace for the call above at the provided link.
However, you may have noticed that the bot's knowledge is limited to what's in its
training data. In the next part, we'll add a web search tool to expand the bot's
knowledge and make it more capable.
Below is the full code for this section for your reference:
def stream_graph_updates(user_input: str):
for event in graph.stream({"messages": [("user",
user_input)]}):
for value in event.values():
print("Assistant:", value["messages"][-1].content)
while True:
try:
user_input = input("User: ")
if user_input.lower() in ["quit", "exit", "q"]:
print("Goodbye!")
break
stream_graph_updates(user_input)
except:
# fallback if input() is not available
user_input = "What do you know about LangGraph?"
print("User: " + user_input)
stream_graph_updates(user_input)
break
Assistant: LangGraph is a library designed to help build stateful
multi-agent applications using language models. It provides tools
for creating workflows and state machines to coordinate multiple A
I agents or language model interactions. LangGraph is built on top
of LangChain, leveraging its components while adding graph-based c
oordination capabilities. It's particularly useful for developing
more complex, stateful AI applications that go beyond simple query
-response interactions.
Goodbye!
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 5/61
Part 2: Enhancing the Chatbot with Tools
To handle queries our chatbot can't answer "from memory", we'll integrate a web
search tool. Our bot can use this tool to ?nd relevant information and provide better
Full Code
from typing import Annotated
from langchain_anthropic import ChatAnthropic
from typing_extensions import TypedDict
from langgraph.graph import StateGraph
from langgraph.graph.message import add_messages
class State(TypedDict):
messages: Annotated[list, add_messages]
graph_builder = StateGraph(State)
llm = ChatAnthropic(model="claude-3-5-sonnet-20240620")
def chatbot(state: State):
return {"messages": [llm.invoke(state["messages"])]}
# The first argument is the unique node name
# The second argument is the function or object that will be
called whenever
# the node is used.
graph_builder.add_node("chatbot", chatbot)
graph_builder.set_entry_point("chatbot")
graph_builder.set_finish_point("chatbot")
graph = graph_builder.compile()
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 6/61
responses.
Requirements
Before we start, make sure you have the necessary packages installed and API keys
set up:
First, install the requirements to use the Tavily Search Engine, and set your
TAVILY_API_KEY.
Next, de?ne the tool:
The results are page summaries our chat bot can use to answer questions.
Next, we'll start de?ning our graph. The following is all the same as in Part 1, except
we have added bind_tools on our LLM. This lets the LLM know the correct JSON
format to use if it wants to use our search engine.
%%capture --no-stderr
%pip install -U tavily-python langchain_community
_set_env("TAVILY_API_KEY")
TAVILY_API_KEY: ········
from langchain_community.tools.tavily_search import
TavilySearchResults
tool = TavilySearchResults(max_results=2)
tools = [tool]
tool.invoke("What's a 'node' in LangGraph?")
[{'url': 'https://medium.com/@cplog/introduction-to-langgraph-a-be
ginners-guide-14f9be027141',
'content': 'Nodes: Nodes are the building blocks of your LangGra
ph. Each node represents a function or a computation step. You def
ine nodes to perform specific tasks, such as processing input, mak
ing ...'},
{'url': 'https://saksheepatil05.medium.com/demystifying-langgraph
-a-beginner-friendly-dive-into-langgraph-concepts-5ffe890ddac0',
'content': 'Nodes (Tasks): Nodes are like the workstations on th
e assembly line. Each node performs a specific task on the produc
t. In LangGraph, nodes are Python functions that take the current
state, do some work, and return an updated state. Next, we define
the nodes, each representing a task in our sandwich-making proces
s.'}]
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 7/61
Next we need to create a function to actually run the tools if they are called. We'll do
this by adding the tools to a new node.
Below, we implement a BasicToolNode that checks the most recent message in the
state and calls tools if the message contains tool_calls . It relies on the LLM's
tool_calling support, which is available in Anthropic, OpenAI, Google Gemini, and a
number of other LLM providers.
We will later replace this with LangGraph's prebuilt ToolNode to speed things up, but
building it ourselves ?rst is instructive.
from typing import Annotated
from langchain_anthropic import ChatAnthropic
from typing_extensions import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
class State(TypedDict):
messages: Annotated[list, add_messages]
graph_builder = StateGraph(State)
llm = ChatAnthropic(model="claude-3-5-sonnet-20240620")
# Modification: tell the LLM which tools it can call
llm_with_tools = llm.bind_tools(tools)
def chatbot(state: State):
return {"messages":
[llm_with_tools.invoke(state["messages"])]}
graph_builder.add_node("chatbot", chatbot)
import json
from langchain_core.messages import ToolMessage
class BasicToolNode:
"""A node that runs the tools requested in the last
AIMessage."""
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 8/61
With the tool node added, we can de?ne the conditional_edges .
Recall that edges route the control ?ow from one node to the next. Conditional
edges usually contain "if" statements to route to different nodes depending on the
current graph state. These functions receive the current graph state and return a
string or list of strings indicating which node(s) to call next.
Below, call de?ne a router function called route_tools , that checks for tool_calls in
the chatbot's output. Provide this function to the graph by calling
add_conditional_edges , which tells the graph that whenever the chatbot node
completes to check this function to see where to go next.
The condition will route to tools if tool calls are present and END if not.
Later, we will replace this with the prebuilt tools_condition to be more concise, but
implementing it ourselves ?rst makes things more clear.
def __init__(self, tools: list) -> None:
self.tools_by_name = {tool.name: tool for tool in tools}
def __call__(self, inputs: dict):
if messages := inputs.get("messages", []):
message = messages[-1]
else:
raise ValueError("No message found in input")
outputs = []
for tool_call in message.tool_calls:
tool_result =
self.tools_by_name[tool_call["name"]].invoke(
tool_call["args"]
)
outputs.append(
ToolMessage(
content=json.dumps(tool_result),
name=tool_call["name"],
tool_call_id=tool_call["id"],
)
)
return {"messages": outputs}
tool_node = BasicToolNode(tools=[tool])
graph_builder.add_node("tools", tool_node)
from typing import Literal
def route_tools(
state: State,
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 9/61
Notice that conditional edges start from a single node. This tells the graph "any time
the ' chatbot ' node runs, either go to 'tools' if it calls a tool, or end the loop if it
responds directly.
Like the prebuilt tools_condition , our function returns the END string if no tool
calls are made. When the graph transitions to END , it has no more tasks to complete
and ceases execution. Because the condition can return END , we don't need to
explicitly set a finish_point this time. Our graph already has a way to ?nish!
):
"""
Use in the conditional_edge to route to the ToolNode if the
last message
has tool calls. Otherwise, route to the end.
"""
if isinstance(state, list):
ai_message = state[-1]
elif messages := state.get("messages", []):
ai_message = messages[-1]
else:
raise ValueError(f"No messages found in input state to
tool_edge: {state}")
if hasattr(ai_message, "tool_calls") and
len(ai_message.tool_calls) > 0:
return "tools"
return END
# The `tools_condition` function returns "tools" if the chatbot
asks to use a tool, and "END" if
# it is fine directly responding. This conditional routing
defines the main agent loop.
graph_builder.add_conditional_edges(
"chatbot",
route_tools,
# The following dictionary lets you tell the graph to
interpret the condition's outputs as a specific node
# It defaults to the identity function, but if you
# want to use a node named something else apart from "tools",
# You can update the value of the dictionary to something
else
# e.g., "tools": "my_tools"
{"tools": "tools", END: END},
)
# Any time a tool is called, we return to the chatbot to decide
the next step
graph_builder.add_edge("tools", "chatbot")
graph_builder.add_edge(START, "chatbot")
graph = graph_builder.compile()
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 10/61
Let's visualize the graph we've built. The following function has some additional
dependencies to run that are unimportant for this tutorial.
Now we can ask the bot questions outside its training data.
from IPython.display import Image, display
try:
display(Image(graph.get_graph().draw_mermaid_png()))
except Exception:
# This requires some extra dependencies and is optional
pass
while True:
try:
user_input = input("User: ")
if user_input.lower() in ["quit", "exit", "q"]:
print("Goodbye!")
break
stream_graph_updates(user_input)
except:
# fallback if input() is not available
user_input = "What do you know about LangGraph?"
print("User: " + user_input)
stream_graph_updates(user_input)
break
Assistant: [{'text': "To provide you with accurate and up-to-date
information about LangGraph, I'll need to search for the latest de
tails. Let me do that for you.", 'type': 'text'}, {'id': 'toolu_01
Q588CszHaSvvP2MxRq9zRD', 'input': {'query': 'LangGraph AI tool inf
ormation'}, 'name': 'tavily_search_results_json', 'type': 'tool_us
e'}]
Assistant: [{"url": "https://www.langchain.com/langgraph", "conten
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 11/61
t": "LangGraph sets the foundation for how we can build and scale
AI workloads \u2014 from conversational agents, complex task autom
ation, to custom LLM-backed experiences that 'just work'. The next
chapter in building complex production-ready features with LLMs is
agentic, and with LangGraph and LangSmith, LangChain delivers an o
ut-of-the-box solution ..."}, {"url": "https://github.com/langchai
n-ai/langgraph", "content": "Overview. LangGraph is a library for
building stateful, multi-actor applications with LLMs, used to cre
ate agent and multi-agent workflows. Compared to other LLM framewo
rks, it offers these core benefits: cycles, controllability, and p
ersistence. LangGraph allows you to define flows that involve cycl
es, essential for most agentic architectures ..."}]
Assistant: Based on the search results, I can provide you with inf
ormation about LangGraph:
1. Purpose:
LangGraph is a library designed for building stateful, multi-ac
tor applications with Large Language Models (LLMs). It's particula
rly useful for creating agent and multi-agent workflows.
2. Developer:
LangGraph is developed by LangChain, a company known for its to
ols and frameworks in the AI and LLM space.
3. Key Features:
- Cycles: LangGraph allows the definition of flows that involve
cycles, which is essential for most agentic architectures.
- Controllability: It offers enhanced control over the applicat
ion flow.
- Persistence: The library provides ways to maintain state and
persistence in LLM-based applications.
4. Use Cases:
LangGraph can be used for various applications, including:
- Conversational agents
- Complex task automation
- Custom LLM-backed experiences
5. Integration:
LangGraph works in conjunction with LangSmith, another tool by
LangChain, to provide an out-of-the-box solution for building comp
lex, production-ready features with LLMs.
6. Significance:
LangGraph is described as setting the foundation for building a
nd scaling AI workloads. It's positioned as a key tool in the next
chapter of LLM-based application development, particularly in the
realm of agentic AI.
7. Availability:
LangGraph is open-source and available on GitHub, which suggest
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 12/61
Congrats! You've created a conversational agent in langgraph that can use a search
engine to retrieve updated information when needed. Now it can handle a wider
range of user queries. To inspect all the steps your agent just took, check out this
LangSmith trace.
Our chatbot still can't remember past interactions on its own, limiting its ability to
have coherent, multi-turn conversations. In the next part, we'll add memory to
address this.
The full code for the graph we've created in this section is reproduced below,
replacing our BasicToolNode for the prebuilt ToolNode, and our route_tools
condition with the prebuilt tools_condition
s that developers can access and contribute to its codebase.
8. Comparison to Other Frameworks:
LangGraph is noted to offer unique benefits compared to other L
LM frameworks, particularly in its ability to handle cycles, provi
de controllability, and maintain persistence.
LangGraph appears to be a significant tool in the evolving landsca
pe of LLM-based application development, offering developers new w
ays to create more complex, stateful, and interactive AI systems.
Goodbye!
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 13/61
Full Code
from typing import Annotated
from langchain_anthropic import ChatAnthropic
from langchain_community.tools.tavily_search import
TavilySearchResults
from langchain_core.messages import BaseMessage
from typing_extensions import TypedDict
from langgraph.graph import StateGraph
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode, tools_condition
class State(TypedDict):
messages: Annotated[list, add_messages]
graph_builder = StateGraph(State)
tool = TavilySearchResults(max_results=2)
tools = [tool]
llm = ChatAnthropic(model="claude-3-5-sonnet-20240620")
llm_with_tools = llm.bind_tools(tools)
def chatbot(state: State):
return {"messages":
[llm_with_tools.invoke(state["messages"])]}
graph_builder.add_node("chatbot", chatbot)
tool_node = ToolNode(tools=[tool])
graph_builder.add_node("tools", tool_node)
graph_builder.add_conditional_edges(
"chatbot",
tools_condition,
)
# Any time a tool is called, we return to the chatbot to decide
the next step
graph_builder.add_edge("tools", "chatbot")
graph_builder.set_entry_point("chatbot")
graph = graph_builder.compile()
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 14/61
Part 3: Adding Memory to the Chatbot
Our chatbot can now use tools to answer user questions, but it doesn't remember the
context of previous interactions. This limits its ability to have coherent, multi-turn
conversations.
LangGraph solves this problem through persistent checkpointing. If you provide a
checkpointer when compiling the graph and a thread_id when calling your graph,
LangGraph automatically saves the state after each step. When you invoke the graph
again using the same thread_id , the graph loads its saved state, allowing the
chatbot to pick up where it left off.
We will see later that checkpointing is much more powerful than simple chat
memory - it lets you save and resume complex state at any time for error recovery,
human-in-the-loop work?ows, time travel interactions, and more. But before we get
too ahead of ourselves, let's add checkpointing to enable multi-turn conversations.
To get started, create a MemorySaver checkpointer.
Notice we're using an in-memory checkpointer. This is convenient for our tutorial (it
saves it all in-memory). In a production application, you would likely change this to
use SqliteSaver or PostgresSaver and connect to your own DB.
Next de?ne the graph. Now that you've already built your own BasicToolNode , we'll
replace it with LangGraph's prebuilt ToolNode and tools_condition , since these do
some nice things like parallel API execution. Apart from that, the following is all
copied from Part 2.
from langgraph.checkpoint.memory import MemorySaver
memory = MemorySaver()
from typing import Annotated
from langchain_anthropic import ChatAnthropic
from langchain_community.tools.tavily_search import
TavilySearchResults
from langchain_core.messages import BaseMessage
from typing_extensions import TypedDict
from langgraph.graph import StateGraph, START, END
from langgraph.graph.message import add_messages
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 15/61
Finally, compile the graph with the provided checkpointer.
Notice the connectivity of the graph hasn't changed since Part 2. All we are doing is
checkpointing the State as the graph works through each node.
from langgraph.prebuilt import ToolNode, tools_condition
class State(TypedDict):
messages: Annotated[list, add_messages]
graph_builder = StateGraph(State)
tool = TavilySearchResults(max_results=2)
tools = [tool]
llm = ChatAnthropic(model="claude-3-5-sonnet-20240620")
llm_with_tools = llm.bind_tools(tools)
def chatbot(state: State):
return {"messages":
[llm_with_tools.invoke(state["messages"])]}
graph_builder.add_node("chatbot", chatbot)
tool_node = ToolNode(tools=[tool])
graph_builder.add_node("tools", tool_node)
graph_builder.add_conditional_edges(
"chatbot",
tools_condition,
)
# Any time a tool is called, we return to the chatbot to decide
the next step
graph_builder.add_edge("tools", "chatbot")
graph_builder.add_edge(START, "chatbot")
graph = graph_builder.compile(checkpointer=memory)
from IPython.display import Image, display
try:
display(Image(graph.get_graph().draw_mermaid_png()))
except Exception:
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 16/61
Now you can interact with your bot! First, pick a thread to use as the key for this
conversation.
Next, call your chat bot.
Note: The con?g was provided as the second positional argument when calling our
graph. It importantly is not nested within the graph inputs ( {'messages': []} ).
Let's ask a followup: see if it remembers your name.
# This requires some extra dependencies and is optional
pass
config = {"configurable": {"thread_id": "1"}}
user_input = "Hi there! My name is Will."
# The config is the **second positional argument** to stream() or
invoke()!
events = graph.stream(
{"messages": [("user", user_input)]}, config,
stream_mode="values"
)
for event in events:
event["messages"][-1].pretty_print()
================================ Human Message ===================
==============
Hi there! My name is Will.
================================== Ai Message ====================
==============
Hello Will! It's nice to meet you. How can I assist you today? Is
there anything specific you'd like to know or discuss?
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 17/61
Notice that we aren't using an external list for memory: it's all handled by the
checkpointer! You can inspect the full execution in this LangSmith trace to see
what's going on.
Don't believe me? Try this using a different con?g.
user_input = "Remember my name?"
# The config is the **second positional argument** to stream() or
invoke()!
events = graph.stream(
{"messages": [("user", user_input)]}, config,
stream_mode="values"
)
for event in events:
event["messages"][-1].pretty_print()
================================ Human Message ===================
==============
Remember my name?
================================== Ai Message ====================
==============
Of course, I remember your name, Will. I always try to pay attenti
on to important details that users share with me. Is there anythin
g else you'd like to talk about or any questions you have? I'm her
e to help with a wide range of topics or tasks.
# The only difference is we change the `thread_id` here to "2"
instead of "1"
events = graph.stream(
{"messages": [("user", user_input)]},
{"configurable": {"thread_id": "2"}},
stream_mode="values",
)
for event in events:
event["messages"][-1].pretty_print()
================================ Human Message ===================
==============
Remember my name?
================================== Ai Message ====================
==============
I apologize, but I don't have any previous context or memory of yo
ur name. As an AI assistant, I don't retain information from past
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 18/61
Notice that the only change we've made is to modify the thread_id in the con?g.
See this call's LangSmith trace for comparison.
By now, we have made a few checkpoints across two different threads. But what
goes into a checkpoint? To inspect a graph's state for a given con?g at any time,
call get_state(config) .
conversations. Each interaction starts fresh. Could you please tel
l me your name so I can address you properly in this conversation?
snapshot = graph.get_state(config)
snapshot
StateSnapshot(values={'messages': [HumanMessage(content='Hi there!
My name is Will.', additional_kwargs={}, response_metadata={}, id
='8c1ca919-c553-4ebf-95d4-b59a2d61e078'), AIMessage(content="Hello
Will! It's nice to meet you. How can I assist you today? Is there
anything specific you'd like to know or discuss?", additional_kwar
gs={}, response_metadata={'id': 'msg_01WTQebPhNwmMrmmWojJ9KXJ', 'm
odel': 'claude-3-5-sonnet-20240620', 'stop_reason': 'end_turn', 's
top_sequence': None, 'usage': {'input_tokens': 405, 'output_token
s': 32}}, id='run-58587b77-8c82-41e6-8a90-d62c444a261d-0', usage_m
etadata={'input_tokens': 405, 'output_tokens': 32, 'total_tokens':
437}), HumanMessage(content='Remember my name?', additional_kwargs
={}, response_metadata={}, id='daba7df6-ad75-4d6b-8057-745881cea1c
a'), AIMessage(content="Of course, I remember your name, Will. I a
lways try to pay attention to important details that users share w
ith me. Is there anything else you'd like to talk about or any que
stions you have? I'm here to help with a wide range of topics or t
asks.", additional_kwargs={}, response_metadata={'id': 'msg_01E41K
itY74HpENRgXx94vag', 'model': 'claude-3-5-sonnet-20240620', 'stop_
reason': 'end_turn', 'stop_sequence': None, 'usage': {'input_token
s': 444, 'output_tokens': 58}}, id='run-ffeaae5c-4d2d-4ddb-bd59-5d
5cbf2a5af8-0', usage_metadata={'input_tokens': 444, 'output_token
s': 58, 'total_tokens': 502})]}, next=(), config={'configurable':
{'thread_id': '1', 'checkpoint_ns': '', 'checkpoint_id': '1ef7d06e
-93e0-6acc-8004-f2ac846575d2'}}, metadata={'source': 'loop', 'writ
es': {'chatbot': {'messages': [AIMessage(content="Of course, I rem
ember your name, Will. I always try to pay attention to important
details that users share with me. Is there anything else you'd lik
e to talk about or any questions you have? I'm here to help with a
wide range of topics or tasks.", additional_kwargs={}, response_me
tadata={'id': 'msg_01E41KitY74HpENRgXx94vag', 'model': 'claude-3-5
-sonnet-20240620', 'stop_reason': 'end_turn', 'stop_sequence': Non
e, 'usage': {'input_tokens': 444, 'output_tokens': 58}}, id='run-f
feaae5c-4d2d-4ddb-bd59-5d5cbf2a5af8-0', usage_metadata={'input_tok
ens': 444, 'output_tokens': 58, 'total_tokens': 502})]}}, 'step':
4, 'parents': {}}, created_at='2024-09-27T19:30:10.820758+00:00',
parent_config={'configurable': {'thread_id': '1', 'checkpoint_ns':
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 19/61
The snapshot above contains the current state values, corresponding con?g, and the
next node to process. In our case, the graph has reached an END state, so next is
empty.
Congratulations! Your chatbot can now maintain conversation state across sessions
thanks to LangGraph's checkpointing system. This opens up exciting possibilities for
more natural, contextual interactions. LangGraph's checkpointing even handles
arbitrarily complex graph states, which is much more expressive and powerful than
simple chat memory.
In the next part, we'll introduce human oversight to our bot to handle situations
where it may need guidance or veri?cation before proceeding.
Check out the code snippet below to review our graph from this section.
'', 'checkpoint_id': '1ef7d06e-859f-6206-8003-e1bd3c264b8f'}}, tas
ks=())
snapshot.next # (since the graph ended this turn, `next` is
empty. If you fetch a state from within a graph invocation, next
tells which node will execute next)
()
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 20/61
Full Code
from typing import Annotated
from langchain_anthropic import ChatAnthropic
from langchain_community.tools.tavily_search import
TavilySearchResults
from langchain_core.messages import BaseMessage
from typing_extensions import TypedDict
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import StateGraph
from langgraph.graph.message import add_messages
from langgraph.prebuilt import ToolNode
class State(TypedDict):
messages: Annotated[list, add_messages]
graph_builder = StateGraph(State)
tool = TavilySearchResults(max_results=2)
tools = [tool]
llm = ChatAnthropic(model="claude-3-5-sonnet-20240620")
llm_with_tools = llm.bind_tools(tools)
def chatbot(state: State):
return {"messages":
[llm_with_tools.invoke(state["messages"])]}
graph_builder.add_node("chatbot", chatbot)
tool_node = ToolNode(tools=[tool])
graph_builder.add_node("tools", tool_node)
graph_builder.add_conditional_edges(
"chatbot",
tools_condition,
)
graph_builder.add_edge("tools", "chatbot")
graph_builder.set_entry_point("chatbot")
graph = graph_builder.compile(checkpointer=memory)
10/3/24, 10:39 AM Quick Start
https://langchain-ai.github.io/langgraph/tutorials/introduction/#requirements 21/61
Basic Multi-agent Collaboration
A single agent can usually operate effectively using a handful of tools within a single
domain, but even using powerful models like gpt-4 , it can be less effective at using
many tools.
One way to approach complicated tasks is through a "divide-and-conquer" approach:
create an specialized agent for each task or domain and route tasks to the correct
"expert".
This notebook (inspired by the paper AutoGen: Enabling Next-Gen LLM Applications
via Multi-Agent Conversation, by Wu, et. al.) shows one way to do this using
LangGraph.
The resulting graph will look something like the following diagram:
Before we get started, a quick note: this and other multi-agent notebooks are
designed to show how you can implement certain design patterns in LangGraph. If
the pattern suits your needs, we recommend combining it with some of the other
fundamental patterns described elsewhere in the docs for best performance.
10/3/24, 10:33 AM Collaboration
https://langchain-ai.github.io/langgraph/tutorials/multi_agent/multi-agent-collaboration/ 1/48
Setup
First, let's install our required packages and set our API keys:
Sign up for LangSmith to quickly spot issues and improve the performance of your
LangGraph projects. LangSmith lets you use trace data to debug, test, and monitor
your LLM apps built with LangGraph ? read more about how to get started here.
Create Agents
The following helper functions will help create agents. These agents will then be
nodes in the graph.
You can skip ahead if you just want to see what the graph looks like.
%%capture --no-stderr
%pip install -U langchain langchain_openai langsmith pandas
langchain_experimental matplotlib langgraph langchain_core
import getpass
import os
def _set_if_undefined(var: str):
if not os.environ.get(var):
os.environ[var] = getpass.getpass(f"Please provide your 
{var}")
_set_if_undefined("OPENAI_API_KEY")
_set_if_undefined("TAVILY_API_KEY")
Set up LangSmith for LangGraph development
from langchain_core.messages import (
BaseMessage,
HumanMessage,
ToolMessage,
)
from langchain_core.prompts import ChatPromptTemplate,
MessagesPlaceholder
10/3/24, 10:33 AM Collaboration
https://langchain-ai.github.io/langgraph/tutorials/multi_agent/multi-agent-collaboration/ 2/48
De?ne tools
We will also de?ne some tools that our agents will use in the future
from langgraph.graph import END, StateGraph, START
def create_agent(llm, tools, system_message: str):
"""Create an agent."""
prompt = ChatPromptTemplate.from_messages(
[
(
"system",
"You are a helpful AI assistant, collaborating
with other assistants."
" Use the provided tools to progress towards
answering the question."
" If you are unable to fully answer, that's OK,
another assistant with different tools "
" will help where you left off. Execute what you
can to make progress."
" If you or any of the other assistants have the
final answer or deliverable,"
" prefix your response with FINAL ANSWER so the
team knows to stop."
" You have access to the following tools: 
{tool_names}.\n{system_message}",
),
MessagesPlaceholder(variable_name="messages"),
]
)
prompt = prompt.partial(system_message=system_message)
prompt = prompt.partial(tool_names=", ".join([tool.name for
tool in tools]))
return prompt | llm.bind_tools(tools)
from typing import Annotated
from langchain_community.tools.tavily_search import
TavilySearchResults
from langchain_core.tools import tool
from langchain_experimental.utilities import PythonREPL
tavily_tool = TavilySearchResults(max_results=5)
# Warning: This executes code locally, which can be unsafe when
not sandboxed
10/3/24, 10:33 AM Collaboration
https://langchain-ai.github.io/langgraph/tutorials/multi_agent/multi-agent-collaboration/ 3/48
Create graph
Now that we've de?ned our tools and made some helper functions, will create the
individual agents below and tell them how to talk to each other using LangGraph.
De?ne State
We ?rst de?ne the state of the graph. This will just a list of messages, along with a
key to track the most recent sender
repl = PythonREPL()
@tool
def python_repl(
code: Annotated[str, "The python code to execute to generate
your chart."],
):
"""Use this to execute python code. If you want to see the
output of a value,
you should print it out with `print(...)`. This is visible
to the user."""
try:
result = repl.run(code)
except BaseException as e:
return f"Failed to execute. Error: {repr(e)}"
result_str = f"Successfully
executed:\n```python\n{code}\n```\nStdout: {result}"
return (
result_str + "\n\nIf you have completed all tasks,
respond with FINAL ANSWER."
)
import operator
from typing import Annotated, Sequence
from typing_extensions import TypedDict
from langchain_openai import ChatOpenAI
# This defines the object that is passed between each node
# in the graph. We will create different nodes for each agent and
tool
class AgentState(TypedDict):
10/3/24, 10:33 AM Collaboration
https://langchain-ai.github.io/langgraph/tutorials/multi_agent/multi-agent-collaboration/ 4/48
De?ne Agent Nodes
We now need to de?ne the nodes. First, let's de?ne the nodes for the agents.
messages: Annotated[Sequence[BaseMessage], operator.add]
sender: str
import functools
from langchain_core.messages import AIMessage
# Helper function to create a node for a given agent
def agent_node(state, agent, name):
result = agent.invoke(state)
# We convert the agent output into a format that is suitable
to append to the global state
if isinstance(result, ToolMessage):
pass
else:
result = AIMessage(**result.dict(exclude={"type",
"name"}), name=name)
return {
"messages": [result],
# Since we have a strict workflow, we can
# track the sender so we know who to pass to next.
"sender": name,
}
llm = ChatOpenAI(model="gpt-4o")
# Research agent and node
research_agent = create_agent(
llm,
[tavily_tool],
system_message="You should provide accurate data for the
chart_generator to use.",
)
research_node = functools.partial(agent_node,
agent=research_agent, name="Researcher")
# chart_generator
chart_agent = create_agent(
llm,
[python_repl],
system_message="Any charts you display will be visible by
the user.",
10/3/24, 10:33 AM Collaboration
https://langchain-ai.github.io/langgraph/tutorials/multi_agent/multi-agent-collaboration/ 5/48
De?ne Tool Node
We now de?ne a node to run the tools
De?ne Edge Logic
We can de?ne some of the edge logic that is needed to decide what to do based on
results of the agents
De?ne the Graph
We can now put it all together and de?ne the graph!
)
chart_node = functools.partial(agent_node, agent=chart_agent,
name="chart_generator")
from langgraph.prebuilt import ToolNode
tools = [tavily_tool, python_repl]
tool_node = ToolNode(tools)
# Either agent can decide to end
from typing import Literal
def router(state):
# This is the router
messages = state["messages"]
last_message = messages[-1]
if last_message.tool_calls:
# The previous agent is invoking a tool
return "call_tool"
if "FINAL ANSWER" in last_message.content:
# Any agent decided the work is done
return END
return "continue"
workflow = StateGraph(AgentState)
workflow.add_node("Researcher", research_node)
workflow.add_node("chart_generator", chart_node)
workflow.add_node("call_tool", tool_node)
10/3/24, 10:33 AM Collaboration
https://langchain-ai.github.io/langgraph/tutorials/multi_agent/multi-agent-collaboration/ 6/48
workflow.add_conditional_edges(
"Researcher",
router,
{"continue": "chart_generator", "call_tool": "call_tool",
END: END},
)
workflow.add_conditional_edges(
"chart_generator",
router,
{"continue": "Researcher", "call_tool": "call_tool", END:
END},
)
workflow.add_conditional_edges(
"call_tool",
# Each agent node updates the 'sender' field
# the tool calling node does not, meaning
# this edge will route back to the original agent
# who invoked the tool
lambda x: x["sender"],
{
"Researcher": "Researcher",
"chart_generator": "chart_generator",
},
)
workflow.add_edge(START, "Researcher")
graph = workflow.compile()
from IPython.display import Image, display
try:
display(Image(graph.get_graph(xray=True).draw_mermaid_png()))
except Exception:
# This requires some extra dependencies and is optional
pass
10/3/24, 10:33 AM Collaboration
https://langchain-ai.github.io/langgraph/tutorials/multi_agent/multi-agent-collaboration/ 7/48
Invoke
With the graph created, you can invoke it! Let's have it chart some stats for us.
events = graph.stream(
{
"messages": [
HumanMessage(
content="Fetch the UK's GDP over the past 5
years,"
" then draw a line graph of it."
" Once you code it up, finish."
)
],
},
# Maximum number of steps to take in the graph
{"recursion_limit": 150},
)
for s in events:
print(s)
print("----")
{'Researcher': {'messages': [AIMessage(content='', additional_kwar
gs={'tool_calls': [{'id': 'call_3zDlnDMUkWEJxnHASo59doCL', 'functi
on': {'arguments': '{"query":"UK GDP 2018 to 2023"}', 'name': 'tav
ily_search_results_json'}, 'type': 'function'}]}, response_metadat
10/3/24, 10:33 AM Collaboration
https://langchain-ai.github.io/langgraph/tutorials/multi_agent/multi-agent-collaboration/ 8/48
Tutorials Conversational RAG
Conversational RAG
PREREQUISITES
This guide assumes familiarity with the following concepts:
Chat history
Chat models
Embeddings
Vector stores
Retrieval-augmented generation
Tools
Agents
In many Q&A applications we want to allow the user to have a back-and-forth conversation,
meaning the application needs some sort of "memory" of past questions and answers, and
some logic for incorporating those into its current thinking.
In this guide we focus on adding logic for incorporating historical messages. Further details
on chat history management is covered here.
We will cover two approaches:
?? Chains, in which we always execute a retrieval step;
?? Agents, in which we give an LLM discretion over whether and how to execute a retrieval
step (or multiple steps).
For the external knowledge source, we will use the same LLM Powered Autonomous Agents
blog post by Lilian Weng from the RAG tutorial.
Setup
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 1/27
Dependencies
We'll use OpenAI embeddings and a simple in-memory vector store in this walkthrough, but
everything shown here works with any Embeddings, and VectorStore or Retriever.
We'll use the following packages:
We need to set environment variable OPENAI_API_KEY , which can be done directly or loaded
from a .env file like so:
LangSmith
Many of the applications you build with LangChain will contain multiple steps with multiple
invocations of LLM calls. As these applications get more and more complex, it becomes
crucial to be able to inspect what exactly is going on inside your chain or agent. The best way
to do this is with LangSmith.
Note that LangSmith is not needed, but it is helpful. If you do want to use LangSmith, after
you sign up at the link above, make sure to set your environment variables to start logging
traces:
Chains
%%capture --no-stderr
%pip install --upgrade --quiet langchain langchain-community beautifulsoup4
import getpass
import os
if not os.environ.get("OPENAI_API_KEY"):
os.environ["OPENAI_API_KEY"] = getpass.getpass()
os.environ["LANGCHAIN_TRACING_V2"] = "true"
if not os.environ.get("LANGCHAIN_API_KEY"):
os.environ["LANGCHAIN_API_KEY"] = getpass.getpass()
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 2/27
Let's first revisit the Q&A app we built over the LLM Powered Autonomous Agents blog post
by Lilian Weng in the RAG tutorial.
OpenAI Anthropic Azure Google Cohere NVIDIA FireworksAI Gr
pip install -qU langchain-openai
import getpass
import os
os.environ["OPENAI_API_KEY"] = getpass.getpass()
from langchain_openai import ChatOpenAI
llm = ChatOpenAI(model="gpt-4o-mini")
import bs4
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_community.document_loaders import WebBaseLoader
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_openai import OpenAIEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter
# 1. Load, chunk and index the contents of the blog to create a retriever.
loader = WebBaseLoader(
web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
bs_kwargs=dict(
parse_only=bs4.SoupStrainer(
class_=("post-content", "post-title", "post-header")
)
),
)
docs = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, 
chunk_overlap=200)
splits = text_splitter.split_documents(docs)
vectorstore = InMemoryVectorStore.from_documents(
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 3/27
API Reference: create_retrieval_chain | create_stuff_documents_chain | WebBaseLoader |
ChatPromptTemplate | InMemoryVectorStore | OpenAIEmbeddings |
RecursiveCharacterTextSplitter
Note that we have used the built-in chain constructors create_stuff_documents_chain and
create_retrieval_chain , so that the basic ingredients to our solution are:
documents=splits, embedding=OpenAIEmbeddings()
)
retriever = vectorstore.as_retriever()
# 2. Incorporate the retriever into a question-answering chain.
system_prompt = (
"You are an assistant for question-answering tasks. "
"Use the following pieces of retrieved context to answer "
"the question. If you don't know the answer, say that you "
"don't know. Use three sentences maximum and keep the "
"answer concise."
"\n\n"
"{context}"
)
prompt = ChatPromptTemplate.from_messages(
[
("system", system_prompt),
("human", "{input}"),
]
)
question_answer_chain = create_stuff_documents_chain(llm, prompt)
rag_chain = create_retrieval_chain(retriever, question_answer_chain)
response = rag_chain.invoke({"input": "What is Task Decomposition?"})
response["answer"]
"Task decomposition is the process of breaking down a complicated task into 
smaller, more manageable steps. Techniques like Chain of Thought (CoT) and 
Tree of Thoughts enhance this process by guiding models to think step by step 
and explore multiple reasoning possibilities. This approach helps in 
simplifying complex tasks and provides insight into the model's reasoning."
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 4/27
?? retriever;
?? prompt;
?? LLM.
This will simplify the process of incorporating chat history.
Adding chat history
The chain we have built uses the input query directly to retrieve relevant context. But in a
conversational setting, the user query might require conversational context to be
understood. For example, consider this exchange:
Human: "What is Task Decomposition?"
AI: "Task decomposition involves breaking down complex tasks into smaller and simpler
steps to make them more manageable for an agent or model."
Human: "What are common ways of doing it?"
In order to answer the second question, our system needs to understand that "it" refers to
"Task Decomposition."
We'll need to update two things about our existing app:
?? Prompt: Update our prompt to support historical messages as an input.
?? Contextualizing questions: Add a sub-chain that takes the latest user question and
reformulates it in the context of the chat history. This can be thought of simply as
building a new "history aware" retriever. Whereas before we had:
query -> retriever
Now we will have:
(query, conversation history) -> LLM -> rephrased query -> retriever
Contextualizing the question
First we'll need to define a sub-chain that takes historical messages and the latest user
question, and reformulates the question if it makes reference to any information in the
historical information.
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 5/27
We'll use a prompt that includes a MessagesPlaceholder variable under the name
"chat_history". This allows us to pass in a list of Messages to the prompt using the
"chat_history" input key, and these messages will be inserted after the system message and
before the human message containing the latest question.
Note that we leverage a helper function create_history_aware_retriever for this step, which
manages the case where chat_history is empty, and otherwise applies prompt | llm |
StrOutputParser() | retriever in sequence.
create_history_aware_retriever constructs a chain that accepts keys input and
chat_history as input, and has the same output schema as a retriever.
API Reference: create_history_aware_retriever | MessagesPlaceholder
This chain prepends a rephrasing of the input query to our retriever, so that the retrieval
incorporates the context of the conversation.
from langchain.chains import create_history_aware_retriever
from langchain_core.prompts import MessagesPlaceholder
contextualize_q_system_prompt = (
"Given a chat history and the latest user question "
"which might reference context in the chat history, "
"formulate a standalone question which can be understood "
"without the chat history. Do NOT answer the question, "
"just reformulate it if needed and otherwise return it as is."
)
contextualize_q_prompt = ChatPromptTemplate.from_messages(
[
("system", contextualize_q_system_prompt),
MessagesPlaceholder("chat_history"),
("human", "{input}"),
]
)
history_aware_retriever = create_history_aware_retriever(
llm, retriever, contextualize_q_prompt
)
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 6/27
Now we can build our full QA chain. This is as simple as updating the retriever to be our new
history_aware_retriever .
Again, we will use create_stuff_documents_chain to generate a question_answer_chain ,
with input keys context , chat_history , and input -- it accepts the retrieved context
alongside the conversation history and query to generate an answer. A more detailed
explaination is over here
We build our final rag_chain with create_retrieval_chain. This chain applies the
history_aware_retriever and question_answer_chain in sequence, retaining intermediate
outputs such as the retrieved context for convenience. It has input keys input and
chat_history , and includes input , chat_history , context , and answer in its output.
API Reference: create_retrieval_chain | create_stuff_documents_chain
Let's try this. Below we ask a question and a follow-up question that requires
contextualization to return a sensible response. Because our chain includes a
"chat_history" input, the caller needs to manage the chat history. We can achieve this by
appending input and output messages to a list:
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
qa_prompt = ChatPromptTemplate.from_messages(
[
("system", system_prompt),
MessagesPlaceholder("chat_history"),
("human", "{input}"),
]
)
question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)
rag_chain = create_retrieval_chain(history_aware_retriever, 
question_answer_chain)
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 7/27
API Reference: AIMessage | HumanMessage
TIP
Check out the LangSmith trace.
Stateful management of chat history
NOTE
This section of the tutorial previously used the RunnableWithMessageHistory
abstraction. You can access that version of the documentation in the v0.2 docs.
As of the v0.3 release of LangChain, we recommend that LangChain users take
advantage of LangGraph persistence to incorporate memory into new LangChain
from langchain_core.messages import AIMessage, HumanMessage
chat_history = []
question = "What is Task Decomposition?"
ai_msg_1 = rag_chain.invoke({"input": question, "chat_history": 
chat_history})
chat_history.extend(
[
HumanMessage(content=question),
AIMessage(content=ai_msg_1["answer"]),
]
)
second_question = "What are common ways of doing it?"
ai_msg_2 = rag_chain.invoke({"input": second_question, "chat_history": 
chat_history})
print(ai_msg_2["answer"])
Common ways of task decomposition include using simple prompting techniques, 
such as asking for "Steps for XYZ" or "What are the subgoals for achieving 
XYZ?" Additionally, task-specific instructions can be employed, like "Write a 
story outline" for writing tasks, or human inputs can guide the decomposition 
process.
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 8/27
applications.
If your code is already relying on RunnableWithMessageHistory or
BaseChatMessageHistory , you do not need to make any changes. We do not plan on
deprecating this functionality in the near future as it works for simple chat applications
and any code that uses RunnableWithMessageHistory will continue to work as
expected.
Please see How to migrate to LangGraph Memory for more details.
We have added application logic for incorporating chat history, but we are still manually
plumbing it through our application. In production, the Q&A application will usually persist
the chat history into a database, and be able to read and update it appropriately.
LangGraph implements a built-in persistence layer, making it ideal for chat applications that
support multiple conversational turns.
Wrapping our chat model in a minimal LangGraph application allows us to automatically
persist the message history, simplifying the development of multi-turn applications.
LangGraph comes with a simple in-memory checkpointer, which we use below. See its
documentation for more detail, including how to use different persistence backends (e.g.,
SQLite or Postgres).
For a detailed walkthrough of how to manage message history, head to the How to add
message history (memory) guide.
from typing import Sequence
from langchain_core.messages import BaseMessage
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import START, StateGraph
from langgraph.graph.message import add_messages
from typing_extensions import Annotated, TypedDict
# We define a dict representing the state of the application.
# This state has the same input and output keys as `rag_chain`.
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 9/27
API Reference: BaseMessage | MemorySaver | StateGraph | add_messages
This application out-of-the-box supports multiple conversation threads. We pass in a
configuration dict specifying a unique identifier for a thread to control what thread is run.
This enables the application to support interactions with multiple users.
class State(TypedDict):
input: str
chat_history: Annotated[Sequence[BaseMessage], add_messages]
context: str
answer: str
# We then define a simple node that runs the `rag_chain`.
# The `return` values of the node update the graph state, so here we just
# update the chat history with the input message and response.
def call_model(state: State):
response = rag_chain.invoke(state)
return {
"chat_history": [
HumanMessage(state["input"]),
AIMessage(response["answer"]),
],
"context": response["context"],
"answer": response["answer"],
}
# Our graph consists only of one node:
workflow = StateGraph(state_schema=State)
workflow.add_edge(START, "model")
workflow.add_node("model", call_model)
# Finally, we compile the graph with a checkpointer object.
# This persists the state, in this case in memory.
memory = MemorySaver()
app = workflow.compile(checkpointer=memory)
config = {"configurable": {"thread_id": "abc123"}}
result = app.invoke(
{"input": "What is Task Decomposition?"},
config=config,
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 10/27
The conversation history can be inspected via the state of the application:
)
print(result["answer"])
Task decomposition is the process of breaking down a complicated task into 
smaller, more manageable steps. Techniques like Chain of Thought (CoT) and 
Tree of Thoughts enhance this process by guiding models to think step by step 
and explore multiple reasoning possibilities. This approach helps in 
simplifying complex tasks and provides insight into the model's reasoning.
result = app.invoke(
{"input": "What is one way of doing it?"},
config=config,
)
print(result["answer"])
One way of doing task decomposition is by using simple prompting, such as 
asking the model, "What are the subgoals for achieving XYZ?" This method 
encourages the model to identify and outline the smaller tasks needed to 
accomplish the larger goal.
chat_history = app.get_state(config).values["chat_history"]
for message in chat_history:
message.pretty_print()
================================?[1m Human Message 
?[0m=================================
What is Task Decomposition?
==================================?[1m Ai Message 
?[0m==================================
Task decomposition is the process of breaking down a complicated task into 
smaller, more manageable steps. Techniques like Chain of Thought (CoT) and 
Tree of Thoughts enhance this process by guiding models to think step by step 
and explore multiple reasoning possibilities. This approach helps in 
simplifying complex tasks and provides insight into the model's reasoning.
================================?[1m Human Message 
?[0m=================================
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 11/27
Tying it together
For convenience, we tie together all of the necessary steps in a single code cell:
What is one way of doing it?
==================================?[1m Ai Message 
?[0m==================================
One way of doing task decomposition is by using simple prompting, such as 
asking the model, "What are the subgoals for achieving XYZ?" This method 
encourages the model to identify and outline the smaller tasks needed to 
accomplish the larger goal.
from typing import Sequence
import bs4
from langchain.chains import create_history_aware_retriever, 
create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_community.document_loaders import WebBaseLoader
from langchain_core.messages import AIMessage, BaseMessage, HumanMessage
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langgraph.checkpoint.memory import MemorySaver
from langgraph.graph import START, StateGraph
from langgraph.graph.message import add_messages
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 12/27
from typing_extensions import Annotated, TypedDict
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
### Construct retriever ###
loader = WebBaseLoader(
web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
bs_kwargs=dict(
parse_only=bs4.SoupStrainer(
class_=("post-content", "post-title", "post-header")
)
),
)
docs = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, 
chunk_overlap=200)
splits = text_splitter.split_documents(docs)
vectorstore = InMemoryVectorStore.from_documents(
documents=splits, embedding=OpenAIEmbeddings()
)
retriever = vectorstore.as_retriever()
### Contextualize question ###
contextualize_q_system_prompt = (
"Given a chat history and the latest user question "
"which might reference context in the chat history, "
"formulate a standalone question which can be understood "
"without the chat history. Do NOT answer the question, "
"just reformulate it if needed and otherwise return it as is."
)
contextualize_q_prompt = ChatPromptTemplate.from_messages(
[
("system", contextualize_q_system_prompt),
MessagesPlaceholder("chat_history"),
("human", "{input}"),
]
)
history_aware_retriever = create_history_aware_retriever(
llm, retriever, contextualize_q_prompt
)
### Answer question ###
system_prompt = (
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 13/27
"You are an assistant for question-answering tasks. "
"Use the following pieces of retrieved context to answer "
"the question. If you don't know the answer, say that you "
"don't know. Use three sentences maximum and keep the "
"answer concise."
"\n\n"
"{context}"
)
qa_prompt = ChatPromptTemplate.from_messages(
[
("system", system_prompt),
MessagesPlaceholder("chat_history"),
("human", "{input}"),
]
)
question_answer_chain = create_stuff_documents_chain(llm, qa_prompt)
rag_chain = create_retrieval_chain(history_aware_retriever, 
question_answer_chain)
### Statefully manage chat history ###
class State(TypedDict):
input: str
chat_history: Annotated[Sequence[BaseMessage], add_messages]
context: str
answer: str
def call_model(state: State):
response = rag_chain.invoke(state)
return {
"chat_history": [
HumanMessage(state["input"]),
AIMessage(response["answer"]),
],
"context": response["context"],
"answer": response["answer"],
}
workflow = StateGraph(state_schema=State)
workflow.add_edge(START, "model")
workflow.add_node("model", call_model)
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 14/27
API Reference: create_history_aware_retriever | create_retrieval_chain |
create_stuff_documents_chain | WebBaseLoader | AIMessage | BaseMessage |
HumanMessage | ChatPromptTemplate | MessagesPlaceholder | InMemoryVectorStore |
ChatOpenAI | OpenAIEmbeddings | RecursiveCharacterTextSplitter | MemorySaver |
StateGraph | add_messages
Agents
memory = MemorySaver()
app = workflow.compile(checkpointer=memory)
config = {"configurable": {"thread_id": "abc123"}}
result = app.invoke(
{"input": "What is Task Decomposition?"},
config=config,
)
print(result["answer"])
Task decomposition is the process of breaking down a complicated task into 
smaller, more manageable steps. Techniques like Chain of Thought (CoT) and 
Tree of Thoughts enhance this process by guiding models to think step by step 
and explore multiple reasoning possibilities. This approach helps in 
simplifying complex tasks and improving the model's performance.
result = app.invoke(
{"input": "What is one way of doing it?"},
config=config,
)
print(result["answer"])
One way of doing task decomposition is by using simple prompting, such as 
asking the model, "What are the subgoals for achieving XYZ?" This method 
encourages the model to identify and outline the smaller steps needed to 
complete the larger task.
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 15/27
Agents leverage the reasoning capabilities of LLMs to make decisions during execution.
Using agents allow you to offload some discretion over the retrieval process. Although their
behavior is less predictable than chains, they offer some advantages in this context:
Agents generate the input to the retriever directly, without necessarily needing us to
explicitly build in contextualization, as we did above;
Agents can execute multiple retrieval steps in service of a query, or refrain from
executing a retrieval step altogether (e.g., in response to a generic greeting from a user).
Retrieval tool
Agents can access "tools" and manage their execution. In this case, we will convert our
retriever into a LangChain tool to be wielded by the agent:
API Reference: create_retriever_tool
Tools are LangChain Runnables, and implement the usual interface:
from langchain.tools.retriever import create_retriever_tool
tool = create_retriever_tool(
retriever,
"blog_post_retriever",
"Searches and returns excerpts from the Autonomous Agents blog post.",
)
tools = [tool]
tool.invoke("task decomposition")
'Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple 
reasoning possibilities at each step. It first decomposes the problem into 
multiple thought steps and generates multiple thoughts per step, creating a 
tree structure. The search process can be BFS (breadth-first search) or DFS 
(depth-first search) with each state evaluated by a classifier (via a prompt) 
or majority vote.\nTask decomposition can be done (1) by LLM with simple 
prompting like "Steps for XYZ.\\n1.", "What are the subgoals for achieving 
XYZ?", (2) by using task-specific instructions; e.g. "Write a story outline." 
for writing a novel, or (3) with human inputs.\n\nFig. 1. Overview of a LLM-
powered autonomous agent system.\nComponent One: Planning#\nA complicated task 
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 16/27
Agent constructor
Now that we have defined the tools and the LLM, we can create the agent. We will be using
LangGraph to construct the agent. Currently we are using a high level interface to construct
the agent, but the nice thing about LangGraph is that this high-level interface is backed by a
low-level, highly controllable API in case you want to modify the agent logic.
API Reference: create_react_agent
We can now try it out. Note that so far it is not stateful (we still need to add in memory)
usually involves many steps. An agent needs to know what they are and plan 
ahead.\nTask Decomposition#\nChain of thought (CoT; Wei et al. 2022) has 
become a standard prompting technique for enhancing model performance on 
complex tasks. The model is instructed to “think step by step” to utilize more 
test-time computation to decompose hard tasks into smaller and simpler steps. 
CoT transforms big tasks into multiple manageable tasks and shed lights into 
an interpretation of the model’s thinking process.\n\n(3) Task execution: 
Expert models execute on the specific tasks and log 
results.\nInstruction:\n\nWith the input and the inference results, the AI 
assistant needs to describe the process and results. The previous stages can 
be formed as - User Input: {{ User Input }}, Task Planning: {{ Tasks }}, Model 
Selection: {{ Model Assignment }}, Task Execution: {{ Predictions }}. You must 
first answer the user\'s request in a straightforward manner. Then describe 
the task process and show your analysis and model inference results to the 
user in the first person. If inference results contain a file path, must tell 
the user the complete file path.\n\nFig. 11. Illustration of how HuggingGPT 
works. (Image source: Shen et al. 2023)\nThe system comprises of 4 
stages:\n(1) Task planning: LLM works as the brain and parses the user 
requests into multiple tasks. There are four attributes associated with each 
task: task type, ID, dependencies, and arguments. They use few-shot examples 
to guide LLM to do task parsing and planning.\nInstruction:'
from langgraph.prebuilt import create_react_agent
agent_executor = create_react_agent(llm, tools)
query = "What is Task Decomposition?"
for event in agent_executor.stream(
{"messages": [HumanMessage(content=query)]},
stream_mode="values",
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 17/27
):
event["messages"][-1].pretty_print()
================================?[1m Human Message 
?[0m=================================
What is Task Decomposition?
==================================?[1m Ai Message 
?[0m==================================
Tool Calls:
blog_post_retriever (call_WKHdiejvg4In982Hr3EympuI)
Call ID: call_WKHdiejvg4In982Hr3EympuI
Args:
query: Task Decomposition
=================================?[1m Tool Message 
?[0m=================================
Name: blog_post_retriever
Fig. 1. Overview of a LLM-powered autonomous agent system.
Component One: Planning#
A complicated task usually involves many steps. An agent needs to know what 
they are and plan ahead.
Task Decomposition#
Chain of thought (CoT; Wei et al. 2022) has become a standard prompting 
technique for enhancing model performance on complex tasks. The model is 
instructed to “think step by step” to utilize more test-time computation to 
decompose hard tasks into smaller and simpler steps. CoT transforms big tasks 
into multiple manageable tasks and shed lights into an interpretation of the 
model’s thinking process.
Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning 
possibilities at each step. It first decomposes the problem into multiple 
thought steps and generates multiple thoughts per step, creating a tree 
structure. The search process can be BFS (breadth-first search) or DFS (depth-
first search) with each state evaluated by a classifier (via a prompt) or 
majority vote.
Task decomposition can be done (1) by LLM with simple prompting like "Steps 
for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-
specific instructions; e.g. "Write a story outline." for writing a novel, or 
(3) with human inputs.
(3) Task execution: Expert models execute on the specific tasks and log 
results.
Instruction:
With the input and the inference results, the AI assistant needs to describe 
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 18/27
the process and results. The previous stages can be formed as - User Input: {{ 
User Input }}, Task Planning: {{ Tasks }}, Model Selection: {{ Model 
Assignment }}, Task Execution: {{ Predictions }}. You must first answer the 
user's request in a straightforward manner. Then describe the task process and 
show your analysis and model inference results to the user in the first 
person. If inference results contain a file path, must tell the user the 
complete file path.
Fig. 11. Illustration of how HuggingGPT works. (Image source: Shen et al. 
2023)
The system comprises of 4 stages:
(1) Task planning: LLM works as the brain and parses the user requests into 
multiple tasks. There are four attributes associated with each task: task 
type, ID, dependencies, and arguments. They use few-shot examples to guide LLM 
to do task parsing and planning.
Instruction:
==================================?[1m Ai Message 
?[0m==================================
Task Decomposition is a process used in complex problem-solving where a larger 
task is broken down into smaller, more manageable sub-tasks. This approach 
enhances the ability of models, particularly large language models (LLMs), to 
handle intricate tasks by allowing them to think step by step.
There are several methods for task decomposition:
1. **Chain of Thought (CoT)**: This technique encourages the model to 
articulate its reasoning process by thinking through the task in a sequential 
manner. It transforms a big task into smaller, manageable steps, which also 
provides insight into the model's thought process.
2. **Tree of Thoughts**: An extension of CoT, this method explores multiple 
reasoning possibilities at each step. It decomposes the problem into various 
thought steps and generates multiple thoughts for each step, creating a tree 
structure. The evaluation of each state can be done using breadth-first search 
(BFS) or depth-first search (DFS).
3. **Prompting Techniques**: Task decomposition can be achieved through simple 
prompts like "Steps for XYZ" or "What are the subgoals for achieving XYZ?" 
Additionally, task-specific instructions can guide the model, such as asking 
it to "Write a story outline" for creative tasks.
4. **Human Inputs**: In some cases, human guidance can be used to assist in 
breaking down tasks.
Overall, task decomposition is a crucial component in planning and executing 
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 19/27
We can again take advantage of LangGraph's built-in persistence to save stateful updates to
memory:
API Reference: MemorySaver
This is all we need to construct a conversational RAG agent.
Let's observe its behavior. Note that if we input a query that does not require a retrieval step,
the agent does not execute one:
Further, if we input a query that does require a retrieval step, the agent generates the input
to the tool:
complex tasks, allowing for better organization and clarity in the problem-
solving process.
from langgraph.checkpoint.memory import MemorySaver
memory = MemorySaver()
agent_executor = create_react_agent(llm, tools, checkpointer=memory)
config = {"configurable": {"thread_id": "abc123"}}
for event in agent_executor.stream(
{"messages": [HumanMessage(content="Hi! I'm bob")]},
config=config,
stream_mode="values",
):
event["messages"][-1].pretty_print()
================================?[1m Human Message 
?[0m=================================
Hi! I'm bob
==================================?[1m Ai Message 
?[0m==================================
Hello Bob! How can I assist you today?
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 20/27
query = "What is Task Decomposition?"
for event in agent_executor.stream(
{"messages": [HumanMessage(content=query)]},
config=config,
stream_mode="values",
):
event["messages"][-1].pretty_print()
================================?[1m Human Message 
?[0m=================================
What is Task Decomposition?
==================================?[1m Ai Message 
?[0m==================================
Tool Calls:
blog_post_retriever (call_0rhrUJiHkoOQxwqCpKTkSkiu)
Call ID: call_0rhrUJiHkoOQxwqCpKTkSkiu
Args:
query: Task Decomposition
=================================?[1m Tool Message 
?[0m=================================
Name: blog_post_retriever
Fig. 1. Overview of a LLM-powered autonomous agent system.
Component One: Planning#
A complicated task usually involves many steps. An agent needs to know what 
they are and plan ahead.
Task Decomposition#
Chain of thought (CoT; Wei et al. 2022) has become a standard prompting 
technique for enhancing model performance on complex tasks. The model is 
instructed to “think step by step” to utilize more test-time computation to 
decompose hard tasks into smaller and simpler steps. CoT transforms big tasks 
into multiple manageable tasks and shed lights into an interpretation of the 
model’s thinking process.
Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning 
possibilities at each step. It first decomposes the problem into multiple 
thought steps and generates multiple thoughts per step, creating a tree 
structure. The search process can be BFS (breadth-first search) or DFS (depth-
first search) with each state evaluated by a classifier (via a prompt) or 
majority vote.
Task decomposition can be done (1) by LLM with simple prompting like "Steps 
for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-
specific instructions; e.g. "Write a story outline." for writing a novel, or 
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 21/27
(3) with human inputs.
(3) Task execution: Expert models execute on the specific tasks and log 
results.
Instruction:
With the input and the inference results, the AI assistant needs to describe 
the process and results. The previous stages can be formed as - User Input: {{ 
User Input }}, Task Planning: {{ Tasks }}, Model Selection: {{ Model 
Assignment }}, Task Execution: {{ Predictions }}. You must first answer the 
user's request in a straightforward manner. Then describe the task process and 
show your analysis and model inference results to the user in the first 
person. If inference results contain a file path, must tell the user the 
complete file path.
Fig. 11. Illustration of how HuggingGPT works. (Image source: Shen et al. 
2023)
The system comprises of 4 stages:
(1) Task planning: LLM works as the brain and parses the user requests into 
multiple tasks. There are four attributes associated with each task: task 
type, ID, dependencies, and arguments. They use few-shot examples to guide LLM 
to do task parsing and planning.
Instruction:
==================================?[1m Ai Message 
?[0m==================================
Task Decomposition is a technique used to break down complex tasks into 
smaller, more manageable steps. This approach is particularly useful in the 
context of autonomous agents and large language models (LLMs). Here are some 
key points about Task Decomposition:
1. **Chain of Thought (CoT)**: This is a prompting technique that encourages 
the model to "think step by step." By doing so, it can utilize more 
computational resources to decompose difficult tasks into simpler ones, making 
them easier to handle.
2. **Tree of Thoughts**: An extension of CoT, this method explores multiple 
reasoning possibilities at each step. It decomposes a problem into various 
thought steps and generates multiple thoughts for each step, creating a tree 
structure. This can be evaluated using search methods like breadth-first 
search (BFS) or depth-first search (DFS).
3. **Methods of Decomposition**: Task decomposition can be achieved through:
- Simple prompting (e.g., asking for steps to achieve a goal).
- Task-specific instructions (e.g., requesting a story outline for 
writing).
- Human inputs to guide the decomposition process.
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 22/27
Above, instead of inserting our query verbatim into the tool, the agent stripped unnecessary
words like "what" and "is".
This same principle allows the agent to use the context of the conversation when necessary:
4. **Execution**: After decomposition, expert models execute the specific 
tasks and log the results, allowing for a structured approach to complex 
problem-solving.
Overall, Task Decomposition enhances the model's ability to tackle intricate 
tasks by breaking them down into simpler, actionable components.
query = "What according to the blog post are common ways of doing it? redo 
the search"
for event in agent_executor.stream(
{"messages": [HumanMessage(content=query)]},
config=config,
stream_mode="values",
):
event["messages"][-1].pretty_print()
================================?[1m Human Message 
?[0m=================================
What according to the blog post are common ways of doing it? redo the search
==================================?[1m Ai Message 
?[0m==================================
Tool Calls:
blog_post_retriever (call_bZRDF6Xr0QdurM9LItM8cN7a)
Call ID: call_bZRDF6Xr0QdurM9LItM8cN7a
Args:
query: common ways of Task Decomposition
=================================?[1m Tool Message 
?[0m=================================
Name: blog_post_retriever
Tree of Thoughts (Yao et al. 2023) extends CoT by exploring multiple reasoning 
possibilities at each step. It first decomposes the problem into multiple 
thought steps and generates multiple thoughts per step, creating a tree 
structure. The search process can be BFS (breadth-first search) or DFS (depth-
first search) with each state evaluated by a classifier (via a prompt) or 
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 23/27
majority vote.
Task decomposition can be done (1) by LLM with simple prompting like "Steps 
for XYZ.\n1.", "What are the subgoals for achieving XYZ?", (2) by using task-
specific instructions; e.g. "Write a story outline." for writing a novel, or 
(3) with human inputs.
Fig. 1. Overview of a LLM-powered autonomous agent system.
Component One: Planning#
A complicated task usually involves many steps. An agent needs to know what 
they are and plan ahead.
Task Decomposition#
Chain of thought (CoT; Wei et al. 2022) has become a standard prompting 
technique for enhancing model performance on complex tasks. The model is 
instructed to “think step by step” to utilize more test-time computation to 
decompose hard tasks into smaller and simpler steps. CoT transforms big tasks 
into multiple manageable tasks and shed lights into an interpretation of the 
model’s thinking process.
Resources:
1. Internet access for searches and information gathering.
2. Long Term memory management.
3. GPT-3.5 powered Agents for delegation of simple tasks.
4. File output.
Performance Evaluation:
1. Continuously review and analyze your actions to ensure you are performing 
to the best of your abilities.
2. Constructively self-criticize your big-picture behavior constantly.
3. Reflect on past decisions and strategies to refine your approach.
4. Every command has a cost, so be smart and efficient. Aim to complete tasks 
in the least number of steps.
(3) Task execution: Expert models execute on the specific tasks and log 
results.
Instruction:
With the input and the inference results, the AI assistant needs to describe 
the process and results. The previous stages can be formed as - User Input: {{ 
User Input }}, Task Planning: {{ Tasks }}, Model Selection: {{ Model 
Assignment }}, Task Execution: {{ Predictions }}. You must first answer the 
user's request in a straightforward manner. Then describe the task process and 
show your analysis and model inference results to the user in the first 
person. If inference results contain a file path, must tell the user the 
complete file path.
==================================?[1m Ai Message 
?[0m==================================
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 24/27
Note that the agent was able to infer that "it" in our query refers to "task decomposition", and
generated a reasonable search query as a result-- in this case, "common ways of task
decomposition".
Tying it together
For convenience, we tie together all of the necessary steps in a single code cell:
According to the blog post, common ways to perform Task Decomposition include:
1. **Simple Prompting**: Using straightforward prompts such as "Steps for 
XYZ.\n1." or "What are the subgoals for achieving XYZ?" to guide the model in 
breaking down the task.
2. **Task-Specific Instructions**: Providing specific instructions tailored to 
the task at hand, such as asking for a "story outline" when writing a novel.
3. **Human Inputs**: Involving human guidance or input to assist in the 
decomposition process, allowing for a more nuanced understanding of the task 
requirements.
These methods help in transforming complex tasks into smaller, manageable 
components, facilitating better planning and execution.
import bs4
from langchain.tools.retriever import create_retriever_tool
from langchain_community.document_loaders import WebBaseLoader
from langchain_core.vectorstores import InMemoryVectorStore
from langchain_openai import ChatOpenAI, OpenAIEmbeddings
from langchain_text_splitters import RecursiveCharacterTextSplitter
from langgraph.checkpoint.memory import MemorySaver
from langgraph.prebuilt import create_react_agent
memory = MemorySaver()
llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
### Construct retriever ###
loader = WebBaseLoader(
web_paths=("https://lilianweng.github.io/posts/2023-06-23-agent/",),
bs_kwargs=dict(
parse_only=bs4.SoupStrainer(
class_=("post-content", "post-title", "post-header")
)
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 25/27
API Reference: create_retriever_tool | WebBaseLoader | InMemoryVectorStore |
ChatOpenAI | OpenAIEmbeddings | RecursiveCharacterTextSplitter | MemorySaver |
create_react_agent
Next steps
We've covered the steps to build a basic conversational Q&A application:
We used chains to build a predictable application that generates search queries for each
user input;
We used agents to build an application that "decides" when and how to generate search
queries.
To explore different types of retrievers and retrieval strategies, visit the retrievers section of
the how-to guides.
),
)
docs = loader.load()
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, 
chunk_overlap=200)
splits = text_splitter.split_documents(docs)
vectorstore = InMemoryVectorStore.from_documents(
documents=splits, embedding=OpenAIEmbeddings()
)
retriever = vectorstore.as_retriever()
### Build retriever tool ###
tool = create_retriever_tool(
retriever,
"blog_post_retriever",
"Searches and returns excerpts from the Autonomous Agents blog post.",
)
tools = [tool]
agent_executor = create_react_agent(llm, tools, checkpointer=memory)
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 26/27
For a detailed walkthrough of LangChain's conversation memory abstractions, visit the How
to add message history (memory) guide.
To learn more about agents, head to the Agents Modules.
Edit this page
Was this page helpful?
You can also leave detailed feedback on GitHub.
10/3/24, 10:26 AM Conversational RAG | 🦜 🔗 LangChain
https://python.langchain.com/docs/tutorials/qa_chat_history/#agents 27/27