Package: langgraph
🦜🕸️LangGraph⚡ Building language agents as graphs ⚡OverviewLangGraphis a library for building stateful, multi-actor applications with LLMs, built on top of (and intended to be used with)LangChain.
It extends theLangChain Expression Languagewith the ability to coordinate multiple chains (or actors) across multiple steps of computation in a cyclic manner.
It is inspired byPregelandApache Beam.
The current interface exposed is one inspired byNetworkX.The main use is for addingcyclesto your LLM application.
Crucially, this is NOT aDAGframework.
If you want to build a DAG, you should just useLangChain Expression Language.Cycles are important for agent-like behaviors, where you call an LLM in a loop, asking it what action to take next.InstallationpipinstalllanggraphQuick StartHere we will go over an example of creating a simple agent that uses chat models and function calling.
This agent will represent all its state as a list of messages.We will need to install some LangChain packages, as well asTavilyto use as an example tool.pipinstall-Ulangchainlangchain_openaitavily-pythonWe also need to export some environment variables for OpenAI and Tavily API access.exportOPENAI_API_KEY=sk-...exportTAVILY_API_KEY=tvly-...Optionally, we can set upLangSmithfor best-in-class observability.exportLANGCHAIN_TRACING_V2="true"exportLANGCHAIN_API_KEY=ls__...Set up the toolsWe will first define the tools we want to use.
For this simple example, we will use a built-in search tool via Tavily.
However, it is really easy to create your own tools - see documentationhereon how to do that.fromlangchain_community.tools.tavily_searchimportTavilySearchResultstools=[TavilySearchResults(max_results=1)]We can now wrap these tools in a simple LangGraphToolExecutor.
This is a simple class that receivesToolInvocationobjects, calls that tool, and returns the output.ToolInvocationis any class withtoolandtool_inputattributes.fromlanggraph.prebuiltimportToolExecutortool_executor=ToolExecutor(tools)Set up the modelNow we need to load the chat model we want to use.
Importantly, this should satisfy two criteria:It should work with lists of messages. We will represent all agent state in the form of messages, so it needs to be able to work well with them.It should work with the OpenAI function calling interface. This means it should either be an OpenAI model or a model that exposes a similar interface.Note: these model requirements are not requirements for using LangGraph - they are just requirements for this one example.fromlangchain_openaiimportChatOpenAI# We will set streaming=True so that we can stream tokens# See the streaming section for more information on this.model=ChatOpenAI(temperature=0,streaming=True)After we've done this, we should make sure the model knows that it has these tools available to call.
We can do this by converting the LangChain tools into the format for OpenAI function calling, and then bind them to the model class.fromlangchain.tools.renderimportformat_tool_to_openai_functionfunctions=[format_tool_to_openai_function(t)fortintools]model=model.bind_functions(functions)Define the agent stateThe main type of graph inlanggraphis theStatefulGraph.
This graph is parameterized by a state object that it passes around to each node.
Each node then returns operations to update that state.
These operations can either SET specific attributes on the state (e.g. overwrite the existing values) or ADD to the existing attribute.
Whether to set or add is denoted by annotating the state object you construct the graph with.For this example, the state we will track will just be a list of messages.
We want each node to just add messages to that list.
Therefore, we will use aTypedDictwith one key (messages) and annotate it so that themessagesattribute is always added to.fromtypingimportTypedDict,Annotated,Sequenceimportoperatorfromlangchain_core.messagesimportBaseMessageclassAgentState(TypedDict):messages:Annotated[Sequence[BaseMessage],operator.add]Define the nodesWe now need to define a few different nodes in our graph.
Inlanggraph, a node can be either a function or arunnable.
There are two main nodes we need for this:The agent: responsible for deciding what (if any) actions to take.A function to invoke tools: if the agent decides to take an action, this node will then execute that action.We will also need to define some edges.
Some of these edges may be conditional.
The reason they are conditional is that based on the output of a node, one of several paths may be taken.
The path that is taken is not known until that node is run (the LLM decides).Conditional Edge: after the agent is called, we should either:a. If the agent said to take an action, then the function to invoke tools should be calledb. If the agent said that it was finished, then it should finishNormal Edge: after the tools are invoked, it should always go back to the agent to decide what to do nextLet's define the nodes, as well as a function to decide how what conditional edge to take.fromlanggraph.prebuiltimportToolInvocationimportjsonfromlangchain_core.messagesimportFunctionMessage# Define the function that determines whether to continue or notdefshould_continue(state):messages=state['messages']last_message=messages[-1]# If there is no function call, then we finishif"function_call"notinlast_message.additional_kwargs:return"end"# Otherwise if there is, we continueelse:return"continue"# Define the function that calls the modeldefcall_model(state):messages=state['messages']response=model.invoke(messages)# We return a list, because this will get added to the existing listreturn{"messages":[response]}# Define the function to execute toolsdefcall_tool(state):messages=state['messages']# Based on the continue condition# we know the last message involves a function calllast_message=messages[-1]# We construct an ToolInvocation from the function_callaction=ToolInvocation(tool=last_message.additional_kwargs["function_call"]["name"],tool_input=json.loads(last_message.additional_kwargs["function_call"]["arguments"]),)# We call the tool_executor and get back a responseresponse=tool_executor.invoke(action)# We use the response to create a FunctionMessagefunction_message=FunctionMessage(content=str(response),name=action.tool)# We return a list, because this will get added to the existing listreturn{"messages":[function_message]}Define the graphWe can now put it all together and define the graph!fromlanggraph.graphimportStateGraph,END# Define a new graphworkflow=StateGraph(AgentState)# Define the two nodes we will cycle betweenworkflow.add_node("agent",call_model)workflow.add_node("action",call_tool)# Set the entrypoint as `agent`# This means that this node is the first one calledworkflow.set_entry_point("agent")# We now add a conditional edgeworkflow.add_conditional_edges(# First, we define the start node. We use `agent`.# This means these are the edges taken after the `agent` node is called."agent",# Next, we pass in the function that will determine which node is called next.should_continue,# Finally we pass in a mapping.# The keys are strings, and the values are other nodes.# END is a special node marking that the graph should finish.# What will happen is we will call `should_continue`, and then the output of that# will be matched against the keys in this mapping.# Based on which one it matches, that node will then be called.{# If `tools`, then we call the tool node."continue":"action",# Otherwise we finish."end":END})# We now add a normal edge from `tools` to `agent`.# This means that after `tools` is called, `agent` node is called next.workflow.add_edge('action','agent')# Finally, we compile it!# This compiles it into a LangChain Runnable,# meaning you can use it as you would any other runnableapp=workflow.compile()Use it!We can now use it!
This now exposes thesame interfaceas all other LangChain runnables.
This runnable accepts a list of messages.fromlangchain_core.messagesimportHumanMessageinputs={"messages":[HumanMessage(content="what is the weather in sf")]}app.invoke(inputs)This may take a little bit - it's making a few calls behind the scenes.
In order to start seeing some intermediate results as they happen, we can use streaming - see below for more information on that.StreamingLangGraph has support for several different types of streaming.Streaming Node OutputOne of the benefits of using LangGraph is that it is easy to stream output as it's produced by each node.inputs={"messages":[HumanMessage(content="what is the weather in sf")]}foroutputinapp.stream(inputs):# stream() yields dictionaries with output keyed by node nameforkey,valueinoutput.items():print(f"Output from node '{key}':")print("---")print(value)print("\n---\n")Output from node 'agent':
---
{'messages': [AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\n  "query": "weather in San Francisco"\n}', 'name': 'tavily_search_results_json'}})]}

---

Output from node 'action':
---
{'messages': [FunctionMessage(content="[{'url': 'https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States', 'content': 'January 2024 Weather History in San Francisco California, United States  Daily Precipitation in January 2024 in San Francisco Observed Weather in January 2024 in San Francisco  San Francisco Temperature History January 2024 Hourly Temperature in January 2024 in San Francisco  Hours of Daylight and Twilight in January 2024 in San FranciscoThis report shows the past weather for San Francisco, providing a weather history for January 2024. It features all historical weather data series we have available, including the San Francisco temperature history for January 2024. You can drill down from year to month and even day level reports by clicking on the graphs.'}]", name='tavily_search_results_json')]}

---

Output from node 'agent':
---
{'messages': [AIMessage(content="I couldn't find the current weather in San Francisco. However, you can visit [WeatherSpark](https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States) to check the historical weather data for January 2024 in San Francisco.")]}

---

Output from node '__end__':
---
{'messages': [HumanMessage(content='what is the weather in sf'), AIMessage(content='', additional_kwargs={'function_call': {'arguments': '{\n  "query": "weather in San Francisco"\n}', 'name': 'tavily_search_results_json'}}), FunctionMessage(content="[{'url': 'https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States', 'content': 'January 2024 Weather History in San Francisco California, United States  Daily Precipitation in January 2024 in San Francisco Observed Weather in January 2024 in San Francisco  San Francisco Temperature History January 2024 Hourly Temperature in January 2024 in San Francisco  Hours of Daylight and Twilight in January 2024 in San FranciscoThis report shows the past weather for San Francisco, providing a weather history for January 2024. It features all historical weather data series we have available, including the San Francisco temperature history for January 2024. You can drill down from year to month and even day level reports by clicking on the graphs.'}]", name='tavily_search_results_json'), AIMessage(content="I couldn't find the current weather in San Francisco. However, you can visit [WeatherSpark](https://weatherspark.com/h/m/557/2024/1/Historical-Weather-in-January-2024-in-San-Francisco-California-United-States) to check the historical weather data for January 2024 in San Francisco.")]}

---Streaming LLM TokensYou can also access the LLM tokens as they are produced by each node.
In this case only the "agent" node produces LLM tokens.
In order for this to work properly, you must be using an LLM that supports streaming as well as have set it when constructing the LLM (e.g.ChatOpenAI(model="gpt-3.5-turbo-1106", streaming=True))inputs={"messages":[HumanMessage(content="what is the weather in sf")]}asyncforoutputinapp.astream_log(inputs,include_types=["llm"]):# astream_log() yields the requested logs (here LLMs) in JSONPatch formatforopinoutput.ops:ifop["path"]=="/streamed_output/-":# this is the output from .stream()...elifop["path"].startswith("/logs/")andop["path"].endswith("/streamed_output/-"):# because we chose to only include LLMs, these are LLM tokensprint(op["value"])content='' additional_kwargs={'function_call': {'arguments': '', 'name': 'tavily_search_results_json'}}
content='' additional_kwargs={'function_call': {'arguments': '{\n', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': ' ', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': ' "', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': 'query', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': '":', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': ' "', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': 'weather', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': ' in', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': ' San', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': ' Francisco', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': '"\n', 'name': ''}}
content='' additional_kwargs={'function_call': {'arguments': '}', 'name': ''}}
content=''
content=''
content='I'
content="'m"
content=' sorry'
content=','
content=' but'
content=' I'
content=' couldn'
content="'t"
content=' find'
content=' the'
content=' current'
content=' weather'
content=' in'
content=' San'
content=' Francisco'
content='.'
content=' However'
content=','
content=' you'
content=' can'
content=' check'
content=' the'
content=' historical'
content=' weather'
content=' data'
content=' for'
content=' January'
content=' '
content='202'
content='4'
content=' in'
content=' San'
content=' Francisco'
content=' ['
content='here'
content=']('
content='https'
content='://'
content='we'
content='athers'
content='park'
content='.com'
content='/h'
content='/m'
content='/'
content='557'
content='/'
content='202'
content='4'
content='/'
content='1'
content='/H'
content='istorical'
content='-'
content='Weather'
content='-in'
content='-Jan'
content='uary'
content='-'
content='202'
content='4'
content='-in'
content='-S'
content='an'
content='-F'
content='r'
content='anc'
content='isco'
content='-Cal'
content='ifornia'
content='-'
content='United'
content='-'
content='States'
content=').'
content=''When to UseWhen should you use this versusLangChain Expression Language?If you need cycles.Langchain Expression Language allows you to easily define chains (DAGs) but does not have a good mechanism for adding in cycles.langgraphadds that syntax.ExamplesChatAgentExecutor: with function callingThis agent executor takes a list of messages as input and outputs a list of messages.
All agent state is represented as a list of messages.
This specifically uses OpenAI function calling.
This is recommended agent executor for newer chat based models that support function calling.Getting Started Notebook: Walks through creating this type of executor from scratchHigh Level Entrypoint: Walks through how to use the high level entrypoint for the chat agent executor.ModificationsWe also have a lot of examples highlighting how to slightly modify the base chat agent executor. These all build off thegetting started notebookso it is recommended you start with that first.Human-in-the-loop: How to add a human-in-the-loop componentForce calling a tool first: How to always call a specific tool firstRespond in a specific format: How to force the agent to respond in a specific formatDynamically returning tool output directly: How to dynamically let the agent choose whether to return the result of a tool directly to the userManaging agent steps: How to more explicitly manage intermediate steps that an agent takesAgentExecutorThis agent executor uses existing LangChain agents.Getting Started Notebook: Walks through creating this type of executor from scratchHigh Level Entrypoint: Walks through how to use the high level entrypoint for the chat agent executor.ModificationsWe also have a lot of examples highlighting how to slightly modify the base chat agent executor. These all build off thegetting started notebookso it is recommended you start with that first.Human-in-the-loop: How to add a human-in-the-loop componentForce calling a tool first: How to always call a specific tool firstManaging agent steps: How to more explicitly manage intermediate steps that an agent takesAsyncIf you are running LangGraph in async workflows, you may want to create the nodes to be async by default.
For a walkthrough on how to do that, seethis documentationStreaming TokensSometimes language models take a while to respond and you may want to stream tokens to end users.
For a guide on how to do this, seethis documentationPersistenceLangGraph comes with built-in persistence, allowing you to save the state of the graph at point and resume from there.
For a walkthrough on how to do that, seethis documentationHuman-in-the-loopLangGraph comes with built-in support for human-in-the-loop workflows. This is useful when you want to have a human review the current state before proceeding to a particular node.
For a walkthrough on how to do that, seethis documentationPlanning Agent ExamplesThe following notebooks implement agent architectures prototypical of the "plan-and-execute" style, where an LLM planner decomposes a user request into a program, an executor executes the program, and an LLM synthesizes a response (and/or dynamically replans) based on the program outputs.Plan-and-execute: a simple agent with aplannerthat generates a multi-step task list, anexecutorthat invokes the tools in the plan, and areplannerthat responds or generates an updated plan. Based on thePlan-and-solvepaper by Wang, et. al.Reasoning without Observation: planner generates a task list whose observations are saved asvariables. Variables can be used in subsequent tasks to reduce the need for further re-planning. Based on theReWOOpaper by Xu, et. al.LLMCompiler: planner generates aDAGof tasks with variable responses. Tasks arestreamedand executed eagerly to minimize tool execution runtime. Based on thepaperby Kim, et. al.Reflection / Self-CritiqueWhen output quality is a major concern, it's common to incorporate some combination of self-critique or reflection and external validation to refine your system's outputs. The following examples demonstrate research that implement this type of design.Basic Reflection: add a simple "reflect" step in your graph to prompt your system to revise its outputs.Reflexion: critique missing and superflous aspects of the agent's response to guide subsequent steps. Based onReflexion, by Shinn, et. al.Language Agent Tree Search: execute multiple agents in parallel, using reflection and environmental rewards to drive a Monte Carlo Tree Search. Based onLATS, by Zhou, et. al.Multi-agent ExamplesMulti-agent collaboration: how to create two agents that work together to accomplish a taskMulti-agent with supervisor: how to orchestrate individual agents by using an LLM as a "supervisor" to distribute workHierarchical agent teams: how to orchestrate "teams" of agents as nested graphs that can collaborate to solve a problemChatbot Evaluation via SimulationIt can often be tough to evaluation chat bots in multi-turn situations. One way to do this is with simulations.Chat bot evaluation as multi-agent simulation: how to simulate a dialogue between a "virtual user" and your chat botMultimodal ExamplesWebVoyager: vision-enabled web browsing agent that usesSet-of-marksprompting to navigate a web browser and execute tasksDocumentationThere are only a few new APIs to use.StateGraphThe main entrypoint isStateGraph.fromlanggraph.graphimportStateGraphThis class is responsible for constructing the graph.
It exposes an interface inspired byNetworkX.
This graph is parameterized by a state object that it passes around to each node.__init__def__init__(self,schema:Type[Any])->None:When constructing the graph, you need to pass in a schema for a state.
Each node then returns operations to update that state.
These operations can either SET specific attributes on the state (e.g. overwrite the existing values) or ADD to the existing attribute.
Whether to set or add is denoted by annotating the state object you construct the graph with.The recommended way to specify the schema is with a typed dictionary:from typing import TypedDictYou can then annotate the different attributes usingfrom typing imoport Annotated.
Currently, the only supported annotation isimport operator; operator.add.
This annotation will make it so that any node that returns this attribute ADDS that new result to the existing value.Let's take a look at an example:fromtypingimportTypedDict,Annotated,Unionfromlangchain_core.agentsimportAgentAction,AgentFinishimportoperatorclassAgentState(TypedDict):# The input stringinput:str# The outcome of a given call to the agent# Needs `None` as a valid type, since this is what this will start asagent_outcome:Union[AgentAction,AgentFinish,None]# List of actions and corresponding observations# Here we annotate this with `operator.add` to indicate that operations to# this state should be ADDED to the existing values (not overwrite it)intermediate_steps:Annotated[list[tuple[AgentAction,str]],operator.add]We can then use this like:# Initialize the StateGraph with this stategraph=StateGraph(AgentState)# Create nodes and edges...# Compile the graphapp=graph.compile()# The inputs should be a dictionary, because the state is a TypedDictinputs={# Let's assume this the input"input":"hi"# Let's assume agent_outcome is set by the graph as some point# It doesn't need to be provided, and it will be None by default# Let's assume `intermediate_steps` is built up over time by the graph# It doesn't need to provided, and it will be empty list by default# The reason `intermediate_steps` is an empty list and not `None` is because# it's annotated with `operator.add`}.add_nodedefadd_node(self,key:str,action:RunnableLike)->None:This method adds a node to the graph.
It takes two arguments:key: A string representing the name of the node. This must be unique.action: The action to take when this node is called. This should either be a function or a runnable..add_edgedefadd_edge(self,start_key:str,end_key:str)->None:Creates an edge from one node to the next.
This means that output of the first node will be passed to the next node.
It takes two arguments.start_key: A string representing the name of the start node. This key must have already been registered in the graph.end_key: A string representing the name of the end node. This key must have already been registered in the graph..add_conditional_edgesdefadd_conditional_edges(self,start_key:str,condition:Callable[...,str],conditional_edge_mapping:Dict[str,str],)->None:This method adds conditional edges.
What this means is that only one of the downstream edges will be taken, and which one that is depends on the results of the start node.
This takes three arguments:start_key: A string representing the name of the start node. This key must have already been registered in the graph.condition: A function to call to decide what to do next. The input will be the output of the start node. It should return a string that is present inconditional_edge_mappingand represents the edge to take.conditional_edge_mapping: A mapping of string to string. The keys should be strings that may be returned bycondition. The values should be the downstream node to call if that condition is returned..set_entry_pointdefset_entry_point(self,key:str)->None:The entrypoint to the graph.
This is the node that is first called.
It only takes one argument:key: The name of the node that should be called first..set_finish_pointdefset_finish_point(self,key:str)->None:This is the exit point of the graph.
When this node is called, the results will be the final result from the graph.
It only has one argument:key: The name of the node that, when called, will return the results of calling it as the final outputNote: This does not need to be called if at any point you previously created an edge (conditional or normal) toENDGraphfromlanggraph.graphimportGraphgraph=Graph()This has the same interface asStateGraphwith the exception that it doesn't update a state object over time, and rather relies on passing around the full state from each step.
This means that whatever is returned from one node is the input to the next as is.ENDfromlanggraph.graphimportENDThis is a special node representing the end of the graph.
This means that anything passed to this node will be the final output of the graph.
It can be used in two places:As theend_keyinadd_edgeAs a value inconditional_edge_mappingas passed toadd_conditional_edgesPrebuilt ExamplesThere are also a few methods we've added to make it easy to use common, prebuilt graphs and components.ToolExecutorfromlanggraph.prebuiltimportToolExecutorThis is a simple helper class to help with calling tools.
It is parameterized by a list of tools:tools=[...]tool_executor=ToolExecutor(tools)It then exposes arunnable interface.
It can be used to call tools: you can pass in anAgentActionand it will look up the relevant tool and call it with the appropriate input.chat_agent_executor.create_function_calling_executorfromlanggraph.prebuiltimportchat_agent_executorThis is a helper function for creating a graph that works with a chat model that utilizes function calling.
Can be created by passing in a model and a list of tools.
The model must be one that supports OpenAI function calling.fromlangchain_openaiimportChatOpenAIfromlangchain_community.tools.tavily_searchimportTavilySearchResultsfromlanggraph.prebuiltimportchat_agent_executorfromlangchain_core.messagesimportHumanMessagetools=[TavilySearchResults(max_results=1)]model=ChatOpenAI()app=chat_agent_executor.create_function_calling_executor(model,tools)inputs={"messages":[HumanMessage(content="what is the weather in sf")]}forsinapp.stream(inputs):print(list(s.values())[0])print("----")create_agent_executorfromlanggraph.prebuiltimportcreate_agent_executorThis is a helper function for creating a graph that works withLangChain Agents.
Can be created by passing in an agent and a list of tools.fromlanggraph.prebuiltimportcreate_agent_executorfromlangchain_openaiimportChatOpenAIfromlangchainimporthubfromlangchain.agentsimportcreate_openai_functions_agentfromlangchain_community.tools.tavily_searchimportTavilySearchResultstools=[TavilySearchResults(max_results=1)]# Get the prompt to use - you can modify this!prompt=hub.pull("hwchase17/openai-functions-agent")# Choose the LLM that will drive the agentllm=ChatOpenAI(model="gpt-3.5-turbo-1106")# Construct the OpenAI Functions agentagent_runnable=create_openai_functions_agent(llm,tools,prompt)app=create_agent_executor(agent_runnable,tools)inputs={"input":"what is the weather in sf","chat_history":[]}forsinapp.stream(inputs):print(list(s.values())[0])print("----")