follow the tutorial and build me a react agent that I can chat on in terminal, use rich text to print debug stuff out. and use this llm = ChatOllama(
    model="llama3.1",
    temperature=0,
)

 from langchain_ollama


tutorial:
LangGraph and Research Agents
James Briggs
James Briggs
Jul 11, 2024
Deep Dives
Share:

Research agents are multi-step AI agents that aim to provide in-depth answers to user queries. These agents differ from typical conversational agents, where we typically find fast response times and short AI answers.

There are several good reasons our conversational agents respond in the way they do. The first reason is that we aim to imitate the style of human conversation. A typical conversation between two people consists of short back-and-forths — it's rare (and often not wanted) that one of those people suddenly breaks out into a multi-page monologue.

Another typical quality of human conversations is a short wait time between responses. When someone says "hi," it would be odd to wait even a second before getting a response. We expect low-latency responses from people, and the same applies to our conversational AI agents.

Now, let's move away from conversational agents and consider what qualities we might want in a research agent. A longer, more researched, and cited response can be much more helpful if we ask a question in research or study. In this scenario, many might choose to delay their response by a few (or more) seconds if they're getting a more detailed response. In these scenarios, a user prefers to speak to a research agent rather than the typical conversational agent.

Because we have less time pressure with research agents, we can design them to work through a multi-step research pipeline and synthesize information from different places. Given a user's query, a research agent should be able to search the web, read papers, and access other relevant sources of information for its particular use case.

That doesn't mean a research agent cannot be conversational. In most cases, users prefer a conversational interface, and we can design agents that do both.
Graph-based research agent that will be built
Graph showing the research agent we will build in this article.
We'll focus on building a research agent and how we can achieve that using a graph-based approach to agent construction.
Video companion to the article.
Graphs for Agents

Agents are still a relatively new concept. The modern form of agents, primarily LLM-powered, only became popular in early 2022, shortly after the release of ChatGPT.

These earlier agents focused on providing an LLM with a decision-making framework allowing iterative decision-making and tool-use steps. The most popular of these being the Reason Action (ReAct) method [1].

ReAct encourages an LLM to generate steps broken into iterative reasoning and action steps. The reasoning step encourages the LLM to outline what steps it needs to take to answer a question. The agent then follows up with an action where our tool/function call executes a step like searching the internet.
ReAct agent process
ReAct agent process. The reasoning step manifests as the Thought output from the LLM. The reasoning step is followed by an Action like Search[Apple Remote]. Multiple iterations can occur to arrive at a final answer.

After these two steps, the agent receives an observation, i.e., the result of our action step. This reasoning-action loop can be repeated with various actions, all culminating in a final answer.

The ReAct framework is powerful because it allows us to combine LLMs with a wide range of data sources and executable programs.

Popular implementations of ReAct (or a similar method called "MRKL") made their way to libraries like LangChain, Llama-Index, and Haystack. These implementations allowed us to built ReAct-like agents using an object oriented approach.

The object-oriented approach functions well but leaves little flexibility or transparency in our agent. It's very possible to use ReAct agents via LangChain and have no idea how any of it is functioning, given the framework's abstractions.
Graph-based ReAct-like agent
ReAct-like agent constructed as a graph.

An interesting solution to this is to construct agents as graphs. When building an agent as a graph, you can still produce the iterative ReAct-like loop, but it is much easier to customize that loop. We can add more deterministic flows and hierarchical decision-making and simply build with more flexibility and transparency.

More deterministic flows in our agent allow us to address two key limitations of ReAct-like agents: interpretability and entropy.

The interpretability of LLMs can often be hard to understand or measure. LLMs lack hard-coded rules, in contrast to traditional software which is very interpretable as it is constructed solely from hard-coded rules. This lack of deterministic behavior in LLMs is both a blessing and a curse. They can generate a seemingly infinite number of outputs, resulting in seemingly infinite potential — but those outputs are hard to engineer and impossible to fully predict.

To deploy LLM-based software into the world, we need confidence that it will behave a certain way. However, LLMs are compressed representations of an entire world, making it impossible (for now) to predict every possible output of an LLM.

Because of the lack of interpretability, LLM-based software needs as much determinstic logic as possible. Anything that does not require an LLM, should not use an LLM.

Entropy is the tendency of a system to become more chaotic over time.When executed iteratively without human guidance, an LLM can hallucinate or simply struggle to keep track of an increasingly long set of prior thoughts, actions, and observations. Due to this, LLMs typically become increasingly divergent from their original purpose as the number of iterative generations increases.
LangGraph

LangGraph is LangChain's graph-based agent framework and one of the most popular frameworks for building graph-based agents. It focuses on providing more "fine-grained" control over an agent's flow and state.
State

At the core of a graph in LangGraph is the agent state. The state is a mutable object where we track the current state of the agent execution as we pass through the graph. We can include different parameters within the state, in our research agent we will use a minimal example containing three parameters:

    input: This is the user's most recent query. Usually, this is a question that we want to answer with our research agent.
    chat_history: We are building a conversational agent that can support multiple interactions. To allow previous interactions to provide additional context throughout our agent logic, we include the chat history in the agent state.
    intermediate_steps provides a record of all steps the research agent will take between the user asking a question via input and the agent providing a final answer. These can include "search arxiv", "perform general purpose web search," etc. These intermediate steps are crucial to allowing the agent to follow a path of coherent actions and ultimately producing an informed final answer.

Follow Along with Code!

The remainder of this article will be focused on the implementation of a LangGraph research agent using Python. You can find a copy of the code here.

If following along, make sure to install prerequisite packages first:

!pip install -qU \
    datasets==2.19.1 \
    langchain-pinecone==0.1.1 \
    langchain-openai==0.1.9 \
    langchain==0.2.5 \
    langchain-core==0.2.9 \
    langgraph==0.1.1 \
    semantic-router==0.0.48 \
    serpapi==0.1.5 \
    google-search-results==2.4.2 \

We define our minimal agent state object like so:

from typing import TypedDict, Annotated
from langchain_core.agents import AgentAction
from langchain_core.messages import BaseMessage
import operator


class AgentState(TypedDict):
   input: str
   chat_history: list[BaseMessage]
   intermediate_steps: Annotated[list[tuple[AgentAction, str]], operator.add]

Creating our Tools

The agent graph consists of several nodes, five of which are custom tools we need to create. Those tools are:

    ArXiv paper fetch: Given an arXiv paper ID, this tool provides our agent with the paper's abstract.

    Web search: This tool provides our agent access to Google search for more generalized queries.

    RAG search: We will create a knowledge base containing AI arXiv papers. This tool provides our agent with access to this knowledge.

    RAG search with filter: Sometimes, our agent may need more information from a specific paper. This tool allows our agent to do just that.

    Final answer: We create a custom final answer tool that forces our agent to output information in a specific format like:

 INTRODUCTION
 ------------
 <some intro to our report>
  RESEARCH STEPS
 --------------
 <the steps the agent took during research>
  REPORT
 ------
 <the report main body>
  CONCLUSION
 ----------
 <the report conclusion>
  SOURCES
 -------
 <any sources the agent used>

We'll setup each of these tools, which our LLM and graph-logic will later be able to execute.
ArXiv Paper Fetch Tool

The fetch_arxiv tool will allow our agent to get the summary of a specific paper given an ArXiv paper ID. We will simply send a GET request to arXiv and use regex to extract the paper abstract. That GET request will return something like this:

import requests

# we will test with the mixtral paper
arxiv_id = "2401.04088"

res = requests.get(
    f"https://export.arxiv.org/abs/{arxiv_id}"
)
res.text

'<?xml version="1.0" encoding="UTF-8"?>\n<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">\n<html xmlns="http://www.w3.org/1999/xhtml" lang="en">\n<head>\n<title>[2401.04088] Mixtral of Experts</title>\n<link rel="shortcut icon" href="/favicon.ico" type="image/x-icon" />\n<link rel="stylesheet" type="text/css" media="screen" href="/css/arXiv-export.css" />\n<link rel="stylesheet" type="text/css" media="screen" href="/bibex/bibex.css?20181009">\n<link rel="stylesheet" type="text/css" media="screen" href="https://static.arxiv.org/static/browse/0.3.8/css/browse_search.css" />\n<meta name="citation_title" content="Mixtral of Experts" ...\n</body>\n</html>\n'

A lot is going on there. Fortunately, we can a small regex to find the paper abstract.

import re

# our regex
abstract_pattern = re.compile(
    r'<blockquote class="abstract mathjax">\s*<span class="descriptor">Abstract:</span>\s*(.*?)\s*</blockquote>',
    re.DOTALL
)

# we search
re_match = abstract_pattern.search(res.text)

# and now let's see what we got
print(re_match.group(1))

We introduce Mixtral 8x7B, a Sparse Mixture of Experts (SMoE) language model.
Mixtral has the same architecture as Mistral 7B, with the difference that each
layer is composed of 8 feedforward blocks (i.e. experts). For every token, at
each layer, a router network selects two experts to process the current state
and combine their outputs. Even though each token only sees two experts, the
selected experts can be different at each timestep. As a result, each token has
access to 47B parameters, but only uses 13B active parameters during inference.
Mixtral was trained with a context size of 32k tokens and it outperforms or
matches Llama 2 70B and GPT-3.5 across all evaluated benchmarks. In particular,
Mixtral vastly outperforms Llama 2 70B on mathematics, code generation, and
multilingual benchmarks. We also provide a model fine-tuned to follow
instructions, Mixtral 8x7B - Instruct, that surpasses GPT-3.5 Turbo,
Claude-2.1, Gemini Pro, and Llama 2 70B - chat model on human benchmarks. Both
the base and instruct models are released under the Apache 2.0 license.

Now we pack all of this logic into a tool for our agent.

from langchain_core.tools import tool


@tool("fetch_arxiv")
def fetch_arxiv(arxiv_id: str):
   """Gets the abstract from an ArXiv paper given the arxiv ID. Useful for
   finding high-level context about a specific paper."""
   # get paper page in html
   res = requests.get(
       f"https://export.arxiv.org/abs/{arxiv_id}"
   )
   # search html for abstract
   re_match = abstract_pattern.search(res.text)
   # return abstract text
   return re_match.group(1)

We can test the tool to confirm it works using the .invoke method like so:

print(
    fetch_arxiv.invoke(input={"arxiv_id": arxiv_id})
)

We introduce Mixtral 8x7B, a Sparse Mixture of Experts (SMoE) language model.
Mixtral has the same architecture as Mistral 7B, with the difference that each
layer is composed of 8 feedforward blocks (i.e. experts). For every token, at
each layer, a router network selects two experts to process the current state
and combine their outputs. Even though each token only sees two experts, the
selected experts can be different at each timestep. As a result, each token has
access to 47B parameters, but only uses 13B active parameters during inference.
Mixtral was trained with a context size of 32k tokens and it outperforms or
matches Llama 2 70B and GPT-3.5 across all evaluated benchmarks. In particular,
Mixtral vastly outperforms Llama 2 70B on mathematics, code generation, and
multilingual benchmarks. We also provide a model fine-tuned to follow
instructions, Mixtral 8x7B - Instruct, that surpasses GPT-3.5 Turbo,
Claude-2.1, Gemini Pro, and Llama 2 70B - chat model on human benchmarks. Both
the base and instruct models are released under the Apache 2.0 license.

Web Search Tool

The web search tool will provide the agent with access to web search. We will instruct our LLM to use this for more general knowledge queries. To implement the search component of the tool, we will be using the SerpAPI. You can get a free API key for the service here. Once you have the API key we run a query for "coffee" like so:

from serpapi import GoogleSearch

serpapi_params = {
    "engine": "google",
    "api_key": os.getenv("SERPAPI_KEY") or getpass("SerpAPI key: ")
}

search = GoogleSearch({
    **serpapi_params,
    "q": "coffee"
})

results = search.get_dict()["organic_results"]

contexts = "\n---\n".join(
    ["\n".join([x["title"], x["snippet"], x["link"]]) for x in results]
)

print(contexts)

Coffee
Coffee is a beverage brewed from roasted coffee beans. Darkly colored, bitter, and slightly acidic, coffee has a stimulating effect on humans, primarily due ...
https://en.wikipedia.org/wiki/Coffee
---
Starbucks Coffee Company
A carbonated iced drink with tea on top, an iced coffee with a creamy foam. Sunny-day essentials. Flavorful creations ...
https://www.starbucks.com/
---
What is Coffee?
Coffee traces its origin to a genus of plants known as Coffea. Within the genus there are over 500 genera and 6,000 species of tropical trees and shrubs.
https://www.ncausa.org/About-Coffee/What-is-Coffee
---
Peet's Coffee | The Original Craft Coffee, Est. 1966
Since 1966, Peet's Coffee has source and offered superior coffees and teas and adhering to strict high-quality and taste standards. Shop today.
https://www.peets.com/
---
Philz Coffee: Home
We specialize in customized blends with coffee shops in California and Chicago, IL Buy coffee online or find your nearest Philz location.
https://philzcoffee.com/
---
Madcap Coffee Company - Grand Rapids
Madcap Coffee's mission has been to unveil the inspiring craftsmanship and human touch in every cup of specialty coffee, transforming an ordinary daily ...
https://www.madcapcoffee.com/
---
Blue Bottle Coffee | Fresh Roasted Specialty Coffee
Blue Bottle Coffee is a specialty coffee roaster with cafes in LA, SF, NYC, & Japan. Shop our freshly roasted specialty coffee online & in-store.
https://bluebottlecoffee.com/
---
Gimme! Coffee
Gimme Coffee's goal is to permit customers to successfully gather information and conduct business through our website, including individuals with visual ...
https://gimmecoffee.com/

We place all of this logic into another tool:

@tool("web_search")
def web_search(query: str):
   """Finds general knowledge information using Google search. Can also be used
   to augment more 'general' knowledge to a previous specialist query."""
   search = GoogleSearch({
       **serpapi_params,
       "q": query,
       "num": 5
   })
   results = search.get_dict()["organic_results"]
   contexts = "\n---\n".join(
       ["\n".join([x["title"], x["snippet"], x["link"]]) for x in results]
   )
   return contexts

RAG Tools

We provide two RAG-focused tools for our agent. The rag_search allows the agent to perform a simple RAG search for some information across all indexed research papers. The rag_search_filter also searches, but within a specific paper filtered for via the arxiv_id parameter.

We also define the format_rag_contexts function to handle the transformation of our Pinecone results from a JSON object to a readble plaintext format. Note: you can find the code for setting up the index we use here.

def format_rag_contexts(matches: list):
   contexts = []
   for x in matches:
       text = (
           f"Title: {x['metadata']['title']}\n"
           f"Content: {x['metadata']['content']}\n"
           f"ArXiv ID: {x['metadata']['arxiv_id']}\n"
           f"Related Papers: {x['metadata']['references']}\n"
       )
       contexts.append(text)
   context_str = "\n---\n".join(contexts)
   return context_str


@tool("rag_search_filter")
def rag_search_filter(query: str, arxiv_id: str):
   """Finds information from our ArXiv database using a natural language query
   and a specific ArXiv ID. Allows us to learn more details about a specific paper."""
   xq = encoder([query])
   xc = index.query(vector=xq, top_k=6, include_metadata=True, filter={"arxiv_id": arxiv_id})
   context_str = format_rag_contexts(xc["matches"])
   return context_str


@tool("rag_search")
def rag_search(query: str):
   """Finds specialist information on AI using a natural language query."""
   xq = encoder([query])
   xc = index.query(vector=xq, top_k=2, include_metadata=True)
   context_str = format_rag_contexts(xc["matches"])
   return context_str

Final Answer "Tool"

Finally, we define a "final answer" tool. This tool isn't a tool in the usual sense, instead we use it to force a particular output format from our LLM via the function/tool calling.

@tool("final_answer")
def final_answer(
   introduction: str,
   research_steps: str,
   main_body: str,
   conclusion: str,
   sources: str
):
   """Returns a natural language response to the user in the form of a research
   report. There are several sections to this report, those are:
   - `introduction`: a short paragraph introducing the user's question and the
   topic we are researching.
   - `research_steps`: a few bullet points explaining the steps that were taken
   to research your report.
   - `main_body`: this is where the bulk of high quality and concise
   information that answers the user's question belongs. It is 3-4 paragraphs
   long in length.
   - `conclusion`: this is a short single paragraph conclusion providing a
   concise but sophisticated view on what was found.
   - `sources`: a bulletpoint list provided detailed sources for all information
   referenced during the research process
   """
   if type(research_steps) is list:
       research_steps = "\n".join([f"- {r}" for r in research_steps])
   if type(sources) is list:
       sources = "\n".join([f"- {s}" for s in sources])
   return ""

We have finished defining all our tools and can move onto our next components.
Defining the Oracle

The Oracle LLM is our graph's decision maker. It decides which path we should take down our graph. It functions similarly to an agent but is much simpler; consisting of a single prompt, LLM, and tool instructions. When executed the oracle will only run once before we move to another node in our graph.

The Oracle consists of an LLM provided with a set of potential function calls (i.e., our tools) that it can decide to use. We force it to use at least one of those tools using the tool_choice="any" setting (see below). Our Oracle only makes the decision to use a tool; it doesn't execute the tool code itself (we do that separately in our graph).

Our prompt for the Oracle will emphasize its decision-making ability within the system_prompt. We will also add a placeholder for us to later insert chat_history, and provide a place for us to insert the intermediate steps (scratchpad) and user input.

from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder


system_prompt = """You are the oracle, the great AI decision maker.
Given the user's query you must decide what to do with it based on the
list of tools provided to you.


If you see that a tool has been used (in the scratchpad) with a particular
query, do NOT use that same tool with the same query again. Also, do NOT use
any tool more than twice (ie, if the tool appears in the scratchpad twice, do
not use it again).


You should aim to collect information from a diverse range of sources before
providing the answer to the user. Once you have collected plenty of information
to answer the user's question (stored in the scratchpad) use the final_answer
tool."""


prompt = ChatPromptTemplate.from_messages([
   ("system", system_prompt),
   MessagesPlaceholder(variable_name="chat_history"),
   ("user", "{input}"),
   ("assistant", "scratchpad: {scratchpad}"),
])

Next, we must initialize our llm (for this we use gpt-4o) and then create the runnable pipeline of our Oracle.

The runnable connects our inputs (the user input and chat_history) to our prompt, and our prompt to our llm. It is also where we bind our tools to the LLM and enforce function calling via tool_choice="any".

from langchain_core.messages import ToolCall, ToolMessage
from langchain_openai import ChatOpenAI


llm = ChatOpenAI(
   model="gpt-4o",
   openai_api_key=os.environ["OPENAI_API_KEY"],
   temperature=0
)


tools=[
   rag_search_filter,
   rag_search,
   fetch_arxiv,
   web_search,
   final_answer
]


# define a function to transform intermediate_steps from list
# of AgentAction to scratchpad string
def create_scratchpad(intermediate_steps: list[AgentAction]):
   research_steps = []
   for i, action in enumerate(intermediate_steps):
       if action.log != "TBD":
           # this was the ToolExecution
           research_steps.append(
               f"Tool: {action.tool}, input: {action.tool_input}\n"
               f"Output: {action.log}"
           )
   return "\n---\n".join(research_steps)


oracle = (
   {
       "input": lambda x: x["input"],
       "chat_history": lambda x: x["chat_history"],
       "scratchpad": lambda x: create_scratchpad(
           intermediate_steps=x["intermediate_steps"]
       ),
   }
   | prompt
   | llm.bind_tools(tools, tool_choice="any")
)

We can quickly test the agent to confirm it is functional:

inputs = {
    "input": "tell me something interesting about dogs",
    "chat_history": [],
    "intermediate_steps": [],
}
out = oracle.invoke(inputs)
out

AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_UL4qocVvucVLhykCbn1WEkaV', 'function': {'arguments': '{"query": "interesting facts about dogs"}', 'name': 'web_search'}, 'type': 'function'}, {'id': 'call_VeoMlK9MKFopRVLBXAqtnSm2', 'function': {'arguments': '{"query": "interesting facts about dogs"}', 'name': 'rag_search'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 49, 'prompt_tokens': 541, 'total_tokens': 590}, 'model_name': 'gpt-4o-2024-05-13', 'system_fingerprint': 'fp_ce0793330f', 'finish_reason': 'stop', 'logprobs': None}, id='run-ddd60742-06ec-4b16-bf37-6d5eefa423eb-0', tool_calls=[{'name': 'web_search', 'args': {'query': 'interesting facts about dogs'}, 'id': 'call_UL4qocVvucVLhykCbn1WEkaV'}, {'name': 'rag_search', 'args': {'query': 'interesting facts about dogs'}, 'id': 'call_VeoMlK9MKFopRVLBXAqtnSm2'}], usage_metadata={'input_tokens': 541, 'output_tokens': 49, 'total_tokens': 590})

It is running, but we are returning a lot of output here. We can narrow this down to what we need—i.e., the chosen tool name and generated input args for the tool.

out.tool_calls[0]["name"]

'web_search'

out.tool_calls[0]["args"]

{'query': 'interesting facts about dogs'}

We can see now that our Oracle decided to use the web_search tool with a query of "interesting facts about dogs" — a good choice.
Preparing the Graph Components

Our graph needs functions that consume the AgentState we defined earlier, and output intermediate_steps. To do this, we wrap our oracle and tool functions to use the AgentState. We'll also define a router which will route our state to different tool nodes based on the output from our oracle. We get this value from the out.tool_calls[0]["name"] value.

from typing import TypedDict


def run_oracle(state: TypedDict):
   print("run_oracle")
   print(f"intermediate_steps: {state['intermediate_steps']}")
   out = oracle.invoke(state)
   tool_name = out.tool_calls[0]["name"]
   tool_args = out.tool_calls[0]["args"]
   action_out = AgentAction(
       tool=tool_name,
       tool_input=tool_args,
       log="TBD"
   )
   return {
       "intermediate_steps": [action_out]
   }


def router(state: TypedDict):
   # return the tool name to use
   if isinstance(state["intermediate_steps"], list):
       return state["intermediate_steps"][-1].tool
   else:
       # if we output bad format go to final answer
       print("Router invalid format")
       return "final_answer"

Our tools can run using the same function logic we define with run_tool. We add the input parameters to our tool call and the resultant output to our graph state's intermediate_steps parameter.

tool_str_to_func = {
   "rag_search_filter": rag_search_filter,
   "rag_search": rag_search,
   "fetch_arxiv": fetch_arxiv,
   "web_search": web_search,
   "final_answer": final_answer
}


def run_tool(state: TypedDict):
   # use this as helper function so we repeat less code
   tool_name = state["intermediate_steps"][-1].tool
   tool_args = state["intermediate_steps"][-1].tool_input
   print(f"{tool_name}.invoke(input={tool_args})")
   # run tool
   out = tool_str_to_func[tool_name].invoke(input=tool_args)
   action_out = AgentAction(
       tool=tool_name,
       tool_input=tool_args,
       log=str(out)
   )
   return {"intermediate_steps": [action_out]}

Constructing the Graph

The graph in LangGraph consists of several components, we will be using the following:

    Nodes: these are the steps in the graph where we execute some logic. These could be an LLM call, tool execution, or our final answer output. We add them via graph.add_node(...).
    Edges: these are the connecting flows/paths in our graph. If we connect a node to another via an edge, we can move between those nodes. An edge only allows travel in a single direction, so from node A -> node B we would write graph.add_edge("node A", "node B") . However, we can allow bidirectional travel by adding a second edge from node B -> node A with graph.add_edge("node B", "node A").
    Entry point: this specifies the node from which we would enter the graph, i.e. our starting node. If starting at node A we write graph.set_entry_point("node A").
    Conditional edge: a conditional edge allows us to specify multiple potential edges from a source node to a set of destination nodes defined by a mapping passed to the path parameter. If adding conditional edges from node B to nodes C, D, and E — all decided using a router function (more on this later), we would write graph.add_conditional_edges(source="node A", path=router).

from langgraph.graph import StateGraph, END


# initialize the graph with our AgentState
graph = StateGraph(AgentState)


# add nodes
graph.add_node("oracle", run_oracle)
graph.add_node("rag_search_filter", run_tool)
graph.add_node("rag_search", run_tool)
graph.add_node("fetch_arxiv", run_tool)
graph.add_node("web_search", run_tool)
graph.add_node("final_answer", run_tool)


# specify the entry node
graph.set_entry_point("oracle")


# add the conditional edges which use the router
graph.add_conditional_edges(
   source="oracle",  # where in graph to start
   path=router,  # function to determine which node is called
)


# create edges from each tool back to the oracle
for tool_obj in tools:
   if tool_obj.name != "final_answer":
       graph.add_edge(tool_obj.name, "oracle")


# if anything goes to final answer, it must then move to END
graph.add_edge("final_answer", END)


# finally, we compile our graph
runnable = graph.compile()

From all of this we will have an agent graph that looks like this:
Research agent graph
The agent graph that we built.
Building Research Reports

Our research agent is now ready, and we can begin testing it. First, we'll start with something simple that is out of our agent's scope but allows us to test it quickly.

out = runnable.invoke({
    "input": "tell me something interesting about dogs",
    "chat_history": [],
})

run_oracle
intermediate_steps: []
rag_search.invoke(input={'query': 'interesting facts about dogs'})
run_oracle
web_search.invoke(input={'query': 'interesting facts about dogs'})
run_oracle
final_answer.invoke(input={'introduction': 'Dogs are fascinating creatures that have been companions to humans for thousands of years. They are known for their loyalty, intelligence, and diverse abilities. Here are some interesting facts about dogs that highlight their unique characteristics and behaviors.', 'research_steps': '1. Conducted a specialist search on ArXiv for any research papers or articles related to interesting facts about dogs.\n2. Performed a web search to gather general knowledge and fun facts about dogs from various reputable sources.', 'main_body': 'Dogs were the first animals to be domesticated by humans, with evidence suggesting this occurred over 20,000 years ago. As they evolved from wolves, their physical features such as skulls, teeth, and paws became smaller. Today, there are more than 150 dog breeds, categorized into eight classes: sporting, hound, working, terrier, toy, non-sporting, herding, and miscellaneous.\n\nOne of the most remarkable abilities of dogs is their sense of smell, which is 40 times better than that of humans. This extraordinary olfactory capability allows them to detect a wide range of scents, making them invaluable in roles such as search and rescue, detection of explosives, and even medical diagnosis.\n\nDogs also have a keen sense of time. Studies have shown that they can distinguish between different durations and can be conditioned to anticipate future events, such as their feeding times. This sense of time, combined with their ability to understand human emotions and commands, makes them highly trainable and responsive companions.\n\nAdditionally, dogs exhibit a range of behaviors that are both intriguing and endearing. For instance, they have been observed to dream, much like humans, and often exhibit twitching or paw movements during their sleep. Their social nature and pack mentality also mean that they thrive on companionship and can form strong bonds with both humans and other animals.', 'conclusion': 'Dogs are truly remarkable animals with a rich history of domestication and a wide array of unique abilities. Their exceptional sense of smell, understanding of time, and social behaviors make them not only fascinating but also invaluable companions to humans.', 'sources': '- https://www.mspca.org/pet_resources/interesting-facts-about-dogs/\n- https://www.akc.org/expert-advice/lifestyle/dog-facts/\n- https://www.montecitopethospital.com/site/blog/2023/09/15/fun-dog-facts-kids\n- https://www.natgeokids.com/uk/discover/animals/general-animals/dog-facts/\n- https://www.thedrakecenter.com/services/dogs/blog/23-amazing-facts-about-dogs-you-probably-didnt-know'})

Let's create a function to consume the agent output and format it into our report. Note: we could add this into the final answer tool logic if preferred.

def build_report(output: dict):
   research_steps = output["research_steps"]
   if type(research_steps) is list:
       research_steps = "\n".join([f"- {r}" for r in research_steps])
   sources = output["sources"]
   if type(sources) is list:
       sources = "\n".join([f"- {s}" for s in sources])
   return f"""
INTRODUCTION
------------
{output["introduction"]}


RESEARCH STEPS
--------------
{research_steps}


REPORT
------
{output["main_body"]}


CONCLUSION
----------
{output["conclusion"]}


SOURCES
-------
{sources}
"""

Now we run our build_report function:

print(build_report(
    output=out["intermediate_steps"][-1].tool_input
))


INTRODUCTION
------------
Dogs are fascinating creatures that have been companions to humans for thousands of years. They are known for their loyalty, intelligence, and diverse abilities. Here are some interesting facts about dogs that highlight their unique characteristics and behaviors.

RESEARCH STEPS
--------------
1. Conducted a specialist search on ArXiv for any research papers or articles related to interesting facts about dogs.
2. Performed a web search to gather general knowledge and fun facts about dogs from various reputable sources.

REPORT
------
Dogs were the first animals to be domesticated by humans, with evidence suggesting this occurred over 20,000 years ago. As they evolved from wolves, their physical features such as skulls, teeth, and paws became smaller. Today, there are more than 150 dog breeds, categorized into eight classes: sporting, hound, working, terrier, toy, non-sporting, herding, and miscellaneous.

One of the most remarkable abilities of dogs is their sense of smell, which is 40 times better than that of humans. This extraordinary olfactory capability allows them to detect a wide range of scents, making them invaluable in roles such as search and rescue, detection of explosives, and even medical diagnosis.

Dogs also have a keen sense of time. Studies have shown that they can distinguish between different durations and can be conditioned to anticipate future events, such as their feeding times. This sense of time, combined with their ability to understand human emotions and commands, makes them highly trainable and responsive companions.

Additionally, dogs exhibit a range of behaviors that are both intriguing and endearing. For instance, they have been observed to dream, much like humans, and often exhibit twitching or paw movements during their sleep. Their social nature and pack mentality also mean that they thrive on companionship and can form strong bonds with both humans and other animals.

CONCLUSION
----------
Dogs are truly remarkable animals with a rich history of domestication and a wide array of unique abilities. Their exceptional sense of smell, understanding of time, and social behaviors make them not only fascinating but also invaluable companions to humans.

SOURCES
-------
- https://www.mspca.org/pet_resources/interesting-facts-about-dogs/
- https://www.akc.org/expert-advice/lifestyle/dog-facts/
- https://www.montecitopethospital.com/site/blog/2023/09/15/fun-dog-facts-kids
- https://www.natgeokids.com/uk/discover/animals/general-animals/dog-facts/
- https://www.thedrakecenter.com/services/dogs/blog/23-amazing-facts-about-dogs-you-probably-didnt-know

Now let's try with an on-topic question regarding AI.

out = runnable.invoke({
    "input": "tell me about AI",
    "chat_history": []
})

run_oracle
rag_search.invoke(input={'query': 'AI'})
run_oracle
fetch_arxiv.invoke(input={'arxiv_id': '2309.02427'})
run_oracle
web_search.invoke(input={'query': 'Artificial Intelligence overview'})
run_oracle
final_answer.invoke(input={'introduction': 'Artificial Intelligence (AI) is a rapidly evolving field that encompasses a wide range of technologies and applications aimed at simulating human intelligence in machines. From cognitive architectures to large language models, AI is transforming various domains by enabling machines to perform tasks that typically require human intelligence.', 'research_steps': '1. Conducted a specialist search on AI using the RAG search tool.\n2. Retrieved detailed information on specific AI papers and their abstracts.\n3. Performed a web search to gather general knowledge and an overview of AI from various reputable sources.', 'main_body': "Artificial Intelligence (AI) refers to the simulation of human intelligence processes by machines, particularly computer systems. These processes include learning (the acquisition of information and rules for using the information), reasoning (using rules to reach approximate or definite conclusions), and self-correction. AI applications are numerous and diverse, ranging from natural language processing and robotics to autonomous vehicles and predictive analytics.\n\nOne significant area of AI research is the development of large language models (LLMs) and cognitive architectures. For instance, the paper titled 'Cognitive Architectures for Language Agents' introduces the CoALA framework, which organizes language agents with modular memory components and structured action spaces. This framework aims to contextualize current language agents within the broader history of AI and outline a path towards more capable, language-based general intelligence.\n\nAnother notable development is the use of AI agents that leverage advanced language models like GPT-4. These agents can perform a wide range of tasks across various domains, from research and coding to emotional companionship. For example, the chatbot Pi3 is designed not only for productivity tasks but also for providing emotional support, showcasing the versatility and potential of AI in enhancing human-machine interactions.\n\nThe general overview of AI highlights its importance in modern technology. AI enables machines to sense, comprehend, act, and learn, making it a cornerstone of innovations in fields such as healthcare, finance, and transportation. The continuous advancements in AI research and applications are driving significant changes in how we interact with technology and how various industries operate.", 'conclusion': 'Artificial Intelligence is a multifaceted field that is revolutionizing the way machines interact with the world. From cognitive architectures to versatile AI agents, the advancements in AI are paving the way for more intelligent and capable systems. As research continues to evolve, AI will undoubtedly play an increasingly integral role in shaping the future of technology and society.', 'sources': '- Cognitive Architectures for Language Agents, ArXiv ID: 2309.02427\n- An In-depth Survey of Large Language Model-based Artificial Intelligence Agents, ArXiv ID: 2309.14365\n- Overview of Artificial Intelligence Technology, FINRA\n- What is Artificial Intelligence (AI) & Why is it Important?, Accenture\n- What is Artificial Intelligence (AI)?, IBM\n- What is Artificial Intelligence and Why It Matters in 2024?, Simplilearn\n- What is artificial intelligence (AI)? Everything you need to know, TechTarget'})

print(build_report(
    output=out["intermediate_steps"][-1].tool_input
))


INTRODUCTION
------------
Artificial Intelligence (AI) is a rapidly evolving field that encompasses a wide range of technologies and applications aimed at simulating human intelligence in machines. From cognitive architectures to large language models, AI is transforming various domains by enabling machines to perform tasks that typically require human intelligence.

RESEARCH STEPS
--------------
1. Conducted a specialist search on AI using the RAG search tool.
2. Retrieved detailed information on specific AI papers and their abstracts.
3. Performed a web search to gather general knowledge and an overview of AI from various reputable sources.

REPORT
------
Artificial Intelligence (AI) refers to the simulation of human intelligence processes by machines, particularly computer systems. These processes include learning (the acquisition of information and rules for using the information), reasoning (using rules to reach approximate or definite conclusions), and self-correction. AI applications are numerous and diverse, ranging from natural language processing and robotics to autonomous vehicles and predictive analytics.

One significant area of AI research is the development of large language models (LLMs) and cognitive architectures. For instance, the paper titled 'Cognitive Architectures for Language Agents' introduces the CoALA framework, which organizes language agents with modular memory components and structured action spaces. This framework aims to contextualize current language agents within the broader history of AI and outline a path towards more capable, language-based general intelligence.

Another notable development is the use of AI agents that leverage advanced language models like GPT-4. These agents can perform a wide range of tasks across various domains, from research and coding to emotional companionship. For example, the chatbot Pi3 is designed not only for productivity tasks but also for providing emotional support, showcasing the versatility and potential of AI in enhancing human-machine interactions.

The general overview of AI highlights its importance in modern technology. AI enables machines to sense, comprehend, act, and learn, making it a cornerstone of innovations in fields such as healthcare, finance, and transportation. The continuous advancements in AI research and applications are driving significant changes in how we interact with technology and how various industries operate.

CONCLUSION
----------
Artificial Intelligence is a multifaceted field that is revolutionizing the way machines interact with the world. From cognitive architectures to versatile AI agents, the advancements in AI are paving the way for more intelligent and capable systems. As research continues to evolve, AI will undoubtedly play an increasingly integral role in shaping the future of technology and society.

SOURCES
-------
- Cognitive Architectures for Language Agents, ArXiv ID: 2309.02427
- An In-depth Survey of Large Language Model-based Artificial Intelligence Agents, ArXiv ID: 2309.14365
- Overview of Artificial Intelligence Technology, FINRA
- What is Artificial Intelligence (AI) & Why is it Important?, Accenture
- What is Artificial Intelligence (AI)?, IBM
- What is Artificial Intelligence and Why It Matters in 2024?, Simplilearn
- What is artificial intelligence (AI)? Everything you need to know, TechTarget

Let's try one more, we can ask about RAG:

out = runnable.invoke({
    "input": "what is retrieval augmented generation?",
    "chat_history": []
})

run_oracle
rag_search.invoke(input={'query': 'retrieval augmented generation'})
run_oracle
web_search.invoke(input={'query': 'retrieval augmented generation'})
run_oracle
final_answer.invoke(input={'introduction': 'Retrieval-Augmented Generation (RAG) is an advanced technique in the field of artificial intelligence that aims to enhance the performance of generative models by incorporating external information. This method is particularly useful for improving the accuracy and reliability of responses generated by large language models (LLMs).', 'research_steps': '1. Conducted a specialized search using the RAG search tool to gather detailed information on Retrieval-Augmented Generation.\n2. Performed a web search to collect general knowledge and additional perspectives on RAG.\n3. Compiled and synthesized the information from both specialized and general sources to provide a comprehensive overview.', 'main_body': 'Retrieval-Augmented Generation (RAG) is a framework designed to improve the quality of responses generated by large language models (LLMs) by integrating external information sources. Traditional generative models rely solely on the data they were trained on, which can lead to limitations in accuracy and relevance, especially when dealing with specialized or up-to-date information. RAG addresses this issue by incorporating a retrieval mechanism that fetches relevant documents or data from external databases or knowledge bases during the generation process.\n\nThe RAG system typically consists of two main components: a retriever and a generator. The retriever is responsible for searching and retrieving relevant documents or pieces of information based on the input query. This retrieved information is then fed into the generator, which uses it to produce a more accurate and contextually relevant response. This approach allows the model to leverage both its inherent knowledge and the most current or specialized information available externally.\n\nSeveral studies and implementations have demonstrated the effectiveness of RAG. For instance, the AutoGen system employs a multi-agent conversation framework to build a RAG system, enhancing its performance in tasks like code generation and question answering. Another example is the SimplyRetrieve tool, which compares different approaches, including RAG, to highlight its advantages in integrating external knowledge seamlessly.\n\nRAG is particularly beneficial in scenarios where the information required is dynamic or highly specialized, such as medical diagnosis, legal advice, or technical support. By grounding the generative process in authoritative and up-to-date sources, RAG significantly improves the reliability and accuracy of the generated content.', 'conclusion': 'Retrieval-Augmented Generation represents a significant advancement in the field of AI, addressing the limitations of traditional generative models by incorporating external information. This technique enhances the accuracy, relevance, and reliability of AI-generated responses, making it a valuable tool in various specialized and dynamic fields.', 'sources': '- AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, ArXiv ID: 2308.08155\n- SimplyRetrieve: A Private and Lightweight Retrieval-Centric Generative AI Tool, ArXiv ID: 2308.03983\n- NVIDIA Blog: What Is Retrieval-Augmented Generation aka RAG\n- AWS: What is RAG (Retrieval-Augmented Generation)?\n- GitHub Blog: What is retrieval-augmented generation, and what does it do for generative AI?\n- IBM Research Blog: What is retrieval-augmented generation (RAG)?'})

print(build_report(
    output=out["intermediate_steps"][-1].tool_input
))


INTRODUCTION
------------
Retrieval-Augmented Generation (RAG) is an advanced technique in the field of artificial intelligence that aims to enhance the performance of generative models by incorporating external information. This method is particularly useful for improving the accuracy and reliability of responses generated by large language models (LLMs).

RESEARCH STEPS
--------------
1. Conducted a specialized search using the RAG search tool to gather detailed information on Retrieval-Augmented Generation.
2. Performed a web search to collect general knowledge and additional perspectives on RAG.
3. Compiled and synthesized the information from both specialized and general sources to provide a comprehensive overview.

REPORT
------
Retrieval-Augmented Generation (RAG) is a framework designed to improve the quality of responses generated by large language models (LLMs) by integrating external information sources. Traditional generative models rely solely on the data they were trained on, which can lead to limitations in accuracy and relevance, especially when dealing with specialized or up-to-date information. RAG addresses this issue by incorporating a retrieval mechanism that fetches relevant documents or data from external databases or knowledge bases during the generation process.

The RAG system typically consists of two main components: a retriever and a generator. The retriever is responsible for searching and retrieving relevant documents or pieces of information based on the input query. This retrieved information is then fed into the generator, which uses it to produce a more accurate and contextually relevant response. This approach allows the model to leverage both its inherent knowledge and the most current or specialized information available externally.

Several studies and implementations have demonstrated the effectiveness of RAG. For instance, the AutoGen system employs a multi-agent conversation framework to build a RAG system, enhancing its performance in tasks like code generation and question answering. Another example is the SimplyRetrieve tool, which compares different approaches, including RAG, to highlight its advantages in integrating external knowledge seamlessly.

RAG is particularly beneficial in scenarios where the information required is dynamic or highly specialized, such as medical diagnosis, legal advice, or technical support. By grounding the generative process in authoritative and up-to-date sources, RAG significantly improves the reliability and accuracy of the generated content.

CONCLUSION
----------
Retrieval-Augmented Generation represents a significant advancement in the field of AI, addressing the limitations of traditional generative models by incorporating external information. This technique enhances the accuracy, relevance, and reliability of AI-generated responses, making it a valuable tool in various specialized and dynamic fields.

SOURCES
-------
- AutoGen: Enabling Next-Gen LLM Applications via Multi-Agent Conversation, ArXiv ID: 2308.08155
- SimplyRetrieve: A Private and Lightweight Retrieval-Centric Generative AI Tool, ArXiv ID: 2308.03983
- NVIDIA Blog: What Is Retrieval-Augmented Generation aka RAG
- AWS: What is RAG (Retrieval-Augmented Generation)?
- GitHub Blog: What is retrieval-augmented generation, and what does it do for generative AI?
- IBM Research Blog: What is retrieval-augmented generation (RAG)?

There we go, we're seeing our research agent use multiple tools such as web search and arxiv search, synthesize all of the information it retrieves, and finally produce a mini-research report!

That's it for this article covering LangGraph and research agents. We've looked at the essentials behind conversational agents and research agents and where they share similarities and differences—particularly on the engineering front. We've discussed the idea behind graph-based agent frameworks and the additional visibility and flexibility they provide. Finally, we built our own research agent using LangGraph.

Graphs and agents are very promising concepts, and we expect pairing both to become an increasingly popular choice for engineers when building AI applications—particularly as AI becomes an increasingly critical component of reliable and trustworthy software.