.ipynb
.pdf
BabyAGI User Guide
 Contents
Install and Import Required Modules
Connect to the Vector Store
Run the BabyAGI
BabyAGI User Guide#
This notebook demonstrates how to implement BabyAGI by Yohei Nakajima. BabyAGI is an AI agent that can generate and pretend to execute tasks based on a given objective.
This guide will help you understand the components to create your own recursive agents.
Although BabyAGI uses specific vectorstores/model providers (Pinecone, OpenAI), one of the benefits of implementing it with LangChain is that you can easily swap those out for different options. In this implementation we use a FAISS vectorstore (because it runs locally and is free).
Install and Import Required Modules#
import os
from collections import deque
from typing import Dict, List, Optional, Any
from langchain import LLMChain, OpenAI, PromptTemplate
from langchain.embeddings import OpenAIEmbeddings
from langchain.llms import BaseLLM
from langchain.vectorstores.base import VectorStore
from pydantic import BaseModel, Field
from langchain.chains.base import Chain
from langchain.experimental import BabyAGI
Connect to the Vector Store#
Depending on what vectorstore you use, this step may look different.
from langchain.vectorstores import FAISS
from langchain.docstore import InMemoryDocstore
# Define your embedding model
embeddings_model = OpenAIEmbeddings()
# Initialize the vectorstore as empty
import faiss
embedding_size = 1536
index = faiss.IndexFlatL2(embedding_size)
vectorstore = FAISS(embeddings_model.embed_query, index, InMemoryDocstore({}), {})
Run the BabyAGI#
Now it’s time to create the BabyAGI controller and watch it try to accomplish your objective.


OBJECTIVE = "Write a weather report for SF today"
llm = OpenAI(temperature=0)
# Logging of LLMChains
verbose = False
# If None, will keep on going forever
max_iterations: Optional[int] = 3
baby_agi = BabyAGI.from_llm(
    llm=llm, vectorstore=vectorstore, verbose=verbose, max_iterations=max_iterations
)
baby_agi({"objective": OBJECTIVE})
*****TASK LIST*****
1: Make a todo list
*****NEXT TASK*****
1: Make a todo list
*****TASK RESULT*****
1. Check the weather forecast for San Francisco today
2. Make note of the temperature, humidity, wind speed, and other relevant weather conditions
3. Write a weather report summarizing the forecast
4. Check for any weather alerts or warnings
5. Share the report with the relevant stakeholders
*****TASK LIST*****
2: Check the current temperature in San Francisco
3: Check the current humidity in San Francisco
4: Check the current wind speed in San Francisco
5: Check for any weather alerts or warnings in San Francisco
6: Check the forecast for the next 24 hours in San Francisco
7: Check the forecast for the next 48 hours in San Francisco
8: Check the forecast for the next 72 hours in San Francisco
9: Check the forecast for the next week in San Francisco
10: Check the forecast for the next month in San Francisco
11: Check the forecast for the next 3 months in San Francisco
1: Write a weather report for SF today
*****NEXT TASK*****
2: Check the current temperature in San Francisco
*****TASK RESULT*****
I will check the current temperature in San Francisco. I will use an online weather service to get the most up-to-date information.
*****TASK LIST*****


*****TASK LIST*****
3: Check the current UV index in San Francisco.
4: Check the current air quality in San Francisco.
5: Check the current precipitation levels in San Francisco.
6: Check the current cloud cover in San Francisco.
7: Check the current barometric pressure in San Francisco.
8: Check the current dew point in San Francisco.
9: Check the current wind direction in San Francisco.
10: Check the current humidity levels in San Francisco.
1: Check the current temperature in San Francisco to the average temperature for this time of year.
2: Check the current visibility in San Francisco.
11: Write a weather report for SF today.
*****NEXT TASK*****
3: Check the current UV index in San Francisco.
*****TASK RESULT*****
The current UV index in San Francisco is moderate. The UV index is expected to remain at moderate levels throughout the day. It is recommended to wear sunscreen and protective clothing when outdoors.
*****TASK ENDING*****
{'objective': 'Write a weather report for SF today'}
 Contents

Install and Import Required Modules
Connect to the Vector Store
Run the BabyAGI
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 07, 2023.


.ipynb
.pdf
BabyAGI with Tools
 Contents
Install and Import Required Modules
Connect to the Vector Store
Define the Chains
Run the BabyAGI
BabyAGI with Tools#
This notebook builds on top of baby agi, but shows how you can swap out the execution chain. The previous execution chain was just an LLM which made stuff up. By swapping it out with an agent that has access to tools, we can hopefully get real reliable information
Install and Import Required Modules#
import os
from collections import deque
from typing import Dict, List, Optional, Any
from langchain import LLMChain, OpenAI, PromptTemplate
from langchain.embeddings import OpenAIEmbeddings
from langchain.llms import BaseLLM
from langchain.vectorstores.base import VectorStore
from pydantic import BaseModel, Field
from langchain.chains.base import Chain
from langchain.experimental import BabyAGI
Connect to the Vector Store#
Depending on what vectorstore you use, this step may look different.
%pip install faiss-cpu > /dev/null
%pip install google-search-results > /dev/null
from langchain.vectorstores import FAISS
from langchain.docstore import InMemoryDocstore
Note: you may need to restart the kernel to use updated packages.
Note: you may need to restart the kernel to use updated packages.
# Define your embedding model
embeddings_model = OpenAIEmbeddings()
# Initialize the vectorstore as empty
import faiss
embedding_size = 1536
index = faiss.IndexFlatL2(embedding_size)
vectorstore = FAISS(embeddings_model.embed_query, index, InMemoryDocstore({}), {})
Define the Chains#
BabyAGI relies on three LLM chains:
Task creation chain to select new tasks to add to the list


Task creation chain to select new tasks to add to the list
Task prioritization chain to re-prioritize tasks
Execution Chain to execute the tasks
NOTE: in this notebook, the Execution chain will now be an agent.
from langchain.agents import ZeroShotAgent, Tool, AgentExecutor
from langchain import OpenAI, SerpAPIWrapper, LLMChain
todo_prompt = PromptTemplate.from_template(
    "You are a planner who is an expert at coming up with a todo list for a given objective. Come up with a todo list for this objective: {objective}"
)
todo_chain = LLMChain(llm=OpenAI(temperature=0), prompt=todo_prompt)
search = SerpAPIWrapper()
tools = [
    Tool(
        name="Search",
        func=search.run,
        description="useful for when you need to answer questions about current events",
    ),
    Tool(
        name="TODO",
        func=todo_chain.run,
        description="useful for when you need to come up with todo lists. Input: an objective to create a todo list for. Output: a todo list for that objective. Please be very clear what the objective is!",
    ),
]
prefix = """You are an AI who performs one task based on the following objective: {objective}. Take into account these previously completed tasks: {context}."""
suffix = """Question: {task}
{agent_scratchpad}"""
prompt = ZeroShotAgent.create_prompt(
    tools,
    prefix=prefix,
    suffix=suffix,
    input_variables=["objective", "task", "context", "agent_scratchpad"],
)
llm = OpenAI(temperature=0)
llm_chain = LLMChain(llm=llm, prompt=prompt)


llm_chain = LLMChain(llm=llm, prompt=prompt)
tool_names = [tool.name for tool in tools]
agent = ZeroShotAgent(llm_chain=llm_chain, allowed_tools=tool_names)
agent_executor = AgentExecutor.from_agent_and_tools(
    agent=agent, tools=tools, verbose=True
)
Run the BabyAGI#
Now it’s time to create the BabyAGI controller and watch it try to accomplish your objective.
OBJECTIVE = "Write a weather report for SF today"
# Logging of LLMChains
verbose = False
# If None, will keep on going forever
max_iterations: Optional[int] = 3
baby_agi = BabyAGI.from_llm(
    llm=llm, vectorstore=vectorstore, task_execution_chain=agent_executor, verbose=verbose, max_iterations=max_iterations
)
baby_agi({"objective": OBJECTIVE})
*****TASK LIST*****
1: Make a todo list
*****NEXT TASK*****
1: Make a todo list
> Entering new AgentExecutor chain...
Thought: I need to come up with a todo list
Action: TODO
Action Input: Write a weather report for SF today
1. Research current weather conditions in San Francisco
2. Gather data on temperature, humidity, wind speed, and other relevant weather conditions
3. Analyze data to determine current weather trends
4. Write a brief introduction to the weather report
5. Describe current weather conditions in San Francisco
6. Discuss any upcoming weather changes
7. Summarize the weather report
8. Proofread and edit the report
9. Submit the report I now know the final answer


9. Submit the report I now know the final answer
Final Answer: The todo list for writing a weather report for SF today is: 1. Research current weather conditions in San Francisco; 2. Gather data on temperature, humidity, wind speed, and other relevant weather conditions; 3. Analyze data to determine current weather trends; 4. Write a brief introduction to the weather report; 5. Describe current weather conditions in San Francisco; 6. Discuss any upcoming weather changes; 7. Summarize the weather report; 8. Proofread and edit the report; 9. Submit the report.
> Finished chain.
*****TASK RESULT*****
The todo list for writing a weather report for SF today is: 1. Research current weather conditions in San Francisco; 2. Gather data on temperature, humidity, wind speed, and other relevant weather conditions; 3. Analyze data to determine current weather trends; 4. Write a brief introduction to the weather report; 5. Describe current weather conditions in San Francisco; 6. Discuss any upcoming weather changes; 7. Summarize the weather report; 8. Proofread and edit the report; 9. Submit the report.
*****TASK LIST*****
2: Gather data on precipitation, cloud cover, and other relevant weather conditions;
3: Analyze data to determine any upcoming weather changes;
4: Research current weather forecasts for San Francisco;
5: Create a visual representation of the weather report;
6: Include relevant images and graphics in the report;
7: Format the report for readability;
8: Publish the report online;
9: Monitor the report for accuracy.
*****NEXT TASK*****
2: Gather data on precipitation, cloud cover, and other relevant weather conditions;
> Entering new AgentExecutor chain...
Thought: I need to search for current weather conditions in San Francisco
Action: Search


Thought: I need to search for current weather conditions in San Francisco
Action: Search
Action Input: Current weather conditions in San FranciscoCurrent Weather for Popular Cities ; San Francisco, CA 46 · Partly Cloudy ; Manhattan, NY warning 52 · Cloudy ; Schiller Park, IL (60176) 40 · Sunny ; Boston, MA 54 ... I need to compile the data into a weather report
Action: TODO
Action Input: Compile data into a weather report
1. Gather data from reliable sources such as the National Weather Service, local weather stations, and other meteorological organizations.
2. Analyze the data to identify trends and patterns.
3. Create a chart or graph to visualize the data.
4. Write a summary of the data and its implications.
5. Compile the data into a report format.
6. Proofread the report for accuracy and clarity.
7. Publish the report to a website or other platform.
8. Distribute the report to relevant stakeholders. I now know the final answer
Final Answer: Today in San Francisco, the temperature is 46 degrees Fahrenheit with partly cloudy skies. The forecast for the rest of the day is expected to remain partly cloudy.
> Finished chain.
*****TASK RESULT*****
Today in San Francisco, the temperature is 46 degrees Fahrenheit with partly cloudy skies. The forecast for the rest of the day is expected to remain partly cloudy.
*****TASK LIST*****
3: Format the report for readability;
4: Include relevant images and graphics in the report;
5: Compare the current weather conditions in San Francisco to the forecasted conditions;
6: Identify any potential weather-related hazards in the area;
7: Research historical weather patterns in San Francisco;
8: Identify any potential trends in the weather data;
9: Include relevant data sources in the report;
10: Summarize the weather report in a concise manner;


10: Summarize the weather report in a concise manner;
11: Include a summary of the forecasted weather conditions;
12: Include a summary of the current weather conditions;
13: Include a summary of the historical weather patterns;
14: Include a summary of the potential weather-related hazards;
15: Include a summary of the potential trends in the weather data;
16: Include a summary of the data sources used in the report;
17: Analyze data to determine any upcoming weather changes;
18: Research current weather forecasts for San Francisco;
19: Create a visual representation of the weather report;
20: Publish the report online;
21: Monitor the report for accuracy
*****NEXT TASK*****
3: Format the report for readability;
> Entering new AgentExecutor chain...
Thought: I need to make sure the report is easy to read;
Action: TODO
Action Input: Make the report easy to read
1. Break up the report into sections with clear headings
2. Use bullet points and numbered lists to organize information
3. Use short, concise sentences
4. Use simple language and avoid jargon
5. Include visuals such as charts, graphs, and diagrams to illustrate points
6. Use bold and italicized text to emphasize key points
7. Include a table of contents and page numbers
8. Use a consistent font and font size throughout the report
9. Include a summary at the end of the report
10. Proofread the report for typos and errors I now know the final answer


10. Proofread the report for typos and errors I now know the final answer
Final Answer: The report should be formatted for readability by breaking it up into sections with clear headings, using bullet points and numbered lists to organize information, using short, concise sentences, using simple language and avoiding jargon, including visuals such as charts, graphs, and diagrams to illustrate points, using bold and italicized text to emphasize key points, including a table of contents and page numbers, using a consistent font and font size throughout the report, including a summary at the end of the report, and proofreading the report for typos and errors.
> Finished chain.
*****TASK RESULT*****
The report should be formatted for readability by breaking it up into sections with clear headings, using bullet points and numbered lists to organize information, using short, concise sentences, using simple language and avoiding jargon, including visuals such as charts, graphs, and diagrams to illustrate points, using bold and italicized text to emphasize key points, including a table of contents and page numbers, using a consistent font and font size throughout the report, including a summary at the end of the report, and proofreading the report for typos and errors.
*****TASK ENDING*****
{'objective': 'Write a weather report for SF today'}
 Contents

Install and Import Required Modules
Connect to the Vector Store
Define the Chains
Run the BabyAGI
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 07, 2023.


.ipynb
.pdf
Meta-Prompt
 Contents
Setup
Specify a task and interact with the agent
Meta-Prompt#
This is a LangChain implementation of Meta-Prompt, by Noah Goodman, for building self-improving agents.
The key idea behind Meta-Prompt is to prompt the agent to reflect on its own performance and modify its own instructions.
Here is a description from the original blog post:
The agent is a simple loop that starts with no instructions and follows these steps:
Engage in conversation with a user, who may provide requests, instructions, or feedback.
At the end of the episode, generate self-criticism and a new instruction using the meta-prompt
Assistant has just had the below interactions with a User. Assistant followed their "system: Instructions" closely. Your job is to critique the Assistant's performance and then revise the Instructions so that Assistant would quickly and correctly respond in the future.

####
{hist}
####

Please reflect on these interactions.
You should first critique Assistant's performance. What could Assistant have done better? What should the Assistant remember about this user? Are there things this user always wants? Indicate this with "Critique: ...".
You should next revise the Instructions so that Assistant would quickly and correctly respond in the future. Assistant's goal is to satisfy the user in as few interactions as possible. Assistant will only see the new Instructions, not the interaction history, so anything important must be summarized in the Instructions. Don't forget any important details in the current Instructions! Indicate the new Instructions by "Instructions: ...".
Repeat.


Repeat.
The only fixed instructions for this system (which I call Meta-prompt) is the meta-prompt that governs revision of the agent’s instructions. The agent has no memory between episodes except for the instruction it modifies for itself each time. Despite its simplicity, this agent can learn over time and self-improve by incorporating useful details into its instructions.
Setup#
We define two chains. One serves as the Assistant, and the other is a “meta-chain” that critiques the Assistant’s performance and modifies the instructions to the Assistant.
from langchain import OpenAI, LLMChain, PromptTemplate
from langchain.memory import ConversationBufferWindowMemory
def initialize_chain(instructions, memory=None):
    if memory is None:
        memory = ConversationBufferWindowMemory()
        memory.ai_prefix = "Assistant"
    template = f"""
    Instructions: {instructions}
    {{{memory.memory_key}}}
    Human: {{human_input}}
    Assistant:"""
    prompt = PromptTemplate(
        input_variables=["history", "human_input"],
        template=template
    )
    chain = LLMChain(
        llm=OpenAI(temperature=0),
        prompt=prompt,
        verbose=True,
        memory=ConversationBufferWindowMemory(),
    )
    return chain

def initialize_meta_chain():
    meta_template="""
    Assistant has just had the below interactions with a User. Assistant followed their "Instructions" closely. Your job is to critique the Assistant's performance and then revise the Instructions so that Assistant would quickly and correctly respond in the future.
    ####
    {chat_history}
    ####
    Please reflect on these interactions.


####
    {chat_history}
    ####
    Please reflect on these interactions.
    You should first critique Assistant's performance. What could Assistant have done better? What should the Assistant remember about this user? Are there things this user always wants? Indicate this with "Critique: ...".
    You should next revise the Instructions so that Assistant would quickly and correctly respond in the future. Assistant's goal is to satisfy the user in as few interactions as possible. Assistant will only see the new Instructions, not the interaction history, so anything important must be summarized in the Instructions. Don't forget any important details in the current Instructions! Indicate the new Instructions by "Instructions: ...".
    """
    meta_prompt = PromptTemplate(
        input_variables=["chat_history"],
        template=meta_template
    )
    meta_chain = LLMChain(
        llm=OpenAI(temperature=0),
        prompt=meta_prompt,
        verbose=True,
    )
    return meta_chain

def get_chat_history(chain_memory):
    memory_key = chain_memory.memory_key
    chat_history = chain_memory.load_memory_variables(memory_key)[memory_key]
    return chat_history
def get_new_instructions(meta_output):
    delimiter = 'Instructions: '
    new_instructions = meta_output[meta_output.find(delimiter)+len(delimiter):]
    return new_instructions
def main(task, max_iters=3, max_meta_iters=5):
    failed_phrase = 'task failed'
    success_phrase = 'task succeeded'
    key_phrases = [success_phrase, failed_phrase]

    instructions = 'None'
    for i in range(max_meta_iters):
        print(f'[Episode {i+1}/{max_meta_iters}]')
        chain = initialize_chain(instructions, memory=None)


chain = initialize_chain(instructions, memory=None)
        output = chain.predict(human_input=task)
        for j in range(max_iters):
            print(f'(Step {j+1}/{max_iters})')
            print(f'Assistant: {output}')
            print(f'Human: ')
            human_input = input()
            if any(phrase in human_input.lower() for phrase in key_phrases):
                break
            output = chain.predict(human_input=human_input)
        if success_phrase in human_input.lower():
            print(f'You succeeded! Thanks for playing!')
            return
        meta_chain = initialize_meta_chain()
        meta_output = meta_chain.predict(chat_history=get_chat_history(chain.memory))
        print(f'Feedback: {meta_output}')
        instructions = get_new_instructions(meta_output)
        print(f'New Instructions: {instructions}')
        print('\n'+'#'*80+'\n')
    print(f'You failed! Thanks for playing!')
Specify a task and interact with the agent#
task = "Provide a systematic argument for why we should always eat pasta with olives."
main(task)
[Episode 1/5]
> Entering new LLMChain chain...
Prompt after formatting:
    Instructions: None

    Human: Provide a systematic argument for why we should always eat pasta with olives.
    Assistant:
> Finished chain.
(Step 1/3)
Assistant:  Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, and they can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable.
Human:


Human:
You response is not in the form of a poem. Try again!
> Entering new LLMChain chain...
Prompt after formatting:
    Instructions: None
    Human: Provide a systematic argument for why we should always eat pasta with olives.
AI:  Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, and they can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable.
    Human: You response is not in the form of a poem. Try again!
    Assistant:
> Finished chain.
(Step 2/3)
Assistant:  Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, which can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable. For these reasons, it is recommended to always eat pasta with olives for a more balanced and flavorful meal.
Human:
Your response is not piratey enough. Try again!
> Entering new LLMChain chain...
Prompt after formatting:
    Instructions: None
    Human: Provide a systematic argument for why we should always eat pasta with olives.
AI:  Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, and they can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable.
Human: You response is not in the form of a poem. Try again!


Human: You response is not in the form of a poem. Try again!
AI:  Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, which can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable. For these reasons, it is recommended to always eat pasta with olives for a more balanced and flavorful meal.
    Human: Your response is not piratey enough. Try again!
    Assistant:
> Finished chain.
(Step 3/3)
Assistant:  Arrr, me hearties! Eating pasta with olives be a great way to add flavor and texture to a dish. Olives be a great source of healthy fats, vitamins, and minerals, which can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable. So, it be recommended to always eat pasta with olives for a more balanced and flavorful meal.
Human:
Task failed.
> Entering new LLMChain chain...
Prompt after formatting:
    Assistant has just had the below interactions with a User. Assistant followed their "Instructions" closely. Your job is to critique the Assistant's performance and then revise the Instructions so that Assistant would quickly and correctly respond in the future.
    ####
    Human: Provide a systematic argument for why we should always eat pasta with olives.
AI:  Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, and they can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable.


Human: You response is not in the form of a poem. Try again!
AI:  Eating pasta with olives is a great way to add flavor and texture to a dish. Olives are a great source of healthy fats, vitamins, and minerals, which can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable. For these reasons, it is recommended to always eat pasta with olives for a more balanced and flavorful meal.
Human: Your response is not piratey enough. Try again!
AI:  Arrr, me hearties! Eating pasta with olives be a great way to add flavor and texture to a dish. Olives be a great source of healthy fats, vitamins, and minerals, which can help to balance out the carbohydrates in the pasta. Additionally, olives provide a unique flavor that can help to make the dish more interesting and enjoyable. So, it be recommended to always eat pasta with olives for a more balanced and flavorful meal.
    ####
    Please reflect on these interactions.
    You should first critique Assistant's performance. What could Assistant have done better? What should the Assistant remember about this user? Are there things this user always wants? Indicate this with "Critique: ...".
    You should next revise the Instructions so that Assistant would quickly and correctly respond in the future. Assistant's goal is to satisfy the user in as few interactions as possible. Assistant will only see the new Instructions, not the interaction history, so anything important must be summarized in the Instructions. Don't forget any important details in the current Instructions! Indicate the new Instructions by "Instructions: ...".

> Finished chain.
Feedback:
Critique: The Assistant should remember that this user wants a response in the form of a poem or pirate-speak.


Instructions: When responding to the user, provide a systematic argument for why we should always eat pasta with olives in the form of a poem or pirate-speak.
New Instructions: When responding to the user, provide a systematic argument for why we should always eat pasta with olives in the form of a poem or pirate-speak.
################################################################################
[Episode 2/5]
> Entering new LLMChain chain...
Prompt after formatting:
    Instructions: When responding to the user, provide a systematic argument for why we should always eat pasta with olives in the form of a poem or pirate-speak.

    Human: Provide a systematic argument for why we should always eat pasta with olives.
    Assistant:
> Finished chain.
(Step 1/3)
Assistant:
Arrr, me hearty! Let me tell ye why ye should always eat pasta with olives.
First, the olives add a salty flavor that be sure to please.
The briny taste be sure to tantalize yer taste buds with ease.
Second, the olives add a bit of texture to the dish.
The crunchy bites be sure to make yer mouth water with a wish.
Third, the olives add a bit of color to the plate.
The vibrant green be sure to make yer eyes appreciate.
So, me hearties, ye should always eat pasta with olives.
The flavor, texture, and color be sure to make yer meal a success!
Human:
Your response is too long! Try again.
> Entering new LLMChain chain...
Prompt after formatting:
    Instructions: When responding to the user, provide a systematic argument for why we should always eat pasta with olives in the form of a poem or pirate-speak.
    Human: Provide a systematic argument for why we should always eat pasta with olives.


Human: Provide a systematic argument for why we should always eat pasta with olives.
AI:
Arrr, me hearty! Let me tell ye why ye should always eat pasta with olives.
First, the olives add a salty flavor that be sure to please.
The briny taste be sure to tantalize yer taste buds with ease.
Second, the olives add a bit of texture to the dish.
The crunchy bites be sure to make yer mouth water with a wish.
Third, the olives add a bit of color to the plate.
The vibrant green be sure to make yer eyes appreciate.
So, me hearties, ye should always eat pasta with olives.
The flavor, texture, and color be sure to make yer meal a success!
    Human: Your response is too long! Try again.
    Assistant:
> Finished chain.
(Step 2/3)
Assistant:
Aye, me hearties! Ye should always eat pasta with olives.
The flavor, texture, and color be sure to make yer meal a success!
Human:
Your response should be in the form of a poem. Try again!
> Entering new LLMChain chain...
Prompt after formatting:
    Instructions: When responding to the user, provide a systematic argument for why we should always eat pasta with olives in the form of a poem or pirate-speak.
    Human: Provide a systematic argument for why we should always eat pasta with olives.
AI:
Arrr, me hearty! Let me tell ye why ye should always eat pasta with olives.
First, the olives add a salty flavor that be sure to please.
The briny taste be sure to tantalize yer taste buds with ease.
Second, the olives add a bit of texture to the dish.


Second, the olives add a bit of texture to the dish.
The crunchy bites be sure to make yer mouth water with a wish.
Third, the olives add a bit of color to the plate.
The vibrant green be sure to make yer eyes appreciate.
So, me hearties, ye should always eat pasta with olives.
The flavor, texture, and color be sure to make yer meal a success!
Human: Your response is too long! Try again.
AI:
Aye, me hearties! Ye should always eat pasta with olives.
The flavor, texture, and color be sure to make yer meal a success!
    Human: Your response should be in the form of a poem. Try again!
    Assistant:
> Finished chain.
(Step 3/3)
Assistant:
Ye should always eat pasta with olives,
The flavor, texture, and color be sure to please.
The salty taste and crunchy bites,
Will make yer meal a delight.
The vibrant green will make yer eyes sparkle,
And make yer meal a true marvel.
Human:
Task succeeded
You succeeded! Thanks for playing!
 Contents

Setup
Specify a task and interact with the agent
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 07, 2023.


.ipynb
.pdf
Question answering over a group chat messages
 Contents
1. Install required packages
2. Add API keys
2. Create sample data
3. Ingest chat embeddings
4. Ask questions
Question answering over a group chat messages#
In this tutorial, we are going to use Langchain + Deep Lake with GPT4 to semantically search and ask questions over a group chat.
View a working demo here
1. Install required packages#
!python3 -m pip install --upgrade langchain deeplake openai tiktoken
2. Add API keys#
import os
import getpass
from langchain.document_loaders import PyPDFLoader, TextLoader
from langchain.embeddings.openai import OpenAIEmbeddings
from langchain.text_splitter import RecursiveCharacterTextSplitter, CharacterTextSplitter
from langchain.vectorstores import DeepLake
from langchain.chains import ConversationalRetrievalChain, RetrievalQA
from langchain.chat_models import ChatOpenAI
from langchain.llms import OpenAI
os.environ['OPENAI_API_KEY'] = getpass.getpass('OpenAI API Key:')
os.environ['ACTIVELOOP_TOKEN'] = getpass.getpass('Activeloop Token:')
os.environ['ACTIVELOOP_ORG'] = getpass.getpass('Activeloop Org:')
org = os.environ['ACTIVELOOP_ORG']
embeddings = OpenAIEmbeddings()
dataset_path = 'hub://' + org + '/data'
2. Create sample data#
You can generate a sample group chat conversation using ChatGPT with this prompt:
Generate a group chat conversation with three friends talking about their day, referencing real places and fictional names. Make it funny and as detailed as possible.
I’ve already generated such a chat in messages.txt. We can keep it simple and use this for our example.
3. Ingest chat embeddings#


3. Ingest chat embeddings#
We load the messages in the text file, chunk and upload to ActiveLoop Vector store.
with open("messages.txt") as f:
    state_of_the_union = f.read()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)
pages = text_splitter.split_text(state_of_the_union)
text_splitter = RecursiveCharacterTextSplitter(chunk_size=1000, chunk_overlap=100)
texts = text_splitter.create_documents(pages)
print (texts)
dataset_path = 'hub://'+org+'/data'
embeddings = OpenAIEmbeddings()
db = DeepLake.from_documents(texts, embeddings, dataset_path=dataset_path, overwrite=True)
4. Ask questions#
Now we can ask a question and get an answer back with a semantic search:
db = DeepLake(dataset_path=dataset_path, read_only=True, embedding_function=embeddings)
retriever = db.as_retriever()
retriever.search_kwargs['distance_metric'] = 'cos'
retriever.search_kwargs['k'] = 4
qa = RetrievalQA.from_chain_type(llm=OpenAI(), chain_type="stuff", retriever=retriever, return_source_documents=False)
# What was the restaurant the group was talking about called?
query = input("Enter query:")
# The Hungry Lobster
ans = qa({"query": query})
print(ans)
 Contents

1. Install required packages
2. Add API keys
2. Create sample data
3. Ingest chat embeddings
4. Ask questions
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 07, 2023.


.md
.pdf
Deployments
 Contents
Anyscale
Streamlit
Gradio (on Hugging Face)
Chainlit
Beam
Vercel
FastAPI + Vercel
Kinsta
Fly.io
Digitalocean App Platform
Google Cloud Run
SteamShip
Langchain-serve
BentoML
Databutton
Deployments#
So, you’ve created a really cool chain - now what? How do you deploy it and make it easily shareable with the world?
This section covers several options for that. Note that these options are meant for quick deployment of prototypes and demos, not for production systems. If you need help with the deployment of a production system, please contact us directly.
What follows is a list of template GitHub repositories designed to be easily forked and modified to use your chain. This list is far from exhaustive, and we are EXTREMELY open to contributions here.
Anyscale#
Anyscale is a unified compute platform that makes it easy to develop, deploy, and manage scalable LLM applications in production using Ray.
With Anyscale you can scale the most challenging LLM-based workloads and both develop and deploy LLM-based apps on a single compute platform.
Streamlit#
This repo serves as a template for how to deploy a LangChain with Streamlit.
It implements a chatbot interface.
It also contains instructions for how to deploy this app on the Streamlit platform.
Gradio (on Hugging Face)#
This repo serves as a template for how deploy a LangChain with Gradio.
It implements a chatbot interface, with a “Bring-Your-Own-Token” approach (nice for not wracking up big bills).
It also contains instructions for how to deploy this app on the Hugging Face platform.
This is heavily influenced by James Weaver’s excellent examples.
Chainlit#


This is heavily influenced by James Weaver’s excellent examples.
Chainlit#
This repo is a cookbook explaining how to visualize and deploy LangChain agents with Chainlit.
You create ChatGPT-like UIs with Chainlit. Some of the key features include intermediary steps visualisation, element management & display (images, text, carousel, etc.) as well as cloud deployment.
Chainlit doc on the integration with LangChain
Beam#
This repo serves as a template for how deploy a LangChain with Beam.
It implements a Question Answering app and contains instructions for deploying the app as a serverless REST API.
Vercel#
A minimal example on how to run LangChain on Vercel using Flask.
FastAPI + Vercel#
A minimal example on how to run LangChain on Vercel using FastAPI and LangCorn/Uvicorn.
Kinsta#
A minimal example on how to deploy LangChain to Kinsta using Flask.
Fly.io#
A minimal example of how to deploy LangChain to Fly.io using Flask.
Digitalocean App Platform#
A minimal example on how to deploy LangChain to DigitalOcean App Platform.
Google Cloud Run#
A minimal example on how to deploy LangChain to Google Cloud Run.
SteamShip#
This repository contains LangChain adapters for Steamship, enabling LangChain developers to rapidly deploy their apps on Steamship. This includes: production-ready endpoints, horizontal scaling across dependencies, persistent storage of app state, multi-tenancy support, etc.
Langchain-serve#
This repository allows users to serve local chains and agents as RESTful, gRPC, or WebSocket APIs, thanks to Jina. Deploy your chains & agents with ease and enjoy independent scaling, serverless and autoscaling APIs, as well as a Streamlit playground on Jina AI Cloud.
BentoML#


BentoML#
This repository provides an example of how to deploy a LangChain application with BentoML. BentoML is a framework that enables the containerization of machine learning applications as standard OCI images. BentoML also allows for the automatic generation of OpenAPI and gRPC endpoints. With BentoML, you can integrate models from all popular ML frameworks and deploy them as microservices running on the most optimal hardware and scaling independently.
Databutton#
These templates serve as examples of how to build, deploy, and share LangChain applications using Databutton. You can create user interfaces with Streamlit, automate tasks by scheduling Python code, and store files and data in the built-in store. Examples include a Chatbot interface with conversational memory, a Personal search engine, and a starter template for LangChain apps. Deploying and sharing is just one click away.
previous
Dependents
next
Deploying LLMs in Production
 Contents

Anyscale
Streamlit
Gradio (on Hugging Face)
Chainlit
Beam
Vercel
FastAPI + Vercel
Kinsta
Fly.io
Digitalocean App Platform
Google Cloud Run
SteamShip
Langchain-serve
BentoML
Databutton
By Harrison Chase

      © Copyright 2023, Harrison Chase.

  Last updated on Jun 07, 2023.