Elasticsearch database | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsHow toFoundationalDocumentsPopularAdditionalAnalyze DocumentSelf-critique chain with constitutional AICausal program-aided language (CPAL) chainElasticsearch databaseExtractionFLAREArangoDB QA chainGraph DB QA chainHugeGraph QA ChainKuzuQAChainNebulaGraphQAChainGraph QAGraphSparqlQAChainHypothetical Document EmbeddingsBash chainSelf-checking chainMath chainHTTP request chainSummarization checker chainLLM Symbolic MathModerationDynamically selecting from multiple promptsDynamically selecting from multiple retrieversNeptune Open Cypher QA ChainRetrieval QA using OpenAI functionsOpenAPI chainOpenAPI calls with OpenAI functionsProgram-aided language model (PAL) chainQuestion-Answering CitationsDocument QATaggingVector store-augmented text generationMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesChainsAdditionalElasticsearch databaseOn this pageElasticsearch databaseInteract with Elasticsearch analytics database via Langchain. This chain builds search queries via the Elasticsearch DSL API (filters and aggregations).The Elasticsearch client must have permissions for index listing, mapping description and search queries.See here for instructions on how to run Elasticsearch locally.Make sure to install the Elasticsearch Python client before:pip install elasticsearchfrom elasticsearch import Elasticsearchfrom langchain.chains.elasticsearch_database import ElasticsearchDatabaseChainfrom langchain.chat_models import ChatOpenAI# Initialize Elasticsearch python client.# See https://elasticsearch-py.readthedocs.io/en/v8.8.2/api.html#elasticsearch.ElasticsearchELASTIC_SEARCH_SERVER = "https://elastic:pass@localhost:9200"db =


= "https://elastic:pass@localhost:9200"db = Elasticsearch(ELASTIC_SEARCH_SERVER)Uncomment the next cell to initially populate your db.# customers = [#     {"firstname": "Jennifer", "lastname": "Walters"},#     {"firstname": "Monica","lastname":"Rambeau"},#     {"firstname": "Carol","lastname":"Danvers"},#     {"firstname": "Wanda","lastname":"Maximoff"},#     {"firstname": "Jennifer","lastname":"Takeda"},# ]# for i, customer in enumerate(customers):#     db.create(index="customers", document=customer, id=i)llm = ChatOpenAI(model_name="gpt-4", temperature=0)chain = ElasticsearchDatabaseChain.from_llm(llm=llm, database=db, verbose=True)question = "What are the first names of all the customers?"chain.run(question)            > Entering new ElasticsearchDatabaseChain chain...    What are the first names of all the customers?    ESQuery:{'size': 10, 'query': {'match_all': {}}, '_source': ['firstname']}    ESResult: {'took': 5, 'timed_out': False, '_shards': {'total': 1, 'successful': 1, 'skipped': 0, 'failed': 0}, 'hits': {'total': {'value': 6, 'relation': 'eq'}, 'max_score': 1.0, 'hits': [{'_index': 'customers', '_id': '0', '_score': 1.0, '_source': {'firstname': 'Jennifer'}}, {'_index': 'customers', '_id': '1', '_score': 1.0,


'customers', '_id': '1', '_score': 1.0, '_source': {'firstname': 'Monica'}}, {'_index': 'customers', '_id': '2', '_score': 1.0, '_source': {'firstname': 'Carol'}}, {'_index': 'customers', '_id': '3', '_score': 1.0, '_source': {'firstname': 'Wanda'}}, {'_index': 'customers', '_id': '4', '_score': 1.0, '_source': {'firstname': 'Jennifer'}}, {'_index': 'customers', '_id': 'firstname', '_score': 1.0, '_source': {'firstname': 'Jennifer'}}]}}    Answer:The first names of all the customers are Jennifer, Monica, Carol, Wanda, and Jennifer.    > Finished chain.    'The first names of all the customers are Jennifer, Monica, Carol, Wanda, and Jennifer.'Custom promptâ€‹For best results you'll likely need to customize the prompt.from langchain.chains.elasticsearch_database.prompts import DEFAULT_DSL_TEMPLATEfrom langchain.prompts.prompt import PromptTemplatePROMPT_TEMPLATE = """Given an input question, create a syntactically correct Elasticsearch query to run. Unless the user specifies in their question a specific number of examples they wish to obtain, always limit your query to at most {top_k} results. You can order the results by a relevant column to return the most interesting examples in the database.Unless told to do not query for all the columns from a specific index, only ask for a the few relevant columns given the question.Pay attention to use only the column names that you can see in the mapping description. Be careful to not query for columns that do not exist. Also, pay attention to which column is in which index. Return the query as valid json.Use the following format:Question:


which column is in which index. Return the query as valid json.Use the following format:Question: Question hereESQuery: Elasticsearch Query formatted as json"""PROMPT = PromptTemplate.from_template(    PROMPT_TEMPLATE,)chain = ElasticsearchDatabaseChain.from_llm(llm=llm, database=db, query_prompt=PROMPT)Adding example rows from each indexâ€‹Sometimes, the format of the data is not obvious and it is optimal to include a sample of rows from the indices in the prompt to allow the LLM to understand the data before providing a final query. Here we will use this feature to let the LLM know that artists are saved with their full names by providing ten rows from the index.chain = ElasticsearchDatabaseChain.from_llm(    llm=ChatOpenAI(temperature=0),    database=db,    sample_documents_in_index_info=2,  # 2 rows from each index will be included in the prompt as sample data)PreviousCausal program-aided language (CPAL) chainNextExtractionCustom promptAdding example rows from each indexCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Model I/O | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OModel I/OThe core element of any language model application is...the model. LangChain gives you the building blocks to interface with any language model.Prompts: Templatize, dynamically select, and manage model inputsLanguage models: Make calls to language models through common interfacesOutput parsers: Extract information from model outputsPreviousModulesNextPromptsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Output parsers | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsOutput parsersList parserDatetime parserEnum parserAuto-fixing parserPydantic (JSON) parserRetry parserStructured output parserData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OOutput parsersOn this pageOutput parsersLanguage models output text. But many times you may want to get more structured information than just text back. This is where output parsers come in.Output parsers are classes that help structure language model responses. There are two main methods an output parser must implement:"Get format instructions": A method which returns a string containing instructions for how the output of a language model should be formatted."Parse": A method which takes in a string (assumed to be the response from a language model) and parses it into some structure.And then one optional one:"Parse with prompt": A method which takes in a string (assumed to be the response from a language model) and a prompt (assumed to the prompt that generated such a response) and parses it into some structure. The prompt is largely provided in the event the OutputParser wants to retry or fix the output in some way, and needs information from the prompt to do so.Get startedâ€‹Below we go over the main type of output parser, the PydanticOutputParser.from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplatefrom langchain.llms import OpenAIfrom langchain.chat_models import ChatOpenAIfrom langchain.output_parsers import PydanticOutputParserfrom pydantic import BaseModel, Field, validatorfrom typing import Listmodel_name = 'text-davinci-003'temperature = 0.0model =


Listmodel_name = 'text-davinci-003'temperature = 0.0model = OpenAI(model_name=model_name, temperature=temperature)# Define your desired data structure.class Joke(BaseModel):    setup: str = Field(description="question to set up a joke")    punchline: str = Field(description="answer to resolve the joke")        # You can add custom validation logic easily with Pydantic.    @validator('setup')    def question_ends_with_question_mark(cls, field):        if field[-1] != '?':            raise ValueError("Badly formed question!")        return field# Set up a parser + inject instructions into the prompt template.parser = PydanticOutputParser(pydantic_object=Joke)prompt = PromptTemplate(    template="Answer the user query.\n{format_instructions}\n{query}\n",    input_variables=["query"],    partial_variables={"format_instructions": parser.get_format_instructions()})# And a query intended to prompt a language model to populate the data structure.joke_query = "Tell me a joke."_input = prompt.format_prompt(query=joke_query)output = model(_input.to_string())parser.parse(output)    Joke(setup='Why did the chicken cross the road?', punchline='To get to the other side!')PreviousStreamingNextList parserGet startedCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Retry parser | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsOutput parsersList parserDatetime parserEnum parserAuto-fixing parserPydantic (JSON) parserRetry parserStructured output parserData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OOutput parsersRetry parserRetry parserWhile in some cases it is possible to fix any parsing mistakes by only looking at the output, in other cases it can't. An example of this is when the output is not just in the incorrect format, but is partially complete. Consider the below example.from langchain.prompts import (    PromptTemplate,    ChatPromptTemplate,    HumanMessagePromptTemplate,)from langchain.llms import OpenAIfrom langchain.chat_models import ChatOpenAIfrom langchain.output_parsers import (    PydanticOutputParser,    OutputFixingParser,    RetryOutputParser,)from pydantic import BaseModel, Field, validatorfrom typing import Listtemplate = """Based on the user question, provide an Action and Action Input for what step should be taken.{format_instructions}Question: {query}Response:"""class Action(BaseModel):    action: str = Field(description="action to take")    action_input: str = Field(description="input to the action")parser = PydanticOutputParser(pydantic_object=Action)prompt = PromptTemplate(    template="Answer the user query.\n{format_instructions}\n{query}\n",    input_variables=["query"],    partial_variables={"format_instructions": parser.get_format_instructions()},)prompt_value = prompt.format_prompt(query="who is leo di caprios gf?")bad_response


= prompt.format_prompt(query="who is leo di caprios gf?")bad_response = '{"action": "search"}'If we try to parse this response as is, we will get an errorparser.parse(bad_response)    ---------------------------------------------------------------------------    ValidationError                           Traceback (most recent call last)    File ~/workplace/langchain/langchain/output_parsers/pydantic.py:24, in PydanticOutputParser.parse(self, text)         23     json_object = json.loads(json_str)    ---> 24     return self.pydantic_object.parse_obj(json_object)         26 except (json.JSONDecodeError, ValidationError) as e:    File ~/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/pydantic/main.py:527, in pydantic.main.BaseModel.parse_obj()    File ~/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/pydantic/main.py:342, in pydantic.main.BaseModel.__init__()    ValidationError: 1 validation error for Action    action_input      field required (type=value_error.missing)        During handling of the above exception, another exception occurred:    OutputParserException                     Traceback (most recent call last)    Cell In[6], line 1    ----> 1 parser.parse(bad_response)    File


line 1    ----> 1 parser.parse(bad_response)    File ~/workplace/langchain/langchain/output_parsers/pydantic.py:29, in PydanticOutputParser.parse(self, text)         27 name = self.pydantic_object.__name__         28 msg = f"Failed to parse {name} from completion {text}. Got: {e}"    ---> 29 raise OutputParserException(msg)    OutputParserException: Failed to parse Action from completion {"action": "search"}. Got: 1 validation error for Action    action_input      field required (type=value_error.missing)If we try to use the OutputFixingParser to fix this error, it will be confused - namely, it doesn't know what to actually put for action input.fix_parser = OutputFixingParser.from_llm(parser=parser, llm=ChatOpenAI())fix_parser.parse(bad_response)    Action(action='search', action_input='')Instead, we can use the RetryOutputParser, which passes in the prompt (as well as the original output) to try again to get a better response.from langchain.output_parsers import RetryWithErrorOutputParserretry_parser = RetryWithErrorOutputParser.from_llm(    parser=parser, llm=OpenAI(temperature=0))retry_parser.parse_with_prompt(bad_response, prompt_value)    Action(action='search', action_input='who is leo di caprios gf?')PreviousPydantic (JSON) parserNextStructured output parserCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Enum parser | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsOutput parsersList parserDatetime parserEnum parserAuto-fixing parserPydantic (JSON) parserRetry parserStructured output parserData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OOutput parsersEnum parserEnum parserThis notebook shows how to use an Enum output parserfrom langchain.output_parsers.enum import EnumOutputParserfrom enum import Enumclass Colors(Enum):    RED = "red"    GREEN = "green"    BLUE = "blue"parser = EnumOutputParser(enum=Colors)parser.parse("red")    <Colors.RED: 'red'># Can handle spacesparser.parse(" green")    <Colors.GREEN: 'green'># And new linesparser.parse("blue\n")    <Colors.BLUE: 'blue'># And raises errors when appropriateparser.parse("yellow")    ---------------------------------------------------------------------------    ValueError                                Traceback (most recent call last)    File ~/workplace/langchain/langchain/output_parsers/enum.py:25, in EnumOutputParser.parse(self, response)         24 try:    ---> 25     return self.enum(response.strip())         26 except ValueError:    File ~/.pyenv/versions/3.9.1/lib/python3.9/enum.py:315, in EnumMeta.__call__(cls, value, names, module, qualname, type, start)


EnumMeta.__call__(cls, value, names, module, qualname, type, start)        314 if names is None:  # simple value lookup    --> 315     return cls.__new__(cls, value)        316 # otherwise, functional API: we're creating a new Enum type    File ~/.pyenv/versions/3.9.1/lib/python3.9/enum.py:611, in Enum.__new__(cls, value)        610 if result is None and exc is None:    --> 611     raise ve_exc        612 elif exc is None:    ValueError: 'yellow' is not a valid Colors        During handling of the above exception, another exception occurred:    OutputParserException                     Traceback (most recent call last)    Cell In[8], line 2          1 # And raises errors when appropriate    ----> 2 parser.parse("yellow")    File ~/workplace/langchain/langchain/output_parsers/enum.py:27, in EnumOutputParser.parse(self, response)         25     return self.enum(response.strip())         26 except ValueError:    ---> 27     raise OutputParserException(         28         f"Response '{response}' is not one of the "         29         f"expected values:


29         f"expected values: {self._valid_values}"         30     )    OutputParserException: Response 'yellow' is not one of the expected values: ['red', 'green', 'blue']PreviousDatetime parserNextAuto-fixing parserCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Datetime parser | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsOutput parsersList parserDatetime parserEnum parserAuto-fixing parserPydantic (JSON) parserRetry parserStructured output parserData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OOutput parsersDatetime parserDatetime parserThis OutputParser shows out to parse LLM output into datetime format.from langchain.prompts import PromptTemplatefrom langchain.output_parsers import DatetimeOutputParserfrom langchain.chains import LLMChainfrom langchain.llms import OpenAIoutput_parser = DatetimeOutputParser()template = """Answer the users question:{question}{format_instructions}"""prompt = PromptTemplate.from_template(    template,    partial_variables={"format_instructions": output_parser.get_format_instructions()},)chain = LLMChain(prompt=prompt, llm=OpenAI())output = chain.run("around when was bitcoin founded?")output    '\n\n2008-01-03T18:15:05.000000Z'output_parser.parse(output)    datetime.datetime(2008, 1, 3, 18, 15, 5)PreviousList parserNextEnum parserCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Auto-fixing parser | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsOutput parsersList parserDatetime parserEnum parserAuto-fixing parserPydantic (JSON) parserRetry parserStructured output parserData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OOutput parsersAuto-fixing parserAuto-fixing parserThis output parser wraps another output parser, and in the event that the first one fails it calls out to another LLM to fix any errors.But we can do other things besides throw errors. Specifically, we can pass the misformatted output, along with the formatted instructions, to the model and ask it to fix it.For this example, we'll use the above Pydantic output parser. Here's what happens if we pass it a result that does not comply with the schema:from langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplatefrom langchain.llms import OpenAIfrom langchain.chat_models import ChatOpenAIfrom langchain.output_parsers import PydanticOutputParserfrom pydantic import BaseModel, Field, validatorfrom typing import Listclass Actor(BaseModel):    name: str = Field(description="name of an actor")    film_names: List[str] = Field(description="list of names of films they starred in")        actor_query = "Generate the filmography for a random actor."parser = PydanticOutputParser(pydantic_object=Actor)misformatted = "{'name': 'Tom Hanks', 'film_names': ['Forrest Gump']}"parser.parse(misformatted)    ---------------------------------------------------------------------------    JSONDecodeError


JSONDecodeError                           Traceback (most recent call last)    File ~/workplace/langchain/langchain/output_parsers/pydantic.py:23, in PydanticOutputParser.parse(self, text)         22     json_str = match.group()    ---> 23 json_object = json.loads(json_str)         24 return self.pydantic_object.parse_obj(json_object)    File ~/.pyenv/versions/3.9.1/lib/python3.9/json/__init__.py:346, in loads(s, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)        343 if (cls is None and object_hook is None and        344         parse_int is None and parse_float is None and        345         parse_constant is None and object_pairs_hook is None and not kw):    --> 346     return _default_decoder.decode(s)        347 if cls is None:    File ~/.pyenv/versions/3.9.1/lib/python3.9/json/decoder.py:337, in JSONDecoder.decode(self, s, _w)        333 """Return the Python representation of ``s`` (a ``str`` instance        334 containing a JSON document).        335         336 """    --> 337 obj, end = self.raw_decode(s,


336 """    --> 337 obj, end = self.raw_decode(s, idx=_w(s, 0).end())        338 end = _w(s, end).end()    File ~/.pyenv/versions/3.9.1/lib/python3.9/json/decoder.py:353, in JSONDecoder.raw_decode(self, s, idx)        352 try:    --> 353     obj, end = self.scan_once(s, idx)        354 except StopIteration as err:    JSONDecodeError: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)        During handling of the above exception, another exception occurred:    OutputParserException                     Traceback (most recent call last)    Cell In[6], line 1    ----> 1 parser.parse(misformatted)    File ~/workplace/langchain/langchain/output_parsers/pydantic.py:29, in PydanticOutputParser.parse(self, text)         27 name = self.pydantic_object.__name__         28 msg = f"Failed to parse {name} from completion {text}. Got: {e}"    ---> 29 raise OutputParserException(msg)    OutputParserException: Failed to parse Actor from completion {'name': 'Tom Hanks', 'film_names': ['Forrest Gump']}. Got: Expecting property name enclosed in double quotes: line 1 column 2 (char 1)Now we can construct and use a OutputFixingParser. This output parser


(char 1)Now we can construct and use a OutputFixingParser. This output parser takes as an argument another output parser but also an LLM with which to try to correct any formatting mistakes.from langchain.output_parsers import OutputFixingParsernew_parser = OutputFixingParser.from_llm(parser=parser, llm=ChatOpenAI())new_parser.parse(misformatted)    Actor(name='Tom Hanks', film_names=['Forrest Gump'])PreviousEnum parserNextPydantic (JSON) parserCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Structured output parser | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsOutput parsersList parserDatetime parserEnum parserAuto-fixing parserPydantic (JSON) parserRetry parserStructured output parserData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OOutput parsersStructured output parserStructured output parserThis output parser can be used when you want to return multiple fields. While the Pydantic/JSON parser is more powerful, we initially experimented with data structures having text fields only.from langchain.output_parsers import StructuredOutputParser, ResponseSchemafrom langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplatefrom langchain.llms import OpenAIfrom langchain.chat_models import ChatOpenAIHere we define the response schema we want to receive.response_schemas = [    ResponseSchema(name="answer", description="answer to the user's question"),    ResponseSchema(name="source", description="source used to answer the user's question, should be a website.")]output_parser = StructuredOutputParser.from_response_schemas(response_schemas)We now get a string that contains instructions for how the response should be formatted, and we then insert that into our prompt.format_instructions = output_parser.get_format_instructions()prompt = PromptTemplate(    template="answer the users question as best as possible.\n{format_instructions}\n{question}",    input_variables=["question"],    partial_variables={"format_instructions": format_instructions})We can now use this to format a prompt to send to the language model, and then parse the returned result.model = OpenAI(temperature=0)_input = prompt.format_prompt(question="what's the capital of france?")output =


= prompt.format_prompt(question="what's the capital of france?")output = model(_input.to_string())output_parser.parse(output)    {'answer': 'Paris',     'source': 'https://www.worldatlas.com/articles/what-is-the-capital-of-france.html'}And here's an example of using this in a chat modelchat_model = ChatOpenAI(temperature=0)prompt = ChatPromptTemplate(    messages=[        HumanMessagePromptTemplate.from_template("answer the users question as best as possible.\n{format_instructions}\n{question}")      ],    input_variables=["question"],    partial_variables={"format_instructions": format_instructions})_input = prompt.format_prompt(question="what's the capital of france?")output = chat_model(_input.to_messages())output_parser.parse(output.content)    {'answer': 'Paris', 'source': 'https://en.wikipedia.org/wiki/Paris'}PreviousRetry parserNextData connectionCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Pydantic (JSON) parser | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsOutput parsersList parserDatetime parserEnum parserAuto-fixing parserPydantic (JSON) parserRetry parserStructured output parserData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OOutput parsersPydantic (JSON) parserPydantic (JSON) parserThis output parser allows users to specify an arbitrary JSON schema and query LLMs for JSON outputs that conform to that schema.Keep in mind that large language models are leaky abstractions! You'll have to use an LLM with sufficient capacity to generate well-formed JSON. In the OpenAI family, DaVinci can do reliably but Curie's ability already drops off dramatically. Use Pydantic to declare your data model. Pydantic's BaseModel like a Python dataclass, but with actual type checking + coercion.from langchain.prompts import (    PromptTemplate,    ChatPromptTemplate,    HumanMessagePromptTemplate,)from langchain.llms import OpenAIfrom langchain.chat_models import ChatOpenAIfrom langchain.output_parsers import PydanticOutputParserfrom pydantic import BaseModel, Field, validatorfrom typing import Listmodel_name = "text-davinci-003"temperature = 0.0model = OpenAI(model_name=model_name, temperature=temperature)# Define your desired data structure.class Joke(BaseModel):    setup: str = Field(description="question to set up a joke")    punchline: str = Field(description="answer to resolve the joke")    # You can add custom validation logic easily with Pydantic.    @validator("setup")    def


validation logic easily with Pydantic.    @validator("setup")    def question_ends_with_question_mark(cls, field):        if field[-1] != "?":            raise ValueError("Badly formed question!")        return field# And a query intented to prompt a language model to populate the data structure.joke_query = "Tell me a joke."# Set up a parser + inject instructions into the prompt template.parser = PydanticOutputParser(pydantic_object=Joke)prompt = PromptTemplate(    template="Answer the user query.\n{format_instructions}\n{query}\n",    input_variables=["query"],    partial_variables={"format_instructions": parser.get_format_instructions()},)_input = prompt.format_prompt(query=joke_query)output = model(_input.to_string())parser.parse(output)    Joke(setup='Why did the chicken cross the road?', punchline='To get to the other side!')# Here's another example, but with a compound typed field.class Actor(BaseModel):    name: str = Field(description="name of an actor")    film_names: List[str] = Field(description="list of names of films they starred in")actor_query = "Generate the filmography for a random actor."parser = PydanticOutputParser(pydantic_object=Actor)prompt = PromptTemplate(    template="Answer the user query.\n{format_instructions}\n{query}\n",    input_variables=["query"],    partial_variables={"format_instructions": parser.get_format_instructions()},)_input = prompt.format_prompt(query=actor_query)output = model(_input.to_string())parser.parse(output)    Actor(name='Tom Hanks', film_names=['Forrest Gump', 'Saving Private Ryan', 'The


Hanks', film_names=['Forrest Gump', 'Saving Private Ryan', 'The Green Mile', 'Cast Away', 'Toy Story'])PreviousAuto-fixing parserNextRetry parserCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


List parser | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsOutput parsersList parserDatetime parserEnum parserAuto-fixing parserPydantic (JSON) parserRetry parserStructured output parserData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OOutput parsersList parserList parserThis output parser can be used when you want to return a list of comma-separated items.from langchain.output_parsers import CommaSeparatedListOutputParserfrom langchain.prompts import PromptTemplate, ChatPromptTemplate, HumanMessagePromptTemplatefrom langchain.llms import OpenAIfrom langchain.chat_models import ChatOpenAIoutput_parser = CommaSeparatedListOutputParser()format_instructions = output_parser.get_format_instructions()prompt = PromptTemplate(    template="List five {subject}.\n{format_instructions}",    input_variables=["subject"],    partial_variables={"format_instructions": format_instructions})model = OpenAI(temperature=0)_input = prompt.format(subject="ice cream flavors")output = model(_input)output_parser.parse(output)    ['Vanilla',     'Chocolate',     'Strawberry',     'Mint Chocolate Chip',     'Cookies and Cream']PreviousOutput parsersNextDatetime parserCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Language models | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsChat modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsOn this pageLanguage modelsLangChain provides interfaces and integrations for two types of models:LLMs: Models that take a text string as input and return a text stringChat models: Models that are backed by a language model but take a list of Chat Messages as input and return a Chat MessageLLMs vs Chat Modelsâ€‹LLMs and Chat Models are subtly but importantly different. LLMs in LangChain refer to pure text completion models.
The APIs they wrap take a string prompt as input and output a string completion. OpenAI's GPT-3 is implemented as an LLM.
Chat models are often backed by LLMs but tuned specifically for having conversations.
And, crucially, their provider APIs expose a different interface than pure text completion models. Instead of a single string,
they take a list of chat messages as input. Usually these messages are labeled with the speaker (usually one of "System",
"AI", and "Human"). And they return a ("AI") chat message as output. GPT-4 and Anthropic's Claude are both implemented as Chat Models.To make it possible to swap LLMs and Chat Models, both implement the Base Language Model interface. This exposes common
methods "predict", which takes a string and returns a string, and "predict messages", which takes messages and returns a message.


If you are using a specific model it's recommended you use the methods specific to that model class (i.e., "predict" for LLMs and "predict messages" for Chat Models),
but if you're creating an application that should work with different types of models the shared interface can be helpful.PreviousSelect by similarityNextLLMsLLMs vs Chat ModelsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Chat models | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsChat modelsCachingHuman input Chat ModelLLMChainPromptsStreamingOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsChat modelsOn this pageChat modelsinfoHead to Integrations for documentation on built-in integrations with chat model providers.Chat models are a variation on language models.
While chat models use language models under the hood, the interface they expose is a bit different.
Rather than expose a "text in, text out" API, they expose an interface where "chat messages" are the inputs and outputs.Chat model APIs are fairly new, so we are still figuring out the correct abstractions.Get startedâ€‹Setupâ€‹To start we'll need to install the OpenAI Python package:pip install openaiAccessing the API requires an API key, which you can get by creating an account and heading here. Once we have a key we'll want to set it as an environment variable by running:export OPENAI_API_KEY="..."If you'd prefer not to set an environment variable you can pass the key in directly via the openai_api_key named parameter when initiating the OpenAI LLM class:from langchain.chat_models import ChatOpenAIchat = ChatOpenAI(openai_api_key="...")otherwise you can initialize without any params:from langchain.chat_models import ChatOpenAIchat = ChatOpenAI()Messagesâ€‹The chat model interface is based around messages rather than raw text.


The types of messages currently supported in LangChain are AIMessage, HumanMessage, SystemMessage, and ChatMessage -- ChatMessage takes in an arbitrary role parameter. Most of the time, you'll just be dealing with HumanMessage, AIMessage, and SystemMessage__call__â€‹Messages in -> message outâ€‹You can get chat completions by passing one or more messages to the chat model. The response will be a message.from langchain.schema import (    AIMessage,    HumanMessage,    SystemMessage)chat([HumanMessage(content="Translate this sentence from English to French: I love programming.")])    AIMessage(content="J'aime programmer.", additional_kwargs={})OpenAI's chat model supports multiple messages as input. See here for more information. Here is an example of sending a system and user message to the chat model:messages = [    SystemMessage(content="You are a helpful assistant that translates English to French."),    HumanMessage(content="I love programming.")]chat(messages)    AIMessage(content="J'aime programmer.", additional_kwargs={})generateâ€‹Batch calls, richer outputsâ€‹You can go one step further and generate completions for multiple sets of messages using generate. This returns an LLMResult with an additional message parameter.batch_messages = [    [        SystemMessage(content="You are a helpful assistant that translates English to French."),        HumanMessage(content="I love programming.")    ],    [        SystemMessage(content="You are a helpful assistant that translates English to French."),        HumanMessage(content="I love artificial intelligence.")    ],]result = chat.generate(batch_messages)result    LLMResult(generations=[[ChatGeneration(text="J'aime


LLMResult(generations=[[ChatGeneration(text="J'aime programmer.", generation_info=None, message=AIMessage(content="J'aime programmer.", additional_kwargs={}))], [ChatGeneration(text="J'aime l'intelligence artificielle.", generation_info=None, message=AIMessage(content="J'aime l'intelligence artificielle.", additional_kwargs={}))]], llm_output={'token_usage': {'prompt_tokens': 57, 'completion_tokens': 20, 'total_tokens': 77}})You can recover things like token usage from this LLMResultresult.llm_output    {'token_usage': {'prompt_tokens': 57,      'completion_tokens': 20,      'total_tokens': 77}}PreviousTracking token usageNextCachingGet startedCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


LLMChain | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsChat modelsCachingHuman input Chat ModelLLMChainPromptsStreamingOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsChat modelsLLMChainLLMChainYou can use the existing LLMChain in a very similar way to before - provide a prompt and a model.chain = LLMChain(llm=chat, prompt=chat_prompt)chain.run(input_language="English", output_language="French", text="I love programming.")    "J'adore la programmation."PreviousHuman input Chat ModelNextPromptsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Caching | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsChat modelsCachingHuman input Chat ModelLLMChainPromptsStreamingOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsChat modelsCachingCachingLangChain provides an optional caching layer for Chat Models. This is useful for two reasons:It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times.


It can speed up your application by reducing the number of API calls you make to the LLM provider.import langchainfrom langchain.chat_models import ChatOpenAIllm = ChatOpenAI()In Memory Cacheâ€‹from langchain.cache import InMemoryCachelangchain.llm_cache = InMemoryCache()# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")    CPU times: user 35.9 ms, sys: 28.6 ms, total: 64.6 ms    Wall time: 4.83 s        "\n\nWhy couldn't the bicycle stand up by itself? It was...two tired!"# The second time it is, so it goes fasterllm.predict("Tell me a joke")    CPU times: user 238 Âµs, sys: 143 Âµs, total: 381 Âµs    Wall time: 1.76 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'SQLite Cacheâ€‹rm .langchain.db# We can do the same thing with a SQLite cachefrom langchain.cache import SQLiteCachelangchain.llm_cache = SQLiteCache(database_path=".langchain.db")# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")    CPU times: user 17 ms, sys: 9.76 ms, total: 26.7 ms    Wall time: 825 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'# The second time it is, so it goes fasterllm.predict("Tell me a joke")    CPU times: user 2.46


me a joke")    CPU times: user 2.46 ms, sys: 1.23 ms, total: 3.7 ms    Wall time: 2.67 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'PreviousChat modelsNextHuman input Chat ModelCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Streaming | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsChat modelsCachingHuman input Chat ModelLLMChainPromptsStreamingOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsChat modelsStreamingStreamingSome Chat models provide a streaming response. This means that instead of waiting for the entire response to be returned, you can start processing it as soon as it's available. This is useful if you want to display the response to the user as it's being generated, or if you want to process the response as it's being generated.from langchain.chat_models import ChatOpenAIfrom langchain.schema import (    HumanMessage,)from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandlerchat = ChatOpenAI(streaming=True, callbacks=[StreamingStdOutCallbackHandler()], temperature=0)resp = chat([HumanMessage(content="Write me a song about sparkling water.")])    Verse 1:    Bubbles rising to the top    A refreshing drink that never stops    Clear and crisp, it's pure delight    A taste that's sure to excite        Chorus:    Sparkling water, oh so fine    A drink that's always on my mind    With every sip, I feel alive    Sparkling water, you're my vibe        Verse 2:    No sugar, no calories, just pure bliss    A drink that's hard to resist    It's the perfect way to quench my thirst    A drink that always comes first


quench my thirst    A drink that always comes first        Chorus:    Sparkling water, oh so fine    A drink that's always on my mind    With every sip, I feel alive    Sparkling water, you're my vibe        Bridge:    From the mountains to the sea    Sparkling water, you're the key    To a healthy life, a happy soul    A drink that makes me feel whole        Chorus:    Sparkling water, oh so fine    A drink that's always on my mind    With every sip, I feel alive    Sparkling water, you're my vibe        Outro:    Sparkling water, you're the one    A drink that's always so much fun    I'll never let you go, my friend    SparklingPreviousPromptsNextOutput parsersCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Prompts | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsChat modelsCachingHuman input Chat ModelLLMChainPromptsStreamingOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsChat modelsPromptsPromptsPrompts for Chat models are built around messages, instead of just plain text.You can make use of templating by using a MessagePromptTemplate. You can build a ChatPromptTemplate from one or more MessagePromptTemplates. You can use ChatPromptTemplate's format_prompt -- this returns a PromptValue, which you can convert to a string or Message object, depending on whether you want to use the formatted value as input to an llm or chat model.For convenience, there is a from_template method exposed on the template. If you were to use this template, this is what it would look like:from langchain import PromptTemplatefrom langchain.prompts.chat import (    ChatPromptTemplate,    SystemMessagePromptTemplate,    AIMessagePromptTemplate,    HumanMessagePromptTemplate,)template="You are a helpful assistant that translates {input_language} to {output_language}."system_message_prompt = SystemMessagePromptTemplate.from_template(template)human_template="{text}"human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)chat_prompt = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])# get a chat completion from the formatted messageschat(chat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.").to_messages())    AIMessage(content="J'adore la programmation.", additional_kwargs={})If you wanted to construct the MessagePromptTemplate more directly,


la programmation.", additional_kwargs={})If you wanted to construct the MessagePromptTemplate more directly, you could create a PromptTemplate outside and then pass it in, eg:prompt=PromptTemplate(    template="You are a helpful assistant that translates {input_language} to {output_language}.",    input_variables=["input_language", "output_language"],)system_message_prompt = SystemMessagePromptTemplate(prompt=prompt)PreviousLLMChainNextStreamingCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Human input Chat Model | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsChat modelsCachingHuman input Chat ModelLLMChainPromptsStreamingOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsChat modelsHuman input Chat ModelHuman input Chat ModelAlong with HumanInputLLM, LangChain also provides a pseudo Chat Model class that can be used for testing, debugging, or educational purposes. This allows you to mock out calls to the Chat Model and simulate how a human would respond if they received the messages.In this notebook, we go over how to use this.We start this with using the HumanInputChatModel in an agent.from langchain.chat_models.human import HumanInputChatModelSince we will use the WikipediaQueryRun tool in this notebook, you might need to install the wikipedia package if you haven't done so already.%pip install wikipedia    /Users/mskim58/dev/research/chatbot/github/langchain/.venv/bin/python: No module named pip    Note: you may need to restart the kernel to use updated packages.from langchain.agents import load_toolsfrom langchain.agents import initialize_agentfrom langchain.agents import AgentTypetools = load_tools(["wikipedia"])llm = HumanInputChatModel()agent = initialize_agent(    tools, llm, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=True)agent("What is Bocchi the Rock?")            > Entering new  chain...         ======= start of message =======             type: system


message =======             type: system    data:      content: "Answer the following questions as best you can. You have access to the following tools:\n\nWikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, facts, historical events, or other subjects. Input should be a search query.\n\nThe way you use the tools is by specifying a json blob.\nSpecifically, this json should have a `action` key (with the name of the tool to use) and a `action_input` key (with the input to the tool going here).\n\nThe only values that should be in the \"action\" field are: Wikipedia\n\nThe $JSON_BLOB should only contain a SINGLE action, do NOT return a list of multiple actions. Here is an example of a valid $JSON_BLOB:\n\n```\n{\n  \"action\": $TOOL_NAME,\n  \"action_input\": $INPUT\n}\n```\n\nALWAYS use the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction:\n```\n$JSON_BLOB\n```\nObservation: the result of the action\n... (this Thought/Action/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin! Reminder to always use the exact characters `Final Answer` when responding."      additional_kwargs: {}        ======= end of message =======                  ======= start of message =======             type: human    data:


type: human    data:      content: 'What is Bocchi the Rock?                '      additional_kwargs: {}      example: false        ======= end of message =======             Action:    ```    {      "action": "Wikipedia",      "action_input": "What is Bocchi the Rock?"    }    ```    Observation: Page: Bocchi the Rock!    Summary: Bocchi the Rock! (ã�¼ã�£ã�¡ãƒ»ã�–ãƒ»ã‚�ã�£ã��!, Botchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankÅ�bon volumes as of November 2022.    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.        Page: Hitori Bocchi no Marumaru Seikatsu    Summary: Hitori Bocchi no Marumaru Seikatsu (Japanese: ã�²ã�¨ã‚Šã�¼ã�£ã�¡ã�®â—‹â—‹ç”Ÿæ´», lit. "Bocchi Hitori's ____ Life" or "The


lit. "Bocchi Hitori's ____ Life" or "The ____ Life of Being Alone") is a Japanese yonkoma manga series written and illustrated by Katsuwo. It was serialized in ASCII Media Works' Comic Dengeki Daioh "g" magazine from September 2013 to April 2021. Eight tankÅ�bon volumes have been released. An anime television series adaptation by C2C aired from April to June 2019.        Page: Kessoku Band (album)    Summary: Kessoku Band (Japanese: çµ�æ�Ÿãƒ�ãƒ³ãƒ‰, Hepburn: Kessoku Bando) is the debut studio album by Kessoku Band, a fictional musical group from the anime television series Bocchi the Rock!, released digitally on December 25, 2022, and physically on CD on December 28 by Aniplex. Featuring vocals from voice actresses Yoshino Aoyama, Sayumi Suzushiro, Saku Mizuno, and Ikumi Hasegawa, the album consists of 14 tracks previously heard in the anime, including a cover of Asian Kung-Fu Generation's "Rockn' Roll, Morning Light Falls on You", as well as newly recorded songs; nine singles preceded the album's physical release. Commercially, Kessoku Band peaked at number one on the Billboard Japan Hot Albums Chart and Oricon Albums Chart, and was certified gold by the Recording Industry Association of Japan.            Thought:     ======= start of message =======             type: system    data:      content: "Answer the following questions as best you can. You have access to the following tools:\n\nWikipedia: A wrapper around Wikipedia. Useful


You have access to the following tools:\n\nWikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, facts, historical events, or other subjects. Input should be a search query.\n\nThe way you use the tools is by specifying a json blob.\nSpecifically, this json should have a `action` key (with the name of the tool to use) and a `action_input` key (with the input to the tool going here).\n\nThe only values that should be in the \"action\" field are: Wikipedia\n\nThe $JSON_BLOB should only contain a SINGLE action, do NOT return a list of multiple actions. Here is an example of a valid $JSON_BLOB:\n\n```\n{\n  \"action\": $TOOL_NAME,\n  \"action_input\": $INPUT\n}\n```\n\nALWAYS use the following format:\n\nQuestion: the input question you must answer\nThought: you should always think about what to do\nAction:\n```\n$JSON_BLOB\n```\nObservation: the result of the action\n... (this Thought/Action/Observation can repeat N times)\nThought: I now know the final answer\nFinal Answer: the final answer to the original input question\n\nBegin! Reminder to always use the exact characters `Final Answer` when responding."      additional_kwargs: {}        ======= end of message =======                  ======= start of message =======             type: human    data:      content: "What is Bocchi the Rock?\n\nThis was your previous work (but I haven't seen any of it! I only see what you return as final


previous work (but I haven't seen any of it! I only see what you return as final answer):\nAction:\n```\n{\n  \"action\": \"Wikipedia\",\n  \"action_input\": \"What is Bocchi the Rock?\"\n}\n```\nObservation: Page: Bocchi the Rock!\nSummary: Bocchi the Rock! (ã�¼ã�£ã�¡ãƒ»ã�–ãƒ»ã‚�ã�£ã��!, Botchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankÅ�bon volumes as of November 2022.\nAn anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.\n\nPage: Hitori Bocchi no Marumaru Seikatsu\nSummary: Hitori Bocchi no Marumaru Seikatsu (Japanese: ã�²ã�¨ã‚Šã�¼ã�£ã�¡ã�®â—‹â—‹ç”Ÿæ´», lit. \"Bocchi Hitori's ____ Life\" or \"The ____ Life of Being Alone\") is a Japanese yonkoma manga series written and illustrated by Katsuwo. It was serialized in ASCII Media Works' Comic Dengeki Daioh \"g\" magazine from September 2013 to April 2021. Eight tankÅ�bon volumes have been released. An anime television series adaptation by C2C aired from April to June 2019.\n\nPage:


television series adaptation by C2C aired from April to June 2019.\n\nPage: Kessoku Band (album)\nSummary: Kessoku Band (Japanese: çµ�æ�Ÿãƒ�ãƒ³ãƒ‰, Hepburn: Kessoku Bando) is the debut studio album by Kessoku Band, a fictional musical group from the anime television series Bocchi the Rock!, released digitally on December 25, 2022, and physically on CD on December 28 by Aniplex. Featuring vocals from voice actresses Yoshino Aoyama, Sayumi Suzushiro, Saku Mizuno, and Ikumi Hasegawa, the album consists of 14 tracks previously heard in the anime, including a cover of Asian Kung-Fu Generation's \"Rockn' Roll, Morning Light Falls on You\", as well as newly recorded songs; nine singles preceded the album's physical release. Commercially, Kessoku Band peaked at number one on the Billboard Japan Hot Albums Chart and Oricon Albums Chart, and was certified gold by the Recording Industry Association of Japan.\n\n\nThought:"      additional_kwargs: {}      example: false        ======= end of message =======             This finally works.    Final Answer: Bocchi the Rock! is a four-panel manga series and anime television series. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.        > Finished chain.    {'input': 'What is Bocchi the Rock?',     'output': "Bocchi the Rock! is a four-panel manga series and anime television series. The series has been praised for its writing, comedy, characters, and


series and anime television series. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim."}PreviousCachingNextLLMChainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


LLMs | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsLLMsOn this pageLLMsinfoHead to Integrations for documentation on built-in integrations with LLM providers.Large Language Models (LLMs) are a core component of LangChain.


LangChain does not serve its own LLMs, but rather provides a standard interface for interacting with many different LLMs.Get startedâ€‹There are lots of LLM providers (OpenAI, Cohere, Hugging Face, etc) - the LLM class is designed to provide a standard interface for all of them.In this walkthrough we'll work with an OpenAI LLM wrapper, although the functionalities highlighted are generic for all LLM types.Setupâ€‹To start we'll need to install the OpenAI Python package:pip install openaiAccessing the API requires an API key, which you can get by creating an account and heading here. Once we have a key we'll want to set it as an environment variable by running:export OPENAI_API_KEY="..."If you'd prefer not to set an environment variable you can pass the key in directly via the openai_api_key named parameter when initiating the OpenAI LLM class:from langchain.llms import OpenAIllm = OpenAI(openai_api_key="...")otherwise you can initialize without any params:from langchain.llms import OpenAIllm = OpenAI()__call__: string in -> string outâ€‹The simplest way to use an LLM is a callable: pass in a string, get a string completion.llm("Tell me a joke")    'Why did the chicken cross the road?\n\nTo get to the other side.'generate: batch calls, richer outputsâ€‹generate lets you can call the model with a list of strings, getting back a more complete response than just the text. This complete response can includes things like multiple top responses and other LLM provider-specific information:llm_result = llm.generate(["Tell me a joke", "Tell me a poem"]*15)len(llm_result.generations)    30llm_result.generations[0]


30llm_result.generations[0]    [Generation(text='\n\nWhy did the chicken cross the road?\n\nTo get to the other side!'),     Generation(text='\n\nWhy did the chicken cross the road?\n\nTo get to the other side.')]llm_result.generations[-1]    [Generation(text="\n\nWhat if love neverspeech\n\nWhat if love never ended\n\nWhat if love was only a feeling\n\nI'll never know this love\n\nIt's not a feeling\n\nBut it's what we have for each other\n\nWe just know that love is something strong\n\nAnd we can't help but be happy\n\nWe just feel what love is for us\n\nAnd we love each other with all our heart\n\nWe just don't know how\n\nHow it will go\n\nBut we know that love is something strong\n\nAnd we'll always have each other\n\nIn our lives."),     Generation(text='\n\nOnce upon a time\n\nThere was a love so pure and true\n\nIt lasted for centuries\n\nAnd never became stale or dry\n\nIt was moving and alive\n\nAnd the heart of the love-ick\n\nIs still beating strong and true.')]You can also access provider specific information that is returned. This information is NOT standardized across providers.llm_result.llm_output    {'token_usage': {'completion_tokens': 3903,      'total_tokens': 4023,      'prompt_tokens': 120}}PreviousLanguage modelsNextAsync APIGet startedCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Human input LLM | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsLLMsHuman input LLMHuman input LLMSimilar to the fake LLM, LangChain provides a pseudo LLM class that can be used for testing, debugging, or educational purposes. This allows you to mock out calls to the LLM and simulate how a human would respond if they received the prompts.In this notebook, we go over how to use this.We start this with using the HumanInputLLM in an agent.from langchain.llms.human import HumanInputLLMfrom langchain.agents import load_toolsfrom langchain.agents import initialize_agentfrom langchain.agents import AgentTypeSince we will use the WikipediaQueryRun tool in this notebook, you might need to install the wikipedia package if you haven't done so already.%pip install wikipediatools = load_tools(["wikipedia"])llm = HumanInputLLM(    prompt_func=lambda prompt: print(        f"\n===PROMPT====\n{prompt}\n=====END OF PROMPT======"    ))agent = initialize_agent(    tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)agent.run("What is 'Bocchi the Rock!'?")            > Entering new AgentExecutor chain...        ===PROMPT====    Answer the following questions as


===PROMPT====    Answer the following questions as best you can. You have access to the following tools:        Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.        Use the following format:        Question: the input question you must answer    Thought: you should always think about what to do    Action: the action to take, should be one of [Wikipedia]    Action Input: the input to the action    Observation: the result of the action    ... (this Thought/Action/Action Input/Observation can repeat N times)    Thought: I now know the final answer    Final Answer: the final answer to the original input question        Begin!        Question: What is 'Bocchi the Rock!'?    Thought:    =====END OF PROMPT======    I need to use a tool.    Action: Wikipedia    Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.    Observation: Page: Bocchi the Rock!    Summary: Bocchi the Rock! (ã�¼ã�£ã�¡ãƒ»ã�–ãƒ»ã‚�ã�£ã��!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankÅ�bon volumes as of November 2022.


Its chapters have been collected in five tankÅ�bon volumes as of November 2022.    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.        Page: Manga Time Kirara    Summary: Manga Time Kirara (ã�¾ã‚“ã�Œã‚¿ã‚¤ãƒ ã��ã‚‰ã‚‰, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.        Page: Manga Time Kirara Max    Summary: Manga Time Kirara Max (ã�¾ã‚“ã�Œã‚¿ã‚¤ãƒ ã��ã‚‰ã‚‰MAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the "Kirara" series, after "Manga Time Kirara" and "Manga Time Kirara Carat". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.    Thought:    ===PROMPT====    Answer the following questions as best you can. You have access to the following tools:        Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or


Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.        Use the following format:        Question: the input question you must answer    Thought: you should always think about what to do    Action: the action to take, should be one of [Wikipedia]    Action Input: the input to the action    Observation: the result of the action    ... (this Thought/Action/Action Input/Observation can repeat N times)    Thought: I now know the final answer    Final Answer: the final answer to the original input question        Begin!        Question: What is 'Bocchi the Rock!'?    Thought:I need to use a tool.    Action: Wikipedia    Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.    Observation: Page: Bocchi the Rock!    Summary: Bocchi the Rock! (ã�¼ã�£ã�¡ãƒ»ã�–ãƒ»ã‚�ã�£ã��!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankÅ�bon volumes as of November 2022.    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.        Page: Manga


with the anime's visual creativity receiving acclaim.        Page: Manga Time Kirara    Summary: Manga Time Kirara (ã�¾ã‚“ã�Œã‚¿ã‚¤ãƒ ã��ã‚‰ã‚‰, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.        Page: Manga Time Kirara Max    Summary: Manga Time Kirara Max (ã�¾ã‚“ã�Œã‚¿ã‚¤ãƒ ã��ã‚‰ã‚‰MAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the "Kirara" series, after "Manga Time Kirara" and "Manga Time Kirara Carat". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.    Thought:    =====END OF PROMPT======    These are not relevant articles.    Action: Wikipedia    Action Input: Bocchi the Rock!, Japanese four-panel manga series written and illustrated by Aki Hamaji.    Observation: Page: Bocchi the Rock!    Summary: Bocchi the Rock! (ã�¼ã�£ã�¡ãƒ»ã�–ãƒ»ã‚�ã�£ã��!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated


Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankÅ�bon volumes as of November 2022.    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.    Thought:    ===PROMPT====    Answer the following questions as best you can. You have access to the following tools:        Wikipedia: A wrapper around Wikipedia. Useful for when you need to answer general questions about people, places, companies, historical events, or other subjects. Input should be a search query.        Use the following format:        Question: the input question you must answer    Thought: you should always think about what to do    Action: the action to take, should be one of [Wikipedia]    Action Input: the input to the action    Observation: the result of the action    ... (this Thought/Action/Action Input/Observation can repeat N times)    Thought: I now know the final answer    Final Answer: the final answer to the original input question        Begin!        Question: What is 'Bocchi the Rock!'?    Thought:I need to use a tool.    Action: Wikipedia    Action Input: Bocchi the Rock!, Japanese four-panel manga and anime series.    Observation: Page: Bocchi the Rock!


and anime series.    Observation: Page: Bocchi the Rock!    Summary: Bocchi the Rock! (ã�¼ã�£ã�¡ãƒ»ã�–ãƒ»ã‚�ã�£ã��!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankÅ�bon volumes as of November 2022.    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.        Page: Manga Time Kirara    Summary: Manga Time Kirara (ã�¾ã‚“ã�Œã‚¿ã‚¤ãƒ ã��ã‚‰ã‚‰, Manga Taimu Kirara) is a Japanese seinen manga magazine published by Houbunsha which mainly serializes four-panel manga. The magazine is sold on the ninth of each month and was first published as a special edition of Manga Time, another Houbunsha magazine, on May 17, 2002. Characters from this magazine have appeared in a crossover role-playing game called Kirara Fantasia.        Page: Manga Time Kirara Max    Summary: Manga Time Kirara Max (ã�¾ã‚“ã�Œã‚¿ã‚¤ãƒ ã��ã‚‰ã‚‰MAX) is a Japanese four-panel seinen manga magazine published by Houbunsha. It is the third magazine of the "Kirara" series, after "Manga Time


It is the third magazine of the "Kirara" series, after "Manga Time Kirara" and "Manga Time Kirara Carat". The first issue was released on September 29, 2004. Currently the magazine is released on the 19th of each month.    Thought:These are not relevant articles.    Action: Wikipedia    Action Input: Bocchi the Rock!, Japanese four-panel manga series written and illustrated by Aki Hamaji.    Observation: Page: Bocchi the Rock!    Summary: Bocchi the Rock! (ã�¼ã�£ã�¡ãƒ»ã�–ãƒ»ã‚�ã�£ã��!, Bocchi Za Rokku!) is a Japanese four-panel manga series written and illustrated by Aki Hamaji. It has been serialized in Houbunsha's seinen manga magazine Manga Time Kirara Max since December 2017. Its chapters have been collected in five tankÅ�bon volumes as of November 2022.    An anime television series adaptation produced by CloverWorks aired from October to December 2022. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.    Thought:    =====END OF PROMPT======    It worked.    Final Answer: Bocchi the Rock! is a four-panel manga series and anime television series. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim.        > Finished chain.    "Bocchi the Rock! is a four-panel manga series and anime television series. The series has been praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual


praised for its writing, comedy, characters, and depiction of social anxiety, with the anime's visual creativity receiving acclaim."PreviousFake LLMNextCachingCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Streaming | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsLLMsStreamingStreamingSome LLMs provide a streaming response. This means that instead of waiting for the entire response to be returned, you can start processing it as soon as it's available. This is useful if you want to display the response to the user as it's being generated, or if you want to process the response as it's being generated.Currently, we support streaming for a broad range of LLM implementations, including but not limited to OpenAI, ChatOpenAI, ChatAnthropic, Hugging Face Text Generation Inference, and Replicate. This feature has been expanded to accommodate most of the models. To utilize streaming, use a CallbackHandler that implements on_llm_new_token. In this example, we are using StreamingStdOutCallbackHandler.from langchain.llms import OpenAIfrom langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandlerllm = OpenAI(streaming=True, callbacks=[StreamingStdOutCallbackHandler()], temperature=0)resp = llm("Write me a song about sparkling water.")    Verse 1    I'm sippin' on sparkling water,    It's so refreshing and light,    It's the perfect way to quench my thirst    On a hot summer night.        Chorus    Sparkling water, sparkling water,    It's the best way to stay hydrated,


water, sparkling water,    It's the best way to stay hydrated,    It's so crisp and so clean,    It's the perfect way to stay refreshed.        Verse 2    I'm sippin' on sparkling water,    It's so bubbly and bright,    It's the perfect way to cool me down    On a hot summer night.        Chorus    Sparkling water, sparkling water,    It's the best way to stay hydrated,    It's so crisp and so clean,    It's the perfect way to stay refreshed.        Verse 3    I'm sippin' on sparkling water,    It's so light and so clear,    It's the perfect way to keep me cool    On a hot summer night.        Chorus    Sparkling water, sparkling water,    It's the best way to stay hydrated,    It's so crisp and so clean,    It's the perfect way to stay refreshed.We still have access to the end LLMResult if using generate. However, token_usage is not currently supported for streaming.llm.generate(["Tell me a joke."])    Q: What did the fish say when it hit the wall?    A: Dam!    LLMResult(generations=[[Generation(text='\n\nQ: What did the fish say when it hit the wall?\nA: Dam!', generation_info={'finish_reason': 'stop', 'logprobs': None})]], llm_output={'token_usage': {}, 'model_name': 'text-davinci-003'})PreviousSerializationNextTracking token


{}, 'model_name': 'text-davinci-003'})PreviousSerializationNextTracking token usageCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Caching | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsLLMsCachingCachingLangChain provides an optional caching layer for LLMs. This is useful for two reasons:It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times.


It can speed up your application by reducing the number of API calls you make to the LLM provider.import langchainfrom langchain.llms import OpenAI# To make the caching really obvious, lets use a slower model.llm = OpenAI(model_name="text-davinci-002", n=2, best_of=2)In Memory Cacheâ€‹from langchain.cache import InMemoryCachelangchain.llm_cache = InMemoryCache()# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")    CPU times: user 35.9 ms, sys: 28.6 ms, total: 64.6 ms    Wall time: 4.83 s        "\n\nWhy couldn't the bicycle stand up by itself? It was...two tired!"# The second time it is, so it goes fasterllm.predict("Tell me a joke")    CPU times: user 238 Âµs, sys: 143 Âµs, total: 381 Âµs    Wall time: 1.76 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'SQLite Cacheâ€‹rm .langchain.db# We can do the same thing with a SQLite cachefrom langchain.cache import SQLiteCachelangchain.llm_cache = SQLiteCache(database_path=".langchain.db")# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")    CPU times: user 17 ms, sys: 9.76 ms, total: 26.7 ms    Wall time: 825 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'# The


did the chicken cross the road?\n\nTo get to the other side.'# The second time it is, so it goes fasterllm.predict("Tell me a joke")    CPU times: user 2.46 ms, sys: 1.23 ms, total: 3.7 ms    Wall time: 2.67 ms    '\n\nWhy did the chicken cross the road?\n\nTo get to the other side.'Optional Caching in Chainsâ€‹You can also turn off caching for particular nodes in chains. Note that because of certain interfaces, its often easier to construct the chain first, and then edit the LLM afterwards.As an example, we will load a summarizer map-reduce chain. We will cache results for the map-step, but then not freeze it for the combine step.llm = OpenAI(model_name="text-davinci-002")no_cache_llm = OpenAI(model_name="text-davinci-002", cache=False)from langchain.text_splitter import CharacterTextSplitterfrom langchain.chains.mapreduce import MapReduceChaintext_splitter = CharacterTextSplitter()with open('../../../state_of_the_union.txt') as f:    state_of_the_union = f.read()texts = text_splitter.split_text(state_of_the_union)from langchain.docstore.document import Documentdocs = [Document(page_content=t) for t in texts[:3]]from langchain.chains.summarize import load_summarize_chainchain = load_summarize_chain(llm, chain_type="map_reduce", reduce_llm=no_cache_llm)chain.run(docs)    CPU times: user 452 ms, sys: 60.3 ms, total: 512 ms    Wall time: 5.09 s    '\n\nPresident Biden is discussing the American Rescue Plan and the Bipartisan Infrastructure Law, which will


'\n\nPresident Biden is discussing the American Rescue Plan and the Bipartisan Infrastructure Law, which will create jobs and help Americans. He also talks about his vision for America, which includes investing in education and infrastructure. In response to Russian aggression in Ukraine, the United States is joining with European allies to impose sanctions and isolate Russia. American forces are being mobilized to protect NATO countries in the event that Putin decides to keep moving west. The Ukrainians are bravely fighting back, but the next few weeks will be hard for them. Putin will pay a high price for his actions in the long run. Americans should not be alarmed, as the United States is taking action to protect its interests and allies.'When we run it again, we see that it runs substantially faster but the final answer is different. This is due to caching at the map steps, but not at the reduce step.chain.run(docs)    CPU times: user 11.5 ms, sys: 4.33 ms, total: 15.8 ms    Wall time: 1.04 s    '\n\nPresident Biden is discussing the American Rescue Plan and the Bipartisan Infrastructure Law, which will create jobs and help Americans. He also talks about his vision for America, which includes investing in education and infrastructure.'rm .langchain.db sqlite.dbPreviousHuman input LLMNextSerializationCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Async API | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsLLMsAsync APIAsync APILangChain provides async support for LLMs by leveraging the asyncio library.Async support is particularly useful for calling multiple LLMs concurrently, as these calls are network-bound. Currently, OpenAI, PromptLayerOpenAI, ChatOpenAI and Anthropic are supported, but async support for other LLMs is on the roadmap.You can use the agenerate method to call an OpenAI LLM asynchronously.import timeimport asynciofrom langchain.llms import OpenAIdef generate_serially():    llm = OpenAI(temperature=0.9)    for _ in range(10):        resp = llm.generate(["Hello, how are you?"])        print(resp.generations[0][0].text)async def async_generate(llm):    resp = await llm.agenerate(["Hello, how are you?"])    print(resp.generations[0][0].text)async def generate_concurrently():    llm = OpenAI(temperature=0.9)    tasks = [async_generate(llm) for _ in range(10)]    await asyncio.gather(*tasks)s = time.perf_counter()# If running this outside of Jupyter, use asyncio.run(generate_concurrently())await generate_concurrently()elapsed = time.perf_counter() -


asyncio.run(generate_concurrently())await generate_concurrently()elapsed = time.perf_counter() - sprint("\033[1m" + f"Concurrent executed in {elapsed:0.2f} seconds." + "\033[0m")s = time.perf_counter()generate_serially()elapsed = time.perf_counter() - sprint("\033[1m" + f"Serial executed in {elapsed:0.2f} seconds." + "\033[0m")            I'm doing well, thank you. How about you?            I'm doing well, thank you. How about you?            I'm doing well, how about you?            I'm doing well, thank you. How about you?            I'm doing well, thank you. How about you?            I'm doing well, thank you. How about yourself?            I'm doing well, thank you! How about you?            I'm doing well, thank you. How about you?            I'm doing well, thank you! How about you?            I'm doing well, thank you. How about you?    Concurrent executed in 1.39 seconds.            I'm doing well, thank you. How about you?            I'm doing well, thank you. How about you?        I'm doing well, thank you.


How about you?        I'm doing well, thank you. How about you?            I'm doing well, thank you. How about you?            I'm doing well, thank you. How about yourself?            I'm doing well, thanks for asking. How about you?            I'm doing well, thanks! How about you?            I'm doing well, thank you. How about you?            I'm doing well, thank you. How about yourself?            I'm doing well, thanks for asking. How about you?    Serial executed in 5.77 seconds.PreviousLLMsNextCustom LLMCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Custom LLM | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsLLMsCustom LLMCustom LLMThis notebook goes over how to create a custom LLM wrapper, in case you want to use your own LLM or a different wrapper than one that is supported in LangChain.There is only one required thing that a custom LLM needs to implement:A _call method that takes in a string, some optional stop words, and returns a stringThere is a second optional thing it can implement:An _identifying_params property that is used to help with printing of this class. Should return a dictionary.Let's implement a very simple custom LLM that just returns the first N characters of the input.from typing import Any, List, Mapping, Optionalfrom langchain.callbacks.manager import CallbackManagerForLLMRunfrom langchain.llms.base import LLMclass CustomLLM(LLM):    n: int    @property    def _llm_type(self) -> str:        return "custom"    def _call(        self,        prompt: str,        stop: Optional[List[str]] = None,        run_manager: Optional[CallbackManagerForLLMRun] = None,    ) -> str:        if stop is not None:


if stop is not None:            raise ValueError("stop kwargs are not permitted.")        return prompt[: self.n]    @property    def _identifying_params(self) -> Mapping[str, Any]:        """Get the identifying parameters."""        return {"n": self.n}We can now use this as an any other LLM.llm = CustomLLM(n=10)llm("This is a foobar thing")    'This is a 'We can also print the LLM and see its custom print.print(llm)    CustomLLM    Params: {'n': 10}PreviousAsync APINextFake LLMCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Fake LLM | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsLLMsFake LLMFake LLMWe expose a fake LLM class that can be used for testing. This allows you to mock out calls to the LLM and simulate what would happen if the LLM responded in a certain way.In this notebook we go over how to use this.We start this with using the FakeLLM in an agent.from langchain.llms.fake import FakeListLLMfrom langchain.agents import load_toolsfrom langchain.agents import initialize_agentfrom langchain.agents import AgentTypetools = load_tools(["python_repl"])responses = ["Action: Python REPL\nAction Input: print(2 + 2)", "Final Answer: 4"]llm = FakeListLLM(responses=responses)agent = initialize_agent(    tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)agent.run("whats 2 + 2")            > Entering new AgentExecutor chain...    Action: Python REPL    Action Input: print(2 + 2)    Observation: 4        Thought:Final Answer: 4        > Finished chain.    '4'PreviousCustom LLMNextHuman input LLMCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Tracking token usage | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsLLMsTracking token usageTracking token usageThis notebook goes over how to track your token usage for specific calls. It is currently only implemented for the OpenAI API.Let's first look at an extremely simple example of tracking token usage for a single LLM call.from langchain.llms import OpenAIfrom langchain.callbacks import get_openai_callbackllm = OpenAI(model_name="text-davinci-002", n=2, best_of=2)with get_openai_callback() as cb:    result = llm("Tell me a joke")    print(cb)    Tokens Used: 42        Prompt Tokens: 4        Completion Tokens: 38    Successful Requests: 1    Total Cost (USD): $0.00084Anything inside the context manager will get tracked. Here's an example of using it to track multiple calls in sequence.with get_openai_callback() as cb:    result = llm("Tell me a joke")    result2 = llm("Tell me a joke")    print(cb.total_tokens)    91If a chain or agent with multiple steps in it is used, it will track all those steps.from langchain.agents import load_toolsfrom langchain.agents import initialize_agentfrom langchain.agents import AgentTypefrom langchain.llms import


import initialize_agentfrom langchain.agents import AgentTypefrom langchain.llms import OpenAIllm = OpenAI(temperature=0)tools = load_tools(["serpapi", "llm-math"], llm=llm)agent = initialize_agent(    tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)with get_openai_callback() as cb:    response = agent.run(        "Who is Olivia Wilde's boyfriend? What is his current age raised to the 0.23 power?"    )    print(f"Total Tokens: {cb.total_tokens}")    print(f"Prompt Tokens: {cb.prompt_tokens}")    print(f"Completion Tokens: {cb.completion_tokens}")    print(f"Total Cost (USD): ${cb.total_cost}")            > Entering new AgentExecutor chain...     I need to find out who Olivia Wilde's boyfriend is and then calculate his age raised to the 0.23 power.    Action: Search    Action Input: "Olivia Wilde boyfriend"    Observation: Sudeikis and Wilde's relationship ended in November 2020. Wilde was publicly served with court documents regarding child custody while she was presenting Don't Worry Darling at CinemaCon 2022. In January 2021, Wilde began dating singer Harry Styles after meeting during the filming of Don't Worry Darling.    Thought: I need to find out Harry Styles' age.    Action: Search    Action Input: "Harry Styles age"    Observation: 29 years    Thought: I need to calculate 29 raised to the 0.23 power.    Action: Calculator    Action Input: 29^0.23


power.    Action: Calculator    Action Input: 29^0.23    Observation: Answer: 2.169459462491557        Thought: I now know the final answer.    Final Answer: Harry Styles, Olivia Wilde's boyfriend, is 29 years old and his age raised to the 0.23 power is 2.169459462491557.        > Finished chain.    Total Tokens: 1506    Prompt Tokens: 1350    Completion Tokens: 156    Total Cost (USD): $0.03012PreviousStreamingNextChat modelsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Serialization | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsLanguage modelsLLMsAsync APICustom LLMFake LLMHuman input LLMCachingSerializationStreamingTracking token usageChat modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OLanguage modelsLLMsSerializationOn this pageSerializationThis notebook walks through how to write and read an LLM Configuration to and from disk. This is useful if you want to save the configuration for a given LLM (e.g., the provider, the temperature, etc).from langchain.llms import OpenAIfrom langchain.llms.loading import load_llmLoadingâ€‹First, lets go over loading an LLM from disk. LLMs can be saved on disk in two formats: json or yaml. No matter the extension, they are loaded in the same way.cat llm.json    {        "model_name": "text-davinci-003",        "temperature": 0.7,        "max_tokens": 256,        "top_p": 1.0,        "frequency_penalty": 0.0,        "presence_penalty": 0.0,        "n": 1,        "best_of": 1,        "request_timeout": null,        "_type": "openai"    }llm = load_llm("llm.json")cat llm.yaml


}llm = load_llm("llm.json")cat llm.yaml    _type: openai    best_of: 1    frequency_penalty: 0.0    max_tokens: 256    model_name: text-davinci-003    n: 1    presence_penalty: 0.0    request_timeout: null    temperature: 0.7    top_p: 1.0llm = load_llm("llm.yaml")Savingâ€‹If you want to go from an LLM in memory to a serialized version of it, you can do so easily by calling the .save method. Again, this supports both json and yaml.llm.save("llm.json")llm.save("llm.yaml")PreviousCachingNextStreamingLoadingSavingCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Prompts | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPromptsThe new way of programming models is through prompts.
A prompt refers to the input to the model.
This input is often constructed from multiple components.
LangChain provides several classes and functions to make constructing and working with prompts easy.Prompt templates: Parametrize model inputsExample selectors: Dynamically select examples to include in promptsPreviousModel I/ONextPrompt templatesCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Prompt templates | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesOn this pagePrompt templatesLanguage models take text as input - that text is commonly referred to as a prompt.
Typically this is not simply a hardcoded string but rather a combination of a template, some examples, and user input.


LangChain provides several classes and functions to make constructing and working with prompts easy.What is a prompt template?â€‹A prompt template refers to a reproducible way to generate a prompt. It contains a text string ("the template"), that can take in a set of parameters from the end user and generates a prompt.A prompt template can contain:instructions to the language model,a set of few shot examples to help the language model generate a better response,a question to the language model.Here's the simplest example:from langchain import PromptTemplatetemplate = """\You are a naming consultant for new companies.What is a good name for a company that makes {product}?"""prompt = PromptTemplate.from_template(template)prompt.format(product="colorful socks")You are a naming consultant for new companies.What is a good name for a company that makes colorful socks?Create a prompt templateâ€‹You can create simple hardcoded prompts using the PromptTemplate class. Prompt templates can take any number of input variables, and can be formatted to generate a prompt.from langchain import PromptTemplate# An example prompt with no input variablesno_input_prompt = PromptTemplate(input_variables=[], template="Tell me a joke.")no_input_prompt.format()# -> "Tell me a joke."# An example prompt with one input variableone_input_prompt = PromptTemplate(input_variables=["adjective"], template="Tell me a {adjective} joke.")one_input_prompt.format(adjective="funny")# -> "Tell me a funny joke."# An example prompt with multiple input variablesmultiple_input_prompt = PromptTemplate(    input_variables=["adjective", "content"],     template="Tell me a {adjective} joke about {content}.")multiple_input_prompt.format(adjective="funny", content="chickens")# -> "Tell me a funny joke about chickens."If you do not wish to specify input_variables manually, you can also create a PromptTemplate using from_template


you do not wish to specify input_variables manually, you can also create a PromptTemplate using from_template class method. langchain will automatically infer the input_variables based on the template passed.template = "Tell me a {adjective} joke about {content}."prompt_template = PromptTemplate.from_template(template)prompt_template.input_variables# -> ['adjective', 'content']prompt_template.format(adjective="funny", content="chickens")# -> Tell me a funny joke about chickens.You can create custom prompt templates that format the prompt in any way you want. For more information, see Custom Prompt Templates.Chat prompt templateâ€‹Chat Models take a list of chat messages as input - this list commonly referred to as a prompt.


These chat messages differ from raw string (which you would pass into a LLM model) in that every message is associated with a role.For example, in OpenAI Chat Completion API, a chat message can be associated with the AI, human or system role. The model is supposed to follow instruction from system chat message more closely.LangChain provides several prompt templates to make constructing and working with prompts easily. You are encouraged to use these chat related prompt templates instead of PromptTemplate when querying chat models to fully exploit the potential of underlying chat model.from langchain.prompts import (    ChatPromptTemplate,    PromptTemplate,    SystemMessagePromptTemplate,    AIMessagePromptTemplate,    HumanMessagePromptTemplate,)from langchain.schema import (    AIMessage,    HumanMessage,    SystemMessage)To create a message template associated with a role, you use MessagePromptTemplate.For convenience, there is a from_template method exposed on the template. If you were to use this template, this is what it would look like:template="You are a helpful assistant that translates {input_language} to {output_language}."system_message_prompt = SystemMessagePromptTemplate.from_template(template)human_template="{text}"human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)If you wanted to construct the MessagePromptTemplate more directly, you could create a PromptTemplate outside and then pass it in, eg:prompt=PromptTemplate(    template="You are a helpful assistant that translates {input_language} to {output_language}.",    input_variables=["input_language", "output_language"],)system_message_prompt_2 = SystemMessagePromptTemplate(prompt=prompt)assert system_message_prompt == system_message_prompt_2After that, you can build a ChatPromptTemplate from one or more MessagePromptTemplates. You can use ChatPromptTemplate's format_prompt -- this returns a PromptValue, which you can


You can use ChatPromptTemplate's format_prompt -- this returns a PromptValue, which you can convert to a string or Message object, depending on whether you want to use the formatted value as input to an llm or chat model.chat_prompt = ChatPromptTemplate.from_messages([system_message_prompt, human_message_prompt])# get a chat completion from the formatted messageschat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.").to_messages()    [SystemMessage(content='You are a helpful assistant that translates English to French.', additional_kwargs={}),     HumanMessage(content='I love programming.', additional_kwargs={})]PreviousPromptsNextConnecting to a Feature StoreWhat is a prompt template?CommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Connecting to a Feature Store | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreOn this pageConnecting to a Feature StoreFeature stores are a concept from traditional machine learning that make sure data fed into models is up-to-date and relevant. For more on this, see here.This concept is extremely relevant when considering putting LLM applications in production. In order to personalize LLM applications, you may want to combine LLMs with up-to-date information about particular users. Feature stores can be a great way to keep that data fresh, and LangChain provides an easy way to combine that data with LLMs.In this notebook we will show how to connect prompt templates to feature stores. The basic idea is to call a feature store from inside a prompt template to retrieve values that are then formatted into the prompt.Feastâ€‹To start, we will use the popular open source feature store framework Feast.This assumes you have already run the steps in the README around getting started. We will build of off that example in getting started, and create and LLMChain to write a note to a specific driver regarding their up-to-date statistics.Load Feast Storeâ€‹Again, this should be set up according to the instructions in the Feast READMEfrom feast import FeatureStore# You may need to update the path depending on where you stored itfeast_repo_path = "../../../../../my_feature_repo/feature_repo/"store =