where you stored itfeast_repo_path = "../../../../../my_feature_repo/feature_repo/"store = FeatureStore(repo_path=feast_repo_path)Promptsâ€‹Here we will set up a custom FeastPromptTemplate. This prompt template will take in a driver id, look up their stats, and format those stats into a prompt.Note that the input to this prompt template is just driver_id, since that is the only user defined piece (all other variables are looked up inside the prompt template).from langchain.prompts import PromptTemplate, StringPromptTemplatetemplate = """Given the driver's up to date stats, write them note relaying those stats to them.If they have a conversation rate above .5, give them a compliment. Otherwise, make a silly joke about chickens at the end to make them feel betterHere are the drivers stats:Conversation rate: {conv_rate}Acceptance rate: {acc_rate}Average Daily Trips: {avg_daily_trips}Your response:"""prompt = PromptTemplate.from_template(template)class FeastPromptTemplate(StringPromptTemplate):    def format(self, **kwargs) -> str:        driver_id = kwargs.pop("driver_id")        feature_vector = store.get_online_features(            features=[                "driver_hourly_stats:conv_rate",                "driver_hourly_stats:acc_rate",                "driver_hourly_stats:avg_daily_trips",            ],            entity_rows=[{"driver_id": driver_id}],        ).to_dict()


driver_id}],        ).to_dict()        kwargs["conv_rate"] = feature_vector["conv_rate"][0]        kwargs["acc_rate"] = feature_vector["acc_rate"][0]        kwargs["avg_daily_trips"] = feature_vector["avg_daily_trips"][0]        return prompt.format(**kwargs)prompt_template = FeastPromptTemplate(input_variables=["driver_id"])print(prompt_template.format(driver_id=1001))    Given the driver's up to date stats, write them note relaying those stats to them.    If they have a conversation rate above .5, give them a compliment. Otherwise, make a silly joke about chickens at the end to make them feel better        Here are the drivers stats:    Conversation rate: 0.4745151400566101    Acceptance rate: 0.055561766028404236    Average Daily Trips: 936        Your response:Use in a chainâ€‹We can now use this in a chain, successfully creating a chain that achieves personalization backed by a feature storefrom langchain.chat_models import ChatOpenAIfrom langchain.chains import LLMChainchain = LLMChain(llm=ChatOpenAI(), prompt=prompt_template)chain.run(1001)    "Hi there! I wanted to update you on your current stats. Your acceptance rate is 0.055561766028404236 and your average daily trips are 936. While your conversation rate is currently 0.4745151400566101, I have no doubt that with a little extra effort, you'll be able to exceed that .5 mark! Keep up the great work! And remember, even chickens can't always cross the


.5 mark! Keep up the great work! And remember, even chickens can't always cross the road, but they still give it their best shot."Tectonâ€‹Above, we showed how you could use Feast, a popular open source and self-managed feature store, with LangChain. Our examples below will show a similar integration using Tecton. Tecton is a fully managed feature platform built to orchestrate the complete ML feature lifecycle, from transformation to online serving, with enterprise-grade SLAs.Prerequisitesâ€‹Tecton Deployment (sign up at https://tecton.ai)TECTON_API_KEY environment variable set to a valid Service Account keyDefine and Load Featuresâ€‹We will use the user_transaction_counts Feature View from the Tecton tutorial as part of a Feature Service. For simplicity, we are only using a single Feature View; however, more sophisticated applications may require more feature views to retrieve the features needed for its prompt.user_transaction_metrics = FeatureService(    name = "user_transaction_metrics",    features = [user_transaction_counts])The above Feature Service is expected to be applied to a live workspace. For this example, we will be using the "prod" workspace.import tectonworkspace = tecton.get_workspace("prod")feature_service = workspace.get_feature_service("user_transaction_metrics")Promptsâ€‹Here we will set up a custom TectonPromptTemplate. This prompt template will take in a user_id , look up their stats, and format those stats into a prompt.Note that the input to this prompt template is just user_id, since that is the only user defined piece (all other variables are looked up inside the prompt template).from langchain.prompts import PromptTemplate, StringPromptTemplatetemplate = """Given the vendor's up to date transaction stats, write them a note based on the following rules:1. If they had a transaction in the


write them a note based on the following rules:1. If they had a transaction in the last day, write a short congratulations message on their recent sales2. If no transaction in the last day, but they had a transaction in the last 30 days, playfully encourage them to sell more.3. Always add a silly joke about chickens at the endHere are the vendor's stats:Number of Transactions Last Day: {transaction_count_1d}Number of Transactions Last 30 Days: {transaction_count_30d}Your response:"""prompt = PromptTemplate.from_template(template)class TectonPromptTemplate(StringPromptTemplate):    def format(self, **kwargs) -> str:        user_id = kwargs.pop("user_id")        feature_vector = feature_service.get_online_features(            join_keys={"user_id": user_id}        ).to_dict()        kwargs["transaction_count_1d"] = feature_vector[            "user_transaction_counts.transaction_count_1d_1d"        ]        kwargs["transaction_count_30d"] = feature_vector[            "user_transaction_counts.transaction_count_30d_1d"        ]        return prompt.format(**kwargs)prompt_template = TectonPromptTemplate(input_variables=["user_id"])print(prompt_template.format(user_id="user_469998441571"))    Given the vendor's up to date transaction stats, write them a note based on the following rules:        1. If they had a transaction in the last day, write a short congratulations message on their recent sales


If they had a transaction in the last day, write a short congratulations message on their recent sales    2. If no transaction in the last day, but they had a transaction in the last 30 days, playfully encourage them to sell more.    3. Always add a silly joke about chickens at the end        Here are the vendor's stats:    Number of Transactions Last Day: 657    Number of Transactions Last 30 Days: 20326        Your response:Use in a chainâ€‹We can now use this in a chain, successfully creating a chain that achieves personalization backed by the Tecton Feature Platformfrom langchain.chat_models import ChatOpenAIfrom langchain.chains import LLMChainchain = LLMChain(llm=ChatOpenAI(), prompt=prompt_template)chain.run("user_469998441571")    'Wow, congratulations on your recent sales! Your business is really soaring like a chicken on a hot air balloon! Keep up the great work!'Featureformâ€‹Finally, we will use Featureform an open-source and enterprise-grade feature store to run the same example. Featureform allows you to work with your infrastructure like Spark or locally to define your feature transformations.Initialize Featureformâ€‹You can follow in the instructions in the README to initialize your transformations and features in Featureform.import featureform as ffclient = ff.Client(host="demo.featureform.com")Promptsâ€‹Here we will set up a custom FeatureformPromptTemplate. This prompt template will take in the average amount a user pays per transactions.Note that the input to this prompt template is just avg_transaction, since that is the only user defined piece (all other variables are looked up inside the prompt template).from langchain.prompts import PromptTemplate, StringPromptTemplatetemplate = """Given the amount a


langchain.prompts import PromptTemplate, StringPromptTemplatetemplate = """Given the amount a user spends on average per transaction, let them know if they are a high roller. Otherwise, make a silly joke about chickens at the end to make them feel betterHere are the user's stats:Average Amount per Transaction: ${avg_transcation}Your response:"""prompt = PromptTemplate.from_template(template)class FeatureformPromptTemplate(StringPromptTemplate):    def format(self, **kwargs) -> str:        user_id = kwargs.pop("user_id")        fpf = client.features([("avg_transactions", "quickstart")], {"user": user_id})        return prompt.format(**kwargs)prompt_template = FeatureformPrompTemplate(input_variables=["user_id"])print(prompt_template.format(user_id="C1410926"))Use in a chainâ€‹We can now use this in a chain, successfully creating a chain that achieves personalization backed by the Featureform Feature Platformfrom langchain.chat_models import ChatOpenAIfrom langchain.chains import LLMChainchain = LLMChain(llm=ChatOpenAI(), prompt=prompt_template)chain.run("C1410926")PreviousPrompt templatesNextCustom prompt templateFeastLoad Feast StorePromptsUse in a chainTectonPrerequisitesDefine and Load FeaturesPromptsUse in a chainFeatureformInitialize FeatureformPromptsUse in a chainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Format template output | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesFormat template outputFormat template outputThe output of the format method is available as string, list of messages and ChatPromptValueAs string:output = chat_prompt.format(input_language="English", output_language="French", text="I love programming.")output    'System: You are a helpful assistant that translates English to French.\nHuman: I love programming.'# or alternativelyoutput_2 = chat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.").to_string()assert output == output_2As ChatPromptValuechat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.")    ChatPromptValue(messages=[SystemMessage(content='You are a helpful assistant that translates English to French.', additional_kwargs={}), HumanMessage(content='I love programming.', additional_kwargs={})])As list of Message objectschat_prompt.format_prompt(input_language="English", output_language="French", text="I love programming.").to_messages()    [SystemMessage(content='You are a helpful assistant that translates English to French.', additional_kwargs={}),     HumanMessage(content='I love programming.', additional_kwargs={})]PreviousFew shot examples for chat modelsNextTemplate FormatsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Types of MessagePromptTemplate | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesTypes of MessagePromptTemplateTypes of MessagePromptTemplateLangChain provides different types of MessagePromptTemplate. The most commonly used are AIMessagePromptTemplate, SystemMessagePromptTemplate and HumanMessagePromptTemplate, which create an AI message, system message and human message respectively.However, in cases where the chat model supports taking chat message with arbitrary role, you can use ChatMessagePromptTemplate, which allows user to specify the role name.from langchain.prompts import ChatMessagePromptTemplateprompt = "May the {subject} be with you"chat_message_prompt = ChatMessagePromptTemplate.from_template(role="Jedi", template=prompt)chat_message_prompt.format(subject="force")    ChatMessage(content='May the force be with you', additional_kwargs={}, role='Jedi')LangChain also provides MessagesPlaceholder, which gives you full control of what messages to be rendered during formatting. This can be useful when you are uncertain of what role you should be using for your message prompt templates or when you wish to insert a list of messages during formatting.from langchain.prompts import MessagesPlaceholderhuman_prompt = "Summarize our conversation so far in {word_count} words."human_message_template = HumanMessagePromptTemplate.from_template(human_prompt)chat_prompt = ChatPromptTemplate.from_messages([MessagesPlaceholder(variable_name="conversation"), human_message_template])human_message =


ChatPromptTemplate.from_messages([MessagesPlaceholder(variable_name="conversation"), human_message_template])human_message = HumanMessage(content="What is the best way to learn programming?")ai_message = AIMessage(content="""\1. Choose a programming language: Decide on a programming language that you want to learn.2. Start with the basics: Familiarize yourself with the basic programming concepts such as variables, data types and control structures.3. Practice, practice, practice: The best way to learn programming is through hands-on experience\""")chat_prompt.format_prompt(conversation=[human_message, ai_message], word_count="10").to_messages()    [HumanMessage(content='What is the best way to learn programming?', additional_kwargs={}),     AIMessage(content='1. Choose a programming language: Decide on a programming language that you want to learn. \n\n2. Start with the basics: Familiarize yourself with the basic programming concepts such as variables, data types and control structures.\n\n3. Practice, practice, practice: The best way to learn programming is through hands-on experience', additional_kwargs={}),     HumanMessage(content='Summarize our conversation so far in 10 words.', additional_kwargs={})]PreviousTemplate FormatsNextPartial prompt templatesCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Custom prompt template | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesCustom prompt templateOn this pageCustom prompt templateLet's suppose we want the LLM to generate English language explanations of a function given its name. To achieve this task, we will create a custom prompt template that takes in the function name as input, and formats the prompt template to provide the source code of the function.Why are custom prompt templates needed?â€‹LangChain provides a set of default prompt templates that can be used to generate prompts for a variety of tasks. However, there may be cases where the default prompt templates do not meet your needs. For example, you may want to create a prompt template with specific dynamic instructions for your language model. In such cases, you can create a custom prompt template.Take a look at the current set of default prompt templates here.Creating a Custom Prompt Templateâ€‹There are essentially two distinct prompt templates available - string prompt templates and chat prompt templates. String prompt templates provides a simple prompt in string format, while chat prompt templates produces a more structured prompt to be used with a chat API.In this guide, we will create a custom prompt using a string prompt template. To create a custom string prompt template, there are two requirements:It has an input_variables attribute that exposes what input variables the prompt template expects.It exposes a format method that takes in keyword arguments corresponding to the expected input_variables and returns


template expects.It exposes a format method that takes in keyword arguments corresponding to the expected input_variables and returns the formatted prompt.We will create a custom prompt template that takes in the function name as input and formats the prompt to provide the source code of the function. To achieve this, let's first create a function that will return the source code of a function given its name.import inspectdef get_source_code(function_name):    # Get the source code of the function    return inspect.getsource(function_name)Next, we'll create a custom prompt template that takes in the function name as input, and formats the prompt template to provide the source code of the function.from langchain.prompts import StringPromptTemplatefrom pydantic import BaseModel, validatorPROMPT = """\Given the function name and source code, generate an English language explanation of the function.Function Name: {function_name}Source Code:{source_code}Explanation:"""class FunctionExplainerPromptTemplate(StringPromptTemplate, BaseModel):    """A custom prompt template that takes in the function name as input, and formats the prompt template to provide the source code of the function."""    @validator("input_variables")    def validate_input_variables(cls, v):        """Validate that the input variables are correct."""        if len(v) != 1 or "function_name" not in v:            raise ValueError("function_name must be the only input_variable.")        return v    def format(self, **kwargs) -> str:        # Get the source code of the function        source_code = get_source_code(kwargs["function_name"])        # Generate the prompt to be sent to the language model        prompt = PROMPT.format(


to be sent to the language model        prompt = PROMPT.format(            function_name=kwargs["function_name"].__name__, source_code=source_code        )        return prompt    def _prompt_type(self):        return "function-explainer"Use the custom prompt templateâ€‹Now that we have created a custom prompt template, we can use it to generate prompts for our task.fn_explainer = FunctionExplainerPromptTemplate(input_variables=["function_name"])# Generate a prompt for the function "get_source_code"prompt = fn_explainer.format(function_name=get_source_code)print(prompt)    Given the function name and source code, generate an English language explanation of the function.    Function Name: get_source_code    Source Code:    def get_source_code(function_name):        # Get the source code of the function        return inspect.getsource(function_name)        Explanation:    PreviousConnecting to a Feature StoreNextFew-shot prompt templatesWhy are custom prompt templates needed?Creating a Custom Prompt TemplateUse the custom prompt templateCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Validate template | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesValidate templateValidate templateBy default, PromptTemplate will validate the template string by checking whether the input_variables match the variables defined in template. You can disable this behavior by setting validate_template to Falsetemplate = "I am learning langchain because {reason}."prompt_template = PromptTemplate(template=template,                                 input_variables=["reason", "foo"]) # ValueError due to extra variablesprompt_template = PromptTemplate(template=template,                                 input_variables=["reason", "foo"],                                 validate_template=False) # No errorPreviousPrompt PipeliningNextExample selectorsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Serialization | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesSerializationOn this pageSerializationIt is often preferrable to store prompts not as python code but as files. This can make it easy to share, store, and version prompts. This notebook covers how to do that in LangChain, walking through all the different types of prompts and the different serialization options.At a high level, the following design principles are applied to serialization:Both JSON and YAML are supported. We want to support serialization methods that are human readable on disk, and YAML and JSON are two of the most popular methods for that. Note that this rule applies to prompts. For other assets, like Examples, different serialization methods may be supported.We support specifying everything in one file, or storing different components (templates, examples, etc) in different files and referencing them. For some cases, storing everything in file makes the most sense, but for others it is preferrable to split up some of the assets (long templates, large examples, reusable components). LangChain supports both.There is also a single entry point to load prompts from disk, making it easy to load any type of prompt.# All prompts are loaded through the `load_prompt` function.from langchain.prompts import load_promptPromptTemplateâ€‹This section covers examples for loading a PromptTemplate.Loading from YAMLâ€‹This shows an example of loading a PromptTemplate from


PromptTemplate.Loading from YAMLâ€‹This shows an example of loading a PromptTemplate from YAML.cat simple_prompt.yaml    _type: prompt    input_variables:        ["adjective", "content"]    template:         Tell me a {adjective} joke about {content}.prompt = load_prompt("simple_prompt.yaml")print(prompt.format(adjective="funny", content="chickens"))    Tell me a funny joke about chickens.Loading from JSONâ€‹This shows an example of loading a PromptTemplate from JSON.cat simple_prompt.json    {        "_type": "prompt",        "input_variables": ["adjective", "content"],        "template": "Tell me a {adjective} joke about {content}."    }prompt = load_prompt("simple_prompt.json")print(prompt.format(adjective="funny", content="chickens"))Tell me a funny joke about chickens.Loading Template from a Fileâ€‹This shows an example of storing the template in a separate file and then referencing it in the config. Notice that the key changes from template to template_path.cat simple_template.txt    Tell me a {adjective} joke about {content}.cat simple_prompt_with_template_file.json    {        "_type": "prompt",        "input_variables": ["adjective", "content"],        "template_path": "simple_template.txt"    }prompt = load_prompt("simple_prompt_with_template_file.json")print(prompt.format(adjective="funny", content="chickens"))    Tell me a funny joke about chickens.FewShotPromptTemplateâ€‹This section covers examples for


me a funny joke about chickens.FewShotPromptTemplateâ€‹This section covers examples for loading few shot prompt templates.Examplesâ€‹This shows an example of what examples stored as json might look like.cat examples.json    [        {"input": "happy", "output": "sad"},        {"input": "tall", "output": "short"}    ]And here is what the same examples stored as yaml might look like.cat examples.yaml    - input: happy      output: sad    - input: tall      output: shortLoading from YAMLâ€‹This shows an example of loading a few shot example from YAML.cat few_shot_prompt.yaml    _type: few_shot    input_variables:        ["adjective"]    prefix:         Write antonyms for the following words.    example_prompt:        _type: prompt        input_variables:            ["input", "output"]        template:            "Input: {input}\nOutput: {output}"    examples:        examples.json    suffix:        "Input: {adjective}\nOutput:"prompt = load_prompt("few_shot_prompt.yaml")print(prompt.format(adjective="funny"))    Write antonyms for the following words.        Input: happy    Output: sad        Input: tall    Output: short        Input: funny    Output:The same


Output: short        Input: funny    Output:The same would work if you loaded examples from the yaml file.cat few_shot_prompt_yaml_examples.yaml    _type: few_shot    input_variables:        ["adjective"]    prefix:         Write antonyms for the following words.    example_prompt:        _type: prompt        input_variables:            ["input", "output"]        template:            "Input: {input}\nOutput: {output}"    examples:        examples.yaml    suffix:        "Input: {adjective}\nOutput:"prompt = load_prompt("few_shot_prompt_yaml_examples.yaml")print(prompt.format(adjective="funny"))    Write antonyms for the following words.        Input: happy    Output: sad        Input: tall    Output: short        Input: funny    Output:Loading from JSONâ€‹This shows an example of loading a few shot example from JSON.cat few_shot_prompt.json    {        "_type": "few_shot",        "input_variables": ["adjective"],        "prefix": "Write antonyms for the following words.",        "example_prompt": {            "_type": "prompt",            "input_variables": ["input", "output"],


"input_variables": ["input", "output"],            "template": "Input: {input}\nOutput: {output}"        },        "examples": "examples.json",        "suffix": "Input: {adjective}\nOutput:"    }   prompt = load_prompt("few_shot_prompt.json")print(prompt.format(adjective="funny"))    Write antonyms for the following words.        Input: happy    Output: sad        Input: tall    Output: short        Input: funny    Output:Examples in the Configâ€‹This shows an example of referencing the examples directly in the config.cat few_shot_prompt_examples_in.json    {        "_type": "few_shot",        "input_variables": ["adjective"],        "prefix": "Write antonyms for the following words.",        "example_prompt": {            "_type": "prompt",            "input_variables": ["input", "output"],            "template": "Input: {input}\nOutput: {output}"        },        "examples": [            {"input": "happy", "output": "sad"},            {"input": "tall", "output": "short"}        ],


"tall", "output": "short"}        ],        "suffix": "Input: {adjective}\nOutput:"    }   prompt = load_prompt("few_shot_prompt_examples_in.json")print(prompt.format(adjective="funny"))    Write antonyms for the following words.        Input: happy    Output: sad        Input: tall    Output: short        Input: funny    Output:Example Prompt from a Fileâ€‹This shows an example of loading the PromptTemplate that is used to format the examples from a separate file. Note that the key changes from example_prompt to example_prompt_path.cat example_prompt.json    {        "_type": "prompt",        "input_variables": ["input", "output"],        "template": "Input: {input}\nOutput: {output}"     }cat few_shot_prompt_example_prompt.json    {        "_type": "few_shot",        "input_variables": ["adjective"],        "prefix": "Write antonyms for the following words.",        "example_prompt_path": "example_prompt.json",        "examples": "examples.json",        "suffix": "Input: {adjective}\nOutput:"    }   prompt = load_prompt("few_shot_prompt_example_prompt.json")print(prompt.format(adjective="funny"))    Write antonyms for the following words.        Input: happy    Output: sad


Input: happy    Output: sad        Input: tall    Output: short        Input: funny    Output:PromptTempalte with OutputParserâ€‹This shows an example of loading a prompt along with an OutputParser from a file.cat prompt_with_output_parser.json    {        "input_variables": [            "question",            "student_answer"        ],        "output_parser": {            "regex": "(.*?)\\nScore: (.*)",            "output_keys": [                "answer",                "score"            ],            "default_output_key": null,            "_type": "regex_parser"        },        "partial_variables": {},        "template": "Given the following question and student answer, provide a correct answer and score the student answer.\nQuestion: {question}\nStudent Answer: {student_answer}\nCorrect Answer:",        "template_format": "f-string",        "validate_template": true,        "_type": "prompt"    }prompt = load_prompt("prompt_with_output_parser.json")prompt.output_parser.parse(    "George Washington was


"George Washington was born in 1732 and died in 1799.\nScore: 1/2")    {'answer': 'George Washington was born in 1732 and died in 1799.',     'score': '1/2'}PreviousCompositionNextPrompt PipeliningPromptTemplateLoading from YAMLLoading from JSONLoading Template from a FileFewShotPromptTemplateExamplesLoading from YAMLLoading from JSONExamples in the ConfigExample Prompt from a FilePromptTempalte with OutputParserCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Partial prompt templates | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesPartial prompt templatesPartial prompt templatesLike other methods, it can make sense to "partial" a prompt template - eg pass in a subset of the required values, as to create a new prompt template which expects only the remaining subset of values.LangChain supports this in two ways:Partial formatting with string values.Partial formatting with functions that return string values.These two different ways support different use cases. In the examples below, we go over the motivations for both use cases as well as how to do it in LangChain.Partial With Stringsâ€‹One common use case for wanting to partial a prompt template is if you get some of the variables before others. For example, suppose you have a prompt template that requires two variables, foo and baz. If you get the foo value early on in the chain, but the baz value later, it can be annoying to wait until you have both variables in the same place to pass them to the prompt template. Instead, you can partial the prompt template with the foo value, and then pass the partialed prompt template along and just use that. Below is an example of doing this:from langchain.prompts import PromptTemplateprompt = PromptTemplate(template="{foo}{bar}", input_variables=["foo", "bar"])partial_prompt = prompt.partial(foo="foo");print(partial_prompt.format(bar="baz"))


= prompt.partial(foo="foo");print(partial_prompt.format(bar="baz"))    foobazYou can also just initialize the prompt with the partialed variables.prompt = PromptTemplate(template="{foo}{bar}", input_variables=["bar"], partial_variables={"foo": "foo"})print(prompt.format(bar="baz"))    foobazPartial With Functionsâ€‹The other common use is to partial with a function. The use case for this is when you have a variable you know that you always want to fetch in a common way. A prime example of this is with date or time. Imagine you have a prompt which you always want to have the current date. You can't hard code it in the prompt, and passing it along with the other input variables is a bit annoying. In this case, it's very handy to be able to partial the prompt with a function that always returns the current date.from datetime import datetimedef _get_datetime():    now = datetime.now()    return now.strftime("%m/%d/%Y, %H:%M:%S")prompt = PromptTemplate(    template="Tell me a {adjective} joke about the day {date}",     input_variables=["adjective", "date"]);partial_prompt = prompt.partial(date=_get_datetime)print(partial_prompt.format(adjective="funny"))    Tell me a funny joke about the day 02/27/2023, 22:15:16You can also just initialize the prompt with the partialed variables, which often makes more sense in this workflow.prompt = PromptTemplate(    template="Tell me a {adjective} joke about the day {date}",     input_variables=["adjective"],    partial_variables={"date": _get_datetime});print(prompt.format(adjective="funny"))    Tell me a funny joke about the day


Tell me a funny joke about the day 02/27/2023, 22:15:16PreviousTypes of MessagePromptTemplateNextCompositionCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Page Not Found | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKPage Not FoundWe could not find what you were looking for.Please contact the owner of the site that linked you to the original URL and let them know their link is broken.CommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Few shot examples for chat models | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesFew shot examples for chat modelsOn this pageFew shot examples for chat modelsThis notebook covers how to use few shot examples in chat models.There does not appear to be solid consensus on how best to do few shot prompting. As a result, we are not solidifying any abstractions around this yet but rather using existing abstractions.Alternating Human/AI messagesâ€‹The first way of doing few shot prompting relies on using alternating human/ai messages. See an example of this below.from langchain.chat_models import ChatOpenAIfrom langchain import PromptTemplate, LLMChainfrom langchain.prompts.chat import (    ChatPromptTemplate,    SystemMessagePromptTemplate,    AIMessagePromptTemplate,    HumanMessagePromptTemplate,)from langchain.schema import AIMessage, HumanMessage, SystemMessagechat = ChatOpenAI(temperature=0)template = "You are a helpful assistant that translates english to pirate."system_message_prompt = SystemMessagePromptTemplate.from_template(template)example_human = HumanMessagePromptTemplate.from_template("Hi")example_ai = AIMessagePromptTemplate.from_template("Argh me mateys")human_template = "{text}"human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)chat_prompt = ChatPromptTemplate.from_messages(    [system_message_prompt,


= ChatPromptTemplate.from_messages(    [system_message_prompt, example_human, example_ai, human_message_prompt])chain = LLMChain(llm=chat, prompt=chat_prompt)# get a chat completion from the formatted messageschain.run("I love programming.")    "I be lovin' programmin', me hearty!"System Messagesâ€‹OpenAI provides an optional name parameter that they also recommend using in conjunction with system messages to do few shot prompting. Here is an example of how to do that below.template = "You are a helpful assistant that translates english to pirate."system_message_prompt = SystemMessagePromptTemplate.from_template(template)example_human = SystemMessagePromptTemplate.from_template(    "Hi", additional_kwargs={"name": "example_user"})example_ai = SystemMessagePromptTemplate.from_template(    "Argh me mateys", additional_kwargs={"name": "example_assistant"})human_template = "{text}"human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)chat_prompt = ChatPromptTemplate.from_messages(    [system_message_prompt, example_human, example_ai, human_message_prompt])chain = LLMChain(llm=chat, prompt=chat_prompt)# get a chat completion from the formatted messageschain.run("I love programming.")    "I be lovin' programmin', me hearty."PreviousFew-shot prompt templatesNextFormat template outputAlternating Human/AI messagesSystem MessagesCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Composition | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesCompositionCompositionThis notebook goes over how to compose multiple prompts together. This can be useful when you want to reuse parts of prompts. This can be done with a PipelinePrompt. A PipelinePrompt consists of two main parts:Final prompt: This is the final prompt that is returnedPipeline prompts: This is a list of tuples, consisting of a string name and a prompt template. Each prompt template will be formatted and then passed to future prompt templates as a variable with the same name.from langchain.prompts.pipeline import PipelinePromptTemplatefrom langchain.prompts.prompt import PromptTemplatefull_template = """{introduction}{example}{start}"""full_prompt = PromptTemplate.from_template(full_template)introduction_template = """You are impersonating {person}."""introduction_prompt = PromptTemplate.from_template(introduction_template)example_template = """Here's an example of an interaction: Q: {example_q}A: {example_a}"""example_prompt = PromptTemplate.from_template(example_template)start_template = """Now, do this for real!Q: {input}A:"""start_prompt = PromptTemplate.from_template(start_template)input_prompts = [    ("introduction", introduction_prompt),    ("example", example_prompt),    ("start", start_prompt)]pipeline_prompt = PipelinePromptTemplate(final_prompt=full_prompt,


("start", start_prompt)]pipeline_prompt = PipelinePromptTemplate(final_prompt=full_prompt, pipeline_prompts=input_prompts)pipeline_prompt.input_variables    ['example_a', 'person', 'example_q', 'input']print(pipeline_prompt.format(    person="Elon Musk",    example_q="What's your favorite car?",    example_a="Tesla",    input="What's your favorite social media site?"))    You are impersonating Elon Musk.    Here's an example of an interaction:         Q: What's your favorite car?    A: Tesla    Now, do this for real!        Q: What's your favorite social media site?    A:    PreviousPartial prompt templatesNextSerializationCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Prompt Pipelining | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesPrompt PipeliningOn this pagePrompt PipeliningThe idea behind prompt pipelining is to expose a user friendly interface for composing different parts of prompts together. You can do this with either string prompts or chat prompts. Constructing prompts this way allows for easy reuse of components.String Prompt Pipeliningâ€‹When working with string prompts, each template is joined togther. You can work with either prompts directly or strings (the first element in the list needs to be a prompt).from langchain.prompts import PromptTemplate    /Users/harrisonchase/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/deeplake/util/check_latest_version.py:32: UserWarning: A newer version of deeplake (3.6.12) is available. It's recommended that you update to the latest version using `pip install -U deeplake`.      warnings.warn(prompt = (    PromptTemplate.from_template("Tell me a joke about {topic}")    + ", make it funny"    + "\n\nand in {language}")prompt    PromptTemplate(input_variables=['language', 'topic'], output_parser=None, partial_variables={}, template='Tell me a joke about {topic}, make it


output_parser=None, partial_variables={}, template='Tell me a joke about {topic}, make it funny\n\nand in {language}', template_format='f-string', validate_template=True)prompt.format(topic="sports", language="spanish")    'Tell me a joke about sports, make it funny\n\nand in spanish'You can also use it in an LLMChain, just like before.from langchain.chat_models import ChatOpenAIfrom langchain.chains import LLMChainmodel = ChatOpenAI()chain = LLMChain(llm=model, prompt=prompt)chain.run(topic="sports", language="spanish")    'Â¿Por quÃ© el futbolista llevaba un paraguas al partido?\n\nPorque pronosticaban lluvia de goles.'Chat Prompt Pipeliningâ€‹A chat prompt is made up a of a list of messages. Purely for developer experience, we've added a convinient way to create these prompts. In this pipeline, each new element is a new message in the final prompt.from langchain.prompts import ChatPromptTemplate, HumanMessagePromptTemplatefrom langchain.schema import HumanMessage, AIMessage, SystemMessage    /Users/harrisonchase/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/deeplake/util/check_latest_version.py:32: UserWarning: A newer version of deeplake (3.6.10) is available. It's recommended that you update to the latest version using `pip install -U deeplake`.      warnings.warn(First, let's initialize the base ChatPromptTemplate with a system message. It doesn't have to start with a system, but it's often good practiceprompt = SystemMessage(content="You are a nice pirate")You can then easily create a pipeline combining it with other messages OR message templates.


Use a Message when there is no variables to be formatted, use a MessageTemplate when there are variables to be formatted. You can also use just a string -> note that this will automatically get inferred as a HumanMessagePromptTemplate.new_prompt = (    prompt    + HumanMessage(content="hi")    + AIMessage(content="what?")    + "{input}")Under the hood, this creates an instance of the ChatPromptTemplate class, so you can use it just as you did before!new_prompt.format_messages(input="i said hi")    [SystemMessage(content='You are a nice pirate', additional_kwargs={}),     HumanMessage(content='hi', additional_kwargs={}, example=False),     AIMessage(content='what?', additional_kwargs={}, example=False),     HumanMessage(content='i said hi', additional_kwargs={}, example=False)]You can also use it in an LLMChain, just like beforefrom langchain.chat_models import ChatOpenAIfrom langchain.chains import LLMChainmodel = ChatOpenAI()chain = LLMChain(llm=model, prompt=new_prompt)chain.run("i said hi")    'Oh, hello! How can I assist you today?'PreviousSerializationNextValidate templateString Prompt PipeliningChat Prompt PipeliningCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Few-shot prompt templates | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesFew-shot prompt templatesFew-shot prompt templatesIn this tutorial, we'll learn how to create a prompt template that uses few shot examples. A few shot prompt template can be constructed from either a set of examples, or from an Example Selector object.Use Caseâ€‹In this tutorial, we'll configure few shot examples for self-ask with search.Using an example setâ€‹Create the example setâ€‹To get started, create a list of few shot examples. Each example should be a dictionary with the keys being the input variables and the values being the values for those input variables.from langchain.prompts.few_shot import FewShotPromptTemplatefrom langchain.prompts.prompt import PromptTemplateexamples = [  {    "question": "Who lived longer, Muhammad Ali or Alan Turing?",    "answer": """Are follow up questions needed here: Yes.Follow up: How old was Muhammad Ali when he died?Intermediate answer: Muhammad Ali was 74 years old when he died.Follow up: How old was Alan Turing when he died?Intermediate answer: Alan Turing was 41 years old when he died.So the final answer is: Muhammad Ali"""  },  {    "question": "When was the founder of craigslist born?",    "answer": """Are follow up questions


was the founder of craigslist born?",    "answer": """Are follow up questions needed here: Yes.Follow up: Who was the founder of craigslist?Intermediate answer: Craigslist was founded by Craig Newmark.Follow up: When was Craig Newmark born?Intermediate answer: Craig Newmark was born on December 6, 1952.So the final answer is: December 6, 1952"""  },  {    "question": "Who was the maternal grandfather of George Washington?",    "answer":"""Are follow up questions needed here: Yes.Follow up: Who was the mother of George Washington?Intermediate answer: The mother of George Washington was Mary Ball Washington.Follow up: Who was the father of Mary Ball Washington?Intermediate answer: The father of Mary Ball Washington was Joseph Ball.So the final answer is: Joseph Ball"""  },  {    "question": "Are both the directors of Jaws and Casino Royale from the same country?",    "answer":"""Are follow up questions needed here: Yes.Follow up: Who is the director of Jaws?Intermediate Answer: The director of Jaws is Steven Spielberg.Follow up: Where is Steven Spielberg from?Intermediate Answer: The United States.Follow up: Who is the director of Casino Royale?Intermediate Answer: The director of Casino Royale is Martin Campbell.Follow up: Where is Martin Campbell from?Intermediate Answer: New Zealand.So the final answer is: No"""  }]Create a formatter for the few shot examplesâ€‹Configure a formatter that will format the few shot examples into a string. This formatter should be a PromptTemplate object.example_prompt = PromptTemplate(input_variables=["question", "answer"], template="Question: {question}\n{answer}")print(example_prompt.format(**examples[0]))    Question: Who lived longer, Muhammad Ali or Alan Turing?        Are follow up questions


lived longer, Muhammad Ali or Alan Turing?        Are follow up questions needed here: Yes.    Follow up: How old was Muhammad Ali when he died?    Intermediate answer: Muhammad Ali was 74 years old when he died.    Follow up: How old was Alan Turing when he died?    Intermediate answer: Alan Turing was 41 years old when he died.    So the final answer is: Muhammad Ali    Feed examples and formatter to FewShotPromptTemplateâ€‹Finally, create a FewShotPromptTemplate object. This object takes in the few shot examples and the formatter for the few shot examples.prompt = FewShotPromptTemplate(    examples=examples,     example_prompt=example_prompt,     suffix="Question: {input}",     input_variables=["input"])print(prompt.format(input="Who was the father of Mary Ball Washington?"))    Question: Who lived longer, Muhammad Ali or Alan Turing?        Are follow up questions needed here: Yes.    Follow up: How old was Muhammad Ali when he died?    Intermediate answer: Muhammad Ali was 74 years old when he died.    Follow up: How old was Alan Turing when he died?    Intermediate answer: Alan Turing was 41 years old when he died.    So the final answer is: Muhammad Ali            Question: When was the founder of craigslist born?        Are follow up questions needed here: Yes.    Follow up: Who was the founder of craigslist?    Intermediate answer: Craigslist was founded by Craig Newmark.    Follow up: When was Craig Newmark born?    Intermediate answer: Craig Newmark was born on December


When was Craig Newmark born?    Intermediate answer: Craig Newmark was born on December 6, 1952.    So the final answer is: December 6, 1952            Question: Who was the maternal grandfather of George Washington?        Are follow up questions needed here: Yes.    Follow up: Who was the mother of George Washington?    Intermediate answer: The mother of George Washington was Mary Ball Washington.    Follow up: Who was the father of Mary Ball Washington?    Intermediate answer: The father of Mary Ball Washington was Joseph Ball.    So the final answer is: Joseph Ball            Question: Are both the directors of Jaws and Casino Royale from the same country?        Are follow up questions needed here: Yes.    Follow up: Who is the director of Jaws?    Intermediate Answer: The director of Jaws is Steven Spielberg.    Follow up: Where is Steven Spielberg from?    Intermediate Answer: The United States.    Follow up: Who is the director of Casino Royale?    Intermediate Answer: The director of Casino Royale is Martin Campbell.    Follow up: Where is Martin Campbell from?    Intermediate Answer: New Zealand.    So the final answer is: No            Question: Who was the father of Mary Ball Washington?Using an example selectorâ€‹Feed examples into ExampleSelectorâ€‹We will reuse the example set and the formatter from the previous section. However, instead of feeding the examples directly into the FewShotPromptTemplate object, we will feed them into an ExampleSelector object.In this tutorial, we will use the SemanticSimilarityExampleSelector


feed them into an ExampleSelector object.In this tutorial, we will use the SemanticSimilarityExampleSelector class. This class selects few shot examples based on their similarity to the input. It uses an embedding model to compute the similarity between the input and the few shot examples, as well as a vector store to perform the nearest neighbor search.from langchain.prompts.example_selector import SemanticSimilarityExampleSelectorfrom langchain.vectorstores import Chromafrom langchain.embeddings import OpenAIEmbeddingsexample_selector = SemanticSimilarityExampleSelector.from_examples(    # This is the list of examples available to select from.    examples,    # This is the embedding class used to produce embeddings which are used to measure semantic similarity.    OpenAIEmbeddings(),    # This is the VectorStore class that is used to store the embeddings and do a similarity search over.    Chroma,    # This is the number of examples to produce.    k=1)# Select the most similar example to the input.question = "Who was the father of Mary Ball Washington?"selected_examples = example_selector.select_examples({"question": question})print(f"Examples most similar to the input: {question}")for example in selected_examples:    print("\n")    for k, v in example.items():        print(f"{k}: {v}")    Running Chroma using direct local API.    Using DuckDB in-memory for database. Data will be transient.    Examples most similar to the input: Who was the father of Mary Ball Washington?            question: Who was the maternal grandfather of George Washington?    answer:     Are follow up questions needed here: Yes.    Follow up: Who was the mother of George Washington?    Intermediate answer: The mother


Follow up: Who was the mother of George Washington?    Intermediate answer: The mother of George Washington was Mary Ball Washington.    Follow up: Who was the father of Mary Ball Washington?    Intermediate answer: The father of Mary Ball Washington was Joseph Ball.    So the final answer is: Joseph Ball    Feed example selector into FewShotPromptTemplateâ€‹Finally, create a FewShotPromptTemplate object. This object takes in the example selector and the formatter for the few shot examples.prompt = FewShotPromptTemplate(    example_selector=example_selector,     example_prompt=example_prompt,     suffix="Question: {input}",     input_variables=["input"])print(prompt.format(input="Who was the father of Mary Ball Washington?"))    Question: Who was the maternal grandfather of George Washington?        Are follow up questions needed here: Yes.    Follow up: Who was the mother of George Washington?    Intermediate answer: The mother of George Washington was Mary Ball Washington.    Follow up: Who was the father of Mary Ball Washington?    Intermediate answer: The father of Mary Ball Washington was Joseph Ball.    So the final answer is: Joseph Ball            Question: Who was the father of Mary Ball Washington?PreviousCustom prompt templateNextFew shot examples for chat modelsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Template Formats | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesConnecting to a Feature StoreCustom prompt templateFew-shot prompt templatesFew shot examples for chat modelsFormat template outputTemplate FormatsTypes of MessagePromptTemplatePartial prompt templatesCompositionSerializationPrompt PipeliningValidate templateExample selectorsLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsPrompt templatesTemplate FormatsTemplate FormatsPromptTemplate by default uses Python f-string as its template format. However, it can also use other formats like jinja2, specified through the template_format argument.To use the jinja2 template:from langchain.prompts import PromptTemplatejinja2_template = "Tell me a {{ adjective }} joke about {{ content }}"prompt = PromptTemplate.from_template(jinja2_template, template_format="jinja2")prompt.format(adjective="funny", content="chickens")# Output: Tell me a funny joke about chickens.To use the Python f-string template:from langchain.prompts import PromptTemplatefstring_template = """Tell me a {adjective} joke about {content}"""prompt = PromptTemplate.from_template(fstring_template)prompt.format(adjective="funny", content="chickens")# Output: Tell me a funny joke about chickens.Currently, only jinja2 and f-string are supported. For other formats, kindly raise an issue on the Github page.PreviousFormat template outputNextTypes of MessagePromptTemplateCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Example selectors | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesExample selectorsCustom example selectorSelect by lengthSelect by maximal marginal relevance (MMR)Select by n-gram overlapSelect by similarityLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsExample selectorsExample selectorsIf you have a large number of examples, you may need to select which ones to include in the prompt. The Example Selector is the class responsible for doing so.The base interface is defined as below:class BaseExampleSelector(ABC):    """Interface for selecting examples to include in prompts."""    @abstractmethod    def select_examples(self, input_variables: Dict[str, str]) -> List[dict]:        """Select which examples to use based on the inputs."""The only method it needs to expose is a select_examples method. This takes in the input variables and then returns a list of examples. It is up to each specific implementation as to how those examples are selected. Let's take a look at some below.PreviousValidate templateNextCustom example selectorCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Select by length | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesExample selectorsCustom example selectorSelect by lengthSelect by maximal marginal relevance (MMR)Select by n-gram overlapSelect by similarityLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsExample selectorsSelect by lengthSelect by lengthThis example selector selects which examples to use based on length. This is useful when you are worried about constructing a prompt that will go over the length of the context window. For longer inputs, it will select fewer examples to include, while for shorter inputs it will select more.from langchain.prompts import PromptTemplatefrom langchain.prompts import FewShotPromptTemplatefrom langchain.prompts.example_selector import LengthBasedExampleSelector# These are a lot of examples of a pretend task of creating antonyms.examples = [    {"input": "happy", "output": "sad"},    {"input": "tall", "output": "short"},    {"input": "energetic", "output": "lethargic"},    {"input": "sunny", "output": "gloomy"},    {"input": "windy", "output": "calm"},example_prompt = PromptTemplate(    input_variables=["input", "output"],    template="Input: {input}\nOutput: {output}",)example_selector = LengthBasedExampleSelector(    # These are the examples it has available to choose from.    examples=examples,     # This is the PromptTemplate being used to format the examples.    example_prompt=example_prompt,     #


used to format the examples.    example_prompt=example_prompt,     # This is the maximum length that the formatted examples should be.    # Length is measured by the get_text_length function below.    max_length=25,    # This is the function used to get the length of a string, which is used    # to determine which examples to include. It is commented out because    # it is provided as a default value if none is specified.    # get_text_length: Callable[[str], int] = lambda x: len(re.split("\n| ", x)))dynamic_prompt = FewShotPromptTemplate(    # We provide an ExampleSelector instead of examples.    example_selector=example_selector,    example_prompt=example_prompt,    prefix="Give the antonym of every input",    suffix="Input: {adjective}\nOutput:",     input_variables=["adjective"],)# An example with small input, so it selects all examples.print(dynamic_prompt.format(adjective="big"))    Give the antonym of every input        Input: happy    Output: sad        Input: tall    Output: short        Input: energetic    Output: lethargic        Input: sunny    Output: gloomy        Input: windy    Output: calm        Input: big    Output:# An example with long input, so it selects only one example.long_string = "big and huge and massive and large and gigantic and tall and much much much much much bigger than everything else"print(dynamic_prompt.format(adjective=long_string))    Give the antonym of every input


Give the antonym of every input        Input: happy    Output: sad        Input: big and huge and massive and large and gigantic and tall and much much much much much bigger than everything else    Output:# You can add an example to an example selector as well.new_example = {"input": "big", "output": "small"}dynamic_prompt.example_selector.add_example(new_example)print(dynamic_prompt.format(adjective="enthusiastic"))    Give the antonym of every input        Input: happy    Output: sad        Input: tall    Output: short        Input: energetic    Output: lethargic        Input: sunny    Output: gloomy        Input: windy    Output: calm        Input: big    Output: small        Input: enthusiastic    Output:PreviousCustom example selectorNextSelect by maximal marginal relevance (MMR)CommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Select by maximal marginal relevance (MMR) | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesExample selectorsCustom example selectorSelect by lengthSelect by maximal marginal relevance (MMR)Select by n-gram overlapSelect by similarityLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsExample selectorsSelect by maximal marginal relevance (MMR)Select by maximal marginal relevance (MMR)The MaxMarginalRelevanceExampleSelector selects examples based on a combination of which examples are most similar to the inputs, while also optimizing for diversity. It does this by finding the examples with the embeddings that have the greatest cosine similarity with the inputs, and then iteratively adding them while penalizing them for closeness to already selected examples.from langchain.prompts.example_selector import (    MaxMarginalRelevanceExampleSelector,    SemanticSimilarityExampleSelector,)from langchain.vectorstores import FAISSfrom langchain.embeddings import OpenAIEmbeddingsfrom langchain.prompts import FewShotPromptTemplate, PromptTemplateexample_prompt = PromptTemplate(    input_variables=["input", "output"],    template="Input: {input}\nOutput: {output}",)# These are a lot of examples of a pretend task of creating antonyms.examples = [    {"input": "happy", "output": "sad"},    {"input": "tall", "output": "short"},    {"input": "energetic", "output": "lethargic"},    {"input": "sunny", "output": "gloomy"},    {"input": "windy", "output": "calm"},]example_selector =


{"input": "windy", "output": "calm"},]example_selector = MaxMarginalRelevanceExampleSelector.from_examples(    # This is the list of examples available to select from.    examples,    # This is the embedding class used to produce embeddings which are used to measure semantic similarity.    OpenAIEmbeddings(),    # This is the VectorStore class that is used to store the embeddings and do a similarity search over.    FAISS,    # This is the number of examples to produce.    k=2,)mmr_prompt = FewShotPromptTemplate(    # We provide an ExampleSelector instead of examples.    example_selector=example_selector,    example_prompt=example_prompt,    prefix="Give the antonym of every input",    suffix="Input: {adjective}\nOutput:",    input_variables=["adjective"],)# Input is a feeling, so should select the happy/sad example as the first oneprint(mmr_prompt.format(adjective="worried"))    Give the antonym of every input        Input: happy    Output: sad        Input: windy    Output: calm        Input: worried    Output:# Let's compare this to what we would just get if we went solely off of similarity,# by using SemanticSimilarityExampleSelector instead of MaxMarginalRelevanceExampleSelector.example_selector = SemanticSimilarityExampleSelector.from_examples(    # This is the list of examples available to select from.    examples,    # This is the embedding class used to produce embeddings which are used to measure semantic similarity.    OpenAIEmbeddings(),    # This is the VectorStore class that is used to store


OpenAIEmbeddings(),    # This is the VectorStore class that is used to store the embeddings and do a similarity search over.    FAISS,    # This is the number of examples to produce.    k=2,)similar_prompt = FewShotPromptTemplate(    # We provide an ExampleSelector instead of examples.    example_selector=example_selector,    example_prompt=example_prompt,    prefix="Give the antonym of every input",    suffix="Input: {adjective}\nOutput:",    input_variables=["adjective"],)print(similar_prompt.format(adjective="worried"))    Give the antonym of every input        Input: happy    Output: sad        Input: sunny    Output: gloomy        Input: worried    Output:PreviousSelect by lengthNextSelect by n-gram overlapCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Select by similarity | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesExample selectorsCustom example selectorSelect by lengthSelect by maximal marginal relevance (MMR)Select by n-gram overlapSelect by similarityLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsExample selectorsSelect by similaritySelect by similarityThis object selects examples based on similarity to the inputs. It does this by finding the examples with the embeddings that have the greatest cosine similarity with the inputs.from langchain.prompts.example_selector import SemanticSimilarityExampleSelectorfrom langchain.vectorstores import Chromafrom langchain.embeddings import OpenAIEmbeddingsfrom langchain.prompts import FewShotPromptTemplate, PromptTemplateexample_prompt = PromptTemplate(    input_variables=["input", "output"],    template="Input: {input}\nOutput: {output}",)# These are a lot of examples of a pretend task of creating antonyms.examples = [    {"input": "happy", "output": "sad"},    {"input": "tall", "output": "short"},    {"input": "energetic", "output": "lethargic"},    {"input": "sunny", "output": "gloomy"},    {"input": "windy", "output": "calm"},]example_selector = SemanticSimilarityExampleSelector.from_examples(    # This is the list of examples available to select from.    examples,     # This is the embedding class used to produce embeddings which are used to measure semantic similarity.    OpenAIEmbeddings(),     # This is the VectorStore


similarity.    OpenAIEmbeddings(),     # This is the VectorStore class that is used to store the embeddings and do a similarity search over.    Chroma,     # This is the number of examples to produce.    k=1)similar_prompt = FewShotPromptTemplate(    # We provide an ExampleSelector instead of examples.    example_selector=example_selector,    example_prompt=example_prompt,    prefix="Give the antonym of every input",    suffix="Input: {adjective}\nOutput:",     input_variables=["adjective"],)    Running Chroma using direct local API.    Using DuckDB in-memory for database. Data will be transient.# Input is a feeling, so should select the happy/sad exampleprint(similar_prompt.format(adjective="worried"))    Give the antonym of every input        Input: happy    Output: sad        Input: worried    Output:# Input is a measurement, so should select the tall/short exampleprint(similar_prompt.format(adjective="fat"))    Give the antonym of every input        Input: happy    Output: sad        Input: fat    Output:# You can add new examples to the SemanticSimilarityExampleSelector as wellsimilar_prompt.example_selector.add_example({"input": "enthusiastic", "output": "apathetic"})print(similar_prompt.format(adjective="joyful"))    Give the antonym of every input        Input: happy    Output: sad        Input: joyful    Output:PreviousSelect by n-gram overlapNextLanguage


Input: joyful    Output:PreviousSelect by n-gram overlapNextLanguage modelsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Select by n-gram overlap | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesExample selectorsCustom example selectorSelect by lengthSelect by maximal marginal relevance (MMR)Select by n-gram overlapSelect by similarityLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsExample selectorsSelect by n-gram overlapSelect by n-gram overlapThe NGramOverlapExampleSelector selects and orders examples based on which examples are most similar to the input, according to an ngram overlap score. The ngram overlap score is a float between 0.0 and 1.0, inclusive. The selector allows for a threshold score to be set. Examples with an ngram overlap score less than or equal to the threshold are excluded. The threshold is set to -1.0, by default, so will not exclude any examples, only reorder them. Setting the threshold to 0.0 will exclude examples that have no ngram overlaps with the input.from langchain.prompts import PromptTemplatefrom langchain.prompts.example_selector.ngram_overlap import NGramOverlapExampleSelectorfrom langchain.prompts import FewShotPromptTemplate, PromptTemplateexample_prompt = PromptTemplate(    input_variables=["input", "output"],    template="Input: {input}\nOutput: {output}",)# These are a lot of examples of a pretend task of creating antonyms.examples = [    {"input": "happy", "output": "sad"},    {"input": "tall", "output": "short"},    {"input": "energetic", "output": "lethargic"},    {"input": "sunny", "output":


"lethargic"},    {"input": "sunny", "output": "gloomy"},    {"input": "windy", "output": "calm"},]# These are examples of a fictional translation task.examples = [    {"input": "See Spot run.", "output": "Ver correr a Spot."},    {"input": "My dog barks.", "output": "Mi perro ladra."},    {"input": "Spot can run.", "output": "Spot puede correr."},]example_prompt = PromptTemplate(    input_variables=["input", "output"],    template="Input: {input}\nOutput: {output}",)example_selector = NGramOverlapExampleSelector(    # These are the examples it has available to choose from.    examples=examples,    # This is the PromptTemplate being used to format the examples.    example_prompt=example_prompt,    # This is the threshold, at which selector stops.    # It is set to -1.0 by default.    threshold=-1.0,    # For negative threshold:    # Selector sorts examples by ngram overlap score, and excludes none.    # For threshold greater than 1.0:    # Selector excludes all examples, and returns an empty list.    # For threshold equal to 0.0:    # Selector sorts examples by ngram overlap score,    # and excludes those with no ngram overlap with input.)dynamic_prompt = FewShotPromptTemplate(    # We provide an ExampleSelector instead of examples.    example_selector=example_selector,    example_prompt=example_prompt,    prefix="Give the Spanish translation of every input",


prefix="Give the Spanish translation of every input",    suffix="Input: {sentence}\nOutput:",    input_variables=["sentence"],)# An example input with large ngram overlap with "Spot can run."# and no overlap with "My dog barks."print(dynamic_prompt.format(sentence="Spot can run fast."))    Give the Spanish translation of every input        Input: Spot can run.    Output: Spot puede correr.        Input: See Spot run.    Output: Ver correr a Spot.        Input: My dog barks.    Output: Mi perro ladra.        Input: Spot can run fast.    Output:# You can add examples to NGramOverlapExampleSelector as well.new_example = {"input": "Spot plays fetch.", "output": "Spot juega a buscar."}example_selector.add_example(new_example)print(dynamic_prompt.format(sentence="Spot can run fast."))    Give the Spanish translation of every input        Input: Spot can run.    Output: Spot puede correr.        Input: See Spot run.    Output: Ver correr a Spot.        Input: Spot plays fetch.    Output: Spot juega a buscar.        Input: My dog barks.    Output: Mi perro ladra.        Input: Spot can run fast.    Output:# You can set a threshold at which examples are excluded.# For example, setting threshold equal to 0.0# excludes examples with no ngram overlaps with input.# Since "My dog barks." has no ngram overlaps with "Spot can


overlaps with input.# Since "My dog barks." has no ngram overlaps with "Spot can run fast."# it is excluded.example_selector.threshold = 0.0print(dynamic_prompt.format(sentence="Spot can run fast."))    Give the Spanish translation of every input        Input: Spot can run.    Output: Spot puede correr.        Input: See Spot run.    Output: Ver correr a Spot.        Input: Spot plays fetch.    Output: Spot juega a buscar.        Input: Spot can run fast.    Output:# Setting small nonzero thresholdexample_selector.threshold = 0.09print(dynamic_prompt.format(sentence="Spot can play fetch."))    Give the Spanish translation of every input        Input: Spot can run.    Output: Spot puede correr.        Input: Spot plays fetch.    Output: Spot juega a buscar.        Input: Spot can play fetch.    Output:# Setting threshold greater than 1.0example_selector.threshold = 1.0 + 1e-9print(dynamic_prompt.format(sentence="Spot can play fetch."))    Give the Spanish translation of every input        Input: Spot can play fetch.    Output:PreviousSelect by maximal marginal relevance (MMR)NextSelect by similarityCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Custom example selector | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OPromptsPrompt templatesExample selectorsCustom example selectorSelect by lengthSelect by maximal marginal relevance (MMR)Select by n-gram overlapSelect by similarityLanguage modelsOutput parsersData connectionChainsMemoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesModel I/â€‹OPromptsExample selectorsCustom example selectorOn this pageCustom example selectorIn this tutorial, we'll create a custom example selector that selects every alternate example from a given list of examples.An ExampleSelector must implement two methods:An add_example method which takes in an example and adds it into the ExampleSelectorA select_examples method which takes in input variables (which are meant to be user input) and returns a list of examples to use in the few shot prompt.Let's implement a custom ExampleSelector that just selects two examples at random.:::{note}
Take a look at the current set of example selector implementations supported in LangChain here.


Take a look at the current set of example selector implementations supported in LangChain here.
:::Implement custom example selectorâ€‹from langchain.prompts.example_selector.base import BaseExampleSelectorfrom typing import Dict, Listimport numpy as npclass CustomExampleSelector(BaseExampleSelector):        def __init__(self, examples: List[Dict[str, str]]):        self.examples = examples        def add_example(self, example: Dict[str, str]) -> None:        """Add new example to store for a key."""        self.examples.append(example)    def select_examples(self, input_variables: Dict[str, str]) -> List[dict]:        """Select which examples to use based on the inputs."""        return np.random.choice(self.examples, size=2, replace=False)Use custom example selectorâ€‹examples = [    {"foo": "1"},    {"foo": "2"},    {"foo": "3"}]# Initialize example selector.example_selector = CustomExampleSelector(examples)# Select examplesexample_selector.select_examples({"foo": "foo"})# -> array([{'foo': '2'}, {'foo': '3'}], dtype=object)# Add new example to the set of examplesexample_selector.add_example({"foo": "4"})example_selector.examples# -> [{'foo': '1'}, {'foo': '2'}, {'foo': '3'}, {'foo': '4'}]# Select examplesexample_selector.select_examples({"foo": "foo"})# -> array([{'foo': '1'}, {'foo': '4'}], dtype=object)PreviousExample selectorsNextSelect by lengthImplement custom example selectorUse custom example selectorCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Callbacks | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksAsync callbacksCustom callback handlersCallbacks for custom chainsLogging to fileMultiple callback handlersTagsToken countingModulesGuidesEcosystemAdditional resourcesModulesCallbacksCallbacksinfoHead to Integrations for documentation on built-in callbacks integrations with 3rd-party tools.LangChain provides a callbacks system that allows you to hook into the various stages of your LLM application. This is useful for logging, monitoring, streaming, and other tasks.You can subscribe to these events by using the callbacks argument available throughout the API. This argument is list of handler objects, which are expected to implement one or more of the methods described below in more detail.Callback handlersâ€‹CallbackHandlers are objects that implement the CallbackHandler interface, which has a method for each event that can be subscribed to. The CallbackManager will call the appropriate method on each handler when the event is triggered.class BaseCallbackHandler:    """Base callback handler that can be used to handle callbacks from langchain."""    def on_llm_start(        self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any    ) -> Any:        """Run when LLM starts running."""    def on_chat_model_start(        self, serialized: Dict[str, Any], messages: List[List[BaseMessage]], **kwargs: Any    ) -> Any:        """Run when Chat Model starts running."""    def on_llm_new_token(self, token: str, **kwargs: Any) -> Any:        """Run on new LLM


Any) -> Any:        """Run on new LLM token. Only available when streaming is enabled."""    def on_llm_end(self, response: LLMResult, **kwargs: Any) -> Any:        """Run when LLM ends running."""    def on_llm_error(        self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any    ) -> Any:        """Run when LLM errors."""    def on_chain_start(        self, serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any    ) -> Any:        """Run when chain starts running."""    def on_chain_end(self, outputs: Dict[str, Any], **kwargs: Any) -> Any:        """Run when chain ends running."""    def on_chain_error(        self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any    ) -> Any:        """Run when chain errors."""    def on_tool_start(        self, serialized: Dict[str, Any], input_str: str, **kwargs: Any    ) -> Any:        """Run when tool starts running."""    def on_tool_end(self, output: str, **kwargs: Any) -> Any:        """Run when tool ends running."""    def on_tool_error(        self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any    ) -> Any:        """Run when tool errors."""


) -> Any:        """Run when tool errors."""    def on_text(self, text: str, **kwargs: Any) -> Any:        """Run on arbitrary text."""    def on_agent_action(self, action: AgentAction, **kwargs: Any) -> Any:        """Run on agent action."""    def on_agent_finish(self, finish: AgentFinish, **kwargs: Any) -> Any:        """Run on agent end."""Get startedâ€‹LangChain provides a few built-in handlers that you can use to get started. These are available in the langchain/callbacks module. The most basic handler is the StdOutCallbackHandler, which simply logs all events to stdout.Note when the verbose flag on the object is set to true, the StdOutCallbackHandler will be invoked even without being explicitly passed in.from langchain.callbacks import StdOutCallbackHandlerfrom langchain.chains import LLMChainfrom langchain.llms import OpenAIfrom langchain.prompts import PromptTemplatehandler = StdOutCallbackHandler()llm = OpenAI()prompt = PromptTemplate.from_template("1 + {number} = ")# Constructor callback: First, let's explicitly set the StdOutCallbackHandler when initializing our chainchain = LLMChain(llm=llm, prompt=prompt, callbacks=[handler])chain.run(number=2)# Use verbose flag: Then, let's use the `verbose` flag to achieve the same resultchain = LLMChain(llm=llm, prompt=prompt, verbose=True)chain.run(number=2)# Request callbacks: Finally, let's use the request `callbacks` to achieve the same resultchain = LLMChain(llm=llm, prompt=prompt)chain.run(number=2, callbacks=[handler])    > Entering new LLMChain


callbacks=[handler])    > Entering new LLMChain chain...    Prompt after formatting:    1 + 2 =         > Finished chain.            > Entering new LLMChain chain...    Prompt after formatting:    1 + 2 =         > Finished chain.            > Entering new LLMChain chain...    Prompt after formatting:    1 + 2 =         > Finished chain.    '\n\n3'Where to pass in callbacksâ€‹The callbacks argument is available on most objects throughout the API (Chains, Models, Tools, Agents, etc.) in two different places:Constructor callbacks: defined in the constructor, eg. LLMChain(callbacks=[handler], tags=['a-tag']), which will be used for all calls made on that object, and will be scoped to that object only, eg. if you pass a handler to the LLMChain constructor, it will not be used by the Model attached to that chain.Request callbacks: defined in the run()/apply() methods used for issuing a request, eg. chain.run(input, callbacks=[handler]), which will be used for that specific request only, and all sub-requests that it contains (eg. a call to an LLMChain triggers a call to a Model, which uses the same handler passed in the call() method).The verbose argument is available on most objects throughout the API (Chains, Models, Tools, Agents, etc.) as a constructor argument, eg. LLMChain(verbose=True), and it is equivalent to passing a ConsoleCallbackHandler to the callbacks argument of that object and all child objects. This is useful for debugging, as it will log all events to


that object and all child objects. This is useful for debugging, as it will log all events to the console.When do you want to use each of these?â€‹Constructor callbacks are most useful for use cases such as logging, monitoring, etc., which are not specific to a single request, but rather to the entire chain. For example, if you want to log all the requests made to an LLMChain, you would pass a handler to the constructor.Request callbacks are most useful for use cases such as streaming, where you want to stream the output of a single request to a specific websocket connection, or other similar use cases. For example, if you want to stream the output of a single request to a websocket, you would pass a handler to the call() methodPreviousToolkitsNextAsync callbacksCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Async callbacks | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksAsync callbacksCustom callback handlersCallbacks for custom chainsLogging to fileMultiple callback handlersTagsToken countingModulesGuidesEcosystemAdditional resourcesModulesCallbacksAsync callbacksAsync callbacksIf you are planning to use the async API, it is recommended to use AsyncCallbackHandler to avoid blocking the runloop. Advanced if you use a sync CallbackHandler while using an async method to run your llm/chain/tool/agent, it will still work. However, under the hood, it will be called with run_in_executor which can cause issues if your CallbackHandler is not thread-safe.import asynciofrom typing import Any, Dict, Listfrom langchain.chat_models import ChatOpenAIfrom langchain.schema import LLMResult, HumanMessagefrom langchain.callbacks.base import AsyncCallbackHandler, BaseCallbackHandlerclass MyCustomSyncHandler(BaseCallbackHandler):    def on_llm_new_token(self, token: str, **kwargs) -> None:        print(f"Sync handler being called in a `thread_pool_executor`: token: {token}")class MyCustomAsyncHandler(AsyncCallbackHandler):    """Async callback handler that can be used to handle callbacks from langchain."""    async def on_llm_start(        self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any    ) -> None:        """Run when chain starts running."""        print("zzzz....")        await asyncio.sleep(0.3)        class_name = serialized["name"]


class_name = serialized["name"]        print("Hi! I just woke up. Your llm is starting")    async def on_llm_end(self, response: LLMResult, **kwargs: Any) -> None:        """Run when chain ends running."""        print("zzzz....")        await asyncio.sleep(0.3)        print("Hi! I just woke up. Your llm is ending")# To enable streaming, we pass in `streaming=True` to the ChatModel constructor# Additionally, we pass in a list with our custom handlerchat = ChatOpenAI(    max_tokens=25,    streaming=True,    callbacks=[MyCustomSyncHandler(), MyCustomAsyncHandler()],)await chat.agenerate([[HumanMessage(content="Tell me a joke")]])    zzzz....    Hi! I just woke up. Your llm is starting    Sync handler being called in a `thread_pool_executor`: token:     Sync handler being called in a `thread_pool_executor`: token: Why    Sync handler being called in a `thread_pool_executor`: token:  don    Sync handler being called in a `thread_pool_executor`: token: 't    Sync handler being called in a `thread_pool_executor`: token:  scientists    Sync handler being called in a `thread_pool_executor`: token:  trust    Sync handler being called in a `thread_pool_executor`: token:  atoms    Sync handler being called in a `thread_pool_executor`: token: ?    Sync handler being called in a `thread_pool_executor`: token:              Sync handler being


token:              Sync handler being called in a `thread_pool_executor`: token: Because    Sync handler being called in a `thread_pool_executor`: token:  they    Sync handler being called in a `thread_pool_executor`: token:  make    Sync handler being called in a `thread_pool_executor`: token:  up    Sync handler being called in a `thread_pool_executor`: token:  everything    Sync handler being called in a `thread_pool_executor`: token: .    Sync handler being called in a `thread_pool_executor`: token:     zzzz....    Hi! I just woke up. Your llm is ending    LLMResult(generations=[[ChatGeneration(text="Why don't scientists trust atoms? \n\nBecause they make up everything.", generation_info=None, message=AIMessage(content="Why don't scientists trust atoms? \n\nBecause they make up everything.", additional_kwargs={}, example=False))]], llm_output={'token_usage': {}, 'model_name': 'gpt-3.5-turbo'})PreviousCallbacksNextCustom callback handlersCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Multiple callback handlers | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksAsync callbacksCustom callback handlersCallbacks for custom chainsLogging to fileMultiple callback handlersTagsToken countingModulesGuidesEcosystemAdditional resourcesModulesCallbacksMultiple callback handlersMultiple callback handlersIn the previous examples, we passed in callback handlers upon creation of an object by using callbacks=. In this case, the callbacks will be scoped to that particular object. However, in many cases, it is advantageous to pass in handlers instead when running the object. When we pass through CallbackHandlers using the callbacks keyword arg when executing an run, those callbacks will be issued by all nested objects involved in the execution. For example, when a handler is passed through to an Agent, it will be used for all callbacks related to the agent and all the objects involved in the agent's execution, in this case, the Tools, LLMChain, and LLM.This prevents us from having to manually attach the handlers to each individual nested object.from typing import Dict, Union, Any, Listfrom langchain.callbacks.base import BaseCallbackHandlerfrom langchain.schema import AgentActionfrom langchain.agents import AgentType, initialize_agent, load_toolsfrom langchain.callbacks import tracing_enabledfrom langchain.llms import OpenAI# First, define custom callback handler implementationsclass MyCustomHandlerOne(BaseCallbackHandler):    def on_llm_start(        self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any    ) -> Any:        print(f"on_llm_start {serialized['name']}")    def on_llm_new_token(self, token: str, **kwargs: Any) -> Any:


token: str, **kwargs: Any) -> Any:        print(f"on_new_token {token}")    def on_llm_error(        self, error: Union[Exception, KeyboardInterrupt], **kwargs: Any    ) -> Any:        """Run when LLM errors."""    def on_chain_start(        self, serialized: Dict[str, Any], inputs: Dict[str, Any], **kwargs: Any    ) -> Any:        print(f"on_chain_start {serialized['name']}")    def on_tool_start(        self, serialized: Dict[str, Any], input_str: str, **kwargs: Any    ) -> Any:        print(f"on_tool_start {serialized['name']}")    def on_agent_action(self, action: AgentAction, **kwargs: Any) -> Any:        print(f"on_agent_action {action}")class MyCustomHandlerTwo(BaseCallbackHandler):    def on_llm_start(        self, serialized: Dict[str, Any], prompts: List[str], **kwargs: Any    ) -> Any:        print(f"on_llm_start (I'm the second handler!!) {serialized['name']}")# Instantiate the handlershandler1 = MyCustomHandlerOne()handler2 = MyCustomHandlerTwo()# Setup the agent. Only the `llm` will issue callbacks for handler2llm = OpenAI(temperature=0, streaming=True, callbacks=[handler2])tools = load_tools(["llm-math"], llm=llm)agent = initialize_agent(tools, llm,


llm=llm)agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION)# Callbacks for handler1 will be issued by every object involved in the# Agent execution (llm, llmchain, tool, agent executor)agent.run("What is 2 raised to the 0.235 power?", callbacks=[handler1])    on_chain_start AgentExecutor    on_chain_start LLMChain    on_llm_start OpenAI    on_llm_start (I'm the second handler!!) OpenAI    on_new_token  I    on_new_token  need    on_new_token  to    on_new_token  use    on_new_token  a    on_new_token  calculator    on_new_token  to    on_new_token  solve    on_new_token  this    on_new_token .    on_new_token     Action    on_new_token :    on_new_token  Calculator    on_new_token     Action    on_new_token  Input    on_new_token :    on_new_token  2    on_new_token ^    on_new_token 0    on_new_token .    on_new_token 235    on_new_token     on_agent_action AgentAction(tool='Calculator', tool_input='2^0.235', log=' I need to use a calculator to solve this.\nAction: Calculator\nAction Input: 2^0.235')    on_tool_start Calculator    on_chain_start LLMMathChain    on_chain_start LLMChain    on_llm_start OpenAI    on_llm_start (I'm the second handler!!) OpenAI


on_llm_start (I'm the second handler!!) OpenAI    on_new_token     on_new_token ```text    on_new_token         on_new_token 2    on_new_token **    on_new_token 0    on_new_token .    on_new_token 235    on_new_token         on_new_token ```        on_new_token ...    on_new_token num    on_new_token expr    on_new_token .    on_new_token evaluate    on_new_token ("    on_new_token 2    on_new_token **    on_new_token 0    on_new_token .    on_new_token 235    on_new_token ")    on_new_token ...    on_new_token         on_new_token     on_chain_start LLMChain    on_llm_start OpenAI    on_llm_start (I'm the second handler!!) OpenAI    on_new_token  I    on_new_token  now    on_new_token  know    on_new_token  the    on_new_token  final    on_new_token  answer    on_new_token .    on_new_token     Final    on_new_token  Answer    on_new_token :    on_new_token  1    on_new_token .    on_new_token 17    on_new_token 690    on_new_token 67    on_new_token 372    on_new_token 187    on_new_token 674


on_new_token 187    on_new_token 674    on_new_token     '1.1769067372187674'PreviousLogging to fileNextTagsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Tags | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksAsync callbacksCustom callback handlersCallbacks for custom chainsLogging to fileMultiple callback handlersTagsToken countingModulesGuidesEcosystemAdditional resourcesModulesCallbacksTagsTagsYou can add tags to your callbacks by passing a tags argument to the call()/run()/apply() methods. This is useful for filtering your logs, eg. if you want to log all requests made to a specific LLMChain, you can add a tag, and then filter your logs by that tag. You can pass tags to both constructor and request callbacks, see the examples above for details. These tags are then passed to the tags argument of the "start" callback methods, ie. on_llm_start, on_chat_model_start, on_chain_start, on_tool_start.PreviousMultiple callback handlersNextToken countingCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Logging to file | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksAsync callbacksCustom callback handlersCallbacks for custom chainsLogging to fileMultiple callback handlersTagsToken countingModulesGuidesEcosystemAdditional resourcesModulesCallbacksLogging to fileLogging to fileThis example shows how to print logs to file. It shows how to use the FileCallbackHandler, which does the same thing as StdOutCallbackHandler, but instead writes the output to file. It also uses the loguru library to log other outputs that are not captured by the handler.from loguru import loggerfrom langchain.callbacks import FileCallbackHandlerfrom langchain.chains import LLMChainfrom langchain.llms import OpenAIfrom langchain.prompts import PromptTemplatelogfile = "output.log"logger.add(logfile, colorize=True, enqueue=True)handler = FileCallbackHandler(logfile)llm = OpenAI()prompt = PromptTemplate.from_template("1 + {number} = ")# this chain will both print to stdout (because verbose=True) and write to 'output.log'# if verbose=False, the FileCallbackHandler will still write to 'output.log'chain = LLMChain(llm=llm, prompt=prompt, callbacks=[handler], verbose=True)answer = chain.run(number=2)logger.info(answer)            > Entering new LLMChain chain...    Prompt after formatting:    1 + 2 =     2023-06-01 18:36:38.929 | INFO     |


| INFO     | __main__:<module>:20 -         3        > Finished chain.Now we can open the file output.log to see that the output has been captured.pip install ansi2html > /dev/nullfrom IPython.display import display, HTMLfrom ansi2html import Ansi2HTMLConverterwith open("output.log", "r") as f:    content = f.read()conv = Ansi2HTMLConverter()html = conv.convert(content, full=True)display(HTML(html))<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN" "http://www.w3.org/TR/html4/loose.dtd"><meta http-equiv="Content-Type" content="text/html; charset=utf-8"><title></title>
> Entering new LLMChain chain...Prompt after formatting:<span class="ansi1 ansi3


class="ansi1 ansi32">1 + 2 = > Finished chain.2023-06-01 18:36:38.929 | INFO     | __main__:<module>:20 - 3
PreviousCallbacks for custom chainsNextMultiple callback handlersCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Custom callback handlers | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksAsync callbacksCustom callback handlersCallbacks for custom chainsLogging to fileMultiple callback handlersTagsToken countingModulesGuidesEcosystemAdditional resourcesModulesCallbacksCustom callback handlersCustom callback handlersYou can create a custom handler to set on the object as well. In the example below, we'll implement streaming with a custom handler.from langchain.callbacks.base import BaseCallbackHandlerfrom langchain.chat_models import ChatOpenAIfrom langchain.schema import HumanMessageclass MyCustomHandler(BaseCallbackHandler):    def on_llm_new_token(self, token: str, **kwargs) -> None:        print(f"My custom handler, token: {token}")# To enable streaming, we pass in `streaming=True` to the ChatModel constructor# Additionally, we pass in a list with our custom handlerchat = ChatOpenAI(max_tokens=25, streaming=True, callbacks=[MyCustomHandler()])chat([HumanMessage(content="Tell me a joke")])    My custom handler, token:     My custom handler, token: Why    My custom handler, token:  don    My custom handler, token: 't    My custom handler, token:  scientists    My custom handler, token:  trust    My custom handler, token:  atoms    My custom handler, token: ?    My custom handler, token:              My custom handler, token: Because    My custom handler, token:  they    My custom handler, token:  make    My custom handler, token:  up


My custom handler, token:  make    My custom handler, token:  up    My custom handler, token:  everything    My custom handler, token: .    My custom handler, token:     AIMessage(content="Why don't scientists trust atoms? \n\nBecause they make up everything.", additional_kwargs={}, example=False)PreviousAsync callbacksNextCallbacks for custom chainsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.