Token counting | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksAsync callbacksCustom callback handlersCallbacks for custom chainsLogging to fileMultiple callback handlersTagsToken countingModulesGuidesEcosystemAdditional resourcesModulesCallbacksToken countingToken countingLangChain offers a context manager that allows you to count tokens.import asynciofrom langchain.callbacks import get_openai_callbackfrom langchain.llms import OpenAIllm = OpenAI(temperature=0)with get_openai_callback() as cb:    llm("What is the square root of 4?")total_tokens = cb.total_tokensassert total_tokens > 0with get_openai_callback() as cb:    llm("What is the square root of 4?")    llm("What is the square root of 4?")assert cb.total_tokens == total_tokens * 2# You can kick off concurrent runs from within the context managerwith get_openai_callback() as cb:    await asyncio.gather(        *[llm.agenerate(["What is the square root of 4?"]) for _ in range(3)]    )assert cb.total_tokens == total_tokens * 3# The context manager is concurrency safetask = asyncio.create_task(llm.agenerate(["What is the square root of 4?"]))with get_openai_callback() as cb:    await llm.agenerate(["What is the square root of 4?"])await taskassert cb.total_tokens == total_tokensPreviousTagsNextModulesCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Callbacks for custom chains | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksAsync callbacksCustom callback handlersCallbacks for custom chainsLogging to fileMultiple callback handlersTagsToken countingModulesGuidesEcosystemAdditional resourcesModulesCallbacksCallbacks for custom chainsCallbacks for custom chains When you create a custom chain you can easily set it up to use the same callback system as all the built-in chains.
_call, _generate, _run, and equivalent async methods on Chains / LLMs / Chat Models / Agents / Tools now receive a 2nd argument called run_manager which is bound to that run, and contains the logging methods that can be used by that object (i.e. on_llm_new_token). This is useful when constructing a custom chain. See this guide for more information on how to create custom chains and use callbacks inside them.PreviousCustom callback handlersNextLogging to fileCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Evaluation | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksModulesGuidesEvaluationString EvaluatorsComparison EvaluatorsTrajectory EvaluatorsExamplesDebuggingDeploymentLangSmithModel ComparisonEcosystemAdditional resourcesGuidesEvaluationOn this pageEvaluationLanguage models can be unpredictable. This makes it challenging to ship reliable applications to production, where repeatable, useful outcomes across diverse inputs are a minimum requirement. Tests help demonstrate each component in an LLM application can produce the required or expected functionality. These tests also safeguard against regressions while you improve interconnected pieces of an integrated system. However, measuring the quality of generated text can be challenging. It can be hard to agree on the right set of metrics for your application, and it can be difficult to translate those into better performance. Furthermore, it's common to lack sufficient evaluation data to adequately test the range of inputs and expected outputs for each component when you're just getting started. The LangChain community is building open source tools and guides to help address these challenges.LangChain exposes different types of evaluators for common types of evaluation. Each type has off-the-shelf implementations you can use to get started, as well as an


extensible API so you can create your own or contribute improvements for everyone to use. The following sections have example notebooks for you to get started.String Evaluators: Evaluate the predicted string for a given input, usually against a reference stringTrajectory Evaluators: Evaluate the whole trajectory of agent actionsComparison Evaluators: Compare predictions from two runs on a common inputThis section also provides some additional examples of how you could use these evaluators for different scenarios or apply to different chain implementations in the LangChain library. Some examples include:Preference Scoring Chain Outputs: An example using a comparison evaluator on different models or prompts to select statistically significant differences in aggregate preference scoresReference Docsâ€‹For detailed information of the available evaluators, including how to instantiate, configure, and customize them. Check out the reference documentation directly.ğŸ—ƒï¸� String Evaluators5 itemsğŸ—ƒï¸� Comparison Evaluators3 itemsğŸ—ƒï¸� Trajectory Evaluators2 itemsğŸ—ƒï¸� Examples9 itemsPreviousGuidesNextString EvaluatorsReference DocsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Comparison Evaluators | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksModulesGuidesEvaluationString EvaluatorsComparison EvaluatorsCustom Pairwise EvaluatorPairwise Embedding DistancePairwise String ComparisonTrajectory EvaluatorsExamplesDebuggingDeploymentLangSmithModel ComparisonEcosystemAdditional resourcesGuidesEvaluationComparison EvaluatorsComparison EvaluatorsğŸ“„ï¸� Custom Pairwise EvaluatorYou can make your own pairwise string evaluators by inheriting from PairwiseStringEvaluator class and overwriting the evaluatestringpairs method (and the aevaluatestringpairs method if you want to use the evaluator asynchronously).ğŸ“„ï¸� Pairwise Embedding DistanceOne way to measure the similarity (or dissimilarity) between two predictions on a shared or similar input is to embed the predictions and compute a vector distance between the two embeddings.[1]ğŸ“„ï¸� Pairwise String ComparisonOften you will want to compare predictions of an LLM, Chain, or Agent for a given input. The StringComparison evaluators facilitate this so you can answer questions like:PreviousString DistanceNextCustom Pairwise EvaluatorCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Comparing Chain Outputs | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksModulesGuidesEvaluationString EvaluatorsComparison EvaluatorsTrajectory EvaluatorsExamplesAgent VectorDB Question Answering BenchmarkingComparing Chain OutputsData Augmented Question AnsweringEvaluating an OpenAPI ChainQuestion Answering Benchmarking: Paul Graham EssayQuestion Answering Benchmarking: State of the Union AddressQA GenerationQuestion AnsweringSQL Question Answering Benchmarking: ChinookDebuggingDeploymentLangSmithModel ComparisonEcosystemAdditional resourcesGuidesEvaluationExamplesComparing Chain OutputsOn this pageComparing Chain OutputsSuppose you have two different prompts (or LLMs). How do you know which will generate "better" results?One automated way to predict the preferred configuration is to use a PairwiseStringEvaluator like the PairwiseStringEvalChain[1]. This chain prompts an LLM to select which output is preferred, given a specific input.For this evaluation, we will need 3 things:An evaluatorA dataset of inputs2 (or more) LLMs, Chains, or Agents to compareThen we will aggregate the restults to determine the preferred model.Step 1. Create the Evaluatorâ€‹In this example, you will use gpt-4 to select which output is preferred.from langchain.chat_models import ChatOpenAIfrom langchain.evaluation.comparison import PairwiseStringEvalChainllm = ChatOpenAI(model="gpt-4")eval_chain = PairwiseStringEvalChain.from_llm(llm=llm)Step 2. Select Datasetâ€‹If you already have real usage data for your LLM, you can use a representative sample. More examples


provide more reliable results. We will use some example queries someone might have about how to use langchain here.from langchain.evaluation.loading import load_datasetdataset = load_dataset("langchain-howto-queries")    Found cached dataset parquet (/Users/wfh/.cache/huggingface/datasets/LangChainDatasets___parquet/LangChainDatasets--langchain-howto-queries-bbb748bbee7e77aa/0.0.0/2a3b91fbd88a2c90d1dbbb32b460cf621d31bd5b05b934492fdef7d8d6f236ec)      0%|          | 0/1 [00:00<?, ?it/s]Step 3. Define Models to Compareâ€‹We will be comparing two agents in this case.from langchain import SerpAPIWrapperfrom langchain.agents import initialize_agent, Toolfrom langchain.agents import AgentTypefrom langchain.chat_models import ChatOpenAI# Initialize the language model# You can add your own OpenAI API key by adding openai_api_key="<your_api_key>"llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613")# Initialize the SerpAPIWrapper for search functionality# Replace <your_api_key> in openai_api_key="<your_api_key>" with your actual SerpAPI key.search = SerpAPIWrapper()# Define a list of tools offered by the agenttools = [    Tool(        name="Search",        func=search.run,        coroutine=search.arun,        description="Useful when you need to answer questions about current events. You should ask targeted questions.",


when you need to answer questions about current events. You should ask targeted questions.",    ),]functions_agent = initialize_agent(    tools, llm, agent=AgentType.OPENAI_MULTI_FUNCTIONS, verbose=False)conversations_agent = initialize_agent(    tools, llm, agent=AgentType.CHAT_ZERO_SHOT_REACT_DESCRIPTION, verbose=False)Step 4. Generate Responsesâ€‹We will generate outputs for each of the models before evaluating them.from tqdm.notebook import tqdmimport asyncioresults = []agents = [functions_agent, conversations_agent]concurrency_level = 6  # How many concurrent agents to run. May need to decrease if OpenAI is rate limiting.# We will only run the first 20 examples of this dataset to speed things up# This will lead to larger confidence intervals downstream.batch = []for example in tqdm(dataset[:20]):    batch.extend([agent.acall(example["inputs"]) for agent in agents])    if len(batch) >= concurrency_level:        batch_results = await asyncio.gather(*batch, return_exceptions=True)        results.extend(list(zip(*[iter(batch_results)] * 2)))        batch = []if batch:    batch_results = await asyncio.gather(*batch, return_exceptions=True)    results.extend(list(zip(*[iter(batch_results)] * 2)))      0%|          | 0/20 [00:00<?, ?it/s]    Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in 1.0 seconds as it raised ServiceUnavailableError: The server is overloaded or not ready yet..    Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry


Retrying langchain.chat_models.openai.acompletion_with_retry.<locals>._completion_with_retry in 1.0 seconds as it raised ServiceUnavailableError: The server is overloaded or not ready yet..Step 5. Evaluate Pairsâ€‹Now it's time to evaluate the results. For each agent response, run the evaluation chain to select which output is preferred (or return a tie).Randomly select the input order to reduce the likelihood that one model will be preferred just because it is presented first.import randomdef predict_preferences(dataset, results) -> list:    preferences = []    for example, (res_a, res_b) in zip(dataset, results):        input_ = example["inputs"]        # Flip a coin to reduce persistent position bias        if random.random() < 0.5:            pred_a, pred_b = res_a, res_b            a, b = "a", "b"        else:            pred_a, pred_b = res_b, res_a            a, b = "b", "a"        eval_res = eval_chain.evaluate_string_pairs(            prediction=pred_a["output"] if isinstance(pred_a, dict) else str(pred_a),            prediction_b=pred_b["output"] if isinstance(pred_b, dict) else str(pred_b),            input=input_,        )        if eval_res["value"] == "A":


)        if eval_res["value"] == "A":            preferences.append(a)        elif eval_res["value"] == "B":            preferences.append(b)        else:            preferences.append(None)  # No preference    return preferencespreferences = predict_preferences(dataset, results)Print out the ratio of preferences.from collections import Countername_map = {    "a": "OpenAI Functions Agent",    "b": "Structured Chat Agent",}counts = Counter(preferences)pref_ratios = {k: v / len(preferences) for k, v in counts.items()}for k, v in pref_ratios.items():    print(f"{name_map.get(k)}: {v:.2%}")    OpenAI Functions Agent: 90.00%    Structured Chat Agent: 10.00%Estimate Confidence Intervalsâ€‹The results seem pretty clear, but if you want to have a better sense of how confident we are, that model "A" (the OpenAI Functions Agent) is the preferred model, we can calculate confidence intervals. Below, use the Wilson score to estimate the confidence interval.from math import sqrtdef wilson_score_interval(    preferences: list, which: str = "a", z: float = 1.96) -> tuple:    """Estimate the confidence interval using the Wilson score.    See: https://en.wikipedia.org/wiki/Binomial_proportion_confidence_interval#Wilson_score_interval    for more details, including when to use it and when it should not be used.    """    total_preferences = preferences.count("a") +


not be used.    """    total_preferences = preferences.count("a") + preferences.count("b")    n_s = preferences.count(which)    if total_preferences == 0:        return (0, 0)    p_hat = n_s / total_preferences    denominator = 1 + (z**2) / total_preferences    adjustment = (z / denominator) * sqrt(        p_hat * (1 - p_hat) / total_preferences        + (z**2) / (4 * total_preferences * total_preferences)    )    center = (p_hat + (z**2) / (2 * total_preferences)) / denominator    lower_bound = min(max(center - adjustment, 0.0), 1.0)    upper_bound = min(max(center + adjustment, 0.0), 1.0)    return (lower_bound, upper_bound)for which_, name in name_map.items():    low, high = wilson_score_interval(preferences, which=which_)    print(        f'The "{name}" would be preferred between {low:.2%} and {high:.2%} percent of the time (with 95% confidence).'    )    The "OpenAI Functions Agent" would be preferred between 69.90% and 97.21% percent of the time (with 95% confidence).    The "Structured Chat Agent" would be preferred between 2.79% and 30.10% percent of the time (with 95% confidence).Print out the p-value.from scipy import statspreferred_model = max(pref_ratios, key=pref_ratios.get)successes =


import statspreferred_model = max(pref_ratios, key=pref_ratios.get)successes = preferences.count(preferred_model)n = len(preferences) - preferences.count(None)p_value = stats.binom_test(successes, n, p=0.5, alternative="two-sided")print(    f"""The p-value is {p_value:.5f}. If the null hypothesis is true (i.e., if the selected eval chain actually has no preference between the models),then there is a {p_value:.5%} chance of observing the {name_map.get(preferred_model)} be preferred at least {successes}times out of {n} trials.""")    The p-value is 0.00040. If the null hypothesis is true (i.e., if the selected eval chain actually has no preference between the models),    then there is a 0.04025% chance of observing the OpenAI Functions Agent be preferred at least 18    times out of 20 trials._1. Note: Automated evals are still an open research topic and are best used alongside other evaluation approaches. LLM preferences exhibit biases, including banal ones like the order of outputs. In choosing preferences, "ground truth" may not be taken into account, which may lead to scores that aren't grounded in utility._PreviousAgent VectorDB Question Answering BenchmarkingNextData Augmented Question AnsweringStep 1. Create the EvaluatorStep 2. Select DatasetStep 3. Define Models to CompareStep 4. Generate ResponsesStep 5. Evaluate PairsEstimate Confidence IntervalsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Trajectory Evaluators | ğŸ¦œï¸�ğŸ”— Langchain

Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksModulesGuidesEvaluationString EvaluatorsComparison EvaluatorsTrajectory EvaluatorsCustom Trajectory EvaluatorAgent TrajectoryExamplesDebuggingDeploymentLangSmithModel ComparisonEcosystemAdditional resourcesGuidesEvaluationTrajectory EvaluatorsTrajectory EvaluatorsğŸ“„ï¸� Custom Trajectory EvaluatorYou can make your own custom trajectory evaluators by inheriting from the AgentTrajectoryEvaluator class and overwriting the evaluateagenttrajectory (and aevaluateagentaction) method.ğŸ“„ï¸� Agent TrajectoryAgents can be difficult to holistically evaluate due to the breadth of actions and generation they can make. We recommend using multiple evaluation techniques appropriate to your use case. One way to evaluate an agent is to look at the whole trajectory of actions taken along with their responses.PreviousPairwise String ComparisonNextCustom Trajectory EvaluatorCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


String Evaluators | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryAgentsCallbacksModulesGuidesEvaluationString EvaluatorsEvaluating Custom CriteriaCustom String EvaluatorEmbedding DistanceQA CorrectnessString DistanceComparison EvaluatorsTrajectory EvaluatorsExamplesDebuggingDeploymentLangSmithModel ComparisonEcosystemAdditional resourcesGuidesEvaluationString EvaluatorsString EvaluatorsğŸ“„ï¸� Evaluating Custom CriteriaSuppose you want to test a model's output against a custom rubric or custom set of criteria, how would you go about testing this?ğŸ“„ï¸� Custom String EvaluatorYou can make your own custom string evaluators by inheriting from the StringEvaluator class and implementing the evaluatestrings (and aevaluatestrings for async support) methods.ğŸ“„ï¸� Embedding DistanceTo measure semantic similarity (or dissimilarity) between a prediction and a reference label string, you could use a vector vector distance metric the two embedded representations using the embeddingdistance evaluator.[1]ğŸ“„ï¸� QA CorrectnessWhen thinking about a QA system, one of the most important questions to ask is whether the final generated result is correct. The "qa" evaluator compares a question-answering model's response to a reference answer to provide this level of information. If you are able to annotate a test dataset, this evaluator will be useful.ğŸ“„ï¸� String DistanceOne of the simplest ways to compare an LLM or chain's string output against a reference label is by using string distance measurements such as Levenshtein or postfix distance.  This can be used alongside approximate/fuzzy matching criteria for very basic unit testing.PreviousEvaluationNextEvaluating Custom


used alongside approximate/fuzzy matching criteria for very basic unit testing.PreviousEvaluationNextEvaluating Custom CriteriaCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Memory | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryOn this pageMemoryğŸš§ Docs under construction ğŸš§infoHead to Integrations for documentation on built-in memory integrations with 3rd-party tools.By default, Chains and Agents are stateless,
meaning that they treat each incoming query independently (like the underlying LLMs and chat models themselves).
In some applications, like chatbots, it is essential
to remember previous interactions, both in the short and long-term.
The Memory class does exactly that.LangChain provides memory components in two forms.
First, LangChain provides helper utilities for managing and manipulating previous chat messages.
These are designed to be modular and useful regardless of how they are used.


Secondly, LangChain provides easy ways to incorporate these utilities into chains.Get startedâ€‹Memory involves keeping a concept of state around throughout a user's interactions with an language model. A user's interactions with a language model are captured in the concept of ChatMessages, so this boils down to ingesting, capturing, transforming and extracting knowledge from a sequence of chat messages. There are many different ways to do this, each of which exists as its own memory type.In general, for each type of memory there are two ways to understanding using memory. These are the standalone functions which extract information from a sequence of messages, and then there is the way you can use this type of memory in a chain.Memory can return multiple pieces of information (for example, the most recent N messages and a summary of all previous messages). The returned information can either be a string or a list of messages.We will walk through the simplest form of memory: "buffer" memory, which just involves keeping a buffer of all prior messages. We will show how to use the modular utility functions here, then show how it can be used in a chain (both returning a string as well as a list of messages).ChatMessageHistoryâ€‹One of the core utility classes underpinning most (if not all) memory modules is the ChatMessageHistory class. This is a super lightweight wrapper which exposes convenience methods for saving Human messages, AI messages, and then fetching them all.You may want to use this class directly if you are managing memory outside of a chain.from langchain.memory import ChatMessageHistoryhistory = ChatMessageHistory()history.add_user_message("hi!")history.add_ai_message("whats up?")history.messages    [HumanMessage(content='hi!', additional_kwargs={}),     AIMessage(content='whats up?', additional_kwargs={})]ConversationBufferMemoryâ€‹We now show how to use this simple concept in a chain. We first showcase


now show how to use this simple concept in a chain. We first showcase ConversationBufferMemory which is just a wrapper around ChatMessageHistory that extracts the messages in a variable.We can first extract it as a string.from langchain.memory import ConversationBufferMemorymemory = ConversationBufferMemory()memory.chat_memory.add_user_message("hi!")memory.chat_memory.add_ai_message("whats up?")memory.load_memory_variables({})    {'history': 'Human: hi!\nAI: whats up?'}We can also get the history as a list of messagesmemory = ConversationBufferMemory(return_messages=True)memory.chat_memory.add_user_message("hi!")memory.chat_memory.add_ai_message("whats up?")memory.load_memory_variables({})    {'history': [HumanMessage(content='hi!', additional_kwargs={}),      AIMessage(content='whats up?', additional_kwargs={})]}Using in a chainâ€‹Finally, let's take a look at using this in a chain (setting verbose=True so we can see the prompt).from langchain.llms import OpenAIfrom langchain.chains import ConversationChainllm = OpenAI(temperature=0)conversation = ConversationChain(    llm=llm,    verbose=True,    memory=ConversationBufferMemory())conversation.predict(input="Hi there!")    > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.    Current conversation:    Human: Hi there!    AI:    > Finished chain.    " Hi there! It's nice to meet you. How can I help you today?"conversation.predict(input="I'm


It's nice to meet you. How can I help you today?"conversation.predict(input="I'm doing well! Just having a conversation with an AI.")    > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.    Current conversation:    Human: Hi there!    AI:  Hi there! It's nice to meet you. How can I help you today?    Human: I'm doing well! Just having a conversation with an AI.    AI:    > Finished chain.    " That's great! It's always nice to have a conversation with someone new. What would you like to talk about?"conversation.predict(input="Tell me about yourself.")    > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.    Current conversation:    Human: Hi there!    AI:  Hi there! It's nice to meet you. How can I help you today?    Human: I'm doing well! Just having a conversation with an AI.    AI:  That's great! It's always nice to have a conversation with someone new. What would you like to talk about?    Human: Tell me about yourself.    AI:    > Finished chain.    " Sure! I'm an AI created to help people with their everyday


Finished chain.    " Sure! I'm an AI created to help people with their everyday tasks. I'm programmed to understand natural language and provide helpful information. I'm also constantly learning and updating my knowledge base so I can provide more accurate and helpful answers."Saving Message Historyâ€‹You may often have to save messages, and then load them to use again. This can be done easily by first converting the messages to normal python dictionaries, saving those (as json or something) and then loading those. Here is an example of doing that.import jsonfrom langchain.memory import ChatMessageHistoryfrom langchain.schema import messages_from_dict, messages_to_dicthistory = ChatMessageHistory()history.add_user_message("hi!")history.add_ai_message("whats up?")dicts = messages_to_dict(history.messages)dicts    [{'type': 'human', 'data': {'content': 'hi!', 'additional_kwargs': {}}},     {'type': 'ai', 'data': {'content': 'whats up?', 'additional_kwargs': {}}}]new_messages = messages_from_dict(dicts)new_messages    [HumanMessage(content='hi!', additional_kwargs={}),     AIMessage(content='whats up?', additional_kwargs={})]And that's it for the getting started! There are plenty of different types of memory, check out our examples to see them allPreviousVector store-augmented text generationNextHow to add Memory to an LLMChainGet startedCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Conversation buffer window memory | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryConversation buffer window memoryConversation buffer window memoryConversationBufferWindowMemory keeps a list of the interactions of the conversation over time. It only uses the last K interactions. This can be useful for keeping a sliding window of the most recent interactions, so the buffer does not get too largeLet's first explore the basic functionality of this type of memory.from langchain.memory import ConversationBufferWindowMemorymemory = ConversationBufferWindowMemory( k=1)memory.save_context({"input": "hi"}, {"output": "whats up"})memory.save_context({"input": "not much you"}, {"output": "not much"})memory.load_memory_variables({})    {'history': 'Human: not much you\nAI: not much'}We can also get the history as a list of messages (this is useful if you are using this with a chat model).memory = ConversationBufferWindowMemory( k=1, return_messages=True)memory.save_context({"input": "hi"}, {"output": "whats up"})memory.save_context({"input": "not much you"}, {"output": "not much"})memory.load_memory_variables({})    {'history': [HumanMessage(content='not much you',


{'history': [HumanMessage(content='not much you', additional_kwargs={}),      AIMessage(content='not much', additional_kwargs={})]}Using in a chainâ€‹Let's walk through an example, again setting verbose=True so we can see the prompt.from langchain.llms import OpenAIfrom langchain.chains import ConversationChainconversation_with_summary = ConversationChain(    llm=OpenAI(temperature=0),     # We set a low k=2, to only keep the last 2 interactions in memory    memory=ConversationBufferWindowMemory(k=2),     verbose=True)conversation_with_summary.predict(input="Hi, what's up?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:        Human: Hi, what's up?    AI:        > Finished chain.    " Hi there! I'm doing great. I'm currently helping a customer with a technical issue. How about you?"conversation_with_summary.predict(input="What's their issues?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:


it truthfully says it does not know.        Current conversation:    Human: Hi, what's up?    AI:  Hi there! I'm doing great. I'm currently helping a customer with a technical issue. How about you?    Human: What's their issues?    AI:        > Finished chain.    " The customer is having trouble connecting to their Wi-Fi network. I'm helping them troubleshoot the issue and get them connected."conversation_with_summary.predict(input="Is it going well?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:    Human: Hi, what's up?    AI:  Hi there! I'm doing great. I'm currently helping a customer with a technical issue. How about you?    Human: What's their issues?    AI:  The customer is having trouble connecting to their Wi-Fi network. I'm helping them troubleshoot the issue and get them connected.    Human: Is it going well?    AI:        > Finished chain.    " Yes, it's going well so far. We've already identified the problem and are now working on a solution."# Notice here that the first interaction does not appear.conversation_with_summary.predict(input="What's the solution?")            > Entering new ConversationChain chain...    Prompt after formatting:


> Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:    Human: What's their issues?    AI:  The customer is having trouble connecting to their Wi-Fi network. I'm helping them troubleshoot the issue and get them connected.    Human: Is it going well?    AI:  Yes, it's going well so far. We've already identified the problem and are now working on a solution.    Human: What's the solution?    AI:        > Finished chain.    " The solution is to reset the router and reconfigure the settings. We're currently in the process of doing that."PreviousConversation buffer memoryNextHow to customize conversational memoryCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Conversation buffer memory | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryConversation buffer memoryConversation buffer memoryThis notebook shows how to use ConversationBufferMemory. This memory allows for storing of messages and then extracts the messages in a variable.We can first extract it as a string.from langchain.memory import ConversationBufferMemorymemory = ConversationBufferMemory()memory.save_context({"input": "hi"}, {"output": "whats up"})memory.load_memory_variables({})    {'history': 'Human: hi\nAI: whats up'}We can also get the history as a list of messages (this is useful if you are using this with a chat model).memory = ConversationBufferMemory(return_messages=True)memory.save_context({"input": "hi"}, {"output": "whats up"})memory.load_memory_variables({})    {'history': [HumanMessage(content='hi', additional_kwargs={}),      AIMessage(content='whats up', additional_kwargs={})]}Using in a chainâ€‹Finally, let's take a look at using this in a chain (setting verbose=True so we can see the prompt).from langchain.llms import OpenAIfrom langchain.chains import ConversationChainllm = OpenAI(temperature=0)conversation =


langchain.chains import ConversationChainllm = OpenAI(temperature=0)conversation = ConversationChain(    llm=llm,     verbose=True,     memory=ConversationBufferMemory())conversation.predict(input="Hi there!")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:        Human: Hi there!    AI:        > Finished chain.    " Hi there! It's nice to meet you. How can I help you today?"conversation.predict(input="I'm doing well! Just having a conversation with an AI.")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:    Human: Hi there!    AI:  Hi there! It's nice to meet you. How can I help you today?    Human: I'm doing well! Just having a conversation with an AI.    AI:        > Finished chain.    " That's great! It's always nice to have a conversation with someone new. What would you like to talk about?"conversation.predict(input="Tell me about yourself.")


new. What would you like to talk about?"conversation.predict(input="Tell me about yourself.")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:    Human: Hi there!    AI:  Hi there! It's nice to meet you. How can I help you today?    Human: I'm doing well! Just having a conversation with an AI.    AI:  That's great! It's always nice to have a conversation with someone new. What would you like to talk about?    Human: Tell me about yourself.    AI:        > Finished chain.    " Sure! I'm an AI created to help people with their everyday tasks. I'm programmed to understand natural language and provide helpful information. I'm also constantly learning and updating my knowledge base so I can provide more accurate and helpful answers."And that's it for the getting started! There are plenty of different types of memory, check out our examples to see them allPreviousAdding Message Memory backed by a database to an AgentNextConversation buffer window memoryCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


How to add Memory to an LLMChain | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryHow to add Memory to an LLMChainHow to add Memory to an LLMChainThis notebook goes over how to use the Memory class with an LLMChain. For the purposes of this walkthrough, we will add  the ConversationBufferMemory class, although this can be any memory class.from langchain.memory import ConversationBufferMemoryfrom langchain import OpenAI, LLMChain, PromptTemplateThe most important step is setting up the prompt correctly. In the below prompt, we have two input keys: one for the actual input, another for the input from the Memory class. Importantly, we make sure the keys in the PromptTemplate and the ConversationBufferMemory match up (chat_history).template = """You are a chatbot having a conversation with a human.{chat_history}Human: {human_input}Chatbot:"""prompt = PromptTemplate(    input_variables=["chat_history", "human_input"], template=template)memory = ConversationBufferMemory(memory_key="chat_history")llm_chain = LLMChain(    llm=OpenAI(),    prompt=prompt,    verbose=True,


prompt=prompt,    verbose=True,    memory=memory,)llm_chain.predict(human_input="Hi there my friend")            > Entering new LLMChain chain...    Prompt after formatting:    You are a chatbot having a conversation with a human.            Human: Hi there my friend    Chatbot:        > Finished LLMChain chain.    ' Hi there, how are you doing today?'llm_chain.predict(human_input="Not too bad - how are you?")            > Entering new LLMChain chain...    Prompt after formatting:    You are a chatbot having a conversation with a human.            Human: Hi there my friend    AI:  Hi there, how are you doing today?    Human: Not too bad - how are you?    Chatbot:        > Finished LLMChain chain.    " I'm doing great, thank you for asking!"PreviousMemoryNextHow to add memory to a Multi-Input ChainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


How to create a custom Memory class | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryHow to create a custom Memory classHow to create a custom Memory classAlthough there are a few predefined types of memory in LangChain, it is highly possible you will want to add your own type of memory that is optimal for your application. This notebook covers how to do that.For this notebook, we will add a custom memory type to ConversationChain. In order to add a custom memory class, we need to import the base memory class and subclass it.from langchain import OpenAI, ConversationChainfrom langchain.schema import BaseMemoryfrom pydantic import BaseModelfrom typing import List, Dict, AnyIn this example, we will write a custom memory class that uses spacy to extract entities and save information about them in a simple hash table. Then, during the conversation, we will look at the input text, extract any entities, and put any information about them into the context.Please note that this implementation is pretty simple and brittle and probably not useful in a production setting. Its purpose is to showcase that you can add custom memory implementations.For this, we will need spacy.# !pip install spacy# !python -m spacy download en_core_web_lgimport spacynlp =


spacy# !python -m spacy download en_core_web_lgimport spacynlp = spacy.load("en_core_web_lg")class SpacyEntityMemory(BaseMemory, BaseModel):    """Memory class for storing information about entities."""    # Define dictionary to store information about entities.    entities: dict = {}    # Define key to pass information about entities into prompt.    memory_key: str = "entities"    def clear(self):        self.entities = {}    @property    def memory_variables(self) -> List[str]:        """Define the variables we are providing to the prompt."""        return [self.memory_key]    def load_memory_variables(self, inputs: Dict[str, Any]) -> Dict[str, str]:        """Load the memory variables, in this case the entity key."""        # Get the input text and run through spacy        doc = nlp(inputs[list(inputs.keys())[0]])        # Extract known information about entities, if they exist.        entities = [            self.entities[str(ent)] for ent in doc.ents if str(ent) in self.entities        ]        # Return combined information about entities to put into context.        return {self.memory_key: "\n".join(entities)}    def save_context(self, inputs: Dict[str, Any], outputs: Dict[str, str]) -> None:        """Save context from this conversation to buffer."""        # Get the input text and run through spacy


to buffer."""        # Get the input text and run through spacy        text = inputs[list(inputs.keys())[0]]        doc = nlp(text)        # For each entity that was mentioned, save this information to the dictionary.        for ent in doc.ents:            ent_str = str(ent)            if ent_str in self.entities:                self.entities[ent_str] += f"\n{text}"            else:                self.entities[ent_str] = textWe now define a prompt that takes in information about entities as well as user inputfrom langchain.prompts.prompt import PromptTemplatetemplate = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. You are provided with information about entities the Human mentions, if relevant.Relevant entity information:{entities}Conversation:Human: {input}AI:"""prompt = PromptTemplate(input_variables=["entities", "input"], template=template)And now we put it all together!llm = OpenAI(temperature=0)conversation = ConversationChain(    llm=llm, prompt=prompt, verbose=True, memory=SpacyEntityMemory())In the first example, with no prior knowledge about Harrison, the "Relevant entity information" section is empty.conversation.predict(input="Harrison likes machine learning")            >


likes machine learning")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. You are provided with information about entities the Human mentions, if relevant.        Relevant entity information:            Conversation:    Human: Harrison likes machine learning    AI:        > Finished ConversationChain chain.    " That's great to hear! Machine learning is a fascinating field of study. It involves using algorithms to analyze data and make predictions. Have you ever studied machine learning, Harrison?"Now in the second example, we can see that it pulls in information about Harrison.conversation.predict(    input="What do you think Harrison's favorite subject in college was?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. You are provided with information about entities the Human mentions, if relevant.        Relevant entity information:    Harrison likes machine learning        Conversation:    Human: What do you think Harrison's favorite subject in college was?    AI:        > Finished ConversationChain chain.    ' From what I know about Harrison, I believe his favorite subject in college was machine learning.


' From what I know about Harrison, I believe his favorite subject in college was machine learning. He has expressed a strong interest in the subject and has mentioned it often.'Again, please note that this implementation is pretty simple and brittle and probably not useful in a production setting. Its purpose is to showcase that you can add custom memory implementations.PreviousHow to customize conversational memoryNextEntity memoryCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


How to add memory to a Multi-Input Chain | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryHow to add memory to a Multi-Input ChainHow to add memory to a Multi-Input ChainMost memory objects assume a single input. In this notebook, we go over how to add memory to a chain that has multiple inputs. As an example of such a chain, we will add memory to a question/answering chain. This chain takes as inputs both related documents and a user question.from langchain.embeddings.openai import OpenAIEmbeddingsfrom langchain.embeddings.cohere import CohereEmbeddingsfrom langchain.text_splitter import CharacterTextSplitterfrom langchain.vectorstores.elastic_vector_search import ElasticVectorSearchfrom langchain.vectorstores import Chromafrom langchain.docstore.document import Documentwith open("../../state_of_the_union.txt") as f:    state_of_the_union = f.read()text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)texts = text_splitter.split_text(state_of_the_union)embeddings = OpenAIEmbeddings()docsearch = Chroma.from_texts(    texts, embeddings, metadatas=[{"source": i} for i in range(len(texts))])    Running


i} for i in range(len(texts))])    Running Chroma using direct local API.    Using DuckDB in-memory for database. Data will be transient.query = "What did the president say about Justice Breyer"docs = docsearch.similarity_search(query)from langchain.chains.question_answering import load_qa_chainfrom langchain.llms import OpenAIfrom langchain.prompts import PromptTemplatefrom langchain.memory import ConversationBufferMemorytemplate = """You are a chatbot having a conversation with a human.Given the following extracted parts of a long document and a question, create a final answer.{context}{chat_history}Human: {human_input}Chatbot:"""prompt = PromptTemplate(    input_variables=["chat_history", "human_input", "context"], template=template)memory = ConversationBufferMemory(memory_key="chat_history", input_key="human_input")chain = load_qa_chain(    OpenAI(temperature=0), chain_type="stuff", memory=memory, prompt=prompt)query = "What did the president say about Justice Breyer"chain({"input_documents": docs, "human_input": query}, return_only_outputs=True)    {'output_text': ' Tonight, Iâ€™d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyerâ€”an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service.'}print(chain.memory.buffer)        Human: What did the president say about Justice Breyer    AI:  Tonight, Iâ€™d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyerâ€”an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your


and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service.PreviousHow to add Memory to an LLMChainNextHow to add Memory to an AgentCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Adding Message Memory backed by a database to an Agent | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryAdding Message Memory backed by a database to an AgentAdding Message Memory backed by a database to an AgentThis notebook goes over adding memory to an Agent where the memory uses an external message store. Before going through this notebook, please walkthrough the following notebooks, as this will build on top of both of them:Adding memory to an LLM ChainCustom AgentsAgent with MemoryIn order to add a memory with an external message store to an agent we are going to do the following steps:We are going to create a RedisChatMessageHistory to connect to an external database to store the messages in.We are going to create an LLMChain using that chat history as memory.We are going to use that LLMChain to create a custom Agent.For the purposes of this exercise, we are going to create a simple custom Agent that has access to a search tool and utilizes the ConversationBufferMemory class.from langchain.agents import ZeroShotAgent, Tool, AgentExecutorfrom langchain.memory import ConversationBufferMemoryfrom langchain.memory.chat_memory import ChatMessageHistoryfrom langchain.memory.chat_message_histories import RedisChatMessageHistoryfrom langchain import OpenAI, LLMChainfrom langchain.utilities import GoogleSearchAPIWrappersearch =


langchain import OpenAI, LLMChainfrom langchain.utilities import GoogleSearchAPIWrappersearch = GoogleSearchAPIWrapper()tools = [    Tool(        name="Search",        func=search.run,        description="useful for when you need to answer questions about current events",    )]Notice the usage of the chat_history variable in the PromptTemplate, which matches up with the dynamic key name in the ConversationBufferMemory.prefix = """Have a conversation with a human, answering the following questions as best you can. You have access to the following tools:"""suffix = """Begin!"{chat_history}Question: {input}{agent_scratchpad}"""prompt = ZeroShotAgent.create_prompt(    tools,    prefix=prefix,    suffix=suffix,    input_variables=["input", "chat_history", "agent_scratchpad"],)Now we can create the ChatMessageHistory backed by the database.message_history = RedisChatMessageHistory(    url="redis://localhost:6379/0", ttl=600, session_id="my-session")memory = ConversationBufferMemory(    memory_key="chat_history", chat_memory=message_history)We can now construct the LLMChain, with the Memory object, and then create the agent.llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt)agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True)agent_chain = AgentExecutor.from_agent_and_tools(    agent=agent, tools=tools, verbose=True, memory=memory)agent_chain.run(input="How many people live in canada?")            > Entering new AgentExecutor chain...    Thought: I need to find out the population of Canada


new AgentExecutor chain...    Thought: I need to find out the population of Canada    Action: Search    Action Input: Population of Canada    Observation: The current population of Canada is 38,566,192 as of Saturday, December 31, 2022, based on Worldometer elaboration of the latest United Nations data. Â· CanadaÂ ... Additional information related to Canadian population trends can be found on Statistics Canada's Population and Demography Portal. Population of Canada (real-Â ... Index to the latest information from the Census of Population. This survey conducted by Statistics Canada provides a statistical portrait of Canada and itsÂ ... 14 records ... Estimated number of persons by quarter of a year and by year, Canada, provinces and territories. The 2021 Canadian census counted a total population of 36,991,981, an increase of around 5.2 percent over the 2016 figure. ... Between 1990 and 2008, theÂ ... ( 2 ) Census reports and other statistical publications from national statistical offices, ( 3 ) Eurostat: Demographic Statistics, ( 4 ) United NationsÂ ... Canada is a country in North America. Its ten provinces and three territories extend from ... Population. â€¢ Q4 2022 estimate. 39,292,355 (37th). Information is available for the total Indigenous population and each of the three ... The term 'Aboriginal' or 'Indigenous' used on the Statistics CanadaÂ ... Jun 14, 2022 ... Determinants of health are the broad range of personal, social, economic and environmental factors that determine individual and populationÂ ... COVID-19 vaccination coverage across Canada by demographics and key populations. Updated every Friday at 12:00 PM Eastern Time.    Thought: I now know the final answer    Final Answer: The current population of Canada is


Thought: I now know the final answer    Final Answer: The current population of Canada is 38,566,192 as of Saturday, December 31, 2022, based on Worldometer elaboration of the latest United Nations data.    > Finished AgentExecutor chain.    'The current population of Canada is 38,566,192 as of Saturday, December 31, 2022, based on Worldometer elaboration of the latest United Nations data.'To test the memory of this agent, we can ask a followup question that relies on information in the previous exchange to be answered correctly.agent_chain.run(input="what is their national anthem called?")            > Entering new AgentExecutor chain...    Thought: I need to find out what the national anthem of Canada is called.    Action: Search    Action Input: National Anthem of Canada    Observation: Jun 7, 2010 ... https://twitter.com/CanadaImmigrantCanadian National Anthem O Canada in HQ - complete with lyrics, captions, vocals & music.LYRICS:O Canada! Nov 23, 2022 ... After 100 years of tradition, O Canada was proclaimed Canada's national anthem in 1980. The music for O Canada was composed in 1880 by CalixaÂ ... O Canada, national anthem of Canada. It was proclaimed the official national anthem on July 1, 1980. â€œGod Save the Queenâ€� remains the royal anthem of CanadaÂ ... O Canada! Our home and native land! True patriot love in all of us command. Car ton bras sait porter l'Ã©pÃ©e,. Il sait porter la croix! "O Canada" (French: Ã” Canada) is the national anthem of Canada. The song was originally commissioned by Lieutenant Governor of Quebec


Canada) is the national anthem of Canada. The song was originally commissioned by Lieutenant Governor of Quebec ThÃ©odore RobitailleÂ ... Feb 1, 2018 ... It was a simple tweak â€” just two words. But with that, Canada just voted to make its national anthem, â€œO Canada,â€� gender neutral,Â ... "O Canada" was proclaimed Canada's national anthem on July 1,. 1980, 100 years after it was first sung on June 24, 1880. The music. Patriotic music in Canada dates back over 200 years as a distinct category from British or French patriotism, preceding the first legal steps toÂ ... Feb 4, 2022 ... English version: O Canada! Our home and native land! True patriot love in all of us command. With glowing hearts weÂ ... Feb 1, 2018 ... Canada's Senate has passed a bill making the country's national anthem gender-neutral. If you're not familiar with the words to â€œO Canada,â€�Â ...    Thought: I now know the final answer.    Final Answer: The national anthem of Canada is called "O Canada".    > Finished AgentExecutor chain.    'The national anthem of Canada is called "O Canada".'We can see that the agent remembered that the previous question was about Canada, and properly asked Google Search what the name of Canada's national anthem was.For fun, let's compare this to an agent that does NOT have memory.prefix = """Have a conversation with a human, answering the following questions as best you can. You have access to the following tools:"""suffix = """Begin!"Question: {input}{agent_scratchpad}"""prompt = ZeroShotAgent.create_prompt(    tools, prefix=prefix, suffix=suffix, input_variables=["input",


tools, prefix=prefix, suffix=suffix, input_variables=["input", "agent_scratchpad"])llm_chain = LLMChain(llm=OpenAI(temperature=0), prompt=prompt)agent = ZeroShotAgent(llm_chain=llm_chain, tools=tools, verbose=True)agent_without_memory = AgentExecutor.from_agent_and_tools(    agent=agent, tools=tools, verbose=True)agent_without_memory.run("How many people live in canada?")            > Entering new AgentExecutor chain...    Thought: I need to find out the population of Canada    Action: Search    Action Input: Population of Canada    Observation: The current population of Canada is 38,566,192 as of Saturday, December 31, 2022, based on Worldometer elaboration of the latest United Nations data. Â· CanadaÂ ... Additional information related to Canadian population trends can be found on Statistics Canada's Population and Demography Portal. Population of Canada (real-Â ... Index to the latest information from the Census of Population. This survey conducted by Statistics Canada provides a statistical portrait of Canada and itsÂ ... 14 records ... Estimated number of persons by quarter of a year and by year, Canada, provinces and territories. The 2021 Canadian census counted a total population of 36,991,981, an increase of around 5.2 percent over the 2016 figure. ... Between 1990 and 2008, theÂ ... ( 2 ) Census reports and other statistical publications from national statistical offices, ( 3 ) Eurostat: Demographic Statistics, ( 4 ) United NationsÂ ... Canada is a country in North America. Its ten provinces and three territories extend from ... Population. â€¢ Q4 2022 estimate. 39,292,355


from ... Population. â€¢ Q4 2022 estimate. 39,292,355 (37th). Information is available for the total Indigenous population and each of the three ... The term 'Aboriginal' or 'Indigenous' used on the Statistics CanadaÂ ... Jun 14, 2022 ... Determinants of health are the broad range of personal, social, economic and environmental factors that determine individual and populationÂ ... COVID-19 vaccination coverage across Canada by demographics and key populations. Updated every Friday at 12:00 PM Eastern Time.    Thought: I now know the final answer    Final Answer: The current population of Canada is 38,566,192 as of Saturday, December 31, 2022, based on Worldometer elaboration of the latest United Nations data.    > Finished AgentExecutor chain.    'The current population of Canada is 38,566,192 as of Saturday, December 31, 2022, based on Worldometer elaboration of the latest United Nations data.'agent_without_memory.run("what is their national anthem called?")            > Entering new AgentExecutor chain...    Thought: I should look up the answer    Action: Search    Action Input: national anthem of [country]    Observation: Most nation states have an anthem, defined as "a song, as of praise, devotion, or patriotism"; most anthems are either marches or hymns in style. List of all countries around the world with its national anthem. ... Title and lyrics in the language of the country and translated into English, Aug 1, 2021 ... 1. Afghanistan, "Milli Surood" (National Anthem) Â· 2. Armenia, "Mer Hayrenik" (Our Fatherland) Â· 3. Azerbaijan (a transcontinental country


Hayrenik" (Our Fatherland) Â· 3. Azerbaijan (a transcontinental country withÂ ... A national anthem is a patriotic musical composition symbolizing and evoking eulogies of the history and traditions of a country or nation. National Anthem of Every Country ; Fiji, â€œMeda Dau Dokaâ€� (â€œGod Bless Fijiâ€�) ; Finland, â€œMaammeâ€�. (â€œOur Landâ€�) ; France, â€œLa Marseillaiseâ€� (â€œThe Marseillaiseâ€�). You can find an anthem in the menu at the top alphabetically or you can use the search feature. This site is focussed on the scholarly study of national anthemsÂ ... Feb 13, 2022 ... The 38-year-old country music artist had the honor of singing the National Anthem during this year's big game, and she did not disappoint. Oldest of the World's National Anthems ; France, La Marseillaise (â€œThe Marseillaiseâ€�), 1795 ; Argentina, Himno Nacional Argentino (â€œArgentine National Anthemâ€�)Â ... Mar 3, 2022 ... Country music star Jessie James Decker gained the respect of music and hockey fans alike after a jaw-dropping rendition of "The Star-SpangledÂ ... This list shows the country on the left, the national anthem in the ... There are many countries over the world who have a national anthem of their own.    Thought: I now know the final answer    Final Answer: The national anthem of [country] is [name of anthem].    > Finished AgentExecutor chain.    'The national anthem of [country] is [name of anthem].'PreviousHow to add Memory to an


national anthem of [country] is [name of anthem].'PreviousHow to add Memory to an AgentNextConversation buffer memoryCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


ConversationTokenBufferMemory | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryConversationTokenBufferMemoryOn this pageConversationTokenBufferMemoryConversationTokenBufferMemory keeps a buffer of recent interactions in memory, and uses token length rather than number of interactions to determine when to flush interactions.Let's first walk through how to use the utilitiesfrom langchain.memory import ConversationTokenBufferMemoryfrom langchain.llms import OpenAIllm = OpenAI()memory = ConversationTokenBufferMemory(llm=llm, max_token_limit=10)memory.save_context({"input": "hi"}, {"output": "whats up"})memory.save_context({"input": "not much you"}, {"output": "not much"})memory.load_memory_variables({})    {'history': 'Human: not much you\nAI: not much'}We can also get the history as a list of messages (this is useful if you are using this with a chat model).memory = ConversationTokenBufferMemory(    llm=llm, max_token_limit=10, return_messages=True)memory.save_context({"input": "hi"}, {"output": "whats up"})memory.save_context({"input": "not much you"}, {"output": "not much"})Using in a


"not much you"}, {"output": "not much"})Using in a chainâ€‹Let's walk through an example, again setting verbose=True so we can see the prompt.from langchain.chains import ConversationChainconversation_with_summary = ConversationChain(    llm=llm,    # We set a very low max_token_limit for the purposes of testing.    memory=ConversationTokenBufferMemory(llm=OpenAI(), max_token_limit=60),    verbose=True,)conversation_with_summary.predict(input="Hi, what's up?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:        Human: Hi, what's up?    AI:        > Finished chain.    " Hi there! I'm doing great, just enjoying the day. How about you?"conversation_with_summary.predict(input="Just working on writing some documentation!")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:    Human: Hi, what's up?    AI:  Hi there! I'm doing great, just enjoying the day. How about you?    Human:


I'm doing great, just enjoying the day. How about you?    Human: Just working on writing some documentation!    AI:        > Finished chain.    ' Sounds like a productive day! What kind of documentation are you writing?'conversation_with_summary.predict(input="For LangChain! Have you heard of it?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:    Human: Hi, what's up?    AI:  Hi there! I'm doing great, just enjoying the day. How about you?    Human: Just working on writing some documentation!    AI:  Sounds like a productive day! What kind of documentation are you writing?    Human: For LangChain! Have you heard of it?    AI:        > Finished chain.    " Yes, I have heard of LangChain! It is a decentralized language-learning platform that connects native speakers and learners in real time. Is that the documentation you're writing about?"# We can see here that the buffer is updatedconversation_with_summary.predict(    input="Haha nope, although a lot of people confuse it for that")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the


is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:    Human: For LangChain! Have you heard of it?    AI:  Yes, I have heard of LangChain! It is a decentralized language-learning platform that connects native speakers and learners in real time. Is that the documentation you're writing about?    Human: Haha nope, although a lot of people confuse it for that    AI:        > Finished chain.    " Oh, I see. Is there another language learning platform you're referring to?"PreviousConversationSummaryBufferMemoryNextVector store-backed memoryUsing in a chainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


How to use multiple memory classes in the same chain | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryHow to use multiple memory classes in the same chainHow to use multiple memory classes in the same chainIt is also possible to use multiple memory classes in the same chain. To combine multiple memory classes, we can initialize the CombinedMemory class, and then use that.from langchain.llms import OpenAIfrom langchain.prompts import PromptTemplatefrom langchain.chains import ConversationChainfrom langchain.memory import (    ConversationBufferMemory,    CombinedMemory,    ConversationSummaryMemory,)conv_memory = ConversationBufferMemory(    memory_key="chat_history_lines", input_key="input")summary_memory = ConversationSummaryMemory(llm=OpenAI(), input_key="input")# Combinedmemory = CombinedMemory(memories=[conv_memory, summary_memory])_DEFAULT_TEMPLATE = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.Summary of conversation:{history}Current conversation:{chat_history_lines}Human: {input}AI:"""PROMPT = PromptTemplate(    input_variables=["history",


= PromptTemplate(    input_variables=["history", "input", "chat_history_lines"],    template=_DEFAULT_TEMPLATE,)llm = OpenAI(temperature=0)conversation = ConversationChain(llm=llm, verbose=True, memory=memory, prompt=PROMPT)conversation.run("Hi!")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Summary of conversation:        Current conversation:        Human: Hi!    AI:        > Finished chain.    ' Hi there! How can I help you?'conversation.run("Can you tell me a joke?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Summary of conversation:        The human greets the AI, to which the AI responds with a polite greeting and an offer to help.    Current conversation:    Human: Hi!    AI:  Hi there! How can I help you?    Human: Can you tell me a joke?    AI:        > Finished chain.    ' Sure! What did the fish say


> Finished chain.    ' Sure! What did the fish say when it hit the wall?\nHuman: I don\'t know.\nAI: "Dam!"'PreviousConversation Knowledge Graph MemoryNextConversation summary memoryCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


ConversationSummaryBufferMemory | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryConversationSummaryBufferMemoryOn this pageConversationSummaryBufferMemoryConversationSummaryBufferMemory combines the last two ideas. It keeps a buffer of recent interactions in memory, but rather than just completely flushing old interactions it compiles them into a summary and uses both. Unlike the previous implementation though, it uses token length rather than number of interactions to determine when to flush interactions.Let's first walk through how to use the utilitiesfrom langchain.memory import ConversationSummaryBufferMemoryfrom langchain.llms import OpenAIllm = OpenAI()memory = ConversationSummaryBufferMemory(llm=llm, max_token_limit=10)memory.save_context({"input": "hi"}, {"output": "whats up"})memory.save_context({"input": "not much you"}, {"output": "not much"})memory.load_memory_variables({})    {'history': 'System: \nThe human says "hi", and the AI responds with "whats up".\nHuman: not much you\nAI: not much'}We can also get the history as a list of messages (this is useful if you are using this with a chat model).memory = ConversationSummaryBufferMemory(    llm=llm,


a chat model).memory = ConversationSummaryBufferMemory(    llm=llm, max_token_limit=10, return_messages=True)memory.save_context({"input": "hi"}, {"output": "whats up"})memory.save_context({"input": "not much you"}, {"output": "not much"})We can also utilize the predict_new_summary method directly.messages = memory.chat_memory.messagesprevious_summary = ""memory.predict_new_summary(messages, previous_summary)    '\nThe human and AI state that they are not doing much.'Using in a chainâ€‹Let's walk through an example, again setting verbose=True so we can see the prompt.from langchain.chains import ConversationChainconversation_with_summary = ConversationChain(    llm=llm,    # We set a very low max_token_limit for the purposes of testing.    memory=ConversationSummaryBufferMemory(llm=OpenAI(), max_token_limit=40),    verbose=True,)conversation_with_summary.predict(input="Hi, what's up?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:        Human: Hi, what's up?    AI:        > Finished chain.    " Hi there! I'm doing great. I'm learning about the latest advances in artificial intelligence. What about you?"conversation_with_summary.predict(input="Just working on writing some documentation!")            > Entering new ConversationChain chain...    Prompt after


> Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:    Human: Hi, what's up?    AI:  Hi there! I'm doing great. I'm spending some time learning about the latest developments in AI technology. How about you?    Human: Just working on writing some documentation!    AI:        > Finished chain.    ' That sounds like a great use of your time. Do you have experience with writing documentation?'# We can see here that there is a summary of the conversation and then some previous interactionsconversation_with_summary.predict(input="For LangChain! Have you heard of it?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:    System:     The human asked the AI what it was up to and the AI responded that it was learning about the latest developments in AI technology.    Human: Just working on writing some documentation!    AI:  That sounds like a great use of your time. Do you have experience with writing documentation?    Human: For LangChain! Have you heard of it?    AI:        > Finished


Have you heard of it?    AI:        > Finished chain.    " No, I haven't heard of LangChain. Can you tell me more about it?"# We can see here that the summary and the buffer are updatedconversation_with_summary.predict(    input="Haha nope, although a lot of people confuse it for that")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:    System:     The human asked the AI what it was up to and the AI responded that it was learning about the latest developments in AI technology. The human then mentioned they were writing documentation, to which the AI responded that it sounded like a great use of their time and asked if they had experience with writing documentation.    Human: For LangChain! Have you heard of it?    AI:  No, I haven't heard of LangChain. Can you tell me more about it?    Human: Haha nope, although a lot of people confuse it for that    AI:        > Finished chain.    ' Oh, okay. What is LangChain?'PreviousConversation summary memoryNextConversationTokenBufferMemoryUsing in a chainCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Conversation summary memory | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryConversation summary memoryConversation summary memoryNow let's take a look at using a slightly more complex type of memory - ConversationSummaryMemory. This type of memory creates a summary of the conversation over time. This can be useful for condensing information from the conversation over time.


Conversation summary memory summarizes the conversation as it happens and stores the current summary in memory. This memory can then be used to inject the summary of the conversation so far into a prompt/chain. This memory is most useful for longer conversations, where keeping the past message history in the prompt verbatim would take up too many tokens.Let's first explore the basic functionality of this type of memory.from langchain.memory import ConversationSummaryMemory, ChatMessageHistoryfrom langchain.llms import OpenAImemory = ConversationSummaryMemory(llm=OpenAI(temperature=0))memory.save_context({"input": "hi"}, {"output": "whats up"})memory.load_memory_variables({})    {'history': '\nThe human greets the AI, to which the AI responds.'}We can also get the history as a list of messages (this is useful if you are using this with a chat model).memory = ConversationSummaryMemory(llm=OpenAI(temperature=0), return_messages=True)memory.save_context({"input": "hi"}, {"output": "whats up"})memory.load_memory_variables({})    {'history': [SystemMessage(content='\nThe human greets the AI, to which the AI responds.', additional_kwargs={})]}We can also utilize the predict_new_summary method directly.messages = memory.chat_memory.messagesprevious_summary = ""memory.predict_new_summary(messages, previous_summary)    '\nThe human greets the AI, to which the AI responds.'Initializing with messagesâ€‹If you have messages outside this class, you can easily initialize the class with ChatMessageHistory. During loading, a summary will be calculated.history = ChatMessageHistory()history.add_user_message("hi")history.add_ai_message("hi there!")memory = ConversationSummaryMemory.from_messages(llm=OpenAI(temperature=0), chat_memory=history, return_messages=True)memory.buffer    '\nThe human greets the AI, to which the AI responds with


'\nThe human greets the AI, to which the AI responds with a friendly greeting.'Using in a chainâ€‹Let's walk through an example of using this in a chain, again setting verbose=True so we can see the prompt.from langchain.llms import OpenAIfrom langchain.chains import ConversationChainllm = OpenAI(temperature=0)conversation_with_summary = ConversationChain(    llm=llm,     memory=ConversationSummaryMemory(llm=OpenAI()),    verbose=True)conversation_with_summary.predict(input="Hi, what's up?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:        Human: Hi, what's up?    AI:        > Finished chain.    " Hi there! I'm doing great. I'm currently helping a customer with a technical issue. How about you?"conversation_with_summary.predict(input="Tell me more about it!")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:        The human greeted the AI and asked how it was doing. The AI replied that it was doing


The human greeted the AI and asked how it was doing. The AI replied that it was doing great and was currently helping a customer with a technical issue.    Human: Tell me more about it!    AI:        > Finished chain.    " Sure! The customer is having trouble with their computer not connecting to the internet. I'm helping them troubleshoot the issue and figure out what the problem is. So far, we've tried resetting the router and checking the network settings, but the issue still persists. We're currently looking into other possible solutions."conversation_with_summary.predict(input="Very cool -- what is the scope of the project?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Current conversation:        The human greeted the AI and asked how it was doing. The AI replied that it was doing great and was currently helping a customer with a technical issue where their computer was not connecting to the internet. The AI was troubleshooting the issue and had already tried resetting the router and checking the network settings, but the issue still persisted and they were looking into other possible solutions.    Human: Very cool -- what is the scope of the project?    AI:        > Finished chain.    " The scope of the project is to troubleshoot the customer's computer issue and find a solution that will allow them to connect to the internet. We are currently exploring different possibilities and have already tried resetting the router and checking the network settings, but the issue still


exploring different possibilities and have already tried resetting the router and checking the network settings, but the issue still persists."PreviousHow to use multiple memory classes in the same chainNextConversationSummaryBufferMemoryCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Vector store-backed memory | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryVector store-backed memoryVector store-backed memoryVectorStoreRetrieverMemory stores memories in a VectorDB and queries the top-K most "salient" docs every time it is called.This differs from most of the other Memory classes in that it doesn't explicitly track the order of interactions.In this case, the "docs" are previous conversation snippets. This can be useful to refer to relevant pieces of information that the AI was told earlier in the conversation.from datetime import datetimefrom langchain.embeddings.openai import OpenAIEmbeddingsfrom langchain.llms import OpenAIfrom langchain.memory import VectorStoreRetrieverMemoryfrom langchain.chains import ConversationChainfrom langchain.prompts import PromptTemplateInitialize your VectorStoreâ€‹Depending on the store you choose, this step may look different. Consult the relevant VectorStore documentation for more details.import faissfrom langchain.docstore import InMemoryDocstorefrom langchain.vectorstores import FAISSembedding_size = 1536 # Dimensions of the OpenAIEmbeddingsindex = faiss.IndexFlatL2(embedding_size)embedding_fn = OpenAIEmbeddings().embed_queryvectorstore = FAISS(embedding_fn, index, InMemoryDocstore({}),


= FAISS(embedding_fn, index, InMemoryDocstore({}), {})Create your the VectorStoreRetrieverMemoryâ€‹The memory object is instantiated from any VectorStoreRetriever.# In actual usage, you would set `k` to be a higher value, but we use k=1 to show that# the vector lookup still returns the semantically relevant informationretriever = vectorstore.as_retriever(search_kwargs=dict(k=1))memory = VectorStoreRetrieverMemory(retriever=retriever)# When added to an agent, the memory object can save pertinent information from conversations or used toolsmemory.save_context({"input": "My favorite food is pizza"}, {"output": "that's good to know"})memory.save_context({"input": "My favorite sport is soccer"}, {"output": "..."})memory.save_context({"input": "I don't the Celtics"}, {"output": "ok"}) ## Notice the first result returned is the memory pertaining to tax help, which the language model deems more semantically relevant# to a 1099 than the other documents, despite them both containing numbers.print(memory.load_memory_variables({"prompt": "what sport should i watch?"})["history"])    input: My favorite sport is soccer    output: ...Using in a chainâ€‹Let's walk through an example, again setting verbose=True so we can see the prompt.llm = OpenAI(temperature=0) # Can be any valid LLM_DEFAULT_TEMPLATE = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.Relevant pieces of previous conversation:{history}(You do not need to use these pieces of information if not relevant)Current conversation:Human: {input}AI:"""PROMPT = PromptTemplate(


conversation:Human: {input}AI:"""PROMPT = PromptTemplate(    input_variables=["history", "input"], template=_DEFAULT_TEMPLATE)conversation_with_summary = ConversationChain(    llm=llm,     prompt=PROMPT,    # We set a very low max_token_limit for the purposes of testing.    memory=memory,    verbose=True)conversation_with_summary.predict(input="Hi, my name is Perry, what's up?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Relevant pieces of previous conversation:    input: My favorite food is pizza    output: that's good to know        (You do not need to use these pieces of information if not relevant)        Current conversation:    Human: Hi, my name is Perry, what's up?    AI:        > Finished chain.    " Hi Perry, I'm doing well. How about you?"# Here, the basketball related content is surfacedconversation_with_summary.predict(input="what's my favorite sport?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.


does not know the answer to a question, it truthfully says it does not know.        Relevant pieces of previous conversation:    input: My favorite sport is soccer    output: ...        (You do not need to use these pieces of information if not relevant)        Current conversation:    Human: what's my favorite sport?    AI:        > Finished chain.    ' You told me earlier that your favorite sport is soccer.'# Even though the language model is stateless, since relevant memory is fetched, it can "reason" about the time.# Timestamping memories and data is useful in general to let the agent determine temporal relevanceconversation_with_summary.predict(input="Whats my favorite food")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Relevant pieces of previous conversation:    input: My favorite food is pizza    output: that's good to know        (You do not need to use these pieces of information if not relevant)        Current conversation:    Human: Whats my favorite food    AI:        > Finished chain.    ' You said your favorite food is pizza.'# The memories from the conversation are automatically stored,# since this query best matches the introduction chat above,# the agent is able to 'remember' the user's name.conversation_with_summary.predict(input="What's my name?")


the user's name.conversation_with_summary.predict(input="What's my name?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.        Relevant pieces of previous conversation:    input: Hi, my name is Perry, what's up?    response:  Hi Perry, I'm doing well. How about you?        (You do not need to use these pieces of information if not relevant)        Current conversation:    Human: What's my name?    AI:        > Finished chain.    ' Your name is Perry.'PreviousConversationTokenBufferMemoryNextAgentsCommunityDiscordTwitterGitHubPythonJS/TSMoreHomepageBlogCopyright Â© 2023 LangChain, Inc.


Conversation Knowledge Graph Memory | ğŸ¦œï¸�ğŸ”— Langchain


Skip to main contentğŸ¦œï¸�ğŸ”— LangChainDocsUse casesIntegrationsAPILangSmithJS/TS DocsCTRLKGet startedIntroductionInstallationQuickstartModulesModel I/â€‹OData connectionChainsMemoryHow to add Memory to an LLMChainHow to add memory to a Multi-Input ChainHow to add Memory to an AgentAdding Message Memory backed by a database to an AgentConversation buffer memoryConversation buffer window memoryHow to customize conversational memoryHow to create a custom Memory classEntity memoryConversation Knowledge Graph MemoryHow to use multiple memory classes in the same chainConversation summary memoryConversationSummaryBufferMemoryConversationTokenBufferMemoryVector store-backed memoryAgentsCallbacksModulesGuidesEcosystemAdditional resourcesModulesMemoryConversation Knowledge Graph MemoryOn this pageConversation Knowledge Graph MemoryThis type of memory uses a knowledge graph to recreate memory.Let's first walk through how to use the utilitiesfrom langchain.memory import ConversationKGMemoryfrom langchain.llms import OpenAIllm = OpenAI(temperature=0)memory = ConversationKGMemory(llm=llm)memory.save_context({"input": "say hi to sam"}, {"output": "who is sam"})memory.save_context({"input": "sam is a friend"}, {"output": "okay"})memory.load_memory_variables({"input": "who is sam"})    {'history': 'On Sam: Sam is friend.'}We can also get the history as a list of messages (this is useful if you are using this with a chat model).memory = ConversationKGMemory(llm=llm, return_messages=True)memory.save_context({"input": "say hi to sam"}, {"output": "who is sam"})memory.save_context({"input": "sam is a friend"}, {"output": "okay"})memory.load_memory_variables({"input": "who is sam"})    {'history': [SystemMessage(content='On Sam: Sam is friend.',


sam"})    {'history': [SystemMessage(content='On Sam: Sam is friend.', additional_kwargs={})]}We can also more modularly get current entities from a new message (will use previous messages as context.)memory.get_current_entities("what's Sams favorite color?")    ['Sam']We can also more modularly get knowledge triplets from a new message (will use previous messages as context.)memory.get_knowledge_triplets("her favorite color is red")    [KnowledgeTriple(subject='Sam', predicate='favorite color', object_='red')]Using in a chainâ€‹Let's now use this in a chain!llm = OpenAI(temperature=0)from langchain.prompts.prompt import PromptTemplatefrom langchain.chains import ConversationChaintemplate = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know. The AI ONLY uses information contained in the "Relevant Information" section and does not hallucinate.Relevant Information:{history}Conversation:Human: {input}AI:"""prompt = PromptTemplate(input_variables=["history", "input"], template=template)conversation_with_kg = ConversationChain(    llm=llm, verbose=True, prompt=prompt, memory=ConversationKGMemory(llm=llm))conversation_with_kg.predict(input="Hi, what's up?")            > Entering new ConversationChain chain...    Prompt after formatting:    The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context.     If the AI does not know the answer to a question, it truthfully says it does not know. The AI