seansullivan/Langchain-Docs huggingface dataset:
## 🦜🛠️ LangSmith

[LangSmith](https://smith.langchain.com/) helps you trace and evaluate your language model applications and intelligent agents to help you move from prototype to production.

Check out the [interactive walkthrough](https://python.langchain.com/docs/langsmith/walkthrough/) to get started.

For more information, please refer to the [LangSmith documentation](https://docs.smith.langchain.com/).

For tutorials and other end-to-end examples demonstrating ways to integrate LangSmith in your workflow, check out the [LangSmith Cookbook](https://github.com/langchain-ai/langsmith-cookbook). Some of the guides therein include:

*   Leveraging user feedback in your JS application ([link](https://github.com/langchain-ai/langsmith-cookbook/blob/main/feedback-examples/nextjs/README.md)).
*   Building an automated feedback pipeline ([link](https://github.com/langchain-ai/langsmith-cookbook/blob/main/feedback-examples/algorithmic-feedback/algorithmic_feedback.ipynb)).
*   How to evaluate and audit your RAG workflows ([link](https://github.com/langchain-ai/langsmith-cookbook/tree/main/testing-examples/qa-correctness)).
*   How to fine-tune an LLM on real usage data ([link](https://github.com/langchain-ai/langsmith-cookbook/blob/main/fine-tuning-examples/export-to-openai/fine-tuning-on-chat-runs.ipynb)).
*   How to use the [LangChain Hub](https://smith.langchain.com/hub) to version your prompts ([link](https://github.com/langchain-ai/langsmith-cookbook/blob/main/hub-examples/retrieval-qa-chain/retrieval-qa.ipynb))

* * *

#### Help us out by providing feedback on this documentation page:
## File logging

LangChain provides the `FileCallbackHandler` to write logs to a file. The `FileCallbackHandler` is similar to the [`StdOutCallbackHandler`](https://python.langchain.com/docs/modules/callbacks/), but instead of printing logs to standard output it writes logs to a file.

We see how to use the `FileCallbackHandler` in this example. Additionally we use the `StdOutCallbackHandler` to print logs to the standard output. It also uses the `loguru` library to log other outputs that are not captured by the handler.

```
from langchain_core.callbacks import FileCallbackHandler, StdOutCallbackHandlerfrom langchain_core.prompts import PromptTemplatefrom langchain_openai import OpenAIfrom loguru import loggerlogfile = "output.log"logger.add(logfile, colorize=True, enqueue=True)handler_1 = FileCallbackHandler(logfile)handler_2 = StdOutCallbackHandler()prompt = PromptTemplate.from_template("1 + {number} = ")model = OpenAI()# this chain will both print to stdout (because verbose=True) and write to 'output.log'# if verbose=False, the FileCallbackHandler will still write to 'output.log'chain = prompt | modelresponse = chain.invoke({"number": 2}, {"callbacks": [handler_1, handler_2]})logger.info(response)
```

```
> Entering new LLMChain chain...Prompt after formatting:1 + 2 = > Finished chain.
```

```
2023-06-01 18:36:38.929 | INFO     | __main__:<module>:20 - 3
```

Now we can open the file `output.log` to see that the output has been captured.

```
%pip install --upgrade --quiet  ansi2html > /dev/null
```

```
from ansi2html import Ansi2HTMLConverterfrom IPython.display import HTML, displaywith open("output.log", "r") as f:    content = f.read()conv = Ansi2HTMLConverter()html = conv.convert(content, full=True)display(HTML(html))
```

\> Entering new LLMChain chain...
Prompt after formatting:
1 + 2 = 
\> Finished chain.
2023-06-01 18:36:38.929 | INFO     | \_\_main\_\_:<module>:20 - 
3
LangChain offers a context manager that allows you to count tokens.

```
import asynciofrom langchain_community.callbacks import get_openai_callbackfrom langchain_openai import OpenAIllm = OpenAI(temperature=0)with get_openai_callback() as cb:    llm.invoke("What is the square root of 4?")total_tokens = cb.total_tokensassert total_tokens > 0with get_openai_callback() as cb:    llm.invoke("What is the square root of 4?")    llm.invoke("What is the square root of 4?")assert cb.total_tokens == total_tokens * 2# You can kick off concurrent runs from within the context managerwith get_openai_callback() as cb:    await asyncio.gather(        *[llm.agenerate(["What is the square root of 4?"]) for _ in range(3)]    )assert cb.total_tokens == total_tokens * 3# The context manager is concurrency safetask = asyncio.create_task(llm.agenerate(["What is the square root of 4?"]))with get_openai_callback() as cb:    await llm.agenerate(["What is the square root of 4?"])await taskassert cb.total_tokens == total_tokens
```
## Agents

The core idea of agents is to use a language model to choose a sequence of actions to take. In chains, a sequence of actions is hardcoded (in code). In agents, a language model is used as a reasoning engine to determine which actions to take and in which order.

## [Quickstart](https://python.langchain.com/docs/modules/agents/quick_start/)[​](#quickstart "Direct link to quickstart")

For a quick start to working with agents, please check out [this getting started guide](https://python.langchain.com/docs/modules/agents/quick_start/). This covers basics like initializing an agent, creating tools, and adding memory.

## [Concepts](https://python.langchain.com/docs/modules/agents/concepts/)[​](#concepts "Direct link to concepts")

There are several key concepts to understand when building agents: Agents, AgentExecutor, Tools, Toolkits. For an in depth explanation, please check out [this conceptual guide](https://python.langchain.com/docs/modules/agents/concepts/)

## [Agent Types](https://python.langchain.com/docs/modules/agents/agent_types/)[​](#agent-types "Direct link to agent-types")

There are many different types of agents to use. For a overview of the different types and when to use them, please check out [this section](https://python.langchain.com/docs/modules/agents/agent_types/).

Agents are only as good as the tools they have. For a comprehensive guide on tools, please see [this section](https://python.langchain.com/docs/modules/tools/).

## How To Guides[​](#how-to-guides "Direct link to How To Guides")

Agents have a lot of related functionality! Check out comprehensive guides including:

*   [Building a custom agent](https://python.langchain.com/docs/modules/agents/how_to/custom_agent/)
*   [Streaming (of both intermediate steps and tokens](https://python.langchain.com/docs/modules/agents/how_to/streaming/)
*   [Building an agent that returns structured output](https://python.langchain.com/docs/modules/agents/how_to/agent_structured/)
*   Lots functionality around using AgentExecutor, including: [using it as an iterator](https://python.langchain.com/docs/modules/agents/how_to/agent_iter/), [handle parsing errors](https://python.langchain.com/docs/modules/agents/how_to/handle_parsing_errors/), [returning intermediate steps](https://python.langchain.com/docs/modules/agents/how_to/intermediate_steps/), [capping the max number of iterations](https://python.langchain.com/docs/modules/agents/how_to/max_iterations/), and [timeouts for agents](https://python.langchain.com/docs/modules/agents/how_to/max_time_limit/)
## Tags

You can add tags to your callbacks by passing a `tags` argument to the `call()`/`run()`/`apply()` methods. This is useful for filtering your logs, e.g. if you want to log all requests made to a specific `LLMChain`, you can add a tag, and then filter your logs by that tag. You can pass tags to both constructor and request callbacks, see the examples above for details. These tags are then passed to the `tags` argument of the "start" callback methods, ie. `on_llm_start`, `on_chat_model_start`, `on_chain_start`, `on_tool_start`.
## viking DB

> [viking DB](https://www.volcengine.com/docs/6459/1163946) is a database that stores, indexes, and manages massive embedding vectors generated by deep neural networks and other machine learning (ML) models.

This notebook shows how to use functionality related to the VikingDB vector database.

To run, you should have a [viking DB instance up and running](https://www.volcengine.com/docs/6459/1165058).

```
!pip install --upgrade volcengine
```

We want to use VikingDBEmbeddings so we have to get the VikingDB API Key.

```
import getpassimport osos.environ["OPENAI_API_KEY"] = getpass.getpass("OpenAI API Key:")
```

```
from langchain.document_loaders import TextLoaderfrom langchain_community.vectorstores.vikingdb import VikingDB, VikingDBConfigfrom langchain_openai import OpenAIEmbeddingsfrom langchain_text_splitters import RecursiveCharacterTextSplitter
```

```
loader = TextLoader("./test.txt")documents = loader.load()text_splitter = RecursiveCharacterTextSplitter(chunk_size=10, chunk_overlap=0)docs = text_splitter.split_documents(documents)embeddings = OpenAIEmbeddings()
```

```
db = VikingDB.from_documents(    docs,    embeddings,    connection_args=VikingDBConfig(        host="host", region="region", ak="ak", sk="sk", scheme="http"    ),    drop_old=True,)
```

```
query = "What did the president say about Ketanji Brown Jackson"docs = db.similarity_search(query)
```

### Compartmentalize the data with viking DB Collections[​](#compartmentalize-the-data-with-viking-db-collections "Direct link to Compartmentalize the data with viking DB Collections")

You can store different unrelated documents in different collections within same viking DB instance to maintain the context

Here’s how you can create a new collection

```
db = VikingDB.from_documents(    docs,    embeddings,    connection_args=VikingDBConfig(        host="host", region="region", ak="ak", sk="sk", scheme="http"    ),    collection_name="collection_1",    drop_old=True,)
```

And here is how you retrieve that stored collection

```
db = VikingDB.from_documents(    embeddings,    connection_args=VikingDBConfig(        host="host", region="region", ak="ak", sk="sk", scheme="http"    ),    collection_name="collection_1",)
```

After retrieval you can go on querying it as usual.
## Composition

This section contains higher-level components that combine other arbitrary systems (e.g. external APIs and services) and/or LangChain primitives together.

A good primer for this section would be reading the sections on [LangChain Expression Language](https://python.langchain.com/docs/expression_language/get_started/) and becoming familiar with constructing sequences via piping and the various primitives offered.

The components covered in this section are:

Tools provide an interface for LLMs and other components to interact with other systems. Examples include Wikipedia, a calculator, and a Python REPL.

## [Agents](https://python.langchain.com/docs/modules/agents/)[​](#agents "Direct link to agents")

Agents use a language model to decide actions to take, often defined by a tool. They require an `executor`, which is the runtime for the agent. The executor is what actually calls the agent, executes the tools it chooses, passes the action outputs back to the agent, and repeats. The agent is responsible for parsing output from the previous results and choosing the next steps.

## [Chains](https://python.langchain.com/docs/modules/chains/)[​](#chains "Direct link to chains")

Building block-style compositions of other primitives and components.
Use document loaders to load data from a source as `Document`'s. A `Document` is a piece of text and associated metadata. For example, there are document loaders for loading a simple `.txt` file, for loading the text contents of any web page, or even for loading a transcript of a YouTube video.

Document loaders provide a "load" method for loading data as documents from a configured source. They optionally implement a "lazy load" as well for lazily loading data into memory.

```
[    Document(page_content='---\nsidebar_position: 0\n---\n# Document loaders\n\nUse document loaders to load data from a source as `Document`\'s. A `Document` is a piece of text\nand associated metadata. For example, there are document loaders for loading a simple `.txt` file, for loading the text\ncontents of any web page, or even for loading a transcript of a YouTube video.\n\nEvery document loader exposes two methods:\n1. "Load": load documents from the configured source\n2. "Load and split": load documents from the configured source and split them using the passed in text splitter\n\nThey optionally implement:\n\n3. "Lazy load": load documents into memory lazily\n', metadata={'source': '../docs/docs/modules/data_connection/document_loaders/index.md'})]
```
## Self-ask with search

This walkthrough showcases the self-ask with search agent.

```
from langchain import hubfrom langchain.agents import AgentExecutor, create_self_ask_with_search_agentfrom langchain_community.llms import Fireworksfrom langchain_community.tools.tavily_search import TavilyAnswer
```

We will initialize the tools we want to use. This is a good tool because it gives us **answers** (not documents)

For this agent, only one tool can be used and it needs to be named “Intermediate Answer”

```
tools = [TavilyAnswer(max_results=1, name="Intermediate Answer")]
```

## Create Agent[​](#create-agent "Direct link to Create Agent")

```
# Get the prompt to use - you can modify this!prompt = hub.pull("hwchase17/self-ask-with-search")
```

```
# Choose the LLM that will drive the agentllm = Fireworks()# Construct the Self Ask With Search Agentagent = create_self_ask_with_search_agent(llm, tools, prompt)
```

## Run Agent[​](#run-agent "Direct link to Run Agent")

```
# Create an agent executor by passing in the agent and toolsagent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
```

```
agent_executor.invoke(    {"input": "What is the hometown of the reigning men's U.S. Open champion?"})
```

```
> Entering new AgentExecutor chain... Yes.Follow up: Who is the reigning men's U.S. Open champion?The reigning men's U.S. Open champion is Novak Djokovic. He won his 24th Grand Slam singles title by defeating Daniil Medvedev in the final of the 2023 U.S. Open.So the final answer is: Novak Djokovic.> Finished chain.
```

```
{'input': "What is the hometown of the reigning men's U.S. Open champion?", 'output': 'Novak Djokovic.'}
```

* * *

#### Help us out by providing feedback on this documentation page:
[Hologres](https://www.alibabacloud.com/help/en/hologres/latest/introduction) is a unified real-time data warehousing service developed by Alibaba Cloud. You can use Hologres to write, update, process, and analyze large amounts of data in real time. Hologres supports standard SQL syntax, is compatible with PostgreSQL, and supports most PostgreSQL functions. Hologres supports online analytical processing (OLAP) and ad hoc analysis for up to petabytes of data, and provides high-concurrency and low-latency online data services.

This notebook shows how to use functionality related to the `Hologres Proxima` vector database. Click [here](https://www.alibabacloud.com/zh/product/hologres) to fast deploy a Hologres cloud instance.

```
from langchain_community.document_loaders import TextLoaderloader = TextLoader("../../modules/state_of_the_union.txt")documents = loader.load()text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)docs = text_splitter.split_documents(documents)embeddings = OpenAIEmbeddings()
```

Connect to Hologres by setting related ENVIRONMENTS.

```
export PG_HOST={host}export PG_PORT={port} # Optional, default is 80export PG_DATABASE={db_name} # Optional, default is postgresexport PG_USER={username}export PG_PASSWORD={password}
```

```
import osconnection_string = Hologres.connection_string_from_db_params(    host=os.environ.get("PGHOST", "localhost"),    port=int(os.environ.get("PGPORT", "80")),    database=os.environ.get("PGDATABASE", "postgres"),    user=os.environ.get("PGUSER", "postgres"),    password=os.environ.get("PGPASSWORD", "postgres"),)vector_db = Hologres.from_documents(    docs,    embeddings,    connection_string=connection_string,    table_name="langchain_example_embeddings",)
```

```
Tonight. I call on the Senate to: Pass the Freedom to Vote Act. Pass the John Lewis Voting Rights Act. And while you’re at it, pass the Disclose Act so Americans can know who is funding our elections. Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service. One of the most serious constitutional responsibilities a President has is nominating someone to serve on the United States Supreme Court. And I did that 4 days ago, when I nominated Circuit Court of Appeals Judge Ketanji Brown Jackson. One of our nation’s top legal minds, who will continue Justice Breyer’s legacy of excellence.
```
## Microsoft Office

> [The Microsoft Office](https://www.office.com/) suite of productivity software includes Microsoft Word, Microsoft Excel, Microsoft PowerPoint, Microsoft Outlook, and Microsoft OneNote. It is available for Microsoft Windows and macOS operating systems. It is also available on Android and iOS.

This covers how to load commonly used file formats including `DOCX`, `XLSX` and `PPTX` documents into a document format that we can use downstream.

## Loading DOCX, XLSX, PPTX with AzureAIDocumentIntelligenceLoader[​](#loading-docx-xlsx-pptx-with-azureaidocumentintelligenceloader "Direct link to Loading DOCX, XLSX, PPTX with AzureAIDocumentIntelligenceLoader")

[Azure AI Document Intelligence](https://aka.ms/doc-intelligence) (formerly known as `Azure Form Recognizer`) is machine-learning based service that extracts texts (including handwriting), tables, document structures (e.g., titles, section headings, etc.) and key-value-pairs from digital or scanned PDFs, images, Office and HTML files. Document Intelligence supports `PDF`, `JPEG/JPG`, `PNG`, `BMP`, `TIFF`, `HEIF`, `DOCX`, `XLSX`, `PPTX` and `HTML`.

This [current implementation](https://aka.ms/di-langchain) of a loader using `Document Intelligence` can incorporate content page-wise and turn it into LangChain documents. The default output format is markdown, which can be easily chained with `MarkdownHeaderTextSplitter` for semantic document chunking. You can also use `mode="single"` or `mode="page"` to return pure texts in a single page or document split by page.

### Prerequisite[​](#prerequisite "Direct link to Prerequisite")

An Azure AI Document Intelligence resource in one of the 3 preview regions: **East US**, **West US2**, **West Europe** - follow [this document](https://learn.microsoft.com/azure/ai-services/document-intelligence/create-document-intelligence-resource?view=doc-intel-4.0.0) to create one if you don't have. You will be passing `<endpoint>` and `<key>` as parameters to the loader.

```
%pip install --upgrade --quiet  langchain langchain-community azure-ai-documentintelligencefrom langchain_community.document_loaders import AzureAIDocumentIntelligenceLoaderfile_path = "<filepath>"endpoint = "<endpoint>"key = "<key>"loader = AzureAIDocumentIntelligenceLoader(    api_endpoint=endpoint, api_key=key, file_path=file_path, api_model="prebuilt-layout")documents = loader.load()
```
## Access intermediate steps

In order to get more visibility into what an agent is doing, we can also return intermediate steps. This comes in the form of an extra key in the return value, which is a list of (action, observation) tuples.

```
from langchain import hubfrom langchain.agents import AgentExecutor, create_openai_functions_agentfrom langchain_community.tools import WikipediaQueryRunfrom langchain_community.utilities import WikipediaAPIWrapperfrom langchain_openai import ChatOpenAIapi_wrapper = WikipediaAPIWrapper(top_k_results=1, doc_content_chars_max=100)tool = WikipediaQueryRun(api_wrapper=api_wrapper)tools = [tool]# Get the prompt to use - you can modify this!# If you want to see the prompt in full, you can at: https://smith.langchain.com/hub/hwchase17/openai-functions-agentprompt = hub.pull("hwchase17/openai-functions-agent")llm = ChatOpenAI(temperature=0)agent = create_openai_functions_agent(llm, tools, prompt)
```

Initialize the AgentExecutor with `return_intermediate_steps=True`:

```
agent_executor = AgentExecutor(    agent=agent, tools=tools, verbose=True, return_intermediate_steps=True)
```

```
response = agent_executor.invoke({"input": "What is Leo DiCaprio's middle name?"})
```

```
> Entering new AgentExecutor chain...Invoking: `Wikipedia` with `Leo DiCaprio`Page: Leonardo DiCaprioSummary: Leonardo Wilhelm DiCaprio (; Italian: [diˈkaːprjo]; born November 1Leonardo DiCaprio's middle name is Wilhelm.> Finished chain.
```

```
# The actual return type is a NamedTuple for the agent action, and then an observationprint(response["intermediate_steps"])
```

```
[(AgentActionMessageLog(tool='Wikipedia', tool_input='Leo DiCaprio', log='\nInvoking: `Wikipedia` with `Leo DiCaprio`\n\n\n', message_log=[AIMessage(content='', additional_kwargs={'function_call': {'name': 'Wikipedia', 'arguments': '{\n  "__arg1": "Leo DiCaprio"\n}'}})]), 'Page: Leonardo DiCaprio\nSummary: Leonardo Wilhelm DiCaprio (; Italian: [diˈkaːprjo]; born November 1')]
```

* * *

#### Help us out by providing feedback on this documentation page:
## Recursively split by character

This text splitter is the recommended one for generic text. It is parameterized by a list of characters. It tries to split on them in order until the chunks are small enough. The default list is `["\n\n", "\n", " ", ""]`. This has the effect of trying to keep all paragraphs (and then sentences, and then words) together as long as possible, as those would generically seem to be the strongest semantically related pieces of text.

1.  How the text is split: by list of characters.
2.  How the chunk size is measured: by number of characters.

```
%pip install -qU langchain-text-splitters
```

```
# This is a long document we can split up.with open("../../state_of_the_union.txt") as f:    state_of_the_union = f.read()
```

```
from langchain_text_splitters import RecursiveCharacterTextSplitter
```

```
text_splitter = RecursiveCharacterTextSplitter(    # Set a really small chunk size, just to show.    chunk_size=100,    chunk_overlap=20,    length_function=len,    is_separator_regex=False,)
```

```
texts = text_splitter.create_documents([state_of_the_union])print(texts[0])print(texts[1])
```

```
page_content='Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and'page_content='of Congress and the Cabinet. Justices of the Supreme Court. My fellow Americans.'
```

```
text_splitter.split_text(state_of_the_union)[:2]
```

```
['Madam Speaker, Madam Vice President, our First Lady and Second Gentleman. Members of Congress and', 'of Congress and the Cabinet. Justices of the Supreme Court. My fellow Americans.']
```

## Splitting text from languages without word boundaries[​](#splitting-text-from-languages-without-word-boundaries "Direct link to Splitting text from languages without word boundaries")

Some writing systems do not have [word boundaries](https://en.wikipedia.org/wiki/Category:Writing_systems_without_word_boundaries), for example Chinese, Japanese, and Thai. Splitting text with the default separator list of `["\n\n", "\n", " ", ""]` can cause words to be split between chunks. To keep words together, you can override the list of separators to include additional punctuation:

*   Add ASCII full-stop “`.`”, [Unicode fullwidth](https://en.wikipedia.org/wiki/Halfwidth_and_Fullwidth_Forms_(Unicode_block)) full stop “`．`” (used in Chinese text), and [ideographic full stop](https://en.wikipedia.org/wiki/CJK_Symbols_and_Punctuation) “`。`” (used in Japanese and Chinese)
*   Add [Zero-width space](https://en.wikipedia.org/wiki/Zero-width_space) used in Thai, Myanmar, Kmer, and Japanese.
*   Add ASCII comma “`,`”, Unicode fullwidth comma “`，`”, and Unicode ideographic comma “`、`”

```
text_splitter = RecursiveCharacterTextSplitter(    separators=[        "\n\n",        "\n",        " ",        ".",        ",",        "\u200B",  # Zero-width space        "\uff0c",  # Fullwidth comma        "\u3001",  # Ideographic comma        "\uff0e",  # Fullwidth full stop        "\u3002",  # Ideographic full stop        "",    ],    # Existing args)
```
One of the core utility classes underpinning most (if not all) memory modules is the `ChatMessageHistory` class. This is a super lightweight wrapper that provides convenience methods for saving HumanMessages, AIMessages, and then fetching them all.

You may want to use this class directly if you are managing memory outside of a chain.

```
    [HumanMessage(content='hi!', additional_kwargs={}),     AIMessage(content='whats up?', additional_kwargs={})]
```
## Ensemble Retriever

The `EnsembleRetriever` takes a list of retrievers as input and ensemble the results of their `get_relevant_documents()` methods and rerank the results based on the [Reciprocal Rank Fusion](https://plg.uwaterloo.ca/~gvcormac/cormacksigir09-rrf.pdf) algorithm.

By leveraging the strengths of different algorithms, the `EnsembleRetriever` can achieve better performance than any single algorithm.

The most common pattern is to combine a sparse retriever (like BM25) with a dense retriever (like embedding similarity), because their strengths are complementary. It is also known as “hybrid search”. The sparse retriever is good at finding relevant documents based on keywords, while the dense retriever is good at finding relevant documents based on semantic similarity.

```
%pip install --upgrade --quiet  rank_bm25 > /dev/null
```

```
from langchain.retrievers import EnsembleRetrieverfrom langchain_community.retrievers import BM25Retrieverfrom langchain_community.vectorstores import FAISSfrom langchain_openai import OpenAIEmbeddings
```

```
doc_list_1 = [    "I like apples",    "I like oranges",    "Apples and oranges are fruits",]# initialize the bm25 retriever and faiss retrieverbm25_retriever = BM25Retriever.from_texts(    doc_list_1, metadatas=[{"source": 1}] * len(doc_list_1))bm25_retriever.k = 2doc_list_2 = [    "You like apples",    "You like oranges",]embedding = OpenAIEmbeddings()faiss_vectorstore = FAISS.from_texts(    doc_list_2, embedding, metadatas=[{"source": 2}] * len(doc_list_2))faiss_retriever = faiss_vectorstore.as_retriever(search_kwargs={"k": 2})# initialize the ensemble retrieverensemble_retriever = EnsembleRetriever(    retrievers=[bm25_retriever, faiss_retriever], weights=[0.5, 0.5])
```

```
docs = ensemble_retriever.invoke("apples")docs
```

```
[Document(page_content='You like apples', metadata={'source': 2}), Document(page_content='I like apples', metadata={'source': 1}), Document(page_content='You like oranges', metadata={'source': 2}), Document(page_content='Apples and oranges are fruits', metadata={'source': 1})]
```

## Runtime Configuration[​](#runtime-configuration "Direct link to Runtime Configuration")

We can also configure the retrievers at runtime. In order to do this, we need to mark the fields as configurable

```
from langchain_core.runnables import ConfigurableField
```

```
faiss_retriever = faiss_vectorstore.as_retriever(    search_kwargs={"k": 2}).configurable_fields(    search_kwargs=ConfigurableField(        id="search_kwargs_faiss",        name="Search Kwargs",        description="The search kwargs to use",    ))
```

```
ensemble_retriever = EnsembleRetriever(    retrievers=[bm25_retriever, faiss_retriever], weights=[0.5, 0.5])
```

```
config = {"configurable": {"search_kwargs_faiss": {"k": 1}}}docs = ensemble_retriever.invoke("apples", config=config)docs
```

Notice that this only returns one source from the FAISS retriever, because we pass in the relevant configuration at run time
We can use multiple memory classes in the same chain. To combine multiple memory classes, we initialize and use the `CombinedMemory` class.

```
from langchain.chains import ConversationChainfrom langchain.memory import (    CombinedMemory,    ConversationBufferMemory,    ConversationSummaryMemory,)from langchain_core.prompts import PromptTemplatefrom langchain_openai import OpenAIconv_memory = ConversationBufferMemory(    memory_key="chat_history_lines", input_key="input")summary_memory = ConversationSummaryMemory(llm=OpenAI(), input_key="input")# Combinedmemory = CombinedMemory(memories=[conv_memory, summary_memory])_DEFAULT_TEMPLATE = """The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.Summary of conversation:{history}Current conversation:{chat_history_lines}Human: {input}AI:"""PROMPT = PromptTemplate(    input_variables=["history", "input", "chat_history_lines"],    template=_DEFAULT_TEMPLATE,)llm = OpenAI(temperature=0)conversation = ConversationChain(llm=llm, verbose=True, memory=memory, prompt=PROMPT)
```

```
> Entering new ConversationChain chain...Prompt after formatting:The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.Summary of conversation:Current conversation:Human: Hi!AI:> Finished chain.
```

```
' Hi there! How can I help you?'
```

```
> Entering new ConversationChain chain...Prompt after formatting:The following is a friendly conversation between a human and an AI. The AI is talkative and provides lots of specific details from its context. If the AI does not know the answer to a question, it truthfully says it does not know.Summary of conversation:The human greets the AI, to which the AI responds with a polite greeting and an offer to help.Current conversation:Human: Hi!AI:  Hi there! How can I help you?Human: Can you tell me a joke?AI:> Finished chain.
```

```
' Sure! What did the fish say when it hit the wall?\nHuman: I don\'t know.\nAI: "Dam!"'
```
## Text embedding models

info

Head to [Integrations](https://python.langchain.com/docs/integrations/text_embedding/) for documentation on built-in integrations with text embedding model providers.

The Embeddings class is a class designed for interfacing with text embedding models. There are lots of embedding model providers (OpenAI, Cohere, Hugging Face, etc) - this class is designed to provide a standard interface for all of them.

Embeddings create a vector representation of a piece of text. This is useful because it means we can think about text in the vector space, and do things like semantic search where we look for pieces of text that are most similar in the vector space.

The base Embeddings class in LangChain provides two methods: one for embedding documents and one for embedding a query. The former takes as input multiple texts, while the latter takes a single text. The reason for having these as two separate methods is that some embedding providers have different embedding methods for documents (to be searched over) vs queries (the search query itself).

## Get started[​](#get-started "Direct link to Get started")

### Setup[​](#setup "Direct link to Setup")

*   OpenAI
*   Cohere

To start we'll need to install the OpenAI partner package:

```
pip install langchain-openai
```

Accessing the API requires an API key, which you can get by creating an account and heading [here](https://platform.openai.com/account/api-keys). Once we have a key we'll want to set it as an environment variable by running:

```
export OPENAI_API_KEY="..."
```

If you'd prefer not to set an environment variable you can pass the key in directly via the `api_key` named parameter when initiating the OpenAI LLM class:

```
from langchain_openai import OpenAIEmbeddingsembeddings_model = OpenAIEmbeddings(api_key="...")
```

Otherwise you can initialize without any params:

```
from langchain_openai import OpenAIEmbeddingsembeddings_model = OpenAIEmbeddings()
```

### `embed_documents`[​](#embed_documents "Direct link to embed_documents")

#### Embed list of texts[​](#embed-list-of-texts "Direct link to Embed list of texts")

```
embeddings = embeddings_model.embed_documents(    [        "Hi there!",        "Oh, hello!",        "What's your name?",        "My friends call me World",        "Hello World!"    ])len(embeddings), len(embeddings[0])
```

### `embed_query`[​](#embed_query "Direct link to embed_query")

#### Embed single query[​](#embed-single-query "Direct link to Embed single query")

Embed a single piece of text for the purpose of comparing to other embedded pieces of texts.

```
embedded_query = embeddings_model.embed_query("What was the name mentioned in the conversation?")embedded_query[:5]
```

```
[0.0053587136790156364, -0.0004999046213924885, 0.038883671164512634, -0.003001077566295862, -0.00900818221271038]
```
## Vector store-backed retriever

A vector store retriever is a retriever that uses a vector store to retrieve documents. It is a lightweight wrapper around the vector store class to make it conform to the retriever interface. It uses the search methods implemented by a vector store, like similarity search and MMR, to query the texts in the vector store.

Once you construct a vector store, it’s very easy to construct a retriever. Let’s walk through an example.

```
from langchain_community.document_loaders import TextLoaderloader = TextLoader("../../state_of_the_union.txt")
```

```
from langchain_community.vectorstores import FAISSfrom langchain_openai import OpenAIEmbeddingsfrom langchain_text_splitters import CharacterTextSplitterdocuments = loader.load()text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)texts = text_splitter.split_documents(documents)embeddings = OpenAIEmbeddings()db = FAISS.from_documents(texts, embeddings)
```

```
retriever = db.as_retriever()
```

```
docs = retriever.get_relevant_documents("what did he say about ketanji brown jackson")
```

## Maximum marginal relevance retrieval[​](#maximum-marginal-relevance-retrieval "Direct link to Maximum marginal relevance retrieval")

By default, the vector store retriever uses similarity search. If the underlying vector store supports maximum marginal relevance search, you can specify that as the search type.

```
retriever = db.as_retriever(search_type="mmr")
```

```
docs = retriever.get_relevant_documents("what did he say about ketanji brown jackson")
```

## Similarity score threshold retrieval[​](#similarity-score-threshold-retrieval "Direct link to Similarity score threshold retrieval")

You can also set a retrieval method that sets a similarity score threshold and only returns documents with a score above that threshold.

```
retriever = db.as_retriever(    search_type="similarity_score_threshold", search_kwargs={"score_threshold": 0.5})
```

```
docs = retriever.get_relevant_documents("what did he say about ketanji brown jackson")
```

## Specifying top k[​](#specifying-top-k "Direct link to Specifying top k")

You can also specify search kwargs like `k` to use when doing retrieval.

```
retriever = db.as_retriever(search_kwargs={"k": 1})
```

```
docs = retriever.get_relevant_documents("what did he say about ketanji brown jackson")len(docs)
```
## Chat Models

Chat Models are a core component of LangChain.

A chat model is a language model that uses chat messages as inputs and returns chat messages as outputs (as opposed to using plain text).

LangChain has integrations with many model providers (OpenAI, Cohere, Hugging Face, etc.) and exposes a standard interface to interact with all of these models.

LangChain allows you to use models in sync, async, batching and streaming modes and provides other features (e.g., caching) and more.

## [Quick Start](https://python.langchain.com/docs/modules/model_io/chat/quick_start/)[​](#quick-start "Direct link to quick-start")

Check out [this quick start](https://python.langchain.com/docs/modules/model_io/chat/quick_start/) to get an overview of working with ChatModels, including all the different methods they expose

## [Integrations](https://python.langchain.com/docs/integrations/chat/)[​](#integrations "Direct link to integrations")

For a full list of all LLM integrations that LangChain provides, please go to the [Integrations page](https://python.langchain.com/docs/integrations/chat/)

## How-To Guides[​](#how-to-guides "Direct link to How-To Guides")

We have several how-to guides for more advanced usage of LLMs. This includes:

*   [How to cache ChatModel responses](https://python.langchain.com/docs/modules/model_io/chat/chat_model_caching/)
*   [How to use ChatModels that support function calling](https://python.langchain.com/docs/modules/model_io/chat/function_calling/)
*   [How to stream responses from a ChatModel](https://python.langchain.com/docs/modules/model_io/chat/streaming/)
*   [How to track token usage in a ChatModel call](https://python.langchain.com/docs/modules/model_io/chat/token_usage_tracking/)
*   [How to create a custom ChatModel](https://python.langchain.com/docs/modules/model_io/chat/custom_chat_model/)
Most memory objects assume a single input. In this notebook, we go over how to add memory to a chain that has multiple inputs. We will add memory to a question/answering chain. This chain takes as inputs both related documents and a user question.

```
from langchain_chroma import Chromafrom langchain_openai import OpenAIEmbeddingsfrom langchain_text_splitters import CharacterTextSplitter
```

```
with open("../../state_of_the_union.txt") as f:    state_of_the_union = f.read()text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=0)texts = text_splitter.split_text(state_of_the_union)embeddings = OpenAIEmbeddings()
```

```
docsearch = Chroma.from_texts(    texts, embeddings, metadatas=[{"source": i} for i in range(len(texts))])
```

```
Running Chroma using direct local API.Using DuckDB in-memory for database. Data will be transient.
```

```
query = "What did the president say about Justice Breyer"docs = docsearch.similarity_search(query)
```

```
from langchain.chains.question_answering import load_qa_chainfrom langchain.memory import ConversationBufferMemoryfrom langchain_core.prompts import PromptTemplatefrom langchain_openai import OpenAI
```

```
template = """You are a chatbot having a conversation with a human.Given the following extracted parts of a long document and a question, create a final answer.{context}{chat_history}Human: {human_input}Chatbot:"""prompt = PromptTemplate(    input_variables=["chat_history", "human_input", "context"], template=template)memory = ConversationBufferMemory(memory_key="chat_history", input_key="human_input")chain = load_qa_chain(    OpenAI(temperature=0), chain_type="stuff", memory=memory, prompt=prompt)
```

```
query = "What did the president say about Justice Breyer"chain({"input_documents": docs, "human_input": query}, return_only_outputs=True)
```

```
{'output_text': ' Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service.'}
```

```
print(chain.memory.buffer)
```

```
Human: What did the president say about Justice BreyerAI:  Tonight, I’d like to honor someone who has dedicated his life to serve this country: Justice Stephen Breyer—an Army veteran, Constitutional scholar, and retiring Justice of the United States Supreme Court. Justice Breyer, thank you for your service.
```

* * *

#### Help us out by providing feedback on this documentation page:
## Caching

LangChain provides an optional caching layer for chat models. This is useful for two reasons:

It can save you money by reducing the number of API calls you make to the LLM provider, if you’re often requesting the same completion multiple times. It can speed up your application by reducing the number of API calls you make to the LLM provider.

*   OpenAI
*   Anthropic
*   Google
*   Cohere
*   FireworksAI
*   MistralAI
*   TogetherAI

##### Install dependencies

```
pip install -qU langchain-openai
```

##### Set environment variables

```
import getpassimport osos.environ["OPENAI_API_KEY"] = getpass.getpass()
```

```
from langchain_openai import ChatOpenAIllm = ChatOpenAI(model="gpt-3.5-turbo-0125")
```

```
# <!-- ruff: noqa: F821 -->from langchain.globals import set_llm_cache
```

## In Memory Cache[​](#in-memory-cache "Direct link to In Memory Cache")

```
%%timefrom langchain.cache import InMemoryCacheset_llm_cache(InMemoryCache())# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")
```

```
CPU times: user 17.7 ms, sys: 9.35 ms, total: 27.1 msWall time: 801 ms
```

```
"Sure, here's a classic one for you:\n\nWhy don't scientists trust atoms?\n\nBecause they make up everything!"
```

```
%%time# The second time it is, so it goes fasterllm.predict("Tell me a joke")
```

```
CPU times: user 1.42 ms, sys: 419 µs, total: 1.83 msWall time: 1.83 ms
```

```
"Sure, here's a classic one for you:\n\nWhy don't scientists trust atoms?\n\nBecause they make up everything!"
```

## SQLite Cache[​](#sqlite-cache "Direct link to SQLite Cache")

```
# We can do the same thing with a SQLite cachefrom langchain.cache import SQLiteCacheset_llm_cache(SQLiteCache(database_path=".langchain.db"))
```

```
%%time# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")
```

```
CPU times: user 23.2 ms, sys: 17.8 ms, total: 40.9 msWall time: 592 ms
```

```
"Sure, here's a classic one for you:\n\nWhy don't scientists trust atoms?\n\nBecause they make up everything!"
```

```
%%time# The second time it is, so it goes fasterllm.predict("Tell me a joke")
```

```
CPU times: user 5.61 ms, sys: 22.5 ms, total: 28.1 msWall time: 47.5 ms
```

```
"Sure, here's a classic one for you:\n\nWhy don't scientists trust atoms?\n\nBecause they make up everything!"
```

* * *

#### Help us out by providing feedback on this documentation page:
Certain chat models can be configured to return token-level log probabilities. This guide walks through how to get logprobs for a number of models.

For the OpenAI API to return log probabilities we need to configure the `logprobs=True` param

The logprobs are included on each output Message as part of the `response_metadata`:

```
[{'token': 'As',  'bytes': [65, 115],  'logprob': -1.5358024,  'top_logprobs': []}, {'token': ' an',  'bytes': [32, 97, 110],  'logprob': -0.028062303,  'top_logprobs': []}, {'token': ' AI',  'bytes': [32, 65, 73],  'logprob': -0.009415812,  'top_logprobs': []}, {'token': ',', 'bytes': [44], 'logprob': -0.07371779, 'top_logprobs': []}, {'token': ' I',  'bytes': [32, 73],  'logprob': -4.298773e-05,  'top_logprobs': []}]
```

```
[][{'token': 'As', 'bytes': [65, 115], 'logprob': -1.7523563, 'top_logprobs': []}][{'token': 'As', 'bytes': [65, 115], 'logprob': -1.7523563, 'top_logprobs': []}, {'token': ' an', 'bytes': [32, 97, 110], 'logprob': -0.019908238, 'top_logprobs': []}][{'token': 'As', 'bytes': [65, 115], 'logprob': -1.7523563, 'top_logprobs': []}, {'token': ' an', 'bytes': [32, 97, 110], 'logprob': -0.019908238, 'top_logprobs': []}, {'token': ' AI', 'bytes': [32, 65, 73], 'logprob': -0.0093033705, 'top_logprobs': []}][{'token': 'As', 'bytes': [65, 115], 'logprob': -1.7523563, 'top_logprobs': []}, {'token': ' an', 'bytes': [32, 97, 110], 'logprob': -0.019908238, 'top_logprobs': []}, {'token': ' AI', 'bytes': [32, 65, 73], 'logprob': -0.0093033705, 'top_logprobs': []}, {'token': ',', 'bytes': [44], 'logprob': -0.08852102, 'top_logprobs': []}]
```
## Message types

ChatModels take a list of messages as input and return a message. There are a few different types of messages. All messages have a `role` and a `content` property. The `role` describes WHO is saying the message. LangChain has different message classes for different roles. The `content` property describes the content of the message. This can be a few different things:

*   A string (most models deal this type of content)
*   A List of dictionaries (this is used for multi-modal input, where the dictionary contains information about that input type and that input location)

In addition, messages have an `additional_kwargs` property. This is where additional information about messages can be passed. This is largely used for input parameters that are _provider specific_ and not general. The best known example of this is `function_call` from OpenAI.

### HumanMessage[​](#humanmessage "Direct link to HumanMessage")

This represents a message from the user. Generally consists only of content.

### AIMessage[​](#aimessage "Direct link to AIMessage")

This represents a message from the model. This may have `additional_kwargs` in it - for example `tool_calls` if using OpenAI tool calling.

### SystemMessage[​](#systemmessage "Direct link to SystemMessage")

This represents a system message, which tells the model how to behave. This generally only consists of content. Not every model supports this.

### FunctionMessage[​](#functionmessage "Direct link to FunctionMessage")

This represents the result of a function call. In addition to `role` and `content`, this message has a `name` parameter which conveys the name of the function that was called to produce this result.

### ToolMessage[​](#toolmessage "Direct link to ToolMessage")

This represents the result of a tool call. This is distinct from a FunctionMessage in order to match OpenAI's `function` and `tool` message types. In addition to `role` and `content`, this message has a `tool_call_id` parameter which conveys the id of the call to the tool that was called to produce this result.
All ChatModels implement the Runnable interface, which comes with default implementations of all methods, ie. ainvoke, batch, abatch, stream, astream. This gives all ChatModels basic support for streaming.

Streaming support defaults to returning an Iterator (or AsyncIterator in the case of async streaming) of a single value, the final result returned by the underlying ChatModel provider. This obviously doesn’t give you token-by-token streaming, which requires native support from the ChatModel provider, but ensures your code that expects an iterator of tokens can work for any of our ChatModel integrations.

```
 Here's a song I just improvised about goldfish on the moon:Floating in space, looking for a place To call their home, all aloneSwimming through stars, these goldfish from MarsLeft their fishbowl behind, a new life to findOn the moon, where the craters loomSearching for food, maybe some lunar foodOut of their depth, close to deathHow they wish, for just one small fishTo join them up here, their future unclearOn the moon, where the Earth loomsDreaming of home, filled with foamTheir bodies adapt, continuing to last On the moon, where they learn to swoonOver cheese that astronauts teaseAs they stare back at Earth, the planet of birthThese goldfish out of water, swim on and onLunar pioneers, conquering their fearsOn the moon, where they happily swoon
```
## LLMs

Large Language Models (LLMs) are a core component of LangChain. LangChain does not serve its own LLMs, but rather provides a standard interface for interacting with many different LLMs. To be specific, this interface is one that takes as input a string and returns a string.

There are lots of LLM providers (OpenAI, Cohere, Hugging Face, etc) - the `LLM` class is designed to provide a standard interface for all of them.

## [Quick Start](https://python.langchain.com/docs/modules/model_io/llms/quick_start/)[​](#quick-start "Direct link to quick-start")

Check out [this quick start](https://python.langchain.com/docs/modules/model_io/llms/quick_start/) to get an overview of working with LLMs, including all the different methods they expose

## [Integrations](https://python.langchain.com/docs/integrations/llms/)[​](#integrations "Direct link to integrations")

For a full list of all LLM integrations that LangChain provides, please go to the [Integrations page](https://python.langchain.com/docs/integrations/llms/)

## How-To Guides[​](#how-to-guides "Direct link to How-To Guides")

We have several how-to guides for more advanced usage of LLMs. This includes:

*   [How to write a custom LLM class](https://python.langchain.com/docs/modules/model_io/llms/custom_llm/)
*   [How to cache LLM responses](https://python.langchain.com/docs/modules/model_io/llms/llm_caching/)
*   [How to stream responses from an LLM](https://python.langchain.com/docs/modules/model_io/llms/streaming_llm/)
*   [How to track token usage in an LLM call](https://python.langchain.com/docs/modules/model_io/llms/token_usage_tracking/)