seansullivan/Langchain-Docs huggingface dataset:
## NLP Cloud

The [NLP Cloud](https://nlpcloud.io/) serves high performance pre-trained or custom models for NER, sentiment-analysis, classification, summarization, paraphrasing, grammar and spelling correction, keywords and keyphrases extraction, chatbot, product description and ad generation, intent classification, text generation, image generation, blog post generation, code generation, question answering, automatic speech recognition, machine translation, language detection, semantic search, semantic similarity, tokenization, POS tagging, embeddings, and dependency parsing. It is ready for production, served through a REST API.

This example goes over how to use LangChain to interact with `NLP Cloud` [models](https://docs.nlpcloud.com/#models).

```
%pip install --upgrade --quiet  nlpcloud
```

```
# get a token: https://docs.nlpcloud.com/#authenticationfrom getpass import getpassNLPCLOUD_API_KEY = getpass()
```

```
import osos.environ["NLPCLOUD_API_KEY"] = NLPCLOUD_API_KEY
```

```
from langchain.chains import LLMChainfrom langchain_community.llms import NLPCloudfrom langchain_core.prompts import PromptTemplate
```

```
template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate.from_template(template)
```

```
llm_chain = LLMChain(prompt=prompt, llm=llm)
```

```
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)
```

```
' Justin Bieber was born in 1994, so the team that won the Super Bowl that year was the San Francisco 49ers.'
```

* * *

#### Help us out by providing feedback on this documentation page:
## Volc Engine Maas

This notebook provides you with a guide on how to get started with Volc Engine’s MaaS llm models.

```
# Install the package%pip install --upgrade --quiet  volcengine
```

```
from langchain_community.llms import VolcEngineMaasLLMfrom langchain_core.output_parsers import StrOutputParserfrom langchain_core.prompts import PromptTemplate
```

```
llm = VolcEngineMaasLLM(volc_engine_maas_ak="your ak", volc_engine_maas_sk="your sk")
```

or you can set access\_key and secret\_key in your environment variables

```
export VOLC_ACCESSKEY=YOUR_AKexport VOLC_SECRETKEY=YOUR_SK
```

```
chain = PromptTemplate.from_template("给我讲个笑话") | llm | StrOutputParser()chain.invoke({})
```

```
'好的，下面是一个笑话：\n\n大学暑假我配了隐形眼镜，回家给爷爷说，我现在配了隐形眼镜。\n爷爷让我给他看看，于是，我用小镊子夹了一片给爷爷看。\n爷爷看完便准备出门，边走还边说：“真高级啊，还真是隐形眼镜！”\n等爷爷出去后我才发现，我刚没夹起来！'
```

* * *

#### Help us out by providing feedback on this documentation page:
Oracle Cloud Infrastructure (OCI) Generative AI is a fully managed service that provides a set of state-of-the-art, customizable large language models (LLMs) that cover a wide range of use cases, and which is available through a single API. Using the OCI Generative AI service you can access ready-to-use pretrained models, or create and host your own fine-tuned custom models based on your own data on dedicated AI clusters. Detailed documentation of the service and API is available **[here](https://docs.oracle.com/en-us/iaas/Content/generative-ai/home.htm)** and **[here](https://docs.oracle.com/en-us/iaas/api/#/en/generative-ai/20231130/)**.

This notebook explains how to use OCI’s Genrative AI models with LangChain.

```
from langchain_community.llms import OCIGenAI# use default authN method API-keyllm = OCIGenAI(    model_id="MY_MODEL",    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",    compartment_id="MY_OCID",)response = llm.invoke("Tell me one fact about earth", temperature=0.7)print(response)
```

```
from langchain.chains import LLMChainfrom langchain_core.prompts import PromptTemplate# Use Session Token to authNllm = OCIGenAI(    model_id="MY_MODEL",    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",    compartment_id="MY_OCID",    auth_type="SECURITY_TOKEN",    auth_profile="MY_PROFILE",  # replace with your profile name    model_kwargs={"temperature": 0.7, "top_p": 0.75, "max_tokens": 200},)prompt = PromptTemplate(input_variables=["query"], template="{query}")llm_chain = LLMChain(llm=llm, prompt=prompt)response = llm_chain.invoke("what is the capital of france?")print(response)
```

```
from langchain.schema.output_parser import StrOutputParserfrom langchain.schema.runnable import RunnablePassthroughfrom langchain_community.embeddings import OCIGenAIEmbeddingsfrom langchain_community.vectorstores import FAISSembeddings = OCIGenAIEmbeddings(    model_id="MY_EMBEDDING_MODEL",    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",    compartment_id="MY_OCID",)vectorstore = FAISS.from_texts(    [        "Larry Ellison co-founded Oracle Corporation in 1977 with Bob Miner and Ed Oates.",        "Oracle Corporation is an American multinational computer technology company headquartered in Austin, Texas, United States.",    ],    embedding=embeddings,)retriever = vectorstore.as_retriever()template = """Answer the question based only on the following context:{context} Question: {question}"""prompt = PromptTemplate.from_template(template)llm = OCIGenAI(    model_id="MY_MODEL",    service_endpoint="https://inference.generativeai.us-chicago-1.oci.oraclecloud.com",    compartment_id="MY_OCID",)chain = (    {"context": retriever, "question": RunnablePassthrough()}    | prompt    | llm    | StrOutputParser())print(chain.invoke("when was oracle founded?"))print(chain.invoke("where is oracle headquartered?"))
```
[OCI Data Science](https://docs.oracle.com/en-us/iaas/data-science/using/home.htm) is a fully managed and serverless platform for data science teams to build, train, and manage machine learning models in the Oracle Cloud Infrastructure.

To authenticate, [oracle-ads](https://accelerated-data-science.readthedocs.io/en/latest/user_guide/cli/authentication.html) has been used to automatically load credentials for invoking endpoint.

Make sure to have the required [policies](https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-policies-auth.htm#model_dep_policies_auth__predict-endpoint) to access the OCI Data Science Model Deployment endpoint.

After having deployed model, you have to set up following required parameters of the `OCIModelDeploymentVLLM` call:

You have to set up following required parameters of the `OCIModelDeploymentTGI` call:

You can set authentication through either ads or environment variables. When you are working in OCI Data Science Notebook Session, you can leverage resource principal to access other OCI resources. Check out [here](https://accelerated-data-science.readthedocs.io/en/latest/user_guide/cli/authentication.html) to see more options.

```
import adsfrom langchain_community.llms import OCIModelDeploymentVLLM# Set authentication through ads# Use resource principal are operating within a# OCI service that has resource principal based# authentication configuredads.set_auth("resource_principal")# Create an instance of OCI Model Deployment Endpoint# Replace the endpoint uri and model name with your ownllm = OCIModelDeploymentVLLM(endpoint="https://<MD_OCID>/predict", model="model_name")# Run the LLMllm.invoke("Who is the first president of United States?")
```
## OctoAI

[OctoAI](https://docs.octoai.cloud/docs) offers easy access to efficient compute and enables users to integrate their choice of AI models into applications. The `OctoAI` compute service helps you run, tune, and scale AI applications easily.

This example goes over how to use LangChain to interact with `OctoAI` [LLM endpoints](https://octoai.cloud/templates)

## Setup[​](#setup "Direct link to Setup")

To run our example app, there are two simple steps to take:

1.  Get an API Token from [your OctoAI account page](https://octoai.cloud/settings).
    
2.  Paste your API key in in the code cell below.
    

Note: If you want to use a different LLM model, you can containerize the model and make a custom OctoAI endpoint yourself, by following [Build a Container from Python](https://octo.ai/docs/bring-your-own-model/advanced-build-a-container-from-scratch-in-python) and [Create a Custom Endpoint from a Container](https://octo.ai/docs/bring-your-own-model/create-custom-endpoints-from-a-container/create-custom-endpoints-from-a-container) and then updating your `OCTOAI_API_BASE` environment variable.

```
import osos.environ["OCTOAI_API_TOKEN"] = "OCTOAI_API_TOKEN"
```

```
from langchain.chains import LLMChainfrom langchain_community.llms.octoai_endpoint import OctoAIEndpointfrom langchain_core.prompts import PromptTemplate
```

## Example[​](#example "Direct link to Example")

```
template = """Below is an instruction that describes a task. Write a response that appropriately completes the request.\n Instruction:\n{question}\n Response: """prompt = PromptTemplate.from_template(template)
```

```
llm = OctoAIEndpoint(    model="llama-2-13b-chat-fp16",    max_tokens=200,    presence_penalty=0,    temperature=0.1,    top_p=0.9,)
```

```
question = "Who was Leonardo da Vinci?"llm_chain = LLMChain(prompt=prompt, llm=llm)print(llm_chain.run(question))
```

Leonardo da Vinci was a true Renaissance man. He was born in 1452 in Vinci, Italy and was known for his work in various fields, including art, science, engineering, and mathematics. He is considered one of the greatest painters of all time, and his most famous works include the Mona Lisa and The Last Supper. In addition to his art, da Vinci made significant contributions to engineering and anatomy, and his designs for machines and inventions were centuries ahead of his time. He is also known for his extensive journals and drawings, which provide valuable insights into his thoughts and ideas. Da Vinci’s legacy continues to inspire and influence artists, scientists, and thinkers around the world today.

* * *

#### Help us out by providing feedback on this documentation page:
## Writer

[Writer](https://writer.com/) is a platform to generate different language content.

This example goes over how to use LangChain to interact with `Writer` [models](https://dev.writer.com/docs/models).

You have to get the WRITER\_API\_KEY [here](https://dev.writer.com/docs).

```
from getpass import getpassWRITER_API_KEY = getpass()
```

```
import osos.environ["WRITER_API_KEY"] = WRITER_API_KEY
```

```
from langchain.chains import LLMChainfrom langchain_community.llms import Writerfrom langchain_core.prompts import PromptTemplate
```

```
template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate.from_template(template)
```

```
# If you get an error, probably, you need to set up the "base_url" parameter that can be taken from the error log.llm = Writer()
```

```
llm_chain = LLMChain(prompt=prompt, llm=llm)
```

```
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)
```

* * *

#### Help us out by providing feedback on this documentation page:
## Xorbits Inference (Xinference)

[Xinference](https://github.com/xorbitsai/inference) is a powerful and versatile library designed to serve LLMs, speech recognition models, and multimodal models, even on your laptop. It supports a variety of models compatible with GGML, such as chatglm, baichuan, whisper, vicuna, orca, and many others. This notebook demonstrates how to use Xinference with LangChain.

## Installation[​](#installation "Direct link to Installation")

Install `Xinference` through PyPI:

```
%pip install --upgrade --quiet  "xinference[all]"
```

## Deploy Xinference Locally or in a Distributed Cluster.[​](#deploy-xinference-locally-or-in-a-distributed-cluster. "Direct link to Deploy Xinference Locally or in a Distributed Cluster.")

For local deployment, run `xinference`.

To deploy Xinference in a cluster, first start an Xinference supervisor using the `xinference-supervisor`. You can also use the option -p to specify the port and -H to specify the host. The default port is 9997.

Then, start the Xinference workers using `xinference-worker` on each server you want to run them on.

You can consult the README file from [Xinference](https://github.com/xorbitsai/inference) for more information. #\# Wrapper

To use Xinference with LangChain, you need to first launch a model. You can use command line interface (CLI) to do so:

```
!xinference launch -n vicuna-v1.3 -f ggmlv3 -q q4_0
```

```
Model uid: 7167b2b0-2a04-11ee-83f0-d29396a3f064
```

A model UID is returned for you to use. Now you can use Xinference with LangChain:

```
from langchain_community.llms import Xinferencellm = Xinference(    server_url="http://0.0.0.0:9997", model_uid="7167b2b0-2a04-11ee-83f0-d29396a3f064")llm(    prompt="Q: where can we visit in the capital of France? A:",    generate_config={"max_tokens": 1024, "stream": True},)
```

```
' You can visit the Eiffel Tower, Notre-Dame Cathedral, the Louvre Museum, and many other historical sites in Paris, the capital of France.'
```

### Integrate with a LLMChain[​](#integrate-with-a-llmchain "Direct link to Integrate with a LLMChain")

```
from langchain.chains import LLMChainfrom langchain_core.prompts import PromptTemplatetemplate = "Where can we visit in the capital of {country}?"prompt = PromptTemplate.from_template(template)llm_chain = LLMChain(prompt=prompt, llm=llm)generated = llm_chain.run(country="France")print(generated)
```

```
A: You can visit many places in Paris, such as the Eiffel Tower, the Louvre Museum, Notre-Dame Cathedral, the Champs-Elysées, Montmartre, Sacré-Cœur, and the Palace of Versailles.
```

Lastly, terminate the model when you do not need to use it:

```
!xinference terminate --model-uid "7167b2b0-2a04-11ee-83f0-d29396a3f064"
```
[Yuan2.0](https://github.com/IEIT-Yuan/Yuan-2.0) is a new generation Fundamental Large Language Model developed by IEIT System. We have published all three models, Yuan 2.0-102B, Yuan 2.0-51B, and Yuan 2.0-2B. And we provide relevant scripts for pretraining, fine-tuning, and inference services for other developers. Yuan2.0 is based on Yuan1.0, utilizing a wider range of high-quality pre training data and instruction fine-tuning datasets to enhance the model’s understanding of semantics, mathematics, reasoning, code, knowledge, and other aspects.

This example goes over how to use LangChain to interact with `Yuan2.0`(2B/51B/102B) Inference for text generation.

Yuan2.0 set up an inference service so user just need request the inference api to get result, which is introduced in [Yuan2.0 Inference-Server](https://github.com/IEIT-Yuan/Yuan-2.0/blob/main/docs/inference_server.md).

```
# default infer_api for a local deployed Yuan2.0 inference serverinfer_api = "http://127.0.0.1:8000/yuan"# direct access endpoint in a proxied environment# import os# os.environ["no_proxy"]="localhost,127.0.0.1,::1"yuan_llm = Yuan2(    infer_api=infer_api,    max_tokens=2048,    temp=1.0,    top_p=0.9,    use_history=False,)# turn on use_history only when you want the Yuan2.0 to keep track of the conversation history# and send the accumulated context to the backend model api, which make it stateful. By default it is stateless.# llm.use_history = True
```
## OpenAI

[OpenAI](https://platform.openai.com/docs/introduction) offers a spectrum of models with different levels of power suitable for different tasks.

This example goes over how to use LangChain to interact with `OpenAI` [models](https://platform.openai.com/docs/models)

```
# get a token: https://platform.openai.com/account/api-keysfrom getpass import getpassOPENAI_API_KEY = getpass()
```

```
import osos.environ["OPENAI_API_KEY"] = OPENAI_API_KEY
```

Should you need to specify your organization ID, you can use the following cell. However, it is not required if you are only part of a single organization or intend to use your default organization. You can check your default organization [here](https://platform.openai.com/account/api-keys).

To specify your organization, you can use this:

```
OPENAI_ORGANIZATION = getpass()os.environ["OPENAI_ORGANIZATION"] = OPENAI_ORGANIZATION
```

```
from langchain.chains import LLMChainfrom langchain_core.prompts import PromptTemplatefrom langchain_openai import OpenAI
```

```
template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate.from_template(template)
```

If you manually want to specify your OpenAI API key and/or organization ID, you can use the following:

```
llm = OpenAI(openai_api_key="YOUR_API_KEY", openai_organization="YOUR_ORGANIZATION_ID")
```

Remove the openai\_organization parameter should it not apply to you.

```
llm_chain = LLMChain(prompt=prompt, llm=llm)
```

```
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)
```

```
' Justin Bieber was born in 1994, so the NFL team that won the Super Bowl in 1994 was the Dallas Cowboys.'
```

If you are behind an explicit proxy, you can specify the http\_client to pass through

```
pip install httpximport httpxopenai = OpenAI(model_name="gpt-3.5-turbo-instruct", http_client=httpx.Client(proxies="http://proxy.yourcompany.com:8080"))
```
## YandexGPT

This notebook goes over how to use Langchain with [YandexGPT](https://cloud.yandex.com/en/services/yandexgpt).

To use, you should have the `yandexcloud` python package installed.

```
%pip install --upgrade --quiet  yandexcloud
```

First, you should [create service account](https://cloud.yandex.com/en/docs/iam/operations/sa/create) with the `ai.languageModels.user` role.

Next, you have two authentication options: - [IAM token](https://cloud.yandex.com/en/docs/iam/operations/iam-token/create-for-sa). You can specify the token in a constructor parameter `iam_token` or in an environment variable `YC_IAM_TOKEN`.

*   [API key](https://cloud.yandex.com/en/docs/iam/operations/api-key/create) You can specify the key in a constructor parameter `api_key` or in an environment variable `YC_API_KEY`.

To specify the model you can use `model_uri` parameter, see [the documentation](https://cloud.yandex.com/en/docs/yandexgpt/concepts/models#yandexgpt-generation) for more details.

By default, the latest version of `yandexgpt-lite` is used from the folder specified in the parameter `folder_id` or `YC_FOLDER_ID` environment variable.

```
from langchain.chains import LLMChainfrom langchain_community.llms import YandexGPTfrom langchain_core.prompts import PromptTemplate
```

```
template = "What is the capital of {country}?"prompt = PromptTemplate.from_template(template)
```

```
llm_chain = LLMChain(prompt=prompt, llm=llm)
```

```
country = "Russia"llm_chain.invoke(country)
```

```
'The capital of Russia is Moscow.'
```
## OpenLLM

[🦾 OpenLLM](https://github.com/bentoml/OpenLLM) is an open platform for operating large language models (LLMs) in production. It enables developers to easily run inference with any open-source LLMs, deploy to the cloud or on-premises, and build powerful AI apps.

## Installation[​](#installation "Direct link to Installation")

Install `openllm` through [PyPI](https://pypi.org/project/openllm/)

```
%pip install --upgrade --quiet  openllm
```

## Launch OpenLLM server locally[​](#launch-openllm-server-locally "Direct link to Launch OpenLLM server locally")

To start an LLM server, use `openllm start` command. For example, to start a dolly-v2 server, run the following command from a terminal:

## Wrapper[​](#wrapper "Direct link to Wrapper")

```
from langchain_community.llms import OpenLLMserver_url = "http://localhost:3000"  # Replace with remote host if you are running on a remote serverllm = OpenLLM(server_url=server_url)
```

### Optional: Local LLM Inference[​](#optional-local-llm-inference "Direct link to Optional: Local LLM Inference")

You may also choose to initialize an LLM managed by OpenLLM locally from current process. This is useful for development purpose and allows developers to quickly try out different types of LLMs.

When moving LLM applications to production, we recommend deploying the OpenLLM server separately and access via the `server_url` option demonstrated above.

To load an LLM locally via the LangChain wrapper:

```
from langchain_community.llms import OpenLLMllm = OpenLLM(    model_name="dolly-v2",    model_id="databricks/dolly-v2-3b",    temperature=0.94,    repetition_penalty=1.2,)
```

### Integrate with a LLMChain[​](#integrate-with-a-llmchain "Direct link to Integrate with a LLMChain")

```
from langchain.chains import LLMChainfrom langchain_core.prompts import PromptTemplatetemplate = "What is a good name for a company that makes {product}?"prompt = PromptTemplate.from_template(template)llm_chain = LLMChain(prompt=prompt, llm=llm)generated = llm_chain.run(product="mechanical keyboard")print(generated)
```
## Astra DB

> DataStax [Astra DB](https://docs.datastax.com/en/astra/home/astra.html) is a serverless vector-capable database built on Cassandra and made conveniently available through an easy-to-use JSON API.

This notebook goes over how to use Astra DB to store chat message history.

## Setting up[​](#setting-up "Direct link to Setting up")

To run this notebook you need a running Astra DB. Get the connection secrets on your Astra dashboard:

*   the API Endpoint looks like `https://01234567-89ab-cdef-0123-456789abcdef-us-east1.apps.astra.datastax.com`;
*   the Token looks like `AstraCS:6gBhNmsk135...`.

```
%pip install --upgrade --quiet  "astrapy>=0.7.1"
```

### Set up the database connection parameters and secrets[​](#set-up-the-database-connection-parameters-and-secrets "Direct link to Set up the database connection parameters and secrets")

```
import getpassASTRA_DB_API_ENDPOINT = input("ASTRA_DB_API_ENDPOINT = ")ASTRA_DB_APPLICATION_TOKEN = getpass.getpass("ASTRA_DB_APPLICATION_TOKEN = ")
```

```
ASTRA_DB_API_ENDPOINT =  https://01234567-89ab-cdef-0123-456789abcdef-us-east1.apps.astra.datastax.comASTRA_DB_APPLICATION_TOKEN =  ········
```

Depending on whether local or cloud-based Astra DB, create the corresponding database connection “Session” object.

## Example[​](#example "Direct link to Example")

```
from langchain_community.chat_message_histories import AstraDBChatMessageHistorymessage_history = AstraDBChatMessageHistory(    session_id="test-session",    api_endpoint=ASTRA_DB_API_ENDPOINT,    token=ASTRA_DB_APPLICATION_TOKEN,)message_history.add_user_message("hi!")message_history.add_ai_message("whats up?")
```

```
[HumanMessage(content='hi!'), AIMessage(content='whats up?')]
```
## OpenLM

[OpenLM](https://github.com/r2d4/openlm) is a zero-dependency OpenAI-compatible LLM provider that can call different inference endpoints directly via HTTP.

It implements the OpenAI Completion class so that it can be used as a drop-in replacement for the OpenAI API. This changeset utilizes BaseOpenAI for minimal added code.

This examples goes over how to use LangChain to interact with both OpenAI and HuggingFace. You’ll need API keys from both.

### Setup[​](#setup "Direct link to Setup")

Install dependencies and set API keys.

```
# Uncomment to install openlm and openai if you haven't already%pip install --upgrade --quiet  openlm%pip install --upgrade --quiet  langchain-openai
```

```
import osfrom getpass import getpass# Check if OPENAI_API_KEY environment variable is setif "OPENAI_API_KEY" not in os.environ:    print("Enter your OpenAI API key:")    os.environ["OPENAI_API_KEY"] = getpass()# Check if HF_API_TOKEN environment variable is setif "HF_API_TOKEN" not in os.environ:    print("Enter your HuggingFace Hub API key:")    os.environ["HF_API_TOKEN"] = getpass()
```

### Using LangChain with OpenLM[​](#using-langchain-with-openlm "Direct link to Using LangChain with OpenLM")

Here we’re going to call two models in an LLMChain, `text-davinci-003` from OpenAI and `gpt2` on HuggingFace.

```
from langchain.chains import LLMChainfrom langchain_community.llms import OpenLMfrom langchain_core.prompts import PromptTemplate
```

```
question = "What is the capital of France?"template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate.from_template(template)for model in ["text-davinci-003", "huggingface.co/gpt2"]:    llm = OpenLM(model=model)    llm_chain = LLMChain(prompt=prompt, llm=llm)    result = llm_chain.run(question)    print(        """Model: {}Result: {}""".format(model, result)    )
```

```
Model: text-davinci-003Result:  France is a country in Europe. The capital of France is Paris.Model: huggingface.co/gpt2Result: Question: What is the capital of France?Answer: Let's think step by step. I am not going to lie, this is a complicated issue, and I don't see any solutions to all this, but it is still far more
```
## Cassandra

> [Apache Cassandra®](https://cassandra.apache.org/) is a `NoSQL`, row-oriented, highly scalable and highly available database, well suited for storing large amounts of data.

> `Cassandra` is a good choice for storing chat message history because it is easy to scale and can handle a large number of writes.

This notebook goes over how to use Cassandra to store chat message history.

## Setting up[​](#setting-up "Direct link to Setting up")

To run this notebook you need either a running `Cassandra` cluster or a `DataStax Astra DB` instance running in the cloud (you can get one for free at [datastax.com](https://astra.datastax.com/)). Check [cassio.org](https://cassio.org/start_here/) for more information.

```
%pip install --upgrade --quiet  "cassio>=0.1.0"
```

### Set up the database connection parameters and secrets[​](#set-up-the-database-connection-parameters-and-secrets "Direct link to Set up the database connection parameters and secrets")

```
import getpassdatabase_mode = (input("\n(C)assandra or (A)stra DB? ")).upper()keyspace_name = input("\nKeyspace name? ")if database_mode == "A":    ASTRA_DB_APPLICATION_TOKEN = getpass.getpass('\nAstra DB Token ("AstraCS:...") ')    #    ASTRA_DB_SECURE_BUNDLE_PATH = input("Full path to your Secure Connect Bundle? ")elif database_mode == "C":    CASSANDRA_CONTACT_POINTS = input(        "Contact points? (comma-separated, empty for localhost) "    ).strip()
```

Depending on whether local or cloud-based Astra DB, create the corresponding database connection “Session” object.

```
from cassandra.auth import PlainTextAuthProviderfrom cassandra.cluster import Clusterif database_mode == "C":    if CASSANDRA_CONTACT_POINTS:        cluster = Cluster(            [cp.strip() for cp in CASSANDRA_CONTACT_POINTS.split(",") if cp.strip()]        )    else:        cluster = Cluster()    session = cluster.connect()elif database_mode == "A":    ASTRA_DB_CLIENT_ID = "token"    cluster = Cluster(        cloud={            "secure_connect_bundle": ASTRA_DB_SECURE_BUNDLE_PATH,        },        auth_provider=PlainTextAuthProvider(            ASTRA_DB_CLIENT_ID,            ASTRA_DB_APPLICATION_TOKEN,        ),    )    session = cluster.connect()else:    raise NotImplementedError
```

## Example[​](#example "Direct link to Example")

```
from langchain_community.chat_message_histories import (    CassandraChatMessageHistory,)message_history = CassandraChatMessageHistory(    session_id="test-session",    session=session,    keyspace=keyspace_name,)message_history.add_user_message("hi!")message_history.add_ai_message("whats up?")
```

#### Attribution statement[​](#attribution-statement "Direct link to Attribution statement")

> Apache Cassandra, Cassandra and Apache are either registered trademarks or trademarks of the [Apache Software Foundation](http://www.apache.org/) in the United States and/or other countries.
## Petals

`Petals` runs 100B+ language models at home, BitTorrent-style.

This notebook goes over how to use Langchain with [Petals](https://github.com/bigscience-workshop/petals).

## Install petals[​](#install-petals "Direct link to Install petals")

The `petals` package is required to use the Petals API. Install `petals` using `pip3 install petals`.

For Apple Silicon(M1/M2) users please follow this guide [https://github.com/bigscience-workshop/petals/issues/147#issuecomment-1365379642](https://github.com/bigscience-workshop/petals/issues/147#issuecomment-1365379642) to install petals

## Imports[​](#imports "Direct link to Imports")

```
import osfrom langchain.chains import LLMChainfrom langchain_community.llms import Petalsfrom langchain_core.prompts import PromptTemplate
```

## Set the Environment API Key[​](#set-the-environment-api-key "Direct link to Set the Environment API Key")

Make sure to get [your API key](https://huggingface.co/docs/api-inference/quicktour#get-your-api-token) from Huggingface.

```
from getpass import getpassHUGGINGFACE_API_KEY = getpass()
```

```
os.environ["HUGGINGFACE_API_KEY"] = HUGGINGFACE_API_KEY
```

## Create the Petals instance[​](#create-the-petals-instance "Direct link to Create the Petals instance")

You can specify different parameters such as the model name, max new tokens, temperature, etc.

```
# this can take several minutes to download big files!llm = Petals(model_name="bigscience/bloom-petals")
```

```
Downloading:   1%|▏                        | 40.8M/7.19G [00:24<15:44, 7.57MB/s]
```

## Create a Prompt Template[​](#create-a-prompt-template "Direct link to Create a Prompt Template")

We will create a prompt template for Question and Answer.

```
template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate.from_template(template)
```

## Initiate the LLMChain[​](#initiate-the-llmchain "Direct link to Initiate the LLMChain")

```
llm_chain = LLMChain(prompt=prompt, llm=llm)
```

## Run the LLMChain[​](#run-the-llmchain "Direct link to Run the LLMChain")

Provide a question and run the LLMChain.

```
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)
```

* * *

#### Help us out by providing feedback on this documentation page:
## PipelineAI

> [PipelineAI](https://pipeline.ai/) allows you to run your ML models at scale in the cloud. It also provides API access to [several LLM models](https://pipeline.ai/).

This notebook goes over how to use Langchain with [PipelineAI](https://docs.pipeline.ai/docs).

## PipelineAI example[​](#pipelineai-example "Direct link to PipelineAI example")

[This example shows how PipelineAI integrated with LangChain](https://docs.pipeline.ai/docs/langchain) and it is created by PipelineAI.

## Setup[​](#setup "Direct link to Setup")

The `pipeline-ai` library is required to use the `PipelineAI` API, AKA `Pipeline Cloud`. Install `pipeline-ai` using `pip install pipeline-ai`.

```
# Install the package%pip install --upgrade --quiet  pipeline-ai
```

## Example[​](#example "Direct link to Example")

### Imports[​](#imports "Direct link to Imports")

```
import osfrom langchain.chains import LLMChainfrom langchain_community.llms import PipelineAIfrom langchain_core.prompts import PromptTemplate
```

### Set the Environment API Key[​](#set-the-environment-api-key "Direct link to Set the Environment API Key")

Make sure to get your API key from PipelineAI. Check out the [cloud quickstart guide](https://docs.pipeline.ai/docs/cloud-quickstart). You’ll be given a 30 day free trial with 10 hours of serverless GPU compute to test different models.

```
os.environ["PIPELINE_API_KEY"] = "YOUR_API_KEY_HERE"
```

## Create the PipelineAI instance[​](#create-the-pipelineai-instance "Direct link to Create the PipelineAI instance")

When instantiating PipelineAI, you need to specify the id or tag of the pipeline you want to use, e.g. `pipeline_key = "public/gpt-j:base"`. You then have the option of passing additional pipeline-specific keyword arguments:

```
llm = PipelineAI(pipeline_key="YOUR_PIPELINE_KEY", pipeline_kwargs={...})
```

### Create a Prompt Template[​](#create-a-prompt-template "Direct link to Create a Prompt Template")

We will create a prompt template for Question and Answer.

```
template = """Question: {question}Answer: Let's think step by step."""prompt = PromptTemplate.from_template(template)
```

### Initiate the LLMChain[​](#initiate-the-llmchain "Direct link to Initiate the LLMChain")

```
llm_chain = LLMChain(prompt=prompt, llm=llm)
```

### Run the LLMChain[​](#run-the-llmchain "Direct link to Run the LLMChain")

Provide a question and run the LLMChain.

```
question = "What NFL team won the Super Bowl in the year Justin Beiber was born?"llm_chain.run(question)
```

* * *

#### Help us out by providing feedback on this documentation page:
This notebook goes over how to use [Momento Cache](https://www.gomomento.com/services/cache) to store chat message history using the `MomentoChatMessageHistory` class. See the Momento [docs](https://docs.momentohq.com/getting-started) for more detail on how to get set up with Momento.

Note that, by default we will create a cache if one with the given name doesn’t already exist.

You’ll need to get a Momento API key to use this class. This can either be passed in to a momento.CacheClient if you’d like to instantiate that directly, as a named parameter `api_key` to `MomentoChatMessageHistory.from_client_params`, or can just be set as an environment variable `MOMENTO_API_KEY`.

```
from datetime import timedeltafrom langchain_community.chat_message_histories import MomentoChatMessageHistorysession_id = "foo"cache_name = "langchain"ttl = timedelta(days=1)history = MomentoChatMessageHistory.from_client_params(    session_id,    cache_name,    ttl,)history.add_user_message("hi!")history.add_ai_message("whats up?")
```

```
[HumanMessage(content='hi!', additional_kwargs={}, example=False), AIMessage(content='whats up?', additional_kwargs={}, example=False)]
```
## Motörhead

> [Motörhead](https://github.com/getmetal/motorhead) is a memory server implemented in Rust. It automatically handles incremental summarization in the background and allows for stateless applications.

## Setup[​](#setup "Direct link to Setup")

See instructions at [Motörhead](https://github.com/getmetal/motorhead) for running the server locally.

```
from langchain.memory.motorhead_memory import MotorheadMemory
```

## Example[​](#example "Direct link to Example")

```
from langchain.chains import LLMChainfrom langchain_core.prompts import PromptTemplatefrom langchain_openai import OpenAItemplate = """You are a chatbot having a conversation with a human.{chat_history}Human: {human_input}AI:"""prompt = PromptTemplate(    input_variables=["chat_history", "human_input"], template=template)memory = MotorheadMemory(    session_id="testing-1", url="http://localhost:8080", memory_key="chat_history")await memory.init()# loads previous state from Motörhead 🤘llm_chain = LLMChain(    llm=OpenAI(),    prompt=prompt,    verbose=True,    memory=memory,)
```

```
llm_chain.run("hi im bob")
```

```
> Entering new LLMChain chain...Prompt after formatting:You are a chatbot having a conversation with a human.Human: hi im bobAI:> Finished chain.
```

```
' Hi Bob, nice to meet you! How are you doing today?'
```

```
llm_chain.run("whats my name?")
```

```
> Entering new LLMChain chain...Prompt after formatting:You are a chatbot having a conversation with a human.Human: hi im bobAI:  Hi Bob, nice to meet you! How are you doing today?Human: whats my name?AI:> Finished chain.
```

```
' You said your name is Bob. Is that correct?'
```

```
llm_chain.run("whats for dinner?")
```

```
> Entering new LLMChain chain...Prompt after formatting:You are a chatbot having a conversation with a human.Human: hi im bobAI:  Hi Bob, nice to meet you! How are you doing today?Human: whats my name?AI:  You said your name is Bob. Is that correct?Human: whats for dinner?AI:> Finished chain.
```

```
"  I'm sorry, I'm not sure what you're asking. Could you please rephrase your question?"
```

* * *

#### Help us out by providing feedback on this documentation page:
[Neo4j](https://en.wikipedia.org/wiki/Neo4j) is an open-source graph database management system, renowned for its efficient management of highly connected data. Unlike traditional databases that store data in tables, Neo4j uses a graph structure with nodes, edges, and properties to represent and store data. This design allows for high-performance queries on complex data relationships.

This notebook goes over how to use `Neo4j` to store chat message history.

```
from langchain_community.chat_message_histories import Neo4jChatMessageHistoryhistory = Neo4jChatMessageHistory(    url="bolt://localhost:7687",    username="neo4j",    password="password",    session_id="session_id_1",)history.add_user_message("hi!")history.add_ai_message("whats up?")
```
[Redis (Remote Dictionary Server)](https://en.wikipedia.org/wiki/Redis) is an open-source in-memory storage, used as a distributed, in-memory key–value database, cache and message broker, with optional durability. Because it holds all data in memory and because of its design, `Redis` offers low-latency reads and writes, making it particularly suitable for use cases that require a cache. Redis is the most popular NoSQL database, and one of the most popular databases overall.

This notebook goes over how to use `Redis` to store chat message history.

First we need to install dependencies, and start a redis instance using commands like: `redis-server`.

```
[HumanMessage(content='hi!'), AIMessage(content='whats up?')]
```

```
from typing import Optionalfrom langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholderfrom langchain_core.runnables.history import RunnableWithMessageHistoryfrom langchain_openai import ChatOpenAI
```

```
prompt = ChatPromptTemplate.from_messages(    [        ("system", "You're an assistant。"),        MessagesPlaceholder(variable_name="history"),        ("human", "{question}"),    ])chain = prompt | ChatOpenAI()chain_with_history = RunnableWithMessageHistory(    chain,    lambda session_id: RedisChatMessageHistory(        session_id, url="redis://localhost:6379"    ),    input_messages_key="question",    history_messages_key="history",)config = {"configurable": {"session_id": "foo"}}chain_with_history.invoke({"question": "Hi! I'm bob"}, config=config)chain_with_history.invoke({"question": "Whats my name"}, config=config)
```

```
AIMessage(content='Your name is Bob, as you mentioned earlier. Is there anything specific you would like assistance with, Bob?')
```
```
from langchain_community.chat_message_histories import (    RocksetChatMessageHistory,)from rockset import Regions, RocksetClienthistory = RocksetChatMessageHistory(    session_id="MySession",    client=RocksetClient(        api_key="YOUR API KEY",        host=Regions.usw2a1,  # us-west-2 Oregon    ),    collection="langchain_demo",    sync=True,)history.add_user_message("hi!")history.add_ai_message("whats up?")print(history.messages)
```

```
[    HumanMessage(content='hi!', additional_kwargs={'id': '2e62f1c2-e9f7-465e-b551-49bae07fe9f0'}, example=False),     AIMessage(content='whats up?', additional_kwargs={'id': 'b9be8eda-4c18-4cf8-81c3-e91e876927d0'}, example=False)]
```
## Remembrall

This page covers how to use the [Remembrall](https://remembrall.dev/) ecosystem within LangChain.

## What is Remembrall?[​](#what-is-remembrall "Direct link to What is Remembrall?")

Remembrall gives your language model long-term memory, retrieval augmented generation, and complete observability with just a few lines of code.

![Screenshot of the Remembrall dashboard showing request statistics and model interactions.](https://python.langchain.com/assets/images/RemembrallDashboard-0100fe50b3bc6728c8861c07f9bb2a1a.png "Remembrall Dashboard Interface")

It works as a light-weight proxy on top of your OpenAI calls and simply augments the context of the chat calls at runtime with relevant facts that have been collected.

## Setup[​](#setup "Direct link to Setup")

To get started, [sign in with Github on the Remembrall platform](https://remembrall.dev/login) and copy your [API key from the settings page](https://remembrall.dev/dashboard/settings).

Any request that you send with the modified `openai_api_base` (see below) and Remembrall API key will automatically be tracked in the Remembrall dashboard. You **never** have to share your OpenAI key with our platform and this information is **never** stored by the Remembrall systems.

To do this, we need to install the following dependencies:

```
pip install -U langchain-openai
```

### Enable Long Term Memory[​](#enable-long-term-memory "Direct link to Enable Long Term Memory")

In addition to setting the `openai_api_base` and Remembrall API key via `x-gp-api-key`, you should specify a UID to maintain memory for. This will usually be a unique user identifier (like email).

```
from langchain_openai import ChatOpenAIchat_model = ChatOpenAI(openai_api_base="https://remembrall.dev/api/openai/v1",                        model_kwargs={                            "headers":{                                "x-gp-api-key": "remembrall-api-key-here",                                "x-gp-remember": "user@email.com",                            }                        })chat_model.predict("My favorite color is blue.")import time; time.sleep(5)  # wait for system to save fact via auto saveprint(chat_model.predict("What is my favorite color?"))
```

### Enable Retrieval Augmented Generation[​](#enable-retrieval-augmented-generation "Direct link to Enable Retrieval Augmented Generation")

First, create a document context in the [Remembrall dashboard](https://remembrall.dev/dashboard/spells). Paste in the document texts or upload documents as PDFs to be processed. Save the Document Context ID and insert it as shown below.

```
from langchain_openai import ChatOpenAIchat_model = ChatOpenAI(openai_api_base="https://remembrall.dev/api/openai/v1",                        model_kwargs={                            "headers":{                                "x-gp-api-key": "remembrall-api-key-here",                                "x-gp-context": "document-context-id-goes-here",                            }                        })print(chat_model.predict("This is a question that can be answered with my document."))
```

* * *

#### Help us out by providing feedback on this documentation page:
## Anthropic

All functionality related to Anthropic models.

[Anthropic](https://www.anthropic.com/) is an AI safety and research company, and is the creator of Claude. This page covers all integrations between Anthropic models and LangChain.

## Installation[​](#installation "Direct link to Installation")

To use Anthropic models, you will need to install the `langchain-anthropic` package. You can do this with the following command:

```
pip install langchain-anthropic
```

## Environment Setup[​](#environment-setup "Direct link to Environment Setup")

To use Anthropic models, you will need to set the `ANTHROPIC_API_KEY` environment variable. You can get an Anthropic API key [here](https://console.anthropic.com/settings/keys)

## `ChatAnthropic`[​](#chatanthropic "Direct link to chatanthropic")

`ChatAnthropic` is a subclass of LangChain's `ChatModel`. You can import this wrapper with the following code:

```
from langchain_anthropic import ChatAnthropicmodel = ChatAnthropic(model='claude-3-opus-20240229')
```

Read more in the [ChatAnthropic documentation](https://python.langchain.com/docs/integrations/chat/anthropic/).

## \[Legacy\] `AnthropicLLM`[​](#legacy-anthropicllm "Direct link to legacy-anthropicllm")

`AnthropicLLM` is a subclass of LangChain's `LLM`. It is a wrapper around Anthropic's text-based completion endpoints.

```
from langchain_anthropic import AnthropicLLMmodel = AnthropicLLM(model='claude-2.1')
```

* * *

#### Help us out by providing feedback on this documentation page:
## Providers

LangChain integrates with many providers.

## Partner Packages[​](#partner-packages "Direct link to Partner Packages")

These providers have standalone `langchain-{provider}` packages for improved versioning, dependency management and testing.

*   [AI21](https://python.langchain.com/docs/integrations/providers/ai21/)
*   [Airbyte](https://python.langchain.com/docs/integrations/providers/airbyte/)
*   [Amazon Web Services](https://python.langchain.com/docs/integrations/platforms/aws/)
*   [Anthropic](https://python.langchain.com/docs/integrations/platforms/anthropic/)
*   [Astra DB](https://python.langchain.com/docs/integrations/providers/astradb/)
*   [Cohere](https://python.langchain.com/docs/integrations/providers/cohere/)
*   [Elasticsearch](https://python.langchain.com/docs/integrations/providers/elasticsearch/)
*   [Exa Search](https://python.langchain.com/docs/integrations/providers/exa_search/)
*   [Fireworks](https://python.langchain.com/docs/integrations/providers/fireworks/)
*   [Google](https://python.langchain.com/docs/integrations/platforms/google/)
*   [Groq](https://python.langchain.com/docs/integrations/providers/groq/)
*   [IBM](https://python.langchain.com/docs/integrations/providers/ibm/)
*   [MistralAI](https://python.langchain.com/docs/integrations/providers/mistralai/)
*   [MongoDB](https://python.langchain.com/docs/integrations/providers/mongodb_atlas/)
*   [Nomic](https://python.langchain.com/docs/integrations/providers/nomic/)
*   [Nvidia](https://python.langchain.com/docs/integrations/providers/nvidia/)
*   [OpenAI](https://python.langchain.com/docs/integrations/platforms/openai/)
*   [Pinecone](https://python.langchain.com/docs/integrations/providers/pinecone/)
*   [Robocorp](https://python.langchain.com/docs/integrations/providers/robocorp/)
*   [Together AI](https://python.langchain.com/docs/integrations/providers/together/)
*   [Upstage](https://python.langchain.com/docs/integrations/providers/upstage/)
*   [Voyage AI](https://python.langchain.com/docs/integrations/providers/voyageai/)

## Featured Community Providers[​](#featured-community-providers "Direct link to Featured Community Providers")

*   [Hugging Face](https://python.langchain.com/docs/integrations/platforms/huggingface/)
*   [Microsoft](https://python.langchain.com/docs/integrations/platforms/microsoft/)

## All Providers[​](#all-providers "Direct link to All Providers")

Click [here](https://python.langchain.com/docs/integrations/providers/) to see all providers.

* * *

#### Help us out by providing feedback on this documentation page: