seansullivan/Langchain-Docs huggingface dataset:
In this guide we will demonstrate how to use `StreamlitCallbackHandler` to display the thoughts and actions of an agent in an interactive Streamlit app. Try it out with the running app below using the MRKL agent:

You can run `streamlit hello` to load a sample app and validate your install succeeded. See full instructions in Streamlit's [Getting started documentation](https://docs.streamlit.io/library/get-started).

To create a `StreamlitCallbackHandler`, you just need to provide a parent container to render the output.

Additional keyword arguments to customize the display behavior are described in the [API reference](https://api.python.langchain.com/en/latest/callbacks/langchain.callbacks.streamlit.streamlit_callback_handler.StreamlitCallbackHandler.html).

The primary supported use case today is visualizing the actions of an Agent with Tools (or Agent Executor). You can create an agent in your Streamlit app and simply pass the `StreamlitCallbackHandler` to `agent.run()` in order to visualize the thoughts and actions live in your app.

```
import streamlit as stfrom langchain import hubfrom langchain.agents import AgentExecutor, create_react_agent, load_toolsfrom langchain_openai import OpenAIllm = OpenAI(temperature=0, streaming=True)tools = load_tools(["ddg-search"])prompt = hub.pull("hwchase17/react")agent = create_react_agent(llm, tools, prompt)agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)if prompt := st.chat_input():    st.chat_message("user").write(prompt)    with st.chat_message("assistant"):        st_callback = StreamlitCallbackHandler(st.container())        response = agent_executor.invoke(            {"input": prompt}, {"callbacks": [st_callback]}        )        st.write(response["output"])
```

**Note:** You will need to set `OPENAI_API_KEY` for the above app code to run successfully. The easiest way to do this is via [Streamlit secrets.toml](https://docs.streamlit.io/library/advanced-features/secrets-management), or any other local ENV management tool.

Currently `StreamlitCallbackHandler` is geared towards use with a LangChain Agent Executor. Support for additional agent types, use directly with Chains, etc will be added in the future.
## GMail

This loader goes over how to load data from GMail. There are many ways you could want to load data from GMail. This loader is currently fairly opinionated in how to do so. The way it does it is it first looks for all messages that you have sent. It then looks for messages where you are responding to a previous email. It then fetches that previous email, and creates a training example of that email, followed by your email.

Note that there are clear limitations here. For example, all examples created are only looking at the previous email for context.

To use:

*   Set up a Google Developer Account: Go to the Google Developer Console, create a project, and enable the Gmail API for that project. This will give you a credentials.json file that you’ll need later.
    
*   Install the Google Client Library: Run the following command to install the Google Client Library:
    

```
%pip install --upgrade --quiet  google-auth google-auth-oauthlib google-auth-httplib2 google-api-python-client
```

```
import os.pathfrom google.auth.transport.requests import Requestfrom google.oauth2.credentials import Credentialsfrom google_auth_oauthlib.flow import InstalledAppFlowSCOPES = ["https://www.googleapis.com/auth/gmail.readonly"]creds = None# The file token.json stores the user's access and refresh tokens, and is# created automatically when the authorization flow completes for the first# time.if os.path.exists("email_token.json"):    creds = Credentials.from_authorized_user_file("email_token.json", SCOPES)# If there are no (valid) credentials available, let the user log in.if not creds or not creds.valid:    if creds and creds.expired and creds.refresh_token:        creds.refresh(Request())    else:        flow = InstalledAppFlow.from_client_secrets_file(            # your creds file here. Please create json file as here https://cloud.google.com/docs/authentication/getting-started            "creds.json",            SCOPES,        )        creds = flow.run_local_server(port=0)    # Save the credentials for the next run    with open("email_token.json", "w") as token:        token.write(creds.to_json())
```

```
from langchain_community.chat_loaders.gmail import GMailLoader
```

```
loader = GMailLoader(creds=creds, n=3)
```

```
# Sometimes there can be errors which we silently ignorelen(data)
```

```
from langchain_community.chat_loaders.utils import (    map_ai_messages,)
```

```
# This makes messages sent by hchase@langchain.com the AI Messages# This means you will train an LLM to predict as if it's responding as hchasetraining_data = list(    map_ai_messages(data, sender="Harrison Chase <hchase@langchain.com>"))
```
This notebook shows how to load chat messages from Twitter to fine-tune on. We do this by utilizing Apify.

First, use Apify to export tweets. An example

```
# Filter out tweets that reference other tweets, because it's a bit weirdtweets = [d["full_text"] for d in data if "t.co" not in d["full_text"]]# Create them as AI messagesmessages = [AIMessage(content=t) for t in tweets]# Add in a system message at the start# TODO: we could try to extract the subject from the tweets, and put that in the system message.system_message = {"role": "system", "content": "write a tweet"}data = [[system_message, convert_message_to_dict(m)] for m in messages]
```
## ChatAI21

This notebook covers how to get started with AI21 chat models.

## Installation[​](#installation "Direct link to Installation")

```
!pip install -qU langchain-ai21
```

## Environment Setup[​](#environment-setup "Direct link to Environment Setup")

We’ll need to get a [AI21 API key](https://docs.ai21.com/) and set the `AI21_API_KEY` environment variable:

```
import osfrom getpass import getpassos.environ["AI21_API_KEY"] = getpass()
```

## Usage[​](#usage "Direct link to Usage")

```
from langchain_ai21 import ChatAI21from langchain_core.prompts import ChatPromptTemplatechat = ChatAI21(model="j2-ultra")prompt = ChatPromptTemplate.from_messages(    [        ("system", "You are a helpful assistant that translates English to French."),        ("human", "Translate this sentence from English to French. {english_text}."),    ])chain = prompt | chatchain.invoke({"english_text": "Hello, how are you?"})
```

```
AIMessage(content='Bonjour, comment vas-tu?')
```

* * *

#### Help us out by providing feedback on this documentation page:
## Alibaba Cloud PAI EAS

> [Alibaba Cloud PAI (Platform for AI)](https://www.alibabacloud.com/help/en/pai/?spm=a2c63.p38356.0.0.c26a426ckrxUwZ) is a lightweight and cost-efficient machine learning platform that uses cloud-native technologies. It provides you with an end-to-end modelling service. It accelerates model training based on tens of billions of features and hundreds of billions of samples in more than 100 scenarios.

> [Machine Learning Platform for AI of Alibaba Cloud](https://www.alibabacloud.com/help/en/machine-learning-platform-for-ai/latest/what-is-machine-learning-pai) is a machine learning or deep learning engineering platform intended for enterprises and developers. It provides easy-to-use, cost-effective, high-performance, and easy-to-scale plug-ins that can be applied to various industry scenarios. With over 140 built-in optimization algorithms, `Machine Learning Platform for AI` provides whole-process AI engineering capabilities including data labelling (`PAI-iTAG`), model building (`PAI-Designer` and `PAI-DSW`), model training (`PAI-DLC`), compilation optimization, and inference deployment (`PAI-EAS`).
> 
> `PAI-EAS` supports different types of hardware resources, including CPUs and GPUs, and features high throughput and low latency. It allows you to deploy large-scale complex models with a few clicks and perform elastic scale-ins and scale-outs in real-time. It also provides a comprehensive O&M and monitoring system.

## Setup EAS Service[​](#setup-eas-service "Direct link to Setup EAS Service")

Set up environment variables to init EAS service URL and token. Use [this document](https://www.alibabacloud.com/help/en/pai/user-guide/service-deployment/) for more information.

```
export EAS_SERVICE_URL=XXXexport EAS_SERVICE_TOKEN=XXX
```

Another option is to use this code:

```
import osfrom langchain_community.chat_models import PaiEasChatEndpointfrom langchain_core.language_models.chat_models import HumanMessageos.environ["EAS_SERVICE_URL"] = "Your_EAS_Service_URL"os.environ["EAS_SERVICE_TOKEN"] = "Your_EAS_Service_Token"chat = PaiEasChatEndpoint(    eas_service_url=os.environ["EAS_SERVICE_URL"],    eas_service_token=os.environ["EAS_SERVICE_TOKEN"],)
```

## Run Chat Model[​](#run-chat-model "Direct link to Run Chat Model")

You can use the default settings to call EAS service as follows:

```
output = chat([HumanMessage(content="write a funny joke")])print("output:", output)
```

Or, call EAS service with new inference params:

```
kwargs = {"temperature": 0.8, "top_p": 0.8, "top_k": 5}output = chat([HumanMessage(content="write a funny joke")], **kwargs)print("output:", output)
```

Or, run a stream call to get a stream response:

```
outputs = chat.stream([HumanMessage(content="hi")], streaming=True)for output in outputs:    print("stream output:", output)
```
danger

The Anthropic API officially supports tool-calling so this workaround is no longer needed. Please use [ChatAnthropic](https://python.langchain.com/docs/integrations/chat/anthropic/) with `langchain-anthropic>=0.1.5`.

This notebook shows how to use an experimental wrapper around Anthropic that gives it tool calling and structured output capabilities. It follows Anthropic’s guide [here](https://docs.anthropic.com/claude/docs/functions-external-tools)

The wrapper is available from the `langchain-anthropic` package, and it also requires the optional dependency `defusedxml` for parsing XML output from the llm.

Note: this is a beta feature that will be replaced by Anthropic’s formal implementation of tool calling, but it is useful for testing and experimentation in the meantime.

```
%pip install -qU langchain-anthropic defusedxmlfrom langchain_anthropic.experimental import ChatAnthropicTools
```

`ChatAnthropicTools` exposes a `bind_tools` method that allows you to pass in Pydantic models or BaseTools to the llm.

```
from langchain_core.pydantic_v1 import BaseModelclass Person(BaseModel):    name: str    age: intmodel = ChatAnthropicTools(model="claude-3-opus-20240229").bind_tools(tools=[Person])model.invoke("I am a 27 year old named Erick")
```

```
AIMessage(content='', additional_kwargs={'tool_calls': [{'function': {'name': 'Person', 'arguments': '{"name": "Erick", "age": "27"}'}, 'type': 'function'}]})
```

## Structured Output[​](#structured-output "Direct link to Structured Output")

`ChatAnthropicTools` also implements the [`with_structured_output` spec](https://python.langchain.com/docs/modules/model_io/chat/structured_output/) for extracting values. Note: this may not be as stable as with models that explicitly offer tool calling.

```
chain = ChatAnthropicTools(model="claude-3-opus-20240229").with_structured_output(    Person)chain.invoke("I am a 27 year old named Erick")
```

```
Person(name='Erick', age=27)
```
## Chat with Baichuan-192K

Baichuan chat models API by Baichuan Intelligent Technology. For more information, see [https://platform.baichuan-ai.com/docs/api](https://platform.baichuan-ai.com/docs/api)

```
from langchain_community.chat_models import ChatBaichuanfrom langchain_core.messages import HumanMessage
```

```
chat = ChatBaichuan(baichuan_api_key="YOUR_API_KEY")
```

Alternatively, you can set your API key with:

```
import osos.environ["BAICHUAN_API_KEY"] = "YOUR_API_KEY"
```

```
chat([HumanMessage(content="我日薪8块钱，请问在闰年的二月，我月薪多少")])
```

```
AIMessage(content='首先，我们需要确定闰年的二月有多少天。闰年的二月有29天。\n\n然后，我们可以计算你的月薪：\n\n日薪 = 月薪 / (当月天数)\n\n所以，你的月薪 = 日薪 * 当月天数\n\n将数值代入公式：\n\n月薪 = 8元/天 * 29天 = 232元\n\n因此，你在闰年的二月的月薪是232元。')
```

## Chat with Baichuan-192K with Streaming[​](#chat-with-baichuan-192k-with-streaming "Direct link to Chat with Baichuan-192K with Streaming")

```
chat = ChatBaichuan(    baichuan_api_key="YOUR_API_KEY",    streaming=True,)
```

```
chat([HumanMessage(content="我日薪8块钱，请问在闰年的二月，我月薪多少")])
```

```
AIMessageChunk(content='首先，我们需要确定闰年的二月有多少天。闰年的二月有29天。\n\n然后，我们可以计算你的月薪：\n\n日薪 = 月薪 / (当月天数)\n\n所以，你的月薪 = 日薪 * 当月天数\n\n将数值代入公式：\n\n月薪 = 8元/天 * 29天 = 232元\n\n因此，你在闰年的二月的月薪是232元。')
```

* * *

#### Help us out by providing feedback on this documentation page:
## ChatBedrock

> [Amazon Bedrock](https://aws.amazon.com/bedrock/) is a fully managed service that offers a choice of high-performing foundation models (FMs) from leading AI companies like `AI21 Labs`, `Anthropic`, `Cohere`, `Meta`, `Stability AI`, and `Amazon` via a single API, along with a broad set of capabilities you need to build generative AI applications with security, privacy, and responsible AI. Using `Amazon Bedrock`, you can easily experiment with and evaluate top FMs for your use case, privately customize them with your data using techniques such as fine-tuning and `Retrieval Augmented Generation` (`RAG`), and build agents that execute tasks using your enterprise systems and data sources. Since `Amazon Bedrock` is serverless, you don’t have to manage any infrastructure, and you can securely integrate and deploy generative AI capabilities into your applications using the AWS services you are already familiar with.

```
%pip install --upgrade --quiet  langchain-aws
```

```
Note: you may need to restart the kernel to use updated packages.
```

```
from langchain_aws import ChatBedrockfrom langchain_core.messages import HumanMessage
```

```
chat = ChatBedrock(    model_id="anthropic.claude-3-sonnet-20240229-v1:0",    model_kwargs={"temperature": 0.1},)
```

```
messages = [    HumanMessage(        content="Translate this sentence from English to French. I love programming."    )]chat.invoke(messages)
```

```
AIMessage(content="Voici la traduction en français :\n\nJ'aime la programmation.", additional_kwargs={'usage': {'prompt_tokens': 20, 'completion_tokens': 21, 'total_tokens': 41}}, response_metadata={'model_id': 'anthropic.claude-3-sonnet-20240229-v1:0', 'usage': {'prompt_tokens': 20, 'completion_tokens': 21, 'total_tokens': 41}}, id='run-994f0362-0e50-4524-afad-3c4f5bb11328-0')
```

### Streaming[​](#streaming "Direct link to Streaming")

To stream responses, you can use the runnable `.stream()` method.

```
for chunk in chat.stream(messages):    print(chunk.content, end="", flush=True)
```

```
Voici la traduction en français :J'aime la programmation.
```

* * *

#### Help us out by providing feedback on this documentation page:
## Dappier AI

**Dappier: Powering AI with Dynamic, Real-Time Data Models**

Dappier offers a cutting-edge platform that grants developers immediate access to a wide array of real-time data models spanning news, entertainment, finance, market data, weather, and beyond. With our pre-trained data models, you can supercharge your AI applications, ensuring they deliver precise, up-to-date responses and minimize inaccuracies.

Dappier data models help you build next-gen LLM apps with trusted, up-to-date content from the world’s leading brands. Unleash your creativity and enhance any GPT App or AI workflow with actionable, proprietary, data through a simple API. Augment your AI with proprietary data from trusted sources is the best way to ensure factual, up-to-date, responses with fewer hallucinations no matter the question.

For Developers, By Developers Designed with developers in mind, Dappier simplifies the journey from data integration to monetization, providing clear, straightforward paths to deploy and earn from your AI models. Experience the future of monetization infrastructure for the new internet at **[https://dappier.com/](https://dappier.com/)**.

This example goes over how to use LangChain to interact with Dappier AI models

* * *

To use one of our Dappier AI Data Models, you will need an API key. Please visit Dappier Platform ([https://platform.dappier.com/](https://platform.dappier.com/)) to log in and create an API key in your profile.

You can find more details on the API reference : [https://docs.dappier.com/introduction](https://docs.dappier.com/introduction)

To work with our Dappier Chat Model you can pass the key directly through the parameter named dappier\_api\_key when initiating the class or set as an environment variable.

```
export DAPPIER_API_KEY="..."
```

```
from langchain_community.chat_models.dappier import ChatDappierAIfrom langchain_core.messages import HumanMessage
```

```
chat = ChatDappierAI(    dappier_endpoint="https://api.dappier.com/app/datamodelconversation",    dappier_model="dm_01hpsxyfm2fwdt2zet9cg6fdxt",    dappier_api_key="...",)
```

```
messages = [HumanMessage(content="Who won the super bowl in 2024?")]chat.invoke(messages)
```

```
AIMessage(content='Hey there! The Kansas City Chiefs won Super Bowl LVIII in 2024. They beat the San Francisco 49ers in overtime with a final score of 25-22. It was quite the game! 🏈')
```

```
await chat.ainvoke(messages)
```

```
AIMessage(content='The Kansas City Chiefs won Super Bowl LVIII in 2024! 🏈')
```
## DeepInfra

[DeepInfra](https://deepinfra.com/?utm_source=langchain) is a serverless inference as a service that provides access to a [variety of LLMs](https://deepinfra.com/models?utm_source=langchain) and [embeddings models](https://deepinfra.com/models?type=embeddings&utm_source=langchain). This notebook goes over how to use LangChain with DeepInfra for chat models.

## Set the Environment API Key[​](#set-the-environment-api-key "Direct link to Set the Environment API Key")

Make sure to get your API key from DeepInfra. You have to [Login](https://deepinfra.com/login?from=%2Fdash) and get a new token.

You are given a 1 hour free of serverless GPU compute to test different models. (see [here](https://github.com/deepinfra/deepctl#deepctl)) You can print your token with `deepctl auth token`

```
# get a new token: https://deepinfra.com/login?from=%2Fdashfrom getpass import getpassDEEPINFRA_API_TOKEN = getpass()
```

```
import os# or pass deepinfra_api_token parameter to the ChatDeepInfra constructoros.environ["DEEPINFRA_API_TOKEN"] = DEEPINFRA_API_TOKEN
```

```
from langchain_community.chat_models import ChatDeepInfrafrom langchain_core.messages import HumanMessage
```

```
chat = ChatDeepInfra(model="meta-llama/Llama-2-7b-chat-hf")
```

```
messages = [    HumanMessage(        content="Translate this sentence from English to French. I love programming."    )]chat(messages)
```

```
AIMessage(content=" J'aime la programmation.", additional_kwargs={}, example=False)
```

## `ChatDeepInfra` also supports async and streaming functionality:[​](#chatdeepinfra-also-supports-async-and-streaming-functionality "Direct link to chatdeepinfra-also-supports-async-and-streaming-functionality")

```
from langchain.callbacks.streaming_stdout import StreamingStdOutCallbackHandler
```

```
await chat.agenerate([messages])
```

```
LLMResult(generations=[[ChatGeneration(text=" J'aime programmer.", generation_info=None, message=AIMessage(content=" J'aime programmer.", additional_kwargs={}, example=False))]], llm_output={}, run=[RunInfo(run_id=UUID('8cc8fb68-1c35-439c-96a0-695036a93652'))])
```

```
chat = ChatDeepInfra(    streaming=True,    verbose=True,    callbacks=[StreamingStdOutCallbackHandler()],)chat(messages)
```

```
AIMessage(content=" J'aime la programmation.", additional_kwargs={}, example=False)
```
## ErnieBotChat

[ERNIE-Bot](https://cloud.baidu.com/doc/WENXINWORKSHOP/s/jlil56u11) is a large language model developed by Baidu, covering a huge amount of Chinese data. This notebook covers how to get started with ErnieBot chat models.

**Deprecated Warning**

We recommend users using `langchain_community.chat_models.ErnieBotChat` to use `langchain_community.chat_models.QianfanChatEndpoint` instead.

documentation for `QianfanChatEndpoint` is [here](https://python.langchain.com/docs/integrations/chat/baidu_qianfan_endpoint/).

they are 4 why we recommend users to use `QianfanChatEndpoint`:

1.  `QianfanChatEndpoint` support more LLM in the Qianfan platform.
2.  `QianfanChatEndpoint` support streaming mode.
3.  `QianfanChatEndpoint` support function calling usgage.
4.  `ErnieBotChat` is lack of maintenance and deprecated.

Some tips for migration:

*   change `ernie_client_id` to `qianfan_ak`, also change `ernie_client_secret` to `qianfan_sk`.
*   install `qianfan` package. like `pip install qianfan`
*   change `ErnieBotChat` to `QianfanChatEndpoint`.

```
from langchain_community.chat_models.baidu_qianfan_endpoint import QianfanChatEndpointchat = QianfanChatEndpoint(    qianfan_ak="your qianfan ak",    qianfan_sk="your qianfan sk",)
```

## Usage[​](#usage "Direct link to Usage")

```
from langchain_community.chat_models import ErnieBotChatfrom langchain_core.messages import HumanMessagechat = ErnieBotChat(    ernie_client_id="YOUR_CLIENT_ID", ernie_client_secret="YOUR_CLIENT_SECRET")
```

or you can set `client_id` and `client_secret` in your environment variables

```
export ERNIE_CLIENT_ID=YOUR_CLIENT_IDexport ERNIE_CLIENT_SECRET=YOUR_CLIENT_SECRET
```

```
chat([HumanMessage(content="hello there, who are you?")])
```

```
AIMessage(content='Hello, I am an artificial intelligence language model. My purpose is to help users answer questions or provide information. What can I do for you?', additional_kwargs={}, example=False)
```
## ChatFireworks

> [Fireworks](https://app.fireworks.ai/) accelerates product development on generative AI by creating an innovative AI experiment and production platform.

This example goes over how to use LangChain to interact with `ChatFireworks` models.

%pip install langchain-fireworks

```
from langchain_core.messages import HumanMessage, SystemMessagefrom langchain_fireworks import ChatFireworks
```

## Setup

1.  Make sure the `langchain-fireworks` package is installed in your environment.
2.  Sign in to [Fireworks AI](http://fireworks.ai/) for the an API Key to access our models, and make sure it is set as the `FIREWORKS_API_KEY` environment variable.
3.  Set up your model using a model id. If the model is not set, the default model is fireworks-llama-v2-7b-chat. See the full, most up-to-date model list on [app.fireworks.ai](https://app.fireworks.ai/).

```
import getpassimport osif "FIREWORKS_API_KEY" not in os.environ:    os.environ["FIREWORKS_API_KEY"] = getpass.getpass("Fireworks API Key:")# Initialize a Fireworks chat modelchat = ChatFireworks(model="accounts/fireworks/models/mixtral-8x7b-instruct")
```

## Calling the Model Directly

You can call the model directly with a system and human message to get answers.

```
# ChatFireworks Wrappersystem_message = SystemMessage(content="You are to chat with the user.")human_message = HumanMessage(content="Who are you?")chat.invoke([system_message, human_message])
```

```
AIMessage(content="Hello! I'm an AI language model, a helpful assistant designed to chat and assist you with any questions or information you might need. I'm here to make your experience as smooth and enjoyable as possible. How can I assist you today?")
```

```
# Setting additional parameters: temperature, max_tokens, top_pchat = ChatFireworks(    model="accounts/fireworks/models/mixtral-8x7b-instruct",    temperature=1,    max_tokens=20,)system_message = SystemMessage(content="You are to chat with the user.")human_message = HumanMessage(content="How's the weather today?")chat.invoke([system_message, human_message])
```

```
AIMessage(content="I'm an AI and do not have the ability to experience the weather firsthand. However,")
```

## Tool Calling

Fireworks offers the [`FireFunction-v1` tool calling model](https://fireworks.ai/blog/firefunction-v1-gpt-4-level-function-calling). You can use it for structured output and function calling use cases:

```
from pprint import pprintfrom langchain_core.pydantic_v1 import BaseModelclass ExtractFields(BaseModel):    name: str    age: intchat = ChatFireworks(    model="accounts/fireworks/models/firefunction-v1",).bind_tools([ExtractFields])result = chat.invoke("I am a 27 year old named Erick")pprint(result.additional_kwargs["tool_calls"][0])
```

```
{'function': {'arguments': '{"name": "Erick", "age": 27}',              'name': 'ExtractFields'}, 'id': 'call_J0WYP2TLenaFw3UeVU0UnWqx', 'index': 0, 'type': 'function'}
```
## PromptLayerChatOpenAI

This example showcases how to connect to [PromptLayer](https://www.promptlayer.com/) to start recording your ChatOpenAI requests.

## Install PromptLayer[​](#install-promptlayer "Direct link to Install PromptLayer")

The `promptlayer` package is required to use PromptLayer with OpenAI. Install `promptlayer` using pip.

## Imports[​](#imports "Direct link to Imports")

```
import osfrom langchain_community.chat_models import PromptLayerChatOpenAIfrom langchain_core.messages import HumanMessage
```

## Set the Environment API Key[​](#set-the-environment-api-key "Direct link to Set the Environment API Key")

You can create a PromptLayer API Key at [www.promptlayer.com](https://www.promptlayer.com/) by clicking the settings cog in the navbar.

Set it as an environment variable called `PROMPTLAYER_API_KEY`.

```
os.environ["PROMPTLAYER_API_KEY"] = "**********"
```

## Use the PromptLayerOpenAI LLM like normal[​](#use-the-promptlayeropenai-llm-like-normal "Direct link to Use the PromptLayerOpenAI LLM like normal")

_You can optionally pass in `pl_tags` to track your requests with PromptLayer’s tagging feature._

```
chat = PromptLayerChatOpenAI(pl_tags=["langchain"])chat([HumanMessage(content="I am a cat and I want")])
```

```
AIMessage(content='to take a nap in a cozy spot. I search around for a suitable place and finally settle on a soft cushion on the window sill. I curl up into a ball and close my eyes, relishing the warmth of the sun on my fur. As I drift off to sleep, I can hear the birds chirping outside and feel the gentle breeze blowing through the window. This is the life of a contented cat.', additional_kwargs={})
```

**The above request should now appear on your [PromptLayer dashboard](https://www.promptlayer.com/).**

## Using PromptLayer Track[​](#using-promptlayer-track "Direct link to Using PromptLayer Track")

If you would like to use any of the [PromptLayer tracking features](https://magniv.notion.site/Track-4deee1b1f7a34c1680d085f82567dab9), you need to pass the argument `return_pl_id` when instantiating the PromptLayer LLM to get the request id.

```
import promptlayerchat = PromptLayerChatOpenAI(return_pl_id=True)chat_results = chat.generate([[HumanMessage(content="I am a cat and I want")]])for res in chat_results.generations:    pl_request_id = res[0].generation_info["pl_request_id"]    promptlayer.track.score(request_id=pl_request_id, score=100)
```

Using this allows you to track the performance of your model in the PromptLayer dashboard. If you are using a prompt template, you can attach a template to a request as well. Overall, this gives you the opportunity to track the performance of different templates and models in the PromptLayer dashboard.
## GigaChat

This notebook shows how to use LangChain with [GigaChat](https://developers.sber.ru/portal/products/gigachat). To use you need to install `gigachat` python package.

```
%pip install --upgrade --quiet  gigachat
```

To get GigaChat credentials you need to [create account](https://developers.sber.ru/studio/login) and [get access to API](https://developers.sber.ru/docs/ru/gigachat/individuals-quickstart)

## Example[​](#example "Direct link to Example")

```
import osfrom getpass import getpassos.environ["GIGACHAT_CREDENTIALS"] = getpass()
```

```
from langchain_community.chat_models import GigaChatchat = GigaChat(verify_ssl_certs=False, scope="GIGACHAT_API_PERS")
```

```
from langchain_core.messages import HumanMessage, SystemMessagemessages = [    SystemMessage(        content="You are a helpful AI that shares everything you know. Talk in English."    ),    HumanMessage(content="What is capital of Russia?"),]print(chat.invoke(messages).content)
```

```
The capital of Russia is Moscow.
```

* * *

#### Help us out by providing feedback on this documentation page:
## SparkLLM Chat

SparkLLM chat models API by iFlyTek. For more information, see [iFlyTek Open Platform](https://www.xfyun.cn/).

## Basic use[​](#basic-use "Direct link to Basic use")

```
"""For basic init and call"""from langchain_community.chat_models import ChatSparkLLMfrom langchain_core.messages import HumanMessagechat = ChatSparkLLM(    spark_app_id="<app_id>", spark_api_key="<api_key>", spark_api_secret="<api_secret>")message = HumanMessage(content="Hello")chat([message])
```

```
AIMessage(content='Hello! How can I help you today?')
```

*   Get SparkLLM’s app\_id, api\_key and api\_secret from [iFlyTek SparkLLM API Console](https://console.xfyun.cn/services/bm3) (for more info, see [iFlyTek SparkLLM Intro](https://xinghuo.xfyun.cn/sparkapi) ), then set environment variables `IFLYTEK_SPARK_APP_ID`, `IFLYTEK_SPARK_API_KEY` and `IFLYTEK_SPARK_API_SECRET` or pass parameters when creating `ChatSparkLLM` as the demo above.

## For ChatSparkLLM with Streaming[​](#for-chatsparkllm-with-streaming "Direct link to For ChatSparkLLM with Streaming")

```
chat = ChatSparkLLM(    spark_app_id="<app_id>",    spark_api_key="<api_key>",    spark_api_secret="<api_secret>",    streaming=True,)for chunk in chat.stream("Hello!"):    print(chunk.content, end="")
```

```
Hello! How can I help you today?
```

## For v2[​](#for-v2 "Direct link to For v2")

```
"""For basic init and call"""from langchain_community.chat_models import ChatSparkLLMfrom langchain_core.messages import HumanMessagechat = ChatSparkLLM(    spark_app_id="<app_id>",    spark_api_key="<api_key>",    spark_api_secret="<api_secret>",    spark_api_url="wss://spark-api.xf-yun.com/v2.1/chat",    spark_llm_domain="generalv2",)message = HumanMessage(content="Hello")chat([message])
```

* * *

#### Help us out by providing feedback on this documentation page:
## Tencent Hunyuan

> [Tencent’s hybrid model API](https://cloud.tencent.com/document/product/1729) (`Hunyuan API`) implements dialogue communication, content generation, analysis and understanding, and can be widely used in various scenarios such as intelligent customer service, intelligent marketing, role playing, advertising copywriting, product description, script creation, resume generation, article writing, code generation, data analysis, and content analysis.

See for [more information](https://cloud.tencent.com/document/product/1729).

```
from langchain_community.chat_models import ChatHunyuanfrom langchain_core.messages import HumanMessage
```

```
chat = ChatHunyuan(    hunyuan_app_id=111111111,    hunyuan_secret_id="YOUR_SECRET_ID",    hunyuan_secret_key="YOUR_SECRET_KEY",)
```

```
chat(    [        HumanMessage(            content="You are a helpful assistant that translates English to French.Translate this sentence from English to French. I love programming."        )    ])
```

```
AIMessage(content="J'aime programmer.")
```

## For ChatHunyuan with Streaming[​](#for-chathunyuan-with-streaming "Direct link to For ChatHunyuan with Streaming")

```
chat = ChatHunyuan(    hunyuan_app_id="YOUR_APP_ID",    hunyuan_secret_id="YOUR_SECRET_ID",    hunyuan_secret_key="YOUR_SECRET_KEY",    streaming=True,)
```

```
chat(    [        HumanMessage(            content="You are a helpful assistant that translates English to French.Translate this sentence from English to French. I love programming."        )    ])
```

```
AIMessageChunk(content="J'aime programmer.")
```

* * *

#### Help us out by providing feedback on this documentation page:
## ChatTongyi

Tongyi Qwen is a large language model developed by Alibaba’s Damo Academy. It is capable of understanding user intent through natural language understanding and semantic analysis, based on user input in natural language. It provides services and assistance to users in different domains and tasks. By providing clear and detailed instructions, you can obtain results that better align with your expectations. In this notebook, we will introduce how to use langchain with [Tongyi](https://www.aliyun.com/product/dashscope) mainly in `Chat` corresponding to the package `langchain/chat_models` in langchain

```
# Install the package%pip install --upgrade --quiet  dashscope
```

```
# Get a new token: https://help.aliyun.com/document_detail/611472.html?spm=a2c4g.2399481.0.0from getpass import getpassDASHSCOPE_API_KEY = getpass()
```

```
import osos.environ["DASHSCOPE_API_KEY"] = DASHSCOPE_API_KEY
```

```
from langchain_community.chat_models.tongyi import ChatTongyifrom langchain_core.messages import HumanMessagechatLLM = ChatTongyi(    streaming=True,)res = chatLLM.stream([HumanMessage(content="hi")], streaming=True)for r in res:    print("chat resp:", r)
```

```
chat resp: content='Hello! How' additional_kwargs={} example=Falsechat resp: content=' can I assist you today?' additional_kwargs={} example=False
```

```
from langchain_core.messages import HumanMessage, SystemMessagemessages = [    SystemMessage(        content="You are a helpful assistant that translates English to French."    ),    HumanMessage(        content="Translate this sentence from English to French. I love programming."    ),]chatLLM(messages)
```

```
AIMessageChunk(content="J'aime programmer.", additional_kwargs={}, example=False)
```

* * *

#### Help us out by providing feedback on this documentation page:
Alternatively, you may configure the API key when you initialize ChatGroq.

If you do not want to set your API key in the environment, you can pass it directly to the client:

```
AIMessage(content='Low Latency Large Language Models (LLMs) are a type of artificial intelligence model that can understand and generate human-like text. The term "low latency" refers to the model\'s ability to process and respond to inputs quickly, with minimal delay.\n\nThe importance of low latency in LLMs can be explained through the following points:\n\n1. Improved user experience: In real-time applications such as chatbots, virtual assistants, and interactive games, users expect quick and responsive interactions. Low latency LLMs can provide instant feedback and responses, creating a more seamless and engaging user experience.\n\n2. Better decision-making: In time-sensitive scenarios, such as financial trading or autonomous vehicles, low latency LLMs can quickly process and analyze vast amounts of data, enabling faster and more informed decision-making.\n\n3. Enhanced accessibility: For individuals with disabilities, low latency LLMs can help create more responsive and inclusive interfaces, such as voice-controlled assistants or real-time captioning systems.\n\n4. Competitive advantage: In industries where real-time data analysis and decision-making are crucial, low latency LLMs can provide a competitive edge by enabling businesses to react more quickly to market changes, customer needs, or emerging opportunities.\n\n5. Scalability: Low latency LLMs can efficiently handle a higher volume of requests and interactions, making them more suitable for large-scale applications and services.\n\nIn summary, low latency is an essential aspect of LLMs, as it significantly impacts user experience, decision-making, accessibility, competitiveness, and scalability. By minimizing delays and response times, low latency LLMs can unlock new possibilities and applications for artificial intelligence in various industries and scenarios.')
```

```
AIMessage(content="There's a star that shines up in the sky,\nThe Sun, that makes the day bright and spry.\nIt rises and sets,\nIn a daily, predictable bet,\nGiving life to the world, oh my!")
```

```
The moon's gentle glowIlluminates the night skyPeaceful and serene
```
## ChatUpstage

This notebook covers how to get started with Upstage chat models.

## Installation[​](#installation "Direct link to Installation")

Install `langchain-upstage` package.

```
pip install -U langchain-upstage
```

## Environment Setup[​](#environment-setup "Direct link to Environment Setup")

Make sure to set the following environment variables:

*   `UPSTAGE_API_KEY`: Your Upstage API key from [Upstage console](https://console.upstage.ai/).

## Usage[​](#usage "Direct link to Usage")

```
import osos.environ["UPSTAGE_API_KEY"] = "YOUR_API_KEY"
```

```
from langchain_core.prompts import ChatPromptTemplatefrom langchain_upstage import ChatUpstagechat = ChatUpstage()
```

```
# using chat invokechat.invoke("Hello, how are you?")
```

```
# using chat streamfor m in chat.stream("Hello, how are you?"):    print(m)
```

## Chaining[​](#chaining "Direct link to Chaining")

```
# using chainprompt = ChatPromptTemplate.from_messages(    [        ("system", "You are a helpful assistant that translates English to French."),        ("human", "Translate this sentence from English to French. {english_text}."),    ])chain = prompt | chatchain.invoke({"english_text": "Hello, how are you?"})
```

* * *

#### Help us out by providing feedback on this documentation page:
## ChatYandexGPT

This notebook goes over how to use Langchain with [YandexGPT](https://cloud.yandex.com/en/services/yandexgpt) chat model.

To use, you should have the `yandexcloud` python package installed.

```
%pip install --upgrade --quiet  yandexcloud
```

First, you should [create service account](https://cloud.yandex.com/en/docs/iam/operations/sa/create) with the `ai.languageModels.user` role.

Next, you have two authentication options: - [IAM token](https://cloud.yandex.com/en/docs/iam/operations/iam-token/create-for-sa). You can specify the token in a constructor parameter `iam_token` or in an environment variable `YC_IAM_TOKEN`.

*   [API key](https://cloud.yandex.com/en/docs/iam/operations/api-key/create) You can specify the key in a constructor parameter `api_key` or in an environment variable `YC_API_KEY`.

To specify the model you can use `model_uri` parameter, see [the documentation](https://cloud.yandex.com/en/docs/yandexgpt/concepts/models#yandexgpt-generation) for more details.

By default, the latest version of `yandexgpt-lite` is used from the folder specified in the parameter `folder_id` or `YC_FOLDER_ID` environment variable.

```
from langchain_community.chat_models import ChatYandexGPTfrom langchain_core.messages import HumanMessage, SystemMessage
```

```
chat_model = ChatYandexGPT()
```

```
answer = chat_model.invoke(    [        SystemMessage(            content="You are a helpful assistant that translates English to French."        ),        HumanMessage(content="I love programming."),    ])answer
```

```
AIMessage(content='Je adore le programmement.')
```
## vLLM Chat

vLLM can be deployed as a server that mimics the OpenAI API protocol. This allows vLLM to be used as a drop-in replacement for applications using OpenAI API. This server can be queried in the same format as OpenAI API.

This notebook covers how to get started with vLLM chat models using langchain’s `ChatOpenAI` **as it is**.

```
from langchain_core.messages import HumanMessage, SystemMessagefrom langchain_core.prompts.chat import (    ChatPromptTemplate,    HumanMessagePromptTemplate,    SystemMessagePromptTemplate,)from langchain_openai import ChatOpenAI
```

```
inference_server_url = "http://localhost:8000/v1"chat = ChatOpenAI(    model="mosaicml/mpt-7b",    openai_api_key="EMPTY",    openai_api_base=inference_server_url,    max_tokens=5,    temperature=0,)
```

```
messages = [    SystemMessage(        content="You are a helpful assistant that translates English to Italian."    ),    HumanMessage(        content="Translate the following sentence from English to Italian: I love programming."    ),]chat(messages)
```

```
AIMessage(content=' Io amo programmare', additional_kwargs={}, example=False)
```

You can make use of templating by using a `MessagePromptTemplate`. You can build a `ChatPromptTemplate` from one or more `MessagePromptTemplates`. You can use ChatPromptTemplate’s format\_prompt – this returns a `PromptValue`, which you can convert to a string or `Message` object, depending on whether you want to use the formatted value as input to an llm or chat model.

For convenience, there is a `from_template` method exposed on the template. If you were to use this template, this is what it would look like:

```
template = (    "You are a helpful assistant that translates {input_language} to {output_language}.")system_message_prompt = SystemMessagePromptTemplate.from_template(template)human_template = "{text}"human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)
```

```
chat_prompt = ChatPromptTemplate.from_messages(    [system_message_prompt, human_message_prompt])# get a chat completion from the formatted messageschat(    chat_prompt.format_prompt(        input_language="English", output_language="Italian", text="I love programming."    ).to_messages())
```

```
AIMessage(content=' I love programming too.', additional_kwargs={}, example=False)
```
## JinaChat

This notebook covers how to get started with JinaChat chat models.

```
from langchain_community.chat_models import JinaChatfrom langchain_core.messages import HumanMessage, SystemMessagefrom langchain_core.prompts.chat import (    ChatPromptTemplate,    HumanMessagePromptTemplate,    SystemMessagePromptTemplate,)
```

```
chat = JinaChat(temperature=0)
```

```
messages = [    SystemMessage(        content="You are a helpful assistant that translates English to French."    ),    HumanMessage(        content="Translate this sentence from English to French. I love programming."    ),]chat(messages)
```

```
AIMessage(content="J'aime programmer.", additional_kwargs={}, example=False)
```

You can make use of templating by using a `MessagePromptTemplate`. You can build a `ChatPromptTemplate` from one or more `MessagePromptTemplates`. You can use `ChatPromptTemplate`’s `format_prompt` – this returns a `PromptValue`, which you can convert to a string or Message object, depending on whether you want to use the formatted value as input to an llm or chat model.

For convenience, there is a `from_template` method exposed on the template. If you were to use this template, this is what it would look like:

```
template = (    "You are a helpful assistant that translates {input_language} to {output_language}.")system_message_prompt = SystemMessagePromptTemplate.from_template(template)human_template = "{text}"human_message_prompt = HumanMessagePromptTemplate.from_template(human_template)
```

```
chat_prompt = ChatPromptTemplate.from_messages(    [system_message_prompt, human_message_prompt])# get a chat completion from the formatted messageschat(    chat_prompt.format_prompt(        input_language="English", output_language="French", text="I love programming."    ).to_messages())
```

```
AIMessage(content="J'aime programmer.", additional_kwargs={}, example=False)
```
## VolcEngineMaasChat

This notebook provides you with a guide on how to get started with volc engine maas chat models.

```
# Install the package%pip install --upgrade --quiet  volcengine
```

```
from langchain_community.chat_models import VolcEngineMaasChatfrom langchain_core.messages import HumanMessage
```

```
chat = VolcEngineMaasChat(volc_engine_maas_ak="your ak", volc_engine_maas_sk="your sk")
```

or you can set access\_key and secret\_key in your environment variables

```
export VOLC_ACCESSKEY=YOUR_AKexport VOLC_SECRETKEY=YOUR_SK
```

```
chat([HumanMessage(content="给我讲个笑话")])
```

```
AIMessage(content='好的，这是一个笑话：\n\n为什么鸟儿不会玩电脑游戏？\n\n因为它们没有翅膀！')
```

## volc engine maas chat with stream

```
chat = VolcEngineMaasChat(    volc_engine_maas_ak="your ak",    volc_engine_maas_sk="your sk",    streaming=True,)
```

```
chat([HumanMessage(content="给我讲个笑话")])
```

```
AIMessage(content='好的，这是一个笑话：\n\n三岁的女儿说她会造句了，妈妈让她用“年轻”造句，女儿说：“妈妈减肥，一年轻了好几斤”。')
```

* * *

#### Help us out by providing feedback on this documentation page:
[LiteLLM](https://github.com/BerriAI/litellm) is a library that simplifies calling Anthropic, Azure, Huggingface, Replicate, etc.

This notebook covers how to get started with using Langchain + the LiteLLM Router I/O library.

```
model_list = [    {        "model_name": "gpt-4",        "litellm_params": {            "model": "azure/gpt-4-1106-preview",            "api_key": "<your-api-key>",            "api_version": "2023-05-15",            "api_base": "https://<your-endpoint>.openai.azure.com/",        },    },    {        "model_name": "gpt-4",        "litellm_params": {            "model": "azure/gpt-4-1106-preview",            "api_key": "<your-api-key>",            "api_version": "2023-05-15",            "api_base": "https://<your-endpoint>.openai.azure.com/",        },    },]litellm_router = Router(model_list=model_list)chat = ChatLiteLLMRouter(router=litellm_router)
```

```
AIMessage(content="J'aime programmer.")
```

```
LLMResult(generations=[[ChatGeneration(text="J'adore programmer.", generation_info={'finish_reason': 'stop'}, message=AIMessage(content="J'adore programmer."))]], llm_output={'token_usage': {'completion_tokens': 6, 'prompt_tokens': 19, 'total_tokens': 25}, 'model_name': None}, run=[RunInfo(run_id=UUID('75003ec9-1e2b-43b7-a216-10dcc0f75e00'))])
```

```
chat = ChatLiteLLMRouter(    router=litellm_router,    streaming=True,    verbose=True,    callback_manager=CallbackManager([StreamingStdOutCallbackHandler()]),)chat(messages)
```

```
AIMessage(content="J'adore programmer.")
```
This example goes over how to use LangChain to interact with `Konko` ChatCompletion [models](https://docs.konko.ai/docs/list-of-models#konko-hosted-models-for-chatcompletion)

To run this notebook, you’ll need Konko API key. Sign in to our web app to [create an API key](https://platform.konko.ai/settings/api-keys) to access models

Another way to find the list of models running on the Konko instance is through this [endpoint](https://docs.konko.ai/reference/get-models).

```
AIMessage(content="  Sure thing! The Big Bang Theory is a scientific theory that explains the origins of the universe. In short, it suggests that the universe began as an infinitely hot and dense point around 13.8 billion years ago and expanded rapidly. This expansion continues to this day, and it's what makes the universe look the way it does.\n\nHere's a brief overview of the key points:\n\n1. The universe started as a singularity, a point of infinite density and temperature.\n2. The singularity expanded rapidly, causing the universe to cool and expand.\n3. As the universe expanded, particles began to form, including protons, neutrons, and electrons.\n4. These particles eventually came together to form atoms, and later, stars and galaxies.\n5. The universe is still expanding today, and the rate of this expansion is accelerating.\n\nThat's the Big Bang Theory in a nutshell! It's a pretty mind-blowing idea when you think about it, and it's supported by a lot of scientific evidence. Do you have any other questions about it?")
```
## ChatLlamaAPI

This notebook shows how to use LangChain with [LlamaAPI](https://llama-api.com/) - a hosted version of Llama2 that adds in support for function calling.

%pip install –upgrade –quiet llamaapi

```
from llamaapi import LlamaAPI# Replace 'Your_API_Token' with your actual API tokenllama = LlamaAPI("Your_API_Token")
```

```
from langchain_experimental.llms import ChatLlamaAPI
```

```
/Users/harrisonchase/.pyenv/versions/3.9.1/envs/langchain/lib/python3.9/site-packages/deeplake/util/check_latest_version.py:32: UserWarning: A newer version of deeplake (3.6.12) is available. It's recommended that you update to the latest version using `pip install -U deeplake`.  warnings.warn(
```

```
model = ChatLlamaAPI(client=llama)
```

```
from langchain.chains import create_tagging_chainschema = {    "properties": {        "sentiment": {            "type": "string",            "description": "the sentiment encountered in the passage",        },        "aggressiveness": {            "type": "integer",            "description": "a 0-10 score of how aggressive the passage is",        },        "language": {"type": "string", "description": "the language of the passage"},    }}chain = create_tagging_chain(schema, model)
```

```
chain.run("give me your money")
```

```
{'sentiment': 'aggressive', 'aggressiveness': 8, 'language': 'english'}
```

* * *

#### Help us out by providing feedback on this documentation page: