Chat Models
class langchain.chat_models.ChatOpenAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model='gpt-3.5-turbo', temperature=0.7, model_kwargs=None, openai_api_key=None, openai_api_base=None, openai_organization=None, openai_proxy=None, request_timeout=None, max_retries=6, streaming=False, n=1, max_tokens=None, tiktoken_model_name=None)[source]
Bases: langchain.chat_models.base.BaseChatModel
Wrapper around OpenAI Chat large language models.
To use, you should have the openai python package installed, and the
environment variable OPENAI_API_KEY set with your API key.
Any parameters that are valid to be passed to the openai.create call can be passed
in, even if not explicitly saved on this class.
Example
from langchain.chat_models import ChatOpenAI
openai = ChatOpenAI(model_name="gpt-3.5-turbo")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (str) – 
temperature (float) – 
model_kwargs (Dict[str, Any]) – 
openai_api_key (Optional[str]) – 
openai_api_base (Optional[str]) – 
openai_organization (Optional[str]) – 
openai_proxy (Optional[str]) – 
request_timeout (Optional[Union[float, Tuple[float, float]]]) – 
max_retries (int) – 
streaming (bool) – 
n (int) – 
max_tokens (Optional[int]) – 
tiktoken_model_name (Optional[str]) – 
Return type
None
attribute max_retries: int = 6
Maximum number of retries to make when generating.
attribute max_tokens: Optional[int] = None
Maximum number of tokens to generate.
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not explicitly specified.
attribute model_name: str = 'gpt-3.5-turbo' (alias 'model')
Model name to use.
attribute n: int = 1
Number of chat completions to generate for each prompt.
attribute openai_api_base: Optional[str] = None
attribute openai_api_key: Optional[str] = None
Base URL path for API requests,
leave blank if not using a proxy or service emulator.
attribute openai_organization: Optional[str] = None
attribute openai_proxy: Optional[str] = None
attribute request_timeout: Optional[Union[float, Tuple[float, float]]] = None
Timeout for requests to OpenAI completion API. Default is 600 seconds.
attribute streaming: bool = False
Whether to stream the results or not.
attribute temperature: float = 0.7
What sampling temperature to use.
attribute tiktoken_model_name: Optional[str] = None
The model name to pass to tiktoken when using this class.
Tiktoken is used to count the number of tokens in documents to constrain
them to be under a certain limit. By default, when set to None, this will
be the same as the embedding model name. However, there are some cases
where you may want to use this Embedding class with a model name not
supported by tiktoken. This can include when using Azure embeddings or
when using one of the many model providers that expose an OpenAI-like
API but with different models. In those cases, in order to avoid erroring
when tiktoken is called, you can specify a model name to use here.
completion_with_retry(**kwargs)[source]
Use tenacity to retry the completion call.
Parameters
kwargs (Any) – 
Return type
Any
get_num_tokens_from_messages(messages)[source]
Calculate num tokens for gpt-3.5-turbo and gpt-4 with tiktoken package.
Official documentation: https://github.com/openai/openai-cookbook/blob/
main/examples/How_to_format_inputs_to_ChatGPT_models.ipynb
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)[source]
Get the tokens present in the text with tiktoken package.
Parameters
text (str) – 
Return type
List[int]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.chat_models.AzureChatOpenAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model='gpt-3.5-turbo', temperature=0.7, model_kwargs=None, openai_api_key='', openai_api_base='', openai_organization='', openai_proxy='', request_timeout=None, max_retries=6, streaming=False, n=1, max_tokens=None, tiktoken_model_name=None, deployment_name='', openai_api_type='azure', openai_api_version='')[source]
Bases: langchain.chat_models.openai.ChatOpenAI
Wrapper around Azure OpenAI Chat Completion API. To use this class you
must have a deployed model on Azure OpenAI. Use deployment_name in the
constructor to refer to the “Model deployment name” in the Azure portal.
In addition, you should have the openai python package installed, and the
following environment variables set or passed in constructor in lower case:
- OPENAI_API_TYPE (default: azure)
- OPENAI_API_KEY
- OPENAI_API_BASE
- OPENAI_API_VERSION
- OPENAI_PROXY
For exmaple, if you have gpt-35-turbo deployed, with the deployment name
35-turbo-dev, the constructor should look like:
AzureChatOpenAI(
    deployment_name="35-turbo-dev",
    openai_api_version="2023-03-15-preview",
)
Be aware the API version may change.
Any parameters that are valid to be passed to the openai.create call can be passed
in, even if not explicitly saved on this class.
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (str) – 
temperature (float) – 
model_kwargs (Dict[str, Any]) – 
openai_api_key (str) – 
openai_api_base (str) – 
openai_organization (str) – 
openai_proxy (str) – 
request_timeout (Optional[Union[float, Tuple[float, float]]]) – 
max_retries (int) – 
streaming (bool) – 
n (int) – 
max_tokens (Optional[int]) – 
tiktoken_model_name (Optional[str]) – 
deployment_name (str) – 
openai_api_type (str) – 
openai_api_version (str) – 
Return type
None
attribute deployment_name: str = ''
attribute openai_api_base: str = ''
attribute openai_api_key: str = ''
Base URL path for API requests,
leave blank if not using a proxy or service emulator.
attribute openai_api_type: str = 'azure'
attribute openai_api_version: str = ''
attribute openai_organization: str = ''
attribute openai_proxy: str = ''
class langchain.chat_models.FakeListChatModel(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, responses, i=0)[source]
Bases: langchain.chat_models.base.SimpleChatModel
Fake ChatModel for testing purposes.
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
responses (List) – 
i (int) – 
Return type
None
attribute i: int = 0
attribute responses: List [Required]
class langchain.chat_models.PromptLayerChatOpenAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model='gpt-3.5-turbo', temperature=0.7, model_kwargs=None, openai_api_key=None, openai_api_base=None, openai_organization=None, openai_proxy=None, request_timeout=None, max_retries=6, streaming=False, n=1, max_tokens=None, tiktoken_model_name=None, pl_tags=None, return_pl_id=False)[source]
Bases: langchain.chat_models.openai.ChatOpenAI
Wrapper around OpenAI Chat large language models and PromptLayer.
To use, you should have the openai and promptlayer python
package installed, and the environment variable OPENAI_API_KEY
and PROMPTLAYER_API_KEY set with your openAI API key and
promptlayer key respectively.
All parameters that can be passed to the OpenAI LLM can also
be passed here. The PromptLayerChatOpenAI adds to optional
Parameters
pl_tags (Optional[List[str]]) – List of strings to tag the request with.
return_pl_id (Optional[bool]) – If True, the PromptLayer request ID will be
returned in the generation_info field of the
Generation object.
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (str) – 
temperature (float) – 
model_kwargs (Dict[str, Any]) – 
openai_api_key (Optional[str]) – 
openai_api_base (Optional[str]) – 
openai_organization (Optional[str]) – 
openai_proxy (Optional[str]) – 
request_timeout (Optional[Union[float, Tuple[float, float]]]) – 
max_retries (int) – 
streaming (bool) – 
n (int) – 
max_tokens (Optional[int]) – 
tiktoken_model_name (Optional[str]) – 
Return type
None
Example
from langchain.chat_models import PromptLayerChatOpenAI
openai = PromptLayerChatOpenAI(model_name="gpt-3.5-turbo")
attribute pl_tags: Optional[List[str]] = None
attribute return_pl_id: Optional[bool] = False
class langchain.chat_models.ChatAnthropic(*, client=None, model='claude-v1', max_tokens_to_sample=256, temperature=None, top_k=None, top_p=None, streaming=False, default_request_timeout=None, anthropic_api_url=None, anthropic_api_key=None, HUMAN_PROMPT=None, AI_PROMPT=None, count_tokens=None, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None)[source]
Bases: langchain.chat_models.base.BaseChatModel, langchain.llms.anthropic._AnthropicCommon
Wrapper around Anthropic’s large language model.
To use, you should have the anthropic python package installed, and the
environment variable ANTHROPIC_API_KEY set with your API key, or pass
it as a named parameter to the constructor.
Example
import anthropic
from langchain.llms import Anthropic
model = ChatAnthropic(model="<model_name>", anthropic_api_key="my-api-key")
Parameters
client (Any) – 
model (str) – 
max_tokens_to_sample (int) – 
temperature (Optional[float]) – 
top_k (Optional[int]) – 
top_p (Optional[float]) – 
streaming (bool) – 
default_request_timeout (Optional[Union[float, Tuple[float, float]]]) – 
anthropic_api_url (Optional[str]) – 
anthropic_api_key (Optional[str]) – 
HUMAN_PROMPT (Optional[str]) – 
AI_PROMPT (Optional[str]) – 
count_tokens (Optional[Callable[[str], int]]) – 
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
Return type
None
get_num_tokens(text)[source]
Calculate number of tokens.
Parameters
text (str) – 
Return type
int
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.chat_models.ChatGooglePalm(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model_name='models/chat-bison-001', google_api_key=None, temperature=None, top_p=None, top_k=None, n=1)[source]
Bases: langchain.chat_models.base.BaseChatModel, pydantic.main.BaseModel
Wrapper around Google’s PaLM Chat API.
To use you must have the google.generativeai Python package installed and
either:
The GOOGLE_API_KEY` environment varaible set with your API key, or
Pass your API key using the google_api_key kwarg to the ChatGoogle
constructor.
Example
from langchain.chat_models import ChatGooglePalm
chat = ChatGooglePalm()
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model_name (str) – 
google_api_key (Optional[str]) – 
temperature (Optional[float]) – 
top_p (Optional[float]) – 
top_k (Optional[int]) – 
n (int) – 
Return type
None
attribute google_api_key: Optional[str] = None
attribute model_name: str = 'models/chat-bison-001'
Model name to use.
attribute n: int = 1
Number of chat completions to generate for each prompt. Note that the API may
not return the full n completions if duplicates are generated.
attribute temperature: Optional[float] = None
Run inference with this temperature. Must by in the closed
interval [0.0, 1.0].
attribute top_k: Optional[int] = None
Decode using top-k sampling: consider the set of top_k most probable tokens.
Must be positive.
attribute top_p: Optional[float] = None
Decode using nucleus sampling: consider the smallest set of tokens whose
probability sum is at least top_p. Must be in the closed interval [0.0, 1.0].
class langchain.chat_models.ChatVertexAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model_name='chat-bison', temperature=0.0, max_output_tokens=128, top_p=0.95, top_k=40, stop=None, project=None, location='us-central1', credentials=None)[source]
Bases: langchain.llms.vertexai._VertexAICommon, langchain.chat_models.base.BaseChatModel
Wrapper around Vertex AI large language models.
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (_LanguageModel) – 
model_name (str) – 
temperature (float) – 
max_output_tokens (int) – 
top_p (float) – 
top_k (int) – 
stop (Optional[List[str]]) – 
project (Optional[str]) – 
location (str) – 
credentials (Any) – 
Return type
None
attribute model_name: str = 'chat-bison'
Model name to use.