LLMs
Wrappers on top of large language models APIs.
class langchain.llms.AI21(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model='j2-jumbo-instruct', temperature=0.7, maxTokens=256, minTokens=0, topP=1.0, presencePenalty=AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True), countPenalty=AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True), frequencyPenalty=AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True), numResults=1, logitBias=None, ai21_api_key=None, stop=None, base_url=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around AI21 large language models.
To use, you should have the environment variable AI21_API_KEY
set with your API key.
Example
from langchain.llms import AI21
ai21 = AI21(model="j2-jumbo-instruct")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model (str) – 
temperature (float) – 
maxTokens (int) – 
minTokens (int) – 
topP (float) – 
presencePenalty (langchain.llms.ai21.AI21PenaltyData) – 
countPenalty (langchain.llms.ai21.AI21PenaltyData) – 
frequencyPenalty (langchain.llms.ai21.AI21PenaltyData) – 
numResults (int) – 
logitBias (Optional[Dict[str, float]]) – 
ai21_api_key (Optional[str]) – 
stop (Optional[List[str]]) – 
base_url (Optional[str]) – 
Return type
None
attribute base_url: Optional[str] = None
Base url to use, if None decides based on model name.
attribute countPenalty: langchain.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)
Penalizes repeated tokens according to count.
attribute frequencyPenalty: langchain.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)
Penalizes repeated tokens according to frequency.
attribute logitBias: Optional[Dict[str, float]] = None
Adjust the probability of specific tokens being generated.
attribute maxTokens: int = 256
The maximum number of tokens to generate in the completion.
attribute minTokens: int = 0
The minimum number of tokens to generate in the completion.
attribute model: str = 'j2-jumbo-instruct'
Model name to use.
attribute numResults: int = 1
How many completions to generate for each prompt.
attribute presencePenalty: langchain.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)
Penalizes repeated tokens.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.7
What sampling temperature to use.
attribute topP: float = 1.0
Total probability mass of tokens to consider at each step.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.AlephAlpha(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model='luminous-base', maximum_tokens=64, temperature=0.0, top_k=0, top_p=0.0, presence_penalty=0.0, frequency_penalty=0.0, repetition_penalties_include_prompt=False, use_multiplicative_presence_penalty=False, penalty_bias=None, penalty_exceptions=None, penalty_exceptions_include_stop_sequences=None, best_of=None, n=1, logit_bias=None, log_probs=None, tokens=False, disable_optimizations=False, minimum_tokens=0, echo=False, use_multiplicative_frequency_penalty=False, sequence_penalty=0.0, sequence_penalty_min_length=2, use_multiplicative_sequence_penalty=False, completion_bias_inclusion=None, completion_bias_inclusion_first_token_only=False, completion_bias_exclusion=None, completion_bias_exclusion_first_token_only=False, contextual_control_threshold=None, control_log_additive=True, repetition_penalties_include_completion=True, raw_completion=False, aleph_alpha_api_key=None, stop_sequences=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Aleph Alpha large language models.
To use, you should have the aleph_alpha_client python package installed, and the
environment variable ALEPH_ALPHA_API_KEY set with your API key, or pass
it as a named parameter to the constructor.
Parameters are explained more in depth here:
https://github.com/Aleph-Alpha/aleph-alpha-client/blob/c14b7dd2b4325c7da0d6a119f6e76385800e097b/aleph_alpha_client/completion.py#L10
Example
from langchain.llms import AlephAlpha
aleph_alpha = AlephAlpha(aleph_alpha_api_key="my-api-key")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (Optional[str]) – 
maximum_tokens (int) – 
temperature (float) – 
top_k (int) – 
top_p (float) – 
presence_penalty (float) – 
frequency_penalty (float) – 
repetition_penalties_include_prompt (Optional[bool]) – 
use_multiplicative_presence_penalty (Optional[bool]) – 
penalty_bias (Optional[str]) – 
penalty_exceptions (Optional[List[str]]) – 
penalty_exceptions_include_stop_sequences (Optional[bool]) – 
best_of (Optional[int]) – 
n (int) – 
logit_bias (Optional[Dict[int, float]]) – 
log_probs (Optional[int]) – 
tokens (Optional[bool]) – 
disable_optimizations (Optional[bool]) – 
minimum_tokens (Optional[int]) – 
echo (bool) – 
use_multiplicative_frequency_penalty (bool) – 
sequence_penalty (float) – 
sequence_penalty_min_length (int) – 
use_multiplicative_sequence_penalty (bool) – 
completion_bias_inclusion (Optional[Sequence[str]]) – 
completion_bias_inclusion_first_token_only (bool) – 
completion_bias_exclusion (Optional[Sequence[str]]) – 
completion_bias_exclusion_first_token_only (bool) – 
contextual_control_threshold (Optional[float]) – 
control_log_additive (Optional[bool]) – 
repetition_penalties_include_completion (bool) – 
raw_completion (bool) – 
aleph_alpha_api_key (Optional[str]) – 
stop_sequences (Optional[List[str]]) – 
Return type
None
attribute aleph_alpha_api_key: Optional[str] = None
API key for Aleph Alpha API.
attribute best_of: Optional[int] = None
returns the one with the “best of” results
(highest log probability per token)
attribute completion_bias_exclusion_first_token_only: bool = False
Only consider the first token for the completion_bias_exclusion.
attribute contextual_control_threshold: Optional[float] = None
If set to None, attention control parameters only apply to those tokens that have
explicitly been set in the request.
If set to a non-None value, control parameters are also applied to similar tokens.
attribute control_log_additive: Optional[bool] = True
True: apply control by adding the log(control_factor) to attention scores.
False: (attention_scores - - attention_scores.min(-1)) * control_factor
attribute echo: bool = False
Echo the prompt in the completion.
attribute frequency_penalty: float = 0.0
Penalizes repeated tokens according to frequency.
attribute log_probs: Optional[int] = None
Number of top log probabilities to be returned for each generated token.
attribute logit_bias: Optional[Dict[int, float]] = None
The logit bias allows to influence the likelihood of generating tokens.
attribute maximum_tokens: int = 64
The maximum number of tokens to be generated.
attribute minimum_tokens: Optional[int] = 0
Generate at least this number of tokens.
attribute model: Optional[str] = 'luminous-base'
Model name to use.
attribute n: int = 1
How many completions to generate for each prompt.
attribute penalty_bias: Optional[str] = None
Penalty bias for the completion.
attribute penalty_exceptions: Optional[List[str]] = None
List of strings that may be generated without penalty,
regardless of other penalty settings
attribute penalty_exceptions_include_stop_sequences: Optional[bool] = None
Should stop_sequences be included in penalty_exceptions.
attribute presence_penalty: float = 0.0
Penalizes repeated tokens.
attribute raw_completion: bool = False
Force the raw completion of the model to be returned.
attribute repetition_penalties_include_completion: bool = True
Flag deciding whether presence penalty or frequency penalty
are updated from the completion.
attribute repetition_penalties_include_prompt: Optional[bool] = False
Flag deciding whether presence penalty or frequency penalty are
updated from the prompt.
attribute stop_sequences: Optional[List[str]] = None
Stop sequences to use.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.0
A non-negative float that tunes the degree of randomness in generation.
attribute tokens: Optional[bool] = False
return tokens of completion.
attribute top_k: int = 0
Number of most likely tokens to consider at each step.
attribute top_p: float = 0.0
Total probability mass of tokens to consider at each step.
attribute use_multiplicative_presence_penalty: Optional[bool] = False
Flag deciding whether presence penalty is applied
multiplicatively (True) or additively (False).
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.AmazonAPIGateway(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, api_url, model_kwargs=None, content_handler=<langchain.llms.amazon_api_gateway.ContentHandlerAmazonAPIGateway object>)[source]
Bases: langchain.llms.base.LLM
Wrapper around custom Amazon API Gateway
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
api_url (str) – 
model_kwargs (Optional[Dict]) – 
content_handler (langchain.llms.amazon_api_gateway.ContentHandlerAmazonAPIGateway) – 
Return type
None
attribute api_url: str [Required]
API Gateway URL
attribute content_handler: langchain.llms.amazon_api_gateway.ContentHandlerAmazonAPIGateway = <langchain.llms.amazon_api_gateway.ContentHandlerAmazonAPIGateway object>
The content handler class that provides an input and
output transform functions to handle formats between LLM
and the endpoint.
attribute model_kwargs: Optional[Dict] = None
Key word arguments to pass to the model.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Anthropic(*, client=None, model='claude-v1', max_tokens_to_sample=256, temperature=None, top_k=None, top_p=None, streaming=False, default_request_timeout=None, anthropic_api_url=None, anthropic_api_key=None, HUMAN_PROMPT=None, AI_PROMPT=None, count_tokens=None, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None)[source]
Bases: langchain.llms.base.LLM, langchain.llms.anthropic._AnthropicCommon
Wrapper around Anthropic’s large language models.
To use, you should have the anthropic python package installed, and the
environment variable ANTHROPIC_API_KEY set with your API key, or pass
it as a named parameter to the constructor.
Example
import anthropic
from langchain.llms import Anthropic
model = Anthropic(model="<model_name>", anthropic_api_key="my-api-key")
# Simplest invocation, automatically wrapped with HUMAN_PROMPT
# and AI_PROMPT.
response = model("What are the biggest risks facing humanity?")
# Or if you want to use the chat mode, build a few-shot-prompt, or
# put words in the Assistant's mouth, use HUMAN_PROMPT and AI_PROMPT:
raw_prompt = "What are the biggest risks facing humanity?"
prompt = f"{anthropic.HUMAN_PROMPT} {prompt}{anthropic.AI_PROMPT}"
response = model(prompt)
Parameters
client (Any) – 
model (str) – 
max_tokens_to_sample (int) – 
temperature (Optional[float]) – 
top_k (Optional[int]) – 
top_p (Optional[float]) – 
streaming (bool) – 
default_request_timeout (Optional[Union[float, Tuple[float, float]]]) – 
anthropic_api_url (Optional[str]) – 
anthropic_api_key (Optional[str]) – 
HUMAN_PROMPT (Optional[str]) – 
AI_PROMPT (Optional[str]) – 
count_tokens (Optional[Callable[[str], int]]) – 
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
Return type
None
attribute default_request_timeout: Optional[Union[float, Tuple[float, float]]] = None
Timeout for requests to Anthropic Completion API. Default is 600 seconds.
attribute max_tokens_to_sample: int = 256
Denotes the number of tokens to predict per generation.
attribute model: str = 'claude-v1'
Model name to use.
attribute streaming: bool = False
Whether to stream the results.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: Optional[float] = None
A non-negative float that tunes the degree of randomness in generation.
attribute top_k: Optional[int] = None
Number of most likely tokens to consider at each step.
attribute top_p: Optional[float] = None
Total probability mass of tokens to consider at each step.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)[source]
Calculate number of tokens.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
stream(prompt, stop=None)[source]
Call Anthropic completion_stream and return the resulting generator.
BETA: this is a beta feature while we figure out the right abstraction.
Once that happens, this interface could change.
Parameters
prompt (str) – The prompt to pass into the model.
stop (Optional[List[str]]) – Optional list of stop words to use when generating.
Returns
A generator representing the stream of tokens from Anthropic.
Return type
Generator
Example
prompt = "Write a poem about a stream."
prompt = f"\n\nHuman: {prompt}\n\nAssistant:"
generator = anthropic.stream(prompt)
for token in generator:
    yield token
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Anyscale(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model_kwargs=None, anyscale_service_url=None, anyscale_service_route=None, anyscale_service_token=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Anyscale Services.
To use, you should have the environment variable ANYSCALE_SERVICE_URL,
ANYSCALE_SERVICE_ROUTE and ANYSCALE_SERVICE_TOKEN set with your Anyscale
Service, or pass it as a named parameter to the constructor.
Example
from langchain.llms import Anyscale
anyscale = Anyscale(anyscale_service_url="SERVICE_URL",
                    anyscale_service_route="SERVICE_ROUTE",
                    anyscale_service_token="SERVICE_TOKEN")
# Use Ray for distributed processing
import ray
prompt_list=[]
@ray.remote
def send_query(llm, prompt):
    resp = llm(prompt)
    return resp
futures = [send_query.remote(anyscale, prompt) for prompt in prompt_list]
results = ray.get(futures)
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model_kwargs (Optional[dict]) – 
anyscale_service_url (Optional[str]) – 
anyscale_service_route (Optional[str]) – 
anyscale_service_token (Optional[str]) – 
Return type
None
attribute model_kwargs: Optional[dict] = None
Key word arguments to pass to the model. Reserved for future use
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Aviary(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model='amazon/LightGPT', aviary_url=None, aviary_token=None, use_prompt_format=True, version=None)[source]
Bases: langchain.llms.base.LLM
Allow you to use an Aviary.
Aviary is a backend for hosted models. You can
find out more about aviary at
http://github.com/ray-project/aviary
To get a list of the models supported on an
aviary, follow the instructions on the web site to
install the aviary CLI and then use:
aviary models
AVIARY_URL and AVIARY_TOKEN environement variables must be set.
Example
from langchain.llms import Aviary
os.environ["AVIARY_URL"] = "<URL>"
os.environ["AVIARY_TOKEN"] = "<TOKEN>"
light = Aviary(model='amazon/LightGPT')
output = light('How do you make fried rice?')
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model (str) – 
aviary_url (Optional[str]) – 
aviary_token (Optional[str]) – 
use_prompt_format (bool) – 
version (Optional[str]) – 
Return type
None
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.AzureMLOnlineEndpoint(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, endpoint_url='', endpoint_api_key='', deployment_name='', http_client=None, content_formatter=None, model_kwargs=None)[source]
Bases: langchain.llms.base.LLM, pydantic.main.BaseModel
Wrapper around Azure ML Hosted models using Managed Online Endpoints.
Example
azure_llm = AzureMLModel(
    endpoint_url="https://<your-endpoint>.<your_region>.inference.ml.azure.com/score",
    endpoint_api_key="my-api-key",
    deployment_name="my-deployment-name",
    content_formatter=content_formatter,
)
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
endpoint_url (str) – 
endpoint_api_key (str) – 
deployment_name (str) – 
http_client (Any) – 
content_formatter (Any) – 
model_kwargs (Optional[dict]) – 
Return type
None
attribute content_formatter: Any = None
The content formatter that provides an input and output
transform function to handle formats between the LLM and
the endpoint
attribute deployment_name: str = ''
Deployment Name for Endpoint. Should be passed to constructor or specified as
env var AZUREML_DEPLOYMENT_NAME.
attribute endpoint_api_key: str = ''
Authentication Key for Endpoint. Should be passed to constructor or specified as
env var AZUREML_ENDPOINT_API_KEY.
attribute endpoint_url: str = ''
URL of pre-existing Endpoint. Should be passed to constructor or specified as
env var AZUREML_ENDPOINT_URL.
attribute model_kwargs: Optional[dict] = None
Key word arguments to pass to the model.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.AzureOpenAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model='text-davinci-003', temperature=0.7, max_tokens=256, top_p=1, frequency_penalty=0, presence_penalty=0, n=1, best_of=1, model_kwargs=None, openai_api_key=None, openai_api_base=None, openai_organization=None, openai_proxy=None, batch_size=20, request_timeout=None, logit_bias=None, max_retries=6, streaming=False, allowed_special={}, disallowed_special='all', tiktoken_model_name=None, deployment_name='', openai_api_type='azure', openai_api_version='')[source]
Bases: langchain.llms.openai.BaseOpenAI
Wrapper around Azure-specific OpenAI large language models.
To use, you should have the openai python package installed, and the
environment variable OPENAI_API_KEY set with your API key.
Any parameters that are valid to be passed to the openai.create call can be passed
in, even if not explicitly saved on this class.
Example
from langchain.llms import AzureOpenAI
openai = AzureOpenAI(model_name="text-davinci-003")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (str) – 
temperature (float) – 
max_tokens (int) – 
top_p (float) – 
frequency_penalty (float) – 
presence_penalty (float) – 
n (int) – 
best_of (int) – 
model_kwargs (Dict[str, Any]) – 
openai_api_key (Optional[str]) – 
openai_api_base (Optional[str]) – 
openai_organization (Optional[str]) – 
openai_proxy (Optional[str]) – 
batch_size (int) – 
request_timeout (Optional[Union[float, Tuple[float, float]]]) – 
logit_bias (Optional[Dict[str, float]]) – 
max_retries (int) – 
streaming (bool) – 
allowed_special (Union[Literal['all'], typing.AbstractSet[str]]) – 
disallowed_special (Union[Literal['all'], typing.Collection[str]]) – 
tiktoken_model_name (Optional[str]) – 
deployment_name (str) – 
openai_api_type (str) – 
openai_api_version (str) – 
Return type
None
attribute allowed_special: Union[Literal['all'], AbstractSet[str]] = {}
Set of special tokens that are allowed。
attribute batch_size: int = 20
Batch size to use when passing multiple documents to generate.
attribute best_of: int = 1
Generates best_of completions server-side and returns the “best”.
attribute deployment_name: str = ''
Deployment name to use.
attribute disallowed_special: Union[Literal['all'], Collection[str]] = 'all'
Set of special tokens that are not allowed。
attribute frequency_penalty: float = 0
Penalizes repeated tokens according to frequency.
attribute logit_bias: Optional[Dict[str, float]] [Optional]
Adjust the probability of specific tokens being generated.
attribute max_retries: int = 6
Maximum number of retries to make when generating.
attribute max_tokens: int = 256
The maximum number of tokens to generate in the completion.
-1 returns as many tokens as possible given the prompt and
the models maximal context size.
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not explicitly specified.
attribute model_name: str = 'text-davinci-003' (alias 'model')
Model name to use.
attribute n: int = 1
How many completions to generate for each prompt.
attribute presence_penalty: float = 0
Penalizes repeated tokens.
attribute request_timeout: Optional[Union[float, Tuple[float, float]]] = None
Timeout for requests to OpenAI completion API. Default is 600 seconds.
attribute streaming: bool = False
Whether to stream the results or not.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.7
What sampling temperature to use.
attribute tiktoken_model_name: Optional[str] = None
The model name to pass to tiktoken when using this class.
Tiktoken is used to count the number of tokens in documents to constrain
them to be under a certain limit. By default, when set to None, this will
be the same as the embedding model name. However, there are some cases
where you may want to use this Embedding class with a model name not
supported by tiktoken. This can include when using Azure embeddings or
when using one of the many model providers that expose an OpenAI-like
API but with different models. In those cases, in order to avoid erroring
when tiktoken is called, you can specify a model name to use here.
attribute top_p: float = 1
Total probability mass of tokens to consider at each step.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
create_llm_result(choices, prompts, token_usage)
Create the LLMResult from the choices and prompts.
Parameters
choices (Any) – 
prompts (List[str]) – 
token_usage (Dict[str, int]) – 
Return type
langchain.schema.LLMResult
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_sub_prompts(params, prompts, stop=None)
Get the sub prompts for llm call.
Parameters
params (Dict[str, Any]) – 
prompts (List[str]) – 
stop (Optional[List[str]]) – 
Return type
List[List[str]]
get_token_ids(text)
Get the token IDs using the tiktoken package.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
max_tokens_for_prompt(prompt)
Calculate the maximum number of tokens possible to generate for a prompt.
Parameters
prompt (str) – The prompt to pass into the model.
Returns
The maximum number of tokens to generate for a prompt.
Return type
int
Example
max_tokens = openai.max_token_for_prompt("Tell me a joke.")
static modelname_to_contextsize(modelname)
Calculate the maximum number of tokens possible to generate for a model.
Parameters
modelname (str) – The modelname we want to know the context size for.
Returns
The maximum context size
Return type
int
Example
max_tokens = openai.modelname_to_contextsize("text-davinci-003")
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
prep_streaming_params(stop=None)
Prepare the params for streaming.
Parameters
stop (Optional[List[str]]) – 
Return type
Dict[str, Any]
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
stream(prompt, stop=None)
Call OpenAI with streaming flag and return the resulting generator.
BETA: this is a beta feature while we figure out the right abstraction.
Once that happens, this interface could change.
Parameters
prompt (str) – The prompts to pass into the model.
stop (Optional[List[str]]) – Optional list of stop words to use when generating.
Returns
A generator representing the stream of tokens from OpenAI.
Return type
Generator
Example
generator = openai.stream("Tell me a joke.")
for token in generator:
    yield token
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
property max_context_size: int
Get max context size for this model.
class langchain.llms.Banana(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model_key='', model_kwargs=None, banana_api_key=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Banana large language models.
To use, you should have the banana-dev python package installed,
and the environment variable BANANA_API_KEY set with your API key.
Any parameters that are valid to be passed to the call can be passed
in, even if not explicitly saved on this class.
Example
from langchain.llms import Banana
banana = Banana(model_key="")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model_key (str) – 
model_kwargs (Dict[str, Any]) – 
banana_api_key (Optional[str]) – 
Return type
None
attribute model_key: str = ''
model endpoint to use
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not
explicitly specified.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Baseten(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model, input=None, model_kwargs=None)[source]
Bases: langchain.llms.base.LLM
Use your Baseten models in Langchain
To use, you should have the baseten python package installed,
and run baseten.login() with your Baseten API key.
The required model param can be either a model id or model
version id. Using a model version ID will result in
slightly faster invocation.
Any other model parameters can also
be passed in with the format input={model_param: value, …}
The Baseten model must accept a dictionary of input with the key
“prompt” and return a dictionary with a key “data” which maps
to a list of response strings.
Example
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model (str) – 
input (Dict[str, Any]) – 
model_kwargs (Dict[str, Any]) – 
Return type
None
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Beam(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model_name='', name='', cpu='', memory='', gpu='', python_version='', python_packages=[], max_length='', url='', model_kwargs=None, beam_client_id='', beam_client_secret='', app_id=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Beam API for gpt2 large language model.
To use, you should have the beam-sdk python package installed,
and the environment variable BEAM_CLIENT_ID set with your client id
and BEAM_CLIENT_SECRET set with your client secret. Information on how
to get these is available here: https://docs.beam.cloud/account/api-keys.
The wrapper can then be called as follows, where the name, cpu, memory, gpu,
python version, and python packages can be updated accordingly. Once deployed,
the instance can be called.
Example
llm = Beam(model_name="gpt2",
    name="langchain-gpt2",
    cpu=8,
    memory="32Gi",
    gpu="A10G",
    python_version="python3.8",
    python_packages=[
        "diffusers[torch]>=0.10",
        "transformers",
        "torch",
        "pillow",
        "accelerate",
        "safetensors",
        "xformers",],
    max_length=50)
llm._deploy()
call_result = llm._call(input)
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model_name (str) – 
name (str) – 
cpu (str) – 
memory (str) – 
gpu (str) – 
python_version (str) – 
python_packages (List[str]) – 
max_length (str) – 
url (str) – 
model_kwargs (Dict[str, Any]) – 
beam_client_id (str) – 
beam_client_secret (str) – 
app_id (Optional[str]) – 
Return type
None
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not
explicitly specified.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute url: str = ''
model endpoint to use
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
app_creation()[source]
Creates a Python file which will contain your Beam app definition.
Return type
None
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
run_creation()[source]
Creates a Python file which will be deployed on beam.
Return type
None
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Bedrock(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, region_name=None, credentials_profile_name=None, model_id, model_kwargs=None)[source]
Bases: langchain.llms.base.LLM
LLM provider to invoke Bedrock models.
To authenticate, the AWS client uses the following methods to
automatically load credentials:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
If a specific credential profile should be used, you must pass
the name of the profile from the ~/.aws/credentials file that is to be used.
Make sure the credentials / roles used have the required policies to
access the Bedrock service.
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
region_name (Optional[str]) – 
credentials_profile_name (Optional[str]) – 
model_id (str) – 
model_kwargs (Optional[Dict]) – 
Return type
None
attribute credentials_profile_name: Optional[str] = None
The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which
has either access keys or role information specified.
If not specified, the default credential profile or, if on an EC2 instance,
credentials from IMDS will be used.
See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
attribute model_id: str [Required]
Id of the model to call, e.g., amazon.titan-tg1-large, this is
equivalent to the modelId property in the list-foundation-models api
attribute model_kwargs: Optional[Dict] = None
Key word arguments to pass to the model.
attribute region_name: Optional[str] = None
The aws region e.g., us-west-2. Fallsback to AWS_DEFAULT_REGION env variable
or region specified in ~/.aws/config in case it is not provided here.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.CTransformers(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model, model_type=None, model_file=None, config=None, lib=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around the C Transformers LLM interface.
To use, you should have the ctransformers python package installed.
See https://github.com/marella/ctransformers
Example
from langchain.llms import CTransformers
llm = CTransformers(model="/path/to/ggml-gpt-2.bin", model_type="gpt2")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (str) – 
model_type (Optional[str]) – 
model_file (Optional[str]) – 
config (Optional[Dict[str, Any]]) – 
lib (Optional[str]) – 
Return type
None
attribute config: Optional[Dict[str, Any]] = None
The config parameters.
See https://github.com/marella/ctransformers#config
attribute lib: Optional[str] = None
The path to a shared library or one of avx2, avx, basic.
attribute model: str [Required]
The path to a model file or directory or the name of a Hugging Face Hub
model repo.
attribute model_file: Optional[str] = None
The name of the model file in repo or directory.
attribute model_type: Optional[str] = None
The model type.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.CerebriumAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, endpoint_url='', model_kwargs=None, cerebriumai_api_key=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around CerebriumAI large language models.
To use, you should have the cerebrium python package installed, and the
environment variable CEREBRIUMAI_API_KEY set with your API key.
Any parameters that are valid to be passed to the call can be passed
in, even if not explicitly saved on this class.
Example
from langchain.llms import CerebriumAI
cerebrium = CerebriumAI(endpoint_url="")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
endpoint_url (str) – 
model_kwargs (Dict[str, Any]) – 
cerebriumai_api_key (Optional[str]) – 
Return type
None
attribute endpoint_url: str = ''
model endpoint to use
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not
explicitly specified.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Clarifai(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, stub=None, metadata=None, userDataObject=None, model_id=None, model_version_id=None, app_id=None, user_id=None, clarifai_pat_key=None, api_base='https://api.clarifai.com', stop=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Clarifai’s large language models.
To use, you should have an account on the Clarifai platform,
the clarifai python package installed, and the
environment variable CLARIFAI_PAT_KEY set with your PAT key,
or pass it as a named parameter to the constructor.
Example
from langchain.llms import Clarifai
clarifai_llm = Clarifai(clarifai_pat_key=CLARIFAI_PAT_KEY,                 user_id=USER_ID, app_id=APP_ID, model_id=MODEL_ID)
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
stub (Any) – 
metadata (Any) – 
userDataObject (Any) – 
model_id (Optional[str]) – 
model_version_id (Optional[str]) – 
app_id (Optional[str]) – 
user_id (Optional[str]) – 
clarifai_pat_key (Optional[str]) – 
api_base (str) – 
stop (Optional[List[str]]) – 
Return type
None
attribute app_id: Optional[str] = None
Clarifai application id to use.
attribute model_id: Optional[str] = None
Model id to use.
attribute model_version_id: Optional[str] = None
Model version id to use.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute user_id: Optional[str] = None
Clarifai user id to use.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Cohere(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model=None, max_tokens=256, temperature=0.75, k=0, p=1, frequency_penalty=0.0, presence_penalty=0.0, truncate=None, max_retries=10, cohere_api_key=None, stop=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Cohere large language models.
To use, you should have the cohere python package installed, and the
environment variable COHERE_API_KEY set with your API key, or pass
it as a named parameter to the constructor.
Example
from langchain.llms import Cohere
cohere = Cohere(model="gptd-instruct-tft", cohere_api_key="my-api-key")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (Optional[str]) – 
max_tokens (int) – 
temperature (float) – 
k (int) – 
p (int) – 
frequency_penalty (float) – 
presence_penalty (float) – 
truncate (Optional[str]) – 
max_retries (int) – 
cohere_api_key (Optional[str]) – 
stop (Optional[List[str]]) – 
Return type
None
attribute frequency_penalty: float = 0.0
Penalizes repeated tokens according to frequency. Between 0 and 1.
attribute k: int = 0
Number of most likely tokens to consider at each step.
attribute max_retries: int = 10
Maximum number of retries to make when generating.
attribute max_tokens: int = 256
Denotes the number of tokens to predict per generation.
attribute model: Optional[str] = None
Model name to use.
attribute p: int = 1
Total probability mass of tokens to consider at each step.
attribute presence_penalty: float = 0.0
Penalizes repeated tokens. Between 0 and 1.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.75
A non-negative float that tunes the degree of randomness in generation.
attribute truncate: Optional[str] = None
Specify how the client handles inputs longer than the maximum token
length: Truncate from START, END or NONE
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Databricks(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, host=None, api_token=None, endpoint_name=None, cluster_id=None, cluster_driver_port=None, model_kwargs=None, transform_input_fn=None, transform_output_fn=None)[source]
Bases: langchain.llms.base.LLM
LLM wrapper around a Databricks serving endpoint or a cluster driver proxy app.
It supports two endpoint types:
Serving endpoint (recommended for both production and development).
We assume that an LLM was registered and deployed to a serving endpoint.
To wrap it as an LLM you must have “Can Query” permission to the endpoint.
Set endpoint_name accordingly and do not set cluster_id and
cluster_driver_port.
The expected model signature is:
inputs:
[{"name": "prompt", "type": "string"},
 {"name": "stop", "type": "list[string]"}]
outputs: [{"type": "string"}]
Cluster driver proxy app (recommended for interactive development).
One can load an LLM on a Databricks interactive cluster and start a local HTTP
server on the driver node to serve the model at / using HTTP POST method
with JSON input/output.
Please use a port number between [3000, 8000] and let the server listen to
the driver IP address or simply 0.0.0.0 instead of localhost only.
To wrap it as an LLM you must have “Can Attach To” permission to the cluster.
Set cluster_id and cluster_driver_port and do not set endpoint_name.
The expected server schema (using JSON schema) is:
inputs:
{"type": "object",
 "properties": {
    "prompt": {"type": "string"},
    "stop": {"type": "array", "items": {"type": "string"}}},
 "required": ["prompt"]}`
outputs: {"type": "string"}
If the endpoint model signature is different or you want to set extra params,
you can use transform_input_fn and transform_output_fn to apply necessary
transformations before and after the query.
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
host (str) – 
api_token (str) – 
endpoint_name (Optional[str]) – 
cluster_id (Optional[str]) – 
cluster_driver_port (Optional[str]) – 
model_kwargs (Optional[Dict[str, Any]]) – 
transform_input_fn (Optional[Callable]) – 
transform_output_fn (Optional[Callable[[...], str]]) – 
Return type
None
attribute api_token: str [Optional]
Databricks personal access token.
If not provided, the default value is determined by
the DATABRICKS_TOKEN environment variable if present, or
an automatically generated temporary token if running inside a Databricks
notebook attached to an interactive cluster in “single user” or
“no isolation shared” mode.
attribute cluster_driver_port: Optional[str] = None
The port number used by the HTTP server running on the cluster driver node.
The server should listen on the driver IP address or simply 0.0.0.0 to connect.
We recommend the server using a port number between [3000, 8000].
attribute cluster_id: Optional[str] = None
ID of the cluster if connecting to a cluster driver proxy app.
If neither endpoint_name nor cluster_id is not provided and the code runs
inside a Databricks notebook attached to an interactive cluster in “single user”
or “no isolation shared” mode, the current cluster ID is used as default.
You must not set both endpoint_name and cluster_id.
attribute endpoint_name: Optional[str] = None
Name of the model serving endpont.
You must specify the endpoint name to connect to a model serving endpoint.
You must not set both endpoint_name and cluster_id.
attribute host: str [Optional]
Databricks workspace hostname.
If not provided, the default value is determined by
the DATABRICKS_HOST environment variable if present, or
the hostname of the current Databricks workspace if running inside
a Databricks notebook attached to an interactive cluster in “single user”
or “no isolation shared” mode.
attribute model_kwargs: Optional[Dict[str, Any]] = None
Extra parameters to pass to the endpoint.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute transform_input_fn: Optional[Callable] = None
A function that transforms {prompt, stop, **kwargs} into a JSON-compatible
request object that the endpoint accepts.
For example, you can apply a prompt template to the input prompt.
attribute transform_output_fn: Optional[Callable[[...], str]] = None
A function that transforms the output from the endpoint to the generated text.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.DeepInfra(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model_id='google/flan-t5-xl', model_kwargs=None, deepinfra_api_token=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around DeepInfra deployed models.
To use, you should have the requests python package installed, and the
environment variable DEEPINFRA_API_TOKEN set with your API token, or pass
it as a named parameter to the constructor.
Only supports text-generation and text2text-generation for now.
Example
from langchain.llms import DeepInfra
di = DeepInfra(model_id="google/flan-t5-xl",
                    deepinfra_api_token="my-api-key")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model_id (str) – 
model_kwargs (Optional[dict]) – 
deepinfra_api_token (Optional[str]) – 
Return type
None
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.FakeListLLM(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, responses, i=0)[source]
Bases: langchain.llms.base.LLM
Fake LLM wrapper for testing purposes.
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
responses (List) – 
i (int) – 
Return type
None
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.ForefrontAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, endpoint_url='', temperature=0.7, length=256, top_p=1.0, top_k=40, repetition_penalty=1, forefrontai_api_key=None, base_url=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around ForefrontAI large language models.
To use, you should have the environment variable FOREFRONTAI_API_KEY
set with your API key.
Example
from langchain.llms import ForefrontAI
forefrontai = ForefrontAI(endpoint_url="")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
endpoint_url (str) – 
temperature (float) – 
length (int) – 
top_p (float) – 
top_k (int) – 
repetition_penalty (int) – 
forefrontai_api_key (Optional[str]) – 
base_url (Optional[str]) – 
Return type
None
attribute base_url: Optional[str] = None
Base url to use, if None decides based on model name.
attribute endpoint_url: str = ''
Model name to use.
attribute length: int = 256
The maximum number of tokens to generate in the completion.
attribute repetition_penalty: int = 1
Penalizes repeated tokens according to frequency.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.7
What sampling temperature to use.
attribute top_k: int = 40
The number of highest probability vocabulary tokens to
keep for top-k-filtering.
attribute top_p: float = 1.0
Total probability mass of tokens to consider at each step.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.GPT4All(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model, backend=None, n_ctx=512, n_parts=- 1, seed=0, f16_kv=False, logits_all=False, vocab_only=False, use_mlock=False, embedding=False, n_threads=4, n_predict=256, temp=0.8, top_p=0.95, top_k=40, echo=False, stop=[], repeat_last_n=64, repeat_penalty=1.3, n_batch=1, streaming=False, context_erase=0.5, allow_download=False, client=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around GPT4All language models.
To use, you should have the gpt4all python package installed, the
pre-trained model file, and the model’s config information.
Example
from langchain.llms import GPT4All
model = GPT4All(model="./models/gpt4all-model.bin", n_ctx=512, n_threads=8)
# Simplest invocation
response = model("Once upon a time, ")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model (str) – 
backend (Optional[str]) – 
n_ctx (int) – 
n_parts (int) – 
seed (int) – 
f16_kv (bool) – 
logits_all (bool) – 
vocab_only (bool) – 
use_mlock (bool) – 
embedding (bool) – 
n_threads (Optional[int]) – 
n_predict (Optional[int]) – 
temp (Optional[float]) – 
top_p (Optional[float]) – 
top_k (Optional[int]) – 
echo (Optional[bool]) – 
stop (Optional[List[str]]) – 
repeat_last_n (Optional[int]) – 
repeat_penalty (Optional[float]) – 
n_batch (int) – 
streaming (bool) – 
context_erase (float) – 
allow_download (bool) – 
client (Any) – 
Return type
None
attribute allow_download: bool = False
If model does not exist in ~/.cache/gpt4all/, download it.
attribute context_erase: float = 0.5
Leave (n_ctx * context_erase) tokens
starting from beginning if the context has run out.
attribute echo: Optional[bool] = False
Whether to echo the prompt.
attribute embedding: bool = False
Use embedding mode only.
attribute f16_kv: bool = False
Use half-precision for key/value cache.
attribute logits_all: bool = False
Return logits for all tokens, not just the last token.
attribute model: str [Required]
Path to the pre-trained GPT4All model file.
attribute n_batch: int = 1
Batch size for prompt processing.
attribute n_ctx: int = 512
Token context window.
attribute n_parts: int = -1
Number of parts to split the model into.
If -1, the number of parts is automatically determined.
attribute n_predict: Optional[int] = 256
The maximum number of tokens to generate.
attribute n_threads: Optional[int] = 4
Number of threads to use.
attribute repeat_last_n: Optional[int] = 64
Last n tokens to penalize
attribute repeat_penalty: Optional[float] = 1.3
The penalty to apply to repeated tokens.
attribute seed: int = 0
Seed. If -1, a random seed is used.
attribute stop: Optional[List[str]] = []
A list of strings to stop generation when encountered.
attribute streaming: bool = False
Whether to stream the results or not.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temp: Optional[float] = 0.8
The temperature to use for sampling.
attribute top_k: Optional[int] = 40
The top-k value to use for sampling.
attribute top_p: Optional[float] = 0.95
The top-p value to use for sampling.
attribute use_mlock: bool = False
Force system to keep model in RAM.
attribute verbose: bool [Optional]
Whether to print out response text.
attribute vocab_only: bool = False
Only load the vocabulary, no weights.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.GooglePalm(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, google_api_key=None, model_name='models/text-bison-001', temperature=0.7, top_p=None, top_k=None, max_output_tokens=None, n=1)[source]
Bases: langchain.llms.base.BaseLLM, pydantic.main.BaseModel
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
google_api_key (Optional[str]) – 
model_name (str) – 
temperature (float) – 
top_p (Optional[float]) – 
top_k (Optional[int]) – 
max_output_tokens (Optional[int]) – 
n (int) – 
Return type
None
attribute max_output_tokens: Optional[int] = None
Maximum number of tokens to include in a candidate. Must be greater than zero.
If unset, will default to 64.
attribute model_name: str = 'models/text-bison-001'
Model name to use.
attribute n: int = 1
Number of chat completions to generate for each prompt. Note that the API may
not return the full n completions if duplicates are generated.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.7
Run inference with this temperature. Must by in the closed interval
[0.0, 1.0].
attribute top_k: Optional[int] = None
Decode using top-k sampling: consider the set of top_k most probable tokens.
Must be positive.
attribute top_p: Optional[float] = None
Decode using nucleus sampling: consider the smallest set of tokens whose
probability sum is at least top_p. Must be in the closed interval [0.0, 1.0].
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.GooseAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model_name='gpt-neo-20b', temperature=0.7, max_tokens=256, top_p=1, min_tokens=1, frequency_penalty=0, presence_penalty=0, n=1, model_kwargs=None, logit_bias=None, gooseai_api_key=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around OpenAI large language models.
To use, you should have the openai python package installed, and the
environment variable GOOSEAI_API_KEY set with your API key.
Any parameters that are valid to be passed to the openai.create call can be passed
in, even if not explicitly saved on this class.
Example
from langchain.llms import GooseAI
gooseai = GooseAI(model_name="gpt-neo-20b")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model_name (str) – 
temperature (float) – 
max_tokens (int) – 
top_p (float) – 
min_tokens (int) – 
frequency_penalty (float) – 
presence_penalty (float) – 
n (int) – 
model_kwargs (Dict[str, Any]) – 
logit_bias (Optional[Dict[str, float]]) – 
gooseai_api_key (Optional[str]) – 
Return type
None
attribute frequency_penalty: float = 0
Penalizes repeated tokens according to frequency.
attribute logit_bias: Optional[Dict[str, float]] [Optional]
Adjust the probability of specific tokens being generated.
attribute max_tokens: int = 256
The maximum number of tokens to generate in the completion.
-1 returns as many tokens as possible given the prompt and
the models maximal context size.
attribute min_tokens: int = 1
The minimum number of tokens to generate in the completion.
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not explicitly specified.
attribute model_name: str = 'gpt-neo-20b'
Model name to use
attribute n: int = 1
How many completions to generate for each prompt.
attribute presence_penalty: float = 0
Penalizes repeated tokens.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.7
What sampling temperature to use
attribute top_p: float = 1
Total probability mass of tokens to consider at each step.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.HuggingFaceEndpoint(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, endpoint_url='', task=None, model_kwargs=None, huggingfacehub_api_token=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around HuggingFaceHub Inference Endpoints.
To use, you should have the huggingface_hub python package installed, and the
environment variable HUGGINGFACEHUB_API_TOKEN set with your API token, or pass
it as a named parameter to the constructor.
Only supports text-generation and text2text-generation for now.
Example
from langchain.llms import HuggingFaceEndpoint
endpoint_url = (
    "https://abcdefghijklmnop.us-east-1.aws.endpoints.huggingface.cloud"
)
hf = HuggingFaceEndpoint(
    endpoint_url=endpoint_url,
    huggingfacehub_api_token="my-api-key"
)
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
endpoint_url (str) – 
task (Optional[str]) – 
model_kwargs (Optional[dict]) – 
huggingfacehub_api_token (Optional[str]) – 
Return type
None
attribute endpoint_url: str = ''
Endpoint URL to use.
attribute model_kwargs: Optional[dict] = None
Key word arguments to pass to the model.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute task: Optional[str] = None
Task to call the model with.
Should be a task that returns generated_text or summary_text.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.HuggingFaceHub(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, repo_id='gpt2', task=None, model_kwargs=None, huggingfacehub_api_token=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around HuggingFaceHub  models.
To use, you should have the huggingface_hub python package installed, and the
environment variable HUGGINGFACEHUB_API_TOKEN set with your API token, or pass
it as a named parameter to the constructor.
Only supports text-generation, text2text-generation and summarization for now.
Example
from langchain.llms import HuggingFaceHub
hf = HuggingFaceHub(repo_id="gpt2", huggingfacehub_api_token="my-api-key")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
repo_id (str) – 
task (Optional[str]) – 
model_kwargs (Optional[dict]) – 
huggingfacehub_api_token (Optional[str]) – 
Return type
None
attribute model_kwargs: Optional[dict] = None
Key word arguments to pass to the model.
attribute repo_id: str = 'gpt2'
Model name to use.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute task: Optional[str] = None
Task to call the model with.
Should be a task that returns generated_text or summary_text.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.HuggingFacePipeline(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, pipeline=None, model_id='gpt2', model_kwargs=None, pipeline_kwargs=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around HuggingFace Pipeline API.
To use, you should have the transformers python package installed.
Only supports text-generation, text2text-generation and summarization for now.
Example using from_model_id:from langchain.llms import HuggingFacePipeline
hf = HuggingFacePipeline.from_model_id(
    model_id="gpt2",
    task="text-generation",
    pipeline_kwargs={"max_new_tokens": 10},
)
Example passing pipeline in directly:from langchain.llms import HuggingFacePipeline
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
model_id = "gpt2"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
pipe = pipeline(
    "text-generation", model=model, tokenizer=tokenizer, max_new_tokens=10
)
hf = HuggingFacePipeline(pipeline=pipe)
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
pipeline (Any) – 
model_id (str) – 
model_kwargs (Optional[dict]) – 
pipeline_kwargs (Optional[dict]) – 
Return type
None
attribute model_id: str = 'gpt2'
Model name to use.
attribute model_kwargs: Optional[dict] = None
Key word arguments passed to the model.
attribute pipeline_kwargs: Optional[dict] = None
Key word arguments passed to the pipeline.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
classmethod from_model_id(model_id, task, device=- 1, model_kwargs=None, pipeline_kwargs=None, **kwargs)[source]
Construct the pipeline object from model_id and task.
Parameters
model_id (str) – 
task (str) – 
device (int) – 
model_kwargs (Optional[dict]) – 
pipeline_kwargs (Optional[dict]) – 
kwargs (Any) – 
Return type
langchain.llms.base.LLM
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.HuggingFaceTextGenInference(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, max_new_tokens=512, top_k=None, top_p=0.95, typical_p=0.95, temperature=0.8, repetition_penalty=None, stop_sequences=None, seed=None, inference_server_url='', timeout=120, server_kwargs=None, stream=False, client=None, async_client=None)[source]
Bases: langchain.llms.base.LLM
HuggingFace text generation inference API.
This class is a wrapper around the HuggingFace text generation inference API.
It is used to generate text from a given prompt.
Attributes:
- max_new_tokens: The maximum number of tokens to generate.
- top_k: The number of top-k tokens to consider when generating text.
- top_p: The cumulative probability threshold for generating text.
- typical_p: The typical probability threshold for generating text.
- temperature: The temperature to use when generating text.
- repetition_penalty: The repetition penalty to use when generating text.
- stop_sequences: A list of stop sequences to use when generating text.
- seed: The seed to use when generating text.
- inference_server_url: The URL of the inference server to use.
- timeout: The timeout value in seconds to use while connecting to inference server.
- server_kwargs: The keyword arguments to pass to the inference server.
- client: The client object used to communicate with the inference server.
- async_client: The async client object used to communicate with the server.
Methods:
- _call: Generates text based on a given prompt and stop sequences.
- _acall: Async generates text based on a given prompt and stop sequences.
- _llm_type: Returns the type of LLM.
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
max_new_tokens (int) – 
top_k (Optional[int]) – 
top_p (Optional[float]) – 
typical_p (Optional[float]) – 
temperature (float) – 
repetition_penalty (Optional[float]) – 
stop_sequences (List[str]) – 
seed (Optional[int]) – 
inference_server_url (str) – 
timeout (int) – 
server_kwargs (Dict[str, Any]) – 
stream (bool) – 
client (Any) – 
async_client (Any) – 
Return type
None
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.HumanInputLLM(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, input_func=None, prompt_func=None, separator='\n', input_kwargs={}, prompt_kwargs={})[source]
Bases: langchain.llms.base.LLM
A LLM wrapper which returns user input as the response.
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
input_func (Callable) – 
prompt_func (Callable[[str], None]) – 
separator (str) – 
input_kwargs (Mapping[str, Any]) – 
prompt_kwargs (Mapping[str, Any]) – 
Return type
None
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.LlamaCpp(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model_path, lora_base=None, lora_path=None, n_ctx=512, n_parts=- 1, seed=- 1, f16_kv=True, logits_all=False, vocab_only=False, use_mlock=False, n_threads=None, n_batch=8, n_gpu_layers=None, suffix=None, max_tokens=256, temperature=0.8, top_p=0.95, logprobs=None, echo=False, stop=[], repeat_penalty=1.1, top_k=40, last_n_tokens_size=64, use_mmap=True, streaming=True)[source]
Bases: langchain.llms.base.LLM
Wrapper around the llama.cpp model.
To use, you should have the llama-cpp-python library installed, and provide the
path to the Llama model as a named parameter to the constructor.
Check out: https://github.com/abetlen/llama-cpp-python
Example
from langchain.llms import LlamaCppEmbeddings
llm = LlamaCppEmbeddings(model_path="/path/to/llama/model")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model_path (str) – 
lora_base (Optional[str]) – 
lora_path (Optional[str]) – 
n_ctx (int) – 
n_parts (int) – 
seed (int) – 
f16_kv (bool) – 
logits_all (bool) – 
vocab_only (bool) – 
use_mlock (bool) – 
n_threads (Optional[int]) – 
n_batch (Optional[int]) – 
n_gpu_layers (Optional[int]) – 
suffix (Optional[str]) – 
max_tokens (Optional[int]) – 
temperature (Optional[float]) – 
top_p (Optional[float]) – 
logprobs (Optional[int]) – 
echo (Optional[bool]) – 
stop (Optional[List[str]]) – 
repeat_penalty (Optional[float]) – 
top_k (Optional[int]) – 
last_n_tokens_size (Optional[int]) – 
use_mmap (Optional[bool]) – 
streaming (bool) – 
Return type
None
attribute echo: Optional[bool] = False
Whether to echo the prompt.
attribute f16_kv: bool = True
Use half-precision for key/value cache.
attribute last_n_tokens_size: Optional[int] = 64
The number of tokens to look back when applying the repeat_penalty.
attribute logits_all: bool = False
Return logits for all tokens, not just the last token.
attribute logprobs: Optional[int] = None
The number of logprobs to return. If None, no logprobs are returned.
attribute lora_base: Optional[str] = None
The path to the Llama LoRA base model.
attribute lora_path: Optional[str] = None
The path to the Llama LoRA. If None, no LoRa is loaded.
attribute max_tokens: Optional[int] = 256
The maximum number of tokens to generate.
attribute model_path: str [Required]
The path to the Llama model file.
attribute n_batch: Optional[int] = 8
Number of tokens to process in parallel.
Should be a number between 1 and n_ctx.
attribute n_ctx: int = 512
Token context window.
attribute n_gpu_layers: Optional[int] = None
Number of layers to be loaded into gpu memory. Default None.
attribute n_parts: int = -1
Number of parts to split the model into.
If -1, the number of parts is automatically determined.
attribute n_threads: Optional[int] = None
Number of threads to use.
If None, the number of threads is automatically determined.
attribute repeat_penalty: Optional[float] = 1.1
The penalty to apply to repeated tokens.
attribute seed: int = -1
Seed. If -1, a random seed is used.
attribute stop: Optional[List[str]] = []
A list of strings to stop generation when encountered.
attribute streaming: bool = True
Whether to stream the results, token by token.
attribute suffix: Optional[str] = None
A suffix to append to the generated text. If None, no suffix is appended.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: Optional[float] = 0.8
The temperature to use for sampling.
attribute top_k: Optional[int] = 40
The top-k value to use for sampling.
attribute top_p: Optional[float] = 0.95
The top-p value to use for sampling.
attribute use_mlock: bool = False
Force system to keep model in RAM.
attribute use_mmap: Optional[bool] = True
Whether to keep the model loaded in RAM
attribute verbose: bool [Optional]
Whether to print out response text.
attribute vocab_only: bool = False
Only load the vocabulary, no weights.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)[source]
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
stream(prompt, stop=None, run_manager=None)[source]
Yields results objects as they are generated in real time.
BETA: this is a beta feature while we figure out the right abstraction.
Once that happens, this interface could change.
It also calls the callback manager’s on_llm_new_token event with
similar parameters to the OpenAI LLM class method of the same name.
Args:prompt: The prompts to pass into the model.
stop: Optional list of stop words to use when generating.
Returns:A generator representing the stream of tokens being generated.
Yields:A dictionary like objects containing a string token and metadata.
See llama-cpp-python docs and below for more.
Example:from langchain.llms import LlamaCpp
llm = LlamaCpp(
    model_path="/path/to/local/model.bin",
    temperature = 0.5
)
for chunk in llm.stream("Ask 'Hi, how are you?' like a pirate:'",
        stop=["'","
“]):result = chunk[“choices”][0]
print(result[“text”], end=’’, flush=True)
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
run_manager (Optional[langchain.callbacks.manager.CallbackManagerForLLMRun]) – 
Return type
Generator[Dict, None, None]
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.TextGen(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model_url, max_new_tokens=250, do_sample=True, temperature=1.3, top_p=0.1, typical_p=1, epsilon_cutoff=0, eta_cutoff=0, repetition_penalty=1.18, top_k=40, min_length=0, no_repeat_ngram_size=0, num_beams=1, penalty_alpha=0, length_penalty=1, early_stopping=False, seed=- 1, add_bos_token=True, truncation_length=2048, ban_eos_token=False, skip_special_tokens=True, stopping_strings=[], streaming=False)[source]
Bases: langchain.llms.base.LLM
Wrapper around the text-generation-webui model.
To use, you should have the text-generation-webui installed, a model loaded,
and –api added as a command-line option.
Suggested installation, use one-click installer for your OS:
https://github.com/oobabooga/text-generation-webui#one-click-installers
Paremeters below taken from text-generation-webui api example:
https://github.com/oobabooga/text-generation-webui/blob/main/api-examples/api-example.py
Example
from langchain.llms import TextGen
llm = TextGen(model_url="http://localhost:8500")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model_url (str) – 
max_new_tokens (Optional[int]) – 
do_sample (bool) – 
temperature (Optional[float]) – 
top_p (Optional[float]) – 
typical_p (Optional[float]) – 
epsilon_cutoff (Optional[float]) – 
eta_cutoff (Optional[float]) – 
repetition_penalty (Optional[float]) – 
top_k (Optional[float]) – 
min_length (Optional[int]) – 
no_repeat_ngram_size (Optional[int]) – 
num_beams (Optional[int]) – 
penalty_alpha (Optional[float]) – 
length_penalty (Optional[float]) – 
early_stopping (bool) – 
seed (int) – 
add_bos_token (bool) – 
truncation_length (Optional[int]) – 
ban_eos_token (bool) – 
skip_special_tokens (bool) – 
stopping_strings (Optional[List[str]]) – 
streaming (bool) – 
Return type
None
attribute add_bos_token: bool = True
Add the bos_token to the beginning of prompts.
Disabling this can make the replies more creative.
attribute ban_eos_token: bool = False
Ban the eos_token. Forces the model to never end the generation prematurely.
attribute do_sample: bool = True
Do sample
attribute early_stopping: bool = False
Early stopping
attribute epsilon_cutoff: Optional[float] = 0
Epsilon cutoff
attribute eta_cutoff: Optional[float] = 0
ETA cutoff
attribute length_penalty: Optional[float] = 1
Length Penalty
attribute max_new_tokens: Optional[int] = 250
The maximum number of tokens to generate.
attribute min_length: Optional[int] = 0
Minimum generation length in tokens.
attribute model_url: str [Required]
The full URL to the textgen webui including http[s]://host:port
attribute no_repeat_ngram_size: Optional[int] = 0
If not set to 0, specifies the length of token sets that are completely blocked
from repeating at all. Higher values = blocks larger phrases,
lower values = blocks words or letters from repeating.
Only 0 or high values are a good idea in most cases.
attribute num_beams: Optional[int] = 1
Number of beams
attribute penalty_alpha: Optional[float] = 0
Penalty Alpha
attribute repetition_penalty: Optional[float] = 1.18
Exponential penalty factor for repeating prior tokens. 1 means no penalty,
higher value = less repetition, lower value = more repetition.
attribute seed: int = -1
Seed (-1 for random)
attribute skip_special_tokens: bool = True
Skip special tokens. Some specific models need this unset.
attribute stopping_strings: Optional[List[str]] = []
A list of strings to stop generation when encountered.
attribute streaming: bool = False
Whether to stream the results, token by token (currently unimplemented).
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: Optional[float] = 1.3
Primary factor to control randomness of outputs. 0 = deterministic
(only the most likely token is used). Higher value = more randomness.
attribute top_k: Optional[float] = 40
Similar to top_p, but select instead only the top_k most likely tokens.
Higher value = higher range of possible random results.
attribute top_p: Optional[float] = 0.1
If not set to 1, select tokens with probabilities adding up to less than this
number. Higher value = higher range of possible random results.
attribute truncation_length: Optional[int] = 2048
Truncate the prompt up to this length. The leftmost tokens are removed if
the prompt exceeds this length. Most models require this to be at most 2048.
attribute typical_p: Optional[float] = 1
If not set to 1, select only tokens that are at least this much more likely to
appear than random tokens, given the prior text.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.ManifestWrapper(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, llm_kwargs=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around HazyResearch’s Manifest library.
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
llm_kwargs (Optional[Dict]) – 
Return type
None
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Modal(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, endpoint_url='', model_kwargs=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Modal large language models.
To use, you should have the modal-client python package installed.
Any parameters that are valid to be passed to the call can be passed
in, even if not explicitly saved on this class.
Example
from langchain.llms import Modal
modal = Modal(endpoint_url="")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
endpoint_url (str) – 
model_kwargs (Dict[str, Any]) – 
Return type
None
attribute endpoint_url: str = ''
model endpoint to use
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not
explicitly specified.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.MosaicML(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, endpoint_url='https://models.hosted-on.mosaicml.hosting/mpt-7b-instruct/v1/predict', inject_instruction_format=False, model_kwargs=None, retry_sleep=1.0, mosaicml_api_token=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around MosaicML’s LLM inference service.
To use, you should have the
environment variable MOSAICML_API_TOKEN set with your API token, or pass
it as a named parameter to the constructor.
Example
from langchain.llms import MosaicML
endpoint_url = (
    "https://models.hosted-on.mosaicml.hosting/mpt-7b-instruct/v1/predict"
)
mosaic_llm = MosaicML(
    endpoint_url=endpoint_url,
    mosaicml_api_token="my-api-key"
)
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
endpoint_url (str) – 
inject_instruction_format (bool) – 
model_kwargs (Optional[dict]) – 
retry_sleep (float) – 
mosaicml_api_token (Optional[str]) – 
Return type
None
attribute endpoint_url: str = 'https://models.hosted-on.mosaicml.hosting/mpt-7b-instruct/v1/predict'
Endpoint URL to use.
attribute inject_instruction_format: bool = False
Whether to inject the instruction format into the prompt.
attribute model_kwargs: Optional[dict] = None
Key word arguments to pass to the model.
attribute retry_sleep: float = 1.0
How long to try sleeping for if a rate limit is encountered
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.NLPCloud(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model_name='finetuned-gpt-neox-20b', temperature=0.7, min_length=1, max_length=256, length_no_input=True, remove_input=True, remove_end_sequence=True, bad_words=[], top_p=1, top_k=50, repetition_penalty=1.0, length_penalty=1.0, do_sample=True, num_beams=1, early_stopping=False, num_return_sequences=1, nlpcloud_api_key=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around NLPCloud large language models.
To use, you should have the nlpcloud python package installed, and the
environment variable NLPCLOUD_API_KEY set with your API key.
Example
from langchain.llms import NLPCloud
nlpcloud = NLPCloud(model="gpt-neox-20b")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model_name (str) – 
temperature (float) – 
min_length (int) – 
max_length (int) – 
length_no_input (bool) – 
remove_input (bool) – 
remove_end_sequence (bool) – 
bad_words (List[str]) – 
top_p (int) – 
top_k (int) – 
repetition_penalty (float) – 
length_penalty (float) – 
do_sample (bool) – 
num_beams (int) – 
early_stopping (bool) – 
num_return_sequences (int) – 
nlpcloud_api_key (Optional[str]) – 
Return type
None
attribute bad_words: List[str] = []
List of tokens not allowed to be generated.
attribute do_sample: bool = True
Whether to use sampling (True) or greedy decoding.
attribute early_stopping: bool = False
Whether to stop beam search at num_beams sentences.
attribute length_no_input: bool = True
Whether min_length and max_length should include the length of the input.
attribute length_penalty: float = 1.0
Exponential penalty to the length.
attribute max_length: int = 256
The maximum number of tokens to generate in the completion.
attribute min_length: int = 1
The minimum number of tokens to generate in the completion.
attribute model_name: str = 'finetuned-gpt-neox-20b'
Model name to use.
attribute num_beams: int = 1
Number of beams for beam search.
attribute num_return_sequences: int = 1
How many completions to generate for each prompt.
attribute remove_end_sequence: bool = True
Whether or not to remove the end sequence token.
attribute remove_input: bool = True
Remove input text from API response
attribute repetition_penalty: float = 1.0
Penalizes repeated tokens. 1.0 means no penalty.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.7
What sampling temperature to use.
attribute top_k: int = 50
The number of highest probability tokens to keep for top-k filtering.
attribute top_p: int = 1
Total probability mass of tokens to consider at each step.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.OpenAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model='text-davinci-003', temperature=0.7, max_tokens=256, top_p=1, frequency_penalty=0, presence_penalty=0, n=1, best_of=1, model_kwargs=None, openai_api_key=None, openai_api_base=None, openai_organization=None, openai_proxy=None, batch_size=20, request_timeout=None, logit_bias=None, max_retries=6, streaming=False, allowed_special={}, disallowed_special='all', tiktoken_model_name=None)[source]
Bases: langchain.llms.openai.BaseOpenAI
Wrapper around OpenAI large language models.
To use, you should have the openai python package installed, and the
environment variable OPENAI_API_KEY set with your API key.
Any parameters that are valid to be passed to the openai.create call can be passed
in, even if not explicitly saved on this class.
Example
from langchain.llms import OpenAI
openai = OpenAI(model_name="text-davinci-003")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (str) – 
temperature (float) – 
max_tokens (int) – 
top_p (float) – 
frequency_penalty (float) – 
presence_penalty (float) – 
n (int) – 
best_of (int) – 
model_kwargs (Dict[str, Any]) – 
openai_api_key (Optional[str]) – 
openai_api_base (Optional[str]) – 
openai_organization (Optional[str]) – 
openai_proxy (Optional[str]) – 
batch_size (int) – 
request_timeout (Optional[Union[float, Tuple[float, float]]]) – 
logit_bias (Optional[Dict[str, float]]) – 
max_retries (int) – 
streaming (bool) – 
allowed_special (Union[Literal['all'], typing.AbstractSet[str]]) – 
disallowed_special (Union[Literal['all'], typing.Collection[str]]) – 
tiktoken_model_name (Optional[str]) – 
Return type
None
attribute allowed_special: Union[Literal['all'], AbstractSet[str]] = {}
Set of special tokens that are allowed。
attribute batch_size: int = 20
Batch size to use when passing multiple documents to generate.
attribute best_of: int = 1
Generates best_of completions server-side and returns the “best”.
attribute disallowed_special: Union[Literal['all'], Collection[str]] = 'all'
Set of special tokens that are not allowed。
attribute frequency_penalty: float = 0
Penalizes repeated tokens according to frequency.
attribute logit_bias: Optional[Dict[str, float]] [Optional]
Adjust the probability of specific tokens being generated.
attribute max_retries: int = 6
Maximum number of retries to make when generating.
attribute max_tokens: int = 256
The maximum number of tokens to generate in the completion.
-1 returns as many tokens as possible given the prompt and
the models maximal context size.
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not explicitly specified.
attribute model_name: str = 'text-davinci-003' (alias 'model')
Model name to use.
attribute n: int = 1
How many completions to generate for each prompt.
attribute presence_penalty: float = 0
Penalizes repeated tokens.
attribute request_timeout: Optional[Union[float, Tuple[float, float]]] = None
Timeout for requests to OpenAI completion API. Default is 600 seconds.
attribute streaming: bool = False
Whether to stream the results or not.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.7
What sampling temperature to use.
attribute tiktoken_model_name: Optional[str] = None
The model name to pass to tiktoken when using this class.
Tiktoken is used to count the number of tokens in documents to constrain
them to be under a certain limit. By default, when set to None, this will
be the same as the embedding model name. However, there are some cases
where you may want to use this Embedding class with a model name not
supported by tiktoken. This can include when using Azure embeddings or
when using one of the many model providers that expose an OpenAI-like
API but with different models. In those cases, in order to avoid erroring
when tiktoken is called, you can specify a model name to use here.
attribute top_p: float = 1
Total probability mass of tokens to consider at each step.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
create_llm_result(choices, prompts, token_usage)
Create the LLMResult from the choices and prompts.
Parameters
choices (Any) – 
prompts (List[str]) – 
token_usage (Dict[str, int]) – 
Return type
langchain.schema.LLMResult
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_sub_prompts(params, prompts, stop=None)
Get the sub prompts for llm call.
Parameters
params (Dict[str, Any]) – 
prompts (List[str]) – 
stop (Optional[List[str]]) – 
Return type
List[List[str]]
get_token_ids(text)
Get the token IDs using the tiktoken package.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
max_tokens_for_prompt(prompt)
Calculate the maximum number of tokens possible to generate for a prompt.
Parameters
prompt (str) – The prompt to pass into the model.
Returns
The maximum number of tokens to generate for a prompt.
Return type
int
Example
max_tokens = openai.max_token_for_prompt("Tell me a joke.")
static modelname_to_contextsize(modelname)
Calculate the maximum number of tokens possible to generate for a model.
Parameters
modelname (str) – The modelname we want to know the context size for.
Returns
The maximum context size
Return type
int
Example
max_tokens = openai.modelname_to_contextsize("text-davinci-003")
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
prep_streaming_params(stop=None)
Prepare the params for streaming.
Parameters
stop (Optional[List[str]]) – 
Return type
Dict[str, Any]
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
stream(prompt, stop=None)
Call OpenAI with streaming flag and return the resulting generator.
BETA: this is a beta feature while we figure out the right abstraction.
Once that happens, this interface could change.
Parameters
prompt (str) – The prompts to pass into the model.
stop (Optional[List[str]]) – Optional list of stop words to use when generating.
Returns
A generator representing the stream of tokens from OpenAI.
Return type
Generator
Example
generator = openai.stream("Tell me a joke.")
for token in generator:
    yield token
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
property max_context_size: int
Get max context size for this model.
class langchain.llms.OpenAIChat(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model_name='gpt-3.5-turbo', model_kwargs=None, openai_api_key=None, openai_api_base=None, openai_proxy=None, max_retries=6, prefix_messages=None, streaming=False, allowed_special={}, disallowed_special='all')[source]
Bases: langchain.llms.base.BaseLLM
Wrapper around OpenAI Chat large language models.
To use, you should have the openai python package installed, and the
environment variable OPENAI_API_KEY set with your API key.
Any parameters that are valid to be passed to the openai.create call can be passed
in, even if not explicitly saved on this class.
Example
from langchain.llms import OpenAIChat
openaichat = OpenAIChat(model_name="gpt-3.5-turbo")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model_name (str) – 
model_kwargs (Dict[str, Any]) – 
openai_api_key (Optional[str]) – 
openai_api_base (Optional[str]) – 
openai_proxy (Optional[str]) – 
max_retries (int) – 
prefix_messages (List) – 
streaming (bool) – 
allowed_special (Union[Literal['all'], typing.AbstractSet[str]]) – 
disallowed_special (Union[Literal['all'], typing.Collection[str]]) – 
Return type
None
attribute allowed_special: Union[Literal['all'], AbstractSet[str]] = {}
Set of special tokens that are allowed。
attribute disallowed_special: Union[Literal['all'], Collection[str]] = 'all'
Set of special tokens that are not allowed。
attribute max_retries: int = 6
Maximum number of retries to make when generating.
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not explicitly specified.
attribute model_name: str = 'gpt-3.5-turbo'
Model name to use.
attribute prefix_messages: List [Optional]
Series of messages for Chat input.
attribute streaming: bool = False
Whether to stream the results or not.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)[source]
Get the token IDs using the tiktoken package.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.OpenLLM(model_name=None, *, model_id=None, server_url=None, server_type='http', embedded=True, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, llm_kwargs)[source]
Bases: langchain.llms.base.LLM
Wrapper for accessing OpenLLM, supporting both in-process model
instance and remote OpenLLM servers.
To use, you should have the openllm library installed:
pip install openllm
Learn more at: https://github.com/bentoml/openllm
Example running an LLM model locally managed by OpenLLM:from langchain.llms import OpenLLM
llm = OpenLLM(
    model_name='flan-t5',
    model_id='google/flan-t5-large',
)
llm("What is the difference between a duck and a goose?")
For all available supported models, you can run ‘openllm models’.
If you have a OpenLLM server running, you can also use it remotely:from langchain.llms import OpenLLM
llm = OpenLLM(server_url='http://localhost:3000')
llm("What is the difference between a duck and a goose?")
Parameters
model_name (Optional[str]) – 
model_id (Optional[str]) – 
server_url (Optional[str]) – 
server_type (Literal['grpc', 'http']) – 
embedded (bool) – 
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
llm_kwargs (Dict[str, Any]) – 
Return type
None
attribute embedded: bool = True
Initialize this LLM instance in current process by default. Should
only set to False when using in conjunction with BentoML Service.
attribute llm_kwargs: Dict[str, Any] [Required]
Key word arguments to be passed to openllm.LLM
attribute model_id: Optional[str] = None
Model Id to use. If not provided, will use the default model for the model name.
See ‘openllm models’ for all available model variants.
attribute model_name: Optional[str] = None
Model name to use. See ‘openllm models’ for all available models.
attribute server_type: ServerType = 'http'
Optional server type. Either ‘http’ or ‘grpc’.
attribute server_url: Optional[str] = None
Optional server URL that currently runs a LLMServer with ‘openllm start’.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
property runner: openllm.LLMRunner
Get the underlying openllm.LLMRunner instance for integration with BentoML.
Example:
.. code-block:: python
llm = OpenLLM(model_name=’flan-t5’,
model_id=’google/flan-t5-large’,
embedded=False,
)
tools = load_tools([“serpapi”, “llm-math”], llm=llm)
agent = initialize_agent(
tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION
)
svc = bentoml.Service(“langchain-openllm”, runners=[llm.runner])
@svc.api(input=Text(), output=Text())
def chat(input_text: str):
return agent.run(input_text)
class langchain.llms.OpenLM(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model='text-davinci-003', temperature=0.7, max_tokens=256, top_p=1, frequency_penalty=0, presence_penalty=0, n=1, best_of=1, model_kwargs=None, openai_api_key=None, openai_api_base=None, openai_organization=None, openai_proxy=None, batch_size=20, request_timeout=None, logit_bias=None, max_retries=6, streaming=False, allowed_special={}, disallowed_special='all', tiktoken_model_name=None)[source]
Bases: langchain.llms.openai.BaseOpenAI
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (str) – 
temperature (float) – 
max_tokens (int) – 
top_p (float) – 
frequency_penalty (float) – 
presence_penalty (float) – 
n (int) – 
best_of (int) – 
model_kwargs (Dict[str, Any]) – 
openai_api_key (Optional[str]) – 
openai_api_base (Optional[str]) – 
openai_organization (Optional[str]) – 
openai_proxy (Optional[str]) – 
batch_size (int) – 
request_timeout (Optional[Union[float, Tuple[float, float]]]) – 
logit_bias (Optional[Dict[str, float]]) – 
max_retries (int) – 
streaming (bool) – 
allowed_special (Union[Literal['all'], typing.AbstractSet[str]]) – 
disallowed_special (Union[Literal['all'], typing.Collection[str]]) – 
tiktoken_model_name (Optional[str]) – 
Return type
None
attribute allowed_special: Union[Literal['all'], AbstractSet[str]] = {}
Set of special tokens that are allowed。
attribute batch_size: int = 20
Batch size to use when passing multiple documents to generate.
attribute best_of: int = 1
Generates best_of completions server-side and returns the “best”.
attribute disallowed_special: Union[Literal['all'], Collection[str]] = 'all'
Set of special tokens that are not allowed。
attribute frequency_penalty: float = 0
Penalizes repeated tokens according to frequency.
attribute logit_bias: Optional[Dict[str, float]] [Optional]
Adjust the probability of specific tokens being generated.
attribute max_retries: int = 6
Maximum number of retries to make when generating.
attribute max_tokens: int = 256
The maximum number of tokens to generate in the completion.
-1 returns as many tokens as possible given the prompt and
the models maximal context size.
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not explicitly specified.
attribute model_name: str = 'text-davinci-003' (alias 'model')
Model name to use.
attribute n: int = 1
How many completions to generate for each prompt.
attribute presence_penalty: float = 0
Penalizes repeated tokens.
attribute request_timeout: Optional[Union[float, Tuple[float, float]]] = None
Timeout for requests to OpenAI completion API. Default is 600 seconds.
attribute streaming: bool = False
Whether to stream the results or not.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.7
What sampling temperature to use.
attribute tiktoken_model_name: Optional[str] = None
The model name to pass to tiktoken when using this class.
Tiktoken is used to count the number of tokens in documents to constrain
them to be under a certain limit. By default, when set to None, this will
be the same as the embedding model name. However, there are some cases
where you may want to use this Embedding class with a model name not
supported by tiktoken. This can include when using Azure embeddings or
when using one of the many model providers that expose an OpenAI-like
API but with different models. In those cases, in order to avoid erroring
when tiktoken is called, you can specify a model name to use here.
attribute top_p: float = 1
Total probability mass of tokens to consider at each step.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
create_llm_result(choices, prompts, token_usage)
Create the LLMResult from the choices and prompts.
Parameters
choices (Any) – 
prompts (List[str]) – 
token_usage (Dict[str, int]) – 
Return type
langchain.schema.LLMResult
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_sub_prompts(params, prompts, stop=None)
Get the sub prompts for llm call.
Parameters
params (Dict[str, Any]) – 
prompts (List[str]) – 
stop (Optional[List[str]]) – 
Return type
List[List[str]]
get_token_ids(text)
Get the token IDs using the tiktoken package.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
max_tokens_for_prompt(prompt)
Calculate the maximum number of tokens possible to generate for a prompt.
Parameters
prompt (str) – The prompt to pass into the model.
Returns
The maximum number of tokens to generate for a prompt.
Return type
int
Example
max_tokens = openai.max_token_for_prompt("Tell me a joke.")
static modelname_to_contextsize(modelname)
Calculate the maximum number of tokens possible to generate for a model.
Parameters
modelname (str) – The modelname we want to know the context size for.
Returns
The maximum context size
Return type
int
Example
max_tokens = openai.modelname_to_contextsize("text-davinci-003")
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
prep_streaming_params(stop=None)
Prepare the params for streaming.
Parameters
stop (Optional[List[str]]) – 
Return type
Dict[str, Any]
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
stream(prompt, stop=None)
Call OpenAI with streaming flag and return the resulting generator.
BETA: this is a beta feature while we figure out the right abstraction.
Once that happens, this interface could change.
Parameters
prompt (str) – The prompts to pass into the model.
stop (Optional[List[str]]) – Optional list of stop words to use when generating.
Returns
A generator representing the stream of tokens from OpenAI.
Return type
Generator
Example
generator = openai.stream("Tell me a joke.")
for token in generator:
    yield token
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
property max_context_size: int
Get max context size for this model.
class langchain.llms.Petals(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, tokenizer=None, model_name='bigscience/bloom-petals', temperature=0.7, max_new_tokens=256, top_p=0.9, top_k=None, do_sample=True, max_length=None, model_kwargs=None, huggingface_api_key=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Petals Bloom models.
To use, you should have the petals python package installed, and the
environment variable HUGGINGFACE_API_KEY set with your API key.
Any parameters that are valid to be passed to the call can be passed
in, even if not explicitly saved on this class.
Example
from langchain.llms import petals
petals = Petals()
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
tokenizer (Any) – 
model_name (str) – 
temperature (float) – 
max_new_tokens (int) – 
top_p (float) – 
top_k (Optional[int]) – 
do_sample (bool) – 
max_length (Optional[int]) – 
model_kwargs (Dict[str, Any]) – 
huggingface_api_key (Optional[str]) – 
Return type
None
attribute client: Any = None
The client to use for the API calls.
attribute do_sample: bool = True
Whether or not to use sampling; use greedy decoding otherwise.
attribute max_length: Optional[int] = None
The maximum length of the sequence to be generated.
attribute max_new_tokens: int = 256
The maximum number of new tokens to generate in the completion.
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call
not explicitly specified.
attribute model_name: str = 'bigscience/bloom-petals'
The model to use.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.7
What sampling temperature to use
attribute tokenizer: Any = None
The tokenizer to use for the API calls.
attribute top_k: Optional[int] = None
The number of highest probability vocabulary tokens
to keep for top-k-filtering.
attribute top_p: float = 0.9
The cumulative probability for top-p sampling.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.PipelineAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, pipeline_key='', pipeline_kwargs=None, pipeline_api_key=None)[source]
Bases: langchain.llms.base.LLM, pydantic.main.BaseModel
Wrapper around PipelineAI large language models.
To use, you should have the pipeline-ai python package installed,
and the environment variable PIPELINE_API_KEY set with your API key.
Any parameters that are valid to be passed to the call can be passed
in, even if not explicitly saved on this class.
Example
from langchain import PipelineAI
pipeline = PipelineAI(pipeline_key="")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
pipeline_key (str) – 
pipeline_kwargs (Dict[str, Any]) – 
pipeline_api_key (Optional[str]) – 
Return type
None
attribute pipeline_key: str = ''
The id or tag of the target pipeline
attribute pipeline_kwargs: Dict[str, Any] [Optional]
Holds any pipeline parameters valid for create call not
explicitly specified.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.PredictionGuard(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model='MPT-7B-Instruct', output=None, max_tokens=256, temperature=0.75, token=None, stop=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Prediction Guard large language models.
To use, you should have the predictionguard python package installed, and the
environment variable PREDICTIONGUARD_TOKEN set with your access token, or pass
it as a named parameter to the constructor. To use Prediction Guard’s API along
with OpenAI models, set the environment variable OPENAI_API_KEY with your
OpenAI API key as well.
Example
pgllm = PredictionGuard(model="MPT-7B-Instruct",
                        token="my-access-token",
                        output={
                            "type": "boolean"
                        })
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (Optional[str]) – 
output (Optional[Dict[str, Any]]) – 
max_tokens (int) – 
temperature (float) – 
token (Optional[str]) – 
stop (Optional[List[str]]) – 
Return type
None
attribute max_tokens: int = 256
Denotes the number of tokens to predict per generation.
attribute model: Optional[str] = 'MPT-7B-Instruct'
Model name to use.
attribute output: Optional[Dict[str, Any]] = None
The output type or structure for controlling the LLM output.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.75
A non-negative float that tunes the degree of randomness in generation.
attribute token: Optional[str] = None
Your Prediction Guard access token.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.PromptLayerOpenAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model='text-davinci-003', temperature=0.7, max_tokens=256, top_p=1, frequency_penalty=0, presence_penalty=0, n=1, best_of=1, model_kwargs=None, openai_api_key=None, openai_api_base=None, openai_organization=None, openai_proxy=None, batch_size=20, request_timeout=None, logit_bias=None, max_retries=6, streaming=False, allowed_special={}, disallowed_special='all', tiktoken_model_name=None, pl_tags=None, return_pl_id=False)[source]
Bases: langchain.llms.openai.OpenAI
Wrapper around OpenAI large language models.
To use, you should have the openai and promptlayer python
package installed, and the environment variable OPENAI_API_KEY
and PROMPTLAYER_API_KEY set with your openAI API key and
promptlayer key respectively.
All parameters that can be passed to the OpenAI LLM can also
be passed here. The PromptLayerOpenAI LLM adds two optional
Parameters
pl_tags (Optional[List[str]]) – List of strings to tag the request with.
return_pl_id (Optional[bool]) – If True, the PromptLayer request ID will be
returned in the generation_info field of the
Generation object.
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model (str) – 
temperature (float) – 
max_tokens (int) – 
top_p (float) – 
frequency_penalty (float) – 
presence_penalty (float) – 
n (int) – 
best_of (int) – 
model_kwargs (Dict[str, Any]) – 
openai_api_key (Optional[str]) – 
openai_api_base (Optional[str]) – 
openai_organization (Optional[str]) – 
openai_proxy (Optional[str]) – 
batch_size (int) – 
request_timeout (Optional[Union[float, Tuple[float, float]]]) – 
logit_bias (Optional[Dict[str, float]]) – 
max_retries (int) – 
streaming (bool) – 
allowed_special (Union[Literal['all'], typing.AbstractSet[str]]) – 
disallowed_special (Union[Literal['all'], typing.Collection[str]]) – 
tiktoken_model_name (Optional[str]) – 
Return type
None
Example
from langchain.llms import PromptLayerOpenAI
openai = PromptLayerOpenAI(model_name="text-davinci-003")
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
create_llm_result(choices, prompts, token_usage)
Create the LLMResult from the choices and prompts.
Parameters
choices (Any) – 
prompts (List[str]) – 
token_usage (Dict[str, int]) – 
Return type
langchain.schema.LLMResult
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_sub_prompts(params, prompts, stop=None)
Get the sub prompts for llm call.
Parameters
params (Dict[str, Any]) – 
prompts (List[str]) – 
stop (Optional[List[str]]) – 
Return type
List[List[str]]
get_token_ids(text)
Get the token IDs using the tiktoken package.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
max_tokens_for_prompt(prompt)
Calculate the maximum number of tokens possible to generate for a prompt.
Parameters
prompt (str) – The prompt to pass into the model.
Returns
The maximum number of tokens to generate for a prompt.
Return type
int
Example
max_tokens = openai.max_token_for_prompt("Tell me a joke.")
static modelname_to_contextsize(modelname)
Calculate the maximum number of tokens possible to generate for a model.
Parameters
modelname (str) – The modelname we want to know the context size for.
Returns
The maximum context size
Return type
int
Example
max_tokens = openai.modelname_to_contextsize("text-davinci-003")
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
prep_streaming_params(stop=None)
Prepare the params for streaming.
Parameters
stop (Optional[List[str]]) – 
Return type
Dict[str, Any]
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
stream(prompt, stop=None)
Call OpenAI with streaming flag and return the resulting generator.
BETA: this is a beta feature while we figure out the right abstraction.
Once that happens, this interface could change.
Parameters
prompt (str) – The prompts to pass into the model.
stop (Optional[List[str]]) – Optional list of stop words to use when generating.
Returns
A generator representing the stream of tokens from OpenAI.
Return type
Generator
Example
generator = openai.stream("Tell me a joke.")
for token in generator:
    yield token
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
property max_context_size: int
Get max context size for this model.
class langchain.llms.PromptLayerOpenAIChat(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model_name='gpt-3.5-turbo', model_kwargs=None, openai_api_key=None, openai_api_base=None, openai_proxy=None, max_retries=6, prefix_messages=None, streaming=False, allowed_special={}, disallowed_special='all', pl_tags=None, return_pl_id=False)[source]
Bases: langchain.llms.openai.OpenAIChat
Wrapper around OpenAI large language models.
To use, you should have the openai and promptlayer python
package installed, and the environment variable OPENAI_API_KEY
and PROMPTLAYER_API_KEY set with your openAI API key and
promptlayer key respectively.
All parameters that can be passed to the OpenAIChat LLM can also
be passed here. The PromptLayerOpenAIChat adds two optional
Parameters
pl_tags (Optional[List[str]]) – List of strings to tag the request with.
return_pl_id (Optional[bool]) – If True, the PromptLayer request ID will be
returned in the generation_info field of the
Generation object.
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
model_name (str) – 
model_kwargs (Dict[str, Any]) – 
openai_api_key (Optional[str]) – 
openai_api_base (Optional[str]) – 
openai_proxy (Optional[str]) – 
max_retries (int) – 
prefix_messages (List) – 
streaming (bool) – 
allowed_special (Union[Literal['all'], typing.AbstractSet[str]]) – 
disallowed_special (Union[Literal['all'], typing.Collection[str]]) – 
Return type
None
Example
from langchain.llms import PromptLayerOpenAIChat
openaichat = PromptLayerOpenAIChat(model_name="gpt-3.5-turbo")
attribute allowed_special: Union[Literal['all'], AbstractSet[str]] = {}
Set of special tokens that are allowed。
attribute disallowed_special: Union[Literal['all'], Collection[str]] = 'all'
Set of special tokens that are not allowed。
attribute max_retries: int = 6
Maximum number of retries to make when generating.
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not explicitly specified.
attribute model_name: str = 'gpt-3.5-turbo'
Model name to use.
attribute prefix_messages: List [Optional]
Series of messages for Chat input.
attribute streaming: bool = False
Whether to stream the results or not.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token IDs using the tiktoken package.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.RWKV(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model, tokens_path, strategy='cpu fp32', rwkv_verbose=True, temperature=1.0, top_p=0.5, penalty_alpha_frequency=0.4, penalty_alpha_presence=0.4, CHUNK_LEN=256, max_tokens_per_generation=256, client=None, tokenizer=None, pipeline=None, model_tokens=None, model_state=None)[source]
Bases: langchain.llms.base.LLM, pydantic.main.BaseModel
Wrapper around RWKV language models.
To use, you should have the rwkv python package installed, the
pre-trained model file, and the model’s config information.
Example
from langchain.llms import RWKV
model = RWKV(model="./models/rwkv-3b-fp16.bin", strategy="cpu fp32")
# Simplest invocation
response = model("Once upon a time, ")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model (str) – 
tokens_path (str) – 
strategy (str) – 
rwkv_verbose (bool) – 
temperature (float) – 
top_p (float) – 
penalty_alpha_frequency (float) – 
penalty_alpha_presence (float) – 
CHUNK_LEN (int) – 
max_tokens_per_generation (int) – 
client (Any) – 
tokenizer (Any) – 
pipeline (Any) – 
model_tokens (Any) – 
model_state (Any) – 
Return type
None
attribute CHUNK_LEN: int = 256
Batch size for prompt processing.
attribute max_tokens_per_generation: int = 256
Maximum number of tokens to generate.
attribute model: str [Required]
Path to the pre-trained RWKV model file.
attribute penalty_alpha_frequency: float = 0.4
Positive values penalize new tokens based on their existing frequency
in the text so far, decreasing the model’s likelihood to repeat the same
line verbatim..
attribute penalty_alpha_presence: float = 0.4
Positive values penalize new tokens based on whether they appear
in the text so far, increasing the model’s likelihood to talk about
new topics..
attribute rwkv_verbose: bool = True
Print debug information.
attribute strategy: str = 'cpu fp32'
Token context window.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 1.0
The temperature to use for sampling.
attribute tokens_path: str [Required]
Path to the RWKV tokens file.
attribute top_p: float = 0.5
The top-p value to use for sampling.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Replicate(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, model, input=None, model_kwargs=None, replicate_api_token=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Replicate models.
To use, you should have the replicate python package installed,
and the environment variable REPLICATE_API_TOKEN set with your API token.
You can find your token here: https://replicate.com/account
The model param is required, but any other model parameters can also
be passed in with the format input={model_param: value, …}
Example
from langchain.llms import Replicate
replicate = Replicate(model="stability-ai/stable-diffusion:                                          27b93a2413e7f36cd83da926f365628                                         0b2931564ff050bf9575f1fdf9bcd7478",
                      input={"image_dimensions": "512x512"})
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
model (str) – 
input (Dict[str, Any]) – 
model_kwargs (Dict[str, Any]) – 
replicate_api_token (Optional[str]) – 
Return type
None
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.SagemakerEndpoint(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, endpoint_name='', region_name='', credentials_profile_name=None, content_handler, model_kwargs=None, endpoint_kwargs=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around custom Sagemaker Inference Endpoints.
To use, you must supply the endpoint name from your deployed
Sagemaker model & the region where it is deployed.
To authenticate, the AWS client uses the following methods to
automatically load credentials:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
If a specific credential profile should be used, you must pass
the name of the profile from the ~/.aws/credentials file that is to be used.
Make sure the credentials / roles used have the required policies to
access the Sagemaker endpoint.
See: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (Any) – 
endpoint_name (str) – 
region_name (str) – 
credentials_profile_name (Optional[str]) – 
content_handler (langchain.llms.sagemaker_endpoint.LLMContentHandler) – 
model_kwargs (Optional[Dict]) – 
endpoint_kwargs (Optional[Dict]) – 
Return type
None
attribute content_handler: langchain.llms.sagemaker_endpoint.LLMContentHandler [Required]
The content handler class that provides an input and
output transform functions to handle formats between LLM
and the endpoint.
attribute credentials_profile_name: Optional[str] = None
The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which
has either access keys or role information specified.
If not specified, the default credential profile or, if on an EC2 instance,
credentials from IMDS will be used.
See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
attribute endpoint_kwargs: Optional[Dict] = None
Optional attributes passed to the invoke_endpoint
function. See `boto3`_. docs for more info.
.. _boto3: <https://boto3.amazonaws.com/v1/documentation/api/latest/index.html>
attribute endpoint_name: str = ''
The name of the endpoint from the deployed Sagemaker model.
Must be unique within an AWS Region.
attribute model_kwargs: Optional[Dict] = None
Key word arguments to pass to the model.
attribute region_name: str = ''
The aws region where the Sagemaker model is deployed, eg. us-west-2.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.SelfHostedHuggingFaceLLM(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, pipeline_ref=None, client=None, inference_fn=<function _generate_text>, hardware=None, model_load_fn=<function _load_transformer>, load_fn_kwargs=None, model_reqs=['./', 'transformers', 'torch'], model_id='gpt2', task='text-generation', device=0, model_kwargs=None)[source]
Bases: langchain.llms.self_hosted.SelfHostedPipeline
Wrapper around HuggingFace Pipeline API to run on self-hosted remote hardware.
Supported hardware includes auto-launched instances on AWS, GCP, Azure,
and Lambda, as well as servers specified
by IP address and SSH credentials (such as on-prem, or another cloud
like Paperspace, Coreweave, etc.).
To use, you should have the runhouse python package installed.
Only supports text-generation, text2text-generation and summarization for now.
Example using from_model_id:from langchain.llms import SelfHostedHuggingFaceLLM
import runhouse as rh
gpu = rh.cluster(name="rh-a10x", instance_type="A100:1")
hf = SelfHostedHuggingFaceLLM(
    model_id="google/flan-t5-large", task="text2text-generation",
    hardware=gpu
)
Example passing fn that generates a pipeline (bc the pipeline is not serializable):from langchain.llms import SelfHostedHuggingFaceLLM
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
import runhouse as rh
def get_pipeline():
    model_id = "gpt2"
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(model_id)
    pipe = pipeline(
        "text-generation", model=model, tokenizer=tokenizer
    )
    return pipe
hf = SelfHostedHuggingFaceLLM(
    model_load_fn=get_pipeline, model_id="gpt2", hardware=gpu)
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
pipeline_ref (Any) – 
client (Any) – 
inference_fn (Callable) – 
hardware (Any) – 
model_load_fn (Callable) – 
load_fn_kwargs (Optional[dict]) – 
model_reqs (List[str]) – 
model_id (str) – 
task (str) – 
device (int) – 
model_kwargs (Optional[dict]) – 
Return type
None
attribute device: int = 0
Device to use for inference. -1 for CPU, 0 for GPU, 1 for second GPU, etc.
attribute hardware: Any = None
Remote hardware to send the inference function to.
attribute inference_fn: Callable = <function _generate_text>
Inference function to send to the remote hardware.
attribute load_fn_kwargs: Optional[dict] = None
Key word arguments to pass to the model load function.
attribute model_id: str = 'gpt2'
Hugging Face model_id to load the model.
attribute model_kwargs: Optional[dict] = None
Key word arguments to pass to the model.
attribute model_load_fn: Callable = <function _load_transformer>
Function to load the model remotely on the server.
attribute model_reqs: List[str] = ['./', 'transformers', 'torch']
Requirements to install on hardware to inference the model.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute task: str = 'text-generation'
Hugging Face task (“text-generation”, “text2text-generation” or
“summarization”).
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
classmethod from_pipeline(pipeline, hardware, model_reqs=None, device=0, **kwargs)
Init the SelfHostedPipeline from a pipeline object or string.
Parameters
pipeline (Any) – 
hardware (Any) – 
model_reqs (Optional[List[str]]) – 
device (int) – 
kwargs (Any) – 
Return type
langchain.llms.base.LLM
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.SelfHostedPipeline(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, pipeline_ref=None, client=None, inference_fn=<function _generate_text>, hardware=None, model_load_fn, load_fn_kwargs=None, model_reqs=['./', 'torch'])[source]
Bases: langchain.llms.base.LLM
Run model inference on self-hosted remote hardware.
Supported hardware includes auto-launched instances on AWS, GCP, Azure,
and Lambda, as well as servers specified
by IP address and SSH credentials (such as on-prem, or another
cloud like Paperspace, Coreweave, etc.).
To use, you should have the runhouse python package installed.
Example for custom pipeline and inference functions:from langchain.llms import SelfHostedPipeline
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
import runhouse as rh
def load_pipeline():
    tokenizer = AutoTokenizer.from_pretrained("gpt2")
    model = AutoModelForCausalLM.from_pretrained("gpt2")
    return pipeline(
        "text-generation", model=model, tokenizer=tokenizer,
        max_new_tokens=10
    )
def inference_fn(pipeline, prompt, stop = None):
    return pipeline(prompt)[0]["generated_text"]
gpu = rh.cluster(name="rh-a10x", instance_type="A100:1")
llm = SelfHostedPipeline(
    model_load_fn=load_pipeline,
    hardware=gpu,
    model_reqs=model_reqs, inference_fn=inference_fn
)
Example for <2GB model (can be serialized and sent directly to the server):from langchain.llms import SelfHostedPipeline
import runhouse as rh
gpu = rh.cluster(name="rh-a10x", instance_type="A100:1")
my_model = ...
llm = SelfHostedPipeline.from_pipeline(
    pipeline=my_model,
    hardware=gpu,
    model_reqs=["./", "torch", "transformers"],
)
Example passing model path for larger models:from langchain.llms import SelfHostedPipeline
import runhouse as rh
import pickle
from transformers import pipeline
generator = pipeline(model="gpt2")
rh.blob(pickle.dumps(generator), path="models/pipeline.pkl"
    ).save().to(gpu, path="models")
llm = SelfHostedPipeline.from_pipeline(
    pipeline="models/pipeline.pkl",
    hardware=gpu,
    model_reqs=["./", "torch", "transformers"],
)
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
pipeline_ref (Any) – 
client (Any) – 
inference_fn (Callable) – 
hardware (Any) – 
model_load_fn (Callable) – 
load_fn_kwargs (Optional[dict]) – 
model_reqs (List[str]) – 
Return type
None
attribute hardware: Any = None
Remote hardware to send the inference function to.
attribute inference_fn: Callable = <function _generate_text>
Inference function to send to the remote hardware.
attribute load_fn_kwargs: Optional[dict] = None
Key word arguments to pass to the model load function.
attribute model_load_fn: Callable [Required]
Function to load the model remotely on the server.
attribute model_reqs: List[str] = ['./', 'torch']
Requirements to install on hardware to inference the model.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
classmethod from_pipeline(pipeline, hardware, model_reqs=None, device=0, **kwargs)[source]
Init the SelfHostedPipeline from a pipeline object or string.
Parameters
pipeline (Any) – 
hardware (Any) – 
model_reqs (Optional[List[str]]) – 
device (int) – 
kwargs (Any) – 
Return type
langchain.llms.base.LLM
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.StochasticAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, api_url='', model_kwargs=None, stochasticai_api_key=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around StochasticAI large language models.
To use, you should have the environment variable STOCHASTICAI_API_KEY
set with your API key.
Example
from langchain.llms import StochasticAI
stochasticai = StochasticAI(api_url="")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
api_url (str) – 
model_kwargs (Dict[str, Any]) – 
stochasticai_api_key (Optional[str]) – 
Return type
None
attribute api_url: str = ''
Model name to use.
attribute model_kwargs: Dict[str, Any] [Optional]
Holds any model parameters valid for create call not
explicitly specified.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.VertexAI(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, client=None, model_name='text-bison', temperature=0.0, max_output_tokens=128, top_p=0.95, top_k=40, stop=None, project=None, location='us-central1', credentials=None, tuned_model_name=None)[source]
Bases: langchain.llms.vertexai._VertexAICommon, langchain.llms.base.LLM
Wrapper around Google Vertex AI large language models.
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
client (_LanguageModel) – 
model_name (str) – 
temperature (float) – 
max_output_tokens (int) – 
top_p (float) – 
top_k (int) – 
stop (Optional[List[str]]) – 
project (Optional[str]) – 
location (str) – 
credentials (Any) – 
tuned_model_name (Optional[str]) – 
Return type
None
attribute credentials: Any = None
The default custom credentials (google.auth.credentials.Credentials) to use
attribute location: str = 'us-central1'
The default location to use when making API calls.
attribute max_output_tokens: int = 128
Token limit determines the maximum amount of text output from one prompt.
attribute model_name: str = 'text-bison'
The name of the Vertex AI large language model.
attribute project: Optional[str] = None
The default GCP project to use when making Vertex API calls.
attribute stop: Optional[List[str]] = None
Optional list of stop words to use when generating.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: float = 0.0
Sampling temperature, it controls the degree of randomness in token selection.
attribute top_k: int = 40
How the model selects tokens for output, the next token is selected from
attribute top_p: float = 0.95
Tokens are selected from most probable to least until the sum of their
attribute tuned_model_name: Optional[str] = None
The name of a tuned model. If provided, model_name is ignored.
attribute verbose: bool [Optional]
Whether to print out response text.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.
class langchain.llms.Writer(*, cache=None, verbose=None, callbacks=None, callback_manager=None, tags=None, writer_org_id=None, model_id='palmyra-instruct', min_tokens=None, max_tokens=None, temperature=None, top_p=None, stop=None, presence_penalty=None, repetition_penalty=None, best_of=None, logprobs=False, n=None, writer_api_key=None, base_url=None)[source]
Bases: langchain.llms.base.LLM
Wrapper around Writer large language models.
To use, you should have the environment variable WRITER_API_KEY and
WRITER_ORG_ID set with your API key and organization ID respectively.
Example
from langchain import Writer
writer = Writer(model_id="palmyra-base")
Parameters
cache (Optional[bool]) – 
verbose (bool) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
callback_manager (Optional[langchain.callbacks.base.BaseCallbackManager]) – 
tags (Optional[List[str]]) – 
writer_org_id (Optional[str]) – 
model_id (str) – 
min_tokens (Optional[int]) – 
max_tokens (Optional[int]) – 
temperature (Optional[float]) – 
top_p (Optional[float]) – 
stop (Optional[List[str]]) – 
presence_penalty (Optional[float]) – 
repetition_penalty (Optional[float]) – 
best_of (Optional[int]) – 
logprobs (bool) – 
n (Optional[int]) – 
writer_api_key (Optional[str]) – 
base_url (Optional[str]) – 
Return type
None
attribute base_url: Optional[str] = None
Base url to use, if None decides based on model name.
attribute best_of: Optional[int] = None
Generates this many completions server-side and returns the “best”.
attribute logprobs: bool = False
Whether to return log probabilities.
attribute max_tokens: Optional[int] = None
Maximum number of tokens to generate.
attribute min_tokens: Optional[int] = None
Minimum number of tokens to generate.
attribute model_id: str = 'palmyra-instruct'
Model name to use.
attribute n: Optional[int] = None
How many completions to generate.
attribute presence_penalty: Optional[float] = None
Penalizes repeated tokens regardless of frequency.
attribute repetition_penalty: Optional[float] = None
Penalizes repeated tokens according to frequency.
attribute stop: Optional[List[str]] = None
Sequences when completion generation will stop.
attribute tags: Optional[List[str]] = None
Tags to add to the run trace.
attribute temperature: Optional[float] = None
What sampling temperature to use.
attribute top_p: Optional[float] = None
Total probability mass of tokens to consider at each step.
attribute verbose: bool [Optional]
Whether to print out response text.
attribute writer_api_key: Optional[str] = None
Writer API key.
attribute writer_org_id: Optional[str] = None
Writer organization ID.
__call__(prompt, stop=None, callbacks=None, **kwargs)
Check Cache and run the LLM on the given prompt and input.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
str
async agenerate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async agenerate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
async apredict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
async apredict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
classmethod construct(_fields_set=None, **values)
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include=None, exclude=None, update=None, deep=False)
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs)
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
generate(prompts, stop=None, callbacks=None, *, tags=None, **kwargs)
Run the LLM on the given prompt and input.
Parameters
prompts (List[str]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
generate_prompt(prompts, stop=None, callbacks=None, **kwargs)
Take in a list of prompt values and return an LLMResult.
Parameters
prompts (List[langchain.schema.PromptValue]) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[langchain.callbacks.base.BaseCallbackHandler], langchain.callbacks.base.BaseCallbackManager]]) – 
kwargs (Any) – 
Return type
langchain.schema.LLMResult
get_num_tokens(text)
Get the number of tokens present in the text.
Parameters
text (str) – 
Return type
int
get_num_tokens_from_messages(messages)
Get the number of tokens in the message.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
Return type
int
get_token_ids(text)
Get the token present in the text.
Parameters
text (str) – 
Return type
List[int]
json(*, include=None, exclude=None, by_alias=False, skip_defaults=None, exclude_unset=False, exclude_defaults=False, exclude_none=False, encoder=None, models_as_dict=True, **dumps_kwargs)
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
predict(text, *, stop=None, **kwargs)
Predict text from text.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages, *, stop=None, **kwargs)
Predict message from messages.
Parameters
messages (List[langchain.schema.BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
langchain.schema.BaseMessage
save(file_path)
Save the LLM.
Parameters
file_path (Union[pathlib.Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod update_forward_refs(**localns)
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
property lc_attributes: Dict
Return a list of attribute names that should be included in the
serialized kwargs. These attributes must be accepted by the
constructor.
property lc_namespace: List[str]
Return the namespace of the langchain object.
eg. [“langchain”, “llms”, “openai”]
property lc_secrets: Dict[str, str]
Return a map of constructor argument names to secret ids.
eg. {“openai_api_key”: “OPENAI_API_KEY”}
property lc_serializable: bool
Return whether or not the class is serializable.