viruPVS/langchain-docs huggingface dataset:
Get a pydantic model that can be used to validate output to the runnable.
Runnables that leverage the configurable_fields and configurable_alternatives
methods will have a dynamic output schema that depends on which
configuration the runnable is invoked with.
This method allows to get an output schema for a specific configuration.
Parameters
config – A config to use when generating the schema.
Returns
A pydantic model that can be used to validate output.
get_sub_prompts(params: Dict[str, Any], prompts: List[str], stop: Optional[List[str]] = None) → List[List[str]]¶
Get the sub prompts for llm call.
get_token_ids(text: str) → List[int]¶
Get the token IDs using the tiktoken package.
invoke(input: Union[PromptValue, str, List[BaseMessage]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → str¶
Transform a single input into an output. Override to implement.
Parameters
input – The input to the runnable.
config – A config to use when invoking the runnable.
The config supports standard keys like ‘tags’, ‘metadata’ for tracing
purposes, ‘max_concurrency’ for controlling how much work to do
in parallel, and other keys. Please refer to the RunnableConfig
for more details.
Returns
The output of the runnable.
classmethod is_lc_serializable() → bool[source]¶
Is this class serializable?
between retries
stop_after_attempt – The maximum number of attempts to make before giving up
Returns
A new Runnable that retries the original runnable on exceptions.
with_types(*, input_type: Optional[Type[Input]] = None, output_type: Optional[Type[Output]] = None) → Runnable[Input, Output]¶
Bind input and output types to a Runnable, returning a new Runnable.
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict[str, Any]¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property max_context_size: int¶
Get max context size for this model.
property output_schema: Type[pydantic.main.BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using OpenLM¶
OpenLM
langchain_community.llms.fireworks.acompletion_with_retry_batching¶
async langchain_community.llms.fireworks.acompletion_with_retry_batching(llm: Fireworks, use_retry: bool, *, run_manager: Optional[AsyncCallbackManagerForLLMRun] = None, **kwargs: Any) → Any[source]¶
Use tenacity to retry the completion call.
langchain_community.llms.self_hosted_hugging_face.SelfHostedHuggingFaceLLM¶
class langchain_community.llms.self_hosted_hugging_face.SelfHostedHuggingFaceLLM[source]¶
Bases: SelfHostedPipeline
HuggingFace Pipeline API to run on self-hosted remote hardware.
Supported hardware includes auto-launched instances on AWS, GCP, Azure,
and Lambda, as well as servers specified
by IP address and SSH credentials (such as on-prem, or another cloud
like Paperspace, Coreweave, etc.).
To use, you should have the runhouse python package installed.
Only supports text-generation, text2text-generation and summarization for now.
Example using from_model_id:from langchain_community.llms import SelfHostedHuggingFaceLLM
import runhouse as rh
gpu = rh.cluster(name="rh-a10x", instance_type="A100:1")
hf = SelfHostedHuggingFaceLLM(
    model_id="google/flan-t5-large", task="text2text-generation",
    hardware=gpu
)
Example passing fn that generates a pipeline (bc the pipeline is not serializable):from langchain_community.llms import SelfHostedHuggingFaceLLM
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
import runhouse as rh
def get_pipeline():
    model_id = "gpt2"
    tokenizer = AutoTokenizer.from_pretrained(model_id)
    model = AutoModelForCausalLM.from_pretrained(model_id)
    pipe = pipeline(
        "text-generation", model=model, tokenizer=tokenizer
    )
    return pipe
hf = SelfHostedHuggingFaceLLM(
    model_load_fn=get_pipeline, model_id="gpt2", hardware=gpu)
Construct the pipeline remotely using an auxiliary function.
Construct the pipeline remotely using an auxiliary function.
The load function needs to be importable to be imported
and run on the server, i.e. in a module and not a REPL or closure.
Then, initialize the remote inference function.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param device: int = 0¶
Device to use for inference. -1 for CPU, 0 for GPU, 1 for second GPU, etc.
param hardware: Any = None¶
Remote hardware to send the inference function to.
param inference_fn: Callable = <function _generate_text>¶
Inference function to send to the remote hardware.
param load_fn_kwargs: Optional[dict] = None¶
Keyword arguments to pass to the model load function.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_id: str = 'gpt2'¶
Hugging Face model_id to load the model.
param model_kwargs: Optional[dict] = None¶
Keyword arguments to pass to the model.
param model_load_fn: Callable = <function _load_transformer>¶
Function to load the model remotely on the server.
param model_reqs: List[str] = ['./', 'transformers', 'torch']¶
Requirements to install on hardware to inference the model.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param task: str = 'text-generation'¶
Hugging Face task (“text-generation”, “text2text-generation” or
“summarization”).
param verbose: bool [Optional]¶
Whether to print out response text.
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep – set to True to make a deep copy of the model
Returns
new model instance
dict(**kwargs: Any) → Dict¶
Return a dictionary of the LLM.
classmethod from_orm(obj: Any) → Model¶
classmethod from_pipeline(pipeline: Any, hardware: Any, model_reqs: Optional[List[str]] = None, device: int = 0, **kwargs: Any) → LLM¶
Init the SelfHostedPipeline from a pipeline object or string.
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property output_schema: Type[pydantic.main.BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using SelfHostedHuggingFaceLLM¶
Runhouse
langchain_community.llms.anthropic.Anthropic¶
class langchain_community.llms.anthropic.Anthropic[source]¶
Bases: LLM, _AnthropicCommon
Anthropic large language models.
To use, you should have the anthropic python package installed, and the
environment variable ANTHROPIC_API_KEY set with your API key, or pass
it as a named parameter to the constructor.
Example
import anthropic
from langchain_community.llms import Anthropic
model = Anthropic(model="<model_name>", anthropic_api_key="my-api-key")
# Simplest invocation, automatically wrapped with HUMAN_PROMPT
# and AI_PROMPT.
response = model("What are the biggest risks facing humanity?")
# Or if you want to use the chat mode, build a few-shot-prompt, or
# put words in the Assistant's mouth, use HUMAN_PROMPT and AI_PROMPT:
raw_prompt = "What are the biggest risks facing humanity?"
prompt = f"{anthropic.HUMAN_PROMPT} {prompt}{anthropic.AI_PROMPT}"
response = model(prompt)
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param AI_PROMPT: Optional[str] = None¶
param HUMAN_PROMPT: Optional[str] = None¶
param anthropic_api_key: Optional[SecretStr] = None¶
Constraints
type = string
writeOnly = True
format = password
param anthropic_api_url: Optional[str] = None¶
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param count_tokens: Optional[Callable[[str], int]] = None¶
param default_request_timeout: Optional[float] = None¶
param default_request_timeout: Optional[float] = None¶
Timeout for requests to Anthropic Completion API. Default is 600 seconds.
param max_tokens_to_sample: int = 256 (alias 'max_tokens')¶
Denotes the number of tokens to predict per generation.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model: str = 'claude-2' (alias 'model_name')¶
Model name to use.
param model_kwargs: Dict[str, Any] [Optional]¶
param streaming: bool = False¶
Whether to stream the results.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: Optional[float] = None¶
A non-negative float that tunes the degree of randomness in generation.
param top_k: Optional[int] = None¶
Number of most likely tokens to consider at each step.
param top_p: Optional[float] = None¶
Total probability mass of tokens to consider at each step.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
Check Cache and run the LLM on the given prompt and input.
async abatch(inputs: List[Union[PromptValue, str, List[BaseMessage]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
and configurable_alternatives methods.
Parameters
include – A list of fields to include in the config schema.
Returns
A pydantic model that can be used to validate config.
configurable_alternatives(which: ConfigurableField, *, default_key: str = 'default', prefix_keys: bool = False, **kwargs: Union[Runnable[Input, Output], Callable[[], Runnable[Input, Output]]]) → RunnableSerializable[Input, Output]¶
configurable_fields(**kwargs: Union[ConfigurableField, ConfigurableFieldSingleOption, ConfigurableFieldMultiOption]) → RunnableSerializable[Input, Output]¶
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
convert_prompt(prompt: PromptValue) → str[source]¶
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep – set to True to make a deep copy of the model
Returns
new model instance
dict(**kwargs: Any) → Dict¶
Return a dictionary of the LLM.
dict(**kwargs: Any) → Dict¶
Return a dictionary of the LLM.
classmethod from_orm(obj: Any) → Model¶
generate(prompts: List[str], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, *, tags: Optional[Union[List[str], List[List[str]]]] = None, metadata: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, run_name: Optional[Union[str, List[str]]] = None, **kwargs: Any) → LLMResult¶
Run the LLM on the given prompt and input.
generate_prompt(prompts: List[PromptValue], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, **kwargs: Any) → LLMResult¶
Pass a sequence of prompts to the model and return model generations.
This method should make use of batched calls for models that expose a batched
API.
Use this method when you want to:
take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language modeltype (e.g., pure text completion models vs chat models).
Parameters
prompts – List of PromptValues. A PromptValue is an object that can be
converted to match the format of any language model (string for pure
text generation models and BaseMessages for chat models).
stop – Stop words to use when generating. Model output is cut off at the
first occurrence of any of these substrings.
callbacks – Callbacks to pass through. Used for executing additional
callbacks – Callbacks to pass through. Used for executing additional
functionality, such as logging or streaming, throughout generation.
**kwargs – Arbitrary additional keyword arguments. These are usually passed
to the model provider API call.
Returns
An LLMResult, which contains a list of candidate Generations for each inputprompt and additional model provider-specific output.
get_input_schema(config: Optional[RunnableConfig] = None) → Type[BaseModel]¶
Get a pydantic model that can be used to validate input to the runnable.
Runnables that leverage the configurable_fields and configurable_alternatives
methods will have a dynamic input schema that depends on which
configuration the runnable is invoked with.
This method allows to get an input schema for a specific configuration.
Parameters
config – A config to use when generating the schema.
Returns
A pydantic model that can be used to validate input.
classmethod get_lc_namespace() → List[str]¶
Get the namespace of the langchain object.
For example, if the class is langchain.llms.openai.OpenAI, then the
namespace is [“langchain”, “llms”, “openai”]
get_num_tokens(text: str) → int[source]¶
Calculate number of tokens.
get_num_tokens_from_messages(messages: List[BaseMessage]) → int¶
Get the number of tokens in the messages.
Useful for checking if an input will fit in a model’s context window.
Parameters
messages – The message inputs to tokenize.
Returns
The sum of the number of tokens across the messages.
get_output_schema(config: Optional[RunnableConfig] = None) → Type[BaseModel]¶
Get a pydantic model that can be used to validate output to the runnable.
Runnables that leverage the configurable_fields and configurable_alternatives
methods will have a dynamic output schema that depends on which
methods will have a dynamic output schema that depends on which
configuration the runnable is invoked with.
This method allows to get an output schema for a specific configuration.
Parameters
config – A config to use when generating the schema.
Returns
A pydantic model that can be used to validate output.
get_token_ids(text: str) → List[int]¶
Return the ordered ids of the tokens in a text.
Parameters
text – The string input to tokenize.
Returns
A list of ids corresponding to the tokens in the text, in order they occurin the text.
invoke(input: Union[PromptValue, str, List[BaseMessage]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → str¶
Transform a single input into an output. Override to implement.
Parameters
input – The input to the runnable.
config – A config to use when invoking the runnable.
The config supports standard keys like ‘tags’, ‘metadata’ for tracing
purposes, ‘max_concurrency’ for controlling how much work to do
in parallel, and other keys. Please refer to the RunnableConfig
for more details.
Returns
The output of the runnable.
classmethod is_lc_serializable() → bool¶
Is this class serializable?
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
langchain_community.llms.azureml_endpoint.OSSContentFormatter¶
class langchain_community.llms.azureml_endpoint.OSSContentFormatter[source]¶
Deprecated: Kept for backwards compatibility
Content handler for LLMs from the OSS catalog.
Attributes
accepts
The MIME type of the response data returned from the endpoint
content_formatter
content_type
The MIME type of the input data passed to the endpoint
Methods
__init__()
escape_special_characters(prompt)
Escapes any special characters in prompt
format_request_payload(prompt, model_kwargs)
Formats the request body according to the input schema of the model.
format_response_payload(output)
Formats the response body according to the output schema of the model.
__init__() → None[source]¶
static escape_special_characters(prompt: str) → str¶
Escapes any special characters in prompt
format_request_payload(prompt: str, model_kwargs: Dict) → bytes¶
Formats the request body according to the input schema of
the model. Returns bytes or seekable file like object in the
format specified in the content_type request header.
format_response_payload(output: bytes) → str¶
Formats the response body according to the output
schema of the model. Returns the data type that is
received from the response.
langchain_community.llms.ai21.AI21¶
class langchain_community.llms.ai21.AI21[source]¶
Bases: LLM
AI21 large language models.
To use, you should have the environment variable AI21_API_KEY
set with your API key or pass it as a named parameter to the constructor.
Example
from langchain_community.llms import AI21
ai21 = AI21(ai21_api_key="my-api-key", model="j2-jumbo-instruct")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param ai21_api_key: Optional[pydantic.types.SecretStr] = None¶
Constraints
type = string
writeOnly = True
format = password
param base_url: Optional[str] = None¶
Base url to use, if None decides based on model name.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param countPenalty: langchain_community.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)¶
Penalizes repeated tokens according to count.
param frequencyPenalty: langchain_community.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)¶
Penalizes repeated tokens according to frequency.
param logitBias: Optional[Dict[str, float]] = None¶
Adjust the probability of specific tokens being generated.
Adjust the probability of specific tokens being generated.
param maxTokens: int = 256¶
The maximum number of tokens to generate in the completion.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param minTokens: int = 0¶
The minimum number of tokens to generate in the completion.
param model: str = 'j2-jumbo-instruct'¶
Model name to use.
param numResults: int = 1¶
How many completions to generate for each prompt.
param presencePenalty: langchain_community.llms.ai21.AI21PenaltyData = AI21PenaltyData(scale=0, applyToWhitespaces=True, applyToPunctuations=True, applyToNumbers=True, applyToStopwords=True, applyToEmojis=True)¶
Penalizes repeated tokens.
param stop: Optional[List[str]] = None¶
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: float = 0.7¶
What sampling temperature to use.
param topP: float = 1.0¶
Total probability mass of tokens to consider at each step.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
Check Cache and run the LLM on the given prompt and input.
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property output_schema: Type[pydantic.main.BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using AI21¶
AI21 Labs
AI21
langchain_community.llms.writer.Writer¶
class langchain_community.llms.writer.Writer[source]¶
Bases: LLM
Writer large language models.
To use, you should have the environment variable WRITER_API_KEY and
WRITER_ORG_ID set with your API key and organization ID respectively.
Example
from langchain_community.llms import Writer
writer = Writer(model_id="palmyra-base")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param base_url: Optional[str] = None¶
Base url to use, if None decides based on model name.
param best_of: Optional[int] = None¶
Generates this many completions server-side and returns the “best”.
param cache: Optional[bool] = None¶
param callback_manager: Optional[BaseCallbackManager] = None¶
param callbacks: Callbacks = None¶
param logprobs: bool = False¶
Whether to return log probabilities.
param max_tokens: Optional[int] = None¶
Maximum number of tokens to generate.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param min_tokens: Optional[int] = None¶
Minimum number of tokens to generate.
param model_id: str = 'palmyra-instruct'¶
Model name to use.
param n: Optional[int] = None¶
How many completions to generate.
param presence_penalty: Optional[float] = None¶
Penalizes repeated tokens regardless of frequency.
param repetition_penalty: Optional[float] = None¶
Penalizes repeated tokens according to frequency.
param stop: Optional[List[str]] = None¶
Sequences when completion generation will stop.
param tags: Optional[List[str]] = None¶