Tony20100/langchain_full huggingface dataset:
Construct the pipeline object from model_id and task.
Parameters
model_id (str) – 
task (str) – 
device (Optional[int]) – 
device_map (Optional[str]) – 
model_kwargs (Optional[dict]) – 
pipeline_kwargs (Optional[dict]) – 
load_in_4bit (Optional[bool]) – 
load_in_8bit (Optional[bool]) – 
quantization_config (Optional[Any]) – 
kwargs (Any) – 
Return type
LLM
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
generate(prompts: List[str], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, *, tags: Optional[Union[List[str], List[List[str]]]] = None, metadata: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, run_name: Optional[Union[str, List[str]]] = None, run_id: Optional[Union[UUID, List[Optional[UUID]]]] = None, **kwargs: Any) → LLMResult¶
Pass a sequence of prompts to a model and return generations.
This method should make use of batched calls for models that expose a batched
API.
Use this method when you want to:
take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language modeltype (e.g., pure text completion models vs chat models).
Parameters
prompts (List[str]) – List of string prompts.
Parameters
prompts (List[str]) – List of string prompts.
stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the
first occurrence of any of these substrings.
callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional
functionality, such as logging or streaming, throughout generation.
**kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed
to the model provider API call.
tags (Optional[Union[List[str], List[List[str]]]]) – 
metadata (Optional[Union[Dict[str, Any], List[Dict[str, Any]]]]) – 
run_name (Optional[Union[str, List[str]]]) – 
run_id (Optional[Union[UUID, List[Optional[UUID]]]]) – 
**kwargs – 
Returns
An LLMResult, which contains a list of candidate Generations for each inputprompt and additional model provider-specific output.
Return type
LLMResult
generate_prompt(prompts: List[PromptValue], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, **kwargs: Any) → LLMResult¶
Pass a sequence of prompts to the model and return model generations.
This method should make use of batched calls for models that expose a batched
API.
Use this method when you want to:
take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language modeltype (e.g., pure text completion models vs chat models).
Parameters
Parameters
prompts (List[PromptValue]) – List of PromptValues. A PromptValue is an object that can be
converted to match the format of any language model (string for pure
text generation models and BaseMessages for chat models).
stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the
first occurrence of any of these substrings.
callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional
functionality, such as logging or streaming, throughout generation.
**kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed
to the model provider API call.
Returns
An LLMResult, which contains a list of candidate Generations for each inputprompt and additional model provider-specific output.
Return type
LLMResult
get_graph(config: Optional[RunnableConfig] = None) → Graph¶
Return a graph representation of this runnable.
Parameters
config (Optional[RunnableConfig]) – 
Return type
Graph
get_input_schema(config: Optional[RunnableConfig] = None) → Type[BaseModel]¶
Get a pydantic model that can be used to validate input to the runnable.
Runnables that leverage the configurable_fields and configurable_alternatives
methods will have a dynamic input schema that depends on which
configuration the runnable is invoked with.
This method allows to get an input schema for a specific configuration.
Parameters
config (Optional[RunnableConfig]) – A config to use when generating the schema.
Returns
A pydantic model that can be used to validate input.
Return type
Type[BaseModel]
classmethod get_lc_namespace() → List[str]¶
Get the namespace of the langchain object.
Get the namespace of the langchain object.
For example, if the class is langchain.llms.openai.OpenAI, then the
namespace is [“langchain”, “llms”, “openai”]
Return type
List[str]
get_name(suffix: Optional[str] = None, *, name: Optional[str] = None) → str¶
Get the name of the runnable.
Parameters
suffix (Optional[str]) – 
name (Optional[str]) – 
Return type
str
get_num_tokens(text: str) → int¶
Get the number of tokens present in the text.
Useful for checking if an input will fit in a model’s context window.
Parameters
text (str) – The string input to tokenize.
Returns
The integer number of tokens in the text.
Return type
int
get_num_tokens_from_messages(messages: List[BaseMessage]) → int¶
Get the number of tokens in the messages.
Useful for checking if an input will fit in a model’s context window.
Parameters
messages (List[BaseMessage]) – The message inputs to tokenize.
Returns
The sum of the number of tokens across the messages.
Return type
int
get_output_schema(config: Optional[RunnableConfig] = None) → Type[BaseModel]¶
Get a pydantic model that can be used to validate output to the runnable.
Runnables that leverage the configurable_fields and configurable_alternatives
methods will have a dynamic output schema that depends on which
configuration the runnable is invoked with.
This method allows to get an output schema for a specific configuration.
Parameters
config (Optional[RunnableConfig]) – A config to use when generating the schema.
Returns
A pydantic model that can be used to validate output.
Return type
Type[BaseModel]
Return type
Type[BaseModel]
get_prompts(config: Optional[RunnableConfig] = None) → List[BasePromptTemplate]¶
Parameters
config (Optional[RunnableConfig]) – 
Return type
List[BasePromptTemplate]
get_token_ids(text: str) → List[int]¶
Return the ordered ids of the tokens in a text.
Parameters
text (str) – The string input to tokenize.
Returns
A list of ids corresponding to the tokens in the text, in order they occurin the text.
Return type
List[int]
invoke(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → str¶
Transform a single input into an output. Override to implement.
Parameters
input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) – The input to the runnable.
config (Optional[RunnableConfig]) – A config to use when invoking the runnable.
The config supports standard keys like ‘tags’, ‘metadata’ for tracing
purposes, ‘max_concurrency’ for controlling how much work to do
in parallel, and other keys. Please refer to the RunnableConfig
for more details.
stop (Optional[List[str]]) – 
kwargs (Any) – 
Returns
The output of the runnable.
Return type
str
classmethod is_lc_serializable() → bool¶
Is this class serializable?
Return type
bool
return x * 2
runnable_1 = RunnableLambda(add_one)
runnable_2 = RunnableLambda(mul_two)
sequence = runnable_1.pipe(runnable_2)
# Or equivalently:
# sequence = runnable_1 | runnable_2
# sequence = RunnableSequence(first=runnable_1, last=runnable_2)
sequence.invoke(1)
await sequence.ainvoke(1)
# -> 4
sequence.batch([1, 2, 3])
await sequence.abatch([1, 2, 3])
# -> [4, 6, 8]
Parameters
others (Union[Runnable[Any, Other], Callable[[Any], Other]]) – 
name (Optional[str]) – 
Return type
RunnableSerializable[Input, Other]
predict(text: str, *, stop: Optional[Sequence[str]] = None, **kwargs: Any) → str¶
[Deprecated]
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
text (str) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
str
predict_messages(messages: List[BaseMessage], *, stop: Optional[Sequence[str]] = None, **kwargs: Any) → BaseMessage¶
[Deprecated]
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
messages (List[BaseMessage]) – 
stop (Optional[Sequence[str]]) – 
kwargs (Any) – 
Return type
BaseMessage
save(file_path: Union[Path, str]) → None¶
Save the LLM.
Parameters
file_path (Union[Path, str]) – Path to file to save the LLM to.
Return type
None
Example:
Return type
None
Example:
.. code-block:: python
llm.save(file_path=”path/llm.yaml”)
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
Return type
DictStrAny
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
dumps_kwargs (Any) – 
Return type
unicode
stream(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → Iterator[str]¶
Default implementation of stream, which calls invoke.
Subclasses should override this method if they support streaming output.
Parameters
input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) – 
config (Optional[RunnableConfig]) – 
stop (Optional[List[str]]) – 
kwargs (Any) – 
Return type
Iterator[str]
to_json() → Union[SerializedConstructor, SerializedNotImplemented]¶
Serialize the runnable to JSON.
Return type
Union[SerializedConstructor, SerializedNotImplemented]
to_json_not_implemented() → SerializedNotImplemented¶
Return type
SerializedNotImplemented
transform(input: Iterator[Input], config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → Iterator[Output]¶
Default implementation of transform, which buffers input and then calls stream.
Not implemented on this class.
Parameters
schema (Union[Dict, Type[BaseModel]]) – 
kwargs (Any) – 
Return type
Runnable[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], Union[Dict, BaseModel]]
with_types(*, input_type: Optional[Type[Input]] = None, output_type: Optional[Type[Output]] = None) → Runnable[Input, Output]¶
Bind input and output types to a Runnable, returning a new Runnable.
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using WeightOnlyQuantPipeline¶
Intel Weight-Only Quantization
langchain_community.llms.azureml_endpoint.ContentFormatterBase¶
class langchain_community.llms.azureml_endpoint.ContentFormatterBase[source]¶
Transform request and response of AzureML endpoint to match with
required schema.
Attributes
accepts
The MIME type of the response data returned from the endpoint
content_type
The MIME type of the input data passed to the endpoint
format_error_msg
supported_api_types
Supported APIs for the given formatter.
Methods
__init__()
escape_special_characters(prompt)
Escapes any special characters in prompt
format_request_payload(prompt, model_kwargs)
Formats the request body according to the input schema of the model.
format_response_payload(output[, api_type])
Formats the response body according to the output schema of the model.
__init__()¶
static escape_special_characters(prompt: str) → str[source]¶
Escapes any special characters in prompt
Parameters
prompt (str) – 
Return type
str
format_request_payload(prompt: str, model_kwargs: Dict, api_type: AzureMLEndpointApiType = AzureMLEndpointApiType.dedicated) → Any[source]¶
Formats the request body according to the input schema of
the model. Returns bytes or seekable file like object in the
format specified in the content_type request header.
Parameters
prompt (str) – 
model_kwargs (Dict) – 
api_type (AzureMLEndpointApiType) – 
Return type
Any
abstract format_response_payload(output: bytes, api_type: AzureMLEndpointApiType = AzureMLEndpointApiType.dedicated) → Generation[source]¶
Formats the response body according to the output
schema of the model. Returns the data type that is
received from the response.
Parameters
output (bytes) – 
api_type (AzureMLEndpointApiType) – 
Return type
Generation
langchain_community.llms.azureml_endpoint.OSSContentFormatter¶
class langchain_community.llms.azureml_endpoint.OSSContentFormatter[source]¶
Deprecated: Kept for backwards compatibility
Content handler for LLMs from the OSS catalog.
Attributes
accepts
The MIME type of the response data returned from the endpoint
content_formatter
content_type
The MIME type of the input data passed to the endpoint
format_error_msg
supported_api_types
Supported APIs for the given formatter.
Methods
__init__()
escape_special_characters(prompt)
Escapes any special characters in prompt
format_request_payload(prompt, model_kwargs, ...)
Formats the request body according to the input schema of the model.
format_response_payload(output, api_type)
Formats the response body according to the output schema of the model.
__init__() → None[source]¶
Return type
None
static escape_special_characters(prompt: str) → str¶
Escapes any special characters in prompt
Parameters
prompt (str) – 
Return type
str
format_request_payload(prompt: str, model_kwargs: Dict, api_type: AzureMLEndpointApiType) → bytes¶
Formats the request body according to the input schema of
the model. Returns bytes or seekable file like object in the
format specified in the content_type request header.
Parameters
prompt (str) – 
model_kwargs (Dict) – 
api_type (AzureMLEndpointApiType) – 
Return type
bytes
format_response_payload(output: bytes, api_type: AzureMLEndpointApiType) → Generation¶
Formats the response body according to the output
schema of the model. Returns the data type that is
received from the response.
Parameters
output (bytes) – 
api_type (AzureMLEndpointApiType) – 
Return type
Generation
langchain_community.llms.aviary.Aviary¶
class langchain_community.llms.aviary.Aviary[source]¶
Bases: LLM
Aviary hosted models.
Aviary is a backend for hosted models. You can
find out more about aviary at
http://github.com/ray-project/aviary
To get a list of the models supported on an
aviary, follow the instructions on the website to
install the aviary CLI and then use:
aviary models
AVIARY_URL and AVIARY_TOKEN environment variables must be set.
model¶
The name of the model to use. Defaults to “amazon/LightGPT”.
aviary_url¶
The URL for the Aviary backend. Defaults to None.
aviary_token¶
The bearer token for the Aviary backend. Defaults to None.
use_prompt_format¶
If True, the prompt template for the model will be ignored.
Defaults to True.
version¶
API version to use for Aviary. Defaults to None.
Example
from langchain_community.llms import Aviary
os.environ["AVIARY_URL"] = "<URL>"
os.environ["AVIARY_TOKEN"] = "<TOKEN>"
light = Aviary(model='amazon/LightGPT')
output = light('How do you make fried rice?')
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param aviary_token: Optional[str] = None¶
param aviary_url: Optional[str] = None¶
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model: str = 'amazon/LightGPT'¶
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param use_prompt_format: bool = True¶
param verbose: bool [Optional]¶
Whether to print out response text.
param version: Optional[str] = None¶
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
langchain_community.llms.vllm.VLLM¶
class langchain_community.llms.vllm.VLLM[source]¶
Bases: BaseLLM
VLLM language model.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param best_of: Optional[int] = None¶
Number of output sequences that are generated from the prompt.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param download_dir: Optional[str] = None¶
Directory to download and load the weights. (Default to the default
cache dir of huggingface)
param dtype: str = 'auto'¶
The data type for the model weights and activations.
param frequency_penalty: float = 0.0¶
Float that penalizes new tokens based on their frequency in the
generated text so far
param ignore_eos: bool = False¶
Whether to ignore the EOS token and continue generating tokens after
the EOS token is generated.
param logprobs: Optional[int] = None¶
Number of log probabilities to return per output token.
param max_new_tokens: int = 512¶
param max_new_tokens: int = 512¶
Maximum number of tokens to generate per output sequence.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model: str = ''¶
The name or path of a HuggingFace Transformers model.
param n: int = 1¶
Number of output sequences to return for the given prompt.
param presence_penalty: float = 0.0¶
Float that penalizes new tokens based on whether they appear in the
generated text so far
param stop: Optional[List[str]] = None¶
List of strings that stop the generation when they are generated.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: float = 1.0¶
Float that controls the randomness of the sampling.
param tensor_parallel_size: Optional[int] = 1¶
The number of GPUs to use for distributed execution with tensor parallelism.
param top_k: int = -1¶
Integer that controls the number of top tokens to consider.
param top_p: float = 1.0¶
Float that controls the cumulative probability of the top tokens to consider.
param trust_remote_code: Optional[bool] = False¶
Trust remote code (e.g., from HuggingFace) when downloading the model
and tokenizer.
param use_beam_search: bool = False¶
Whether to use beam search instead of sampling.
param verbose: bool [Optional]¶
Whether to print out response text.
param vllm_kwargs: Dict[str, Any] [Optional]¶
Holds any model parameters valid for vllm.LLM call not explicitly specified.
Holds any model parameters valid for vllm.LLM call not explicitly specified.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]]) – 
config (Optional[Union[RunnableConfig, List[RunnableConfig]]]) – 
return_exceptions (bool) – 
kwargs (Any) – 
Return type
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using VLLM¶
vLLM