Tony20100/langchain_full huggingface dataset:
def _lambda(x: int) -> int:
    return x + 1
runnable = RunnableLambda(_lambda)
print(runnable.map().invoke([1, 2, 3])) # [2, 3, 4]
Return type
Runnable[List[Input], List[Output]]
max_tokens_for_prompt(prompt: str) → int¶
Calculate the maximum number of tokens possible to generate for a prompt.
Parameters
prompt (str) – The prompt to pass into the model.
Returns
The maximum number of tokens to generate for a prompt.
Return type
int
Example
max_tokens = openai.max_token_for_prompt("Tell me a joke.")
static modelname_to_contextsize(modelname: str) → int¶
Calculate the maximum number of tokens possible to generate for a model.
Parameters
modelname (str) – The modelname we want to know the context size for.
Returns
The maximum context size
Return type
int
Example
max_tokens = openai.modelname_to_contextsize("gpt-3.5-turbo-instruct")
classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
Parameters
path (Union[str, Path]) – 
content_type (unicode) – 
encoding (unicode) – 
proto (Protocol) – 
allow_pickle (bool) – 
Return type
Model
classmethod parse_obj(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
Parameters
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property max_context_size: int¶
Get max context size for this model.
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using OpenLM¶
OpenLM
langchain_community.llms.azureml_endpoint.CustomOpenAIContentFormatter¶
class langchain_community.llms.azureml_endpoint.CustomOpenAIContentFormatter[source]¶
Content formatter for models that use the OpenAI like API scheme.
Attributes
accepts
The MIME type of the response data returned from the endpoint
content_type
The MIME type of the input data passed to the endpoint
format_error_msg
supported_api_types
Supported APIs for the given formatter.
Methods
__init__()
escape_special_characters(prompt)
Escapes any special characters in prompt
format_request_payload(prompt, model_kwargs, ...)
Formats the request according to the chosen api
format_response_payload(output, api_type)
Formats response
__init__()¶
static escape_special_characters(prompt: str) → str¶
Escapes any special characters in prompt
Parameters
prompt (str) – 
Return type
str
format_request_payload(prompt: str, model_kwargs: Dict, api_type: AzureMLEndpointApiType) → bytes[source]¶
Formats the request according to the chosen api
Parameters
prompt (str) – 
model_kwargs (Dict) – 
api_type (AzureMLEndpointApiType) – 
Return type
bytes
format_response_payload(output: bytes, api_type: AzureMLEndpointApiType) → Generation[source]¶
Formats response
Parameters
output (bytes) – 
api_type (AzureMLEndpointApiType) – 
Return type
Generation
Examples using CustomOpenAIContentFormatter¶
Azure ML
pipeline=my_model,
    hardware=gpu,
    model_reqs=["./", "torch", "transformers"],
)
Example passing model path for larger models:from langchain_community.llms import SelfHostedPipeline
import runhouse as rh
import pickle
from transformers import pipeline
generator = pipeline(model="gpt2")
rh.blob(pickle.dumps(generator), path="models/pipeline.pkl"
    ).save().to(gpu, path="models")
llm = SelfHostedPipeline.from_pipeline(
    pipeline="models/pipeline.pkl",
    hardware=gpu,
    model_reqs=["./", "torch", "transformers"],
)
Init the pipeline with an auxiliary function.
The load function must be in global scope to be imported
and run on the server, i.e. in a module and not a REPL or closure.
Then, initialize the remote inference function.
param allow_dangerous_deserialization: bool = False¶
Allow deserialization using pickle which can be dangerous if
loading compromised data.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param hardware: Any = None¶
Remote hardware to send the inference function to.
param hardware: Any = None¶
Remote hardware to send the inference function to.
param inference_fn: Callable = <function _generate_text>¶
Inference function to send to the remote hardware.
param load_fn_kwargs: Optional[dict] = None¶
Keyword arguments to pass to the model load function.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_load_fn: Callable [Required]¶
Function to load the model remotely on the server.
param model_reqs: List[str] = ['./', 'torch']¶
Requirements to install on hardware to inference the model.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
values (Any) – 
Return type
Model
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs: Any) → Dict¶
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
classmethod from_pipeline(pipeline: Any, hardware: Any, model_reqs: Optional[List[str]] = None, device: int = 0, **kwargs: Any) → LLM[source]¶
Init the SelfHostedPipeline from a pipeline object or string.
Parameters
pipeline (Any) – 
hardware (Any) – 
model_reqs (Optional[List[str]]) – 
device (int) – 
kwargs (Any) – 
Return type
LLM
device (int) – 
kwargs (Any) – 
Return type
LLM
generate(prompts: List[str], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, *, tags: Optional[Union[List[str], List[List[str]]]] = None, metadata: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, run_name: Optional[Union[str, List[str]]] = None, run_id: Optional[Union[UUID, List[Optional[UUID]]]] = None, **kwargs: Any) → LLMResult¶
Pass a sequence of prompts to a model and return generations.
This method should make use of batched calls for models that expose a batched
API.
Use this method when you want to:
take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language modeltype (e.g., pure text completion models vs chat models).
Parameters
prompts (List[str]) – List of string prompts.
stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the
first occurrence of any of these substrings.
callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional
functionality, such as logging or streaming, throughout generation.
**kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed
to the model provider API call.
tags (Optional[Union[List[str], List[List[str]]]]) –
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using SelfHostedPipeline¶
Runhouse
langchain_community.llms.bittensor.NIBittensorLLM¶
class langchain_community.llms.bittensor.NIBittensorLLM[source]¶
Bases: LLM
NIBittensor LLMs
NIBittensorLLM is created by Neural Internet (https://neuralinternet.ai/),
powered by Bittensor, a decentralized network full of different AI models.
To analyze API_KEYS and logs of your usage visithttps://api.neuralinternet.ai/api-keys
https://api.neuralinternet.ai/logs
Example
from langchain_community.llms import NIBittensorLLM
llm = NIBittensorLLM()
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param system_prompt: Optional[str] = None¶
Provide system prompt that you want to supply it to model before every prompt
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param top_responses: Optional[int] = 0¶
Provide top_responses to get Top N miner responses on one request.May get delayed
Don’t use in Production
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using NIBittensorLLM¶
Bittensor
langchain_community.llms.volcengine_maas.VolcEngineMaasLLM¶
class langchain_community.llms.volcengine_maas.VolcEngineMaasLLM[source]¶
Bases: LLM, VolcEngineMaasBase
volc engine maas hosts a plethora of models.
You can utilize these models through this class.
To use, you should have the volcengine python package installed.
and set access key and secret key by environment variable or direct pass those to
this class.
access key, secret key are required parameters which you could get help
https://www.volcengine.com/docs/6291/65568
In order to use them, it is necessary to install the ‘volcengine’ Python package.
The access key and secret key must be set either via environment variables or
passed directly to this class.
access key and secret key are mandatory parameters for which assistance can be
sought at https://www.volcengine.com/docs/6291/65568.
Example
from langchain_community.llms import VolcEngineMaasLLM
model = VolcEngineMaasLLM(model="skylark-lite-public",
                              volc_engine_maas_ak="your_ak",
                              volc_engine_maas_sk="your_sk")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param client: Any = None¶
param connect_timeout: Optional[int] = 60¶
Timeout for connect to volc engine maas endpoint. Default is 60 seconds.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param endpoint: Optional[str] = 'maas-api.ml-platform-cn-beijing.volces.com'¶
Endpoint of the VolcEngineMaas LLM.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model: str = 'skylark-lite-public'¶
Model name. you could check this model details here
https://www.volcengine.com/docs/82379/1133187
and you could choose other models by change this field
param model_kwargs: Dict[str, Any] [Optional]¶
model special arguments, you could check detail on model page
param model_version: Optional[str] = None¶
Model version. Only used in moonshot large language model.
you could check details here https://www.volcengine.com/docs/82379/1158281
param read_timeout: Optional[int] = 60¶
Timeout for read response from volc engine maas endpoint.
Default is 60 seconds.
param region: Optional[str] = 'Region'¶
Region of the VolcEngineMaas LLM.
param streaming: bool = False¶
Whether to stream the results.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: Optional[float] = 0.95¶
A non-negative float that tunes the degree of randomness in generation.
param top_p: Optional[float] = 0.8¶
Total probability mass of tokens to consider at each step.
param verbose: bool [Optional]¶
Whether to print out response text.
param volc_engine_maas_ak: Optional[SecretStr] = None¶
access key for volc engine
Constraints
type = string
writeOnly = True
format = password
param volc_engine_maas_sk: Optional[SecretStr] = None¶
secret key for volc engine
Constraints
type = string
writeOnly = True
format = password
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using VolcEngineMaasLLM¶
Volc Engine Maas
langchain_aws.llms.bedrock.Bedrock¶
class langchain_aws.llms.bedrock.Bedrock[source]¶
Bases: BedrockLLM
[Deprecated]
Notes
Deprecated since version 0.1.0: Use BedrockLLM instead.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param config: Any = None¶
An optional botocore.config.Config instance to pass to the client.
param credentials_profile_name: Optional[str] = None¶
The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which
has either access keys or role information specified.
If not specified, the default credential profile or, if on an EC2 instance,
credentials from IMDS will be used.
See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param endpoint_url: Optional[str] = None¶
Needed if you don’t want to default to us-east-1 endpoint
Needed if you don’t want to default to us-east-1 endpoint
param guardrails: Optional[Mapping[str, Any]] = {'guardrailIdentifier': None, 'guardrailVersion': None, 'trace': None}¶
An optional dictionary to configure guardrails for Bedrock.
This field ‘guardrails’ consists of two keys: ‘id’ and ‘version’,
which should be strings, but are initialized to None. It’s used to
determine if specific guardrails are enabled and properly set.
Type:Optional[Mapping[str, str]]: A mapping with ‘id’ and ‘version’ keys.
Example:
llm = Bedrock(model_id=”<model_id>”, client=<bedrock_client>,
model_kwargs={},
guardrails={
“id”: “<guardrail_id>”,
“version”: “<guardrail_version>”})
To enable tracing for guardrails, set the ‘trace’ key to True and pass a callback handler to the
‘run_manager’ parameter of the ‘generate’, ‘_call’ methods.
Example:
llm = Bedrock(model_id=”<model_id>”, client=<bedrock_client>,
model_kwargs={},
guardrails={
“id”: “<guardrail_id>”,
“version”: “<guardrail_version>”,
“trace”: True},
callbacks=[BedrockAsyncCallbackHandler()])
[https://python.langchain.com/docs/modules/callbacks/] for more information on callback handlers.
class BedrockAsyncCallbackHandler(AsyncCallbackHandler):
async def on_llm_error(self,
error: BaseException,
**kwargs: Any,
) -> Any:reason = kwargs.get(“reason”)
if reason == “GUARDRAIL_INTERVENED”:
…Logic to handle guardrail intervention…
param metadata: Optional[Dict[str, Any]] = None¶
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_id: str [Required]¶
Id of the model to call, e.g., amazon.titan-text-express-v1, this is
equivalent to the modelId property in the list-foundation-models api. For custom and
provisioned models, an ARN value is expected.
param model_kwargs: Optional[Dict[str, Any]] = None¶
Keyword arguments to pass to the model.
param provider: Optional[str] = None¶
The model provider, e.g., amazon, cohere, ai21, etc. When not supplied, provider
is extracted from the first part of the model_id e.g. ‘amazon’ in
‘amazon.titan-text-express-v1’. This value should be provided for model ids that do
not have the provider in them, e.g., custom and provisioned models that have an ARN
associated with them.
param provider_stop_reason_key_map: Mapping[str, str] = {'ai21': 'finishReason', 'amazon': 'completionReason', 'anthropic': 'stop_reason', 'cohere': 'finish_reason', 'mistral': 'stop_reason'}¶
param provider_stop_sequence_key_name_map: Mapping[str, str] = {'ai21': 'stop_sequences', 'amazon': 'stopSequences', 'anthropic': 'stop_sequences', 'cohere': 'stop_sequences', 'mistral': 'stop_sequences'}¶
param region_name: Optional[str] = None¶
The aws region e.g., us-west-2. Fallsback to AWS_DEFAULT_REGION env variable
or region specified in ~/.aws/config in case it is not provided here.
param streaming: bool = False¶
Whether to stream the results.
param tags: Optional[List[str]] = None¶