Tony20100/langchain_full huggingface dataset:
langchain_aws.llms.sagemaker_endpoint.LineIterator¶
class langchain_aws.llms.sagemaker_endpoint.LineIterator(stream: Any)[source]¶
A helper class for parsing the byte stream input.
The output of the model will be in the following format:
b’{“outputs”: [” a”]}
‘b’{“outputs”: [” challenging”]}
‘b’{“outputs”: [” problem”]}
‘…
While usually each PayloadPart event from the event stream will
contain a byte array with a full json, this is not guaranteed
and some of the json objects may be split acrossPayloadPart events.
For example:
{‘PayloadPart’: {‘Bytes’: b’{“outputs”: ‘}}
{‘PayloadPart’: {‘Bytes’: b’[” problem”]}
‘}}
This class accounts for this by concatenating bytes written via the ‘write’ function
and then exposing a method which will return lines (ending with a ‘
‘ character)within the buffer via the ‘scan_lines’ function.
It maintains the position of the last read position to ensure
that previous bytes are not exposed again.
For more details see:
https://aws.amazon.com/blogs/machine-learning/elevating-the-generative-ai-experience-introducing-streaming-support-in-amazon-sagemaker-hosting/
Methods
__init__(stream)
Parameters
stream (Any) – 
__init__(stream: Any) → None[source]¶
Parameters
stream (Any) – 
Return type
None
langchain_aws.llms.sagemaker_endpoint.SagemakerEndpoint¶
class langchain_aws.llms.sagemaker_endpoint.SagemakerEndpoint[source]¶
Bases: LLM
Sagemaker Inference Endpoint models.
To use, you must supply the endpoint name from your deployed
Sagemaker model & the region where it is deployed.
To authenticate, the AWS client uses the following methods to
automatically load credentials:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
If a specific credential profile should be used, you must pass
the name of the profile from the ~/.aws/credentials file that is to be used.
Make sure the credentials / roles used have the required policies to
access the Sagemaker endpoint.
See: https://docs.aws.amazon.com/IAM/latest/UserGuide/access_policies.html
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param client: Any = None¶
Boto3 client for sagemaker runtime
param content_handler: LLMContentHandler [Required]¶
The content handler class that provides an input and
output transform functions to handle formats between LLM
and the endpoint.
output transform functions to handle formats between LLM
and the endpoint.
param credentials_profile_name: Optional[str] = None¶
The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which
has either access keys or role information specified.
If not specified, the default credential profile or, if on an EC2 instance,
credentials from IMDS will be used.
See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param endpoint_kwargs: Optional[Dict] = None¶
Optional attributes passed to the invoke_endpoint
function. See `boto3`_. docs for more info.
.. _boto3: <https://boto3.amazonaws.com/v1/documentation/api/latest/index.html>
param endpoint_name: str = ''¶
The name of the endpoint from the deployed Sagemaker model.
Must be unique within an AWS Region.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_kwargs: Optional[Dict] = None¶
Keyword arguments to pass to the model.
param region_name: str = ''¶
The aws region where the Sagemaker model is deployed, eg. us-west-2.
param streaming: bool = False¶
Whether to stream the results.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param verbose: bool [Optional]¶
Whether to print out response text.
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using SagemakerEndpoint¶
SageMakerEndpoint
langchain_community.llms.anyscale.Anyscale¶
class langchain_community.llms.anyscale.Anyscale[source]¶
Bases: BaseOpenAI
Anyscale large language models.
To use, you should have the environment variable ANYSCALE_API_KEY``set with your
Anyscale Endpoint, or pass it as a named parameter to the constructor.
To use with Anyscale Private Endpoint, please also set ``ANYSCALE_BASE_URL.
Example
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param allowed_special: Union[Literal['all'], AbstractSet[str]] = {}¶
Set of special tokens that are allowed。
param anyscale_api_base: str = 'https://api.endpoints.anyscale.com/v1'¶
param anyscale_api_key: SecretStr = None¶
Constraints
type = string
writeOnly = True
format = password
param batch_size: int = 20¶
Batch size to use when passing multiple documents to generate.
param best_of: int = 1¶
Generates best_of completions server-side and returns the “best”.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
Optional encoder to use for counting tokens.
param default_headers: Union[Mapping[str, str], None] = None¶
param default_query: Union[Mapping[str, object], None] = None¶
param disallowed_special: Union[Literal['all'], Collection[str]] = 'all'¶
Set of special tokens that are not allowed。
param frequency_penalty: float = 0¶
Penalizes repeated tokens according to frequency.
param http_client: Union[Any, None] = None¶
Optional httpx.Client.
param logit_bias: Optional[Dict[str, float]] [Optional]¶
Adjust the probability of specific tokens being generated.
param max_retries: int = 2¶
Maximum number of retries to make when generating.
param max_tokens: int = 256¶
The maximum number of tokens to generate in the completion.
-1 returns as many tokens as possible given the prompt and
the models maximal context size.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_kwargs: Dict[str, Any] [Optional]¶
Holds any model parameters valid for create call not explicitly specified.
param model_name: str = 'mistralai/Mixtral-8x7B-Instruct-v0.1'¶
Model name to use.
param n: int = 1¶
How many completions to generate for each prompt.
param openai_api_base: Optional[str] = None (alias 'base_url')¶
Base URL path for API requests, leave blank if not using a proxy or service
emulator.
param openai_api_key: Optional[str] = None (alias 'api_key')¶
Automatically inferred from env var OPENAI_API_KEY if not provided.
param openai_organization: Optional[str] = None (alias 'organization')¶
param openai_organization: Optional[str] = None (alias 'organization')¶
Automatically inferred from env var OPENAI_ORG_ID if not provided.
param openai_proxy: Optional[str] = None¶
param prefix_messages: List [Optional]¶
param presence_penalty: float = 0¶
Penalizes repeated tokens.
param request_timeout: Union[float, Tuple[float, float], Any, None] = None (alias 'timeout')¶
Timeout for requests to OpenAI completion API. Can be float, httpx.Timeout or
None.
param streaming: bool = False¶
Whether to stream the results or not.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: float = 0.7¶
What sampling temperature to use.
param tiktoken_model_name: Optional[str] = None¶
The model name to pass to tiktoken when using this class.
Tiktoken is used to count the number of tokens in documents to constrain
them to be under a certain limit. By default, when set to None, this will
be the same as the embedding model name. However, there are some cases
where you may want to use this Embedding class with a model name not
supported by tiktoken. This can include when using Azure embeddings or
when using one of the many model providers that expose an OpenAI-like
API but with different models. In those cases, in order to avoid erroring
when tiktoken is called, you can specify a model name to use here.
param top_p: float = 1¶
Total probability mass of tokens to consider at each step.
param verbose: bool [Optional]¶
Whether to print out response text.
langchain_community.llms.openai.acompletion_with_retry¶
async langchain_community.llms.openai.acompletion_with_retry(llm: Union[BaseOpenAI, OpenAIChat], run_manager: Optional[AsyncCallbackManagerForLLMRun] = None, **kwargs: Any) → Any[source]¶
Use tenacity to retry the async completion call.
Parameters
llm (Union[BaseOpenAI, OpenAIChat]) – 
run_manager (Optional[AsyncCallbackManagerForLLMRun]) – 
kwargs (Any) – 
Return type
Any
langchain_community.llms.baichuan.BaichuanLLM¶
class langchain_community.llms.baichuan.BaichuanLLM[source]¶
Bases: LLM
Baichuan large language models.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param baichuan_api_host: Optional[str] = None¶
param baichuan_api_key: Optional[SecretStr] = None¶
Constraints
type = string
writeOnly = True
format = password
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model: str = 'Baichuan2-Turbo-192k'¶
Other models are available at https://platform.baichuan-ai.com/docs/api.
param model_kwargs: Dict[str, Any] [Optional]¶
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: float = 0.3¶
param timeout: int = 60¶
param temperature: float = 0.3¶
param timeout: int = 60¶
param top_p: float = 0.95¶
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]]) –
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using BaichuanLLM¶
Baichuan
Baichuan LLM
langchain_community.llms.bedrock.Bedrock¶
class langchain_community.llms.bedrock.Bedrock[source]¶
Bases: LLM, BedrockBase
[Deprecated] Bedrock models.
To authenticate, the AWS client uses the following methods to
automatically load credentials:
https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
If a specific credential profile should be used, you must pass
the name of the profile from the ~/.aws/credentials file that is to be used.
Make sure the credentials / roles used have the required policies to
access the Bedrock service.
Notes
Deprecated since version 0.0.34.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param config: Any = None¶
An optional botocore.config.Config instance to pass to the client.
param credentials_profile_name: Optional[str] = None¶
The name of the profile in the ~/.aws/credentials or ~/.aws/config files, which
has either access keys or role information specified.
If not specified, the default credential profile or, if on an EC2 instance,
credentials from IMDS will be used.
credentials from IMDS will be used.
See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param endpoint_url: Optional[str] = None¶
Needed if you don’t want to default to us-east-1 endpoint
param guardrails: Optional[Mapping[str, Any]] = {'id': None, 'trace': False, 'version': None}¶
An optional dictionary to configure guardrails for Bedrock.
This field ‘guardrails’ consists of two keys: ‘id’ and ‘version’,
which should be strings, but are initialized to None. It’s used to
determine if specific guardrails are enabled and properly set.
Type:Optional[Mapping[str, str]]: A mapping with ‘id’ and ‘version’ keys.
Example:
llm = Bedrock(model_id=”<model_id>”, client=<bedrock_client>,
model_kwargs={},
guardrails={
“id”: “<guardrail_id>”,
“version”: “<guardrail_version>”})
To enable tracing for guardrails, set the ‘trace’ key to True and pass a callback handler to the
‘run_manager’ parameter of the ‘generate’, ‘_call’ methods.
Example:
llm = Bedrock(model_id=”<model_id>”, client=<bedrock_client>,
model_kwargs={},
guardrails={
“id”: “<guardrail_id>”,
“version”: “<guardrail_version>”,
“trace”: True},
callbacks=[BedrockAsyncCallbackHandler()])
[https://python.langchain.com/docs/modules/callbacks/] for more information on callback handlers.
class BedrockAsyncCallbackHandler(AsyncCallbackHandler):
async def on_llm_error(self,
async def on_llm_error(self,
error: BaseException,
**kwargs: Any,
) -> Any:reason = kwargs.get(“reason”)
if reason == “GUARDRAIL_INTERVENED”:
…Logic to handle guardrail intervention…
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_id: str [Required]¶
Id of the model to call, e.g., amazon.titan-text-express-v1, this is
equivalent to the modelId property in the list-foundation-models api. For custom and
provisioned models, an ARN value is expected.
param model_kwargs: Optional[Dict] = None¶
Keyword arguments to pass to the model.
param provider: Optional[str] = None¶
The model provider, e.g., amazon, cohere, ai21, etc. When not supplied, provider
is extracted from the first part of the model_id e.g. ‘amazon’ in
‘amazon.titan-text-express-v1’. This value should be provided for model ids that do
not have the provider in them, e.g., custom and provisioned models that have an ARN
associated with them.
param provider_stop_sequence_key_name_map: Mapping[str, str] = {'ai21': 'stop_sequences', 'amazon': 'stopSequences', 'anthropic': 'stop_sequences', 'cohere': 'stop_sequences', 'mistral': 'stop'}¶
param region_name: Optional[str] = None¶
The aws region e.g., us-west-2. Fallsback to AWS_DEFAULT_REGION env variable
or region specified in ~/.aws/config in case it is not provided here.
param streaming: bool = False¶
Whether to stream the results.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]]) –
Returns
The integer number of tokens in the text.
Return type
int
get_num_tokens_from_messages(messages: List[BaseMessage]) → int¶
Get the number of tokens in the messages.
Useful for checking if an input will fit in a model’s context window.
Parameters
messages (List[BaseMessage]) – The message inputs to tokenize.
Returns
The sum of the number of tokens across the messages.
Return type
int
get_output_schema(config: Optional[RunnableConfig] = None) → Type[BaseModel]¶
Get a pydantic model that can be used to validate output to the runnable.
Runnables that leverage the configurable_fields and configurable_alternatives
methods will have a dynamic output schema that depends on which
configuration the runnable is invoked with.
This method allows to get an output schema for a specific configuration.
Parameters
config (Optional[RunnableConfig]) – A config to use when generating the schema.
Returns
A pydantic model that can be used to validate output.
Return type
Type[BaseModel]
get_prompts(config: Optional[RunnableConfig] = None) → List[BasePromptTemplate]¶
Parameters
config (Optional[RunnableConfig]) – 
Return type
List[BasePromptTemplate]
get_token_ids(text: str) → List[int][source]¶
Return the ordered ids of the tokens in a text.
Parameters
text (str) – The string input to tokenize.
Returns
A list of ids corresponding to the tokens in the text, in order they occurin the text.
Return type
List[int]
Return type
List[int]
invoke(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → str¶
Transform a single input into an output. Override to implement.
Parameters
input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) – The input to the runnable.
config (Optional[RunnableConfig]) – A config to use when invoking the runnable.
The config supports standard keys like ‘tags’, ‘metadata’ for tracing
purposes, ‘max_concurrency’ for controlling how much work to do
in parallel, and other keys. Please refer to the RunnableConfig
for more details.
stop (Optional[List[str]]) – 
kwargs (Any) – 
Returns
The output of the runnable.
Return type
str
classmethod is_lc_serializable() → bool[source]¶
Return whether this model can be serialized by Langchain.
Return type
bool
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict[str, Any]¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using Bedrock¶
Bedrock
Bedrock (Knowledge Bases)