Tony20100/langchain_full huggingface dataset:
Whether to stream the results.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]]) –
Return type
List[int]
invoke(input: Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], config: Optional[RunnableConfig] = None, *, stop: Optional[List[str]] = None, **kwargs: Any) → str¶
Transform a single input into an output. Override to implement.
Parameters
input (Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]) – The input to the runnable.
config (Optional[RunnableConfig]) – A config to use when invoking the runnable.
The config supports standard keys like ‘tags’, ‘metadata’ for tracing
purposes, ‘max_concurrency’ for controlling how much work to do
in parallel, and other keys. Please refer to the RunnableConfig
for more details.
stop (Optional[List[str]]) – 
kwargs (Any) – 
Returns
The output of the runnable.
Return type
str
classmethod is_lc_serializable() → bool¶
Return whether this model can be serialized by Langchain.
Return type
bool
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
langchain_community.llms.oci_data_science_model_deployment_endpoint.OCIModelDeploymentVLLM¶
class langchain_community.llms.oci_data_science_model_deployment_endpoint.OCIModelDeploymentVLLM[source]¶
Bases: OCIModelDeploymentLLM
VLLM deployed on OCI Data Science Model Deployment
To use, you must provide the model HTTP endpoint from your deployed
model, e.g. https://<MD_OCID>/predict.
To authenticate, oracle-ads has been used to automatically load
credentials: https://accelerated-data-science.readthedocs.io/en/latest/user_guide/cli/authentication.html
Make sure to have the required policies to access the OCI Data
Science Model Deployment endpoint. See:
https://docs.oracle.com/en-us/iaas/data-science/using/model-dep-policies-auth.htm#model_dep_policies_auth__predict-endpoint
Example
from langchain_community.llms import OCIModelDeploymentVLLM
oci_md = OCIModelDeploymentVLLM(
    endpoint="https://<MD_OCID>/predict",
    model="mymodel"
)
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param auth: dict [Optional]¶
ADS auth dictionary for OCI authentication:
https://accelerated-data-science.readthedocs.io/en/latest/user_guide/cli/authentication.html.
This can be generated by calling ads.common.auth.api_keys()
or ads.common.auth.resource_principal(). If this is not
provided then the ads.common.default_signer() will be used.
param best_of: int = 1¶
Generates best_of completions server-side and returns the “best”
(the one with the highest log probability per token).
param cache: Union[BaseCache, bool, None] = None¶
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param endpoint: str = ''¶
The uri of the endpoint from the deployed Model Deployment model.
param frequency_penalty: float = 0.0¶
Penalizes repeated tokens according to frequency. Between 0 and 1.
param ignore_eos: bool = False¶
Whether to ignore the EOS token and continue generating tokens after
the EOS token is generated.
param k: int = -1¶
Number of most likely tokens to consider at each step.
param logprobs: Optional[int] = None¶
Number of log probabilities to return per output token.
param max_tokens: int = 256¶
Denotes the number of tokens to predict per generation.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model: str [Required]¶
The name of the model.
param n: int = 1¶
Number of output sequences to return for the given prompt.
param p: float = 0.75¶
Total probability mass of tokens to consider at each step.
param presence_penalty: float = 0.0¶
param presence_penalty: float = 0.0¶
Penalizes repeated tokens. Between 0 and 1.
param stop: Optional[List[str]] = None¶
Stop words to use when generating. Model output is cut off
at the first occurrence of any of these substrings.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: float = 0.2¶
A non-negative float that tunes the degree of randomness in generation.
param use_beam_search: bool = False¶
Whether to use beam search instead of sampling.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using OCIModelDeploymentVLLM¶
OCI Data Science Model Deployment Endpoint
Oracle Cloud Infrastructure (OCI)
langchain_openai.llms.azure.AzureOpenAI¶
class langchain_openai.llms.azure.AzureOpenAI[source]¶
Bases: BaseOpenAI
Azure-specific OpenAI large language models.
To use, you should have the openai python package installed, and the
environment variable OPENAI_API_KEY set with your API key.
Any parameters that are valid to be passed to the openai.create call can be passed
in, even if not explicitly saved on this class.
Example
from langchain_openai import AzureOpenAI
openai = AzureOpenAI(model_name="gpt-3.5-turbo-instruct")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param allowed_special: Union[Literal['all'], AbstractSet[str]] = {}¶
Set of special tokens that are allowed。
param azure_ad_token: Optional[SecretStr] = None¶
Your Azure Active Directory token.
Automatically inferred from env var AZURE_OPENAI_AD_TOKEN if not provided.
For more:
https://www.microsoft.com/en-us/security/business/identity-access/microsoft-entra-id.
Constraints
type = string
writeOnly = True
format = password
param azure_ad_token_provider: Union[Callable[[], str], None] = None¶
A function that returns an Azure Active Directory token.
Will be invoked on every request.
param azure_endpoint: Union[str, None] = None¶
Your Azure endpoint, including the resource.
Automatically inferred from env var AZURE_OPENAI_ENDPOINT if not provided.
Example: https://example-resource.azure.openai.com/
param batch_size: int = 20¶
Batch size to use when passing multiple documents to generate.
param best_of: int = 1¶
param best_of: int = 1¶
Generates best_of completions server-side and returns the “best”.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param default_headers: Union[Mapping[str, str], None] = None¶
param default_query: Union[Mapping[str, object], None] = None¶
param deployment_name: Union[str, None] = None (alias 'azure_deployment')¶
A model deployment.
If given sets the base client URL to include /deployments/{azure_deployment}.
Note: this means you won’t be able to use non-deployment endpoints.
param disallowed_special: Union[Literal['all'], Collection[str]] = 'all'¶
Set of special tokens that are not allowed。
param frequency_penalty: float = 0¶
Penalizes repeated tokens according to frequency.
param http_async_client: Union[Any, None] = None¶
Optional httpx.AsyncClient. Only used for async invocations. Must specify
http_client as well if you’d like a custom client for sync invocations.
param http_client: Union[Any, None] = None¶
Optional httpx.Client. Only used for sync invocations. Must specify
Optional httpx.Client. Only used for sync invocations. Must specify
http_async_client as well if you’d like a custom client for async invocations.
param logit_bias: Optional[Dict[str, float]] [Optional]¶
Adjust the probability of specific tokens being generated.
param max_retries: int = 2¶
Maximum number of retries to make when generating.
param max_tokens: int = 256¶
The maximum number of tokens to generate in the completion.
-1 returns as many tokens as possible given the prompt and
the models maximal context size.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_kwargs: Dict[str, Any] [Optional]¶
Holds any model parameters valid for create call not explicitly specified.
param model_name: str = 'gpt-3.5-turbo-instruct' (alias 'model')¶
Model name to use.
param n: int = 1¶
How many completions to generate for each prompt.
param openai_api_base: Optional[str] = None (alias 'base_url')¶
Base URL path for API requests, leave blank if not using a proxy or service
emulator.
param openai_api_key: Optional[SecretStr] = None (alias 'api_key')¶
Automatically inferred from env var AZURE_OPENAI_API_KEY if not provided.
Constraints
type = string
writeOnly = True
format = password
param openai_api_type: str = ''¶
Legacy, for openai<1.0.0 support.
param openai_api_version: str = '' (alias 'api_version')¶
Automatically inferred from env var OPENAI_API_VERSION if not provided.
param openai_organization: Optional[str] = None (alias 'organization')¶
param openai_organization: Optional[str] = None (alias 'organization')¶
Automatically inferred from env var OPENAI_ORG_ID if not provided.
param openai_proxy: Optional[str] = None¶
param presence_penalty: float = 0¶
Penalizes repeated tokens.
param request_timeout: Union[float, Tuple[float, float], Any, None] = None (alias 'timeout')¶
Timeout for requests to OpenAI completion API. Can be float, httpx.Timeout or
None.
param streaming: bool = False¶
Whether to stream the results or not.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param temperature: float = 0.7¶
What sampling temperature to use.
param tiktoken_model_name: Optional[str] = None¶
The model name to pass to tiktoken when using this class.
Tiktoken is used to count the number of tokens in documents to constrain
them to be under a certain limit. By default, when set to None, this will
be the same as the embedding model name. However, there are some cases
where you may want to use this Embedding class with a model name not
supported by tiktoken. This can include when using Azure embeddings or
when using one of the many model providers that expose an OpenAI-like
API but with different models. In those cases, in order to avoid erroring
when tiktoken is called, you can specify a model name to use here.
param top_p: float = 1¶
Total probability mass of tokens to consider at each step.
param validate_base_url: bool = True¶
For backwards compatibility. If legacy val openai_api_base is passed in, try to
infer if it is a base_url or azure_endpoint and update accordingly.
param verbose: bool [Optional]¶
langchain_community.llms.sagemaker_endpoint.LineIterator¶
class langchain_community.llms.sagemaker_endpoint.LineIterator(stream: Any)[source]¶
Parse the byte stream input.
The output of the model will be in the following format:
b’{“outputs”: [” a”]}
‘b’{“outputs”: [” challenging”]}
‘b’{“outputs”: [” problem”]}
‘…
While usually each PayloadPart event from the event stream will
contain a byte array with a full json, this is not guaranteed
and some of the json objects may be split acrossPayloadPart events.
For example:
{‘PayloadPart’: {‘Bytes’: b’{“outputs”: ‘}}
{‘PayloadPart’: {‘Bytes’: b’[” problem”]}
‘}}
This class accounts for this by concatenating bytes written via the ‘write’ function
and then exposing a method which will return lines (ending with a ‘
‘ character)within the buffer via the ‘scan_lines’ function.
It maintains the position of the last read position to ensure
that previous bytes are not exposed again.
For more details see:
https://aws.amazon.com/blogs/machine-learning/elevating-the-generative-ai-experience-introducing-streaming-support-in-amazon-sagemaker-hosting/
Methods
__init__(stream)
Parameters
stream (Any) – 
__init__(stream: Any) → None[source]¶
Parameters
stream (Any) – 
Return type
None
langchain_community.llms.ipex_llm.IpexLLM¶
class langchain_community.llms.ipex_llm.IpexLLM[source]¶
Bases: LLM
IpexLLM model.
Example
from langchain_community.llms import IpexLLM
llm = IpexLLM.from_model_id(model_id="THUDM/chatglm-6b")
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model: Any = None¶
IpexLLM model.
param model_id: str = 'gpt2'¶
Model name or model path to use.
param model_kwargs: Optional[dict] = None¶
Keyword arguments passed to the model.
param streaming: bool = True¶
Whether to stream the results, token by token.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param tokenizer: Any = None¶
Tags to add to the run trace.
param tokenizer: Any = None¶
Huggingface tokenizer model.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]]) –
values (Any) – 
Return type
Model
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs: Any) → Dict¶
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
classmethod from_model_id(model_id: str, model_kwargs: Optional[dict] = None, *, tokenizer_id: Optional[str] = None, load_in_4bit: bool = True, load_in_low_bit: Optional[str] = None, **kwargs: Any) → LLM[source]¶
Construct object from model_id
Parameters
model_id (str) – Path for the huggingface repo id to be downloaded or
the huggingface checkpoint folder.
tokenizer_id (Optional[str]) – Path for the huggingface repo id to be downloaded or
the huggingface checkpoint folder which contains the tokenizer.
the huggingface checkpoint folder which contains the tokenizer.
load_in_4bit (bool) – “Whether to load model in 4bit.
Unused if load_in_low_bit is not None.
load_in_low_bit (Optional[str]) – Which low bit precisions to use when loading model.
Example values: ‘sym_int4’, ‘asym_int4’, ‘fp4’, ‘nf4’, ‘fp8’, etc.
Overrides load_in_4bit if specified.
model_kwargs (Optional[dict]) – Keyword arguments to pass to the model and tokenizer.
kwargs (Any) – Extra arguments to pass to the model and tokenizer.
Returns
An object of IpexLLM.
Return type
LLM
classmethod from_model_id_low_bit(model_id: str, model_kwargs: Optional[dict] = None, *, tokenizer_id: Optional[str] = None, **kwargs: Any) → LLM[source]¶
Construct low_bit object from model_id
Parameters
model_id (str) – Path for the ipex-llm transformers low-bit model folder.
tokenizer_id (Optional[str]) – Path for the huggingface repo id or local model folder
which contains the tokenizer.
model_kwargs (Optional[dict]) – Keyword arguments to pass to the model and tokenizer.
kwargs (Any) – Extra arguments to pass to the model and tokenizer.
Returns
An object of IpexLLM.
Return type
LLM
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using IpexLLM¶
IPEX-LLM
langchain_community.llms.openai.update_token_usage¶
langchain_community.llms.openai.update_token_usage(keys: Set[str], response: Dict[str, Any], token_usage: Dict[str, Any]) → None[source]¶
Update token usage.
Parameters
keys (Set[str]) – 
response (Dict[str, Any]) – 
token_usage (Dict[str, Any]) – 
Return type
None