viruPVS/langchain-docs huggingface dataset:
Call out to DashScope’s embedding endpoint for embedding query text.
Parameters
text – The text to embed.
Returns
Embedding for the text.
classmethod from_orm(obj: Any) → Model¶
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod parse_obj(obj: Any) → Model¶
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
classmethod validate(value: Any) → Model¶
langchain_community.embeddings.google_palm.GooglePalmEmbeddings¶
class langchain_community.embeddings.google_palm.GooglePalmEmbeddings[source]¶
Bases: BaseModel, Embeddings
Google’s PaLM Embeddings APIs.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param client: Any = None¶
param google_api_key: Optional[str] = None¶
param model_name: str = 'models/embedding-gecko-001'¶
Model name to use.
param show_progress_bar: bool = False¶
Whether to show a tqdm progress bar. Must have tqdm installed.
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep – set to True to make a deep copy of the model
Returns
new model instance
dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny¶
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
embed_documents(texts: List[str]) → List[List[float]][source]¶
Embed search docs.
embed_query(text: str) → List[float][source]¶
Embed query text.
classmethod from_orm(obj: Any) → Model¶
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod parse_obj(obj: Any) → Model¶
langchain.embeddings.cache.CacheBackedEmbeddings¶
class langchain.embeddings.cache.CacheBackedEmbeddings(underlying_embeddings: Embeddings, document_embedding_store: BaseStore[str, List[float]])[source]¶
Interface for caching results from embedding models.
The interface allows works with any store that implements
the abstract store interface accepting keys of type str and values of list of
floats.
If need be, the interface can be extended to accept other implementations
of the value serializer and deserializer, as well as the key encoder.
Examples
Initialize the embedder.
Parameters
underlying_embeddings – the embedder to use for computing embeddings.
document_embedding_store – The store to use for caching document embeddings.
Methods
__init__(underlying_embeddings, ...)
Initialize the embedder.
aembed_documents(texts)
Asynchronous Embed search docs.
aembed_query(text)
Asynchronous Embed query text.
embed_documents(texts)
Embed a list of texts.
embed_query(text)
Embed query text.
from_bytes_store(underlying_embeddings, ...)
On-ramp that adds the necessary serialization and encoding to the store.
__init__(underlying_embeddings: Embeddings, document_embedding_store: BaseStore[str, List[float]]) → None[source]¶
Initialize the embedder.
Parameters
underlying_embeddings – the embedder to use for computing embeddings.
document_embedding_store – The store to use for caching document embeddings.
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
embed_documents(texts: List[str]) → List[List[float]][source]¶
Embed a list of texts.
The method first checks the cache for the embeddings.
Embed a list of texts.
The method first checks the cache for the embeddings.
If the embeddings are not found, the method uses the underlying embedder
to embed the documents and stores the results in the cache.
Parameters
texts – A list of texts to embed.
Returns
A list of embeddings for the given texts.
embed_query(text: str) → List[float][source]¶
Embed query text.
This method does not support caching at the moment.
Support for caching queries is easily to implement, but might make
sense to hold off to see the most common patterns.
If the cache has an eviction policy, we may need to be a bit more careful
about sharing the cache between documents and queries. Generally,
one is OK evicting query caches, but document caches should be kept.
Parameters
text – The text to embed.
Returns
The embedding for the given text.
classmethod from_bytes_store(underlying_embeddings: Embeddings, document_embedding_cache: BaseStore[str, bytes], *, namespace: str = '') → CacheBackedEmbeddings[source]¶
On-ramp that adds the necessary serialization and encoding to the store.
Parameters
underlying_embeddings – The embedder to use for embedding.
document_embedding_cache – The cache to use for storing document embeddings.
* – 
:param :
:param namespace: The namespace to use for document cache.
This namespace is used to avoid collisions with other caches.
For example, set it to the name of the embedding model used.
Examples using CacheBackedEmbeddings¶
Caching
langchain_community.embeddings.aleph_alpha.AlephAlphaAsymmetricSemanticEmbedding¶
class langchain_community.embeddings.aleph_alpha.AlephAlphaAsymmetricSemanticEmbedding[source]¶
Bases: BaseModel, Embeddings
Aleph Alpha’s asymmetric semantic embedding.
AA provides you with an endpoint to embed a document and a query.
The models were optimized to make the embeddings of documents and
the query for a document as similar as possible.
To learn more, check out: https://docs.aleph-alpha.com/docs/tasks/semantic_embed/
Example
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param aleph_alpha_api_key: Optional[str] = None¶
API key for Aleph Alpha API.
param compress_to_size: Optional[int] = None¶
Should the returned embeddings come back as an original 5120-dim vector,
or should it be compressed to 128-dim.
param contextual_control_threshold: Optional[int] = None¶
Attention control parameters only apply to those tokens that have
explicitly been set in the request.
param control_log_additive: bool = True¶
Apply controls on prompt items by adding the log(control_factor)
to attention scores.
param host: str = 'https://api.aleph-alpha.com'¶
The hostname of the API host.
The default one is “https://api.aleph-alpha.com”)
param hosting: Optional[str] = None¶
Determines in which datacenters the request may be processed.
You can either set the parameter to “aleph-alpha” or omit it (defaulting to None).
Not setting this value, or setting it to None, gives us maximal flexibility
in processing your request in our
own datacenters and on servers hosted with other providers.
in processing your request in our
own datacenters and on servers hosted with other providers.
Choose this option for maximal availability.
Setting it to “aleph-alpha” allows us to only process the request
in our own datacenters.
Choose this option for maximal data privacy.
param model: str = 'luminous-base'¶
Model name to use.
param nice: bool = False¶
Setting this to True, will signal to the API that you intend to be
nice to other users
by de-prioritizing your request below concurrent ones.
param normalize: Optional[bool] = None¶
Should returned embeddings be normalized
param request_timeout_seconds: int = 305¶
Client timeout that will be set for HTTP requests in the
requests library’s API calls.
Server will close all requests after 300 seconds with an internal server error.
param total_retries: int = 8¶
The number of retries made in case requests fail with certain retryable
status codes. If the last
retry fails a corresponding exception is raised. Note, that between retries
an exponential backoff
is applied, starting with 0.5 s after the first retry and doubling for each
retry made. So with the
default setting of 8 retries a total wait time of 63.5 s is added between
the retries.
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep – set to True to make a deep copy of the model
Returns
new model instance
dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny¶
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
embed_documents(texts: List[str]) → List[List[float]][source]¶
Call out to Aleph Alpha’s asymmetric Document endpoint.
Parameters
texts – The list of texts to embed.
Returns
List of embeddings, one for each text.
embed_query(text: str) → List[float][source]¶
Call out to Aleph Alpha’s asymmetric, query embedding endpoint
:param text: The text to embed.
Returns
Embeddings for the text.
:param text: The text to embed.
Returns
Embeddings for the text.
classmethod from_orm(obj: Any) → Model¶
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod parse_obj(obj: Any) → Model¶
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
classmethod validate(value: Any) → Model¶
Examples using AlephAlphaAsymmetricSemanticEmbedding¶
Aleph Alpha
langchain_community.embeddings.sagemaker_endpoint.EmbeddingsContentHandler¶
class langchain_community.embeddings.sagemaker_endpoint.EmbeddingsContentHandler[source]¶
Content handler for LLM class.
Attributes
accepts
The MIME type of the response data returned from endpoint
content_type
The MIME type of the input data passed to endpoint
Methods
__init__()
transform_input(prompt, model_kwargs)
Transforms the input to a format that model can accept as the request Body.
transform_output(output)
Transforms the output from the model to string that the LLM class expects.
__init__()¶
abstract transform_input(prompt: INPUT_TYPE, model_kwargs: Dict) → bytes¶
Transforms the input to a format that model can accept
as the request Body. Should return bytes or seekable file
like object in the format specified in the content_type
request header.
abstract transform_output(output: bytes) → OUTPUT_TYPE¶
Transforms the output from the model to string that
the LLM class expects.
Examples using EmbeddingsContentHandler¶
SageMaker
langchain_community.embeddings.vertexai.VertexAIEmbeddings¶
class langchain_community.embeddings.vertexai.VertexAIEmbeddings[source]¶
Bases: _VertexAICommon, Embeddings
Google Cloud VertexAI embedding models.
Initialize the sentence_transformer.
param credentials: Any = None¶
The default custom credentials (google.auth.credentials.Credentials) to use
param location: str = 'us-central1'¶
The default location to use when making API calls.
param max_output_tokens: int = 128¶
Token limit determines the maximum amount of text output from one prompt.
param max_retries: int = 6¶
The maximum number of retries to make when generating.
param model_name: str [Required]¶
Underlying model name.
param n: int = 1¶
How many completions to generate for each prompt.
param project: Optional[str] = None¶
The default GCP project to use when making Vertex API calls.
param request_parallelism: int = 5¶
The amount of parallelism allowed for requests issued to VertexAI models.
param stop: Optional[List[str]] = None¶
Optional list of stop words to use when generating.
param streaming: bool = False¶
Whether to stream the results or not.
param temperature: float = 0.0¶
Sampling temperature, it controls the degree of randomness in token selection.
param top_k: int = 40¶
How the model selects tokens for output, the next token is selected from
param top_p: float = 0.95¶
Tokens are selected from most probable to least until the sum of their
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
Asynchronous Embed query text.
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep – set to True to make a deep copy of the model
Returns
new model instance
dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny¶
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
embed(texts: List[str], batch_size: int = 0, embeddings_task_type: Optional[Literal['RETRIEVAL_QUERY', 'RETRIEVAL_DOCUMENT', 'SEMANTIC_SIMILARITY', 'CLASSIFICATION', 'CLUSTERING']] = None) → List[List[float]][source]¶
Embed a list of strings.
Parameters
texts – List[str] The list of strings to embed.
batch_size – [int] The batch size of embeddings to send to the model.
If zero, then the largest batch size will be detected dynamically
at the first request, starting from 250, down to 5.
embeddings_task_type – [str] optional embeddings task type,
one of the following
RETRIEVAL_QUERY     - Text is a queryin a search/retrieval setting.
RETRIEVAL_DOCUMENT - Text is a documentin a search/retrieval setting.
SEMANTIC_SIMILARITY - Embeddings will be usedfor Semantic Textual Similarity (STS).
CLASSIFICATION - Embeddings will be used for classification.
CLUSTERING - Embeddings will be used for clustering.
Returns
List of embeddings, one for each text.
embed_documents(texts: List[str], batch_size: int = 0) → List[List[float]][source]¶
Embed a list of documents.
Parameters
texts – List[str] The list of texts to embed.
batch_size – [int] The batch size of embeddings to send to the model.
If zero, then the largest batch size will be detected dynamically
at the first request, starting from 250, down to 5.
Returns
List of embeddings, one for each text.
embed_query(text: str) → List[float][source]¶
Embed a text.
Parameters
Embed a text.
Parameters
text – The text to embed.
Returns
Embedding for the text.
classmethod from_orm(obj: Any) → Model¶
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod parse_obj(obj: Any) → Model¶
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
classmethod validate(value: Any) → Model¶
property is_codey_model: bool¶
langchain_community.embeddings.localai.LocalAIEmbeddings¶
class langchain_community.embeddings.localai.LocalAIEmbeddings[source]¶
Bases: BaseModel, Embeddings
LocalAI embedding models.
Since LocalAI and OpenAI have 1:1 compatibility between APIs, this class
uses the openai Python package’s openai.Embedding as its client.
Thus, you should have the openai python package installed, and defeat
the environment variable OPENAI_API_KEY by setting to a random string.
You also need to specify OPENAI_API_BASE to point to your LocalAI
service endpoint.
Example
from langchain_community.embeddings import LocalAIEmbeddings
openai = LocalAIEmbeddings(
    openai_api_key="random-string",
    openai_api_base="http://localhost:8080"
)
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param allowed_special: Union[Literal['all'], Set[str]] = {}¶
param chunk_size: int = 1000¶
Maximum number of texts to embed in each batch
param deployment: str = 'text-embedding-ada-002'¶
param disallowed_special: Union[Literal['all'], Set[str], Sequence[str]] = 'all'¶
param embedding_ctx_length: int = 8191¶
The maximum number of tokens to embed at once.
param headers: Any = None¶
param max_retries: int = 6¶
Maximum number of retries to make when generating.
param model: str = 'text-embedding-ada-002'¶
param model_kwargs: Dict[str, Any] [Optional]¶
Holds any model parameters valid for create call not explicitly specified.
param openai_api_base: Optional[str] = None¶