Tony20100/langchain_full huggingface dataset:
langchain_community.embeddings.fastembed.FastEmbedEmbeddings¶
class langchain_community.embeddings.fastembed.FastEmbedEmbeddings[source]¶
Bases: BaseModel, Embeddings
Qdrant FastEmbedding models.
FastEmbed is a lightweight, fast, Python library built for embedding generation.
See more documentation at:
* https://github.com/qdrant/fastembed/
* https://qdrant.github.io/fastembed/
To use this class, you must install the fastembed Python package.
pip install fastembed
.. rubric:: Example
from langchain_community.embeddings import FastEmbedEmbeddings
fastembed = FastEmbedEmbeddings()
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param cache_dir: Optional[str] = None¶
The path to the cache directory.
Defaults to local_cache in the parent directory
param doc_embed_type: Literal['default', 'passage'] = 'default'¶
Type of embedding to use for documents
The available options are: “default” and “passage”
param max_length: int = 512¶
The maximum number of tokens. Defaults to 512.
Unknown behavior for values > 512.
param model_name: str = 'BAAI/bge-small-en-v1.5'¶
Name of the FastEmbedding model to use
Defaults to “BAAI/bge-small-en-v1.5”
Find the list of supported models at
https://qdrant.github.io/fastembed/examples/Supported_Models/
param threads: Optional[int] = None¶
The number of threads single onnxruntime session can use.
Defaults to None
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
Asynchronous Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[List[float]]
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
Parameters
text (str) – 
Return type
List[float]
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
self (Model) – 
Returns
new model instance
Return type
Model
dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny¶
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
Return type
DictStrAny
embed_documents(texts: List[str]) → List[List[float]][source]¶
Generate embeddings for documents using FastEmbed.
Parameters
texts (List[str]) – The list of texts to embed.
Returns
List of embeddings, one for each text.
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Generate query embeddings using FastEmbed.
Parameters
text (str) – The text to embed.
Returns
Embeddings for the text.
Return type
List[float]
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
Parameters
obj (Any) – 
Return type
Model
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
Parameters
b (Union[str, bytes]) – 
content_type (unicode) – 
encoding (unicode) – 
proto (Protocol) – 
allow_pickle (bool) – 
Return type
Model
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
Return type
DictStrAny
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
dumps_kwargs (Any) – 
Return type
unicode
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
classmethod validate(value: Any) → Model¶
Parameters
value (Any) – 
Return type
Model
Examples using FastEmbedEmbeddings¶
FastEmbed by Qdrant
langchain_community.embeddings.infinity.InfinityEmbeddings¶
class langchain_community.embeddings.infinity.InfinityEmbeddings[source]¶
Bases: BaseModel, Embeddings
Self-hosted embedding models for infinity package.
See https://github.com/michaelfeil/infinity
This also works for text-embeddings-inference and other
self-hosted openai-compatible servers.
Infinity is a package to interact with Embedding Models on https://github.com/michaelfeil/infinity
Example
from langchain_community.embeddings import InfinityEmbeddings
InfinityEmbeddings(
    model="BAAI/bge-small",
    infinity_api_url="http://localhost:7997",
)
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param client: Any = None¶
Infinity client.
param infinity_api_url: str = 'http://localhost:7997'¶
Endpoint URL to use.
param model: str [Required]¶
Underlying Infinity model id.
async aembed_documents(texts: List[str]) → List[List[float]][source]¶
Async call out to Infinity’s embedding endpoint.
Parameters
texts (List[str]) – The list of texts to embed.
Returns
List of embeddings, one for each text.
Return type
List[List[float]]
async aembed_query(text: str) → List[float][source]¶
Async call out to Infinity’s embedding endpoint.
Parameters
text (str) – The text to embed.
Returns
Embeddings for the text.
Return type
List[float]
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
Return type
DictStrAny
embed_documents(texts: List[str]) → List[List[float]][source]¶
Call out to Infinity’s embedding endpoint.
Parameters
texts (List[str]) – The list of texts to embed.
Returns
List of embeddings, one for each text.
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Call out to Infinity’s embedding endpoint.
Parameters
text (str) – The text to embed.
Returns
Embeddings for the text.
Return type
List[float]
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) –
ref_template (unicode) – 
Return type
DictStrAny
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
dumps_kwargs (Any) – 
Return type
unicode
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
classmethod validate(value: Any) → Model¶
Parameters
value (Any) – 
Return type
Model
Examples using InfinityEmbeddings¶
Infinity
langchain_community.embeddings.clarifai.ClarifaiEmbeddings¶
class langchain_community.embeddings.clarifai.ClarifaiEmbeddings[source]¶
Bases: BaseModel, Embeddings
Clarifai embedding models.
To use, you should have the clarifai python package installed, and the
environment variable CLARIFAI_PAT set with your personal access token or pass it
as a named parameter to the constructor.
Example
from langchain_community.embeddings import ClarifaiEmbeddings
clarifai = ClarifaiEmbeddings(user_id=USER_ID,
                              app_id=APP_ID,
                              model_id=MODEL_ID)
                 (or)
Example_URL = "https://clarifai.com/clarifai/main/models/BAAI-bge-base-en-v15"
clarifai = ClarifaiEmbeddings(model_url=EXAMPLE_URL)
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param api_base: str = 'https://api.clarifai.com'¶
param app_id: Optional[str] = None¶
Clarifai application id to use.
param model_id: Optional[str] = None¶
Model id to use.
param model_url: Optional[str] = None¶
Model url to use.
param model_version_id: Optional[str] = None¶
Model version id to use.
param pat: Optional[str] = None¶
Clarifai personal access token to use.
param token: Optional[str] = None¶
Clarifai session token to use.
param user_id: Optional[str] = None¶
Clarifai user id to use.
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
self (Model) – 
Returns
new model instance
Return type
Model
dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny¶
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
Return type
DictStrAny
embed_documents(texts: List[str]) → List[List[float]][source]¶
Call out to Clarifai’s embedding models.
Parameters
texts (List[str]) – The list of texts to embed.
Returns
List of embeddings, one for each text.
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Call out to Clarifai’s embedding models.
Parameters
text (str) – The text to embed.
Returns
Embeddings for the text.
Return type
List[float]
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
Parameters
obj (Any) – 
Return type
Model
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
Parameters
b (Union[str, bytes]) – 
content_type (unicode) – 
encoding (unicode) – 
proto (Protocol) – 
allow_pickle (bool) – 
Return type
Model
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
Return type
DictStrAny
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
dumps_kwargs (Any) – 
Return type
unicode
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
classmethod validate(value: Any) → Model¶
Parameters
value (Any) – 
Return type
Model
Examples using ClarifaiEmbeddings¶
Clarifai
param openai_api_base: Optional[str] = None¶
param openai_api_key: Optional[str] = None¶
param openai_api_version: Optional[str] = None¶
param openai_organization: Optional[str] = None¶
param openai_proxy: Optional[str] = None¶
param request_timeout: Optional[Union[float, Tuple[float, float]]] = None¶
Timeout in seconds for the LocalAI request.
param show_progress_bar: bool = False¶
Whether to show a progress bar when embedding.
async aembed_documents(texts: List[str], chunk_size: Optional[int] = 0) → List[List[float]][source]¶
Call out to LocalAI’s embedding endpoint async for embedding search docs.
Parameters
texts (List[str]) – The list of texts to embed.
chunk_size (Optional[int]) – The chunk size of embeddings. If None, will use the chunk size
specified by the class.
Returns
List of embeddings, one for each text.
Return type
List[List[float]]
async aembed_query(text: str) → List[float][source]¶
Call out to LocalAI’s embedding endpoint async for embedding query text.
Parameters
text (str) – The text to embed.
Returns
Embedding for the text.
Return type
List[float]
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
Return type
DictStrAny
embed_documents(texts: List[str], chunk_size: Optional[int] = 0) → List[List[float]][source]¶
Call out to LocalAI’s embedding endpoint for embedding search docs.
Parameters
texts (List[str]) – The list of texts to embed.
chunk_size (Optional[int]) – The chunk size of embeddings. If None, will use the chunk size
specified by the class.
Returns
List of embeddings, one for each text.
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Call out to LocalAI’s embedding endpoint for embedding query text.
Parameters
text (str) – The text to embed.
Returns
Embedding for the text.
Return type
List[float]
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) –
ref_template (unicode) – 
Return type
DictStrAny
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
dumps_kwargs (Any) – 
Return type
unicode
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
classmethod validate(value: Any) → Model¶
Parameters
value (Any) – 
Return type
Model
Examples using LocalAIEmbeddings¶
LocalAI
langchain_community.embeddings.openai.OpenAIEmbeddings¶
class langchain_community.embeddings.openai.OpenAIEmbeddings[source]¶
Bases: BaseModel, Embeddings
[Deprecated] OpenAI embedding models.
To use, you should have the openai python package installed, and the
environment variable OPENAI_API_KEY set with your API key or pass it
as a named parameter to the constructor.
Example
from langchain_community.embeddings import OpenAIEmbeddings
openai = OpenAIEmbeddings(openai_api_key="my-api-key")
In order to use the library with Microsoft Azure endpoints, you need to set
the OPENAI_API_TYPE, OPENAI_API_BASE, OPENAI_API_KEY and OPENAI_API_VERSION.
The OPENAI_API_TYPE must be set to ‘azure’ and the others correspond to
the properties of your endpoint.
In addition, the deployment name must be passed as the model parameter.
Example
import os
os.environ["OPENAI_API_TYPE"] = "azure"
os.environ["OPENAI_API_BASE"] = "https://<your-endpoint.openai.azure.com/"
os.environ["OPENAI_API_KEY"] = "your AzureOpenAI key"
os.environ["OPENAI_API_VERSION"] = "2023-05-15"
os.environ["OPENAI_PROXY"] = "http://your-corporate-proxy:8080"
from langchain_community.embeddings.openai import OpenAIEmbeddings
embeddings = OpenAIEmbeddings(
    deployment="your-embeddings-deployment-name",
    model="your-embeddings-model-name",
    openai_api_base="https://your-endpoint.openai.azure.com/",
    openai_api_type="azure",
)
text = "This is a test query."
query_result = embeddings.embed_query(text)
Notes
Deprecated since version 0.0.9.
Notes
Deprecated since version 0.0.9.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param allowed_special: Union[Literal['all'], Set[str]] = {}¶
param chunk_size: int = 1000¶
Maximum number of texts to embed in each batch
param default_headers: Optional[Mapping[str, str]] = None¶
param default_query: Optional[Mapping[str, object]] = None¶
param deployment: Optional[str] = 'text-embedding-ada-002'¶
param disallowed_special: Union[Literal['all'], Set[str], Sequence[str]] = 'all'¶
param embedding_ctx_length: int = 8191¶
The maximum number of tokens to embed at once.
param headers: Any = None¶
param http_client: Optional[Any] = None¶
Optional httpx.Client.
param max_retries: int = 2¶
Maximum number of retries to make when generating.
param model: str = 'text-embedding-ada-002'¶
param model_kwargs: Dict[str, Any] [Optional]¶
Holds any model parameters valid for create call not explicitly specified.
param openai_api_base: Optional[str] = None (alias 'base_url')¶
Base URL path for API requests, leave blank if not using a proxy or service
emulator.
param openai_api_key: Optional[str] = None (alias 'api_key')¶
Automatically inferred from env var OPENAI_API_KEY if not provided.
param openai_api_type: Optional[str] = None¶
param openai_api_version: Optional[str] = None (alias 'api_version')¶
Automatically inferred from env var OPENAI_API_VERSION if not provided.
Automatically inferred from env var OPENAI_API_VERSION if not provided.
param openai_organization: Optional[str] = None (alias 'organization')¶
Automatically inferred from env var OPENAI_ORG_ID if not provided.
param openai_proxy: Optional[str] = None¶
param request_timeout: Optional[Union[float, Tuple[float, float], Any]] = None (alias 'timeout')¶
Timeout for requests to OpenAI completion API. Can be float, httpx.Timeout or
None.
param retry_max_seconds: int = 20¶
Max number of seconds to wait between retries
param retry_min_seconds: int = 4¶
Min number of seconds to wait between retries
param show_progress_bar: bool = False¶
Whether to show a progress bar when embedding.
param skip_empty: bool = False¶
Whether to skip empty strings when embedding or raise an error.
Defaults to not skipping.
param tiktoken_enabled: bool = True¶
Set this to False for non-OpenAI implementations of the embeddings API, e.g.
the –extensions openai extension for text-generation-webui
param tiktoken_model_name: Optional[str] = None¶
The model name to pass to tiktoken when using this class.
Tiktoken is used to count the number of tokens in documents to constrain
them to be under a certain limit. By default, when set to None, this will
be the same as the embedding model name. However, there are some cases
where you may want to use this Embedding class with a model name not
supported by tiktoken. This can include when using Azure embeddings or
when using one of the many model providers that expose an OpenAI-like
API but with different models. In those cases, in order to avoid erroring
when tiktoken is called, you can specify a model name to use here.
when tiktoken is called, you can specify a model name to use here.
async aembed_documents(texts: List[str], chunk_size: Optional[int] = 0) → List[List[float]][source]¶
Call out to OpenAI’s embedding endpoint async for embedding search docs.
Parameters
texts (List[str]) – The list of texts to embed.
chunk_size (Optional[int]) – The chunk size of embeddings. If None, will use the chunk size
specified by the class.
Returns
List of embeddings, one for each text.
Return type
List[List[float]]
async aembed_query(text: str) → List[float][source]¶
Call out to OpenAI’s embedding endpoint async for embedding query text.
Parameters
text (str) – The text to embed.
Returns
Embedding for the text.
Return type
List[float]
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny¶
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
Return type
DictStrAny
embed_documents(texts: List[str], chunk_size: Optional[int] = 0) → List[List[float]][source]¶
Call out to OpenAI’s embedding endpoint for embedding search docs.
Parameters
texts (List[str]) – The list of texts to embed.
chunk_size (Optional[int]) – The chunk size of embeddings. If None, will use the chunk size
specified by the class.
Returns
List of embeddings, one for each text.
Return type