Tony20100/langchain_full huggingface dataset:
Returns
List of embeddings, one for each text.
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Call out to OpenAI’s embedding endpoint for embedding query text.
Parameters
text (str) – The text to embed.
Returns
Embedding for the text.
Return type
List[float]
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
Return type
None
classmethod validate(value: Any) → Model¶
Parameters
value (Any) – 
Return type
Model
Examples using OpenAIEmbeddings¶
Activeloop Deep Lake
Activeloop Deep Memory
Alibaba Cloud OpenSearch
Amazon Document DB
AnalyticDB
Apache Cassandra
Apache Doris
Astra DB
Astra DB (Cassandra)
Azure AI Search
Azure Cosmos DB
Azure OpenAI
Build a Query Analysis System
Build a Question/Answering system over SQL data
Build a Retrieval Augmented Generation (RAG) App
Build an Agent
Caching
Cassandra
China Mobile ECloud ElasticSearch VectorSearch
Chroma
ClickHouse
Confident
Conversational RAG
Couchbase 
Databricks Vector Search
Deep Lake
DingoDB
DocArray
DocArray HnswSearch
DocArray InMemorySearch
Docugami
Document Comparison
DuckDB
Elasticsearch
Epsilla
Faiss
Faiss (Async)
FlashRank reranker
Fleet AI Context
Hippo
Hologres
How deal with high cardinality categoricals when doing query analysis
How to add chat history
How to add retrieval to chatbots
How to add scores to retriever results
How to add values to a chain’s state
How to best prompt for Graph-RAG
How to better prompt when doing SQL question-answering
How to combine results from multiple retrievers
How to create and query vector stores
How to deal with large databases when doing SQL question-answering
How to do “self-querying” retrieval
How to do per-user retrieval
How to do retrieval with contextual compression
How to get a RAG application to add citations
How to get your RAG application to return sources
How to handle cases where no queries are generated
How to handle cases where no queries are generated
How to handle long text when doing extraction
How to handle multiple queries when doing query analysis
How to handle multiple retrievers when doing query analysis
How to inspect runnables
How to invoke runnables in parallel
How to load PDFs
How to pass through arguments from one step to the next
How to retrieve using multiple vectors per document
How to route between sub-chains
How to select examples by maximal marginal relevance (MMR)
How to select examples by similarity
How to split text based on semantic similarity
How to stream results from your RAG application
How to stream runnables
How to use a time-weighted vector store retriever
How to use a vectorstore as a retriever
How to use few shot examples
How to use few shot examples in chat models
How to use the LangChain indexing API
How to use the MultiQueryRetriever
How to use the Parent Document Retriever
Hybrid Search
Jaguar Vector Database
JaguarDB Vector Database
Javelin AI Gateway
KDB.AI
Kinetica Vectorstore API
Kinetica Vectorstore based Retriever
LLMLingua Document Compressor
LOTR (Merger Retriever)
LanceDB
Lantern
Meilisearch
Milvus
Milvus Hybrid Search
Model caches
Momento Vector Index (MVI)
MongoDB Atlas
MyScale
Neo4j Vector Index
OpenAI
OpenSearch
PGVector (Postgres)
Pinecone
Pinecone Hybrid Search
Postgres Embedding
Psychic
Qdrant
RAGatouille
RankLLM Reranker
RePhraseQuery
Redis
Rockset
SAP HANA Cloud Vector Engine
SVM
SingleStoreDB
StarRocks
Supabase (Postgres)
SVM
SingleStoreDB
StarRocks
Supabase (Postgres)
Tencent Cloud VectorDB
Text embedding models
TiDB Vector
Tigris
Timescale Vector (Postgres)
Timescale Vector (Postgres) 
Typesense
USearch
UpTrain
Upstash Vector
Vector stores and retrievers
Weaviate
Xata
Yellowbrick
YouTube audio
Zilliz
kNN
scikit-learn
viking DB
langchain_community.embeddings.deepinfra.DeepInfraEmbeddings¶
class langchain_community.embeddings.deepinfra.DeepInfraEmbeddings[source]¶
Bases: BaseModel, Embeddings
Deep Infra’s embedding inference service.
To use, you should have the
environment variable DEEPINFRA_API_TOKEN set with your API token, or pass
it as a named parameter to the constructor.
There are multiple embeddings models available,
see https://deepinfra.com/models?type=embeddings.
Example
from langchain_community.embeddings import DeepInfraEmbeddings
deepinfra_emb = DeepInfraEmbeddings(
    model_id="sentence-transformers/clip-ViT-B-32",
    deepinfra_api_token="my-api-key"
)
r1 = deepinfra_emb.embed_documents(
    [
        "Alpha is the first letter of Greek alphabet",
        "Beta is the second letter of Greek alphabet",
    ]
)
r2 = deepinfra_emb.embed_query(
    "What is the second letter of Greek alphabet"
)
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param batch_size: int = 1024¶
Batch size for embedding requests.
param deepinfra_api_token: Optional[str] = None¶
API token for Deep Infra. If not provided, the token is
fetched from the environment variable ‘DEEPINFRA_API_TOKEN’.
param embed_instruction: str = 'passage: '¶
Instruction used to embed documents.
param model_id: str = 'sentence-transformers/clip-ViT-B-32'¶
Embeddings model to use.
param model_kwargs: Optional[dict] = None¶
Other model keyword args
param normalize: bool = False¶
whether to normalize the computed embeddings
Other model keyword args
param normalize: bool = False¶
whether to normalize the computed embeddings
param query_instruction: str = 'query: '¶
Instruction used to embed the query.
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[List[float]]
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
Parameters
text (str) – 
Return type
List[float]
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny¶
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
Return type
DictStrAny
embed_documents(texts: List[str]) → List[List[float]][source]¶
Embed documents using a Deep Infra deployed embedding model.
For larger batches, the input list of texts is chunked into smaller
batches to avoid exceeding the maximum request size.
Parameters
texts (List[str]) – The list of texts to embed.
Returns
List of embeddings, one for each text.
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Embed a query using a Deep Infra deployed embedding model.
Parameters
text (str) – The text to embed.
Returns
Embeddings for the text.
Return type
List[float]
classmethod from_orm(obj: Any) → Model¶
Parameters
List[float]
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
encoder (Optional[Callable[[Any], Any]]) – 
models_as_dict (bool) – 
dumps_kwargs (Any) – 
Return type
unicode
classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
Parameters
path (Union[str, Path]) – 
content_type (unicode) – 
encoding (unicode) – 
proto (Protocol) – 
allow_pickle (bool) – 
Return type
Model
classmethod parse_obj(obj: Any) → Model¶
Parameters
Return type
Model
classmethod parse_obj(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
Parameters
b (Union[str, bytes]) – 
content_type (unicode) – 
encoding (unicode) – 
proto (Protocol) – 
allow_pickle (bool) – 
Return type
Model
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
Return type
DictStrAny
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
dumps_kwargs (Any) – 
Return type
unicode
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
classmethod validate(value: Any) → Model¶
Parameters
value (Any) – 
Return type
Model
Examples using DeepInfraEmbeddings¶
DeepInfra
langchain_community.embeddings.huggingface.HuggingFaceEmbeddings¶
class langchain_community.embeddings.huggingface.HuggingFaceEmbeddings[source]¶
Bases: BaseModel, Embeddings
[Deprecated] HuggingFace sentence_transformers embedding models.
To use, you should have the sentence_transformers python package installed.
Example
from langchain_community.embeddings import HuggingFaceEmbeddings
model_name = "sentence-transformers/all-mpnet-base-v2"
model_kwargs = {'device': 'cpu'}
encode_kwargs = {'normalize_embeddings': False}
hf = HuggingFaceEmbeddings(
    model_name=model_name,
    model_kwargs=model_kwargs,
    encode_kwargs=encode_kwargs
)
Notes
Deprecated since version 0.2.2.
Initialize the sentence_transformer.
param cache_folder: Optional[str] = None¶
Path to store models.
Can be also set by SENTENCE_TRANSFORMERS_HOME environment variable.
param encode_kwargs: Dict[str, Any] [Optional]¶
Keyword arguments to pass when calling the encode method of the Sentence
Transformer model, such as prompt_name, prompt, batch_size, precision,
normalize_embeddings, and more.
See also the Sentence Transformer documentation: https://sbert.net/docs/package_reference/SentenceTransformer.html#sentence_transformers.SentenceTransformer.encode
param model_kwargs: Dict[str, Any] [Optional]¶
Keyword arguments to pass to the Sentence Transformer model, such as device,
prompts, default_prompt_name, revision, trust_remote_code, or token.
See also the Sentence Transformer documentation: https://sbert.net/docs/package_reference/SentenceTransformer.html#sentence_transformers.SentenceTransformer
param model_name: str = 'sentence-transformers/all-mpnet-base-v2'¶
Model name to use.
param multi_process: bool = False¶
Run encode() on multiple GPUs.
param multi_process: bool = False¶
Run encode() on multiple GPUs.
param show_progress: bool = False¶
Whether to show a progress bar.
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[List[float]]
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
Parameters
text (str) – 
Return type
List[float]
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
langchain_community.embeddings.spacy_embeddings.SpacyEmbeddings¶
class langchain_community.embeddings.spacy_embeddings.SpacyEmbeddings[source]¶
Bases: BaseModel, Embeddings
Embeddings by spaCy models.
model_name¶
Name of a spaCy model.
Type
str
nlp¶
The spaCy model loaded into memory.
Type
Any
embed_documents(texts
List[str]) -> List[List[float]]:
Generates embeddings for a list of documents.
embed_query(text
str) -> List[float]:
Generates an embedding for a single piece of text.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param model_name: str = 'en_core_web_sm'¶
param nlp: Optional[Any] = None¶
async aembed_documents(texts: List[str]) → List[List[float]][source]¶
Asynchronously generates embeddings for a list of documents.
This method is not implemented and raises a NotImplementedError.
Parameters
texts (List[str]) – The documents to generate embeddings for.
Raises
NotImplementedError – This method is not implemented.
Return type
List[List[float]]
async aembed_query(text: str) → List[float][source]¶
Asynchronously generates an embedding for a single piece of text.
This method is not implemented and raises a NotImplementedError.
Parameters
text (str) – The text to generate an embedding for.
Raises
NotImplementedError – This method is not implemented.
Return type
List[float]
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – 
by_alias (bool) – 
skip_defaults (Optional[bool]) – 
exclude_unset (bool) – 
exclude_defaults (bool) – 
exclude_none (bool) – 
Return type
DictStrAny
embed_documents(texts: List[str]) → List[List[float]][source]¶
Generates embeddings for a list of documents.
Parameters
texts (List[str]) – The documents to generate embeddings for.
Returns
A list of embeddings, one for each document.
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Generates an embedding for a single piece of text.
Parameters
text (str) – The text to generate an embedding for.
Returns
The embedding for the text.
Return type
List[float]
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Model
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
Parameters
ref_template (unicode) – 
Return type
DictStrAny
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
dumps_kwargs (Any) – 
Return type
unicode
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
classmethod validate(value: Any) → Model¶
Parameters
value (Any) – 
Return type
Model
Examples using SpacyEmbeddings¶
SpaCy
spaCy
langchain_community.embeddings.xinference.XinferenceEmbeddings¶
class langchain_community.embeddings.xinference.XinferenceEmbeddings(server_url: Optional[str] = None, model_uid: Optional[str] = None)[source]¶
Xinference embedding models.
To use, you should have the xinference library installed:
pip install xinference
If you’re simply using the services provided by Xinference, you can utilize the xinference_client package:
pip install xinference_client
Check out: https://github.com/xorbitsai/inference
To run, you need to start a Xinference supervisor on one server and Xinference workers on the other servers.
Example
To start a local instance of Xinference, run
$ xinference
You can also deploy Xinference in a distributed cluster. Here are the steps:
Starting the supervisor:
$ xinference-supervisor
If you’re simply using the services provided by Xinference, you can utilize the xinference_client package:
pip install xinference_client
Starting the worker:
$ xinference-worker
Then, launch a model using command line interface (CLI).
Example:
$ xinference launch -n orca -s 3 -q q4_0
It will return a model UID. Then you can use Xinference Embedding with LangChain.
Example:
from langchain_community.embeddings import XinferenceEmbeddings
xinference = XinferenceEmbeddings(
    server_url="http://0.0.0.0:9997",
    model_uid = {model_uid} # replace model_uid with the model UID return from launching the model
)
Attributes
client
server_url
URL of the xinference server
model_uid
UID of the launched model
Methods
__init__([server_url, model_uid])
aembed_documents(texts)
__init__([server_url, model_uid])
aembed_documents(texts)
Asynchronous Embed search docs.
aembed_query(text)
Asynchronous Embed query text.
embed_documents(texts)
Embed a list of documents using Xinference.
embed_query(text)
Embed a query of documents using Xinference.
Parameters
server_url (Optional[str]) – 
model_uid (Optional[str]) – 
__init__(server_url: Optional[str] = None, model_uid: Optional[str] = None)[source]¶
Parameters
server_url (Optional[str]) – 
model_uid (Optional[str]) – 
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[List[float]]
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
Parameters
text (str) – 
Return type
List[float]
embed_documents(texts: List[str]) → List[List[float]][source]¶
Embed a list of documents using Xinference.
:param texts: The list of texts to embed.
Returns
List of embeddings, one for each text.
Parameters
texts (List[str]) – 
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Embed a query of documents using Xinference.
:param text: The text to embed.
Returns
Embeddings for the text.
Parameters
text (str) – 
Return type
List[float]
Examples using XinferenceEmbeddings¶
Xorbits inference (Xinference)
langchain_community.embeddings.embaas.EmbaasEmbeddings¶
class langchain_community.embeddings.embaas.EmbaasEmbeddings[source]¶
Bases: BaseModel, Embeddings
Embaas’s embedding service.
To use, you should have the
environment variable EMBAAS_API_KEY set with your API key, or pass
it as a named parameter to the constructor.
Example
# initialize with default model and instruction
from langchain_community.embeddings import EmbaasEmbeddings
emb = EmbaasEmbeddings()
# initialize with custom model and instruction
from langchain_community.embeddings import EmbaasEmbeddings
emb_model = "instructor-large"
emb_inst = "Represent the Wikipedia document for retrieval"
emb = EmbaasEmbeddings(
    model=emb_model,
    instruction=emb_inst
)
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param api_url: str = 'https://api.embaas.io/v1/embeddings/'¶
The URL for the embaas embeddings API.
param embaas_api_key: Optional[SecretStr] = None¶
max number of retries for requests
Constraints
type = string
writeOnly = True
format = password
param instruction: Optional[str] = None¶
Instruction used for domain-specific embeddings.
param max_retries: Optional[int] = 3¶
request timeout in seconds
param model: str = 'e5-large-v2'¶
The model used for embeddings.
param timeout: Optional[int] = 30¶
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[List[float]]