Tony20100/langchain_full huggingface dataset:
langchain_nomic.embeddings.NomicEmbeddings¶
class langchain_nomic.embeddings.NomicEmbeddings(*, model: str, nomic_api_key: Optional[str] = ..., dimensionality: Optional[int] = ..., inference_mode: Literal['remote'] = ...)[source]¶
class langchain_nomic.embeddings.NomicEmbeddings(*, model: str, nomic_api_key: Optional[str] = ..., dimensionality: Optional[int] = ..., inference_mode: Literal['local', 'dynamic'], device: Optional[str] = ...)
class langchain_nomic.embeddings.NomicEmbeddings(*, model: str, nomic_api_key: Optional[str] = ..., dimensionality: Optional[int] = ..., inference_mode: str, device: Optional[str] = ...)
NomicEmbeddings embedding model.
Example
from langchain_nomic import NomicEmbeddings
model = NomicEmbeddings()
Initialize NomicEmbeddings model.
Parameters
model (str) – model name
nomic_api_key (Optional[str]) – optionally, set the Nomic API key. Uses the NOMIC_API_KEY
environment variable by default.
dimensionality (Optional[int]) – The embedding dimension, for use with Matryoshka-capable
models. Defaults to full-size.
inference_mode (str) – How to generate embeddings. One of remote, local
(Embed4All), or dynamic (automatic). Defaults to remote.
device (Optional[str]) – The device to use for local embeddings. Choices include
cpu, gpu, nvidia, amd, or a specific device name. See
the docstring for GPT4All.__init__ for more info. Typically
defaults to CPU. Do not use on macOS.
vision_model (Optional[str]) – 
Methods
__init__()
Initialize NomicEmbeddings model.
aembed_documents(texts)
Asynchronous Embed search docs.
aembed_documents(texts)
Asynchronous Embed search docs.
aembed_query(text)
Asynchronous Embed query text.
embed(texts, *, task_type)
Embed texts.
embed_documents(texts)
Embed search docs.
embed_image(uris)
embed_query(text)
Embed query text.
__init__(*, model: str, nomic_api_key: Optional[str] = None, dimensionality: Optional[int] = None, inference_mode: Literal['remote'] = 'remote')[source]¶
__init__(*, model: str, nomic_api_key: Optional[str] = None, dimensionality: Optional[int] = None, inference_mode: Literal['local', 'dynamic'], device: Optional[str] = None)
__init__(*, model: str, nomic_api_key: Optional[str] = None, dimensionality: Optional[int] = None, inference_mode: str, device: Optional[str] = None)
Initialize NomicEmbeddings model.
Parameters
model – model name
nomic_api_key – optionally, set the Nomic API key. Uses the NOMIC_API_KEY
environment variable by default.
dimensionality – The embedding dimension, for use with Matryoshka-capable
models. Defaults to full-size.
inference_mode – How to generate embeddings. One of remote, local
(Embed4All), or dynamic (automatic). Defaults to remote.
device – The device to use for local embeddings. Choices include
cpu, gpu, nvidia, amd, or a specific device name. See
the docstring for GPT4All.__init__ for more info. Typically
defaults to CPU. Do not use on macOS.
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[List[float]]
Parameters
texts (List[str]) – 
Return type
List[List[float]]
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
Parameters
text (str) – 
Return type
List[float]
embed(texts: List[str], *, task_type: str) → List[List[float]][source]¶
Embed texts.
Parameters
texts (List[str]) – list of texts to embed
task_type (str) – the task type to use when embedding. One of search_query,
search_document, classification, clustering
Return type
List[List[float]]
embed_documents(texts: List[str]) → List[List[float]][source]¶
Embed search docs.
Parameters
texts (List[str]) – list of texts to embed as documents
Return type
List[List[float]]
embed_image(uris: List[str]) → List[List[float]][source]¶
Parameters
uris (List[str]) – 
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Embed query text.
Parameters
text (str) – query text
Return type
List[float]
embeddings = SelfHostedHFEmbeddings.from_pipeline(
    pipeline="models/pipeline.pkl",
    hardware=gpu,
    model_reqs=["./", "torch", "transformers"],
)
Init the pipeline with an auxiliary function.
The load function must be in global scope to be imported
and run on the server, i.e. in a module and not a REPL or closure.
Then, initialize the remote inference function.
param allow_dangerous_deserialization: bool = False¶
Allow deserialization using pickle which can be dangerous if
loading compromised data.
param cache: Union[BaseCache, bool, None] = None¶
Whether to cache the response.
If true, will use the global cache.
If false, will not use a cache
If None, will use the global cache if it’s set, otherwise no cache.
If instance of BaseCache, will use the provided cache.
Caching is not currently supported for streaming methods of models.
param callback_manager: Optional[BaseCallbackManager] = None¶
[DEPRECATED]
param callbacks: Callbacks = None¶
Callbacks to add to the run trace.
param custom_get_token_ids: Optional[Callable[[str], List[int]]] = None¶
Optional encoder to use for counting tokens.
param hardware: Any = None¶
Remote hardware to send the inference function to.
param inference_fn: Callable = <function _embed_documents>¶
Inference function to extract the embeddings on the remote hardware.
param inference_kwargs: Any = None¶
Any kwargs to pass to the model’s inference function.
param load_fn_kwargs: Optional[dict] = None¶
Keyword arguments to pass to the model load function.
param metadata: Optional[Dict[str, Any]] = None¶
Metadata to add to the run trace.
param model_load_fn: Callable [Required]¶
Metadata to add to the run trace.
param model_load_fn: Callable [Required]¶
Function to load the model remotely on the server.
param model_reqs: List[str] = ['./', 'torch']¶
Requirements to install on hardware to inference the model.
param tags: Optional[List[str]] = None¶
Tags to add to the run trace.
param verbose: bool [Optional]¶
Whether to print out response text.
__call__(prompt: str, stop: Optional[List[str]] = None, callbacks: Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]] = None, *, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, **kwargs: Any) → str¶
[Deprecated] Check Cache and run the LLM on the given prompt and input.
Notes
Deprecated since version langchain-core==0.1.7: Use invoke instead.
Parameters
prompt (str) – 
stop (Optional[List[str]]) – 
callbacks (Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]) – 
tags (Optional[List[str]]) – 
metadata (Optional[Dict[str, Any]]) – 
kwargs (Any) – 
Return type
str
async abatch(inputs: List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Any) → List[str]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
Parameters
inputs (List[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]]]) – 
config (Optional[Union[RunnableConfig, List[RunnableConfig]]]) – 
return_exceptions (bool) – 
kwargs (Any) – 
Return type
List[str]
async abatch_as_completed(inputs: Sequence[Input], config: Optional[Union[RunnableConfig, Sequence[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → AsyncIterator[Tuple[int, Union[Output, Exception]]]¶
Run ainvoke in parallel on a list of inputs,
yielding results as they complete.
Parameters
inputs (Sequence[Input]) – 
config (Optional[Union[RunnableConfig, Sequence[RunnableConfig]]]) – 
return_exceptions (bool) – 
kwargs (Optional[Any]) – 
Return type
AsyncIterator[Tuple[int, Union[Output, Exception]]]
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[List[float]]
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
Parameters
text (str) – 
Return type
List[float]
Parameters
text (str) – 
Return type
List[float]
async agenerate(prompts: List[str], stop: Optional[List[str]] = None, callbacks: Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]] = None, *, tags: Optional[Union[List[str], List[List[str]]]] = None, metadata: Optional[Union[Dict[str, Any], List[Dict[str, Any]]]] = None, run_name: Optional[Union[str, List[str]]] = None, run_id: Optional[Union[UUID, List[Optional[UUID]]]] = None, **kwargs: Any) → LLMResult¶
Asynchronously pass a sequence of prompts to a model and return generations.
This method should make use of batched calls for models that expose a batched
API.
Use this method when you want to:
take advantage of batched calls,
need more output from the model than just the top generated value,
are building chains that are agnostic to the underlying language modeltype (e.g., pure text completion models vs chat models).
Parameters
prompts (List[str]) – List of string prompts.
stop (Optional[List[str]]) – Stop words to use when generating. Model output is cut off at the
first occurrence of any of these substrings.
callbacks (Union[List[BaseCallbackHandler], BaseCallbackManager, None, List[Optional[Union[List[BaseCallbackHandler], BaseCallbackManager]]]]) – Callbacks to pass through. Used for executing additional
functionality, such as logging or streaming, throughout generation.
**kwargs (Any) – Arbitrary additional keyword arguments. These are usually passed
to the model provider API call.
tags (Optional[Union[List[str], List[List[str]]]]) –
values (Any) – 
Return type
Model
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to include in new model
exclude (Optional[Union[AbstractSetIntStr, MappingIntStrAny]]) – fields to exclude from new model, as with values this takes precedence over include
update (Optional[DictStrAny]) – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep (bool) – set to True to make a deep copy of the model
self (Model) – 
Returns
new model instance
Return type
Model
dict(**kwargs: Any) → Dict¶
Return a dictionary of the LLM.
Parameters
kwargs (Any) – 
Return type
Dict
embed_documents(texts: List[str]) → List[List[float]][source]¶
Compute doc embeddings using a HuggingFace transformer model.
Parameters
texts (List[str]) – The list of texts to embed.s
Returns
List of embeddings, one for each text.
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Compute query embeddings using a HuggingFace transformer model.
Parameters
text (str) – The text to embed.
Returns
Embeddings for the text.
Return type
List[float]
classmethod from_orm(obj: Any) → Model¶
Parameters
obj (Any) – 
Return type
Not implemented on this class.
Parameters
schema (Union[Dict, Type[BaseModel]]) – 
kwargs (Any) – 
Return type
Runnable[Union[PromptValue, str, Sequence[Union[BaseMessage, List[str], Tuple[str, str], str, Dict[str, Any]]]], Union[Dict, BaseModel]]
with_types(*, input_type: Optional[Type[Input]] = None, output_type: Optional[Type[Output]] = None) → Runnable[Input, Output]¶
Bind input and output types to a Runnable, returning a new Runnable.
Parameters
input_type (Optional[Type[Input]]) – 
output_type (Optional[Type[Output]]) – 
Return type
Runnable[Input, Output]
property InputType: TypeAlias¶
Get the input type for this runnable.
property OutputType: Type[str]¶
Get the input type for this runnable.
property config_specs: List[ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
name: Optional[str] = None¶
The name of the runnable. Used for debugging and tracing.
property output_schema: Type[BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using SelfHostedEmbeddings¶
Self Hosted
langchain_community.embeddings.elasticsearch.ElasticsearchEmbeddings¶
class langchain_community.embeddings.elasticsearch.ElasticsearchEmbeddings(client: MlClient, model_id: str, *, input_field: str = 'text_field')[source]¶
[Deprecated] Elasticsearch embedding models.
This class provides an interface to generate embeddings using a model deployed
in an Elasticsearch cluster. It requires an Elasticsearch connection object
and the model_id of the model deployed in the cluster.
In Elasticsearch you need to have an embedding model loaded and deployed.
- https://www.elastic.co/guide/en/elasticsearch/reference/current/infer-trained-model.html
- https://www.elastic.co/guide/en/machine-learning/current/ml-nlp-deploy-models.html
Notes
Deprecated since version 0.1.11: Use Use class in langchain-elasticsearch package instead.
Initialize the ElasticsearchEmbeddings instance.
Parameters
client (MlClient) – An Elasticsearch ML client object.
model_id (str) – The model_id of the model deployed in the Elasticsearch
cluster.
input_field (str) – The name of the key for the input text field in the
document. Defaults to ‘text_field’.
Methods
__init__(client, model_id, *[, input_field])
Initialize the ElasticsearchEmbeddings instance.
aembed_documents(texts)
Asynchronous Embed search docs.
aembed_query(text)
Asynchronous Embed query text.
embed_documents(texts)
Generate embeddings for a list of documents.
embed_query(text)
Generate an embedding for a single query text.
from_credentials(model_id, *[, es_cloud_id, ...])
Instantiate embeddings from Elasticsearch credentials.
from_es_connection(model_id, es_connection)
Instantiate embeddings from an existing Elasticsearch connection.
from_es_connection(model_id, es_connection)
Instantiate embeddings from an existing Elasticsearch connection.
__init__(client: MlClient, model_id: str, *, input_field: str = 'text_field')[source]¶
Initialize the ElasticsearchEmbeddings instance.
Parameters
client (MlClient) – An Elasticsearch ML client object.
model_id (str) – The model_id of the model deployed in the Elasticsearch
cluster.
input_field (str) – The name of the key for the input text field in the
document. Defaults to ‘text_field’.
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[List[float]]
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
Parameters
text (str) – 
Return type
List[float]
embed_documents(texts: List[str]) → List[List[float]][source]¶
Generate embeddings for a list of documents.
Parameters
texts (List[str]) – A list of document text strings to generate embeddings
for.
Returns
A list of embeddings, one for each document in the inputlist.
Return type
List[List[float]]
embed_query(text: str) → List[float][source]¶
Generate an embedding for a single query text.
Parameters
text (str) – The query text to generate an embedding for.
Returns
The embedding for the input query text.
Return type
List[float]
classmethod from_credentials(model_id: str, *, es_cloud_id: Optional[str] = None, es_user: Optional[str] = None, es_password: Optional[str] = None, input_field: str = 'text_field') → ElasticsearchEmbeddings[source]¶
Instantiate embeddings from Elasticsearch credentials.
Parameters
Instantiate embeddings from Elasticsearch credentials.
Parameters
model_id (str) – The model_id of the model deployed in the Elasticsearch
cluster.
input_field (str) – The name of the key for the input text field in the
document. Defaults to ‘text_field’.
es_cloud_id (Optional[str]) – (str, optional): The Elasticsearch cloud ID to connect to.
es_user (Optional[str]) – (str, optional): Elasticsearch username.
es_password (Optional[str]) – (str, optional): Elasticsearch password.
Return type
ElasticsearchEmbeddings
Example
from langchain_community.embeddings import ElasticsearchEmbeddings
# Define the model ID and input field name (if different from default)
model_id = "your_model_id"
# Optional, only if different from 'text_field'
input_field = "your_input_field"
# Credentials can be passed in two ways. Either set the env vars
# ES_CLOUD_ID, ES_USER, ES_PASSWORD and they will be automatically
# pulled in, or pass them in directly as kwargs.
embeddings = ElasticsearchEmbeddings.from_credentials(
    model_id,
    input_field=input_field,
    # es_cloud_id="foo",
    # es_user="bar",
    # es_password="baz",
)
documents = [
    "This is an example document.",
    "Another example document to generate embeddings for.",
]
embeddings_generator.embed_documents(documents)
classmethod from_es_connection(model_id: str, es_connection: Elasticsearch, input_field: str = 'text_field') → ElasticsearchEmbeddings[source]¶
Instantiate embeddings from an existing Elasticsearch connection.
This method provides a way to create an instance of the ElasticsearchEmbeddings
class using an existing Elasticsearch connection. The connection object is used
to create an MlClient, which is then used to initialize the
ElasticsearchEmbeddings instance.
ElasticsearchEmbeddings instance.
Args:
model_id (str): The model_id of the model deployed in the Elasticsearch cluster.
es_connection (elasticsearch.Elasticsearch): An existing Elasticsearch
connection object. input_field (str, optional): The name of the key for the
input text field in the document. Defaults to ‘text_field’.
Returns:
ElasticsearchEmbeddings: An instance of the ElasticsearchEmbeddings class.
Example
from elasticsearch import Elasticsearch
from langchain_community.embeddings import ElasticsearchEmbeddings
# Define the model ID and input field name (if different from default)
model_id = "your_model_id"
# Optional, only if different from 'text_field'
input_field = "your_input_field"
# Create Elasticsearch connection
es_connection = Elasticsearch(
    hosts=["localhost:9200"], http_auth=("user", "password")
)
# Instantiate ElasticsearchEmbeddings using the existing connection
embeddings = ElasticsearchEmbeddings.from_es_connection(
    model_id,
    es_connection,
    input_field=input_field,
)
documents = [
    "This is an example document.",
    "Another example document to generate embeddings for.",
]
embeddings_generator.embed_documents(documents)
Parameters
model_id (str) – 
es_connection (Elasticsearch) – 
input_field (str) – 
Return type
ElasticsearchEmbeddings
langchain_community.embeddings.volcengine.VolcanoEmbeddings¶
class langchain_community.embeddings.volcengine.VolcanoEmbeddings[source]¶
Bases: BaseModel, Embeddings
Volcengine Embeddings embedding models.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param chunk_size: int = 100¶
Chunk size when multiple texts are input
param client: Any = None¶
volcano client
param host: str = 'maas-api.ml-platform-cn-beijing.volces.com'¶
host
learn more from https://www.volcengine.com/docs/82379/1174746
param model: str = 'bge-large-zh'¶
Model name
you could get from https://www.volcengine.com/docs/82379/1174746
for now, we support bge_large_zh
param region: str = 'cn-beijing'¶
region
learn more from https://www.volcengine.com/docs/82379/1174746
param version: str = '1.0'¶
model version
param volcano_ak: Optional[str] = None¶
volcano access key
learn more from: https://www.volcengine.com/docs/6459/76491#ak-sk
param volcano_sk: Optional[str] = None¶
volcano secret key
learn more from: https://www.volcengine.com/docs/6459/76491#ak-sk
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[List[float]]
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
Parameters
text (str) – 
Return type
Parameters
obj (Any) – 
Return type
Model
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
Parameters
b (Union[str, bytes]) – 
content_type (unicode) – 
encoding (unicode) – 
proto (Protocol) – 
allow_pickle (bool) – 
Return type
Model
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
Return type
DictStrAny
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
Parameters
by_alias (bool) – 
ref_template (unicode) – 
dumps_kwargs (Any) – 
Return type
unicode
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
Parameters
localns (Any) – 
Return type
None
classmethod validate(value: Any) → Model¶
Parameters
value (Any) – 
Return type
Model
Examples using VolcanoEmbeddings¶
Volc Engine
langchain_community.embeddings.modelscope_hub.ModelScopeEmbeddings¶
class langchain_community.embeddings.modelscope_hub.ModelScopeEmbeddings[source]¶
Bases: BaseModel, Embeddings
ModelScopeHub embedding models.
To use, you should have the modelscope python package installed.
Example
from langchain_community.embeddings import ModelScopeEmbeddings
model_id = "damo/nlp_corom_sentence-embedding_english-base"
embed = ModelScopeEmbeddings(model_id=model_id, model_revision="v1.0.0")
Initialize the modelscope
param embed: Any = None¶
param model_id: str = 'damo/nlp_corom_sentence-embedding_english-base'¶
Model name to use.
param model_revision: Optional[str] = None¶
async aembed_documents(texts: List[str]) → List[List[float]]¶
Asynchronous Embed search docs.
Parameters
texts (List[str]) – 
Return type
List[List[float]]
async aembed_query(text: str) → List[float]¶
Asynchronous Embed query text.
Parameters
text (str) – 
Return type
List[float]
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
Parameters
_fields_set (Optional[SetStr]) – 
values (Any) – 
Return type
Model