viruPVS/langchain-docs huggingface dataset:
Return docs most similar to query using specified search type.
similarity_search(query[, k, filter])
Run similarity search with Chroma.
similarity_search_by_vector(embedding[, k, ...])
Return docs most similar to embedding vector.
similarity_search_by_vector_with_relevance_scores(...)
Return docs most similar to embedding vector and similarity score.
similarity_search_with_relevance_scores(query)
Return docs and relevance scores in the range [0, 1].
similarity_search_with_score(query[, k, ...])
Run similarity search with Chroma with distance.
update_document(document_id, document)
Update a document in the collection.
update_documents(ids, documents)
Update a document in the collection.
__init__(collection_name: str = 'langchain', embedding_function: Optional[Embeddings] = None, persist_directory: Optional[str] = None, client_settings: Optional[chromadb.config.Settings] = None, collection_metadata: Optional[Dict] = None, client: Optional[chromadb.Client] = None, relevance_score_fn: Optional[Callable[[float], float]] = None) → None[source]¶
Initialize with a Chroma client.
async aadd_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
async aadd_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, **kwargs: Any) → List[str]¶
Run more texts through the embeddings and add to the vectorstore.
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
add_documents(documents: List[Document], **kwargs: Any) → List[str]¶
Run more documents through the embeddings and add to the vectorstore.
Parameters
(List[Document] (documents) – Documents to add to the vectorstore.
Returns
List of IDs of the added texts.
Return type
List[str]
add_images(uris: List[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶
Run more images through the embeddings and add to the vectorstore.
Parameters
List[str] (uris) – File path to the image.
metadatas (Optional[List[dict]], optional) – Optional list of metadatas.
ids (Optional[List[str]], optional) – Optional list of IDs.
Returns
List of IDs of the added images.
Return type
List[str]
add_texts(texts: Iterable[str], metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, **kwargs: Any) → List[str][source]¶
Run more texts through the embeddings and add to the vectorstore.
Parameters
texts (Iterable[str]) – Texts to add to the vectorstore.
metadatas (Optional[List[dict]], optional) – Optional list of metadatas.
ids (Optional[List[str]], optional) – Optional list of IDs.
Returns
List of IDs of the added texts.
Return type
List[str]
async adelete(ids: Optional[List[str]] = None, **kwargs: Any) → Optional[bool]¶
Delete by vector ID or other criteria.
Parameters
ids – List of ids to delete.
**kwargs – Other keyword arguments that subclasses might use.
Returns
True if deletion is successful,
False otherwise, None if not implemented.
Return docs most similar to embedding vector.
async asimilarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1], asynchronously.
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
async asimilarity_search_with_score(*args: Any, **kwargs: Any) → List[Tuple[Document, float]]¶
Run similarity search with distance asynchronously.
delete(ids: Optional[List[str]] = None, **kwargs: Any) → None[source]¶
Delete by vector IDs.
Parameters
ids – List of ids to delete.
delete_collection() → None[source]¶
Delete the collection.
encode_image(uri: str) → str[source]¶
Get base64 string from image URI.
classmethod from_documents(documents: List[Document], embedding: Optional[Embeddings] = None, ids: Optional[List[str]] = None, collection_name: str = 'langchain', persist_directory: Optional[str] = None, client_settings: Optional[chromadb.config.Settings] = None, client: Optional[chromadb.Client] = None, collection_metadata: Optional[Dict] = None, **kwargs: Any) → Chroma[source]¶
Create a Chroma vectorstore from a list of documents.
If a persist_directory is specified, the collection will be persisted there.
Otherwise, the data will be ephemeral in-memory.
Parameters
Otherwise, the data will be ephemeral in-memory.
Parameters
collection_name (str) – Name of the collection to create.
persist_directory (Optional[str]) – Directory to persist the collection.
ids (Optional[List[str]]) – List of document IDs. Defaults to None.
documents (List[Document]) – List of documents to add to the vectorstore.
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
client_settings (Optional[chromadb.config.Settings]) – Chroma client settings
collection_metadata (Optional[Dict]) – Collection configurations.
Defaults to None.
Returns
Chroma vectorstore.
Return type
Chroma
classmethod from_texts(texts: List[str], embedding: Optional[Embeddings] = None, metadatas: Optional[List[dict]] = None, ids: Optional[List[str]] = None, collection_name: str = 'langchain', persist_directory: Optional[str] = None, client_settings: Optional[chromadb.config.Settings] = None, client: Optional[chromadb.Client] = None, collection_metadata: Optional[Dict] = None, **kwargs: Any) → Chroma[source]¶
Create a Chroma vectorstore from a raw documents.
If a persist_directory is specified, the collection will be persisted there.
Otherwise, the data will be ephemeral in-memory.
Parameters
texts (List[str]) – List of texts to add to the collection.
collection_name (str) – Name of the collection to create.
persist_directory (Optional[str]) – Directory to persist the collection.
embedding (Optional[Embeddings]) – Embedding function. Defaults to None.
metadatas (Optional[List[dict]]) – List of metadatas. Defaults to None.
ids (Optional[List[str]]) – List of document IDs. Defaults to None.
ids (Optional[List[str]]) – List of document IDs. Defaults to None.
client_settings (Optional[chromadb.config.Settings]) – Chroma client settings
collection_metadata (Optional[Dict]) – Collection configurations.
Defaults to None.
Returns
Chroma vectorstore.
Return type
Chroma
get(ids: Optional[OneOrMany[ID]] = None, where: Optional[Where] = None, limit: Optional[int] = None, offset: Optional[int] = None, where_document: Optional[WhereDocument] = None, include: Optional[List[str]] = None) → Dict[str, Any][source]¶
Gets the collection.
Parameters
ids – The ids of the embeddings to get. Optional.
where – A Where type dict used to filter results by.
E.g. {“color” : “red”, “price”: 4.20}. Optional.
limit – The number of documents to return. Optional.
offset – The offset to start returning results from.
Useful for paging results with limit. Optional.
where_document – A WhereDocument type dict used to filter by the documents.
E.g. {$contains: “hello”}. Optional.
include – A list of what to include in the results.
Can contain “embeddings”, “metadatas”, “documents”.
Ids are always included.
Defaults to [“metadatas”, “documents”]. Optional.
max_marginal_relevance_search(query: str, k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
query – Text to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of Documents selected by maximal marginal relevance.
max_marginal_relevance_search_by_vector(embedding: List[float], k: int = 4, fetch_k: int = 20, lambda_mult: float = 0.5, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Return docs selected using the maximal marginal relevance.
Maximal marginal relevance optimizes for similarity to query AND diversity
among selected documents.
Parameters
embedding – Embedding to look up documents similar to.
k – Number of Documents to return. Defaults to 4.
fetch_k – Number of Documents to fetch to pass to MMR algorithm.
lambda_mult – Number between 0 and 1 that determines the degree
of diversity among the results with 0 corresponding
to maximum diversity and 1 to minimum diversity.
Defaults to 0.5.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of Documents selected by maximal marginal relevance.
persist() → None[source]¶
Persist the collection.
This can be used to explicitly persist the data to disk.
Persist the collection.
This can be used to explicitly persist the data to disk.
It will also be called automatically when the object is destroyed.
search(query: str, search_type: str, **kwargs: Any) → List[Document]¶
Return docs most similar to query using specified search type.
similarity_search(query: str, k: int = 4, filter: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Run similarity search with Chroma.
Parameters
query (str) – Query text to search for.
k (int) – Number of results to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of documents most similar to the query text.
Return type
List[Document]
similarity_search_by_vector(embedding: List[float], k: int = 4, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Document][source]¶
Return docs most similar to embedding vector.
:param embedding: Embedding to look up documents similar to.
:type embedding: List[float]
:param k: Number of Documents to return. Defaults to 4.
:type k: int
:param filter: Filter by metadata. Defaults to None.
:type filter: Optional[Dict[str, str]]
Returns
List of Documents most similar to the query vector.
similarity_search_by_vector_with_relevance_scores(embedding: List[float], k: int = 4, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Return docs most similar to embedding vector and similarity score.
Parameters
embedding (List[float]) – Embedding to look up documents similar to.
k (int) – Number of Documents to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of documents most similar to
the query text and cosine distance in float for each.
Lower score represents more similarity.
Return type
List[Tuple[Document, float]]
similarity_search_with_relevance_scores(query: str, k: int = 4, **kwargs: Any) → List[Tuple[Document, float]]¶
Return docs and relevance scores in the range [0, 1].
0 is dissimilar, 1 is most similar.
Parameters
query – input text
k – Number of Documents to return. Defaults to 4.
**kwargs – kwargs to be passed to similarity search. Should include:
score_threshold: Optional, a floating point value between 0 to 1 to
filter the resulting set of retrieved docs
Returns
List of Tuples of (doc, similarity_score)
similarity_search_with_score(query: str, k: int = 4, filter: Optional[Dict[str, str]] = None, where_document: Optional[Dict[str, str]] = None, **kwargs: Any) → List[Tuple[Document, float]][source]¶
Run similarity search with Chroma with distance.
Parameters
query (str) – Query text to search for.
k (int) – Number of results to return. Defaults to 4.
filter (Optional[Dict[str, str]]) – Filter by metadata. Defaults to None.
Returns
List of documents most similar to
the query text and cosine distance in float for each.
Lower score represents more similarity.
Return type
Lower score represents more similarity.
Return type
List[Tuple[Document, float]]
update_document(document_id: str, document: Document) → None[source]¶
Update a document in the collection.
Parameters
document_id (str) – ID of the document to update.
document (Document) – Document to update.
update_documents(ids: List[str], documents: List[Document]) → None[source]¶
Update a document in the collection.
Parameters
ids (List[str]) – List of ids of the document to update.
documents (List[Document]) – List of documents to update.
Examples using Chroma¶
RePhraseQueryRetriever
LOTR (Merger Retriever)
Ollama
Confident
Chroma
Vectorstore
StarRocks
Psychic
Docugami
Set env var OPENAI_API_KEY or load from a .env file:
Set env var OPENAI_API_KEY or load from a .env file
Question Answering
Perform context-aware text splitting
Use local LLMs
Retrieve from vector stores directly
Improve document indexing with HyDE
Structure answers with OpenAI functions
SalesGPT - Your Context-Aware AI Sales Assistant With Knowledge Base
MultiVector Retriever
MultiQueryRetriever
Parent Document Retriever
WebResearchRetriever
Lost in the middle: The problem with long contexts
Memory in the Multi-Input Chain
Combine agents and vector stores
Few-shot examples for chat models
Router
Loading from LangChainHub
langchain_community.vectorstores.elasticsearch.SparseRetrievalStrategy¶
class langchain_community.vectorstores.elasticsearch.SparseRetrievalStrategy(model_id: Optional[str] = None)[source]¶
Sparse retrieval strategy using the text_expansion processor.
Methods
__init__([model_id])
before_index_setup(client, text_field, ...)
Executes before the index is created.
index(dims_length, vector_query_field, ...)
Executes when the index is created.
query(query_vector, query, k, fetch_k, ...)
Executes when a search is performed on the store.
require_inference()
Returns whether or not the strategy requires inference to be performed on the text before it is added to the index.
__init__(model_id: Optional[str] = None)[source]¶
before_index_setup(client: Elasticsearch, text_field: str, vector_query_field: str) → None[source]¶
Executes before the index is created. Used for setting up
any required Elasticsearch resources like a pipeline.
Parameters
client – The Elasticsearch client.
text_field – The field containing the text data in the index.
vector_query_field – The field containing the vector
representations in the index.
index(dims_length: Optional[int], vector_query_field: str, similarity: Optional[DistanceStrategy]) → Dict[source]¶
Executes when the index is created.
Parameters
dims_length – Numeric length of the embedding vectors,
or None if not using vector-based query.
vector_query_field – The field containing the vector
representations in the index.
similarity – The similarity strategy to use,
or None if not using one.
Returns
The Elasticsearch settings and mappings for the strategy.
Return type
Dict
Returns
The Elasticsearch settings and mappings for the strategy.
Return type
Dict
query(query_vector: Optional[List[float]], query: Optional[str], k: int, fetch_k: int, vector_query_field: str, text_field: str, filter: List[dict], similarity: Optional[DistanceStrategy]) → Dict[source]¶
Executes when a search is performed on the store.
Parameters
query_vector – The query vector,
or None if not using vector-based query.
query – The text query, or None if not using text-based query.
k – The total number of results to retrieve.
fetch_k – The number of results to fetch initially.
vector_query_field – The field containing the vector
representations in the index.
text_field – The field containing the text data in the index.
filter – List of filter clauses to apply to the query.
similarity – The similarity strategy to use, or None if not using one.
Returns
The Elasticsearch query body.
Return type
Dict
require_inference() → bool[source]¶
Returns whether or not the strategy requires inference
to be performed on the text before it is added to the index.
Returns
Whether or not the strategy requires inference
to be performed on the text before it is added to the index.
Return type
bool
langchain.model_laboratory.ModelLaboratory¶
class langchain.model_laboratory.ModelLaboratory(chains: Sequence[Chain], names: Optional[List[str]] = None)[source]¶
Experiment with different models.
Initialize with chains to experiment with.
Parameters
chains – list of chains to experiment with.
Methods
__init__(chains[, names])
Initialize with chains to experiment with.
compare(text)
Compare model outputs on an input text.
from_llms(llms[, prompt])
Initialize with LLMs to experiment with and optional prompt.
__init__(chains: Sequence[Chain], names: Optional[List[str]] = None)[source]¶
Initialize with chains to experiment with.
Parameters
chains – list of chains to experiment with.
compare(text: str) → None[source]¶
Compare model outputs on an input text.
If a prompt was provided with starting the laboratory, then this text will be
fed into the prompt. If no prompt was provided, then the input text is the
entire prompt.
Parameters
text – input text to run all models on.
classmethod from_llms(llms: List[BaseLLM], prompt: Optional[PromptTemplate] = None) → ModelLaboratory[source]¶
Initialize with LLMs to experiment with and optional prompt.
Parameters
llms – list of LLMs to experiment with
prompt – Optional prompt to use to prompt the LLMs. Defaults to None.
If a prompt was provided, it should only have one input variable.
Examples using ModelLaboratory¶
Manifest
Model comparison
langchain_experimental.graph_transformers.diffbot.SimplifiedSchema¶
class langchain_experimental.graph_transformers.diffbot.SimplifiedSchema[source]¶
Provides functionality for working with a simplified schema mapping.
schema¶
A dictionary containing the mapping to simplified schema types.
Type
Dict
Initializes the schema dictionary based on the predefined list.
Methods
__init__()
Initializes the schema dictionary based on the predefined list.
get_type(type)
Retrieves the simplified schema type for a given original type.
__init__() → None[source]¶
Initializes the schema dictionary based on the predefined list.
get_type(type: str) → str[source]¶
Retrieves the simplified schema type for a given original type.
Parameters
type (str) – The original schema type to find the simplified type for.
Returns
The simplified schema type if it exists;otherwise, returns the original type.
Return type
str
langchain_experimental.graph_transformers.diffbot.DiffbotGraphTransformer¶
class langchain_experimental.graph_transformers.diffbot.DiffbotGraphTransformer(diffbot_api_key: Optional[str] = None, fact_confidence_threshold: float = 0.7, include_qualifiers: bool = True, include_evidence: bool = True, simplified_schema: bool = True)[source]¶
Transforms documents into graph documents using Diffbot’s NLP API.
A graph document transformation system takes a sequence of Documents and returns a
sequence of Graph Documents.
Example
class DiffbotGraphTransformer(BaseGraphDocumentTransformer):
    def transform_documents(
        self, documents: Sequence[Document], **kwargs: Any
    ) -> Sequence[GraphDocument]:
        results = []
        for document in documents:
            raw_results = self.nlp_request(document.page_content)
            graph_document = self.process_response(raw_results, document)
            results.append(graph_document)
        return results
    async def atransform_documents(
        self, documents: Sequence[Document], **kwargs: Any
    ) -> Sequence[Document]:
        raise NotImplementedError
Initialize the graph transformer with various options.
Parameters
diffbot_api_key (str) – The API key for Diffbot’s NLP services.
fact_confidence_threshold (float) – Minimum confidence level for facts to be included.
include_qualifiers (bool) – Whether to include qualifiers in the relationships.
include_evidence (bool) – Whether to include evidence for the relationships.
simplified_schema (bool) – Whether to use a simplified schema for relationships.
Methods
__init__([diffbot_api_key, ...])
Initialize the graph transformer with various options.
convert_to_graph_documents(documents)
Convert a sequence of documents into graph documents.
nlp_request(text)
Make an API request to the Diffbot NLP endpoint.
nlp_request(text)
Make an API request to the Diffbot NLP endpoint.
process_response(payload, document)
Transform the Diffbot NLP response into a GraphDocument.
__init__(diffbot_api_key: Optional[str] = None, fact_confidence_threshold: float = 0.7, include_qualifiers: bool = True, include_evidence: bool = True, simplified_schema: bool = True) → None[source]¶
Initialize the graph transformer with various options.
Parameters
diffbot_api_key (str) – The API key for Diffbot’s NLP services.
fact_confidence_threshold (float) – Minimum confidence level for facts to be included.
include_qualifiers (bool) – Whether to include qualifiers in the relationships.
include_evidence (bool) – Whether to include evidence for the relationships.
simplified_schema (bool) – Whether to use a simplified schema for relationships.
convert_to_graph_documents(documents: Sequence[Document]) → List[GraphDocument][source]¶
Convert a sequence of documents into graph documents.
Parameters
documents (Sequence[Document]) – The original documents.
**kwargs – Additional keyword arguments.
Returns
The transformed documents as graphs.
Return type
Sequence[GraphDocument]
nlp_request(text: str) → Dict[str, Any][source]¶
Make an API request to the Diffbot NLP endpoint.
Parameters
text (str) – The text to be processed.
Returns
The JSON response from the API.
Return type
Dict[str, Any]
process_response(payload: Dict[str, Any], document: Document) → GraphDocument[source]¶
Transform the Diffbot NLP response into a GraphDocument.
Parameters
payload (Dict[str, Any]) – The JSON response from Diffbot’s NLP API.
document (Document) – The original document.
Returns
langchain_experimental.graph_transformers.diffbot.NodesList¶
class langchain_experimental.graph_transformers.diffbot.NodesList[source]¶
Manages a list of nodes with associated properties.
nodes¶
Stores nodes as keys and their properties as values.
Each key is a tuple where the first element is the
node ID and the second is the node type.
Type
Dict[Tuple, Any]
Methods
__init__()
add_node_property(node, properties)
Adds or updates node properties.
return_node_list()
Returns the nodes as a list of Node objects.
__init__() → None[source]¶
add_node_property(node: Tuple[Union[str, int], str], properties: Dict[str, Any]) → None[source]¶
Adds or updates node properties.
If the node does not exist in the list, it’s added along with its properties.
If the node already exists, its properties are updated with the new values.
Parameters
node (Tuple) – A tuple containing the node ID and node type.
properties (Dict) – A dictionary of properties to add or update for the node.
return_node_list() → List[Node][source]¶
Returns the nodes as a list of Node objects.
Each Node object will have its ID, type, and properties populated.
Returns
A list of Node objects.
Return type
List[Node]