viruPVS/langchain-docs huggingface dataset:
use case.
param mmr_lambda: Optional[float] = None¶
Lambda value for MMR search.
param search_scope: langchain_community.retrievers.zep.SearchScope = SearchScope.messages¶
Which documents to search. Messages or Summaries?
param search_type: langchain_community.retrievers.zep.SearchType = SearchType.similarity¶
Type of search to perform (similarity / mmr)
param session_id: str [Required]¶
Zep session ID.
param tags: Optional[List[str]] = None¶
Optional list of tags associated with the retriever. Defaults to None
These tags will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a retriever with its
use case.
param top_k: Optional[int] = None¶
Number of items to return.
param url: str [Required]¶
URL of your Zep server.
param zep_client: Optional[Any] = None¶
Zep client.
async abatch(inputs: List[Input], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → List[Output]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
async aget_relevant_documents(query: str, *, callbacks: Callbacks = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, run_name: Optional[str] = None, **kwargs: Any) → List[Document]¶
property OutputType: Type[langchain_core.runnables.utils.Output]¶
The type of output this runnable produces specified as a type annotation.
property config_specs: List[langchain_core.runnables.utils.ConfigurableFieldSpec]¶
List configurable fields for this runnable.
property input_schema: Type[pydantic.main.BaseModel]¶
The type of input this runnable accepts specified as a pydantic model.
property lc_attributes: Dict¶
List of attribute names that should be included in the serialized kwargs.
These attributes must be accepted by the constructor.
property lc_secrets: Dict[str, str]¶
A map of constructor argument names to secret ids.
For example,{“openai_api_key”: “OPENAI_API_KEY”}
property output_schema: Type[pydantic.main.BaseModel]¶
The type of output this runnable produces specified as a pydantic model.
Examples using ZepRetriever¶
Zep
Zep Memory
langchain_community.retrievers.tavily_search_api.SearchDepth¶
class langchain_community.retrievers.tavily_search_api.SearchDepth(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]¶
Search depth as enumerator.
BASIC = 'basic'¶
ADVANCED = 'advanced'¶
langchain_community.retrievers.azure_cognitive_search.AzureCognitiveSearchRetriever¶
class langchain_community.retrievers.azure_cognitive_search.AzureCognitiveSearchRetriever[source]¶
Bases: BaseRetriever
Azure Cognitive Search service retriever.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param aiosession: Optional[aiohttp.ClientSession] = None¶
ClientSession, in case we want to reuse connection for better performance.
param api_key: str = ''¶
API Key. Both Admin and Query keys work, but for reading data it’s
recommended to use a Query key.
param api_version: str = '2020-06-30'¶
API version
param content_key: str = 'content'¶
Key in a retrieved result to set as the Document page_content.
param index_name: str = ''¶
Name of Index inside Azure Cognitive Search service
param metadata: Optional[Dict[str, Any]] = None¶
Optional metadata associated with the retriever. Defaults to None
This metadata will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a retriever with its
use case.
param service_name: str = ''¶
Name of Azure Cognitive Search service
param tags: Optional[List[str]] = None¶
Optional list of tags associated with the retriever. Defaults to None
These tags will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a retriever with its
use case.
param top_k: Optional[int] = None¶
use case.
param top_k: Optional[int] = None¶
Number of results to retrieve. Set to None to retrieve all results.
async abatch(inputs: List[Input], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → List[Output]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
async aget_relevant_documents(query: str, *, callbacks: Callbacks = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, run_name: Optional[str] = None, **kwargs: Any) → List[Document]¶
Asynchronously get documents relevant to a query.
:param query: string to find relevant documents for
:param callbacks: Callback manager or list of callbacks
:param tags: Optional list of tags associated with the retriever. Defaults to None
These tags will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
Parameters
metadata – Optional metadata associated with the retriever. Defaults to None
This metadata will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
Returns
List of relevant documents
async ainvoke(input: str, config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → List[Document]¶
Default implementation of ainvoke, calls invoke from a thread.
The default implementation allows usage of async code even if
the runnable did not implement a native async version of invoke.
Subclasses should override this method if they can run asynchronously.
langchain_community.retrievers.kendra.Highlight¶
class langchain_community.retrievers.kendra.Highlight[source]¶
Bases: BaseModel
Information that highlights the keywords in the excerpt.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param BeginOffset: int [Required]¶
The zero-based location in the excerpt where the highlight starts.
param EndOffset: int [Required]¶
The zero-based location in the excerpt where the highlight ends.
param TopAnswer: Optional[bool] = None¶
Indicates whether the result is the best one.
param Type: Optional[str] = None¶
The highlight type: STANDARD or THESAURUS_SYNONYM.
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep – set to True to make a deep copy of the model
Returns
new model instance
langchain_community.retrievers.weaviate_hybrid_search.WeaviateHybridSearchRetriever¶
class langchain_community.retrievers.weaviate_hybrid_search.WeaviateHybridSearchRetriever[source]¶
Bases: BaseRetriever
Weaviate hybrid search retriever.
See the documentation:https://weaviate.io/blog/hybrid-search-explained
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param alpha: float = 0.5¶
The weight of the text key in the hybrid search.
param attributes: List[str] [Required]¶
The attributes to return in the results.
param client: Any = None¶
keyword arguments to pass to the Weaviate client.
param create_schema_if_missing: bool = True¶
Whether to create the schema if it doesn’t exist.
param index_name: str [Required]¶
The name of the index to use.
param k: int = 4¶
The number of results to return.
param metadata: Optional[Dict[str, Any]] = None¶
Optional metadata associated with the retriever. Defaults to None
This metadata will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a retriever with its
use case.
param tags: Optional[List[str]] = None¶
Optional list of tags associated with the retriever. Defaults to None
These tags will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
You can use these to eg identify a specific instance of a retriever with its
use case.
param text_key: str [Required]¶
use case.
param text_key: str [Required]¶
The name of the text key to use.
async abatch(inputs: List[Input], config: Optional[Union[RunnableConfig, List[RunnableConfig]]] = None, *, return_exceptions: bool = False, **kwargs: Optional[Any]) → List[Output]¶
Default implementation runs ainvoke in parallel using asyncio.gather.
The default implementation of batch works well for IO bound runnables.
Subclasses should override this method if they can batch more efficiently;
e.g., if the underlying runnable uses an API which supports a batch mode.
add_documents(docs: List[Document], **kwargs: Any) → List[str][source]¶
Upload documents to Weaviate.
async aget_relevant_documents(query: str, *, callbacks: Callbacks = None, tags: Optional[List[str]] = None, metadata: Optional[Dict[str, Any]] = None, run_name: Optional[str] = None, **kwargs: Any) → List[Document]¶
Asynchronously get documents relevant to a query.
:param query: string to find relevant documents for
:param callbacks: Callback manager or list of callbacks
:param tags: Optional list of tags associated with the retriever. Defaults to None
These tags will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
Parameters
metadata – Optional metadata associated with the retriever. Defaults to None
This metadata will be associated with each call to this retriever,
and passed as arguments to the handlers defined in callbacks.
Returns
List of relevant documents
async ainvoke(input: str, config: Optional[RunnableConfig] = None, **kwargs: Optional[Any]) → List[Document]¶
Default implementation of ainvoke, calls invoke from a thread.
The default implementation allows usage of async code even if
langchain_community.document_transformers.doctran_text_translate.DoctranTextTranslator¶
class langchain_community.document_transformers.doctran_text_translate.DoctranTextTranslator(openai_api_key: Optional[str] = None, language: str = 'english', openai_api_model: Optional[str] = None)[source]¶
Translate text documents using doctran.
Parameters
openai_api_key – OpenAI API key. Can also be specified via environment variable
OPENAI_API_KEY. – 
language – The language to translate to.
Example
from langchain_community.document_transformers import DoctranTextTranslator
# Pass in openai_api_key or set env var OPENAI_API_KEY
qa_translator = DoctranTextTranslator(language=”spanish”)
translated_document = await qa_translator.atransform_documents(documents)
Methods
__init__([openai_api_key, language, ...])
atransform_documents(documents, **kwargs)
Translates text documents using doctran.
transform_documents(documents, **kwargs)
Transform a list of documents.
__init__(openai_api_key: Optional[str] = None, language: str = 'english', openai_api_model: Optional[str] = None) → None[source]¶
async atransform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document][source]¶
Translates text documents using doctran.
transform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document][source]¶
Transform a list of documents.
Parameters
documents – A sequence of Documents to be transformed.
Returns
A list of transformed Documents.
Examples using DoctranTextTranslator¶
Doctran Translate Documents
langchain_community.document_transformers.google_translate.GoogleTranslateTransformer¶
class langchain_community.document_transformers.google_translate.GoogleTranslateTransformer(project_id: str, *, location: str = 'global', model_id: Optional[str] = None, glossary_id: Optional[str] = None, api_endpoint: Optional[str] = None)[source]¶
Translate text documents using Google Cloud Translation.
Parameters
project_id – Google Cloud Project ID.
location – (Optional) Translate model location.
model_id – (Optional) Translate model ID to use.
glossary_id – (Optional) Translate glossary ID to use.
api_endpoint – (Optional) Regional endpoint to use.
Methods
__init__(project_id, *[, location, ...])
param project_id
Google Cloud Project ID.
atransform_documents(documents, **kwargs)
Asynchronously transform a list of documents.
transform_documents(documents, **kwargs)
Translate text documents using Google Translate.
__init__(project_id: str, *, location: str = 'global', model_id: Optional[str] = None, glossary_id: Optional[str] = None, api_endpoint: Optional[str] = None) → None[source]¶
Parameters
project_id – Google Cloud Project ID.
location – (Optional) Translate model location.
model_id – (Optional) Translate model ID to use.
glossary_id – (Optional) Translate glossary ID to use.
api_endpoint – (Optional) Regional endpoint to use.
async atransform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document]¶
Asynchronously transform a list of documents.
Parameters
documents – A sequence of Documents to be transformed.
Returns
A list of transformed Documents.
transform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document][source]¶
Translate text documents using Google Translate.
Parameters
source_language_code – ISO 639 language code of the input document.
target_language_code – ISO 639 language code of the output document.
For supported languages, refer to:
https://cloud.google.com/translate/docs/languages
mime_type – (Optional) Media Type of input text.
Options: text/plain, text/html
Examples using GoogleTranslateTransformer¶
Google Cloud Translation
langchain_community.document_transformers.html2text.Html2TextTransformer¶
class langchain_community.document_transformers.html2text.Html2TextTransformer(ignore_links: bool = True, ignore_images: bool = True)[source]¶
Replace occurrences of a particular search pattern with a replacement string
Parameters
ignore_links – Whether links should be ignored; defaults to True.
ignore_images – Whether images should be ignored; defaults to True.
Example
Methods
__init__([ignore_links, ignore_images])
atransform_documents(documents, **kwargs)
Asynchronously transform a list of documents.
transform_documents(documents, **kwargs)
Transform a list of documents.
__init__(ignore_links: bool = True, ignore_images: bool = True) → None[source]¶
async atransform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document][source]¶
Asynchronously transform a list of documents.
Parameters
documents – A sequence of Documents to be transformed.
Returns
A list of transformed Documents.
transform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document][source]¶
Transform a list of documents.
Parameters
documents – A sequence of Documents to be transformed.
Returns
A list of transformed Documents.
Examples using Html2TextTransformer¶
html2text
Async Chromium
Set env var OPENAI_API_KEY or load from a .env file:
langchain_community.document_transformers.embeddings_redundant_filter.EmbeddingsRedundantFilter¶
class langchain_community.document_transformers.embeddings_redundant_filter.EmbeddingsRedundantFilter[source]¶
Bases: BaseDocumentTransformer, BaseModel
Filter that drops redundant documents by comparing their embeddings.
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
param embeddings: langchain_core.embeddings.Embeddings [Required]¶
Embeddings to use for embedding document contents.
param similarity_fn: Callable = <function cosine_similarity>¶
Similarity function for comparing documents. Function expected to take as input
two matrices (List[List[float]]) and return a matrix of scores where higher values
indicate greater similarity.
param similarity_threshold: float = 0.95¶
Threshold for determining when two documents are similar enough
to be considered redundant.
async atransform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document]¶
Asynchronously transform a list of documents.
Parameters
documents – A sequence of Documents to be transformed.
Returns
A list of transformed Documents.
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep – set to True to make a deep copy of the model
Returns
new model instance
dict(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False) → DictStrAny¶
Generate a dictionary representation of the model, optionally specifying which fields to include or exclude.
classmethod from_orm(obj: Any) → Model¶
json(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, by_alias: bool = False, skip_defaults: Optional[bool] = None, exclude_unset: bool = False, exclude_defaults: bool = False, exclude_none: bool = False, encoder: Optional[Callable[[Any], Any]] = None, models_as_dict: bool = True, **dumps_kwargs: Any) → unicode¶
Generate a JSON representation of the model, include and exclude arguments as per dict().
encoder is an optional function to supply as default to json.dumps(), other arguments as per json.dumps().
classmethod parse_file(path: Union[str, Path], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod parse_obj(obj: Any) → Model¶
classmethod parse_raw(b: Union[str, bytes], *, content_type: unicode = None, encoding: unicode = 'utf8', proto: Protocol = None, allow_pickle: bool = False) → Model¶
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
transform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document][source]¶
Filter down documents.
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
classmethod validate(value: Any) → Model¶
Examples using EmbeddingsRedundantFilter¶
LOTR (Merger Retriever)
langchain_community.document_transformers.embeddings_redundant_filter.get_stateful_documents¶
langchain_community.document_transformers.embeddings_redundant_filter.get_stateful_documents(documents: Sequence[Document]) → Sequence[_DocumentWithState][source]¶
Convert a list of documents to a list of documents with state.
Parameters
documents – The documents to convert.
Returns
A list of documents with state.
langchain_community.document_transformers.long_context_reorder.LongContextReorder¶
class langchain_community.document_transformers.long_context_reorder.LongContextReorder[source]¶
Bases: BaseDocumentTransformer, BaseModel
Lost in the middle:
Performance degrades when models must access relevant information
in the middle of long contexts.
See: https://arxiv.org/abs//2307.03172
Create a new model by parsing and validating input data from keyword arguments.
Raises ValidationError if the input data cannot be parsed to form a valid model.
async atransform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document][source]¶
Asynchronously transform a list of documents.
Parameters
documents – A sequence of Documents to be transformed.
Returns
A list of transformed Documents.
classmethod construct(_fields_set: Optional[SetStr] = None, **values: Any) → Model¶
Creates a new model setting __dict__ and __fields_set__ from trusted or pre-validated data.
Default values are respected, but no other validation is performed.
Behaves as if Config.extra = ‘allow’ was set since it adds all passed values
copy(*, include: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, exclude: Optional[Union[AbstractSetIntStr, MappingIntStrAny]] = None, update: Optional[DictStrAny] = None, deep: bool = False) → Model¶
Duplicate a model, optionally choose which fields to include, exclude and change.
Parameters
include – fields to include in new model
exclude – fields to exclude from new model, as with values this takes precedence over include
update – values to change/add in the new model. Note: the data is not validated before creating
the new model: you should trust this data
deep – set to True to make a deep copy of the model
classmethod schema(by_alias: bool = True, ref_template: unicode = '#/definitions/{model}') → DictStrAny¶
classmethod schema_json(*, by_alias: bool = True, ref_template: unicode = '#/definitions/{model}', **dumps_kwargs: Any) → unicode¶
transform_documents(documents: Sequence[Document], **kwargs: Any) → Sequence[Document][source]¶
Reorders documents.
classmethod update_forward_refs(**localns: Any) → None¶
Try to update ForwardRefs on fields based on this Model, globalns and localns.
classmethod validate(value: Any) → Model¶
Examples using LongContextReorder¶
LOTR (Merger Retriever)
Lost in the middle: The problem with long contexts