viruPVS/langchain-docs huggingface dataset:
A parser for the output of the CriteriaEvalChain.
evaluation.criteria.eval_chain.LabeledCriteriaEvalChain
Criteria evaluation chain that requires references.
evaluation.embedding_distance.base.EmbeddingDistance(value)
Embedding Distance Metric.
evaluation.embedding_distance.base.EmbeddingDistanceEvalChain
Use embedding distances to score semantic difference between a prediction and reference.
evaluation.embedding_distance.base.PairwiseEmbeddingDistanceEvalChain
Use embedding distances to score semantic difference between two predictions.
evaluation.exact_match.base.ExactMatchStringEvaluator(*)
Compute an exact match between the prediction and the reference.
evaluation.parsing.base.JsonEqualityEvaluator([...])
Evaluates whether the prediction is equal to the reference after
evaluation.parsing.base.JsonValidityEvaluator(...)
Evaluates whether the prediction is valid JSON.
evaluation.parsing.json_distance.JsonEditDistanceEvaluator([...])
An evaluator that calculates the edit distance between JSON strings.
evaluation.parsing.json_schema.JsonSchemaEvaluator(...)
An evaluator that validates a JSON prediction against a JSON schema reference.
evaluation.qa.eval_chain.ContextQAEvalChain
LLM Chain for evaluating QA w/o GT based on context
evaluation.qa.eval_chain.CotQAEvalChain
LLM Chain for evaluating QA using chain of thought reasoning.
evaluation.qa.eval_chain.QAEvalChain
LLM Chain for evaluating question answering.
evaluation.qa.generate_chain.QAGenerateChain
LLM Chain for generating examples for question answering.
evaluation.regex_match.base.RegexMatchStringEvaluator(*)
Compute a regex match between the prediction and the reference.
evaluation.schema.AgentTrajectoryEvaluator()
Interface for evaluating agent trajectories.
evaluation.schema.EvaluatorType(value[, ...])
The types of the evaluators.
evaluation.schema.LLMEvalChain
A base class for evaluators that use an LLM.
evaluation.schema.PairwiseStringEvaluator()
evaluation.schema.PairwiseStringEvaluator()
Compare the output of two models (or two outputs of the same model).
evaluation.schema.StringEvaluator()
Grade, tag, or otherwise evaluate predictions relative to their inputs and/or reference labels.
evaluation.scoring.eval_chain.LabeledScoreStringEvalChain
A chain for scoring the output of a model on a scale of 1-10.
evaluation.scoring.eval_chain.ScoreStringEvalChain
A chain for scoring on a scale of 1-10 the output of a model.
evaluation.scoring.eval_chain.ScoreStringResultOutputParser
A parser for the output of the ScoreStringEvalChain.
evaluation.string_distance.base.PairwiseStringDistanceEvalChain
Compute string edit distances between two predictions.
evaluation.string_distance.base.StringDistance(value)
Distance metric to use.
evaluation.string_distance.base.StringDistanceEvalChain
Compute string distances between the prediction and the reference.
Functions¶
evaluation.comparison.eval_chain.resolve_pairwise_criteria(...)
Resolve the criteria for the pairwise evaluator.
evaluation.criteria.eval_chain.resolve_criteria(...)
Resolve the criteria to evaluate.
evaluation.loading.load_dataset(uri)
Load a dataset from the LangChainDatasets on HuggingFace.
evaluation.loading.load_evaluator(evaluator, *)
Load the requested evaluation chain specified by a string.
evaluation.loading.load_evaluators(evaluators, *)
Load evaluators specified by a list of evaluator types.
evaluation.scoring.eval_chain.resolve_criteria(...)
Resolve the criteria for the pairwise evaluator.
langchain.hub¶
Interface with the LangChain Hub.
Functions¶
hub.pull(owner_repo_commit, *[, api_url, ...])
Pulls an object from the hub and returns it as a LangChain object.
hub.push(repo_full_name, object, *[, ...])
hub.push(repo_full_name, object, *[, ...])
Pushes an object to the hub and returns the URL it can be viewed at in a browser.
langchain.indexes¶
Code to support various indexing workflows.
Provides code to:
Create knowledge graphs from data.
Support indexing workflows from LangChain data loaders to vectorstores.
For indexing workflows, this code is used to avoid writing duplicated content
into the vectostore and to avoid over-writing content if it’s unchanged.
Importantly, this keeps on working even if the content being written is derived
via a set of transformations from some source content (e.g., indexing children
documents that were derived from parent documents by chunking.)
Classes¶
indexes.base.RecordManager(namespace)
An abstract base class representing the interface for a record manager.
indexes.graph.GraphIndexCreator
Functionality to create graph index.
indexes.vectorstore.VectorStoreIndexWrapper
Wrapper around a vectorstore for easy access.
indexes.vectorstore.VectorstoreIndexCreator
Logic for creating indexes.
Functions¶
langchain.memory¶
Memory maintains Chain state, incorporating context from past runs.
Class hierarchy for Memory:
BaseMemory --> BaseChatMemory --> <name>Memory  # Examples: ZepMemory, MotorheadMemory
Main helpers:
BaseChatMessageHistory
Chat Message History stores the chat message history in different stores.
Class hierarchy for ChatMessageHistory:
BaseChatMessageHistory --> <name>ChatMessageHistory  # Example: ZepChatMessageHistory
Main helpers:
AIMessage, BaseMessage, HumanMessage
Classes¶
memory.buffer.ConversationBufferMemory
Buffer for storing conversation memory.
memory.buffer.ConversationStringBufferMemory
Buffer for storing conversation memory.
memory.buffer_window.ConversationBufferWindowMemory
Buffer for storing conversation memory inside a limited size window.
memory.chat_memory.BaseChatMemory
Buffer for storing conversation memory inside a limited size window.
memory.chat_memory.BaseChatMemory
Abstract base class for chat memory.
memory.combined.CombinedMemory
Combining multiple memories' data together.
memory.entity.BaseEntityStore
Abstract base class for Entity store.
memory.entity.ConversationEntityMemory
Entity extractor & summarizer memory.
memory.entity.InMemoryEntityStore
In-memory Entity store.
memory.entity.RedisEntityStore
Redis-backed Entity store.
memory.entity.SQLiteEntityStore
SQLite-backed Entity store
memory.entity.UpstashRedisEntityStore
Upstash Redis backed Entity store.
memory.kg.ConversationKGMemory
Knowledge graph conversation memory.
memory.motorhead_memory.MotorheadMemory
Chat message memory backed by Motorhead service.
memory.readonly.ReadOnlySharedMemory
A memory wrapper that is read-only and cannot be changed.
memory.simple.SimpleMemory
Simple memory for storing context or other information that shouldn't ever change between prompts.
memory.summary.ConversationSummaryMemory
Conversation summarizer to chat memory.
memory.summary.SummarizerMixin
Mixin for summarizer.
memory.summary_buffer.ConversationSummaryBufferMemory
Buffer with summarizer for storing conversation memory.
memory.token_buffer.ConversationTokenBufferMemory
Conversation chat memory with token limit.
memory.vectorstore.VectorStoreRetrieverMemory
VectorStoreRetriever-backed memory.
memory.zep_memory.ZepMemory
Persist your chain history to the Zep MemoryStore.
Functions¶
memory.utils.get_prompt_input_key(inputs, ...)
Get the prompt input key.
langchain.model_laboratory¶
Experiment with different models.
Classes¶
model_laboratory.ModelLaboratory(chains[, names])
Experiment with different models.
langchain.output_parsers¶
OutputParser classes parse the output of an LLM call.
Class hierarchy:
OutputParser classes parse the output of an LLM call.
Class hierarchy:
BaseLLMOutputParser --> BaseOutputParser --> <name>OutputParser  # ListOutputParser, PydanticOutputParser
Main helpers:
Serializable, Generation, PromptValue
Classes¶
output_parsers.boolean.BooleanOutputParser
Parse the output of an LLM call to a boolean.
output_parsers.combining.CombiningOutputParser
Combine multiple output parsers into one.
output_parsers.datetime.DatetimeOutputParser
Parse the output of an LLM call to a datetime.
output_parsers.enum.EnumOutputParser
Parse an output that is one of a set of values.
output_parsers.ernie_functions.JsonKeyOutputFunctionsParser
Parse an output as the element of the Json object.
output_parsers.ernie_functions.JsonOutputFunctionsParser
Parse an output as the Json object.
output_parsers.ernie_functions.OutputFunctionsParser
Parse an output that is one of sets of values.
output_parsers.ernie_functions.PydanticAttrOutputFunctionsParser
Parse an output as an attribute of a pydantic object.
output_parsers.ernie_functions.PydanticOutputFunctionsParser
Parse an output as a pydantic object.
output_parsers.fix.OutputFixingParser
Wraps a parser and tries to fix parsing errors.
output_parsers.json.SimpleJsonOutputParser
Parse the output of an LLM call to a JSON object.
output_parsers.openai_functions.JsonKeyOutputFunctionsParser
Parse an output as the element of the Json object.
output_parsers.openai_functions.JsonOutputFunctionsParser
Parse an output as the Json object.
output_parsers.openai_functions.OutputFunctionsParser
Parse an output that is one of sets of values.
output_parsers.openai_functions.PydanticAttrOutputFunctionsParser
output_parsers.openai_functions.PydanticAttrOutputFunctionsParser
Parse an output as an attribute of a pydantic object.
output_parsers.openai_functions.PydanticOutputFunctionsParser
Parse an output as a pydantic object.
output_parsers.openai_tools.JsonOutputKeyToolsParser
Parse tools from OpenAI response.
output_parsers.openai_tools.JsonOutputToolsParser
Parse tools from OpenAI response.
output_parsers.openai_tools.PydanticToolsParser
Parse tools from OpenAI response.
output_parsers.pandas_dataframe.PandasDataFrameOutputParser
Parse an output using Pandas DataFrame format.
output_parsers.pydantic.PydanticOutputParser
Parse an output using a pydantic model.
output_parsers.rail_parser.GuardrailsOutputParser
Parse the output of an LLM call using Guardrails.
output_parsers.regex.RegexParser
Parse the output of an LLM call using a regex.
output_parsers.regex_dict.RegexDictParser
Parse the output of an LLM call into a Dictionary using a regex.
output_parsers.retry.RetryOutputParser
Wraps a parser and tries to fix parsing errors.
output_parsers.retry.RetryWithErrorOutputParser
Wraps a parser and tries to fix parsing errors.
output_parsers.structured.ResponseSchema
A schema for a response from a structured output parser.
output_parsers.structured.StructuredOutputParser
Parse the output of an LLM call to a structured output.
output_parsers.xml.XMLOutputParser
Parse an output using xml format.
output_parsers.yaml.YamlOutputParser
Parse YAML output using a pydantic model.
Functions¶
output_parsers.json.parse_and_check_json_markdown(...)
Parse a JSON string from a Markdown string and check that it contains the expected keys.
output_parsers.json.parse_json_markdown(...)
output_parsers.json.parse_json_markdown(...)
Parse a JSON string from a Markdown string.
output_parsers.json.parse_partial_json(s, *)
Parse a JSON string that may be missing closing braces.
output_parsers.loading.load_output_parser(config)
Load an output parser.
langchain.prompts¶
Prompt is the input to the model.
Prompt is often constructed
from multiple components. Prompt classes and functions make constructing
and working with prompts easy.
Class hierarchy:
BasePromptTemplate --> PipelinePromptTemplate
                       StringPromptTemplate --> PromptTemplate
                                                FewShotPromptTemplate
                                                FewShotPromptWithTemplates
                       BaseChatPromptTemplate --> AutoGPTPrompt
                                                  ChatPromptTemplate --> AgentScratchPadChatPromptTemplate
BaseMessagePromptTemplate --> MessagesPlaceholder
                              BaseStringMessagePromptTemplate --> ChatMessagePromptTemplate
                                                                  HumanMessagePromptTemplate
                                                                  AIMessagePromptTemplate
                                                                  SystemMessagePromptTemplate
PromptValue --> StringPromptValue
                ChatPromptValue
Classes¶
prompts.example_selector.ngram_overlap.NGramOverlapExampleSelector
Select and order examples based on ngram overlap score (sentence_bleu score).
Functions¶
prompts.example_selector.ngram_overlap.ngram_overlap_score(...)
Compute ngram overlap score of source and example as sentence_bleu score.
langchain.retrievers¶
Retriever class returns Documents given a text query.
It is more general than a vector store. A retriever does not need to be able to
store documents, only to return (or retrieve) it. Vector stores can be used as
the backbone of a retriever, but there are other types of retrievers as well.
Class hierarchy:
BaseRetriever --> <name>Retriever  # Examples: ArxivRetriever, MergerRetriever
Main helpers:
Document, Serializable, Callbacks,
Main helpers:
Document, Serializable, Callbacks,
CallbackManagerForRetrieverRun, AsyncCallbackManagerForRetrieverRun
Classes¶
retrievers.contextual_compression.ContextualCompressionRetriever
Retriever that wraps a base retriever and compresses the results.
retrievers.document_compressors.base.BaseDocumentCompressor
Base class for document compressors.
retrievers.document_compressors.base.DocumentCompressorPipeline
Document compressor that uses a pipeline of Transformers.
retrievers.document_compressors.chain_extract.LLMChainExtractor
Document compressor that uses an LLM chain to extract the relevant parts of documents.
retrievers.document_compressors.chain_extract.NoOutputParser
Parse outputs that could return a null string of some sort.
retrievers.document_compressors.chain_filter.LLMChainFilter
Filter that drops documents that aren't relevant to the query.
retrievers.document_compressors.cohere_rerank.CohereRerank
Document compressor that uses Cohere Rerank API.
retrievers.document_compressors.embeddings_filter.EmbeddingsFilter
Document compressor that uses embeddings to drop documents unrelated to the query.
retrievers.ensemble.EnsembleRetriever
Retriever that ensembles the multiple retrievers.
retrievers.merger_retriever.MergerRetriever
Retriever that merges the results of multiple retrievers.
retrievers.multi_query.LineList
List of lines.
retrievers.multi_query.LineListOutputParser
Output parser for a list of lines.
retrievers.multi_query.MultiQueryRetriever
Given a query, use an LLM to write a set of queries.
retrievers.multi_vector.MultiVectorRetriever
Retrieve from a set of multiple embeddings for the same document.
retrievers.multi_vector.SearchType(value[, ...])
Enumerator of the types of search to perform.
Enumerator of the types of search to perform.
retrievers.parent_document_retriever.ParentDocumentRetriever
Retrieve small chunks then retrieve their parent documents.
retrievers.re_phraser.RePhraseQueryRetriever
Given a query, use an LLM to re-phrase it.
retrievers.self_query.base.SelfQueryRetriever
Retriever that uses a vector store and an LLM to generate the vector store queries.
retrievers.self_query.chroma.ChromaTranslator()
Translate Chroma internal query language elements to valid filters.
retrievers.self_query.dashvector.DashvectorTranslator()
Logic for converting internal query language elements to valid filters.
retrievers.self_query.deeplake.DeepLakeTranslator()
Translate DeepLake internal query language elements to valid filters.
retrievers.self_query.elasticsearch.ElasticsearchTranslator()
Translate Elasticsearch internal query language elements to valid filters.
retrievers.self_query.milvus.MilvusTranslator()
Translate Milvus internal query language elements to valid filters.
retrievers.self_query.mongodb_atlas.MongoDBAtlasTranslator()
Translate Mongo internal query language elements to valid filters.
retrievers.self_query.myscale.MyScaleTranslator([...])
Translate MyScale internal query language elements to valid filters.
retrievers.self_query.opensearch.OpenSearchTranslator()
Translate OpenSearch internal query domain-specific language elements to valid filters.
retrievers.self_query.pinecone.PineconeTranslator()
Translate Pinecone internal query language elements to valid filters.
retrievers.self_query.qdrant.QdrantTranslator(...)
Translate Qdrant internal query language elements to valid filters.
retrievers.self_query.redis.RedisTranslator(schema)
Translate
retrievers.self_query.supabase.SupabaseVectorTranslator()
Translate Langchain filters to Supabase PostgREST filters.
Translate Langchain filters to Supabase PostgREST filters.
retrievers.self_query.timescalevector.TimescaleVectorTranslator()
Translate the internal query language elements to valid filters.
retrievers.self_query.vectara.VectaraTranslator()
Translate Vectara internal query language elements to valid filters.
retrievers.self_query.weaviate.WeaviateTranslator()
Translate Weaviate internal query language elements to valid filters.
retrievers.time_weighted_retriever.TimeWeightedVectorStoreRetriever
Retriever that combines embedding similarity with recency in retrieving values.
retrievers.web_research.LineList
List of questions.
retrievers.web_research.QuestionListOutputParser
Output parser for a list of numbered questions.
retrievers.web_research.SearchQueries
Search queries to research for the user's goal.
retrievers.web_research.WebResearchRetriever
Google Search API retriever.
Functions¶
retrievers.document_compressors.chain_extract.default_get_input(...)
Return the compression chain input.
retrievers.document_compressors.chain_filter.default_get_input(...)
Return the compression chain input.
retrievers.self_query.deeplake.can_cast_to_float(string)
Check if a string can be cast to a float.
retrievers.self_query.milvus.process_value(value)
Convert a value to a string and add double quotes if it is a string.
retrievers.self_query.vectara.process_value(value)
Convert a value to a string and add single quotes if it is a string.
langchain.runnables¶
Classes¶
runnables.hub.HubRunnable
An instance of a runnable stored in the LangChain Hub.
runnables.openai_functions.OpenAIFunction
A function description for ChatOpenAI
runnables.openai_functions.OpenAIFunctionsRouter
runnables.openai_functions.OpenAIFunctionsRouter
A runnable that routes to the selected function.
langchain.smith¶
LangSmith utilities.
This module provides utilities for connecting to LangSmith. For more information on LangSmith, see the LangSmith documentation.
Evaluation
LangSmith helps you evaluate Chains and other language model application components using a number of LangChain evaluators.
An example of this is shown below, assuming you’ve created a LangSmith dataset called <my_dataset_name>:
from langsmith import Client
from langchain.chat_models import ChatOpenAI
from langchain.chains import LLMChain
from langchain.smith import RunEvalConfig, run_on_dataset
# Chains may have memory. Passing in a constructor function lets the
# evaluation framework avoid cross-contamination between runs.
def construct_chain():
    llm = ChatOpenAI(temperature=0)
    chain = LLMChain.from_string(
        llm,
        "What's the answer to {your_input_key}"
    )
    return chain
# Load off-the-shelf evaluators via config or the EvaluatorType (string or enum)
evaluation_config = RunEvalConfig(
    evaluators=[
        "qa",  # "Correctness" against a reference answer
        "embedding_distance",
        RunEvalConfig.Criteria("helpfulness"),
        RunEvalConfig.Criteria({
            "fifth-grader-score": "Do you have to be smarter than a fifth grader to answer this question?"
        }),
    ]
)
client = Client()
run_on_dataset(
    client,
    "<my_dataset_name>",
    construct_chain,
    evaluation=evaluation_config,
)
You can also create custom evaluators by subclassing the
StringEvaluator
or LangSmith’s RunEvaluator classes.
from typing import Optional
StringEvaluator
or LangSmith’s RunEvaluator classes.
from typing import Optional
from langchain.evaluation import StringEvaluator
class MyStringEvaluator(StringEvaluator):
    @property
    def requires_input(self) -> bool:
        return False
    @property
    def requires_reference(self) -> bool:
        return True
    @property
    def evaluation_name(self) -> str:
        return "exact_match"
    def _evaluate_strings(self, prediction, reference=None, input=None, **kwargs) -> dict:
        return {"score": prediction == reference}
evaluation_config = RunEvalConfig(
    custom_evaluators = [MyStringEvaluator()],
)
run_on_dataset(
    client,
    "<my_dataset_name>",
    construct_chain,
    evaluation=evaluation_config,
)
Primary Functions
arun_on_dataset: Asynchronous function to evaluate a chain, agent, or other LangChain component over a dataset.
run_on_dataset: Function to evaluate a chain, agent, or other LangChain component over a dataset.
RunEvalConfig: Class representing the configuration for running evaluation. You can select evaluators by EvaluatorType or config, or you can pass in custom_evaluators
Classes¶
smith.evaluation.config.EvalConfig
Configuration for a given run evaluator.
smith.evaluation.config.RunEvalConfig
Configuration for a run evaluation.
smith.evaluation.config.SingleKeyEvalConfig
Create a new model by parsing and validating input data from keyword arguments.
smith.evaluation.progress.ProgressBarCallback(total)
A simple progress bar for the console.
smith.evaluation.runner_utils.EvalError(...)
Your architecture raised an error.
smith.evaluation.runner_utils.InputFormatError
Raised when the input format is invalid.
smith.evaluation.runner_utils.TestResult
A dictionary of the results of a single test run.
smith.evaluation.runner_utils.TestResult
A dictionary of the results of a single test run.
smith.evaluation.string_run_evaluator.ChainStringRunMapper
Extract items to evaluate from the run object from a chain.
smith.evaluation.string_run_evaluator.LLMStringRunMapper
Extract items to evaluate from the run object.
smith.evaluation.string_run_evaluator.StringExampleMapper
Map an example, or row in the dataset, to the inputs of an evaluation.
smith.evaluation.string_run_evaluator.StringRunEvaluatorChain
Evaluate Run and optional examples.
smith.evaluation.string_run_evaluator.StringRunMapper
Extract items to evaluate from the run object.
smith.evaluation.string_run_evaluator.ToolStringRunMapper
Map an input to the tool.
Functions¶
smith.evaluation.name_generation.random_name()
Generate a random name.
smith.evaluation.runner_utils.arun_on_dataset(...)
Run the Chain or language model on a dataset and store traces to the specified project name.
smith.evaluation.runner_utils.run_on_dataset(...)
Run the Chain or language model on a dataset and store traces to the specified project name.
langchain.storage¶
Implementations of key-value stores and storage helpers.
Module provides implementations of various key-value stores that conform
to a simple key-value interface.
The primary goal of these storages is to support implementation of caching.
Classes¶
storage.encoder_backed.EncoderBackedStore(...)
Wraps a store with key and value encoders/decoders.
storage.file_system.LocalFileStore(root_path)
BaseStore interface that works on the local file system.
storage.in_memory.InMemoryBaseStore()
In-memory implementation of the BaseStore using a dictionary.
langchain.text_splitter¶
Text Splitters are classes for splitting text.
Class hierarchy:
BaseDocumentTransformer --> TextSplitter --> <name>TextSplitter  # Example: CharacterTextSplitter