!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

You are viewing the **preview** v0.2 docs. View the **stable** v0.1 docs [here](/v0.1/docs/get_started/introduction/). Leave feedback on the v0.2 docs [here](https://github.com/langchain-ai/langchainjs/discussions/5386).

[

![🦜️🔗 Langchain](/v0.2/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.2/img/brand/wordmark-dark.png)

](/v0.2/)[Integrations](/v0.2/docs/integrations/platforms/)[API Reference](https://v02.api.js.langchain.com)

[More](#)

*   [People](/v0.2/docs/people/)
*   [Community](/v0.2/docs/community)
*   [Tutorials](/v0.2/docs/additional_resources/tutorials)
*   [Contributing](/v0.2/docs/contributing)

[v0.2](#)

*   [v0.2](/v0.2/docs/introduction)
*   [v0.1](https://js.langchain.com/v0.1/docs/get_started/introduction)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Introduction](/v0.2/docs/introduction)
*   [Tutorials](/v0.2/docs/tutorials/)
    
    *   [Build a Question Answering application over a Graph Database](/v0.2/docs/tutorials/graph)
    *   [Tutorials](/v0.2/docs/tutorials/)
    *   [Build a Simple LLM Application](/v0.2/docs/tutorials/llm_chain)
    *   [Build a Query Analysis System](/v0.2/docs/tutorials/query_analysis)
    *   [Build a Chatbot](/v0.2/docs/tutorials/chatbot)
    *   [Build an Agent](/v0.2/docs/tutorials/agents)
    *   [Build an Extraction Chain](/v0.2/docs/tutorials/extraction)
    *   [Summarize Text](/v0.2/docs/tutorials/summarization)
    *   [Tagging](/v0.2/docs/tutorials/classification)
    *   [Build a Local RAG Application](/v0.2/docs/tutorials/local_rag)
    *   [Conversational RAG](/v0.2/docs/tutorials/qa_chat_history)
    *   [Build a Retrieval Augmented Generation (RAG) App](/v0.2/docs/tutorials/rag)
    *   [Build a Question/Answering system over SQL data](/v0.2/docs/tutorials/sql_qa)
*   [How-to guides](/v0.2/docs/how_to/)
    
    *   [How-to guides](/v0.2/docs/how_to/)
    *   [How to use example selectors](/v0.2/docs/how_to/example_selectors)
    *   [Installation](/v0.2/docs/how_to/installation)
    *   [How to stream responses from an LLM](/v0.2/docs/how_to/streaming_llm)
    *   [How to stream chat model responses](/v0.2/docs/how_to/chat_streaming)
    *   [How to embed text data](/v0.2/docs/how_to/embed_text)
    *   [How to use few shot examples in chat models](/v0.2/docs/how_to/few_shot_examples_chat)
    *   [How to cache model responses](/v0.2/docs/how_to/llm_caching)
    *   [How to cache chat model responses](/v0.2/docs/how_to/chat_model_caching)
    *   [How to create a custom LLM class](/v0.2/docs/how_to/custom_llm)
    *   [How to use few shot examples](/v0.2/docs/how_to/few_shot_examples)
    *   [How to use output parsers to parse an LLM response into structured format](/v0.2/docs/how_to/output_parser_structured)
    *   [How to return structured data from a model](/v0.2/docs/how_to/structured_output)
    *   [How to add ad-hoc tool calling capability to LLMs and Chat Models](/v0.2/docs/how_to/tools_prompting)
    *   [How to create a custom chat model class](/v0.2/docs/how_to/custom_chat)
    *   [How to do per-user retrieval](/v0.2/docs/how_to/qa_per_user)
    *   [How to track token usage](/v0.2/docs/how_to/chat_token_usage_tracking)
    *   [How to track token usage](/v0.2/docs/how_to/llm_token_usage_tracking)
    *   [How to pass through arguments from one step to the next](/v0.2/docs/how_to/passthrough)
    *   [How to compose prompts together](/v0.2/docs/how_to/prompts_composition)
    *   [How to use legacy LangChain Agents (AgentExecutor)](/v0.2/docs/how_to/agent_executor)
    *   [How to add values to a chain's state](/v0.2/docs/how_to/assign)
    *   [How to attach runtime arguments to a Runnable](/v0.2/docs/how_to/binding)
    *   [How to cache embedding results](/v0.2/docs/how_to/caching_embeddings)
    *   [How to split by character](/v0.2/docs/how_to/character_text_splitter)
    *   [How to manage memory](/v0.2/docs/how_to/chatbots_memory)
    *   [How to do retrieval](/v0.2/docs/how_to/chatbots_retrieval)
    *   [How to use tools](/v0.2/docs/how_to/chatbots_tools)
    *   [How to split code](/v0.2/docs/how_to/code_splitter)
    *   [How to do retrieval with contextual compression](/v0.2/docs/how_to/contextual_compression)
    *   [How to write a custom retriever class](/v0.2/docs/how_to/custom_retriever)
    *   [How to create custom Tools](/v0.2/docs/how_to/custom_tools)
    *   [How to debug your LLM apps](/v0.2/docs/how_to/debugging)
    *   [How to load CSV data](/v0.2/docs/how_to/document_loader_csv)
    *   [How to write a custom document loader](/v0.2/docs/how_to/document_loader_custom)
    *   [How to load data from a directory](/v0.2/docs/how_to/document_loader_directory)
    *   [How to load PDF files](/v0.2/docs/how_to/document_loader_pdf)
    *   [How to load JSON data](/v0.2/docs/how_to/document_loaders_json)
    *   [How to select examples by length](/v0.2/docs/how_to/example_selectors_length_based)
    *   [How to select examples by similarity](/v0.2/docs/how_to/example_selectors_similarity)
    *   [How to use reference examples](/v0.2/docs/how_to/extraction_examples)
    *   [How to handle long text](/v0.2/docs/how_to/extraction_long_text)
    *   [How to do extraction without using function calling](/v0.2/docs/how_to/extraction_parse)
    *   [Fallbacks](/v0.2/docs/how_to/fallbacks)
    *   [Few Shot Prompt Templates](/v0.2/docs/how_to/few_shot)
    *   [How to run custom functions](/v0.2/docs/how_to/functions)
    *   [How to construct knowledge graphs](/v0.2/docs/how_to/graph_constructing)
    *   [How to map values to a database](/v0.2/docs/how_to/graph_mapping)
    *   [How to improve results with prompting](/v0.2/docs/how_to/graph_prompting)
    *   [How to add a semantic layer over the database](/v0.2/docs/how_to/graph_semantic)
    *   [How to reindex data to keep your vectorstore in-sync with the underlying data source](/v0.2/docs/how_to/indexing)
    *   [How to get log probabilities](/v0.2/docs/how_to/logprobs)
    *   [How to add message history](/v0.2/docs/how_to/message_history)
    *   [How to generate multiple embeddings per document](/v0.2/docs/how_to/multi_vector)
    *   [How to generate multiple queries to retrieve data for](/v0.2/docs/how_to/multiple_queries)
    *   [How to parse JSON output](/v0.2/docs/how_to/output_parser_json)
    *   [How to retry when output parsing errors occur](/v0.2/docs/how_to/output_parser_retry)
    *   [How to parse XML output](/v0.2/docs/how_to/output_parser_xml)
    *   [How to invoke runnables in parallel](/v0.2/docs/how_to/parallel)
    *   [How to retrieve the whole document for a chunk](/v0.2/docs/how_to/parent_document_retriever)
    *   [How to partially format prompt templates](/v0.2/docs/how_to/prompts_partial)
    *   [How to add chat history to a question-answering chain](/v0.2/docs/how_to/qa_chat_history_how_to)
    *   [How to return citations](/v0.2/docs/how_to/qa_citations)
    *   [How to return sources](/v0.2/docs/how_to/qa_sources)
    *   [How to stream from a question-answering chain](/v0.2/docs/how_to/qa_streaming)
    *   [How to construct filters](/v0.2/docs/how_to/query_constructing_filters)
    *   [How to add examples to the prompt](/v0.2/docs/how_to/query_few_shot)
    *   [How to deal with high cardinality categorical variables](/v0.2/docs/how_to/query_high_cardinality)
    *   [How to handle multiple queries](/v0.2/docs/how_to/query_multiple_queries)
    *   [How to handle multiple retrievers](/v0.2/docs/how_to/query_multiple_retrievers)
    *   [How to handle cases where no queries are generated](/v0.2/docs/how_to/query_no_queries)
    *   [How to recursively split text by characters](/v0.2/docs/how_to/recursive_text_splitter)
    *   [How to reduce retrieval latency](/v0.2/docs/how_to/reduce_retrieval_latency)
    *   [How to route execution within a chain](/v0.2/docs/how_to/routing)
    *   [How to chain runnables](/v0.2/docs/how_to/sequence)
    *   [How to split text by tokens](/v0.2/docs/how_to/split_by_token)
    *   [How to deal with large databases](/v0.2/docs/how_to/sql_large_db)
    *   [How to use prompting to improve results](/v0.2/docs/how_to/sql_prompting)
    *   [How to do query validation](/v0.2/docs/how_to/sql_query_checking)
    *   [How to stream](/v0.2/docs/how_to/streaming)
    *   [How to create a time-weighted retriever](/v0.2/docs/how_to/time_weighted_vectorstore)
    *   [How to use a chat model to call tools](/v0.2/docs/how_to/tool_calling)
    *   [How to call tools with multi-modal data](/v0.2/docs/how_to/tool_calls_multi_modal)
    *   [How to use LangChain tools](/v0.2/docs/how_to/tools_builtin)
    *   [How use a vector store to retrieve data](/v0.2/docs/how_to/vectorstore_retriever)
    *   [How to create and query vector stores](/v0.2/docs/how_to/vectorstores)
*   [Conceptual guide](/v0.2/docs/concepts)
*   Ecosystem
    
    *   [🦜🛠️ LangSmith](/v0.2/docs/langsmith/)
        
    *   [🦜🕸️LangGraph.js](/v0.2/docs/langgraph)
*   Versions
    
    *   [Overview](/v0.2/docs/versions/overview)
    *   [v0.2](/v0.2/docs/versions/v0_2)
    *   [Release Policy](/v0.2/docs/versions/release_policy)
    *   [Packages](/v0.2/docs/versions/packages)
*   [Security](/v0.2/docs/security)

*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to handle long text

On this page

How to handle long text
=======================

Prerequisites

This guide assumes familiarity with the following:

*   [Extraction](/v0.2/docs/tutorials/extraction)

When working with files, like PDFs, you’re likely to encounter text that exceeds your language model’s context window. To process this text, consider these strategies:

1.  **Change LLM** Choose a different LLM that supports a larger context window.
2.  **Brute Force** Chunk the document, and extract content from each chunk.
3.  **RAG** Chunk the document, index the chunks, and only extract content from a subset of chunks that look “relevant”.

Keep in mind that these strategies have different trade offs and the best strategy likely depends on the application that you’re designing!

Set up[​](#set-up "Direct link to Set up")
------------------------------------------

First, let’s install some required dependencies:

tip

See [this section for general instructions on installing integration packages](/v0.2/docs/how_to/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/openai zod cheerio

    yarn add @langchain/openai zod cheerio

    pnpm add @langchain/openai zod cheerio

Next, we need some example data! Let’s download an article about [cars from Wikipedia](https://en.wikipedia.org/wiki/Car) and load it as a LangChain `Document`.

    import { CheerioWebBaseLoader } from "langchain/document_loaders/web/cheerio";// Only required in a Deno notebook environment to load the peer dep.import "cheerio";const loader = new CheerioWebBaseLoader("https://en.wikipedia.org/wiki/Car");const docs = await loader.load();docs[0].pageContent.length;

    97336

Define the schema[​](#define-the-schema "Direct link to Define the schema")
---------------------------------------------------------------------------

Here, we’ll define schema to extract key developments from the text.

    import { z } from "zod";import { ChatPromptTemplate } from "@langchain/core/prompts";import { ChatOpenAI } from "@langchain/openai";const keyDevelopmentSchema = z  .object({    year: z      .number()      .describe("The year when there was an important historic development."),    description: z      .string()      .describe("What happened in this year? What was the development?"),    evidence: z      .string()      .describe(        "Repeat verbatim the sentence(s) from which the year and description information were extracted"      ),  })  .describe("Information about a development in the history of cars.");const extractionDataSchema = z  .object({    key_developments: z.array(keyDevelopmentSchema),  })  .describe(    "Extracted information about key developments in the history of cars"  );const SYSTEM_PROMPT_TEMPLATE = [  "You are an expert at identifying key historic development in text.",  "Only extract important historic developments. Extract nothing if no important information can be found in the text.",].join("\n");// Define a custom prompt to provide instructions and any additional context.// 1) You can add examples into the prompt template to improve extraction quality// 2) Introduce additional parameters to take context into account (e.g., include metadata//    about the document from which the text was extracted.)const prompt = ChatPromptTemplate.fromMessages([  ["system", SYSTEM_PROMPT_TEMPLATE],  // Keep on reading through this use case to see how to use examples to improve performance  // MessagesPlaceholder('examples'),  ["human", "{text}"],]);// We will be using tool calling mode, which// requires a tool calling capable model.const llm = new ChatOpenAI({  model: "gpt-4-0125-preview",  temperature: 0,});const extractionChain = prompt.pipe(  llm.withStructuredOutput(extractionDataSchema));

Brute force approach[​](#brute-force-approach "Direct link to Brute force approach")
------------------------------------------------------------------------------------

Split the documents into chunks such that each chunk fits into the context window of the LLMs.

    import { TokenTextSplitter } from "langchain/text_splitter";const textSplitter = new TokenTextSplitter({  chunkSize: 2000,  chunkOverlap: 20,});// Note that this method takes an array of docsconst splitDocs = await textSplitter.splitDocuments(docs);

Use the `.batch` method present on all runnables to run the extraction in **parallel** across each chunk!

tip

You can often use `.batch()` to parallelize the extractions!

If your model is exposed via an API, this will likely speed up your extraction flow.

    // Limit just to the first 3 chunks// so the code can be re-run quicklyconst firstFewTexts = splitDocs.slice(0, 3).map((doc) => doc.pageContent);const extractionChainParams = firstFewTexts.map((text) => {  return { text };});const results = await extractionChain.batch(extractionChainParams, {  maxConcurrency: 5,});

### Merge results[​](#merge-results "Direct link to Merge results")

After extracting data from across the chunks, we’ll want to merge the extractions together.

    const keyDevelopments = results.flatMap((result) => result.key_developments);keyDevelopments.slice(0, 20);

    [  { year: 0, description: "", evidence: "" },  {    year: 1769,    description: "French inventor Nicolas-Joseph Cugnot built the first steam-powered road vehicle.",    evidence: "French inventor Nicolas-Joseph Cugnot built the first steam-powered road vehicle in 1769."  },  {    year: 1808,    description: "French-born Swiss inventor François Isaac de Rivaz designed and constructed the first internal combu"... 25 more characters,    evidence: "French-born Swiss inventor François Isaac de Rivaz designed and constructed the first internal combu"... 33 more characters  },  {    year: 1886,    description: "German inventor Carl Benz patented his Benz Patent-Motorwagen, inventing the modern car—a practical,"... 40 more characters,    evidence: "The modern car—a practical, marketable automobile for everyday use—was invented in 1886, when German"... 56 more characters  },  {    year: 1908,    description: "The 1908 Model T, an American car manufactured by the Ford Motor Company, became one of the first ca"... 28 more characters,    evidence: "One of the first cars affordable by the masses was the 1908 Model T, an American car manufactured by"... 24 more characters  }]

RAG based approach[​](#rag-based-approach "Direct link to RAG based approach")
------------------------------------------------------------------------------

Another simple idea is to chunk up the text, but instead of extracting information from every chunk, just focus on the the most relevant chunks.

caution

It can be difficult to identify which chunks are relevant.

For example, in the `car` article we’re using here, most of the article contains key development information. So by using **RAG**, we’ll likely be throwing out a lot of relevant information.

We suggest experimenting with your use case and determining whether this approach works or not.

Here’s a simple example that relies on an in-memory demo `MemoryVectorStore` vectorstore.

    import { MemoryVectorStore } from "langchain/vectorstores/memory";import { OpenAIEmbeddings } from "@langchain/openai";// Only load the first 10 docs for speed in this demo use-caseconst vectorstore = await MemoryVectorStore.fromDocuments(  splitDocs.slice(0, 10),  new OpenAIEmbeddings());// Only extract from top documentconst retriever = vectorstore.asRetriever({ k: 1 });

In this case the RAG extractor is only looking at the top document.

    import { RunnableSequence } from "@langchain/core/runnables";const ragExtractor = RunnableSequence.from([  {    text: retriever.pipe((docs) => docs[0].pageContent),  },  extractionChain,]);

    const results = await ragExtractor.invoke(  "Key developments associated with cars");

    results.key_developments;

    [  {    year: 2020,    description: "The lifetime of a car built in the 2020s is expected to be about 16 years, or about 2 million km (1."... 33 more characters,    evidence: "The lifetime of a car built in the 2020s is expected to be about 16 years, or about 2 millionkm (1.2"... 31 more characters  },  {    year: 2030,    description: "All fossil fuel vehicles will be banned in Amsterdam from 2030.",    evidence: "all fossil fuel vehicles will be banned in Amsterdam from 2030."  },  {    year: 2020,    description: "In 2020, there were 56 million cars manufactured worldwide, down from 67 million the previous year.",    evidence: "In 2020, there were 56 million cars manufactured worldwide, down from 67 million the previous year."  }]

Common issues[​](#common-issues "Direct link to Common issues")
---------------------------------------------------------------

Different methods have their own pros and cons related to cost, speed, and accuracy.

Watch out for these issues:

*   Chunking content means that the LLM can fail to extract information if the information is spread across multiple chunks.
*   Large chunk overlap may cause the same information to be extracted twice, so be prepared to de-duplicate!
*   LLMs can make up data. If looking for a single fact across a large text and using a brute force approach, you may end up getting more made up data.

Next steps[​](#next-steps "Direct link to Next steps")
------------------------------------------------------

You’ve now learned how to improve extraction quality using few-shot examples.

Next, check out some of the other guides in this section, such as [some tips on how to improve extraction quality with examples](/v0.2/docs/how_to/extraction_examples).

* * *

#### Was this page helpful?

  

#### You can leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to use reference examples

](/v0.2/docs/how_to/extraction_examples)[

Next

How to do extraction without using function calling

](/v0.2/docs/how_to/extraction_parse)

*   [Set up](#set-up)
*   [Define the schema](#define-the-schema)
*   [Brute force approach](#brute-force-approach)
    *   [Merge results](#merge-results)
*   [RAG based approach](#rag-based-approach)
*   [Common issues](#common-issues)
*   [Next steps](#next-steps)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.