!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

LangChain v0.2 is coming soon! Preview the new docs [here](/v0.2/docs/introduction/).

[

![🦜️🔗 Langchain](/v0.1/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.1/img/brand/wordmark-dark.png)

](/v0.1/)[Docs](/v0.1/docs/get_started/introduction/)[Use cases](/v0.1/docs/use_cases/)[Integrations](/v0.1/docs/integrations/platforms/)[API Reference](https://api.js.langchain.com)

[More](#)

*   [People](/v0.1/docs/people/)
*   [Community](/v0.1/docs/community/)
*   [Tutorials](/v0.1/docs/additional_resources/tutorials/)
*   [Contributing](/v0.1/docs/contributing/)

[v0.1](#)

*   [v0.2](https://js.langchain.com/v0.2/docs/introduction)
*   [v0.1](/v0.1/docs/get_started/introduction/)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Providers](/v0.1/docs/integrations/platforms/)
    
    *   [Providers](/v0.1/docs/integrations/platforms/)
    *   [Anthropic](/v0.1/docs/integrations/platforms/anthropic/)
    *   [AWS](/v0.1/docs/integrations/platforms/aws/)
    *   [Google](/v0.1/docs/integrations/platforms/google/)
    *   [Microsoft](/v0.1/docs/integrations/platforms/microsoft/)
    *   [OpenAI](/v0.1/docs/integrations/platforms/openai/)
*   [Components](/v0.1/docs/integrations/components/)
    
    *   [LLMs](/v0.1/docs/integrations/llms/)
        
    *   [Chat models](/v0.1/docs/integrations/chat/)
        
    *   [Document loaders](/v0.1/docs/integrations/document_loaders/)
        
        *   [File Loaders](/v0.1/docs/integrations/document_loaders/file_loaders/)
            
        *   [Web Loaders](/v0.1/docs/integrations/document_loaders/web_loaders/)
            
            *   [Cheerio](/v0.1/docs/integrations/document_loaders/web_loaders/web_cheerio/)
            *   [Puppeteer](/v0.1/docs/integrations/document_loaders/web_loaders/web_puppeteer/)
            *   [Playwright](/v0.1/docs/integrations/document_loaders/web_loaders/web_playwright/)
            *   [Apify Dataset](/v0.1/docs/integrations/document_loaders/web_loaders/apify_dataset/)
            *   [AssemblyAI Audio Transcript](/v0.1/docs/integrations/document_loaders/web_loaders/assemblyai_audio_transcription/)
            *   [Azure Blob Storage Container](/v0.1/docs/integrations/document_loaders/web_loaders/azure_blob_storage_container/)
            *   [Azure Blob Storage File](/v0.1/docs/integrations/document_loaders/web_loaders/azure_blob_storage_file/)
            *   [Browserbase Loader](/v0.1/docs/integrations/document_loaders/web_loaders/browserbase/)
            *   [College Confidential](/v0.1/docs/integrations/document_loaders/web_loaders/college_confidential/)
            *   [Confluence](/v0.1/docs/integrations/document_loaders/web_loaders/confluence/)
            *   [Couchbase](/v0.1/docs/integrations/document_loaders/web_loaders/couchbase/)
            *   [Figma](/v0.1/docs/integrations/document_loaders/web_loaders/figma/)
            *   [Firecrawl](/v0.1/docs/integrations/document_loaders/web_loaders/firecrawl/)
            *   [GitBook](/v0.1/docs/integrations/document_loaders/web_loaders/gitbook/)
            *   [GitHub](/v0.1/docs/integrations/document_loaders/web_loaders/github/)
            *   [Hacker News](/v0.1/docs/integrations/document_loaders/web_loaders/hn/)
            *   [IMSDB](/v0.1/docs/integrations/document_loaders/web_loaders/imsdb/)
            *   [Notion API](/v0.1/docs/integrations/document_loaders/web_loaders/notionapi/)
            *   [PDF files](/v0.1/docs/integrations/document_loaders/web_loaders/pdf/)
            *   [Recursive URL Loader](/v0.1/docs/integrations/document_loaders/web_loaders/recursive_url_loader/)
            *   [S3 File](/v0.1/docs/integrations/document_loaders/web_loaders/s3/)
            *   [SearchApi Loader](/v0.1/docs/integrations/document_loaders/web_loaders/searchapi/)
            *   [SerpAPI Loader](/v0.1/docs/integrations/document_loaders/web_loaders/serpapi/)
            *   [Sitemap Loader](/v0.1/docs/integrations/document_loaders/web_loaders/sitemap/)
            *   [Sonix Audio](/v0.1/docs/integrations/document_loaders/web_loaders/sonix_audio_transcription/)
            *   [Blockchain Data](/v0.1/docs/integrations/document_loaders/web_loaders/sort_xyz_blockchain/)
            *   [YouTube transcripts](/v0.1/docs/integrations/document_loaders/web_loaders/youtube/)
    *   [Document transformers](/v0.1/docs/integrations/document_transformers/)
        
    *   [Document compressors](/v0.1/docs/integrations/document_compressors/)
        
    *   [Text embedding models](/v0.1/docs/integrations/text_embedding/)
        
    *   [Vector stores](/v0.1/docs/integrations/vectorstores/)
        
    *   [Retrievers](/v0.1/docs/integrations/retrievers/)
        
    *   [Tools](/v0.1/docs/integrations/tools/)
        
    *   [Agents and toolkits](/v0.1/docs/integrations/toolkits/)
        
    *   [Chat Memory](/v0.1/docs/integrations/chat_memory/)
        
    *   [Stores](/v0.1/docs/integrations/stores/)
        

*   [](/v0.1/)
*   [Components](/v0.1/docs/integrations/components/)
*   [Document loaders](/v0.1/docs/integrations/document_loaders/)
*   [Web Loaders](/v0.1/docs/integrations/document_loaders/web_loaders/)
*   Puppeteer

Webpages, with Puppeteer
========================

Compatibility

Only available on Node.js.

This example goes over how to load data from webpages using Puppeteer. One document will be created for each webpage.

Puppeteer is a Node.js library that provides a high-level API for controlling headless Chrome or Chromium. You can use Puppeteer to automate web page interactions, including extracting data from dynamic web pages that require JavaScript to render.

If you want a lighterweight solution, and the webpages you want to load do not require JavaScript to render, you can use the [CheerioWebBaseLoader](/v0.1/docs/integrations/document_loaders/web_loaders/web_cheerio/) instead.

Setup[​](#setup "Direct link to Setup")
---------------------------------------

*   npm
*   Yarn
*   pnpm

    npm install puppeteer

    yarn add puppeteer

    pnpm add puppeteer

Usage[​](#usage "Direct link to Usage")
---------------------------------------

    import { PuppeteerWebBaseLoader } from "langchain/document_loaders/web/puppeteer";/** * Loader uses `page.evaluate(() => document.body.innerHTML)` * as default evaluate function **/const loader = new PuppeteerWebBaseLoader("https://www.tabnews.com.br/");const docs = await loader.load();

Options[​](#options "Direct link to Options")
---------------------------------------------

Here's an explanation of the parameters you can pass to the PuppeteerWebBaseLoader constructor using the PuppeteerWebBaseLoaderOptions interface:

    type PuppeteerWebBaseLoaderOptions = {  launchOptions?: PuppeteerLaunchOptions;  gotoOptions?: PuppeteerGotoOptions;  evaluate?: (page: Page, browser: Browser) => Promise<string>;};

1.  `launchOptions`: an optional object that specifies additional options to pass to the puppeteer.launch() method. This can include options such as the headless flag to launch the browser in headless mode, or the slowMo option to slow down Puppeteer's actions to make them easier to follow.
    
2.  `gotoOptions`: an optional object that specifies additional options to pass to the page.goto() method. This can include options such as the timeout option to specify the maximum navigation time in milliseconds, or the waitUntil option to specify when to consider the navigation as successful.
    
3.  `evaluate`: an optional function that can be used to evaluate JavaScript code on the page using the page.evaluate() method. This can be useful for extracting data from the page or interacting with page elements. The function should return a Promise that resolves to a string containing the result of the evaluation.
    

By passing these options to the `PuppeteerWebBaseLoader` constructor, you can customize the behavior of the loader and use Puppeteer's powerful features to scrape and interact with web pages.

Here is a basic example to do it:

    import { PuppeteerWebBaseLoader } from "langchain/document_loaders/web/puppeteer";const loaderWithOptions = new PuppeteerWebBaseLoader(  "https://www.tabnews.com.br/",  {    launchOptions: {      headless: true,    },    gotoOptions: {      waitUntil: "domcontentloaded",    },    /**  Pass custom evaluate , in this case you get page and browser instances */    async evaluate(page, browser) {      await page.waitForResponse("https://www.tabnews.com.br/va/view");      const result = await page.evaluate(() => document.body.innerHTML);      await browser.close();      return result;    },  });const docsFromLoaderWithOptions = await loaderWithOptions.load();console.log({ docsFromLoaderWithOptions });

#### API Reference:

*   [PuppeteerWebBaseLoader](https://api.js.langchain.com/classes/langchain_document_loaders_web_puppeteer.PuppeteerWebBaseLoader.html) from `langchain/document_loaders/web/puppeteer`

### Screenshots[​](#screenshots "Direct link to Screenshots")

To take a screenshot of a site, initialize the loader the same as above, and call the `.screenshot()` method. This will return an instance of `Document` where the page content is a base64 encoded image, and the metadata contains a `source` field with the URL of the page.

    import { PuppeteerWebBaseLoader } from "langchain/document_loaders/web/puppeteer";const loaderWithOptions = new PuppeteerWebBaseLoader("https://langchain.com", {  launchOptions: {    headless: true,  },  gotoOptions: {    waitUntil: "domcontentloaded",  },});const screenshot = await loaderWithOptions.screenshot();console.log({ screenshot });

#### API Reference:

*   [PuppeteerWebBaseLoader](https://api.js.langchain.com/classes/langchain_document_loaders_web_puppeteer.PuppeteerWebBaseLoader.html) from `langchain/document_loaders/web/puppeteer`

* * *

#### Help us out by providing feedback on this documentation page:

[

Previous

Cheerio

](/v0.1/docs/integrations/document_loaders/web_loaders/web_cheerio/)[

Next

Playwright

](/v0.1/docs/integrations/document_loaders/web_loaders/web_playwright/)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.