!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

LangChain v0.2 is coming soon! Preview the new docs [here](/v0.2/docs/introduction/).

[

![🦜️🔗 Langchain](/v0.1/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.1/img/brand/wordmark-dark.png)

](/v0.1/)[Docs](/v0.1/docs/get_started/introduction/)[Use cases](/v0.1/docs/use_cases/)[Integrations](/v0.1/docs/integrations/platforms/)[API Reference](https://api.js.langchain.com)

[More](#)

*   [People](/v0.1/docs/people/)
*   [Community](/v0.1/docs/community/)
*   [Tutorials](/v0.1/docs/additional_resources/tutorials/)
*   [Contributing](/v0.1/docs/contributing/)

[v0.1](#)

*   [v0.2](https://js.langchain.com/v0.2/docs/introduction)
*   [v0.1](/v0.1/docs/get_started/introduction/)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Providers](/v0.1/docs/integrations/platforms/)
    
    *   [Providers](/v0.1/docs/integrations/platforms/)
    *   [Anthropic](/v0.1/docs/integrations/platforms/anthropic/)
    *   [AWS](/v0.1/docs/integrations/platforms/aws/)
    *   [Google](/v0.1/docs/integrations/platforms/google/)
    *   [Microsoft](/v0.1/docs/integrations/platforms/microsoft/)
    *   [OpenAI](/v0.1/docs/integrations/platforms/openai/)
*   [Components](/v0.1/docs/integrations/components/)
    
    *   [LLMs](/v0.1/docs/integrations/llms/)
        
    *   [Chat models](/v0.1/docs/integrations/chat/)
        
    *   [Document loaders](/v0.1/docs/integrations/document_loaders/)
        
        *   [File Loaders](/v0.1/docs/integrations/document_loaders/file_loaders/)
            
        *   [Web Loaders](/v0.1/docs/integrations/document_loaders/web_loaders/)
            
            *   [Cheerio](/v0.1/docs/integrations/document_loaders/web_loaders/web_cheerio/)
            *   [Puppeteer](/v0.1/docs/integrations/document_loaders/web_loaders/web_puppeteer/)
            *   [Playwright](/v0.1/docs/integrations/document_loaders/web_loaders/web_playwright/)
            *   [Apify Dataset](/v0.1/docs/integrations/document_loaders/web_loaders/apify_dataset/)
            *   [AssemblyAI Audio Transcript](/v0.1/docs/integrations/document_loaders/web_loaders/assemblyai_audio_transcription/)
            *   [Azure Blob Storage Container](/v0.1/docs/integrations/document_loaders/web_loaders/azure_blob_storage_container/)
            *   [Azure Blob Storage File](/v0.1/docs/integrations/document_loaders/web_loaders/azure_blob_storage_file/)
            *   [Browserbase Loader](/v0.1/docs/integrations/document_loaders/web_loaders/browserbase/)
            *   [College Confidential](/v0.1/docs/integrations/document_loaders/web_loaders/college_confidential/)
            *   [Confluence](/v0.1/docs/integrations/document_loaders/web_loaders/confluence/)
            *   [Couchbase](/v0.1/docs/integrations/document_loaders/web_loaders/couchbase/)
            *   [Figma](/v0.1/docs/integrations/document_loaders/web_loaders/figma/)
            *   [Firecrawl](/v0.1/docs/integrations/document_loaders/web_loaders/firecrawl/)
            *   [GitBook](/v0.1/docs/integrations/document_loaders/web_loaders/gitbook/)
            *   [GitHub](/v0.1/docs/integrations/document_loaders/web_loaders/github/)
            *   [Hacker News](/v0.1/docs/integrations/document_loaders/web_loaders/hn/)
            *   [IMSDB](/v0.1/docs/integrations/document_loaders/web_loaders/imsdb/)
            *   [Notion API](/v0.1/docs/integrations/document_loaders/web_loaders/notionapi/)
            *   [PDF files](/v0.1/docs/integrations/document_loaders/web_loaders/pdf/)
            *   [Recursive URL Loader](/v0.1/docs/integrations/document_loaders/web_loaders/recursive_url_loader/)
            *   [S3 File](/v0.1/docs/integrations/document_loaders/web_loaders/s3/)
            *   [SearchApi Loader](/v0.1/docs/integrations/document_loaders/web_loaders/searchapi/)
            *   [SerpAPI Loader](/v0.1/docs/integrations/document_loaders/web_loaders/serpapi/)
            *   [Sitemap Loader](/v0.1/docs/integrations/document_loaders/web_loaders/sitemap/)
            *   [Sonix Audio](/v0.1/docs/integrations/document_loaders/web_loaders/sonix_audio_transcription/)
            *   [Blockchain Data](/v0.1/docs/integrations/document_loaders/web_loaders/sort_xyz_blockchain/)
            *   [YouTube transcripts](/v0.1/docs/integrations/document_loaders/web_loaders/youtube/)
    *   [Document transformers](/v0.1/docs/integrations/document_transformers/)
        
    *   [Document compressors](/v0.1/docs/integrations/document_compressors/)
        
    *   [Text embedding models](/v0.1/docs/integrations/text_embedding/)
        
    *   [Vector stores](/v0.1/docs/integrations/vectorstores/)
        
    *   [Retrievers](/v0.1/docs/integrations/retrievers/)
        
    *   [Tools](/v0.1/docs/integrations/tools/)
        
    *   [Agents and toolkits](/v0.1/docs/integrations/toolkits/)
        
    *   [Chat Memory](/v0.1/docs/integrations/chat_memory/)
        
    *   [Stores](/v0.1/docs/integrations/stores/)
        

*   [](/v0.1/)
*   [Components](/v0.1/docs/integrations/components/)
*   [Document loaders](/v0.1/docs/integrations/document_loaders/)
*   [Web Loaders](/v0.1/docs/integrations/document_loaders/web_loaders/)
*   Firecrawl

Firecrawl
=========

This guide shows how to use [Firecrawl](https://firecrawl.dev) with LangChain to load web data into an LLM-ready format using Firecrawl.

Overview[​](#overview "Direct link to Overview")
------------------------------------------------

[FireCrawl](https://firecrawl.dev) crawls and convert any website into LLM-ready data. It crawls all accessible subpages and give you clean markdown and metadata for each. No sitemap required.

FireCrawl handles complex tasks such as reverse proxies, caching, rate limits, and content blocked by JavaScript. Built by the [mendable.ai](https://mendable.ai) team.

This guide shows how to scrap and crawl entire websites and load them using the `FireCrawlLoader` in LangChain.

Setup[​](#setup "Direct link to Setup")
---------------------------------------

Sign up and get your free [FireCrawl API key](https://firecrawl.dev) to start. FireCrawl offers 300 free credits to get you started, and it's [open-source](https://github.com/mendableai/firecrawl) in case you want to self-host.

Usage[​](#usage "Direct link to Usage")
---------------------------------------

Here's an example of how to use the `FireCrawlLoader` to load web search results:

Firecrawl offers 2 modes: `scrape` and `crawl`. In `scrape` mode, Firecrawl will only scrape the page you provide. In `crawl` mode, Firecrawl will crawl the entire website.

*   npm
*   Yarn
*   pnpm

    npm install @mendable/firecrawl-js

    yarn add @mendable/firecrawl-js

    pnpm add @mendable/firecrawl-js

    import { FireCrawlLoader } from "langchain/document_loaders/web/firecrawl";const loader = new FireCrawlLoader({  url: "https://firecrawl.dev", // The URL to scrape  apiKey: process.env.FIRECRAWL_API_KEY, // Optional, defaults to `FIRECRAWL_API_KEY` in your env.  mode: "scrape", // The mode to run the crawler in. Can be "scrape" for single urls or "crawl" for all accessible subpages  params: {    // optional parameters based on Firecrawl API docs    // For API documentation, visit https://docs.firecrawl.dev  },});const docs = await loader.load();

#### API Reference:

*   [FireCrawlLoader](https://api.js.langchain.com/classes/langchain_document_loaders_web_firecrawl.FireCrawlLoader.html) from `langchain/document_loaders/web/firecrawl`

### Additional Parameters[​](#additional-parameters "Direct link to Additional Parameters")

For `params` you can pass any of the params according to the [Firecrawl documentation](https://docs.firecrawl.dev).

* * *

#### Help us out by providing feedback on this documentation page:

[

Previous

Figma

](/v0.1/docs/integrations/document_loaders/web_loaders/figma/)[

Next

GitBook

](/v0.1/docs/integrations/document_loaders/web_loaders/gitbook/)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.