!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

You are viewing the **preview** v0.2 docs. View the **stable** v0.1 docs [here](/v0.1/docs/get_started/introduction/). Leave feedback on the v0.2 docs [here](https://github.com/langchain-ai/langchainjs/discussions/5386).

[

![🦜️🔗 Langchain](/v0.2/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.2/img/brand/wordmark-dark.png)

](/v0.2/)[Integrations](/v0.2/docs/integrations/platforms/)[API Reference](https://v02.api.js.langchain.com)

[More](#)

*   [People](/v0.2/docs/people/)
*   [Community](/v0.2/docs/community)
*   [Tutorials](/v0.2/docs/additional_resources/tutorials)
*   [Contributing](/v0.2/docs/contributing)

[v0.2](#)

*   [v0.2](/v0.2/docs/introduction)
*   [v0.1](https://js.langchain.com/v0.1/docs/get_started/introduction)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Providers](/v0.2/docs/integrations/platforms/)
    
    *   [Providers](/v0.2/docs/integrations/platforms/)
    *   [Anthropic](/v0.2/docs/integrations/platforms/anthropic)
    *   [AWS](/v0.2/docs/integrations/platforms/aws)
    *   [Google](/v0.2/docs/integrations/platforms/google)
    *   [Microsoft](/v0.2/docs/integrations/platforms/microsoft)
    *   [OpenAI](/v0.2/docs/integrations/platforms/openai)
*   [Components](/v0.2/docs/integrations/components)
    
    *   [Chat models](/v0.2/docs/integrations/chat/)
        
    *   [LLMs](/v0.2/docs/integrations/llms/)
        
    *   [Embedding models](/v0.2/docs/integrations/text_embedding)
        
        *   [Alibaba Tongyi](/v0.2/docs/integrations/text_embedding/alibaba_tongyi)
        *   [Azure OpenAI](/v0.2/docs/integrations/text_embedding/azure_openai)
        *   [Baidu Qianfan](/v0.2/docs/integrations/text_embedding/baidu_qianfan)
        *   [Bedrock](/v0.2/docs/integrations/text_embedding/bedrock)
        *   [Cloudflare Workers AI](/v0.2/docs/integrations/text_embedding/cloudflare_ai)
        *   [Cohere](/v0.2/docs/integrations/text_embedding/cohere)
        *   [Fireworks](/v0.2/docs/integrations/text_embedding/fireworks)
        *   [Google AI](/v0.2/docs/integrations/text_embedding/google_generativeai)
        *   [Google PaLM](/v0.2/docs/integrations/text_embedding/google_palm)
        *   [Google Vertex AI](/v0.2/docs/integrations/text_embedding/google_vertex_ai)
        *   [Gradient AI](/v0.2/docs/integrations/text_embedding/gradient_ai)
        *   [HuggingFace Inference](/v0.2/docs/integrations/text_embedding/hugging_face_inference)
        *   [Llama CPP](/v0.2/docs/integrations/text_embedding/llama_cpp)
        *   [Minimax](/v0.2/docs/integrations/text_embedding/minimax)
        *   [Mistral AI](/v0.2/docs/integrations/text_embedding/mistralai)
        *   [Nomic](/v0.2/docs/integrations/text_embedding/nomic)
        *   [Ollama](/v0.2/docs/integrations/text_embedding/ollama)
        *   [OpenAI](/v0.2/docs/integrations/text_embedding/openai)
        *   [Prem AI](/v0.2/docs/integrations/text_embedding/premai)
        *   [TensorFlow](/v0.2/docs/integrations/text_embedding/tensorflow)
        *   [Together AI](/v0.2/docs/integrations/text_embedding/togetherai)
        *   [HuggingFace Transformers](/v0.2/docs/integrations/text_embedding/transformers)
        *   [Voyage AI](/v0.2/docs/integrations/text_embedding/voyageai)
        *   [ZhipuAI](/v0.2/docs/integrations/text_embedding/zhipuai)
    *   [Document loaders](/v0.2/docs/integrations/document_loaders)
        
    *   [Document transformers](/v0.2/docs/integrations/document_transformers)
        
    *   [Vector stores](/v0.2/docs/integrations/vectorstores)
        
    *   [Retrievers](/v0.2/docs/integrations/retrievers)
        
    *   [Tools](/v0.2/docs/integrations/tools)
        
    *   [Toolkits](/v0.2/docs/integrations/toolkits)
        
    *   [Stores](/v0.2/docs/integrations/stores/)
        

*   [](/v0.2/)
*   [Components](/v0.2/docs/integrations/components)
*   [Embedding models](/v0.2/docs/integrations/text_embedding)
*   Llama CPP

On this page

Llama CPP
=========

Compatibility

Only available on Node.js.

This module is based on the [node-llama-cpp](https://github.com/withcatai/node-llama-cpp) Node.js bindings for [llama.cpp](https://github.com/ggerganov/llama.cpp), allowing you to work with a locally running LLM. This allows you to work with a much smaller quantized model capable of running on a laptop environment, ideal for testing and scratch padding ideas without running up a bill!

Setup[​](#setup "Direct link to Setup")
---------------------------------------

You'll need to install the [node-llama-cpp](https://github.com/withcatai/node-llama-cpp) module to communicate with your local model.

*   npm
*   Yarn
*   pnpm

    npm install -S node-llama-cpp

    yarn add node-llama-cpp

    pnpm add node-llama-cpp

tip

See [this section for general instructions on installing integration packages](/v0.2/docs/how_to/installation#installing-integration-packages).

*   npm
*   Yarn
*   pnpm

    npm install @langchain/community

    yarn add @langchain/community

    pnpm add @langchain/community

You will also need a local Llama 2 model (or a model supported by [node-llama-cpp](https://github.com/withcatai/node-llama-cpp)). You will need to pass the path to this model to the LlamaCpp module as a part of the parameters (see example).

Out-of-the-box `node-llama-cpp` is tuned for running on a MacOS platform with support for the Metal GPU of Apple M-series of processors. If you need to turn this off or need support for the CUDA architecture then refer to the documentation at [node-llama-cpp](https://withcatai.github.io/node-llama-cpp/).

For advice on getting and preparing `llama2` see the documentation for the LLM version of this module.

A note to LangChain.js contributors: if you want to run the tests associated with this module you will need to put the path to your local model in the environment variable `LLAMA_PATH`.

Usage[​](#usage "Direct link to Usage")
---------------------------------------

### Basic use[​](#basic-use "Direct link to Basic use")

We need to provide a path to our local Llama2 model, also the `embeddings` property is always set to `true` in this module.

    import { LlamaCppEmbeddings } from "@langchain/community/embeddings/llama_cpp";const llamaPath = "/Replace/with/path/to/your/model/gguf-llama2-q4_0.bin";const embeddings = new LlamaCppEmbeddings({  modelPath: llamaPath,});const res = embeddings.embedQuery("Hello Llama!");console.log(res);/*	[ 15043, 365, 29880, 3304, 29991 ]*/

#### API Reference:

*   [LlamaCppEmbeddings](https://v02.api.js.langchain.com/classes/langchain_community_embeddings_llama_cpp.LlamaCppEmbeddings.html) from `@langchain/community/embeddings/llama_cpp`

### Document embedding[​](#document-embedding "Direct link to Document embedding")

    import { LlamaCppEmbeddings } from "@langchain/community/embeddings/llama_cpp";const llamaPath = "/Replace/with/path/to/your/model/gguf-llama2-q4_0.bin";const documents = ["Hello World!", "Bye Bye!"];const embeddings = new LlamaCppEmbeddings({  modelPath: llamaPath,});const res = await embeddings.embedDocuments(documents);console.log(res);/*	[ [ 15043, 2787, 29991 ], [ 2648, 29872, 2648, 29872, 29991 ] ]*/

#### API Reference:

*   [LlamaCppEmbeddings](https://v02.api.js.langchain.com/classes/langchain_community_embeddings_llama_cpp.LlamaCppEmbeddings.html) from `@langchain/community/embeddings/llama_cpp`

* * *

#### Was this page helpful?

  

#### You can leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

HuggingFace Inference

](/v0.2/docs/integrations/text_embedding/hugging_face_inference)[

Next

Minimax

](/v0.2/docs/integrations/text_embedding/minimax)

*   [Setup](#setup)
*   [Usage](#usage)
    *   [Basic use](#basic-use)
    *   [Document embedding](#document-embedding)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.