!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

You are viewing the **preview** v0.2 docs. View the **stable** v0.1 docs [here](/v0.1/docs/get_started/introduction/). Leave feedback on the v0.2 docs [here](https://github.com/langchain-ai/langchainjs/discussions/5386).

[

![🦜️🔗 Langchain](/v0.2/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.2/img/brand/wordmark-dark.png)

](/v0.2/)[Integrations](/v0.2/docs/integrations/platforms/)[API Reference](https://v02.api.js.langchain.com)

[More](#)

*   [People](/v0.2/docs/people/)
*   [Community](/v0.2/docs/community)
*   [Tutorials](/v0.2/docs/additional_resources/tutorials)
*   [Contributing](/v0.2/docs/contributing)

[v0.2](#)

*   [v0.2](/v0.2/docs/introduction)
*   [v0.1](https://js.langchain.com/v0.1/docs/get_started/introduction)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Providers](/v0.2/docs/integrations/platforms/)
    
    *   [Providers](/v0.2/docs/integrations/platforms/)
    *   [Anthropic](/v0.2/docs/integrations/platforms/anthropic)
    *   [AWS](/v0.2/docs/integrations/platforms/aws)
    *   [Google](/v0.2/docs/integrations/platforms/google)
    *   [Microsoft](/v0.2/docs/integrations/platforms/microsoft)
    *   [OpenAI](/v0.2/docs/integrations/platforms/openai)
*   [Components](/v0.2/docs/integrations/components)
    
    *   [Chat models](/v0.2/docs/integrations/chat/)
        
    *   [LLMs](/v0.2/docs/integrations/llms/)
        
        *   [LLMs](/v0.2/docs/integrations/llms/)
        *   [AI21](/v0.2/docs/integrations/llms/ai21)
        *   [AlephAlpha](/v0.2/docs/integrations/llms/aleph_alpha)
        *   [AWS SageMakerEndpoint](/v0.2/docs/integrations/llms/aws_sagemaker)
        *   [Azure OpenAI](/v0.2/docs/integrations/llms/azure)
        *   [Bedrock](/v0.2/docs/integrations/llms/bedrock)
        *   [Cloudflare Workers AI](/v0.2/docs/integrations/llms/cloudflare_workersai)
        *   [Cohere](/v0.2/docs/integrations/llms/cohere)
        *   [Fireworks](/v0.2/docs/integrations/llms/fireworks)
        *   [Friendli](/v0.2/docs/integrations/llms/friendli)
        *   [(Legacy) Google PaLM/VertexAI](/v0.2/docs/integrations/llms/google_palm)
        *   [Google Vertex AI](/v0.2/docs/integrations/llms/google_vertex_ai)
        *   [Gradient AI](/v0.2/docs/integrations/llms/gradient_ai)
        *   [HuggingFaceInference](/v0.2/docs/integrations/llms/huggingface_inference)
        *   [Llama CPP](/v0.2/docs/integrations/llms/llama_cpp)
        *   [NIBittensor](/v0.2/docs/integrations/llms/ni_bittensor)
        *   [Ollama](/v0.2/docs/integrations/llms/ollama)
        *   [OpenAI](/v0.2/docs/integrations/llms/openai)
        *   [PromptLayer OpenAI](/v0.2/docs/integrations/llms/prompt_layer_openai)
        *   [RaycastAI](/v0.2/docs/integrations/llms/raycast)
        *   [Replicate](/v0.2/docs/integrations/llms/replicate)
        *   [Together AI](/v0.2/docs/integrations/llms/togetherai)
        *   [WatsonX AI](/v0.2/docs/integrations/llms/watsonx_ai)
        *   [Writer](/v0.2/docs/integrations/llms/writer)
        *   [YandexGPT](/v0.2/docs/integrations/llms/yandex)
    *   [Embedding models](/v0.2/docs/integrations/text_embedding)
        
    *   [Document loaders](/v0.2/docs/integrations/document_loaders)
        
    *   [Document transformers](/v0.2/docs/integrations/document_transformers)
        
    *   [Vector stores](/v0.2/docs/integrations/vectorstores)
        
    *   [Retrievers](/v0.2/docs/integrations/retrievers)
        
    *   [Tools](/v0.2/docs/integrations/tools)
        
    *   [Toolkits](/v0.2/docs/integrations/toolkits)
        
    *   [Stores](/v0.2/docs/integrations/stores/)
        

*   [](/v0.2/)
*   [Components](/v0.2/docs/integrations/components)
*   [LLMs](/v0.2/docs/integrations/llms/)
*   AWS SageMakerEndpoint

On this page

AWS SageMakerEndpoint
=====================

LangChain.js supports integration with AWS SageMaker-hosted endpoints. Check [Amazon SageMaker JumpStart](https://aws.amazon.com/sagemaker/jumpstart/) for a list of available models, and how to deploy your own.

Setup[​](#setup "Direct link to Setup")
---------------------------------------

You'll need to install the official SageMaker SDK as a peer dependency:

*   npm
*   Yarn
*   pnpm

    npm install @aws-sdk/client-sagemaker-runtime

    yarn add @aws-sdk/client-sagemaker-runtime

    pnpm add @aws-sdk/client-sagemaker-runtime

tip

See [this section for general instructions on installing integration packages](/v0.2/docs/how_to/installation#installing-integration-packages).

*   npm
*   Yarn
*   pnpm

    npm install @langchain/community

    yarn add @langchain/community

    pnpm add @langchain/community

Usage[​](#usage "Direct link to Usage")
---------------------------------------

    import {  SageMakerEndpoint,  SageMakerLLMContentHandler,} from "@langchain/community/llms/sagemaker_endpoint";interface ResponseJsonInterface {  generation: {    content: string;  };}// Custom for whatever model you'll be usingclass LLama213BHandler implements SageMakerLLMContentHandler {  contentType = "application/json";  accepts = "application/json";  async transformInput(    prompt: string,    modelKwargs: Record<string, unknown>  ): Promise<Uint8Array> {    const payload = {      inputs: [[{ role: "user", content: prompt }]],      parameters: modelKwargs,    };    const stringifiedPayload = JSON.stringify(payload);    return new TextEncoder().encode(stringifiedPayload);  }  async transformOutput(output: Uint8Array): Promise<string> {    const response_json = JSON.parse(      new TextDecoder("utf-8").decode(output)    ) as ResponseJsonInterface[];    const content = response_json[0]?.generation.content ?? "";    return content;  }}const contentHandler = new LLama213BHandler();const model = new SageMakerEndpoint({  endpointName: "aws-llama-2-13b-chat",  modelKwargs: {    temperature: 0.5,    max_new_tokens: 700,    top_p: 0.9,  },  endpointKwargs: {    CustomAttributes: "accept_eula=true",  },  contentHandler,  clientOptions: {    region: "YOUR AWS ENDPOINT REGION",    credentials: {      accessKeyId: "YOUR AWS ACCESS ID",      secretAccessKey: "YOUR AWS SECRET ACCESS KEY",    },  },});const res = await model.invoke(  "Hello, my name is John Doe, tell me a joke about llamas ");console.log(res);/*  [    {      content: "Hello, John Doe! Here's a llama joke for you:        Why did the llama become a gardener?        Because it was great at llama-scaping!"    }  ] */

#### API Reference:

*   [SageMakerEndpoint](https://v02.api.js.langchain.com/classes/langchain_community_llms_sagemaker_endpoint.SageMakerEndpoint.html) from `@langchain/community/llms/sagemaker_endpoint`
*   [SageMakerLLMContentHandler](https://v02.api.js.langchain.com/types/langchain_community_llms_sagemaker_endpoint.SageMakerLLMContentHandler.html) from `@langchain/community/llms/sagemaker_endpoint`

* * *

#### Was this page helpful?

  

#### You can leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

AlephAlpha

](/v0.2/docs/integrations/llms/aleph_alpha)[

Next

Azure OpenAI

](/v0.2/docs/integrations/llms/azure)

*   [Setup](#setup)
*   [Usage](#usage)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.