!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();t(null!==e?e:"light")}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())

[Skip to main content](#__docusaurus_skipToContent_fallback)

You are viewing the **preview** v0.2 docs. View the **stable** v0.1 docs [here](/v0.1/docs/get_started/introduction/). Leave feedback on the v0.2 docs [here](https://github.com/langchain-ai/langchainjs/discussions/5386).

[

![🦜️🔗 Langchain](/v0.2/img/brand/wordmark.png)![🦜️🔗 Langchain](/v0.2/img/brand/wordmark-dark.png)

](/v0.2/)[Integrations](/v0.2/docs/integrations/platforms/)[API Reference](https://v02.api.js.langchain.com)

[More](#)

*   [People](/v0.2/docs/people/)
*   [Community](/v0.2/docs/community)
*   [Tutorials](/v0.2/docs/additional_resources/tutorials)
*   [Contributing](/v0.2/docs/contributing)

[v0.2](#)

*   [v0.2](/v0.2/docs/introduction)
*   [v0.1](https://js.langchain.com/v0.1/docs/get_started/introduction)

[🦜🔗](#)

*   [LangSmith](https://smith.langchain.com)
*   [LangSmith Docs](https://docs.smith.langchain.com)
*   [LangChain Hub](https://smith.langchain.com/hub)
*   [LangServe](https://github.com/langchain-ai/langserve)
*   [Python Docs](https://python.langchain.com/)

[Chat](https://chatjs.langchain.com)[](https://github.com/langchain-ai/langchainjs)

Search

*   [Introduction](/v0.2/docs/introduction)
*   [Tutorials](/v0.2/docs/tutorials/)
    
    *   [Build a Question Answering application over a Graph Database](/v0.2/docs/tutorials/graph)
    *   [Tutorials](/v0.2/docs/tutorials/)
    *   [Build a Simple LLM Application](/v0.2/docs/tutorials/llm_chain)
    *   [Build a Query Analysis System](/v0.2/docs/tutorials/query_analysis)
    *   [Build a Chatbot](/v0.2/docs/tutorials/chatbot)
    *   [Build an Agent](/v0.2/docs/tutorials/agents)
    *   [Build an Extraction Chain](/v0.2/docs/tutorials/extraction)
    *   [Summarize Text](/v0.2/docs/tutorials/summarization)
    *   [Tagging](/v0.2/docs/tutorials/classification)
    *   [Build a Local RAG Application](/v0.2/docs/tutorials/local_rag)
    *   [Conversational RAG](/v0.2/docs/tutorials/qa_chat_history)
    *   [Build a Retrieval Augmented Generation (RAG) App](/v0.2/docs/tutorials/rag)
    *   [Build a Question/Answering system over SQL data](/v0.2/docs/tutorials/sql_qa)
*   [How-to guides](/v0.2/docs/how_to/)
    
    *   [How-to guides](/v0.2/docs/how_to/)
    *   [How to use example selectors](/v0.2/docs/how_to/example_selectors)
    *   [Installation](/v0.2/docs/how_to/installation)
    *   [How to stream responses from an LLM](/v0.2/docs/how_to/streaming_llm)
    *   [How to stream chat model responses](/v0.2/docs/how_to/chat_streaming)
    *   [How to embed text data](/v0.2/docs/how_to/embed_text)
    *   [How to use few shot examples in chat models](/v0.2/docs/how_to/few_shot_examples_chat)
    *   [How to cache model responses](/v0.2/docs/how_to/llm_caching)
    *   [How to cache chat model responses](/v0.2/docs/how_to/chat_model_caching)
    *   [How to create a custom LLM class](/v0.2/docs/how_to/custom_llm)
    *   [How to use few shot examples](/v0.2/docs/how_to/few_shot_examples)
    *   [How to use output parsers to parse an LLM response into structured format](/v0.2/docs/how_to/output_parser_structured)
    *   [How to return structured data from a model](/v0.2/docs/how_to/structured_output)
    *   [How to add ad-hoc tool calling capability to LLMs and Chat Models](/v0.2/docs/how_to/tools_prompting)
    *   [How to create a custom chat model class](/v0.2/docs/how_to/custom_chat)
    *   [How to do per-user retrieval](/v0.2/docs/how_to/qa_per_user)
    *   [How to track token usage](/v0.2/docs/how_to/chat_token_usage_tracking)
    *   [How to track token usage](/v0.2/docs/how_to/llm_token_usage_tracking)
    *   [How to pass through arguments from one step to the next](/v0.2/docs/how_to/passthrough)
    *   [How to compose prompts together](/v0.2/docs/how_to/prompts_composition)
    *   [How to use legacy LangChain Agents (AgentExecutor)](/v0.2/docs/how_to/agent_executor)
    *   [How to add values to a chain's state](/v0.2/docs/how_to/assign)
    *   [How to attach runtime arguments to a Runnable](/v0.2/docs/how_to/binding)
    *   [How to cache embedding results](/v0.2/docs/how_to/caching_embeddings)
    *   [How to split by character](/v0.2/docs/how_to/character_text_splitter)
    *   [How to manage memory](/v0.2/docs/how_to/chatbots_memory)
    *   [How to do retrieval](/v0.2/docs/how_to/chatbots_retrieval)
    *   [How to use tools](/v0.2/docs/how_to/chatbots_tools)
    *   [How to split code](/v0.2/docs/how_to/code_splitter)
    *   [How to do retrieval with contextual compression](/v0.2/docs/how_to/contextual_compression)
    *   [How to write a custom retriever class](/v0.2/docs/how_to/custom_retriever)
    *   [How to create custom Tools](/v0.2/docs/how_to/custom_tools)
    *   [How to debug your LLM apps](/v0.2/docs/how_to/debugging)
    *   [How to load CSV data](/v0.2/docs/how_to/document_loader_csv)
    *   [How to write a custom document loader](/v0.2/docs/how_to/document_loader_custom)
    *   [How to load data from a directory](/v0.2/docs/how_to/document_loader_directory)
    *   [How to load PDF files](/v0.2/docs/how_to/document_loader_pdf)
    *   [How to load JSON data](/v0.2/docs/how_to/document_loaders_json)
    *   [How to select examples by length](/v0.2/docs/how_to/example_selectors_length_based)
    *   [How to select examples by similarity](/v0.2/docs/how_to/example_selectors_similarity)
    *   [How to use reference examples](/v0.2/docs/how_to/extraction_examples)
    *   [How to handle long text](/v0.2/docs/how_to/extraction_long_text)
    *   [How to do extraction without using function calling](/v0.2/docs/how_to/extraction_parse)
    *   [Fallbacks](/v0.2/docs/how_to/fallbacks)
    *   [Few Shot Prompt Templates](/v0.2/docs/how_to/few_shot)
    *   [How to run custom functions](/v0.2/docs/how_to/functions)
    *   [How to construct knowledge graphs](/v0.2/docs/how_to/graph_constructing)
    *   [How to map values to a database](/v0.2/docs/how_to/graph_mapping)
    *   [How to improve results with prompting](/v0.2/docs/how_to/graph_prompting)
    *   [How to add a semantic layer over the database](/v0.2/docs/how_to/graph_semantic)
    *   [How to reindex data to keep your vectorstore in-sync with the underlying data source](/v0.2/docs/how_to/indexing)
    *   [How to get log probabilities](/v0.2/docs/how_to/logprobs)
    *   [How to add message history](/v0.2/docs/how_to/message_history)
    *   [How to generate multiple embeddings per document](/v0.2/docs/how_to/multi_vector)
    *   [How to generate multiple queries to retrieve data for](/v0.2/docs/how_to/multiple_queries)
    *   [How to parse JSON output](/v0.2/docs/how_to/output_parser_json)
    *   [How to retry when output parsing errors occur](/v0.2/docs/how_to/output_parser_retry)
    *   [How to parse XML output](/v0.2/docs/how_to/output_parser_xml)
    *   [How to invoke runnables in parallel](/v0.2/docs/how_to/parallel)
    *   [How to retrieve the whole document for a chunk](/v0.2/docs/how_to/parent_document_retriever)
    *   [How to partially format prompt templates](/v0.2/docs/how_to/prompts_partial)
    *   [How to add chat history to a question-answering chain](/v0.2/docs/how_to/qa_chat_history_how_to)
    *   [How to return citations](/v0.2/docs/how_to/qa_citations)
    *   [How to return sources](/v0.2/docs/how_to/qa_sources)
    *   [How to stream from a question-answering chain](/v0.2/docs/how_to/qa_streaming)
    *   [How to construct filters](/v0.2/docs/how_to/query_constructing_filters)
    *   [How to add examples to the prompt](/v0.2/docs/how_to/query_few_shot)
    *   [How to deal with high cardinality categorical variables](/v0.2/docs/how_to/query_high_cardinality)
    *   [How to handle multiple queries](/v0.2/docs/how_to/query_multiple_queries)
    *   [How to handle multiple retrievers](/v0.2/docs/how_to/query_multiple_retrievers)
    *   [How to handle cases where no queries are generated](/v0.2/docs/how_to/query_no_queries)
    *   [How to recursively split text by characters](/v0.2/docs/how_to/recursive_text_splitter)
    *   [How to reduce retrieval latency](/v0.2/docs/how_to/reduce_retrieval_latency)
    *   [How to route execution within a chain](/v0.2/docs/how_to/routing)
    *   [How to chain runnables](/v0.2/docs/how_to/sequence)
    *   [How to split text by tokens](/v0.2/docs/how_to/split_by_token)
    *   [How to deal with large databases](/v0.2/docs/how_to/sql_large_db)
    *   [How to use prompting to improve results](/v0.2/docs/how_to/sql_prompting)
    *   [How to do query validation](/v0.2/docs/how_to/sql_query_checking)
    *   [How to stream](/v0.2/docs/how_to/streaming)
    *   [How to create a time-weighted retriever](/v0.2/docs/how_to/time_weighted_vectorstore)
    *   [How to use a chat model to call tools](/v0.2/docs/how_to/tool_calling)
    *   [How to call tools with multi-modal data](/v0.2/docs/how_to/tool_calls_multi_modal)
    *   [How to use LangChain tools](/v0.2/docs/how_to/tools_builtin)
    *   [How use a vector store to retrieve data](/v0.2/docs/how_to/vectorstore_retriever)
    *   [How to create and query vector stores](/v0.2/docs/how_to/vectorstores)
*   [Conceptual guide](/v0.2/docs/concepts)
*   Ecosystem
    
    *   [🦜🛠️ LangSmith](/v0.2/docs/langsmith/)
        
    *   [🦜🕸️LangGraph.js](/v0.2/docs/langgraph)
*   Versions
    
    *   [Overview](/v0.2/docs/versions/overview)
    *   [v0.2](/v0.2/docs/versions/v0_2)
    *   [Release Policy](/v0.2/docs/versions/release_policy)
    *   [Packages](/v0.2/docs/versions/packages)
*   [Security](/v0.2/docs/security)

*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to get log probabilities

On this page

How to get log probabilities
============================

Prerequisites

This guide assumes familiarity with the following concepts:

*   [Chat models](/v0.2/docs/concepts/#chat-models)

Certain chat models can be configured to return token-level log probabilities representing the likelihood of a given token. This guide walks through how to get this information in LangChain.

OpenAI[​](#openai "Direct link to OpenAI")
------------------------------------------

Install the `@langchain/openai` package and set your API key:

tip

See [this section for general instructions on installing integration packages](/v0.2/docs/how_to/installation#installing-integration-packages).

*   npm
*   yarn
*   pnpm

    npm i @langchain/openai

    yarn add @langchain/openai

    pnpm add @langchain/openai

For the OpenAI API to return log probabilities, we need to set the `logprobs` param to `true`. Then, the logprobs are included on each output [`AIMessage`](https://api.python.langchain.com/en/latest/messages/langchain_core.messages.ai.AIMessage.html) as part of the `response_metadata`:

    import { ChatOpenAI } from "@langchain/openai";const model = new ChatOpenAI({  model: "gpt-4o",  logprobs: true,});const responseMessage = await model.invoke("how are you today?");responseMessage.response_metadata.logprobs.content.slice(0, 5);

    [  {    token: "Thank",    logprob: -0.70174205,    bytes: [ 84, 104, 97, 110, 107 ],    top_logprobs: []  },  {    token: " you",    logprob: 0,    bytes: [ 32, 121, 111, 117 ],    top_logprobs: []  },  {    token: " for",    logprob: -0.000004723352,    bytes: [ 32, 102, 111, 114 ],    top_logprobs: []  },  {    token: " asking",    logprob: -0.0000013856493,    bytes: [       32,  97, 115,      107, 105, 110,      103    ],    top_logprobs: []  },  {    token: "!",    logprob: -0.00030102333,    bytes: [ 33 ],    top_logprobs: []  }]

And are part of streamed Message chunks as well:

    let count = 0;const stream = await model.stream("How are you today?");let aggregateResponse;for await (const chunk of stream) {  if (count > 5) {    break;  }  if (aggregateResponse === undefined) {    aggregateResponse = chunk;  } else {    aggregateResponse = aggregateResponse.concat(chunk);  }  console.log(aggregateResponse.response_metadata.logprobs?.content);  count++;}

    [][  {    token: "Thank",    logprob: -0.23375113,    bytes: [ 84, 104, 97, 110, 107 ],    top_logprobs: []  }][  {    token: "Thank",    logprob: -0.23375113,    bytes: [ 84, 104, 97, 110, 107 ],    top_logprobs: []  },  {    token: " you",    logprob: 0,    bytes: [ 32, 121, 111, 117 ],    top_logprobs: []  }][  {    token: "Thank",    logprob: -0.23375113,    bytes: [ 84, 104, 97, 110, 107 ],    top_logprobs: []  },  {    token: " you",    logprob: 0,    bytes: [ 32, 121, 111, 117 ],    top_logprobs: []  },  {    token: " for",    logprob: -0.000004723352,    bytes: [ 32, 102, 111, 114 ],    top_logprobs: []  }][  {    token: "Thank",    logprob: -0.23375113,    bytes: [ 84, 104, 97, 110, 107 ],    top_logprobs: []  },  {    token: " you",    logprob: 0,    bytes: [ 32, 121, 111, 117 ],    top_logprobs: []  },  {    token: " for",    logprob: -0.000004723352,    bytes: [ 32, 102, 111, 114 ],    top_logprobs: []  },  {    token: " asking",    logprob: -0.0000029352968,    bytes: [       32,  97, 115,      107, 105, 110,      103    ],    top_logprobs: []  }][  {    token: "Thank",    logprob: -0.23375113,    bytes: [ 84, 104, 97, 110, 107 ],    top_logprobs: []  },  {    token: " you",    logprob: 0,    bytes: [ 32, 121, 111, 117 ],    top_logprobs: []  },  {    token: " for",    logprob: -0.000004723352,    bytes: [ 32, 102, 111, 114 ],    top_logprobs: []  },  {    token: " asking",    logprob: -0.0000029352968,    bytes: [       32,  97, 115,      107, 105, 110,      103    ],    top_logprobs: []  },  {    token: "!",    logprob: -0.00039694557,    bytes: [ 33 ],    top_logprobs: []  }]

`topLogprobs`[​](#toplogprobs "Direct link to toplogprobs")
-----------------------------------------------------------

To see alternate potential generations at each step, you can use the `topLogprobs` parameter:

    const model = new ChatOpenAI({  model: "gpt-4o",  logprobs: true,  topLogprobs: 3,});const responseMessage = await model.invoke("how are you today?");responseMessage.response_metadata.logprobs.content.slice(0, 5);

    [  {    token: "I'm",    logprob: -2.2864406,    bytes: [ 73, 39, 109 ],    top_logprobs: [      {        token: "Thank",        logprob: -0.28644064,        bytes: [ 84, 104, 97, 110, 107 ]      },      {        token: "Hello",        logprob: -2.0364406,        bytes: [ 72, 101, 108, 108, 111 ]      },      { token: "I'm", logprob: -2.2864406, bytes: [ 73, 39, 109 ] }    ]  },  {    token: " just",    logprob: -0.14442946,    bytes: [ 32, 106, 117, 115, 116 ],    top_logprobs: [      {        token: " just",        logprob: -0.14442946,        bytes: [ 32, 106, 117, 115, 116 ]      },      { token: " an", logprob: -2.2694294, bytes: [ 32, 97, 110 ] },      {        token: " here",        logprob: -4.0194297,        bytes: [ 32, 104, 101, 114, 101 ]      }    ]  },  {    token: " a",    logprob: -0.00066632946,    bytes: [ 32, 97 ],    top_logprobs: [      { token: " a", logprob: -0.00066632946, bytes: [ 32, 97 ] },      {        token: " lines",        logprob: -7.750666,        bytes: [ 32, 108, 105, 110, 101, 115 ]      },      { token: " an", logprob: -9.250667, bytes: [ 32, 97, 110 ] }    ]  },  {    token: " computer",    logprob: -0.015423919,    bytes: [       32,  99, 111, 109,      112, 117, 116, 101,      114    ],    top_logprobs: [      {        token: " computer",        logprob: -0.015423919,        bytes: [           32,  99, 111, 109,          112, 117, 116, 101,          114        ]      },      {        token: " program",        logprob: -5.265424,        bytes: [           32, 112, 114, 111,          103, 114,  97, 109        ]      },      {        token: " machine",        logprob: -5.390424,        bytes: [           32, 109,  97,  99,          104, 105, 110, 101        ]      }    ]  },  {    token: " program",    logprob: -0.0010724656,    bytes: [       32, 112, 114, 111,      103, 114,  97, 109    ],    top_logprobs: [      {        token: " program",        logprob: -0.0010724656,        bytes: [           32, 112, 114, 111,          103, 114,  97, 109        ]      },      {        token: "-based",        logprob: -6.8760724,        bytes: [ 45, 98, 97, 115, 101, 100 ]      },      {        token: " algorithm",        logprob: -10.626073,        bytes: [           32,  97, 108, 103,          111, 114, 105, 116,          104, 109        ]      }    ]  }]

Next steps[​](#next-steps "Direct link to Next steps")
------------------------------------------------------

You’ve now learned how to get logprobs from OpenAI models in LangChain.

Next, check out the other how-to guides chat models in this section, like [how to get a model to return structured output](/v0.2/docs/how_to/structured_output) or [how to track token usage](/v0.2/docs/how_to/chat_token_usage_tracking).

* * *

#### Was this page helpful?

  

#### You can leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchainjs/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to reindex data to keep your vectorstore in-sync with the underlying data source

](/v0.2/docs/how_to/indexing)[

Next

How to add message history

](/v0.2/docs/how_to/message_history)

*   [OpenAI](#openai)
*   [`topLogprobs`](#toplogprobs)
*   [Next steps](#next-steps)

Community

*   [Discord](https://discord.gg/cU2adEyC7w)
*   [Twitter](https://twitter.com/LangChainAI)

GitHub

*   [Python](https://github.com/langchain-ai/langchain)
*   [JS/TS](https://github.com/langchain-ai/langchainjs)

More

*   [Homepage](https://langchain.com)
*   [Blog](https://blog.langchain.dev)

Copyright © 2024 LangChain, Inc.