seansullivan/Langchain-Docs-June-26-2024 huggingface dataset:
!function(){function t(t){document.documentElement.setAttribute("data-theme",t)}var e=function(){var t=null;try{t=new URLSearchParams(window.location.search).get("docusaurus-theme")}catch(t){}return t}()||function(){var t=null;try{t=localStorage.getItem("theme")}catch(t){}return t}();null!==e?t(e):window.matchMedia("(prefers-color-scheme: dark)").matches?t("dark"):(window.matchMedia("(prefers-color-scheme: light)").matches,t("light"))}(),document.documentElement.setAttribute("data-announcement-bar-initially-dismissed",function(){try{return"true"===localStorage.getItem("docusaurus.announcement.dismiss")}catch(t){}return!1}())
*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to cache LLM responses

On this page

How to cache LLM responses
==========================

LangChain provides an optional caching layer for LLMs. This is useful for two reasons:

It can save you money by reducing the number of API calls you make to the LLM provider, if you're often requesting the same completion multiple times. It can speed up your application by reducing the number of API calls you make to the LLM provider.

    from langchain.globals import set_llm_cachefrom langchain_openai import OpenAI# To make the caching really obvious, lets use a slower model.llm = OpenAI(model_name="gpt-3.5-turbo-instruct", n=2, best_of=2)

**API Reference:**[set\_llm\_cache](https://api.python.langchain.com/en/latest/globals/langchain.globals.set_llm_cache.html) | [OpenAI](https://api.python.langchain.com/en/latest/llms/langchain_openai.llms.base.OpenAI.html)

    %%timefrom langchain.cache import InMemoryCacheset_llm_cache(InMemoryCache())# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")

**API Reference:**[InMemoryCache](https://api.python.langchain.com/en/latest/cache/langchain_community.cache.InMemoryCache.html)

    CPU times: user 13.7 ms, sys: 6.54 ms, total: 20.2 msWall time: 330 ms

    "\n\nWhy couldn't the bicycle stand up by itself? Because it was two-tired!"

    %%time# The second time it is, so it goes fasterllm.predict("Tell me a joke")

    CPU times: user 436 µs, sys: 921 µs, total: 1.36 msWall time: 1.36 ms

    "\n\nWhy couldn't the bicycle stand up by itself? Because it was two-tired!"

SQLite Cache[​](#sqlite-cache "Direct link to SQLite Cache")
------------------------------------------------------------

    !rm .langchain.db

    # We can do the same thing with a SQLite cachefrom langchain_community.cache import SQLiteCacheset_llm_cache(SQLiteCache(database_path=".langchain.db"))

**API Reference:**[SQLiteCache](https://api.python.langchain.com/en/latest/cache/langchain_community.cache.SQLiteCache.html)

    %%time# The first time, it is not yet in cache, so it should take longerllm.predict("Tell me a joke")

    CPU times: user 29.3 ms, sys: 17.3 ms, total: 46.7 msWall time: 364 ms

    '\n\nWhy did the tomato turn red?\n\nBecause it saw the salad dressing!'

    %%time# The second time it is, so it goes fasterllm.predict("Tell me a joke")

    CPU times: user 4.58 ms, sys: 2.23 ms, total: 6.8 msWall time: 4.68 ms

    '\n\nWhy did the tomato turn red?\n\nBecause it saw the salad dressing!'

[Edit this page](https://github.com/langchain-ai/langchain/edit/master/docs/docs/how_to/llm_caching.ipynb)

* * *

#### Was this page helpful?

  

#### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchain/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

LangChain Expression Language Cheatsheet

](/v0.2/docs/how_to/lcel_cheatsheet/)[

Next

How to track token usage for LLMs

](/v0.2/docs/how_to/llm_token_usage_tracking/)

*   [SQLite Cache](#sqlite-cache)
*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to use toolkits

How to use toolkits
===================

Toolkits are collections of tools that are designed to be used together for specific tasks. They have convenient loading methods. For a complete list of available ready-made toolkits, visit [Integrations](/v0.2/docs/integrations/toolkits/).

All Toolkits expose a `get_tools` method which returns a list of tools. You can therefore do:

    # Initialize a toolkittoolkit = ExampleTookit(...)# Get list of toolstools = toolkit.get_tools()# Create agentagent = create_agent_method(llm, tools, prompt)

[Edit this page](https://github.com/langchain-ai/langchain/edit/master/docs/docs/how_to/toolkits.mdx)

* * *

#### Was this page helpful?

  

#### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchain/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to return structured data from a model

](/v0.2/docs/how_to/structured_output/)[

Next

How to add ad-hoc tool calling capability to LLMs and Chat Models

](/v0.2/docs/how_to/tools_prompting/)
*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   tool\_calling\_parallel

On this page

tool\_calling\_parallel
=======================

### Disabling parallel tool calling (OpenAI only)[​](#disabling-parallel-tool-calling-openai-only "Direct link to Disabling parallel tool calling (OpenAI only)")

OpenAI tool calling performs tool calling in parallel by default. That means that if we ask a question like "What is the weather in Tokyo, New York, and Chicago?" and we have a tool for getting the weather, it will call the tool 3 times in parallel. We can force it to call only a single tool once by using the `parallel_tool_call` parameter.

First let's set up our tools and model:

    from langchain_core.tools import tool@tooldef add(a: int, b: int) -> int:    """Adds a and b."""    return a + b@tooldef multiply(a: int, b: int) -> int:    """Multiplies a and b."""    return a * btools = [add, multiply]

**API Reference:**[tool](https://api.python.langchain.com/en/latest/tools/langchain_core.tools.tool.html)

    import osfrom getpass import getpassfrom langchain_openai import ChatOpenAIos.environ["OPENAI_API_KEY"] = getpass()llm = ChatOpenAI(model="gpt-3.5-turbo-0125", temperature=0)

**API Reference:**[ChatOpenAI](https://api.python.langchain.com/en/latest/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html)

Now let's show a quick example of how disabling parallel tool calls work:

    llm_with_tools = llm.bind_tools(tools, parallel_tool_calls=False)llm_with_tools.invoke("Please call the first tool two times").tool_calls

    [{'name': 'add',  'args': {'a': 2, 'b': 2},  'id': 'call_Hh4JOTCDM85Sm9Pr84VKrWu5'}]

As we can see, even though we explicitly told the model to call a tool twice, by disabling parallel tool calls the model was constrained to only calling one.

[Edit this page](https://github.com/langchain-ai/langchain/edit/master/docs/docs/how_to/tool_calling_parallel.ipynb)

* * *

#### Was this page helpful?

  

#### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchain/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to use a model to call tools

](/v0.2/docs/how_to/tool_calling/)[

Next

How to force tool calling behavior

](/v0.2/docs/how_to/tool_choice/)

*   [Disabling parallel tool calling (OpenAI only)](#disabling-parallel-tool-calling-openai-only)
*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to force tool calling behavior

How to force tool calling behavior
==================================

In order to force our LLM to spelect a specific tool, we can use the `tool_choice` parameter to ensure certain behavior. First, let's define our model and tools:

    from langchain_core.tools import tool@tooldef add(a: int, b: int) -> int:    """Adds a and b."""    return a + b@tooldef multiply(a: int, b: int) -> int:    """Multiplies a and b."""    return a * btools = [add, multiply]

**API Reference:**[tool](https://api.python.langchain.com/en/latest/tools/langchain_core.tools.tool.html)

For example, we can force our tool to call the multiply tool by using the following code:

    llm_forced_to_multiply = llm.bind_tools(tools, tool_choice="Multiply")llm_forced_to_multiply.invoke("what is 2 + 4")

    AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_9cViskmLvPnHjXk9tbVla5HA', 'function': {'arguments': '{"a":2,"b":4}', 'name': 'Multiply'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 9, 'prompt_tokens': 103, 'total_tokens': 112}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-095b827e-2bdd-43bb-8897-c843f4504883-0', tool_calls=[{'name': 'Multiply', 'args': {'a': 2, 'b': 4}, 'id': 'call_9cViskmLvPnHjXk9tbVla5HA'}], usage_metadata={'input_tokens': 103, 'output_tokens': 9, 'total_tokens': 112})

Even if we pass it something that doesn't require multiplcation - it will still call the tool!

We can also just force our tool to select at least one of our tools by passing in the "any" (or "required" which is OpenAI specific) keyword to the `tool_choice` parameter.

    llm_forced_to_use_tool = llm.bind_tools(tools, tool_choice="any")llm_forced_to_use_tool.invoke("What day is today?")

    AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_mCSiJntCwHJUBfaHZVUB2D8W', 'function': {'arguments': '{"a":1,"b":2}', 'name': 'Add'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 15, 'prompt_tokens': 94, 'total_tokens': 109}, 'model_name': 'gpt-3.5-turbo-0125', 'system_fingerprint': None, 'finish_reason': 'stop', 'logprobs': None}, id='run-28f75260-9900-4bed-8cd3-f1579abb65e5-0', tool_calls=[{'name': 'Add', 'args': {'a': 1, 'b': 2}, 'id': 'call_mCSiJntCwHJUBfaHZVUB2D8W'}], usage_metadata={'input_tokens': 94, 'output_tokens': 15, 'total_tokens': 109})

[Edit this page](https://github.com/langchain-ai/langchain/edit/master/docs/docs/how_to/tool_choice.ipynb)

* * *

#### Was this page helpful?

  

#### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchain/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

tool\_calling\_parallel

](/v0.2/docs/how_to/tool_calling_parallel/)[

Next

How to pass tool outputs to the model

](/v0.2/docs/how_to/tool_results_pass_to_model/)
*   [](/v0.2/)
*   [How-to guides](/v0.2/docs/how_to/)
*   How to bind model-specific tools

How to bind model-specific tools
================================

Providers adopt different conventions for formatting tool schemas. For instance, OpenAI uses a format like this:

*   `type`: The type of the tool. At the time of writing, this is always `"function"`.
*   `function`: An object containing tool parameters.
*   `function.name`: The name of the schema to output.
*   `function.description`: A high level description of the schema to output.
*   `function.parameters`: The nested details of the schema you want to extract, formatted as a [JSON schema](https://json-schema.org/) dict.

We can bind this model-specific format directly to the model as well if preferred. Here's an example:

    from langchain_openai import ChatOpenAImodel = ChatOpenAI()model_with_tools = model.bind(    tools=[        {            "type": "function",            "function": {                "name": "multiply",                "description": "Multiply two integers together.",                "parameters": {                    "type": "object",                    "properties": {                        "a": {"type": "number", "description": "First integer"},                        "b": {"type": "number", "description": "Second integer"},                    },                    "required": ["a", "b"],                },            },        }    ])model_with_tools.invoke("Whats 119 times 8?")

**API Reference:**[ChatOpenAI](https://api.python.langchain.com/en/latest/chat_models/langchain_openai.chat_models.base.ChatOpenAI.html)

    AIMessage(content='', additional_kwargs={'tool_calls': [{'id': 'call_mn4ELw1NbuE0DFYhIeK0GrPe', 'function': {'arguments': '{"a":119,"b":8}', 'name': 'multiply'}, 'type': 'function'}]}, response_metadata={'token_usage': {'completion_tokens': 17, 'prompt_tokens': 62, 'total_tokens': 79}, 'model_name': 'gpt-3.5-turbo', 'system_fingerprint': 'fp_c2295e73ad', 'finish_reason': 'tool_calls', 'logprobs': None}, id='run-353e8a9a-7125-4f94-8c68-4f3da4c21120-0', tool_calls=[{'name': 'multiply', 'args': {'a': 119, 'b': 8}, 'id': 'call_mn4ELw1NbuE0DFYhIeK0GrPe'}])

This is functionally equivalent to the `bind_tools()` method.

[Edit this page](https://github.com/langchain-ai/langchain/edit/master/docs/docs/how_to/tools_model_specific.ipynb)

* * *

#### Was this page helpful?

  

#### You can also leave detailed feedback [on GitHub](https://github.com/langchain-ai/langchain/issues/new?assignees=&labels=03+-+Documentation&projects=&template=documentation.yml&title=DOC%3A+%3CPlease+write+a+comprehensive+title+after+the+%27DOC%3A+%27+prefix%3E).

[

Previous

How to add a human-in-the-loop for tools

](/v0.2/docs/how_to/tools_human/)[

Next

How to trim messages

](/v0.2/docs/how_to/trim_messages/)