Repository: langchain-ai/langchain
Pull request #31209: ollama: Add separate kwargs parameter for async client
Author: login: AlexeyBond
Created: 2025-05-13T12:44:50Z | Merged: 2025-05-15T20:10:11Z

**Description**:

Add a `async_client_kwargs` field to ollama chat/llm/embeddings adapters that is passed to async httpx client constructor.

**Motivation:**

In my use-case:
- chat/embedding model adapters may be created frequently, sometimes to be called just once or to never be called at all
- they may be used in bots sunc and async mode (not known at the moment they are created)

So, I want to keep a static transport instance maintaining connection pool, so model adapters can be created and destroyed freely. But that doesn't work when both sync and async functions are in use as I can only pass one transport instance for both sync and async client, while transport types must be different for them. So I can't make both sync and async calls use shared transport with current model adapter interfaces.

In this PR I add a separate `async_client_kwargs` that gets passed to async client constructor, so it will be possible to pass a separate transport instance. For sake of backwards compatibility, it is merged with `client_kwargs`, so nothing changes when it is not set.

I am unable to run linter right now, but the changes look ok.


Diff:
diff --git a/libs/partners/ollama/langchain_ollama/chat_models.py b/libs/partners/ollama/langchain_ollama/chat_models.py
index a7046ef79ca21..0bd1f9fd381dd 100644
--- a/libs/partners/ollama/langchain_ollama/chat_models.py
+++ b/libs/partners/ollama/langchain_ollama/chat_models.py
@@ -436,8 +436,22 @@ class Multiply(BaseModel):
     """Base url the model is hosted under."""
 
     client_kwargs: Optional[dict] = {}
-    """Additional kwargs to pass to the httpx Client.
-    For a full list of the params, see [this link](https://pydoc.dev/httpx/latest/httpx.Client.html)
+    """Additional kwargs to pass to the httpx clients. 
+    These arguments are passed to both synchronous and async clients.
+    Use sync_client_kwargs and async_client_kwargs to pass different arguments
+    to synchronous and asynchronous clients.
+    """
+
+    async_client_kwargs: Optional[dict] = {}
+    """Additional kwargs to merge with client_kwargs before
+    passing to the httpx AsyncClient.
+    For a full list of the params, see [this link](https://www.python-httpx.org/api/#asyncclient)
+    """
+
+    sync_client_kwargs: Optional[dict] = {}
+    """Additional kwargs to merge with client_kwargs before
+    passing to the httpx Client.
+    For a full list of the params, see [this link](https://www.python-httpx.org/api/#client)
     """
 
     _client: Client = PrivateAttr(default=None)  # type: ignore
@@ -503,8 +517,17 @@ def _chat_params(
     def _set_clients(self) -> Self:
         """Set clients to use for ollama."""
         client_kwargs = self.client_kwargs or {}
-        self._client = Client(host=self.base_url, **client_kwargs)
-        self._async_client = AsyncClient(host=self.base_url, **client_kwargs)
+
+        sync_client_kwargs = client_kwargs
+        if self.sync_client_kwargs:
+            sync_client_kwargs = {**sync_client_kwargs, **self.sync_client_kwargs}
+
+        async_client_kwargs = client_kwargs
+        if self.async_client_kwargs:
+            async_client_kwargs = {**async_client_kwargs, **self.async_client_kwargs}
+
+        self._client = Client(host=self.base_url, **sync_client_kwargs)
+        self._async_client = AsyncClient(host=self.base_url, **async_client_kwargs)
         return self
 
     def _convert_messages_to_ollama_messages(
diff --git a/libs/partners/ollama/langchain_ollama/embeddings.py b/libs/partners/ollama/langchain_ollama/embeddings.py
index c6327343d48d9..15643529cdafd 100644
--- a/libs/partners/ollama/langchain_ollama/embeddings.py
+++ b/libs/partners/ollama/langchain_ollama/embeddings.py
@@ -127,8 +127,22 @@ class OllamaEmbeddings(BaseModel, Embeddings):
     """Base url the model is hosted under."""
 
     client_kwargs: Optional[dict] = {}
-    """Additional kwargs to pass to the httpx Client. 
-    For a full list of the params, see [this link](https://pydoc.dev/httpx/latest/httpx.Client.html)
+    """Additional kwargs to pass to the httpx clients. 
+    These arguments are passed to both synchronous and async clients.
+    Use sync_client_kwargs and async_client_kwargs to pass different arguments
+    to synchronous and asynchronous clients.
+    """
+
+    async_client_kwargs: Optional[dict] = {}
+    """Additional kwargs to merge with client_kwargs before
+    passing to the httpx AsyncClient.
+    For a full list of the params, see [this link](https://www.python-httpx.org/api/#asyncclient)
+    """
+
+    sync_client_kwargs: Optional[dict] = {}
+    """Additional kwargs to merge with client_kwargs before
+    passing to the httpx Client.
+    For a full list of the params, see [this link](https://www.python-httpx.org/api/#client)
     """
 
     _client: Client = PrivateAttr(default=None)  # type: ignore
@@ -233,8 +247,17 @@ def _default_params(self) -> dict[str, Any]:
     def _set_clients(self) -> Self:
         """Set clients to use for ollama."""
         client_kwargs = self.client_kwargs or {}
-        self._client = Client(host=self.base_url, **client_kwargs)
-        self._async_client = AsyncClient(host=self.base_url, **client_kwargs)
+
+        sync_client_kwargs = client_kwargs
+        if self.sync_client_kwargs:
+            sync_client_kwargs = {**sync_client_kwargs, **self.sync_client_kwargs}
+
+        async_client_kwargs = client_kwargs
+        if self.async_client_kwargs:
+            async_client_kwargs = {**async_client_kwargs, **self.async_client_kwargs}
+
+        self._client = Client(host=self.base_url, **sync_client_kwargs)
+        self._async_client = AsyncClient(host=self.base_url, **async_client_kwargs)
         return self
 
     def embed_documents(self, texts: list[str]) -> list[list[float]]:
diff --git a/libs/partners/ollama/langchain_ollama/llms.py b/libs/partners/ollama/langchain_ollama/llms.py
index 5298fec0e3c9c..4a40af3e99db5 100644
--- a/libs/partners/ollama/langchain_ollama/llms.py
+++ b/libs/partners/ollama/langchain_ollama/llms.py
@@ -113,8 +113,22 @@ class OllamaLLM(BaseLLM):
     """Base url the model is hosted under."""
 
     client_kwargs: Optional[dict] = {}
-    """Additional kwargs to pass to the httpx Client. 
-    For a full list of the params, see [this link](https://pydoc.dev/httpx/latest/httpx.Client.html)
+    """Additional kwargs to pass to the httpx clients. 
+    These arguments are passed to both synchronous and async clients.
+    Use sync_client_kwargs and async_client_kwargs to pass different arguments
+    to synchronous and asynchronous clients.
+    """
+
+    async_client_kwargs: Optional[dict] = {}
+    """Additional kwargs to merge with client_kwargs before
+    passing to the httpx AsyncClient.
+    For a full list of the params, see [this link](https://www.python-httpx.org/api/#asyncclient)
+    """
+
+    sync_client_kwargs: Optional[dict] = {}
+    """Additional kwargs to merge with client_kwargs before
+    passing to the httpx Client.
+    For a full list of the params, see [this link](https://www.python-httpx.org/api/#client)
     """
 
     _client: Client = PrivateAttr(default=None)  # type: ignore
@@ -189,8 +203,17 @@ def _get_ls_params(
     def _set_clients(self) -> Self:
         """Set clients to use for ollama."""
         client_kwargs = self.client_kwargs or {}
-        self._client = Client(host=self.base_url, **client_kwargs)
-        self._async_client = AsyncClient(host=self.base_url, **client_kwargs)
+
+        sync_client_kwargs = client_kwargs
+        if self.sync_client_kwargs:
+            sync_client_kwargs = {**sync_client_kwargs, **self.sync_client_kwargs}
+
+        async_client_kwargs = client_kwargs
+        if self.async_client_kwargs:
+            async_client_kwargs = {**async_client_kwargs, **self.async_client_kwargs}
+
+        self._client = Client(host=self.base_url, **sync_client_kwargs)
+        self._async_client = AsyncClient(host=self.base_url, **async_client_kwargs)
         return self
 
     async def _acreate_generate_stream(


Reviews:
user: login: AlexeyBond
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-05-13T12:45:34Z
user: login: ccurme
body: It seems a little unintuitive that we should merge these vs. just maintaining separate (potentially redundant) sets of kwargs.

For instance, if there's a key in `client_kwargs`, it's not possible to initialize the async client without this key, right?

Not sure how realistic this is but let me know what you think.
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-05-13T19:19:34Z
user: login: ccurme
body: Thank you!
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-05-15T20:10:05Z

Review comments:
diff_hunk: @@ -436,8 +436,13 @@ class Multiply(BaseModel):
     """Base url the model is hosted under."""
 
     client_kwargs: Optional[dict] = {}
-    """Additional kwargs to pass to the httpx Client.
-    For a full list of the params, see [this link](https://pydoc.dev/httpx/latest/httpx.Client.html)
path: libs/partners/ollama/langchain_ollama/chat_models.py
user: login: AlexeyBond
body: Documentation link here seems broken, so I've changed it to a valid one.

created_at: 2025-05-13T12:45:34Z
author_association: CONTRIBUTOR
diff_hunk: @@ -436,8 +436,13 @@ class Multiply(BaseModel):
     """Base url the model is hosted under."""
 
     client_kwargs: Optional[dict] = {}
-    """Additional kwargs to pass to the httpx Client.
-    For a full list of the params, see [this link](https://pydoc.dev/httpx/latest/httpx.Client.html)
path: libs/partners/ollama/langchain_ollama/chat_models.py
user: login: ccurme
body: Thank you!
created_at: 2025-05-13T19:14:48Z
author_association: COLLABORATOR

PR comments:
user: login: vercel[bot]
created_at: 2025-05-13T12:44:55Z
body: [vc]: #ew2aTSdmw/E8zMCRT882Jdwz/vumdoZO3VqX+wkJEzY=:eyJpc01vbm9yZXBvIjp0cnVlLCJ0eXBlIjoiZ2l0aHViIiwicHJvamVjdHMiOlt7Im5hbWUiOiJsYW5nY2hhaW4iLCJyb290RGlyZWN0b3J5IjoiZG9jcyIsImluc3BlY3RvclVybCI6Imh0dHBzOi8vdmVyY2VsLmNvbS9sYW5nY2hhaW4vbGFuZ2NoYWluL0F6d0FyVkRURVJjYlhiemJHVkF2VnZMV2FNOUwiLCJwcmV2aWV3VXJsIjoibGFuZ2NoYWluLWdpdC1mb3JrLWFsZXhleWJvbmQtb2xsYW1hLWFzeW5jLWNsaS00YTgzYjgtbGFuZ2NoYWluLnZlcmNlbC5hcHAiLCJuZXh0Q29tbWl0U3RhdHVzIjoiSUdOT1JFRCIsImxpdmVGZWVkYmFjayI6eyJyZXNvbHZlZCI6MCwidW5yZXNvbHZlZCI6MCwidG90YWwiOjAsImxpbmsiOiIifX1dfQ==
**The latest updates on your projects**. Learn more about [Vercel for Git ↗︎](https://vercel.link/github-learn-more)


<details><summary>1 Skipped Deployment</summary>

| Name | Status | Preview | Comments | Updated (UTC) |
| :--- | :----- | :------ | :------- | :------ |
| **langchain** | ⬜️ Ignored ([Inspect](https://vercel.com/langchain/langchain/AzwArVDTERcbXbzbGVAvVvLWaM9L)) | [Visit Preview](https://langchain-git-fork-alexeybond-ollama-async-cli-4a83b8-langchain.vercel.app) |  | May 14, 2025 6:22am |
</details>


author_association: NONE
user: login: AlexeyBond
created_at: 2025-05-14T06:33:13Z
body: > It seems a little unintuitive that we should merge these vs. just maintaining separate (potentially redundant) sets of kwargs.
> 
> For instance, if there's a key in `client_kwargs`, it's not possible to initialize the async client without this key, right?
> 
> Not sure how realistic this is but let me know what you think.

I don't think it is a significant issue, at least for now - as most of the parameters (except for `transport`) of sync and async clients are the same and will have the same values. But I have added another field - `sync_client_kwargs` that will be merged to synchronous client parameters. So it will be possible to specify some parameters for sync client only.

I think `client_args` field should be kept and merged into parameters of both sync and async client. Both to keep backwards compatibility and for some convenience - as in most cases the additional parameters for sync and async clients are the same.

author_association: CONTRIBUTOR
user: login: ccurme
created_at: 2025-05-15T20:29:25Z
body: Just released.
author_association: COLLABORATOR