Repository: opensearch-project/opensearch-py
Pull request #603 (2023-11-17 19:31:00)

Fix: re-add service to fix integration with LangChain.
### Description

Re-add `.service`, which is [used by LangChain](https://github.com/langchain-ai/langchain/blob/b4312aac5c0567088353178fb70fdb356b372e12/libs/langchain/langchain/vectorstores/opensearch_vector_search.py#L83) to figure out whether we're talking to AOSS. This broke in https://github.com/opensearch-project/opensearch-py/pull/547.

### Issues Resolved

Closes https://github.com/opensearch-project/opensearch-py/issues/600

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check [here](https://github.com/opensearch-project/OpenSearch/blob/main/CONTRIBUTING.md#developer-certificate-of-origin).


navneet1v: @dblock its not necessary related to AOSS. Even in AOS, if user is asking for OpenSearch service to generate the ids for the data client should not send _id in the bulk request.
dblock: > @dblock its not necessary related to AOSS. Even in AOS/OSS, if user is asking for OpenSearch service to generate the ids for the data, os client should not send _id in the bulk request.

The code is pretty explicit :) 

```
def _is_aoss_enabled(http_auth: Any) -> bool:
    """Check if the service is http_auth is set as `aoss`."""
    if (
        http_auth is not None
        and hasattr(http_auth, "service")
        and http_auth.service == "aoss"
    ):
        return True
    return False
```
navneet1v: > > @dblock its not necessary related to AOSS. Even in AOS/OSS, if user is asking for OpenSearch service to generate the ids for the data, os client should not send _id in the bulk request.
> 
> The code is pretty explicit :)
> 
> ```python
> def _is_aoss_enabled(http_auth: Any) -> bool:
>     """Check if the service is http_auth is set as `aoss`."""
>     if (
>         http_auth is not None
>         and hasattr(http_auth, "service")
>         and http_auth.service == "aoss"
>     ):
>         return True
>     return False
> ```

This code is explicit, because AOSS doesn't support _id. But given that client is generic and AOSS support _id for search collection. So if we make this change then search collections will break.

@dblock 

This is the reason why I am saying the expectation from opensearch-py is if user has not provided _id, client should not add of its own whether it is AOS, OSS or AOSS. 
dblock: > This is the reason why I am saying the expectation from opensearch-py is if user has not provided _id, client should not add of its own whether it is AOS, OSS or AOSS.

Agreed. It does not. Am I missing anything?

Patch:
diff --git a/CHANGELOG.md b/CHANGELOG.md
index 93658586..69bdda60 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -8,6 +8,7 @@ Inspired from [Keep a Changelog](https://keepachangelog.com/en/1.0.0/)
 ### Removed
 ### Fixed
 - Fix `TypeError` on `parallel_bulk` ([#601](https://github.com/opensearch-project/opensearch-py/pull/601))
+- Fix Amazon OpenSearch Serverless integration with LangChain ([#603](https://github.com/opensearch-project/opensearch-py/pull/603))
 ### Security
 
 ## [2.4.1]
diff --git a/opensearchpy/helpers/signer.py b/opensearchpy/helpers/signer.py
index 930b8d25..43b5ee3c 100644
--- a/opensearchpy/helpers/signer.py
+++ b/opensearchpy/helpers/signer.py
@@ -78,6 +78,7 @@ class RequestsAWSV4SignerAuth(requests.auth.AuthBase):
 
     def __init__(self, credentials, region, service: str = "es") -> None:  # type: ignore
         self.signer = AWSV4Signer(credentials, region, service)
+        self.service = service  # tools like LangChain rely on this, see https://github.com/opensearch-project/opensearch-py/issues/600
 
     def __call__(self, request):  # type: ignore
         return self._sign_request(request)  # type: ignore
@@ -133,6 +134,7 @@ class AWSV4SignerAuth(RequestsAWSV4SignerAuth):
 class Urllib3AWSV4SignerAuth(Callable):  # type: ignore
     def __init__(self, credentials, region, service: str = "es") -> None:  # type: ignore
         self.signer = AWSV4Signer(credentials, region, service)
+        self.service = service  # tools like LangChain rely on this, see https://github.com/opensearch-project/opensearch-py/issues/600
 
     def __call__(self, method: str, url: str, body: Any) -> Dict[str, str]:
         return self.signer.sign(method, url, body)


Test patch:
diff --git a/test_opensearchpy/test_connection/test_requests_http_connection.py b/test_opensearchpy/test_connection/test_requests_http_connection.py
index bdfb97d7..62adf39f 100644
--- a/test_opensearchpy/test_connection/test_requests_http_connection.py
+++ b/test_opensearchpy/test_connection/test_requests_http_connection.py
@@ -460,6 +460,7 @@ class TestRequestsHttpConnection(TestCase):
         from opensearchpy.helpers.signer import RequestsAWSV4SignerAuth
 
         auth = RequestsAWSV4SignerAuth(self.mock_session(), region)
+        self.assertEqual(auth.service, "es")
         con = RequestsHttpConnection(http_auth=auth)
         prepared_request = requests.Request("GET", "http://localhost").prepare()
         auth(prepared_request)
@@ -478,6 +479,7 @@ class TestRequestsHttpConnection(TestCase):
         from opensearchpy.helpers.signer import RequestsAWSV4SignerAuth
 
         auth = RequestsAWSV4SignerAuth(self.mock_session(), region, service)
+        self.assertEqual(auth.service, service)
         con = RequestsHttpConnection(http_auth=auth)
         prepared_request = requests.Request("GET", "http://localhost").prepare()
         auth(prepared_request)
diff --git a/test_opensearchpy/test_connection/test_urllib3_http_connection.py b/test_opensearchpy/test_connection/test_urllib3_http_connection.py
index e22e943f..971a3254 100644
--- a/test_opensearchpy/test_connection/test_urllib3_http_connection.py
+++ b/test_opensearchpy/test_connection/test_urllib3_http_connection.py
@@ -192,6 +192,7 @@ class TestUrllib3HttpConnection(TestCase):
         from opensearchpy.helpers.signer import Urllib3AWSV4SignerAuth
 
         auth = Urllib3AWSV4SignerAuth(self.mock_session(), "us-west-2")
+        self.assertEqual(auth.service, "es")
         con = Urllib3HttpConnection(http_auth=auth, headers={"x": "y"})
         con.perform_request("GET", "/")
         self.assertEqual(mock_open.call_count, 1)
@@ -249,6 +250,7 @@ class TestUrllib3HttpConnection(TestCase):
         from opensearchpy.helpers.signer import Urllib3AWSV4SignerAuth
 
         auth = Urllib3AWSV4SignerAuth(self.mock_session(), region, service)
+        self.assertEqual(auth.service, service)
         headers = auth("GET", "http://localhost", None)
         self.assertIn("Authorization", headers)
         self.assertIn("X-Amz-Date", headers)