Repository: open-telemetry/opentelemetry-python-contrib

Issue:
Handle elasticsearch client native instrumentation
**Describe your environment**

Elasticsearch client added native opentelemetry instrumentation in [8.13.0](https://www.elastic.co/guide/en/elasticsearch/client/python-api/master/release-notes.html#rn-8-13-0) and so currently without manual intervention it is possible to have our instrumentation and the client one create spans concurrently.

**Steps to reproduce**

Add opentelemetry instrumentation using elasticsearch client 8.13.0

**What is the expected behavior?**

Only the elasticsearch client creates spans.

**What is the actual behavior?**

Both client and opentelemetry instrumentation creates their own spans.

**Additional context**

Java opentelemetry instrumentation disables its instrumentation if a new enough elasticsearch client is found https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/9337/
Handle elasticsearch client native instrumentation
**Describe your environment**

Elasticsearch client added native opentelemetry instrumentation in [8.13.0](https://www.elastic.co/guide/en/elasticsearch/client/python-api/master/release-notes.html#rn-8-13-0) and so currently without manual intervention it is possible to have our instrumentation and the client one create spans concurrently.

**Steps to reproduce**

Add opentelemetry instrumentation using elasticsearch client 8.13.0

**What is the expected behavior?**

Only the elasticsearch client creates spans.

**What is the actual behavior?**

Both client and opentelemetry instrumentation creates their own spans.

**Additional context**

Java opentelemetry instrumentation disables its instrumentation if a new enough elasticsearch client is found https://github.com/open-telemetry/opentelemetry-java-instrumentation/pull/9337/


Discussion:
Since we are hooking into transport `perform_request` we can check if one of its kwargs is `otel_span` (https://github.com/elastic/elasticsearch-py/blob/main/elasticsearch/_sync/client/_base.py#L271) and if its `enabled` attribute is True (https://github.com/elastic/elasticsearch-py/blob/main/elasticsearch/_otel.py#L46).
Before fixing this we need to make tests pass with elasticsearch 7 and 8
Since we are hooking into transport `perform_request` we can check if one of its kwargs is `otel_span` (https://github.com/elastic/elasticsearch-py/blob/main/elasticsearch/_sync/client/_base.py#L271) and if its `enabled` attribute is True (https://github.com/elastic/elasticsearch-py/blob/main/elasticsearch/_otel.py#L46).
Before fixing this we need to make tests pass with elasticsearch 7 and 8

Resolving patch:
diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py
--- a/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py
+++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py
@@ -16,6 +16,15 @@
 This library allows tracing HTTP elasticsearch made by the
 `elasticsearch <https://elasticsearch-py.readthedocs.io/en/master/>`_ library.
 
+.. warning::
+    The elasticsearch package got native OpenTelemetry support since version
+    `8.13 <https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/release-notes.html#rn-8-13-0>`_.
+    To avoid duplicated tracing this instrumentation disables itself if it finds an elasticsearch client
+    that has OpenTelemetry support enabled.
+
+    Please be aware that the two libraries may use a different semantic convention, see
+    `elasticsearch documentation <https://www.elastic.co/guide/en/elasticsearch/client/python-api/current/opentelemetry.html>`_.
+
 Usage
 -----
 
@@ -54,7 +63,7 @@ def response_hook(span: Span, response: dict)
 
 for example:
 
-.. code: python
+.. code-block: python
 
     from opentelemetry.instrumentation.elasticsearch import ElasticsearchInstrumentor
     import elasticsearch
@@ -81,6 +90,7 @@ def response_hook(span, response):
 """
 
 import re
+import warnings
 from logging import getLogger
 from os import environ
 from typing import Collection
@@ -197,6 +207,16 @@ def _wrap_perform_request(
 ):
     # pylint: disable=R0912,R0914
     def wrapper(wrapped, _, args, kwargs):
+        # if wrapped elasticsearch has native OTel instrumentation just call the wrapped function
+        otel_span = kwargs.get("otel_span")
+        if otel_span and otel_span.otel_span:
+            warnings.warn(
+                "Instrumentation disabled, relying on elasticsearch native OTel support, see "
+                "https://opentelemetry-python-contrib.readthedocs.io/en/latest/instrumentation/elasticsearch/elasticsearch.html",
+                Warning,
+            )
+            return wrapped(*args, **kwargs)
+
         method = url = None
         try:
             method, url, *_ = args
@@ -249,6 +269,11 @@ def normalize_kwargs(k, v):
                             v = str(v)
                         elif isinstance(v, elastic_transport.HttpHeaders):
                             v = dict(v)
+                        elif isinstance(
+                            v, elastic_transport.OpenTelemetrySpan
+                        ):
+                            # the transport Span is always a dummy one
+                            v = None
                         return (k, v)
 
                     hook_kwargs = dict(


Test patch:
diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py
--- a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py
+++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py
@@ -23,6 +23,7 @@
 import elasticsearch.exceptions
 from elasticsearch import Elasticsearch
 from elasticsearch_dsl import Search
+from pytest import mark
 
 import opentelemetry.instrumentation.elasticsearch
 from opentelemetry import trace
@@ -36,7 +37,7 @@
 
 from . import sanitization_queries  # pylint: disable=no-name-in-module
 
-major_version = elasticsearch.VERSION[0]
+major_version, minor_version = elasticsearch.VERSION[:2]
 
 if major_version == 8:
     from . import helpers_es8 as helpers  # pylint: disable=no-name-in-module
@@ -70,6 +71,9 @@ def get_elasticsearch_client(*args, **kwargs):
 
 
 @mock.patch(helpers.perform_request_mock_path)
+@mock.patch.dict(
+    os.environ, {"OTEL_PYTHON_INSTRUMENTATION_ELASTICSEARCH_ENABLED": "false"}
+)
 class TestElasticsearchIntegration(TestBase):
     search_attributes = {
         SpanAttributes.DB_SYSTEM: "elasticsearch",
@@ -110,7 +114,6 @@ def test_instrumentor(self, request_mock):
         span = spans_list[0]
 
         # Check version and name in span's instrumentation info
-        # self.assertEqualSpanInstrumentationInfo(span, opentelemetry.instrumentation.elasticsearch)
         self.assertEqualSpanInstrumentationInfo(
             span, opentelemetry.instrumentation.elasticsearch
         )
@@ -475,6 +478,7 @@ def request_hook(span, method, url, kwargs):
                 "headers": {
                     "accept": "application/vnd.elasticsearch+json; compatible-with=8"
                 },
+                "otel_span": None,
             }
         elif major_version == 7:
             expected_kwargs = {
@@ -607,3 +611,30 @@ def test_bulk(self, request_mock):
         self.assertEqualSpanInstrumentationInfo(
             span, opentelemetry.instrumentation.elasticsearch
         )
+
+    @mark.skipif(
+        (major_version, minor_version) < (8, 13),
+        reason="Native OTel since elasticsearch 8.13",
+    )
+    @mock.patch.dict(
+        os.environ,
+        {"OTEL_PYTHON_INSTRUMENTATION_ELASTICSEARCH_ENABLED": "true"},
+    )
+    def test_instrumentation_is_disabled_if_native_support_enabled(
+        self, request_mock
+    ):
+        request_mock.return_value = helpers.mock_response("{}")
+
+        es = get_elasticsearch_client(hosts=["http://localhost:9200"])
+        es.index(
+            index="sw",
+            id=1,
+            **normalize_arguments(body={"name": "adam"}, doc_type="_doc"),
+        )
+
+        spans_list = self.get_finished_spans()
+        self.assertEqual(len(spans_list), 1)
+        span = spans_list[0]
+
+        # Check that name in span's instrumentation info is not from this instrumentation
+        self.assertEqual(span.instrumentation_info.name, "elasticsearch-api")