Repository: open-telemetry/opentelemetry-python-contrib

Issue:
fix: elasticsearch `bulk` API produces lists
# Description

Fixes an issue seen in production where the elasticsearch bulk API would end up sending a list through this code path and not a dictionary.

For example:

```
File \"/code/platform_be/core/logic/ingest.py\", line 378, in update_source_doc_version_history
        bulk(es_client, bulk_es_updates)
      File \"/opt/python/lib/python3.10/site-packages/elasticsearch/helpers/actions.py\", line 521, in bulk
        for ok, item in streaming_bulk(
      File \"/opt/python/lib/python3.10/site-packages/elasticsearch/helpers/actions.py\", line 436, in streaming_bulk
        for data, (ok, info) in zip(
      File \"/opt/python/lib/python3.10/site-packages/elasticsearch/helpers/actions.py\", line 339, in _process_bulk_chunk
        resp = client.bulk(*args, operations=bulk_actions, **kwargs)  # type: ignore[arg-type]
      File \"/opt/python/lib/python3.10/site-packages/elasticsearch/_sync/client/utils.py\", line 414, in wrapped
        return api(*args, **kwargs)
      File \"/opt/python/lib/python3.10/site-packages/elasticsearch/_sync/client/__init__.py\", line 704, in bulk
        return self.perform_request(  # type: ignore[return-value]
      File \"/opt/python/lib/python3.10/site-packages/elasticsearch/_sync/client/_base.py\", line 285, in perform_request
        meta, resp_body = self.transport.perform_request(
      File \"/opt/python/lib/python3.10/site-packages/opentelemetry/instrumentation/elasticsearch/__init__.py\", line 242, in wrapper
        attributes[SpanAttributes.DB_STATEMENT] = sanitize_body(
      File \"/opt/python/lib/python3.10/site-packages/opentelemetry/instrumentation/elasticsearch/utils.py\", line 58, in sanitize_body
        flatten_body = _flatten_dict(body)
      File \"/opt/python/lib/python3.10/site-packages/opentelemetry/instrumentation/elasticsearch/utils.py\", line 31, in _flatten_dict
        for k, v in d.items():
    AttributeError: 'list' object has no attribute 'items'"
```

## Type of change

Please delete options that are not relevant.

- [x] Bug fix (non-breaking change which fixes an issue)

# How Has This Been Tested?

Please describe the tests that you ran to verify your changes. Provide instructions so we can reproduce. Please also list any relevant details for your test configuration

- [x] Unit tests

# Does This PR Require a Core Repo Change?

- [x] No.

# Checklist:

See [contributing.md](https://github.com/open-telemetry/opentelemetry-python-contrib/blob/main/CONTRIBUTING.md) for styleguide, changelog guidelines, and more.

- [ ] Followed the style guidelines of this project
- [ ] Changelogs have been updated
- [ ] Unit tests have been added
- [ ] Documentation has been updated



Discussion:
Hi @lzchen / @ocelotl / @shalevr -- Is there anything I can do to help move this forward?  This is preventing us from using ES instrumentation in our service.  Thanks!
Hello! I maintain the elasticsearch and elastic-transport packages at Elastic (and am actually working on native instrumentation - see https://github.com/elastic/elasticsearch-py/issues/2435). That said, I'm new to OpenTelemetry and I've never contributed to opentelemtry-python-contrib.

While most Elasticsearch APIs accept dictionaries, indeed the [bulk API](https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-bulk.html) accepts a list of dictionaries, as you found out. However, the [Semantic Conventions for Elasticsearch](https://opentelemetry.io/docs/specs/semconv/database/elasticsearch/) define `db.statement` as follows:

> The request body for a [search-type query](https://www.elastic.co/guide/en/elasticsearch/reference/current/search.html), as a json string.

This makes sense. The bulk API can be used to send arbitrarily large amount of data, and you would not want that to be part of your span attributes. I believe the correct fix here is to only set `db.statement` when one of those APIs is used: Search, Async search, Multi search, EQL search, ES|QL query, Terms enum, Search template, Multi search template or Render search template. This isn't very easy to do in opentelemetry-python-contrib because the client does not pass the name of the endpoint to the transport (yet, I'm working on it). Which means it has to be guessed from the URL.

I guess an easy way out here could also be to *not* set `db.statement` for bulk requests, which would solve your immediate issue, but would still not be compliant.
> I guess an easy way out here could also be to _not_ set `db.statement` for bulk requests, which would solve your immediate issue, but would still not be compliant.

Thanks for the context @pquentin -- I assume that the path forward for us would be to either wait for your native instrumentation or to do a stopgap fix here.  From what you described it sounds like the stopgap is to just not generate any data if `db.statement` happens to be a list.  This feels a bit like a heuristic, but if you think it's OK stopgap I'm happy to implement this.
Native instrumentation will only be available with a future version of elasticsearch-py, so I think you want to keep the existing instrumentation for older versions? (Though it would be nice to automatically disable for newer versions maybe? But it would be a breaking change since the native implementation follows the semantic convention which probably did not exist when this repo was started.)

Regarding the fix, the proper fix is definitely inspecting the URL to figure out if the request is a search request, but I also offered the stopgap as way to fix the most pressing issue with less effort. It's really up to you to decide what you implement! (But if you do ping me in the pull request, I'll be happy to take a look.)
Made this change much simpler and ran tests with `CORE_REPO_SHA=a1253585f66d63e7c05a19f070f3bfe0ab6460c1 tox3 -e py311-test-instrumentation-elasticsearch-6` -- @pquentin or @xrmx if you have ideas on how to test this I'm all ears, or is this good enough?
Ok, I'll look into adding a simple test

Resolving patch:
diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py
--- a/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py
+++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/src/opentelemetry/instrumentation/elasticsearch/__init__.py
@@ -245,9 +245,11 @@ def wrapper(wrapped, _, args, kwargs):
                 if method:
                     attributes["elasticsearch.method"] = method
                 if body:
-                    attributes[SpanAttributes.DB_STATEMENT] = sanitize_body(
-                        body
-                    )
+                    # Don't set db.statement for bulk requests, as it can be very large
+                    if isinstance(body, dict):
+                        attributes[
+                            SpanAttributes.DB_STATEMENT
+                        ] = sanitize_body(body)
                 if params:
                     attributes["elasticsearch.params"] = str(params)
                 if doc_id:


Test patch:
diff --git a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py
--- a/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py
+++ b/instrumentation/opentelemetry-instrumentation-elasticsearch/tests/test_elasticsearch.py
@@ -51,6 +51,8 @@
 
 Article = helpers.Article
 
+# pylint: disable=too-many-public-methods
+
 
 @mock.patch(
     "elasticsearch.connection.http_urllib3.Urllib3HttpConnection.perform_request"
@@ -486,3 +488,35 @@ def test_body_sanitization(self, _):
             sanitize_body(json.dumps(sanitization_queries.interval_query)),
             str(sanitization_queries.interval_query_sanitized),
         )
+
+    def test_bulk(self, request_mock):
+        request_mock.return_value = (1, {}, "")
+
+        es = Elasticsearch()
+        es.bulk(
+            [
+                {
+                    "_op_type": "index",
+                    "_index": "sw",
+                    "_doc_type": "_doc",
+                    "_id": 1,
+                    "doc": {"name": "adam"},
+                },
+                {
+                    "_op_type": "index",
+                    "_index": "sw",
+                    "_doc_type": "_doc",
+                    "_id": 1,
+                    "doc": {"name": "adam"},
+                },
+            ]
+        )
+
+        spans_list = self.get_finished_spans()
+        self.assertEqual(len(spans_list), 1)
+        span = spans_list[0]
+
+        # Check version and name in span's instrumentation info
+        self.assertEqualSpanInstrumentationInfo(
+            span, opentelemetry.instrumentation.elasticsearch
+        )