Repository: elastic/apm-agent-python

Issue:
ElasticAPM crash because of a missing context since version 6.3.0
**Describe the bug**: 

Since ElasticAPM release 6.3.0 our application has started to see a ton of crashes that we believed are cause by a recent change in ElasticAPM. The elasticsearch instrumentation package is trying to assign the type of DB to elasticsearch into an None context which causes it to crash. 

Here's a snippet of a strack trace from our app:
```
...
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 168, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticsearch/client/__init__.py", line 1026, in get
    "GET", _make_path(index, doc_type, id), params=params, headers=headers
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticapm/instrumentation/packages/base.py", line 210, in call_if_sampling
    return self.call(module, method, wrapped, instance, args, kwargs)
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticapm/instrumentation/packages/elasticsearch.py", line 113, in call
    result_data = wrapped(*args, **kwargs)
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticsearch/transport.py", line 388, in perform_request
    timeout=timeout,
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticapm/instrumentation/packages/base.py", line 210, in call_if_sampling
    return self.call(module, method, wrapped, instance, args, kwargs)
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticapm/instrumentation/packages/elasticsearch.py", line 56, in call
    self._update_context_by_request_data(span.context, instance, args, kwargs)
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticapm/instrumentation/packages/elasticsearch.py", line 72, in _update_context_by_request_data
    context["db"] = {"type": "elasticsearch"}
TypeError: 'NoneType' object does not support item assignment
```

I believe this crash is related to a recent change in this commit: https://github.com/elastic/apm-agent-python/commit/ee75cb8f238303b79d0f697a7f2eca547a1dfe8c#diff-c8fb731f92134757656c157f5c3175bcb62e131c1fed1aec5041367603c204d0L62

You can see here, the context was previously assigned it's DB type in a way where even if the context was None it would still work but now it assumes the context is a dictionary. I'm not creating a PR to fix this because I'm not 100% sure if the old way was changed for a reason. 

**Possible fix**
I have very limited understanding on what that context should be before reaching this function but possible fixes include: 

1. Revert to the old way of assigning the DB type.

2. To test for None context before assigning a type.
```
if context is None:
    context = {}
```

3. or make sure span.context default value is an empty dict instead of None

**To Reproduce**

I have no easy way to reproduce this crash because it does not happen all the time. 

**Environment (please complete the following information)**
- OS: Linux
- Python version: 3.7.11
- Framework and version [e.g. Django 2.1]: Flask 2.0.1
- APM Server version: 7.12
- Agent version: 6.3.0+

**Aditional Information**

Our app in launched in Gunicorn using gevent workers.
ElasticAPM crash because of a missing context since version 6.3.0
**Describe the bug**: 

Since ElasticAPM release 6.3.0 our application has started to see a ton of crashes that we believed are cause by a recent change in ElasticAPM. The elasticsearch instrumentation package is trying to assign the type of DB to elasticsearch into an None context which causes it to crash. 

Here's a snippet of a strack trace from our app:
```
...
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 168, in _wrapped
    return func(*args, params=params, headers=headers, **kwargs)
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticsearch/client/__init__.py", line 1026, in get
    "GET", _make_path(index, doc_type, id), params=params, headers=headers
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticapm/instrumentation/packages/base.py", line 210, in call_if_sampling
    return self.call(module, method, wrapped, instance, args, kwargs)
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticapm/instrumentation/packages/elasticsearch.py", line 113, in call
    result_data = wrapped(*args, **kwargs)
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticsearch/transport.py", line 388, in perform_request
    timeout=timeout,
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticapm/instrumentation/packages/base.py", line 210, in call_if_sampling
    return self.call(module, method, wrapped, instance, args, kwargs)
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticapm/instrumentation/packages/elasticsearch.py", line 56, in call
    self._update_context_by_request_data(span.context, instance, args, kwargs)
  File "/var/lib/assemblyline/.local/lib/python3.7/site-packages/elasticapm/instrumentation/packages/elasticsearch.py", line 72, in _update_context_by_request_data
    context["db"] = {"type": "elasticsearch"}
TypeError: 'NoneType' object does not support item assignment
```

I believe this crash is related to a recent change in this commit: https://github.com/elastic/apm-agent-python/commit/ee75cb8f238303b79d0f697a7f2eca547a1dfe8c#diff-c8fb731f92134757656c157f5c3175bcb62e131c1fed1aec5041367603c204d0L62

You can see here, the context was previously assigned it's DB type in a way where even if the context was None it would still work but now it assumes the context is a dictionary. I'm not creating a PR to fix this because I'm not 100% sure if the old way was changed for a reason. 

**Possible fix**
I have very limited understanding on what that context should be before reaching this function but possible fixes include: 

1. Revert to the old way of assigning the DB type.

2. To test for None context before assigning a type.
```
if context is None:
    context = {}
```

3. or make sure span.context default value is an empty dict instead of None

**To Reproduce**

I have no easy way to reproduce this crash because it does not happen all the time. 

**Environment (please complete the following information)**
- OS: Linux
- Python version: 3.7.11
- Framework and version [e.g. Django 2.1]: Flask 2.0.1
- APM Server version: 7.12
- Agent version: 6.3.0+

**Aditional Information**

Our app in launched in Gunicorn using gevent workers.


Discussion:
Same problem here
Same problem here

Resolving patch:
diff --git a/elasticapm/instrumentation/packages/elasticsearch.py b/elasticapm/instrumentation/packages/elasticsearch.py
--- a/elasticapm/instrumentation/packages/elasticsearch.py
+++ b/elasticapm/instrumentation/packages/elasticsearch.py
@@ -34,7 +34,7 @@
 
 import elasticapm
 from elasticapm.instrumentation.packages.base import AbstractInstrumentedModule
-from elasticapm.traces import execution_context
+from elasticapm.traces import DroppedSpan, execution_context
 from elasticapm.utils.logging import get_logger
 
 logger = get_logger("elasticapm.instrument")
@@ -52,6 +52,8 @@ class ElasticsearchConnectionInstrumentation(AbstractInstrumentedModule):
 
     def call(self, module, method, wrapped, instance, args, kwargs):
         span = execution_context.get_span()
+        if isinstance(span, DroppedSpan):
+            return wrapped(*args, **kwargs)
 
         self._update_context_by_request_data(span.context, instance, args, kwargs)
 


Test patch:
diff --git a/tests/instrumentation/elasticsearch_tests.py b/tests/instrumentation/elasticsearch_tests.py
--- a/tests/instrumentation/elasticsearch_tests.py
+++ b/tests/instrumentation/elasticsearch_tests.py
@@ -39,6 +39,7 @@
 from elasticsearch import Elasticsearch
 from elasticsearch.serializer import JSONSerializer
 
+import elasticapm
 from elasticapm.conf.constants import TRANSACTION
 from elasticapm.utils import compat
 
@@ -550,3 +551,17 @@ def test_custom_serializer(instrument, elasticapm_client, elasticsearch):
     span = spans[0]
     assert json.loads(span["context"]["db"]["statement"]) == json.loads('{"query":{"term":{"2":{"value":1}}}}')
     assert span["context"]["http"]["status_code"] == 200
+
+
+@pytest.mark.integrationtest
+def test_dropped_span(instrument, elasticapm_client, elasticsearch):
+    elasticapm_client.begin_transaction("test")
+    with elasticapm.capture_span("test", leaf=True):
+        elasticsearch.ping()
+    elasticapm_client.end_transaction("test", "OK")
+
+    transaction = elasticapm_client.events[TRANSACTION][0]
+    spans = elasticapm_client.spans_for_transaction(transaction)
+    assert len(spans) == 1
+    span = spans[0]
+    assert span["name"] == "test"