Repository: elastic/apm-agent-python

Issue:
Warning when capture_message in leaf span
**Describe the bug**: ...
Hi, we have following warning on some messages:

```
// Error: elasticapm.transport : Failed to submit message: 'HTTP 400: {"accepted":2,"errors":[{"message":"validation error: error: \'parent_id\' required when \'transaction_id\' is set","document":"{\\"error\\": {\\"context\\": {\\"custom\\": {\\"args\\": [], \\"pathname\\": \\"\\u003cmaya console\\u003e\\", \\"filename\\": \\"\\u003cmaya console\\u003e\\", \\"module\\": \\"\\u003cmaya console\\u003e\\", \\"stack_info\\": null, \\"lineno\\": 6, \\"funcName\\": \\"a\\", \\"thread\\": 19792, \\"threadName\\": \\"MainThread\\", \\"processName\\": \\"MainProcess\\", \\"process\\": 19788, \\"elasticapm_transaction_id\\": \\"c5c5fbeffeb7e0b7\\", \\"elasticapm_trace_id\\": \\"80736d829572c5953d620caaa0bee1b5\\", \\"elasticapm_span_id\\": null, \\"elasticapm_service_name\\": \\"maya\\", \\"elasticapm_labels\\": {\\"transaction.id\\": \\"c5c5fbeffeb7e0b7\\", \\"trace.id\\": \\"80736d829572c5953d620caaa0bee1b5\\", \\"span.id\\": null, \\"service.name\\": \\"maya\\"}, \\"message\\": \\"Test\\"}}, \\"id\\": \\"ecb569d067f9980d8d589bfc74a063f4\\", \\"log\\": {\\"level\\": \\"warning\\", \\"logger_name\\": \\"root\\", \\"message\\": \\"Test\\", \\"param_message\\": \\"Test\\"}, \\"timestamp\\": 1668508095566545, \\"trace_id\\": \\"80736d829572c5953d620caaa0bee1b5\\", \\"parent_id\\": null, \\"transaction_id\\": \\"c5c5fbeffeb7e0b7\\", \\"transaction\\": {\\"sampled\\": true, \\"type\\": \\"python\\", \\"name\\": \\"__main__.a\\"}}}"}]}\n'
Traceback (most recent call last):
  File "C:\Python37\lib\site-packages\elasticapm\transport\base.py", line 275, in _flush
  File "C:\Python37\lib\site-packages\elasticapm\transport\http.py", line 108, in send
elasticapm.transport.exceptions.TransportException: HTTP 400: {"accepted":2,"errors":[{"message":"validation error: error: 'parent_id' required when 'transaction_id' is set","document":"{\"error\": {\"context\": {\"custom\": {\"args\": [], \"pathname\": \"\u003cmaya console\u003e\", \"filename\": \"\u003cmaya console\u003e\", \"module\": \"\u003cmaya console\u003e\", \"stack_info\": null, \"lineno\": 6, \"funcName\": \"a\", \"thread\": 19792, \"threadName\": \"MainThread\", \"processName\": \"MainProcess\", \"process\": 19788, \"elasticapm_transaction_id\": \"c5c5fbeffeb7e0b7\", \"elasticapm_trace_id\": \"80736d829572c5953d620caaa0bee1b5\", \"elasticapm_span_id\": null, \"elasticapm_service_name\": \"maya\", \"elasticapm_labels\": {\"transaction.id\": \"c5c5fbeffeb7e0b7\", \"trace.id\": \"80736d829572c5953d620caaa0bee1b5\", \"span.id\": null, \"service.name\": \"maya\"}, \"message\": \"Test\"}}, \"id\": \"ecb569d067f9980d8d589bfc74a063f4\", \"log\": {\"level\": \"warning\", \"logger_name\": \"root\", \"message\": \"Test\", \"param_message\": \"Test\"}, \"timestamp\": 1668508095566545, \"trace_id\": \"80736d829572c5953d620caaa0bee1b5\", \"parent_id\": null, \"transaction_id\": \"c5c5fbeffeb7e0b7\", \"transaction\": {\"sampled\": true, \"type\": \"python\", \"name\": \"__main__.a\"}}}"}]}
 // 
```

**To Reproduce**

```
# start transaction
with capture_span('span', leaf=True):
    with capture_span('span', leaf=True):
        logging.warning('Test')
# end transaction
```

**Environment (please complete the following information)**
- Windows 21H2
- Python version: 3.7.7 (tags/v3.7.7:d7c567b08f, Mar 10 2020, 10:41:24) [MSC v.1900 64 bit (AMD64)]
- APM Server version: 7.15
- Agent version: 6.10.1

Some inspection was posted in [this ](https://github.com/elastic/apm-agent-python/issues/1415) ticket


Discussion:
@kalemas nice catch! I think the source of the bug is here:

https://github.com/elastic/apm-agent-python/blob/c0cfd4ecfe7885ab771b685cc7c63c6b42d31b54/elasticapm/base.py#L571

A `DroppedSpan` is not falsy, but doesn't have an `id`, so we end up setting the `parent_id` to `None`. 

The `span` object does have a reference to its parent, so we could walk up the chain until we find a non-dropped span, similar to what we do here:

https://github.com/elastic/apm-agent-python/blob/c0cfd4ecfe7885ab771b685cc7c63c6b42d31b54/elasticapm/instrumentation/packages/urllib3.py#L121-L123

Would you like to open a PR for this? 

Resolving patch:
diff --git a/elasticapm/base.py b/elasticapm/base.py
--- a/elasticapm/base.py
+++ b/elasticapm/base.py
@@ -50,7 +50,7 @@
 from elasticapm.conf import Config, VersionedConfig, constants
 from elasticapm.conf.constants import ERROR
 from elasticapm.metrics.base_metrics import MetricsRegistry
-from elasticapm.traces import Tracer, execution_context
+from elasticapm.traces import DroppedSpan, Tracer, execution_context
 from elasticapm.utils import cgroup, cloud, compat, is_master_process, stacks, varmap
 from elasticapm.utils.disttracing import TraceParent
 from elasticapm.utils.encoding import enforce_label_format, keyword_field, shorten, transform
@@ -469,6 +469,8 @@ def _build_msg_for_logging(
         """
         transaction = execution_context.get_transaction()
         span = execution_context.get_span()
+        while isinstance(span, DroppedSpan):
+            span = span.parent
         if transaction:
             transaction_context = deepcopy(transaction.context)
         else:


Test patch: