Repository: open-telemetry/opentelemetry-python-contrib

Issue:
Datadog exporter shouldn't parse span description to determine if there was an exception
The Datadog exporter currently [parses the status description](https://github.com/open-telemetry/opentelemetry-python-contrib/blob/master/exporter/opentelemetry-exporter-datadog/src/opentelemetry/exporter/datadog/exporter.py#L252) to determine if an exception was raised, yet [the specification](https://github.com/open-telemetry/opentelemetry-specification/blob/master/specification/trace/api.md#record-exception) requires these to be recorded as events (and [the python code already does this](https://github.com/open-telemetry/opentelemetry-python/blob/master/opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py#L690-L715)), so the Datadog exporter should read this data from the event list instead of hoping the description is properly formatted.

The current code does say that [span events are not supported ](https://github.com/open-telemetry/opentelemetry-python-contrib/blob/master/exporter/opentelemetry-exporter-datadog/src/opentelemetry/exporter/datadog/exporter.py#L252), and while that would suggest that *reporting* events are not supported, we could certainly parse them for this data.

I'll probably tackle this, I'm already working on #154.  Also mentioned in #236, where I worked around the current behavior.


Discussion:
//cc @majorgreys 
Thanks for reporting this @alertedsnake! Your suggestion makes sense. If you end up tackling this, just give a heads up here and I'll check in before giving it a go myself.
@owais @majorgreys can you assign this to me? I'll try to address this and bring in line with spec compliane , sorry for the delay
@ericmustin thanks!  I got bogged down trying to fix #262 that I haven't had a chance to look at this one.
@alertedsnake no worries, only one of us gets time during their day job to do this 🙃 . May ask for a review as my python is quite rusty, will ping on PR
@ericmustin assigned!

Resolving patch:
diff --git a/exporter/opentelemetry-exporter-datadog/src/opentelemetry/exporter/datadog/constants.py b/exporter/opentelemetry-exporter-datadog/src/opentelemetry/exporter/datadog/constants.py
--- a/exporter/opentelemetry-exporter-datadog/src/opentelemetry/exporter/datadog/constants.py
+++ b/exporter/opentelemetry-exporter-datadog/src/opentelemetry/exporter/datadog/constants.py
@@ -7,3 +7,10 @@
 ENV_KEY = "env"
 VERSION_KEY = "version"
 SERVICE_NAME_TAG = "service.name"
+EVENT_NAME_EXCEPTION = "exception"
+EXCEPTION_TYPE_ATTR_KEY = "exception.type"
+EXCEPTION_MSG_ATTR_KEY = "exception.message"
+EXCEPTION_STACK_ATTR_KEY = "exception.stacktrace"
+DD_ERROR_TYPE_TAG_KEY = "error.type"
+DD_ERROR_MSG_TAG_KEY = "error.msg"
+DD_ERROR_STACK_TAG_KEY = "error.stack"
diff --git a/exporter/opentelemetry-exporter-datadog/src/opentelemetry/exporter/datadog/exporter.py b/exporter/opentelemetry-exporter-datadog/src/opentelemetry/exporter/datadog/exporter.py
--- a/exporter/opentelemetry-exporter-datadog/src/opentelemetry/exporter/datadog/exporter.py
+++ b/exporter/opentelemetry-exporter-datadog/src/opentelemetry/exporter/datadog/exporter.py
@@ -22,8 +22,15 @@
 
 import opentelemetry.trace as trace_api
 from opentelemetry.exporter.datadog.constants import (
+    DD_ERROR_MSG_TAG_KEY,
+    DD_ERROR_STACK_TAG_KEY,
+    DD_ERROR_TYPE_TAG_KEY,
     DD_ORIGIN,
     ENV_KEY,
+    EVENT_NAME_EXCEPTION,
+    EXCEPTION_MSG_ATTR_KEY,
+    EXCEPTION_STACK_ATTR_KEY,
+    EXCEPTION_TYPE_ATTR_KEY,
     SAMPLE_RATE_METRIC_KEY,
     SERVICE_NAME_TAG,
     VERSION_KEY,
@@ -145,11 +152,12 @@ def _translate_to_datadog(self, spans):
 
             if not span.status.is_ok:
                 datadog_span.error = 1
-                if span.status.description:
-                    exc_type, exc_val = _get_exc_info(span)
-                    # no mapping for error.stack since traceback not recorded
-                    datadog_span.set_tag("error.msg", exc_val)
-                    datadog_span.set_tag("error.type", exc_type)
+                # loop over events and look for exception events, extract info.
+                # https://github.com/open-telemetry/opentelemetry-python/blob/71e3a7a192c0fc8a7503fac967ada36a74b79e58/opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py#L810-L819
+                if span.events:
+                    _extract_tags_from_exception_events(
+                        span.events, datadog_span
+                    )
 
             # combine resource attributes and span attributes, don't modify existing span attributes
             combined_span_tags = {}
@@ -178,7 +186,7 @@ def _translate_to_datadog(self, spans):
             if sampling_rate is not None:
                 datadog_span.set_metric(SAMPLE_RATE_METRIC_KEY, sampling_rate)
 
-            # span events and span links are not supported
+            # span events and span links are not supported except for extracting exception event context
 
             datadog_spans.append(datadog_span)
 
@@ -318,3 +326,17 @@ def _extract_tags_from_resource(resource):
         else:
             tags[attribute_key] = attribute_value
     return [tags, service_name]
+
+
+def _extract_tags_from_exception_events(events, datadog_span):
+    """Parse error tags from exception events, error.msg error.type
+    and error.stack have special significance within datadog"""
+    for event in events:
+        if event.name is not None and event.name == EVENT_NAME_EXCEPTION:
+            for key, value in event.attributes.items():
+                if key == EXCEPTION_TYPE_ATTR_KEY:
+                    datadog_span.set_tag(DD_ERROR_TYPE_TAG_KEY, value)
+                elif key == EXCEPTION_MSG_ATTR_KEY:
+                    datadog_span.set_tag(DD_ERROR_MSG_TAG_KEY, value)
+                elif key == EXCEPTION_STACK_ATTR_KEY:
+                    datadog_span.set_tag(DD_ERROR_STACK_TAG_KEY, value)


Test patch:
diff --git a/exporter/opentelemetry-exporter-datadog/tests/test_datadog_exporter.py b/exporter/opentelemetry-exporter-datadog/tests/test_datadog_exporter.py
--- a/exporter/opentelemetry-exporter-datadog/tests/test_datadog_exporter.py
+++ b/exporter/opentelemetry-exporter-datadog/tests/test_datadog_exporter.py
@@ -388,6 +388,7 @@ def test_errors(self):
         self.assertEqual(span["error"], 1)
         self.assertEqual(span["meta"]["error.msg"], "bar")
         self.assertEqual(span["meta"]["error.type"], "ValueError")
+        self.assertTrue(span["meta"]["error.stack"] is not None)
 
     def test_shutdown(self):
         span_names = ["xxx", "bar", "foo"]