Repository: open-telemetry/opentelemetry-python-contrib

Issue:
Ignore exception attributes for a given route or status (Pyramid)
It would be convenient to provide a way to specify routes and/or status codes are not exceptional.

A common pattern within Pyramid is to raise the desired response for certain business flows (e.g. raise a 301 to force a redirect, raise a 204 after deleting an entity, etc) rather than properly return. Because the response was raised and handled as an `HTTPException` the span is decorated with exception attributes even though the request was not actually exceptional. This causes many vendors to interpret a false-positive when calculating things like error rate.


There is history of this functionality in other instrumentation:

 - https://ddtrace.readthedocs.io/en/stable/advanced_usage.html#custom-error-codes
 - https://docs.newrelic.com/docs/apm/agents/python-agent/configuration/python-agent-configuration/#error-ignore
 - https://docs.newrelic.com/docs/apm/agents/python-agent/configuration/python-agent-configuration/#error-ignore-status-codes



Discussion:
Having investigated this further, it looks like Pyramid instrumentation is miscategorizing most 200s and 300s exceptions as errors. Created a pull to fix that.
I if understand correctly, we should ignore 4xx as well and only record 5xx exceptions as errors. According to the spec:

> For HTTP status codes in the 4xx range span status MUST be left unset in case of SpanKind.SERVER and MUST be set to Error in case of SpanKind.CLIENT.

> For HTTP status codes in the 5xx range, as well as any other code the client failed to interpret, span status MUST be set to Error.

I think we are talking only about server spans here right? In that case only 5xx should be set to the span status as an error unless I misunderstood something.

https://opentelemetry.io/docs/reference/specification/trace/semantic_conventions/http/#status

Resolving patch:
diff --git a/instrumentation/opentelemetry-instrumentation-pyramid/src/opentelemetry/instrumentation/pyramid/callbacks.py b/instrumentation/opentelemetry-instrumentation-pyramid/src/opentelemetry/instrumentation/pyramid/callbacks.py
--- a/instrumentation/opentelemetry-instrumentation-pyramid/src/opentelemetry/instrumentation/pyramid/callbacks.py
+++ b/instrumentation/opentelemetry-instrumentation-pyramid/src/opentelemetry/instrumentation/pyramid/callbacks.py
@@ -15,7 +15,7 @@
 from logging import getLogger
 
 from pyramid.events import BeforeTraversal
-from pyramid.httpexceptions import HTTPError, HTTPException
+from pyramid.httpexceptions import HTTPException, HTTPServerError
 from pyramid.settings import asbool
 from pyramid.tweens import EXCVIEW
 
@@ -198,9 +198,9 @@ def trace_tween(request):
 
                 activation = request.environ.get(_ENVIRON_ACTIVATION_KEY)
 
-                # Only considering HTTPClientError and HTTPServerError
-                # to make sure HTTPRedirection is not reported as error
-                if isinstance(response, HTTPError):
+                # Only considering HTTPServerError
+                # to make sure 200, 300 and 400 exceptions are not reported as error
+                if isinstance(response, HTTPServerError):
                     activation.__exit__(
                         type(response),
                         response,


Test patch:
diff --git a/instrumentation/opentelemetry-instrumentation-pyramid/tests/pyramid_base_test.py b/instrumentation/opentelemetry-instrumentation-pyramid/tests/pyramid_base_test.py
--- a/instrumentation/opentelemetry-instrumentation-pyramid/tests/pyramid_base_test.py
+++ b/instrumentation/opentelemetry-instrumentation-pyramid/tests/pyramid_base_test.py
@@ -24,6 +24,8 @@ def _hello_endpoint(request):
         helloid = int(request.matchdict["helloid"])
         if helloid == 500:
             raise exc.HTTPInternalServerError()
+        if helloid == 404:
+            raise exc.HTTPNotFound()
         if helloid == 302:
             raise exc.HTTPFound()
         if helloid == 204:
diff --git a/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_automatic.py b/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_automatic.py
--- a/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_automatic.py
+++ b/instrumentation/opentelemetry-instrumentation-pyramid/tests/test_automatic.py
@@ -136,6 +136,27 @@ def test_204_empty_response_is_not_an_error(self):
         span_list = self.memory_exporter.get_finished_spans()
         self.assertEqual(len(span_list), 1)
 
+    def test_400s_response_is_not_an_error(self):
+        tween_list = "pyramid.tweens.excview_tween_factory"
+        config = Configurator(settings={"pyramid.tweens": tween_list})
+        self._common_initialization(config)
+        resp = self.client.get("/hello/404")
+        self.assertEqual(404, resp.status_code)
+        span_list = self.memory_exporter.get_finished_spans()
+        self.assertEqual(len(span_list), 1)
+        self.assertEqual(span_list[0].status.status_code, StatusCode.UNSET)
+
+        PyramidInstrumentor().uninstrument()
+
+        self.config = Configurator()
+
+        self._common_initialization(self.config)
+
+        resp = self.client.get("/hello/404")
+        self.assertEqual(404, resp.status_code)
+        span_list = self.memory_exporter.get_finished_spans()
+        self.assertEqual(len(span_list), 1)
+
 
 class TestWrappedWithOtherFramework(
     InstrumentationTest, TestBase, WsgiTestBase