Repository: open-telemetry/opentelemetry-python

Issue:
Define where to handle invalid span contexts
Joining a conversation that's currently happening across two PRs:

* https://github.com/open-telemetry/opentelemetry-python/pull/226#issuecomment-544159432
* https://github.com/open-telemetry/opentelemetry-python/pull/228#pullrequestreview-304544636

It's important for us to standardize on what httptextformatters will return back when they are unable to parse the span context from their respective headers. If the choice is to return an invalid span context, then we should figure out what is responsible for handling that condition, and what the correct behavior is.

## Proposal

As a start, I'm proposing:

1. if a formatter is unable to retrieve a valid spancontext, then an invalid spancontext should be returned.
2. this should be handled in the propagator shim: https://github.com/open-telemetry/opentelemetry-python/blob/master/opentelemetry-api/src/opentelemetry/propagators/__init__.py#L26 and return back tracer.CURRENT_SPAN.

Here's alternatives and why I'm arguing against them:

## Handling invalid spans in the integrations

Handling this in the integrations would require the same boilerplate code to handle and convert the invalid span into the CURRENT_SPAN constant. I worry about someone missing that implementation detail resulting in incongruent behavior.

## Handling invalid spans in the SDK

I'm not 100% clear on what this would look like, but it would probably require making propagators and API interface and moving implementation code into the SDK. Not including this behavior would mean that the API alone is not enough to implement correct w3c tracecontext propagation (as invalid span contexts would not be converted to new ones).

I'm having trouble finding where it was stated that the API should propagate tracecontext by defaults. Maybe @c24t or @reyang can point me in the right direction?
Define where to handle invalid span contexts
Joining a conversation that's currently happening across two PRs:

* https://github.com/open-telemetry/opentelemetry-python/pull/226#issuecomment-544159432
* https://github.com/open-telemetry/opentelemetry-python/pull/228#pullrequestreview-304544636

It's important for us to standardize on what httptextformatters will return back when they are unable to parse the span context from their respective headers. If the choice is to return an invalid span context, then we should figure out what is responsible for handling that condition, and what the correct behavior is.

## Proposal

As a start, I'm proposing:

1. if a formatter is unable to retrieve a valid spancontext, then an invalid spancontext should be returned.
2. this should be handled in the propagator shim: https://github.com/open-telemetry/opentelemetry-python/blob/master/opentelemetry-api/src/opentelemetry/propagators/__init__.py#L26 and return back tracer.CURRENT_SPAN.

Here's alternatives and why I'm arguing against them:

## Handling invalid spans in the integrations

Handling this in the integrations would require the same boilerplate code to handle and convert the invalid span into the CURRENT_SPAN constant. I worry about someone missing that implementation detail resulting in incongruent behavior.

## Handling invalid spans in the SDK

I'm not 100% clear on what this would look like, but it would probably require making propagators and API interface and moving implementation code into the SDK. Not including this behavior would mean that the API alone is not enough to implement correct w3c tracecontext propagation (as invalid span contexts would not be converted to new ones).

I'm having trouble finding where it was stated that the API should propagate tracecontext by defaults. Maybe @c24t or @reyang can point me in the right direction?


Discussion:
I think the API will not be required to propagate context see https://github.com/open-telemetry/opentelemetry-specification/issues/208#issuecomment-538187193
> I'm having trouble finding where it was stated that the API should propagate tracecontext by defaults. Maybe @c24t or @reyang can point me in the right direction?

https://github.com/open-telemetry/opentelemetry-specification/issues/208 is probably the best source, this isn't really documented in the spec.

> I think the API will not be required to propagate context

I think it's too soon to call this issue. The discussion in the specs issue is better, but as I understand it there's no way for us to do all of:

- enable context propagation in applications via extensions alone (e.g. `ext-grpc` for gRPC, `ext-wsgi`/`ext-requests` for HTTP)
- not require libraries to depend on the SDK
- implement context propagation in the SDK only

From https://github.com/open-telemetry/opentelemetry-python/pull/228#pullrequestreview-304458212:

> I think we should rather fix the handling of INVALID_SPANCONTEXT in the SDK. Moving generate_spancontext to the API would break [open-telemetry/oteps#58](https://github.com/open-telemetry/oteps/pull/58)

@Oberon00 can you spell this out? Why does moving ID generation into the API make that OTEP impossible to implement?

Returning `INVALID_SPAN` from the formatter and converting to `CURRENT_SPAN` in the propagator sounds like a great solution to me, but it sounds like I'm missing some of the implications here.

Does it matter that integrations wouldn't be able to distinguish between invalid and unspecified spans?
Because a vendor-SDK might encode custom information in the span context, maybe even in the Trace or Span ID. Thinking about it, moving the generate_span_id might not be as bad as I first thought because any vendor SDK would still have to deal with incoming span contexts that were not created by it anyway.

Anyway, my suggestion would be: Handle INVALID_SPANCONTEXT  just like None when it is passed as parent. Maybe even disallow None as parent and only allow INVALID_SPANCONTEXT.
That's a good idea, it makes sure that things are handled in a standard way without adding additional logic to integrations. Although it does come at the cost of not being able to disambiguate no parent vs invalid parent, but I feel like that isn't a big deal.

@c24t @mauriciovasquezbernal any concerns with handling this in Tracer.start_span (or those collection of methods?)
I don't have concerns at this point. Just a question, propagators must then return `INVALID_SPANCONTEXT` when something fails. Is an SpanContext with trace_id and span_id 0 valid?

Just asking because the b3 propagator returns the later, a check like `context is INVALID_SPAN_CONTEXT` is False then.
I think the API will not be required to propagate context see https://github.com/open-telemetry/opentelemetry-specification/issues/208#issuecomment-538187193
> I'm having trouble finding where it was stated that the API should propagate tracecontext by defaults. Maybe @c24t or @reyang can point me in the right direction?

https://github.com/open-telemetry/opentelemetry-specification/issues/208 is probably the best source, this isn't really documented in the spec.

> I think the API will not be required to propagate context

I think it's too soon to call this issue. The discussion in the specs issue is better, but as I understand it there's no way for us to do all of:

- enable context propagation in applications via extensions alone (e.g. `ext-grpc` for gRPC, `ext-wsgi`/`ext-requests` for HTTP)
- not require libraries to depend on the SDK
- implement context propagation in the SDK only

From https://github.com/open-telemetry/opentelemetry-python/pull/228#pullrequestreview-304458212:

> I think we should rather fix the handling of INVALID_SPANCONTEXT in the SDK. Moving generate_spancontext to the API would break [open-telemetry/oteps#58](https://github.com/open-telemetry/oteps/pull/58)

@Oberon00 can you spell this out? Why does moving ID generation into the API make that OTEP impossible to implement?

Returning `INVALID_SPAN` from the formatter and converting to `CURRENT_SPAN` in the propagator sounds like a great solution to me, but it sounds like I'm missing some of the implications here.

Does it matter that integrations wouldn't be able to distinguish between invalid and unspecified spans?
Because a vendor-SDK might encode custom information in the span context, maybe even in the Trace or Span ID. Thinking about it, moving the generate_span_id might not be as bad as I first thought because any vendor SDK would still have to deal with incoming span contexts that were not created by it anyway.

Anyway, my suggestion would be: Handle INVALID_SPANCONTEXT  just like None when it is passed as parent. Maybe even disallow None as parent and only allow INVALID_SPANCONTEXT.
That's a good idea, it makes sure that things are handled in a standard way without adding additional logic to integrations. Although it does come at the cost of not being able to disambiguate no parent vs invalid parent, but I feel like that isn't a big deal.

@c24t @mauriciovasquezbernal any concerns with handling this in Tracer.start_span (or those collection of methods?)
I don't have concerns at this point. Just a question, propagators must then return `INVALID_SPANCONTEXT` when something fails. Is an SpanContext with trace_id and span_id 0 valid?

Just asking because the b3 propagator returns the later, a check like `context is INVALID_SPAN_CONTEXT` is False then.

Resolving patch:
diff --git a/opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py b/opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py
--- a/opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py
+++ b/opentelemetry-sdk/src/opentelemetry/sdk/trace/__init__.py
@@ -376,30 +376,33 @@ def create_span(
         as a child of the current span in this tracer's context, or as a root
         span if no current span exists.
         """
+        span_id = generate_span_id()
 
         if parent is Tracer.CURRENT_SPAN:
             parent = self.get_current_span()
 
-        if parent is None:
-            parent_context = None
-            new_span_context = trace_api.SpanContext(
-                generate_trace_id(), generate_span_id()
-            )
+        parent_context = parent
+        if isinstance(parent_context, trace_api.Span):
+            parent_context = parent.get_context()
+
+        if parent_context is not None and not isinstance(
+            parent_context, trace_api.SpanContext
+        ):
+            raise TypeError
+
+        if parent_context is None or not parent_context.is_valid():
+            parent = parent_context = None
+            trace_id = generate_trace_id()
+            trace_options = None
+            trace_state = None
         else:
-            if isinstance(parent, trace_api.Span):
-                parent_context = parent.get_context()
-            elif isinstance(parent, trace_api.SpanContext):
-                parent_context = parent
-            else:
-                # TODO: error handling
-                raise TypeError
-
-            new_span_context = trace_api.SpanContext(
-                parent_context.trace_id,
-                generate_span_id(),
-                parent_context.trace_options,
-                parent_context.trace_state,
-            )
+            trace_id = parent_context.trace_id
+            trace_options = parent_context.trace_options
+            trace_state = parent_context.trace_state
+
+        context = trace_api.SpanContext(
+            trace_id, span_id, trace_options, trace_state
+        )
 
         # The sampler decides whether to create a real or no-op span at the
         # time of span creation. No-op spans do not record events, and are not
@@ -408,8 +411,8 @@ def create_span(
         # to include information about the sampling decision.
         sampling_decision = self.sampler.should_sample(
             parent_context,
-            new_span_context.trace_id,
-            new_span_context.span_id,
+            context.trace_id,
+            context.span_id,
             name,
             {},  # TODO: links
         )
@@ -417,14 +420,15 @@ def create_span(
         if sampling_decision.sampled:
             return Span(
                 name=name,
-                context=new_span_context,
+                context=context,
                 parent=parent,
                 sampler=self.sampler,
                 attributes=sampling_decision.attributes,
                 span_processor=self._active_span_processor,
                 kind=kind,
             )
-        return trace_api.DefaultSpan(context=new_span_context)
+
+        return trace_api.DefaultSpan(context=context)
 
     @contextmanager
     def use_span(


Test patch:
diff --git a/opentelemetry-sdk/tests/trace/test_trace.py b/opentelemetry-sdk/tests/trace/test_trace.py
--- a/opentelemetry-sdk/tests/trace/test_trace.py
+++ b/opentelemetry-sdk/tests/trace/test_trace.py
@@ -51,6 +51,20 @@ def test_sampler_no_sampling(self):
 
 
 class TestSpanCreation(unittest.TestCase):
+    def test_create_span_invalid_spancontext(self):
+        """If an invalid span context is passed as the parent, the created
+        span should use a new span id.
+
+        Invalid span contexts should also not be added as a parent. This
+        eliminates redundant error handling logic in exporters.
+        """
+        tracer = trace.Tracer("test_create_span_invalid_spancontext")
+        new_span = tracer.create_span(
+            "root", parent=trace_api.INVALID_SPAN_CONTEXT
+        )
+        self.assertTrue(new_span.context.is_valid())
+        self.assertIsNone(new_span.parent)
+
     def test_start_span_implicit(self):
         tracer = trace.Tracer("test_start_span_implicit")