Repository: DataDog/dd-trace-go
Pull request #3219: baggage otel drop-in functionality
Author: login: rachelyangdog
Created: 2025-02-21T18:14:10Z | Merged: 2025-03-26T09:57:38Z

<!--
* New contributors are highly encouraged to read our
  [CONTRIBUTING](/CONTRIBUTING.md) documentation.
* Commit and PR titles should be prefixed with the general area of the pull request's change.

-->
### What does this PR do?
- Adds otel drop-in support. Baggage set on otel context objects will be merged in to Datadog context objects and vice versa. 
<!--
* A brief description of the change being made with this pull request.
* If the description here cannot be expressed in a succinct form, consider
  opening multiple pull requests instead of a single one.
-->

### Motivation

<!--
* What inspired you to submit this pull request?
* Link any related GitHub issues or PRs here.
* If this resolves a GitHub issue, include "Fixes #XXXX" to link the issue and auto-close it on merge.
-->

### Reviewer's Checklist
<!--
* Authors can use this list as a reference to ensure that there are no problems
  during the review but the signing off is to be done by the reviewer(s).
-->

- [ ] Changed code has unit tests for its functionality at or near 100% coverage.
- [ ] [System-Tests](https://github.com/DataDog/system-tests/) covering this feature have been added and enabled with the va.b.c-dev version tag.
- [ ] There is a benchmark for any new code, or changes to existing code.
- [ ] If this interacts with the agent in a new way, a system test has been added.
- [ ] Add an appropriate team label so this PR gets put in the right place for the release notes.
- [ ] Non-trivial go.mod changes, e.g. adding new modules, are reviewed by @DataDog/dd-trace-go-guild.
- [ ] For internal contributors, a matching PR should be created to the `v2-dev` branch and reviewed by @DataDog/apm-go.


Unsure? Have a question? Request a review!



Diff:
diff --git a/ddtrace/opentelemetry/otel_test.go b/ddtrace/opentelemetry/otel_test.go
index 5b766730251..9db6ce2173f 100644
--- a/ddtrace/opentelemetry/otel_test.go
+++ b/ddtrace/opentelemetry/otel_test.go
@@ -11,10 +11,12 @@ import (
 	"net/http/httptest"
 	"testing"
 
+	ddbaggage "github.com/DataDog/dd-trace-go/v2/ddtrace/baggage"
 	"github.com/stretchr/testify/assert"
 	"github.com/stretchr/testify/require"
 	"go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp"
 	"go.opentelemetry.io/otel"
+	otelbaggage "go.opentelemetry.io/otel/baggage"
 	"go.opentelemetry.io/otel/propagation"
 	oteltrace "go.opentelemetry.io/otel/trace"
 )
@@ -52,3 +54,71 @@ func TestHttpDistributedTrace(t *testing.T) {
 	assert.Equal("testRootSpan", p[1][0]["resource"])
 	assert.Equal("HTTP GET", p[1][1]["resource"])
 }
+
+// setupBaggageContext creates a context with both OpenTelemetry and Datadog baggage
+func setupBaggageContext(t *testing.T) (context.Context, oteltrace.Span) {
+	assert := assert.New(t)
+	tp, _, cleanup := mockTracerProvider(t)
+	t.Cleanup(cleanup)
+	otel.SetTracerProvider(tp)
+	otel.SetTextMapPropagator(propagation.Baggage{})
+	tr := tp.Tracer("")
+
+	ctx := context.Background()
+	otelMember, err := otelbaggage.NewMember("otel-key", "otel-value")
+	assert.NoError(err)
+	otelBag, err := otelbaggage.New(otelMember)
+	assert.NoError(err)
+	ctx = otelbaggage.ContextWithBaggage(ctx, otelBag)
+	ctx = ddbaggage.Set(ctx, "dd-key", "dd-value")
+
+	sctx, rootSpan := tr.Start(ctx, "testRootSpan")
+	return sctx, rootSpan
+}
+
+// setupTestServer creates a test HTTP server that captures baggage headers
+func setupTestServer(t *testing.T) (*httptest.Server, *string) {
+	var extractedBaggage string
+	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		extractedBaggage = r.Header.Get("baggage")
+		w.WriteHeader(http.StatusOK)
+	})
+	wrappedHandler := otelhttp.NewHandler(handler, "testOperation")
+	testServer := httptest.NewServer(wrappedHandler)
+	t.Cleanup(testServer.Close)
+	return testServer, &extractedBaggage
+}
+
+func TestHttpDistributedTraceWithBaggage(t *testing.T) {
+	assert := assert.New(t)
+	req := require.New(t)
+
+	// Setup context with baggage
+	sctx, rootSpan := setupBaggageContext(t)
+	defer rootSpan.End()
+
+	// Setup test server
+	testServer, extractedBaggage := setupTestServer(t)
+
+	// Make HTTP request with baggage context
+	client := http.Client{Transport: otelhttp.NewTransport(nil)}
+	httpReq, err := http.NewRequestWithContext(sctx, http.MethodGet, testServer.URL, nil)
+	req.NoError(err)
+	resp, err := client.Do(httpReq)
+	req.NoError(err)
+	req.NoError(resp.Body.Close())
+
+	// Verify baggage propagation
+	carrier := propagation.MapCarrier{}
+	otel.GetTextMapPropagator().Inject(sctx, carrier)
+
+	// Check that the injected "baggage" header includes both baggage items
+	injectedBaggage, ok := carrier["baggage"]
+	req.True(ok, "baggage header must be present in the injected carrier")
+	assert.Contains(injectedBaggage, "otel-key=otel-value")
+	assert.Contains(injectedBaggage, "dd-key=dd-value")
+
+	// Verify that the HTTP server received the merged baggage header
+	assert.Contains(*extractedBaggage, "otel-key=otel-value")
+	assert.Contains(*extractedBaggage, "dd-key=dd-value")
+}
diff --git a/ddtrace/opentelemetry/tracer.go b/ddtrace/opentelemetry/tracer.go
index c5b308e4234..8580827fa6b 100644
--- a/ddtrace/opentelemetry/tracer.go
+++ b/ddtrace/opentelemetry/tracer.go
@@ -10,10 +10,13 @@ import (
 	"encoding/binary"
 	"encoding/hex"
 
+	"github.com/DataDog/dd-trace-go/v2/ddtrace/baggage"
 	"github.com/DataDog/dd-trace-go/v2/ddtrace/ext"
 	"github.com/DataDog/dd-trace-go/v2/ddtrace/tracer"
 	"github.com/DataDog/dd-trace-go/v2/internal/telemetry"
+	"github.com/DataDog/dd-trace-go/v2/internal/telemetry/log"
 
+	otelbaggage "go.opentelemetry.io/otel/baggage"
 	oteltrace "go.opentelemetry.io/otel/trace"
 	"go.opentelemetry.io/otel/trace/noop"
 )
@@ -92,6 +95,14 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	// we have to record the attributes  locally.
 	// The span operation name will be calculated when it's ended.
 	s := tracer.StartSpan(spanName, ddopts...)
+
+	// Merge baggage from otel and dd, update Datadog baggage, and update the context.
+	mergedBag := mergeBaggageFromContext(ctx)
+	for _, m := range mergedBag.Members() {
+		ctx = baggage.Set(ctx, m.Key(), m.Value())
+	}
+	ctx = otelbaggage.ContextWithBaggage(ctx, mergedBag)
+
 	os := oteltrace.Span(&span{
 		DD:         s,
 		oteltracer: t,
@@ -105,6 +116,26 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	return ctx, os
 }
 
+// mergeBaggageFromContext merges Datadog baggage found on the context into the OpenTelemetry baggage found on the context.
+// adding key-value pairs only if the key doesn't already exist in the OpenTelemetry baggage. It ensures no existing values are overwritten.
+func mergeBaggageFromContext(ctx context.Context) otelbaggage.Baggage {
+	otelBag := otelbaggage.FromContext(ctx)
+	for key, value := range baggage.All(ctx) {
+		if otelBag.Member(key).Value() == "" {
+			member, err := otelbaggage.NewMemberRaw(key, value)
+			if err == nil {
+				b, err := otelBag.SetMember(member)
+				if err == nil {
+					otelBag = b
+				} else {
+					log.Debug("Error adding baggage member with key", key, "; dropping")
+				}
+			}
+		}
+	}
+	return otelBag
+}
+
 type otelCtxToDDCtx struct {
 	oc oteltrace.SpanContext
 }
diff --git a/ddtrace/opentelemetry/tracer_test.go b/ddtrace/opentelemetry/tracer_test.go
index 40e05f102d6..3c54299571e 100644
--- a/ddtrace/opentelemetry/tracer_test.go
+++ b/ddtrace/opentelemetry/tracer_test.go
@@ -7,11 +7,13 @@ package opentelemetry
 
 import (
 	"context"
+	"encoding/hex"
 	"fmt"
 	"sync"
 	"testing"
 	"time"
 
+	"github.com/DataDog/dd-trace-go/v2/ddtrace/baggage"
 	"github.com/DataDog/dd-trace-go/v2/ddtrace/tracer"
 	"github.com/DataDog/dd-trace-go/v2/internal/log"
 	"github.com/DataDog/dd-trace-go/v2/internal/telemetry"
@@ -20,6 +22,7 @@ import (
 	"github.com/stretchr/testify/assert"
 	"go.opentelemetry.io/otel"
 	"go.opentelemetry.io/otel/attribute"
+	otelbaggage "go.opentelemetry.io/otel/baggage"
 	oteltrace "go.opentelemetry.io/otel/trace"
 	"go.opentelemetry.io/otel/trace/noop"
 )
@@ -28,10 +31,9 @@ func TestGetTracer(t *testing.T) {
 	assert := assert.New(t)
 	tp := NewTracerProvider()
 	tr := tp.Tracer("ot")
-	dd := tracer.GetGlobalTracer()
 	ott, ok := tr.(*oteltracer)
 	assert.True(ok)
-	assert.Equal(ott.DD, dd)
+	assert.NotNil(ott.DD)
 }
 
 func TestGetTracerMultiple(t *testing.T) {
@@ -67,7 +69,11 @@ func TestSpanWithNewRoot(t *testing.T) {
 	assert.True(ok)
 	assert.Equal(got, child.(*span).DD)
 
-	assert.NotEqual(noopParent.Context().TraceID(), child.SpanContext().TraceID())
+	// Convert string TraceID to bytes for comparison
+	parentTraceID := noopParent.Context().TraceID()
+	parentBytes := make([]byte, 16)
+	hex.Decode(parentBytes, []byte(parentTraceID))
+	assert.NotEqual(parentBytes, child.SpanContext().TraceID())
 }
 
 func TestSpanWithoutNewRoot(t *testing.T) {
@@ -77,8 +83,7 @@ func TestSpanWithoutNewRoot(t *testing.T) {
 
 	parent, ddCtx := tracer.StartSpanFromContext(context.Background(), "otel.child")
 	_, child := tr.Start(ddCtx, "otel.child")
-	parentCtxW3C := parent.Context()
-	assert.Equal(parentCtxW3C.TraceIDBytes(), [16]byte(child.SpanContext().TraceID()))
+	assert.Equal(parent.Context().TraceID(), child.SpanContext().TraceID().String())
 }
 
 func TestTracerOptions(t *testing.T) {
@@ -308,3 +313,87 @@ func BenchmarkOTelConcurrentTracing(b *testing.B) {
 		}
 	}
 }
+
+func TestMergeOtelDDBaggage(t *testing.T) {
+	t.Run("otelBag and ddBag contain members", func(t *testing.T) {
+		assert := assert.New(t)
+
+		// Set up Datadog baggage first
+		ctx := context.Background()
+		ctx = baggage.Set(ctx, "testKey1", "ddValue1")
+		ctx = baggage.Set(ctx, "testKey2", "ddValue2")
+
+		// Set up OpenTelemetry baggage with one key that duplicated dd baggage, and one unique key
+		m1, err := otelbaggage.NewMember("testKey1", "otelValue1")
+		assert.NoError(err)
+		m2, err := otelbaggage.NewMember("testKey3", "otelValue3")
+		assert.NoError(err)
+		bag, err := otelbaggage.New(m1, m2)
+		assert.NoError(err)
+		ctx = otelbaggage.ContextWithBaggage(ctx, bag)
+
+		// Create tracer and start span
+		tp := NewTracerProvider()
+		otel.SetTracerProvider(tp)
+		tr := otel.Tracer("baggage.test")
+		ctx, span := tr.Start(ctx, "baggage.span")
+		defer span.End()
+
+		// Check DD Baggage API
+		value, ok := baggage.Get(ctx, "testKey1")
+		assert.True(ok)
+		assert.Equal("otelValue1", value) // Otel takes precedence on key conflict
+		value, ok = baggage.Get(ctx, "testKey2")
+		assert.True(ok)
+		assert.Equal("ddValue2", value)
+		value, ok = baggage.Get(ctx, "testKey3")
+		assert.True(ok)
+		assert.Equal("otelValue3", value)
+
+		otelBag := otelbaggage.FromContext(ctx)
+		assert.Equal("otelValue1", otelBag.Member("testKey1").Value())
+		assert.Equal("ddValue2", otelBag.Member("testKey2").Value())
+		assert.Equal("otelValue3", otelBag.Member("testKey3").Value())
+	})
+	t.Run("otelBag empty", func(t *testing.T) {
+		assert := assert.New(t)
+
+		// Set up Datadog baggage
+		ctx := context.Background()
+		ctx = baggage.Set(ctx, "testKey", "ddValue")
+
+		// Create tracer and start span
+		tp := NewTracerProvider()
+		otel.SetTracerProvider(tp)
+		tr := otel.Tracer("baggage.test")
+		ctx, span := tr.Start(ctx, "baggage.span")
+		defer span.End()
+
+		// Assert dd key is retrievable via otel baggage API
+		otelBag := otelbaggage.FromContext(ctx)
+		assert.Equal("ddValue", otelBag.Member("testKey").Value())
+	})
+	t.Run("ddBag empty", func(t *testing.T) {
+		assert := assert.New(t)
+
+		// Set up OpenTelemetry baggage
+		ctx := context.Background()
+		m, err := otelbaggage.NewMember("testKey", "otelValue")
+		assert.NoError(err)
+		bag, err := otelbaggage.New(m)
+		assert.NoError(err)
+		ctx = otelbaggage.ContextWithBaggage(ctx, bag)
+
+		// Create tracer and start span
+		tp := NewTracerProvider()
+		otel.SetTracerProvider(tp)
+		tr := otel.Tracer("baggage.test")
+		ctx, span := tr.Start(ctx, "baggage.span")
+		defer span.End()
+
+		// Assert otel key is retrievable via dd baggage API
+		value, ok := baggage.Get(ctx, "testKey")
+		assert.True(ok)
+		assert.Equal("otelValue", value)
+	})
+}


Reviews:
user: login: mtoffl01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-12T17:54:45Z
user: login: rachelyangdog
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T17:42:56Z
user: login: rachelyangdog
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-17T17:43:57Z
user: login: rachelyangdog
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-19T20:42:59Z
user: login: mtoffl01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T14:26:45Z
user: login: mtoffl01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-20T17:40:16Z
user: login: mtoffl01
body: Make sure to:

1. Address the issues raised by the github actions bot
2. Add tests for cases where you are deliberately trying to break your implementation, to prove that it's robust. E.g. what happens if you try to set a baggage item via the otel API with a value or key type that is not supported?  
state: CHANGES_REQUESTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-24T17:00:31Z
user: login: mtoffl01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T17:34:47Z
user: login: mtoffl01
state: COMMENTED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T17:35:41Z
user: login: mtoffl01
body: Failing tests seem unrelated. Try to clean up the otel_test file if you can, but good to merge otherwise!
state: APPROVED
author_association: CONTRIBUTOR
submitted_at: 2025-03-25T19:49:01Z
user: login: darccio
state: APPROVED
author_association: MEMBER
submitted_at: 2025-03-26T09:57:23Z

Review comments:
diff_hunk: @@ -92,6 +95,28 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	// we have to record the attributes  locally.
 	// The span operation name will be calculated when it's ended.
 	s := tracer.StartSpan(spanName, ddopts...)
+
+	// Get the current OpenTelemetry baggage.
+	otelBag := otelbaggage.FromContext(ctx)
+	// Get the ddtrace baggage as a map[string]string.
+	ddBag := baggage.All(ctx)
+
+	// Merge the two baggage maps.
+	// If there are conflicts, the OpenTelemetry baggage wins.
path: ddtrace/opentelemetry/tracer.go
user: login: mtoffl01
body: Really? Why? I'd expect the opposite: That DD baggage should override in the context of Datadog
created_at: 2025-03-12T17:50:38Z
author_association: CONTRIBUTOR
diff_hunk: @@ -92,6 +95,28 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	// we have to record the attributes  locally.
 	// The span operation name will be calculated when it's ended.
 	s := tracer.StartSpan(spanName, ddopts...)
+
+	// Get the current OpenTelemetry baggage.
+	otelBag := otelbaggage.FromContext(ctx)
+	// Get the ddtrace baggage as a map[string]string.
+	ddBag := baggage.All(ctx)
+
+	// Merge the two baggage maps.
+	// If there are conflicts, the OpenTelemetry baggage wins.
+	if len(ddBag) > 0 || otelBag.Len() > 0 {
+		var members []otelbaggage.Member
+		for key, value := range ddBag {
+			member, _ := otelbaggage.NewMember(key, value)
+			members = append(members, member)
+		}
+		members = append(members, otelBag.Members()...)
+		mergedBag, _ := otelbaggage.New(members...)
+		for _, member := range mergedBag.Members() {
+			s.SetBaggageItem(member.Key(), member.Value())
+		}
+		ctx = otelbaggage.ContextWithBaggage(ctx, mergedBag)
+	}
path: ddtrace/opentelemetry/tracer.go
user: login: mtoffl01
body: This is a lot of copying to do on every call to start an otel span. Can we find opportunities to optimize? e.g:
- rather than getting all the dd baggage and all the otel baggage, just check the size of each. If one is empty, just use the other. If neither is empty, then merge the smaller map onto the bigger map. 

Is something like this possible?
created_at: 2025-03-12T17:53:31Z
author_association: CONTRIBUTOR
diff_hunk: @@ -92,6 +95,28 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	// we have to record the attributes  locally.
 	// The span operation name will be calculated when it's ended.
 	s := tracer.StartSpan(spanName, ddopts...)
+
+	// Get the current OpenTelemetry baggage.
+	otelBag := otelbaggage.FromContext(ctx)
+	// Get the ddtrace baggage as a map[string]string.
+	ddBag := baggage.All(ctx)
+
+	// Merge the two baggage maps.
+	// If there are conflicts, the OpenTelemetry baggage wins.
path: ddtrace/opentelemetry/tracer.go
user: login: rachelyangdog
body: Hmm... This is up for debate. I just have it set to Otel right now. Idk why, it just sounded more intuitive to me. But, I've started a conversation about this in #apm-baggage! I can update the implementation accordingly! 
created_at: 2025-03-17T17:42:56Z
author_association: CONTRIBUTOR
diff_hunk: @@ -92,6 +95,28 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	// we have to record the attributes  locally.
 	// The span operation name will be calculated when it's ended.
 	s := tracer.StartSpan(spanName, ddopts...)
+
+	// Get the current OpenTelemetry baggage.
+	otelBag := otelbaggage.FromContext(ctx)
+	// Get the ddtrace baggage as a map[string]string.
+	ddBag := baggage.All(ctx)
+
+	// Merge the two baggage maps.
+	// If there are conflicts, the OpenTelemetry baggage wins.
+	if len(ddBag) > 0 || otelBag.Len() > 0 {
+		var members []otelbaggage.Member
+		for key, value := range ddBag {
+			member, _ := otelbaggage.NewMember(key, value)
+			members = append(members, member)
+		}
+		members = append(members, otelBag.Members()...)
+		mergedBag, _ := otelbaggage.New(members...)
+		for _, member := range mergedBag.Members() {
+			s.SetBaggageItem(member.Key(), member.Value())
+		}
+		ctx = otelbaggage.ContextWithBaggage(ctx, mergedBag)
+	}
path: ddtrace/opentelemetry/tracer.go
user: login: rachelyangdog
body: Ah ok, I can look into this! 
created_at: 2025-03-17T17:43:57Z
author_association: CONTRIBUTOR
diff_hunk: @@ -92,6 +95,28 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	// we have to record the attributes  locally.
 	// The span operation name will be calculated when it's ended.
 	s := tracer.StartSpan(spanName, ddopts...)
+
+	// Get the current OpenTelemetry baggage.
+	otelBag := otelbaggage.FromContext(ctx)
+	// Get the ddtrace baggage as a map[string]string.
+	ddBag := baggage.All(ctx)
+
+	// Merge the two baggage maps.
+	// If there are conflicts, the OpenTelemetry baggage wins.
path: ddtrace/opentelemetry/tracer.go
user: login: rachelyangdog
body: We decided that otel should take precedence to show that we're cooperating more with OpenTelemetry users. Also, there's some precedence where W3C trace context span-id takes precedence if there's conflict! 
created_at: 2025-03-19T20:42:58Z
author_association: CONTRIBUTOR
diff_hunk: @@ -92,6 +94,69 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	// we have to record the attributes  locally.
 	// The span operation name will be calculated when it's ended.
 	s := tracer.StartSpan(spanName, ddopts...)
+
+	// Get the current OpenTelemetry baggage.
+	otelBag := otelbaggage.FromContext(ctx)
+	// Get the ddtrace baggage as a map[string]string.
+	ddBag := baggage.All(ctx)
+
+	// Merge the two baggage maps.
+	// If there are conflicts, the OpenTelemetry baggage wins.
+	var mergedBag otelbaggage.Baggage
+	switch {
path: ddtrace/opentelemetry/tracer.go
user: login: mtoffl01
body: Nice! Can we refactor some of the logic in this switch statement into helper functions? This would:
- clean up the code inside of tracer.Start, which is already a massive function
- remove the need for so many in-line comments, as the helper function names would be sufficiently descriptive to showcase what you're doing, and why.

You could even extract the baggage logic (starting at line 98)out into its own helper function and use sub-"helper"-functions inside of that.
created_at: 2025-03-20T14:26:45Z
author_association: CONTRIBUTOR
diff_hunk: @@ -10,10 +10,12 @@ import (
 	"encoding/binary"
 	"encoding/hex"
 
-	"github.com/DataDog/dd-trace-go/v2/ddtrace/ext"
-	"github.com/DataDog/dd-trace-go/v2/ddtrace/tracer"
-	"github.com/DataDog/dd-trace-go/v2/internal/telemetry"
+	"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/baggage"
+	"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/ext"
+	"gopkg.in/DataDog/dd-trace-go.v1/ddtrace/tracer"
+	"gopkg.in/DataDog/dd-trace-go.v1/internal/telemetry"
path: ddtrace/opentelemetry/tracer.go
user: login: mtoffl01
body: Is this import intentional? I don't see it being used in any of the new code 🤔 
created_at: 2025-03-20T17:40:16Z
author_association: CONTRIBUTOR
diff_hunk: @@ -11,10 +11,12 @@ import (
 	"net/http/httptest"
 	"testing"
 
+	ddBaggage "github.com/DataDog/dd-trace-go/v2/ddtrace/baggage"
path: ddtrace/opentelemetry/otel_test.go
user: login: mtoffl01
body: ```suggestion
	ddbaggage "github.com/DataDog/dd-trace-go/v2/ddtrace/baggage"
```
created_at: 2025-03-24T14:40:41Z
author_association: CONTRIBUTOR
diff_hunk: @@ -52,3 +54,50 @@ func TestHttpDistributedTrace(t *testing.T) {
 	assert.Equal("testRootSpan", p[1][0]["resource"])
 	assert.Equal("HTTP GET", p[1][1]["resource"])
 }
+
+func TestHttpDistributedTraceWithBaggage(t *testing.T) {
+	assert := assert.New(t)
+	req := require.New(t)
+	tp, _, cleanup := mockTracerProvider(t)
+	defer cleanup()
+	otel.SetTracerProvider(tp)
+	otel.SetTextMapPropagator(propagation.Baggage{})
+	tr := tp.Tracer("")
+	ctx := context.Background()
+	otelMember, _ := otelbaggage.NewMember("otel-key", "otel-value")
+	otelBag, _ := otelbaggage.New(otelMember)
path: ddtrace/opentelemetry/otel_test.go
user: login: mtoffl01
body: still good practice to retrieve the error and assert it's nil, even in testing

```suggestion
	otelMember, err := otelbaggage.NewMember("otel-key", "otel-value")
	req.NoError(t, err)
	otelBag, err := otelbaggage.New(otelMember)
	req.NoError(t, err)
```
created_at: 2025-03-24T14:44:15Z
author_association: CONTRIBUTOR
diff_hunk: @@ -52,3 +54,50 @@ func TestHttpDistributedTrace(t *testing.T) {
 	assert.Equal("testRootSpan", p[1][0]["resource"])
 	assert.Equal("HTTP GET", p[1][1]["resource"])
 }
+
+func TestHttpDistributedTraceWithBaggage(t *testing.T) {
+	assert := assert.New(t)
+	req := require.New(t)
+	tp, _, cleanup := mockTracerProvider(t)
+	defer cleanup()
+	otel.SetTracerProvider(tp)
+	otel.SetTextMapPropagator(propagation.Baggage{})
+	tr := tp.Tracer("")
+	ctx := context.Background()
+	otelMember, _ := otelbaggage.NewMember("otel-key", "otel-value")
+	otelBag, _ := otelbaggage.New(otelMember)
+	ctx = otelbaggage.ContextWithBaggage(ctx, otelBag)
+	ctx = ddBaggage.Set(ctx, "dd-key", "dd-value")
+	sctx, rootSpan := tr.Start(ctx, "testRootSpan")
+	defer rootSpan.End()
+
+	var extractedBaggage string
+	handler := http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		// Capture the injected baggage header.
+		extractedBaggage = r.Header.Get("baggage")
+		w.WriteHeader(http.StatusOK)
+	})
+	wrappedHandler := otelhttp.NewHandler(handler, "testOperation")
+	testServer := httptest.NewServer(wrappedHandler)
+	defer testServer.Close()
+
+	client := http.Client{Transport: otelhttp.NewTransport(nil)}
+	httpReq, err := http.NewRequestWithContext(sctx, http.MethodGet, testServer.URL, nil)
+	req.NoError(err)
+	resp, err := client.Do(httpReq)
+	req.NoError(err)
+	req.NoError(resp.Body.Close())
+
+	carrier := propagation.MapCarrier{}
+	otel.GetTextMapPropagator().Inject(sctx, carrier)
+
+	// Check that the injected "baggage" header includes both baggage items.
path: ddtrace/opentelemetry/otel_test.go
user: login: mtoffl01
body: All of the code prior to this line is setup/boiler plate code; any way to refactor this test such that it's more readable/easier to know what you're actually testing for (really just lines 94-102, although lines 67-70 are also relevant)
created_at: 2025-03-24T14:47:17Z
author_association: CONTRIBUTOR
diff_hunk: @@ -105,6 +116,71 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	return ctx, os
 }
 
+// mergeBaggageFromContext consolidates baggage from OpenTelemetry and Datadog sources.
+func mergeBaggageFromContext(ctx context.Context) (otelbaggage.Baggage, error) {
+	otelBag := otelbaggage.FromContext(ctx)
+	ddBag := baggage.All(ctx)
+
+	switch {
+	case len(ddBag) == 0 && otelBag.Len() > 0:
+		return otelBag, nil
+	case otelBag.Len() == 0 && len(ddBag) > 0:
+		return convertDDBaggage(ddBag)
+	case len(ddBag) > 0 && otelBag.Len() > 0:
+		if len(ddBag) <= otelBag.Len() {
+			return mergeDDBagIntoOtel(otelBag, ddBag)
+		}
+		return mergeOtelBagIntoDD(otelBag, ddBag)
+	}
+	emptyBag, _ := otelbaggage.New()
path: ddtrace/opentelemetry/tracer.go
user: login: mtoffl01
body: Make sure to handle errors
created_at: 2025-03-24T16:55:00Z
author_association: CONTRIBUTOR
diff_hunk: @@ -105,6 +116,71 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	return ctx, os
 }
 
+// mergeBaggageFromContext consolidates baggage from OpenTelemetry and Datadog sources.
+func mergeBaggageFromContext(ctx context.Context) (otelbaggage.Baggage, error) {
+	otelBag := otelbaggage.FromContext(ctx)
+	ddBag := baggage.All(ctx)
+
+	switch {
+	case len(ddBag) == 0 && otelBag.Len() > 0:
+		return otelBag, nil
+	case otelBag.Len() == 0 && len(ddBag) > 0:
+		return convertDDBaggage(ddBag)
+	case len(ddBag) > 0 && otelBag.Len() > 0:
path: ddtrace/opentelemetry/tracer.go
user: login: mtoffl01
body: Might be cleaner to store lengths in local variables so you don't need to get them via len(<>) so many times. Eg. `lenDDBag` and `lenOtelBag`
created_at: 2025-03-24T16:55:39Z
author_association: CONTRIBUTOR
diff_hunk: @@ -308,3 +313,88 @@ func BenchmarkOTelConcurrentTracing(b *testing.B) {
 		}
 	}
 }
+
+func TestOteltoDDBaggagePropagation(t *testing.T) {
+	assert := assert.New(t)
+
+	// Create a baggage with a key/value pair using the OTel baggage API.
+	m, err := otelbaggage.NewMember("testKey", "testValue")
+	assert.NoError(err)
+	bag, err := otelbaggage.New(m)
+	assert.NoError(err)
+
+	// Create a context that includes the baggage.
+	ctx := otelbaggage.ContextWithBaggage(context.Background(), bag)
+
+	tp := NewTracerProvider()
+	otel.SetTracerProvider(tp)
+	tr := otel.Tracer("baggage.test", oteltrace.WithInstrumentationVersion("0.1"))
+
+	ctx, _ = tr.Start(ctx, "baggage.span")
+
+	ddSpan, ok := tracer.SpanFromContext(ctx)
+	assert.True(ok)
+
+	// Verify that the baggage item was propagated to the Datadog span.
+	baggageValue := ddSpan.BaggageItem("testKey")
+	assert.Equal("testValue", baggageValue)
+	value, _ := baggage.Get(ctx, "testKey")
+	assert.Equal("testValue", value)
+}
+
+func TestDDtoOtelBaggagePropagation(t *testing.T) {
+	assert := assert.New(t)
+
+	// Set up the tracer provider and tracer.
+	tp := NewTracerProvider()
+	otel.SetTracerProvider(tp)
+	_ = otel.Tracer("baggage.test", oteltrace.WithInstrumentationVersion("0.1"))
path: ddtrace/opentelemetry/tracer_test.go
user: login: mtoffl01
body: ```suggestion
	otel.Tracer("baggage.test", oteltrace.WithInstrumentationVersion("0.1"))
```
?
created_at: 2025-03-24T16:56:59Z
author_association: CONTRIBUTOR
diff_hunk: @@ -308,3 +313,88 @@ func BenchmarkOTelConcurrentTracing(b *testing.B) {
 		}
 	}
 }
+
+func TestOteltoDDBaggagePropagation(t *testing.T) {
+	assert := assert.New(t)
+
+	// Create a baggage with a key/value pair using the OTel baggage API.
+	m, err := otelbaggage.NewMember("testKey", "testValue")
+	assert.NoError(err)
+	bag, err := otelbaggage.New(m)
+	assert.NoError(err)
+
+	// Create a context that includes the baggage.
+	ctx := otelbaggage.ContextWithBaggage(context.Background(), bag)
+
+	tp := NewTracerProvider()
+	otel.SetTracerProvider(tp)
+	tr := otel.Tracer("baggage.test", oteltrace.WithInstrumentationVersion("0.1"))
+
+	ctx, _ = tr.Start(ctx, "baggage.span")
+
+	ddSpan, ok := tracer.SpanFromContext(ctx)
+	assert.True(ok)
+
+	// Verify that the baggage item was propagated to the Datadog span.
+	baggageValue := ddSpan.BaggageItem("testKey")
+	assert.Equal("testValue", baggageValue)
+	value, _ := baggage.Get(ctx, "testKey")
+	assert.Equal("testValue", value)
+}
+
+func TestDDtoOtelBaggagePropagation(t *testing.T) {
+	assert := assert.New(t)
+
+	// Set up the tracer provider and tracer.
+	tp := NewTracerProvider()
+	otel.SetTracerProvider(tp)
+	_ = otel.Tracer("baggage.test", oteltrace.WithInstrumentationVersion("0.1"))
+
+	ctx := context.Background()
+	tracer.Start()
+	defer tracer.Stop()
+	ddSpan, ctx := tracer.StartSpanFromContext(ctx, "ddtrace-baggage")
+
+	// Set baggage items on the ddtrace span.
path: ddtrace/opentelemetry/tracer_test.go
user: login: mtoffl01
body: "Set baggage items on the ddtrace span" <-- That's not exactly what we're doing here, right? We're just configuring baggage the "DD API way"
created_at: 2025-03-24T16:57:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -105,6 +115,79 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	return ctx, os
 }
 
+// mergeBaggageFromContext consolidates baggage from OpenTelemetry and Datadog sources.
+func mergeBaggageFromContext(ctx context.Context) (otelbaggage.Baggage, error) {
+	otelBag := otelbaggage.FromContext(ctx)
+	ddBag := baggage.All(ctx)
+	lenDDBag := len(ddBag)
+	lenOtelBag := otelBag.Len()
+
+	switch {
+	case lenDDBag == 0 && lenOtelBag > 0:
+		return otelBag, nil
+	case lenOtelBag == 0 && lenDDBag > 0:
+		return convertDDBaggage(ddBag)
+	case lenDDBag > 0 && lenOtelBag > 0:
+		if lenDDBag <= lenOtelBag {
+			return mergeDDBagIntoOtel(otelBag, ddBag)
+		}
+		return mergeOtelBagIntoDD(otelBag, ddBag)
+	}
+	emptyBag, err := otelbaggage.New()
+	if err != nil {
+		return otelbaggage.Baggage{}, err
+	}
+	return emptyBag, nil
+}
+
+// convertDDBaggage converts a Datadog baggage map into an OpenTelemetry baggage.
+func convertDDBaggage(ddBag map[string]string) (otelbaggage.Baggage, error) {
+	var members []otelbaggage.Member
+	for key, value := range ddBag {
+		member, err := otelbaggage.NewMember(key, value)
+		if err != nil {
+			return otelbaggage.Baggage{}, err
+		}
+		members = append(members, member)
+	}
+	return otelbaggage.New(members...)
+}
+
+// mergeDDBagIntoOtel merges Datadog baggage into the existing OpenTelemetry baggage,
+// adding only keys that are missing in the OpenTelemetry baggage.
+func mergeDDBagIntoOtel(otelBag otelbaggage.Baggage, ddBag map[string]string) (otelbaggage.Baggage, error) {
+	members := otelBag.Members()
+	otelKeys := make(map[string]struct{}, otelBag.Len())
+	for _, m := range otelBag.Members() {
+		otelKeys[m.Key()] = struct{}{}
+	}
+	for key, value := range ddBag {
+		if _, exists := otelKeys[key]; !exists {
+			member, _ := otelbaggage.NewMember(key, value)
path: ddtrace/opentelemetry/tracer.go
user: login: mtoffl01
body: Make sure to perform error checks everywhere!
created_at: 2025-03-25T17:34:46Z
author_association: CONTRIBUTOR
diff_hunk: @@ -105,6 +115,79 @@ func (t *oteltracer) Start(ctx context.Context, spanName string, opts ...oteltra
 	return ctx, os
 }
 
+// mergeBaggageFromContext consolidates baggage from OpenTelemetry and Datadog sources.
+func mergeBaggageFromContext(ctx context.Context) (otelbaggage.Baggage, error) {
+	otelBag := otelbaggage.FromContext(ctx)
+	ddBag := baggage.All(ctx)
+	lenDDBag := len(ddBag)
+	lenOtelBag := otelBag.Len()
+
+	switch {
+	case lenDDBag == 0 && lenOtelBag > 0:
+		return otelBag, nil
+	case lenOtelBag == 0 && lenDDBag > 0:
+		return convertDDBaggage(ddBag)
+	case lenDDBag > 0 && lenOtelBag > 0:
+		if lenDDBag <= lenOtelBag {
+			return mergeDDBagIntoOtel(otelBag, ddBag)
+		}
+		return mergeOtelBagIntoDD(otelBag, ddBag)
+	}
+	emptyBag, err := otelbaggage.New()
+	if err != nil {
+		return otelbaggage.Baggage{}, err
+	}
+	return emptyBag, nil
+}
+
+// convertDDBaggage converts a Datadog baggage map into an OpenTelemetry baggage.
+func convertDDBaggage(ddBag map[string]string) (otelbaggage.Baggage, error) {
+	var members []otelbaggage.Member
+	for key, value := range ddBag {
+		member, err := otelbaggage.NewMember(key, value)
+		if err != nil {
+			return otelbaggage.Baggage{}, err
+		}
+		members = append(members, member)
+	}
+	return otelbaggage.New(members...)
+}
+
+// mergeDDBagIntoOtel merges Datadog baggage into the existing OpenTelemetry baggage,
+// adding only keys that are missing in the OpenTelemetry baggage.
+func mergeDDBagIntoOtel(otelBag otelbaggage.Baggage, ddBag map[string]string) (otelbaggage.Baggage, error) {
+	members := otelBag.Members()
+	otelKeys := make(map[string]struct{}, otelBag.Len())
+	for _, m := range otelBag.Members() {
+		otelKeys[m.Key()] = struct{}{}
+	}
+	for key, value := range ddBag {
+		if _, exists := otelKeys[key]; !exists {
+			member, _ := otelbaggage.NewMember(key, value)
+			members = append(members, member)
+		}
+	}
+	return otelbaggage.New(members...)
+}
+
+// mergeOtelBagIntoDD merges OpenTelemetry baggage into a Datadog baggage map,
+// overriding keys with the OpenTelemetry values in case of conflict.
+func mergeOtelBagIntoDD(otelBag otelbaggage.Baggage, ddBag map[string]string) (otelbaggage.Baggage, error) {
+	mergedMap := make(map[string]otelbaggage.Member, len(ddBag)+otelBag.Len())
+	for key, value := range ddBag {
+		member, _ := otelbaggage.NewMember(key, value)
path: ddtrace/opentelemetry/tracer.go
user: login: mtoffl01
body: error check
created_at: 2025-03-25T17:35:41Z
author_association: CONTRIBUTOR

PR comments:
user: login: datadog-datadog-prod-us1[bot]
created_at: 2025-02-21T18:21:43Z
body: 
## Datadog Report

Branch report: [`rachel.yang/otel-drop-in-baggage`](https://app.datadoghq.com/ci/test-branch/github.com%2Fdatadog%2Fdd-trace-go/dd-trace-go/rachel.yang%2Fotel-drop-in-baggage?env=none&index=citest&_user_click_origin_=github)
Commit report: [`10f89af`](https://app.datadoghq.com/ci/test-commit/github.com%2Fdatadog%2Fdd-trace-go/dd-trace-go/rachel.yang%2Fotel-drop-in-baggage/10f89afefc1af1ef6cb20975326b5dc785c9558c?env=none&index=citest&_user_click_origin_=github)
Test service: `dd-trace-go`

:white_check_mark: 0 Failed, 4479 Passed, 66 Skipped, 3m 53.48s Total Time




author_association: NONE
user: login: pr-commenter[bot]
created_at: 2025-02-21T18:40:31Z
body: ## Benchmarks

Benchmark execution time: 2025-03-25 19:41:44

Comparing candidate commit f566ee77668014e043e7e0c13617556e8f58c803 in PR branch `rachel.yang/otel-drop-in-baggage` with baseline commit 0a39bf61cf56914a20de7f8aebb82223fdf2cbc7 in branch `v2-dev`.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 52 metrics, 2 unstable metrics.

#### scenario:BenchmarkOTelApiWithCustomTags/otel_api-24

- 🟥 `allocations` [+2; +2] or [+4.444%; +4.444%]
- 🟥 `execution_time` [+238.646ns; +286.954ns] or [+3.096%; +3.722%]




author_association: NONE
user: login: darccio
created_at: 2025-03-10T16:39:43Z
body: @rachelyangdog As part of our upcoming release, we are reviewing all in-flight PRs. Please consider closing it or rebasing to `v2-dev` if you want to merge it for the next release after `v2.0.0`.
author_association: MEMBER
user: login: darccio
created_at: 2025-03-26T09:56:54Z
body: I confirm that failing tests are unrelated.
author_association: MEMBER