Repository: fluent/fluent-bit (C)

Issue:
OTEL traceId and spanId sent via HTTP/JSON are doubled in size and malformed
## Bug Report

**Describe the bug**
TraceId and spanId provided with OpenTelemetry logs are doubled in size and malformed. The first part looks correct, but the second part contains garbage data.

**To Reproduce**
Let’s use two Fluent Bit instances. The first instance receives HTTP/JSON OTEL messages and forwards them via gRPC OTEL to the second instance.

We send the following message to the first instance:

```shell
curl --location 'http://localhost:4317/v1/logs' \
--header 'Content-Type: application/json' \
--data '{
  "resourceLogs": [
    {
      "scopeLogs": [
        {
          "logRecords": [
            {
              "timeUnixNano": "1755519434000000000",
              "body": {
                "stringValue": "ABC123"
              },
              "traceId": "000102030405060708090a0b0c0d0e0f",
              "spanId": "0f0e0d0c0b0a0908"
            }
          ]
        }
      ]
    }
  ]
}'
```
Log in the first instance (already showing malformed trace_id and span_id):
```
Fluent Bit v4.0.7
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___  _____
|  ___| |                | |   | ___ (_) |           /   ||  _  |
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| || |/' |
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| ||  /| |
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |\ |_/ /
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/


[2025/08/18 14:34:40] [ info] Configuration:
[2025/08/18 14:34:40] [ info]  flush time     | 1.000000 seconds
[2025/08/18 14:34:40] [ info]  grace          | 5 seconds
[2025/08/18 14:34:40] [ info]  daemon         | 0
[2025/08/18 14:34:40] [ info] ___________
[2025/08/18 14:34:40] [ info]  inputs:
[2025/08/18 14:34:40] [ info]      opentelemetry
[2025/08/18 14:34:40] [ info] ___________
[2025/08/18 14:34:40] [ info]  filters:
[2025/08/18 14:34:40] [ info] ___________
[2025/08/18 14:34:40] [ info]  outputs:
[2025/08/18 14:34:40] [ info]      stdout.0
[2025/08/18 14:34:40] [ info]      opentelemetry.1
[2025/08/18 14:34:40] [ info] ___________
[2025/08/18 14:34:40] [ info]  collectors:
[2025/08/18 14:34:40] [ info] [fluent bit] version=4.0.7, commit=8c5cbc1bd8, pid=1
[2025/08/18 14:34:40] [debug] [engine] coroutine stack size: 196608 bytes (192.0K)
[2025/08/18 14:34:40] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/08/18 14:34:40] [ info] [output:stdout:stdout.0] worker #0 started
[2025/08/18 14:34:40] [ info] [simd    ] NEON
[2025/08/18 14:34:40] [ info] [cmetrics] version=1.0.5
[2025/08/18 14:34:40] [ info] [ctraces ] version=0.6.6
[2025/08/18 14:34:40] [ info] [input:opentelemetry:opentelemetry.0] initializing
[2025/08/18 14:34:40] [ info] [input:opentelemetry:opentelemetry.0] storage_strategy='memory' (memory only)
[2025/08/18 14:34:40] [debug] [opentelemetry:opentelemetry.0] created event channels: read=25 write=26
[2025/08/18 14:34:40] [debug] [downstream] listening on 0.0.0.0:4317
[2025/08/18 14:34:40] [ info] [input:opentelemetry:opentelemetry.0] listening on 0.0.0.0:4317
[2025/08/18 14:34:40] [debug] [stdout:stdout.0] created event channels: read=28 write=29
[2025/08/18 14:34:40] [debug] [opentelemetry:opentelemetry.1] created event channels: read=37 write=38
[2025/08/18 14:34:40] [ info] [output:opentelemetry:opentelemetry.1] gRPC enabled, HTTP/2 has been auto-enabled
[2025/08/18 14:34:40] [debug] [router] match rule opentelemetry.0:stdout.0
[2025/08/18 14:34:40] [debug] [router] match rule opentelemetry.0:opentelemetry.1
[2025/08/18 14:34:40] [ info] [sp] stream processor started
[2025/08/18 14:34:40] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[2025/08/18 14:34:53] [debug] [task] created task=0xfdc331223640 id=0 OK
[2025/08/18 14:34:53] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
GROUP METADATA :

{"schema"=>"otlp", "resource_id"=>0, "scope_id"=>0}

GROUP ATTRIBUTES :

{"resource"=>{}}

[0] v1_logs: [[1755519434.1530546742, {"otlp"=>{"trace_id"=>"\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f8\xca 1\xc3\xfd\x00\x000\x95+1\xc3\xfd\x00\x00", "span_id"=>"\x0f\x0e\x0d\x0c\x0b\x0a\x09\x08\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f"}}], {"log"=>"ABC123"}]
[2025/08/18 14:34:53] [debug] [out flush] cb_destroy coro_id=0
[2025/08/18 14:34:53] [debug] [upstream] KA connection #50 to host.docker.internal:4318 is connected
[2025/08/18 14:34:53] [ info] [output:opentelemetry:opentelemetry.1] host.docker.internal:4318, HTTP status=200
[2025/08/18 14:34:53] [debug] [upstream] KA connection #50 to host.docker.internal:4318 is now available
[2025/08/18 14:34:53] [debug] [out flush] cb_destroy coro_id=0
[2025/08/18 14:34:53] [debug] [task] destroy task=0xfdc331223640 (task_id=0)
```
Log in the second instance:
```
Fluent Bit v4.0.7
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___  _____
|  ___| |                | |   | ___ (_) |           /   ||  _  |
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| || |/' |
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| ||  /| |
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |\ |_/ /
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/


[2025/08/18 14:34:38] [ info] Configuration:
[2025/08/18 14:34:38] [ info]  flush time     | 1.000000 seconds
[2025/08/18 14:34:38] [ info]  grace          | 5 seconds
[2025/08/18 14:34:38] [ info]  daemon         | 0
[2025/08/18 14:34:38] [ info] ___________
[2025/08/18 14:34:38] [ info]  inputs:
[2025/08/18 14:34:38] [ info]      opentelemetry
[2025/08/18 14:34:38] [ info] ___________
[2025/08/18 14:34:38] [ info]  filters:
[2025/08/18 14:34:38] [ info] ___________
[2025/08/18 14:34:38] [ info]  outputs:
[2025/08/18 14:34:38] [ info]      stdout.0
[2025/08/18 14:34:38] [ info] ___________
[2025/08/18 14:34:38] [ info]  collectors:
[2025/08/18 14:34:38] [ info] [fluent bit] version=4.0.7, commit=8c5cbc1bd8, pid=1
[2025/08/18 14:34:38] [debug] [engine] coroutine stack size: 196608 bytes (192.0K)
[2025/08/18 14:34:38] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/08/18 14:34:38] [ info] [simd    ] NEON
[2025/08/18 14:34:38] [ info] [cmetrics] version=1.0.5
[2025/08/18 14:34:38] [ info] [ctraces ] version=0.6.6
[2025/08/18 14:34:38] [ info] [input:opentelemetry:opentelemetry.0] initializing
[2025/08/18 14:34:38] [ info] [input:opentelemetry:opentelemetry.0] storage_strategy='memory' (memory only)
[2025/08/18 14:34:38] [debug] [opentelemetry:opentelemetry.0] created event channels: read=25 write=26
[2025/08/18 14:34:38] [debug] [downstream] listening on 0.0.0.0:4318
[2025/08/18 14:34:38] [ info] [input:opentelemetry:opentelemetry.0] listening on 0.0.0.0:4318
[2025/08/18 14:34:38] [debug] [stdout:stdout.0] created event channels: read=28 write=29
[2025/08/18 14:34:38] [ info] [sp] stream processor started
[2025/08/18 14:34:38] [ info] [output:stdout:stdout.0] worker #0 started
[2025/08/18 14:34:38] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[2025/08/18 14:34:54] [debug] [task] created task=0xf1d0df023640 id=0 OK
[2025/08/18 14:34:54] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
GROUP METADATA :

{"schema"=>"otlp", "resource_id"=>0, "scope_id"=>0}

GROUP ATTRIBUTES :

{"resource"=>{"schema_url"=>""}, "scope"=>{}}

[0] v1_logs: [[1755519435.18446744072475677803, {"otlp"=>{"observed_timestamp"=>0, "timestamp"=>1755519435530546742, "trace_id"=>"\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f8\xca 1\xc3\xfd\x00\x000\x95+1\xc3\xfd\x00\x00", "span_id"=>"\x0f\x0e\x0d\x0c\x0b\x0a\x09\x08\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f", "trace_flags"=>0}}], {"log"=>"ABC123"}]
[2025/08/18 14:34:54] [debug] [out flush] cb_destroy coro_id=0
[2025/08/18 14:34:54] [debug] [task] destroy task=0xf1d0df023640 (task_id=0)
```
Malformed trace_id and span_id are sent to the second instance. Inspecting gRPC traffic shows that trace_id is sent as a 32-byte array and span_id as a 16-byte array. This is incorrect according to the OpenTelemetry Protocol (OTLP) specification, which requires trace_id to be 16 bytes and span_id to be 8 bytes. The first half of each array is correct, while the second half contains garbage that changes after each restart of the first instance.

For reference, the previous major version of Fluent Bit (3.2.10) works correctly. If I replace the first instance with Fluent Bit 3.2.10, leaving the second on Fluent Bit 4.0.7, and send the same message:
```shell
curl --location 'http://localhost:4317/v1/logs' \
--header 'Content-Type: application/json' \
--data '{
  "resourceLogs": [
    {
      "scopeLogs": [
        {
          "logRecords": [
            {
              "timeUnixNano": "1755519434000000000",
              "body": {
                "stringValue": "ABC123"
              },
              "traceId": "000102030405060708090a0b0c0d0e0f",
              "spanId": "0f0e0d0c0b0a0908"
            }
          ]
        }
      ]
    }
  ]
}'
```
Log of the first Fluent Bit 3.2.10 instance:
```
Fluent Bit v3.2.10
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _           _____  _____
|  ___| |                | |   | ___ (_) |         |____ |/ __  \
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __   / /`' / /'
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / /   \ \  / /
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /.___/ /./ /___
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/ \____(_)_____/


[2025/08/18 14:59:43] [ info] Configuration:
[2025/08/18 14:59:43] [ info]  flush time     | 1.000000 seconds
[2025/08/18 14:59:43] [ info]  grace          | 5 seconds
[2025/08/18 14:59:43] [ info]  daemon         | 0
[2025/08/18 14:59:43] [ info] ___________
[2025/08/18 14:59:43] [ info]  inputs:
[2025/08/18 14:59:43] [ info]      opentelemetry
[2025/08/18 14:59:43] [ info] ___________
[2025/08/18 14:59:43] [ info]  filters:
[2025/08/18 14:59:43] [ info] ___________
[2025/08/18 14:59:43] [ info]  outputs:
[2025/08/18 14:59:43] [ info]      stdout.0
[2025/08/18 14:59:43] [ info]      opentelemetry.1
[2025/08/18 14:59:43] [ info] ___________
[2025/08/18 14:59:43] [ info]  collectors:
[2025/08/18 14:59:43] [ info] [fluent bit] version=3.2.10, commit=a988291a19, pid=1
[2025/08/18 14:59:43] [debug] [engine] coroutine stack size: 196608 bytes (192.0K)
[2025/08/18 14:59:43] [ info] [storage] ver=1.5.2, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/08/18 14:59:43] [ info] [simd    ] disabled
[2025/08/18 14:59:43] [ info] [cmetrics] version=0.9.9
[2025/08/18 14:59:43] [ info] [ctraces ] version=0.6.1
[2025/08/18 14:59:43] [ info] [input:opentelemetry:opentelemetry.0] initializing
[2025/08/18 14:59:43] [ info] [input:opentelemetry:opentelemetry.0] storage_strategy='memory' (memory only)
[2025/08/18 14:59:43] [debug] [opentelemetry:opentelemetry.0] created event channels: read=25 write=26
[2025/08/18 14:59:43] [debug] [downstream] listening on 0.0.0.0:4317
[2025/08/18 14:59:43] [ info] [input:opentelemetry:opentelemetry.0] listening on 0.0.0.0:4317
[2025/08/18 14:59:43] [debug] [stdout:stdout.0] created event channels: read=28 write=29
[2025/08/18 14:59:43] [debug] [opentelemetry:opentelemetry.1] created event channels: read=37 write=38
[2025/08/18 14:59:43] [ info] [output:stdout:stdout.0] worker #0 started
[2025/08/18 14:59:43] [ info] [output:opentelemetry:opentelemetry.1] gRPC enabled, HTTP/2 has been auto-enabled
[2025/08/18 14:59:43] [debug] [router] match rule opentelemetry.0:stdout.0
[2025/08/18 14:59:43] [debug] [router] match rule opentelemetry.0:opentelemetry.1
[2025/08/18 14:59:43] [ info] [sp] stream processor started
[2025/08/18 14:59:47] [debug] [input:opentelemetry:opentelemetry.0] attributes missing
[2025/08/18 14:59:47] [debug] [task] created task=0xe85b7ca39640 id=0 OK
[2025/08/18 14:59:47] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
GROUP METADATA :

{"schema"=>"otlp", "resource_id"=>0, "scope_id"=>0}

GROUP ATTRIBUTES :

{"resource"=>{}}

[0] v1_logs: [[1755519434.1530546742, {"otlp"=>{"trace_id"=>"000102030405060708090a0b0c0d0e0f", "span_id"=>"0f0e0d0c0b0a0908"}}], {"log"=>"ABC123"}]
[2025/08/18 14:59:47] [debug] [out flush] cb_destroy coro_id=0
[2025/08/18 14:59:47] [debug] [upstream] KA connection #50 to host.docker.internal:4318 is connected
[2025/08/18 14:59:47] [ info] [output:opentelemetry:opentelemetry.1] host.docker.internal:4318, HTTP status=200
[2025/08/18 14:59:47] [debug] [upstream] KA connection #50 to host.docker.internal:4318 is now available
[2025/08/18 14:59:47] [debug] [out flush] cb_destroy coro_id=0
[2025/08/18 14:59:47] [debug] [task] destroy task=0xe85b7ca39640 (task_id=0)
```
Log from the second instance with Fluent Bit 4.0.7:
```
Fluent Bit v4.0.7
* Copyright (C) 2015-2025 The Fluent Bit Authors
* Fluent Bit is a CNCF sub-project under the umbrella of Fluentd
* https://fluentbit.io

______ _                  _    ______ _ _             ___  _____
|  ___| |                | |   | ___ (_) |           /   ||  _  |
| |_  | |_   _  ___ _ __ | |_  | |_/ /_| |_  __   __/ /| || |/' |
|  _| | | | | |/ _ \ '_ \| __| | ___ \ | __| \ \ / / /_| ||  /| |
| |   | | |_| |  __/ | | | |_  | |_/ / | |_   \ V /\___  |\ |_/ /
\_|   |_|\__,_|\___|_| |_|\__| \____/|_|\__|   \_/     |_(_)___/


[2025/08/18 14:59:44] [ info] Configuration:
[2025/08/18 14:59:44] [ info]  flush time     | 1.000000 seconds
[2025/08/18 14:59:44] [ info]  grace          | 5 seconds
[2025/08/18 14:59:44] [ info]  daemon         | 0
[2025/08/18 14:59:44] [ info] ___________
[2025/08/18 14:59:44] [ info]  inputs:
[2025/08/18 14:59:44] [ info]      opentelemetry
[2025/08/18 14:59:44] [ info] ___________
[2025/08/18 14:59:44] [ info]  filters:
[2025/08/18 14:59:44] [ info] ___________
[2025/08/18 14:59:44] [ info]  outputs:
[2025/08/18 14:59:44] [ info]      stdout.0
[2025/08/18 14:59:44] [ info] ___________
[2025/08/18 14:59:44] [ info]  collectors:
[2025/08/18 14:59:44] [ info] [fluent bit] version=4.0.7, commit=8c5cbc1bd8, pid=1
[2025/08/18 14:59:44] [debug] [engine] coroutine stack size: 196608 bytes (192.0K)
[2025/08/18 14:59:44] [ info] [storage] ver=1.5.3, type=memory, sync=normal, checksum=off, max_chunks_up=128
[2025/08/18 14:59:44] [ info] [simd    ] NEON
[2025/08/18 14:59:44] [ info] [cmetrics] version=1.0.5
[2025/08/18 14:59:44] [ info] [ctraces ] version=0.6.6
[2025/08/18 14:59:44] [ info] [input:opentelemetry:opentelemetry.0] initializing
[2025/08/18 14:59:44] [ info] [input:opentelemetry:opentelemetry.0] storage_strategy='memory' (memory only)
[2025/08/18 14:59:44] [debug] [opentelemetry:opentelemetry.0] created event channels: read=25 write=26
[2025/08/18 14:59:44] [debug] [downstream] listening on 0.0.0.0:4318
[2025/08/18 14:59:44] [ info] [input:opentelemetry:opentelemetry.0] listening on 0.0.0.0:4318
[2025/08/18 14:59:44] [debug] [stdout:stdout.0] created event channels: read=28 write=29
[2025/08/18 14:59:44] [ info] [sp] stream processor started
[2025/08/18 14:59:44] [ info] [engine] Shutdown Grace Period=5, Shutdown Input Grace Period=2
[2025/08/18 14:59:44] [ info] [output:stdout:stdout.0] worker #0 started
[2025/08/18 14:59:48] [debug] [task] created task=0xe675baa23640 id=0 OK
[2025/08/18 14:59:48] [debug] [output:stdout:stdout.0] task_id=0 assigned to thread #0
GROUP METADATA :

{"schema"=>"otlp", "resource_id"=>0, "scope_id"=>0}

GROUP ATTRIBUTES :

{"resource"=>{"schema_url"=>""}, "scope"=>{}}

[0] v1_logs: [[1755519435.18446744072475677803, {"otlp"=>{"observed_timestamp"=>0, "timestamp"=>1755519435530546742, "trace_id"=>"\x00\x01\x02\x03\x04\x05\x06\x07\x08\x09\x0a\x0b\x0c\x0d\x0e\x0f", "span_id"=>"\x0f\x0e\x0d\x0c\x0b\x0a\x09\x08", "trace_flags"=>0}}], {"log"=>"ABC123"}]
[2025/08/18 14:59:48] [debug] [out flush] cb_destroy coro_id=0
[2025/08/18 14:59:48] [debug] [task] destroy task=0xe675baa23640 (task_id=0)
```
In this case, the first instance correctly sends a 16-byte trace_id and an 8-byte span_id to the second instance.

**Expected behavior**
Fluent Bit should collect and resend messages with the correct 16-byte trace_id and 8-byte span_id.

**Screenshots**
N/A

**Your Environment**
<!--- Include as many relevant details about the environment you experienced the bug in -->
* Version used: v4.0.7
* Configuration:

First instance:
```yaml
service:
  log_level: trace

pipeline:
  inputs:
    - name: opentelemetry
      port: 4317

  outputs:
    - name: stdout
      match: '*'

    - name: opentelemetry
      match: '*'
      host: host.docker.internal
      port: 4318
      grpc: on
```
Second instance:
```yaml
service:
  log_level: trace

pipeline:
  inputs:
    - name: opentelemetry
      port: 4318

  outputs:
    - name: stdout
      match: '*'
```
* Environment name and version (e.g. Kubernetes? What version?): Docker
* Server type and version: N/A
* Operating System and version: fluent/fluent-bit:4.0.7
* Filters and plugins: built-in OpenTelemetry input and output plugins



Fix (patch):
diff --git a/src/opentelemetry/flb_opentelemetry_logs.c b/src/opentelemetry/flb_opentelemetry_logs.c
index 8e7fde45aff..2c009f0c8ad 100644
--- a/src/opentelemetry/flb_opentelemetry_logs.c
+++ b/src/opentelemetry/flb_opentelemetry_logs.c
@@ -293,18 +293,18 @@ static int process_json_payload_log_records_entry(
         result = flb_otel_utils_json_payload_append_converted_kvlist(encoder, FLB_LOG_EVENT_METADATA, metadata_object);
     }
 
-    if (trace_id != NULL) {
-        flb_otel_utils_hex_to_id(trace_id->via.str.ptr, trace_id->via.str.size, tmp_id, 32);
+    if (trace_id != NULL && trace_id->type == MSGPACK_OBJECT_STR && trace_id->via.str.size == 32) {
+        flb_otel_utils_hex_to_id(trace_id->via.str.ptr, trace_id->via.str.size, tmp_id, 16);
         flb_log_event_encoder_append_metadata_values(encoder,
                                                         FLB_LOG_EVENT_STRING_VALUE("trace_id", 8),
-                                                        FLB_LOG_EVENT_BINARY_VALUE(tmp_id, 32));
+                                                        FLB_LOG_EVENT_BINARY_VALUE(tmp_id, 16));
     }
 
-    if (span_id != NULL) {
-        flb_otel_utils_hex_to_id(span_id->via.str.ptr, span_id->via.str.size, tmp_id, 16);
+    if (span_id != NULL && span_id->type == MSGPACK_OBJECT_STR && span_id->via.str.size == 16) {
+        flb_otel_utils_hex_to_id(span_id->via.str.ptr, span_id->via.str.size, tmp_id, 8);
         flb_log_event_encoder_append_metadata_values(encoder,
                                                         FLB_LOG_EVENT_STRING_VALUE("span_id", 7),
-                                                        FLB_LOG_EVENT_BINARY_VALUE(tmp_id, 16));
+                                                        FLB_LOG_EVENT_BINARY_VALUE(tmp_id, 8));
     }
 
     result = flb_log_event_encoder_commit_map(encoder, FLB_LOG_EVENT_METADATA);