Repository: apache/skywalking (language: python)

Issue:
[Feature] [Tracing] Add OTLP Tracing support as a Zipkin trace input
### Search before asking

- [X] I had searched in the [issues](https://github.com/apache/skywalking/issues?q=is%3Aissue) and found no similar feature requirement.


### Description

We are declaring to support [OTLP trace format through OTLP collector and forward to Zipkin receiver](https://skywalking.apache.org/docs/main/next/en/setup/backend/otlp-trace/). 

This works, but an extra component seems unnecessary.

OTLP trace format is logically the same as the Zipkin trace format, so, we just need to add a OLTP trace handler and an in-memory converter to Zipkin receiver.

### Use case

```yaml
receiver-otel:
  selector: ${SW_OTEL_RECEIVER:default}
  default:
    enabledHandlers: ${SW_OTEL_RECEIVER_ENABLED_HANDLERS:"otlp-metrics,otlp-logs"}
    enabledOtelMetricsRules: ....
```

We should add `otlp-trace` as an option to `enabledHandlers` to support this.

@kezhenxu94 Any suggestion?

### Related issues

_No response_

### Are you willing to submit a pull request to implement this on your own?

- [ ] Yes I am willing to submit a pull request on my own!

### Code of Conduct

- [X] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)



Discussion / hints:
To be clear, this would not forward any OAL sources like the previous Zipkin receiver.
Not sure what you want, OpenTelemetry agents can send traces directly to Zipkin server

https://github.com/open-telemetry/opentelemetry-java/blob/3b1c6d9108f7e25afe5d0fdae830458e8c1a481b/sdk-extensions/autoconfigure/README.md#zipkin-exporter
Some agents could, I am thinking wider support only.
It sounds kind of out of skywalking’s scope to convert one third party format (OpenTelemetry) to another third party format (Zipkin)

Fix patch:
diff --git a/.github/workflows/skywalking.yaml b/.github/workflows/skywalking.yaml
index fc679c01512a..ceef78e80416 100644
--- a/.github/workflows/skywalking.yaml
+++ b/.github/workflows/skywalking.yaml
@@ -661,6 +661,9 @@ jobs:
           - name: UI Menu OpenSearch 2.4.0
             config: test/e2e-v2/cases/menu/opensearch/e2e.yaml
             env: OPENSEARCH_VERSION=2.4.0
+
+          - name: OTLP Trace
+            config:  test/e2e-v2/cases/otlp-traces/e2e.yaml
     steps:
       - uses: actions/checkout@v3
         with:
diff --git a/docs/en/changes/changes.md b/docs/en/changes/changes.md
index ad0b10366726..9b392058930f 100644
--- a/docs/en/changes/changes.md
+++ b/docs/en/changes/changes.md
@@ -80,6 +80,7 @@
 * Fix GraphQL query `listInstances` not using endTime query
 * Do not start server and Kafka consumer in init mode.
 * Add Iris component ID(5018).
+* Add OTLP Tracing support as a Zipkin trace input.
 
 #### UI
 
diff --git a/docs/en/setup/backend/otlp-trace.md b/docs/en/setup/backend/otlp-trace.md
index ad35ae0675fe..6afd5f984ad6 100644
--- a/docs/en/setup/backend/otlp-trace.md
+++ b/docs/en/setup/backend/otlp-trace.md
@@ -1,12 +1,22 @@
 # OpenTelemetry Trace Format
-OpenTelemetry metrics and log formats are supported, the trace format is not supported directly.
 
-OpenTelemetry and Zipkin formats are generally logically consistent. 
-If the Zipkin server alternative mode is expected, user could use OpenTelemetry Collector's [Zipkin Exporter](https://opentelemetry.io/docs/specs/otel/trace/sdk_exporters/zipkin/)
-to transfer the format and forward to OAP as alternative Zipkin server.
+SkyWalking can receive traces from Traces in OTLP format and convert them to Zipkin Trace format eventually. 
+For data analysis and queries related to Zipkin Trace, please [refer to the relevant documentation](./zipkin-trace.md#zipkin-query).
 
-Read [Zipkin Trace Doc](zipkin-trace.md) for more details about **Zipkin Server Alternative Mode**.
+OTLP Trace handler references the [Zipkin Exporter in the OpenTelemetry Collector](https://opentelemetry.io/docs/specs/otel/trace/sdk_exporters/zipkin/#summary) to convert the data format.
 
-To contributors, if you want to contribute `otlp-trace` handler in `receiver-otel` receiver, we could accept that PR.
-But still, we could require the trace transferred into Zipkin format in the handler.
- 
\ No newline at end of file
+## Set up backend receiver
+
+1. Make sure to enable **otlp-traces** handler in OTLP receiver of `application.yml`.
+```yaml
+receiver-otel:
+  selector: default
+  default:
+    enabledHandlers: otlp-traces
+```
+
+2. Make sure to enable zipkin receiver and zipkin query in `application.yml` for config the zipkin.
+
+## Setup Query and Lens UI 
+
+Please read [deploy Lens UI documentation](./zipkin-trace.md#lens-ui) for query OTLP traces.
\ No newline at end of file
diff --git a/oap-server/server-receiver-plugin/otel-receiver-plugin/pom.xml b/oap-server/server-receiver-plugin/otel-receiver-plugin/pom.xml
index 3b73c24f5bb2..b3b0bd3aa930 100644
--- a/oap-server/server-receiver-plugin/otel-receiver-plugin/pom.xml
+++ b/oap-server/server-receiver-plugin/otel-receiver-plugin/pom.xml
@@ -49,5 +49,10 @@
             <artifactId>log-analyzer</artifactId>
             <version>${project.version}</version>
         </dependency>
+        <dependency>
+            <groupId>org.apache.skywalking</groupId>
+            <artifactId>zipkin-receiver-plugin</artifactId>
+            <version>${project.version}</version>
+        </dependency>
     </dependencies>
 </project>
diff --git a/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/OtelMetricReceiverProvider.java b/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/OtelMetricReceiverProvider.java
index 0ba59525d980..0cf91ca315d9 100644
--- a/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/OtelMetricReceiverProvider.java
+++ b/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/OtelMetricReceiverProvider.java
@@ -25,6 +25,7 @@
 import org.apache.skywalking.oap.server.receiver.otel.otlp.OpenTelemetryLogHandler;
 import org.apache.skywalking.oap.server.receiver.otel.otlp.OpenTelemetryMetricHandler;
 import org.apache.skywalking.oap.server.receiver.otel.otlp.OpenTelemetryMetricRequestProcessor;
+import org.apache.skywalking.oap.server.receiver.otel.otlp.OpenTelemetryTraceHandler;
 import org.apache.skywalking.oap.server.receiver.sharing.server.SharingServerModule;
 
 import java.util.ArrayList;
@@ -80,6 +81,10 @@ public void prepare() throws ServiceNotProvidedException, ModuleStartException {
         if (enabledHandlers.contains(openTelemetryLogHandler.type())) {
             handlers.add(openTelemetryLogHandler);
         }
+        final var openTelemetryTraceHandler = new OpenTelemetryTraceHandler(getManager());
+        if (enabledHandlers.contains(openTelemetryTraceHandler.type())) {
+            handlers.add(openTelemetryTraceHandler);
+        }
         this.handlers = handlers;
     }
 
diff --git a/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryTraceHandler.java b/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryTraceHandler.java
new file mode 100644
index 000000000000..18e76bee346f
--- /dev/null
+++ b/oap-server/server-receiver-plugin/otel-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/otel/otlp/OpenTelemetryTraceHandler.java
@@ -0,0 +1,377 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.otel.otlp;
+
+import com.google.gson.JsonObject;
+import com.google.protobuf.ByteString;
+import io.grpc.stub.StreamObserver;
+import io.opentelemetry.proto.collector.trace.v1.ExportTraceServiceRequest;
+import io.opentelemetry.proto.collector.trace.v1.ExportTraceServiceResponse;
+import io.opentelemetry.proto.collector.trace.v1.TraceServiceGrpc;
+import io.opentelemetry.proto.common.v1.AnyValue;
+import io.opentelemetry.proto.common.v1.InstrumentationScope;
+import io.opentelemetry.proto.common.v1.KeyValue;
+import io.opentelemetry.proto.resource.v1.Resource;
+import io.opentelemetry.proto.trace.v1.ScopeSpans;
+import io.opentelemetry.proto.trace.v1.Status;
+import lombok.RequiredArgsConstructor;
+import lombok.extern.slf4j.Slf4j;
+import org.apache.skywalking.oap.server.core.server.GRPCHandlerRegister;
+import org.apache.skywalking.oap.server.library.module.ModuleManager;
+import org.apache.skywalking.oap.server.library.module.ModuleStartException;
+import org.apache.skywalking.oap.server.library.util.StringUtil;
+import org.apache.skywalking.oap.server.receiver.otel.Handler;
+import org.apache.skywalking.oap.server.receiver.sharing.server.SharingServerModule;
+import org.apache.skywalking.oap.server.receiver.zipkin.SpanForwardService;
+import org.apache.skywalking.oap.server.receiver.zipkin.ZipkinReceiverModule;
+import zipkin2.Endpoint;
+import zipkin2.Span;
+
+import java.math.BigInteger;
+import java.nio.ByteBuffer;
+import java.nio.charset.StandardCharsets;
+import java.util.ArrayList;
+import java.util.Base64;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
+import java.util.stream.Collectors;
+
+@Slf4j
+@RequiredArgsConstructor
+public class OpenTelemetryTraceHandler
+    extends TraceServiceGrpc.TraceServiceImplBase
+    implements Handler {
+    private final ModuleManager manager;
+    private SpanForwardService forwardService;
+
+    @Override
+    public String type() {
+        return "otlp-traces";
+    }
+
+    @Override
+    public void active() throws ModuleStartException {
+        GRPCHandlerRegister grpcHandlerRegister = manager.find(SharingServerModule.NAME)
+            .provider()
+            .getService(GRPCHandlerRegister.class);
+        grpcHandlerRegister.addHandler(this);
+    }
+
+    @Override
+    public void export(ExportTraceServiceRequest request, StreamObserver<ExportTraceServiceResponse> responseObserver) {
+        final ArrayList<Span> result = new ArrayList<>();
+        request.getResourceSpansList().forEach(resourceSpans -> {
+            final Resource resource = resourceSpans.getResource();
+            final List<ScopeSpans> scopeSpansList = resourceSpans.getScopeSpansList();
+            if (resource.getAttributesCount() == 0 && scopeSpansList.size() == 0) {
+                return;
+            }
+
+            final Map<String, String> resourceTags = convertAttributeToMap(resource.getAttributesList());
+            String serviceName = extractZipkinServiceName(resourceTags);
+            if (StringUtil.isEmpty(serviceName)) {
+                log.warn("No service name found in resource attributes, discarding the trace");
+                return;
+            }
+
+            try {
+                for (ScopeSpans scopeSpans : scopeSpansList) {
+                    extractScopeTag(scopeSpans.getScope(), resourceTags);
+                    for (io.opentelemetry.proto.trace.v1.Span span : scopeSpans.getSpansList()) {
+                        Span zipkinSpan = convertSpan(span, serviceName, resourceTags);
+                        result.add(zipkinSpan);
+                    }
+                }
+            } catch (Exception e) {
+                log.warn("convert span error, discarding the span: {}", e.getMessage());
+            }
+        });
+
+        getForwardService().send(result);
+        responseObserver.onNext(ExportTraceServiceResponse.getDefaultInstance());
+        responseObserver.onCompleted();
+    }
+
+    private Span convertSpan(io.opentelemetry.proto.trace.v1.Span span, String serviceName, Map<String, String> resourceTags) {
+        final Span.Builder spanBuilder = Span.newBuilder();
+        final Map<String, String> tags = aggregateSpanTags(span.getAttributesList(), resourceTags);
+
+        if (span.getTraceId().isEmpty()) {
+            throw new IllegalArgumentException("No trace id found in span");
+        }
+        spanBuilder.traceId(
+            ByteBuffer.wrap(span.getTraceId().toByteArray(), 0, 8).getLong(),
+            ByteBuffer.wrap(span.getTraceId().toByteArray(), 8, span.getTraceId().size() - 8).getLong()
+        );
+
+        if (span.getSpanId().isEmpty()) {
+            throw new IllegalArgumentException("No span id found in span");
+        }
+        spanBuilder.id(convertSpanId(span.getSpanId()));
+
+        tags.put("w3c.tracestate", span.getTraceState());
+
+        if (!span.getParentSpanId().isEmpty()) {
+            spanBuilder.parentId(convertSpanId(span.getParentSpanId()));
+        }
+
+        spanBuilder.name(span.getName());
+        final long startMicro = TimeUnit.NANOSECONDS.toMicros(span.getStartTimeUnixNano());
+        final long endMicro = TimeUnit.NANOSECONDS.toMicros(span.getEndTimeUnixNano());
+        spanBuilder.timestamp(startMicro);
+        spanBuilder.duration(endMicro - startMicro);
+
+        spanBuilder.kind(convertKind(span.getKind()));
+        if (span.getKind() == io.opentelemetry.proto.trace.v1.Span.SpanKind.SPAN_KIND_INTERNAL) {
+            tags.put("span.kind", "internal");
+        }
+
+        final Set<String> redundantKeys = new HashSet<>();
+        spanBuilder.localEndpoint(convertEndpointFromTags(tags, serviceName, false, redundantKeys));
+        spanBuilder.remoteEndpoint(convertEndpointFromTags(tags, "", true, redundantKeys));
+
+        removeRedundantTags(tags, redundantKeys);
+        populateStatus(span.getStatus(), tags);
+
+        convertAnnotations(spanBuilder, span.getEventsList());
+        convertLink(tags, span.getLinksList());
+
+        tags.forEach(spanBuilder::putTag);
+
+        return spanBuilder.build();
+    }
+
+    private void convertAnnotations(Span.Builder spanBuilder, List<io.opentelemetry.proto.trace.v1.Span.Event> events) {
+        events.forEach(event -> {
+            final long eventTime = TimeUnit.NANOSECONDS.toMicros(event.getTimeUnixNano());
+            if (event.getAttributesList().size() == 0 && event.getDroppedAttributesCount() == 0) {
+                spanBuilder.addAnnotation(eventTime, event.getName());
+                return;
+            }
+
+            final JsonObject attrObj = convertToString(event.getAttributesList());
+            spanBuilder.addAnnotation(eventTime,
+                event.getName() + "|" + attrObj + "|" + event.getDroppedAttributesCount());
+        });
+    }
+
+    private void convertLink(Map<String, String> tags, List<io.opentelemetry.proto.trace.v1.Span.Link> links) {
+        for (int i = 0; i < links.size(); i++) {
+            final io.opentelemetry.proto.trace.v1.Span.Link link = links.get(i);
+            tags.put("otlp.link." + i,
+                idToHexString(link.getTraceId()) + "|" + idToHexString(link.getSpanId()) + "|" +
+                link.getTraceState() + "|" + convertToString(link.getAttributesList()) + "|" +
+                link.getDroppedAttributesCount());
+        }
+    }
+
+    private String idToHexString(ByteString id) {
+        if (id == null) {
+            return "";
+        }
+        return new BigInteger(1, id.toByteArray()).toString();
+    }
+
+    private void populateStatus(Status status, Map<String, String> tags) {
+        if (status.getCode() == Status.StatusCode.STATUS_CODE_ERROR) {
+            tags.put("error", "true");
+        } else {
+            tags.remove("error");
+        }
+
+        if (status.getCode() == Status.StatusCode.STATUS_CODE_UNSET) {
+            return;
+        }
+
+        tags.put("otel.status_code", status.getCode().name());
+        if (StringUtil.isNotEmpty(status.getMessage())) {
+            tags.put("otel.status_description", status.getMessage());
+        }
+    }
+
+    private void removeRedundantTags(Map<String, String> resourceKeys, Set<String> redundantKeys) {
+        for (String key : redundantKeys) {
+            resourceKeys.remove(key);
+        }
+    }
+
+    private Endpoint convertEndpointFromTags(Map<String, String> resourceTags, String localServiceName, boolean isRemote, Set<String> redundantKeys) {
+        final Endpoint.Builder builder = Endpoint.newBuilder();
+        String serviceName = localServiceName;
+        String tmpVal;
+        if (isRemote && StringUtil.isNotEmpty(tmpVal = getAndPutRedundantKey(resourceTags, "peer.service", redundantKeys))) {
+            serviceName = tmpVal;
+        } else if (isRemote &&
+            StringUtil.isNotEmpty(tmpVal = getAndPutRedundantKey(resourceTags, "net.peer.name", redundantKeys)) &&
+            // if it's not IP, then define it as service name
+            !builder.parseIp(tmpVal)) {
+            serviceName = tmpVal;
+        }
+
+        String ipKey, portKey;
+        if (isRemote) {
+            ipKey = "net.peer.ip";
+            portKey = "net.peer.port";
+        } else {
+            ipKey = "net.host.ip";
+            portKey = "net.host.port";
+        }
+
+        boolean ipParseSuccess = false;
+        if (StringUtil.isNotEmpty(tmpVal = getAndPutRedundantKey(resourceTags, ipKey, redundantKeys))) {
+            if (!(ipParseSuccess = builder.parseIp(tmpVal))) {
+                // if ip parse failed, use the value as service name
+                serviceName = StringUtil.isEmpty(serviceName) ? tmpVal : serviceName;
+            }
+        }
+        if (StringUtil.isNotEmpty(tmpVal = getAndPutRedundantKey(resourceTags, portKey, redundantKeys))) {
+            builder.port(Integer.parseInt(tmpVal));
+        }
+        if (StringUtil.isEmpty(serviceName) && !ipParseSuccess) {
+            return null;
+        }
+
+        builder.serviceName(serviceName);
+        return builder.build();
+    }
+
+    private String getAndPutRedundantKey(Map<String, String> resourceTags, String key, Set<String> redundantKeys) {
+        String val = resourceTags.get(key);
+        if (StringUtil.isEmpty(val)) {
+            return null;
+        }
+        redundantKeys.add(key);
+        return val;
+    }
+
+    private Span.Kind convertKind(io.opentelemetry.proto.trace.v1.Span.SpanKind kind) {
+        switch (kind) {
+            case SPAN_KIND_CLIENT:
+                return Span.Kind.CLIENT;
+            case SPAN_KIND_SERVER:
+                return Span.Kind.SERVER;
+            case SPAN_KIND_PRODUCER:
+                return Span.Kind.PRODUCER;
+            case SPAN_KIND_CONSUMER:
+                return Span.Kind.CONSUMER;
+        }
+        return null;
+    }
+
+    private long convertSpanId(ByteString spanId) {
+        return ByteBuffer.wrap(spanId.toByteArray()).getLong();
+    }
+
+    private Map<String, String> aggregateSpanTags(List<KeyValue> spanAttrs, Map<String, String> resourceTags) {
+        final HashMap<String, String> result = new HashMap<>();
+        result.putAll(resourceTags);
+        result.putAll(convertAttributeToMap(spanAttrs));
+        return result;
+    }
+
+    private void extractScopeTag(InstrumentationScope scope, Map<String, String> resourceTags) {
+        if (scope == null) {
+            return;
+        }
+
+        if (StringUtil.isNotEmpty(scope.getName())) {
+            resourceTags.put("otel.library.name", scope.getName());
+        }
+        if (StringUtil.isNotEmpty(scope.getVersion())) {
+            resourceTags.put("otel.library.version", scope.getVersion());
+        }
+    }
+
+    private Map<String, String> convertAttributeToMap(List<KeyValue> attrs) {
+        return attrs.stream().collect(Collectors.toMap(
+            KeyValue::getKey,
+            attributeKeyValue -> convertToString(attributeKeyValue.getValue()),
+            (v1, v2) -> v1
+        ));
+    }
+
+    private String extractZipkinServiceName(Map<String, String> resourceTags) {
+        String name = null;
+        name = getServiceNameFromTags(name, resourceTags, "service.name", false);
+        name = getServiceNameFromTags(name, resourceTags, "faas.name", true);
+        name = getServiceNameFromTags(name, resourceTags, "k8s.deployment.name", true);
+        name = getServiceNameFromTags(name, resourceTags, "process.executable.name", true);
+
+        return name;
+    }
+
+    private String getServiceNameFromTags(String serviceName, Map<String, String> resourceTags, String tagKey, boolean addingSource) {
+        if (StringUtil.isNotEmpty(serviceName)) {
+            return serviceName;
+        }
+
+        String name = resourceTags.get(tagKey);
+        if (StringUtil.isNotEmpty(name)) {
+            if (addingSource) {
+                resourceTags.remove(tagKey);
+                resourceTags.put("otlp.service.name.source", tagKey);
+            }
+            return name;
+        }
+        return "";
+    }
+
+    private String convertToString(AnyValue value) {
+        if (value == null) {
+            return "";
+        }
+
+        if (value.hasBoolValue()) {
+            return String.valueOf(value.getBoolValue());
+        } else if (value.hasDoubleValue()) {
+            return String.valueOf(value.getDoubleValue());
+        } else if (value.hasStringValue()) {
+            return value.getStringValue();
+        } else if (value.hasArrayValue()) {
+            return value.getArrayValue().getValuesList().stream().map(this::convertToString).collect(Collectors.joining(","));
+        } else if (value.hasIntValue()) {
+            return String.valueOf(value.getIntValue());
+        } else if (value.hasKvlistValue()) {
+            final JsonObject kvObj = convertToString(value.getKvlistValue().getValuesList());
+            return kvObj.getAsString();
+        } else if (value.hasBytesValue()) {
+            return new String(Base64.getEncoder().encode(value.getBytesValue().toByteArray()), StandardCharsets.UTF_8);
+        }
+        return "";
+    }
+
+    private JsonObject convertToString(List<KeyValue> keyValues) {
+        final JsonObject json = new JsonObject();
+        for (KeyValue keyValue : keyValues) {
+            json.addProperty(keyValue.getKey(), convertToString(keyValue.getValue()));
+        }
+        return json;
+    }
+
+    private SpanForwardService getForwardService() {
+        if (forwardService == null) {
+            forwardService = manager.find(ZipkinReceiverModule.NAME).provider().getService(SpanForwardService.class);
+        }
+        return forwardService;
+    }
+}
diff --git a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/SpanForwardService.java b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/SpanForwardService.java
new file mode 100644
index 000000000000..91d800b07d3c
--- /dev/null
+++ b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/SpanForwardService.java
@@ -0,0 +1,31 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ *     http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ *
+ */
+
+package org.apache.skywalking.oap.server.receiver.zipkin;
+
+import org.apache.skywalking.oap.server.library.module.Service;
+import zipkin2.Span;
+
+import java.util.List;
+
+public interface SpanForwardService extends Service {
+    /**
+     * Forward and process zipkin span
+     */
+    void send(List<Span> spanList);
+}
diff --git a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/ZipkinReceiverModule.java b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/ZipkinReceiverModule.java
index 0e0d9512558c..37eb3b76c2f6 100644
--- a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/ZipkinReceiverModule.java
+++ b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/ZipkinReceiverModule.java
@@ -36,6 +36,8 @@ public ZipkinReceiverModule() {
 
     @Override
     public Class[] services() {
-        return new Class[0];
+        return new Class[] {
+            SpanForwardService.class
+        };
     }
 }
diff --git a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/ZipkinReceiverProvider.java b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/ZipkinReceiverProvider.java
index e494c99f50a8..7ba0e1bbb55c 100644
--- a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/ZipkinReceiverProvider.java
+++ b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/ZipkinReceiverProvider.java
@@ -30,6 +30,7 @@
 import org.apache.skywalking.oap.server.library.server.http.HTTPServerConfig;
 import org.apache.skywalking.oap.server.receiver.zipkin.handler.ZipkinSpanHTTPHandler;
 import org.apache.skywalking.oap.server.receiver.zipkin.kafka.KafkaHandler;
+import org.apache.skywalking.oap.server.receiver.zipkin.trace.SpanForward;
 import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
 
 public class ZipkinReceiverProvider extends ModuleProvider {
@@ -37,6 +38,7 @@ public class ZipkinReceiverProvider extends ModuleProvider {
     private ZipkinReceiverConfig config;
     private HTTPServer httpServer;
     private KafkaHandler kafkaHandler;
+    private SpanForward spanForward;
 
     @Override
     public String name() {
@@ -65,6 +67,8 @@ public void onInitialized(final ZipkinReceiverConfig initialized) {
 
     @Override
     public void prepare() throws ServiceNotProvidedException {
+        this.spanForward = new SpanForward(config, getManager());
+        this.registerServiceImplementation(SpanForwardService.class, spanForward);
     }
 
     @Override
@@ -88,13 +92,13 @@ public void start() throws ServiceNotProvidedException, ModuleStartException {
             httpServer.initialize();
 
             httpServer.addHandler(
-                new ZipkinSpanHTTPHandler(config, getManager()),
+                new ZipkinSpanHTTPHandler(this.spanForward, getManager()),
                 Arrays.asList(HttpMethod.POST, HttpMethod.GET)
             );
         }
 
         if (config.isEnableKafkaCollector()) {
-            kafkaHandler = new KafkaHandler(config, getManager());
+            kafkaHandler = new KafkaHandler(config, this.spanForward, getManager());
         }
     }
 
diff --git a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/handler/ZipkinSpanHTTPHandler.java b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/handler/ZipkinSpanHTTPHandler.java
index 4befc24dde1a..bf77bc448105 100644
--- a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/handler/ZipkinSpanHTTPHandler.java
+++ b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/handler/ZipkinSpanHTTPHandler.java
@@ -28,7 +28,6 @@
 import java.util.List;
 import lombok.extern.slf4j.Slf4j;
 import org.apache.skywalking.oap.server.library.module.ModuleManager;
-import org.apache.skywalking.oap.server.receiver.zipkin.ZipkinReceiverConfig;
 import org.apache.skywalking.oap.server.receiver.zipkin.trace.SpanForward;
 import org.apache.skywalking.oap.server.telemetry.TelemetryModule;
 import org.apache.skywalking.oap.server.telemetry.api.CounterMetrics;
@@ -46,8 +45,8 @@ public class ZipkinSpanHTTPHandler {
     private final CounterMetrics errorCounter;
     private final SpanForward spanForward;
 
-    public ZipkinSpanHTTPHandler(ZipkinReceiverConfig config, ModuleManager manager) {
-        this.spanForward = new SpanForward(config, manager);
+    public ZipkinSpanHTTPHandler(SpanForward forward, ModuleManager manager) {
+        this.spanForward = forward;
         MetricsCreator metricsCreator = manager.find(TelemetryModule.NAME)
                                                .provider()
                                                .getService(MetricsCreator.class);
diff --git a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/kafka/KafkaHandler.java b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/kafka/KafkaHandler.java
index 49f383989018..bdcac89ff46f 100644
--- a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/kafka/KafkaHandler.java
+++ b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/kafka/KafkaHandler.java
@@ -58,9 +58,9 @@ public class KafkaHandler {
     private final CounterMetrics errorCounter;
     private final HistogramMetrics histogram;
 
-    public KafkaHandler(final ZipkinReceiverConfig config, ModuleManager manager) {
+    public KafkaHandler(final ZipkinReceiverConfig config, SpanForward forward, ModuleManager manager) {
         this.config = config;
-        this.spanForward = new SpanForward(config, manager);
+        this.spanForward = forward;
 
         properties = new Properties();
         properties.setProperty(ConsumerConfig.GROUP_ID_CONFIG, config.getKafkaGroupId());
diff --git a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/trace/SpanForward.java b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/trace/SpanForward.java
index cb4a3e220601..0500bf35bd42 100644
--- a/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/trace/SpanForward.java
+++ b/oap-server/server-receiver-plugin/zipkin-receiver-plugin/src/main/java/org/apache/skywalking/oap/server/receiver/zipkin/trace/SpanForward.java
@@ -41,23 +41,24 @@
 import org.apache.skywalking.oap.server.library.module.ModuleManager;
 import org.apache.skywalking.oap.server.library.util.CollectionUtils;
 import org.apache.skywalking.oap.server.library.util.StringUtil;
+import org.apache.skywalking.oap.server.receiver.zipkin.SpanForwardService;
 import org.apache.skywalking.oap.server.receiver.zipkin.ZipkinReceiverConfig;
 import zipkin2.Annotation;
 import zipkin2.Span;
 import zipkin2.internal.HexCodec;
 
 @Slf4j
-public class SpanForward {
+public class SpanForward implements SpanForwardService {
     private final ZipkinReceiverConfig config;
-    private final NamingControl namingControl;
-    private final SourceReceiver receiver;
+    private final ModuleManager moduleManager;
     private final List<String> searchTagKeys;
     private final long samplerBoundary;
+    private NamingControl namingControl;
+    private SourceReceiver receiver;
 
     public SpanForward(final ZipkinReceiverConfig config, final ModuleManager manager) {
         this.config = config;
-        this.namingControl = manager.find(CoreModule.NAME).provider().getService(NamingControl.class);
-        this.receiver = manager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+        this.moduleManager = manager;
         this.searchTagKeys = Arrays.asList(config.getSearchableTracesTags().split(Const.COMMA));
         float sampleRate = (float) config.getSampleRate() / 10000;
         samplerBoundary = (long) (Long.MAX_VALUE * sampleRate);
@@ -77,12 +78,12 @@ public void send(List<Span> spanList) {
             zipkinSpan.setTraceId(span.traceId());
             zipkinSpan.setSpanId(span.id());
             zipkinSpan.setParentId(span.parentId());
-            zipkinSpan.setName(namingControl.formatEndpointName(serviceName, span.name()));
+            zipkinSpan.setName(getNamingControl().formatEndpointName(serviceName, span.name()));
             zipkinSpan.setDuration(span.duration());
             if (span.kind() != null) {
                 zipkinSpan.setKind(span.kind().name());
             }
-            zipkinSpan.setLocalEndpointServiceName(namingControl.formatServiceName(serviceName));
+            zipkinSpan.setLocalEndpointServiceName(getNamingControl().formatServiceName(serviceName));
             if (span.localEndpoint() != null) {
                 zipkinSpan.setLocalEndpointIPV4(span.localEndpoint().ipv4());
                 zipkinSpan.setLocalEndpointIPV6(span.localEndpoint().ipv6());
@@ -92,7 +93,7 @@ public void send(List<Span> spanList) {
                 }
             }
             if (span.remoteEndpoint() != null) {
-                zipkinSpan.setRemoteEndpointServiceName(namingControl.formatServiceName(span.remoteServiceName()));
+                zipkinSpan.setRemoteEndpointServiceName(getNamingControl().formatServiceName(span.remoteServiceName()));
                 zipkinSpan.setRemoteEndpointIPV4(span.remoteEndpoint().ipv4());
                 zipkinSpan.setRemoteEndpointIPV6(span.remoteEndpoint().ipv6());
                 Integer remotePort = span.remoteEndpoint().port();
@@ -144,7 +145,7 @@ public void send(List<Span> spanList) {
                 }
                 zipkinSpan.setTags(tagsJson);
             }
-            receiver.receive(zipkinSpan);
+            getReceiver().receive(zipkinSpan);
 
             toService(zipkinSpan, minuteTimeBucket);
             toServiceSpan(zipkinSpan, minuteTimeBucket);
@@ -160,14 +161,14 @@ private void addAutocompleteTags(final long minuteTimeBucket, final String key,
         tagAutocomplete.setTagValue(value);
         tagAutocomplete.setTagType(TagType.ZIPKIN);
         tagAutocomplete.setTimeBucket(minuteTimeBucket);
-        receiver.receive(tagAutocomplete);
+        getReceiver().receive(tagAutocomplete);
     }
 
     private void toService(ZipkinSpan zipkinSpan, final long minuteTimeBucket) {
         ZipkinService service = new ZipkinService();
         service.setServiceName(zipkinSpan.getLocalEndpointServiceName());
         service.setTimeBucket(minuteTimeBucket);
-        receiver.receive(service);
+        getReceiver().receive(service);
     }
 
     private void toServiceSpan(ZipkinSpan zipkinSpan, final long minuteTimeBucket) {
@@ -175,7 +176,7 @@ private void toServiceSpan(ZipkinSpan zipkinSpan, final long minuteTimeBucket) {
         serviceSpan.setServiceName(zipkinSpan.getLocalEndpointServiceName());
         serviceSpan.setSpanName(zipkinSpan.getName());
         serviceSpan.setTimeBucket(minuteTimeBucket);
-        receiver.receive(serviceSpan);
+        getReceiver().receive(serviceSpan);
     }
 
     private void toServiceRelation(ZipkinSpan zipkinSpan, final long minuteTimeBucket) {
@@ -183,7 +184,7 @@ private void toServiceRelation(ZipkinSpan zipkinSpan, final long minuteTimeBucke
         relation.setServiceName(zipkinSpan.getLocalEndpointServiceName());
         relation.setRemoteServiceName(zipkinSpan.getRemoteEndpointServiceName());
         relation.setTimeBucket(minuteTimeBucket);
-        receiver.receive(relation);
+        getReceiver().receive(relation);
     }
 
     private List<Span> getSampledTraces(List<Span> input) {
@@ -205,4 +206,18 @@ private List<Span> getSampledTraces(List<Span> input) {
         }
         return sampledTraces;
     }
+
+    private NamingControl getNamingControl() {
+        if (namingControl == null) {
+            namingControl = moduleManager.find(CoreModule.NAME).provider().getService(NamingControl.class);
+        }
+        return namingControl;
+    }
+
+    private SourceReceiver getReceiver() {
+        if (receiver == null) {
+            receiver = moduleManager.find(CoreModule.NAME).provider().getService(SourceReceiver.class);
+        }
+        return receiver;
+    }
 }


Test patch:
diff --git a/test/e2e-v2/cases/otlp-traces/docker-compose.yml b/test/e2e-v2/cases/otlp-traces/docker-compose.yml
new file mode 100644
index 000000000000..8a12a5540b80
--- /dev/null
+++ b/test/e2e-v2/cases/otlp-traces/docker-compose.yml
@@ -0,0 +1,84 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+version: '3.9'
+x-default-logging: &logging
+  driver: "json-file"
+  options:
+    max-size: "5m"
+    max-file: "2"
+
+services:
+  frontend:
+    image: ghcr.io/open-telemetry/demo:1.4.0-frontend
+    deploy:
+      resources:
+        limits:
+          memory: 200M
+    restart: unless-stopped
+    ports:
+      - 8080
+    environment:
+      PORT: 8080
+      FRONTEND_ADDR: frontend:8080
+      PRODUCT_CATALOG_SERVICE_ADDR: productcatalogservice:3550
+      OTEL_EXPORTER_OTLP_ENDPOINT: http://oap:11800
+      OTEL_RESOURCE_ATTRIBUTES: service.namespace=opentelemetry-demo
+      ENV_PLATFORM: local
+      OTEL_SERVICE_NAME: frontend
+      WEB_OTEL_SERVICE_NAME: frontend-web
+      CURRENCY_SERVICE_ADDR: no.exist:80
+    depends_on:
+      - oap
+      - productcatalogservice
+    logging: *logging
+    networks:
+      - e2e
+
+  productcatalogservice:
+    image: ghcr.io/open-telemetry/demo:1.4.0-productcatalogservice
+    deploy:
+      resources:
+        limits:
+          memory: 20M
+    restart: unless-stopped
+    ports:
+      - "3550"
+    environment:
+      PRODUCT_CATALOG_SERVICE_PORT: 3550
+      OTEL_EXPORTER_OTLP_ENDPOINT: http://oap:11800
+      OTEL_RESOURCE_ATTRIBUTES: service.namespace=opentelemetry-demo
+      OTEL_SERVICE_NAME: productcatalogservice
+      FEATURE_FLAG_GRPC_SERVICE_ADDR: no.exist:80
+    depends_on:
+      - oap
+    logging: *logging
+    networks:
+      - e2e
+
+  oap:
+    extends:
+      file: ../../script/docker-compose/base-compose.yml
+      service: oap
+    ports:
+      - 12800
+      - 9412
+    environment:
+      SW_OTEL_RECEIVER_ENABLED_HANDLERS: otlp-metrics,otlp-logs,otlp-traces
+      SW_RECEIVER_ZIPKIN: default
+      SW_QUERY_ZIPKIN: default
+
+networks:
+  e2e:
diff --git a/test/e2e-v2/cases/otlp-traces/e2e.yaml b/test/e2e-v2/cases/otlp-traces/e2e.yaml
new file mode 100644
index 000000000000..1a1e3151873a
--- /dev/null
+++ b/test/e2e-v2/cases/otlp-traces/e2e.yaml
@@ -0,0 +1,60 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This file is used to show how to write configuration files and can be used to test.
+
+setup:
+  env: compose
+  file: docker-compose.yml
+  timeout: 20m
+  init-system-environment: ../../script/env
+  steps:
+    - name: set PATH
+      command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+    - name: install yq
+      command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+    - name: install swctl
+      command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
+
+trigger:
+  action: http
+  interval: 3s
+  times: 10
+  url: http://${frontend_host}:${frontend_8080}/api/products
+  method: GET
+
+verify:
+  # verify with retry strategy
+  retry:
+    # max retry count
+    count: 20
+    # the interval between two retries, in millisecond.
+    interval: 5s
+  cases:
+    # service name
+    - query: curl http://${oap_host}:${oap_9412}/zipkin/api/v2/services
+      expected: expected/service-name.yml
+    # remote-service name
+    - query: curl http://${oap_host}:${oap_9412}/zipkin/api/v2/remoteServices?serviceName=frontend
+      expected: expected/remote-service-name.yml
+    # span name
+    - query: curl http://${oap_host}:${oap_9412}/zipkin/api/v2/spans?serviceName=productcatalogservice
+      expected: expected/span-name.yml
+    # traces
+    - query: curl http://${oap_host}:${oap_9412}/zipkin/api/v2/traces\?limit\=1\&serviceName\=productcatalogservice\&spanName\=oteldemo.productcatalogservice/listproducts | yq e 'del(..|.tags?, ..|.annotations?)' -
+      expected: expected/traces.yml
+    # autocomplete
+    - query: curl http://${oap_host}:${oap_9412}/zipkin/api/v2/autocompleteValues?key=http.method
+      expected: expected/autocomplete.yml
diff --git a/test/e2e-v2/cases/otlp-traces/expected/autocomplete.yml b/test/e2e-v2/cases/otlp-traces/expected/autocomplete.yml
new file mode 100644
index 000000000000..30e36a52a9d1
--- /dev/null
+++ b/test/e2e-v2/cases/otlp-traces/expected/autocomplete.yml
@@ -0,0 +1,18 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+[
+  "GET"
+]
diff --git a/test/e2e-v2/cases/otlp-traces/expected/remote-service-name.yml b/test/e2e-v2/cases/otlp-traces/expected/remote-service-name.yml
new file mode 100644
index 000000000000..cf93883b414b
--- /dev/null
+++ b/test/e2e-v2/cases/otlp-traces/expected/remote-service-name.yml
@@ -0,0 +1,16 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+["productcatalogservice"]
diff --git a/test/e2e-v2/cases/otlp-traces/expected/service-name.yml b/test/e2e-v2/cases/otlp-traces/expected/service-name.yml
new file mode 100644
index 000000000000..81d7bd5b451f
--- /dev/null
+++ b/test/e2e-v2/cases/otlp-traces/expected/service-name.yml
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+[
+  "frontend",
+  "productcatalogservice"
+]
diff --git a/test/e2e-v2/cases/otlp-traces/expected/span-name.yml b/test/e2e-v2/cases/otlp-traces/expected/span-name.yml
new file mode 100644
index 000000000000..5190e5149760
--- /dev/null
+++ b/test/e2e-v2/cases/otlp-traces/expected/span-name.yml
@@ -0,0 +1,18 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+[
+  "oteldemo.productcatalogservice/listproducts"
+]
diff --git a/test/e2e-v2/cases/otlp-traces/expected/traces.yml b/test/e2e-v2/cases/otlp-traces/expected/traces.yml
new file mode 100644
index 000000000000..0e9ef65ea087
--- /dev/null
+++ b/test/e2e-v2/cases/otlp-traces/expected/traces.yml
@@ -0,0 +1,58 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{{- contains . }}
+-
+{{- contains . }}
+  - traceId: {{ notEmpty .traceId }}
+    parentId: {{ notEmpty .parentId}}
+    id: {{ notEmpty .id }}
+    kind: SERVER
+    name: oteldemo.productcatalogservice/listproducts
+    timestamp: {{ ge .timestamp 0}}
+    duration: {{ ge .duration 0}}
+    localEndpoint:
+      serviceName: productcatalogservice
+    remoteEndpoint:
+      ipv4: {{ notEmpty .remoteEndpoint.ipv4 }}
+      port: {{ ge .remoteEndpoint.port 0}}
+  - traceId: {{ notEmpty .traceId }}
+    parentId: {{ .parentId }}
+    id: {{ notEmpty .id }}
+    kind: CLIENT
+    name: grpc.oteldemo.productcatalogservice/listproducts
+    timestamp: {{ ge .timestamp 0}}
+    duration: {{ ge .duration 0}}
+    localEndpoint:
+      serviceName: frontend
+    remoteEndpoint:
+      serviceName: productcatalogservice
+      port: {{ ge .remoteEndpoint.port 0}}
+  - traceId: {{ notEmpty .traceId }}
+    id: {{ notEmpty .id }}
+    kind: SERVER
+    name: http get
+    timestamp: {{ ge .timestamp 0}}
+    duration: {{ ge .duration 0}}
+    localEndpoint:
+      serviceName: frontend
+      ipv4: {{ notEmpty .localEndpoint.ipv4 }}
+      port: 8080
+    remoteEndpoint:
+      ipv4: {{ notEmpty .remoteEndpoint.ipv4 }}
+      port: {{ ge .remoteEndpoint.port 0}}
+{{- end }}
+{{- end }}
+