Repository: apache/skywalking (language: python)

Issue:
Support Fetch Cilium Observe data to monitoring Cilium Service network traffic
In this PR, I mainly implemented the following features:
1. Combining data collection using [Peer](https://github.com/cilium/cilium/blob/main/api/v1/peer/peer_grpc.pb.go#L38) and [Flow](https://github.com/cilium/cilium/blob/main/api/v1/observer/observer_grpc.pb.go#L41) services provided by Cilium itself and converting it to SkyWalking data, finally displaying it in UI.
2. Added a `labelCount` function in the OAL engine for creating count statistics based on tags.
3. Rewrite FieldsHelper to support ALS as well as Cilium.

NOTE: Following https://github.com/cilium/cilium/issues/32175, we found that some flow data would be missing some metadata, so we cannot get both side metrics. Then, we decide to ignore the server-side data and convert the client-side data to the server-side. 

## Screenshots

### All Cilium Services Overview

![image](https://github.com/apache/skywalking/assets/3417650/0986b595-55f3-4562-b96a-beeb38f33a58)
![image](https://github.com/apache/skywalking/assets/3417650/66fad466-ecd4-4023-8dbe-67a0399fe2d3)
![image](https://github.com/apache/skywalking/assets/3417650/7ecfb8c7-5735-4985-8f96-53256f788e73)

### Service Overview

![image](https://github.com/apache/skywalking/assets/3417650/89506a3b-b0d7-4e24-8994-f0fbca36d3b0)
![image](https://github.com/apache/skywalking/assets/3417650/5571dea1-0cf5-41bb-85de-945ecc25ae3e)
![image](https://github.com/apache/skywalking/assets/3417650/0c8ef6ff-a8e1-4a20-9e49-a316b713dc08)
![image](https://github.com/apache/skywalking/assets/3417650/f5b4951b-c3b8-4b7a-9c8e-4cb9abc46c23)

### Service Relation Overview

![image](https://github.com/apache/skywalking/assets/3417650/02a023aa-1447-4df1-be6f-c5266f78f68b)
![image](https://github.com/apache/skywalking/assets/3417650/7189d7ac-ef50-416d-a3c5-4f64b377afcd)

### Service Instance Overview

![image](https://github.com/apache/skywalking/assets/3417650/81c331cb-f195-4c0a-aec7-f707f1925dac)
![image](https://github.com/apache/skywalking/assets/3417650/7c8f05a5-964d-4b4a-9530-4853b7d85d35)

### Service Instance Relaiton Overview

![image](https://github.com/apache/skywalking/assets/3417650/2a2e626c-81e1-4bdc-84c6-02e574496779)

### Endpoint Overview

![image](https://github.com/apache/skywalking/assets/3417650/98dfb13e-f464-4583-bf15-72b49a82cd63)
![image](https://github.com/apache/skywalking/assets/3417650/5e09d927-b5f4-4b47-9df0-077e1732ac8a)

- [x] If it's UI related, attach the screenshots below.
- [x] If this pull request closes/resolves/fixes an existing issue, replace the issue number. Part of #11697.
- [x] Update the [`CHANGES` log](https://github.com/apache/skywalking/blob/master/docs/en/changes/changes.md).



Discussion / hints:
UI submodule seems including more changes. Could you update the changelogs of UI side?
How about e2e of Cilium?
Cilium integration docs should be added in the marketplace, https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-k8s-monitoring/.
And the dashboard on the UI should point to it.

The docs should include
1. Integration architecture and data flow
2. API dependencies
3. Process logic and the brief about how we process the data
4. How OAP cluster works in fetching mode. 
> UI submodule seems including more changes. Could you update the changelogs of UI side?

Updated. 
> How about e2e of Cilium?

I will trying to create other PR for the E2E Testing. 
> Cilium integration docs should be added in the marketplace, https://skywalking.apache.org/docs/main/next/en/setup/backend/backend-k8s-monitoring/. And the dashboard on the UI should point to it.
> 
> The docs should include
> 
> 1. Integration architecture and data flow
> 2. API dependencies
> 3. Process logic and the brief about how we process the data
> 4. How OAP cluster works in fetching mode.

Thanks. I have update the documentation following your mentioned. 
Two questions
1. Is Hubble peer always deployed as one pod? Because I only see one hostname and one port.
2. How does OAP cluster work with Hubble peer? How many pods of OAP are going to fetch and process metrics from Hubble peer?

> 1. Is Hubble peer always deployed as one pod? Because I only see one hostname and one port.

Hubble peer service is correlated with the cilium node, not a only pod. And Kubernetes will load balance with wich pod should be listen. 

> 2. How does OAP cluster work with Hubble peer? How many pods of OAP are going to fetch and process metrics from Hubble peer?

I have write a selection about this: [**Load Balance for Cilium Fetcher with OAP cluster**](https://github.com/apache/skywalking/pull/12393/files#diff-c709949b64ba7a07a8c5fe555a5760148f81eb7c9c429f167a0b57570aa952bcR97-R103). All the OAP nodes will listen bubble peer service for getting nodes. And each OAP node will calucate how many cilium node should be monitoring. 

Fix patch:
diff --git a/.github/workflows/skywalking.yaml b/.github/workflows/skywalking.yaml
index 943730c75a1f..bc0cfe9b1525 100644
--- a/.github/workflows/skywalking.yaml
+++ b/.github/workflows/skywalking.yaml
@@ -685,6 +685,9 @@ jobs:
 
           - name: OTLP Trace
             config:  test/e2e-v2/cases/otlp-traces/e2e.yaml
+
+          - name: Cilium Service
+            config: test/e2e-v2/cases/cilium/e2e.yaml
     steps:
       - uses: actions/checkout@v3
         with:
@@ -720,7 +723,7 @@ jobs:
         if: matrix.test.docker != null
         run: docker build -t ${{ matrix.test.docker.name }} -f ${{ matrix.test.docker.base }}/${{ matrix.test.docker.file }} ${{ matrix.test.docker.base }}
       - name: ${{ matrix.test.name }}
-        uses: apache/skywalking-infra-e2e@0a5b398fc9668ccb848b16e6da4f09180955dc3e
+        uses: apache/skywalking-infra-e2e@7f6b0da20f170b181a79cf89ef7d47848ac34138
         with:
           e2e-file: $GITHUB_WORKSPACE/${{ matrix.test.config }}
       - if: ${{ failure() }}
diff --git a/oap-server/server-fetcher-plugin/cilium-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/cilium/nodes/CiliumNodeManager.java b/oap-server/server-fetcher-plugin/cilium-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/cilium/nodes/CiliumNodeManager.java
index e5068fd2bbb2..f73159239c6e 100644
--- a/oap-server/server-fetcher-plugin/cilium-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/cilium/nodes/CiliumNodeManager.java
+++ b/oap-server/server-fetcher-plugin/cilium-fetcher-plugin/src/main/java/org/apache/skywalking/oap/server/fetcher/cilium/nodes/CiliumNodeManager.java
@@ -221,6 +221,7 @@ private void reBuildUsingNodes(List<CiliumNode> shouldUsingNodes) {
             }
         });
 
+        newNodes.sort(Comparator.comparing(CiliumNode::getAddress));
         this.usingNodes = ImmutableList.copyOf(newNodes);
     }
 
diff --git a/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-endpoint.json b/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-endpoint.json
index fa20e1aa975e..0ded5f6fe7bb 100644
--- a/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-endpoint.json
+++ b/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-endpoint.json
@@ -81,7 +81,6 @@
                                     "type": "Widget",
                                     "expressions": [
                                         "cilium_endpoint_protocol_http_status_1xx_cpm",
-                                        "cilium_endpoint_protocol_http_status_2xx_cpm",
                                         "cilium_endpoint_protocol_http_status_3xx_cpm",
                                         "cilium_endpoint_protocol_http_status_4xx_cpm",
                                         "cilium_endpoint_protocol_http_status_5xx_cpm"
@@ -98,9 +97,6 @@
                                         {
                                             "label": "1xx"
                                         },
-                                        {
-                                            "label": "2xx"
-                                        },
                                         {
                                             "label": "3xx"
                                         },
@@ -112,7 +108,7 @@
                                         }
                                     ],
                                     "widget": {
-                                        "title": "HTTP Response Code(Count per min)"
+                                        "title": "HTTP Non-2xx Response Code(Count per min)"
                                     }
                                 },
                                 {
diff --git a/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-instance-relation.json b/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-instance-relation.json
index f37596df9575..a96161108324 100644
--- a/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-instance-relation.json
+++ b/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-instance-relation.json
@@ -304,7 +304,6 @@
                                     "type": "Widget",
                                     "expressions": [
                                         "cilium_service_instance_relation_protocol_http_status_1xx_cpm",
-                                        "cilium_service_instance_relation_protocol_http_status_2xx_cpm",
                                         "cilium_service_instance_relation_protocol_http_status_3xx_cpm",
                                         "cilium_service_instance_relation_protocol_http_status_4xx_cpm",
                                         "cilium_service_instance_relation_protocol_http_status_5xx_cpm"
@@ -318,15 +317,12 @@
                                         "showYAxis": true
                                     },
                                     "widget": {
-                                        "title": "HTTP Response Status Server Side(Count Per Min)"
+                                        "title": "HTTP Non-2xx Response Status Server Side(Count Per Min)"
                                     },
                                     "metricConfig": [
                                         {
                                             "label": "1xx"
                                         },
-                                        {
-                                            "label": "2xx"
-                                        },
                                         {
                                             "label": "3xx"
                                         },
diff --git a/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-instance.json b/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-instance.json
index 40fc936210f4..131e81152ddf 100644
--- a/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-instance.json
+++ b/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-instance.json
@@ -188,7 +188,6 @@
                                     "type": "Widget",
                                     "expressions": [
                                         "cilium_service_instance_protocol_http_status_1xx_cpm",
-                                        "cilium_service_instance_protocol_http_status_2xx_cpm",
                                         "cilium_service_instance_protocol_http_status_3xx_cpm",
                                         "cilium_service_instance_protocol_http_status_4xx_cpm",
                                         "cilium_service_instance_protocol_http_status_5xx_cpm"
@@ -205,9 +204,6 @@
                                         {
                                             "label": "1xx"
                                         },
-                                        {
-                                            "label": "2xx"
-                                        },
                                         {
                                             "label": "3xx"
                                         },
@@ -219,7 +215,7 @@
                                         }
                                     ],
                                     "widget": {
-                                        "title": "HTTP Response Code(Count per min)"
+                                        "title": "HTTP Non-2xx Response Code(Count per min)"
                                     }
                                 },
                                 {
diff --git a/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-relation.json b/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-relation.json
index 485720c275a5..e177b653447a 100644
--- a/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-relation.json
+++ b/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service-relation.json
@@ -304,7 +304,6 @@
                                     "type": "Widget",
                                     "expressions": [
                                         "cilium_service_relation_protocol_http_status_1xx_cpm",
-                                        "cilium_service_relation_protocol_http_status_2xx_cpm",
                                         "cilium_service_relation_protocol_http_status_3xx_cpm",
                                         "cilium_service_relation_protocol_http_status_4xx_cpm",
                                         "cilium_service_relation_protocol_http_status_5xx_cpm"
@@ -318,15 +317,12 @@
                                         "showYAxis": true
                                     },
                                     "widget": {
-                                        "title": "HTTP Response Status Server Side(Count Per Min)"
+                                        "title": "HTTP Non-2xx Response Status Server Side(Count Per Min)"
                                     },
                                     "metricConfig": [
                                         {
                                             "label": "1xx"
                                         },
-                                        {
-                                            "label": "2xx"
-                                        },
                                         {
                                             "label": "3xx"
                                         },
diff --git a/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service.json b/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service.json
index 7de44eced554..330751d94558 100644
--- a/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service.json
+++ b/oap-server/server-starter/src/main/resources/ui-initialized-templates/cilium_service/cilium-service.json
@@ -188,7 +188,6 @@
                                     "type": "Widget",
                                     "expressions": [
                                         "cilium_service_protocol_http_status_1xx_cpm",
-                                        "cilium_service_protocol_http_status_2xx_cpm",
                                         "cilium_service_protocol_http_status_3xx_cpm",
                                         "cilium_service_protocol_http_status_4xx_cpm",
                                         "cilium_service_protocol_http_status_5xx_cpm"
@@ -205,9 +204,6 @@
                                         {
                                             "label": "1xx"
                                         },
-                                        {
-                                            "label": "2xx"
-                                        },
                                         {
                                             "label": "3xx"
                                         },
@@ -219,7 +215,7 @@
                                         }
                                     ],
                                     "widget": {
-                                        "title": "HTTP Response Code(Count per min)"
+                                        "title": "HTTP Non-2xx Response Code(Count per min)"
                                     }
                                 },
                                 {


Test patch:
diff --git a/test/e2e-v2/cases/cilium/cilium-network-policy.yaml b/test/e2e-v2/cases/cilium/cilium-network-policy.yaml
new file mode 100644
index 000000000000..70c4ec5f33c4
--- /dev/null
+++ b/test/e2e-v2/cases/cilium/cilium-network-policy.yaml
@@ -0,0 +1,161 @@
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: "cilium.io/v2"
+kind: CiliumNetworkPolicy
+metadata:
+    name: "rule3"
+spec:
+    description: "L7 policy to restrict access to specific HTTP call"
+    endpointSelector:
+        matchLabels:
+            app: productpage
+    ingress:
+        - fromEndpoints:
+              - matchLabels:
+                    "k8s:app": details
+              - matchLabels:
+                    "k8s:app": reviews
+              - matchLabels:
+                    "k8s:app": trafficgenerator
+    egress:
+        - toEndpoints:
+              - matchLabels:
+                    "k8s:io.kubernetes.pod.namespace": kube-system
+                    "k8s:k8s-app": kube-dns
+          toPorts:
+              - ports:
+                    - port: "53"
+                      protocol: ANY
+                rules:
+                    dns:
+                        - matchPattern: "*"
+        - toEndpoints:
+              - matchLabels:
+                    "k8s:app": trafficgenerator
+              - matchLabels:
+                    "k8s:app": details
+              - matchLabels:
+                    "k8s:app": reviews
+          toPorts:
+              - ports:
+                    - port: "9080"
+                      protocol: TCP
+                rules:
+                    http:
+                        - method: ""
+                          path: ""
+
+---
+
+apiVersion: "cilium.io/v2"
+kind: CiliumNetworkPolicy
+metadata:
+    name: "rule5"
+spec:
+    description: "L7 policy to restrict access to specific HTTP call"
+    endpointSelector:
+        matchLabels:
+            app: reviews
+    ingress:
+        - fromEndpoints:
+              - matchLabels:
+                    "k8s:app": productpage
+              - matchLabels:
+                    "k8s:app": ratings
+              - matchLabels:
+                    "k8s:io.kubernetes.pod.namespace": kube-system
+                    "k8s:k8s-app": kube-dns
+        - toPorts:
+              - ports:
+                    - port: "9080"
+                      protocol: TCP
+                rules:
+                    http:
+                        - method: ""
+                          path: ""
+    egress:
+        - toEndpoints:
+              - matchLabels:
+                    "k8s:io.kubernetes.pod.namespace": kube-system
+                    "k8s:k8s-app": kube-dns
+          toPorts:
+              - ports:
+                    - port: "53"
+                      protocol: ANY
+                rules:
+                    dns:
+                        - matchPattern: "*"
+        - toEndpoints:
+              - matchLabels:
+                    "k8s:app": productpage
+              - matchLabels:
+                    "k8s:app": ratings
+          toPorts:
+              - ports:
+                    - port: "9080"
+                      protocol: TCP
+                rules:
+                    http:
+                        - method: ""
+                          path: ""
+
+---
+
+apiVersion: "cilium.io/v2"
+kind: CiliumNetworkPolicy
+metadata:
+    name: "rule4"
+spec:
+    description: "L7 policy to restrict access to specific HTTP call"
+    endpointSelector:
+        matchLabels:
+            app: trafficgenerator
+    ingress:
+        - fromEndpoints:
+              - matchLabels:
+                    "k8s:app": productpage
+        - toPorts:
+              - ports:
+                    - port: "9080"
+                      protocol: TCP
+                rules:
+                    http:
+                        - method: ""
+                          path: ""
+    egress:
+        - toEndpoints:
+              - matchLabels:
+                    "k8s:io.kubernetes.pod.namespace": kube-system
+                    "k8s:k8s-app": kube-dns
+          toPorts:
+              - ports:
+                    - port: "53"
+                      protocol: ANY
+                rules:
+                    dns:
+                        - matchPattern: "*"
+        - toEndpoints:
+              - matchLabels:
+                    "k8s:app": productpage
+          toPorts:
+              - ports:
+                    - port: "9080"
+                      protocol: TCP
+                rules:
+                    http:
+                        - method: ""
+                          path: ""
\ No newline at end of file
diff --git a/test/e2e-v2/cases/cilium/e2e.yaml b/test/e2e-v2/cases/cilium/e2e.yaml
new file mode 100644
index 000000000000..26cc434bc6af
--- /dev/null
+++ b/test/e2e-v2/cases/cilium/e2e.yaml
@@ -0,0 +1,152 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# This file is used to show how to write configuration files and can be used to test.
+
+setup:
+  env: kind
+  file: kind.yaml
+  init-system-environment: ../../script/env
+  timeout: 10m
+  kind:
+    no-wait: true
+    import-images:
+      - skywalking/ui:latest
+      - skywalking/oap:latest
+    expose-ports:
+      - namespace: kube-system
+        resource: service/skywalking-ui
+        port: 80
+  steps:
+    - name: set PATH
+      command: export PATH=/tmp/skywalking-infra-e2e/bin:$PATH
+    - name: install yq
+      command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh yq
+    - name: install swctl
+      command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh swctl
+    - name: install kubectl
+      command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh kubectl
+    - name: Install helm
+      command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh helm
+    - name: Install kubectl
+      command: bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh kubectl
+    - name: install cilium
+      command: |
+        bash test/e2e-v2/script/prepare/setup-e2e-shell/install.sh cilium
+        cilium install --wait
+        cilium status --wait
+    - name: Install SkyWalking
+      command: |
+        helm -n kube-system install skywalking \
+          oci://ghcr.io/apache/skywalking-helm/skywalking-helm \
+          --version "0.0.0-${SW_KUBERNETES_COMMIT_SHA}" \
+          --set fullnameOverride=skywalking \
+          --set elasticsearch.replicas=1 \
+          --set elasticsearch.minimumMasterNodes=1 \
+          --set oap.replicas=1 \
+          --set ui.image.repository=skywalking/ui \
+          --set ui.image.tag=latest \
+          --set oap.image.tag=latest \
+          --set oap.image.repository=skywalking/oap \
+          --set oap.storageType=elasticsearch \
+          -f test/e2e-v2/cases/cilium/values.yaml
+      wait:
+        - namespace: kube-system
+          resource: deployments/skywalking-oap
+          for: condition=available
+    - name: Deploy demo services
+      command: |
+        kubectl apply -f https://raw.githubusercontent.com/istio/istio/1.18.0/samples/bookinfo/platform/kube/bookinfo.yaml
+      wait:
+        - namespace: default
+          resource: pod
+          for: condition=Ready
+    - name: Generate traffic
+      path: traffic-gen.yaml
+      wait:
+        - namespace: default
+          resource: pod
+          for: condition=Ready
+    - name: Apply Cilium Network Policy
+      path: cilium-network-policy.yaml
+
+verify:
+  retry:
+    count: 10
+    interval: 10s
+  cases:
+    # service list
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql service ls
+      expected: expected/service.yml
+    # service instance list
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=productpage.default
+      expected: expected/service-instance.yml
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=reviews.default
+      expected: expected/service-instance.yml
+    # service endpoint list
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql endpoint list --service-name=productpage.default
+      expected: expected/service-endpoint-productpage.yml
+
+    # dependency service
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql dependency global --layer=CILIUM_SERVICE
+      expected: expected/dependency-services.yml
+
+    # service metrics
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_l4_read_pkg_cpm --service-name='productpage.default'
+      expected: expected/metrics-has-value.yml
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_l4_write_pkg_cpm --service-name='productpage.default'
+      expected: expected/metrics-has-value.yml
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_protocol_http_call_cpm --service-name='productpage.default'
+      expected: expected/metrics-has-value.yml
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_protocol_http_status_2xx_cpm --service-name='productpage.default'
+      expected: expected/metrics-has-value.yml
+
+    # service instance metrics
+    - query: |
+        swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_instance_l4_read_pkg_cpm --service-name='productpage.default' --instance-name=$(
+          swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=productpage.default | yq e '.[0].name' -
+        )
+      expected: expected/metrics-has-value.yml
+    - query: |
+        swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_instance_l4_write_pkg_cpm --service-name='productpage.default' --instance-name=$(
+          swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=productpage.default | yq e '.[0].name' -
+        )
+      expected: expected/metrics-has-value.yml
+    - query: |
+        swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_instance_protocol_http_call_cpm --service-name='productpage.default' --instance-name=$(
+          swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=productpage.default | yq e '.[0].name' -
+        )
+      expected: expected/metrics-has-value.yml
+    - query: |
+        swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_instance_protocol_http_status_2xx_cpm --service-name='productpage.default' --instance-name=$(
+          swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql instance list --service-name=productpage.default | yq e '.[0].name' -
+        )
+      expected: expected/metrics-has-value.yml
+
+    # service endpoint metrics
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_endpoint_protocol_http_call_cpm --service-name='productpage.default' --endpoint-name="GET:/productpage"
+      expected: expected/metrics-has-value.yml
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_endpoint_protocol_http_status_2xx_cpm --service-name='productpage.default' --endpoint-name="GET:/productpage"
+      expected: expected/metrics-has-value.yml
+
+    # service relation metrics
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_relation_server_l4_read_pkg_cpm  --service-name='productpage.default' --dest-service-name='details.default'
+      expected: expected/metrics-has-value.yml
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_relation_server_l4_write_pkg_cpm  --service-name='productpage.default' --dest-service-name='details.default'
+      expected: expected/metrics-has-value.yml
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_relation_server_protocol_http_call_cpm  --service-name='productpage.default' --dest-service-name='details.default'
+      expected: expected/metrics-has-value.yml
+    - query: swctl --display yaml --base-url=http://${service_skywalking_ui_host}:${service_skywalking_ui_80}/graphql metrics exec --expression=cilium_service_relation_protocol_http_status_2xx_cpm  --service-name='productpage.default' --dest-service-name='details.default'
+      expected: expected/metrics-has-value.yml
diff --git a/test/e2e-v2/cases/cilium/expected/dependency-services.yml b/test/e2e-v2/cases/cilium/expected/dependency-services.yml
new file mode 100644
index 000000000000..ab14539a5ff8
--- /dev/null
+++ b/test/e2e-v2/cases/cilium/expected/dependency-services.yml
@@ -0,0 +1,70 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+nodes:
+{{- contains .nodes }}
+- id: {{ b64enc "productpage.default"}}.1
+  name: productpage.default
+  type: http
+  isreal: true
+- id: {{ b64enc "details.default"}}.1
+  name: details.default
+  type: http
+  isreal: true
+- id: {{ b64enc "ratings.default" }}.1
+  name: ratings.default
+  type: http
+  isreal: true
+- id: {{ b64enc "reviews.default" }}.1
+  name: reviews.default
+  type: http
+  isreal: true
+{{- end }}
+calls:
+{{- contains .calls }}
+- source: {{ b64enc "productpage.default"}}.1
+  sourcecomponents: []
+  target: {{ b64enc "reviews.default"}}.1
+  targetcomponents:
+  {{- contains .targetcomponents }}
+  - http
+  - tcp
+  {{- end }}
+  id: {{ b64enc "productpage.default"}}.1-{{ b64enc "reviews.default"}}.1
+  detectpoints:
+    - SERVER
+- source: {{ b64enc "reviews.default" }}.1
+  sourcecomponents: []
+  target: {{ b64enc "ratings.default"}}.1
+  targetcomponents:
+  {{- contains .targetcomponents }}
+    - http
+    - tcp
+  {{- end }}
+  id: {{ b64enc "reviews.default" }}.1-{{ b64enc "ratings.default"}}.1
+  detectpoints:
+    - SERVER
+- source: {{ b64enc "productpage.default" }}.1
+  sourcecomponents: []
+  target: {{ b64enc "details.default"}}.1
+  targetcomponents:
+  {{- contains .targetcomponents }}
+  - http
+  - tcp
+  {{- end }}
+  id: {{ b64enc "productpage.default" }}.1-{{ b64enc "details.default"}}.1
+  detectpoints:
+    - SERVER
+{{- end }}
diff --git a/test/e2e-v2/cases/cilium/expected/metrics-has-value.yml b/test/e2e-v2/cases/cilium/expected/metrics-has-value.yml
new file mode 100644
index 000000000000..dc71f566321a
--- /dev/null
+++ b/test/e2e-v2/cases/cilium/expected/metrics-has-value.yml
@@ -0,0 +1,31 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+type: TIME_SERIES_VALUES
+results:
+  {{- contains .results }}
+  - metric:
+      labels: []
+    values:
+      {{- contains .values }}
+      - id: {{ notEmpty .id }}
+        value: {{ .value }}
+        traceid: null
+      - id: {{ notEmpty .id }}
+        value: null
+        traceid: null
+      {{- end}}
+  {{- end}}
+error: null
diff --git a/test/e2e-v2/cases/cilium/expected/service-endpoint-productpage.yml b/test/e2e-v2/cases/cilium/expected/service-endpoint-productpage.yml
new file mode 100644
index 000000000000..f5788fe089aa
--- /dev/null
+++ b/test/e2e-v2/cases/cilium/expected/service-endpoint-productpage.yml
@@ -0,0 +1,19 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{{- contains . }}
+- id: {{ b64enc "productpage.default" }}.1_{{ b64enc "GET:/productpage" }}
+  name: GET:/productpage
+{{- end }}
diff --git a/test/e2e-v2/cases/cilium/expected/service-instance.yml b/test/e2e-v2/cases/cilium/expected/service-instance.yml
new file mode 100644
index 000000000000..658cdbd553b0
--- /dev/null
+++ b/test/e2e-v2/cases/cilium/expected/service-instance.yml
@@ -0,0 +1,22 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{{- contains . }}
+- id: {{ notEmpty .id }}
+  name: {{ notEmpty .name }}
+  attributes: []
+  language: UNKNOWN
+  instanceuuid: {{ notEmpty .instanceuuid }}
+{{- end }}
diff --git a/test/e2e-v2/cases/cilium/expected/service.yml b/test/e2e-v2/cases/cilium/expected/service.yml
new file mode 100644
index 000000000000..f077bf3b9387
--- /dev/null
+++ b/test/e2e-v2/cases/cilium/expected/service.yml
@@ -0,0 +1,45 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+{{- contains . }}
+- id: {{ b64enc "details.default" }}.1
+  name: details.default
+  group: ""
+  shortname: details.default
+  layers:
+    - CILIUM_SERVICE
+  normal: true
+- id: {{ b64enc "reviews.default" }}.1
+  name: reviews.default
+  group: ""
+  shortname: reviews.default
+  layers:
+    - CILIUM_SERVICE
+  normal: true
+- id: {{ b64enc "productpage.default" }}.1
+  name: productpage.default
+  group: ""
+  shortname: productpage.default
+  layers:
+    - CILIUM_SERVICE
+  normal: true
+- id: {{ b64enc "ratings.default" }}.1
+  name: ratings.default
+  group: ""
+  shortname: ratings.default
+  layers:
+    - CILIUM_SERVICE
+  normal: true
+{{- end }}
diff --git a/test/e2e-v2/cases/cilium/kind.yaml b/test/e2e-v2/cases/cilium/kind.yaml
new file mode 100644
index 000000000000..03add7f30222
--- /dev/null
+++ b/test/e2e-v2/cases/cilium/kind.yaml
@@ -0,0 +1,23 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+kind: Cluster
+apiVersion: kind.x-k8s.io/v1alpha4
+nodes:
+  - role: control-plane
+  - role: worker
+  - role: worker
+networking:
+  disableDefaultCNI: true
\ No newline at end of file
diff --git a/test/e2e-v2/cases/cilium/traffic-gen.yaml b/test/e2e-v2/cases/cilium/traffic-gen.yaml
new file mode 100644
index 000000000000..4e3b05021672
--- /dev/null
+++ b/test/e2e-v2/cases/cilium/traffic-gen.yaml
@@ -0,0 +1,40 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: trafficgenerator
+  labels:
+    app: trafficgenerator
+spec:
+  replicas: 1
+  selector:
+    matchLabels:
+      app: trafficgenerator
+  template:
+    metadata:
+      annotations:
+        sidecar.istio.io/inject: "false"
+      labels:
+        app: trafficgenerator
+    spec:
+      containers:
+        - name: trafficgenerator
+          image: elswork/wrk
+          command: ["wrk", "-t1", "-c1", "-d20m", "http://productpage:9080/productpage"]
+          resources:
+            requests:
+              cpu: 0.1
\ No newline at end of file
diff --git a/test/e2e-v2/cases/cilium/values.yaml b/test/e2e-v2/cases/cilium/values.yaml
new file mode 100644
index 000000000000..4a6d75d48aa4
--- /dev/null
+++ b/test/e2e-v2/cases/cilium/values.yaml
@@ -0,0 +1,40 @@
+# Licensed to the Apache Software Foundation (ASF) under one or more
+# contributor license agreements.  See the NOTICE file distributed with
+# this work for additional information regarding copyright ownership.
+# The ASF licenses this file to You under the Apache License, Version 2.0
+# (the "License"); you may not use this file except in compliance with
+# the License.  You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+oap:
+  secretMounts:
+    - name: cilium-tls
+      secretName: hubble-server-certs
+      path: /etc/config/cilium-tls
+  env:
+    SW_CILIUM_FETCHER_CONVERT_CLIENT_AS_SERVER_TRAFFIC: "true"
+    SW_CILIUM_FETCHER: default
+    SW_CILIUM_FETCHER_PEER_PORT: 443
+    SW_CILIUM_FETCHER_SSL_CONNECTION: "true"
+    SW_CILIUM_FETCHER_PRIVATE_KEY_FILE_PATH: /etc/config/cilium-tls/tls.key
+    SW_CILIUM_FETCHER_CERT_CHAIN_FILE_PATH: /etc/config/cilium-tls/tls.crt
+    SW_CILIUM_FETCHER_CA_FILE_PATH: /etc/config/cilium-tls/ca.crt
+
+elasticsearch:
+  esConfig:
+    elasticsearch.yml: |
+      cluster.routing.allocation.disk.threshold_enabled: false
+  volumeClaimTemplate:
+    accessModes: [ "ReadWriteOnce" ]
+    resources:
+      requests:
+        storage: 2Gi
+  persistence:
+    enabled: true
diff --git a/test/e2e-v2/script/env b/test/e2e-v2/script/env
index e906d727c79a..c0cbb4aeed7e 100644
--- a/test/e2e-v2/script/env
+++ b/test/e2e-v2/script/env
@@ -21,7 +21,7 @@ SW_AGENT_GO_COMMIT=774a6d56baba1187eb03bf1861af542c923b3dff
 SW_AGENT_PYTHON_COMMIT=c76a6ec51a478ac91abb20ec8f22a99b8d4d6a58
 SW_AGENT_CLIENT_JS_COMMIT=af0565a67d382b683c1dbd94c379b7080db61449
 SW_AGENT_CLIENT_JS_TEST_COMMIT=4f1eb1dcdbde3ec4a38534bf01dded4ab5d2f016
-SW_KUBERNETES_COMMIT_SHA=1741f5a0959b85faaf0a2fc6bbd7b550ec330280
+SW_KUBERNETES_COMMIT_SHA=1335f15bf821a40a7cd71448fa805f0be265afcc
 SW_ROVER_COMMIT=6bbd39aa701984482330d9dfb4dbaaff0527d55c
 SW_BANYANDB_COMMIT=e7210733022566cd0cee7ea2dc12cfb2f30fa8f3
 SW_AGENT_PHP_COMMIT=3192c553002707d344bd6774cfab5bc61f67a1d3
diff --git a/test/e2e-v2/script/prepare/setup-e2e-shell/install-cilium.sh b/test/e2e-v2/script/prepare/setup-e2e-shell/install-cilium.sh
new file mode 100644
index 000000000000..142008e84f8d
--- /dev/null
+++ b/test/e2e-v2/script/prepare/setup-e2e-shell/install-cilium.sh
@@ -0,0 +1,34 @@
+#!/usr/bin/env bash
+
+# ----------------------------------------------------------------------------
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements.  See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership.  The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License.  You may obtain a copy of the License at
+#
+#    http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied.  See the License for the
+# specific language governing permissions and limitations
+# under the License.
+# ----------------------------------------------------------------------------
+
+BASE_DIR=$1
+BIN_DIR=$2
+
+if ! command -v cilium &> /dev/null; then
+  mkdir -p $BASE_DIR/cilium && cd $BASE_DIR/cilium
+  CILIUM_CLI_VERSION="v0.16.11"
+  CLI_ARCH=amd64
+  if [ "$(uname -m)" = "aarch64" ]; then CLI_ARCH=arm64; fi
+  curl -L --fail --remote-name-all https://github.com/cilium/cilium-cli/releases/download/${CILIUM_CLI_VERSION}/cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
+  sha256sum --check cilium-linux-${CLI_ARCH}.tar.gz.sha256sum
+  sudo tar xzvfC cilium-linux-${CLI_ARCH}.tar.gz $BIN_DIR
+  rm cilium-linux-${CLI_ARCH}.tar.gz{,.sha256sum}
+fi
\ No newline at end of file