Repository: vllm-project/aibrix
Pull request #967: Update envoy proxy and gateway-plugins config
Author: login: varungup90
Created: 2025-04-10T22:04:54Z | Merged: 2025-04-11T21:54:25Z

## Pull Request Description
Update envoy proxy and gateway-plugins config
- Ignore gpu node for their deployment
- Option to change number of replicas and image
- Restrict one replica per node
- Add pod disruption budget

## Related Issues
Resolves: #760 

**Important: Before submitting, please complete the description above and review the checklist below.**

---

<details>
<summary><strong>Contribution Guidelines (Expand for Details)</strong></summary>

<p>We appreciate your contribution to aibrix! To ensure a smooth review process and maintain high code quality, please adhere to the following guidelines:</p>

<h3>Pull Request Title Format</h3>
<p>Your PR title should start with one of these prefixes to indicate the nature of the change:</p>
<ul>
    <li><code>[Bug]</code>: Corrections to existing functionality</li>
    <li><code>[CI]</code>: Changes to build process or CI pipeline</li>
    <li><code>[Docs]</code>: Updates or additions to documentation</li>
    <li><code>[API]</code>: Modifications to aibrix's API or interface</li>
    <li><code>[CLI]</code>: Changes or additions to the Command Line Interface</li>
    <li><code>[Misc]</code>: For changes not covered above (use sparingly)</li>
</ul>
<p><em>Note: For changes spanning multiple categories, use multiple prefixes in order of importance.</em></p>

<h3>Submission Checklist</h3>
<ul>
    <li>[ ] PR title includes appropriate prefix(es)</li>
    <li>[ ] Changes are clearly explained in the PR description</li>
    <li>[ ] New and existing tests pass successfully</li>
    <li>[ ] Code adheres to project style and best practices</li>
    <li>[ ] Documentation updated to reflect changes (if applicable)</li>
    <li>[ ] Thorough testing completed, no regressions introduced</li>
</ul>

<p>By submitting this PR, you confirm that you've read these guidelines and your changes align with the project's contribution standards.</p>

</details>


Diff:
diff --git a/Makefile b/Makefile
index 43e8a7aa7..84db754e3 100644
--- a/Makefile
+++ b/Makefile
@@ -241,6 +241,10 @@ deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in
 undeploy: kustomize ## Undeploy controller from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.
 	$(KUSTOMIZE) build config/default | $(KUBECTL) delete --ignore-not-found=$(ignore-not-found) -f -
 
+.PHONY: deploy-release
+deploy-release: manifests kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config.
+	$(KUSTOMIZE) build config/overlays/release | $(KUBECTL) apply -f -
+
 .PHONY: install-vke
 install-vke: manifests kustomize ## Install CRDs into the K8s cluster specified in ~/.kube/config.
     ## helm creates objects without aibrix prefix, hence deploying gateway components outside of kustomization
@@ -260,11 +264,11 @@ undeploy-vke: kustomize ## Undeploy controller from the K8s cluster specified in
 
 .PHONY: deploy-vke-ipv4
 deploy-vke-ipv4: manifests kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config.
-	$(KUSTOMIZE) build config/overlays/vke-ipv4/default | $(KUBECTL) apply -f -
+	$(KUSTOMIZE) build config/overlays/vke-ipv4 | $(KUBECTL) apply -f -
 
 .PHONY: undeploy-vke-ipv4
 undeploy-vke-ipv4: kustomize ## Undeploy controller from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.
-	$(KUSTOMIZE) build config/overlays/vke-ipv4/default | $(KUBECTL) delete --ignore-not-found=$(ignore-not-found) -f -
+	$(KUSTOMIZE) build config/overlays/vke-ipv4 | $(KUBECTL) delete --ignore-not-found=$(ignore-not-found) -f -
 
 ##@ Dependencies
 
diff --git a/config/dependency/envoy-gateway/envoy_config_patch.yaml b/config/dependency/envoy-gateway/envoy_gateway_patch.yaml
similarity index 100%
rename from config/dependency/envoy-gateway/envoy_config_patch.yaml
rename to config/dependency/envoy-gateway/envoy_gateway_patch.yaml
diff --git a/config/dependency/envoy-gateway/kustomization.yaml b/config/dependency/envoy-gateway/kustomization.yaml
index dc0cfc090..227420b86 100644
--- a/config/dependency/envoy-gateway/kustomization.yaml
+++ b/config/dependency/envoy-gateway/kustomization.yaml
@@ -1,7 +1,12 @@
 kind: Kustomization
 
 resources:
-- https://github.com/envoyproxy/gateway/releases/download/v1.1.0/install.yaml
+- https://github.com/envoyproxy/gateway/releases/download/v1.2.8/install.yaml
 
 patches:
-- path: envoy_config_patch.yaml
\ No newline at end of file
+- path: envoy_gateway_patch.yaml
+
+images:
+- name: envoyproxy/gateway
+  newName: envoyproxy/gateway
+  newTag: v1.2.8
\ No newline at end of file
diff --git a/config/gateway/gateway-plugin/gateway-plugin.yaml b/config/gateway/gateway-plugin/gateway-plugin.yaml
index 2eddadb8e..b8ea44df9 100644
--- a/config/gateway/gateway-plugin/gateway-plugin.yaml
+++ b/config/gateway/gateway-plugin/gateway-plugin.yaml
@@ -22,6 +22,11 @@ metadata:
   name: gateway-plugins
   namespace: aibrix-system
 spec:
+  strategy:
+    type: RollingUpdate
+    rollingUpdate:
+      maxUnavailable: 1
+      maxSurge: 1
   replicas: 1
   selector:
     matchLabels:
@@ -31,6 +36,27 @@ spec:
       labels:
         app: gateway-plugins
     spec:
+      affinity:
+        podAntiAffinity: # pods are placed on different nodes
+           preferredDuringSchedulingIgnoredDuringExecution:
+            - weight: 100
+              podAffinityTerm:
+                labelSelector:
+                  matchExpressions:
+                    - key: app
+                      operator: In
+                      values:
+                        - gateway-plugins
+                topologyKey: "kubernetes.io/hostname"
+        nodeAffinity: # prevent gateway pod to be placed on gpu node.
+          preferredDuringSchedulingIgnoredDuringExecution:
+          - weight: 100
+            preference:
+              matchExpressions:
+                - key: nvidia.com/gpu.present
+                  operator: NotIn
+                  values:
+                    - "true"
       initContainers:
         - name: init-c
           image: busybox
@@ -88,6 +114,17 @@ spec:
             periodSeconds: 10
       serviceAccountName: aibrix-gateway-plugins
 ---
+apiVersion: policy/v1
+kind: PodDisruptionBudget
+metadata:
+  name: gateway-plugin-pdb
+  namespace: aibrix-system
+spec:
+  minAvailable: 1
+  selector:
+    matchLabels:
+      app: gateway-plugins
+---
 # this is a dummy route for incoming request to list models registered to aibrix-control-plane
 # TODO (varun): check if this dummy route can be removed in future
 apiVersion: gateway.networking.k8s.io/v1
diff --git a/config/gateway/gateway.yaml b/config/gateway/gateway.yaml
index 8414e73fc..293942724 100644
--- a/config/gateway/gateway.yaml
+++ b/config/gateway/gateway.yaml
@@ -33,7 +33,14 @@ spec:
     type: Kubernetes
     kubernetes:
       envoyDeployment:
+        replicas: 1
+        strategy:
+          type: RollingUpdate
+          rollingUpdate:
+            maxUnavailable: 1
+            maxSurge: 1
         container:
+          image: envoyproxy/envoy:v1.33.2
           resources:
             requests:
               cpu: 1
@@ -41,6 +48,30 @@ spec:
             limits:
               cpu: 1
               memory: 1Gi
+        pod:
+          affinity:
+            podAntiAffinity: # pods are placed on different nodes
+              preferredDuringSchedulingIgnoredDuringExecution:
+              - weight: 100
+                podAffinityTerm:
+                  labelSelector:
+                    matchExpressions:
+                      - key: app.kubernetes.io/name
+                        operator: In
+                        values:
+                          - envoy
+                  topologyKey: "kubernetes.io/hostname"
+            nodeAffinity: # prevent gateway pod to be placed on gpu node.
+              preferredDuringSchedulingIgnoredDuringExecution:
+              - weight: 100
+                preference:
+                  matchExpressions:
+                    - key: nvidia.com/gpu.present
+                      operator: NotIn
+                      values:
+                        - "true"
+      envoyPDB:
+        minAvailable: 1
 ---
 apiVersion: gateway.envoyproxy.io/v1alpha1
 kind: ClientTrafficPolicy
diff --git a/config/overlays/release/dependency_patch.yaml b/config/overlays/release/dependency_patch.yaml
deleted file mode 100644
index 155bdfc54..000000000
--- a/config/overlays/release/dependency_patch.yaml
+++ /dev/null
@@ -1,27 +0,0 @@
-# To customize EnvoyProxy
-# https://gateway.envoyproxy.io/docs/tasks/operations/customize-envoyproxy/
-apiVersion: gateway.envoyproxy.io/v1alpha1
-kind: EnvoyProxy
-metadata:
-  name: aibrix-custom-proxy-config
-  namespace: aibrix-system
-spec:
-  provider:
-    type: Kubernetes
-    kubernetes:
-      envoyDeployment:
-        patch:
-          type: StrategicMerge
-          value:
-            spec:
-              template:
-                spec:
-                  containers:
-                  - name: envoy
-                    resources:
-                      requests:
-                        cpu: 2
-                        memory: 8Gi
-                      limits:
-                        cpu: 2
-                        memory: 8Gi
\ No newline at end of file
diff --git a/config/overlays/release/envoy_proxy_patch.yaml b/config/overlays/release/envoy_proxy_patch.yaml
new file mode 100644
index 000000000..c57f64a50
--- /dev/null
+++ b/config/overlays/release/envoy_proxy_patch.yaml
@@ -0,0 +1,35 @@
+# To customize EnvoyProxy
+# https://gateway.envoyproxy.io/docs/tasks/operations/customize-envoyproxy/
+apiVersion: gateway.envoyproxy.io/v1alpha1
+kind: EnvoyProxy
+metadata:
+  name: custom-proxy-config
+  namespace: aibrix-system
+spec:
+  provider:
+    type: Kubernetes
+    kubernetes:
+      envoyDeployment:
+        replicas: 1
+        container:
+          image: envoyproxy/envoy:v1.33.2
+          resources:
+            requests:
+              cpu: 2
+              memory: 8Gi
+            limits:
+              cpu: 2
+              memory: 8Gi
+        pod:
+          affinity:
+            nodeAffinity: # prevent gateway pod to be placed on gpu node.
+              preferredDuringSchedulingIgnoredDuringExecution:
+              - weight: 100
+                preference:
+                  matchExpressions:
+                    - key: nvidia.com/gpu.present
+                      operator: NotIn
+                      values:
+                        - "true"
+      envoyPDB:
+        minAvailable: 1
\ No newline at end of file
diff --git a/config/overlays/release/gateway_plugins_patch.yaml b/config/overlays/release/gateway_plugins_patch.yaml
new file mode 100644
index 000000000..af51af504
--- /dev/null
+++ b/config/overlays/release/gateway_plugins_patch.yaml
@@ -0,0 +1,41 @@
+apiVersion: apps/v1
+kind: Deployment
+metadata:
+  name: gateway-plugins
+  namespace: aibrix-system
+spec:
+  replicas: 1
+  template:
+    spec:
+      affinity:
+        nodeAffinity: # prevent gateway pod to be placed on gpu node.
+          preferredDuringSchedulingIgnoredDuringExecution:
+          - weight: 100
+            preference:
+              matchExpressions:
+                - key: nvidia.com/gpu.present
+                  operator: NotIn
+                  values:
+                    - "true"
+      containers:
+        - name: gateway-plugin
+          resources:
+            limits:
+              cpu: 2
+              memory: 8Gi
+            requests:
+              cpu: 2
+              memory: 8Gi
+          env:
+            - name: AIBRIX_GPU_OPTIMIZER_TRACING_FLAG
+              value: "false"
+            - name: AIBRIX_PREFIX_CACHE_TOKENIZER_TYPE
+              value: "character"
+            - name: AIBRIX_PREFIX_CACHE_BLOCK_SIZE
+              value: "128"
+            - name: AIBRIX_PREFIX_CACHE_BLOCK_NUMBER
+              value: "200000"
+            - name: AIBRIX_PREFIX_CACHE_POD_RUNNING_REQUEST_IMBALANCE_ABS_COUNT
+              value: "16"
+            - name: AIBRIX_PREFIX_CACHE_STANDARD_DEVIATION_FACTOR
+              value: "2"
\ No newline at end of file
diff --git a/config/overlays/release/kustomization.yaml b/config/overlays/release/kustomization.yaml
index ad5a20026..0c4674a1e 100644
--- a/config/overlays/release/kustomization.yaml
+++ b/config/overlays/release/kustomization.yaml
@@ -7,15 +7,13 @@ resources:
 - ../../default
 
 patches:
-  - path: dependency_patch.yaml
-  - path: default_patch.yaml
+  - path: envoy_proxy_patch.yaml
+  - path: gateway_plugins_patch.yaml
 
 images:
 - name: quay.io/kuberay/operator
   newName: aibrix/kuberay-operator
   newTag: v1.2.1-patch
-- name: envoyproxy/envoy
-  newTag: distroless-v1.31.0
 - name: busybox
   newTag: stable
 - name: redis
diff --git a/config/overlays/vke-ipv4/default/ipv4_patch.yaml b/config/overlays/vke-ipv4/envoy_proxy_patch.yaml
similarity index 86%
rename from config/overlays/vke-ipv4/default/ipv4_patch.yaml
rename to config/overlays/vke-ipv4/envoy_proxy_patch.yaml
index bafc1d253..bda6d5eb1 100644
--- a/config/overlays/vke-ipv4/default/ipv4_patch.yaml
+++ b/config/overlays/vke-ipv4/envoy_proxy_patch.yaml
@@ -8,8 +8,9 @@ spec:
     kubernetes:
       envoyService:
         patch:
+          type: StrategicMerge
           value:
             spec:
               ipFamilies:
-              - IPv4    # Only IPv4
+                - IPv4
               ipFamilyPolicy: SingleStack  # Changed from PreferDualStack to SingleStack
\ No newline at end of file
diff --git a/config/overlays/vke-ipv4/default/kustomization.yaml b/config/overlays/vke-ipv4/kustomization.yaml
similarity index 81%
rename from config/overlays/vke-ipv4/default/kustomization.yaml
rename to config/overlays/vke-ipv4/kustomization.yaml
index 6017c49a9..ba03d7e8c 100644
--- a/config/overlays/vke-ipv4/default/kustomization.yaml
+++ b/config/overlays/vke-ipv4/kustomization.yaml
@@ -2,13 +2,10 @@ apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 
 resources:
-- ../../vke/default
+- ../vke/default
 
 patches:
-- path: ipv4_patch.yaml
-  target:
-    kind: EnvoyProxy
-    name: aibrix-custom-proxy-config
+- path: envoy_proxy_patch.yaml
 
 images:
 - name: aibrix-container-registry-cn-beijing.cr.volces.com/aibrix/gateway-plugins
diff --git a/config/overlays/vke/default/gateway/envoyproxy.yaml b/config/overlays/vke/default/envoy_proxy_patch.yaml
similarity index 55%
rename from config/overlays/vke/default/gateway/envoyproxy.yaml
rename to config/overlays/vke/default/envoy_proxy_patch.yaml
index 4c0bb4726..9145e200f 100644
--- a/config/overlays/vke/default/gateway/envoyproxy.yaml
+++ b/config/overlays/vke/default/envoy_proxy_patch.yaml
@@ -20,8 +20,26 @@ spec:
                 - IPv6
               ipFamilyPolicy: PreferDualStack
       envoyDeployment:
+        replicas: 1
+        container:
+          image: aibrix-container-registry-cn-beijing.cr.volces.com/envoyproxy/envoy:v1.33.2 
+          resources:
+            limits:
+              cpu: 2
+              memory: 8Gi
+            requests:
+              cpu: 2
+              memory: 8Gi
         pod:
           annotations:
             vci.vke.volcengine.com/pod-ip-family: dualstack
-        container:
-          image: aibrix-container-registry-cn-beijing.cr.volces.com/envoyproxy/envoy:distroless-v1.31.0
\ No newline at end of file
+          affinity:
+            nodeAffinity: # prevent gateway pod to be placed on gpu node.
+              preferredDuringSchedulingIgnoredDuringExecution:
+              - weight: 100
+                preference:
+                  matchExpressions:
+                    - key: vke.node.gpu.schedule
+                      operator: NotIn
+                      values:
+                        - nvidia
\ No newline at end of file
diff --git a/config/overlays/vke/default/gateway/gateway_patch.yaml b/config/overlays/vke/default/gateway/gateway_patch.yaml
deleted file mode 100644
index b4dc44b47..000000000
--- a/config/overlays/vke/default/gateway/gateway_patch.yaml
+++ /dev/null
@@ -1,12 +0,0 @@
-apiVersion: gateway.networking.k8s.io/v1
-kind: Gateway
-metadata:
-  name: eg
-  namespace: aibrix-system
-spec:
-  gatewayClassName: aibrix-eg
-  infrastructure:
-    parametersRef:
-      group: gateway.envoyproxy.io
-      kind: EnvoyProxy
-      name: aibrix-custom-proxy-config
\ No newline at end of file
diff --git a/config/overlays/vke/default/gateway/kustomization.yaml b/config/overlays/vke/default/gateway/kustomization.yaml
deleted file mode 100644
index 6e4df8c22..000000000
--- a/config/overlays/vke/default/gateway/kustomization.yaml
+++ /dev/null
@@ -1,3 +0,0 @@
-resources:
-  - ../../../../gateway
-  - envoyproxy.yaml
diff --git a/config/overlays/release/default_patch.yaml b/config/overlays/vke/default/gateway_plugins_patch.yaml
similarity index 71%
rename from config/overlays/release/default_patch.yaml
rename to config/overlays/vke/default/gateway_plugins_patch.yaml
index dc9e2a48d..433dccf84 100644
--- a/config/overlays/release/default_patch.yaml
+++ b/config/overlays/vke/default/gateway_plugins_patch.yaml
@@ -7,6 +7,16 @@ spec:
   replicas: 1
   template:
     spec:
+      affinity:
+        nodeAffinity: # prevent gateway pod to be placed on gpu node.
+          preferredDuringSchedulingIgnoredDuringExecution:
+          - weight: 100
+            preference:
+              matchExpressions:
+                - key: vke.node.gpu.schedule
+                  operator: NotIn
+                  values:
+                    - nvidia
       containers:
         - name: gateway-plugin
           resources:
diff --git a/config/overlays/vke/default/kustomization.yaml b/config/overlays/vke/default/kustomization.yaml
index 2d6980977..d27686f70 100644
--- a/config/overlays/vke/default/kustomization.yaml
+++ b/config/overlays/vke/default/kustomization.yaml
@@ -3,18 +3,15 @@ kind: Kustomization
 
 resources:
 - ../../../default
-- gateway/envoyproxy.yaml
 
 patches:
-  - path: gateway/gateway_patch.yaml
+  - path: envoy_proxy_patch.yaml
+  - path: gateway_plugins_patch.yaml
 
 images:
 - name: quay.io/kuberay/operator
   newName: aibrix-container-registry-cn-beijing.cr.volces.com/aibrix/kuberay-operator
   newTag: v1.2.1-patch
-- name: envoyproxy/envoy
-  newName: aibrix-container-registry-cn-beijing.cr.volces.com/envoyproxy/envoy
-  newTag: distroless-v1.31.0
 - name: busybox
   newName: aibrix-container-registry-cn-beijing.cr.volces.com/aibrix/busybox
   newTag: stable
diff --git a/config/overlays/vke/dependency/envoy-gateway/kustomization.yaml b/config/overlays/vke/dependency/envoy-gateway/kustomization.yaml
deleted file mode 100644
index f4b91610b..000000000
--- a/config/overlays/vke/dependency/envoy-gateway/kustomization.yaml
+++ /dev/null
@@ -1,10 +0,0 @@
-resources:
-  - ../../../../dependency
-
-patches:
-- path: envoy_config_patch.yaml
-
-images:
-- name: envoyproxy/gateway
-  newName: aibrix-container-registry-cn-beijing.cr.volces.com/envoyproxy/gateway
-  newTag: v1.1.0
\ No newline at end of file
diff --git a/config/overlays/vke/dependency/envoy-gateway/envoy_config_patch.yaml b/config/overlays/vke/dependency/envoy_gateway_patch.yaml
similarity index 94%
rename from config/overlays/vke/dependency/envoy-gateway/envoy_config_patch.yaml
rename to config/overlays/vke/dependency/envoy_gateway_patch.yaml
index d4011d23c..278a87a94 100644
--- a/config/overlays/vke/dependency/envoy-gateway/envoy_config_patch.yaml
+++ b/config/overlays/vke/dependency/envoy_gateway_patch.yaml
@@ -10,7 +10,7 @@ data:
     provider:
       kubernetes:
         shutdownManager:
-          image: aibrix-container-registry-cn-beijing.cr.volces.com/envoyproxy/gateway:v1.1.0
+          image: aibrix-container-registry-cn-beijing.cr.volces.com/envoyproxy/gateway:v1.2.8
       type: Kubernetes
     gateway:
       controllerName: gateway.envoyproxy.io/gatewayclass-controller
diff --git a/config/overlays/vke/dependency/kustomization.yaml b/config/overlays/vke/dependency/kustomization.yaml
index 55b42f011..8f4194c28 100644
--- a/config/overlays/vke/dependency/kustomization.yaml
+++ b/config/overlays/vke/dependency/kustomization.yaml
@@ -1,4 +1,13 @@
+apiVersion: kustomize.config.k8s.io/v1beta1
 kind: Kustomization
 
 resources:
-- envoy-gateway
+  - ../../../dependency
+
+patches:
+- path: envoy_gateway_patch.yaml
+
+images:
+- name: envoyproxy/gateway
+  newName: aibrix-container-registry-cn-beijing.cr.volces.com/envoyproxy/gateway
+  newTag: v1.2.8
\ No newline at end of file


Reviews:
user: login: Jeffwan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T16:22:16Z
user: login: Jeffwan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T16:22:55Z
user: login: Jeffwan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T16:25:25Z
user: login: Jeffwan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T16:33:22Z
user: login: varungup90
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T18:26:37Z
user: login: varungup90
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T18:27:14Z
user: login: varungup90
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T20:06:42Z
user: login: varungup90
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T20:10:02Z
user: login: varungup90
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T20:14:31Z
user: login: Jeffwan
state: COMMENTED
author_association: COLLABORATOR
submitted_at: 2025-04-11T20:55:46Z
user: login: Jeffwan
body: lgtm
state: APPROVED
author_association: COLLABORATOR
submitted_at: 2025-04-11T21:45:21Z

Review comments:
diff_hunk: @@ -241,6 +241,10 @@ deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in
 undeploy: kustomize ## Undeploy controller from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.
 	$(KUSTOMIZE) build config/default | $(KUBECTL) delete --ignore-not-found=$(ignore-not-found) -f -
 
+.PHONY: release
path: Makefile
user: login: Jeffwan
body: `release` itself is a little bit confusing. Please follow similar pattern like `install-xx` or `deploy-xxx` 
created_at: 2025-04-11T16:22:16Z
author_association: COLLABORATOR
diff_hunk: @@ -1,7 +1,12 @@
 kind: Kustomization
 
 resources:
-- https://github.com/envoyproxy/gateway/releases/download/v1.1.0/install.yaml
+- https://github.com/envoyproxy/gateway/releases/download/v1.2.8/install.yaml
 
 patches:
-- path: envoy_config_patch.yaml
\ No newline at end of file
+- path: envoy_gateway_patch.yaml
+
+images:
+- name: envoyproxy/gateway
+  newName: envoyproxy/gateway
+  newTag: v1.2.8
path: config/dependency/envoy-gateway/kustomization.yaml
user: login: Jeffwan
body: please copy latest images to VKE container registry store
created_at: 2025-04-11T16:22:55Z
author_association: COLLABORATOR
diff_hunk: @@ -31,6 +36,24 @@ spec:
       labels:
         app: gateway-plugins
     spec:
+      affinity:
+        podAntiAffinity: # pods are placed on different nodes
+           requiredDuringSchedulingIgnoredDuringExecution:
path: config/gateway/gateway-plugin/gateway-plugin.yaml
user: login: Jeffwan
body: if this is required, single node scenario can not serve 2 replicas. Prod env commonly has more than 1 node, but I am a little bit concern user raise such problem again.
- either update the docs to tell user you need >= 2 nodes in the cluster for HA purpose
- change to `preferred` 
created_at: 2025-04-11T16:25:25Z
author_association: COLLABORATOR
diff_hunk: @@ -31,6 +36,24 @@ spec:
       labels:
         app: gateway-plugins
     spec:
+      affinity:
+        podAntiAffinity: # pods are placed on different nodes
+           requiredDuringSchedulingIgnoredDuringExecution:
+           - labelSelector:
+               matchExpressions:
+                 - key: app
+                   operator: In
+                   values:
+                     - gateway-plugins
+             topologyKey: "kubernetes.io/hostname"
+        nodeAffinity: # prevent gateway pod to be placed on gpu node.
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+              - matchExpressions:
+                  - key: vke.node.gpu.schedule
path: config/gateway/gateway-plugin/gateway-plugin.yaml
user: login: Jeffwan
body: this is not common labels, should only exist in VKE.
created_at: 2025-04-11T16:25:58Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,43 @@
+# To customize EnvoyProxy
+# https://gateway.envoyproxy.io/docs/tasks/operations/customize-envoyproxy/
+apiVersion: gateway.envoyproxy.io/v1alpha1
+kind: EnvoyProxy
+metadata:
+  name: custom-proxy-config
+  namespace: aibrix-system
+spec:
+  provider:
+    type: Kubernetes
+    kubernetes:
+      envoyDeployment:
+        replicas: 1
path: config/overlays/release/envoy_proxy_patch.yaml
user: login: Jeffwan
body: ```
patch:
          type: StrategicMerge
```

is this deprecated in latest version?
created_at: 2025-04-11T16:30:13Z
author_association: COLLABORATOR
diff_hunk: @@ -241,6 +241,10 @@ deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in
 undeploy: kustomize ## Undeploy controller from the K8s cluster specified in ~/.kube/config. Call with ignore-not-found=true to ignore resource not found errors during deletion.
 	$(KUSTOMIZE) build config/default | $(KUBECTL) delete --ignore-not-found=$(ignore-not-found) -f -
 
+.PHONY: release
path: Makefile
user: login: varungup90
body: rename to deploy-release.
created_at: 2025-04-11T18:26:37Z
author_association: COLLABORATOR
diff_hunk: @@ -1,7 +1,12 @@
 kind: Kustomization
 
 resources:
-- https://github.com/envoyproxy/gateway/releases/download/v1.1.0/install.yaml
+- https://github.com/envoyproxy/gateway/releases/download/v1.2.8/install.yaml
 
 patches:
-- path: envoy_config_patch.yaml
\ No newline at end of file
+- path: envoy_gateway_patch.yaml
+
+images:
+- name: envoyproxy/gateway
+  newName: envoyproxy/gateway
+  newTag: v1.2.8
path: config/dependency/envoy-gateway/kustomization.yaml
user: login: varungup90
body: this is done. I tested with new images.
created_at: 2025-04-11T18:27:14Z
author_association: COLLABORATOR
diff_hunk: @@ -31,6 +36,24 @@ spec:
       labels:
         app: gateway-plugins
     spec:
+      affinity:
+        podAntiAffinity: # pods are placed on different nodes
+           requiredDuringSchedulingIgnoredDuringExecution:
path: config/gateway/gateway-plugin/gateway-plugin.yaml
user: login: varungup90
body: Changed to preferred.
created_at: 2025-04-11T20:06:42Z
author_association: COLLABORATOR
diff_hunk: @@ -31,6 +36,24 @@ spec:
       labels:
         app: gateway-plugins
     spec:
+      affinity:
+        podAntiAffinity: # pods are placed on different nodes
+           requiredDuringSchedulingIgnoredDuringExecution:
+           - labelSelector:
+               matchExpressions:
+                 - key: app
+                   operator: In
+                   values:
+                     - gateway-plugins
+             topologyKey: "kubernetes.io/hostname"
+        nodeAffinity: # prevent gateway pod to be placed on gpu node.
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+              - matchExpressions:
+                  - key: vke.node.gpu.schedule
path: config/gateway/gateway-plugin/gateway-plugin.yaml
user: login: varungup90
body: Yes. I have added here such that it does not get missed on using default deployment, and also provided same config in release overlay with a comment to change label value per their env.

Open to suggestion for naming.

created_at: 2025-04-11T20:10:02Z
author_association: COLLABORATOR
diff_hunk: @@ -0,0 +1,43 @@
+# To customize EnvoyProxy
+# https://gateway.envoyproxy.io/docs/tasks/operations/customize-envoyproxy/
+apiVersion: gateway.envoyproxy.io/v1alpha1
+kind: EnvoyProxy
+metadata:
+  name: custom-proxy-config
+  namespace: aibrix-system
+spec:
+  provider:
+    type: Kubernetes
+    kubernetes:
+      envoyDeployment:
+        replicas: 1
path: config/overlays/release/envoy_proxy_patch.yaml
user: login: varungup90
body: It is not deprecated. For our purpose of overriding replica or resources, we do not need strategic merge as simple direct constructs are [available](https://github.com/envoyproxy/gateway/blob/6a15dc769f647905498a0322f425e7eabef04e09/api/v1alpha1/shared_types.go#L63).
created_at: 2025-04-11T20:14:31Z
author_association: COLLABORATOR
diff_hunk: @@ -31,6 +36,24 @@ spec:
       labels:
         app: gateway-plugins
     spec:
+      affinity:
+        podAntiAffinity: # pods are placed on different nodes
+           requiredDuringSchedulingIgnoredDuringExecution:
+           - labelSelector:
+               matchExpressions:
+                 - key: app
+                   operator: In
+                   values:
+                     - gateway-plugins
+             topologyKey: "kubernetes.io/hostname"
+        nodeAffinity: # prevent gateway pod to be placed on gpu node.
+          requiredDuringSchedulingIgnoredDuringExecution:
+            nodeSelectorTerms:
+              - matchExpressions:
+                  - key: vke.node.gpu.schedule
path: config/gateway/gateway-plugin/gateway-plugin.yaml
user: login: Jeffwan
body: @varungup90 I think it's better for default ones to be directly used in public clouds, vke specific labels can be moved to our own overlay.  WDYT?
created_at: 2025-04-11T20:55:45Z
author_association: COLLABORATOR

PR comments:
user: login: Jeffwan
created_at: 2025-04-11T16:33:14Z
body: Seems there're lots of folder movement, that that easy to check the differences. What's the purpose? Are they just for flatten the folder structure? 
author_association: COLLABORATOR
user: login: varungup90
created_at: 2025-04-11T20:16:11Z
body: > Seems there're lots of folder movement, that that easy to check the differences. What's the purpose? Are they just for flatten the folder structure?

it mostly, flatten the structure or rename patch files. Primary changes are in config/default and config/dependency, other overlays for vke or release is copy-paste. I have test all overlays.
author_association: COLLABORATOR
user: login: Jeffwan
created_at: 2025-04-11T20:24:00Z
body: @varungup90 I merged another PR which introduces the conflicts. you need to rebase this branch now
author_association: COLLABORATOR