Repository: envoyproxy/gateway

Issue:
ExtAuth CircuitBreaker Thresholds Not Updated in Envoy Proxy After SecurityPolicy Update
*Description*:
>What issue is being seen? Describe what should be happening instead of
the bug, for example: The expected value isn't returned, etc.

We tried to update the `circuitBreaker` thresholds of the `ExtAuth` service (HTTP) through its `backendSettings`. The updated SecurityPolicy was successfully applied to Kubernetes, but the circuit breaker thresholds of the corresponding cluster config were not updated in the Envoy Proxy pods (verified via config_dump).

*Repro steps*:
> Include sample requests, environment, etc. All data and inputs
required to reproduce the bug.

Here's the SecurityPolicy that we used to set up an ExtAuth service.

```
apiVersion: gateway.envoyproxy.io/v1alpha1
kind: SecurityPolicy
metadata:
...
  name: fake-security-policy
spec:
  authorization:
    ...
  extAuth:
    failOpen: true
    headersToExtAuth: []
    http:
      backendRefs:
      - group: ""
        kind: Service
        name: fake-service
        port: fake-port
      backendSettings:
        circuitBreaker:
          maxConnections: 30000
          maxParallelRequests: 1024
          maxParallelRetries: 1024
          maxPendingRequests: 1024
      headersToBackend:
        ...
      path: /fake_path
  targetRefs:
    ...
```

It was applied two days ago. Here's the status of the security policy from the output of `kubectl describe securitypolicy fake-security-policy`

```
Status:
  Ancestors:
    Ancestor Ref:
      Group:      gateway.networking.k8s.io
      Kind:       Gateway
      Name:       envoy-gateway
      Namespace:  envoy-gateway
    Conditions:
      Last Transition Time:  2025-02-25T17:55:10Z
      Message:               Policy has been accepted.
      Reason:                Accepted
      Status:                True
      Type:                  Accepted
    Controller Name:         gateway.envoyproxy.io/gatewayclass-controller
```

Here is the corresponding cluster configuration dump from an Envoy Proxy pod:
```
  {
   "@type": "type.googleapis.com/envoy.admin.v3.ClustersConfigDump",
   "version_info": "63187",
   ...
   "dynamic_active_clusters": [
    ...
    {
     "version_info": "f91211d7deb02f8d0acb0632d832a4fdaa18ef9d0dc3b2b5fdaf0235116fa70a",
     "cluster": {
      "@type": "type.googleapis.com/envoy.config.cluster.v3.Cluster",
      "name": "securitypolicy/namespace/fake-security-policy/extauth/0",
      "type": "EDS",
      "eds_cluster_config": {
       "eds_config": {
        "ads": {},
        "resource_api_version": "V3"
       },
       "service_name": "securitypolicy/namespace/fake-security-policy/extauth/0"
      },
      "connect_timeout": "...",
      "per_connection_buffer_limit_bytes": ...,
      "lb_policy": "LEAST_REQUEST",
      "circuit_breakers": {
       "thresholds": [
        {
         "max_retries": 1024
        }
       ]
      },
      "dns_lookup_family": "V4_PREFERRED",
      "outlier_detection": {},
      "common_lb_config": {
       "locality_weighted_lb_config": {}
      },
      "ignore_health_on_host_removal": true
     },
     "last_updated": "2025-02-20T22:51:37.009Z"
    },
```

The `maxConnections` threshold we set in the SecurityPolicy is missing from the circuit breaker configuration. The `last_updated` field of the cluster config (2025-02-20T22:51:37.009Z) is older than the security policy application time (2025-02-25T17:55:10Z). It appears that Envoy Gateway somehow skipped applying the circuit breaker `backendSetting` from the security policy accordingly.

>**Note**: If there are privacy concerns, sanitize the data prior to
sharing.

*Environment*:
>Include the environment like gateway version, envoy version and so on.

v1.2.5


Discussion:

Solution patch:
diff --git a/.github/workflows/build_and_test.yaml b/.github/workflows/build_and_test.yaml
index f15efe463..497c6ff48 100644
--- a/.github/workflows/build_and_test.yaml
+++ b/.github/workflows/build_and_test.yaml
@@ -213,7 +213,7 @@ jobs:
       if: github.event_name == 'push'
       uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567  # v3.3.0
       with:
-        username: ${{ secrets.DOCKERHUB_USERNAME }}
+        username: ${{ vars.DOCKERHUB_USERNAME }}
         password: ${{ secrets.DOCKERHUB_PASSWORD }}
 
     - name: Setup Multiarch Environment
diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml
index dbdec72e6..5ca6f5160 100644
--- a/.github/workflows/release.yaml
+++ b/.github/workflows/release.yaml
@@ -63,7 +63,7 @@ jobs:
       - name: Login to DockerHub
         uses: docker/login-action@9780b0c442fbb1117ed29e0efdff1e18412f7567  # v3.3.0
         with:
-          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          username: ${{ vars.DOCKERHUB_USERNAME }}
           password: ${{ secrets.DOCKERHUB_PASSWORD }}
       - name: Retag and push existing gateway-dev image
         run: |
diff --git a/api/v1alpha1/backendtrafficpolicy_types.go b/api/v1alpha1/backendtrafficpolicy_types.go
index 79b0e5a54..d230e2a4c 100644
--- a/api/v1alpha1/backendtrafficpolicy_types.go
+++ b/api/v1alpha1/backendtrafficpolicy_types.go
@@ -15,13 +15,13 @@ const (
 	KindBackendTrafficPolicy = "BackendTrafficPolicy"
 )
 
+// BackendTrafficPolicy allows the user to configure the behavior of the connection
+// between the Envoy Proxy listener and the backend service.
+//
 // +kubebuilder:object:root=true
 // +kubebuilder:resource:categories=envoy-gateway,shortName=btp
 // +kubebuilder:subresource:status
 // +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp`
-
-// BackendTrafficPolicy allows the user to configure the behavior of the connection
-// between the Envoy Proxy listener and the backend service.
 type BackendTrafficPolicy struct {
 	metav1.TypeMeta   `json:",inline"`
 	metav1.ObjectMeta `json:"metadata,omitempty"`
@@ -33,16 +33,15 @@ type BackendTrafficPolicy struct {
 	Status gwapiv1a2.PolicyStatus `json:"status,omitempty"`
 }
 
-// +kubebuilder:validation:XValidation:rule="(has(self.targetRef) && !has(self.targetRefs)) || (!has(self.targetRef) && has(self.targetRefs)) || (has(self.targetSelectors) && self.targetSelectors.size() > 0) ", message="either targetRef or targetRefs must be used"
+// BackendTrafficPolicySpec defines the desired state of BackendTrafficPolicy.
 //
+// +kubebuilder:validation:XValidation:rule="(has(self.targetRef) && !has(self.targetRefs)) || (!has(self.targetRef) && has(self.targetRefs)) || (has(self.targetSelectors) && self.targetSelectors.size() > 0) ", message="either targetRef or targetRefs must be used"
 // +kubebuilder:validation:XValidation:rule="has(self.targetRef) ? self.targetRef.group == 'gateway.networking.k8s.io' : true ", message="this policy can only have a targetRef.group of gateway.networking.k8s.io"
 // +kubebuilder:validation:XValidation:rule="has(self.targetRef) ? self.targetRef.kind in ['Gateway', 'HTTPRoute', 'GRPCRoute', 'UDPRoute', 'TCPRoute', 'TLSRoute'] : true", message="this policy can only have a targetRef.kind of Gateway/HTTPRoute/GRPCRoute/TCPRoute/UDPRoute/TLSRoute"
 // +kubebuilder:validation:XValidation:rule="has(self.targetRef) ? !has(self.targetRef.sectionName) : true",message="this policy does not yet support the sectionName field"
 // +kubebuilder:validation:XValidation:rule="has(self.targetRefs) ? self.targetRefs.all(ref, ref.group == 'gateway.networking.k8s.io') : true ", message="this policy can only have a targetRefs[*].group of gateway.networking.k8s.io"
 // +kubebuilder:validation:XValidation:rule="has(self.targetRefs) ? self.targetRefs.all(ref, ref.kind in ['Gateway', 'HTTPRoute', 'GRPCRoute', 'UDPRoute', 'TCPRoute', 'TLSRoute']) : true ", message="this policy can only have a targetRefs[*].kind of Gateway/HTTPRoute/GRPCRoute/TCPRoute/UDPRoute/TLSRoute"
 // +kubebuilder:validation:XValidation:rule="has(self.targetRefs) ? self.targetRefs.all(ref, !has(ref.sectionName)) : true",message="this policy does not yet support the sectionName field"
-//
-// BackendTrafficPolicySpec defines the desired state of BackendTrafficPolicy.
 type BackendTrafficPolicySpec struct {
 	PolicyTargetReferences `json:",inline"`
 	ClusterSettings        `json:",inline"`
@@ -74,11 +73,28 @@ type BackendTrafficPolicySpec struct {
 	//
 	// +optional
 	ResponseOverride []*ResponseOverride `json:"responseOverride,omitempty"`
+	// HTTPUpgrade defines the configuration for HTTP protocol upgrades.
+	// If not specified, the default upgrade configuration(websocket) will be used.
+	//
+	// +optional
+	// +notImplementedHide
+	HTTPUpgrade []*ProtocolUpgradeConfig `json:"httpUpgrade,omitempty"`
 }
 
-// +kubebuilder:object:root=true
+type ProtocolUpgradeConfig struct {
+	// Type is the case-insensitive type of protocol upgrade.
+	// e.g. `websocket`, `CONNECT`, `spdy/3.1` etc.
+	//
+	// +kubebuilder:validation:Required
+	Type string `json:"type"`
+	// Disabled indicates whether the upgrade is disabled.
+	// +optional
+	Disabled *bool `json:"disabled"`
+}
 
 // BackendTrafficPolicyList contains a list of BackendTrafficPolicy resources.
+//
+// +kubebuilder:object:root=true
 type BackendTrafficPolicyList struct {
 	metav1.TypeMeta `json:",inline"`
 	metav1.ListMeta `json:"metadata,omitempty"`
diff --git a/api/v1alpha1/basic_auth_types.go b/api/v1alpha1/basic_auth_types.go
index f7bec2837..cc98c1e6f 100644
--- a/api/v1alpha1/basic_auth_types.go
+++ b/api/v1alpha1/basic_auth_types.go
@@ -26,4 +26,13 @@ type BasicAuth struct {
 	//
 	// Note: The secret must be in the same namespace as the SecurityPolicy.
 	Users gwapiv1.SecretObjectReference `json:"users"`
+
+	// This field specifies the header name to forward a successfully authenticated user to
+	// the backend. The header will be added to the request with the username as the value.
+	//
+	// If it is not specified, the username will not be forwarded.
+	//
+	// +optional
+	// +notImplementedHide
+	ForwardUsernameHeader *string `json:"forwardUsernameHeader,omitempty"`
 }
diff --git a/api/v1alpha1/zz_generated.deepcopy.go b/api/v1alpha1/zz_generated.deepcopy.go
index 5efdb1bde..bbb7c1145 100644
--- a/api/v1alpha1/zz_generated.deepcopy.go
+++ b/api/v1alpha1/zz_generated.deepcopy.go
@@ -626,6 +626,17 @@ func (in *BackendTrafficPolicySpec) DeepCopyInto(out *BackendTrafficPolicySpec)
 			}
 		}
 	}
+	if in.HTTPUpgrade != nil {
+		in, out := &in.HTTPUpgrade, &out.HTTPUpgrade
+		*out = make([]*ProtocolUpgradeConfig, len(*in))
+		for i := range *in {
+			if (*in)[i] != nil {
+				in, out := &(*in)[i], &(*out)[i]
+				*out = new(ProtocolUpgradeConfig)
+				(*in).DeepCopyInto(*out)
+			}
+		}
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BackendTrafficPolicySpec.
@@ -642,6 +653,11 @@ func (in *BackendTrafficPolicySpec) DeepCopy() *BackendTrafficPolicySpec {
 func (in *BasicAuth) DeepCopyInto(out *BasicAuth) {
 	*out = *in
 	in.Users.DeepCopyInto(&out.Users)
+	if in.ForwardUsernameHeader != nil {
+		in, out := &in.ForwardUsernameHeader, &out.ForwardUsernameHeader
+		*out = new(string)
+		**out = **in
+	}
 }
 
 // DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new BasicAuth.
@@ -4413,6 +4429,26 @@ func (in *ProcessingModeOptions) DeepCopy() *ProcessingModeOptions {
 	return out
 }
 
+// DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
+func (in *ProtocolUpgradeConfig) DeepCopyInto(out *ProtocolUpgradeConfig) {
+	*out = *in
+	if in.Disabled != nil {
+		in, out := &in.Disabled, &out.Disabled
+		*out = new(bool)
+		**out = **in
+	}
+}
+
+// DeepCopy is an autogenerated deepcopy function, copying the receiver, creating a new ProtocolUpgradeConfig.
+func (in *ProtocolUpgradeConfig) DeepCopy() *ProtocolUpgradeConfig {
+	if in == nil {
+		return nil
+	}
+	out := new(ProtocolUpgradeConfig)
+	in.DeepCopyInto(out)
+	return out
+}
+
 // DeepCopyInto is an autogenerated deepcopy function, copying the receiver, writing into out. in must be non-nil.
 func (in *ProxyAccessLog) DeepCopyInto(out *ProxyAccessLog) {
 	*out = *in
diff --git a/charts/gateway-helm/crds/generated/gateway.envoyproxy.io_backendtrafficpolicies.yaml b/charts/gateway-helm/crds/generated/gateway.envoyproxy.io_backendtrafficpolicies.yaml
index c135bead9..3b7415212 100644
--- a/charts/gateway-helm/crds/generated/gateway.envoyproxy.io_backendtrafficpolicies.yaml
+++ b/charts/gateway-helm/crds/generated/gateway.envoyproxy.io_backendtrafficpolicies.yaml
@@ -532,6 +532,24 @@ spec:
                       Default: TerminateConnection
                     type: string
                 type: object
+              httpUpgrade:
+                description: |-
+                  HTTPUpgrade defines the configuration for HTTP protocol upgrades.
+                  If not specified, the default upgrade configuration(websocket) will be used.
+                items:
+                  properties:
+                    disabled:
+                      description: Disabled indicates whether the upgrade is disabled.
+                      type: boolean
+                    type:
+                      description: |-
+                        Type is the case-insensitive type of protocol upgrade.
+                        e.g. `websocket`, `CONNECT`, `spdy/3.1` etc.
+                      type: string
+                  required:
+                  - type
+                  type: object
+                type: array
               loadBalancer:
                 description: |-
                   LoadBalancer policy to apply when routing traffic from the gateway to
diff --git a/charts/gateway-helm/crds/generated/gateway.envoyproxy.io_securitypolicies.yaml b/charts/gateway-helm/crds/generated/gateway.envoyproxy.io_securitypolicies.yaml
index a48bc49ec..6aa0d9dce 100644
--- a/charts/gateway-helm/crds/generated/gateway.envoyproxy.io_securitypolicies.yaml
+++ b/charts/gateway-helm/crds/generated/gateway.envoyproxy.io_securitypolicies.yaml
@@ -384,6 +384,13 @@ spec:
                 description: BasicAuth defines the configuration for the HTTP Basic
                   Authentication.
                 properties:
+                  forwardUsernameHeader:
+                    description: |-
+                      This field specifies the header name to forward a successfully authenticated user to
+                      the backend. The header will be added to the request with the username as the value.
+
+                      If it is not specified, the username will not be forwarded.
+                    type: string
                   users:
                     description: |-
                       The Kubernetes secret which contains the username-password pairs in
diff --git a/internal/gatewayapi/securitypolicy.go b/internal/gatewayapi/securitypolicy.go
index 9d3505318..0bbd78fd7 100644
--- a/internal/gatewayapi/securitypolicy.go
+++ b/internal/gatewayapi/securitypolicy.go
@@ -1152,6 +1152,7 @@ func (t *Translator) buildExtAuth(
 	switch {
 	case http != nil:
 		protocol = ir.HTTP
+		backendSettings = http.BackendSettings
 		switch {
 		case len(http.BackendRefs) > 0:
 			backendRefs = http.BackendCluster.BackendRefs
@@ -1167,6 +1168,7 @@ func (t *Translator) buildExtAuth(
 		}
 	case grpc != nil:
 		protocol = ir.GRPC
+		backendSettings = grpc.BackendSettings
 		switch {
 		case len(grpc.BackendCluster.BackendRefs) > 0:
 			backendRefs = grpc.BackendRefs
diff --git a/release-notes/current.yaml b/release-notes/current.yaml
index 155da1509..913c715c0 100644
--- a/release-notes/current.yaml
+++ b/release-notes/current.yaml
@@ -18,10 +18,10 @@ new features: |
   Added support for HorizontalPodAutoscaler to helm chart
 
 bug fixes: |
-
   Fix traffic splitting when filters are attached to the backendRef.
   Added support for Secret and ConfigMap parsing in Standalone mode.
   Bypass overload manager for stats and ready listeners
+  Fix translating backendSettings for extAuth
 
 # Enhancements that improve performance.
 performance improvements: |



Test patch:
diff --git a/internal/gatewayapi/testdata/backendtrafficpolicy-dns-lookup-family.out.yaml b/internal/gatewayapi/testdata/backendtrafficpolicy-dns-lookup-family.out.yaml
index 7e067b8ea..7951e9ce2 100644
--- a/internal/gatewayapi/testdata/backendtrafficpolicy-dns-lookup-family.out.yaml
+++ b/internal/gatewayapi/testdata/backendtrafficpolicy-dns-lookup-family.out.yaml
@@ -439,6 +439,10 @@ xdsIR:
                   weight: 1
               path: ""
             name: securitypolicy/envoy-gateway/policy-for-gateway-1
+            traffic:
+              dns:
+                dnsRefreshRate: 30s
+                lookupFamily: IPv4Preferred
         traffic:
           dns:
             dnsRefreshRate: 5s
@@ -480,6 +484,10 @@ xdsIR:
                   weight: 1
               path: ""
             name: securitypolicy/envoy-gateway/policy-for-gateway-1
+            traffic:
+              dns:
+                dnsRefreshRate: 30s
+                lookupFamily: IPv4Preferred
         traffic:
           dns:
             dnsRefreshRate: 5s
diff --git a/internal/gatewayapi/testdata/securitypolicy-with-extauth-backendref.in.yaml b/internal/gatewayapi/testdata/securitypolicy-with-extauth-backendref.in.yaml
index 1c24de65e..93a6223fa 100644
--- a/internal/gatewayapi/testdata/securitypolicy-with-extauth-backendref.in.yaml
+++ b/internal/gatewayapi/testdata/securitypolicy-with-extauth-backendref.in.yaml
@@ -169,3 +169,9 @@ securityPolicies:
           headersToBackend:
             - header1
             - header2
+          backendSettings:
+            circuitBreaker:
+              maxConnections: 30001
+              maxParallelRequests: 1022
+              maxParallelRetries: 1023
+              maxPendingRequests: 1024
diff --git a/internal/gatewayapi/testdata/securitypolicy-with-extauth-backendref.out.yaml b/internal/gatewayapi/testdata/securitypolicy-with-extauth-backendref.out.yaml
index 1b74752bd..5dd39981f 100644
--- a/internal/gatewayapi/testdata/securitypolicy-with-extauth-backendref.out.yaml
+++ b/internal/gatewayapi/testdata/securitypolicy-with-extauth-backendref.out.yaml
@@ -188,6 +188,12 @@ securityPolicies:
         - name: http-backend
           namespace: envoy-gateway
           port: 80
+        backendSettings:
+          circuitBreaker:
+            maxConnections: 30001
+            maxParallelRequests: 1022
+            maxParallelRetries: 1023
+            maxPendingRequests: 1024
         headersToBackend:
         - header1
         - header2
@@ -358,6 +364,12 @@ xdsIR:
               - header2
               path: /auth
             name: securitypolicy/default/policy-for-gateway-1
+            traffic:
+              circuitBreaker:
+                maxConnections: 30001
+                maxParallelRequests: 1022
+                maxParallelRetries: 1023
+                maxPendingRequests: 1024
     readyListener:
       address: 0.0.0.0
       ipFamily: IPv4
diff --git a/internal/xds/translator/testdata/in/xds-ir/ext-auth-backend.yaml b/internal/xds/translator/testdata/in/xds-ir/ext-auth-backend.yaml
index d532bfd8d..dfda65a47 100644
--- a/internal/xds/translator/testdata/in/xds-ir/ext-auth-backend.yaml
+++ b/internal/xds/translator/testdata/in/xds-ir/ext-auth-backend.yaml
@@ -111,6 +111,12 @@ http:
           extAuth:
             name: securitypolicy/default/policy-for-gateway-1
             failOpen: true
+            traffic:
+              circuitBreaker:
+                maxConnections: 30001
+                maxParallelRequests: 1022
+                maxParallelRetries: 1023
+                maxPendingRequests: 1024
             http:
               authority: primary.foo.com
               destination:
diff --git a/internal/xds/translator/testdata/out/xds-ir/ext-auth-backend.clusters.yaml b/internal/xds/translator/testdata/out/xds-ir/ext-auth-backend.clusters.yaml
index 3ff821f5e..8c4d92eea 100644
--- a/internal/xds/translator/testdata/out/xds-ir/ext-auth-backend.clusters.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/ext-auth-backend.clusters.yaml
@@ -84,7 +84,10 @@
           initialStreamWindowSize: 65536
 - circuitBreakers:
     thresholds:
-    - maxRetries: 1024
+    - maxConnections: 30001
+      maxPendingRequests: 1024
+      maxRequests: 1022
+      maxRetries: 1023
   commonLbConfig:
     localityWeightedLbConfig: {}
   connectTimeout: 10s
diff --git a/site/content/en/latest/api/extension_types.md b/site/content/en/latest/api/extension_types.md
index 3d309ad68..32c07223e 100644
--- a/site/content/en/latest/api/extension_types.md
+++ b/site/content/en/latest/api/extension_types.md
@@ -3170,6 +3170,21 @@ _Appears in:_
 | `attributes` | _string array_ |  false  |  | Defines which attributes are sent to the external processor. Envoy Gateway currently<br />supports only the following attribute prefixes: connection, source, destination,<br />request, response, upstream and xds.route.<br />https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes |
 
 
+#### ProtocolUpgradeConfig
+
+
+
+
+
+_Appears in:_
+- [BackendTrafficPolicySpec](#backendtrafficpolicyspec)
+
+| Field | Type | Required | Default | Description |
+| ---   | ---  | ---      | ---     | ---         |
+| `type` | _string_ |  true  |  | Type is the case-insensitive type of protocol upgrade.<br />e.g. `websocket`, `CONNECT`, `spdy/3.1` etc. |
+| `disabled` | _boolean_ |  false  |  | Disabled indicates whether the upgrade is disabled. |
+
+
 #### ProviderType
 
 _Underlying type:_ _string_