Repository: envoyproxy/gateway

Issue:
Allow removal of the `x-envoy-ratelimited` header
*Description*:

**Use Case:** I want to disable `x-envoy-ratelimited` header which is being set when a request is rate limited, as I do not need it and it doesn't really bring additional information about the ratelimit (correct me if that header is needed for something else).

**Problem:** 
1. I tried the approach of removing the header using a [HTTPRoute filter](https://gateway.envoyproxy.io/docs/tasks/traffic/http-response-headers/) but the header doesn't get removed, which I am not entirely sure why, as the documentation mentions, that in the default filter chain the header modification filter is executed after the ratelimit one (which supposedly sets the header). My other assumption is that the filter set in the `HTTPRoute` can't re-write the header, as this is not a response from a service that is being intercepted, but that response is generate by a call to the `ratelimit` service, which potentially bypasses the response modification filters? 

2. I found that the `fiter.ratelimit` does have an option to disable setting the header (linked the PR below), but that option is not exposed & it seems the only way to enable it is to use `EnvoyPatchPolicy` which is not something my organisation wants to use due to it's instability

3. There is the possibility to [disable all](https://gateway.envoyproxy.io/docs/api/extension_types/#headersettings) `x-envoy-*` headers, but I don't want to disable all of the headers just this one.

4. The `ratelimit` service does have the option to add custom headers, but the `x-envoy-ratelimited` seems to not be set in it.

**Solution:** 
- Add a configuration option that would allow disabling the `x-envoy-ratelimited` filter without using `EnvoyPatchPolicy` 


[optional *Relevant Links*:]
> [PR that added disabling the header in the filter ](https://github.com/envoyproxy/envoy/pull/13270)


Discussion:
@arkodg: should be possible with ClientTrafficPolicy https://gateway.envoyproxy.io/docs/api/extension_types/#headersettings@edheliel: @arkodg `ClientTrafficPolicy` allows for disabling **all** `x-envoy-*` headers, which is not what I want to do (as I pointed out in 3. in my original message). 

The functionality I am asking for is to be able to disable just the `x-envoy-ratelimited` header which is not an option in `ClientTrafficPolicy` at least to my understanding of the documentation.@arkodg: there's an option in CTP to disable RL headers - `disableRateLimitHeaders`
```

```@edheliel: 1. I have no permissions to open this issue (I am mentioning that as you asked me to reopen this issue on https://github.com/envoyproxy/gateway/issues/7064) 
2. @zirain has already mentioned `disableRateLimitHeaders` and I explained in the other issue why this is not the behaviour I want and it follows what I have already mentioned here, that I want to remove **only** `x-envoy-ratelimited` not all ratelimit headers, as some of them are actually needed and part of the standard. 

The documentation for `disableRateLimitHeaders` states:
> DisableRateLimitHeaders configures Envoy Proxy to omit the “X-RateLimit-” response headers
when rate limiting is enabled.

This implies that it would disable all ratelimit headers. 


So the questions is do you want to support adding a feature to disable the `x-envoy-ratelimited` header specifically , so the other ratelimiting related headers can be kept?


@arkodg: apologies from not reading the issue throughly @edheliel 

https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ratelimit/v3/rate_limit.proto#envoy-v3-api-field-extensions-filters-http-ratelimit-v3-ratelimit-disable-x-envoy-ratelimited-header

so it looks like for the default case `x-envoy-ratelimited` is being set, when it shouldnt, which imo is a bug

regarding only setting `x-envoy-ratelimited` instead of all the others  `x-envoy-*`, would like to better understand this use case, and see if others in the community are need this@edheliel: Thank you @arkodg ! Just to make sure we are on the same page, the behaviour that my organisation needs is to be able to disable the `x-envoy-ratelimited` header, without affecting any other headers.

Right now using the default configuration where `enableEnvoyHeaders` is set to `false`. We get the following response headers (once the ratelimit is reached i.e 429 response) 

```
x-envoy-ratelimited
x-ratelimit-limit
x-ratelimit-remaining
x-ratelimit-reset
```

what we want to achieve is to instead receive **only** the following headers:

```
x-ratelimit-limit
x-ratelimit-remaining
x-ratelimit-reset
```

Based on our previous discussion, it seems that there might also be a semantic issue as to where does the `x-envoy-ratelimited` header belongs, i.e is it in ratelimit headers or in `x-envoy` as it technically fits in both, but I would say that is rather an implementation decision that could be taken.

> https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ratelimit/v3/rate_limit.proto#envoy-v3-api-field-extensions-filters-http-ratelimit-v3-ratelimit-disable-x-envoy-ratelimited-header 

Would it be potentially an option to expose this flag/option in the CRD? 

Anyway, thank you for looking into that & let's see what other people from the community have to say. 
@arkodg: can we close this one @edheliel and create a new one for selectively deciding which `x-envoy-*` header to enable ? @edheliel: Sure @arkodg. Should I create the new issue ? Or would you rather do that ? @arkodg: Can you do that @edheliel and share the motivation for it@edheliel: @arkodg we updated to gateway 1.5.2, and it seems it solves our use-case, as `x-envoy-*` headers are disabled by default. That being said I don't have a use-case for selectively disabling headers, so maybe it is not needed. 

If you want I can still open a new issue, but I would rather leave it to someone, in the future that has an actual use-case for that, that is not solved by the fix you did in 1.5.2. 

Solution patch:
diff --git a/internal/xds/translator/ratelimit.go b/internal/xds/translator/ratelimit.go
index 9f8501c5d..be4570948 100644
--- a/internal/xds/translator/ratelimit.go
+++ b/internal/xds/translator/ratelimit.go
@@ -126,6 +126,12 @@ func createRateLimitFilter(t *Translator, irListener *ir.HTTPListener, domain, f
 		rateLimitFilterProto.Timeout = durationpb.New(t.GlobalRateLimit.Timeout)
 	}
 
+	// Disable the x-envoy-ratelimited header unless envoy headers are explicitly enabled.
+	rateLimitFilterProto.DisableXEnvoyRatelimitedHeader = true
+	if irListener.Headers != nil && irListener.Headers.EnableEnvoyHeaders {
+		rateLimitFilterProto.DisableXEnvoyRatelimitedHeader = false
+	}
+
 	// Configure the X-RateLimit headers based on the listener's header settings
 	if irListener.Headers != nil && irListener.Headers.DisableRateLimitHeaders {
 		rateLimitFilterProto.EnableXRatelimitHeaders = ratelimitfilterv3.RateLimit_OFF
diff --git a/release-notes/current.yaml b/release-notes/current.yaml
index 6e47191bf..1516d01d0 100644
--- a/release-notes/current.yaml
+++ b/release-notes/current.yaml
@@ -31,6 +31,7 @@ bug fixes: |
   Fixed validation for grpc routes with extension ref filters.
   Fixed service account token handling in GatewayNamespaceMode to use SDS for properly refreshing expired token.
   Fixed handling of regex meta characters in prefix match replace for URL rewrite.
+  Disabled the default emission of `x-envoy-ratelimited` headers from the rate limit filter; re-enable with the `enableEnvoyHeaders` setting in ClientTrafficPolicy.
 
 # Enhancements that improve performance.
 performance improvements: |



Test patch:
diff --git a/internal/cmd/egctl/testdata/translate/out/envoy-patch-policy.all.yaml b/internal/cmd/egctl/testdata/translate/out/envoy-patch-policy.all.yaml
index b7e57c2ca..ed3d14678 100644
--- a/internal/cmd/egctl/testdata/translate/out/envoy-patch-policy.all.yaml
+++ b/internal/cmd/egctl/testdata/translate/out/envoy-patch-policy.all.yaml
@@ -189,6 +189,7 @@ xds:
                     typedConfig:
                       '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
                       domain: eag-ratelimit
+                      disableXEnvoyRatelimitedHeader: true
                       failureModeDeny: true
                       rateLimitService:
                         grpcService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/jsonpatch-add-op-without-value.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/jsonpatch-add-op-without-value.listeners.yaml
index 4347e1ff7..4aee3acf7 100644
--- a/internal/xds/translator/testdata/out/xds-ir/jsonpatch-add-op-without-value.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/jsonpatch-add-op-without-value.listeners.yaml
@@ -19,6 +19,7 @@
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
             domain: eg-ratelimit
+            disableXEnvoyRatelimitedHeader: true
             failureModeDeny: true
             rateLimitService:
               grpcService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/jsonpatch-move-op-with-value.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/jsonpatch-move-op-with-value.listeners.yaml
index 4347e1ff7..4aee3acf7 100644
--- a/internal/xds/translator/testdata/out/xds-ir/jsonpatch-move-op-with-value.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/jsonpatch-move-op-with-value.listeners.yaml
@@ -19,6 +19,7 @@
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
             domain: eg-ratelimit
+            disableXEnvoyRatelimitedHeader: true
             failureModeDeny: true
             rateLimitService:
               grpcService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/jwt-ratelimit.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/jwt-ratelimit.listeners.yaml
index a4d81f0e1..800e15de9 100644
--- a/internal/xds/translator/testdata/out/xds-ir/jwt-ratelimit.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/jwt-ratelimit.listeners.yaml
@@ -40,6 +40,7 @@
         - name: envoy.filters.http.ratelimit
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: first-listener
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/ratelimit-custom-domain.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/ratelimit-custom-domain.listeners.yaml
index 5848912b9..68136c025 100644
--- a/internal/xds/translator/testdata/out/xds-ir/ratelimit-custom-domain.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/ratelimit-custom-domain.listeners.yaml
@@ -17,6 +17,7 @@
         - name: envoy.filters.http.ratelimit
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: first-listener
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/ratelimit-disable-headers.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/ratelimit-disable-headers.listeners.yaml
index d7dc5d129..af301069a 100644
--- a/internal/xds/translator/testdata/out/xds-ir/ratelimit-disable-headers.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/ratelimit-disable-headers.listeners.yaml
@@ -17,6 +17,7 @@
         - name: envoy.filters.http.ratelimit
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: first-listener
             rateLimitService:
               grpcService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/ratelimit-endpoint-stats.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/ratelimit-endpoint-stats.listeners.yaml
index 5848912b9..68136c025 100644
--- a/internal/xds/translator/testdata/out/xds-ir/ratelimit-endpoint-stats.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/ratelimit-endpoint-stats.listeners.yaml
@@ -17,6 +17,7 @@
         - name: envoy.filters.http.ratelimit
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: first-listener
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/ratelimit-global-shared.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/ratelimit-global-shared.listeners.yaml
index 8576ebeb0..417121f9f 100644
--- a/internal/xds/translator/testdata/out/xds-ir/ratelimit-global-shared.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/ratelimit-global-shared.listeners.yaml
@@ -17,6 +17,7 @@
         - name: envoy.filters.http.ratelimit
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: first-listener
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
@@ -27,6 +28,7 @@
         - name: envoy.filters.http.ratelimit/test-namespace/test-policy-1
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: test-namespace/test-policy-1
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
@@ -37,6 +39,7 @@
         - name: envoy.filters.http.ratelimit/test-namespace/test-policy-2
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: test-namespace/test-policy-2
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/ratelimit-headers-and-cidr.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/ratelimit-headers-and-cidr.listeners.yaml
index 5848912b9..68136c025 100644
--- a/internal/xds/translator/testdata/out/xds-ir/ratelimit-headers-and-cidr.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/ratelimit-headers-and-cidr.listeners.yaml
@@ -17,6 +17,7 @@
         - name: envoy.filters.http.ratelimit
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: first-listener
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/ratelimit-multi-global-shared.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/ratelimit-multi-global-shared.listeners.yaml
index 8576ebeb0..417121f9f 100644
--- a/internal/xds/translator/testdata/out/xds-ir/ratelimit-multi-global-shared.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/ratelimit-multi-global-shared.listeners.yaml
@@ -17,6 +17,7 @@
         - name: envoy.filters.http.ratelimit
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: first-listener
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
@@ -27,6 +28,7 @@
         - name: envoy.filters.http.ratelimit/test-namespace/test-policy-1
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: test-namespace/test-policy-1
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
@@ -37,6 +39,7 @@
         - name: envoy.filters.http.ratelimit/test-namespace/test-policy-2
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: test-namespace/test-policy-2
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/ratelimit-sourceip.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/ratelimit-sourceip.listeners.yaml
index 5848912b9..68136c025 100644
--- a/internal/xds/translator/testdata/out/xds-ir/ratelimit-sourceip.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/ratelimit-sourceip.listeners.yaml
@@ -17,6 +17,7 @@
         - name: envoy.filters.http.ratelimit
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: first-listener
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService:
diff --git a/internal/xds/translator/testdata/out/xds-ir/ratelimit.listeners.yaml b/internal/xds/translator/testdata/out/xds-ir/ratelimit.listeners.yaml
index 5848912b9..68136c025 100644
--- a/internal/xds/translator/testdata/out/xds-ir/ratelimit.listeners.yaml
+++ b/internal/xds/translator/testdata/out/xds-ir/ratelimit.listeners.yaml
@@ -17,6 +17,7 @@
         - name: envoy.filters.http.ratelimit
           typedConfig:
             '@type': type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
+            disableXEnvoyRatelimitedHeader: true
             domain: first-listener
             enableXRatelimitHeaders: DRAFT_VERSION_03
             rateLimitService: