Repository: envoyproxy/gateway

Issue:
Context cancelled not treated as transient, causing unintended Envoy-Proxy recreation
*Description*:

We encountered an unexpected recreation of the Envoy-Proxy deployment, which caused traffic disruption. After investigating, we found that a network issue triggered Envoy-Gateway to recreate the Envoy-Proxy deployment.

The logs indicate that while processingGateways, the system received a `context cancelled` error. This error was not handled as a transient error, which caused the remaining logic to incorrectly assume that zero Gateways existed, and eventually led to the deletion of the Envoy-Proxy deployment.

Expected: These context errors (`deadline exceeded` or `context cancelled`) should be treated as transient, allowing retry.
Actual: They are treated as non-transient, which may cause unexpected failures—such as the issue we experienced and the one reported [here](https://github.com/envoyproxy/gateway/issues/6284).

*Repro steps*:

1. Send a context with a tight timeout or manually cancel the request context.
2. Observe that the context returns `deadline exceeded` or `context cancelled`.
3. The system doesn't classify this as a transient error and continuing with a corrupted truth.

*Environment*:

EG v1.4.2

*Logs*:

An error correctly marked as a transient error:
>2025-08-25T19:34:24.009Z	ERROR	provider	kubernetes/controller.go:295	transient error processing gateways	{"runner": "provider", "gatewayClass": "eg", "error": "failed to list <resource>: etcdserver: leader changed"}

Examples of errors not marked as transient:

>2025-08-25T14:47:35.097Z	ERROR	provider	kubernetes/controller.go:298	failed processGateways for gatewayClass eg, skipping it	{"runner": "provider", "error": "failed to list <resource>: client rate limiter Wait returned an error: context canceled"}
>2025-08-25T14:47:31.632Z	ERROR	provider	kubernetes/controller.go:298	failed processGateways for gatewayClass eg, skipping it	{"runner": "provider", "error": "failed to list <resource>: context canceled"}

Discussion:
@TomerJLevy: I'm suggesting this fix: https://github.com/envoyproxy/gateway/pull/6850

Solution patch:
diff --git a/internal/provider/kubernetes/controller.go b/internal/provider/kubernetes/controller.go
index dbf310280..051d56e2a 100644
--- a/internal/provider/kubernetes/controller.go
+++ b/internal/provider/kubernetes/controller.go
@@ -256,7 +256,9 @@ func isTransientError(err error) bool {
 		kerrors.IsServiceUnavailable(err) ||
 		kerrors.IsStoreReadError(err) ||
 		kerrors.IsInternalError(err) ||
-		kerrors.IsUnexpectedServerError(err)
+		kerrors.IsUnexpectedServerError(err) ||
+		errors.Is(err, context.Canceled) ||
+		errors.Is(err, context.DeadlineExceeded)
 }
 
 // Reconcile handles reconciling all resources in a single call. Any resource event should enqueue the
diff --git a/release-notes/current.yaml b/release-notes/current.yaml
index e1ee30b5f..e65d38c80 100644
--- a/release-notes/current.yaml
+++ b/release-notes/current.yaml
@@ -12,9 +12,10 @@ new features: |
 
 bug fixes: |
   Fixed %ROUTE_KIND% operator to be lower-cased when used by clusterStatName in EnvoyProxy API.
-  Fixed maxAcceptPerSocketEvent being ignored in ClientTrafficPolicy
+  Fixed maxAcceptPerSocketEvent being ignored in ClientTrafficPolicy.
   Fixed the topologyInjectorDisabled and the local cluster was not defined.
   Fixed log formatting of improper key-value pairs to avoid DPANIC in controller-runtime logger.
+  Fixed handling of context-related transient errors to prevent incorrect state reconciliation and unintended behavior.
 
 # Enhancements that improve performance.
 performance improvements: |



Test patch:
diff --git a/internal/provider/kubernetes/controller_test.go b/internal/provider/kubernetes/controller_test.go
index bb23b27be..a19b8a46f 100644
--- a/internal/provider/kubernetes/controller_test.go
+++ b/internal/provider/kubernetes/controller_test.go
@@ -1603,6 +1603,12 @@ func TestIsTransientError(t *testing.T) {
 	serviceUnavailableErr := kerrors.NewServiceUnavailable("service unavailable")
 	badRequestErr := kerrors.NewBadRequest("bad request")
 
+	// new test errors for context
+	canceledErr := context.Canceled
+	deadlineExceededErr := context.DeadlineExceeded
+	wrappedCanceledErr := fmt.Errorf("wrapped: %w", context.Canceled)
+	wrappedDeadlineExceededErr := fmt.Errorf("wrapped: %w", context.DeadlineExceeded)
+
 	testCases := []struct {
 		name     string
 		err      error
@@ -1614,6 +1620,10 @@ func TestIsTransientError(t *testing.T) {
 		{"ServiceUnavailable", serviceUnavailableErr, true},
 		{"BadRequest", badRequestErr, false},
 		{"NilError", nil, false},
+		{"ContextCanceled", canceledErr, true},
+		{"ContextDeadlineExceeded", deadlineExceededErr, true},
+		{"WrappedContextCanceled", wrappedCanceledErr, true},
+		{"WrappedContextDeadlineExceeded", wrappedDeadlineExceededErr, true},
 	}
 
 	for _, tc := range testCases {