Repository: knative-sandbox/net-kourier (go)

Issue:
Add support for pack_as_bytes configuration for ext_authz.v3
Hey there!
I've been forwarding an internal ArgoCD through a Knative Service (caddy proxy) to expose it to the internet.
I'm using Kourier along with External Authorization (using OPA)

Today I was trying to execute an ArgoCD CLI command: `argocd cluster add <context>` and it failed with a 403 error, with no logs on OPA. There was no rule defined to disallow this request btw.

After hours of searching through logs, I found that when the Argocd CLI tries to make a grpc web request, and this happens in Kourier:
```
[libprotobuf ERROR external/com_google_protobuf/src/google/protobuf/wire_format_lite.cc:577] String field 'envoy.service.auth.v3.AttributeContext.HttpRequest.body' contains invalid UTF-8 data when serializing a protocol buffer. Use the 'bytes' type if you intend to send raw bytes. 

[2022-04-26T16:34:13.257Z] "POST /cluster.ClusterService/Create HTTP/1.1" 403 UAEX 2239 0 1 - "-" "argocd-client/v2.3.3+07ac038.dirty grpc-go/1.15.0" "cf34b22f-2a40-427a-bdd6-a7c9987e92e3" "argocd.dv.kn.voiceintern.com:443" "-"
```
Basically Envoy is trying to convert every request body into a UTF-8 string. This will not be possible in all cases and will fail on [many edge cases](https://github.com/envoyproxy/envoy/issues/9822).

The good news is that the [`pack_as_bytes` parameter]() enables Envoy to send the request as raw bytes without converting into a UTF-8 string: https://github.com/envoyproxy/envoy/pull/13131

This has also been documented in more detail here: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ext_authz/v3/ext_authz.proto.html?highlight=pack_as_bytes#extensions-filters-http-ext-authz-v3-buffersettings

Proposal: Add a `KOURIER_EXTAUTHZ_PACKASBYTES` environment variable to allow passing the `pack_as_bytes` parameter to Envoy.




Pull request:
Add support for pack_as_bytes configuration for ext-authz
Supersedes #831. This #831 PR is opened for more than 1 year as of today, and making changes on it introduces a lot of delay, since I'm not the author and I need to synchronize with the original PR creator (I've made 3 PR there yet, and when CI tests are failing, I need to create another MR every time :disappointed:). I am currently running a fork of Kourier in production just for this change, so I'd like this to be merged upstream soon, and it can also profit other users. Thanks!

Credits also go to [sidharthramesh](https://github.com/sidharthramesh) who reported the issue and made the initial PR. I've added the tests (unit + integration), fix lint issues, doc, etc.

# Changes

:gift: Add support for pack_as_bytes configuration for ext-authz

When ext-authz uses grpc protocol, it's now possible to configure pack_as_bytes. This is necessary when the body sent to the external authorization service is raw bytes, and not an UTF-8 string.

By default, this value is set to `false`, so there is no breaking changes.

If users try to set `KOURIER_EXTAUTHZ_PACKASBYTES` to `true` if they are using the HTTP protocol for ext-authz, kourier will throw an error at startup. Indeed, pack as bytes option makes only sense when gRPC protocol is used.

/kind enhancement

Fixes #830

**Release Note**

```release-note
add support for pack_as_bytes configuration for ext-authz
```

Patch:
diff --git a/README.md b/README.md
index e9f7512b..615ec798 100644
--- a/README.md
+++ b/README.md
@@ -151,7 +151,11 @@ vars in the `net-kourier-controller` deployment:
 - `KOURIER_EXTAUTHZ_PATHPREFIX`: If `KOURIER_EXTAUTHZ_PROTOCOL` is equal to
   http or https, path to query the ext auth service. Example : if set to
   `/verify`, it will query `/verify/` (**notice the trailing `/`**).
-  If not set, it will query `/`.
+  If not set, it will query `/`
+- `KOURIER_EXTAUTHZ_PACKASBYTES`: If `KOURIER_EXTAUTHZ_PROTOCOL` is equal to
+  grpc, sends the body as raw bytes instead of a UTF-8 string. Accepts only true/false, t/f or 1/0.
+  Attempting to set another value will throw an error. Defaults to false. More info
+  [Envoy Docs](https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/ext_authz/v3/ext_authz.proto.html?highlight=pack_as_bytes#extensions-filters-http-ext-authz-v3-buffersettings).
 
 `*` Required
 
diff --git a/pkg/config/ext_authz.go b/pkg/config/ext_authz.go
index aad6e3ba..bdcd2ea6 100644
--- a/pkg/config/ext_authz.go
+++ b/pkg/config/ext_authz.go
@@ -17,6 +17,7 @@ limitations under the License.
 package config
 
 import (
+	"errors"
 	"fmt"
 	"net"
 	"os"
@@ -78,6 +79,7 @@ type config struct {
 	MaxRequestBytes  uint32           `default:"8192"`
 	Timeout          int              `default:"2000"`
 	Protocol         extAuthzProtocol `default:"grpc"`
+	PackAsBytes      bool             `default:"false"`
 	PathPrefix       string
 }
 
@@ -174,6 +176,8 @@ func extAuthzCluster(host string, port uint32, protocol extAuthzProtocol) *v3Clu
 	}
 }
 
+var errPackAsBytesInvalidWithProtocolHTTP = errors.New("pack as bytes option cannot be set when using http protocol")
+
 func externalAuthZFilter(conf *config) *hcm.HttpFilter {
 	timeout := durationpb.New(time.Duration(conf.Timeout) * time.Millisecond)
 
@@ -187,6 +191,12 @@ func externalAuthZFilter(conf *config) *hcm.HttpFilter {
 		ClearRouteCache: false,
 	}
 
+	if conf.Protocol != extAuthzProtocolGRPC && conf.PackAsBytes {
+		panic(errPackAsBytesInvalidWithProtocolHTTP)
+	}
+
+	extAuthConfig.WithRequestBody.PackAsBytes = conf.PackAsBytes
+
 	headers := []*core.HeaderValue{{
 		Key:   "client",
 		Value: "kourier",
diff --git a/pkg/generator/ingress_translator.go b/pkg/generator/ingress_translator.go
index c73f1fb6..405b9406 100644
--- a/pkg/generator/ingress_translator.go
+++ b/pkg/generator/ingress_translator.go
@@ -18,6 +18,7 @@ package generator
 
 import (
 	"context"
+	"crypto/tls"
 	"fmt"
 	"os"
 	"strings"
@@ -27,7 +28,7 @@ import (
 	envoycorev3 "github.com/envoyproxy/go-control-plane/envoy/config/core/v3"
 	endpoint "github.com/envoyproxy/go-control-plane/envoy/config/endpoint/v3"
 	route "github.com/envoyproxy/go-control-plane/envoy/config/route/v3"
-	tls "github.com/envoyproxy/go-control-plane/envoy/extensions/transport_sockets/tls/v3"
+	tlsv3 "github.com/envoyproxy/go-control-plane/envoy/extensions/transport_sockets/tls/v3"
 	envoymatcherv3 "github.com/envoyproxy/go-control-plane/envoy/type/matcher/v3"
 	"github.com/envoyproxy/go-control-plane/pkg/wellknown"
 	"google.golang.org/protobuf/types/known/anypb"
@@ -92,6 +93,16 @@ func (translator *IngressTranslator) translateIngress(ctx context.Context, ingre
 			return nil, fmt.Errorf("failed to fetch secret: %w", err)
 		}
 
+		// Validate certificate here as these are defined by users.
+		// We should not send Gateway without validation.
+		_, err = tls.X509KeyPair(
+			secret.Data[certFieldInSecret],
+			secret.Data[keyFieldInSecret],
+		)
+		if err != nil {
+			return nil, fmt.Errorf("invalid secret is specified: %w", err)
+		}
+
 		secretRef := types.NamespacedName{
 			Namespace: ingressTLS.SecretNamespace,
 			Name:      ingressTLS.SecretName,
@@ -329,22 +340,22 @@ func (translator *IngressTranslator) createUpstreamTransportSocket(http2 bool) (
 	}, nil
 }
 
-func createUpstreamTLSContext(caCertificate []byte, alpnProtocols ...string) *tls.UpstreamTlsContext {
-	return &tls.UpstreamTlsContext{
-		CommonTlsContext: &tls.CommonTlsContext{
+func createUpstreamTLSContext(caCertificate []byte, alpnProtocols ...string) *tlsv3.UpstreamTlsContext {
+	return &tlsv3.UpstreamTlsContext{
+		CommonTlsContext: &tlsv3.CommonTlsContext{
 			AlpnProtocols: alpnProtocols,
-			TlsParams: &tls.TlsParameters{
-				TlsMinimumProtocolVersion: tls.TlsParameters_TLSv1_2,
+			TlsParams: &tlsv3.TlsParameters{
+				TlsMinimumProtocolVersion: tlsv3.TlsParameters_TLSv1_2,
 			},
-			ValidationContextType: &tls.CommonTlsContext_ValidationContext{
-				ValidationContext: &tls.CertificateValidationContext{
+			ValidationContextType: &tlsv3.CommonTlsContext_ValidationContext{
+				ValidationContext: &tlsv3.CertificateValidationContext{
 					TrustedCa: &envoycorev3.DataSource{
 						Specifier: &envoycorev3.DataSource_InlineBytes{
 							InlineBytes: caCertificate,
 						},
 					},
-					MatchTypedSubjectAltNames: []*tls.SubjectAltNameMatcher{{
-						SanType: tls.SubjectAltNameMatcher_DNS,
+					MatchTypedSubjectAltNames: []*tlsv3.SubjectAltNameMatcher{{
+						SanType: tlsv3.SubjectAltNameMatcher_DNS,
 						Matcher: &envoymatcherv3.StringMatcher{
 							MatchPattern: &envoymatcherv3.StringMatcher_Exact{
 								Exact: certificates.FakeDnsName,


Test patch:
diff --git a/pkg/config/ext_authz_test.go b/pkg/config/ext_authz_test.go
index fa630216..56edc71c 100644
--- a/pkg/config/ext_authz_test.go
+++ b/pkg/config/ext_authz_test.go
@@ -17,6 +17,7 @@ limitations under the License.
 package config
 
 import (
+	"errors"
 	"reflect"
 	"testing"
 	"time"
@@ -127,6 +128,7 @@ func Test_externalAuthZFilter_extAuthz(t *testing.T) {
 		name           string
 		conf           *config
 		extAuthzWanted *extAuthService.ExtAuthz
+		panicErrWanted error
 	}{{
 		name: "grpc",
 		conf: &config{
@@ -156,6 +158,37 @@ func Test_externalAuthZFilter_extAuthz(t *testing.T) {
 				},
 			},
 		},
+	}, {
+		name: "grpc with pack as bytes enabled",
+		conf: &config{
+			Host:            "example.com:50051",
+			MaxRequestBytes: 8192,
+			Timeout:         2000,
+			Protocol:        "grpc",
+			PackAsBytes:     true,
+		},
+		extAuthzWanted: &extAuthService.ExtAuthz{
+			TransportApiVersion: core.ApiVersion_V3,
+			WithRequestBody: &extAuthService.BufferSettings{
+				MaxRequestBytes:     8192,
+				AllowPartialMessage: true,
+				PackAsBytes:         true,
+			},
+			Services: &extAuthService.ExtAuthz_GrpcService{
+				GrpcService: &core.GrpcService{
+					TargetSpecifier: &core.GrpcService_EnvoyGrpc_{
+						EnvoyGrpc: &core.GrpcService_EnvoyGrpc{
+							ClusterName: extAuthzClusterName,
+						},
+					},
+					Timeout: durationpb.New(time.Duration(2000) * time.Millisecond),
+					InitialMetadata: []*core.HeaderValue{{
+						Key:   "client",
+						Value: "kourier",
+					}},
+				},
+			},
+		},
 	}, {
 		name: "http",
 		conf: &config{
@@ -222,6 +255,16 @@ func Test_externalAuthZFilter_extAuthz(t *testing.T) {
 				},
 			},
 		},
+	}, {
+		name: "http with pack as bytes enabled",
+		conf: &config{
+			Host:            "example.com:8080",
+			MaxRequestBytes: 8192,
+			Timeout:         2000,
+			Protocol:        "http",
+			PackAsBytes:     true,
+		},
+		panicErrWanted: errPackAsBytesInvalidWithProtocolHTTP,
 	}, {
 		name: "https",
 		conf: &config{
@@ -291,6 +334,12 @@ func Test_externalAuthZFilter_extAuthz(t *testing.T) {
 	}}
 	for _, tt := range tests {
 		t.Run(tt.name, func(t *testing.T) {
+			defer func() {
+				if r := recover(); r != nil && !errors.Is(r.(error), tt.panicErrWanted) {
+					t.Errorf("externalAuthZFilter() extAuthz have panicked with \"%v\", want \"%v\"", r, tt.panicErrWanted)
+				}
+			}()
+
 			got := externalAuthZFilter(tt.conf)
 
 			extAuthzWantedAny, err := anypb.New(tt.extAuthzWanted)
diff --git a/pkg/generator/ingress_translator_test.go b/pkg/generator/ingress_translator_test.go
index 8a3d7d55..3360888c 100644
--- a/pkg/generator/ingress_translator_test.go
+++ b/pkg/generator/ingress_translator_test.go
@@ -18,6 +18,7 @@ package generator
 
 import (
 	"context"
+	"strings"
 	"testing"
 	"time"
 
@@ -49,10 +50,11 @@ import (
 
 func TestIngressTranslator(t *testing.T) {
 	tests := []struct {
-		name  string
-		in    *v1alpha1.Ingress
-		state []runtime.Object
-		want  *translatedIngress
+		name    string
+		in      *v1alpha1.Ingress
+		state   []runtime.Object
+		want    *translatedIngress
+		wantErr bool
 	}{{
 		name: "simple",
 		in:   ing("simplens", "simplename"),
@@ -349,6 +351,22 @@ func TestIngressTranslator(t *testing.T) {
 				internalVirtualHosts:    vHosts,
 			}
 		}(),
+	}, {
+		name: "invalid tls",
+		in: ing("testspace", "testname", func(ing *v1alpha1.Ingress) {
+			ing.Spec.TLS = []v1alpha1.IngressTLS{{
+				Hosts:           []string{"foo.example.com"},
+				SecretNamespace: "secretns",
+				SecretName:      "secretname",
+			}}
+		}),
+		state: []runtime.Object{
+			ns("testspace"),
+			svc("servicens", "servicename"),
+			eps("servicens", "servicename"),
+			invalidSecret,
+		},
+		wantErr: true,
 	}, {
 		name: "split",
 		in: ing("testspace", "testname", func(ing *v1alpha1.Ingress) {
@@ -659,7 +677,7 @@ func TestIngressTranslator(t *testing.T) {
 			)
 
 			got, err := translator.translateIngress(ctx, test.in, false)
-			assert.NilError(t, err)
+			assert.Equal(t, err != nil, test.wantErr)
 			assert.DeepEqual(t, got, test.want,
 				cmp.AllowUnexported(translatedIngress{}),
 				protocmp.Transform(),
@@ -1585,9 +1603,10 @@ var lbEndpointHTTP01Challenge = []*endpoint.LbEndpoint{
 }
 
 var (
-	cert       = []byte("cert")
-	privateKey = []byte("key")
-	secret     = &corev1.Secret{
+	cert        = []byte(rsaCertPEM)
+	invalidCert = []byte(invalidRsaCertPEM)
+	privateKey  = []byte(rsaKeyPEM)
+	secret      = &corev1.Secret{
 		ObjectMeta: metav1.ObjectMeta{
 			Namespace: "secretns",
 			Name:      "secretname",
@@ -1606,6 +1625,16 @@ var (
 			certificates.CaCertName: cert,
 		},
 	}
+	invalidSecret = &corev1.Secret{
+		ObjectMeta: metav1.ObjectMeta{
+			Namespace: "secretns",
+			Name:      "secretname",
+		},
+		Data: map[string][]byte{
+			"tls.crt": invalidCert,
+			"tls.key": privateKey,
+		},
+	}
 )
 
 func typedConfig(http2 bool) *envoycorev3.TransportSocket_TypedConfig {
@@ -1642,3 +1671,36 @@ func typedConfig(http2 bool) *envoycorev3.TransportSocket_TypedConfig {
 		TypedConfig: tlsAny,
 	}
 }
+
+var invalidRsaCertPEM = `-----BEGIN CERTIFICATE-----
+INVALID
+-----END CERTIFICATE-----
+`
+
+// Copied from https://go.dev/src/crypto/tls/tls_test.go
+var rsaCertPEM = `-----BEGIN CERTIFICATE-----
+MIIB0zCCAX2gAwIBAgIJAI/M7BYjwB+uMA0GCSqGSIb3DQEBBQUAMEUxCzAJBgNV
+BAYTAkFVMRMwEQYDVQQIDApTb21lLVN0YXRlMSEwHwYDVQQKDBhJbnRlcm5ldCBX
+aWRnaXRzIFB0eSBMdGQwHhcNMTIwOTEyMjE1MjAyWhcNMTUwOTEyMjE1MjAyWjBF
+MQswCQYDVQQGEwJBVTETMBEGA1UECAwKU29tZS1TdGF0ZTEhMB8GA1UECgwYSW50
+ZXJuZXQgV2lkZ2l0cyBQdHkgTHRkMFwwDQYJKoZIhvcNAQEBBQADSwAwSAJBANLJ
+hPHhITqQbPklG3ibCVxwGMRfp/v4XqhfdQHdcVfHap6NQ5Wok/4xIA+ui35/MmNa
+rtNuC+BdZ1tMuVCPFZcCAwEAAaNQME4wHQYDVR0OBBYEFJvKs8RfJaXTH08W+SGv
+zQyKn0H8MB8GA1UdIwQYMBaAFJvKs8RfJaXTH08W+SGvzQyKn0H8MAwGA1UdEwQF
+MAMBAf8wDQYJKoZIhvcNAQEFBQADQQBJlffJHybjDGxRMqaRmDhX0+6v02TUKZsW
+r5QuVbpQhH6u+0UgcW0jp9QwpxoPTLTWGXEWBBBurxFwiCBhkQ+V
+-----END CERTIFICATE-----
+`
+
+var rsaKeyPEM = testingKey(`-----BEGIN RSA TESTING KEY-----
+MIIBOwIBAAJBANLJhPHhITqQbPklG3ibCVxwGMRfp/v4XqhfdQHdcVfHap6NQ5Wo
+k/4xIA+ui35/MmNartNuC+BdZ1tMuVCPFZcCAwEAAQJAEJ2N+zsR0Xn8/Q6twa4G
+6OB1M1WO+k+ztnX/1SvNeWu8D6GImtupLTYgjZcHufykj09jiHmjHx8u8ZZB/o1N
+MQIhAPW+eyZo7ay3lMz1V01WVjNKK9QSn1MJlb06h/LuYv9FAiEA25WPedKgVyCW
+SmUwbPw8fnTcpqDWE3yTO3vKcebqMSsCIBF3UmVue8YU3jybC3NxuXq3wNm34R8T
+xVLHwDXh/6NJAiEAl2oHGGLz64BuAfjKrqwz7qMYr9HCLIe/YsoWq/olzScCIQDi
+D2lWusoe2/nEqfDVVWGWlyJ7yOmqaVm/iNUN9B2N2g==
+-----END RSA TESTING KEY-----
+`)
+
+func testingKey(s string) string { return strings.ReplaceAll(s, "TESTING KEY", "PRIVATE KEY") }
diff --git a/test/config/extauthz/grpc/src/main.go b/test/config/extauthz/grpc/src/main.go
index ac0443b4..ddfcaa66 100644
--- a/test/config/extauthz/grpc/src/main.go
+++ b/test/config/extauthz/grpc/src/main.go
@@ -36,7 +36,7 @@ type AuthV2 struct{}
 
 type AuthV3 struct{}
 
-func (ea AuthV3) Check(ctx context.Context, ar *authZ_v3.CheckRequest) (*authZ_v3.CheckResponse, error) {
+func (ea AuthV3) Check(_ context.Context, ar *authZ_v3.CheckRequest) (*authZ_v3.CheckResponse, error) {
 	if ar.Attributes.Request.Http.Path == "/success" || ar.Attributes.Request.Http.Path == "/healthz" {
 		log.Print("TRUE")
 		return &authZ_v3.CheckResponse{
@@ -56,7 +56,7 @@ func (ea AuthV3) Check(ctx context.Context, ar *authZ_v3.CheckRequest) (*authZ_v
 	}, nil
 }
 
-func (ea AuthV2) Check(ctx context.Context, ar *authZ_v2.CheckRequest) (*authZ_v2.CheckResponse, error) {
+func (ea AuthV2) Check(_ context.Context, ar *authZ_v2.CheckRequest) (*authZ_v2.CheckResponse, error) {
 	if ar.Attributes.Request.Http.Path == "/success" || ar.Attributes.Request.Http.Path == "/healthz" {
 		log.Print("TRUE")
 		return &authZ_v2.CheckResponse{
diff --git a/test/e2e-kind.sh b/test/e2e-kind.sh
index 21768025..60384fd5 100755
--- a/test/e2e-kind.sh
+++ b/test/e2e-kind.sh
@@ -37,7 +37,7 @@ if [[ $(kubectl get secret server-certs -n "${TEST_NAMESPACE}" -o name | wc -l)
   export "SERVER_NAME=data-plane.knative.dev"
 fi
 
-IPS=( $(kubectl get nodes -lkubernetes.io/hostname!=kind-control-plane -ojsonpath='{.items[*].status.addresses[?(@.type=="InternalIP")].address}') )
+IPS=($(kubectl get nodes -lkubernetes.io/hostname!=kind-control-plane -ojsonpath='{.items[*].status.addresses[?(@.type=="InternalIP")].address}'))
 
 export "GATEWAY_OVERRIDE=kourier"
 export "GATEWAY_NAMESPACE_OVERRIDE=${KOURIER_GATEWAY_NAMESPACE}"
@@ -67,7 +67,7 @@ echo ">> Running TLS Cipher suites"
 echo ">> Setup cipher suites"
 kubectl -n "${KOURIER_CONTROL_NAMESPACE}" patch configmap/config-kourier --type merge -p '{"data":{"cipher-suites":"ECDHE-ECDSA-AES128-GCM-SHA256,ECDHE-ECDSA-CHACHA20-POLY1305"}}'
 
-go test -v  -tags=e2e ./test/tls/... \
+go test -v -tags=e2e ./test/tls/... \
   --ingressendpoint="${IPS[0]}" \
   --ingressClass=kourier.ingress.networking.knative.dev \
   --cluster-suffix="$CLUSTER_SUFFIX"
@@ -85,6 +85,8 @@ go test -race -count=1 -timeout=20m -tags=e2e ./test/cert/... \
   --ingressClass=kourier.ingress.networking.knative.dev \
   --cluster-suffix="$CLUSTER_SUFFIX"
 
+export "KOURIER_EXTAUTHZ_PROTOCOL=grpc"
+
 echo ">> Setup ExtAuthz gRPC"
 ko apply -f test/config/extauthz/grpc
 kubectl -n "${KOURIER_CONTROL_NAMESPACE}" wait --timeout=300s --for=condition=Available deployment/externalauthz-grpc
@@ -92,6 +94,7 @@ kubectl -n "${KOURIER_CONTROL_NAMESPACE}" set env deployment net-kourier-control
 kubectl -n "${KOURIER_CONTROL_NAMESPACE}" rollout status deployment/net-kourier-controller --timeout=300s
 
 echo ">> Running ExtAuthz tests"
+KOURIER_GATEWAY_IMAGE="$(kubectl get deployment -n ${KOURIER_GATEWAY_NAMESPACE} 3scale-kourier-gateway -o jsonpath='{.spec.template.spec.containers[?(@.name=="kourier-gateway")].image}')" \
 go test -race -count=1 -timeout=20m -tags=e2e ./test/extauthz/... \
   --ingressendpoint="${IPS[0]}" \
   --ingressClass=kourier.ingress.networking.knative.dev \
@@ -101,6 +104,23 @@ echo ">> Unset ExtAuthz gRPC"
 kubectl -n "${KOURIER_CONTROL_NAMESPACE}" set env deployment net-kourier-controller KOURIER_EXTAUTHZ_HOST-
 kubectl -n "${KOURIER_CONTROL_NAMESPACE}" rollout status deployment/net-kourier-controller
 
+echo ">> Setup ExtAuthz gRPC with pack as bytes option"
+kubectl -n "${KOURIER_CONTROL_NAMESPACE}" set env deployment net-kourier-controller \
+  KOURIER_EXTAUTHZ_HOST=externalauthz-grpc.knative-serving:6000 \
+  KOURIER_EXTAUTHZ_PACKASBYTES=true
+
+kubectl -n "${KOURIER_CONTROL_NAMESPACE}" rollout status deployment/net-kourier-controller --timeout=300s
+
+echo ">> Running ExtAuthz tests"
+go test -race -count=1 -timeout=20m -tags=e2e ./test/extauthz/... \
+  --ingressendpoint="${IPS[0]}" \
+  --ingressClass=kourier.ingress.networking.knative.dev \
+  --cluster-suffix="$CLUSTER_SUFFIX"
+
+echo ">> Unset ExtAuthz gRPC"
+kubectl -n "${KOURIER_CONTROL_NAMESPACE}" set env deployment net-kourier-controller KOURIER_EXTAUTHZ_HOST- KOURIER_EXTAUTHZ_PACKASBYTES-
+kubectl -n "${KOURIER_CONTROL_NAMESPACE}" rollout status deployment/net-kourier-controller
+
 echo ">> Setup ExtAuthz HTTP"
 ko apply -f test/config/extauthz/http
 kubectl -n "${KOURIER_CONTROL_NAMESPACE}" wait --timeout=300s --for=condition=Available deployment/externalauthz-http
diff --git a/test/extauthz/extauthz_test.go b/test/extauthz/extauthz_test.go
index 2441e211..b19561a8 100644
--- a/test/extauthz/extauthz_test.go
+++ b/test/extauthz/extauthz_test.go
@@ -20,8 +20,10 @@ limitations under the License.
 package extauthz
 
 import (
+	"bytes"
 	"context"
 	"net/http"
+	"os"
 	"testing"
 
 	"gotest.tools/v3/assert"
@@ -79,4 +81,30 @@ func TestExtAuthz(t *testing.T) {
 	}
 	defer resp.Body.Close()
 	assert.Equal(t, resp.StatusCode, http.StatusOK)
+
+	// TODO: Drop this skip after maistra 2.4 starts using Envoy 1.22.10 or later.
+	if os.Getenv("KOURIER_GATEWAY_IMAGE") == "quay.io/maistra-dev/proxyv2-ubi8:2.4-latest" {
+		t.Skip("Skip POSTing with binary data when using maistra-dev/proxyv2-ubi8:2.4-latest image, " +
+			"see https://github.com/knative-sandbox/net-kourier/pull/1039#issuecomment-1537426109 for details. " +
+			"There is a bug in older Envoy versions, and will be solved when maistra starts using Envoy 1.22.10 or later, " +
+			"specifically this commit: https://github.com/envoyproxy/envoy/commit/5ced3041fc2da22fcf72fd621ce432aa26024caa")
+	}
+
+	postBody := make([]byte, 256)
+	for i := 0; i < 256; i++ {
+		postBody[i] = byte(i)
+	}
+
+	req, err = http.NewRequest("POST", "http://"+name+".example.com/success", bytes.NewReader(postBody))
+	if err != nil {
+		t.Fatal(err)
+	}
+
+	resp, err = client.Do(req)
+	if err != nil {
+		t.Fatal(err)
+	}
+	defer resp.Body.Close()
+
+	assert.Equal(t, resp.StatusCode, http.StatusOK)
 }