Repository: kubernetes/ingress-nginx
Pull request #7560 (2021-08-28 18:11:10)

put modsecurity e2e tests into their own packages
This will allow users not wanting modsecurity to easily remove it from base image and also disable corresponding e2e tests by editing `test/e2e/e2e.go`. Eventually we can even parametrize e2e command to not require editing `e2e.go`. If this is something reviewers approves then I'll apply the same approach to few other e2e tests (i.e all the opentracing related stuff) too where they are not crucial to core ingress-nginx and that some users might want to strip them out from the final image.

k8s-ci-robot: @ElvinEfendi: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the `triage/accepted` label and provide further guidance.

The `triage/accepted` label can be added by org members by writing `/triage accepted` in a comment.


<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes/ingress-nginx/pull/7560#" title="Author self-approved">ElvinEfendi</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes%2Fingress-nginx).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes/ingress-nginx/blob/main/OWNERS)~~ [ElvinEfendi]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->

Patch:
diff --git a/Makefile b/Makefile
index edd16e4cc..6bc6f8cdc 100644
--- a/Makefile
+++ b/Makefile
@@ -51,7 +51,7 @@ endif
 
 REGISTRY ?= gcr.io/k8s-staging-ingress-nginx
 
-BASE_IMAGE ?= k8s.gcr.io/ingress-nginx/nginx:v20210809-g98288bc3c@sha256:f9363669cf26514c9548c1fe4f8f4e2f58dfb76616bcd638a0ff7f0ec3457c17
+BASE_IMAGE ?= k8s.gcr.io/ingress-nginx/nginx:v20210904-gb7c973dce@sha256:fac972a7e43b18408ecb9e87da868df519428294e2e988c16be72479ee873c0e
 
 GOARCH=$(ARCH)
 
diff --git a/build/run-in-docker.sh b/build/run-in-docker.sh
index debbe9752..90382fd5c 100755
--- a/build/run-in-docker.sh
+++ b/build/run-in-docker.sh
@@ -37,7 +37,7 @@ function cleanup {
 }
 trap cleanup EXIT
 
-E2E_IMAGE=${E2E_IMAGE:-k8s.gcr.io/ingress-nginx/e2e-test-runner:v20210822-g5e5faa24d@sha256:55c568d9e35e15d94b3ab41fe549b8ee4cd910cc3e031ddcccd06256755c5d89}
+E2E_IMAGE=${E2E_IMAGE:-k8s.gcr.io/ingress-nginx/e2e-test-runner:v20210906-g7d577d976@sha256:cf7079b5c05b8b1b108b16752c6ff4ca312cf96700e91eef6088b9e0c4a7aff1}
 
 DOCKER_OPTS=${DOCKER_OPTS:-}
 DOCKER_IN_DOCKER_ENABLED=${DOCKER_IN_DOCKER_ENABLED:-}
diff --git a/charts/ingress-nginx/README.md b/charts/ingress-nginx/README.md
index 22e66fb85..fecbbcde2 100644
--- a/charts/ingress-nginx/README.md
+++ b/charts/ingress-nginx/README.md
@@ -84,7 +84,8 @@ else it would make it impossible to evacuate a node. See [gh issue #7127](https:
 
 The Nginx ingress controller can export Prometheus metrics, by setting `controller.metrics.enabled` to `true`.
 
-You can add Prometheus annotations to the metrics service using `controller.metrics.service.annotations`. Alternatively, if you use the Prometheus Operator, you can enable ServiceMonitor creation using `controller.metrics.serviceMonitor.enabled`.
+You can add Prometheus annotations to the metrics service using `controller.metrics.service.annotations`.
+Alternatively, if you use the Prometheus Operator, you can enable ServiceMonitor creation using `controller.metrics.serviceMonitor.enabled`. And set `controller.metrics.serviceMonitor.additionalLabels.release="prometheus"`. "release=prometheus" should match the label configured in the prometheus servicemonitor ( see `kubectl get servicemonitor prometheus-kube-prom-prometheus -oyaml -n prometheus`)
 
 ### ingress-nginx nginx\_status page/stats server
 
diff --git a/charts/ingress-nginx/templates/controller-service-internal.yaml b/charts/ingress-nginx/templates/controller-service-internal.yaml
index 09c942eb2..599449836 100644
--- a/charts/ingress-nginx/templates/controller-service-internal.yaml
+++ b/charts/ingress-nginx/templates/controller-service-internal.yaml
@@ -51,6 +51,28 @@ spec:
       nodePort: {{ .Values.controller.service.nodePorts.https }}
     {{- end }}
   {{- end }}
+  {{- range $key, $value := .Values.tcp }}
+    - name: {{ $key }}-tcp
+      port: {{ $key }}
+      protocol: TCP
+      targetPort: {{ $key }}-tcp
+    {{- if $.Values.controller.service.nodePorts.tcp }}
+    {{- if index $.Values.controller.service.nodePorts.tcp $key }}
+      nodePort: {{ index $.Values.controller.service.nodePorts.tcp $key }}
+    {{- end }}
+    {{- end }}
+  {{- end }}
+  {{- range $key, $value := .Values.udp }}
+    - name: {{ $key }}-udp
+      port: {{ $key }}
+      protocol: UDP
+      targetPort: {{ $key }}-udp
+    {{- if $.Values.controller.service.nodePorts.udp }}
+    {{- if index $.Values.controller.service.nodePorts.udp $key }}
+      nodePort: {{ index $.Values.controller.service.nodePorts.udp $key }}
+    {{- end }}
+    {{- end }}
+  {{- end }}
   selector:
     {{- include "ingress-nginx.selectorLabels" . | nindent 4 }}
     app.kubernetes.io/component: controller
diff --git a/cloudbuild.yaml b/cloudbuild.yaml
index 8dead6df3..22f4771bb 100644
--- a/cloudbuild.yaml
+++ b/cloudbuild.yaml
@@ -4,7 +4,7 @@ timeout: 1800s
 options:
   substitution_option: ALLOW_LOOSE
 steps:
-  - name: 'gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f'
+  - name: 'gcr.io/k8s-testimages/gcb-docker-gcloud:v20210722-085d930'
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/deploy/static/provider/aws/deploy-tls-termination.yaml b/deploy/static/provider/aws/deploy-tls-termination.yaml
index bf442df89..3085270d8 100644
--- a/deploy/static/provider/aws/deploy-tls-termination.yaml
+++ b/deploy/static/provider/aws/deploy-tls-termination.yaml
@@ -37,7 +37,7 @@ metadata:
   name: ingress-nginx-controller
   namespace: ingress-nginx
 data:
-  http-snippet:|
+  http-snippet: |
     server{
       listen 2443;
       return 308 https://$host$request_uri;
diff --git a/deploy/static/provider/kind/deploy.yaml b/deploy/static/provider/kind/deploy.yaml
index 88c512162..4bd7e42c3 100644
--- a/deploy/static/provider/kind/deploy.yaml
+++ b/deploy/static/provider/kind/deploy.yaml
@@ -339,6 +339,7 @@ spec:
             - --validating-webhook-certificate=/usr/local/certificates/cert
             - --validating-webhook-key=/usr/local/certificates/key
             - --publish-status-address=localhost
+            - --watch-ingress-without-class
           securityContext:
             capabilities:
               drop:
diff --git a/docs/examples/affinity/cookie/README.md b/docs/examples/affinity/cookie/README.md
index 4f93fc0f8..b567c44c0 100644
--- a/docs/examples/affinity/cookie/README.md
+++ b/docs/examples/affinity/cookie/README.md
@@ -12,6 +12,7 @@ Session affinity can be configured using the following annotations:
 |nginx.ingress.kubernetes.io/affinity-mode|The affinity mode defines how sticky a session is. Use `balanced` to redistribute some sessions when scaling pods or `persistent` for maximum stickiness.|`balanced` (default) or `persistent`|
 |nginx.ingress.kubernetes.io/affinity-canary-behavior|Defines session affinity behavior of canaries. By default the behavior is `sticky`, and canaries respect session affinity configuration. Set this to `legacy` to restore original canary behavior, when session affinity parameters were not respected.|`sticky` (default) or `legacy`|
 |nginx.ingress.kubernetes.io/session-cookie-name|Name of the cookie that will be created|string (defaults to `INGRESSCOOKIE`)|
+|nginx.ingress.kubernetes.io/session-cookie-secure|Set the cookie as secure regardless the protocol of the incoming request|`"true"` or `"false"`|
 |nginx.ingress.kubernetes.io/session-cookie-path|Path that will be set on the cookie (required if your [Ingress paths][ingress-paths] use regular expressions)|string (defaults to the currently [matched path][ingress-paths])|
 |nginx.ingress.kubernetes.io/session-cookie-samesite|SameSite attribute to apply to the cookie|Browser accepted values are `None`, `Lax`, and `Strict`|
 |nginx.ingress.kubernetes.io/session-cookie-conditional-samesite-none|Will omit `SameSite=None` attribute for older browsers which reject the more-recently defined `SameSite=None` value|`"true"` or `"false"`
diff --git a/docs/examples/affinity/cookie/ingress-samesite.yaml b/docs/examples/affinity/cookie/ingress-samesite.yaml
index b3f8f4b20..edfa1b653 100644
--- a/docs/examples/affinity/cookie/ingress-samesite.yaml
+++ b/docs/examples/affinity/cookie/ingress-samesite.yaml
@@ -5,6 +5,7 @@ metadata:
   annotations:
     nginx.ingress.kubernetes.io/affinity: "cookie"
     nginx.ingress.kubernetes.io/session-cookie-name: "SSNONE"
+    nginx.ingress.kubernetes.io/session-cookie-secure: "true"
     nginx.ingress.kubernetes.io/session-cookie-expires: "172800"
     nginx.ingress.kubernetes.io/session-cookie-max-age: "172800"
     nginx.ingress.kubernetes.io/session-cookie-samesite: "None"
diff --git a/docs/index.md b/docs/index.md
index 3012ffbea..8eb626101 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -104,7 +104,7 @@ DESCRIPTION:
 
 - If you have only one instance of the Ingress-NGINX controller running in your cluster, and you still want to use ingressclass, you should add the annotation "ingressclass.kubernetes.io/is-default-class" in your ingress class, so any new Ingress objects will have this one as default ingressClass.
 
-In this case, you need to make your Controller aware of the objects. If you have several Ingress objects and they don't yet have the [ingressClassName](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#ingress-v1-networking-k8s-io) field, or the ingress annotation (`kubernetes.io/ingress.class`),  then you should start your ingress-controller with the flag [--watch-ingress-without-class=true](## What is the flag '--watch-without-ingress-class' ?) .
+In this case, you need to make your Controller aware of the objects. If you have several Ingress objects and they don't yet have the [ingressClassName](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#ingress-v1-networking-k8s-io) field, or the ingress annotation (`kubernetes.io/ingress.class`),  then you should start your ingress-controller with the flag [--watch-ingress-without-class=true](## What is the flag '--watch-ingress-without-class' ?) .
 
 You can configure your helm chart installation's values file with `.controller.watchIngressWithoutClass: true`. 
 
@@ -126,7 +126,7 @@ And add the value "spec.ingressClassName=nginx" in your Ingress objects
 ## I have multiple ingress objects in my cluster. What should I do ?
 - If you don't care about ingressClass, or you have a lot of ingress objects without ingressClass configuration, you can run the ingress-controller with the flag `--watch-ingress-without-class=true`.
 
-## What is the flag '--watch-without-ingress-class' ?
+## What is the flag '--watch-ingress-without-class' ?
 - Its a flag that is passed,as an argument, to the ingress-controller executable, in the pod spec. It looks like this ;
 ```
 ...
@@ -165,5 +165,5 @@ Bear in mind that, if your `Ingress-Nginx-Controller-nginx2` is started with the
 - and also objects pointing to the ingressClass that have the same .spec.controller as configured in `--controller-class`
 
 
-## Why is the ingressClassResource disabled by default in the helm chart ?
-- If this field is set to enabled value and there are existing ingress objects in the cluster, then the existing ingress objects will not be honored and only any new ingress object you create will inherit the ingressClass value.
+## I am seeing this error message in the logs of the Ingress-NGINX controller "ingress class annotation is not equal to the expected by Ingress Controller". Why ?
+- It is highly likely that you will also see the name of the ingress resource in the same error message. This error messsage has been observed on use the deprecated annotation, to spec the ingressClass, in a ingress resource manifest. It is recommended to use the ingress.spec.ingressClassName field, of the ingress resource, to spec the name of the ingressClass of the ingress resource being configured.
diff --git a/docs/user-guide/nginx-configuration/annotations.md b/docs/user-guide/nginx-configuration/annotations.md
index 67bc10c58..1fb883596 100755
--- a/docs/user-guide/nginx-configuration/annotations.md
+++ b/docs/user-guide/nginx-configuration/annotations.md
@@ -64,6 +64,7 @@ You can add these Kubernetes annotations to specific Ingress objects to customiz
 |[nginx.ingress.kubernetes.io/permanent-redirect](#permanent-redirect)|string|
 |[nginx.ingress.kubernetes.io/permanent-redirect-code](#permanent-redirect-code)|number|
 |[nginx.ingress.kubernetes.io/temporal-redirect](#temporal-redirect)|string|
+|[nginx.ingress.kubernetes.io/preserve-trailing-slash](#server-side-https-enforcement-through-redirect)|"true" or "false"|
 |[nginx.ingress.kubernetes.io/proxy-body-size](#custom-max-body-size)|string|
 |[nginx.ingress.kubernetes.io/proxy-cookie-domain](#proxy-cookie-domain)|string|
 |[nginx.ingress.kubernetes.io/proxy-cookie-path](#proxy-cookie-path)|string|
@@ -243,23 +244,18 @@ It is possible to enable Client Certificate Authentication using additional anno
 
 Client Certificate Authentication is applied per host and it is not possible to specify rules that differ for individual paths.
 
-The annotations are:
+To enable, add the annotation `nginx.ingress.kubernetes.io/auth-tls-secret: namespace/secretName`. This secret must have a file named `ca.crt` containing the full Certificate Authority chain `ca.crt` that is enabled to authenticate against this Ingress.
 
-* `nginx.ingress.kubernetes.io/auth-tls-secret: secretName`:
-  The name of the Secret that contains the full Certificate Authority chain `ca.crt` that is enabled to authenticate against this Ingress.
-  This annotation expects the Secret name in the form "namespace/secretName".
-* `nginx.ingress.kubernetes.io/auth-tls-verify-depth`:
-  The validation depth between the provided client certificate and the Certification Authority chain.
-* `nginx.ingress.kubernetes.io/auth-tls-verify-client`:
-  Enables verification of client certificates. Possible values are:
-  * `off`: Don't request client certificates and don't do client certificate verification. (default)
-  * `on`: Request a client certificate that must be signed by a certificate that is included in the secret key `ca.crt` of the secret specified by `nginx.ingress.kubernetes.io/auth-tls-secret: secretName`. Failed certificate verification will result in a status code 400 (Bad Request).
-  * `optional`: Do optional client certificate validation against the CAs from `auth-tls-secret`. The request fails with status code 400 (Bad Request) when a certificate is provided that is not signed by the CA. When no or an otherwise invalid certificate is provided, the request does not fail, but instead the verification result is sent to the upstream service.
-  * `optional_no_ca`: Do optional client certificate validation, but do not fail the request when the client certificate is not signed by the CAs from `auth-tls-secret`. Certificate verification result is sent to the upstream service.
-* `nginx.ingress.kubernetes.io/auth-tls-error-page`:
-  The URL/Page that user should be redirected in case of a Certificate Authentication Error
-* `nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream`:
-  Indicates if the received certificates should be passed or not to the upstream server in the header `ssl-client-cert`. Possible values are "true" or "false" (default).
+You can further customize client certificate authentication and behaviour with these annotations:
+
+* `nginx.ingress.kubernetes.io/auth-tls-verify-depth`: The validation depth between the provided client certificate and the Certification Authority chain. (default: 1)
+* `nginx.ingress.kubernetes.io/auth-tls-verify-client`: Enables verification of client certificates. Possible values are:
+    * `on`: Request a client certificate that must be signed by a certificate that is included in the secret key `ca.crt` of the secret specified by `nginx.ingress.kubernetes.io/auth-tls-secret: namespace/secretName`. Failed certificate verification will result in a status code 400 (Bad Request) (default)
+    * `off`: Don't request client certificates and don't do client certificate verification.
+    * `optional`: Do optional client certificate validation against the CAs from `auth-tls-secret`. The request fails with status code 400 (Bad Request) when a certificate is provided that is not signed by the CA. When no or an otherwise invalid certificate is provided, the request does not fail, but instead the verification result is sent to the upstream service.
+    * `optional_no_ca`: Do optional client certificate validation, but do not fail the request when the client certificate is not signed by the CAs from `auth-tls-secret`. Certificate verification result is sent to the upstream service.
+* `nginx.ingress.kubernetes.io/auth-tls-error-page`: The URL/Page that user should be redirected in case of a Certificate Authentication Error
+* `nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream`: Indicates if the received certificates should be passed or not to the upstream server in the header `ssl-client-cert`. Possible values are "true" or "false" (default).
 
 The following headers are sent to the upstream service according to the `auth-tls-*` annotations:
 
@@ -332,39 +328,43 @@ location enabling this functionality.
 
 CORS can be controlled with the following annotations:
 
-* `nginx.ingress.kubernetes.io/cors-allow-methods`
-  controls which methods are accepted. This is a multi-valued field, separated by ',' and
-  accepts only letters (upper and lower case).
-  - Default: `GET, PUT, POST, DELETE, PATCH, OPTIONS`
-  - Example: `nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"`
-
-* `nginx.ingress.kubernetes.io/cors-allow-headers`
-  controls which headers are accepted. This is a multi-valued field, separated by ',' and accepts letters,
-  numbers, _ and -.
-  - Default: `DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization`
-  - Example: `nginx.ingress.kubernetes.io/cors-allow-headers: "X-Forwarded-For, X-app123-XPTO"`
-
-* `nginx.ingress.kubernetes.io/cors-expose-headers`
-  controls which headers are exposed to response. This is a multi-valued field, separated by ',' and accepts
-  letters, numbers, _, - and *.
-  - Default: *empty*
-  - Example: `nginx.ingress.kubernetes.io/cors-expose-headers: "*, X-CustomResponseHeader"`
-
-* `nginx.ingress.kubernetes.io/cors-allow-origin`
-  controls what's the accepted Origin for CORS.
-  This is a single field value, with the following format: `http(s)://origin-site.com` or `http(s)://origin-site.com:port`
-  - Default: `*`
-  - Example: `nginx.ingress.kubernetes.io/cors-allow-origin: "https://origin-site.com:4443"`
-
-* `nginx.ingress.kubernetes.io/cors-allow-credentials`
-  controls if credentials can be passed during CORS operations.
-  - Default: `true`
-  - Example: `nginx.ingress.kubernetes.io/cors-allow-credentials: "false"`
-
-* `nginx.ingress.kubernetes.io/cors-max-age`
-  controls how long preflight requests can be cached.
-  Default: `1728000`
-  Example: `nginx.ingress.kubernetes.io/cors-max-age: 600`
+* `nginx.ingress.kubernetes.io/cors-allow-methods`: Controls which methods are accepted.
+
+    This is a multi-valued field, separated by ',' and accepts only letters (upper and lower case).
+
+    - Default: `GET, PUT, POST, DELETE, PATCH, OPTIONS`
+    - Example: `nginx.ingress.kubernetes.io/cors-allow-methods: "PUT, GET, POST, OPTIONS"`
+
+* `nginx.ingress.kubernetes.io/cors-allow-headers`: Controls which headers are accepted.
+
+    This is a multi-valued field, separated by ',' and accepts letters, numbers, _ and -.
+
+    - Default: `DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Authorization`
+    - Example: `nginx.ingress.kubernetes.io/cors-allow-headers: "X-Forwarded-For, X-app123-XPTO"`
+
+* `nginx.ingress.kubernetes.io/cors-expose-headers`: Controls which headers are exposed to response.
+
+    This is a multi-valued field, separated by ',' and accepts letters, numbers, _, - and *.
+
+    - Default: *empty*
+    - Example: `nginx.ingress.kubernetes.io/cors-expose-headers: "*, X-CustomResponseHeader"`
+
+* `nginx.ingress.kubernetes.io/cors-allow-origin`: Controls what's the accepted Origin for CORS.
+
+    This is a single field value, with the following format: `http(s)://origin-site.com` or `http(s)://origin-site.com:port`
+
+    - Default: `*`
+    - Example: `nginx.ingress.kubernetes.io/cors-allow-origin: "https://origin-site.com:4443"`
+
+* `nginx.ingress.kubernetes.io/cors-allow-credentials`: Controls if credentials can be passed during CORS operations.
+
+    - Default: `true`
+    - Example: `nginx.ingress.kubernetes.io/cors-allow-credentials: "false"`
+
+* `nginx.ingress.kubernetes.io/cors-max-age`: Controls how long preflight requests can be cached.
+
+    - Default: `1728000`
+    - Example: `nginx.ingress.kubernetes.io/cors-max-age: 600`
 
 !!! note
     For more information please see [https://enable-cors.org](https://enable-cors.org/server_nginx.html)
@@ -591,6 +591,8 @@ When using SSL offloading outside of cluster (e.g. AWS ELB) it may be useful to
 even when there is no TLS certificate available.
 This can be achieved by using the `nginx.ingress.kubernetes.io/force-ssl-redirect: "true"` annotation in the particular resource.
 
+To preserve the trailing slash in the URI with `ssl-redirect`, set `nginx.ingress.kubernetes.io/preserve-trailing-slash: "true"` annotation for that particular resource.
+
 ### Redirect from/to www
 
 In some scenarios is required to redirect from `www.domain.com` to `domain.com` or vice versa.
diff --git a/hack/init-buildx.sh b/hack/init-buildx.sh
index a24b64956..1a47bf145 100755
--- a/hack/init-buildx.sh
+++ b/hack/init-buildx.sh
@@ -29,6 +29,16 @@ if ! docker buildx 2>&1 >/dev/null; then
   exit 1
 fi
 
+# Ensure qemu is in binfmt_misc
+# Docker desktop already has these in versions recent enough to have buildx
+# We only need to do this setup on linux hosts
+if [ "$(uname)" == 'Linux' ]; then
+  # NOTE: this is pinned to a digest for a reason!
+  # Note2 (@rikatz) - Removing the pin, as apparently it's breaking new alpine builds
+  # docker run --rm --privileged multiarch/qemu-user-static@sha256:28ebe2e48220ae8fd5d04bb2c847293b24d7fbfad84f0b970246e0a4efd48ad6 --reset -p yes
+  docker run --rm --privileged multiarch/qemu-user-static --reset -p yes
+fi
+
 # We can skip setup if the current builder already has multi-arch
 # AND if it isn't the docker driver, which doesn't work
 current_builder="$(docker buildx inspect)"
@@ -41,13 +51,6 @@ if ! grep -q "^Driver: docker$"  <<<"${current_builder}" && \
   exit 0
 fi
 
-# Ensure qemu is in binfmt_misc
-# Docker desktop already has these in versions recent enough to have buildx
-# We only need to do this setup on linux hosts
-if [ "$(uname)" == 'Linux' ]; then
-  # NOTE: this is pinned to a digest for a reason!
-  docker run --rm --privileged multiarch/qemu-user-static@sha256:28ebe2e48220ae8fd5d04bb2c847293b24d7fbfad84f0b970246e0a4efd48ad6 --reset -p yes
-fi
 
 # Ensure we use a builder that can leverage it (the default on linux will not)
 docker buildx rm ingress-nginx || true
diff --git a/images/echo/Makefile b/images/echo/Makefile
index 90a544b24..ccfa711b0 100644
--- a/images/echo/Makefile
+++ b/images/echo/Makefile
@@ -36,7 +36,7 @@ build: ensure-buildx
 		--platform=${PLATFORMS} $(OUTPUT) \
 		--progress=$(PROGRESS) \
 		--pull \
-		--build-arg BASE_IMAGE=k8s.gcr.io/ingress-nginx/nginx:v20210809-g98288bc3c@sha256:f9363669cf26514c9548c1fe4f8f4e2f58dfb76616bcd638a0ff7f0ec3457c17 \
+		--build-arg BASE_IMAGE=k8s.gcr.io/ingress-nginx/nginx:v20210904-gb7c973dce@sha256:fac972a7e43b18408ecb9e87da868df519428294e2e988c16be72479ee873c0e \
 		--build-arg LUAROCKS_VERSION=3.3.1 \
 		--build-arg LUAROCKS_SHA=837481e408f7c06b59befe7ec194537c657687d624894bca7f79034302141a34 \
 		-t $(IMAGE):$(TAG) rootfs
diff --git a/images/nginx/README.md b/images/nginx/README.md
index dde0b618a..fc247699e 100644
--- a/images/nginx/README.md
+++ b/images/nginx/README.md
@@ -18,11 +18,6 @@ This image provides a default configuration file with no backend servers.
 _Using docker_
 
 ```console
-docker run -v /some/nginx.con:/etc/nginx/nginx.conf:ro k8s.gcr.io/ingress-nginx/nginx:v20210809-g98288bc3c@sha256:f9363669cf26514c9548c1fe4f8f4e2f58dfb76616bcd638a0ff7f0ec3457c17
+docker run -v /some/nginx.con:/etc/nginx/nginx.conf:ro k8s.gcr.io/ingress-nginx/nginx:v20210904-gb7c973dce@sha256:fac972a7e43b18408ecb9e87da868df519428294e2e988c16be72479ee873c0e
 ```
 
-_Creating a replication controller_
-
-```console
-kubectl create -f ./rc.yaml
-```
diff --git a/images/nginx/cloudbuild.yaml b/images/nginx/cloudbuild.yaml
index ba09d2acd..c5aca043e 100644
--- a/images/nginx/cloudbuild.yaml
+++ b/images/nginx/cloudbuild.yaml
@@ -4,7 +4,7 @@ options:
   # job builds a multi-arch docker image for amd64,arm,arm64 and s390x.
   machineType: N1_HIGHCPU_32
 steps:
-  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f
+  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20210722-085d930
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/images/nginx/rc.yaml b/images/nginx/rc.yaml
index 98a3df14b..e5654260f 100644
--- a/images/nginx/rc.yaml
+++ b/images/nginx/rc.yaml
@@ -38,7 +38,7 @@ spec:
     spec:
       containers:
         - name: nginx
-          image: k8s.gcr.io/ingress-nginx/nginx:v20210809-g98288bc3c@sha256:f9363669cf26514c9548c1fe4f8f4e2f58dfb76616bcd638a0ff7f0ec3457c17
+          image: k8s.gcr.io/ingress-nginx/nginx:v20210904-gb7c973dce@sha256:fac972a7e43b18408ecb9e87da868df519428294e2e988c16be72479ee873c0e
           ports:
             - containerPort: 80
             - containerPort: 443
diff --git a/images/nginx/rootfs/Dockerfile b/images/nginx/rootfs/Dockerfile
index 676083bf8..4893f3c54 100644
--- a/images/nginx/rootfs/Dockerfile
+++ b/images/nginx/rootfs/Dockerfile
@@ -13,7 +13,7 @@
 # limitations under the License.
 
 
-FROM alpine:3.14.1 as builder
+FROM alpine:3.14.2 as builder
 
 COPY . /
 
@@ -23,7 +23,7 @@ RUN apk update \
   && /build.sh
 
 # Use a multi-stage build
-FROM alpine:3.14.1
+FROM alpine:3.14.2
 
 ENV PATH=$PATH:/usr/local/luajit/bin:/usr/local/nginx/sbin:/usr/local/nginx/bin
 
@@ -53,7 +53,6 @@ RUN apk update \
     nano \
     tzdata \
   && ln -s /usr/local/nginx/sbin/nginx /sbin/nginx \
-  && addgroup -Sg 101 www-data \
   && adduser -S -D -H -u 101 -h /usr/local/nginx \
     -s /sbin/nologin -G www-data -g www-data www-data \
   && bash -eu -c ' \
diff --git a/images/nginx/rootfs/build.sh b/images/nginx/rootfs/build.sh
index 46f304ea5..d8d57bafe 100755
--- a/images/nginx/rootfs/build.sh
+++ b/images/nginx/rootfs/build.sh
@@ -733,7 +733,6 @@ writeDirs=( \
   /var/log/nginx \
 );
 
-addgroup -Sg 101 www-data
 adduser -S -D -H -u 101 -h /usr/local/nginx -s /sbin/nologin -G www-data -g www-data www-data
 
 for dir in "${writeDirs[@]}"; do
diff --git a/internal/ingress/annotations/sessionaffinity/main.go b/internal/ingress/annotations/sessionaffinity/main.go
index 358be1442..9c4d1d2bc 100644
--- a/internal/ingress/annotations/sessionaffinity/main.go
+++ b/internal/ingress/annotations/sessionaffinity/main.go
@@ -37,6 +37,10 @@ const (
 
 	defaultAffinityCookieName = "INGRESSCOOKIE"
 
+	// This is used to force the Secure flag on the cookie even if the
+	// incoming request is not secured. (https://github.com/kubernetes/ingress-nginx/issues/6812)
+	annotationAffinityCookieSecure = "session-cookie-secure"
+
 	// This is used to control the cookie expires, its value is a number of seconds until the
 	// cookie expires
 	annotationAffinityCookieExpires = "session-cookie-expires"
@@ -85,6 +89,8 @@ type Cookie struct {
 	Path string `json:"path"`
 	// Flag that allows cookie regeneration on request failure
 	ChangeOnFailure bool `json:"changeonfailure"`
+	// Secure flag to be set
+	Secure bool `json:"secure"`
 	// SameSite attribute value
 	SameSite string `json:"samesite"`
 	// Flag that conditionally applies SameSite=None attribute on cookie if user agent accepts it.
@@ -126,6 +132,11 @@ func (a affinity) cookieAffinityParse(ing *networking.Ingress) *Cookie {
 		klog.V(3).InfoS("Invalid or no annotation value found. Ignoring", "ingress", klog.KObj(ing), "annotation", annotationAffinityCookieSameSite)
 	}
 
+	cookie.Secure, err = parser.GetBoolAnnotation(annotationAffinityCookieSecure, ing)
+	if err != nil {
+		klog.V(3).InfoS("Invalid or no annotation value found. Ignoring", "ingress", klog.KObj(ing), "annotation", annotationAffinityCookieSecure)
+	}
+
 	cookie.ConditionalSameSiteNone, err = parser.GetBoolAnnotation(annotationAffinityCookieConditionalSameSiteNone, ing)
 	if err != nil {
 		klog.V(3).InfoS("Invalid or no annotation value found. Ignoring", "ingress", klog.KObj(ing), "annotation", annotationAffinityCookieConditionalSameSiteNone)
diff --git a/internal/ingress/controller/controller.go b/internal/ingress/controller/controller.go
index 72a1afa63..1fd22079b 100644
--- a/internal/ingress/controller/controller.go
+++ b/internal/ingress/controller/controller.go
@@ -667,6 +667,7 @@ func (n *NGINXController) getBackendServers(ingresses []*ingress.Ingress) ([]*in
 					ups.SessionAffinity.CookieSessionAffinity.Name = anns.SessionAffinity.Cookie.Name
 					ups.SessionAffinity.CookieSessionAffinity.Expires = anns.SessionAffinity.Cookie.Expires
 					ups.SessionAffinity.CookieSessionAffinity.MaxAge = anns.SessionAffinity.Cookie.MaxAge
+					ups.SessionAffinity.CookieSessionAffinity.Secure = anns.SessionAffinity.Cookie.Secure
 					ups.SessionAffinity.CookieSessionAffinity.Path = cookiePath
 					ups.SessionAffinity.CookieSessionAffinity.SameSite = anns.SessionAffinity.Cookie.SameSite
 					ups.SessionAffinity.CookieSessionAffinity.ConditionalSameSiteNone = anns.SessionAffinity.Cookie.ConditionalSameSiteNone
diff --git a/internal/ingress/controller/template/template.go b/internal/ingress/controller/template/template.go
index b06f5a216..e5da5b993 100644
--- a/internal/ingress/controller/template/template.go
+++ b/internal/ingress/controller/template/template.go
@@ -1112,7 +1112,7 @@ func buildOpentracing(c interface{}, s interface{}) string {
 	buf := bytes.NewBufferString("")
 
 	if cfg.DatadogCollectorHost != "" {
-		buf.WriteString("opentracing_load_tracer /usr/local/lib64/libdd_opentracing.so /etc/nginx/opentracing.json;")
+		buf.WriteString("opentracing_load_tracer /usr/local/lib/libdd_opentracing.so /etc/nginx/opentracing.json;")
 	} else if cfg.ZipkinCollectorHost != "" {
 		buf.WriteString("opentracing_load_tracer /usr/local/lib/libzipkin_opentracing_plugin.so /etc/nginx/opentracing.json;")
 	} else if cfg.JaegerCollectorHost != "" || cfg.JaegerEndpoint != "" {
diff --git a/internal/ingress/types.go b/internal/ingress/types.go
index a1cbdf685..033fa9cc4 100644
--- a/internal/ingress/types.go
+++ b/internal/ingress/types.go
@@ -155,6 +155,7 @@ type CookieSessionAffinity struct {
 	Expires                 string              `json:"expires,omitempty"`
 	MaxAge                  string              `json:"maxage,omitempty"`
 	Locations               map[string][]string `json:"locations,omitempty"`
+	Secure                  bool                `json:"secure,omitempty"`
 	Path                    string              `json:"path,omitempty"`
 	SameSite                string              `json:"samesite,omitempty"`
 	ConditionalSameSiteNone bool                `json:"conditional_samesite_none,omitempty"`
diff --git a/internal/ingress/types_equals.go b/internal/ingress/types_equals.go
index 87af2bcfc..18ffa9042 100644
--- a/internal/ingress/types_equals.go
+++ b/internal/ingress/types_equals.go
@@ -182,6 +182,9 @@ func (csa1 *CookieSessionAffinity) Equal(csa2 *CookieSessionAffinity) bool {
 	if csa1.SameSite != csa2.SameSite {
 		return false
 	}
+	if csa1.Secure != csa2.Secure {
+		return false
+	}
 	if csa1.ConditionalSameSiteNone != csa2.ConditionalSameSiteNone {
 		return false
 	}
diff --git a/internal/task/queue.go b/internal/task/queue.go
index 3c6788409..ff6b20f62 100644
--- a/internal/task/queue.go
+++ b/internal/task/queue.go
@@ -118,7 +118,7 @@ func (t *Queue) worker() {
 		ts := time.Now().UnixNano()
 
 		item := key.(Element)
-		if t.lastSync > item.Timestamp {
+		if item.Timestamp != 0 && t.lastSync > item.Timestamp {
 			klog.V(3).InfoS("skipping sync", "key", item.Key, "last", t.lastSync, "now", item.Timestamp)
 			t.queue.Forget(key)
 			t.queue.Done(key)
@@ -130,7 +130,7 @@ func (t *Queue) worker() {
 			klog.ErrorS(err, "requeuing", "key", item.Key)
 			t.queue.AddRateLimited(Element{
 				Key:       item.Key,
-				Timestamp: time.Now().UnixNano(),
+				Timestamp: 0,
 			})
 		} else {
 			t.queue.Forget(key)
diff --git a/rootfs/etc/nginx/lua/balancer/sticky.lua b/rootfs/etc/nginx/lua/balancer/sticky.lua
index 63f3c6785..3440d86bd 100644
--- a/rootfs/etc/nginx/lua/balancer/sticky.lua
+++ b/rootfs/etc/nginx/lua/balancer/sticky.lua
@@ -87,13 +87,18 @@ function _M.set_cookie(self, value)
     end
   end
 
+  local cookie_secure = self.cookie_session_affinity.secure
+  if cookie_secure == nil then
+    cookie_secure = ngx.var.https == "on"
+  end
+
   local cookie_data = {
     key = self:cookie_name(),
     value = value .. COOKIE_VALUE_DELIMITER .. self.backend_key,
     path = cookie_path,
     httponly = true,
     samesite = cookie_samesite,
-    secure = ngx.var.https == "on",
+    secure = cookie_secure,
   }
 
   if self.cookie_session_affinity.expires and self.cookie_session_affinity.expires ~= "" then


Test patch:
diff --git a/images/test-runner/Makefile b/images/test-runner/Makefile
index 5c7b47d66..0ba3561b0 100644
--- a/images/test-runner/Makefile
+++ b/images/test-runner/Makefile
@@ -23,7 +23,7 @@ REGISTRY ?= local
 
 IMAGE = $(REGISTRY)/e2e-test-runner
 
-NGINX_BASE_IMAGE ?= k8s.gcr.io/ingress-nginx/nginx:v20210809-g98288bc3c@sha256:f9363669cf26514c9548c1fe4f8f4e2f58dfb76616bcd638a0ff7f0ec3457c17
+NGINX_BASE_IMAGE ?= k8s.gcr.io/ingress-nginx/nginx:v20210904-gb7c973dce@sha256:fac972a7e43b18408ecb9e87da868df519428294e2e988c16be72479ee873c0e
 
 # required to enable buildx
 export DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/images/test-runner/cloudbuild.yaml b/images/test-runner/cloudbuild.yaml
index 3e4d67d06..7258ab384 100644
--- a/images/test-runner/cloudbuild.yaml
+++ b/images/test-runner/cloudbuild.yaml
@@ -1,4 +1,4 @@
-timeout: 1200s
+timeout: 3600s
 options:
   substitution_option: ALLOW_LOOSE
 steps:
diff --git a/internal/ingress/annotations/authtls/main_test.go b/internal/ingress/annotations/authtls/main_test.go
index b2c915d18..f7649fe1c 100644
--- a/internal/ingress/annotations/authtls/main_test.go
+++ b/internal/ingress/annotations/authtls/main_test.go
@@ -94,10 +94,6 @@ func TestAnnotations(t *testing.T) {
 	data := map[string]string{}
 
 	data[parser.GetAnnotationWithPrefix("auth-tls-secret")] = "default/demo-secret"
-	data[parser.GetAnnotationWithPrefix("auth-tls-verify-client")] = "off"
-	data[parser.GetAnnotationWithPrefix("auth-tls-verify-depth")] = "1"
-	data[parser.GetAnnotationWithPrefix("auth-tls-error-page")] = "ok.com/error"
-	data[parser.GetAnnotationWithPrefix("auth-tls-pass-certificate-to-upstream")] = "true"
 
 	ing.SetAnnotations(data)
 
@@ -120,12 +116,45 @@ func TestAnnotations(t *testing.T) {
 	if u.AuthSSLCert.Secret != secret.Secret {
 		t.Errorf("expected %v but got %v", secret.Secret, u.AuthSSLCert.Secret)
 	}
-	if u.VerifyClient != "off" {
-		t.Errorf("expected %v but got %v", "off", u.VerifyClient)
+	if u.VerifyClient != "on" {
+		t.Errorf("expected %v but got %v", "on", u.VerifyClient)
 	}
 	if u.ValidationDepth != 1 {
 		t.Errorf("expected %v but got %v", 1, u.ValidationDepth)
 	}
+	if u.ErrorPage != "" {
+		t.Errorf("expected %v but got %v", "", u.ErrorPage)
+	}
+	if u.PassCertToUpstream != false {
+		t.Errorf("expected %v but got %v", false, u.PassCertToUpstream)
+	}
+
+	data[parser.GetAnnotationWithPrefix("auth-tls-verify-client")] = "off"
+	data[parser.GetAnnotationWithPrefix("auth-tls-verify-depth")] = "2"
+	data[parser.GetAnnotationWithPrefix("auth-tls-error-page")] = "ok.com/error"
+	data[parser.GetAnnotationWithPrefix("auth-tls-pass-certificate-to-upstream")] = "true"
+
+	ing.SetAnnotations(data)
+
+	i, err = NewParser(fakeSecret).Parse(ing)
+	if err != nil {
+		t.Errorf("Unexpected error with ingress: %v", err)
+	}
+
+	u, ok = i.(*Config)
+	if !ok {
+		t.Errorf("expected *Config but got %v", u)
+	}
+
+	if u.AuthSSLCert.Secret != secret.Secret {
+		t.Errorf("expected %v but got %v", secret.Secret, u.AuthSSLCert.Secret)
+	}
+	if u.VerifyClient != "off" {
+		t.Errorf("expected %v but got %v", "off", u.VerifyClient)
+	}
+	if u.ValidationDepth != 2 {
+		t.Errorf("expected %v but got %v", 2, u.ValidationDepth)
+	}
 	if u.ErrorPage != "ok.com/error" {
 		t.Errorf("expected %v but got %v", "ok.com/error", u.ErrorPage)
 	}
diff --git a/internal/ingress/annotations/sessionaffinity/main_test.go b/internal/ingress/annotations/sessionaffinity/main_test.go
index 98f0a1eb6..65d11ac2d 100644
--- a/internal/ingress/annotations/sessionaffinity/main_test.go
+++ b/internal/ingress/annotations/sessionaffinity/main_test.go
@@ -79,6 +79,7 @@ func TestIngressAffinityCookieConfig(t *testing.T) {
 	data[parser.GetAnnotationWithPrefix(annotationAffinityCookieMaxAge)] = "3000"
 	data[parser.GetAnnotationWithPrefix(annotationAffinityCookiePath)] = "/foo"
 	data[parser.GetAnnotationWithPrefix(annotationAffinityCookieChangeOnFailure)] = "true"
+	data[parser.GetAnnotationWithPrefix(annotationAffinityCookieSecure)] = "true"
 	ing.SetAnnotations(data)
 
 	affin, _ := NewParser(&resolver.Mock{}).Parse(ing)
@@ -114,4 +115,8 @@ func TestIngressAffinityCookieConfig(t *testing.T) {
 	if !nginxAffinity.Cookie.ChangeOnFailure {
 		t.Errorf("expected change of failure parameter set to true but returned %v", nginxAffinity.Cookie.ChangeOnFailure)
 	}
+
+	if !nginxAffinity.Cookie.Secure {
+		t.Errorf("expected secure parameter set to true but returned %v", nginxAffinity.Cookie.Secure)
+	}
 }
diff --git a/internal/ingress/controller/template/template_test.go b/internal/ingress/controller/template/template_test.go
index 14291dbf2..96dc9acae 100644
--- a/internal/ingress/controller/template/template_test.go
+++ b/internal/ingress/controller/template/template_test.go
@@ -1520,7 +1520,7 @@ func TestBuildOpenTracing(t *testing.T) {
 		EnableOpentracing:    true,
 		DatadogCollectorHost: "datadog-host.com",
 	}
-	expected = "opentracing_load_tracer /usr/local/lib64/libdd_opentracing.so /etc/nginx/opentracing.json;\r\n"
+	expected = "opentracing_load_tracer /usr/local/lib/libdd_opentracing.so /etc/nginx/opentracing.json;\r\n"
 	actual = buildOpentracing(cfgDatadog, []*ingress.Server{})
 
 	if expected != actual {
@@ -1544,7 +1544,7 @@ func TestBuildOpenTracing(t *testing.T) {
 		OpentracingOperationName:         "my-operation-name",
 		OpentracingLocationOperationName: "my-location-operation-name",
 	}
-	expected = "opentracing_load_tracer /usr/local/lib64/libdd_opentracing.so /etc/nginx/opentracing.json;\r\n"
+	expected = "opentracing_load_tracer /usr/local/lib/libdd_opentracing.so /etc/nginx/opentracing.json;\r\n"
 	expected += "opentracing_operation_name \"my-operation-name\";\n"
 	expected += "opentracing_location_operation_name \"my-location-operation-name\";\n"
 	actual = buildOpentracing(cfgOpenTracing, []*ingress.Server{})
diff --git a/rootfs/etc/nginx/lua/test/balancer/sticky_test.lua b/rootfs/etc/nginx/lua/test/balancer/sticky_test.lua
index a0c0ae54f..80d0c0d0e 100644
--- a/rootfs/etc/nginx/lua/test/balancer/sticky_test.lua
+++ b/rootfs/etc/nginx/lua/test/balancer/sticky_test.lua
@@ -422,7 +422,7 @@ describe("Sticky", function()
       cookie.new = mocked_cookie_new
     end)
 
-    local function test_set_cookie_with(sticky_balancer_type, samesite, conditional_samesite_none, expected_path, expected_samesite)
+    local function test_set_cookie_with(sticky_balancer_type, samesite, conditional_samesite_none, expected_path, expected_samesite, secure, expected_secure)
       local s = {}
       cookie.new = function(self)
         local cookie_instance = {
@@ -432,7 +432,7 @@ describe("Sticky", function()
             assert.equal(payload.samesite, expected_samesite)
             assert.equal(payload.domain, nil)
             assert.equal(payload.httponly, true)
-            assert.equal(payload.secure, false)
+            assert.equal(payload.secure, expected_secure)
             return true, nil
           end,
           get = function(k) return false end,
@@ -445,57 +445,61 @@ describe("Sticky", function()
       b.sessionAffinityConfig.cookieSessionAffinity.locations["test.com"] = {"/"}
       b.sessionAffinityConfig.cookieSessionAffinity.samesite = samesite
       b.sessionAffinityConfig.cookieSessionAffinity.conditional_samesite_none = conditional_samesite_none
+      b.sessionAffinityConfig.cookieSessionAffinity.secure = secure
       local sticky_balancer_instance = sticky_balancer_type:new(b)
       assert.has_no.errors(function() sticky_balancer_instance:balance() end)
       assert.spy(s).was_called()
     end
 
-    it("returns a cookie with SameSite=Strict when user specifies samesite strict", function()
-      test_set_cookie_with(sticky_balanced, "Strict", false, "/", "Strict")
+    it("returns a secure cookie with SameSite=Strict when user specifies samesite strict and secure=true", function()
+      test_set_cookie_with(sticky_balanced, "Lax", false, "/", "Lax", true, true)
     end)
     it("returns a cookie with SameSite=Strict when user specifies samesite strict and conditional samesite none", function()
-      test_set_cookie_with(sticky_balanced, "Strict", true, "/", "Strict")
+      test_set_cookie_with(sticky_balanced, "Strict", true, "/", "Strict", nil, false)
     end)
     it("returns a cookie with SameSite=Lax when user specifies samesite lax", function()
-      test_set_cookie_with(sticky_balanced, "Lax", false, "/", "Lax")
+      test_set_cookie_with(sticky_balanced, "Lax", false, "/", "Lax", nil, false)
     end)
     it("returns a cookie with SameSite=Lax when user specifies samesite lax and conditional samesite none", function()
-      test_set_cookie_with(sticky_balanced, "Lax", true, "/", "Lax")
+      test_set_cookie_with(sticky_balanced, "Lax", true, "/", "Lax", nil, false)
     end)
     it("returns a cookie with SameSite=None when user specifies samesite None", function()
-      test_set_cookie_with(sticky_balanced, "None", false, "/", "None")
+      test_set_cookie_with(sticky_balanced, "None", false, "/", "None", nil, false)
     end)
     it("returns a cookie with SameSite=None when user specifies samesite None and conditional samesite none with supported user agent", function()
       mock_ngx({ var = { location_path = "/", host = "test.com" , http_user_agent = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.2704.103 Safari/537.36"} })
-      test_set_cookie_with(sticky_balanced, "None", true, "/", "None")
+      test_set_cookie_with(sticky_balanced, "None", true, "/", "None", nil, false)
     end)
     it("returns a cookie without SameSite=None when user specifies samesite None and conditional samesite none with unsupported user agent", function()
       mock_ngx({ var = { location_path = "/", host = "test.com" , http_user_agent = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"} })
-      test_set_cookie_with(sticky_balanced, "None", true, "/", nil)
+      test_set_cookie_with(sticky_balanced, "None", true, "/", nil, nil, false)
     end)
 
+    it("returns a secure cookie with SameSite=Strict when user specifies samesite strict and secure=true", function()
+      test_set_cookie_with(sticky_persistent, "Lax", false, "/", "Lax", true, true)
+    end)
     it("returns a cookie with SameSite=Strict when user specifies samesite strict", function()
-      test_set_cookie_with(sticky_persistent, "Strict", false, "/", "Strict")
+      test_set_cookie_with(sticky_persistent, "Strict", false, "/", "Strict", nil, false)
     end)
     it("returns a cookie with SameSite=Strict when user specifies samesite strict and conditional samesite none", function()
-      test_set_cookie_with(sticky_persistent, "Strict", true, "/", "Strict")
+      test_set_cookie_with(sticky_persistent, "Strict", true, "/", "Strict", nil, false)
     end)
     it("returns a cookie with SameSite=Lax when user specifies samesite lax", function()
-      test_set_cookie_with(sticky_persistent, "Lax", false, "/", "Lax")
+      test_set_cookie_with(sticky_persistent, "Lax", false, "/", "Lax", nil, false)
     end)
     it("returns a cookie with SameSite=Lax when user specifies samesite lax and conditional samesite none", function()
-      test_set_cookie_with(sticky_persistent, "Lax", true, "/", "Lax")
+      test_set_cookie_with(sticky_persistent, "Lax", true, "/", "Lax", nil, false)
     end)
     it("returns a cookie with SameSite=None when user specifies samesite None", function()
-      test_set_cookie_with(sticky_persistent, "None", false, "/", "None")
+      test_set_cookie_with(sticky_persistent, "None", false, "/", "None", nil, false)
     end)
     it("returns a cookie with SameSite=None when user specifies samesite None and conditional samesite none with supported user agent", function()
       mock_ngx({ var = { location_path = "/", host = "test.com" , http_user_agent = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.2704.103 Safari/537.36"} })
-      test_set_cookie_with(sticky_persistent, "None", true, "/", "None")
+      test_set_cookie_with(sticky_persistent, "None", true, "/", "None", nil, false)
     end)
     it("returns a cookie without SameSite=None when user specifies samesite None and conditional samesite none with unsupported user agent", function()
       mock_ngx({ var = { location_path = "/", host = "test.com" , http_user_agent = "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"} })
-      test_set_cookie_with(sticky_persistent, "None", true, "/", nil)
+      test_set_cookie_with(sticky_persistent, "None", true, "/", nil, nil, false)
     end)
   end)
 
diff --git a/test/e2e-image/Dockerfile b/test/e2e-image/Dockerfile
index 28cd272b6..083ad5792 100644
--- a/test/e2e-image/Dockerfile
+++ b/test/e2e-image/Dockerfile
@@ -1,4 +1,4 @@
-FROM k8s.gcr.io/ingress-nginx/e2e-test-runner:v20210822-g5e5faa24d@sha256:55c568d9e35e15d94b3ab41fe549b8ee4cd910cc3e031ddcccd06256755c5d89 AS BASE
+FROM k8s.gcr.io/ingress-nginx/e2e-test-runner:v20210906-g7d577d976@sha256:cf7079b5c05b8b1b108b16752c6ff4ca312cf96700e91eef6088b9e0c4a7aff1 AS BASE
 
 FROM alpine:3.12
 
diff --git a/test/e2e/annotations/authtls.go b/test/e2e/annotations/authtls.go
index 86c5d9ec5..093afe14e 100644
--- a/test/e2e/annotations/authtls.go
+++ b/test/e2e/annotations/authtls.go
@@ -17,7 +17,6 @@ limitations under the License.
 package annotations
 
 import (
-	"crypto/tls"
 	"fmt"
 	"net/http"
 	"strings"
@@ -34,7 +33,7 @@ var _ = framework.DescribeAnnotation("auth-tls-*", func() {
 		f.NewEchoDeploymentWithReplicas(2)
 	})
 
-	ginkgo.It("should set valid auth-tls-secret", func() {
+	ginkgo.It("should set sslClientCertificate, sslVerifyClient and sslVerifyDepth with auth-tls-secret", func() {
 		host := "authtls.foo.com"
 		nameSpace := f.Namespace
 
@@ -45,16 +44,28 @@ var _ = framework.DescribeAnnotation("auth-tls-*", func() {
 			nameSpace)
 		assert.Nil(ginkgo.GinkgoT(), err)
 
-		annotations := map[string]string{
+		annotations := map[string]string{}
+
+		ing := f.EnsureIngress(framework.NewSingleIngressWithTLS(host, "/", host, []string{host}, nameSpace, framework.EchoService, 80, annotations))
+
+		f.WaitForNginxServer(host,
+			func(server string) bool {
+				return !strings.Contains(server, "ssl_client_certificate") &&
+					!strings.Contains(server, "ssl_verify_client") &&
+					!strings.Contains(server, "ssl_verify_depth")
+			})
+
+		annotations = map[string]string{
 			"nginx.ingress.kubernetes.io/auth-tls-secret": nameSpace + "/" + host,
 		}
 
-		f.EnsureIngress(framework.NewSingleIngressWithTLS(host, "/", host, []string{host}, nameSpace, framework.EchoService, 80, annotations))
+		ing.SetAnnotations(annotations)
+		f.UpdateIngress(ing)
 
 		assertSslClientCertificateConfig(f, host, "on", "1")
 
 		// Send Request without Client Certs
-		f.HTTPTestClientWithTLSConfig(&tls.Config{ServerName: host, InsecureSkipVerify: true}).
+		f.HTTPTestClient().
 			GET("/").
 			WithURL(f.GetURL(framework.HTTPS)).
 			WithHeader("Host", host).
@@ -100,7 +111,7 @@ var _ = framework.DescribeAnnotation("auth-tls-*", func() {
 			Status(http.StatusOK)
 	})
 
-	ginkgo.It("should set valid auth-tls-secret, pass certificate to upstream, and error page", func() {
+	ginkgo.It("should 302 redirect to error page instead of 400 when auth-tls-error-page is set", func() {
 		host := "authtls.foo.com"
 		nameSpace := f.Namespace
 
@@ -114,9 +125,8 @@ var _ = framework.DescribeAnnotation("auth-tls-*", func() {
 		assert.Nil(ginkgo.GinkgoT(), err)
 
 		annotations := map[string]string{
-			"nginx.ingress.kubernetes.io/auth-tls-secret":                       nameSpace + "/" + host,
-			"nginx.ingress.kubernetes.io/auth-tls-error-page":                   f.GetURL(framework.HTTP) + errorPath,
-			"nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream": "true",
+			"nginx.ingress.kubernetes.io/auth-tls-secret":     nameSpace + "/" + host,
+			"nginx.ingress.kubernetes.io/auth-tls-error-page": f.GetURL(framework.HTTP) + errorPath,
 		}
 
 		f.EnsureIngress(framework.NewSingleIngressWithTLS(host, "/", host, []string{host}, nameSpace, framework.EchoService, 80, annotations))
@@ -124,12 +134,10 @@ var _ = framework.DescribeAnnotation("auth-tls-*", func() {
 		assertSslClientCertificateConfig(f, host, "on", "1")
 
 		sslErrorPage := fmt.Sprintf("error_page 495 496 = %s;", f.GetURL(framework.HTTP)+errorPath)
-		sslUpstreamClientCert := "proxy_set_header ssl-client-cert $ssl_client_escaped_cert;"
 
 		f.WaitForNginxServer(host,
 			func(server string) bool {
-				return strings.Contains(server, sslErrorPage) &&
-					strings.Contains(server, sslUpstreamClientCert)
+				return strings.Contains(server, sslErrorPage)
 			})
 
 		// Send Request without Client Certs
@@ -150,6 +158,51 @@ var _ = framework.DescribeAnnotation("auth-tls-*", func() {
 			Status(http.StatusOK)
 	})
 
+	ginkgo.It("should pass URL-encoded certificate to upstream", func() {
+		host := "authtls.foo.com"
+		nameSpace := f.Namespace
+
+		clientConfig, err := framework.CreateIngressMASecret(
+			f.KubeClientSet,
+			host,
+			host,
+			nameSpace)
+		assert.Nil(ginkgo.GinkgoT(), err)
+
+		annotations := map[string]string{
+			"nginx.ingress.kubernetes.io/auth-tls-secret":                       nameSpace + "/" + host,
+			"nginx.ingress.kubernetes.io/auth-tls-pass-certificate-to-upstream": "true",
+		}
+
+		f.EnsureIngress(framework.NewSingleIngressWithTLS(host, "/", host, []string{host}, nameSpace, framework.EchoService, 80, annotations))
+
+		assertSslClientCertificateConfig(f, host, "on", "1")
+
+		sslUpstreamClientCert := "proxy_set_header ssl-client-cert $ssl_client_escaped_cert;"
+
+		f.WaitForNginxServer(host,
+			func(server string) bool {
+				return strings.Contains(server, sslUpstreamClientCert)
+			})
+
+		// Send Request without Client Certs
+		f.HTTPTestClient().
+			GET("/").
+			WithURL(f.GetURL(framework.HTTPS)).
+			WithHeader("Host", host).
+			Expect().
+			Status(http.StatusBadRequest)
+
+		// Send Request Passing the Client Certs
+		f.HTTPTestClientWithTLSConfig(clientConfig).
+			GET("/").
+			WithURL(f.GetURL(framework.HTTPS)).
+			WithHeader("Host", host).
+			Expect().
+			Status(http.StatusOK).
+			Body().Contains("ssl-client-cert=-----BEGIN%20CERTIFICATE-----%0A")
+	})
+
 	ginkgo.It("should validate auth-tls-verify-client", func() {
 		host := "authtls.foo.com"
 		nameSpace := f.Namespace
diff --git a/test/e2e/annotations/modsecurity.go b/test/e2e/annotations/modsecurity/modsecurity.go
similarity index 99%
rename from test/e2e/annotations/modsecurity.go
rename to test/e2e/annotations/modsecurity/modsecurity.go
index d83803c93..fad3a2dee 100644
--- a/test/e2e/annotations/modsecurity.go
+++ b/test/e2e/annotations/modsecurity/modsecurity.go
@@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
-package annotations
+package modsecurity
 
 import (
 	"net/http"
diff --git a/test/e2e/e2e.go b/test/e2e/e2e.go
index 9bb8937ea..c0829f5ee 100644
--- a/test/e2e/e2e.go
+++ b/test/e2e/e2e.go
@@ -32,6 +32,7 @@ import (
 	// tests to run
 	_ "k8s.io/ingress-nginx/test/e2e/admission"
 	_ "k8s.io/ingress-nginx/test/e2e/annotations"
+	_ "k8s.io/ingress-nginx/test/e2e/annotations/modsecurity"
 	_ "k8s.io/ingress-nginx/test/e2e/dbg"
 	_ "k8s.io/ingress-nginx/test/e2e/defaultbackend"
 	_ "k8s.io/ingress-nginx/test/e2e/gracefulshutdown"
@@ -42,6 +43,7 @@ import (
 	_ "k8s.io/ingress-nginx/test/e2e/security"
 	_ "k8s.io/ingress-nginx/test/e2e/servicebackend"
 	_ "k8s.io/ingress-nginx/test/e2e/settings"
+	_ "k8s.io/ingress-nginx/test/e2e/settings/modsecurity"
 	_ "k8s.io/ingress-nginx/test/e2e/settings/ocsp"
 	_ "k8s.io/ingress-nginx/test/e2e/ssl"
 	_ "k8s.io/ingress-nginx/test/e2e/status"
diff --git a/test/e2e/framework/deployment.go b/test/e2e/framework/deployment.go
index 79da5c2ca..95bf37d75 100644
--- a/test/e2e/framework/deployment.go
+++ b/test/e2e/framework/deployment.go
@@ -38,7 +38,7 @@ const SlowEchoService = "slow-echo"
 const HTTPBinService = "httpbin"
 
 // NginxBaseImage use for testing
-const NginxBaseImage = "k8s.gcr.io/ingress-nginx/nginx:v20210809-g98288bc3c@sha256:f9363669cf26514c9548c1fe4f8f4e2f58dfb76616bcd638a0ff7f0ec3457c17"
+const NginxBaseImage = "k8s.gcr.io/ingress-nginx/nginx:v20210904-gb7c973dce@sha256:fac972a7e43b18408ecb9e87da868df519428294e2e988c16be72479ee873c0e"
 
 // NewEchoDeployment creates a new single replica deployment of the echoserver image in a particular namespace
 func (f *Framework) NewEchoDeployment() {
diff --git a/test/e2e/settings/modsecurity_snippet.go b/test/e2e/settings/modsecurity/modsecurity_snippet.go
similarity index 98%
rename from test/e2e/settings/modsecurity_snippet.go
rename to test/e2e/settings/modsecurity/modsecurity_snippet.go
index c0b962236..f912db984 100644
--- a/test/e2e/settings/modsecurity_snippet.go
+++ b/test/e2e/settings/modsecurity/modsecurity_snippet.go
@@ -14,7 +14,7 @@ See the License for the specific language governing permissions and
 limitations under the License.
 */
 
-package settings
+package modsecurity
 
 import (
 	"strings"