Repository: kubernetes/ingress-nginx
Pull request #8021 (2021-12-08 12:48:12)

Disable default modsecurity_rules_file if modsecurity-snippet is specified
The default modsecurity_rules_file (/etc/nginx/modsecurity/modsecurity.conf;) has settings that override the ModSecurity-snippet if it is specified with custom config settings like "SecAuditLog",  "SecAuditLogStorageDir" and "SecRuleEngine On". 

<!--- Provide a general summary of your changes in the Title above --->
<!--- Please don't @-mention people in PR or commit messages (do so in an additional comment). --->

## What this PR does / why we need it:
To disable the  default modsecurity_rules_file (/etc/nginx/modsecurity/modsecurity.conf;) if custom modsecurity-snippet is specified

## Types of changes
<!--- What types of changes does your code introduce? Put an `x` in all the boxes that apply: -->
- [x] Bug fix (non-breaking change which fixes an issue)
- [ ] New feature (non-breaking change which adds functionality)
- [ ] Breaking change (fix or feature that would cause existing functionality to change)
- [ ] Documentation only

## Which issue/s this PR fixes

[6307](https://github.com/kubernetes/ingress-nginx/issues/6307)
[6438](https://github.com/kubernetes/ingress-nginx/issues/6438)
[4640](https://github.com/kubernetes/ingress-nginx/issues/4640)


## How Has This Been Tested?

In the values yaml file have the config below
config:
      enable-modsecurity: "true"
      enable-owasp-modsecurity-crs: "false"
      modsecurity-snippet: |-
          SecRuleEngine On
          SecAuditEngine RelevantOnly
          SecAuditLogParts ABIJDEFHZ
          SecAuditLogRelevantStatus "^(?:05912300)"
          SecAuditLogType Concurrent
          SecAuditLog /var/tmp/modsec_audit.log
          SecAuditLogStorageDir /var/tmp/

Then deploying the latest nginx controller
helm upgrade --install --values your-values-with-config-above.yaml your-release-name ingress-nginx/ingress-nginx --version 4.0.13

After you deploy, exec into one nginx pod and browse " /etc/nginx/nginx.conf ". You will find after your modsecurity custom config the below line is included

      modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;

Browsing this file, you will find some settings that conflict with the settings in your "modsecurity-snippet: | " such as:
        
        SecRuleEngine DetectionOnly
        SecAuditLog /var/log/modsec_audit.log
        SecAuditLogStorageDir /var/log/audit/


Verification:
make some suspicious traffic to your nginx ingress such as

   curl http://your-url/index.html\?exec\=/bin/bash

Then when you exec into the nginx pod, you will see the logs are in "/var/log/modsec_audit.log" (from the default modsecurity file) instead of the customized configured one "/var/tmp/modsec_audit.log" in the (modsecurity-snippet)



This PR will not have this include file " modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf" in the nginx configuration file. After deploying this PR, all the custom modsecurity-snippet will take effect and not be overridden.
Then if you do suspicious traffic to your nginx ingress like the one above,t he audit log will be /var/tmp/modsec_audit.log , and SecRuleEngine On will be on the blocking mode according to the rules in your modsecurity-snippet


## Checklist:
<!--- Go over all the following points, and put an `x` in all the boxes that apply. -->
<!--- If you're unsure about any of these, don't hesitate to ask. We're here to help! -->
- [ ] My change requires a change to the documentation.
- [ ] I have updated the documentation accordingly.
- [x] I've read the [CONTRIBUTION](https://github.com/kubernetes/ingress-nginx/blob/main/CONTRIBUTING.md) guide
- [x] I have added tests to cover my changes.
- [x] All new and existing tests passed.


linux-foundation-easycla[bot]: <a href="https://api.easycla.lfx.linuxfoundation.org/v2/repository-provider/github/sign/18706487/72891330/8021/#/?version=2"><img src="https://s3.amazonaws.com/cla-project-logo-prod/cla-not-signed.svg" alt="CLA Not Signed" align="left" height="28" width="328" ></a><br/><br /><ul><li><a href='https://api.easycla.lfx.linuxfoundation.org/v2/repository-provider/github/sign/18706487/72891330/8021/#/?version=2' target='_blank'>:x:</a> - besha100 The commit (f9b6e70d01a2c4edb779511878ab189b3717cf45) is not authorized under a signed CLA. <a href='https://api.easycla.lfx.linuxfoundation.org/v2/repository-provider/github/sign/18706487/72891330/8021/#/?version=2' target='_blank'>Please click here to be authorized</a>. For further assistance with EasyCLA, <a href='https://jira.linuxfoundation.org/servicedesk/customer/portal/4' target='_blank'>please submit a support request ticket</a>.</li></ul>
k8s-ci-robot: Thanks for your pull request. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA).

:memo: **Please follow instructions at <https://git.k8s.io/community/CLA.md#the-contributor-license-agreement> to sign the CLA.**

It may take a couple minutes for the CLA signature to be fully registered; after that, please reply here with a new comment and we'll verify.  Thanks.

---

- If you've already signed a CLA, it's possible we don't have your GitHub username or you're using a different email address.  Check your existing CLA data and verify that your [email is set on your git commits](https://help.github.com/articles/setting-your-email-in-git/).
- If you signed the CLA as a corporation, please sign in with your organization's credentials at <https://identity.linuxfoundation.org/projects/cncf> to be authorized.
- If you have done the above and are still having issues with the CLA being reported as unsigned, please log a ticket with the Linux Foundation Helpdesk: <https://support.linuxfoundation.org/>
- Should you encounter any issues with the Linux Foundation Helpdesk, send a message to the backup e-mail support address at: login-issues@jira.linuxfoundation.org

<!-- need_sender_cla -->

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
	
k8s-ci-robot: @besha100: This issue is currently awaiting triage.

If Ingress contributors determines this is a relevant issue, they will accept it by applying the `triage/accepted` label and provide further guidance.

The `triage/accepted` label can be added by org members by writing `/triage accepted` in a comment.


<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
k8s-ci-robot: Hi @besha100. Thanks for your PR.

I'm waiting for a [kubernetes](https://github.com/orgs/kubernetes/people) member to verify that this patch is reasonable to test. If it is, they should reply with `/ok-to-test` on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should [join the org](https://git.k8s.io/community/community-membership.md#member) to skip this step.

Once the patch is verified, the new status will be reflected by the `ok-to-test` label.

I understand the commands that are listed [here](https://go.k8s.io/bot-commands?repo=kubernetes%2Fingress-nginx).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>

k8s-ci-robot: Welcome @besha100! <br><br>It looks like this is your first PR to <a href='https://github.com/kubernetes/ingress-nginx'>kubernetes/ingress-nginx</a> 🎉. Please refer to our [pull request process documentation](https://git.k8s.io/community/contributors/guide/pull-requests.md) to help your PR have a smooth ride to approval. <br><br>You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. [Here is the bot commands documentation](https://go.k8s.io/bot-commands). <br><br>You can also check if kubernetes/ingress-nginx has [its own contribution guidelines](https://github.com/kubernetes/ingress-nginx/tree/master/CONTRIBUTING.md). <br><br>You may want to refer to our [testing guide](https://git.k8s.io/community/contributors/devel/sig-testing/testing.md) if you run into trouble with your tests not passing. <br><br>If you are having difficulty getting your pull request seen, please follow the [recommended escalation practices](https://github.com/kubernetes/community/blob/master/contributors/guide/pull-requests.md#why-is-my-pull-request-not-getting-reviewed). Also, for tips and tricks in the contribution process you may want to read the [Kubernetes contributor cheat sheet](https://git.k8s.io/community/contributors/guide/contributor-cheatsheet/README.md). We want to make sure your contribution gets all the attention it needs! <br><br>Thank you, and welcome to Kubernetes. :smiley:
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes/ingress-nginx/pull/8021#" title="Author self-approved">besha100</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **thockin** after the PR has been reviewed.
You can assign the PR to them by writing `/assign @thockin` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes%2Fingress-nginx).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes/ingress-nginx/blob/main/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["thockin"]} -->
besha100: /assign @thockin
iamNoah1: /triage accepted 
/priority important-longterm
/kind bug
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes/ingress-nginx/pull/8021#" title="Author self-approved">besha100</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please ask for approval from **thockin** after the PR has been reviewed.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes%2Fingress-nginx).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes/ingress-nginx/blob/main/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["thockin"]} -->
besha100: @iamNoah1 Totally makes sense, just fixed the unnecessary commented lines. 
I also tested the fix with this issue [6438](https://github.com/kubernetes/ingress-nginx/issues/6438) with logging to a custom direcory. The he default Modsecurity config " /etc/nginx/modsecurity/modsecurity.conf" has these

SecAuditLogType Concurrent
SecAuditLog /var/log/modsec_audit.log
SecAuditLogStorageDir /var/log/audit/

Thus, these settings will override the ones in modsecurity-snippet:

I tested with this config snippet
  config:
    enable-modsecurity: "true"
    enable-owasp-modsecurity-crs: "false"
    modsecurity-snippet: |-
      SecRuleEngine On
      SecAuditEngine RelevantOnly
      SecAuditLogParts ABIJDEFHZ
      SecAuditLogRelevantStatus "^(?:05912300)"
      SecAuditLogType Concurrent
      SecAuditLog /var/tmp/modsec_audit.log
      SecAuditLogStorageDir /var/tmp/
      Include /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf


So modsecurity was still writing to var/log/modsec_audit.log. but when removing  ( modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf; ) line from nginx.config and reloading nginx, modsecurity started to log  /var/tmp/modsec_audit.log instead
besha100: /auto-cc
besha100: @iamNoah1 I have updatede the PR descrition above in "How Has This Been Tested?" with all the testing steps. Could you please review. Thanks :)
iamNoah1: @besha100 When I said e2e testing I meant automatic e2e testing -> extending the test suite, not only manually testing. 
thockin: /assign @rikatz 

Patch:
diff --git a/.github/workflows/ci.yaml b/.github/workflows/ci.yaml
index 67a9ac74d..b7cf3bce1 100644
--- a/.github/workflows/ci.yaml
+++ b/.github/workflows/ci.yaml
@@ -51,7 +51,7 @@ jobs:
           # G601 for zz_generated.deepcopy.go
           # G306 TODO: Expect WriteFile permissions to be 0600 or less
           # G307 TODO: Deferring unsafe method "Close"
-          args: -exclude=G601,G104,G204,G304,G306,G307 -tests=false -exclude-dir=test -exclude-dir=images/  -exclude-dir=docs/ ./...
+          args: -exclude=G109,G601,G104,G204,G304,G306,G307 -tests=false -exclude-dir=test -exclude-dir=images/  -exclude-dir=docs/ ./...
 
   build:
     name: Build
diff --git a/OWNERS b/OWNERS
index 1d90c4cd5..20082fb1f 100644
--- a/OWNERS
+++ b/OWNERS
@@ -1,9 +1,7 @@
 # See the OWNERS docs: https://github.com/kubernetes/community/blob/master/contributors/guide/owners.md
 
 approvers:
-- ingress-nginx-admins
 - ingress-nginx-maintainers
-- sig-network-leads
 
 reviewers:
 - ingress-nginx-reviewers
diff --git a/OWNERS_ALIASES b/OWNERS_ALIASES
index f147d0f89..286a0d998 100644
--- a/OWNERS_ALIASES
+++ b/OWNERS_ALIASES
@@ -9,16 +9,15 @@ aliases:
   ingress-nginx-admins:
   - bowei
   - rikatz
+  - strongjz
 
   ingress-nginx-maintainers:
   - ElvinEfendi
-  - justinsb
   - rikatz
   - strongjz
 
   ingress-nginx-reviewers:
   - ElvinEfendi
-  - cmluciano
   - rikatz
   - strongjz
   - tao12345666333
@@ -31,5 +30,10 @@ aliases:
   - ChiefAlexander
   - cpanato
 
+  ingress-nginx-docs-maintainers:
+  - IamNoah1
+  - longwuyuan
+  - tao12345666333
+
   ingress-nginx-kube-webhook-certgen-reviewers:
   - invidian
diff --git a/cloudbuild.yaml b/cloudbuild.yaml
index 22f4771bb..a9d4a214c 100644
--- a/cloudbuild.yaml
+++ b/cloudbuild.yaml
@@ -4,7 +4,7 @@ timeout: 1800s
 options:
   substitution_option: ALLOW_LOOSE
 steps:
-  - name: 'gcr.io/k8s-testimages/gcb-docker-gcloud:v20210722-085d930'
+  - name: 'gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20211118-2f2d816b90'
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/docs/OWNERS b/docs/OWNERS
index 1d3805a73..e8b886e5b 100644
--- a/docs/OWNERS
+++ b/docs/OWNERS
@@ -1,4 +1,7 @@
 # See the OWNERS docs: https://github.com/kubernetes/community/blob/master/contributors/guide/owners.md
 
+approvers:
+- ingress-nginx-docs-maintainers
+
 labels:
 - area/docs
\ No newline at end of file
diff --git a/docs/index.md b/docs/index.md
index 6aa2ea81c..b06141814 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -1,31 +1,35 @@
-# Welcome
+# Overview
 
 This is the documentation for the NGINX Ingress Controller.
 
-It is built around the [Kubernetes Ingress resource](http://kubernetes.io/docs/user-guide/ingress/), using a [ConfigMap](https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap/#understanding-configmaps-and-pods) to store the NGINX configuration.
+It is built around the [Kubernetes Ingress resource](https://kubernetes.io/docs/concepts/services-networking/ingress/), using a [ConfigMap](https://kubernetes.io/docs/concepts/configuration/configmap/) to store the controller configuration.
 
-Learn more about using Ingress on [k8s.io](http://kubernetes.io/docs/user-guide/ingress/).
+You can learn more about using [Ingress](http://kubernetes.io/docs/user-guide/ingress/) in the official [Kubernetes documentation](https://docs.k8s.io).
 
 ## Getting Started
 
 See [Deployment](./deploy/) for a whirlwind tour that will get you started.
 
 
-# FAQ - Migration to apiVersion networking.k8s.io/v1
+# FAQ - Migration to apiVersion `networking.k8s.io/v1`
 
-- Please read this [official blog on deprecated ingress api versions](https://kubernetes.io/blog/2021/07/26/update-with-ingress-nginx/) If you are using ingress objects in your pre K8s v1.22 cluster, and you upgrade to K8s v1.22, then this document may be relevant to you.
+If you are using Ingress objects in your cluster (running Kubernetes older than v1.22), and you plan to upgrade to Kubernetess v1.22, this section is relevant to you.
+
+- Please read this [official blog on deprecated Ingress API versions](https://kubernetes.io/blog/2021/07/26/update-with-ingress-nginx/) 
 
 - Please read this [official documentation on the IngressClass object](https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-class)
 
-## What is an ingressClass and why is it important for users of Ingress-NGINX controller now ?
+## What is an IngressClass and why is it important for users of Ingress-NGINX controller now ?
 
 IngressClass is a Kubernetes resource. See the description below.
-Its important because until now, a default install of the Ingress-NGINX controller did not require a ingressClass object. But from version 1.0.0 of the Ingress-NGINX Controller, a ingressclass object is required.
+Its important because until now, a default install of the Ingress-NGINX controller did not require any IngressClass object. From version 1.0.0 of the Ingress-NGINX Controller, an IngressClass object is required.
 
-On clusters with more than one instance of the Ingress-NGINX controller, all instances of the controllers must be aware of which Ingress object they must serve. The ingressClass field of a ingress object is the way to let the controller know about that. 
+On clusters with more than one instance of the Ingress-NGINX controller, all instances of the controllers must be aware of which Ingress objects they serve. The `ingressClassName` field of an Ingress is the way to let the controller know about that. 
 
 ```
-_$ k explain ingressClass                                                           
+kubectl explain ingressclass                                                           
+```
+```
 KIND:     IngressClass                                                               
 VERSION:  networking.k8s.io/v1                     
 
@@ -63,23 +67,25 @@ FIELDS:
 
 There are 2 reasons primarily.
 
-(Reason #1) Until K8s version 1.21, it was possible to create a ingress resource, with the "apiVersion:" field set to a value like:
+_(Reason #1)_ Until K8s version 1.21, it was possible to create an Ingress resource using deprecated versions of the Ingress API, such as:
 
-  - extensions/v1beta1
-  - networking.k8s.io/v1beta1
+  - `extensions/v1beta1`
+  - `networking.k8s.io/v1beta1`
 
-You would get a message about deprecation but the ingress resource would get created.
+You would get a message about deprecation, but the Ingress resource would get created.
 
-From K8s version 1.22 onwards, you can ONLY set the "apiVersion:" field of a ingress resource, to the value "networking.k8s.io/v1". The reason is [official blog on deprecated ingress api versions](https://kubernetes.io/blog/2021/07/26/update-with-ingress-nginx/).
+From K8s version 1.22 onwards, you can **only** access the Ingress API via the stable, `networking.k8s.io/v1` API. The reason is explained in the [official blog on deprecated ingress API versions](https://kubernetes.io/blog/2021/07/26/update-with-ingress-nginx/).
 
-(Reason #2) When you upgrade to K8s version v1.22, while you are already using the Ingress-NGINX controller, there are several scenarios where the old existing ingress objects will not work. Read this FAQ to check which scenario matches your use case.
+_(Reason #2)_ if you are already using the Ingress-NGINX controller and then upgrade to K8s version v1.22 , there are several scenarios where your existing Ingress objects will not work how you expect. Read this FAQ to check which scenario matches your use case.
 
 ## What is ingressClassName field ?
 
-ingressClassName is a field in the specs of a ingress object.
+`ingressClassName` is a field in the specs of an Ingress object.
 
+```shell
+kubectl explain ingress.spec.ingressClassName
+```
 ```
-% k explain ingress.spec.ingressClassName
 KIND:     Ingress
 VERSION:  networking.k8s.io/v1
 
@@ -97,25 +103,27 @@ DESCRIPTION:
      for this field. For more information, refer to the IngressClass
      documentation.
 ```
- the spec.ingressClassName behavior has precedence over the annotation.
 
+The `.spec.ingressClassName` behavior has precedence over the deprecated `kubernetes.io/ingress.class` annotation.
 
 
-## I have only one instance of the Ingresss-NGINX controller in my cluster. What should I do ?
 
-- If you have only one instance of the Ingress-NGINX controller running in your cluster, and you still want to use ingressclass, you should add the annotation "ingressclass.kubernetes.io/is-default-class" in your ingress class, so any new Ingress objects will have this one as default ingressClass.
+## I have only one instance of the Ingress-NGINX controller in my cluster. What should I do ?
 
-In this case, you need to make your Controller aware of the objects. If you have several Ingress objects and they don't yet have the [ingressClassName](https://kubernetes.io/docs/reference/generated/kubernetes-api/v1.20/#ingress-v1-networking-k8s-io) field, or the ingress annotation (`kubernetes.io/ingress.class`),  then you should start your ingress-controller with the flag [--watch-ingress-without-class=true](## What is the flag '--watch-ingress-without-class' ?) .
+- If you have only one instance of the Ingress-NGINX controller running in your cluster, and you still want to use IngressClass, you should add the annotation `ingressclass.kubernetes.io/is-default-class` in your IngressClass, so that any new Ingress objects will have this one as default IngressClass.
 
-You can configure your helm chart installation's values file with `.controller.watchIngressWithoutClass: true`. 
+In this case, you need to make your controller aware of the objects. If you have any Ingress objects that don't yet have either the [`.spec.ingressClassName`](https://kubernetes.io/docs/reference/kubernetes-api/service-resources/ingress-v1/#IngressSpec) field set in their manifest, or the ingress annotation (`kubernetes.io/ingress.class`), then you should start your Ingress-NGINX controller with the flag `--watch-ingress-without-class=true`.
 
-We highly recommend that you create the ingressClass as shown below:
+You can configure your Helm chart installation's values file with `.controller.watchIngressWithoutClass: true`. 
+
+We recommend that you create the IngressClass as shown below:
 ```
+---
 apiVersion: networking.k8s.io/v1
 kind: IngressClass
 metadata:
   labels:
-    app.kubernetes.io/component: controller
+    app.kubernetes.io/component: controller 
   name: nginx
   annotations:
     ingressclass.kubernetes.io/is-default-class: "true"
@@ -124,16 +132,19 @@ spec:
 ```
 And add the value "spec.ingressClassName=nginx" in your Ingress objects
 
+
 ## I have multiple ingress objects in my cluster. What should I do ?
-- If you don't care about ingressClass, or you have a lot of ingress objects without ingressClass configuration, you can run the ingress-controller with the flag `--watch-ingress-without-class=true`.
+- If you have lot of ingress objects without ingressClass configuration, you can run the ingress-controller with the flag `--watch-ingress-without-class=true`.
 
-## What is the flag '--watch-ingress-without-class' ?
-- Its a flag that is passed,as an argument, to the ingress-controller executable, in the pod spec. It looks like this ;
+
+### What is the flag '--watch-ingress-without-class' ?
+- Its a flag that is passed,as an argument, to the `nginx-ingress-controller` executable. In the configuration, it looks like this ;
 ```
 ...
 ...
 args:
   - /nginx-ingress-controller
+  - --watch-ingress-without-class=true
   - --publish-service=$(POD_NAMESPACE)/ingress-nginx-dev-v1-test-controller
   - --election-id=ingress-controller-leader
   - --controller-class=k8s.io/ingress-nginx
@@ -141,45 +152,56 @@ args:
   - --validating-webhook=:8443
   - --validating-webhook-certificate=/usr/local/certificates/cert
   - --validating-webhook-key=/usr/local/certificates/key
-  - --watch-ingress-without-class=true
 ...
 ...
 ```
 
 ## I have more than one controller in my cluster and already use the annotation ?
+
 No problem. This should still keep working, but we highly recommend you to test!
 
-## I have more than one controller running in my cluster, and I want to use the new spec ?
-In this scenario, you need to create multiple ingressClasses (see example one). But be aware that ingressClass works in a very specific way: you will need to change the .spec.controller value in your IngressClass and point the controller to the relevant ingressClass. Let's see some example, supposing that you have two Ingress Classes:
+Even though `kubernetes.io/ingress.class` is deprecated, the Ingress-NGINX controller still understands that annotation.
+If you want to follow good practice, you should consider migrating to use IngressClass and `.spec.ingressClassName`.
+
+## I have more than one controller running in my cluster, and I want to use the new API ?
+
+In this scenario, you need to create multiple IngressClasses (see example one). But be aware that IngressClass works in a very specific way: you will need to change the `.spec.controller` value in your IngressClass and configure the controller to expect the exact same value.
 
-- Ingress-Nginx-IngressClass-1 with .spec.controller equals to "k8s.io/ingress-nginx1"
-- Ingress-Nginx-IngressClass-2 with .spec.controller equals to "k8s.io/ingress-nginx2"
+Let's see some example, supposing that you have three IngressClasses:
+
+- IngressClass `ingress-nginx-one`, with `.spec.controller` equal to `example.com/ingress-nginx1`
+- IngressClass `ingress-nginx-two`, with `.spec.controller` equal to `example.com/ingress-nginx2`
+- IngressClass `ingress-nginx-three`, with `.spec.controller` equal to `example.com/ingress-nginx1`
+
+(for private use, you can also use a controller name that doesn't contain a `/`; for example: `ingress-nginx1`)
 
 When deploying your ingress controllers, you will have to change the `--controller-class` field as follows:
 
-- Ingress-Nginx-Controller-nginx1 with `k8s.io/ingress-nginx1`
-- Ingress-Nginx-Controller-nginx2 with `k8s.io/ingress-nginx2`
+- Ingress-Nginx A, configured to use controller class name `example.com/ingress-nginx1`
+- Ingress-Nginx B, configured to use controller class name `example.com/ingress-nginx2`
+
+Then, when you create an Ingress object with its `ingressClassName` set to `ingress-nginx-two`, only controllers looking for the `example.com/ingress-nginx2` controller class pay attention to the new object. Given that Ingress-Nginx B is set up that way, it will serve that object, whereas Ingress-Nginx A ignores the new Ingress.
 
-Then, when you create an Ingress Object with IngressClassName = `ingress-nginx2`, it will look for controllers with `controller-class=k8s.io/ingress-nginx2` and as `Ingress-Nginx-Controller-nginx2` is watching objects that points to `ingressClass="k8s.io/ingress-nginx2`, it will serve that object, while `Ingress-Nginx-Controller-nginx1` will ignore the ingress object.
+Bear in mind that, if you start Ingress-Nginx B with the command line argument `--watch-ingress-without-class=true`, then it will serve:
 
-Bear in mind that, if your `Ingress-Nginx-Controller-nginx2` is started with the flag `--watch-ingress-without-class=true`, then it will serve:
+1. Ingresses without any `ingressClassName` set
+2. Ingresses where the the deprecated annotation (`kubernetes.io/ingress.class`) matches the value set in the command line argument `--ingress-class`
+3. Ingresses that refer to any IngressClass that has the same `spec.controller` as configured in `--controller-class`
 
-- objects without ingress-class
-- objects with the annotation configured in flag `--ingress-class` and same class value
-- and also objects pointing to the ingressClass that have the same .spec.controller as configured in `--controller-class`
+If you start Ingress-Nginx B with the command line argument `--watch-ingress-without-class=true` and you run Ingress-Nginx A with the command line argument `--watch-ingress-without-class=false` then this is a supported configuration. If you have two Ingress-NGINX controllers for the same cluster, both running with `--watch-ingress-without-class=true` then there is likely to be a conflict.
 
+## I am seeing this error message in the logs of the Ingress-NGINX controller: "ingress class annotation is not equal to the expected by Ingress Controller". Why ?
 
-## I am seeing this error message in the logs of the Ingress-NGINX controller "ingress class annotation is not equal to the expected by Ingress Controller". Why ?
-- It is highly likely that you will also see the name of the ingress resource in the same error message. This error messsage has been observed on use the deprecated annotation, to spec the ingressClass, in a ingress resource manifest. It is recommended to use the ingress.spec.ingressClassName field, of the ingress resource, to spec the name of the ingressClass of the ingress resource being configured.
+- It is highly likely that you will also see the name of the ingress resource in the same error message. This error messsage has been observed on use the deprecated annotation (`kubernetes.io/ingress.class`) in a Ingress resource manifest. It is recommended to use the `.spec.ingressClassName` field of the Ingress resource, to specify the name of the IngressClass of the Ingress you are defining.
 
 ## How to easily install multiple instances of the ingress-NGINX controller in the same cluster ?
 - Create a new namespace
   ```
   kubectl create namespace ingress-nginx-2
   ```
-- Use helm to install the additional instance of the ingress controller
-- Ensure you have helm working (refer to helm documentation)
-- We have to assume that you have the helm repo for the ingress-NGINX controller already added to your helm config. But, if you have not added the helm repo then you can do this to add the repo to your helm config;
+- Use Helm to install the additional instance of the ingress controller
+- Ensure you have Helm working (refer to the [Helm documentation](https://helm.sh/docs/))
+- We have to assume that you have the helm repo for the ingress-NGINX controller already added to your Helm config. But, if you have not added the helm repo then you can do this to add the repo to your helm config;
   ```
   helm repo add ingress-nginx https://kubernetes.github.io/ingress-nginx
   ```
@@ -187,13 +209,13 @@ Bear in mind that, if your `Ingress-Nginx-Controller-nginx2` is started with the
   ```
   helm repo update
   ```
-- Now you install the additional instance of the ingress-NGINX controller like this ;
+- Now, install an additional instance of the ingress-NGINX controller like this ;
   ```
   helm install ingress-nginx-2 ingress-nginx/ingress-nginx  \
   --namespace ingress-nginx-2 \
-  --set controller.ingressClassResource.name=nginx-2 \
-  --set controller.ingressClassResource.controllerValue="k8s.io/ingress-nginx-2" \
+  --set controller.ingressClassResource.name=nginx-two \
+  --set controller.ingressClassResource.controllerValue="example.com/ingress-nginx-2" \
   --set controller.ingressClassResource.enabled=true \
   --set controller.ingressClassByName=true
   ```
-- If you need to install yet another instance, then repeat the procedure to create a new namespace, change the values like names & namespaces (for example from "-2" to "-3"), or anything else that meets your needs.
+- If you need to install yet another instance, then repeat the procedure to create a new namespace, change the values such as names & namespaces (for example from "-2" to "-3"), or anything else that meets your needs.
diff --git a/docs/user-guide/multiple-ingress.md b/docs/user-guide/multiple-ingress.md
index b0125c3b2..f13574044 100644
--- a/docs/user-guide/multiple-ingress.md
+++ b/docs/user-guide/multiple-ingress.md
@@ -2,13 +2,14 @@
 
 By default, deploying multiple Ingress controllers (e.g., `ingress-nginx` & `gce`) will result in all controllers simultaneously racing to update Ingress status fields in confusing ways.
 
-To fix this problem, you can either use [IngressClasses](https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-class) (preferred) or use the `kubernetes.io/ingress.class` annotation (in deprecation).
+To fix this problem, use [IngressClasses](https://kubernetes.io/docs/concepts/services-networking/ingress/#ingress-class), the `kubernetes.io/ingress.class` annotation  is deprecated from kubernetes v1.22+.
 
 ## Using IngressClasses
 
 If all ingress controllers respect IngressClasses (e.g. multiple instances of ingress-nginx v1.0), you can deploy two Ingress controllers by granting them control over two different IngressClasses, then selecting one of the two IngressClasses with `ingressClassName`.
+When two or more 
 
-First, ensure the `--controller-class=` is set to something different on each ingress controller:
+First, ensure the `--controller-class=` and `--ingress-class` are set to something different on each ingress controller:
 
 ```yaml
 # ingress-nginx Deployment/Statfulset
@@ -19,7 +20,8 @@ spec:
          - name: ingress-nginx-internal-controller
            args:
              - /nginx-ingress-controller
-             - '--controller-class=k8s.io/internal-ingress-nginx'
+             - '--controller-class=k8s.io/internal-nginx'
+             - '--ingress-class=k8s.io/internal-nginx'
             ...
 ```
 
diff --git a/images/cfssl/cloudbuild.yaml b/images/cfssl/cloudbuild.yaml
index d80d2aeaf..a71001c89 100644
--- a/images/cfssl/cloudbuild.yaml
+++ b/images/cfssl/cloudbuild.yaml
@@ -2,7 +2,7 @@ timeout: 600s
 options:
   substitution_option: ALLOW_LOOSE
 steps:
-  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f
+  - name: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20211118-2f2d816b90
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/images/custom-error-pages/cloudbuild.yaml b/images/custom-error-pages/cloudbuild.yaml
index 9b913b3e6..45cff0021 100644
--- a/images/custom-error-pages/cloudbuild.yaml
+++ b/images/custom-error-pages/cloudbuild.yaml
@@ -2,7 +2,7 @@ timeout: 1800s
 options:
   substitution_option: ALLOW_LOOSE
 steps:
-  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f
+  - name: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20211118-2f2d816b90
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/images/echo/cloudbuild.yaml b/images/echo/cloudbuild.yaml
index 3e4d67d06..2f773cebc 100644
--- a/images/echo/cloudbuild.yaml
+++ b/images/echo/cloudbuild.yaml
@@ -2,7 +2,7 @@ timeout: 1200s
 options:
   substitution_option: ALLOW_LOOSE
 steps:
-  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f
+  - name: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20211118-2f2d816b90
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/images/fastcgi-helloserver/cloudbuild.yaml b/images/fastcgi-helloserver/cloudbuild.yaml
index d80d2aeaf..a71001c89 100644
--- a/images/fastcgi-helloserver/cloudbuild.yaml
+++ b/images/fastcgi-helloserver/cloudbuild.yaml
@@ -2,7 +2,7 @@ timeout: 600s
 options:
   substitution_option: ALLOW_LOOSE
 steps:
-  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f
+  - name: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20211118-2f2d816b90
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/images/go-grpc-greeter-server/cloudbuild.yaml b/images/go-grpc-greeter-server/cloudbuild.yaml
index 14c64db31..b92b9c877 100644
--- a/images/go-grpc-greeter-server/cloudbuild.yaml
+++ b/images/go-grpc-greeter-server/cloudbuild.yaml
@@ -4,7 +4,7 @@ options:
   # job builds a multi-arch docker image for amd64,arm,arm64 and s390x.
   machineType: N1_HIGHCPU_8
 steps:
-  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20210622-762366a
+  - name: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20211118-2f2d816b90
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/images/httpbin/cloudbuild.yaml b/images/httpbin/cloudbuild.yaml
index 02f28b8ca..b92b9c877 100644
--- a/images/httpbin/cloudbuild.yaml
+++ b/images/httpbin/cloudbuild.yaml
@@ -4,7 +4,7 @@ options:
   # job builds a multi-arch docker image for amd64,arm,arm64 and s390x.
   machineType: N1_HIGHCPU_8
 steps:
-  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f
+  - name: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20211118-2f2d816b90
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/images/kube-webhook-certgen/cloudbuild.yaml b/images/kube-webhook-certgen/cloudbuild.yaml
index 8bdd225c5..d1eb9fe6e 100644
--- a/images/kube-webhook-certgen/cloudbuild.yaml
+++ b/images/kube-webhook-certgen/cloudbuild.yaml
@@ -17,7 +17,7 @@ timeout: 10800s
 options:
   substitution_option: ALLOW_LOOSE
 steps:
-  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f
+  - name: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20211118-2f2d816b90
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/images/nginx/cloudbuild.yaml b/images/nginx/cloudbuild.yaml
index c5aca043e..d507b8f1a 100644
--- a/images/nginx/cloudbuild.yaml
+++ b/images/nginx/cloudbuild.yaml
@@ -4,7 +4,7 @@ options:
   # job builds a multi-arch docker image for amd64,arm,arm64 and s390x.
   machineType: N1_HIGHCPU_32
 steps:
-  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20210722-085d930
+  - name: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20211118-2f2d816b90
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/images/nginx/rootfs/build.sh b/images/nginx/rootfs/build.sh
index ab64d5d94..1aa655523 100755
--- a/images/nginx/rootfs/build.sh
+++ b/images/nginx/rootfs/build.sh
@@ -637,7 +637,7 @@ WITH_FLAGS="--with-debug \
 
 # "Combining -flto with -g is currently experimental and expected to produce unexpected results."
 # https://gcc.gnu.org/onlinedocs/gcc/Optimize-Options.html
-CC_OPT="-g -Og -fPIE -fstack-protector-strong \
+CC_OPT="-g -O2 -fPIE -fstack-protector-strong \
   -Wformat \
   -Werror=format-security \
   -Wno-deprecated-declarations \
@@ -656,7 +656,7 @@ if [[ ${ARCH} != "aarch64" ]]; then
 fi
 
 if [[ ${ARCH} == "x86_64" ]]; then
-  CC_OPT+=' -m64 -mtune=native'
+  CC_OPT+=' -m64 -mtune=generic'
 fi
 
 WITH_MODULES=" \
diff --git a/internal/ingress/controller/template/template.go b/internal/ingress/controller/template/template.go
index ae5ec259a..e5be1490a 100644
--- a/internal/ingress/controller/template/template.go
+++ b/internal/ingress/controller/template/template.go
@@ -1537,7 +1537,7 @@ func buildModSecurityForLocation(cfg config.Configuration, location *ingress.Loc
 `, location.ModSecurity.TransactionID))
 	}
 
-	if !isMSEnabled {
+	if !isMSEnabled && location.ModSecurity.Snippet == "" {
 		buffer.WriteString(`modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;
 `)
 	}
diff --git a/rootfs/etc/nginx/template/nginx.tmpl b/rootfs/etc/nginx/template/nginx.tmpl
index c6e978ffe..0cc8d3cab 100755
--- a/rootfs/etc/nginx/template/nginx.tmpl
+++ b/rootfs/etc/nginx/template/nginx.tmpl
@@ -153,10 +153,10 @@ http {
     modsecurity_rules '
       {{ $all.Cfg.ModsecuritySnippet }}
     ';
-    {{ end }}
-
+    {{ else }}
     modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;
-
+    {{ end }}
+    
     {{ if $all.Cfg.EnableOWASPCoreRules }}
     modsecurity_rules_file /etc/nginx/owasp-modsecurity-crs/nginx-modsecurity.conf;
     {{ end }}


Test patch:
diff --git a/images/test-runner/cloudbuild.yaml b/images/test-runner/cloudbuild.yaml
index 7258ab384..253b99799 100644
--- a/images/test-runner/cloudbuild.yaml
+++ b/images/test-runner/cloudbuild.yaml
@@ -2,7 +2,7 @@ timeout: 3600s
 options:
   substitution_option: ALLOW_LOOSE
 steps:
-  - name: gcr.io/k8s-testimages/gcb-docker-gcloud:v20201130-750d12f
+  - name: gcr.io/k8s-staging-test-infra/gcb-docker-gcloud:v20211118-2f2d816b90
     entrypoint: bash
     env:
       - DOCKER_CLI_EXPERIMENTAL=enabled
diff --git a/internal/ingress/controller/template/template_test.go b/internal/ingress/controller/template/template_test.go
index cfb65c08e..b65e33c32 100644
--- a/internal/ingress/controller/template/template_test.go
+++ b/internal/ingress/controller/template/template_test.go
@@ -1781,8 +1781,8 @@ func TestModSecurityForLocation(t *testing.T) {
 		{"configmap enabled, configmap OWASP enabled, annotation enabled, OWASP disabled", true, true, true, true, false, "", "", ""},
 		{"configmap disabled, annotation enabled, OWASP disabled", false, false, true, true, false, "", "", fmt.Sprintf("%v%v", loadModule, modSecCfg)},
 		{"configmap disabled, annotation disabled, OWASP disabled", false, false, false, true, false, "", "", ""},
-		{"configmap disabled, annotation enabled, OWASP disabled", false, false, true, true, false, testRule, "", fmt.Sprintf("%v%v%v", loadModule, modsecRule, modSecCfg)},
-		{"configmap disabled, annotation enabled, OWASP enabled", false, false, true, true, false, testRule, "", fmt.Sprintf("%v%v%v", loadModule, modsecRule, modSecCfg)},
+		{"configmap disabled, annotation enabled, OWASP disabled", false, false, true, true, false, testRule, "", fmt.Sprintf("%v%v", loadModule, modsecRule)},
+		{"configmap disabled, annotation enabled, OWASP enabled", false, false, true, true, false, testRule, "", fmt.Sprintf("%v%v", loadModule, modsecRule)},
 	}
 
 	for _, testCase := range testCases {
diff --git a/test/e2e/annotations/modsecurity/modsecurity.go b/test/e2e/annotations/modsecurity/modsecurity.go
index f88d6541e..4de85818d 100644
--- a/test/e2e/annotations/modsecurity/modsecurity.go
+++ b/test/e2e/annotations/modsecurity/modsecurity.go
@@ -342,4 +342,41 @@ var _ = framework.DescribeAnnotation("modsecurity owasp", func() {
 			Expect().
 			Status(http.StatusOK)
 	})
+
+	ginkgo.It("should disable default modsecurity conf setting when modsecurity-snippet is specified", func() {
+		host := "modsecurity.foo.com"
+		nameSpace := f.Namespace
+
+		snippet := `SecRuleEngine On
+		SecRequestBodyAccess On
+		SecAuditEngine RelevantOnly
+		SecAuditLogParts ABIJDEFHZ
+		SecAuditLogType Concurrent
+		SecAuditLog /var/tmp/modsec_audit.log
+		SecAuditLogStorageDir /var/tmp/
+		SecRule REQUEST_HEADERS:User-Agent \"block-ua\" \"log,deny,id:107,status:403,msg:\'UA blocked\'\"`
+
+		annotations := map[string]string{
+			"nginx.ingress.kubernetes.io/enable-modsecurity":  "true",
+			"nginx.ingress.kubernetes.io/modsecurity-snippet": snippet,
+		}
+		f.UpdateNginxConfigMapData("annotation-value-word-blocklist", "load_module, lua_package, _by_lua, location, root, {, }")
+		// Sleep a while just to guarantee that the configmap is applied
+		framework.Sleep()
+		ing := framework.NewSingleIngress(host, "/", host, nameSpace, framework.EchoService, 80, annotations)
+		f.EnsureIngress(ing)
+
+		f.WaitForNginxServer(host,
+			func(server string) bool {
+				return !strings.Contains(server, "modsecurity_rules_file /etc/nginx/modsecurity/modsecurity.conf;") &&
+					strings.Contains(server, "SecAuditLog /var/tmp/modsec_audit.log")
+			})
+
+		f.HTTPTestClient().
+			GET("/").
+			WithHeader("Host", host).
+			WithHeader("User-Agent", "block-ua").
+			Expect().
+			Status(http.StatusForbidden)
+	})
 })