Repository: kubernetes-sigs/cloud-provider-azure
Pull request #2828 (2022-11-25 07:17:25)

Feat:Add annotation service.beta.kubernetes.io/port_%d_health-probe_only
<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?
/kind feature

<!--
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature
/kind design

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

#### What this PR does / why we need it:
add a new annotation service.beta.kubernetes.io/port_%d_health-probe_only
with this annotation applied, no lbrules will be create for the corresponding port. 
The user can create health probe rules without exposing health probe endpoint to public internet

behavior change in health probe rule generation:
the health probe rule will not be overwritten

the name of health probe rule has not changed because we don't want to trigger unnecessary lb update event after version upgrade.
#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

#### Special notes for your reviewer:
docs will be updated after the changes has been cherry-picked.
#### Does this PR introduce a user-facing change?
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

For more information on release notes see: https://git.k8s.io/community/contributors/guide/release-notes.md
-->
```release-note
Add annotation service.beta.kubernetes.io/port_%d_no_lb_rule, with this annotation in svc manifest, no loud balancing rules will be created for the corresponding port and health status port will not be exposed to public internet.
```

#### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

<!--
This section can be blank if this pull request does not require a release note.

When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.

See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files

Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
-->
```docs

```


netlify[bot]: ### <span aria-hidden="true">👷</span> Deploy Preview for *kubernetes-sigs-cloud-provide-azure* processing.


|  Name | Link |
|---------------------------------|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | 447f7a62477081b571dc3f1c80bfa40f4845801d |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/kubernetes-sigs-cloud-provide-azure/deploys/63806c078958ca00094b7c24 |
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2828#" title="Author self-approved">MartinForReal</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fcloud-provider-azure).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/cloud-provider-azure/blob/master/OWNERS)~~ [MartinForReal]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
k8s-ci-robot: @MartinForReal: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-unit/1596040350185558016) | true | `/test pull-cloud-provider-azure-unit`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-unit/1596040350185558016) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596040350101671936) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-unit/1596040350185558016) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596040350101671936) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1596040350013591552) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-unit/1596040350185558016) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596040350101671936) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1596040350013591552) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1596040349967454208) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-unit/1596040350185558016) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596040350101671936) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1596040350013591552) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1596040349967454208) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1596040350147809280) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
coveralls: 
[![Coverage Status](https://coveralls.io/builds/54559437/badge)](https://coveralls.io/builds/54559437)

Coverage decreased (-0.03%) to 79.567% when pulling **3001831c368cde3cda51259c543a5049ae6355e1 on MartinForReal:fix_health_probe_name** into **ff90c62695d24bae5f9227a4e4308f7d952651a5 on kubernetes-sigs:master**.

MartinForReal: /cc @rainest PTAL Thanks!
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596040350101671936) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1596040350013591552) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1596040349967454208) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 3001831c368cde3cda51259c543a5049ae6355e1 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1596041768023887872) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596040350101671936) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1596040350013591552) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1596040349967454208) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1596066572550541312) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596040350101671936) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1596040349967454208) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1596066572550541312) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1596066572370186240) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 447f7a62477081b571dc3f1c80bfa40f4845801d | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1596040349967454208) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1596066572550541312) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1596066572370186240) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596066572496015360) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1596066572550541312) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1596066572370186240) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596066572496015360) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1596066572324048896) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: PR needs rebase.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1596066572550541312) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596066572496015360) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1596066572324048896) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1600035908248145920) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1596066572550541312) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596066572496015360) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1600035908248145920) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1600035908193619968) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1596066572550541312) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1596066572496015360) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1600035908248145920) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1600035908193619968) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-check | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-check/1600035908134899712) | true | `/test pull-cloud-provider-azure-check`
pull-cloud-provider-azure-unit | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-unit/1600035908445278208) | true | `/test pull-cloud-provider-azure-unit`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 51a04045f68dce2b0a0df854ef749fdea6256847 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1596066572550541312) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1600035908248145920) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1600035908193619968) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-check | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-check/1600035908134899712) | true | `/test pull-cloud-provider-azure-check`
pull-cloud-provider-azure-unit | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-unit/1600035908445278208) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1600035908348809216) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1600035908248145920) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1600035908193619968) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-unit | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-unit/1600035908445278208) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1600035908348809216) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1600035908399140864) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-check | 46bc68440d00a622bf4cb601b833e65db2b0a611 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-check/1600036349644115968) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1600035908248145920) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1600035908193619968) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1600035908348809216) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1600035908399140864) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-check | 69fb5fae75e589704d2d0293330be8a65e5e84d2 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-check/1600040034449231872) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1600035908248145920) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1600035908193619968) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1600035908348809216) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1600035908399140864) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-check | 69fb5fae75e589704d2d0293330be8a65e5e84d2 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-check/1600040034449231872) | true | `/test pull-cloud-provider-azure-check`
pull-cloud-provider-azure-e2e-capz | 69fb5fae75e589704d2d0293330be8a65e5e84d2 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-capz/1600040034612809728) | true | `/test pull-cloud-provider-azure-e2e-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1600035908248145920) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1600035908193619968) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-capz/1600035908348809216) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 4a305c55a054b6fc98b62d34789b61092ab76c25 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1600035908399140864) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-capz | 69fb5fae75e589704d2d0293330be8a65e5e84d2 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-e2e-capz/1600040034612809728) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-check | 6fab01399eb82cb706d9cdcbe7f40430d6ada5a6 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2828/pull-cloud-provider-azure-check/1600056215839182848) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2828). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->

Patch:
diff --git a/pkg/consts/consts.go b/pkg/consts/consts.go
index d832b28d4..04838b5d1 100644
--- a/pkg/consts/consts.go
+++ b/pkg/consts/consts.go
@@ -300,7 +300,7 @@ const (
 
 	// ServiceAnnotationLoadBalancerHealthProbeRequestPath determines the request path of the load balancer health probe.
 	// This is only useful for the HTTP and HTTPS, and would be ignored when using TCP. If not set,
-	// `/healthz` would be configured by default.
+	// `/` would be configured by default.
 	ServiceAnnotationLoadBalancerHealthProbeRequestPath = "service.beta.kubernetes.io/azure-load-balancer-health-probe-request-path"
 
 	// ServiceAnnotationAzurePIPTags determines what tags should be applied to the public IP of the service. The cluster name
@@ -428,9 +428,19 @@ const RateLimited = "rate limited"
 // CreatedByTag tag key for CSI drivers
 const CreatedByTag = "k8s-azure-created-by"
 
+// port specific
+const (
+	PortAnnotationPrefixPattern            = "service.beta.kubernetes.io/port_%d_%s"
+	PortAnnotationNoLBRule      PortParams = "no_lb_rule"
+	// NoHealthProbeRule determines whether the port is only used for health probe. no lb probe rule will be created.
+	PortAnnotationNoHealthProbeRule PortParams = "no_probe_rule"
+)
+
+type PortParams string
+
 // health probe
 const (
-	HealthProbeAnnotationPrefixPattern = "service.beta.kubernetes.io/port_%d_health-probe_"
+	HealthProbeAnnotationPrefixPattern = "health-probe_%s"
 
 	// HealthProbeParamsProtocol determines the protocol for the health probe params.
 	// It always takes priority over spec.appProtocol or any other specified protocol
diff --git a/pkg/consts/helpers.go b/pkg/consts/helpers.go
index 141aff688..b800e9c64 100644
--- a/pkg/consts/helpers.go
+++ b/pkg/consts/helpers.go
@@ -50,6 +50,16 @@ func GetHealthProbeConfigOfPortFromK8sSvcAnnotation(annotations map[string]strin
 	return GetAttributeValueInSvcAnnotation(annotations, BuildHealthProbeAnnotationKeyForPort(port, key), validators...)
 }
 
+// IsHealthProbeRuleOnK8sServicePortDisabled return if port is for health probe only
+func IsHealthProbeRuleOnK8sServicePortDisabled(annotations map[string]string, port int32) (bool, error) {
+	return expectAttributeInSvcAnnotationBeEqualTo(annotations, BuildAnnotationKeyForPort(port, PortAnnotationNoHealthProbeRule), TrueAnnotationValue), nil
+}
+
+// IsHealthProbeRuleOnK8sServicePortDisabled return if port is for health probe only
+func IsLBRuleOnK8sServicePortDisabled(annotations map[string]string, port int32) (bool, error) {
+	return expectAttributeInSvcAnnotationBeEqualTo(annotations, BuildAnnotationKeyForPort(port, PortAnnotationNoLBRule), TrueAnnotationValue), nil
+}
+
 // Getint32ValueFromK8sSvcAnnotation get health probe configuration for port
 func Getint32ValueFromK8sSvcAnnotation(annotations map[string]string, key string, validators ...Int32BusinessValidator) (*int32, error) {
 	val, err := GetAttributeValueInSvcAnnotation(annotations, key)
@@ -59,9 +69,14 @@ func Getint32ValueFromK8sSvcAnnotation(annotations map[string]string, key string
 	return nil, err
 }
 
+// BuildHealthProbeAnnotationKeyForPort get health probe configuration key for port
+func BuildAnnotationKeyForPort(port int32, key PortParams) string {
+	return fmt.Sprintf(PortAnnotationPrefixPattern, port, string(key))
+}
+
 // BuildHealthProbeAnnotationKeyForPort get health probe configuration key for port
 func BuildHealthProbeAnnotationKeyForPort(port int32, key HealthProbeParams) string {
-	return fmt.Sprintf(HealthProbeAnnotationPrefixPattern, port) + string(key)
+	return BuildAnnotationKeyForPort(port, PortParams(fmt.Sprintf(HealthProbeAnnotationPrefixPattern, key)))
 }
 
 // GetInt32HealthProbeConfigOfPortFromK8sSvcAnnotation get health probe configuration for port
diff --git a/pkg/provider/azure_loadbalancer.go b/pkg/provider/azure_loadbalancer.go
index 6ac852af0..f4a32f30b 100644
--- a/pkg/provider/azure_loadbalancer.go
+++ b/pkg/provider/azure_loadbalancer.go
@@ -2062,7 +2062,7 @@ func lbRuleConflictsWithPort(rule network.LoadBalancingRule, frontendIPConfigID
 // buildHealthProbeRulesForPort
 // for following sku: basic loadbalancer vs standard load balancer
 // for following protocols: TCP HTTP HTTPS(SLB only)
-func (az *Cloud) buildHealthProbeRulesForPort(annotations map[string]string, port v1.ServicePort, lbrule string) (*network.Probe, error) {
+func (az *Cloud) buildHealthProbeRulesForPort(serviceManifest *v1.Service, port v1.ServicePort, lbrule string) (*network.Probe, error) {
 	if port.Protocol == v1.ProtocolUDP || port.Protocol == v1.ProtocolSCTP {
 		return nil, nil
 	}
@@ -2080,7 +2080,7 @@ func (az *Cloud) buildHealthProbeRulesForPort(annotations map[string]string, por
 	var protocol *string
 
 	// 1. Look up port-specific override
-	protocol, err = consts.GetHealthProbeConfigOfPortFromK8sSvcAnnotation(annotations, port.Port, consts.HealthProbeParamsProtocol)
+	protocol, err = consts.GetHealthProbeConfigOfPortFromK8sSvcAnnotation(serviceManifest.Annotations, port.Port, consts.HealthProbeParamsProtocol)
 	if err != nil {
 		return nil, fmt.Errorf("failed to parse annotation %s: %w", consts.BuildHealthProbeAnnotationKeyForPort(port.Port, consts.HealthProbeParamsProtocol), err)
 	}
@@ -2093,7 +2093,7 @@ func (az *Cloud) buildHealthProbeRulesForPort(annotations map[string]string, por
 
 	// 3. If protocol is still nil, check the global annotation
 	if protocol == nil {
-		protocol, err = consts.GetAttributeValueInSvcAnnotation(annotations, consts.ServiceAnnotationLoadBalancerHealthProbeProtocol)
+		protocol, err = consts.GetAttributeValueInSvcAnnotation(serviceManifest.Annotations, consts.ServiceAnnotationLoadBalancerHealthProbeProtocol)
 		if err != nil {
 			return nil, fmt.Errorf("failed to parse annotation %s: %w", consts.ServiceAnnotationLoadBalancerHealthProbeProtocol, err)
 		}
@@ -2126,37 +2126,71 @@ func (az *Cloud) buildHealthProbeRulesForPort(annotations map[string]string, por
 	// Lookup or Override Health Probe Port
 	properties.Port = &port.NodePort
 
-	// Look up port-specific override
-	var probePortValidator = func(val *int32) error {
-		//minimum number of unhealthy responses is 2. ref: https://docs.microsoft.com/en-us/rest/api/load-balancer/load-balancers/create-or-update#probe
-		const (
-			MinProbePort = 1
-			MaxProbePort = 65535
-		)
-		if *val < MinProbePort || *val > MaxProbePort {
-			return fmt.Errorf("the value of %s must be between %d and %d inclusive", consts.HealthProbeParamsPort, MinProbePort, MaxProbePort)
+	probePort, err := consts.GetHealthProbeConfigOfPortFromK8sSvcAnnotation(serviceManifest.Annotations, port.Port, consts.HealthProbeParamsPort, func(s *string) error {
+		if s == nil {
+			return nil
 		}
-		return nil
-	}
-
-	probePort, err := consts.GetInt32HealthProbeConfigOfPortFromK8sSvcAnnotation(annotations, port.Port, consts.HealthProbeParamsPort, probePortValidator)
+		//nolint:gosec
+		port, err := strconv.Atoi(*s)
+		if err != nil {
+			//not a integer
+			for _, item := range serviceManifest.Spec.Ports {
+				if strings.EqualFold(item.Name, *s) {
+					//found the port
+					return nil
+				}
+			}
+			return fmt.Errorf("port %s not found in service", *s)
+		}
+		if port < 0 || port > 65535 {
+			return fmt.Errorf("port %d is out of range", port)
+		}
+		for _, item := range serviceManifest.Spec.Ports {
+			//nolint:gosec
+			if item.Port == int32(port) {
+				//found the port
+				return nil
+			}
+		}
+		return fmt.Errorf("port %s not found in service", *s)
+	})
 	if err != nil {
 		return nil, fmt.Errorf("failed to parse annotation %s: %w", consts.BuildHealthProbeAnnotationKeyForPort(port.Port, consts.HealthProbeParamsPort), err)
 	}
 
 	if probePort != nil {
-		properties.Port = probePort
+		//nolint:gosec
+		port, err := strconv.Atoi(*probePort)
+		if err != nil {
+			//not a integer
+			for _, item := range serviceManifest.Spec.Ports {
+				if strings.EqualFold(item.Name, *probePort) {
+					//found the port
+					properties.Port = to.Int32Ptr(item.NodePort)
+				}
+			}
+		} else {
+			if port >= 0 || port <= 65535 {
+				for _, item := range serviceManifest.Spec.Ports {
+					//nolint:gosec
+					if item.Port == int32(port) {
+						//found the port
+						properties.Port = to.Int32Ptr(item.NodePort)
+					}
+				}
+			}
+		}
 	}
 
 	// Select request path
 	if strings.EqualFold(string(properties.Protocol), string(network.ProtocolHTTPS)) || strings.EqualFold(string(properties.Protocol), string(network.ProtocolHTTP)) {
 		// get request path ,only used with http/https probe
-		path, err := consts.GetHealthProbeConfigOfPortFromK8sSvcAnnotation(annotations, port.Port, consts.HealthProbeParamsRequestPath)
+		path, err := consts.GetHealthProbeConfigOfPortFromK8sSvcAnnotation(serviceManifest.Annotations, port.Port, consts.HealthProbeParamsRequestPath)
 		if err != nil {
 			return nil, fmt.Errorf("failed to parse annotation %s: %w", consts.BuildHealthProbeAnnotationKeyForPort(port.Port, consts.HealthProbeParamsRequestPath), err)
 		}
 		if path == nil {
-			if path, err = consts.GetAttributeValueInSvcAnnotation(annotations, consts.ServiceAnnotationLoadBalancerHealthProbeRequestPath); err != nil {
+			if path, err = consts.GetAttributeValueInSvcAnnotation(serviceManifest.Annotations, consts.ServiceAnnotationLoadBalancerHealthProbeRequestPath); err != nil {
 				return nil, fmt.Errorf("failed to parse annotation %s: %w", consts.ServiceAnnotationLoadBalancerHealthProbeRequestPath, err)
 			}
 		}
@@ -2176,12 +2210,12 @@ func (az *Cloud) buildHealthProbeRulesForPort(annotations map[string]string, por
 		}
 		return nil
 	}
-	numberOfProbes, err := consts.GetInt32HealthProbeConfigOfPortFromK8sSvcAnnotation(annotations, port.Port, consts.HealthProbeParamsNumOfProbe, numOfProbeValidator)
+	numberOfProbes, err := consts.GetInt32HealthProbeConfigOfPortFromK8sSvcAnnotation(serviceManifest.Annotations, port.Port, consts.HealthProbeParamsNumOfProbe, numOfProbeValidator)
 	if err != nil {
 		return nil, fmt.Errorf("failed to parse annotation %s: %w", consts.BuildHealthProbeAnnotationKeyForPort(port.Port, consts.HealthProbeParamsNumOfProbe), err)
 	}
 	if numberOfProbes == nil {
-		if numberOfProbes, err = consts.Getint32ValueFromK8sSvcAnnotation(annotations, consts.ServiceAnnotationLoadBalancerHealthProbeNumOfProbe, numOfProbeValidator); err != nil {
+		if numberOfProbes, err = consts.Getint32ValueFromK8sSvcAnnotation(serviceManifest.Annotations, consts.ServiceAnnotationLoadBalancerHealthProbeNumOfProbe, numOfProbeValidator); err != nil {
 			return nil, fmt.Errorf("failed to parse annotation %s: %w", consts.ServiceAnnotationLoadBalancerHealthProbeNumOfProbe, err)
 		}
 	}
@@ -2202,12 +2236,12 @@ func (az *Cloud) buildHealthProbeRulesForPort(annotations map[string]string, por
 		}
 		return nil
 	}
-	probeInterval, err := consts.GetInt32HealthProbeConfigOfPortFromK8sSvcAnnotation(annotations, port.Port, consts.HealthProbeParamsProbeInterval, probeIntervalValidator)
+	probeInterval, err := consts.GetInt32HealthProbeConfigOfPortFromK8sSvcAnnotation(serviceManifest.Annotations, port.Port, consts.HealthProbeParamsProbeInterval, probeIntervalValidator)
 	if err != nil {
 		return nil, fmt.Errorf("failed to parse annotation %s:%w", consts.BuildHealthProbeAnnotationKeyForPort(port.Port, consts.HealthProbeParamsProbeInterval), err)
 	}
 	if probeInterval == nil {
-		if probeInterval, err = consts.Getint32ValueFromK8sSvcAnnotation(annotations, consts.ServiceAnnotationLoadBalancerHealthProbeInterval, probeIntervalValidator); err != nil {
+		if probeInterval, err = consts.Getint32ValueFromK8sSvcAnnotation(serviceManifest.Annotations, consts.ServiceAnnotationLoadBalancerHealthProbeInterval, probeIntervalValidator); err != nil {
 			return nil, fmt.Errorf("failed to parse annotation %s: %w", consts.ServiceAnnotationLoadBalancerHealthProbeInterval, err)
 		}
 	}
@@ -2229,25 +2263,6 @@ func (az *Cloud) buildHealthProbeRulesForPort(annotations map[string]string, por
 	return probe, nil
 }
 
-// probeKey generates a string key containing all functional probe configuration, to allow de-duplication of like probes.
-func probeKey(probe network.Probe) string {
-	var port, interval, number int32
-	var path string
-	if probe.Port != nil {
-		port = *probe.Port
-	}
-	if probe.IntervalInSeconds != nil {
-		interval = *probe.IntervalInSeconds
-	}
-	if probe.NumberOfProbes != nil {
-		number = *probe.NumberOfProbes
-	}
-	if probe.RequestPath != nil {
-		path = *probe.RequestPath
-	}
-	return fmt.Sprintf("%s-%d-%d-%d-%s", probe.Protocol, port, interval, number, path)
-}
-
 // buildLBRules
 // for following sku: basic loadbalancer vs standard load balancer
 // for following scenario: internal vs external
@@ -2259,7 +2274,6 @@ func (az *Cloud) getExpectedLBRules(
 
 	var expectedRules []network.LoadBalancingRule
 	var expectedProbes []network.Probe
-	uniqueProbes := make(map[string]network.Probe)
 
 	// support podPresence health check when External Traffic Policy is local
 	// take precedence over user defined probe configuration
@@ -2280,7 +2294,7 @@ func (az *Cloud) getExpectedLBRules(
 				NumberOfProbes:    to.Int32Ptr(consts.HealthProbeDefaultNumOfProbe),
 			},
 		}
-		uniqueProbes[probeKey(*nodeEndpointHealthprobe)] = *nodeEndpointHealthprobe
+		expectedProbes = append(expectedProbes, *nodeEndpointHealthprobe)
 	}
 
 	// In HA mode, lb forward traffic of all port to backend
@@ -2300,17 +2314,17 @@ func (az *Cloud) getExpectedLBRules(
 		if nodeEndpointHealthprobe == nil {
 			// use user customized health probe rule if any
 			for _, port := range service.Spec.Ports {
-				portprobe, err := az.buildHealthProbeRulesForPort(service.Annotations, port, lbRuleName)
+				portprobe, err := az.buildHealthProbeRulesForPort(service, port, lbRuleName)
 				if err != nil {
 					klog.V(2).ErrorS(err, "error occurred when buildHealthProbeRulesForPort", "service", service.Name, "namespace", service.Namespace,
 						"rule-name", lbRuleName, "port", port.Port)
 					//ignore error because we only need one correct rule
 				}
 				if portprobe != nil {
-					uniqueProbes[probeKey(*portprobe)] = *portprobe
 					props.Probe = &network.SubResource{
 						ID: to.StringPtr(az.getLoadBalancerProbeID(lbName, az.getLoadBalancerResourceGroup(), *portprobe.Name)),
 					}
+					expectedProbes = append(expectedProbes, *portprobe)
 					break
 				}
 			}
@@ -2331,7 +2345,16 @@ func (az *Cloud) getExpectedLBRules(
 		for _, port := range service.Spec.Ports {
 			lbRuleName := az.getLoadBalancerRuleName(service, port.Protocol, port.Port)
 			klog.V(2).Infof("getExpectedLBRules lb name (%s) rule name (%s)", lbName, lbRuleName)
-
+			isNoLBRuleRequired, err := consts.IsLBRuleOnK8sServicePortDisabled(service.Annotations, port.Port)
+			if err != nil {
+				err := fmt.Errorf("failed to parse annotation %s: %w", consts.BuildAnnotationKeyForPort(port.Port, consts.PortAnnotationNoLBRule), err)
+				klog.V(2).ErrorS(err, "error occurred when getExpectedLoadBalancingRulePropertiesForPort", "service", service.Name, "namespace", service.Namespace,
+					"rule-name", lbRuleName, "port", port.Port)
+			}
+			if isNoLBRuleRequired {
+				klog.V(2).Infof("getExpectedLBRules lb name (%s) rule name (%s) no lb rule required", lbName, lbRuleName)
+				continue
+			}
 			if port.Protocol == v1.ProtocolSCTP && !(az.useStandardLoadBalancer() && consts.IsK8sServiceUsingInternalLoadBalancer(service)) {
 				return expectedProbes, expectedRules, fmt.Errorf("SCTP is only supported on standard loadbalancer in internal mode")
 			}
@@ -2345,23 +2368,31 @@ func (az *Cloud) getExpectedLBRules(
 				return expectedProbes, expectedRules, fmt.Errorf("error generate lb rule for ha mod loadbalancer. err: %w", err)
 			}
 
-			if nodeEndpointHealthprobe == nil {
-				portprobe, err := az.buildHealthProbeRulesForPort(service.Annotations, port, lbRuleName)
-				if err != nil {
-					klog.V(2).ErrorS(err, "error occurred when buildHealthProbeRulesForPort", "service", service.Name, "namespace", service.Namespace,
-						"rule-name", lbRuleName, "port", port.Port)
-					return expectedProbes, expectedRules, err
-				}
-				if portprobe != nil {
-					uniqueProbes[probeKey(*portprobe)] = *portprobe
+			isNoHealthProbeRule, err := consts.IsHealthProbeRuleOnK8sServicePortDisabled(service.Annotations, port.Port)
+			if err != nil {
+				err := fmt.Errorf("failed to parse annotation %s: %w", consts.BuildAnnotationKeyForPort(port.Port, consts.PortAnnotationNoHealthProbeRule), err)
+				klog.V(2).ErrorS(err, "error occurred when buildHealthProbeRulesForPort", "service", service.Name, "namespace", service.Namespace,
+					"rule-name", lbRuleName, "port", port.Port)
+			}
+			if !isNoHealthProbeRule {
+				if nodeEndpointHealthprobe == nil {
+					portprobe, err := az.buildHealthProbeRulesForPort(service, port, lbRuleName)
+					if err != nil {
+						klog.V(2).ErrorS(err, "error occurred when buildHealthProbeRulesForPort", "service", service.Name, "namespace", service.Namespace,
+							"rule-name", lbRuleName, "port", port.Port)
+						return expectedProbes, expectedRules, err
+					}
+					if portprobe != nil {
+						props.Probe = &network.SubResource{
+							ID: to.StringPtr(az.getLoadBalancerProbeID(lbName, az.getLoadBalancerResourceGroup(), *portprobe.Name)),
+						}
+						expectedProbes = append(expectedProbes, *portprobe)
+					}
+				} else {
 					props.Probe = &network.SubResource{
-						ID: to.StringPtr(az.getLoadBalancerProbeID(lbName, az.getLoadBalancerResourceGroup(), *portprobe.Name)),
+						ID: to.StringPtr(az.getLoadBalancerProbeID(lbName, az.getLoadBalancerResourceGroup(), *nodeEndpointHealthprobe.Name)),
 					}
 				}
-			} else {
-				props.Probe = &network.SubResource{
-					ID: to.StringPtr(az.getLoadBalancerProbeID(lbName, az.getLoadBalancerResourceGroup(), *nodeEndpointHealthprobe.Name)),
-				}
 			}
 			if consts.IsK8sServiceDisableLoadBalancerFloatingIP(service) {
 				props.BackendPort = to.Int32Ptr(port.NodePort)
@@ -2371,14 +2402,9 @@ func (az *Cloud) getExpectedLBRules(
 				Name:                              &lbRuleName,
 				LoadBalancingRulePropertiesFormat: props,
 			})
-
 		}
 	}
 
-	for _, probe := range uniqueProbes {
-		expectedProbes = append(expectedProbes, probe)
-	}
-
 	return expectedProbes, expectedRules, nil
 }
 


Test patch:
diff --git a/.golangci-test.yml b/.golangci-test.yml
new file mode 100644
index 000000000..05efbaf64
--- /dev/null
+++ b/.golangci-test.yml
@@ -0,0 +1,26 @@
+run:
+  deadline: 30m
+  concurrency: 2
+  go: '1.18'
+  allow-serial-runners: true
+skip-dirs:
+    - vendor
+linters:
+  disable-all: true
+  enable:
+    - gosec
+  # Run with --fast=false for more extensive checks
+  fast: false
+  govet:
+    # Report about shadowed variables.
+    # Default: false
+    check-shadowing: true
+issues:
+  exclude-rules:
+    - path: tests/e2e
+      linters:
+        - revive
+  #include:
+  #  - EXC0002 # include "missing comments" issues from golint
+  max-issues-per-linter: 0
+  max-same-issues: 0
diff --git a/pkg/consts/helpers_test.go b/pkg/consts/helpers_test.go
index 0763de2fd..c38f5074a 100644
--- a/pkg/consts/helpers_test.go
+++ b/pkg/consts/helpers_test.go
@@ -376,3 +376,39 @@ func TestGetInt32HealthProbeConfigOfPortFromK8sSvcAnnotation(t *testing.T) {
 		})
 	}
 }
+
+func TestBuildAnnotationKeyForPort(t *testing.T) {
+	type args struct {
+		port int32
+		key  PortParams
+	}
+	tests := []struct {
+		name string
+		args args
+		want string
+	}{
+		{
+			name: "no lb rule",
+			args: args{
+				port: 80,
+				key:  PortAnnotationNoLBRule,
+			},
+			want: "service.beta.kubernetes.io/port_80_no_lb_rule",
+		},
+		{
+			name: "no lb rule",
+			args: args{
+				port: 80,
+				key:  PortAnnotationNoHealthProbeRule,
+			},
+			want: "service.beta.kubernetes.io/port_80_no_probe_rule",
+		},
+	}
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			if got := BuildAnnotationKeyForPort(tt.args.port, tt.args.key); got != tt.want {
+				t.Errorf("BuildAnnotationKeyForPort() = %v, want %v", got, tt.want)
+			}
+		})
+	}
+}
diff --git a/pkg/provider/azure_loadbalancer_test.go b/pkg/provider/azure_loadbalancer_test.go
index cb53ed129..924dddc31 100644
--- a/pkg/provider/azure_loadbalancer_test.go
+++ b/pkg/provider/azure_loadbalancer_test.go
@@ -2486,21 +2486,61 @@ func TestReconcileLoadBalancerRule(t *testing.T) {
 			expectedProbes: getDefaultTestProbes("Tcp", ""),
 		},
 		{
-			desc: "getExpectedLBRules should allow setting port specific health probe port",
+			desc: "getExpectedLBRules should support customize health probe port in multi-port service",
 			service: getTestService("test1", v1.ProtocolTCP, map[string]string{
-				"service.beta.kubernetes.io/port_80_health-probe_protocol": "Http",
-				"service.beta.kubernetes.io/port_80_health-probe_port":     "15012",
-			}, false, 80),
-			expectedRules:  getDefaultTestRules(false),
-			expectedProbes: getTestProbes("Http", "/", to.Int32Ptr(5), to.Int32Ptr(80), to.Int32Ptr(15012), to.Int32Ptr(2)),
+				"service.beta.kubernetes.io/port_8000_health-probe_port": "port-tcp-80",
+			}, false, 80, 8000),
+			expectedRules: []network.LoadBalancingRule{
+				getTestRule(false, 80),
+				getTestRule(false, 8000),
+			},
+			expectedProbes: []network.Probe{
+				getTestProbe("Tcp", "/", to.Int32Ptr(5), to.Int32Ptr(80), to.Int32Ptr(10080), to.Int32Ptr(2)),
+				getTestProbe("Tcp", "/", to.Int32Ptr(5), to.Int32Ptr(8000), to.Int32Ptr(10080), to.Int32Ptr(2)),
+			},
 		},
 		{
-			desc: "getExpectedLBRules should not include duplicate probes when overrides would create them",
+			desc: "getExpectedLBRules should support customize health probe port in multi-port service",
 			service: getTestService("test1", v1.ProtocolTCP, map[string]string{
-				"service.beta.kubernetes.io/port_8000_health-probe_port": "10080",
+				"service.beta.kubernetes.io/port_8000_health-probe_port": "80",
 			}, false, 80, 8000),
-			expectedRules:  []network.LoadBalancingRule{getTestRule(false, 80), getTestRule(false, 8000)},
-			expectedProbes: getTestProbes("Tcp", "/", to.Int32Ptr(5), to.Int32Ptr(8000), to.Int32Ptr(10080), to.Int32Ptr(2)),
+			expectedRules: []network.LoadBalancingRule{
+				getTestRule(false, 80),
+				getTestRule(false, 8000),
+			},
+			expectedProbes: []network.Probe{
+				getTestProbe("Tcp", "/", to.Int32Ptr(5), to.Int32Ptr(80), to.Int32Ptr(10080), to.Int32Ptr(2)),
+				getTestProbe("Tcp", "/", to.Int32Ptr(5), to.Int32Ptr(8000), to.Int32Ptr(10080), to.Int32Ptr(2)),
+			},
+		},
+		{
+			desc: "getExpectedLBRules should not generate probe rule when no health probe rule is specified.",
+			service: getTestService("test1", v1.ProtocolTCP, map[string]string{
+				"service.beta.kubernetes.io/port_8000_no_probe_rule": "true",
+			}, false, 80, 8000),
+			expectedRules: []network.LoadBalancingRule{
+				getTestRule(false, 80),
+				func() network.LoadBalancingRule {
+					rule := getTestRule(false, 8000)
+					rule.Probe = nil
+					return rule
+				}(),
+			},
+			expectedProbes: []network.Probe{
+				getTestProbe("Tcp", "/", to.Int32Ptr(5), to.Int32Ptr(80), to.Int32Ptr(10080), to.Int32Ptr(2)),
+			},
+		},
+		{
+			desc: "getExpectedLBRules should not generate lb rule and health probe rule when no lb rule is specified.",
+			service: getTestService("test1", v1.ProtocolTCP, map[string]string{
+				"service.beta.kubernetes.io/port_8000_no_lb_rule": "true",
+			}, false, 80, 8000),
+			expectedRules: []network.LoadBalancingRule{
+				getTestRule(false, 80),
+			},
+			expectedProbes: []network.Probe{
+				getTestProbe("Tcp", "/", to.Int32Ptr(5), to.Int32Ptr(80), to.Int32Ptr(10080), to.Int32Ptr(2)),
+			},
 		},
 	}
 	rules := getDefaultTestRules(true)
diff --git a/tests/e2e/network/service_annotations.go b/tests/e2e/network/service_annotations.go
index b020b0be2..787d30205 100644
--- a/tests/e2e/network/service_annotations.go
+++ b/tests/e2e/network/service_annotations.go
@@ -23,6 +23,7 @@ import (
 	"os"
 	"reflect"
 	"regexp"
+	"strconv"
 	"strings"
 	"time"
 
@@ -53,8 +54,9 @@ var (
 )
 
 const (
-	serverPort  = 80
-	testingPort = 81
+	serverPort             = 80
+	alterNativeServicePort = 8080
+	testingPort            = 81
 )
 
 var _ = Describe("Service with annotation", Label(utils.TestSuiteLabelServiceAnnotation), func() {
@@ -71,10 +73,7 @@ var _ = Describe("Service with annotation", Label(utils.TestSuiteLabelServiceAnn
 	labels := map[string]string{
 		"app": serviceName,
 	}
-	ports := []v1.ServicePort{{
-		Port:       serverPort,
-		TargetPort: intstr.FromInt(serverPort),
-	}}
+	var ports []v1.ServicePort
 
 	BeforeEach(func() {
 		var err error
@@ -96,6 +95,12 @@ var _ = Describe("Service with annotation", Label(utils.TestSuiteLabelServiceAnn
 		utils.Logf("Waiting for backend pods to be ready")
 		err = utils.WaitPodsToBeReady(cs, ns.Name)
 		Expect(err).NotTo(HaveOccurred())
+
+		ports = []v1.ServicePort{{
+			Name:       "http",
+			Port:       serverPort,
+			TargetPort: intstr.FromInt(serverPort),
+		}}
 	})
 
 	AfterEach(func() {
@@ -653,6 +658,70 @@ var _ = Describe("Service with annotation", Label(utils.TestSuiteLabelServiceAnn
 		Expect(targetProbes[0].Protocol).To(Equal(network.ProbeProtocolHTTP))
 	})
 
+	It("should generate health probe configs in multi-port scenario", func() {
+		By("Creating a service with health probe annotations")
+		annotation := map[string]string{
+			consts.ServiceAnnotationLoadBalancerHealthProbeNumOfProbe:                                      "5",
+			consts.BuildHealthProbeAnnotationKeyForPort(serverPort, consts.HealthProbeParamsNumOfProbe):    "3",
+			consts.ServiceAnnotationLoadBalancerHealthProbeInterval:                                        "15",
+			consts.BuildHealthProbeAnnotationKeyForPort(serverPort, consts.HealthProbeParamsProbeInterval): "10",
+			consts.BuildHealthProbeAnnotationKeyForPort(serverPort, consts.HealthProbeParamsPort):          strconv.Itoa(alterNativeServicePort),
+			consts.ServiceAnnotationLoadBalancerHealthProbeProtocol:                                        "Http",
+			consts.ServiceAnnotationLoadBalancerHealthProbeRequestPath:                                     "/",
+			consts.BuildAnnotationKeyForPort(alterNativeServicePort, consts.PortAnnotationNoLBRule):        "true",
+		}
+		ports = append(ports, v1.ServicePort{
+			Name:       "port2",
+			Port:       alterNativeServicePort,
+			TargetPort: intstr.FromInt(serverPort),
+		})
+
+		// create service with given annotation and wait it to expose
+		publicIP := createAndExposeDefaultServiceWithAnnotation(cs, serviceName, ns.Name, labels, annotation, ports)
+		defer func() {
+			By("Cleaning up service")
+			err := utils.DeleteService(cs, ns.Name, serviceName)
+			Expect(err).NotTo(HaveOccurred())
+		}()
+		pipFrontendConfigID := getPIPFrontendConfigurationID(tc, publicIP, tc.GetResourceGroup(), "")
+		pipFrontendConfigIDSplit := strings.Split(pipFrontendConfigID, "/")
+		Expect(len(pipFrontendConfigIDSplit)).NotTo(Equal(0))
+
+		var lb *network.LoadBalancer
+		var targetProbes []*network.Probe
+		//wait for backend update
+		err := wait.PollImmediate(5*time.Second, 60*time.Second, func() (bool, error) {
+			lb = getAzureLoadBalancerFromPIP(tc, publicIP, tc.GetResourceGroup(), "")
+			targetProbes = []*network.Probe{}
+			for i := range *lb.LoadBalancerPropertiesFormat.Probes {
+				probe := (*lb.LoadBalancerPropertiesFormat.Probes)[i]
+				utils.Logf("One probe of LB is %q", *probe.Name)
+				targetProbes = append(targetProbes, &probe)
+			}
+			return len(targetProbes) == 1, nil
+		})
+		Expect(err).NotTo(HaveOccurred())
+
+		By("Validating health probe configs")
+		var numberOfProbes *int32
+		var intervalInSeconds *int32
+		for _, probe := range targetProbes {
+			if probe.NumberOfProbes != nil {
+				numberOfProbes = probe.NumberOfProbes
+			}
+			if probe.IntervalInSeconds != nil {
+				intervalInSeconds = probe.IntervalInSeconds
+			}
+		}
+		utils.Logf("Validating health probe config numberOfProbes")
+		Expect(*numberOfProbes).To(Equal(int32(3)))
+		utils.Logf("Validating health probe config intervalInSeconds")
+		Expect(*intervalInSeconds).To(Equal(int32(10)))
+		utils.Logf("Validating health probe config protocol")
+		Expect((len(targetProbes))).To(Equal(1))
+		Expect(targetProbes[0].Protocol).To(Equal(network.ProbeProtocolHTTP))
+	})
+
 	// Check if the following annotations are correctly set with Service LB IP
 	// service.beta.kubernetes.io/azure-load-balancer-ipv4 or service.beta.kubernetes.io/azure-load-balancer-ipv6
 	It("should support service annotation 'service.beta.kubernetes.io/azure-load-balancer-ip'", func() {