Repository: kubernetes-sigs/cloud-provider-azure
Pull request #4536 (2023-08-31 02:52:17)

fix: check endpoint slice update after backend pool update for local service to prevent mismatch
<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?

<!--
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature
/kind design

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

/kind bug
/kind failing-test

#### What this PR does / why we need it:

There are chances that the endpoints in an endpointslice updates slower than the actual state. This may bring an issue if the endpointslice updates after we update backend pool but before the service reconciliation finishes. In this case, the endpointslice update would be ignored. This PR checks the endpointslice state after updating backend pool, and reconciles the IPs in the backend pool for a second time to make sure they keep aligned with the endpoints in the endpointslice.

#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #
Related https://github.com/kubernetes-sigs/cloud-provider-azure/issues/4013

#### Special notes for your reviewer:

#### Does this PR introduce a user-facing change?
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

For more information on release notes see: https://git.k8s.io/community/contributors/guide/release-notes.md
-->
```release-note
fix: check endpoint slice update after backend pool update for local service to prevent mismatch
```

#### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

<!--
This section can be blank if this pull request does not require a release note.

When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.

See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files

Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
-->
```docs

```


k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/cloud-provider-azure/pull/4536#" title="Author self-approved">nilo19</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fcloud-provider-azure).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/cloud-provider-azure/blob/master/OWNERS)~~ [nilo19]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | ee42e7fd9e5910f4deee6b8936ccae389db6f76f | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-unit/1697079841758646272) | true | `/test pull-cloud-provider-azure-unit`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | 4a086ad436de9201cadf3d11647681a968c68158 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1697087719835963392) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | f3eabac6cfac70bf417fd867b98c85da0e11d761 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1697117572698214400) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @nilo19: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | f3eabac6cfac70bf417fd867b98c85da0e11d761 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1697117572698214400) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1698537329322364928) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @nilo19: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1698537329322364928) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1698537329372696576) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @nilo19: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1698537329322364928) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1698537329372696576) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1698537329171369984) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @nilo19: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1698537329322364928) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1698537329372696576) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1698537329171369984) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1698537329121038336) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @nilo19: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1698537329322364928) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1698537329372696576) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1698537329171369984) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1698537329121038336) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-capz/1698537329272033280) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
nilo19: /test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz
k8s-ci-robot: @nilo19: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1698537329322364928) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1698537329171369984) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1698537329121038336) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-capz/1698537329272033280) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1698655590609326080) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
nilo19: /test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz
k8s-ci-robot: @nilo19: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1698537329322364928) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1698537329171369984) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1698537329121038336) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-capz/1698537329272033280) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | 25a1bff115108c1f1e60c53af7cb6db795d47ba0 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1698698340490809344) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
nilo19: /test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz
nilo19: /test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz
k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | 6d3c662fd6f013311b29b5555c5012f8822e8c8c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1699225353274265600) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
nilo19: /test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz
nilo19: /test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz
k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | 6d3c662fd6f013311b29b5555c5012f8822e8c8c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1699287456743952384) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
nilo19: /test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz
nilo19: /test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz
nilo19: /test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz
k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz | 6d3c662fd6f013311b29b5555c5012f8822e8c8c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4536/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz/1699607554704805888) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4536). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
nilo19: /test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz

Patch:
diff --git a/.github/workflows/release.yaml b/.github/workflows/release.yaml
index fc44be437..ab01a6a94 100644
--- a/.github/workflows/release.yaml
+++ b/.github/workflows/release.yaml
@@ -38,7 +38,7 @@ jobs:
           rm -rf ./bin
           ARCH=${{ matrix.arch }} make bin/azure-cloud-controller-manager
           mv bin/azure-cloud-controller-manager bin/azure-cloud-controller-manager-${{ matrix.os }}-${{ matrix.arch }}
-      - uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
+      - uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
         with:
           name: cloud-controller-manager-${{ matrix.os }}-${{ matrix.arch }}
           path: bin/azure-cloud-controller-manager-${{ matrix.os }}-${{ matrix.arch }}
@@ -84,13 +84,13 @@ jobs:
           mv bin/azure-cloud-node-manager-${{ matrix.arch }}.exe bin/azure-cloud-node-manager-${{ matrix.os }}-${{ matrix.arch }}.exe
       - name: Upload artifact for linux
         if: matrix.os == 'linux'
-        uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
+        uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
         with:
           name: cloud-node-manager-${{ matrix.os }}-${{ matrix.arch }}
           path: bin/azure-cloud-node-manager-${{ matrix.os }}-${{ matrix.arch }}
       - name: Upload artifact for windows
         if: matrix.os == 'windows'
-        uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
+        uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
         with:
           name: cloud-node-manager-${{ matrix.os }}-${{ matrix.arch }}
           path: bin/azure-cloud-node-manager-${{ matrix.os }}-${{ matrix.arch }}.exe
@@ -135,13 +135,13 @@ jobs:
           mv bin/azure-acr-credential-provider.exe bin/azure-acr-credential-provider-${{ matrix.os }}-${{ matrix.arch }}.exe
       - name: Upload artifact for linux
         if: matrix.os == 'linux'
-        uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
+        uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
         with:
           name: acr-credential-provider-${{ matrix.os }}-${{ matrix.arch }}
           path: bin/azure-acr-credential-provider-${{ matrix.os }}-${{ matrix.arch }}
       - name: Upload artifact for windows
         if: matrix.os == 'windows'
-        uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
+        uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
         with:
           name: acr-credential-provider-${{ matrix.os }}-${{ matrix.arch }}
           path: bin/azure-acr-credential-provider-${{ matrix.os }}-${{ matrix.arch }}.exe
diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml
index ed11f4253..37f974bf3 100644
--- a/.github/workflows/scorecards.yml
+++ b/.github/workflows/scorecards.yml
@@ -64,7 +64,7 @@ jobs:
       # Upload the results as artifacts (optional). Commenting out will disable uploads of run results in SARIF
       # format to the repository Actions tab.
       - name: "Upload artifact"
-        uses: actions/upload-artifact@a8a3f3ad30e3422c9c7b888a15615d19a852ae32 # v3.1.3
+        uses: actions/upload-artifact@0b7f8abb1508181956e8e162db84b466c27e18ce # v3.1.2
         with:
           name: SARIF file
           path: results.sarif
diff --git a/Dockerfile b/Dockerfile
index 3fddfbf61..0b2c578eb 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -14,7 +14,7 @@
 
 # syntax=docker/dockerfile:1
 
-FROM --platform=linux/amd64 golang:1.21-bullseye@sha256:b61af81a1e082e061b8addc222f2b1fd472a34eaf6950823e4f2743498ee4924 AS builder
+FROM --platform=linux/amd64 golang:1.21-bullseye@sha256:02f350d8452d3f9693a450586659ecdc6e40e9be8f8dfc6d402300d87223fdfa AS builder
 
 ARG ENABLE_GIT_COMMAND=true
 ARG ARCH=amd64
diff --git a/cloud-node-manager.Dockerfile b/cloud-node-manager.Dockerfile
index 50545f436..055b5003e 100644
--- a/cloud-node-manager.Dockerfile
+++ b/cloud-node-manager.Dockerfile
@@ -14,7 +14,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM --platform=linux/amd64 golang:1.21-bullseye@sha256:b61af81a1e082e061b8addc222f2b1fd472a34eaf6950823e4f2743498ee4924 AS builder
+FROM --platform=linux/amd64 golang:1.21-bullseye@sha256:02f350d8452d3f9693a450586659ecdc6e40e9be8f8dfc6d402300d87223fdfa AS builder
 
 ARG ENABLE_GIT_COMMAND=true
 ARG ARCH=amd64
diff --git a/pkg/azclient/auth.go b/pkg/azclient/auth.go
index 67f12af48..59a7236a6 100644
--- a/pkg/azclient/auth.go
+++ b/pkg/azclient/auth.go
@@ -166,8 +166,7 @@ func NewAuthProvider(config AzureAuthConfig, clientOption *policy.ClientOptions)
 	var clientCertificateCredential azcore.TokenCredential
 	if len(config.AADClientCertPath) > 0 && len(config.AADClientCertPassword) > 0 {
 		credOptions := &azidentity.ClientCertificateCredentialOptions{
-			ClientOptions:        clientOption.ClientOptions,
-			SendCertificateChain: true,
+			ClientOptions: clientOption.ClientOptions,
 		}
 		certData, err := os.ReadFile(config.AADClientCertPath)
 		if err != nil {
diff --git a/pkg/azclient/go.mod b/pkg/azclient/go.mod
index ee16bf48d..626ed11c0 100644
--- a/pkg/azclient/go.mod
+++ b/pkg/azclient/go.mod
@@ -3,7 +3,7 @@ module sigs.k8s.io/cloud-provider-azure/pkg/azclient
 go 1.20
 
 require (
-	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.2
+	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.1
 	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.3.1
 	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/compute/armcompute/v5 v5.1.0
 	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v4 v4.3.0
@@ -16,7 +16,7 @@ require (
 	github.com/onsi/gomega v1.27.10
 	go.opentelemetry.io/otel v1.17.0
 	go.opentelemetry.io/otel/trace v1.17.0
-	golang.org/x/crypto v0.13.0
+	golang.org/x/crypto v0.12.0
 	golang.org/x/time v0.3.0
 	gopkg.in/dnaeon/go-vcr.v3 v3.1.2
 	k8s.io/utils v0.0.0-20230505201702-9f6742963106
@@ -39,8 +39,8 @@ require (
 	go.opentelemetry.io/otel/metric v1.17.0 // indirect
 	golang.org/x/net v0.14.0 // indirect
 	golang.org/x/sync v0.3.0 // indirect
-	golang.org/x/sys v0.12.0 // indirect
-	golang.org/x/text v0.13.0 // indirect
+	golang.org/x/sys v0.11.0 // indirect
+	golang.org/x/text v0.12.0 // indirect
 	golang.org/x/tools v0.12.0 // indirect
 	google.golang.org/protobuf v1.30.0 // indirect
 	gopkg.in/check.v1 v1.0.0-20201130134442-10cb98267c6c // indirect
diff --git a/pkg/azclient/go.sum b/pkg/azclient/go.sum
index 43bc58211..26f941a6c 100644
--- a/pkg/azclient/go.sum
+++ b/pkg/azclient/go.sum
@@ -1,5 +1,5 @@
-github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.2 h1:t5+QXLCK9SVi0PPdaY0PrFvYUo24KwA0QwxnaHRSVd4=
-github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.2/go.mod h1:bjGvMhVMb+EEm3VRNQawDMUyMMjo+S5ewNjflkep/0Q=
+github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.1 h1:/iHxaJhsFr0+xVFfbMr5vxz848jyiWuIEDhYq3y5odY=
+github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.1/go.mod h1:bjGvMhVMb+EEm3VRNQawDMUyMMjo+S5ewNjflkep/0Q=
 github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.3.1 h1:LNHhpdK7hzUcx/k1LIcuh5k7k1LGIWLQfCjaneSj7Fc=
 github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.3.1/go.mod h1:uE9zaUfEQT/nbQjVi2IblCG9iaLtZsuYZ8ne+PuQ02M=
 github.com/Azure/azure-sdk-for-go/sdk/internal v1.3.0 h1:sXr+ck84g/ZlZUOZiNELInmMgOsuGwdjjVkEIde0OtY=
@@ -76,8 +76,8 @@ go.opentelemetry.io/otel/trace v1.17.0 h1:/SWhSRHmDPOImIAetP1QAeMnZYiQXrTy4fMMYO
 go.opentelemetry.io/otel/trace v1.17.0/go.mod h1:I/4vKTgFclIsXRVucpH25X0mpFSczM7aHeaz0ZBLWjY=
 golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACkg1iLfiJU5Ep61QUkGW8qpdssI0+w=
 golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
-golang.org/x/crypto v0.13.0 h1:mvySKfSWJ+UKUii46M40LOvyWfN0s2U+46/jDd0e6Ck=
-golang.org/x/crypto v0.13.0/go.mod h1:y6Z2r+Rw4iayiXXAIxJIDAJ1zMW4yaTpebo8fPOliYc=
+golang.org/x/crypto v0.12.0 h1:tFM/ta59kqch6LlvYnPa0yx5a83cL2nHflFhYKvv9Yk=
+golang.org/x/crypto v0.12.0/go.mod h1:NF0Gs7EO5K4qLn+Ylc+fih8BSTeIjAP05siRnAh98yw=
 golang.org/x/mod v0.4.2/go.mod h1:s0Qsj1ACt9ePp/hMypM3fl4fZqREWJwdYDEqhRiZZUA=
 golang.org/x/mod v0.12.0 h1:rmsUpXtvNzj340zd98LZ4KntptpfRHwpFOHG188oHXc=
 golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
@@ -95,14 +95,14 @@ golang.org/x/sys v0.0.0-20201119102817-f84b799fce68/go.mod h1:h1NjWce9XRLGQEsW7w
 golang.org/x/sys v0.0.0-20210330210617-4fbd30eecc44/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
 golang.org/x/sys v0.0.0-20210510120138-977fb7262007/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/sys v0.0.0-20210616045830-e2b7044e8c71/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
-golang.org/x/sys v0.12.0 h1:CM0HF96J0hcLAwsHPJZjfdNzs0gftsLfgKt57wWHJ0o=
-golang.org/x/sys v0.12.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
+golang.org/x/sys v0.11.0 h1:eG7RXZHdqOJ1i+0lgLgCpSXAp6M3LYlAo6osgSi0xOM=
+golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
 golang.org/x/term v0.0.0-20201126162022-7de9c90e9dd1/go.mod h1:bj7SfCRtBDWHUb9snDiAeCFNEtKQo2Wmx5Cou7ajbmo=
-golang.org/x/term v0.12.0 h1:/ZfYdc3zq+q02Rv9vGqTeSItdzZTSNDmfTi0mBAuidU=
+golang.org/x/term v0.11.0 h1:F9tnn/DA/Im8nCwm+fX+1/eBwi4qFjRT++MhtVC4ZX0=
 golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
 golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
-golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
-golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
+golang.org/x/text v0.12.0 h1:k+n5B8goJNdU7hSvEtMUz3d1Q6D/XW4COJSJR6fN0mc=
+golang.org/x/text v0.12.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
 golang.org/x/time v0.3.0 h1:rg5rLMjNzMS1RkNLzCG38eapWhnYLFYXDXj2gOlr8j4=
 golang.org/x/time v0.3.0/go.mod h1:tRJNPiyCQ0inRvYxbN9jk5I+vvW/OXSQhTDSoE431IQ=
 golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
diff --git a/pkg/azclient/policy/retryonregionalendpoint/retry_endpoint.go b/pkg/azclient/policy/retryonregionalendpoint/retry_endpoint.go
deleted file mode 100644
index 5e26467fc..000000000
--- a/pkg/azclient/policy/retryonregionalendpoint/retry_endpoint.go
+++ /dev/null
@@ -1,92 +0,0 @@
-/*
-Copyright 2023 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package retryonregionalendpoint
-
-import (
-	"bytes"
-	"encoding/json"
-	"io"
-	"net/http"
-	"strings"
-
-	"github.com/Azure/azure-sdk-for-go/sdk/azcore/policy"
-)
-
-type RetryOnRegionalEndpointPolicy struct {
-	regionalEndpoint string
-}
-
-func (c *RetryOnRegionalEndpointPolicy) Do(request *policy.Request) (*http.Response, error) {
-	response, err := request.Next()
-	switch true {
-	case response == nil,
-		err == nil,
-		request.Raw().Method != http.MethodGet,
-		response.StatusCode == http.StatusNotFound,
-		c.regionalEndpoint == "":
-		return response, err
-	}
-
-	currentHost := request.Raw().URL.Host
-	if request.Raw().Host != "" {
-		currentHost = request.Raw().Host
-	}
-	if strings.HasPrefix(strings.ToLower(currentHost), c.regionalEndpoint) {
-		return response, err
-	}
-
-	bodyBytes, _ := io.ReadAll(response.Body)
-	bodyString := string(bodyBytes)
-	trimmed := strings.TrimSpace(bodyString)
-
-	var retryNeeded bool
-	if (response.ContentLength == 0 || trimmed == "" || trimmed == "{}") && response.StatusCode >= 200 && response.StatusCode < 300 {
-		retryNeeded = true
-	}
-
-	// Hack: retry the regional ARM endpoint in case of ARM traffic split and arm resource group replication is too slow
-	// Empty content and 2xx http status code are returned in this case.
-	// Issue: https://github.com/kubernetes-sigs/cloud-provider-azure/issues/1296
-	// Such situation also needs retrying that ContentLength is -1, StatusCode is 200 and an empty body is returned.
-	var body map[string]map[string]interface{}
-	if unmarshalErr := json.Unmarshal(bodyBytes, &body); unmarshalErr == nil {
-		if errSection, ok := body["error"]; ok {
-			if code, ok := errSection["code"]; ok {
-				if strings.EqualFold(code.(string), "ResourceGroupNotFound") {
-					retryNeeded = true
-				}
-			}
-		}
-	}
-	if retryNeeded {
-		regionalRequest := request.Clone(request.Raw().Context())
-		regionalRequest.Raw().URL.Host = c.regionalEndpoint
-		request.RewindBody()
-
-		regionalResponse, regionalError := regionalRequest.Next()
-
-		// only use the result if the regional request actually goes through and returns 2xx status code, for two reasons:
-		// 1. the retry on regional ARM host approach is a hack.
-		// 2. the concatenated regional uri could be wrong as the rule is not officially declared by ARM.
-		if regionalResponse == nil || regionalResponse.StatusCode > 299 {
-			return response, err
-		}
-		return regionalResponse, regionalError
-	}
-	response.Body = io.NopCloser(bytes.NewBuffer(bodyBytes))
-	return response, err
-}
diff --git a/pkg/consts/consts.go b/pkg/consts/consts.go
index d9ddea625..27e7bd86c 100644
--- a/pkg/consts/consts.go
+++ b/pkg/consts/consts.go
@@ -17,6 +17,7 @@ limitations under the License.
 package consts
 
 import (
+	"strings"
 	"time"
 
 	"github.com/Azure/azure-sdk-for-go/services/storage/mgmt/2021-09-01/storage"
@@ -203,6 +204,8 @@ const (
 	IPVersionDualStackString string = "DualStack"
 )
 
+var IPVersionIPv6StringLower = strings.ToLower(IPVersionIPv6String)
+
 // LB variables for dual-stack
 var (
 	// Service.Spec.LoadBalancerIP has been deprecated and may be removed in a future release. Those two annotations are introduced as alternatives to set IPv4/IPv6 LoadBalancer IPs.
diff --git a/pkg/nodeipam/ipam/range_allocator.go b/pkg/nodeipam/ipam/range_allocator.go
index 4cfa92fc9..009392ad3 100644
--- a/pkg/nodeipam/ipam/range_allocator.go
+++ b/pkg/nodeipam/ipam/range_allocator.go
@@ -191,7 +191,7 @@ func (r *rangeAllocator) worker(stopChan <-chan struct{}) {
 				klog.Warning("Channel nodeCIDRUpdateChannel was unexpectedly closed")
 				return
 			}
-			if workItem, err := r.updateCIDRsAllocation(workItem); err != nil {
+			if err := r.updateCIDRsAllocation(workItem); err != nil {
 				// Requeue the failed node for update again.
 				r.nodeCIDRUpdateChannel <- workItem
 			}
@@ -257,20 +257,23 @@ func (r *rangeAllocator) AllocateOrOccupyCIDR(node *v1.Node) error {
 	if len(node.Spec.PodCIDRs) > 0 {
 		return r.occupyCIDRs(node)
 	}
-
-	// allocate pod cidrs
-	allocatedCIDRs, err := r.allocatePodCIDRs()
-	if err != nil {
-		r.removeNodeFromProcessing(node.Name)
-		nodeutil.RecordNodeStatusChange(r.recorder, node, "CIDRNotAvailable")
-		return fmt.Errorf("failed to allocate cidr for node %s: %w", node.Name, err)
-	}
+	// allocate and queue the assignment
 	allocated := nodeReservedCIDRs{
 		nodeName:       node.Name,
-		allocatedCIDRs: allocatedCIDRs,
+		allocatedCIDRs: make([]*net.IPNet, len(r.cidrSets)),
+	}
+
+	for idx := range r.cidrSets {
+		podCIDR, err := r.cidrSets[idx].AllocateNext()
+		if err != nil {
+			r.removeNodeFromProcessing(node.Name)
+			nodeutil.RecordNodeStatusChange(r.recorder, node, "CIDRNotAvailable")
+			return fmt.Errorf("failed to allocate cidr from cluster cidr at idx:%v: %w", idx, err)
+		}
+		allocated.allocatedCIDRs[idx] = podCIDR
 	}
 
-	// queue the assignment
+	//queue the assignment
 	klog.V(4).Infof("Putting node %s with CIDR %v into the work queue", node.Name, allocated.allocatedCIDRs)
 	r.nodeCIDRUpdateChannel <- allocated
 	return nil
@@ -324,55 +327,21 @@ func filterOutServiceRange(clusterCIDRs []*net.IPNet, cidrSets []*cidrset.CidrSe
 	}
 }
 
-func (r *rangeAllocator) allocatePodCIDRs() ([]*net.IPNet, error) {
-	allocatedCIDRs := make([]*net.IPNet, len(r.cidrSets))
-	for idx := range r.cidrSets {
-		podCIDR, err := r.cidrSets[idx].AllocateNext()
-		if err != nil {
-			for i := 0; i < idx; i++ {
-				if releaseErr := r.cidrSets[idx].Release(allocatedCIDRs[i]); releaseErr != nil {
-					// continue releasing the rest
-					klog.Errorf("Error releasing allocated CIDR at index %d for node: %v", i, releaseErr)
-				}
-			}
-			return nil, fmt.Errorf("failed to allocate cidr from cluster cidr at idx:%v: %w", idx, err)
-		}
-		allocatedCIDRs[idx] = podCIDR
-	}
-	return allocatedCIDRs, nil
-}
-
 // updateCIDRsAllocation assigns CIDR to Node and sends an update to the API server.
-func (r *rangeAllocator) updateCIDRsAllocation(data nodeReservedCIDRs) (dataToRetry nodeReservedCIDRs, err error) {
+func (r *rangeAllocator) updateCIDRsAllocation(data nodeReservedCIDRs) error {
+	var err error
 	var node *v1.Node
-
-	defer func() {
-		if err == nil {
-			// only remove node when node patch succeeds
-			r.removeNodeFromProcessing(data.nodeName)
-		}
-	}()
-
+	defer r.removeNodeFromProcessing(data.nodeName)
+	cidrsString := cidrsAsString(data.allocatedCIDRs)
 	node, err = r.nodeLister.Get(data.nodeName)
 	if err != nil {
 		if apierrors.IsNotFound(err) {
 			klog.Warningf("Failed to get node %s: not found", data.nodeName)
-			return data, nil
+			return nil
 		}
 		klog.Errorf("Failed while getting node %v for updating Node.Spec.PodCIDRs: %v", data.nodeName, err)
-		return data, err
-	}
-
-	// this happens when node patch fails, we release the CIDRs allocated and retry
-	if data.allocatedCIDRs == nil {
-		allocatedCIDRs, err := r.allocatePodCIDRs()
-		if err != nil {
-			nodeutil.RecordNodeStatusChange(r.recorder, node, "CIDRNotAvailable")
-			return data, fmt.Errorf("failed to allocate cidr for node %s: %w", data.nodeName, err)
-		}
-		data.allocatedCIDRs = allocatedCIDRs
+		return err
 	}
-	cidrsString := cidrsAsString(data.allocatedCIDRs)
 
 	// if cidr list matches the proposed.
 	// then we possibly updated this node
@@ -387,7 +356,7 @@ func (r *rangeAllocator) updateCIDRsAllocation(data nodeReservedCIDRs) (dataToRe
 		}
 		if match {
 			klog.V(4).Infof("Node %v already has allocated CIDR %v. It matches the proposed one.", node.Name, data.allocatedCIDRs)
-			return data, nil
+			return nil
 		}
 	}
 
@@ -399,13 +368,13 @@ func (r *rangeAllocator) updateCIDRsAllocation(data nodeReservedCIDRs) (dataToRe
 				klog.Errorf("Error when releasing CIDR idx:%v value: %v err:%v", idx, cidr, releaseErr)
 			}
 		}
-		return data, nil
+		return nil
 	}
 
 	// If we reached here, it means that the node has no CIDR currently assigned. So we set it.
 	for i := 0; i < cidrUpdateRetries; i++ {
 		if err = utilnode.PatchNodeCIDRs(r.client, types.NodeName(node.Name), cidrsString); err == nil {
-			return data, nil
+			return nil
 		}
 	}
 	// failed release back to the pool
@@ -421,9 +390,8 @@ func (r *rangeAllocator) updateCIDRsAllocation(data nodeReservedCIDRs) (dataToRe
 				klog.Errorf("Error releasing allocated CIDR for node %v: %v", node.Name, releaseErr)
 			}
 		}
-		data.allocatedCIDRs = nil
 	}
-	return data, err
+	return err
 }
 
 // converts a slice of cidrs into <c-1>,<c-2>,<c-n>
diff --git a/pkg/nodemanager/nodemanager.go b/pkg/nodemanager/nodemanager.go
index 077600e82..55ef8c9b8 100644
--- a/pkg/nodemanager/nodemanager.go
+++ b/pkg/nodemanager/nodemanager.go
@@ -115,11 +115,6 @@ var updateNetworkConditionBackoff = wait.Backoff{
 	Jitter:   1.0,
 }
 
-const (
-	v4Suffix = "IPv4"
-	v6Suffix = "IPv6"
-)
-
 // CloudNodeController reconciles node information.
 type CloudNodeController struct {
 	nodeName      string
@@ -586,18 +581,18 @@ func nodeAddressesChangeDetected(addressSet1, addressSet2 []v1.NodeAddress) bool
 	addressMap1 := map[string]string{}
 
 	for i := range addressSet1 {
-		suffix := v4Suffix
+		suffix := consts.IPVersionIPv4String
 		if net.ParseIP(addressSet1[i].Address).To4() == nil {
-			suffix = v6Suffix
+			suffix = consts.IPVersionIPv6String
 		}
 		addrType := fmt.Sprintf("%s/%s", addressSet1[i].Type, suffix)
 		addressMap1[addrType] = addressSet1[i].Address
 	}
 
 	for _, v := range addressSet2 {
-		suffix := v4Suffix
+		suffix := consts.IPVersionIPv4String
 		if net.ParseIP(v.Address).To4() == nil {
-			suffix = v6Suffix
+			suffix = consts.IPVersionIPv6String
 		}
 		addrType := fmt.Sprintf("%s/%s", v.Type, suffix)
 		if addressMap1[addrType] != v.Address {
diff --git a/pkg/provider/azure.go b/pkg/provider/azure.go
index dec5be640..a322fe1fc 100644
--- a/pkg/provider/azure.go
+++ b/pkg/provider/azure.go
@@ -1258,18 +1258,19 @@ func (az *Cloud) updateNodeCaches(prevNode, newNode *v1.Node) {
 			az.unmanagedNodes.Delete(prevNode.ObjectMeta.Name)
 		}
 
-		// if the node is being deleted from the cluster, exclude it from load balancers
-		if newNode == nil {
-			az.excludeLoadBalancerNodes.Insert(prevNode.ObjectMeta.Name)
-			az.nodesWithCorrectLoadBalancerByPrimaryVMSet.Delete(strings.ToLower(prevNode.ObjectMeta.Name))
-		}
-
 		// Remove from nodePrivateIPs cache.
 		for _, address := range getNodePrivateIPAddresses(prevNode) {
-			klog.V(4).Infof("removing IP address %s of the node %s", address, prevNode.Name)
+			klog.V(6).Infof("removing IP address %s of the node %s", address, prevNode.Name)
 			az.nodePrivateIPs[prevNode.Name].Delete(address)
 			delete(az.nodePrivateIPToNodeNameMap, address)
 		}
+
+		// if the node is being deleted from the cluster, exclude it from load balancers
+		if newNode == nil {
+			az.excludeLoadBalancerNodes.Insert(prevNode.ObjectMeta.Name)
+			az.nodesWithCorrectLoadBalancerByPrimaryVMSet.Delete(strings.ToLower(prevNode.ObjectMeta.Name))
+			delete(az.nodePrivateIPs, strings.ToLower(prevNode.Name))
+		}
 	}
 
 	if newNode != nil {
@@ -1318,15 +1319,15 @@ func (az *Cloud) updateNodeCaches(prevNode, newNode *v1.Node) {
 
 		// Add to nodePrivateIPs cache
 		for _, address := range getNodePrivateIPAddresses(newNode) {
-			if az.nodePrivateIPs[newNode.Name] == nil {
-				az.nodePrivateIPs[newNode.Name] = sets.New[string]()
+			if az.nodePrivateIPs[strings.ToLower(newNode.Name)] == nil {
+				az.nodePrivateIPs[strings.ToLower(newNode.Name)] = sets.New[string]()
 			}
 			if az.nodePrivateIPToNodeNameMap == nil {
 				az.nodePrivateIPToNodeNameMap = make(map[string]string)
 			}
 
 			klog.V(6).Infof("adding IP address %s of the node %s", address, newNode.Name)
-			az.nodePrivateIPs[newNode.Name].Insert(address)
+			az.nodePrivateIPs[strings.ToLower(newNode.Name)].Insert(address)
 			az.nodePrivateIPToNodeNameMap[address] = newNode.Name
 		}
 	}
diff --git a/pkg/provider/azure_loadbalancer.go b/pkg/provider/azure_loadbalancer.go
index 825a60256..8b66c16c1 100644
--- a/pkg/provider/azure_loadbalancer.go
+++ b/pkg/provider/azure_loadbalancer.go
@@ -167,6 +167,12 @@ func (az *Cloud) reconcileService(ctx context.Context, clusterName string, servi
 	key := strings.ToLower(serviceName)
 	if az.useMultipleStandardLoadBalancers() && isLocalService(service) {
 		az.localServiceNameToServiceInfoMap.Store(key, newServiceInfo(getServiceIPFamily(service), lbName))
+		// There are chances that the endpointslice changes after EnsureHostsInPool, so
+		// need to check endpointslice for a second time.
+		if err := az.checkAndApplyLocalServiceBackendPoolUpdates(*lb, service); err != nil {
+			klog.Errorf("failed to checkAndApplyLocalServiceBackendPoolUpdates: %v", err)
+			return nil, err
+		}
 	} else {
 		az.localServiceNameToServiceInfoMap.Delete(key)
 	}
@@ -1801,18 +1807,9 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 		if az.useMultipleStandardLoadBalancers() {
 			lbToReconcile = *existingLBs
 		}
-		for _, lb := range lbToReconcile {
-			lbName := pointer.StringDeref(lb.Name, "")
-			if lb.LoadBalancerPropertiesFormat != nil && lb.LoadBalancerPropertiesFormat.BackendAddressPools != nil {
-				for _, backendPool := range *lb.LoadBalancerPropertiesFormat.BackendAddressPools {
-					isIPv6 := isBackendPoolIPv6(pointer.StringDeref(backendPool.Name, ""))
-					if strings.EqualFold(pointer.StringDeref(backendPool.Name, ""), az.getBackendPoolNameForService(service, clusterName, isIPv6)) {
-						if err := az.LoadBalancerBackendPool.EnsureHostsInPool(service, nodes, lbBackendPoolIDs[isIPv6], vmSetName, clusterName, lbName, backendPool); err != nil {
-							return nil, err
-						}
-					}
-				}
-			}
+		lb, err = az.reconcileBackendPoolHosts(lb, lbToReconcile, service, nodes, clusterName, vmSetName, lbBackendPoolIDs)
+		if err != nil {
+			return nil, err
 		}
 	}
 
@@ -1824,6 +1821,44 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 	return lb, nil
 }
 
+func (az *Cloud) reconcileBackendPoolHosts(
+	currentLB *network.LoadBalancer,
+	lbs []network.LoadBalancer,
+	service *v1.Service,
+	nodes []*v1.Node,
+	clusterName, vmSetName string,
+	lbBackendPoolIDs map[bool]string,
+) (*network.LoadBalancer, error) {
+	var res *network.LoadBalancer
+	res = currentLB
+	for _, lb := range lbs {
+		lb := lb
+		lbName := pointer.StringDeref(lb.Name, "")
+		if lb.LoadBalancerPropertiesFormat != nil && lb.LoadBalancerPropertiesFormat.BackendAddressPools != nil {
+			for i, backendPool := range *lb.LoadBalancerPropertiesFormat.BackendAddressPools {
+				isIPv6 := isBackendPoolIPv6(pointer.StringDeref(backendPool.Name, ""))
+				if strings.EqualFold(pointer.StringDeref(backendPool.Name, ""), az.getBackendPoolNameForService(service, clusterName, isIPv6)) {
+					if err := az.LoadBalancerBackendPool.EnsureHostsInPool(
+						service,
+						nodes,
+						lbBackendPoolIDs[isIPv6],
+						vmSetName,
+						clusterName,
+						lbName,
+						(*lb.LoadBalancerPropertiesFormat.BackendAddressPools)[i],
+					); err != nil {
+						return nil, err
+					}
+				}
+			}
+		}
+		if strings.EqualFold(lbName, *currentLB.Name) {
+			res = &lb
+		}
+	}
+	return res, nil
+}
+
 // addOrUpdateLBInList adds or updates the given lb in the list
 func addOrUpdateLBInList(lbs *[]network.LoadBalancer, targetLB *network.LoadBalancer) {
 	for i, lb := range *lbs {
diff --git a/pkg/provider/azure_loadbalancer_backendpool.go b/pkg/provider/azure_loadbalancer_backendpool.go
index 28d75bdb0..376f2e897 100644
--- a/pkg/provider/azure_loadbalancer_backendpool.go
+++ b/pkg/provider/azure_loadbalancer_backendpool.go
@@ -352,7 +352,7 @@ func (bc *backendPoolTypeNodeIPConfig) GetBackendPrivateIPs(clusterName string,
 						klog.Errorf("bc.GetBackendPrivateIPs for service (%s): GetNodeNameByIPConfigurationID failed with error: %v", serviceName, err)
 						continue
 					}
-					privateIPsSet, ok := bc.nodePrivateIPs[nodeName]
+					privateIPsSet, ok := bc.nodePrivateIPs[strings.ToLower(nodeName)]
 					if !ok {
 						klog.Warningf("bc.GetBackendPrivateIPs for service (%s): failed to get private IPs of node %s", serviceName, nodeName)
 						continue
@@ -658,7 +658,7 @@ func (bi *backendPoolTypeNodeIP) ReconcileBackendPools(clusterName string, servi
 			bp := newBackendPools[i]
 			var nodeIPAddressesToBeDeleted []string
 			for nodeName := range bi.excludeLoadBalancerNodes {
-				for ip := range bi.nodePrivateIPs[nodeName] {
+				for ip := range bi.nodePrivateIPs[strings.ToLower(nodeName)] {
 					klog.V(2).Infof("bi.ReconcileBackendPools for service (%s): found unwanted node private IP %s, decouple it from the LB %s", serviceName, ip, lbName)
 					nodeIPAddressesToBeDeleted = append(nodeIPAddressesToBeDeleted, ip)
 				}
diff --git a/pkg/provider/azure_local_services.go b/pkg/provider/azure_local_services.go
index 66ff100ed..25ff0e816 100644
--- a/pkg/provider/azure_local_services.go
+++ b/pkg/provider/azure_local_services.go
@@ -33,6 +33,7 @@ import (
 	"k8s.io/client-go/informers"
 	"k8s.io/client-go/tools/cache"
 	"k8s.io/klog/v2"
+	utilnet "k8s.io/utils/net"
 	"k8s.io/utils/pointer"
 
 	"sigs.k8s.io/cloud-provider-azure/pkg/consts"
@@ -286,7 +287,6 @@ func (az *Cloud) getLocalServiceInfo(serviceName string) (*serviceInfo, bool) {
 
 // setUpEndpointSlicesInformer creates an informer for EndpointSlices of local services.
 // It watches the update events and send backend pool update operations to the batch updater.
-// TODO (niqi): the update of endpointslice may be slower than tue update of endpoint pods. Need to fix this.
 func (az *Cloud) setUpEndpointSlicesInformer(informerFactory informers.SharedInformerFactory) {
 	endpointSlicesInformer := informerFactory.Discovery().V1().EndpointSlices().Informer()
 	_, _ = endpointSlicesInformer.AddEventHandler(
@@ -311,7 +311,7 @@ func (az *Cloud) setUpEndpointSlicesInformer(informerFactory informers.SharedInf
 				key := strings.ToLower(fmt.Sprintf("%s/%s", newES.Namespace, svcName))
 				si, found := az.getLocalServiceInfo(key)
 				if !found {
-					klog.V(4).Infof("EndpointSlice %s/%s belongs to service %s, but the service is not a local service, skip updating load balancer backend pool", key, newES.Namespace, newES.Name)
+					klog.V(4).Infof("EndpointSlice %s/%s belongs to service %s, but the service is not a local service, or has not finished the initial reconciliation loop. Skip updating load balancer backend pool", newES.Namespace, newES.Name, key)
 					return
 				}
 				lbName, ipFamily := si.lbName, si.ipFamily
@@ -328,18 +328,13 @@ func (az *Cloud) setUpEndpointSlicesInformer(informerFactory informers.SharedInf
 					}
 				}
 				for _, previousNodeName := range previousNodeNames {
-					nodeIPsSet := az.nodePrivateIPs[previousNodeName]
+					nodeIPsSet := az.nodePrivateIPs[strings.ToLower(previousNodeName)]
 					previousIPs = append(previousIPs, setToStrings(nodeIPsSet)...)
 				}
 				for _, currentNodeName := range currentNodeNames {
-					nodeIPsSet := az.nodePrivateIPs[currentNodeName]
+					nodeIPsSet := az.nodePrivateIPs[strings.ToLower(currentNodeName)]
 					currentIPs = append(currentIPs, setToStrings(nodeIPsSet)...)
 				}
-				ipsToBeDeleted := compareNodeIPs(previousIPs, currentIPs)
-				if len(ipsToBeDeleted) == 0 && len(previousIPs) == len(currentIPs) {
-					klog.V(4).Infof("No IP change detected for EndpointSlice %s/%s, skip updating load balancer backend pool", newES.Namespace, newES.Name)
-					return
-				}
 
 				if az.backendPoolUpdater != nil {
 					var bpNames []string
@@ -353,14 +348,11 @@ func (az *Cloud) setUpEndpointSlicesInformer(informerFactory informers.SharedInf
 					default:
 						bpNames = append(bpNames, bpNameIPv4, bpNameIPv6)
 					}
+					currentIPsInBackendPools := make(map[string][]string)
 					for _, bpName := range bpNames {
-						if len(ipsToBeDeleted) > 0 {
-							az.backendPoolUpdater.addOperation(getRemoveIPsFromBackendPoolOperation(key, lbName, bpName, ipsToBeDeleted))
-						}
-						if len(currentIPs) > 0 {
-							az.backendPoolUpdater.addOperation(getAddIPsToBackendPoolOperation(key, lbName, bpName, currentIPs))
-						}
+						currentIPsInBackendPools[bpName] = previousIPs
 					}
+					az.applyIPChangesAmongLocalServiceBackendPoolsByIPFamily(lbName, key, currentIPsInBackendPools, currentIPs)
 				}
 			},
 			DeleteFunc: func(obj interface{}) {
@@ -408,7 +400,7 @@ func compareNodeIPs(previousIPs, currentIPs []string) []string {
 func getLocalServiceBackendPoolName(serviceName string, ipv6 bool) string {
 	serviceName = strings.ToLower(strings.Replace(serviceName, "/", "-", -1))
 	if ipv6 {
-		return fmt.Sprintf("%s-ipv6", serviceName)
+		return fmt.Sprintf("%s-%s", serviceName, consts.IPVersionIPv6StringLower)
 	}
 	return serviceName
 }
@@ -471,12 +463,16 @@ func newServiceInfo(ipFamily, lbName string) *serviceInfo {
 
 // getLocalServiceEndpointsNodeNames gets the node names that host all endpoints of the local service.
 func (az *Cloud) getLocalServiceEndpointsNodeNames(service *v1.Service) (sets.Set[string], error) {
-	var ep *discovery_v1.EndpointSlice
+	var (
+		ep           *discovery_v1.EndpointSlice
+		foundInCache bool
+	)
 	az.endpointSlicesCache.Range(func(key, value interface{}) bool {
 		endpointSlice := value.(*discovery_v1.EndpointSlice)
 		if strings.EqualFold(getServiceNameOfEndpointSlice(endpointSlice), service.Name) &&
 			strings.EqualFold(endpointSlice.Namespace, service.Namespace) {
 			ep = endpointSlice
+			foundInCache = true
 			return false
 		}
 		return true
@@ -499,6 +495,9 @@ func (az *Cloud) getLocalServiceEndpointsNodeNames(service *v1.Service) (sets.Se
 	if ep == nil {
 		return nil, fmt.Errorf("failed to find EndpointSlice for service %s/%s", service.Namespace, service.Name)
 	}
+	if !foundInCache {
+		az.endpointSlicesCache.Store(strings.ToLower(fmt.Sprintf("%s/%s", ep.Namespace, ep.Name)), ep)
+	}
 
 	var nodeNames []string
 	for _, endpoint := range ep.Endpoints {
@@ -544,3 +543,84 @@ func (az *Cloud) cleanupLocalServiceBackendPool(
 	}
 	return lbs, nil
 }
+
+// checkAndApplyLocalServiceBackendPoolUpdates if the IPs in the backend pool are aligned
+// with the corresponding endpointslice, and update the backend pool if necessary.
+func (az *Cloud) checkAndApplyLocalServiceBackendPoolUpdates(lb network.LoadBalancer, service *v1.Service) error {
+	serviceName := getServiceName(service)
+	endpointsNodeNames, err := az.getLocalServiceEndpointsNodeNames(service)
+	if err != nil {
+		return err
+	}
+	var expectedIPs []string
+	for nodeName := range endpointsNodeNames {
+		ips := az.nodePrivateIPs[strings.ToLower(nodeName)]
+		for ip := range ips {
+			expectedIPs = append(expectedIPs, ip)
+		}
+	}
+	currentIPsInBackendPools := make(map[string][]string)
+	for _, bp := range *lb.BackendAddressPools {
+		bpName := pointer.StringDeref(bp.Name, "")
+		if localServiceOwnsBackendPool(serviceName, bpName) {
+			var currentIPs []string
+			for _, address := range *bp.LoadBalancerBackendAddresses {
+				currentIPs = append(currentIPs, *address.IPAddress)
+			}
+			currentIPsInBackendPools[bpName] = currentIPs
+		}
+	}
+	az.applyIPChangesAmongLocalServiceBackendPoolsByIPFamily(*lb.Name, serviceName, currentIPsInBackendPools, expectedIPs)
+
+	return nil
+}
+
+// applyIPChangesAmongLocalServiceBackendPoolsByIPFamily reconciles IPs by IP family
+// amone the backend pools of a local service.
+func (az *Cloud) applyIPChangesAmongLocalServiceBackendPoolsByIPFamily(
+	lbName, serviceName string,
+	currentIPsInBackendPools map[string][]string,
+	expectedIPs []string,
+) {
+	currentIPsInBackendPoolsIPv4 := make(map[string][]string)
+	currentIPsInBackendPoolsIPv6 := make(map[string][]string)
+	for bpName, ips := range currentIPsInBackendPools {
+		if managedResourceHasIPv6Suffix(bpName) {
+			currentIPsInBackendPoolsIPv6[bpName] = ips
+		} else {
+			currentIPsInBackendPoolsIPv4[bpName] = ips
+		}
+	}
+
+	var ipv4, ipv6 []string
+	for _, ip := range expectedIPs {
+		if utilnet.IsIPv6String(ip) {
+			ipv6 = append(ipv6, ip)
+		} else {
+			ipv4 = append(ipv4, ip)
+		}
+	}
+	az.reconcileIPsInLocalServiceBackendPoolsAsync(lbName, serviceName, currentIPsInBackendPoolsIPv6, ipv6)
+	az.reconcileIPsInLocalServiceBackendPoolsAsync(lbName, serviceName, currentIPsInBackendPoolsIPv4, ipv4)
+}
+
+// reconcileIPsInLocalServiceBackendPoolsAsync reconciles IPs in the backend pools of a local service.
+func (az *Cloud) reconcileIPsInLocalServiceBackendPoolsAsync(
+	lbName, serviceName string,
+	currentIPsInBackendPools map[string][]string,
+	expectedIPs []string,
+) {
+	for bpName, currentIPs := range currentIPsInBackendPools {
+		ipsToBeDeleted := compareNodeIPs(currentIPs, expectedIPs)
+		if len(ipsToBeDeleted) == 0 && len(currentIPs) == len(expectedIPs) {
+			klog.V(4).Infof("No IP change detected for service %s, skip updating load balancer backend pool", serviceName)
+			return
+		}
+		if len(ipsToBeDeleted) > 0 {
+			az.backendPoolUpdater.addOperation(getRemoveIPsFromBackendPoolOperation(serviceName, lbName, bpName, ipsToBeDeleted))
+		}
+		if len(expectedIPs) > 0 {
+			az.backendPoolUpdater.addOperation(getAddIPsToBackendPoolOperation(serviceName, lbName, bpName, expectedIPs))
+		}
+	}
+}
diff --git a/pkg/provider/azure_standard.go b/pkg/provider/azure_standard.go
index 1040317be..9f5dd0594 100644
--- a/pkg/provider/azure_standard.go
+++ b/pkg/provider/azure_standard.go
@@ -54,14 +54,6 @@ var (
 	vmasIDRE           = regexp.MustCompile(`/subscriptions/(?:.*)/resourceGroups/(?:.*)/providers/Microsoft.Compute/availabilitySets/(.+)`)
 )
 
-const (
-	v6Suffix = "IPv6"
-)
-
-var (
-	v6SuffixLower = strings.ToLower(v6Suffix)
-)
-
 // returns the full identifier of an availabilitySet
 func (az *Cloud) getAvailabilitySetID(resourceGroup, availabilitySetName string) string {
 	return fmt.Sprintf(
@@ -270,7 +262,7 @@ func isInternalLoadBalancer(lb *network.LoadBalancer) bool {
 // clusters moving from IPv6 to dualstack will require no changes as the IPv4 backend pool will created with <clusterName>
 func getBackendPoolName(clusterName string, isIPv6 bool) string {
 	if isIPv6 {
-		return fmt.Sprintf("%s-%s", clusterName, v6Suffix)
+		return fmt.Sprintf("%s-%s", clusterName, consts.IPVersionIPv6String)
 	}
 
 	return clusterName
@@ -290,7 +282,7 @@ func isBackendPoolIPv6(name string) bool {
 }
 
 func managedResourceHasIPv6Suffix(name string) bool {
-	return strings.HasSuffix(strings.ToLower(name), fmt.Sprintf("-%s", v6SuffixLower))
+	return strings.HasSuffix(strings.ToLower(name), fmt.Sprintf("-%s", consts.IPVersionIPv6StringLower))
 }
 
 func (az *Cloud) getLoadBalancerRuleName(service *v1.Service, protocol v1.Protocol, port int32, isIPv6 bool) string {
diff --git a/pkg/provider/azure_storageaccount.go b/pkg/provider/azure_storageaccount.go
index 7e50bb2dd..ee0fe7db9 100644
--- a/pkg/provider/azure_storageaccount.go
+++ b/pkg/provider/azure_storageaccount.go
@@ -56,7 +56,7 @@ type AccountOptions struct {
 	EnableHTTPSTrafficOnly                    bool
 	// indicate whether create new account when Name is empty or when account does not exists
 	CreateAccount                           bool
-	CreatePrivateEndpoint                   *bool
+	CreatePrivateEndpoint                   bool
 	StorageType                             StorageType
 	StorageEndpointSuffix                   string
 	DisableFileServiceDeleteRetentionPolicy *bool
@@ -216,7 +216,7 @@ func (az *Cloud) EnsureStorageAccount(ctx context.Context, accountOptions *Accou
 	}
 
 	var privateDNSZoneName string
-	if pointer.BoolDeref(accountOptions.CreatePrivateEndpoint, false) {
+	if accountOptions.CreatePrivateEndpoint {
 		if accountOptions.StorageType == "" {
 			klog.V(2).Info("set StorageType as file when not specified")
 			accountOptions.StorageType = StorageTypeFile
@@ -282,7 +282,7 @@ func (az *Cloud) EnsureStorageAccount(ctx context.Context, accountOptions *Accou
 		}
 	}
 
-	if pointer.BoolDeref(accountOptions.CreatePrivateEndpoint, false) {
+	if accountOptions.CreatePrivateEndpoint {
 		if _, err := az.privatednsclient.Get(ctx, vnetResourceGroup, privateDNSZoneName); err != nil {
 			klog.V(2).Infof("get private dns zone %s returned with %v", privateDNSZoneName, err.Error())
 			// Create DNS zone first, this could make sure driver has write permission on vnetResourceGroup
@@ -320,7 +320,7 @@ func (az *Cloud) EnsureStorageAccount(ctx context.Context, accountOptions *Accou
 			}
 		}
 
-		if pointer.BoolDeref(accountOptions.CreatePrivateEndpoint, false) {
+		if accountOptions.CreatePrivateEndpoint {
 			networkRuleSet = &storage.NetworkRuleSet{
 				DefaultAction: storage.DefaultActionDeny,
 			}
@@ -471,7 +471,7 @@ func (az *Cloud) EnsureStorageAccount(ctx context.Context, accountOptions *Accou
 		}
 	}
 
-	if pointer.BoolDeref(accountOptions.CreatePrivateEndpoint, false) {
+	if accountOptions.CreatePrivateEndpoint {
 		// Get properties of the storageAccount
 		storageAccount, err := az.StorageAccountClient.GetProperties(ctx, subsID, resourceGroup, accountName)
 		if err != nil {
@@ -770,15 +770,10 @@ func isEnableNfsV3PropertyEqual(account storage.Account, accountOptions *Account
 }
 
 func isPrivateEndpointAsExpected(account storage.Account, accountOptions *AccountOptions) bool {
-	if accountOptions.CreatePrivateEndpoint == nil {
-		// CreatePrivateEndpoint is not set, match current account
+	if accountOptions.CreatePrivateEndpoint && account.PrivateEndpointConnections != nil && len(*account.PrivateEndpointConnections) > 0 {
 		return true
 	}
-
-	if pointer.BoolDeref(accountOptions.CreatePrivateEndpoint, false) && account.PrivateEndpointConnections != nil && len(*account.PrivateEndpointConnections) > 0 {
-		return true
-	}
-	if !pointer.BoolDeref(accountOptions.CreatePrivateEndpoint, false) && (account.PrivateEndpointConnections == nil || len(*account.PrivateEndpointConnections) == 0) {
+	if !accountOptions.CreatePrivateEndpoint && (account.PrivateEndpointConnections == nil || len(*account.PrivateEndpointConnections) == 0) {
 		return true
 	}
 	return false
diff --git a/pkg/provider/azure_utils.go b/pkg/provider/azure_utils.go
index e8724ef6a..7c0549874 100644
--- a/pkg/provider/azure_utils.go
+++ b/pkg/provider/azure_utils.go
@@ -453,7 +453,7 @@ func getServicePIPPrefixID(service *v1.Service, isIPv6 bool) string {
 // the old PIPs will be recreated.
 func getResourceByIPFamily(resource string, isDualStack, isIPv6 bool) string {
 	if isDualStack && isIPv6 {
-		return fmt.Sprintf("%s-%s", resource, v6Suffix)
+		return fmt.Sprintf("%s-%s", resource, consts.IPVersionIPv6String)
 	}
 	return resource
 }
diff --git a/pkg/util/node/node.go b/pkg/util/node/node.go
index 714f1eb12..9e82db100 100644
--- a/pkg/util/node/node.go
+++ b/pkg/util/node/node.go
@@ -78,7 +78,7 @@ func PatchNodeCIDRs(c clientset.Interface, node types.NodeName, cidrs []string)
 	if err != nil {
 		return fmt.Errorf("failed to json.Marshal CIDR: %w", err)
 	}
-	klog.V(4).Infof("cidrs patch bytes for node %s are:%s", string(node), string(patchBytes))
+	klog.V(4).Infof("cidrs patch bytes are:%s", string(patchBytes))
 	if _, err := c.CoreV1().Nodes().Patch(context.TODO(), string(node), types.StrategicMergePatchType, patchBytes, metav1.PatchOptions{}); err != nil {
 		return fmt.Errorf("failed to patch node CIDR: %w", err)
 	}


Test patch:
diff --git a/e2e.Dockerfile b/e2e.Dockerfile
index 53b4e75b0..958190c2e 100644
--- a/e2e.Dockerfile
+++ b/e2e.Dockerfile
@@ -14,7 +14,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM golang:1.21-bullseye@sha256:b61af81a1e082e061b8addc222f2b1fd472a34eaf6950823e4f2743498ee4924
+FROM golang:1.21-bullseye@sha256:02f350d8452d3f9693a450586659ecdc6e40e9be8f8dfc6d402300d87223fdfa
 
 WORKDIR /go/src/sigs.k8s.io/cloud-provider-azure
 
diff --git a/kubetest2-aks/go.mod b/kubetest2-aks/go.mod
index 7be11ee38..20640fe85 100644
--- a/kubetest2-aks/go.mod
+++ b/kubetest2-aks/go.mod
@@ -3,7 +3,7 @@ module sigs.k8s.io/cloud-provider-azure/kubetest2-aks
 go 1.18
 
 require (
-	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.2
+	github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.1
 	github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.3.1
 	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/containerservice/armcontainerservice/v2 v2.4.0
 	github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/resources/armresources v1.1.1
diff --git a/kubetest2-aks/go.sum b/kubetest2-aks/go.sum
index 5dfa9cf7c..9df4aab08 100644
--- a/kubetest2-aks/go.sum
+++ b/kubetest2-aks/go.sum
@@ -1,7 +1,7 @@
 dario.cat/mergo v1.0.0 h1:AGCNq9Evsj31mOgNPcLyXc+4PNABt905YmuqPYYpBWk=
 dario.cat/mergo v1.0.0/go.mod h1:uNxQE+84aUszobStD9th8a29P2fMDhsBdgRYvZOxGmk=
-github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.2 h1:t5+QXLCK9SVi0PPdaY0PrFvYUo24KwA0QwxnaHRSVd4=
-github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.2/go.mod h1:bjGvMhVMb+EEm3VRNQawDMUyMMjo+S5ewNjflkep/0Q=
+github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.1 h1:/iHxaJhsFr0+xVFfbMr5vxz848jyiWuIEDhYq3y5odY=
+github.com/Azure/azure-sdk-for-go/sdk/azcore v1.7.1/go.mod h1:bjGvMhVMb+EEm3VRNQawDMUyMMjo+S5ewNjflkep/0Q=
 github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.3.1 h1:LNHhpdK7hzUcx/k1LIcuh5k7k1LGIWLQfCjaneSj7Fc=
 github.com/Azure/azure-sdk-for-go/sdk/azidentity v1.3.1/go.mod h1:uE9zaUfEQT/nbQjVi2IblCG9iaLtZsuYZ8ne+PuQ02M=
 github.com/Azure/azure-sdk-for-go/sdk/internal v1.3.0 h1:sXr+ck84g/ZlZUOZiNELInmMgOsuGwdjjVkEIde0OtY=
diff --git a/pkg/nodeipam/ipam/range_allocator_test.go b/pkg/nodeipam/ipam/range_allocator_test.go
index c08d334bd..69371fd6f 100644
--- a/pkg/nodeipam/ipam/range_allocator_test.go
+++ b/pkg/nodeipam/ipam/range_allocator_test.go
@@ -854,91 +854,3 @@ func TestReleaseCIDRSuccess(t *testing.T) {
 		testFunc(tc)
 	}
 }
-
-func TestRetryAllocatingPodCidr(t *testing.T) {
-	testCases := []testCase{
-		{
-			description: "It should release allocated CIDR and retry with new CIDR when node patch fails",
-			fakeNodeHandler: &testutil.FakeNodeHandler{
-				Existing: []*v1.Node{
-					{
-						ObjectMeta: metav1.ObjectMeta{
-							Name: "node0",
-						},
-					},
-				},
-				Clientset:       fake.NewSimpleClientset(),
-				PatchErrorCount: 9, // need 9 because we want first 3 ranges to fail, and each range is retried for 3 times
-			},
-			allocatorParams: CIDRAllocatorParams{
-				ClusterCIDRs: func() []*net.IPNet {
-					_, clusterCIDR, _ := net.ParseCIDR("127.123.234.0/28")
-					return []*net.IPNet{clusterCIDR}
-				}(),
-				ServiceCIDR:          nil,
-				SecondaryServiceCIDR: nil,
-				NodeCIDRMaskSizes:    []int{30},
-			},
-			allocatedCIDRs: map[int][]string{
-				0: {"127.123.234.0/30", "127.123.234.4/30", "127.123.234.8/30"},
-			},
-			expectedAllocatedCIDR: map[int]string{
-				0: "127.123.234.12/30",
-			},
-		},
-	}
-
-	testFunc := func(tc testCase) {
-		// Initialize the range allocator.
-		allocator, err := NewCIDRRangeAllocator(tc.fakeNodeHandler, getFakeNodeInformer(tc.fakeNodeHandler), tc.allocatorParams, nil)
-		if err != nil {
-			t.Logf("%v: failed to create CIDRRangeAllocator with error %v", tc.description, err)
-		}
-		rangeAllocator, ok := allocator.(*rangeAllocator)
-		if !ok {
-			t.Logf("%v: found non-default implementation of CIDRAllocator, skipping white-box test...", tc.description)
-			return
-		}
-		rangeAllocator.nodesSynced = alwaysReady
-		rangeAllocator.recorder = testutil.NewFakeRecorder()
-		go allocator.Run(wait.NeverStop)
-
-		if err := allocator.AllocateOrOccupyCIDR(tc.fakeNodeHandler.Existing[0]); err != nil {
-			t.Errorf("%v: unexpected error in AllocateOrOccupyCIDR: %v", tc.description, err)
-		}
-
-		if err := waitForUpdatedNodeWithTimeout(tc.fakeNodeHandler, 1, wait.ForeverTestTimeout); err != nil {
-			t.Fatalf("%v: timeout while waiting for Node update: %v", tc.description, err)
-		}
-
-		for _, updatedNode := range tc.fakeNodeHandler.GetUpdatedNodesCopy() {
-			if len(updatedNode.Spec.PodCIDRs) == 0 {
-				continue // not assigned yet
-			}
-			//match
-			for podCIDRIdx, expectedPodCIDR := range tc.expectedAllocatedCIDR {
-				if updatedNode.Spec.PodCIDRs[podCIDRIdx] != expectedPodCIDR {
-					t.Errorf("%v: Unable to find allocated CIDR %v, found updated Nodes with CIDRs: %v", tc.description, expectedPodCIDR, updatedNode.Spec.PodCIDRs)
-					break
-				}
-			}
-		}
-
-		// Make sure previously tried CIDRs are all released
-		for idx, allocatedList := range tc.allocatedCIDRs {
-			for _, allocated := range allocatedList {
-				_, cidr, err := net.ParseCIDR(allocated)
-				if err != nil {
-					t.Fatalf("%v: unexpected error when parsing CIDR %v: %v", tc.description, allocated, err)
-				}
-				if err = rangeAllocator.cidrSets[idx].Occupy(cidr); err != nil {
-					t.Fatalf("%v: unexpected error when occupying CIDR %v: %v", tc.description, allocated, err)
-				}
-			}
-		}
-	}
-
-	for _, tc := range testCases {
-		testFunc(tc)
-	}
-}
diff --git a/pkg/provider/azure_loadbalancer_healthprobe_test.go b/pkg/provider/azure_loadbalancer_healthprobe_test.go
index 5603e88d5..27726521e 100644
--- a/pkg/provider/azure_loadbalancer_healthprobe_test.go
+++ b/pkg/provider/azure_loadbalancer_healthprobe_test.go
@@ -39,7 +39,7 @@ func getTestProbes(protocol, path string, interval, servicePort, probePort, numO
 func getTestProbe(protocol, path string, interval, servicePort, probePort, numOfProbe *int32, isIPv6 bool) network.Probe {
 	suffix := ""
 	if isIPv6 {
-		suffix = "-" + v6Suffix
+		suffix = "-" + consts.IPVersionIPv6String
 	}
 	expectedProbes := network.Probe{
 		Name: pointer.String(fmt.Sprintf("atest1-TCP-%d", *servicePort) + suffix),
diff --git a/pkg/provider/azure_loadbalancer_test.go b/pkg/provider/azure_loadbalancer_test.go
index c5e3b9d84..eff140481 100644
--- a/pkg/provider/azure_loadbalancer_test.go
+++ b/pkg/provider/azure_loadbalancer_test.go
@@ -3023,7 +3023,7 @@ func getTCPResetTestRules(enableTCPReset bool) map[bool][]network.LoadBalancingR
 func getTestRule(enableTCPReset bool, port int32, isIPv6 bool) network.LoadBalancingRule {
 	suffix := ""
 	if isIPv6 {
-		suffix = "-" + v6Suffix
+		suffix = "-" + consts.IPVersionIPv6String
 	}
 	expectedRules := network.LoadBalancingRule{
 		Name: pointer.String(fmt.Sprintf("atest1-TCP-%d", port) + suffix),
@@ -3057,7 +3057,7 @@ func getHATestRules(enableTCPReset, hasProbe bool, protocol v1.Protocol, isIPv6,
 	suffix := ""
 	enableFloatingIP := true
 	if isIPv6 {
-		suffix = "-" + v6Suffix
+		suffix = "-" + consts.IPVersionIPv6String
 		if isInternal {
 			enableFloatingIP = false
 		}
@@ -3096,7 +3096,7 @@ func getHATestRules(enableTCPReset, hasProbe bool, protocol v1.Protocol, isIPv6,
 func getFloatingIPTestRule(enableTCPReset, enableFloatingIP bool, port int32, isIPv6 bool) network.LoadBalancingRule {
 	suffix := ""
 	if isIPv6 {
-		suffix = "-" + v6Suffix
+		suffix = "-" + consts.IPVersionIPv6String
 	}
 	expectedRules := network.LoadBalancingRule{
 		Name: pointer.String(fmt.Sprintf("atest1-TCP-%d%s", port, suffix)),
@@ -9002,3 +9002,69 @@ func TestAddOrUpdateLBInList(t *testing.T) {
 	addOrUpdateLBInList(&existingLBs, &targetLB)
 	assert.Equal(t, expectedLBs, existingLBs)
 }
+
+func TestReconcileBackendPoolHosts(t *testing.T) {
+	ctrl := gomock.NewController(t)
+	defer ctrl.Finish()
+
+	svc := getTestService("test", v1.ProtocolTCP, nil, false)
+	lbBackendPoolIDs := map[bool]string{false: "id"}
+	clusterName := "kubernetes"
+	ips := []string{"10.0.0.1"}
+	bp1 := buildTestLoadBalancerBackendPoolWithIPs(clusterName, ips)
+	bp2 := buildTestLoadBalancerBackendPoolWithIPs(clusterName, ips)
+	ips = []string{"10.0.0.2", "10.0.0.3"}
+	bp3 := buildTestLoadBalancerBackendPoolWithIPs(clusterName, ips)
+	lb1 := &network.LoadBalancer{
+		Name: pointer.String(clusterName),
+		LoadBalancerPropertiesFormat: &network.LoadBalancerPropertiesFormat{
+			BackendAddressPools: &[]network.BackendAddressPool{bp1},
+		},
+	}
+	lb2 := &network.LoadBalancer{
+		Name: pointer.String("lb2"),
+		LoadBalancerPropertiesFormat: &network.LoadBalancerPropertiesFormat{
+			BackendAddressPools: &[]network.BackendAddressPool{bp2},
+		},
+	}
+	expectedLB := &network.LoadBalancer{
+		Name: pointer.String(clusterName),
+		LoadBalancerPropertiesFormat: &network.LoadBalancerPropertiesFormat{
+			BackendAddressPools: &[]network.BackendAddressPool{bp3},
+		},
+	}
+	existingLBs := []network.LoadBalancer{*lb1, *lb2}
+
+	cloud := GetTestCloud(ctrl)
+	mockLBBackendPool := NewMockBackendPool(ctrl)
+	mockLBBackendPool.EXPECT().EnsureHostsInPool(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), bp1).DoAndReturn(fakeEnsureHostsInPool())
+	mockLBBackendPool.EXPECT().EnsureHostsInPool(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), bp2).Return(nil)
+	cloud.LoadBalancerBackendPool = mockLBBackendPool
+
+	var err error
+	lb1, err = cloud.reconcileBackendPoolHosts(lb1, existingLBs, &svc, []*v1.Node{}, clusterName, "vmss", lbBackendPoolIDs)
+	assert.NoError(t, err)
+	assert.Equal(t, expectedLB, lb1)
+
+	mockLBBackendPool.EXPECT().EnsureHostsInPool(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).Return(errors.New("error"))
+	_, err = cloud.reconcileBackendPoolHosts(lb1, existingLBs, &svc, []*v1.Node{}, clusterName, "vmss", lbBackendPoolIDs)
+	assert.Equal(t, errors.New("error"), err)
+}
+
+func fakeEnsureHostsInPool() func(*v1.Service, []*v1.Node, string, string, string, string, network.BackendAddressPool) error {
+	return func(svc *v1.Service, nodes []*v1.Node, lbBackendPoolID, vmSet, clusterName, lbName string, backendPool network.BackendAddressPool) error {
+		backendPool.LoadBalancerBackendAddresses = &[]network.LoadBalancerBackendAddress{
+			{
+				LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+					IPAddress: pointer.String("10.0.0.2"),
+				},
+			},
+			{
+				LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+					IPAddress: pointer.String("10.0.0.3"),
+				},
+			},
+		}
+		return nil
+	}
+}
diff --git a/pkg/provider/azure_local_services_test.go b/pkg/provider/azure_local_services_test.go
index ab7d57a9e..a32ed6dba 100644
--- a/pkg/provider/azure_local_services_test.go
+++ b/pkg/provider/azure_local_services_test.go
@@ -34,6 +34,7 @@ import (
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/apimachinery/pkg/util/sets"
 	"k8s.io/client-go/informers"
+	"k8s.io/client-go/kubernetes"
 	"k8s.io/client-go/kubernetes/fake"
 	"k8s.io/utils/pointer"
 
@@ -223,26 +224,6 @@ func TestLoadBalancerBackendPoolUpdater(t *testing.T) {
 				u.removeOperation(tc.removeOperationServiceName)
 			}
 			time.Sleep(3 * time.Second)
-			//err := wait.PollUntilContextTimeout(context.Background(), time.Second, 5*time.Second, true, func(ctx context.Context) (bool, error) {
-			//	resultLen := 0
-			//	results.Range(func(key, value interface{}) bool {
-			//		resultLen++
-			//		return true
-			//	})
-			//	return resultLen == len(tc.expectedResult), nil
-			//})
-			//assert.NoError(t, err)
-
-			//actualResults := make(map[batchOperationResult]bool)
-			//results.Range(func(key, value interface{}) bool {
-			//	actualResults[key.(batchOperationResult)] = true
-			//	return true
-			//})
-
-			//err = wait.PollUntilContextTimeout(ctx, time.Second, 5*time.Second, true, func(ctx context.Context) (bool, error) {
-			//	return assert.Equal(t, tc.expectedResult, actualResults, tc.name), nil
-			//})
-			//assert.NoError(t, err)
 		})
 	}
 }
@@ -262,9 +243,6 @@ func TestLoadBalancerBackendPoolUpdaterFailed(t *testing.T) {
 		putBackendPoolErr                  *retry.Error
 		expectedCreateOrUpdateBackendPools []network.BackendAddressPool
 		expectedBackendPools               []network.BackendAddressPool
-		//expectedResult                     map[batchOperationResult]bool
-		//expectedPoolNameToErrMsg           map[string]string
-		//expectedResultCount                int
 	}{
 		{
 			name:       "Retriable error when getting backend pool",
@@ -279,10 +257,6 @@ func TestLoadBalancerBackendPoolUpdaterFailed(t *testing.T) {
 			expectedBackendPools: []network.BackendAddressPool{
 				getTestBackendAddressPoolWithIPs("lb1", "pool1", []string{"10.0.0.1", "10.0.0.2"}),
 			},
-			//expectedResult: map[batchOperationResult]bool{
-			//	newBatchOperationResult("lb1/pool1", true, nil): true,
-			//},
-			//expectedResultCount: 1,
 		},
 		{
 			name:       "Retriable error when updating backend pool",
@@ -299,10 +273,6 @@ func TestLoadBalancerBackendPoolUpdaterFailed(t *testing.T) {
 			expectedBackendPools: []network.BackendAddressPool{
 				getTestBackendAddressPoolWithIPs("lb1", "pool1", []string{"10.0.0.1", "10.0.0.2"}),
 			},
-			//expectedResult: map[batchOperationResult]bool{
-			//	newBatchOperationResult("lb1/pool1", true, nil): true,
-			//},
-			//expectedResultCount: 1,
 		},
 		{
 			name:       "Non-retriable error when getting backend pool",
@@ -314,8 +284,6 @@ func TestLoadBalancerBackendPoolUpdaterFailed(t *testing.T) {
 			expectedBackendPools: []network.BackendAddressPool{
 				getTestBackendAddressPoolWithIPs("lb1", "pool1", []string{}),
 			},
-			//expectedPoolNameToErrMsg: map[string]string{"lb1/pool1": "Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error"},
-			//expectedResultCount:      1,
 		},
 		{
 			name:       "Non-retriable error when updating backend pool",
@@ -328,8 +296,6 @@ func TestLoadBalancerBackendPoolUpdaterFailed(t *testing.T) {
 			expectedCreateOrUpdateBackendPools: []network.BackendAddressPool{
 				getTestBackendAddressPoolWithIPs("lb1", "pool1", []string{"10.0.0.1", "10.0.0.2"}),
 			},
-			//expectedPoolNameToErrMsg: map[string]string{"lb1/pool1": "Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error"},
-			//expectedResultCount:      1,
 		},
 		{
 			name:       "Backend pool not found",
@@ -416,48 +382,12 @@ func TestLoadBalancerBackendPoolUpdaterFailed(t *testing.T) {
 			defer cancel()
 			go u.run(ctx)
 
-			//results := sync.Map{}
 			for _, op := range tc.operations {
 				op := op
-				//go func() {
 				u.addOperation(op)
-				//result := op.wait()
-				//results.Store(result, true)
-				//}()
 				time.Sleep(100 * time.Millisecond)
 			}
 			time.Sleep(3 * time.Second)
-			//err := wait.PollUntilContextTimeout(context.Background(), time.Second, 5*time.Second, true, func(ctx context.Context) (bool, error) {
-			//	resultLen := 0
-			//	results.Range(func(key, value interface{}) bool {
-			//		resultLen++
-			//		return true
-			//	})
-			//	return resultLen == tc.expectedResultCount, nil
-			//})
-			//assert.NoError(t, err)
-
-			//actualResults := make(map[batchOperationResult]bool)
-			//poolNameToErrMsg := make(map[string]string)
-			//results.Range(func(key, value interface{}) bool {
-			//	actualResults[key.(batchOperationResult)] = true
-			//	if tc.getBackendPoolErr != nil && !tc.getBackendPoolErr.Retriable ||
-			//		tc.putBackendPoolErr != nil && !tc.putBackendPoolErr.Retriable {
-			//		poolNameToErrMsg[key.(batchOperationResult).name] = key.(batchOperationResult).err.Error()
-			//	}
-			//	return true
-			//})
-
-			//err = wait.PollUntilContextTimeout(ctx, time.Second, 5*time.Second, true, func(ctx context.Context) (bool, error) {
-			//	if tc.expectedResult != nil {
-			//		return assert.Equal(t, tc.expectedResult, actualResults, tc.name), nil
-			//	}
-			//	if tc.expectedPoolNameToErrMsg != nil {
-			//		return assert.Equal(t, tc.expectedPoolNameToErrMsg, poolNameToErrMsg, tc.name), nil
-			//	}
-			//	return false, errors.New("unexpected result")
-			//})
-			//assert.NoError(t, err)
 		})
 	}
 }
@@ -604,3 +534,86 @@ func TestGetBackendPoolNamesAndIDsForService(t *testing.T) {
 	_ = cloud.getBackendPoolNamesForService(&svc, "test")
 	_ = cloud.getBackendPoolIDsForService(&svc, "test", "lb")
 }
+
+func TestCheckAndApplyLocalServiceBackendPoolUpdates(t *testing.T) {
+	ctrl := gomock.NewController(t)
+	defer ctrl.Finish()
+
+	for _, tc := range []struct {
+		description string
+		existingEPS *discovery_v1.EndpointSlice
+		expectedErr error
+	}{
+		{
+			description: "should update backend pool as expected",
+			existingEPS: getTestEndpointSlice("eps1", "default", "svc1", "node2"),
+		},
+		{
+			description: "should report an error if failed to get the endpointslice",
+			expectedErr: errors.New("failed to find EndpointSlice for service default/svc1"),
+		},
+	} {
+		t.Run(tc.description, func(t *testing.T) {
+			cloud := GetTestCloud(ctrl)
+			cloud.localServiceNameToServiceInfoMap.Store("default/svc1", &serviceInfo{lbName: "lb1"})
+			svc := getTestService("svc1", v1.ProtocolTCP, nil, false)
+			var client kubernetes.Interface
+			if tc.existingEPS != nil {
+				client = fake.NewSimpleClientset(&svc, tc.existingEPS)
+			} else {
+				client = fake.NewSimpleClientset(&svc)
+			}
+			cloud.KubeClient = client
+			informerFactory := informers.NewSharedInformerFactory(client, 0)
+			cloud.serviceLister = informerFactory.Core().V1().Services().Lister()
+			cloud.LoadBalancerBackendPoolUpdateIntervalInSeconds = 1
+			cloud.LoadBalancerSku = consts.LoadBalancerSkuStandard
+			cloud.MultipleStandardLoadBalancerConfigurations = []MultipleStandardLoadBalancerConfiguration{
+				{
+					Name: "lb1",
+				},
+			}
+			cloud.localServiceNameToServiceInfoMap.Store("default/svc1", newServiceInfo(consts.IPVersionIPv4String, "lb1"))
+			cloud.nodePrivateIPs = map[string]sets.Set[string]{
+				"node1": sets.New[string]("10.0.0.1", "fd00::1"),
+				"node2": sets.New[string]("10.0.0.2", "fd00::2"),
+			}
+
+			existingBackendPool := getTestBackendAddressPoolWithIPs("lb1", "default-svc1", []string{"10.0.0.1"})
+			existingBackendPoolIPv6 := getTestBackendAddressPoolWithIPs("lb1", "default-svc1-ipv6", []string{"fd00::1"})
+			existingLB := network.LoadBalancer{
+				Name: pointer.String("lb1"),
+				LoadBalancerPropertiesFormat: &network.LoadBalancerPropertiesFormat{
+					BackendAddressPools: &[]network.BackendAddressPool{
+						existingBackendPool,
+						existingBackendPoolIPv6,
+					},
+				},
+			}
+			expectedBackendPool := getTestBackendAddressPoolWithIPs("lb1", "default-svc1", []string{"10.0.0.2"})
+			expectedBackendPoolIPv6 := getTestBackendAddressPoolWithIPs("lb1", "default-svc1-ipv6", []string{"fd00::2"})
+			mockLBClient := mockloadbalancerclient.NewMockInterface(ctrl)
+			if tc.existingEPS != nil {
+				mockLBClient.EXPECT().GetLBBackendPool(gomock.Any(), gomock.Any(), "lb1", "default-svc1", "").Return(existingBackendPool, nil)
+				mockLBClient.EXPECT().GetLBBackendPool(gomock.Any(), gomock.Any(), "lb1", "default-svc1-ipv6", "").Return(existingBackendPoolIPv6, nil)
+				mockLBClient.EXPECT().CreateOrUpdateBackendPools(gomock.Any(), gomock.Any(), "lb1", "default-svc1", expectedBackendPool, "").Return(nil)
+				mockLBClient.EXPECT().CreateOrUpdateBackendPools(gomock.Any(), gomock.Any(), "lb1", "default-svc1-ipv6", expectedBackendPoolIPv6, "").Return(nil)
+			}
+			cloud.LoadBalancerClient = mockLBClient
+
+			u := newLoadBalancerBackendPoolUpdater(cloud, time.Second)
+			ctx, cancel := context.WithCancel(context.Background())
+			defer cancel()
+			cloud.backendPoolUpdater = u
+			go cloud.backendPoolUpdater.run(ctx)
+
+			err := cloud.checkAndApplyLocalServiceBackendPoolUpdates(existingLB, &svc)
+			if tc.expectedErr != nil {
+				assert.Equal(t, tc.expectedErr, err)
+			} else {
+				assert.NoError(t, err)
+			}
+			time.Sleep(2 * time.Second)
+		})
+	}
+}
diff --git a/pkg/provider/azure_standard_test.go b/pkg/provider/azure_standard_test.go
index b6c6dba3c..335365b0f 100644
--- a/pkg/provider/azure_standard_test.go
+++ b/pkg/provider/azure_standard_test.go
@@ -568,7 +568,7 @@ func testGetLoadBalancerSubResourceIDs(
 				subscriptionID,
 				rgName,
 				c.loadBalancerName,
-				clusterName) + "-" + v6Suffix
+				clusterName) + "-" + consts.IPVersionIPv6String
 			subResourceIDs := getLoadBalancerSubResourceIDs(clusterName, c.loadBalancerName)
 			assert.Equal(t, expectedV4, subResourceIDs[false])
 			assert.Equal(t, expectedV6, subResourceIDs[true])
diff --git a/pkg/provider/azure_storageaccount_test.go b/pkg/provider/azure_storageaccount_test.go
index 9ad2cd007..0c7a3adb2 100644
--- a/pkg/provider/azure_storageaccount_test.go
+++ b/pkg/provider/azure_storageaccount_test.go
@@ -343,7 +343,7 @@ func TestGetStorageAccountEdgeCases(t *testing.T) {
 			testCase: "account options CreatePrivateEndpoint is true and no private endpoint exists",
 			testAccountOptions: &AccountOptions{
 				ResourceGroup:         "rg",
-				CreatePrivateEndpoint: pointer.BoolPtr(true),
+				CreatePrivateEndpoint: true,
 			},
 			testResourceGroups: []storage.Account{{Name: &name, Kind: "kind", Location: &location, Sku: sku, AccountProperties: &storage.AccountProperties{}}},
 			expectedResult:     []accountWithLocation{},
@@ -409,7 +409,7 @@ func TestEnsureStorageAccount(t *testing.T) {
 	tests := []struct {
 		name                            string
 		createAccount                   bool
-		createPrivateEndpoint           *bool
+		createPrivateEndpoint           bool
 		SubnetPropertiesFormatNil       bool
 		mockStorageAccountsClient       bool
 		setAccountOptions               bool
@@ -426,7 +426,7 @@ func TestEnsureStorageAccount(t *testing.T) {
 		{
 			name:                            "[Success] EnsureStorageAccount with createPrivateEndpoint and storagetype blob",
 			createAccount:                   true,
-			createPrivateEndpoint:           pointer.BoolPtr(true),
+			createPrivateEndpoint:           true,
 			mockStorageAccountsClient:       true,
 			setAccountOptions:               true,
 			pickRandomMatchingAccount:       true,
@@ -441,7 +441,7 @@ func TestEnsureStorageAccount(t *testing.T) {
 		{
 			name:                            "[Success] EnsureStorageAccount with createPrivateEndpoint",
 			createAccount:                   true,
-			createPrivateEndpoint:           pointer.BoolPtr(true),
+			createPrivateEndpoint:           true,
 			mockStorageAccountsClient:       true,
 			setAccountOptions:               true,
 			requireInfrastructureEncryption: pointer.Bool(true),
@@ -454,7 +454,7 @@ func TestEnsureStorageAccount(t *testing.T) {
 		{
 			name:                      "[Failed] EnsureStorageAccount with createPrivateEndpoint: get storage key failed",
 			createAccount:             true,
-			createPrivateEndpoint:     pointer.BoolPtr(true),
+			createPrivateEndpoint:     true,
 			SubnetPropertiesFormatNil: true,
 			mockStorageAccountsClient: true,
 			setAccountOptions:         true,
@@ -496,7 +496,7 @@ func TestEnsureStorageAccount(t *testing.T) {
 			cloud.StorageAccountClient = mockStorageAccountsClient
 		}
 
-		if pointer.BoolDeref(test.createPrivateEndpoint, false) {
+		if test.createPrivateEndpoint {
 			mockStorageAccountsClient.EXPECT().ListByResourceGroup(gomock.Any(), gomock.Any(), gomock.Any()).Return(testStorageAccounts, nil).AnyTimes()
 			mockStorageAccountsClient.EXPECT().Create(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).Return(nil).AnyTimes()
 			mockStorageAccountsClient.EXPECT().GetProperties(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).Return(testStorageAccounts[0], nil).AnyTimes()
@@ -629,7 +629,7 @@ func TestIsPrivateEndpointAsExpected(t *testing.T) {
 				},
 			},
 			accountOptions: &AccountOptions{
-				CreatePrivateEndpoint: pointer.BoolPtr(true),
+				CreatePrivateEndpoint: true,
 			},
 			expectedResult: true,
 		},
@@ -640,7 +640,7 @@ func TestIsPrivateEndpointAsExpected(t *testing.T) {
 				},
 			},
 			accountOptions: &AccountOptions{
-				CreatePrivateEndpoint: pointer.BoolPtr(false),
+				CreatePrivateEndpoint: false,
 			},
 			expectedResult: true,
 		},
@@ -651,21 +651,10 @@ func TestIsPrivateEndpointAsExpected(t *testing.T) {
 				},
 			},
 			accountOptions: &AccountOptions{
-				CreatePrivateEndpoint: pointer.BoolPtr(false),
+				CreatePrivateEndpoint: false,
 			},
 			expectedResult: false,
 		},
-		{
-			account: storage.Account{
-				AccountProperties: &storage.AccountProperties{
-					PrivateEndpointConnections: &[]storage.PrivateEndpointConnection{{}},
-				},
-			},
-			accountOptions: &AccountOptions{
-				CreatePrivateEndpoint: nil,
-			},
-			expectedResult: true,
-		},
 		{
 			account: storage.Account{
 				AccountProperties: &storage.AccountProperties{
@@ -673,7 +662,7 @@ func TestIsPrivateEndpointAsExpected(t *testing.T) {
 				},
 			},
 			accountOptions: &AccountOptions{
-				CreatePrivateEndpoint: pointer.BoolPtr(true),
+				CreatePrivateEndpoint: true,
 			},
 			expectedResult: false,
 		},
diff --git a/pkg/provider/azure_test.go b/pkg/provider/azure_test.go
index ed6eff762..60d4b6e8f 100644
--- a/pkg/provider/azure_test.go
+++ b/pkg/provider/azure_test.go
@@ -241,7 +241,7 @@ func setMockPublicIP(az *Cloud, mockPIPsClient *mockpublicipclient.MockInterface
 	ipAddr1 := "1.2.3.4"
 	ipAddra := "1.2.3.5"
 	if isIPv6 {
-		suffix = "-" + v6Suffix
+		suffix = "-" + consts.IPVersionIPv6String
 		ipVer = network.IPv6
 		ipAddr1 = "fd00::eef0"
 		ipAddra = "fd00::eef1"
@@ -1994,7 +1994,7 @@ func validateLoadBalancer(t *testing.T, loadBalancer *network.LoadBalancer, serv
 			expectedFrontendIPs = append(expectedFrontendIPs, expectedFrontendIP)
 			if svcIPFamilyCount == 2 {
 				expectedFrontendIP := ExpectedFrontendIPInfo{
-					Name:   az.getDefaultFrontendIPConfigName(&services[i]) + "-" + v6Suffix,
+					Name:   az.getDefaultFrontendIPConfigName(&services[i]) + "-" + consts.IPVersionIPv6String,
 					Subnet: pointer.String(expectedSubnetName),
 				}
 				expectedFrontendIPs = append(expectedFrontendIPs, expectedFrontendIP)
diff --git a/pkg/provider/azure_vmss_test.go b/pkg/provider/azure_vmss_test.go
index 4f9b34fa5..86fc98463 100644
--- a/pkg/provider/azure_vmss_test.go
+++ b/pkg/provider/azure_vmss_test.go
@@ -52,9 +52,9 @@ const (
 	testVMSSName           = "vmss"
 	testVMPowerState       = "PowerState/Running"
 	testLBBackendpoolID0   = "/subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb/backendAddressPools/backendpool-0"
-	testLBBackendpoolID0v6 = "/subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb/backendAddressPools/backendpool-0" + "-" + v6Suffix
+	testLBBackendpoolID0v6 = "/subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb/backendAddressPools/backendpool-0" + "-" + consts.IPVersionIPv6String
 	testLBBackendpoolID1   = "/subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb/backendAddressPools/backendpool-1"
-	testLBBackendpoolID1v6 = "/subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb/backendAddressPools/backendpool-1" + "-" + v6Suffix
+	testLBBackendpoolID1v6 = "/subscriptions/sub/resourceGroups/rg/providers/Microsoft.Network/loadBalancers/lb/backendAddressPools/backendpool-1" + "-" + consts.IPVersionIPv6String
 	testLBBackendpoolID2   = "/subscriptions/sub/resourceGroups/rg1/providers/Microsoft.Network/loadBalancers/lb/backendAddressPools/backendpool-2"
 )
 
@@ -62,7 +62,7 @@ func buildTestVMSSWithLB(name, namePrefix string, lbBackendpoolIDs []string, ipv
 	lbBackendpoolsV4, lbBackendpoolsV6 := make([]compute.SubResource, 0), make([]compute.SubResource, 0)
 	for _, id := range lbBackendpoolIDs {
 		lbBackendpoolsV4 = append(lbBackendpoolsV4, compute.SubResource{ID: pointer.String(id)})
-		lbBackendpoolsV6 = append(lbBackendpoolsV6, compute.SubResource{ID: pointer.String(id + "-" + v6Suffix)})
+		lbBackendpoolsV6 = append(lbBackendpoolsV6, compute.SubResource{ID: pointer.String(id + "-" + consts.IPVersionIPv6String)})
 	}
 	ipConfig := []compute.VirtualMachineScaleSetIPConfiguration{
 		{
@@ -2421,7 +2421,7 @@ func TestEnsureVMSSInPool(t *testing.T) {
 				},
 			},
 			isBasicLB:       false,
-			backendPoolID:   testLBBackendpoolID1 + "-" + v6Suffix,
+			backendPoolID:   testLBBackendpoolID1 + "-" + consts.IPVersionIPv6String,
 			clusterIP:       "fd00::e68b",
 			expectedPutVMSS: false,
 			expectedErr:     fmt.Errorf("failed to find a primary IP configuration (IPv6=true) for the VMSS VM or VMSS \"vmss\""),
@@ -2436,7 +2436,7 @@ func TestEnsureVMSSInPool(t *testing.T) {
 				},
 			},
 			isBasicLB:       false,
-			backendPoolID:   testLBBackendpoolID1 + "-" + v6Suffix,
+			backendPoolID:   testLBBackendpoolID1 + "-" + consts.IPVersionIPv6String,
 			setIPv6Config:   true,
 			clusterIP:       "fd00::e68b",
 			expectedPutVMSS: true,
diff --git a/pkg/util/controller/testutil/test_utils.go b/pkg/util/controller/testutil/test_utils.go
index 11acd15ee..e7234db74 100644
--- a/pkg/util/controller/testutil/test_utils.go
+++ b/pkg/util/controller/testutil/test_utils.go
@@ -53,9 +53,8 @@ type FakeNodeHandler struct {
 	*fake.Clientset
 
 	// Input: Hooks determine if request is valid or not
-	CreateHook      func(*FakeNodeHandler, *v1.Node) bool
-	Existing        []*v1.Node
-	PatchErrorCount int
+	CreateHook func(*FakeNodeHandler, *v1.Node) bool
+	Existing   []*v1.Node
 
 	// Output
 	CreatedNodes        []*v1.Node
@@ -275,11 +274,6 @@ func (m *FakeNodeHandler) Patch(_ context.Context, name string, pt types.PatchTy
 		}
 		m.lock.Unlock()
 	}()
-
-	if m.RequestCount < m.PatchErrorCount {
-		return nil, fmt.Errorf("patch failed count is %d, which is smaller than %d", m.RequestCount, m.PatchErrorCount)
-	}
-
 	var nodeCopy v1.Node
 	for i := range m.Existing {
 		if m.Existing[i].Name == name {
diff --git a/tests/e2e/network/multiple_standard_lb.go b/tests/e2e/network/multiple_standard_lb.go
index 1dc0232c5..4f8268a6c 100644
--- a/tests/e2e/network/multiple_standard_lb.go
+++ b/tests/e2e/network/multiple_standard_lb.go
@@ -189,12 +189,13 @@ var _ = Describe("Ensure LoadBalancer", Label(utils.TestSuiteLabelMultiSLB), fun
 		Expect(err).NotTo(HaveOccurred())
 		Expect(len(ips)).NotTo(BeZero())
 
-		nodes, err := utils.GetAgentNodes(cs)
-		By(fmt.Sprintf("Checking the node count in the local service backend pool to equal %d)", len(nodes)))
+		nodeNames, err := getDeploymentPodsNodeNames(cs, ns.Name, testDeploymentName)
+		Expect(err).NotTo(HaveOccurred())
+		By(fmt.Sprintf("Checking the node count in the local service backend pool to equal %d", len(nodeNames)))
 		clusterName := os.Getenv("CLUSTER_NAME")
 		Expect(err).NotTo(HaveOccurred())
 		expectedBPName := fmt.Sprintf("%s-%s", svc.Namespace, svc.Name)
-		err = checkNodeCountInBackendPoolByServiceIPs(tc, clusterName, expectedBPName, ips, len(nodes))
+		err = checkNodeCountInBackendPoolByServiceIPs(tc, clusterName, expectedBPName, ips, len(nodeNames))
 		Expect(err).NotTo(HaveOccurred())
 
 		By("Scaling the deployment to 3 replicas and then to 1")
@@ -221,10 +222,12 @@ var _ = Describe("Ensure LoadBalancer", Label(utils.TestSuiteLabelMultiSLB), fun
 		deployment.Spec.Replicas = pointer.Int32(5)
 		_, err = cs.AppsV1().Deployments(ns.Name).Update(context.Background(), deployment, metav1.UpdateOptions{})
 		Expect(err).NotTo(HaveOccurred())
+		nodeNames, err = getDeploymentPodsNodeNames(cs, ns.Name, testDeploymentName)
+		Expect(err).NotTo(HaveOccurred())
 		err = wait.PollUntilContextTimeout(context.Background(), 5*time.Second, 5*time.Minute, false, func(ctx context.Context) (bool, error) {
-			if err := checkNodeCountInBackendPoolByServiceIPs(tc, clusterName, expectedBPName, ips, len(nodes)); err != nil {
+			if err := checkNodeCountInBackendPoolByServiceIPs(tc, clusterName, expectedBPName, ips, len(nodeNames)); err != nil {
 				if strings.Contains(err.Error(), "expected node count") {
-					utils.Logf("Waiting for the node count in the backend pool to equal %d", len(nodes))
+					utils.Logf("Waiting for the node count in the backend pool to equal %d", len(nodeNames))
 					return false, nil
 				}
 				return false, err
@@ -235,6 +238,37 @@ var _ = Describe("Ensure LoadBalancer", Label(utils.TestSuiteLabelMultiSLB), fun
 	})
 })
 
+// getDeploymentPodsNodeNames returns the node names of the pods in the deployment created in BeforeEach.
+func getDeploymentPodsNodeNames(kubeClient clientset.Interface, namespace, deploymentName string) (map[string]bool, error) {
+	var (
+		podList *v1.PodList
+		res     = make(map[string]bool)
+		err     error
+	)
+	err = wait.PollUntilContextTimeout(context.Background(), 5*time.Second, 5*time.Minute, false, func(ctx context.Context) (bool, error) {
+		podList, err = kubeClient.CoreV1().Pods(namespace).List(context.Background(), metav1.ListOptions{})
+		if err != nil {
+			return false, err
+		}
+		for _, pod := range podList.Items {
+			if strings.HasPrefix(pod.Name, deploymentName) {
+				if pod.Spec.NodeName == "" {
+					utils.Logf("Waiting for pod %s to be running", pod.Name)
+					return false, nil
+				}
+				utils.Logf("Pod %s is running on node %s", pod.Name, pod.Spec.NodeName)
+				res[pod.Spec.NodeName] = true
+			}
+		}
+		return true, nil
+	})
+	if err != nil {
+		return nil, err
+	}
+
+	return res, nil
+}
+
 func checkNodeCountInBackendPoolByServiceIPs(tc *utils.AzureTestClient, expectedLBName, bpName string, svcIPs []string, expectedCount int) error {
 	for _, svcIP := range svcIPs {
 		lb := getAzureLoadBalancerFromPIP(tc, svcIP, tc.GetResourceGroup(), tc.GetResourceGroup())