Repository: kubernetes-sigs/cloud-provider-azure
Pull request #4201 (2023-06-30 08:49:32)

feat: support node selection for multiple standard load balancers
<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?

<!--
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature
/kind design

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

/kind feature

#### What this PR does / why we need it:

Support node selection for multiple standard load balancers.
When nodes are created, each node will be evaluated to see what load balancer it should be placed into.  
Valid placement targets will be determined as follows (rules match from top to bottom, first match wins): 
1. If this node is in an agent pool that is selected as a primary agent pool for a load balancer, that load balancer will be the only potential placement target. 
2. If the nodeSelectors on any load balancer configurations match this node, then all load balancer configurations that match it will be potential placement targets. 
3. If no nodeSelectors on any load balancer configurations match this node, then all load balancers that do not have any nodeSelectors will be potential placement targets. 

After the list of potential placement targets has been calculated, the node should be placed into the kubernetes backend pool of the load balancer with the fewest number of nodes already assigned. 

#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #
Related: https://github.com/kubernetes-sigs/cloud-provider-azure/issues/4013

#### Special notes for your reviewer:

#### Does this PR introduce a user-facing change?
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

For more information on release notes see: https://git.k8s.io/community/contributors/guide/release-notes.md
-->
```release-note
feat: support node selection for multiple standard load balancers
```

#### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

<!--
This section can be blank if this pull request does not require a release note.

When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.

See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files

Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
-->
```docs

```


k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/cloud-provider-azure/pull/4201#" title="Author self-approved">nilo19</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fcloud-provider-azure).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/cloud-provider-azure/blob/master/OWNERS)~~ [nilo19]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 6509c7ebcc4c83ba8ab70d6d2f481d997ab3d10c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1674701700545384448) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4201). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @nilo19: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 6509c7ebcc4c83ba8ab70d6d2f481d997ab3d10c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1674701700545384448) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-capz | 6509c7ebcc4c83ba8ab70d6d2f481d997ab3d10c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-capz/1674701700599910400) | true | `/test pull-cloud-provider-azure-e2e-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4201). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @nilo19: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 6509c7ebcc4c83ba8ab70d6d2f481d997ab3d10c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1674701700545384448) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-capz | 6509c7ebcc4c83ba8ab70d6d2f481d997ab3d10c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-capz/1674701700599910400) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 6509c7ebcc4c83ba8ab70d6d2f481d997ab3d10c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1674701700478275584) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4201). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
nilo19: /retest
k8s-ci-robot: @nilo19: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 6509c7ebcc4c83ba8ab70d6d2f481d997ab3d10c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1674701700545384448) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 6509c7ebcc4c83ba8ab70d6d2f481d997ab3d10c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1674701700478275584) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-capz | 6509c7ebcc4c83ba8ab70d6d2f481d997ab3d10c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-capz/1675518142215884800) | true | `/test pull-cloud-provider-azure-e2e-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4201). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
lzhecheng: /retest
k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 88a7901c3ca3ffcb207f0e780391a7484bb45859 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-unit/1676408284262699008) | true | `/test pull-cloud-provider-azure-unit`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4201). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
coveralls: 
[![Coverage Status](https://coveralls.io/builds/61137366/badge)](https://coveralls.io/builds/61137366)

coverage: 79.853% (+0.05%) from 79.803% when pulling **34b9faa3fd4641f36bf0ec46d7da5fa4b43d5b11 on nilo19:feat/multi-slb/backend** into **545a717fd9b24b6ce3642d249b1e0233de0fef30 on kubernetes-sigs:master**.

k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 205ac9fba264ab0093bb0e581d163a64f718534a | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1676598081505726464) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4201). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
nilo19: /retest
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/cloud-provider-azure/pull/4201#pullrequestreview-1524368005" title="LGTM">feiskyer</a>*, *<a href="https://github.com/kubernetes-sigs/cloud-provider-azure/pull/4201#" title="Author self-approved">nilo19</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fcloud-provider-azure).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/cloud-provider-azure/blob/master/OWNERS)~~ [feiskyer,nilo19]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 205ac9fba264ab0093bb0e581d163a64f718534a | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-ccm-capz/1678768403596185600) | unknown | `/test pull-cloud-provider-azure-e2e-ccm-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4201). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: PR needs rebase.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
k8s-ci-robot: New changes are detected. LGTM label has been removed.
nilo19: @feiskyer squashed commits and rebased.
k8s-ci-robot: @nilo19: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-capz | b0b436b63b5400a56e8d1fdccfcce9ebece868f4 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4201/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1679128399815118848) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4201). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Anilo19). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->

Patch:
diff --git a/Dockerfile b/Dockerfile
index 5fdcbca38..d0d61a55d 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -14,7 +14,7 @@
 
 # syntax=docker/dockerfile:1
 
-FROM --platform=linux/amd64 golang:1.20-bullseye@sha256:851af0a8ca4eba552c84db5b2edac7f3be15deb5892217961a1d4b175585a603 AS builder
+FROM --platform=linux/amd64 golang:1.20-bullseye@sha256:d9f75197f484a4d7e008a0a8f7d1e68344ea1c1ba030273d5ab1567e129eed55 AS builder
 
 ARG ENABLE_GIT_COMMAND=true
 ARG ARCH=amd64
diff --git a/cloud-node-manager.Dockerfile b/cloud-node-manager.Dockerfile
index cd32e9b59..f9eefd4ad 100644
--- a/cloud-node-manager.Dockerfile
+++ b/cloud-node-manager.Dockerfile
@@ -14,7 +14,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM --platform=linux/amd64 golang:1.20-bullseye@sha256:851af0a8ca4eba552c84db5b2edac7f3be15deb5892217961a1d4b175585a603 AS builder
+FROM --platform=linux/amd64 golang:1.20-bullseye@sha256:d9f75197f484a4d7e008a0a8f7d1e68344ea1c1ba030273d5ab1567e129eed55 AS builder
 
 ARG ENABLE_GIT_COMMAND=true
 ARG ARCH=amd64
diff --git a/pkg/provider/azure.go b/pkg/provider/azure.go
index 258e9ebec..455874f1c 100644
--- a/pkg/provider/azure.go
+++ b/pkg/provider/azure.go
@@ -309,7 +309,12 @@ type MultipleStandardLoadBalancerConfigurationSpec struct {
 
 // MultipleStandardLoadBalancerConfigurationStatus stores the properties regarding multiple standard load balancers.
 type MultipleStandardLoadBalancerConfigurationStatus struct {
+	// ActiveServices stores the services that are supposed to use the load balancer.
 	ActiveServices sets.Set[string] `json:"activeServices" yaml:"activeServices"`
+
+	// ActiveNodes stores the nodes that are supposed to be in the load balancer.
+	// It will be used in EnsureHostsInPool to make sure the given ones are in the backend pool.
+	ActiveNodes sets.Set[string] `json:"activeNodes" yaml:"activeNodes"`
 }
 
 type InitSecretConfig struct {
@@ -383,8 +388,9 @@ type Cloud struct {
 	// unmanagedNodes holds a list of nodes not managed by Azure cloud provider.
 	unmanagedNodes sets.Set[string]
 	// excludeLoadBalancerNodes holds a list of nodes that should be excluded from LoadBalancer.
-	excludeLoadBalancerNodes sets.Set[string]
-	nodePrivateIPs           map[string]sets.Set[string]
+	excludeLoadBalancerNodes   sets.Set[string]
+	nodePrivateIPs             map[string]sets.Set[string]
+	nodePrivateIPToNodeNameMap map[string]string
 	// nodeInformerSynced is for determining if the informer has synced.
 	nodeInformerSynced cache.InformerSynced
 
@@ -421,7 +427,14 @@ type Cloud struct {
 	*ManagedDiskController
 	*controllerCommon
 
+	// multipleStandardLoadBalancerConfigurationsSynced make sure the `reconcileMultipleStandardLoadBalancerConfigurations`
+	// runs only once every time the cloud provide restarts.
 	multipleStandardLoadBalancerConfigurationsSynced bool
+	// nodesWithCorrectLoadBalancerByPrimaryVMSet marks nodes that are matched with load balancers by primary vmSet.
+	nodesWithCorrectLoadBalancerByPrimaryVMSet sync.Map
+
+	multipleStandardLoadBalancersActiveServicesLock sync.Mutex
+	multipleStandardLoadBalancersActiveNodesLock    sync.Mutex
 }
 
 // NewCloud returns a Cloud with initialized clients
@@ -487,13 +500,14 @@ func (az *Cloud) configSecretMetadata(secretName, secretNamespace, cloudConfigKe
 
 func NewCloudFromSecret(ctx context.Context, clientBuilder cloudprovider.ControllerClientBuilder, secretName, secretNamespace, cloudConfigKey string) (cloudprovider.Interface, error) {
 	az := &Cloud{
-		nodeNames:                sets.New[string](),
-		nodeZones:                map[string]sets.Set[string]{},
-		nodeResourceGroups:       map[string]string{},
-		unmanagedNodes:           sets.New[string](),
-		routeCIDRs:               map[string]string{},
-		excludeLoadBalancerNodes: sets.New[string](),
-		nodePrivateIPs:           map[string]sets.Set[string]{},
+		nodeNames:                  sets.New[string](),
+		nodeZones:                  map[string]sets.Set[string]{},
+		nodeResourceGroups:         map[string]string{},
+		unmanagedNodes:             sets.New[string](),
+		routeCIDRs:                 map[string]string{},
+		excludeLoadBalancerNodes:   sets.New[string](),
+		nodePrivateIPs:             map[string]sets.Set[string]{},
+		nodePrivateIPToNodeNameMap: map[string]string{},
 	}
 
 	az.configSecretMetadata(secretName, secretNamespace, cloudConfigKey)
@@ -519,13 +533,14 @@ func NewCloudWithoutFeatureGates(ctx context.Context, configReader io.Reader, ca
 	}
 
 	az := &Cloud{
-		nodeNames:                sets.New[string](),
-		nodeZones:                map[string]sets.Set[string]{},
-		nodeResourceGroups:       map[string]string{},
-		unmanagedNodes:           sets.New[string](),
-		routeCIDRs:               map[string]string{},
-		excludeLoadBalancerNodes: sets.New[string](),
-		nodePrivateIPs:           map[string]sets.Set[string]{},
+		nodeNames:                  sets.New[string](),
+		nodeZones:                  map[string]sets.Set[string]{},
+		nodeResourceGroups:         map[string]string{},
+		unmanagedNodes:             sets.New[string](),
+		routeCIDRs:                 map[string]string{},
+		excludeLoadBalancerNodes:   sets.New[string](),
+		nodePrivateIPs:             map[string]sets.Set[string]{},
+		nodePrivateIPToNodeNameMap: map[string]string{},
 	}
 
 	err = az.InitializeCloudFromConfig(ctx, config, false, callFromCCM)
@@ -729,6 +744,24 @@ func (az *Cloud) checkEnableMultipleStandardLoadBalancers() error {
 	if az.isLBBackendPoolTypeNodeIPConfig() {
 		return fmt.Errorf("multiple standard load balancers cannot be used with backend pool type %s", consts.LoadBalancerBackendPoolConfigurationTypeNodeIPConfiguration)
 	}
+
+	names := sets.New[string]()
+	primaryVMSets := sets.New[string]()
+	for _, multiSLBConfig := range az.MultipleStandardLoadBalancerConfigurations {
+		if names.Has(multiSLBConfig.Name) {
+			return fmt.Errorf("duplicated multiple standard load balancer configuration name %s", multiSLBConfig.Name)
+		}
+		names.Insert(multiSLBConfig.Name)
+
+		if multiSLBConfig.PrimaryVMSet == "" {
+			return fmt.Errorf("multiple standard load balancer configuration %s must have primary VMSet", multiSLBConfig.Name)
+		}
+		if primaryVMSets.Has(multiSLBConfig.PrimaryVMSet) {
+			return fmt.Errorf("duplicated primary VMSet %s in multiple standard load balancer configurations %s", multiSLBConfig.PrimaryVMSet, multiSLBConfig.Name)
+		}
+		primaryVMSets.Insert(multiSLBConfig.PrimaryVMSet)
+	}
+
 	return nil
 }
 
@@ -1188,12 +1221,14 @@ func (az *Cloud) updateNodeCaches(prevNode, newNode *v1.Node) {
 		// if the node is being deleted from the cluster, exclude it from load balancers
 		if newNode == nil {
 			az.excludeLoadBalancerNodes.Insert(prevNode.ObjectMeta.Name)
+			az.nodesWithCorrectLoadBalancerByPrimaryVMSet.Delete(strings.ToLower(prevNode.ObjectMeta.Name))
 		}
 
 		// Remove from nodePrivateIPs cache.
 		for _, address := range getNodePrivateIPAddresses(prevNode) {
 			klog.V(4).Infof("removing IP address %s of the node %s", address, prevNode.Name)
 			az.nodePrivateIPs[prevNode.Name].Delete(address)
+			delete(az.nodePrivateIPToNodeNameMap, address)
 		}
 	}
 
@@ -1255,9 +1290,13 @@ func (az *Cloud) updateNodeCaches(prevNode, newNode *v1.Node) {
 			if az.nodePrivateIPs[newNode.Name] == nil {
 				az.nodePrivateIPs[newNode.Name] = sets.New[string]()
 			}
+			if az.nodePrivateIPToNodeNameMap == nil {
+				az.nodePrivateIPToNodeNameMap = make(map[string]string)
+			}
 
 			klog.V(6).Infof("adding IP address %s of the node %s", address, newNode.Name)
 			az.nodePrivateIPs[newNode.Name].Insert(address)
+			az.nodePrivateIPToNodeNameMap[address] = newNode.Name
 		}
 	}
 }
@@ -1392,3 +1431,16 @@ func isNodeReady(node *v1.Node) bool {
 	}
 	return false
 }
+
+func (az *Cloud) getActiveNodesByLoadBalancerName(lbName string) sets.Set[string] {
+	az.multipleStandardLoadBalancersActiveNodesLock.Lock()
+	defer az.multipleStandardLoadBalancersActiveNodesLock.Unlock()
+
+	for _, multiSLBConfig := range az.MultipleStandardLoadBalancerConfigurations {
+		if strings.EqualFold(strings.TrimSuffix(lbName, consts.InternalLoadBalancerNameSuffix), multiSLBConfig.Name) {
+			return multiSLBConfig.ActiveNodes
+		}
+	}
+
+	return sets.New[string]()
+}
diff --git a/pkg/provider/azure_loadbalancer.go b/pkg/provider/azure_loadbalancer.go
index c53f6348b..bca5c3d2e 100644
--- a/pkg/provider/azure_loadbalancer.go
+++ b/pkg/provider/azure_loadbalancer.go
@@ -86,13 +86,13 @@ func (az *Cloud) GetLoadBalancer(ctx context.Context, clusterName string, servic
 		return nil, az.existsPip(clusterName, service), err
 	}
 
-	_, status, _, existsLb, err := az.getServiceLoadBalancer(service, clusterName, nil, false, existingLBs)
+	_, status, _, existsLb, _, err := az.getServiceLoadBalancer(service, clusterName, nil, false, existingLBs)
 	if err != nil || existsLb {
 		return status, existsLb || az.existsPip(clusterName, service), err
 	}
 
 	flippedService := flipServiceInternalAnnotation(service)
-	_, status, _, existsLb, err = az.getServiceLoadBalancer(flippedService, clusterName, nil, false, existingLBs)
+	_, status, _, existsLb, _, err = az.getServiceLoadBalancer(flippedService, clusterName, nil, false, existingLBs)
 	if err != nil || existsLb {
 		return status, existsLb || az.existsPip(clusterName, service), err
 	}
@@ -277,7 +277,7 @@ func (az *Cloud) EnsureLoadBalancerDeleted(ctx context.Context, clusterName stri
 		klog.V(5).InfoS("EnsureLoadBalancerDeleted Finish", "service", serviceName, "cluster", clusterName, "service_spec", service, "error", err)
 	}()
 
-	_, _, lbIPsPrimaryPIPs, _, err := az.getServiceLoadBalancer(service, clusterName, nil, false, []network.LoadBalancer{})
+	_, _, lbIPsPrimaryPIPs, _, _, err := az.getServiceLoadBalancer(service, clusterName, nil, false, []network.LoadBalancer{})
 	if err != nil && !retry.HasStatusForbiddenOrIgnoredError(err) {
 		return err
 	}
@@ -332,8 +332,11 @@ func (az *Cloud) shouldChangeLoadBalancer(service *v1.Service, currLBName, clust
 	}
 
 	if az.useMultipleStandardLoadBalancers() {
-		klog.V(2).Infof("shouldChangeLoadBalancer(%s, %s, %s): change the LB to another one %s", service.Name, currLBName, clusterName, expectedLBName)
-		return currLBName != expectedLBName
+		if currLBName != expectedLBName {
+			klog.V(2).Infof("shouldChangeLoadBalancer(%s, %s, %s): change the LB to another one %s", service.Name, currLBName, clusterName, expectedLBName)
+			return true
+		}
+		return false
 	}
 
 	// basic LB
@@ -364,9 +367,12 @@ func (az *Cloud) shouldChangeLoadBalancer(service *v1.Service, currLBName, clust
 	return true
 }
 
-func (az *Cloud) removeFrontendIPConfigurationFromLoadBalancer(lb *network.LoadBalancer, existingLBs []network.LoadBalancer, fips []*network.FrontendIPConfiguration, clusterName string, service *v1.Service) error {
+// removeFrontendIPConfigurationFromLoadBalancer removes the given ip configs from the load balancer
+// and delete the load balancer if there is no ip config on it. It returns the name of the deleted load balancer
+// and it will be used in reconcileLoadBalancer to remove the load balancer from the list.
+func (az *Cloud) removeFrontendIPConfigurationFromLoadBalancer(lb *network.LoadBalancer, existingLBs []network.LoadBalancer, fips []*network.FrontendIPConfiguration, clusterName string, service *v1.Service) (string, error) {
 	if lb == nil || lb.LoadBalancerPropertiesFormat == nil || lb.FrontendIPConfigurations == nil {
-		return nil
+		return "", nil
 	}
 	fipConfigs := *lb.FrontendIPConfigurations
 	for i, fipConfig := range fipConfigs {
@@ -408,10 +414,11 @@ func (az *Cloud) removeFrontendIPConfigurationFromLoadBalancer(lb *network.LoadB
 		// clean up any private link service associated with the frontEndIPConfig
 		if err := az.reconcilePrivateLinkService(clusterName, service, fip, false /* wantPLS */); err != nil {
 			klog.Errorf("removeFrontendIPConfigurationFromLoadBalancer(%s, %s, %s, %s): failed to clean up PLS: %v", pointer.StringDeref(lb.Name, ""), pointer.StringDeref(fip.Name, ""), clusterName, service.Name, err)
-			return err
+			return "", err
 		}
 	}
 
+	var deletedLBName string
 	fipNames := []string{}
 	for _, fip := range fips {
 		fipNames = append(fipNames, pointer.StringDeref(fip.Name, ""))
@@ -422,18 +429,19 @@ func (az *Cloud) removeFrontendIPConfigurationFromLoadBalancer(lb *network.LoadB
 		err := az.cleanOrphanedLoadBalancer(lb, existingLBs, service, clusterName)
 		if err != nil {
 			klog.Errorf("%s: failed to cleanupOrphanedLoadBalancer: %v", logPrefix, err)
-			return err
+			return "", err
 		}
+		deletedLBName = pointer.StringDeref(lb.Name, "")
 	} else {
 		klog.V(2).Infof("%s: updating the load balancer", logPrefix)
 		err := az.CreateOrUpdateLB(service, *lb)
 		if err != nil {
 			klog.Errorf("%s: failed to CreateOrUpdateLB: %v", logPrefix, err)
-			return err
+			return "", err
 		}
 		_ = az.lbCache.Delete(pointer.StringDeref(lb.Name, ""))
 	}
-	return nil
+	return deletedLBName, nil
 }
 
 func (az *Cloud) cleanOrphanedLoadBalancer(lb *network.LoadBalancer, existingLBs []network.LoadBalancer, service *v1.Service, clusterName string) error {
@@ -535,6 +543,22 @@ func (az *Cloud) safeDeleteLoadBalancer(lb network.LoadBalancer, clusterName, vm
 	}
 	_ = az.lbCache.Delete(pointer.StringDeref(lb.Name, ""))
 
+	// Remove corresponding nodes in ActiveNodes and nodesWithCorrectLoadBalancerByPrimaryVMSet.
+	for i := range az.MultipleStandardLoadBalancerConfigurations {
+		if strings.EqualFold(
+			strings.TrimSuffix(pointer.StringDeref(lb.Name, ""), consts.InternalLoadBalancerNameSuffix),
+			az.MultipleStandardLoadBalancerConfigurations[i].Name,
+		) {
+			if az.MultipleStandardLoadBalancerConfigurations[i].ActiveNodes != nil {
+				for nodeName := range az.MultipleStandardLoadBalancerConfigurations[i].ActiveNodes {
+					az.nodesWithCorrectLoadBalancerByPrimaryVMSet.Delete(strings.ToLower(nodeName))
+				}
+			}
+			az.MultipleStandardLoadBalancerConfigurations[i].ActiveNodes = sets.New[string]()
+			break
+		}
+	}
+
 	return nil
 }
 
@@ -543,20 +567,20 @@ func (az *Cloud) safeDeleteLoadBalancer(lb network.LoadBalancer, clusterName, vm
 // In case the selected load balancer does not exist it returns network.LoadBalancer struct
 // with added metadata (such as name, location) and existsLB set to FALSE.
 // By default - cluster default LB is returned.
-func (az *Cloud) getServiceLoadBalancer(service *v1.Service, clusterName string, nodes []*v1.Node, wantLb bool, existingLBs []network.LoadBalancer) (lb *network.LoadBalancer, status *v1.LoadBalancerStatus, lbIPsPrimaryPIPs []string, exists bool, err error) {
+func (az *Cloud) getServiceLoadBalancer(service *v1.Service, clusterName string, nodes []*v1.Node, wantLb bool, existingLBs []network.LoadBalancer) (lb *network.LoadBalancer, status *v1.LoadBalancerStatus, lbIPsPrimaryPIPs []string, exists bool, deletedLBName string, err error) {
 	isInternal := requiresInternalLoadBalancer(service)
 	var defaultLB *network.LoadBalancer
 	primaryVMSetName := az.VMSet.GetPrimaryVMSetName()
 	defaultLBName, err := az.getAzureLoadBalancerName(service, &existingLBs, clusterName, primaryVMSetName, isInternal)
 	if err != nil {
-		return nil, nil, nil, false, err
+		return nil, nil, nil, false, "", err
 	}
 
 	// reuse the lb list from reconcileSharedLoadBalancer to reduce the api call
 	if len(existingLBs) == 0 {
 		existingLBs, err = az.ListLB(service)
 		if err != nil {
-			return nil, nil, nil, false, err
+			return nil, nil, nil, false, "", err
 		}
 	}
 
@@ -574,7 +598,7 @@ func (az *Cloud) getServiceLoadBalancer(service *v1.Service, clusterName string,
 		var fipConfigs []*network.FrontendIPConfiguration
 		status, lbIPsPrimaryPIPs, fipConfigs, err = az.getServiceLoadBalancerStatus(service, &existingLB)
 		if err != nil {
-			return nil, nil, nil, false, err
+			return nil, nil, nil, false, "", err
 		}
 		if status == nil {
 			// service is not on this load balancer
@@ -584,14 +608,16 @@ func (az *Cloud) getServiceLoadBalancer(service *v1.Service, clusterName string,
 
 		// select another load balancer instead of returning
 		// the current one if the change is needed
+		var err error
 		if wantLb && az.shouldChangeLoadBalancer(service, pointer.StringDeref(existingLB.Name, ""), clusterName, defaultLBName) {
 			fipConfigNames := []string{}
 			for _, fipConfig := range fipConfigs {
 				fipConfigNames = append(fipConfigNames, pointer.StringDeref(fipConfig.Name, ""))
 			}
-			if err := az.removeFrontendIPConfigurationFromLoadBalancer(&existingLB, existingLBs, fipConfigs, clusterName, service); err != nil {
+			deletedLBName, err = az.removeFrontendIPConfigurationFromLoadBalancer(&existingLB, existingLBs, fipConfigs, clusterName, service)
+			if err != nil {
 				klog.Errorf("getServiceLoadBalancer(%s, %s, %v): failed to remove frontend IP configurations %q from load balancer: %v", service.Name, clusterName, wantLb, fipConfigNames, err)
-				return nil, nil, nil, false, err
+				return nil, nil, nil, false, "", err
 			}
 			az.reconcileMultipleStandardLoadBalancerConfigurationStatus(
 				false,
@@ -601,7 +627,7 @@ func (az *Cloud) getServiceLoadBalancer(service *v1.Service, clusterName string,
 			break
 		}
 
-		return &existingLB, status, lbIPsPrimaryPIPs, true, nil
+		return &existingLB, status, lbIPsPrimaryPIPs, true, "", nil
 	}
 
 	// Service does not have a load balancer, select one.
@@ -611,10 +637,10 @@ func (az *Cloud) getServiceLoadBalancer(service *v1.Service, clusterName string,
 		// select new load balancer for service
 		selectedLB, exists, err := az.selectLoadBalancer(clusterName, service, &existingLBs, nodes)
 		if err != nil {
-			return nil, nil, nil, false, err
+			return nil, nil, nil, false, "", err
 		}
 
-		return selectedLB, status, lbIPsPrimaryPIPs, exists, err
+		return selectedLB, status, lbIPsPrimaryPIPs, exists, "", err
 	}
 
 	// create a default LB with meta data if not present
@@ -637,7 +663,7 @@ func (az *Cloud) getServiceLoadBalancer(service *v1.Service, clusterName string,
 		}
 	}
 
-	return defaultLB, nil, nil, false, nil
+	return defaultLB, nil, nil, false, deletedLBName, nil
 }
 
 // selectLoadBalancer selects load balancer for the service in the cluster.
@@ -1415,7 +1441,13 @@ func (az *Cloud) findFrontendIPConfigsOfService(
 // load balancer typed services and add service names to the ActiveServices queue
 // of the corresponding load balancer configuration. It also checks if there is a configuration
 // named <clustername>. If not, an error will be reported.
-func (az *Cloud) reconcileMultipleStandardLoadBalancerConfigurations(clusterName string, existingLBs *[]network.LoadBalancer) (err error) {
+func (az *Cloud) reconcileMultipleStandardLoadBalancerConfigurations(
+	lbs *[]network.LoadBalancer,
+	service *v1.Service,
+	clusterName string,
+	existingLBs *[]network.LoadBalancer,
+	nodes []*v1.Node,
+) (err error) {
 	if !az.useMultipleStandardLoadBalancers() {
 		return nil
 	}
@@ -1474,11 +1506,13 @@ func (az *Cloud) reconcileMultipleStandardLoadBalancerConfigurations(clusterName
 					)
 					for i := range az.MultipleStandardLoadBalancerConfigurations {
 						if strings.EqualFold(strings.TrimSuffix(lbName, consts.InternalLoadBalancerNameSuffix), az.MultipleStandardLoadBalancerConfigurations[i].Name) {
+							az.multipleStandardLoadBalancersActiveServicesLock.Lock()
 							if az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices == nil {
 								az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices = sets.New[string]()
 							}
+							az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices.Insert(strings.ToLower(svcName))
+							az.multipleStandardLoadBalancersActiveServicesLock.Unlock()
 							klog.V(2).Infof("reconcileMultipleStandardLoadBalancerConfigurations: service(%s) is active on lb(%s)", svcName, lbName)
-							az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices.Insert(svcName)
 						}
 					}
 				}
@@ -1486,7 +1520,7 @@ func (az *Cloud) reconcileMultipleStandardLoadBalancerConfigurations(clusterName
 		}
 	}
 
-	return nil
+	return az.reconcileMultipleStandardLoadBalancerBackendNodes("", lbs, service, nodes)
 }
 
 // reconcileLoadBalancer ensures load balancer exists and the frontend ip config is setup.
@@ -1503,16 +1537,19 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 		return nil, fmt.Errorf("reconcileLoadBalancer: failed to list managed LB: %w", err)
 	}
 
-	if err := az.reconcileMultipleStandardLoadBalancerConfigurations(clusterName, &existingLBs); err != nil {
+	if err := az.reconcileMultipleStandardLoadBalancerConfigurations(&existingLBs, service, clusterName, &existingLBs, nodes); err != nil {
 		klog.Errorf("reconcileLoadBalancer: failed to reconcile multiple standard load balancer configurations: %s", err.Error())
 		return nil, err
 	}
 
-	lb, lbStatus, _, _, err := az.getServiceLoadBalancer(service, clusterName, nodes, wantLb, existingLBs)
+	lb, lbStatus, _, _, deletedLBName, err := az.getServiceLoadBalancer(service, clusterName, nodes, wantLb, existingLBs)
 	if err != nil {
 		klog.Errorf("reconcileLoadBalancer: failed to get load balancer for service %q, error: %v", serviceName, err)
 		return nil, err
 	}
+	if deletedLBName != "" {
+		removeLBFromList(&existingLBs, deletedLBName)
+	}
 
 	lbName := *lb.Name
 	lbResourceGroup := az.getLoadBalancerResourceGroup()
@@ -1664,6 +1701,18 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 				return nil, fmt.Errorf("load balancer %q not found", lbName)
 			}
 			lb = newLB
+
+			var found bool
+			for i, existingLB := range existingLBs {
+				if strings.EqualFold(pointer.StringDeref(existingLB.Name, ""), pointer.StringDeref(newLB.Name, "")) {
+					existingLBs[i] = *newLB
+					found = true
+					break
+				}
+			}
+			if !found {
+				existingLBs = append(existingLBs, *newLB)
+			}
 		}
 	}
 
@@ -1675,12 +1724,31 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 			_ = az.lbCache.Delete(lbName)
 		}()
 
-		if lb.LoadBalancerPropertiesFormat != nil && lb.LoadBalancerPropertiesFormat.BackendAddressPools != nil {
-			for _, backendPool := range *lb.LoadBalancerPropertiesFormat.BackendAddressPools {
-				isIPv6 := isBackendPoolIPv6(pointer.StringDeref(backendPool.Name, ""))
-				if strings.EqualFold(pointer.StringDeref(backendPool.Name, ""), getBackendPoolName(clusterName, isIPv6)) {
-					if err := az.LoadBalancerBackendPool.EnsureHostsInPool(service, nodes, lbBackendPoolIDs[isIPv6], vmSetName, clusterName, lbName, backendPool); err != nil {
-						return nil, err
+		if az.useMultipleStandardLoadBalancers() {
+			err := az.reconcileMultipleStandardLoadBalancerBackendNodes(lbName, &existingLBs, service, nodes)
+			if err != nil {
+				return nil, err
+			}
+		}
+
+		// Need to reconcile every managed backend pools of all managed load balancers in
+		// the cluster when using multiple standard load balancers.
+		// This is because there are chances for backend pools from more than one load balancers
+		// change in one reconciliation loop.
+		var lbToReconcile []network.LoadBalancer
+		lbToReconcile = append(lbToReconcile, *lb)
+		if az.useMultipleStandardLoadBalancers() {
+			lbToReconcile = existingLBs
+		}
+		for _, lb := range lbToReconcile {
+			lbName := pointer.StringDeref(lb.Name, "")
+			if lb.LoadBalancerPropertiesFormat != nil && lb.LoadBalancerPropertiesFormat.BackendAddressPools != nil {
+				for _, backendPool := range *lb.LoadBalancerPropertiesFormat.BackendAddressPools {
+					isIPv6 := isBackendPoolIPv6(pointer.StringDeref(backendPool.Name, ""))
+					if strings.EqualFold(pointer.StringDeref(backendPool.Name, ""), getBackendPoolName(clusterName, isIPv6)) {
+						if err := az.LoadBalancerBackendPool.EnsureHostsInPool(service, nodes, lbBackendPoolIDs[isIPv6], vmSetName, clusterName, lbName, backendPool); err != nil {
+							return nil, err
+						}
 					}
 				}
 			}
@@ -1695,21 +1763,259 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 	return lb, nil
 }
 
+// removeLBFromList removes the given lb from the list
+func removeLBFromList(lbs *[]network.LoadBalancer, lbName string) {
+	if lbs != nil {
+		for i := len(*lbs) - 1; i >= 0; i-- {
+			if strings.EqualFold(pointer.StringDeref((*lbs)[i].Name, ""), lbName) {
+				*lbs = append((*lbs)[:i], (*lbs)[i+1:]...)
+				break
+			}
+		}
+	}
+}
+
+// removeNodeFromLBConfig searches for the occurrence of the given node in the lb configs and removes it
+func (az *Cloud) removeNodeFromLBConfig(nodeNameToLBConfigIDXMap map[string]int, nodeName string) {
+	if idx, ok := nodeNameToLBConfigIDXMap[nodeName]; ok {
+		currentLBConfigName := az.MultipleStandardLoadBalancerConfigurations[idx].Name
+		klog.V(4).Infof("reconcileMultipleStandardLoadBalancerBackendNodes: remove node(%s) on lb(%s)", nodeName, currentLBConfigName)
+		az.multipleStandardLoadBalancersActiveNodesLock.Lock()
+		az.MultipleStandardLoadBalancerConfigurations[idx].ActiveNodes.Delete(strings.ToLower(nodeName))
+		az.multipleStandardLoadBalancersActiveNodesLock.Unlock()
+	}
+}
+
+// removeDeletedNodesFromLoadBalancerConfigurations removes the deleted nodes
+// that do not exist in nodes list from the load balancer configurations
+func (az *Cloud) removeDeletedNodesFromLoadBalancerConfigurations(nodes []*v1.Node) map[string]int {
+	nodeNamesSet := sets.New[string]()
+	for _, node := range nodes {
+		nodeNamesSet.Insert(strings.ToLower(node.Name))
+	}
+
+	az.multipleStandardLoadBalancersActiveNodesLock.Lock()
+	defer az.multipleStandardLoadBalancersActiveNodesLock.Unlock()
+
+	// Remove the nodes from the load balancer configurations if they are not in the node list.
+	nodeNameToLBConfigIDXMap := make(map[string]int)
+	for i, multiSLBConfig := range az.MultipleStandardLoadBalancerConfigurations {
+		if multiSLBConfig.ActiveNodes != nil {
+			for nodeName := range multiSLBConfig.ActiveNodes {
+				if nodeNamesSet.Has(nodeName) {
+					nodeNameToLBConfigIDXMap[nodeName] = i
+				} else {
+					klog.V(4).Infof("reconcileMultipleStandardLoadBalancerBackendNodes: node(%s) is gone, remove it from lb(%s)", nodeName, multiSLBConfig.Name)
+					az.MultipleStandardLoadBalancerConfigurations[i].ActiveNodes, _ = safeRemoveKeyFromStringsSet(az.MultipleStandardLoadBalancerConfigurations[i].ActiveNodes, strings.ToLower(nodeName))
+				}
+			}
+		}
+	}
+
+	return nodeNameToLBConfigIDXMap
+}
+
+// accommodateNodesByPrimaryVMSet decides which load balancer configuration the node should be added to by primary vmSet
+func (az *Cloud) accommodateNodesByPrimaryVMSet(
+	lbName string,
+	lbs *[]network.LoadBalancer,
+	nodes []*v1.Node,
+	nodeNameToLBConfigIDXMap map[string]int,
+) error {
+	for _, node := range nodes {
+		if _, ok := az.nodesWithCorrectLoadBalancerByPrimaryVMSet.Load(strings.ToLower(node.Name)); ok {
+			continue
+		}
+
+		// TODO(niqi): reduce the API calls for VMAS and standalone VMs
+		vmSetName, err := az.VMSet.GetNodeVMSetName(node)
+		if err != nil {
+			klog.Errorf("accommodateNodesByPrimaryVMSet: failed to get vmSetName for node(%s): %s", node.Name, err.Error())
+			return err
+		}
+		for i := range az.MultipleStandardLoadBalancerConfigurations {
+			multiSLBConfig := az.MultipleStandardLoadBalancerConfigurations[i]
+			if strings.EqualFold(multiSLBConfig.PrimaryVMSet, vmSetName) {
+				var foundPrimaryLB bool
+				if lbs != nil {
+					for _, lb := range *lbs {
+						if strings.EqualFold(strings.TrimSuffix(pointer.StringDeref(lb.Name, ""), consts.InternalLoadBalancerNameSuffix), multiSLBConfig.Name) {
+							foundPrimaryLB = true
+							break
+						}
+					}
+				}
+				if !foundPrimaryLB && !strings.EqualFold(strings.TrimSuffix(lbName, consts.InternalLoadBalancerNameSuffix), multiSLBConfig.Name) {
+					klog.V(4).Infof("accommodateNodesByPrimaryVMSet: node(%s) should be on lb(%s) because of primary vmSet (%s), but the lb is not found and will not be created this time, will ignore the primaryVMSet", node.Name, multiSLBConfig.Name, vmSetName)
+					continue
+				}
+
+				az.nodesWithCorrectLoadBalancerByPrimaryVMSet.Store(strings.ToLower(node.Name), sets.Empty{})
+				if !multiSLBConfig.ActiveNodes.Has(node.Name) {
+					klog.V(4).Infof("accommodateNodesByPrimaryVMSet: node(%s) should be on lb(%s) because of primary vmSet (%s)", node.Name, multiSLBConfig.Name, vmSetName)
+
+					az.removeNodeFromLBConfig(nodeNameToLBConfigIDXMap, node.Name)
+
+					az.multipleStandardLoadBalancersActiveNodesLock.Lock()
+					az.MultipleStandardLoadBalancerConfigurations[i].ActiveNodes = safeAddKeyToStringsSet(az.MultipleStandardLoadBalancerConfigurations[i].ActiveNodes, strings.ToLower(node.Name))
+					az.multipleStandardLoadBalancersActiveNodesLock.Unlock()
+				}
+				break
+			}
+		}
+	}
+
+	return nil
+}
+
+// accommodateNodesByNodeSelector decides which load balancer configuration the node should be added to by node selector
+func (az *Cloud) accommodateNodesByNodeSelector(
+	lbName string,
+	lbs *[]network.LoadBalancer,
+	service *v1.Service,
+	nodes []*v1.Node,
+	nodeNameToLBConfigIDXMap map[string]int,
+) error {
+	for _, node := range nodes {
+		// Skip nodes that have been matched with a load balancer
+		// by primary vmSet.
+		if _, ok := az.nodesWithCorrectLoadBalancerByPrimaryVMSet.Load(strings.ToLower(node.Name)); ok {
+			continue
+		}
+
+		// If the vmSet of the node does not match any load balancer,
+		// pick all load balancers whose node selector matches the node.
+		var eligibleLBsIDX []int
+		for i, multiSLBConfig := range az.MultipleStandardLoadBalancerConfigurations {
+			if multiSLBConfig.NodeSelector != nil &&
+				(len(multiSLBConfig.NodeSelector.MatchLabels) > 0 || len(multiSLBConfig.NodeSelector.MatchExpressions) > 0) {
+				nodeSelector, err := metav1.LabelSelectorAsSelector(multiSLBConfig.NodeSelector)
+				if err != nil {
+					klog.Errorf("accommodateNodesByNodeSelector: failed to parse nodeSelector for lb(%s): %s", multiSLBConfig.Name, err.Error())
+					return err
+				}
+				if nodeSelector.Matches(labels.Set(node.Labels)) {
+					var foundLB bool
+					if lbs != nil {
+						for _, lb := range *lbs {
+							if strings.EqualFold(strings.TrimSuffix(pointer.StringDeref(lb.Name, ""), consts.InternalLoadBalancerNameSuffix), multiSLBConfig.Name) {
+								foundLB = true
+								break
+							}
+						}
+					}
+					klog.V(4).Infof("accommodateNodesByNodeSelector: lb(%s) matches node(%s) labels", multiSLBConfig.Name, node.Name)
+					if !foundLB && !strings.EqualFold(strings.TrimSuffix(lbName, consts.InternalLoadBalancerNameSuffix), multiSLBConfig.Name) {
+						klog.V(4).Infof("accommodateNodesByNodeSelector: but the lb is not found and will not be created this time, will ignore this load balancer")
+						continue
+					}
+					eligibleLBsIDX = append(eligibleLBsIDX, i)
+				}
+			}
+		}
+		// If no load balancer is matched, all load balancers without node selector are eligible.
+		if len(eligibleLBsIDX) == 0 {
+			for i, multiSLBConfig := range az.MultipleStandardLoadBalancerConfigurations {
+				if multiSLBConfig.NodeSelector == nil {
+					eligibleLBsIDX = append(eligibleLBsIDX, i)
+				}
+			}
+		}
+		if idx, ok := nodeNameToLBConfigIDXMap[node.Name]; ok {
+			if IntInSlice(idx, eligibleLBsIDX) {
+				klog.V(4).Infof("accommodateNodesByNodeSelector: node(%s) is already on the eligible lb(%s)", node.Name, az.MultipleStandardLoadBalancerConfigurations[idx].Name)
+				continue
+			}
+		}
+
+		// Pick one with the fewest nodes among all eligible load balancers.
+		minNodesIDX := -1
+		minNodes := math.MaxInt32
+		az.multipleStandardLoadBalancersActiveNodesLock.Lock()
+		for _, idx := range eligibleLBsIDX {
+			multiSLBConfig := az.MultipleStandardLoadBalancerConfigurations[idx]
+			if multiSLBConfig.ActiveNodes.Len() < minNodes {
+				minNodes = multiSLBConfig.ActiveNodes.Len()
+				minNodesIDX = idx
+			}
+		}
+		az.multipleStandardLoadBalancersActiveNodesLock.Unlock()
+
+		if idx, ok := nodeNameToLBConfigIDXMap[node.Name]; ok && idx != minNodesIDX {
+			az.removeNodeFromLBConfig(nodeNameToLBConfigIDXMap, node.Name)
+		}
+
+		// Emit a warning for the orphaned node.
+		if minNodesIDX == -1 {
+			warningMsg := fmt.Sprintf("failed to find a lb for node %s", node.Name)
+			az.Event(service, v1.EventTypeWarning, "FailedToFindLoadBalancerForNode", warningMsg)
+			continue
+		}
+
+		klog.V(4).Infof("accommodateNodesByNodeSelector: node(%s) should be on lb(%s) it is the eligible LB with fewest number of nodes", node.Name, az.MultipleStandardLoadBalancerConfigurations[minNodesIDX].Name)
+		az.multipleStandardLoadBalancersActiveNodesLock.Lock()
+		az.MultipleStandardLoadBalancerConfigurations[minNodesIDX].ActiveNodes = safeAddKeyToStringsSet(az.MultipleStandardLoadBalancerConfigurations[minNodesIDX].ActiveNodes, strings.ToLower(node.Name))
+		az.multipleStandardLoadBalancersActiveNodesLock.Unlock()
+	}
+
+	return nil
+}
+
+// reconcileMultipleStandardLoadBalancerBackendNodes makes sure the arrangement of nodes
+// across load balancer configurations is expected. This is used in two places:
+// 1. Every time the cloud provide restarts.
+// 2. Every time we ensure hosts in pool.
+// It consists of two parts. First we put corresponding nodes to the load balancers
+// whose primary vmSet matches the node. Then we put the rest of the nodes to the
+// most eligible load balancers according to the node selector and the number of
+// nodes currently in the load balancer.
+// For availability set (no cache) amd vmss flex (with cache) clusters,
+// a list call will be introduced every time we
+// try to get the vmSet of a node. This is acceptable because of two reasons:
+// 1. In AKS, we don't support multiple availability sets in a cluster so the
+// cluster scale is small. For self-managed clusters, it is not recommended to
+// use multiple standard load balancers with availability sets.
+// 2. We only check nodes that are not matched by primary vmSet before we ensure
+// hosts in pool. So the number API calls is under control.
+func (az *Cloud) reconcileMultipleStandardLoadBalancerBackendNodes(
+	lbName string,
+	lbs *[]network.LoadBalancer,
+	service *v1.Service,
+	nodes []*v1.Node,
+) error {
+	// Remove the nodes from the load balancer configurations if they are not in the node list.
+	nodeNameToLBConfigIDXMap := az.removeDeletedNodesFromLoadBalancerConfigurations(nodes)
+
+	err := az.accommodateNodesByPrimaryVMSet(lbName, lbs, nodes, nodeNameToLBConfigIDXMap)
+	if err != nil {
+		return err
+	}
+
+	err = az.accommodateNodesByNodeSelector(lbName, lbs, service, nodes, nodeNameToLBConfigIDXMap)
+	if err != nil {
+		return err
+	}
+
+	return nil
+}
+
 func (az *Cloud) reconcileMultipleStandardLoadBalancerConfigurationStatus(wantLb bool, svcName, lbName string) {
 	lbName = strings.TrimSuffix(lbName, consts.InternalLoadBalancerNameSuffix)
 	for i := range az.MultipleStandardLoadBalancerConfigurations {
 		if strings.EqualFold(lbName, az.MultipleStandardLoadBalancerConfigurations[i].Name) {
+			az.multipleStandardLoadBalancersActiveServicesLock.Lock()
 			if az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices == nil {
 				az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices = sets.New[string]()
 			}
 
 			if wantLb {
 				klog.V(4).Infof("reconcileMultipleStandardLoadBalancerConfigurationStatus: service(%s) is active on lb(%s)", svcName, lbName)
-				az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices.Insert(svcName)
+				az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices.Insert(strings.ToLower(svcName))
 			} else {
 				klog.V(4).Infof("reconcileMultipleStandardLoadBalancerConfigurationStatus: service(%s) is not active on lb(%s) any more", svcName, lbName)
-				az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices.Delete(svcName)
+				az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices, _ = safeRemoveKeyFromStringsSet(az.MultipleStandardLoadBalancerConfigurations[i].ActiveServices, strings.ToLower(svcName))
 			}
+			az.multipleStandardLoadBalancersActiveServicesLock.Unlock()
 			break
 		}
 	}
@@ -2730,7 +3036,7 @@ func (az *Cloud) shouldUpdateLoadBalancer(clusterName string, service *v1.Servic
 		return false, fmt.Errorf("shouldUpdateLoadBalancer: failed to list managed load balancers: %w", err)
 	}
 
-	_, _, _, existsLb, _ := az.getServiceLoadBalancer(service, clusterName, nodes, false, existingManagedLBs)
+	_, _, _, existsLb, _, _ := az.getServiceLoadBalancer(service, clusterName, nodes, false, existingManagedLBs)
 	return existsLb && service.ObjectMeta.DeletionTimestamp == nil && service.Spec.Type == v1.ServiceTypeLoadBalancer, nil
 }
 
@@ -3690,13 +3996,13 @@ func (az *Cloud) getAzureLoadBalancerName(
 	// 1. Filter out the eligible load balancers.
 	// 2. Choose the most eligible load balancer.
 	if az.useMultipleStandardLoadBalancers() {
-		eligibleLBs, err := az.getEligibleLoadBalancers(service)
+		eligibleLBs, err := az.getEligibleLoadBalancersForService(service)
 		if err != nil {
 			return "", err
 		}
 
 		currentLBName := az.getServiceCurrentLoadBalancerName(service)
-		lbNamePrefix = getMostEligibleLBName(currentLBName, eligibleLBs, existingLBs)
+		lbNamePrefix = getMostEligibleLBForService(currentLBName, eligibleLBs, existingLBs)
 	}
 
 	if isInternal {
@@ -3705,14 +4011,14 @@ func (az *Cloud) getAzureLoadBalancerName(
 	return lbNamePrefix, nil
 }
 
-func getMostEligibleLBName(
+func getMostEligibleLBForService(
 	currentLBName string,
 	eligibleLBs []string,
 	existingLBs *[]network.LoadBalancer,
 ) string {
 	// 1. If the LB is eligible and being used, choose it.
 	if StringInSlice(currentLBName, eligibleLBs) {
-		klog.V(4).Infof("getMostEligibleLBName: choose %s as it is eligible and being used", currentLBName)
+		klog.V(4).Infof("getMostEligibleLBForService: choose %s as it is eligible and being used", currentLBName)
 		return currentLBName
 	}
 
@@ -3728,7 +4034,7 @@ func getMostEligibleLBName(
 			}
 		}
 		if !found {
-			klog.V(4).Infof("getMostEligibleLBName: choose %s as it is eligible and not existing", eligibleLB)
+			klog.V(4).Infof("getMostEligibleLBForService: choose %s as it is eligible and not existing", eligibleLB)
 			return eligibleLB
 		}
 	}
@@ -3751,7 +4057,7 @@ func getMostEligibleLBName(
 	}
 
 	if expectedLBName != "" {
-		klog.V(4).Infof("getMostEligibleLBName: choose %s with fewest %d rules", expectedLBName, ruleCount)
+		klog.V(4).Infof("getMostEligibleLBForService: choose %s with fewest %d rules", expectedLBName, ruleCount)
 	}
 
 	return expectedLBName
@@ -3759,14 +4065,14 @@ func getMostEligibleLBName(
 
 func (az *Cloud) getServiceCurrentLoadBalancerName(service *v1.Service) string {
 	for _, multiSLBConfig := range az.MultipleStandardLoadBalancerConfigurations {
-		if isLoadBalancerInUseByService(service, multiSLBConfig) {
+		if az.isLoadBalancerInUseByService(service, multiSLBConfig) {
 			return multiSLBConfig.Name
 		}
 	}
 	return ""
 }
 
-// getEligibleLoadBalancers filter out the eligible load balancers for the service.
+// getEligibleLoadBalancersForService filter out the eligible load balancers for the service.
 // It follows four kinds of constraints:
 // 1. Service annotation `service.beta.kubernetes.io/azure-load-balancer-configurations: lb1,lb2`.
 // 2. AllowServicePlacement flag. Default to true, if set to false, the new services will not be put onto the LB.
@@ -3775,7 +4081,7 @@ func (az *Cloud) getServiceCurrentLoadBalancerName(service *v1.Service) string {
 // If there is no ServiceLabel selector on the LB, all services can be valid.
 // 4. ServiceNamespaceSelector. The service will be put onto the LB only if the service is in the namespaces specified in the selector.
 // If there is no ServiceNamespace selector on the LB, all services can be valid.
-func (az *Cloud) getEligibleLoadBalancers(service *v1.Service) ([]string, error) {
+func (az *Cloud) getEligibleLoadBalancersForService(service *v1.Service) ([]string, error) {
 	var (
 		eligibleLBs               []MultipleStandardLoadBalancerConfiguration
 		eligibleLBNames           []string
@@ -3792,13 +4098,16 @@ func (az *Cloud) getEligibleLoadBalancers(service *v1.Service) ([]string, error)
 		lbNamesSet := sets.New[string](lbsFromAnnotation...)
 		for _, multiSLBConfig := range az.MultipleStandardLoadBalancerConfigurations {
 			if lbNamesSet.Has(strings.ToLower(multiSLBConfig.Name)) {
-				klog.V(4).Infof("getEligibleLoadBalancers: service %q selects load balancer %q by annotation", service.Name, multiSLBConfig.Name)
+				klog.V(4).Infof("getEligibleLoadBalancersForService: service %q selects load balancer %q by annotation", service.Name, multiSLBConfig.Name)
 				eligibleLBs = append(eligibleLBs, multiSLBConfig)
 				lbSelectedByAnnotation = append(lbSelectedByAnnotation, multiSLBConfig.Name)
 			}
 		}
+		if len(lbSelectedByAnnotation) == 0 {
+			return nil, fmt.Errorf("service %q selects %d load balancers by annotation, but none of them is defined in cloud provider configuration", service.Name, len(lbsFromAnnotation))
+		}
 	} else {
-		klog.V(4).Infof("getEligibleLoadBalancers: service %q does not select any load balancer by annotation, all load balancers are eligible", service.Name)
+		klog.V(4).Infof("getEligibleLoadBalancersForService: service %q does not select any load balancer by annotation, all load balancers are eligible", service.Name)
 		eligibleLBs = append(eligibleLBs, az.MultipleStandardLoadBalancerConfigurations...)
 		for _, eligibleLB := range eligibleLBs {
 			lbSelectedByAnnotation = append(lbSelectedByAnnotation, eligibleLB.Name)
@@ -3811,10 +4120,10 @@ func (az *Cloud) getEligibleLoadBalancers(service *v1.Service) ([]string, error)
 		// 2. If the LB does not allow service placement, it is not eligible,
 		// unless the service is already using the LB.
 		if !pointer.BoolDeref(eligibleLB.AllowServicePlacement, true) {
-			if isLoadBalancerInUseByService(service, eligibleLB) {
-				klog.V(4).Infof("getEligibleLoadBalancers: although load balancer %q has AllowServicePlacement=false, service %q is allowed to be placed on load balancer %q because it is using the load balancer", eligibleLB.Name, service.Name, eligibleLB.Name)
+			if az.isLoadBalancerInUseByService(service, eligibleLB) {
+				klog.V(4).Infof("getEligibleLoadBalancersForService: although load balancer %q has AllowServicePlacement=false, service %q is allowed to be placed on load balancer %q because it is using the load balancer", eligibleLB.Name, service.Name, eligibleLB.Name)
 			} else {
-				klog.V(4).Infof("getEligibleLoadBalancers: service %q is not allowed to be placed on load balancer %q", service.Name, eligibleLB.Name)
+				klog.V(4).Infof("getEligibleLoadBalancersForService: service %q is not allowed to be placed on load balancer %q", service.Name, eligibleLB.Name)
 				eligibleLBs = append(eligibleLBs[:i], eligibleLBs[i+1:]...)
 				lbFailedPlacementFlag = append(lbFailedPlacementFlag, eligibleLB.Name)
 				continue
@@ -3830,7 +4139,7 @@ func (az *Cloud) getEligibleLoadBalancers(service *v1.Service) ([]string, error)
 				return []string{}, err
 			}
 			if !serviceLabelSelector.Matches(labels.Set(service.Labels)) {
-				klog.V(2).Infof("getEligibleLoadBalancers: service %q does not match label selector %q for load balancer %q", service.Name, eligibleLB.ServiceLabelSelector.String(), eligibleLB.Name)
+				klog.V(2).Infof("getEligibleLoadBalancersForService: service %q does not match label selector %q for load balancer %q", service.Name, eligibleLB.ServiceLabelSelector.String(), eligibleLB.Name)
 				eligibleLBs = append(eligibleLBs[:i], eligibleLBs[i+1:]...)
 				lbFailedLabelSelector = append(lbFailedLabelSelector, eligibleLB.Name)
 				continue
@@ -3851,7 +4160,7 @@ func (az *Cloud) getEligibleLoadBalancers(service *v1.Service) ([]string, error)
 				return []string{}, err
 			}
 			if !serviceNamespaceSelector.Matches(labels.Set(ns.Labels)) {
-				klog.V(2).Infof("getEligibleLoadBalancers: namespace %q does not match namespace selector %q for load balancer %q", service.Namespace, eligibleLB.ServiceNamespaceSelector.String(), eligibleLB.Name)
+				klog.V(2).Infof("getEligibleLoadBalancersForService: namespace %q does not match namespace selector %q for load balancer %q", service.Namespace, eligibleLB.ServiceNamespaceSelector.String(), eligibleLB.Name)
 				eligibleLBs = append(eligibleLBs[:i], eligibleLBs[i+1:]...)
 				lbFailedNamespaceSelector = append(lbFailedNamespaceSelector, eligibleLB.Name)
 				continue
@@ -3882,7 +4191,10 @@ func (az *Cloud) getEligibleLoadBalancers(service *v1.Service) ([]string, error)
 	return eligibleLBNames, nil
 }
 
-func isLoadBalancerInUseByService(service *v1.Service, lbConfig MultipleStandardLoadBalancerConfiguration) bool {
+func (az *Cloud) isLoadBalancerInUseByService(service *v1.Service, lbConfig MultipleStandardLoadBalancerConfiguration) bool {
+	az.multipleStandardLoadBalancersActiveServicesLock.Lock()
+	defer az.multipleStandardLoadBalancersActiveServicesLock.Unlock()
+
 	serviceName := getServiceName(service)
 	if lbConfig.ActiveServices != nil {
 		return lbConfig.ActiveServices.Has(serviceName)
diff --git a/pkg/provider/azure_loadbalancer_backendpool.go b/pkg/provider/azure_loadbalancer_backendpool.go
index 964e78abb..162832319 100644
--- a/pkg/provider/azure_loadbalancer_backendpool.go
+++ b/pkg/provider/azure_loadbalancer_backendpool.go
@@ -196,7 +196,7 @@ func (bc *backendPoolTypeNodeIPConfig) ReconcileBackendPools(clusterName string,
 			if bp.BackendAddressPoolPropertiesFormat != nil &&
 				bp.LoadBalancerBackendAddresses != nil &&
 				len(*bp.LoadBalancerBackendAddresses) > 0 {
-				if removeNodeIPAddressesFromBackendPool(bp, []string{}, true) {
+				if removeNodeIPAddressesFromBackendPool(bp, []string{}, true, false) {
 					isMigration = true
 					bp.VirtualNetwork = nil
 					if err := bc.CreateOrUpdateLBBackendPool(lbName, bp); err != nil {
@@ -395,7 +395,15 @@ func (bi *backendPoolTypeNodeIP) EnsureHostsInPool(service *v1.Service, nodes []
 	var (
 		changed               bool
 		numOfAdd, numOfDelete int
+		activeNodes           sets.Set[string]
 	)
+	if bi.useMultipleStandardLoadBalancers() {
+		activeNodes = bi.getActiveNodesByLoadBalancerName(lbName)
+		if activeNodes == nil {
+			activeNodes = sets.New[string]()
+		}
+	}
+
 	lbBackendPoolName := getBackendPoolName(clusterName, isIPv6)
 	if strings.EqualFold(pointer.StringDeref(backendPool.Name, ""), lbBackendPoolName) &&
 		backendPool.BackendAddressPoolPropertiesFormat != nil {
@@ -426,6 +434,14 @@ func (bi *backendPoolTypeNodeIP) EnsureHostsInPool(service *v1.Service, nodes []
 
 			privateIP := getNodePrivateIPAddress(node, isIPv6)
 			nodePrivateIPsSet.Insert(privateIP)
+
+			if bi.useMultipleStandardLoadBalancers() {
+				if activeNodes == nil || !activeNodes.Has(node.Name) {
+					klog.V(4).Infof("bi.EnsureHostsInPool: node %s should not be in load balancer %q", node.Name, lbName)
+					continue
+				}
+			}
+
 			if !existingIPs.Has(privateIP) {
 				name := node.Name
 				if utilnet.IsIPv6String(privateIP) {
@@ -448,13 +464,25 @@ func (bi *backendPoolTypeNodeIP) EnsureHostsInPool(service *v1.Service, nodes []
 		for _, loadBalancerBackendAddress := range *backendPool.LoadBalancerBackendAddresses {
 			ip := pointer.StringDeref(loadBalancerBackendAddress.IPAddress, "")
 			if !nodePrivateIPsSet.Has(ip) {
-				klog.V(4).Infof("bi.EnsureHostsInPool: removing IP %s", ip)
+				klog.V(4).Infof("bi.EnsureHostsInPool: removing IP %s because it is deleted or should be excluded", ip)
 				nodeIPsToBeDeleted = append(nodeIPsToBeDeleted, ip)
 				changed = true
 				numOfDelete++
+			} else if bi.useMultipleStandardLoadBalancers() && activeNodes != nil {
+				nodeName, ok := bi.nodePrivateIPToNodeNameMap[ip]
+				if !ok {
+					klog.Warningf("bi.EnsureHostsInPool: cannot find node name for private IP %s", ip)
+					continue
+				}
+				if !activeNodes.Has(nodeName) {
+					klog.V(4).Infof("bi.EnsureHostsInPool: removing IP %s because it should not be in this load balancer", ip)
+					nodeIPsToBeDeleted = append(nodeIPsToBeDeleted, ip)
+					changed = true
+					numOfDelete++
+				}
 			}
 		}
-		removeNodeIPAddressesFromBackendPool(backendPool, nodeIPsToBeDeleted, false)
+		removeNodeIPAddressesFromBackendPool(backendPool, nodeIPsToBeDeleted, false, bi.useMultipleStandardLoadBalancers())
 	}
 	if changed {
 		klog.V(2).Infof("bi.EnsureHostsInPool: updating backend pool %s of load balancer %s to add %d nodes and remove %d nodes", lbBackendPoolName, lbName, numOfAdd, numOfDelete)
@@ -628,7 +656,7 @@ func (bi *backendPoolTypeNodeIP) ReconcileBackendPools(clusterName string, servi
 				}
 			}
 			if len(nodeIPAddressesToBeDeleted) > 0 {
-				if removeNodeIPAddressesFromBackendPool(bp, nodeIPAddressesToBeDeleted, false) {
+				if removeNodeIPAddressesFromBackendPool(bp, nodeIPAddressesToBeDeleted, false, false) {
 					updated = true
 				}
 			}
@@ -744,7 +772,11 @@ func newBackendPool(lb *network.LoadBalancer, isBackendPoolPreConfigured bool, p
 	return isBackendPoolPreConfigured
 }
 
-func removeNodeIPAddressesFromBackendPool(backendPool network.BackendAddressPool, nodeIPAddresses []string, removeAll bool) bool {
+func removeNodeIPAddressesFromBackendPool(
+	backendPool network.BackendAddressPool,
+	nodeIPAddresses []string,
+	removeAll, useMultipleStandardLoadBalancers bool,
+) bool {
 	changed := false
 	nodeIPsSet := sets.New(nodeIPAddresses...)
 
@@ -774,7 +806,9 @@ func removeNodeIPAddressesFromBackendPool(backendPool network.BackendAddressPool
 		return changed
 	}
 
-	if len(addresses) == 0 {
+	// Allow the pool to be empty when EnsureHostsInPool for multiple standard load balancers clusters,
+	// or one node could occur in multiple backend pools.
+	if len(addresses) == 0 && !useMultipleStandardLoadBalancers {
 		klog.V(2).Info("removeNodeIPAddressFromBackendPool: the pool is empty or will be empty after removing the unwanted IP addresses, skipping the removal")
 		changed = false
 	} else if changed {
diff --git a/pkg/provider/azure_utils.go b/pkg/provider/azure_utils.go
index d038fc4ce..eef784774 100644
--- a/pkg/provider/azure_utils.go
+++ b/pkg/provider/azure_utils.go
@@ -26,6 +26,7 @@ import (
 	"github.com/Azure/azure-sdk-for-go/services/network/mgmt/2022-07-01/network"
 
 	v1 "k8s.io/api/core/v1"
+	"k8s.io/apimachinery/pkg/util/sets"
 	"k8s.io/klog/v2"
 	utilnet "k8s.io/utils/net"
 	"k8s.io/utils/pointer"
@@ -547,3 +548,35 @@ func stringSlice(s *[]string) []string {
 	}
 	return nil
 }
+
+// IntInSlice checks if an int is in a list
+func IntInSlice(i int, list []int) bool {
+	for _, item := range list {
+		if item == i {
+			return true
+		}
+	}
+	return false
+}
+
+func safeAddKeyToStringsSet(set sets.Set[string], key string) sets.Set[string] {
+	if set != nil {
+		set.Insert(key)
+	} else {
+		set = sets.New[string](key)
+	}
+
+	return set
+}
+
+func safeRemoveKeyFromStringsSet(set sets.Set[string], key string) (sets.Set[string], bool) {
+	var has bool
+	if set != nil {
+		if set.Has(key) {
+			has = true
+		}
+		set.Delete(key)
+	}
+
+	return set, has
+}


Test patch:
diff --git a/e2e.Dockerfile b/e2e.Dockerfile
index ffa1c1ff4..852c59e1a 100644
--- a/e2e.Dockerfile
+++ b/e2e.Dockerfile
@@ -14,7 +14,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM golang:1.20-bullseye@sha256:851af0a8ca4eba552c84db5b2edac7f3be15deb5892217961a1d4b175585a603
+FROM golang:1.20-bullseye@sha256:d9f75197f484a4d7e008a0a8f7d1e68344ea1c1ba030273d5ab1567e129eed55
 
 WORKDIR /go/src/sigs.k8s.io/cloud-provider-azure
 
diff --git a/pkg/provider/azure_loadbalancer_backendpool_test.go b/pkg/provider/azure_loadbalancer_backendpool_test.go
index e757be9ba..e165a9ce9 100644
--- a/pkg/provider/azure_loadbalancer_backendpool_test.go
+++ b/pkg/provider/azure_loadbalancer_backendpool_test.go
@@ -87,11 +87,29 @@ func TestEnsureHostsInPoolNodeIP(t *testing.T) {
 				},
 			},
 		},
+		{
+			ObjectMeta: metav1.ObjectMeta{
+				Name: "vmss-2",
+			},
+			Status: v1.NodeStatus{
+				Addresses: []v1.NodeAddress{
+					{
+						Type:    v1.NodeInternalIP,
+						Address: "10.0.0.4",
+					},
+					{
+						Type:    v1.NodeInternalIP,
+						Address: "2001::4",
+					},
+				},
+			},
+		},
 	}
 
 	testcases := []struct {
 		desc                string
 		backendPool         network.BackendAddressPool
+		multiSLBConfigs     []MultipleStandardLoadBalancerConfiguration
 		expectedBackendPool network.BackendAddressPool
 	}{
 		{
@@ -129,6 +147,12 @@ func TestEnsureHostsInPoolNodeIP(t *testing.T) {
 								IPAddress: pointer.String("10.0.0.2"),
 							},
 						},
+						{
+							Name: pointer.String("vmss-2"),
+							LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+								IPAddress: pointer.String("10.0.0.4"),
+							},
+						},
 					},
 				},
 			},
@@ -163,6 +187,54 @@ func TestEnsureHostsInPoolNodeIP(t *testing.T) {
 								IPAddress: pointer.String("2001::2"),
 							},
 						},
+						{
+							Name: pointer.String("vmss-2-IPv6"),
+							LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+								IPAddress: pointer.String("2001::4"),
+							},
+						},
+					},
+				},
+			},
+		},
+		{
+			desc: "should add correct nodes to the pool and remove unwanted ones when using multi-slb",
+			backendPool: network.BackendAddressPool{
+				Name: pointer.String("kubernetes"),
+				BackendAddressPoolPropertiesFormat: &network.BackendAddressPoolPropertiesFormat{
+					LoadBalancerBackendAddresses: &[]network.LoadBalancerBackendAddress{
+						{
+							LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+								IPAddress: pointer.String("10.0.0.1"),
+							},
+						},
+						{
+							LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+								IPAddress: pointer.String("10.0.0.3"),
+							},
+						},
+					},
+				},
+			},
+			multiSLBConfigs: []MultipleStandardLoadBalancerConfiguration{
+				{
+					Name: "kubernetes",
+					MultipleStandardLoadBalancerConfigurationStatus: MultipleStandardLoadBalancerConfigurationStatus{
+						ActiveNodes: sets.New[string]("vmss-2"),
+					},
+				},
+			},
+			expectedBackendPool: network.BackendAddressPool{
+				Name: pointer.String("kubernetes"),
+				BackendAddressPoolPropertiesFormat: &network.BackendAddressPoolPropertiesFormat{
+					VirtualNetwork: &network.SubResource{ID: pointer.String("/subscriptions/subscription/resourceGroups/rg/providers/Microsoft.Network/virtualNetworks/vnet")},
+					LoadBalancerBackendAddresses: &[]network.LoadBalancerBackendAddress{
+						{
+							Name: pointer.String("vmss-2"),
+							LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+								IPAddress: pointer.String("10.0.0.4"),
+							},
+						},
 					},
 				},
 			},
@@ -171,8 +243,18 @@ func TestEnsureHostsInPoolNodeIP(t *testing.T) {
 
 	for _, tc := range testcases {
 		t.Run(tc.desc, func(t *testing.T) {
-			mockVMSet := NewMockVMSet(ctrl)
-			az.VMSet = mockVMSet
+			if len(tc.multiSLBConfigs) > 0 {
+				az.MultipleStandardLoadBalancerConfigurations = tc.multiSLBConfigs
+				az.LoadBalancerSku = consts.LoadBalancerSkuStandard
+				az.nodePrivateIPToNodeNameMap = map[string]string{
+					"10.0.0.2": "vmss-0",
+					"2001::2":  "vmss-0",
+					"10.0.0.1": "vmss-1",
+					"2001::1":  "vmss-1",
+					"10.0.0.4": "vmss-2",
+					"2001::4":  "vmss-2",
+				}
+			}
 
 			lbClient := mockloadbalancerclient.NewMockInterface(ctrl)
 			lbClient.EXPECT().CreateOrUpdateBackendPools(gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any(), gomock.Any()).Return(nil)
@@ -712,7 +794,7 @@ func buildTestLoadBalancerBackendPoolWithIPs(name string, ips []string) network.
 func TestRemoveNodeIPAddressFromBackendPool(t *testing.T) {
 	for _, tc := range []struct {
 		description                           string
-		removeAll                             bool
+		removeAll, useMultiSLB                bool
 		unwantedIPs, existingIPs, expectedIPs []string
 	}{
 		{
@@ -727,6 +809,13 @@ func TestRemoveNodeIPAddressFromBackendPool(t *testing.T) {
 			existingIPs: []string{"1.2.3.4", "4.3.2.1"},
 			expectedIPs: []string{"1.2.3.4", "4.3.2.1"},
 		},
+		{
+			description: "removeNodeIPAddressFromBackendPool should make the backend pool empty for multi-SLB",
+			unwantedIPs: []string{"1.2.3.4", "4.3.2.1"},
+			existingIPs: []string{"1.2.3.4", "4.3.2.1"},
+			useMultiSLB: true,
+			expectedIPs: []string{},
+		},
 		{
 			description: "removeNodeIPAddressFromBackendPool should remove all the IP addresses from the backend pool",
 			removeAll:   true,
@@ -739,7 +828,7 @@ func TestRemoveNodeIPAddressFromBackendPool(t *testing.T) {
 			backendPool := buildTestLoadBalancerBackendPoolWithIPs("kubernetes", tc.existingIPs)
 			expectedBackendPool := buildTestLoadBalancerBackendPoolWithIPs("kubernetes", tc.expectedIPs)
 
-			removeNodeIPAddressesFromBackendPool(backendPool, tc.unwantedIPs, tc.removeAll)
+			removeNodeIPAddressesFromBackendPool(backendPool, tc.unwantedIPs, tc.removeAll, tc.useMultiSLB)
 			assert.Equal(t, expectedBackendPool, backendPool)
 		})
 	}
diff --git a/pkg/provider/azure_loadbalancer_test.go b/pkg/provider/azure_loadbalancer_test.go
index 925f570cb..db6238f1c 100644
--- a/pkg/provider/azure_loadbalancer_test.go
+++ b/pkg/provider/azure_loadbalancer_test.go
@@ -1792,7 +1792,7 @@ func TestGetServiceLoadBalancerCommon(t *testing.T) {
 			}
 			az.LoadBalancerSku = test.sku
 			service := test.service
-			lb, status, _, exists, err := az.getServiceLoadBalancer(&service, testClusterName,
+			lb, status, _, exists, _, err := az.getServiceLoadBalancer(&service, testClusterName,
 				clusterResources.nodes, test.wantLB, []network.LoadBalancer{})
 			assert.Equal(t, test.expectedLB, lb)
 			assert.Equal(t, test.expectedStatus, status)
@@ -1825,7 +1825,7 @@ func TestGetServiceLoadBalancerWithExtendedLocation(t *testing.T) {
 	mockLBsClient.EXPECT().List(gomock.Any(), "rg").Return(nil, nil)
 	az.LoadBalancerClient = mockLBsClient
 
-	lb, status, _, exists, err := az.getServiceLoadBalancer(&service, testClusterName,
+	lb, status, _, exists, _, err := az.getServiceLoadBalancer(&service, testClusterName,
 		clusterResources.nodes, false, []network.LoadBalancer{})
 	assert.Equal(t, expectedLB, lb, "GetServiceLoadBalancer shall return a default LB with expected location.")
 	assert.Nil(t, status, "GetServiceLoadBalancer: Status should be nil for default LB.")
@@ -1850,7 +1850,7 @@ func TestGetServiceLoadBalancerWithExtendedLocation(t *testing.T) {
 	mockLBsClient.EXPECT().List(gomock.Any(), "rg").Return(nil, nil)
 	az.LoadBalancerClient = mockLBsClient
 
-	lb, status, _, exists, err = az.getServiceLoadBalancer(&service, testClusterName,
+	lb, status, _, exists, _, err = az.getServiceLoadBalancer(&service, testClusterName,
 		clusterResources.nodes, true, []network.LoadBalancer{})
 	assert.Equal(t, expectedLB, lb, "GetServiceLoadBalancer shall return a new LB with expected location.")
 	assert.Nil(t, status, "GetServiceLoadBalancer: Status should be nil for new LB.")
@@ -6321,7 +6321,7 @@ func TestRemoveFrontendIPConfigurationFromLoadBalancerDelete(t *testing.T) {
 		mockPLSClient := cloud.PrivateLinkServiceClient.(*mockprivatelinkserviceclient.MockInterface)
 		mockPLSClient.EXPECT().List(gomock.Any(), "rg").Return(expectedPLS, nil).MaxTimes(1)
 		existingLBs := []network.LoadBalancer{{Name: pointer.String("lb")}}
-		err := cloud.removeFrontendIPConfigurationFromLoadBalancer(&lb, existingLBs, []*network.FrontendIPConfiguration{fip}, "testCluster", &service)
+		_, err := cloud.removeFrontendIPConfigurationFromLoadBalancer(&lb, existingLBs, []*network.FrontendIPConfiguration{fip}, "testCluster", &service)
 		assert.NoError(t, err)
 	})
 }
@@ -6352,7 +6352,7 @@ func TestRemoveFrontendIPConfigurationFromLoadBalancerUpdate(t *testing.T) {
 		expectedPLS := make([]network.PrivateLinkService, 0)
 		mockPLSClient := cloud.PrivateLinkServiceClient.(*mockprivatelinkserviceclient.MockInterface)
 		mockPLSClient.EXPECT().List(gomock.Any(), "rg").Return(expectedPLS, nil).MaxTimes(1)
-		err := cloud.removeFrontendIPConfigurationFromLoadBalancer(&lb, []network.LoadBalancer{}, []*network.FrontendIPConfiguration{fip}, "testCluster", &service)
+		_, err := cloud.removeFrontendIPConfigurationFromLoadBalancer(&lb, []network.LoadBalancer{}, []*network.FrontendIPConfiguration{fip}, "testCluster", &service)
 		assert.NoError(t, err)
 	})
 }
@@ -7038,10 +7038,14 @@ func TestSafeDeleteLoadBalancer(t *testing.T) {
 	cloud := GetTestCloud(ctrl)
 
 	testCases := []struct {
-		desc                string
-		expectedDeleteCall  bool
-		expectedDecoupleErr error
-		expectedErr         *retry.Error
+		desc                          string
+		expectedDeleteCall            bool
+		expectedDecoupleErr           error
+		multiSLBConfigs               []MultipleStandardLoadBalancerConfiguration
+		nodesWithCorrectVMSet         sets.Set[string]
+		expectedMultiSLBConfigs       []MultipleStandardLoadBalancerConfiguration
+		expectedNodesWithCorrectVMSet sets.Set[string]
+		expectedErr                   *retry.Error
 	}{
 		{
 			desc:               "Standard SKU: should delete the load balancer",
@@ -7057,6 +7061,40 @@ func TestSafeDeleteLoadBalancer(t *testing.T) {
 				fmt.Errorf("safeDeleteLoadBalancer: failed to EnsureBackendPoolDeleted: %w", errors.New("error")),
 			),
 		},
+		{
+			desc:               "should cleanup active nodes when using multi-slb",
+			expectedDeleteCall: true,
+			multiSLBConfigs: []MultipleStandardLoadBalancerConfiguration{
+				{
+					Name: "test",
+					MultipleStandardLoadBalancerConfigurationStatus: MultipleStandardLoadBalancerConfigurationStatus{
+						ActiveNodes: sets.New[string]("node1", "node2"),
+					},
+				},
+				{
+					Name: "test2",
+					MultipleStandardLoadBalancerConfigurationStatus: MultipleStandardLoadBalancerConfigurationStatus{
+						ActiveNodes: sets.New[string]("node3"),
+					},
+				},
+			},
+			nodesWithCorrectVMSet: sets.New[string]("node1", "node2", "node3"),
+			expectedMultiSLBConfigs: []MultipleStandardLoadBalancerConfiguration{
+				{
+					Name: "test",
+					MultipleStandardLoadBalancerConfigurationStatus: MultipleStandardLoadBalancerConfigurationStatus{
+						ActiveNodes: sets.New[string](),
+					},
+				},
+				{
+					Name: "test2",
+					MultipleStandardLoadBalancerConfigurationStatus: MultipleStandardLoadBalancerConfigurationStatus{
+						ActiveNodes: sets.New[string]("node3"),
+					},
+				},
+			},
+			expectedNodesWithCorrectVMSet: sets.New[string]("node3"),
+		},
 	}
 
 	for _, tc := range testCases {
@@ -7075,6 +7113,12 @@ func TestSafeDeleteLoadBalancer(t *testing.T) {
 			).Return(false, tc.expectedDecoupleErr)
 			cloud.VMSet = mockVMSet
 			cloud.LoadBalancerClient = mockLBClient
+			if len(tc.multiSLBConfigs) > 0 {
+				cloud.MultipleStandardLoadBalancerConfigurations = tc.multiSLBConfigs
+				for nodeName := range tc.nodesWithCorrectVMSet {
+					cloud.nodesWithCorrectLoadBalancerByPrimaryVMSet.Store(nodeName, sets.Empty{})
+				}
+			}
 			svc := getTestService("svc", v1.ProtocolTCP, nil, false, 80)
 			lb := network.LoadBalancer{
 				Name: pointer.String("test"),
@@ -7084,6 +7128,15 @@ func TestSafeDeleteLoadBalancer(t *testing.T) {
 			}
 			err := cloud.safeDeleteLoadBalancer(lb, "cluster", "vmss", &svc)
 			assert.Equal(t, tc.expectedErr, err)
+			if len(tc.multiSLBConfigs) > 0 {
+				assert.Equal(t, tc.expectedMultiSLBConfigs, cloud.MultipleStandardLoadBalancerConfigurations)
+				actualNodesWithCorrectLoadBalancerByPrimaryVMSet := sets.New[string]()
+				cloud.nodesWithCorrectLoadBalancerByPrimaryVMSet.Range(func(key, value interface{}) bool {
+					actualNodesWithCorrectLoadBalancerByPrimaryVMSet.Insert(key.(string))
+					return true
+				})
+				assert.Equal(t, tc.expectedNodesWithCorrectVMSet, actualNodesWithCorrectLoadBalancerByPrimaryVMSet)
+			}
 		})
 	}
 }
@@ -7151,6 +7204,21 @@ func TestGetEligibleLoadBalancers(t *testing.T) {
 			},
 			expectedLBs: []string{"a"},
 		},
+		{
+			description: "should report an error if a service selects a load balancer that is not defined in the configuration",
+			svc:         getTestService("test", v1.ProtocolTCP, map[string]string{consts.ServiceAnnotationLoadBalancerConfigurations: "b"}, false),
+			lbConfigs: []MultipleStandardLoadBalancerConfiguration{
+				{
+					Name: "a",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{},
+				},
+				{
+					Name: "c",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{},
+				},
+			},
+			expectedErr: errors.New(`service "test" selects 1 load balancers by annotation, but none of them is defined in cloud provider configuration`),
+		},
 		{
 			description: "should respect namespace selector",
 			svc:         getTestService("test", v1.ProtocolTCP, nil, false),
@@ -7397,7 +7465,7 @@ func TestGetEligibleLoadBalancers(t *testing.T) {
 				tc.svc.Labels = tc.labels
 			}
 
-			lbs, err := az.getEligibleLoadBalancers(&tc.svc)
+			lbs, err := az.getEligibleLoadBalancersForService(&tc.svc)
 			assert.Equal(t, tc.expectedLBs, lbs)
 			if tc.expectedErr != nil {
 				assert.Equal(t, tc.expectedErr.Error(), err.Error())
@@ -7642,7 +7710,7 @@ func TestGetMostEligibleLBName(t *testing.T) {
 		},
 	} {
 		t.Run(tc.description, func(t *testing.T) {
-			lbName := getMostEligibleLBName(tc.currentLBName, tc.eligibleLBs, tc.existingLBs)
+			lbName := getMostEligibleLBForService(tc.currentLBName, tc.eligibleLBs, tc.existingLBs)
 			assert.Equal(t, tc.expectedLBName, lbName)
 		})
 	}
@@ -7656,6 +7724,7 @@ func TestReconcileMultipleStandardLoadBalancerConfigurations(t *testing.T) {
 		description            string
 		useMultipleLB          bool
 		noPrimaryConfig        bool
+		nodes                  []*v1.Node
 		expectedActiveServices map[string]sets.Set[string]
 		expectedErr            error
 	}{
@@ -7663,8 +7732,8 @@ func TestReconcileMultipleStandardLoadBalancerConfigurations(t *testing.T) {
 			description:   "should set active services correctly",
 			useMultipleLB: true,
 			expectedActiveServices: map[string]sets.Set[string]{
-				"kubernetes": sets.New[string]("default/LBSvcOnKubernetes"),
-				"lb1":        sets.New[string]("ns1/LBSvcOnLB1"),
+				"kubernetes": sets.New[string]("default/lbsvconkubernetes"),
+				"lb1":        sets.New[string]("ns1/lbsvconlb1"),
 			},
 		},
 		{
@@ -7757,7 +7826,8 @@ func TestReconcileMultipleStandardLoadBalancerConfigurations(t *testing.T) {
 				}
 			}
 
-			err := az.reconcileMultipleStandardLoadBalancerConfigurations("kubernetes", &existingLBs)
+			svc := getTestService("test", v1.ProtocolTCP, nil, false)
+			err := az.reconcileMultipleStandardLoadBalancerConfigurations(&existingLBs, &svc, "kubernetes", &existingLBs, tc.nodes)
 			assert.Equal(t, err, tc.expectedErr)
 
 			activeServices := make(map[string]sets.Set[string])
@@ -8218,3 +8288,319 @@ func TestServiceOwnsFrontendIP(t *testing.T) {
 		})
 	}
 }
+
+func TestReconcileMultipleStandardLoadBalancerNodes(t *testing.T) {
+	ctrl := gomock.NewController(t)
+	defer ctrl.Finish()
+
+	for _, tc := range []struct {
+		description          string
+		lbName               string
+		existingLBConfigs    []MultipleStandardLoadBalancerConfiguration
+		existingNodes        []*v1.Node
+		existingLBs          []network.LoadBalancer
+		expectedPutLBTimes   int
+		expectedLBToNodesMap map[string]sets.Set[string]
+	}{
+		{
+			description: "should remove unwanted nodes and arrange existing nodes with primary vmSet as expected",
+			existingLBConfigs: []MultipleStandardLoadBalancerConfiguration{
+				{
+					Name: "lb1",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+						PrimaryVMSet: "vmss-1",
+					},
+					MultipleStandardLoadBalancerConfigurationStatus: MultipleStandardLoadBalancerConfigurationStatus{
+						ActiveNodes: sets.Set[string]{"node1": sets.Empty{}, "node2": sets.Empty{}},
+					},
+				},
+				{
+					Name: "lb2",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+						PrimaryVMSet: "vmss-2",
+					},
+					MultipleStandardLoadBalancerConfigurationStatus: MultipleStandardLoadBalancerConfigurationStatus{
+						ActiveNodes: sets.Set[string]{"node3": sets.Empty{}, "node4": sets.Empty{}},
+					},
+				},
+			},
+			existingNodes: []*v1.Node{
+				getTestNodeWithMetadata("node1", "vmss-1", nil, "10.1.0.1"),
+				getTestNodeWithMetadata("node2", "vmss-2", nil, "10.1.0.2"),
+				getTestNodeWithMetadata("node3", "vmss-2", nil, "10.1.0.3"),
+			},
+			existingLBs: []network.LoadBalancer{
+				{
+					Name: pointer.String("lb1-internal"),
+					LoadBalancerPropertiesFormat: &network.LoadBalancerPropertiesFormat{
+						BackendAddressPools: &[]network.BackendAddressPool{
+							{
+								Name: pointer.String("kubernetes"),
+								BackendAddressPoolPropertiesFormat: &network.BackendAddressPoolPropertiesFormat{
+									LoadBalancerBackendAddresses: &[]network.LoadBalancerBackendAddress{
+										{
+											Name: pointer.String("node1"),
+											LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+												IPAddress: pointer.String("10.1.0.1"),
+											},
+										},
+										{
+											Name: pointer.String("node2"),
+											LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+												IPAddress: pointer.String("10.1.0.2"),
+											},
+										},
+									},
+								},
+							},
+						},
+					},
+				},
+				{
+					Name: pointer.String("lb2"),
+					LoadBalancerPropertiesFormat: &network.LoadBalancerPropertiesFormat{
+						BackendAddressPools: &[]network.BackendAddressPool{
+							{
+								Name: pointer.String("kubernetes"),
+								BackendAddressPoolPropertiesFormat: &network.BackendAddressPoolPropertiesFormat{
+									LoadBalancerBackendAddresses: &[]network.LoadBalancerBackendAddress{
+										{
+											Name: pointer.String("node3"),
+											LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+												IPAddress: pointer.String("10.1.0.3"),
+											},
+										},
+										{
+											Name: pointer.String("node4"),
+											LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+												IPAddress: pointer.String("10.1.0.4"),
+											},
+										},
+									},
+								},
+							},
+						},
+					},
+				},
+			},
+			expectedPutLBTimes: 1,
+			expectedLBToNodesMap: map[string]sets.Set[string]{
+				"lb1": {"node1": sets.Empty{}},
+				"lb2": {"node2": sets.Empty{}, "node3": sets.Empty{}},
+			},
+		},
+		{
+			description: "should respect node selector",
+			lbName:      "lb2-internal",
+			existingLBConfigs: []MultipleStandardLoadBalancerConfiguration{
+				{
+					Name: "lb1",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+						PrimaryVMSet: "vmss-1",
+						NodeSelector: &metav1.LabelSelector{
+							MatchLabels: map[string]string{"k1": "v1"},
+						},
+					},
+					MultipleStandardLoadBalancerConfigurationStatus: MultipleStandardLoadBalancerConfigurationStatus{
+						ActiveNodes: sets.Set[string]{"node1": sets.Empty{}, "node2": sets.Empty{}},
+					},
+				},
+				{
+					Name: "lb2",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+						PrimaryVMSet: "vmss-2",
+						NodeSelector: &metav1.LabelSelector{
+							MatchLabels: map[string]string{"k2": "v2"},
+						},
+					},
+					MultipleStandardLoadBalancerConfigurationStatus: MultipleStandardLoadBalancerConfigurationStatus{
+						ActiveNodes: sets.Set[string]{"node3": sets.Empty{}, "node4": sets.Empty{}},
+					},
+				},
+				{
+					Name: "lb3",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+						PrimaryVMSet: "vmss-2",
+						NodeSelector: &metav1.LabelSelector{
+							MatchLabels: map[string]string{"k2": "v2"},
+						},
+					},
+				},
+				{
+					Name: "lb4",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+						PrimaryVMSet: "vmss-2",
+					},
+				},
+			},
+			existingNodes: []*v1.Node{
+				getTestNodeWithMetadata("node1", "vmss-1", map[string]string{"k2": "v2"}, "10.1.0.1"),
+				getTestNodeWithMetadata("node2", "vmss-3", map[string]string{"k3": "v3"}, "10.1.0.2"),
+				getTestNodeWithMetadata("node3", "vmss-3", map[string]string{"k2": "v2"}, "10.1.0.3"),
+				getTestNodeWithMetadata("node5", "vmss-3", map[string]string{"k2": "v2"}, "10.1.0.5"),
+				getTestNodeWithMetadata("node6", "vmss-3", map[string]string{"k3": "v3"}, "10.1.0.6"),
+			},
+			existingLBs: []network.LoadBalancer{
+				{
+					Name: pointer.String("lb1-internal"),
+					LoadBalancerPropertiesFormat: &network.LoadBalancerPropertiesFormat{
+						BackendAddressPools: &[]network.BackendAddressPool{
+							{
+								Name: pointer.String("kubernetes"),
+								BackendAddressPoolPropertiesFormat: &network.BackendAddressPoolPropertiesFormat{
+									LoadBalancerBackendAddresses: &[]network.LoadBalancerBackendAddress{
+										{
+											Name: pointer.String("node1"),
+											LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+												IPAddress: pointer.String("10.1.0.1"),
+											},
+										},
+										{
+											Name: pointer.String("node2"),
+											LoadBalancerBackendAddressPropertiesFormat: &network.LoadBalancerBackendAddressPropertiesFormat{
+												IPAddress: pointer.String("10.1.0.2"),
+											},
+										},
+									},
+								},
+							},
+						},
+					},
+				},
+				{
+					Name: pointer.String("lb3"),
+					LoadBalancerPropertiesFormat: &network.LoadBalancerPropertiesFormat{
+						BackendAddressPools: &[]network.BackendAddressPool{
+							{
+								Name: pointer.String("kubernetes"),
+							},
+						},
+					},
+				},
+			},
+			expectedLBToNodesMap: map[string]sets.Set[string]{
+				"lb1": {"node1": sets.Empty{}},
+				"lb2": {"node3": sets.Empty{}},
+				"lb3": {"node5": sets.Empty{}},
+				"lb4": {"node2": sets.Empty{}, "node6": sets.Empty{}},
+			},
+		},
+		{
+			description: "should remove the node on the lb if it is no longer eligible",
+			existingLBConfigs: []MultipleStandardLoadBalancerConfiguration{
+				{
+					Name: "lb1",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+						PrimaryVMSet: "vmss-1",
+						NodeSelector: &metav1.LabelSelector{
+							MatchLabels: map[string]string{"k1": "v1"},
+						},
+					},
+					MultipleStandardLoadBalancerConfigurationStatus: MultipleStandardLoadBalancerConfigurationStatus{
+						ActiveNodes: sets.Set[string]{"node1": sets.Empty{}, "node2": sets.Empty{}},
+					},
+				},
+			},
+			existingNodes: []*v1.Node{
+				getTestNodeWithMetadata("node1", "vmss-2", map[string]string{"k2": "v2"}, "10.1.0.1"),
+			},
+			expectedLBToNodesMap: map[string]sets.Set[string]{
+				"lb1": {},
+			},
+		},
+		{
+			description: "should skip lbs that do not exist or will not be created",
+			existingLBConfigs: []MultipleStandardLoadBalancerConfiguration{
+				{
+					Name: "lb1",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+						PrimaryVMSet: "vmss-1",
+						NodeSelector: &metav1.LabelSelector{
+							MatchLabels: map[string]string{"k1": "v1"},
+						},
+					},
+				},
+				{
+					Name: "lb2",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+						PrimaryVMSet: "vmss-2",
+						NodeSelector: &metav1.LabelSelector{
+							MatchLabels: map[string]string{"k2": "v2"},
+						},
+					},
+				},
+				{
+					Name: "lb3",
+					MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+						PrimaryVMSet: "vmss-2",
+						NodeSelector: &metav1.LabelSelector{
+							MatchLabels: map[string]string{"k3": "v3"},
+						},
+					},
+				},
+				{
+					Name: "lb4",
+				},
+			},
+			existingLBs: []network.LoadBalancer{
+				{
+					Name: pointer.String("lb2-internal"),
+					LoadBalancerPropertiesFormat: &network.LoadBalancerPropertiesFormat{
+						BackendAddressPools: &[]network.BackendAddressPool{
+							{Name: pointer.String("kubernetes")},
+						},
+					},
+				},
+			},
+			existingNodes: []*v1.Node{
+				getTestNodeWithMetadata("node1", "vmss-1", map[string]string{"k1": "v1"}, "10.1.0.1"),
+				getTestNodeWithMetadata("node2", "vmss-3", map[string]string{"k3": "v3"}, "10.1.0.2"),
+				getTestNodeWithMetadata("node3", "vmss-3", map[string]string{"k2": "v2"}, "10.1.0.3"),
+				getTestNodeWithMetadata("node5", "vmss-3", map[string]string{"k2": "v2"}, "10.1.0.5"),
+				getTestNodeWithMetadata("node6", "vmss-3", map[string]string{"k3": "v3"}, "10.1.0.6"),
+			},
+			expectedLBToNodesMap: map[string]sets.Set[string]{
+				"lb1": nil,
+				"lb2": {"node3": sets.Empty{}, "node5": sets.Empty{}},
+				"lb3": nil,
+				"lb4": {"node1": sets.Empty{}, "node2": sets.Empty{}, "node6": sets.Empty{}},
+			},
+		},
+	} {
+		t.Run(tc.description, func(t *testing.T) {
+			az := GetTestCloud(ctrl)
+			ss, _ := NewTestScaleSet(ctrl)
+			ss.DisableAvailabilitySetNodes = true
+			az.VMSet = ss
+			az.MultipleStandardLoadBalancerConfigurations = tc.existingLBConfigs
+			svc := getTestService("test", v1.ProtocolTCP, nil, false)
+			_ = az.reconcileMultipleStandardLoadBalancerBackendNodes(tc.lbName, &tc.existingLBs, &svc, tc.existingNodes)
+
+			expectedLBToNodesMap := make(map[string]sets.Set[string])
+			for _, multiSLBConfig := range az.MultipleStandardLoadBalancerConfigurations {
+				expectedLBToNodesMap[multiSLBConfig.Name] = multiSLBConfig.ActiveNodes
+			}
+			assert.Equal(t, tc.expectedLBToNodesMap, expectedLBToNodesMap)
+		})
+	}
+}
+
+func getTestNodeWithMetadata(nodeName, vmssName string, labels map[string]string, ip string) *v1.Node {
+	return &v1.Node{
+		ObjectMeta: metav1.ObjectMeta{
+			Name:   nodeName,
+			Labels: labels,
+		},
+		Spec: v1.NodeSpec{
+			ProviderID: fmt.Sprintf("azure:///subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/%s/virtualMachines/0", vmssName),
+		},
+		Status: v1.NodeStatus{
+			Addresses: []v1.NodeAddress{
+				{
+					Type:    v1.NodeInternalIP,
+					Address: ip,
+				},
+			},
+		},
+	}
+}
diff --git a/pkg/provider/azure_test.go b/pkg/provider/azure_test.go
index 211680d33..f23414e4e 100644
--- a/pkg/provider/azure_test.go
+++ b/pkg/provider/azure_test.go
@@ -3896,3 +3896,73 @@ func TestSetLBDefaults(t *testing.T) {
 	_ = az.setLBDefaults(config)
 	assert.Equal(t, config.LoadBalancerSku, consts.LoadBalancerSkuStandard)
 }
+
+func TestCheckEnableMultipleStandardLoadBalancers(t *testing.T) {
+	ctrl := gomock.NewController(t)
+	defer ctrl.Finish()
+	az := GetTestCloud(ctrl)
+	az.LoadBalancerBackendPoolConfigurationType = consts.LoadBalancerBackendPoolConfigurationTypeNodeIP
+
+	az.MultipleStandardLoadBalancerConfigurations = []MultipleStandardLoadBalancerConfiguration{
+		{
+			Name: "kubernetes",
+			MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+				PrimaryVMSet: "vmss-0",
+			},
+		},
+		{
+			Name: "lb1",
+			MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+				PrimaryVMSet: "vmss-1",
+			},
+		},
+		{
+			Name: "kubernetes",
+			MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+				PrimaryVMSet: "vmss-2",
+			},
+		},
+	}
+
+	err := az.checkEnableMultipleStandardLoadBalancers()
+	assert.Equal(t, "duplicated multiple standard load balancer configuration name kubernetes", err.Error())
+
+	az.MultipleStandardLoadBalancerConfigurations = []MultipleStandardLoadBalancerConfiguration{
+		{
+			Name: "kubernetes",
+			MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+				PrimaryVMSet: "vmss-0",
+			},
+		},
+		{
+			Name: "lb1",
+		},
+	}
+
+	err = az.checkEnableMultipleStandardLoadBalancers()
+	assert.Equal(t, "multiple standard load balancer configuration lb1 must have primary VMSet", err.Error())
+
+	az.MultipleStandardLoadBalancerConfigurations = []MultipleStandardLoadBalancerConfiguration{
+		{
+			Name: "kubernetes",
+			MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+				PrimaryVMSet: "vmss-0",
+			},
+		},
+		{
+			Name: "lb1",
+			MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+				PrimaryVMSet: "vmss-2",
+			},
+		},
+		{
+			Name: "lb2",
+			MultipleStandardLoadBalancerConfigurationSpec: MultipleStandardLoadBalancerConfigurationSpec{
+				PrimaryVMSet: "vmss-2",
+			},
+		},
+	}
+
+	err = az.checkEnableMultipleStandardLoadBalancers()
+	assert.Equal(t, "duplicated primary VMSet vmss-2 in multiple standard load balancer configurations lb2", err.Error())
+}