Repository: kubernetes-sigs/cloud-provider-azure
Pull request #7566 (2024-11-13 03:03:18)

[release-1.30] fix: Support switching from loadbalancer to externalName for services
This is an automated cherry-pick of #7518

/assign nilo19

```release-note
fix: Support switching from loadbalancer to externalName for services
```

k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/cloud-provider-azure/pull/7566#" title="Author self-approved">k8s-infra-cherrypick-robot</a>*
**Once this PR has been reviewed and has the lgtm label**, please assign [lzhecheng](https://github.com/lzhecheng) for approval. For more information see [the Kubernetes Code Review Process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process).

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fcloud-provider-azure).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes-sigs/cloud-provider-azure/blob/release-1.30/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["lzhecheng"]} -->
MartinForReal: /lgtm
/approve
k8s-ci-robot: [APPROVALNOTIFIER] This PR is **APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/cloud-provider-azure/pull/7566#" title="Author self-approved">k8s-infra-cherrypick-robot</a>*, *<a href="https://github.com/kubernetes-sigs/cloud-provider-azure/pull/7566#issuecomment-2472279497" title="Approved">MartinForReal</a>*

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fcloud-provider-azure).

The pull request process is described [here](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process)

<details >
Needs approval from an approver in each of these files:

- ~~[OWNERS](https://github.com/kubernetes-sigs/cloud-provider-azure/blob/release-1.30/OWNERS)~~ [MartinForReal]

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":[]} -->
MartinForReal: /kind bug
k8s-ci-robot: @k8s-infra-cherrypick-robot: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30 | b391afeee71c857568d40ea75853ad962ea2f41a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/7566/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30/1856533279017013248) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=7566). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Ak8s-infra-cherrypick-robot). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @k8s-infra-cherrypick-robot: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30 | b391afeee71c857568d40ea75853ad962ea2f41a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/7566/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30/1856533279017013248) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30`
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz-1-30 | b391afeee71c857568d40ea75853ad962ea2f41a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/7566/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz-1-30/1856533279075733504) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz-1-30`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=7566). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Ak8s-infra-cherrypick-robot). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @k8s-infra-cherrypick-robot: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30 | b391afeee71c857568d40ea75853ad962ea2f41a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/7566/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30/1856533279017013248) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30`
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz-1-30 | b391afeee71c857568d40ea75853ad962ea2f41a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/7566/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz-1-30/1856533279075733504) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz-1-30`
pull-cloud-provider-azure-e2e-ccm-capz-1-30 | b391afeee71c857568d40ea75853ad962ea2f41a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/7566/pull-cloud-provider-azure-e2e-ccm-capz-1-30/1856533278945710080) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz-1-30`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=7566). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Ak8s-infra-cherrypick-robot). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @k8s-infra-cherrypick-robot: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30 | b391afeee71c857568d40ea75853ad962ea2f41a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/7566/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30/1856533279017013248) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz-1-30`
pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz-1-30 | b391afeee71c857568d40ea75853ad962ea2f41a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/7566/pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz-1-30/1856533279075733504) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-multi-slb-capz-1-30`
pull-cloud-provider-azure-e2e-ccm-capz-1-30 | b391afeee71c857568d40ea75853ad962ea2f41a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/7566/pull-cloud-provider-azure-e2e-ccm-capz-1-30/1856533278945710080) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz-1-30`
pull-cloud-provider-azure-e2e-ccm-vmss-capz-1-30 | b391afeee71c857568d40ea75853ad962ea2f41a | [link](https://prow.k8s.io/view/gs/kubernetes-ci-logs/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/7566/pull-cloud-provider-azure-e2e-ccm-vmss-capz-1-30/1856533278819880960) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz-1-30`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=7566). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Ak8s-infra-cherrypick-robot). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes-sigs/prow](https://github.com/kubernetes-sigs/prow/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->

Patch:
diff --git a/.github/workflows/codeql-analysis.yml b/.github/workflows/codeql-analysis.yml
index 8a45909db..4c1f87d7d 100644
--- a/.github/workflows/codeql-analysis.yml
+++ b/.github/workflows/codeql-analysis.yml
@@ -70,7 +70,7 @@ jobs:
 
     # Initializes the CodeQL tools for scanning.
     - name: Initialize CodeQL
-      uses: github/codeql-action/init@9278e421667d5d90a2839487a482448c4ec7df4d # v3.27.2
+      uses: github/codeql-action/init@396bb3e45325a47dd9ef434068033c6d5bb0d11a # v3.27.3
       with:
         languages: ${{ matrix.language }}
         # If you wish to specify custom queries, you can do so here or in a config file.
@@ -93,4 +93,4 @@ jobs:
         $(go env GOPATH)/bin/goveralls -coverprofile=profile.cov -service=github
     
     - name: Perform CodeQL Analysis
-      uses: github/codeql-action/analyze@9278e421667d5d90a2839487a482448c4ec7df4d # v3.27.2
+      uses: github/codeql-action/analyze@396bb3e45325a47dd9ef434068033c6d5bb0d11a # v3.27.3
diff --git a/.github/workflows/scorecards.yml b/.github/workflows/scorecards.yml
index ac0168cb8..2e3a1066f 100644
--- a/.github/workflows/scorecards.yml
+++ b/.github/workflows/scorecards.yml
@@ -72,6 +72,6 @@ jobs:
 
       # Upload the results to GitHub's code scanning dashboard.
       - name: "Upload to code-scanning"
-        uses: github/codeql-action/upload-sarif@9278e421667d5d90a2839487a482448c4ec7df4d # v3.27.2
+        uses: github/codeql-action/upload-sarif@396bb3e45325a47dd9ef434068033c6d5bb0d11a # v3.27.3
         with:
           sarif_file: results.sarif
diff --git a/.github/workflows/trivy.yaml b/.github/workflows/trivy.yaml
index 833c24acb..817a33a43 100644
--- a/.github/workflows/trivy.yaml
+++ b/.github/workflows/trivy.yaml
@@ -67,7 +67,7 @@ jobs:
           vuln-type: 'os,library'
           severity: 'CRITICAL,HIGH,MEDIUM,LOW,UNKNOWN'
       - name: Upload Trivy scan results to GitHub Security tab
-        uses: github/codeql-action/upload-sarif@9278e421667d5d90a2839487a482448c4ec7df4d # v3.27.2
+        uses: github/codeql-action/upload-sarif@396bb3e45325a47dd9ef434068033c6d5bb0d11a # v3.27.3
         with:
           sarif_file: 'trivy-ccm-results.sarif'
           category: azure-cloud-controller-manager-image
@@ -81,7 +81,7 @@ jobs:
           vuln-type: 'os,library'
           severity: 'CRITICAL,HIGH,MEDIUM,LOW,UNKNOWN'
       - name: Upload Trivy scan results to GitHub Security tab
-        uses: github/codeql-action/upload-sarif@9278e421667d5d90a2839487a482448c4ec7df4d # v3.27.2
+        uses: github/codeql-action/upload-sarif@396bb3e45325a47dd9ef434068033c6d5bb0d11a # v3.27.3
         with:
           sarif_file: 'trivy-cnm-linux-results.sarif'
           category: azure-cloud-node-manager-linux-image
@@ -95,7 +95,7 @@ jobs:
           vuln-type: 'os,library'
           severity: 'CRITICAL,HIGH,MEDIUM,LOW,UNKNOWN'
       - name: Upload Trivy scan results to GitHub Security tab
-        uses: github/codeql-action/upload-sarif@9278e421667d5d90a2839487a482448c4ec7df4d # v3.27.2
+        uses: github/codeql-action/upload-sarif@396bb3e45325a47dd9ef434068033c6d5bb0d11a # v3.27.3
         with:
           sarif_file: 'trivy-health-probe-proxy-linux-results.sarif'
           category: health-probe-proxy-linux-image
diff --git a/Dockerfile b/Dockerfile
index 9c34c4835..9e73ee082 100644
--- a/Dockerfile
+++ b/Dockerfile
@@ -14,7 +14,7 @@
 
 # syntax=docker/dockerfile:1
 
-FROM --platform=linux/amd64 golang:1.23-bullseye@sha256:1002103155eb5802f6876dceb8cb8bf965cc6d3deeb34f11e4774926aca6a1fd AS builder
+FROM --platform=linux/amd64 golang:1.23-bullseye@sha256:d4387d8b2ee5b78a199fae168a761d0fb0f33f172642d20856f29a67204af16b AS builder
 
 ARG ENABLE_GIT_COMMAND=true
 ARG ARCH=amd64
diff --git a/cloud-node-manager.Dockerfile b/cloud-node-manager.Dockerfile
index be185d9f5..c4fc398b7 100644
--- a/cloud-node-manager.Dockerfile
+++ b/cloud-node-manager.Dockerfile
@@ -14,7 +14,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM --platform=linux/amd64 golang:1.23-bullseye@sha256:1002103155eb5802f6876dceb8cb8bf965cc6d3deeb34f11e4774926aca6a1fd AS builder
+FROM --platform=linux/amd64 golang:1.23-bullseye@sha256:d4387d8b2ee5b78a199fae168a761d0fb0f33f172642d20856f29a67204af16b AS builder
 
 ARG ENABLE_GIT_COMMAND=true
 ARG ARCH=amd64
diff --git a/pkg/provider/azure.go b/pkg/provider/azure.go
index 7cb575398..21249eeea 100644
--- a/pkg/provider/azure.go
+++ b/pkg/provider/azure.go
@@ -71,6 +71,7 @@ import (
 	"sigs.k8s.io/cloud-provider-azure/pkg/azureclients/vmssclient"
 	"sigs.k8s.io/cloud-provider-azure/pkg/azureclients/vmssvmclient"
 	"sigs.k8s.io/cloud-provider-azure/pkg/provider/privatelinkservice"
+	"sigs.k8s.io/cloud-provider-azure/pkg/provider/subnet"
 	"sigs.k8s.io/cloud-provider-azure/pkg/provider/zone"
 
 	"sigs.k8s.io/yaml"
@@ -418,12 +419,13 @@ type Cloud struct {
 	routeUpdater       batchProcessor
 	backendPoolUpdater batchProcessor
 
-	vmCache  azcache.Resource
-	lbCache  azcache.Resource
-	nsgRepo  securitygroup.Repository
-	zoneRepo zone.Repository
-	plsRepo  privatelinkservice.Repository
-	rtCache  azcache.Resource
+	vmCache    azcache.Resource
+	lbCache    azcache.Resource
+	nsgRepo    securitygroup.Repository
+	zoneRepo   zone.Repository
+	plsRepo    privatelinkservice.Repository
+	subnetRepo subnet.Repository
+	rtCache    azcache.Resource
 	// public ip cache
 	// key: [resourceGroupName]
 	// Value: sync.Map of [pipName]*PublicIPAddress
@@ -754,6 +756,10 @@ func (az *Cloud) InitializeCloudFromConfig(ctx context.Context, config *Config,
 		if err != nil {
 			return err
 		}
+		az.subnetRepo, err = subnet.NewRepo(networkClientFactory.GetSubnetClient())
+		if err != nil {
+			return err
+		}
 	}
 	err = az.initCaches()
 	if err != nil {
diff --git a/pkg/provider/azure_fakes.go b/pkg/provider/azure_fakes.go
index 07249d8d3..030128a85 100644
--- a/pkg/provider/azure_fakes.go
+++ b/pkg/provider/azure_fakes.go
@@ -30,6 +30,7 @@ import (
 	"sigs.k8s.io/cloud-provider-azure/pkg/azclient/mock_azclient"
 	"sigs.k8s.io/cloud-provider-azure/pkg/azclient/privatezoneclient/mock_privatezoneclient"
 	"sigs.k8s.io/cloud-provider-azure/pkg/azclient/securitygroupclient/mock_securitygroupclient"
+	"sigs.k8s.io/cloud-provider-azure/pkg/azclient/subnetclient/mock_subnetclient"
 	"sigs.k8s.io/cloud-provider-azure/pkg/azclient/virtualnetworklinkclient/mock_virtualnetworklinkclient"
 	"sigs.k8s.io/cloud-provider-azure/pkg/azureclients/diskclient/mockdiskclient"
 	"sigs.k8s.io/cloud-provider-azure/pkg/azureclients/interfaceclient/mockinterfaceclient"
@@ -45,6 +46,7 @@ import (
 	"sigs.k8s.io/cloud-provider-azure/pkg/provider/config"
 	"sigs.k8s.io/cloud-provider-azure/pkg/provider/privatelinkservice"
 	"sigs.k8s.io/cloud-provider-azure/pkg/provider/securitygroup"
+	"sigs.k8s.io/cloud-provider-azure/pkg/provider/subnet"
 	utilsets "sigs.k8s.io/cloud-provider-azure/pkg/util/sets"
 )
 
@@ -128,6 +130,8 @@ func GetTestCloud(ctrl *gomock.Controller) (az *Cloud) {
 	clientFactory.EXPECT().GetPrivateZoneClient().Return(mockPrivateDNSClient).AnyTimes()
 	virtualNetworkLinkClient := mock_virtualnetworklinkclient.NewMockInterface(ctrl)
 	clientFactory.EXPECT().GetVirtualNetworkLinkClient().Return(virtualNetworkLinkClient).AnyTimes()
+	subnetTrack2Client := mock_subnetclient.NewMockInterface(ctrl)
+	clientFactory.EXPECT().GetSubnetClient().Return(subnetTrack2Client).AnyTimes()
 	az.AuthProvider = &azclient.AuthProvider{
 		ComputeCredential: mock_azclient.NewMockTokenCredential(ctrl),
 	}
@@ -135,6 +139,7 @@ func GetTestCloud(ctrl *gomock.Controller) (az *Cloud) {
 	az.vmCache, _ = az.newVMCache()
 	az.lbCache, _ = az.newLBCache()
 	az.nsgRepo, _ = securitygroup.NewSecurityGroupRepo(az.SecurityGroupResourceGroup, az.SecurityGroupName, az.NsgCacheTTLInSeconds, az.Config.DisableAPICallCache, securtyGrouptrack2Client)
+	az.subnetRepo = subnet.NewMockRepository(ctrl)
 	az.rtCache, _ = az.newRouteTableCache()
 	az.pipCache, _ = az.newPIPCache()
 	az.LoadBalancerBackendPool = NewMockBackendPool(ctrl)
diff --git a/pkg/provider/azure_loadbalancer.go b/pkg/provider/azure_loadbalancer.go
index d93bcdd93..a4389d78b 100644
--- a/pkg/provider/azure_loadbalancer.go
+++ b/pkg/provider/azure_loadbalancer.go
@@ -712,15 +712,12 @@ func (az *Cloud) cleanOrphanedLoadBalancer(ctx context.Context, lb *network.Load
 }
 
 // safeDeleteLoadBalancer deletes the load balancer after decoupling it from the vmSet
-func (az *Cloud) safeDeleteLoadBalancer(ctx context.Context, lb network.LoadBalancer, clusterName, vmSetName string, service *v1.Service) *retry.Error {
-	lbBackendPoolIDs := az.getBackendPoolIDs(clusterName, ptr.Deref(lb.Name, ""))
+func (az *Cloud) safeDeleteLoadBalancer(ctx context.Context, lb network.LoadBalancer, _, vmSetName string, service *v1.Service) *retry.Error {
 	lbBackendPoolIDsToDelete := []string{}
-	v4Enabled, v6Enabled := getIPFamiliesEnabled(service)
-	if v4Enabled {
-		lbBackendPoolIDsToDelete = append(lbBackendPoolIDsToDelete, lbBackendPoolIDs[consts.IPVersionIPv4])
-	}
-	if v6Enabled {
-		lbBackendPoolIDsToDelete = append(lbBackendPoolIDsToDelete, lbBackendPoolIDs[consts.IPVersionIPv6])
+	if lb.LoadBalancerPropertiesFormat != nil && lb.BackendAddressPools != nil {
+		for _, bp := range *lb.BackendAddressPools {
+			lbBackendPoolIDsToDelete = append(lbBackendPoolIDsToDelete, ptr.Deref(bp.ID, ""))
+		}
 	}
 	if _, err := az.VMSet.EnsureBackendPoolDeleted(ctx, service, lbBackendPoolIDsToDelete, vmSetName, lb.BackendAddressPools, true); err != nil {
 		return retry.NewError(false, fmt.Errorf("safeDeleteLoadBalancer: failed to EnsureBackendPoolDeleted: %w", err))
@@ -1508,7 +1505,7 @@ func (az *Cloud) isFrontendIPChanged(
 	if fipIPVersion != "" {
 		isIPv6 = fipIPVersion == network.IPv6
 	} else {
-		if isIPv6, err = az.isFIPIPv6(service, pipRG, &config); err != nil {
+		if isIPv6, err = az.isFIPIPv6(service, &config); err != nil {
 			return false, err
 		}
 	}
@@ -1658,7 +1655,6 @@ func (az *Cloud) findFrontendIPConfigsOfService(
 	service *v1.Service,
 ) (map[bool]*network.FrontendIPConfiguration, error) {
 	fipsOfServiceMap := map[bool]*network.FrontendIPConfiguration{}
-	pipRG := az.getPublicIPAddressResourceGroup(service)
 	for _, config := range *fipConfigs {
 		config := config
 		owns, _, fipIPVersion := az.serviceOwnsFrontendIP(ctx, config, service)
@@ -1668,7 +1664,7 @@ func (az *Cloud) findFrontendIPConfigsOfService(
 			if fipIPVersion != "" {
 				fipIsIPv6 = fipIPVersion == network.IPv6
 			} else {
-				if fipIsIPv6, err = az.isFIPIPv6(service, pipRG, &config); err != nil {
+				if fipIsIPv6, err = az.isFIPIPv6(service, &config); err != nil {
 					return nil, err
 				}
 			}
@@ -1844,7 +1840,6 @@ func (az *Cloud) reconcileLoadBalancer(ctx context.Context, clusterName string,
 	}
 
 	// update probes/rules
-	pipRG := az.getPublicIPAddressResourceGroup(service)
 	for _, ownedFIPConfig := range ownedFIPConfigs {
 		if ownedFIPConfig == nil {
 			continue
@@ -1859,7 +1854,7 @@ func (az *Cloud) reconcileLoadBalancer(ctx context.Context, clusterName string,
 		if fipIPVersion != "" {
 			isIPv6 = fipIPVersion == network.IPv6
 		} else {
-			if isIPv6, err = az.isFIPIPv6(service, pipRG, ownedFIPConfig); err != nil {
+			if isIPv6, err = az.isFIPIPv6(service, ownedFIPConfig); err != nil {
 				return nil, err
 			}
 		}
@@ -2520,8 +2515,6 @@ func (az *Cloud) reconcileFrontendIPConfigs(
 			}
 		}
 
-		pipRG := az.getPublicIPAddressResourceGroup(service)
-
 		for i := len(newConfigs) - 1; i >= 0; i-- {
 			config := newConfigs[i]
 			isServiceOwnsFrontendIP, _, fipIPVersion := az.serviceOwnsFrontendIP(ctx, config, service)
@@ -2535,7 +2528,7 @@ func (az *Cloud) reconcileFrontendIPConfigs(
 			if fipIPVersion != "" {
 				isIPv6 = fipIPVersion == network.IPv6
 			} else {
-				if isIPv6, err = az.isFIPIPv6(service, pipRG, &config); err != nil {
+				if isIPv6, err = az.isFIPIPv6(service, &config); err != nil {
 					return nil, toDeleteConfigs, false, err
 				}
 			}
diff --git a/pkg/provider/azure_privatelinkservice.go b/pkg/provider/azure_privatelinkservice.go
index 0ec458832..48852fe5d 100644
--- a/pkg/provider/azure_privatelinkservice.go
+++ b/pkg/provider/azure_privatelinkservice.go
@@ -18,10 +18,13 @@ package provider
 
 import (
 	"context"
+	"errors"
 	"fmt"
 	"net"
+	"net/http"
 	"strings"
 
+	"github.com/Azure/azure-sdk-for-go/sdk/azcore"
 	"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
 	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v6"
 	"github.com/Azure/azure-sdk-for-go/services/network/mgmt/2022-07-01/network"
@@ -46,7 +49,6 @@ func (az *Cloud) reconcilePrivateLinkService(
 	wantPLS bool,
 ) error {
 	isinternal := requiresInternalLoadBalancer(service)
-	pipRG := az.getPublicIPAddressResourceGroup(service)
 	_, _, fipIPVersion := az.serviceOwnsFrontendIP(ctx, *fipConfig, service)
 	serviceName := getServiceName(service)
 	var isIPv6 bool
@@ -54,7 +56,7 @@ func (az *Cloud) reconcilePrivateLinkService(
 	if fipIPVersion != "" {
 		isIPv6 = fipIPVersion == network.IPv6
 	} else {
-		if isIPv6, err = az.isFIPIPv6(service, pipRG, fipConfig); err != nil {
+		if isIPv6, err = az.isFIPIPv6(service, fipConfig); err != nil {
 			klog.Errorf("reconcilePrivateLinkService for service(%s): failed to get FIP IP family: %v", serviceName, err)
 			return err
 		}
@@ -144,14 +146,14 @@ func (az *Cloud) reconcilePrivateLinkService(
 			return err
 		}
 
-		dirtyPLS, err := az.getExpectedPrivateLinkService(existingPLS, &plsName, &clusterName, service, fipConfig)
+		dirtyPLS, err := az.getExpectedPrivateLinkService(ctx, existingPLS, &plsName, &clusterName, service, fipConfig)
 		if err != nil {
 			return err
 		}
 
 		if dirtyPLS {
 			klog.V(2).Infof("reconcilePrivateLinkService for service(%s): pls(%s) - updating", serviceName, plsName)
-			err := az.disablePLSNetworkPolicy(service)
+			err := az.disablePLSNetworkPolicy(ctx, service)
 			if err != nil {
 				klog.Errorf("reconcilePrivateLinkService for service(%s) disable PLS network policy failed for pls(%s): %v", serviceName, plsName, err.Error())
 				return err
@@ -196,28 +198,39 @@ func (az *Cloud) getPLSResourceGroup(service *v1.Service) string {
 	return az.PrivateLinkServiceResourceGroup
 }
 
-func (az *Cloud) disablePLSNetworkPolicy(service *v1.Service) error {
+func (az *Cloud) disablePLSNetworkPolicy(ctx context.Context, service *v1.Service) error {
 	serviceName := getServiceName(service)
 	subnetName := getPLSSubnetName(service)
 	if subnetName == nil {
 		subnetName = &az.SubnetName
 	}
 
-	subnet, existsSubnet, err := az.getSubnet("", az.VnetName, *subnetName)
+	rg := az.VnetResourceGroup
+	if rg == "" {
+		rg = az.ResourceGroup
+	}
+
+	subnet, err := az.subnetRepo.Get(ctx, rg, az.VnetName, *subnetName)
 	if err != nil {
+		var respErr *azcore.ResponseError
+		if errors.As(err, &respErr) {
+			if respErr != nil && respErr.StatusCode == http.StatusNotFound {
+				return fmt.Errorf("disablePLSNetworkPolicy: failed to get private link service subnet(%s) for service(%s)", *subnetName, serviceName)
+			}
+		}
 		return err
 	}
-	if !existsSubnet {
-		return fmt.Errorf("disablePLSNetworkPolicy: failed to get private link service subnet(%s) for service(%s)", *subnetName, serviceName)
+	if subnet.Properties == nil {
+		subnet.Properties = &armnetwork.SubnetPropertiesFormat{}
 	}
 
 	// Policy already disabled
-	if subnet.PrivateLinkServiceNetworkPolicies == network.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled {
+	if subnet.Properties.PrivateLinkServiceNetworkPolicies != nil && *subnet.Properties.PrivateLinkServiceNetworkPolicies == armnetwork.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled {
 		return nil
 	}
 
-	subnet.PrivateLinkServiceNetworkPolicies = network.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled
-	err = az.CreateOrUpdateSubnet(service, subnet)
+	subnet.Properties.PrivateLinkServiceNetworkPolicies = to.Ptr(armnetwork.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled)
+	err = az.subnetRepo.CreateOrUpdate(ctx, rg, az.VnetName, *subnetName, *subnet)
 	if err != nil {
 		return err
 	}
@@ -281,6 +294,7 @@ func (az *Cloud) getPrivateLinkServiceName(
 
 // getExpectedPrivateLinkService builds expected PLS object from service spec
 func (az *Cloud) getExpectedPrivateLinkService(
+	ctx context.Context,
 	existingPLS *armnetwork.PrivateLinkService,
 	plsName *string,
 	clusterName *string,
@@ -314,7 +328,7 @@ func (az *Cloud) getExpectedPrivateLinkService(
 		dirtyPLS = true
 	}
 
-	changed, err := az.reconcilePLSIpConfigs(existingPLS, service)
+	changed, err := az.reconcilePLSIpConfigs(ctx, existingPLS, service)
 	if err != nil {
 		return false, err
 	}
@@ -347,6 +361,7 @@ func (az *Cloud) getExpectedPrivateLinkService(
 
 // reconcile Private link service's IP configurations
 func (az *Cloud) reconcilePLSIpConfigs(
+	ctx context.Context,
 	existingPLS *armnetwork.PrivateLinkService,
 	service *v1.Service,
 ) (bool, error) {
@@ -357,13 +372,20 @@ func (az *Cloud) reconcilePLSIpConfigs(
 	if subnetName == nil {
 		subnetName = &az.SubnetName
 	}
-	subnet, existsSubnet, err := az.getSubnet("", az.VnetName, *subnetName)
+	rg := az.VnetResourceGroup
+	if rg == "" {
+		rg = az.ResourceGroup
+	}
+	subnet, err := az.subnetRepo.Get(ctx, rg, az.VnetName, *subnetName)
 	if err != nil {
+		var runtimError *azcore.ResponseError
+		if errors.As(err, &runtimError) {
+			if runtimError != nil && runtimError.StatusCode == http.StatusNotFound {
+				return false, fmt.Errorf("checkAndUpdatePLSIPConfigs: failed to get private link service subnet(%s) for service(%s)", *subnetName, serviceName)
+			}
+		}
 		return false, err
 	}
-	if !existsSubnet {
-		return false, fmt.Errorf("checkAndUpdatePLSIPConfigs: failed to get private link service subnet(%s) for service(%s)", *subnetName, serviceName)
-	}
 
 	ipConfigCount, err := getPLSIPConfigCount(service)
 	if err != nil {
diff --git a/pkg/provider/azure_utils.go b/pkg/provider/azure_utils.go
index 253957d19..af288671a 100644
--- a/pkg/provider/azure_utils.go
+++ b/pkg/provider/azure_utils.go
@@ -438,9 +438,12 @@ func getResourceByIPFamily(resource string, isDualStack, isIPv6 bool) string {
 
 // isFIPIPv6 checks if the frontend IP configuration is of IPv6.
 // NOTICE: isFIPIPv6 assumes the FIP is owned by the Service and it is the primary Service.
-func (az *Cloud) isFIPIPv6(service *v1.Service, _ string, fip *network.FrontendIPConfiguration) (bool, error) {
+func (az *Cloud) isFIPIPv6(service *v1.Service, fip *network.FrontendIPConfiguration) (bool, error) {
 	isDualStack := isServiceDualStack(service)
 	if !isDualStack {
+		if len(service.Spec.IPFamilies) == 0 {
+			return false, nil
+		}
 		return service.Spec.IPFamilies[0] == v1.IPv6Protocol, nil
 	}
 	return managedResourceHasIPv6Suffix(pointer.StringDeref(fip.Name, "")), nil
diff --git a/pkg/provider/azure_vmss.go b/pkg/provider/azure_vmss.go
index 99ed13968..dcdbf43fe 100644
--- a/pkg/provider/azure_vmss.go
+++ b/pkg/provider/azure_vmss.go
@@ -1970,7 +1970,9 @@ func (ss *ScaleSet) EnsureBackendPoolDeleted(ctx context.Context, service *v1.Se
 
 	for _, backendPool := range *backendAddressPools {
 		for _, backendPoolID := range backendPoolIDs {
-			if strings.EqualFold(*backendPool.ID, backendPoolID) && backendPool.BackendIPConfigurations != nil {
+			if strings.EqualFold(*backendPool.ID, backendPoolID) &&
+				backendPool.BackendAddressPoolPropertiesFormat != nil &&
+				backendPool.BackendIPConfigurations != nil {
 				for _, ipConf := range *backendPool.BackendIPConfigurations {
 					if ipConf.ID == nil {
 						continue
diff --git a/pkg/provider/subnet/mock.go b/pkg/provider/subnet/mock.go
new file mode 100644
index 000000000..cd2fd4665
--- /dev/null
+++ b/pkg/provider/subnet/mock.go
@@ -0,0 +1,88 @@
+// /*
+// Copyright The Kubernetes Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+// */
+//
+
+// Code generated by MockGen. DO NOT EDIT.
+// Source: subnet.go
+//
+// Generated by this command:
+//
+//	mockgen -package subnet -source subnet.go -copyright_file ../../../hack/boilerplate/boilerplate.generatego.txt Repository
+//
+
+// Package subnet is a generated GoMock package.
+package subnet
+
+import (
+	context "context"
+	reflect "reflect"
+
+	armnetwork "github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v6"
+	gomock "go.uber.org/mock/gomock"
+)
+
+// MockRepository is a mock of Repository interface.
+type MockRepository struct {
+	ctrl     *gomock.Controller
+	recorder *MockRepositoryMockRecorder
+	isgomock struct{}
+}
+
+// MockRepositoryMockRecorder is the mock recorder for MockRepository.
+type MockRepositoryMockRecorder struct {
+	mock *MockRepository
+}
+
+// NewMockRepository creates a new mock instance.
+func NewMockRepository(ctrl *gomock.Controller) *MockRepository {
+	mock := &MockRepository{ctrl: ctrl}
+	mock.recorder = &MockRepositoryMockRecorder{mock}
+	return mock
+}
+
+// EXPECT returns an object that allows the caller to indicate expected use.
+func (m *MockRepository) EXPECT() *MockRepositoryMockRecorder {
+	return m.recorder
+}
+
+// CreateOrUpdate mocks base method.
+func (m *MockRepository) CreateOrUpdate(ctx context.Context, rg, vnetName, subnetName string, subnet armnetwork.Subnet) error {
+	m.ctrl.T.Helper()
+	ret := m.ctrl.Call(m, "CreateOrUpdate", ctx, rg, vnetName, subnetName, subnet)
+	ret0, _ := ret[0].(error)
+	return ret0
+}
+
+// CreateOrUpdate indicates an expected call of CreateOrUpdate.
+func (mr *MockRepositoryMockRecorder) CreateOrUpdate(ctx, rg, vnetName, subnetName, subnet any) *gomock.Call {
+	mr.mock.ctrl.T.Helper()
+	return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CreateOrUpdate", reflect.TypeOf((*MockRepository)(nil).CreateOrUpdate), ctx, rg, vnetName, subnetName, subnet)
+}
+
+// Get mocks base method.
+func (m *MockRepository) Get(ctx context.Context, rg, vnetName, subnetName string) (*armnetwork.Subnet, error) {
+	m.ctrl.T.Helper()
+	ret := m.ctrl.Call(m, "Get", ctx, rg, vnetName, subnetName)
+	ret0, _ := ret[0].(*armnetwork.Subnet)
+	ret1, _ := ret[1].(error)
+	return ret0, ret1
+}
+
+// Get indicates an expected call of Get.
+func (mr *MockRepositoryMockRecorder) Get(ctx, rg, vnetName, subnetName any) *gomock.Call {
+	mr.mock.ctrl.T.Helper()
+	return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Get", reflect.TypeOf((*MockRepository)(nil).Get), ctx, rg, vnetName, subnetName)
+}
diff --git a/pkg/provider/subnet/subnet.go b/pkg/provider/subnet/subnet.go
new file mode 100644
index 000000000..8cbf903ec
--- /dev/null
+++ b/pkg/provider/subnet/subnet.go
@@ -0,0 +1,64 @@
+/*
+Copyright 2024 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package subnet
+
+import (
+	"context"
+
+	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v6"
+	"k8s.io/klog/v2"
+
+	"sigs.k8s.io/cloud-provider-azure/pkg/azclient/subnetclient"
+)
+
+type Repository interface {
+	CreateOrUpdate(ctx context.Context, rg string, vnetName string, subnetName string, subnet armnetwork.Subnet) error
+	Get(ctx context.Context, rg string, vnetName string, subnetName string) (*armnetwork.Subnet, error)
+}
+
+type repo struct {
+	SubnetsClient subnetclient.Interface
+}
+
+func NewRepo(subnetsClient subnetclient.Interface) (Repository, error) {
+	return &repo{
+		SubnetsClient: subnetsClient,
+	}, nil
+}
+
+// CreateOrUpdateSubnet invokes az.SubnetClient.CreateOrUpdate with exponential backoff retry
+func (az *repo) CreateOrUpdate(ctx context.Context, rg string, vnetName string, subnetName string, subnet armnetwork.Subnet) error {
+
+	_, rerr := az.SubnetsClient.CreateOrUpdate(ctx, rg, vnetName, subnetName, subnet)
+	klog.V(10).Infof("SubnetsClient.CreateOrUpdate(%s): end", subnetName)
+	if rerr != nil {
+		klog.Errorf("SubnetClient.CreateOrUpdate(%s) failed: %s", subnetName, rerr.Error())
+		return rerr
+	}
+
+	return nil
+}
+
+func (az *repo) Get(ctx context.Context, rg string, vnetName string, subnetName string) (*armnetwork.Subnet, error) {
+
+	subnet, err := az.SubnetsClient.Get(ctx, rg, vnetName, subnetName, nil)
+	if err != nil {
+		klog.Errorf("SubnetClient.Get(%s) failed: %s", subnetName, err.Error())
+		return nil, err
+	}
+	return subnet, nil
+}
diff --git a/vendor/modules.txt b/vendor/modules.txt
index abec322b4..96d158c25 100644
--- a/vendor/modules.txt
+++ b/vendor/modules.txt
@@ -1549,6 +1549,7 @@ sigs.k8s.io/cloud-provider-azure/pkg/azclient/securitygroupclient/mock_securityg
 sigs.k8s.io/cloud-provider-azure/pkg/azclient/snapshotclient
 sigs.k8s.io/cloud-provider-azure/pkg/azclient/sshpublickeyresourceclient
 sigs.k8s.io/cloud-provider-azure/pkg/azclient/subnetclient
+sigs.k8s.io/cloud-provider-azure/pkg/azclient/subnetclient/mock_subnetclient
 sigs.k8s.io/cloud-provider-azure/pkg/azclient/utils
 sigs.k8s.io/cloud-provider-azure/pkg/azclient/utils/armbalancer
 sigs.k8s.io/cloud-provider-azure/pkg/azclient/vaultclient
diff --git a/vendor/sigs.k8s.io/cloud-provider-azure/pkg/azclient/subnetclient/mock_subnetclient/interface.go b/vendor/sigs.k8s.io/cloud-provider-azure/pkg/azclient/subnetclient/mock_subnetclient/interface.go
new file mode 100644
index 000000000..382b6b4f8
--- /dev/null
+++ b/vendor/sigs.k8s.io/cloud-provider-azure/pkg/azclient/subnetclient/mock_subnetclient/interface.go
@@ -0,0 +1,117 @@
+// /*
+// Copyright The Kubernetes Authors.
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//     http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+// */
+
+// Code generated by MockGen. DO NOT EDIT.
+// Source: subnetclient/interface.go
+//
+// Generated by this command:
+//
+//	mockgen -package mock_subnetclient -source subnetclient/interface.go
+//
+
+// Package mock_subnetclient is a generated GoMock package.
+package mock_subnetclient
+
+import (
+	context "context"
+	reflect "reflect"
+
+	armnetwork "github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v6"
+	gomock "go.uber.org/mock/gomock"
+)
+
+// MockInterface is a mock of Interface interface.
+type MockInterface struct {
+	ctrl     *gomock.Controller
+	recorder *MockInterfaceMockRecorder
+	isgomock struct{}
+}
+
+// MockInterfaceMockRecorder is the mock recorder for MockInterface.
+type MockInterfaceMockRecorder struct {
+	mock *MockInterface
+}
+
+// NewMockInterface creates a new mock instance.
+func NewMockInterface(ctrl *gomock.Controller) *MockInterface {
+	mock := &MockInterface{ctrl: ctrl}
+	mock.recorder = &MockInterfaceMockRecorder{mock}
+	return mock
+}
+
+// EXPECT returns an object that allows the caller to indicate expected use.
+func (m *MockInterface) EXPECT() *MockInterfaceMockRecorder {
+	return m.recorder
+}
+
+// CreateOrUpdate mocks base method.
+func (m *MockInterface) CreateOrUpdate(ctx context.Context, resourceGroupName, parentResourceName, resourceName string, resourceParam armnetwork.Subnet) (*armnetwork.Subnet, error) {
+	m.ctrl.T.Helper()
+	ret := m.ctrl.Call(m, "CreateOrUpdate", ctx, resourceGroupName, parentResourceName, resourceName, resourceParam)
+	ret0, _ := ret[0].(*armnetwork.Subnet)
+	ret1, _ := ret[1].(error)
+	return ret0, ret1
+}
+
+// CreateOrUpdate indicates an expected call of CreateOrUpdate.
+func (mr *MockInterfaceMockRecorder) CreateOrUpdate(ctx, resourceGroupName, parentResourceName, resourceName, resourceParam any) *gomock.Call {
+	mr.mock.ctrl.T.Helper()
+	return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "CreateOrUpdate", reflect.TypeOf((*MockInterface)(nil).CreateOrUpdate), ctx, resourceGroupName, parentResourceName, resourceName, resourceParam)
+}
+
+// Delete mocks base method.
+func (m *MockInterface) Delete(ctx context.Context, resourceGroupName, parentResourceName, resourceName string) error {
+	m.ctrl.T.Helper()
+	ret := m.ctrl.Call(m, "Delete", ctx, resourceGroupName, parentResourceName, resourceName)
+	ret0, _ := ret[0].(error)
+	return ret0
+}
+
+// Delete indicates an expected call of Delete.
+func (mr *MockInterfaceMockRecorder) Delete(ctx, resourceGroupName, parentResourceName, resourceName any) *gomock.Call {
+	mr.mock.ctrl.T.Helper()
+	return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Delete", reflect.TypeOf((*MockInterface)(nil).Delete), ctx, resourceGroupName, parentResourceName, resourceName)
+}
+
+// Get mocks base method.
+func (m *MockInterface) Get(ctx context.Context, resourceGroupName, parentResourceName, resourceName string, expand *string) (*armnetwork.Subnet, error) {
+	m.ctrl.T.Helper()
+	ret := m.ctrl.Call(m, "Get", ctx, resourceGroupName, parentResourceName, resourceName, expand)
+	ret0, _ := ret[0].(*armnetwork.Subnet)
+	ret1, _ := ret[1].(error)
+	return ret0, ret1
+}
+
+// Get indicates an expected call of Get.
+func (mr *MockInterfaceMockRecorder) Get(ctx, resourceGroupName, parentResourceName, resourceName, expand any) *gomock.Call {
+	mr.mock.ctrl.T.Helper()
+	return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "Get", reflect.TypeOf((*MockInterface)(nil).Get), ctx, resourceGroupName, parentResourceName, resourceName, expand)
+}
+
+// List mocks base method.
+func (m *MockInterface) List(ctx context.Context, resourceGroupName, parentResourceName string) ([]*armnetwork.Subnet, error) {
+	m.ctrl.T.Helper()
+	ret := m.ctrl.Call(m, "List", ctx, resourceGroupName, parentResourceName)
+	ret0, _ := ret[0].([]*armnetwork.Subnet)
+	ret1, _ := ret[1].(error)
+	return ret0, ret1
+}
+
+// List indicates an expected call of List.
+func (mr *MockInterfaceMockRecorder) List(ctx, resourceGroupName, parentResourceName any) *gomock.Call {
+	mr.mock.ctrl.T.Helper()
+	return mr.mock.ctrl.RecordCallWithMethodType(mr.mock, "List", reflect.TypeOf((*MockInterface)(nil).List), ctx, resourceGroupName, parentResourceName)
+}


Test patch:
diff --git a/e2e.Dockerfile b/e2e.Dockerfile
index 6e6631e48..2c221c5a5 100644
--- a/e2e.Dockerfile
+++ b/e2e.Dockerfile
@@ -14,7 +14,7 @@
 # See the License for the specific language governing permissions and
 # limitations under the License.
 
-FROM golang:1.23-bullseye@sha256:1002103155eb5802f6876dceb8cb8bf965cc6d3deeb34f11e4774926aca6a1fd
+FROM golang:1.23-bullseye@sha256:d4387d8b2ee5b78a199fae168a761d0fb0f33f172642d20856f29a67204af16b
 
 WORKDIR /go/src/sigs.k8s.io/cloud-provider-azure
 
diff --git a/pkg/provider/azure_loadbalancer_test.go b/pkg/provider/azure_loadbalancer_test.go
index d2402fbb3..79f3b2aa9 100644
--- a/pkg/provider/azure_loadbalancer_test.go
+++ b/pkg/provider/azure_loadbalancer_test.go
@@ -6238,6 +6238,8 @@ func TestCleanOrphanedLoadBalancerLBInUseByVMSS(t *testing.T) {
 		expectedVMSS := buildTestVMSSWithLB(testVMSSName, "vmss-vm-", []string{testLBBackendpoolID0}, false)
 		mockVMSSClient := cloud.VirtualMachineScaleSetsClient.(*mockvmssclient.MockInterface)
 		mockVMSSClient.EXPECT().List(gomock.Any(), "rg").Return([]compute.VirtualMachineScaleSet{expectedVMSS}, nil)
+		mockVMSSClient.EXPECT().Get(gomock.Any(), "rg", testVMSSName).Return(expectedVMSS, nil)
+		mockVMSSClient.EXPECT().CreateOrUpdate(gomock.Any(), "rg", testVMSSName, gomock.Any()).Return(nil)
 
 		service := getTestService("test", v1.ProtocolTCP, nil, false, 80)
 		lb := getTestLoadBalancer(pointer.String("test"), pointer.String("rg"), pointer.String("test"), pointer.String("test"), service, consts.LoadBalancerSkuStandard)
diff --git a/pkg/provider/azure_privatelinkservice_test.go b/pkg/provider/azure_privatelinkservice_test.go
index f4a57dbea..cb8d0a6cb 100644
--- a/pkg/provider/azure_privatelinkservice_test.go
+++ b/pkg/provider/azure_privatelinkservice_test.go
@@ -22,6 +22,7 @@ import (
 	"strings"
 	"testing"
 
+	"github.com/Azure/azure-sdk-for-go/sdk/azcore"
 	"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
 	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/network/armnetwork/v6"
 	"github.com/Azure/azure-sdk-for-go/services/network/mgmt/2022-07-01/network"
@@ -30,12 +31,12 @@ import (
 	v1 "k8s.io/api/core/v1"
 	metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
 	"k8s.io/utils/pointer"
+	"k8s.io/utils/ptr"
 
-	"sigs.k8s.io/cloud-provider-azure/pkg/azureclients/subnetclient/mocksubnetclient"
 	"sigs.k8s.io/cloud-provider-azure/pkg/cache"
 	"sigs.k8s.io/cloud-provider-azure/pkg/consts"
 	"sigs.k8s.io/cloud-provider-azure/pkg/provider/privatelinkservice"
-	"sigs.k8s.io/cloud-provider-azure/pkg/retry"
+	"sigs.k8s.io/cloud-provider-azure/pkg/provider/subnet"
 )
 
 func TestReconcilePrivateLinkService(t *testing.T) {
@@ -47,7 +48,7 @@ func TestReconcilePrivateLinkService(t *testing.T) {
 		annotations       map[string]string
 		wantPLS           bool
 		expectedSubnetGet bool
-		existingSubnet    *network.Subnet
+		existingSubnet    *armnetwork.Subnet
 		expectedPLSList   bool
 		existingPLSList   []*armnetwork.PrivateLinkService
 		expectedPLSCreate bool
@@ -75,11 +76,11 @@ func TestReconcilePrivateLinkService(t *testing.T) {
 			},
 			wantPLS:           true,
 			expectedSubnetGet: true,
-			existingSubnet: &network.Subnet{
-				Name: pointer.String("subnet"),
-				ID:   pointer.String("subnetID"),
-				SubnetPropertiesFormat: &network.SubnetPropertiesFormat{
-					PrivateLinkServiceNetworkPolicies: network.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled,
+			existingSubnet: &armnetwork.Subnet{
+				Name: ptr.To("subnet"),
+				ID:   ptr.To("subnetID"),
+				Properties: &armnetwork.SubnetPropertiesFormat{
+					PrivateLinkServiceNetworkPolicies: to.Ptr(armnetwork.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled),
 				},
 			},
 			expectedPLSList:   true,
@@ -96,11 +97,11 @@ func TestReconcilePrivateLinkService(t *testing.T) {
 			},
 			wantPLS:           true,
 			expectedSubnetGet: true,
-			existingSubnet: &network.Subnet{
-				Name: pointer.String("subnet"),
-				ID:   pointer.String("subnetID"),
-				SubnetPropertiesFormat: &network.SubnetPropertiesFormat{
-					PrivateLinkServiceNetworkPolicies: network.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled,
+			existingSubnet: &armnetwork.Subnet{
+				Name: ptr.To("subnet"),
+				ID:   ptr.To("subnetID"),
+				Properties: &armnetwork.SubnetPropertiesFormat{
+					PrivateLinkServiceNetworkPolicies: to.Ptr(armnetwork.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled),
 				},
 			},
 			expectedPLSList:   true,
@@ -211,11 +212,11 @@ func TestReconcilePrivateLinkService(t *testing.T) {
 			},
 			wantPLS:           true,
 			expectedSubnetGet: true,
-			existingSubnet: &network.Subnet{
-				Name: pointer.String("subnet"),
-				ID:   pointer.String("subnetID"),
-				SubnetPropertiesFormat: &network.SubnetPropertiesFormat{
-					PrivateLinkServiceNetworkPolicies: network.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled,
+			existingSubnet: &armnetwork.Subnet{
+				Name: ptr.To("subnet"),
+				ID:   ptr.To("subnetID"),
+				Properties: &armnetwork.SubnetPropertiesFormat{
+					PrivateLinkServiceNetworkPolicies: to.Ptr(armnetwork.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled),
 				},
 			},
 			expectedPLSList: true,
@@ -252,11 +253,11 @@ func TestReconcilePrivateLinkService(t *testing.T) {
 			},
 			wantPLS:           true,
 			expectedSubnetGet: true,
-			existingSubnet: &network.Subnet{
-				Name: pointer.String("subnet"),
-				ID:   pointer.String("subnetID"),
-				SubnetPropertiesFormat: &network.SubnetPropertiesFormat{
-					PrivateLinkServiceNetworkPolicies: network.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled,
+			existingSubnet: &armnetwork.Subnet{
+				Name: ptr.To("subnet"),
+				ID:   ptr.To("subnetID"),
+				Properties: &armnetwork.SubnetPropertiesFormat{
+					PrivateLinkServiceNetworkPolicies: to.Ptr(armnetwork.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled),
 				},
 			},
 			expectedPLSList: true,
@@ -322,6 +323,7 @@ func TestReconcilePrivateLinkService(t *testing.T) {
 		},
 	}
 	for _, test := range testCases {
+		test := test
 		t.Run(test.desc, func(t *testing.T) {
 			az := GetTestCloud(ctrl)
 			service := getTestServiceWithAnnotation("test", test.annotations, false, 80)
@@ -340,10 +342,10 @@ func TestReconcilePrivateLinkService(t *testing.T) {
 			}
 			clusterName := testClusterName
 
-			mockSubnetsClient := az.SubnetsClient.(*mocksubnetclient.MockInterface)
+			mockSubnetsClient := az.subnetRepo.(*subnet.MockRepository)
 			mockPLSRepo := az.plsRepo.(*privatelinkservice.MockRepository)
 			if test.expectedSubnetGet {
-				mockSubnetsClient.EXPECT().Get(gomock.Any(), "rg", "vnet", "subnet", "").Return(*test.existingSubnet, nil).MaxTimes(2)
+				mockSubnetsClient.EXPECT().Get(gomock.Any(), "rg", "vnet", "subnet").Return(test.existingSubnet, nil).MaxTimes(2)
 			}
 			if test.expectedPLSList {
 				mockPLSRepo.EXPECT().Get(gomock.Any(), "rg", *fipConfig.ID, gomock.Any()).DoAndReturn(func(_ context.Context, _, frontendIPConfigID string, _ cache.AzureCacheReadType) (*armnetwork.PrivateLinkService, error) {
@@ -403,26 +405,26 @@ func TestDisablePLSNetworkPolicy(t *testing.T) {
 
 	testCases := []struct {
 		desc                 string
-		subnet               network.Subnet
+		subnet               *armnetwork.Subnet
 		expectedSubnetUpdate bool
 		expectedError        bool
 	}{
 		{
 			desc: "disablePLSNetworkPolicy shall not update subnet if pls-network-policy is disabled",
-			subnet: network.Subnet{
-				Name: pointer.String("plsSubnet"),
-				SubnetPropertiesFormat: &network.SubnetPropertiesFormat{
-					PrivateLinkServiceNetworkPolicies: network.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled,
+			subnet: &armnetwork.Subnet{
+				Name: ptr.To("plsSubnet"),
+				Properties: &armnetwork.SubnetPropertiesFormat{
+					PrivateLinkServiceNetworkPolicies: to.Ptr(armnetwork.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled),
 				},
 			},
 			expectedSubnetUpdate: false,
 		},
 		{
 			desc: "disablePLSNetworkPolicy shall update subnet if pls-network-policy is enabled",
-			subnet: network.Subnet{
-				Name: pointer.String("plsSubnet"),
-				SubnetPropertiesFormat: &network.SubnetPropertiesFormat{
-					PrivateLinkServiceNetworkPolicies: network.VirtualNetworkPrivateLinkServiceNetworkPoliciesEnabled,
+			subnet: &armnetwork.Subnet{
+				Name: ptr.To("plsSubnet"),
+				Properties: &armnetwork.SubnetPropertiesFormat{
+					PrivateLinkServiceNetworkPolicies: to.Ptr(armnetwork.VirtualNetworkPrivateLinkServiceNetworkPoliciesEnabled),
 				},
 			},
 			expectedSubnetUpdate: true,
@@ -430,22 +432,23 @@ func TestDisablePLSNetworkPolicy(t *testing.T) {
 	}
 
 	for i, test := range testCases {
+		test := test
 		az := GetTestCloud(ctrl)
 		service := &v1.Service{}
 		service.Annotations = map[string]string{
 			consts.ServiceAnnotationPLSIpConfigurationSubnet: "plsSubnet",
 		}
-		mockSubnetsClient := az.SubnetsClient.(*mocksubnetclient.MockInterface)
-		mockSubnetsClient.EXPECT().Get(gomock.Any(), "rg", "vnet", "plsSubnet", "").Return(test.subnet, nil).Times(1)
+		mockSubnetsClient := az.subnetRepo.(*subnet.MockRepository)
+		mockSubnetsClient.EXPECT().Get(gomock.Any(), "rg", "vnet", "plsSubnet").Return(test.subnet, nil).Times(1)
 		if test.expectedSubnetUpdate {
-			mockSubnetsClient.EXPECT().CreateOrUpdate(gomock.Any(), "rg", "vnet", "plsSubnet", network.Subnet{
-				Name: pointer.String("plsSubnet"),
-				SubnetPropertiesFormat: &network.SubnetPropertiesFormat{
-					PrivateLinkServiceNetworkPolicies: network.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled,
+			mockSubnetsClient.EXPECT().CreateOrUpdate(gomock.Any(), "rg", "vnet", "plsSubnet", armnetwork.Subnet{
+				Name: ptr.To("plsSubnet"),
+				Properties: &armnetwork.SubnetPropertiesFormat{
+					PrivateLinkServiceNetworkPolicies: to.Ptr(armnetwork.VirtualNetworkPrivateLinkServiceNetworkPoliciesDisabled),
 				},
 			}).Return(nil).Times(1)
 		}
-		err := az.disablePLSNetworkPolicy(service)
+		err := az.disablePLSNetworkPolicy(context.TODO(), service)
 		assert.Equal(t, test.expectedError, err != nil, "TestCase[%d]: %s", i, test.desc)
 	}
 }
@@ -574,14 +577,14 @@ func TestGetExpectedPrivateLinkService(t *testing.T) {
 		clusterName := testClusterName
 		fipConfig := &network.FrontendIPConfiguration{ID: pointer.String("fipConfigID")}
 		pls := &armnetwork.PrivateLinkService{Properties: &armnetwork.PrivateLinkServiceProperties{}}
-		subnetClient := cloud.SubnetsClient.(*mocksubnetclient.MockInterface)
-		subnetClient.EXPECT().Get(gomock.Any(), "rg", "vnet", "subnet", gomock.Any()).Return(
-			network.Subnet{
-				ID:   pointer.String("subnetID"),
-				Name: pointer.String("subnet"),
+		subnetClient := cloud.subnetRepo.(*subnet.MockRepository)
+		subnetClient.EXPECT().Get(gomock.Any(), "rg", "vnet", "subnet").Return(
+			&armnetwork.Subnet{
+				ID:   ptr.To("subnetID"),
+				Name: ptr.To("subnet"),
 			}, nil).MaxTimes(1)
 
-		dirtyPLS, err := cloud.getExpectedPrivateLinkService(pls, &plsName, &clusterName, service, fipConfig)
+		dirtyPLS, err := cloud.getExpectedPrivateLinkService(context.TODO(), pls, &plsName, &clusterName, service, fipConfig)
 		assert.NoError(t, err)
 		assert.True(t, dirtyPLS)
 
@@ -647,7 +650,7 @@ func TestReconcilePLSIpConfigs(t *testing.T) {
 		plsName           string
 		existingIPConfigs []*armnetwork.PrivateLinkServiceIPConfiguration
 		expectedIPConfigs []*armnetwork.PrivateLinkServiceIPConfiguration
-		getSubnetError    *retry.Error
+		getSubnetError    error
 		expectedChanged   bool
 		expectedErr       bool
 	}{
@@ -657,7 +660,7 @@ func TestReconcilePLSIpConfigs(t *testing.T) {
 			annotations: map[string]string{
 				consts.ServiceAnnotationPLSIpConfigurationSubnet: "subnet",
 			},
-			getSubnetError: &retry.Error{HTTPStatusCode: http.StatusNotFound},
+			getSubnetError: &azcore.ResponseError{StatusCode: http.StatusNotFound},
 			expectedErr:    true,
 		},
 		{
@@ -1006,14 +1009,14 @@ func TestReconcilePLSIpConfigs(t *testing.T) {
 				IPConfigurations: test.existingIPConfigs,
 			},
 		}
-		subnetClient := cloud.SubnetsClient.(*mocksubnetclient.MockInterface)
-		subnetClient.EXPECT().Get(gomock.Any(), "rg", "vnet", "subnet", gomock.Any()).Return(
-			network.Subnet{
-				ID:   pointer.String("subnetID"),
-				Name: pointer.String("subnet"),
+		subnetClient := cloud.subnetRepo.(*subnet.MockRepository)
+		subnetClient.EXPECT().Get(gomock.Any(), "rg", "vnet", "subnet").Return(
+			&armnetwork.Subnet{
+				ID:   ptr.To("subnetID"),
+				Name: ptr.To("subnet"),
 			}, test.getSubnetError).MaxTimes(1)
 
-		changed, err := cloud.reconcilePLSIpConfigs(pls, service)
+		changed, err := cloud.reconcilePLSIpConfigs(context.TODO(), pls, service)
 		if test.expectedErr {
 			assert.Error(t, err, "TestCase[%d]: %s", i, test.desc)
 		} else {
diff --git a/pkg/provider/azure_subnet_repo_test.go b/pkg/provider/azure_subnet_repo_test.go
deleted file mode 100644
index 10c1c6cf7..000000000
--- a/pkg/provider/azure_subnet_repo_test.go
+++ /dev/null
@@ -1,17 +0,0 @@
-/*
-Copyright 2023 The Kubernetes Authors.
-
-Licensed under the Apache License, Version 2.0 (the "License");
-you may not use this file except in compliance with the License.
-You may obtain a copy of the License at
-
-    http://www.apache.org/licenses/LICENSE-2.0
-
-Unless required by applicable law or agreed to in writing, software
-distributed under the License is distributed on an "AS IS" BASIS,
-WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
-See the License for the specific language governing permissions and
-limitations under the License.
-*/
-
-package provider
diff --git a/pkg/provider/azure_utils_test.go b/pkg/provider/azure_utils_test.go
index 4d6e225e6..43b389ebb 100644
--- a/pkg/provider/azure_utils_test.go
+++ b/pkg/provider/azure_utils_test.go
@@ -871,12 +871,21 @@ func TestIsFIPIPv6(t *testing.T) {
 			},
 			expectedIsIPv6: true,
 		},
+		{
+			desc: "enpty ip families",
+			svc: v1.Service{
+				Spec: v1.ServiceSpec{
+					IPFamilies: []v1.IPFamily{},
+				},
+			},
+			expectedIsIPv6: false,
+		},
 	}
 	for _, tc := range testcases {
 		tc := tc
 		t.Run(tc.desc, func(t *testing.T) {
 			az := GetTestCloud(ctrl)
-			isIPv6, err := az.isFIPIPv6(&tc.svc, "rg", tc.fip)
+			isIPv6, err := az.isFIPIPv6(&tc.svc, tc.fip)
 			assert.Nil(t, err)
 			assert.Equal(t, tc.expectedIsIPv6, isIPv6)
 		})