Repository: kubernetes-sigs/cloud-provider-azure
Pull request #523 (2021-02-26 23:37:58)

enable gocyclo linter
<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/release.md#issue-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. Follow the instructions for writing a release note: https://git.k8s.io/community/contributors/guide/release-notes.md
6. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

**What type of PR is this?**
> Uncomment only one ` /kind <>` line, hit enter to put that in a new line, and remove leading whitespaces from that line:
>
> /kind api-change
> /kind bug
> /kind cleanup
> /kind design
> /kind documentation
> /kind failing-test
> /kind feature
> /kind flake

/kind cleanup


**What this PR does / why we need it**: Fix code and enable gocyclo linter. Follow up from #518.

**Which issue(s) this PR fixes**:

```
pkg/provider/azure_loadbalancer.go:1231:1: cyclomatic complexity 76 of func `(*Cloud).reconcileLoadBalancer` is high (> 30) (gocyclo)
func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service, nodes []*v1.Node, wantLb bool) (*network.LoadBalancer, error) {
^
pkg/provider/azure_loadbalancer.go:1857:1: cyclomatic complexity 56 of func `(*Cloud).reconcileSecurityGroup` is high (> 30) (gocyclo)
func (az *Cloud) reconcileSecurityGroup(clusterName string, service *v1.Service, lbIP *string, wantLb bool) (*network.SecurityGroup, error) {
^
pkg/provider/azure.go:372:1: cyclomatic complexity 38 of func `(*Cloud).InitializeCloudFromConfig` is high (> 30) (gocyclo)
func (az *Cloud) InitializeCloudFromConfig(config *Config, fromSecret bool) error {
^
pkg/provider/azure_loadbalancer.go:1623:1: cyclomatic complexity 34 of func `(*Cloud).checkLoadBalancerResourcesConflicted` is high (> 30) (gocyclo)
func (az *Cloud) checkLoadBalancerResourcesConflicted(
^
pkg/provider/azure_test.go:2015:1: cyclomatic complexity 33 of func `validateConfig` is high (> 30) (gocyclo)
func validateConfig(t *testing.T, config string) {
^
pkg/provider/azure_loadbalancer.go:2306:1: cyclomatic complexity 32 of func `(*Cloud).reconcilePublicIP` is high (> 30) (gocyclo)
func (az *Cloud) reconcilePublicIP(clusterName string, service *v1.Service, lbName string, wantLb bool) (*network.PublicIPAddress, error) {
^
pkg/provider/azure_test.go:2890:1: cyclomatic complexity 31 of func `TestCanCombineSharedAndPrivateRulesInSameGroup` is high (> 30) (gocyclo)
func TestCanCombineSharedAndPrivateRulesInSameGroup(t *testing.T) {
^
```

<!-- 
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

**Special notes for your reviewer**:


**Release note**:
```
NONE
```


k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/cloud-provider-azure/pull/523#" title="Author self-approved">CecileRobertMichon</a>*
To complete the [pull request process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process), please assign **nilo19** after the PR has been reviewed.
You can assign the PR to them by writing `/assign @nilo19` in a comment when ready.

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fcloud-provider-azure).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes-sigs/cloud-provider-azure/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["nilo19"]} -->
CecileRobertMichon: this is rebased on top of #522 to minimize merge conflicts.

/hold until #522 merges so I can drop the first commit.
k8s-ci-robot: @CecileRobertMichon: The following test **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-cloud-provider-azure-e2e | 5233b1556f53c97c59b8be88da6b4508c9eff603 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e/1365446930078896128) | `/test pull-cloud-provider-azure-e2e`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=523). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3ACecileRobertMichon). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @CecileRobertMichon: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-cloud-provider-azure-e2e | 5233b1556f53c97c59b8be88da6b4508c9eff603 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e/1365446930078896128) | `/test pull-cloud-provider-azure-e2e`
pull-cloud-provider-azure-e2e-ccm | 5233b1556f53c97c59b8be88da6b4508c9eff603 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm/1365446930166976512) | `/test pull-cloud-provider-azure-e2e-ccm`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=523). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3ACecileRobertMichon). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @CecileRobertMichon: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-cloud-provider-azure-e2e | 5233b1556f53c97c59b8be88da6b4508c9eff603 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e/1365446930078896128) | `/test pull-cloud-provider-azure-e2e`
pull-cloud-provider-azure-e2e-ccm | 5233b1556f53c97c59b8be88da6b4508c9eff603 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm/1365446930166976512) | `/test pull-cloud-provider-azure-e2e-ccm`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 5233b1556f53c97c59b8be88da6b4508c9eff603 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1365446930125033472) | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=523). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3ACecileRobertMichon). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @CecileRobertMichon: PR needs rebase.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
CecileRobertMichon: /hold cancel
k8s-ci-robot: @CecileRobertMichon: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm | 5233b1556f53c97c59b8be88da6b4508c9eff603 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm/1365446930166976512) | `/test pull-cloud-provider-azure-e2e-ccm`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 5233b1556f53c97c59b8be88da6b4508c9eff603 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1365446930125033472) | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e | aaae4e0feff8278dbed4dc0c0277a73ea98ff99e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e/1365901939673403392) | `/test pull-cloud-provider-azure-e2e`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=523). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3ACecileRobertMichon). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @CecileRobertMichon: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 5233b1556f53c97c59b8be88da6b4508c9eff603 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1365446930125033472) | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e | aaae4e0feff8278dbed4dc0c0277a73ea98ff99e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e/1365901939673403392) | `/test pull-cloud-provider-azure-e2e`
pull-cloud-provider-azure-e2e-ccm | aaae4e0feff8278dbed4dc0c0277a73ea98ff99e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm/1365901939757289472) | `/test pull-cloud-provider-azure-e2e-ccm`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=523). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3ACecileRobertMichon). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @CecileRobertMichon: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-cloud-provider-azure-e2e | aaae4e0feff8278dbed4dc0c0277a73ea98ff99e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e/1365901939673403392) | `/test pull-cloud-provider-azure-e2e`
pull-cloud-provider-azure-e2e-ccm | aaae4e0feff8278dbed4dc0c0277a73ea98ff99e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm/1365901939757289472) | `/test pull-cloud-provider-azure-e2e-ccm`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | aaae4e0feff8278dbed4dc0c0277a73ea98ff99e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1365901939706957824) | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=523). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3ACecileRobertMichon). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
CecileRobertMichon: /retest
k8s-ci-robot: @CecileRobertMichon: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | aaae4e0feff8278dbed4dc0c0277a73ea98ff99e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1365901939706957824) | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e-ccm | aaae4e0feff8278dbed4dc0c0277a73ea98ff99e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm/1366807237737058304) | `/test pull-cloud-provider-azure-e2e-ccm`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=523). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3ACecileRobertMichon). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @CecileRobertMichon: The following tests **failed**, say `/retest` to rerun all failed tests:

Test name | Commit | Details | Rerun command
--- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm | aaae4e0feff8278dbed4dc0c0277a73ea98ff99e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm/1366807237737058304) | `/test pull-cloud-provider-azure-e2e-ccm`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | aaae4e0feff8278dbed4dc0c0277a73ea98ff99e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/523/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1366807237661560832) | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=523). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3ACecileRobertMichon). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->

Patch:
diff --git a/.golangci.yml b/.golangci.yml
index d7d79b9d4..5f2b74588 100644
--- a/.golangci.yml
+++ b/.golangci.yml
@@ -9,7 +9,7 @@ linters:
     - errcheck
     - errorlint
     - goconst
-#    - gocyclo
+    - gocyclo
     - gofmt
     - goimports
     - golint
diff --git a/pkg/provider/azure.go b/pkg/provider/azure.go
index 11725a7c3..fb70e5291 100644
--- a/pkg/provider/azure.go
+++ b/pkg/provider/azure.go
@@ -379,7 +379,7 @@ func NewCloudWithoutFeatureGates(configReader io.Reader) (*Cloud, error) {
 func (az *Cloud) InitializeCloudFromConfig(config *Config, fromSecret bool) error {
 	// cloud-config not set, return nil so that it would be initialized from secret.
 	if config == nil {
-		klog.Warning("cloud-config is not provided, Azure cloud provider would be initialized from secret")
+		klog.Warning("cloud-config is not provided, Azure cloud provider will be initialized from secret")
 		return nil
 	}
 
@@ -442,57 +442,11 @@ func (az *Cloud) InitializeCloudFromConfig(config *Config, fromSecret bool) erro
 	// Initialize rate limiting config options.
 	InitializeCloudProviderRateLimitConfig(&config.CloudProviderRateLimitConfig)
 
-	// Conditionally configure resource request backoff
-	resourceRequestBackoff := wait.Backoff{
-		Steps: 1,
-	}
-	if config.CloudProviderBackoff {
-		// Assign backoff defaults if no configuration was passed in
-		if config.CloudProviderBackoffRetries == 0 {
-			config.CloudProviderBackoffRetries = backoffRetriesDefault
-		}
-		if config.CloudProviderBackoffDuration == 0 {
-			config.CloudProviderBackoffDuration = backoffDurationDefault
-		}
-		if config.CloudProviderBackoffExponent == 0 {
-			config.CloudProviderBackoffExponent = backoffExponentDefault
-		}
-
-		if config.CloudProviderBackoffJitter == 0 {
-			config.CloudProviderBackoffJitter = backoffJitterDefault
-		}
-
-		resourceRequestBackoff = wait.Backoff{
-			Steps:    config.CloudProviderBackoffRetries,
-			Factor:   config.CloudProviderBackoffExponent,
-			Duration: time.Duration(config.CloudProviderBackoffDuration) * time.Second,
-			Jitter:   config.CloudProviderBackoffJitter,
-		}
-		klog.V(2).Infof("Azure cloudprovider using try backoff: retries=%d, exponent=%f, duration=%d, jitter=%f",
-			config.CloudProviderBackoffRetries,
-			config.CloudProviderBackoffExponent,
-			config.CloudProviderBackoffDuration,
-			config.CloudProviderBackoffJitter)
-	} else {
-		// CloudProviderBackoffRetries will be set to 1 by default as the requirements of Azure SDK.
-		config.CloudProviderBackoffRetries = 1
-		config.CloudProviderBackoffDuration = backoffDurationDefault
-	}
+	resourceRequestBackoff := az.setCloudProviderBackoffDefaults(config)
 
-	if strings.EqualFold(config.LoadBalancerSku, loadBalancerSkuStandard) {
-		// Do not add master nodes to standard LB by default.
-		if config.ExcludeMasterFromStandardLB == nil {
-			config.ExcludeMasterFromStandardLB = &defaultExcludeMasterFromStandardLB
-		}
-
-		// Enable outbound SNAT by default.
-		if config.DisableOutboundSNAT == nil {
-			config.DisableOutboundSNAT = &defaultDisableOutboundSNAT
-		}
-	} else {
-		if config.DisableOutboundSNAT != nil && *config.DisableOutboundSNAT {
-			return fmt.Errorf("disableOutboundSNAT should only set when loadBalancerSku is standard")
-		}
+	err = az.setLBDefaults(config)
+	if err != nil {
+		return err
 	}
 
 	az.Config = *config
@@ -510,21 +464,11 @@ func (az *Cloud) InitializeCloudFromConfig(config *Config, fromSecret bool) erro
 	}
 
 	// If uses network resources in different AAD Tenant, then prepare corresponding Service Principal Token for VM/VMSS client and network resources client
-	var multiTenantServicePrincipalToken *adal.MultiTenantServicePrincipalToken
-	var networkResourceServicePrincipalToken *adal.ServicePrincipalToken
-	if az.Config.UsesNetworkResourceInDifferentTenant() {
-		multiTenantServicePrincipalToken, err = auth.GetMultiTenantServicePrincipalToken(&az.Config.AzureAuthConfig, &az.Environment)
-		if err != nil {
-			return err
-		}
-		networkResourceServicePrincipalToken, err = auth.GetNetworkResourceServicePrincipalToken(&az.Config.AzureAuthConfig, &az.Environment)
-		if err != nil {
-			return err
-		}
+	err = az.configureMultiTenantClients(servicePrincipalToken)
+	if err != nil {
+		return err
 	}
 
-	az.configAzureClients(servicePrincipalToken, multiTenantServicePrincipalToken, networkResourceServicePrincipalToken)
-
 	if az.MaximumLoadBalancerRuleCount == 0 {
 		az.MaximumLoadBalancerRuleCount = maximumLoadBalancerRuleCount
 	}
@@ -569,6 +513,84 @@ func (az *Cloud) InitializeCloudFromConfig(config *Config, fromSecret bool) erro
 	return nil
 }
 
+func (az *Cloud) setLBDefaults(config *Config) error {
+	if strings.EqualFold(config.LoadBalancerSku, loadBalancerSkuStandard) {
+		// Do not add master nodes to standard LB by default.
+		if config.ExcludeMasterFromStandardLB == nil {
+			config.ExcludeMasterFromStandardLB = &defaultExcludeMasterFromStandardLB
+		}
+
+		// Enable outbound SNAT by default.
+		if config.DisableOutboundSNAT == nil {
+			config.DisableOutboundSNAT = &defaultDisableOutboundSNAT
+		}
+	} else {
+		if config.DisableOutboundSNAT != nil && *config.DisableOutboundSNAT {
+			return fmt.Errorf("disableOutboundSNAT should only set when loadBalancerSku is standard")
+		}
+	}
+	return nil
+}
+
+func (az *Cloud) configureMultiTenantClients(servicePrincipalToken *adal.ServicePrincipalToken) error {
+	var err error
+	var multiTenantServicePrincipalToken *adal.MultiTenantServicePrincipalToken
+	var networkResourceServicePrincipalToken *adal.ServicePrincipalToken
+	if az.Config.UsesNetworkResourceInDifferentTenant() {
+		multiTenantServicePrincipalToken, err = auth.GetMultiTenantServicePrincipalToken(&az.Config.AzureAuthConfig, &az.Environment)
+		if err != nil {
+			return err
+		}
+		networkResourceServicePrincipalToken, err = auth.GetNetworkResourceServicePrincipalToken(&az.Config.AzureAuthConfig, &az.Environment)
+		if err != nil {
+			return err
+		}
+	}
+
+	az.configAzureClients(servicePrincipalToken, multiTenantServicePrincipalToken, networkResourceServicePrincipalToken)
+	return nil
+}
+
+func (az *Cloud) setCloudProviderBackoffDefaults(config *Config) wait.Backoff {
+	// Conditionally configure resource request backoff
+	resourceRequestBackoff := wait.Backoff{
+		Steps: 1,
+	}
+	if config.CloudProviderBackoff {
+		// Assign backoff defaults if no configuration was passed in
+		if config.CloudProviderBackoffRetries == 0 {
+			config.CloudProviderBackoffRetries = backoffRetriesDefault
+		}
+		if config.CloudProviderBackoffDuration == 0 {
+			config.CloudProviderBackoffDuration = backoffDurationDefault
+		}
+		if config.CloudProviderBackoffExponent == 0 {
+			config.CloudProviderBackoffExponent = backoffExponentDefault
+		}
+
+		if config.CloudProviderBackoffJitter == 0 {
+			config.CloudProviderBackoffJitter = backoffJitterDefault
+		}
+
+		resourceRequestBackoff = wait.Backoff{
+			Steps:    config.CloudProviderBackoffRetries,
+			Factor:   config.CloudProviderBackoffExponent,
+			Duration: time.Duration(config.CloudProviderBackoffDuration) * time.Second,
+			Jitter:   config.CloudProviderBackoffJitter,
+		}
+		klog.V(2).Infof("Azure cloudprovider using try backoff: retries=%d, exponent=%f, duration=%d, jitter=%f",
+			config.CloudProviderBackoffRetries,
+			config.CloudProviderBackoffExponent,
+			config.CloudProviderBackoffDuration,
+			config.CloudProviderBackoffJitter)
+	} else {
+		// CloudProviderBackoffRetries will be set to 1 by default as the requirements of Azure SDK.
+		config.CloudProviderBackoffRetries = 1
+		config.CloudProviderBackoffDuration = backoffDurationDefault
+	}
+	return resourceRequestBackoff
+}
+
 func (az *Cloud) configAzureClients(
 	servicePrincipalToken *adal.ServicePrincipalToken,
 	multiTenantServicePrincipalToken *adal.MultiTenantServicePrincipalToken,
diff --git a/pkg/provider/azure_loadbalancer.go b/pkg/provider/azure_loadbalancer.go
index 094d2080a..edb789aaa 100644
--- a/pkg/provider/azure_loadbalancer.go
+++ b/pkg/provider/azure_loadbalancer.go
@@ -1242,98 +1242,217 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 	}
 	lbName := *lb.Name
 	lbResourceGroup := az.getLoadBalancerResourceGroup()
+	lbBackendPoolID := az.getBackendPoolID(lbName, az.getLoadBalancerResourceGroup(), getBackendPoolName(clusterName, service))
 	klog.V(2).Infof("reconcileLoadBalancer for service(%s): lb(%s/%s) wantLb(%t) resolved load balancer name", serviceName, lbResourceGroup, lbName, wantLb)
 	defaultLBFrontendIPConfigName := az.getDefaultFrontendIPConfigName(service)
 	defaultLBFrontendIPConfigID := az.getFrontendIPConfigID(lbName, lbResourceGroup, defaultLBFrontendIPConfigName)
-	lbBackendPoolName := getBackendPoolName(clusterName, service)
-	lbBackendPoolID := az.getBackendPoolID(lbName, lbResourceGroup, lbBackendPoolName)
+	dirtyLb := false
 
 	lbIdleTimeout, err := getIdleTimeout(service)
 	if wantLb && err != nil {
 		return nil, err
 	}
 
-	dirtyLb := false
+	// reconcile the load balancer's backend pool configuration.
+	if wantLb {
+		preConfig, changed, err := az.reconcileBackendPools(clusterName, service, nodes, lb, isBackendPoolPreConfigured)
+		if err != nil {
+			return lb, err
+		}
+		if changed {
+			dirtyLb = true
+		}
+		isBackendPoolPreConfigured = preConfig
+	}
+
+	// reconcile the load balancer's frontend IP configurations.
+	ownedFIPConfig, changed, err := az.reconcileFrontendIPConfigs(clusterName, service, lb, wantLb, defaultLBFrontendIPConfigName)
+	if err != nil {
+		return lb, err
+	}
+	if changed {
+		dirtyLb = true
+	}
+
+	// update probes/rules
+	if ownedFIPConfig != nil {
+		if ownedFIPConfig.ID != nil {
+			defaultLBFrontendIPConfigID = *ownedFIPConfig.ID
+		} else {
+			return nil, fmt.Errorf("reconcileLoadBalancer for service (%s)(%t): nil ID for frontend IP config", serviceName, wantLb)
+		}
+	}
 
-	// Ensure LoadBalancer's Backend Pool Configuration
 	if wantLb {
-		newBackendPools := []network.BackendAddressPool{}
-		if lb.BackendAddressPools != nil {
-			newBackendPools = *lb.BackendAddressPools
-		}
-
-		foundBackendPool := false
-		for _, bp := range newBackendPools {
-			if strings.EqualFold(*bp.Name, lbBackendPoolName) {
-				klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb backendpool - found wanted backendpool. not adding anything", serviceName, wantLb)
-				foundBackendPool = true
-
-				var backendIPConfigurationsToBeDeleted []network.InterfaceIPConfiguration
-				if bp.BackendAddressPoolPropertiesFormat != nil && bp.BackendIPConfigurations != nil {
-					for _, ipConf := range *bp.BackendIPConfigurations {
-						ipConfID := to.String(ipConf.ID)
-						nodeName, _, err := az.VMSet.GetNodeNameByIPConfigurationID(ipConfID)
-						if err != nil {
-							return nil, err
-						}
-						if nodeName == "" {
-							// VM may under deletion
-							continue
-						}
-						// If a node is not supposed to be included in the LB, it
-						// would not be in the `nodes` slice. We need to check the nodes that
-						// have been added to the LB's backendpool, find the unwanted ones and
-						// delete them from the pool.
-						if !nodeNameInNodes(nodeName, nodes) {
-							klog.V(2).Infof("reconcileLoadBalancer for service (%s)(%t): lb backendpool - found unwanted node %s, decouple it from the LB", serviceName, wantLb, nodeName)
-							// construct a backendPool that only contains the IP config of the node to be deleted
-							backendIPConfigurationsToBeDeleted = append(backendIPConfigurationsToBeDeleted, network.InterfaceIPConfiguration{ID: to.StringPtr(ipConfID)})
-						}
-					}
-					if len(backendIPConfigurationsToBeDeleted) > 0 {
-						backendpoolToBeDeleted := &[]network.BackendAddressPool{
-							{
-								ID: to.StringPtr(lbBackendPoolID),
-								BackendAddressPoolPropertiesFormat: &network.BackendAddressPoolPropertiesFormat{
-									BackendIPConfigurations: &backendIPConfigurationsToBeDeleted,
-								},
-							},
-						}
-						vmSetName := az.mapLoadBalancerNameToVMSet(lbName, clusterName)
-						// decouple the backendPool from the node
-						err = az.VMSet.EnsureBackendPoolDeleted(service, lbBackendPoolID, vmSetName, backendpoolToBeDeleted)
-						if err != nil {
-							return nil, err
-						}
-					}
+		err = az.checkLoadBalancerResourcesConflicts(lb, defaultLBFrontendIPConfigID, service)
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	expectedProbes, expectedRules, err := az.getExpectedLBRules(service, wantLb, defaultLBFrontendIPConfigID, lbBackendPoolID, lbName, lbIdleTimeout)
+	if err != nil {
+		return nil, err
+	}
+
+	changed = az.reconcileLBProbes(lb, service, serviceName, wantLb, expectedProbes)
+	if changed {
+		dirtyLb = true
+	}
+
+	changed = az.reconcileLBRules(lb, service, serviceName, wantLb, expectedRules)
+	if changed {
+		dirtyLb = true
+	}
+
+	changed = az.ensureLoadBalancerTagged(lb)
+	if changed {
+		dirtyLb = true
+	}
+
+	// We don't care if the LB exists or not
+	// We only care about if there is any change in the LB, which means dirtyLB
+	// If it is not exist, and no change to that, we don't CreateOrUpdate LB
+	if dirtyLb {
+		if lb.FrontendIPConfigurations == nil || len(*lb.FrontendIPConfigurations) == 0 {
+			err := az.cleanOrphanedLoadBalancer(lb, service, clusterName)
+			if err != nil {
+				klog.V(2).Infof("reconcileLoadBalancer for service(%s): lb(%s) - failed to cleanOrphanedLoadBalancer: %v", serviceName, lbName, err)
+				return nil, err
+			}
+		} else {
+			klog.V(2).Infof("reconcileLoadBalancer: reconcileLoadBalancer for service(%s): lb(%s) - updating", serviceName, lbName)
+			err := az.CreateOrUpdateLB(service, *lb)
+			if err != nil {
+				klog.V(2).Infof("reconcileLoadBalancer for service(%s) abort backoff: lb(%s) - updating", serviceName, lbName)
+				return nil, err
+			}
+
+			if isInternal {
+				// Refresh updated lb which will be used later in other places.
+				newLB, exist, err := az.getAzureLoadBalancer(lbName, azcache.CacheReadTypeDefault)
+				if err != nil {
+					klog.V(2).Infof("reconcileLoadBalancer for service(%s): getAzureLoadBalancer(%s) failed: %v", serviceName, lbName, err)
+					return nil, err
 				}
-				break
-			} else {
-				klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb backendpool - found other backendpool %s", serviceName, wantLb, *bp.Name)
+				if !exist {
+					return nil, fmt.Errorf("load balancer %q not found", lbName)
+				}
+				lb = &newLB
 			}
 		}
-		if !foundBackendPool {
-			if isBackendPoolPreConfigured {
-				klog.V(2).Infof("reconcileLoadBalancer for service (%s)(%t): lb backendpool - PreConfiguredBackendPoolLoadBalancerTypes %s has been set but can not find corresponding backend pool, ignoring it",
-					serviceName,
-					wantLb,
-					az.PreConfiguredBackendPoolLoadBalancerTypes)
-				isBackendPoolPreConfigured = false
+	}
+
+	if wantLb && nodes != nil && !isBackendPoolPreConfigured {
+		// Add the machines to the backend pool if they're not already
+		vmSetName := az.mapLoadBalancerNameToVMSet(lbName, clusterName)
+		// Etag would be changed when updating backend pools, so invalidate lbCache after it.
+		defer func() {
+			_ = az.lbCache.Delete(lbName)
+		}()
+		err := az.VMSet.EnsureHostsInPool(service, nodes, lbBackendPoolID, vmSetName, isInternal)
+		if err != nil {
+			return nil, err
+		}
+	}
+
+	klog.V(2).Infof("reconcileLoadBalancer for service(%s): lb(%s) finished", serviceName, lbName)
+	return lb, nil
+}
+
+func (az *Cloud) reconcileLBProbes(lb *network.LoadBalancer, service *v1.Service, serviceName string, wantLb bool, expectedProbes []network.Probe) bool {
+	// remove unwanted probes
+	dirtyProbes := false
+	var updatedProbes []network.Probe
+	if lb.Probes != nil {
+		updatedProbes = *lb.Probes
+	}
+	for i := len(updatedProbes) - 1; i >= 0; i-- {
+		existingProbe := updatedProbes[i]
+		if az.serviceOwnsRule(service, *existingProbe.Name) {
+			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb probe(%s) - considering evicting", serviceName, wantLb, *existingProbe.Name)
+			keepProbe := false
+			if findProbe(expectedProbes, existingProbe) {
+				klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb probe(%s) - keeping", serviceName, wantLb, *existingProbe.Name)
+				keepProbe = true
 			}
+			if !keepProbe {
+				updatedProbes = append(updatedProbes[:i], updatedProbes[i+1:]...)
+				klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb probe(%s) - dropping", serviceName, wantLb, *existingProbe.Name)
+				dirtyProbes = true
+			}
+		}
+	}
+	// add missing, wanted probes
+	for _, expectedProbe := range expectedProbes {
+		foundProbe := false
+		if findProbe(updatedProbes, expectedProbe) {
+			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb probe(%s) - already exists", serviceName, wantLb, *expectedProbe.Name)
+			foundProbe = true
+		}
+		if !foundProbe {
+			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb probe(%s) - adding", serviceName, wantLb, *expectedProbe.Name)
+			updatedProbes = append(updatedProbes, expectedProbe)
+			dirtyProbes = true
+		}
+	}
+	if dirtyProbes {
+		lb.Probes = &updatedProbes
+	}
+	return dirtyProbes
+}
 
-			newBackendPools = append(newBackendPools, network.BackendAddressPool{
-				Name: to.StringPtr(lbBackendPoolName),
-			})
-			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb backendpool - adding backendpool", serviceName, wantLb)
+func (az *Cloud) reconcileLBRules(lb *network.LoadBalancer, service *v1.Service, serviceName string, wantLb bool, expectedRules []network.LoadBalancingRule) bool {
+	// update rules
+	dirtyRules := false
+	var updatedRules []network.LoadBalancingRule
+	if lb.LoadBalancingRules != nil {
+		updatedRules = *lb.LoadBalancingRules
+	}
 
-			dirtyLb = true
-			lb.BackendAddressPools = &newBackendPools
+	// update rules: remove unwanted
+	for i := len(updatedRules) - 1; i >= 0; i-- {
+		existingRule := updatedRules[i]
+		if az.serviceOwnsRule(service, *existingRule.Name) {
+			keepRule := false
+			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb rule(%s) - considering evicting", serviceName, wantLb, *existingRule.Name)
+			if findRule(expectedRules, existingRule, wantLb) {
+				klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb rule(%s) - keeping", serviceName, wantLb, *existingRule.Name)
+				keepRule = true
+			}
+			if !keepRule {
+				klog.V(2).Infof("reconcileLoadBalancer for service (%s)(%t): lb rule(%s) - dropping", serviceName, wantLb, *existingRule.Name)
+				updatedRules = append(updatedRules[:i], updatedRules[i+1:]...)
+				dirtyRules = true
+			}
+		}
+	}
+	// update rules: add needed
+	for _, expectedRule := range expectedRules {
+		foundRule := false
+		if findRule(updatedRules, expectedRule, wantLb) {
+			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb rule(%s) - already exists", serviceName, wantLb, *expectedRule.Name)
+			foundRule = true
 		}
+		if !foundRule {
+			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb rule(%s) adding", serviceName, wantLb, *expectedRule.Name)
+			updatedRules = append(updatedRules, expectedRule)
+			dirtyRules = true
+		}
+	}
+	if dirtyRules {
+		lb.LoadBalancingRules = &updatedRules
 	}
+	return dirtyRules
+}
 
-	// Ensure LoadBalancer's Frontend IP Configurations
+func (az *Cloud) reconcileFrontendIPConfigs(clusterName string, service *v1.Service, lb *network.LoadBalancer, wantLb bool, defaultLBFrontendIPConfigName string) (*network.FrontendIPConfiguration, bool, error) {
+	var err error
+	lbName := *lb.Name
+	serviceName := getServiceName(service)
+	isInternal := requiresInternalLoadBalancer(service)
 	dirtyConfigs := false
-	newConfigs := []network.FrontendIPConfiguration{}
+	var newConfigs []network.FrontendIPConfiguration
 	if lb.FrontendIPConfigurations != nil {
 		newConfigs = *lb.FrontendIPConfigurations
 	}
@@ -1344,12 +1463,12 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 			config := newConfigs[i]
 			isServiceOwnsFrontendIP, _, err := az.serviceOwnsFrontendIP(config, service)
 			if err != nil {
-				return nil, err
+				return nil, false, err
 			}
 			if isServiceOwnsFrontendIP {
 				unsafe, err := az.isFrontendIPConfigUnsafeToDelete(lb, service, config.ID)
 				if err != nil {
-					return nil, err
+					return nil, false, err
 				}
 
 				// If the frontend IP configuration is not being referenced by:
@@ -1377,7 +1496,7 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 			config := newConfigs[i]
 			isFipChanged, err := az.isFrontendIPChanged(clusterName, config, service, defaultLBFrontendIPConfigName)
 			if err != nil {
-				return nil, err
+				return nil, false, err
 			}
 			if isFipChanged {
 				klog.V(2).Infof("reconcileLoadBalancer for service (%s)(%t): lb frontendconfig(%s) - dropping", serviceName, wantLb, *config.Name)
@@ -1388,7 +1507,7 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 
 		ownedFIPConfig, _, err = az.findFrontendIPConfigOfService(&newConfigs, service)
 		if err != nil {
-			return nil, err
+			return nil, false, err
 		}
 
 		if ownedFIPConfig == nil {
@@ -1401,7 +1520,7 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 				// TODO: remove this check when ILB supports IPv6 *and* the SDK
 				// have been rev'ed to 2019* version
 				if utilnet.IsIPv6String(service.Spec.ClusterIP) {
-					return nil, fmt.Errorf("ensure(%s): lb(%s) - internal load balancers does not support IPv6", serviceName, lbName)
+					return nil, false, fmt.Errorf("ensure(%s): lb(%s) - internal load balancers does not support IPv6", serviceName, lbName)
 				}
 
 				subnetName := subnet(service)
@@ -1410,11 +1529,11 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 				}
 				subnet, existsSubnet, err := az.getSubnet(az.VnetName, *subnetName)
 				if err != nil {
-					return nil, err
+					return nil, false, err
 				}
 
 				if !existsSubnet {
-					return nil, fmt.Errorf("ensure(%s): lb(%s) - failed to get subnet: %s/%s", serviceName, lbName, az.VnetName, az.SubnetName)
+					return nil, false, fmt.Errorf("ensure(%s): lb(%s) - failed to get subnet: %s/%s", serviceName, lbName, az.VnetName, az.SubnetName)
 				}
 
 				configProperties := network.FrontendIPConfigurationPropertiesFormat{
@@ -1434,12 +1553,12 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 			} else {
 				pipName, shouldPIPExisted, err := az.determinePublicIPName(clusterName, service)
 				if err != nil {
-					return nil, err
+					return nil, false, err
 				}
 				domainNameLabel, found := getPublicIPDomainNameLabel(service)
 				pip, err := az.ensurePublicIPExists(service, pipName, domainNameLabel, clusterName, shouldPIPExisted, found)
 				if err != nil {
-					return nil, err
+					return nil, false, err
 				}
 				fipConfigurationProperties = &network.FrontendIPConfigurationPropertiesFormat{
 					PublicIPAddress: &network.PublicIPAddress{ID: pip.ID},
@@ -1457,172 +1576,100 @@ func (az *Cloud) reconcileLoadBalancer(clusterName string, service *v1.Service,
 		}
 	}
 	if dirtyConfigs {
-		dirtyLb = true
 		lb.FrontendIPConfigurations = &newConfigs
 	}
+	return ownedFIPConfig, dirtyConfigs, err
+}
 
-	// update probes/rules
-	if ownedFIPConfig != nil {
-		if ownedFIPConfig.ID != nil {
-			defaultLBFrontendIPConfigID = *ownedFIPConfig.ID
-		} else {
-			return nil, fmt.Errorf("reconcileLoadBalancer for service (%s)(%t): nil ID for frontend IP config", serviceName, wantLb)
-		}
-	}
-
-	if wantLb {
-		err = az.checkLoadBalancerResourcesConflicted(lb, defaultLBFrontendIPConfigID, service)
-		if err != nil {
-			return nil, err
-		}
-	}
-
-	expectedProbes, expectedRules, err := az.reconcileLoadBalancerRule(service, wantLb, defaultLBFrontendIPConfigID, lbBackendPoolID, lbName, lbIdleTimeout)
-	if err != nil {
-		return nil, err
-	}
-
-	// remove unwanted probes
-	dirtyProbes := false
-	var updatedProbes []network.Probe
-	if lb.Probes != nil {
-		updatedProbes = *lb.Probes
-	}
-	for i := len(updatedProbes) - 1; i >= 0; i-- {
-		existingProbe := updatedProbes[i]
-		if az.serviceOwnsRule(service, *existingProbe.Name) {
-			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb probe(%s) - considering evicting", serviceName, wantLb, *existingProbe.Name)
-			keepProbe := false
-			if findProbe(expectedProbes, existingProbe) {
-				klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb probe(%s) - keeping", serviceName, wantLb, *existingProbe.Name)
-				keepProbe = true
-			}
-			if !keepProbe {
-				updatedProbes = append(updatedProbes[:i], updatedProbes[i+1:]...)
-				klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb probe(%s) - dropping", serviceName, wantLb, *existingProbe.Name)
-				dirtyProbes = true
-			}
-		}
-	}
-	// add missing, wanted probes
-	for _, expectedProbe := range expectedProbes {
-		foundProbe := false
-		if findProbe(updatedProbes, expectedProbe) {
-			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb probe(%s) - already exists", serviceName, wantLb, *expectedProbe.Name)
-			foundProbe = true
-		}
-		if !foundProbe {
-			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb probe(%s) - adding", serviceName, wantLb, *expectedProbe.Name)
-			updatedProbes = append(updatedProbes, expectedProbe)
-			dirtyProbes = true
-		}
-	}
-	if dirtyProbes {
-		dirtyLb = true
-		lb.Probes = &updatedProbes
+func (az *Cloud) reconcileBackendPools(clusterName string, service *v1.Service, nodes []*v1.Node, lb *network.LoadBalancer, isBackendPoolPreConfigured bool) (bool, bool, error) {
+	var newBackendPools []network.BackendAddressPool
+	var err error
+	if lb.BackendAddressPools != nil {
+		newBackendPools = *lb.BackendAddressPools
 	}
 
-	// update rules
-	dirtyRules := false
-	var updatedRules []network.LoadBalancingRule
-	if lb.LoadBalancingRules != nil {
-		updatedRules = *lb.LoadBalancingRules
-	}
-
-	// update rules: remove unwanted
-	for i := len(updatedRules) - 1; i >= 0; i-- {
-		existingRule := updatedRules[i]
-		if az.serviceOwnsRule(service, *existingRule.Name) {
-			keepRule := false
-			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb rule(%s) - considering evicting", serviceName, wantLb, *existingRule.Name)
-			if findRule(expectedRules, existingRule, wantLb) {
-				klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb rule(%s) - keeping", serviceName, wantLb, *existingRule.Name)
-				keepRule = true
-			}
-			if !keepRule {
-				klog.V(2).Infof("reconcileLoadBalancer for service (%s)(%t): lb rule(%s) - dropping", serviceName, wantLb, *existingRule.Name)
-				updatedRules = append(updatedRules[:i], updatedRules[i+1:]...)
-				dirtyRules = true
-			}
-		}
-	}
-	// update rules: add needed
-	for _, expectedRule := range expectedRules {
-		foundRule := false
-		if findRule(updatedRules, expectedRule, wantLb) {
-			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb rule(%s) - already exists", serviceName, wantLb, *expectedRule.Name)
-			foundRule = true
-		}
-		if !foundRule {
-			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb rule(%s) adding", serviceName, wantLb, *expectedRule.Name)
-			updatedRules = append(updatedRules, expectedRule)
-			dirtyRules = true
-		}
-	}
-	if dirtyRules {
-		dirtyLb = true
-		lb.LoadBalancingRules = &updatedRules
-	}
+	foundBackendPool := false
+	dirtyLb := false
+	wantLb := true
+	lbName := *lb.Name
 
-	changed := az.ensureLoadBalancerTagged(lb)
-	if changed {
-		dirtyLb = true
-	}
+	serviceName := getServiceName(service)
+	lbBackendPoolName := getBackendPoolName(clusterName, service)
+	lbBackendPoolID := az.getBackendPoolID(lbName, az.getLoadBalancerResourceGroup(), lbBackendPoolName)
+	vmSetName := az.mapLoadBalancerNameToVMSet(lbName, clusterName)
 
-	// We don't care if the LB exists or not
-	// We only care about if there is any change in the LB, which means dirtyLB
-	// If it is not exist, and no change to that, we don't CreateOrUpdate LB
-	if dirtyLb {
-		if lb.FrontendIPConfigurations == nil || len(*lb.FrontendIPConfigurations) == 0 {
-			err := az.cleanOrphanedLoadBalancer(lb, service, clusterName)
-			if err != nil {
-				klog.V(2).Infof("reconcileLoadBalancer for service(%s): lb(%s) - failed to cleanOrphanedLoadBalancer: %v", serviceName, lbName, err)
-				return nil, err
-			}
-		} else {
-			klog.V(2).Infof("reconcileLoadBalancer: reconcileLoadBalancer for service(%s): lb(%s) - updating", serviceName, lbName)
-			err := az.CreateOrUpdateLB(service, *lb)
-			if err != nil {
-				klog.V(2).Infof("reconcileLoadBalancer for service(%s) abort backoff: lb(%s) - updating", serviceName, lbName)
-				return nil, err
-			}
+	for _, bp := range newBackendPools {
+		if strings.EqualFold(*bp.Name, lbBackendPoolName) {
+			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb backendpool - found wanted backendpool. not adding anything", serviceName, wantLb)
+			foundBackendPool = true
 
-			if isInternal {
-				// Refresh updated lb which will be used later in other places.
-				newLB, exist, err := az.getAzureLoadBalancer(lbName, azcache.CacheReadTypeDefault)
-				if err != nil {
-					klog.V(2).Infof("reconcileLoadBalancer for service(%s): getAzureLoadBalancer(%s) failed: %v", serviceName, lbName, err)
-					return nil, err
+			var backendIPConfigurationsToBeDeleted []network.InterfaceIPConfiguration
+			if bp.BackendAddressPoolPropertiesFormat != nil && bp.BackendIPConfigurations != nil {
+				for _, ipConf := range *bp.BackendIPConfigurations {
+					ipConfID := to.String(ipConf.ID)
+					nodeName, _, err := az.VMSet.GetNodeNameByIPConfigurationID(ipConfID)
+					if err != nil {
+						return false, false, err
+					}
+					if nodeName == "" {
+						// VM may under deletion
+						continue
+					}
+					// If a node is not supposed to be included in the LB, it
+					// would not be in the `nodes` slice. We need to check the nodes that
+					// have been added to the LB's backendpool, find the unwanted ones and
+					// delete them from the pool.
+					if !nodeNameInNodes(nodeName, nodes) {
+						klog.V(2).Infof("reconcileLoadBalancer for service (%s)(%t): lb backendpool - found unwanted node %s, decouple it from the LB", serviceName, wantLb, nodeName)
+						// construct a backendPool that only contains the IP config of the node to be deleted
+						backendIPConfigurationsToBeDeleted = append(backendIPConfigurationsToBeDeleted, network.InterfaceIPConfiguration{ID: to.StringPtr(ipConfID)})
+					}
 				}
-				if !exist {
-					return nil, fmt.Errorf("load balancer %q not found", lbName)
+				if len(backendIPConfigurationsToBeDeleted) > 0 {
+					backendpoolToBeDeleted := &[]network.BackendAddressPool{
+						{
+							ID: to.StringPtr(lbBackendPoolID),
+							BackendAddressPoolPropertiesFormat: &network.BackendAddressPoolPropertiesFormat{
+								BackendIPConfigurations: &backendIPConfigurationsToBeDeleted,
+							},
+						},
+					}
+					// decouple the backendPool from the node
+					err = az.VMSet.EnsureBackendPoolDeleted(service, lbBackendPoolID, vmSetName, backendpoolToBeDeleted)
+					if err != nil {
+						return false, false, err
+					}
 				}
-				lb = &newLB
 			}
+			break
+		} else {
+			klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb backendpool - found other backendpool %s", serviceName, wantLb, *bp.Name)
 		}
 	}
-
-	if wantLb && nodes != nil && !isBackendPoolPreConfigured {
-		// Add the machines to the backend pool if they're not already
-		vmSetName := az.mapLoadBalancerNameToVMSet(lbName, clusterName)
-		// Etag would be changed when updating backend pools, so invalidate lbCache after it.
-		defer func() {
-			_ = az.lbCache.Delete(lbName)
-		}()
-		err := az.VMSet.EnsureHostsInPool(service, nodes, lbBackendPoolID, vmSetName, isInternal)
-		if err != nil {
-			return nil, err
+	if !foundBackendPool {
+		if isBackendPoolPreConfigured {
+			klog.V(2).Infof("reconcileLoadBalancer for service (%s)(%t): lb backendpool - PreConfiguredBackendPoolLoadBalancerTypes %s has been set but can not find corresponding backend pool, ignoring it",
+				serviceName,
+				wantLb,
+				az.PreConfiguredBackendPoolLoadBalancerTypes)
+			isBackendPoolPreConfigured = false
 		}
-	}
 
-	klog.V(2).Infof("reconcileLoadBalancer for service(%s): lb(%s) finished", serviceName, lbName)
-	return lb, nil
+		newBackendPools = append(newBackendPools, network.BackendAddressPool{
+			Name: to.StringPtr(lbBackendPoolName),
+		})
+		klog.V(10).Infof("reconcileLoadBalancer for service (%s)(%t): lb backendpool - adding backendpool", serviceName, wantLb)
+
+		dirtyLb = true
+		lb.BackendAddressPools = &newBackendPools
+	}
+	return isBackendPoolPreConfigured, dirtyLb, err
 }
 
-// checkLoadBalancerResourcesConflicted checks if the service is consuming
-// ports which are conflicted with the existing loadBalancer resources,
+// checkLoadBalancerResourcesConflicts checks if the service is consuming
+// ports which conflict with the existing loadBalancer resources,
 // including inbound NAT rule, inbound NAT pools and loadBalancing rules
-func (az *Cloud) checkLoadBalancerResourcesConflicted(
+func (az *Cloud) checkLoadBalancerResourcesConflicts(
 	lb *network.LoadBalancer,
 	frontendIPConfigID string,
 	service *v1.Service,
@@ -1635,18 +1682,12 @@ func (az *Cloud) checkLoadBalancerResourcesConflicted(
 	for _, port := range ports {
 		if lb.LoadBalancingRules != nil {
 			for _, rule := range *lb.LoadBalancingRules {
-				if rule.LoadBalancingRulePropertiesFormat != nil &&
-					rule.FrontendIPConfiguration != nil &&
-					rule.FrontendIPConfiguration.ID != nil &&
-					strings.EqualFold(*rule.FrontendIPConfiguration.ID, frontendIPConfigID) &&
-					strings.EqualFold(string(rule.Protocol), string(port.Protocol)) &&
-					rule.FrontendPort != nil &&
-					*rule.FrontendPort == port.Port {
+				if lbRuleConflictsWithPort(rule, frontendIPConfigID, port) {
 					// ignore self-owned rules for unit test
 					if rule.Name != nil && az.serviceOwnsRule(service, *rule.Name) {
 						continue
 					}
-					return fmt.Errorf("checkLoadBalancerResourcesConflicted: service port %s is trying to "+
+					return fmt.Errorf("checkLoadBalancerResourcesConflicts: service port %s is trying to "+
 						"consume the port %d which is being referenced by an existing loadBalancing rule %s with "+
 						"the same protocol %s and frontend IP config with ID %s",
 						port.Name,
@@ -1660,14 +1701,8 @@ func (az *Cloud) checkLoadBalancerResourcesConflicted(
 
 		if lb.InboundNatRules != nil {
 			for _, inboundNatRule := range *lb.InboundNatRules {
-				if inboundNatRule.InboundNatRulePropertiesFormat != nil &&
-					inboundNatRule.FrontendIPConfiguration != nil &&
-					inboundNatRule.FrontendIPConfiguration.ID != nil &&
-					strings.EqualFold(*inboundNatRule.FrontendIPConfiguration.ID, frontendIPConfigID) &&
-					strings.EqualFold(string(inboundNatRule.Protocol), string(port.Protocol)) &&
-					inboundNatRule.FrontendPort != nil &&
-					*inboundNatRule.FrontendPort == port.Port {
-					return fmt.Errorf("checkLoadBalancerResourcesConflicted: service port %s is trying to "+
+				if inboundNatRuleConflictsWithPort(inboundNatRule, frontendIPConfigID, port) {
+					return fmt.Errorf("checkLoadBalancerResourcesConflicts: service port %s is trying to "+
 						"consume the port %d which is being referenced by an existing inbound NAT rule %s with "+
 						"the same protocol %s and frontend IP config with ID %s",
 						port.Name,
@@ -1681,16 +1716,8 @@ func (az *Cloud) checkLoadBalancerResourcesConflicted(
 
 		if lb.InboundNatPools != nil {
 			for _, pool := range *lb.InboundNatPools {
-				if pool.InboundNatPoolPropertiesFormat != nil &&
-					pool.FrontendIPConfiguration != nil &&
-					pool.FrontendIPConfiguration.ID != nil &&
-					strings.EqualFold(*pool.FrontendIPConfiguration.ID, frontendIPConfigID) &&
-					strings.EqualFold(string(pool.Protocol), string(port.Protocol)) &&
-					pool.FrontendPortRangeStart != nil &&
-					pool.FrontendPortRangeEnd != nil &&
-					*pool.FrontendPortRangeStart <= port.Port &&
-					*pool.FrontendPortRangeEnd >= port.Port {
-					return fmt.Errorf("checkLoadBalancerResourcesConflicted: service port %s is trying to "+
+				if inboundNatPoolConflictsWithPort(pool, frontendIPConfigID, port) {
+					return fmt.Errorf("checkLoadBalancerResourcesConflicts: service port %s is trying to "+
 						"consume the port %d which is being in the range (%d-%d) of an existing "+
 						"inbound NAT pool %s with the same protocol %s and frontend IP config with ID %s",
 						port.Name,
@@ -1708,6 +1735,38 @@ func (az *Cloud) checkLoadBalancerResourcesConflicted(
 	return nil
 }
 
+func inboundNatPoolConflictsWithPort(pool network.InboundNatPool, frontendIPConfigID string, port v1.ServicePort) bool {
+	return pool.InboundNatPoolPropertiesFormat != nil &&
+		pool.FrontendIPConfiguration != nil &&
+		pool.FrontendIPConfiguration.ID != nil &&
+		strings.EqualFold(*pool.FrontendIPConfiguration.ID, frontendIPConfigID) &&
+		strings.EqualFold(string(pool.Protocol), string(port.Protocol)) &&
+		pool.FrontendPortRangeStart != nil &&
+		pool.FrontendPortRangeEnd != nil &&
+		*pool.FrontendPortRangeStart <= port.Port &&
+		*pool.FrontendPortRangeEnd >= port.Port
+}
+
+func inboundNatRuleConflictsWithPort(inboundNatRule network.InboundNatRule, frontendIPConfigID string, port v1.ServicePort) bool {
+	return inboundNatRule.InboundNatRulePropertiesFormat != nil &&
+		inboundNatRule.FrontendIPConfiguration != nil &&
+		inboundNatRule.FrontendIPConfiguration.ID != nil &&
+		strings.EqualFold(*inboundNatRule.FrontendIPConfiguration.ID, frontendIPConfigID) &&
+		strings.EqualFold(string(inboundNatRule.Protocol), string(port.Protocol)) &&
+		inboundNatRule.FrontendPort != nil &&
+		*inboundNatRule.FrontendPort == port.Port
+}
+
+func lbRuleConflictsWithPort(rule network.LoadBalancingRule, frontendIPConfigID string, port v1.ServicePort) bool {
+	return rule.LoadBalancingRulePropertiesFormat != nil &&
+		rule.FrontendIPConfiguration != nil &&
+		rule.FrontendIPConfiguration.ID != nil &&
+		strings.EqualFold(*rule.FrontendIPConfiguration.ID, frontendIPConfigID) &&
+		strings.EqualFold(string(rule.Protocol), string(port.Protocol)) &&
+		rule.FrontendPort != nil &&
+		*rule.FrontendPort == port.Port
+}
+
 func parseHealthProbeProtocolAndPath(service *v1.Service) (string, string) {
 	var protocol, path string
 	if v, ok := service.Annotations[ServiceAnnotationLoadBalancerHealthProbeProtocol]; ok {
@@ -1725,7 +1784,7 @@ func parseHealthProbeProtocolAndPath(service *v1.Service) (string, string) {
 	return protocol, path
 }
 
-func (az *Cloud) reconcileLoadBalancerRule(
+func (az *Cloud) getExpectedLBRules(
 	service *v1.Service,
 	wantLb bool,
 	lbFrontendIPConfigID string,
@@ -1752,7 +1811,7 @@ func (az *Cloud) reconcileLoadBalancerRule(
 	var expectedRules []network.LoadBalancingRule
 	for _, port := range ports {
 		lbRuleName := az.getLoadBalancerRuleName(service, port.Protocol, port.Port)
-		klog.V(2).Infof("reconcileLoadBalancerRule lb name (%s) rule name (%s)", lbName, lbRuleName)
+		klog.V(2).Infof("getExpectedLBRules lb name (%s) rule name (%s)", lbName, lbRuleName)
 
 		transportProto, _, probeProto, err := getProtocolsFromKubernetesProtocol(port.Protocol)
 		if err != nil {
@@ -1906,68 +1965,39 @@ func (az *Cloud) reconcileSecurityGroup(clusterName string, service *v1.Service,
 		}
 		sourceAddressPrefixes = append(sourceAddressPrefixes, serviceTags...)
 	}
-	expectedSecurityRules := []network.SecurityRule{}
 
-	if wantLb {
-		expectedSecurityRules = make([]network.SecurityRule, len(ports)*len(sourceAddressPrefixes))
+	expectedSecurityRules, err := az.getExpectedSecurityRules(wantLb, ports, sourceAddressPrefixes, service, destinationIPAddress, sourceRanges)
+	if err != nil {
+		return nil, err
+	}
 
-		for i, port := range ports {
-			_, securityProto, _, err := getProtocolsFromKubernetesProtocol(port.Protocol)
-			if err != nil {
-				return nil, err
-			}
-			for j := range sourceAddressPrefixes {
-				ix := i*len(sourceAddressPrefixes) + j
-				securityRuleName := az.getSecurityRuleName(service, port, sourceAddressPrefixes[j])
-				expectedSecurityRules[ix] = network.SecurityRule{
-					Name: to.StringPtr(securityRuleName),
-					SecurityRulePropertiesFormat: &network.SecurityRulePropertiesFormat{
-						Protocol:                 *securityProto,
-						SourcePortRange:          to.StringPtr("*"),
-						DestinationPortRange:     to.StringPtr(strconv.Itoa(int(port.Port))),
-						SourceAddressPrefix:      to.StringPtr(sourceAddressPrefixes[j]),
-						DestinationAddressPrefix: to.StringPtr(destinationIPAddress),
-						Access:                   network.SecurityRuleAccessAllow,
-						Direction:                network.SecurityRuleDirectionInbound,
-					},
-				}
-			}
-		}
+	// update security rules
+	dirtySg, updatedRules, err := az.reconcileSecurityRules(sg, service, serviceName, wantLb, expectedSecurityRules, ports, sourceAddressPrefixes, destinationIPAddress)
+	if err != nil {
+		return nil, err
+	}
 
-		shouldAddDenyRule := false
-		if len(sourceRanges) > 0 && !servicehelpers.IsAllowAll(sourceRanges) {
-			if v, ok := service.Annotations[ServiceAnnotationDenyAllExceptLoadBalancerSourceRanges]; ok && strings.EqualFold(v, trueAnnotationValue) {
-				shouldAddDenyRule = true
-			}
-		}
-		if shouldAddDenyRule {
-			for _, port := range ports {
-				_, securityProto, _, err := getProtocolsFromKubernetesProtocol(port.Protocol)
-				if err != nil {
-					return nil, err
-				}
-				securityRuleName := az.getSecurityRuleName(service, port, "deny_all")
-				expectedSecurityRules = append(expectedSecurityRules, network.SecurityRule{
-					Name: to.StringPtr(securityRuleName),
-					SecurityRulePropertiesFormat: &network.SecurityRulePropertiesFormat{
-						Protocol:                 *securityProto,
-						SourcePortRange:          to.StringPtr("*"),
-						DestinationPortRange:     to.StringPtr(strconv.Itoa(int(port.Port))),
-						SourceAddressPrefix:      to.StringPtr("*"),
-						DestinationAddressPrefix: to.StringPtr(destinationIPAddress),
-						Access:                   network.SecurityRuleAccessDeny,
-						Direction:                network.SecurityRuleDirectionInbound,
-					},
-				})
-			}
-		}
+	changed := az.ensureSecurityGroupTagged(&sg)
+	if changed {
+		dirtySg = true
 	}
 
-	for _, r := range expectedSecurityRules {
-		klog.V(10).Infof("Expecting security rule for %s: %s:%s -> %s:%s", service.Name, *r.SourceAddressPrefix, *r.SourcePortRange, *r.DestinationAddressPrefix, *r.DestinationPortRange)
+	if dirtySg {
+		sg.SecurityRules = &updatedRules
+		klog.V(2).Infof("reconcileSecurityGroup for service(%s): sg(%s) - updating", serviceName, *sg.Name)
+		klog.V(10).Infof("CreateOrUpdateSecurityGroup(%q): start", *sg.Name)
+		err := az.CreateOrUpdateSecurityGroup(sg)
+		if err != nil {
+			klog.V(2).Infof("ensure(%s) abort backoff: sg(%s) - updating", serviceName, *sg.Name)
+			return nil, err
+		}
+		klog.V(10).Infof("CreateOrUpdateSecurityGroup(%q): end", *sg.Name)
+		_ = az.nsgCache.Delete(to.String(sg.Name))
 	}
+	return &sg, nil
+}
 
-	// update security rules
+func (az *Cloud) reconcileSecurityRules(sg network.SecurityGroup, service *v1.Service, serviceName string, wantLb bool, expectedSecurityRules []network.SecurityRule, ports []v1.ServicePort, sourceAddressPrefixes []string, destinationIPAddress string) (bool, []network.SecurityRule, error) {
 	dirtySg := false
 	var updatedRules []network.SecurityRule
 	if sg.SecurityGroupPropertiesFormat != nil && sg.SecurityGroupPropertiesFormat.SecurityRules != nil {
@@ -1996,6 +2026,7 @@ func (az *Cloud) reconcileSecurityGroup(clusterName string, service *v1.Service,
 			}
 		}
 	}
+
 	// update security rules: if the service uses a shared rule and is being deleted,
 	// then remove it from the shared rule
 	if useSharedSecurityRule(service) && !wantLb {
@@ -2005,17 +2036,17 @@ func (az *Cloud) reconcileSecurityGroup(clusterName string, service *v1.Service,
 				sharedIndex, sharedRule, sharedRuleFound := findSecurityRuleByName(updatedRules, sharedRuleName)
 				if !sharedRuleFound {
 					klog.V(4).Infof("Expected to find shared rule %s for service %s being deleted, but did not", sharedRuleName, service.Name)
-					return nil, fmt.Errorf("expected to find shared rule %s for service %s being deleted, but did not", sharedRuleName, service.Name)
+					return false, nil, fmt.Errorf("expected to find shared rule %s for service %s being deleted, but did not", sharedRuleName, service.Name)
 				}
 				if sharedRule.DestinationAddressPrefixes == nil {
 					klog.V(4).Infof("Expected to have array of destinations in shared rule for service %s being deleted, but did not", service.Name)
-					return nil, fmt.Errorf("expected to have array of destinations in shared rule for service %s being deleted, but did not", service.Name)
+					return false, nil, fmt.Errorf("expected to have array of destinations in shared rule for service %s being deleted, but did not", service.Name)
 				}
 				existingPrefixes := *sharedRule.DestinationAddressPrefixes
 				addressIndex, found := findIndex(existingPrefixes, destinationIPAddress)
 				if !found {
 					klog.V(4).Infof("Expected to find destination address %s in shared rule %s for service %s being deleted, but did not", destinationIPAddress, sharedRuleName, service.Name)
-					return nil, fmt.Errorf("expected to find destination address %s in shared rule %s for service %s being deleted, but did not", destinationIPAddress, sharedRuleName, service.Name)
+					return false, nil, fmt.Errorf("expected to find destination address %s in shared rule %s for service %s being deleted, but did not", destinationIPAddress, sharedRuleName, service.Name)
 				}
 				if len(existingPrefixes) == 1 {
 					updatedRules = append(updatedRules[:sharedIndex], updatedRules[sharedIndex+1:]...)
@@ -2057,7 +2088,7 @@ func (az *Cloud) reconcileSecurityGroup(clusterName string, service *v1.Service,
 
 			nextAvailablePriority, err := getNextAvailablePriority(updatedRules)
 			if err != nil {
-				return nil, err
+				return false, nil, err
 			}
 
 			expectedRule.Priority = to.Int32Ptr(nextAvailablePriority)
@@ -2069,25 +2100,71 @@ func (az *Cloud) reconcileSecurityGroup(clusterName string, service *v1.Service,
 	for _, r := range updatedRules {
 		klog.V(10).Infof("Updated security rule while processing %s: %s:%s -> %s:%s", service.Name, logSafe(r.SourceAddressPrefix), logSafe(r.SourcePortRange), logSafeCollection(r.DestinationAddressPrefix, r.DestinationAddressPrefixes), logSafe(r.DestinationPortRange))
 	}
+	return dirtySg, updatedRules, nil
+}
 
-	changed := az.ensureSecurityGroupTagged(&sg)
-	if changed {
-		dirtySg = true
-	}
+func (az *Cloud) getExpectedSecurityRules(wantLb bool, ports []v1.ServicePort, sourceAddressPrefixes []string, service *v1.Service, destinationIPAddress string, sourceRanges utilnet.IPNetSet) ([]network.SecurityRule, error) {
+	expectedSecurityRules := []network.SecurityRule{}
 
-	if dirtySg {
-		sg.SecurityRules = &updatedRules
-		klog.V(2).Infof("reconcileSecurityGroup for service(%s): sg(%s) - updating", serviceName, *sg.Name)
-		klog.V(10).Infof("CreateOrUpdateSecurityGroup(%q): start", *sg.Name)
-		err := az.CreateOrUpdateSecurityGroup(sg)
-		if err != nil {
-			klog.V(2).Infof("ensure(%s) abort backoff: sg(%s) - updating", serviceName, *sg.Name)
-			return nil, err
+	if wantLb {
+		expectedSecurityRules = make([]network.SecurityRule, len(ports)*len(sourceAddressPrefixes))
+
+		for i, port := range ports {
+			_, securityProto, _, err := getProtocolsFromKubernetesProtocol(port.Protocol)
+			if err != nil {
+				return nil, err
+			}
+			for j := range sourceAddressPrefixes {
+				ix := i*len(sourceAddressPrefixes) + j
+				securityRuleName := az.getSecurityRuleName(service, port, sourceAddressPrefixes[j])
+				expectedSecurityRules[ix] = network.SecurityRule{
+					Name: to.StringPtr(securityRuleName),
+					SecurityRulePropertiesFormat: &network.SecurityRulePropertiesFormat{
+						Protocol:                 *securityProto,
+						SourcePortRange:          to.StringPtr("*"),
+						DestinationPortRange:     to.StringPtr(strconv.Itoa(int(port.Port))),
+						SourceAddressPrefix:      to.StringPtr(sourceAddressPrefixes[j]),
+						DestinationAddressPrefix: to.StringPtr(destinationIPAddress),
+						Access:                   network.SecurityRuleAccessAllow,
+						Direction:                network.SecurityRuleDirectionInbound,
+					},
+				}
+			}
+		}
+
+		shouldAddDenyRule := false
+		if len(sourceRanges) > 0 && !servicehelpers.IsAllowAll(sourceRanges) {
+			if v, ok := service.Annotations[ServiceAnnotationDenyAllExceptLoadBalancerSourceRanges]; ok && strings.EqualFold(v, trueAnnotationValue) {
+				shouldAddDenyRule = true
+			}
+		}
+		if shouldAddDenyRule {
+			for _, port := range ports {
+				_, securityProto, _, err := getProtocolsFromKubernetesProtocol(port.Protocol)
+				if err != nil {
+					return nil, err
+				}
+				securityRuleName := az.getSecurityRuleName(service, port, "deny_all")
+				expectedSecurityRules = append(expectedSecurityRules, network.SecurityRule{
+					Name: to.StringPtr(securityRuleName),
+					SecurityRulePropertiesFormat: &network.SecurityRulePropertiesFormat{
+						Protocol:                 *securityProto,
+						SourcePortRange:          to.StringPtr("*"),
+						DestinationPortRange:     to.StringPtr(strconv.Itoa(int(port.Port))),
+						SourceAddressPrefix:      to.StringPtr("*"),
+						DestinationAddressPrefix: to.StringPtr(destinationIPAddress),
+						Access:                   network.SecurityRuleAccessDeny,
+						Direction:                network.SecurityRuleDirectionInbound,
+					},
+				})
+			}
 		}
-		klog.V(10).Infof("CreateOrUpdateSecurityGroup(%q): end", *sg.Name)
-		_ = az.nsgCache.Delete(to.String(sg.Name))
 	}
-	return &sg, nil
+
+	for _, r := range expectedSecurityRules {
+		klog.V(10).Infof("Expecting security rule for %s: %s:%s -> %s:%s", service.Name, *r.SourceAddressPrefix, *r.SourcePortRange, *r.DestinationAddressPrefix, *r.DestinationPortRange)
+	}
+	return expectedSecurityRules, nil
 }
 
 func (az *Cloud) shouldUpdateLoadBalancer(clusterName string, service *v1.Service) bool {
@@ -2339,14 +2416,57 @@ func (az *Cloud) reconcilePublicIP(clusterName string, service *v1.Service, lbNa
 		return nil, err
 	}
 
+	discoveredDesiredPublicIP, pipsToBeDeleted, deletedDesiredPublicIP, pipsToBeUpdated, err := az.getPublicIPUpdates(clusterName, service, pips, wantLb, isInternal, desiredPipName, serviceName, serviceIPTagRequest, shouldPIPExisted)
+	if err != nil {
+		return nil, err
+	}
+
+	var deleteFuncs, updateFuncs []func() error
+	for _, pip := range pipsToBeUpdated {
+		pipCopy := *pip
+		updateFuncs = append(updateFuncs, func() error {
+			klog.V(2).Infof("reconcilePublicIP for service(%s): pip(%s) - updating", serviceName, *pip.Name)
+			return az.CreateOrUpdatePIP(service, pipResourceGroup, pipCopy)
+		})
+	}
+	errs := utilerrors.AggregateGoroutines(updateFuncs...)
+	if errs != nil {
+		return nil, utilerrors.Flatten(errs)
+	}
+
+	for _, pip := range pipsToBeDeleted {
+		pipCopy := *pip
+		deleteFuncs = append(deleteFuncs, func() error {
+			klog.V(2).Infof("reconcilePublicIP for service(%s): pip(%s) - deleting", serviceName, *pip.Name)
+			return az.safeDeletePublicIP(service, pipResourceGroup, &pipCopy, lb)
+		})
+	}
+	errs = utilerrors.AggregateGoroutines(deleteFuncs...)
+	if errs != nil {
+		return nil, utilerrors.Flatten(errs)
+	}
+
+	if !isInternal && wantLb {
+		// Confirm desired public ip resource exists
+		var pip *network.PublicIPAddress
+		domainNameLabel, found := getPublicIPDomainNameLabel(service)
+		errorIfPublicIPDoesNotExist := shouldPIPExisted && discoveredDesiredPublicIP && !deletedDesiredPublicIP
+		if pip, err = az.ensurePublicIPExists(service, desiredPipName, domainNameLabel, clusterName, errorIfPublicIPDoesNotExist, found); err != nil {
+			return nil, err
+		}
+		return pip, nil
+	}
+	return nil, nil
+}
+
+func (az *Cloud) getPublicIPUpdates(clusterName string, service *v1.Service, pips []network.PublicIPAddress, wantLb bool, isInternal bool, desiredPipName string, serviceName string, serviceIPTagRequest serviceIPTagRequest, serviceAnnotationRequestsNamedPublicIP bool) (bool, []*network.PublicIPAddress, bool, []*network.PublicIPAddress, error) {
 	var (
-		serviceAnnotationRequestsNamedPublicIP = shouldPIPExisted
-		discoveredDesiredPublicIP              bool
-		deletedDesiredPublicIP                 bool
-		pipsToBeDeleted                        []*network.PublicIPAddress
-		pipsToBeUpdated                        []*network.PublicIPAddress
+		err                       error
+		discoveredDesiredPublicIP bool
+		deletedDesiredPublicIP    bool
+		pipsToBeDeleted           []*network.PublicIPAddress
+		pipsToBeUpdated           []*network.PublicIPAddress
 	)
-
 	for i := range pips {
 		pip := pips[i]
 		pipName := *pip.Name
@@ -2363,7 +2483,7 @@ func (az *Cloud) reconcilePublicIP(clusterName string, service *v1.Service, lbNa
 				klog.V(2).Infof("reconcilePublicIP for service(%s): unbinding the service from pip %s", serviceName, *pip.Name)
 				err = unbindServiceFromPIP(&pip, serviceName)
 				if err != nil {
-					return nil, err
+					return false, nil, false, nil, err
 				}
 				dirtyPIP = true
 			}
@@ -2394,45 +2514,9 @@ func (az *Cloud) reconcilePublicIP(clusterName string, service *v1.Service, lbNa
 	}
 
 	if !isInternal && serviceAnnotationRequestsNamedPublicIP && !discoveredDesiredPublicIP && wantLb {
-		return nil, fmt.Errorf("reconcilePublicIP for service(%s): pip(%s) not found", serviceName, desiredPipName)
-	}
-
-	var deleteFuncs, updateFuncs []func() error
-	for _, pip := range pipsToBeUpdated {
-		pipCopy := *pip
-		updateFuncs = append(updateFuncs, func() error {
-			klog.V(2).Infof("reconcilePublicIP for service(%s): pip(%s) - updating", serviceName, *pip.Name)
-			return az.CreateOrUpdatePIP(service, pipResourceGroup, pipCopy)
-		})
-	}
-	errs := utilerrors.AggregateGoroutines(updateFuncs...)
-	if errs != nil {
-		return nil, utilerrors.Flatten(errs)
-	}
-
-	for _, pip := range pipsToBeDeleted {
-		pipCopy := *pip
-		deleteFuncs = append(deleteFuncs, func() error {
-			klog.V(2).Infof("reconcilePublicIP for service(%s): pip(%s) - deleting", serviceName, *pip.Name)
-			return az.safeDeletePublicIP(service, pipResourceGroup, &pipCopy, lb)
-		})
-	}
-	errs = utilerrors.AggregateGoroutines(deleteFuncs...)
-	if errs != nil {
-		return nil, utilerrors.Flatten(errs)
-	}
-
-	if !isInternal && wantLb {
-		// Confirm desired public ip resource exists
-		var pip *network.PublicIPAddress
-		domainNameLabel, found := getPublicIPDomainNameLabel(service)
-		errorIfPublicIPDoesNotExist := serviceAnnotationRequestsNamedPublicIP && discoveredDesiredPublicIP && !deletedDesiredPublicIP
-		if pip, err = az.ensurePublicIPExists(service, desiredPipName, domainNameLabel, clusterName, errorIfPublicIPDoesNotExist, found); err != nil {
-			return nil, err
-		}
-		return pip, nil
+		return false, nil, false, nil, fmt.Errorf("reconcilePublicIP for service(%s): pip(%s) not found", serviceName, desiredPipName)
 	}
-	return nil, nil
+	return discoveredDesiredPublicIP, pipsToBeDeleted, deletedDesiredPublicIP, pipsToBeUpdated, err
 }
 
 // safeDeletePublicIP deletes public IP by removing its reference first.


Test patch:
diff --git a/pkg/provider/azure_loadbalancer_test.go b/pkg/provider/azure_loadbalancer_test.go
index 9ab439b8b..9b193dc41 100644
--- a/pkg/provider/azure_loadbalancer_test.go
+++ b/pkg/provider/azure_loadbalancer_test.go
@@ -1609,12 +1609,12 @@ func TestReconcileLoadBalancerRule(t *testing.T) {
 		expectedErr     error
 	}{
 		{
-			desc:    "reconcileLoadBalancerRule shall return nil if wantLb is false",
+			desc:    "getExpectedLBRules shall return nil if wantLb is false",
 			service: getTestService("test1", v1.ProtocolTCP, nil, false, 80),
 			wantLb:  false,
 		},
 		{
-			desc:            "reconcileLoadBalancerRule shall return corresponding probe and lbRule(blb)",
+			desc:            "getExpectedLBRules shall return corresponding probe and lbRule(blb)",
 			service:         getTestService("test1", v1.ProtocolTCP, map[string]string{"service.beta.kubernetes.io/azure-load-balancer-disable-tcp-reset": "true"}, false, 80),
 			loadBalancerSku: "basic",
 			wantLb:          true,
@@ -1622,7 +1622,7 @@ func TestReconcileLoadBalancerRule(t *testing.T) {
 			expectedRules:   getDefaultTestRules(false),
 		},
 		{
-			desc:            "reconcileLoadBalancerRule shall return corresponding probe and lbRule (slb without tcp reset)",
+			desc:            "getExpectedLBRules shall return corresponding probe and lbRule (slb without tcp reset)",
 			service:         getTestService("test1", v1.ProtocolTCP, map[string]string{"service.beta.kubernetes.io/azure-load-balancer-disable-tcp-reset": "True"}, false, 80),
 			loadBalancerSku: "standard",
 			wantLb:          true,
@@ -1630,7 +1630,7 @@ func TestReconcileLoadBalancerRule(t *testing.T) {
 			expectedRules:   getDefaultTestRules(true),
 		},
 		{
-			desc:            "reconcileLoadBalancerRule shall return corresponding probe and lbRule(slb with tcp reset)",
+			desc:            "getExpectedLBRules shall return corresponding probe and lbRule(slb with tcp reset)",
 			service:         getTestService("test1", v1.ProtocolTCP, nil, false, 80),
 			loadBalancerSku: "standard",
 			wantLb:          true,
@@ -1638,7 +1638,7 @@ func TestReconcileLoadBalancerRule(t *testing.T) {
 			expectedRules:   getDefaultTestRules(true),
 		},
 		{
-			desc:            "reconcileLoadBalancerRule shall respect the probe protocol and path configuration in the config file",
+			desc:            "getExpectedLBRules shall respect the probe protocol and path configuration in the config file",
 			service:         getTestService("test1", v1.ProtocolTCP, nil, false, 80),
 			loadBalancerSku: "standard",
 			wantLb:          true,
@@ -1658,7 +1658,7 @@ func TestReconcileLoadBalancerRule(t *testing.T) {
 		if test.probePath != "" {
 			service.Annotations[ServiceAnnotationLoadBalancerHealthProbeRequestPath] = test.probePath
 		}
-		probe, lbrule, err := az.reconcileLoadBalancerRule(&test.service, test.wantLb,
+		probe, lbrule, err := az.getExpectedLBRules(&test.service, test.wantLb,
 			"frontendIPConfigID", "backendPoolID", "lbname", to.Int32Ptr(0))
 
 		if test.expectedErr != nil {
@@ -3528,7 +3528,7 @@ func TestCheckLoadBalancerResourcesConflicted(t *testing.T) {
 		expectedErr bool
 	}{
 		{
-			desc: "checkLoadBalancerResourcesConflicted should report the conflict error if " +
+			desc: "checkLoadBalancerResourcesConflicts should report the conflict error if " +
 				"there is a conflicted loadBalancing rule",
 			existingLB: &network.LoadBalancer{
 				Name: to.StringPtr("lb"),
@@ -3548,7 +3548,7 @@ func TestCheckLoadBalancerResourcesConflicted(t *testing.T) {
 			expectedErr: true,
 		},
 		{
-			desc: "checkLoadBalancerResourcesConflicted should report the conflict error if " +
+			desc: "checkLoadBalancerResourcesConflicts should report the conflict error if " +
 				"there is a conflicted inbound NAT rule",
 			existingLB: &network.LoadBalancer{
 				Name: to.StringPtr("lb"),
@@ -3568,7 +3568,7 @@ func TestCheckLoadBalancerResourcesConflicted(t *testing.T) {
 			expectedErr: true,
 		},
 		{
-			desc: "checkLoadBalancerResourcesConflicted should report the conflict error if " +
+			desc: "checkLoadBalancerResourcesConflicts should report the conflict error if " +
 				"there is a conflicted inbound NAT pool",
 			existingLB: &network.LoadBalancer{
 				Name: to.StringPtr("lb"),
@@ -3589,7 +3589,7 @@ func TestCheckLoadBalancerResourcesConflicted(t *testing.T) {
 			expectedErr: true,
 		},
 		{
-			desc: "checkLoadBalancerResourcesConflicted should not report the conflict error if there " +
+			desc: "checkLoadBalancerResourcesConflicts should not report the conflict error if there " +
 				"is no conflicted loadBalancer resources",
 			existingLB: &network.LoadBalancer{
 				Name: to.StringPtr("lb"),
@@ -3631,7 +3631,7 @@ func TestCheckLoadBalancerResourcesConflicted(t *testing.T) {
 	}
 
 	for _, testCase := range testCases {
-		err := az.checkLoadBalancerResourcesConflicted(testCase.existingLB, fipID, &service)
+		err := az.checkLoadBalancerResourcesConflicts(testCase.existingLB, fipID, &service)
 		assert.Equal(t, testCase.expectedErr, err != nil, testCase.desc)
 	}
 }
diff --git a/pkg/provider/azure_test.go b/pkg/provider/azure_test.go
index 78dd8b6e8..51f7fcd05 100644
--- a/pkg/provider/azure_test.go
+++ b/pkg/provider/azure_test.go
@@ -2020,7 +2020,7 @@ disableAvailabilitySetNodes: true
 	validateConfig(t, config)
 }
 
-func validateConfig(t *testing.T, config string) {
+func validateConfig(t *testing.T, config string) { //nolint
 	azureCloud := getCloudFromConfig(t, config)
 
 	if azureCloud.TenantID != "--tenant-id--" {
@@ -2889,6 +2889,7 @@ func TestCanCombineSharedAndPrivateRulesInSameGroup(t *testing.T) {
 	ctrl := gomock.NewController(t)
 	defer ctrl.Finish()
 	az := GetTestCloud(ctrl)
+	var err error
 
 	svc1 := getTestService("servicesr1", v1.ProtocolTCP, nil, false, 4444)
 	svc1.Spec.LoadBalancerIP = testIP1
@@ -2910,6 +2911,8 @@ func TestCanCombineSharedAndPrivateRulesInSameGroup(t *testing.T) {
 	svc5.Spec.LoadBalancerIP = "192.168.22.33"
 	svc5.Annotations[ServiceAnnotationSharedSecurityRule] = "false"
 
+	testServices := []v1.Service{svc1, svc2, svc3, svc4, svc5}
+
 	testRuleName23 := testRuleName2
 	expectedRuleName4 := az.getSecurityRuleName(&svc4, v1.ServicePort{Port: 4444, Protocol: v1.ProtocolTCP}, "Internet")
 	expectedRuleName5 := az.getSecurityRuleName(&svc5, v1.ServicePort{Port: 8888, Protocol: v1.ProtocolTCP}, "Internet")
@@ -2917,29 +2920,11 @@ func TestCanCombineSharedAndPrivateRulesInSameGroup(t *testing.T) {
 	sg := getTestSecurityGroup(az)
 	setMockSecurityGroup(az, ctrl, sg)
 
-	_, err := az.reconcileSecurityGroup(testClusterName, &svc1, to.StringPtr(svc1.Spec.LoadBalancerIP), true)
-	if err != nil {
-		t.Errorf("Unexpected error adding svc1: %q", err)
-	}
-
-	_, err = az.reconcileSecurityGroup(testClusterName, &svc2, to.StringPtr(svc2.Spec.LoadBalancerIP), true)
-	if err != nil {
-		t.Errorf("Unexpected error adding svc2: %q", err)
-	}
-
-	_, err = az.reconcileSecurityGroup(testClusterName, &svc3, to.StringPtr(svc3.Spec.LoadBalancerIP), true)
-	if err != nil {
-		t.Errorf("Unexpected error adding svc3: %q", err)
-	}
-
-	_, err = az.reconcileSecurityGroup(testClusterName, &svc4, to.StringPtr(svc4.Spec.LoadBalancerIP), true)
-	if err != nil {
-		t.Errorf("Unexpected error adding svc4: %q", err)
-	}
-
-	sg, err = az.reconcileSecurityGroup(testClusterName, &svc5, to.StringPtr(svc5.Spec.LoadBalancerIP), true)
-	if err != nil {
-		t.Errorf("Unexpected error adding svc4: %q", err)
+	for i, svc := range testServices {
+		_, err := az.reconcileSecurityGroup(testClusterName, &svc, to.StringPtr(svc.Spec.LoadBalancerIP), true)
+		if err != nil {
+			t.Errorf("Unexpected error adding svc%d: %q", i+1, err)
+		}
 	}
 
 	validateSecurityGroup(t, sg, svc1, svc2, svc3, svc4, svc5)