Repository: kubernetes-sigs/cloud-provider-azure (go)

Issue:
UDP services may trigger unnecessary LoadBalancer updates
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks!

If the matter is security related, please disclose it privately via https://kubernetes.io/security/
-->


#### What happened:

While nothing changed to all LoadBalancer service specs, scaling node number or restart controller manager would trigger LoadBalancer update APIs. This is because `EnableTCPReset` value is different between expected LB rule and existing LB rule when the protocol is not TCP, and hence `equalLoadBalancingRulePropertiesFormat()` would treat the LB rule as updated.

And here are the related logs:

```
I0129 07:27:01.471484       1 azure_loadbalancer.go:1516] reconcileLoadBalancer for service(default/test): lb(test-rg/kubernetes) wantLb(true) resolved load balancer name
I0129 07:27:01.530862       1 azure_loadbalancer.go:2106] getExpectedLBRules lb name (kubernetes) rule name (a30cf55982b684c1993a23dd1e177c98-UDP-24899)
I0129 07:27:01.530954       1 azure_loadbalancer.go:1696] reconcileLoadBalancer for service (default/test)(true): lb rule(a30cf55982b684c1993a23dd1e177c98-UDP-24899) - dropping
```

#### What you expected to happen:

* `EnableTCPReset` should only be set in LB rule when the protocol is TCP.
* `equalLoadBalancingRulePropertiesFormat` should omit `EnableTCPReset` when protocol is not TCP.

#### How to reproduce it (as minimally and precisely as possible):

Create a UDP LoadBalancer service in the cluster and then restart controller manager. After a while, there would be new write operations in LoadBalancer activities （this could be observed on the Azure portal）.

#### Anything else we need to know?:

#### Environment:

- Kubernetes version (use `kubectl version`):
- Cloud provider or hardware configuration:
- OS (e.g: `cat /etc/os-release`):
- Kernel (e.g. `uname -a`):
- Install tools:
- Network plugin and version (if this is a network-related bug):
- Others:


Pull request description:
Fix incorrect EnableTCPReset for non-TCP protocols
Signed-off-by: Zhecheng Li <zhechengli@microsoft.com>

fixes #1087

<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?

<!--
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature
/kind design

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

#### What this PR does / why we need it:

#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

#### Special notes for your reviewer:

#### Does this PR introduce a user-facing change?
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

For more information on release notes see: https://git.k8s.io/community/contributors/guide/release-notes.md
-->
```release-note
This code change fixes the bug that UDP services would trigger unnecessary LoadBalancer updates. The root cause is that a field not working for non-TCP protocols is considered.
```

#### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

<!--
This section can be blank if this pull request does not require a release note.

When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.

See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files

Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
-->
```docs

```


Patch:
diff --git a/pkg/provider/azure_loadbalancer.go b/pkg/provider/azure_loadbalancer.go
index 2b3177ba7..a53d873e1 100644
--- a/pkg/provider/azure_loadbalancer.go
+++ b/pkg/provider/azure_loadbalancer.go
@@ -1994,7 +1994,7 @@ func (az *Cloud) getExpectedLBRules(
 		ports = []v1.ServicePort{}
 	}
 
-	var enableTCPReset *bool
+	var enableTCPReset, nilTCPReset *bool
 	if az.useStandardLoadBalancer() {
 		enableTCPReset = to.BoolPtr(true)
 	}
@@ -2076,6 +2076,10 @@ func (az *Cloud) getExpectedLBRules(
 			loadDistribution = network.LoadDistributionSourceIP
 		}
 
+		tcpReset := enableTCPReset
+		if port.Protocol != v1.ProtocolTCP {
+			tcpReset = nilTCPReset
+		}
 		expectedRule := network.LoadBalancingRule{
 			Name: &lbRuleName,
 			LoadBalancingRulePropertiesFormat: &network.LoadBalancingRulePropertiesFormat{
@@ -2090,7 +2094,7 @@ func (az *Cloud) getExpectedLBRules(
 				FrontendPort:        to.Int32Ptr(port.Port),
 				BackendPort:         to.Int32Ptr(port.Port),
 				DisableOutboundSnat: to.BoolPtr(az.disableLoadBalancerOutboundSNAT()),
-				EnableTCPReset:      enableTCPReset,
+				EnableTCPReset:      tcpReset,
 				EnableFloatingIP:    to.BoolPtr(true),
 			},
 		}
@@ -2874,14 +2878,21 @@ func equalLoadBalancingRulePropertiesFormat(s *network.LoadBalancingRuleProperti
 		return false
 	}
 
-	properties := reflect.DeepEqual(s.Protocol, t.Protocol) &&
-		reflect.DeepEqual(s.FrontendIPConfiguration, t.FrontendIPConfiguration) &&
+	properties := reflect.DeepEqual(s.Protocol, t.Protocol)
+	if !properties {
+		return false
+	}
+
+	if reflect.DeepEqual(s.Protocol, network.TransportProtocolTCP) {
+		properties = properties && reflect.DeepEqual(to.Bool(s.EnableTCPReset), to.Bool(t.EnableTCPReset))
+	}
+
+	properties = properties && reflect.DeepEqual(s.FrontendIPConfiguration, t.FrontendIPConfiguration) &&
 		reflect.DeepEqual(s.BackendAddressPool, t.BackendAddressPool) &&
 		reflect.DeepEqual(s.LoadDistribution, t.LoadDistribution) &&
 		reflect.DeepEqual(s.FrontendPort, t.FrontendPort) &&
 		reflect.DeepEqual(s.BackendPort, t.BackendPort) &&
 		reflect.DeepEqual(s.EnableFloatingIP, t.EnableFloatingIP) &&
-		reflect.DeepEqual(to.Bool(s.EnableTCPReset), to.Bool(t.EnableTCPReset)) &&
 		reflect.DeepEqual(to.Bool(s.DisableOutboundSnat), to.Bool(t.DisableOutboundSnat))
 
 	if wantLB && s.IdleTimeoutInMinutes != nil && t.IdleTimeoutInMinutes != nil {


Test patch:
diff --git a/pkg/provider/azure_loadbalancer_test.go b/pkg/provider/azure_loadbalancer_test.go
index 0cd436e7a..2c28e615a 100644
--- a/pkg/provider/azure_loadbalancer_test.go
+++ b/pkg/provider/azure_loadbalancer_test.go
@@ -1790,7 +1790,7 @@ func TestReconcileLoadBalancerRule(t *testing.T) {
 			}, false, 80),
 			loadBalancerSku: "standard",
 			wantLb:          true,
-			expectedRules:   getHATestRules(true, false, v1.ProtocolSCTP),
+			expectedRules:   getHATestRules(false, false, v1.ProtocolSCTP),
 		},
 		{
 			desc: "getExpectedLBRules shall return corresponding probe and lbRule (slb with HA enabled multi-ports services)",
@@ -4890,3 +4890,48 @@ func Test_getProbeIntervalInSecondsAndNumOfProbe(t *testing.T) {
 		})
 	}
 }
+
+func TestEqualLoadBalancingRulePropertiesFormat(t *testing.T) {
+	var enableTCPReset, disableTCPReset *bool = to.BoolPtr(true), to.BoolPtr(false)
+	var frontPort *int32 = to.Int32Ptr(80)
+
+	testcases := []struct {
+		s        *network.LoadBalancingRulePropertiesFormat
+		t        *network.LoadBalancingRulePropertiesFormat
+		wantLb   bool
+		expected bool
+	}{
+		{
+			s: &network.LoadBalancingRulePropertiesFormat{
+				Protocol:       network.TransportProtocolTCP,
+				EnableTCPReset: enableTCPReset,
+				FrontendPort:   frontPort,
+			},
+			t: &network.LoadBalancingRulePropertiesFormat{
+				Protocol:       network.TransportProtocolTCP,
+				EnableTCPReset: enableTCPReset,
+				FrontendPort:   frontPort,
+			},
+			wantLb:   true,
+			expected: true,
+		},
+		{
+			s: &network.LoadBalancingRulePropertiesFormat{
+				Protocol:       network.TransportProtocolUDP,
+				EnableTCPReset: disableTCPReset,
+				FrontendPort:   frontPort,
+			},
+			t: &network.LoadBalancingRulePropertiesFormat{
+				Protocol:       network.TransportProtocolUDP,
+				EnableTCPReset: enableTCPReset,
+				FrontendPort:   frontPort,
+			},
+			wantLb:   true,
+			expected: true,
+		},
+	}
+
+	for _, tc := range testcases {
+		assert.Equal(t, tc.expected, equalLoadBalancingRulePropertiesFormat(tc.s, tc.t, tc.wantLb))
+	}
+}