Repository: kubernetes-sigs/cloud-provider-azure
Pull request #4380 (2023-07-27 08:09:48)

Support for custom health probe port(not in spec) 
<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?
/kind feature
<!--
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature
/kind design

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

#### What this PR does / why we need it:
Update probe port validation and support custom health probe port(not in spec) 
#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Fixes #

#### Special notes for your reviewer:

#### Does this PR introduce a user-facing change?
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

For more information on release notes see: https://git.k8s.io/community/contributors/guide/release-notes.md
-->
```release-note
Health probe port can be any port assigned by customer.
```

#### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

<!--
This section can be blank if this pull request does not require a release note.

When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.

See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files

Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
-->
```docs

```


k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-unit/1684476173737791488) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-capz/1684476173632933888) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4380). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-unit/1684476173737791488) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-capz/1684476173632933888) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1684476173452578816) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4380). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-unit/1684476173737791488) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-capz/1684476173632933888) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1684476173452578816) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1684476173687459840) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4380). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-unit/1684476173737791488) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-capz/1684476173632933888) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1684476173452578816) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1684476173687459840) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1684476173515493376) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4380). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @MartinForReal: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-unit | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-unit/1684476173737791488) | true | `/test pull-cloud-provider-azure-unit`
pull-cloud-provider-azure-e2e-ccm-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-capz/1684476173632933888) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1684476173452578816) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1684476173687459840) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmssflex-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-ccm-vmssflex-capz/1684476173515493376) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmssflex-capz`
pull-cloud-provider-azure-e2e-capz | 3c60378f426fcf645a0a168e53447d8f94386159 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/4380/pull-cloud-provider-azure-e2e-capz/1684476173578407936) | true | `/test pull-cloud-provider-azure-e2e-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=4380). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3AMartinForReal). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
MartinForReal: /cherrypick release-1.27
MartinForReal: /cherrypick release-1.26
k8s-infra-cherrypick-robot: @MartinForReal: once the present PR merges, I will cherry-pick it on top of release-1.26 in a new PR and assign it to you.

<details>

In response to [this](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/4380#issuecomment-1653558768):

>/cherrypick release-1.26


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>
MartinForReal: /cherrypick release-1.25
k8s-infra-cherrypick-robot: @MartinForReal: once the present PR merges, I will cherry-pick it on top of release-1.25 in a new PR and assign it to you.

<details>

In response to [this](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/4380#issuecomment-1653558911):

>/cherrypick release-1.25


Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>

Patch:
diff --git a/pkg/provider/azure_loadbalancer_healthprobe.go b/pkg/provider/azure_loadbalancer_healthprobe.go
index 6d3eb8146..8664f907a 100644
--- a/pkg/provider/azure_loadbalancer_healthprobe.go
+++ b/pkg/provider/azure_loadbalancer_healthprobe.go
@@ -99,29 +99,22 @@ func (az *Cloud) buildHealthProbeRulesForPort(serviceManifest *v1.Service, port
 		if s == nil {
 			return nil
 		}
+		//not a integer
+		for _, item := range serviceManifest.Spec.Ports {
+			if strings.EqualFold(item.Name, *s) {
+				//found the port
+				return nil
+			}
+		}
 		//nolint:gosec
 		port, err := strconv.Atoi(*s)
 		if err != nil {
-			//not a integer
-			for _, item := range serviceManifest.Spec.Ports {
-				if strings.EqualFold(item.Name, *s) {
-					//found the port
-					return nil
-				}
-			}
 			return fmt.Errorf("port %s not found in service", *s)
 		}
 		if port < 0 || port > 65535 {
 			return fmt.Errorf("port %d is out of range", port)
 		}
-		for _, item := range serviceManifest.Spec.Ports {
-			//nolint:gosec
-			if item.Port == int32(port) {
-				//found the port
-				return nil
-			}
-		}
-		return fmt.Errorf("port %s not found in service", *s)
+		return nil
 	})
 	if err != nil {
 		return nil, fmt.Errorf("failed to parse annotation %s: %w", consts.BuildHealthProbeAnnotationKeyForPort(port.Port, consts.HealthProbeParamsPort), err)
@@ -129,7 +122,7 @@ func (az *Cloud) buildHealthProbeRulesForPort(serviceManifest *v1.Service, port
 
 	if probePort != nil {
 		//nolint:gosec
-		port, err := strconv.Atoi(*probePort)
+		port, err := strconv.ParseInt(*probePort, 10, 32)
 		if err != nil {
 			//not a integer
 			for _, item := range serviceManifest.Spec.Ports {
@@ -140,13 +133,20 @@ func (az *Cloud) buildHealthProbeRulesForPort(serviceManifest *v1.Service, port
 			}
 		} else {
 			// Not need to verify probePort is in correct range again.
+			var found bool
 			for _, item := range serviceManifest.Spec.Ports {
 				//nolint:gosec
 				if item.Port == int32(port) {
 					//found the port
 					properties.Port = pointer.Int32(item.NodePort)
+					found = true
+					break
 				}
 			}
+			if !found {
+				//nolint:gosec
+				properties.Port = pointer.Int32(int32(port))
+			}
 		}
 	}
 


Test patch:
diff --git a/pkg/provider/azure_loadbalancer_test.go b/pkg/provider/azure_loadbalancer_test.go
index 3c0789a3e..961fd93d8 100644
--- a/pkg/provider/azure_loadbalancer_test.go
+++ b/pkg/provider/azure_loadbalancer_test.go
@@ -2670,6 +2670,32 @@ func TestReconcileLoadBalancerRuleCommon(t *testing.T) {
 				},
 			},
 		},
+		{
+			desc: "getExpectedLBRules should support customize health probe port ",
+			service: getTestServiceDualStack("test1", v1.ProtocolTCP, map[string]string{
+				"service.beta.kubernetes.io/port_8000_health-probe_port": "5080",
+			}, 80, 8000),
+			expectedRules: map[bool][]network.LoadBalancingRule{
+				consts.IPVersionIPv4: {
+					getTestRule(false, 80, consts.IPVersionIPv4),
+					getTestRule(false, 8000, consts.IPVersionIPv4),
+				},
+				consts.IPVersionIPv6: {
+					getTestRule(false, 80, consts.IPVersionIPv6),
+					getTestRule(false, 8000, consts.IPVersionIPv6),
+				},
+			},
+			expectedProbes: map[bool][]network.Probe{
+				consts.IPVersionIPv4: {
+					getTestProbe("Tcp", "/", pointer.Int32(5), pointer.Int32(80), pointer.Int32(10080), pointer.Int32(2), consts.IPVersionIPv4),
+					getTestProbe("Tcp", "/", pointer.Int32(5), pointer.Int32(8000), pointer.Int32(5080), pointer.Int32(2), consts.IPVersionIPv4),
+				},
+				consts.IPVersionIPv6: {
+					getTestProbe("Tcp", "/", pointer.Int32(5), pointer.Int32(80), pointer.Int32(10080), pointer.Int32(2), consts.IPVersionIPv6),
+					getTestProbe("Tcp", "/", pointer.Int32(5), pointer.Int32(8000), pointer.Int32(5080), pointer.Int32(2), consts.IPVersionIPv6),
+				},
+			},
+		},
 	}
 	rulesDualStack := getDefaultTestRules(true)
 	for _, rules := range rulesDualStack {