Repository: kubernetes-sigs/cloud-provider-azure
Pull request #2100 (2022-07-29 03:26:13)

VMSS Flex Support: Add vmssflex_cache.go and unit tests
<!--  Thanks for sending a pull request!  Here are some tips for you:

1. If this is your first time, please read our contributor guidelines: https://git.k8s.io/community/contributors/guide/first-contribution.md#your-first-contribution and developer guide https://git.k8s.io/community/contributors/devel/development.md#development-guide
2. Please label this pull request according to what type of issue you are addressing, especially if this is a release targeted pull request. For reference on required PR/issue labels, read here:
https://git.k8s.io/community/contributors/devel/sig-release/release.md#issuepr-kind-label
3. Ensure you have added or ran the appropriate tests for your PR: https://git.k8s.io/community/contributors/devel/sig-testing/testing.md
4. If you want *faster* PR reviews, read how: https://git.k8s.io/community/contributors/guide/pull-requests.md#best-practices-for-faster-reviews
5. If the PR is unfinished, see how to mark it: https://git.k8s.io/community/contributors/guide/pull-requests.md#marking-unfinished-pull-requests
-->

#### What type of PR is this?
/kind feature
<!--
Add one of the following kinds:
/kind bug
/kind cleanup
/kind documentation
/kind feature
/kind design

Optionally add one or more of the following kinds if applicable:
/kind api-change
/kind deprecation
/kind failing-test
/kind flake
/kind regression
-->

#### What this PR does / why we need it:
This PR is to support using VMSS Flex VMs as K8s cluster nodes. This PR added caches used for VMSS Flex nodes

This PR does the following things:
-  Define a FlexScaleSet struct to be used for managing VMSS Flex nodes
- Define the caches needed for VMSS Flex and the operations to query VMSS Flex VM info and VMSS Flex info from the caches
- Unit tests for functions to query vmss Flex related caches.

#### Which issue(s) this PR fixes:
<!--
*Automatically closes linked issue when PR is merged.
Usage: `Fixes #<issue number>`, or `Fixes (paste link of issue)`.
_If PR is about `failing-tests or flakes`, please post the related issues/tests in a comment and do not use `Fixes`_*
-->
Refer to #639 

#### Special notes for your reviewer:
Similar to VMSS Uniform, 2 types of caches are used for VMSS Flex:
- cache for list of VMSS Flex: vmssFlexCache
- cache for list of VMSS Flex VMs: vmssFlexVMCache, vmssFlexVMStatusCache
Both of the above caches are populated in batch behavior.

Note:

1. Batch query needed

VMSS Uniform API support query VM properties and IntanceView with one single API:
GET `https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachineScaleSets/{vmScaleSetName}/virtualMachines/{instanceId}?$expand=InstanceView&api-version=2022-03-01`
	
However, VMSS Flex API does not support adding IntanceView as expand. So 2 queries are needed:

- Get VM properties:

`GET https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Compute/virtualMachines?api-version=2022-03-01&$filter="virtualMachineScaleSet/id' eq /subscriptions/{subId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachineScaleSets/{vmssName}'"`
		

- Get InstanceView properties:

`GET https://management.azure.com/subscriptions/{subscriptionId}/providers/Microsoft.Compute/virtualMachines?api-version=2022-03-01&statusOnly=true&$filter="virtualMachineScaleSet/id' eq /subscriptions/{subId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachineScaleSets/{vmssName}'"`

So, two caches are used for holding VMSS Flex VM properties: vmssFlexVMCache, vmssFlexVMStatusCache

2. Infer VMSS name from VM name/ID
	When using VMSS Uniform, the VMSS name can be easily infered by VM name or VM ID
	
	VMSS ID: `/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachineScaleSets/{vmScaleSetName}`
	
	VMSS Uniform VM ID: (share the same prefix as VMSS ID)
	`/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachineScaleSets/{vmScaleSetName}/virtualMachines/{instanceId}`
	
	VMSS Uniform VM name:
	`{vmScaleSetName}_{random_characters}`
	
	VMSS Flex VM ID:
	`/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.Compute/virtualMachines/{vmName}`
	
	So when using Flex VM, we need to first query the VM to get its parent VMSS ID, then query all the VMs under the VMSS to populate the cache. To do so, vmssFlexVMnameToVmssID is used for store this mapping relationship.



#### Does this PR introduce a user-facing change?
<!--
If no, just write "NONE" in the release-note block below.
If yes, a release note is required:
Enter your extended release note in the block below. If the PR requires additional action from users switching to the new release, include the string "action required".

For more information on release notes see: https://git.k8s.io/community/contributors/guide/release-notes.md
-->
```release-note
NONE
```

#### Additional documentation e.g., KEPs (Kubernetes Enhancement Proposals), usage docs, etc.:

<!--
This section can be blank if this pull request does not require a release note.

When adding links which point to resources within git repositories, like
KEPs or supporting documentation, please reference a specific commit and avoid
linking directly to the master branch. This ensures that links reference a
specific point in time, rather than a document that may change over time.

See here for guidance on getting permanent links to files: https://help.github.com/en/articles/getting-permanent-links-to-files

Please use the following format for linking documentation:
- [KEP]: <link>
- [Usage]: <link>
- [Other doc]: <link>
-->
```docs
- [Other doc]: https://docs.microsoft.com/en-us/azure/virtual-machine-scale-sets/orchestration-modes-api-comparison
```


k8s-ci-robot: Hi @zmyzheng. Thanks for your PR.

I'm waiting for a [kubernetes-sigs](https://github.com/orgs/kubernetes-sigs/people) member to verify that this patch is reasonable to test. If it is, they should reply with `/ok-to-test` on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should [join the org](https://git.k8s.io/community/community-membership.md#member) to skip this step.

Once the patch is verified, the new status will be reflected by the `ok-to-test` label.

I understand the commands that are listed [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fcloud-provider-azure).

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository.
</details>

k8s-ci-robot: [APPROVALNOTIFIER] This PR is **NOT APPROVED**

This pull-request has been approved by: *<a href="https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2100#" title="Author self-approved">zmyzheng</a>*
**Once this PR has been reviewed and has the lgtm label**, please assign lzhecheng for approval by writing `/assign @lzhecheng` in a comment. For more information see:[The Kubernetes Code Review Process](https://git.k8s.io/community/contributors/guide/owners.md#the-code-review-process).

The full list of commands accepted by this bot can be found [here](https://go.k8s.io/bot-commands?repo=kubernetes-sigs%2Fcloud-provider-azure).

<details open>
Needs approval from an approver in each of these files:

- **[OWNERS](https://github.com/kubernetes-sigs/cloud-provider-azure/blob/master/OWNERS)**

Approvers can indicate their approval by writing `/approve` in a comment
Approvers can cancel approval by writing `/approve cancel` in a comment
</details>
<!-- META={"approvers":["lzhecheng"]} -->
netlify[bot]: ### <span aria-hidden="true">👷</span> Deploy Preview for *kubernetes-sigs-cloud-provide-azure* processing.


|  Name | Link |
|---------------------------------|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | ef9c25dab696b1424f54af8da77e1ddf090200b2 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/kubernetes-sigs-cloud-provide-azure/deploys/62e3535783d10d000bd5a1e5 |
netlify[bot]: ### <span aria-hidden="true">👷</span> Deploy Preview for *warm-cascaron-bb779f* processing.


|  Name | Link |
|---------------------------------|------------------------|
|<span aria-hidden="true">🔨</span> Latest commit | ef9c25dab696b1424f54af8da77e1ddf090200b2 |
|<span aria-hidden="true">🔍</span> Latest deploy log | https://app.netlify.com/sites/warm-cascaron-bb779f/deploys/62e3535719938e000960d9f2 |
coveralls: 
[![Coverage Status](https://coveralls.io/builds/51256408/badge)](https://coveralls.io/builds/51256408)

Coverage increased (+0.02%) to 79.929% when pulling **ef9c25dab696b1424f54af8da77e1ddf090200b2 on zmyzheng:vmssflex-cache** into **88b5ff48f290ab8bdb78db4d98f6cf9e936de009 on kubernetes-sigs:master**.

nilo19: /ok-to-test
k8s-ci-robot: @zmyzheng: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-check | ef9c25dab696b1424f54af8da77e1ddf090200b2 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1552895983518486528) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
zmyzheng: /retest
k8s-ci-robot: @zmyzheng: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-check | ef9c25dab696b1424f54af8da77e1ddf090200b2 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1552897862646370304) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-check | a95c795c0f194854302c3df600abea2152e4030e | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1552901670797578240) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-check | f15ba64649a279a667850569f18de4fb9c22bd11 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1552921103465189376) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-check | f15ba64649a279a667850569f18de4fb9c22bd11 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1552921103465189376) | true | `/test pull-cloud-provider-azure-check`
pull-cloud-provider-azure-e2e-capz | f15ba64649a279a667850569f18de4fb9c22bd11 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1552921103599407104) | true | `/test pull-cloud-provider-azure-e2e-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-capz | f15ba64649a279a667850569f18de4fb9c22bd11 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1552921103599407104) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-check | da6a4cf537055349e4a2f4a96ca4bf6e54e52b81 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1553080521309818880) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-capz | f15ba64649a279a667850569f18de4fb9c22bd11 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1552921103599407104) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-check | dd68e16d142da25f702743d3bdf5d6d1390f99e7 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1553084218404245504) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | b3f2e289c6bcbbd6e0ec02f99fb046885efa5daf | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1553110480292155392) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | b3f2e289c6bcbbd6e0ec02f99fb046885efa5daf | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1553110480292155392) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | b3f2e289c6bcbbd6e0ec02f99fb046885efa5daf | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1553110480250212352) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
zmyzheng: /retest
k8s-ci-robot: @zmyzheng: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-capz | b3f2e289c6bcbbd6e0ec02f99fb046885efa5daf | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1553191975358828544) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
zmyzheng: /retest
zmyzheng: /retest
k8s-ci-robot: @zmyzheng: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-capz | 8c8353e10a1374c6763dff39e9212a57c0fc001c | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1554199520345067520) | true | `/test pull-cloud-provider-azure-e2e-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-capz | bc909d568a2f70cdbc6911b1f9784aa427873a3f | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554215934397255680) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
zmyzheng: /retest
k8s-ci-robot: @zmyzheng: The following test **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554717812725911552) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554717812725911552) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554717812763660288) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554717812725911552) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554717812763660288) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554717812595888128) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554717812725911552) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554717812763660288) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554717812595888128) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1554717812633636864) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554717812725911552) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554717812763660288) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554717812595888128) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1554717812633636864) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1554717812688162816) | true | `/test pull-cloud-provider-azure-e2e-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554717812725911552) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554717812763660288) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554717812595888128) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1554717812633636864) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1554717812688162816) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-check | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1554717812558139392) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
andyzhangx: @lzhecheng could you take a look at the test failure, it's like secret expired:
```
ERROR: AADSTS7000222: The provided client secret keys for app 'a46d7624-de73-47f7-addb-4d2993a9b10c' are expired. Visit the Azure portal to create new keys for your app: https://aka.ms/NewClientSecret, or consider using certificate credentials for added security: https://aka.ms/certCreds.
```
zmyzheng: /retest
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554717812763660288) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554717812595888128) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1554717812633636864) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1554717812688162816) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-check | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1554717812558139392) | true | `/test pull-cloud-provider-azure-check`
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554750536014630912) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554717812595888128) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1554717812633636864) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1554717812688162816) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-check | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1554717812558139392) | true | `/test pull-cloud-provider-azure-check`
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554750536014630912) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554750536069156864) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1554717812633636864) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1554717812688162816) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-check | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1554717812558139392) | true | `/test pull-cloud-provider-azure-check`
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554750536014630912) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554750536069156864) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554750535901384704) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1554717812688162816) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-check | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1554717812558139392) | true | `/test pull-cloud-provider-azure-check`
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554750536014630912) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554750536069156864) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554750535901384704) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1554750535939133440) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-check | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1554717812558139392) | true | `/test pull-cloud-provider-azure-check`
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554750536014630912) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554750536069156864) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554750535901384704) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1554750535939133440) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1554750535981076480) | true | `/test pull-cloud-provider-azure-e2e-capz`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554750536014630912) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554750536069156864) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554750535901384704) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1554750535939133440) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1554750535981076480) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-check | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1554750535855247360) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->
zmyzheng: /retest
k8s-ci-robot: @zmyzheng: The following tests **failed**, say `/retest` to rerun all failed tests or `/retest-required` to rerun all mandatory failed tests:

Test name | Commit | Details | Required | Rerun command
--- | --- | --- | --- | ---
pull-cloud-provider-azure-e2e-ccm-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-capz/1554750536014630912) | true | `/test pull-cloud-provider-azure-e2e-ccm-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz/1554750536069156864) | false | `/test pull-cloud-provider-azure-e2e-ccm-vmss-ip-lb-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-capz/1554750535901384704) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-capz`
pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb/1554750535939133440) | true | `/test pull-cloud-provider-azure-e2e-ccm-vmss-basic-lb`
pull-cloud-provider-azure-e2e-capz | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-e2e-capz/1554750535981076480) | true | `/test pull-cloud-provider-azure-e2e-capz`
pull-cloud-provider-azure-check | 36034d1fd6185faa39c6f528c3fb3e4731c05b78 | [link](https://prow.k8s.io/view/gs/kubernetes-jenkins/pr-logs/pull/kubernetes-sigs_cloud-provider-azure/2100/pull-cloud-provider-azure-check/1554998831727251456) | true | `/test pull-cloud-provider-azure-check`

[Full PR test history](https://prow.k8s.io/pr-history?org=kubernetes-sigs&repo=cloud-provider-azure&pr=2100). [Your PR dashboard](https://prow.k8s.io/pr?query=is%3Apr%20state%3Aopen%20author%3Azmyzheng). Please help us cut down on flakes by [linking to](https://git.k8s.io/community/contributors/devel/sig-testing/flaky-tests.md#filing-issues-for-flaky-tests) an [open issue](https://github.com/kubernetes-sigs/cloud-provider-azure/issues?q=is:issue+is:open) when you hit one in your PR.

<details>

Instructions for interacting with me using PR comments are available [here](https://git.k8s.io/community/contributors/guide/pull-requests.md).  If you have questions or suggestions related to my behavior, please file an issue against the [kubernetes/test-infra](https://github.com/kubernetes/test-infra/issues/new?title=Prow%20issue:) repository. I understand the commands that are listed [here](https://go.k8s.io/bot-commands).
</details>
<!-- test report -->

Patch:
diff --git a/.golangci.yml b/.golangci.yml
index e10f54519..5afd635cc 100644
--- a/.golangci.yml
+++ b/.golangci.yml
@@ -1,14 +1,32 @@
 run:
-  deadline: 30m
-  concurrency: 2
-  go: '1.18'
-  allow-serial-runners: true
+  deadline: 10m
 skip-dirs:
     - vendor
 linters:
   disable-all: true
   enable:
-    - LINTER_TO_RUN
+    - deadcode
+    - errcheck
+#    - errorlint
+    - goconst
+#    - gocyclo
+    - gofmt
+    - goimports
+    - revive
+    - gosec
+    - gosimple
+    - govet
+    - ineffassign
+#    - interfacer
+    - misspell
+    - nakedret
+#    - prealloc
+    - staticcheck
+    - structcheck
+    - typecheck
+    - unconvert
+    - unused
+    - varcheck
   # Run with --fast=false for more extensive checks
   fast: false
   govet:
diff --git a/.pipelines/daily-gc.yml b/.pipelines/daily-gc.yml
index bf7139d68..bf8578e8a 100644
--- a/.pipelines/daily-gc.yml
+++ b/.pipelines/daily-gc.yml
@@ -1,11 +1,9 @@
 schedules:
-- cron: "0 16 * * *"
+- cron: "0 0 * * *"
   displayName: Daily GC
   branches:
     include:
-    - master
-
-trigger: none
+    - main
 
 pr: none
 
diff --git a/.pipelines/templates/autoscaling-multipool.json b/.pipelines/templates/autoscaling-multipool.json
index 5d077cdeb..4263059a3 100644
--- a/.pipelines/templates/autoscaling-multipool.json
+++ b/.pipelines/templates/autoscaling-multipool.json
@@ -14,7 +14,7 @@
     "agentPoolProfiles": [
       {
         "name": "agentpool1",
-        "count": 1,
+        "count": 2,
         "mode" : "System",
         "vmSize": "Standard_DS2_v2",
         "osType": "Linux",
@@ -26,7 +26,7 @@
       },
       {
         "name": "agentpool2",
-        "count": 1,
+        "count": 2,
         "mode" : "System",
         "vmSize": "Standard_DS2_v2",
         "osType": "Linux",
diff --git a/.pipelines/templates/autoscaling.json b/.pipelines/templates/autoscaling.json
index bc46fe4cf..545dc2513 100644
--- a/.pipelines/templates/autoscaling.json
+++ b/.pipelines/templates/autoscaling.json
@@ -14,7 +14,7 @@
     "agentPoolProfiles": [
       {
         "name": "agentpool1",
-        "count": 1,
+        "count": 2,
         "mode" : "System",
         "vmSize": "Standard_DS2_v2",
         "osType": "Linux",
diff --git a/cmd/cloud-controller-manager/app/core.go b/cmd/cloud-controller-manager/app/core.go
index 30dacce66..f181164f8 100644
--- a/cmd/cloud-controller-manager/app/core.go
+++ b/cmd/cloud-controller-manager/app/core.go
@@ -17,6 +17,7 @@ limitations under the License.
 // Package app implements a server that runs a set of active
 // components.  This includes node controllers, service and
 // route controller, and so on.
+//
 package app
 
 import (
diff --git a/go.mod b/go.mod
index cc8af5768..c29995aa8 100644
--- a/go.mod
+++ b/go.mod
@@ -5,7 +5,7 @@ go 1.18
 require (
 	github.com/Azure/azure-sdk-for-go v65.0.0+incompatible
 	github.com/Azure/go-autorest/autorest v0.11.28
-	github.com/Azure/go-autorest/autorest/adal v0.9.21
+	github.com/Azure/go-autorest/autorest/adal v0.9.20
 	github.com/Azure/go-autorest/autorest/mocks v0.4.2
 	github.com/Azure/go-autorest/autorest/to v0.4.0
 	github.com/Azure/go-autorest/tracing v0.6.0
diff --git a/go.sum b/go.sum
index 2b2ac2ba5..59be87eca 100644
--- a/go.sum
+++ b/go.sum
@@ -49,8 +49,8 @@ github.com/Azure/go-autorest/autorest v0.11.28 h1:ndAExarwr5Y+GaHE6VCaY1kyS/HwwG
 github.com/Azure/go-autorest/autorest v0.11.28/go.mod h1:MrkzG3Y3AH668QyF9KRk5neJnGgmhQ6krbhR8Q5eMvA=
 github.com/Azure/go-autorest/autorest/adal v0.9.13/go.mod h1:W/MM4U6nLxnIskrw4UwWzlHfGjwUS50aOsc/I3yuU8M=
 github.com/Azure/go-autorest/autorest/adal v0.9.18/go.mod h1:XVVeme+LZwABT8K5Lc3hA4nAe8LDBVle26gTrguhhPQ=
-github.com/Azure/go-autorest/autorest/adal v0.9.21 h1:jjQnVFXPfekaqb8vIsv2G1lxshoW+oGv4MDlhRtnYZk=
-github.com/Azure/go-autorest/autorest/adal v0.9.21/go.mod h1:zua7mBUaCc5YnSLKYgGJR/w5ePdMDA6H56upLsHzA9U=
+github.com/Azure/go-autorest/autorest/adal v0.9.20 h1:gJ3E98kMpFB1MFqQCvA1yFab8vthOeD4VlFRQULxahg=
+github.com/Azure/go-autorest/autorest/adal v0.9.20/go.mod h1:XVVeme+LZwABT8K5Lc3hA4nAe8LDBVle26gTrguhhPQ=
 github.com/Azure/go-autorest/autorest/date v0.3.0 h1:7gUk1U5M/CQbp9WoqinNzJar+8KY+LPI6wiWrP/myHw=
 github.com/Azure/go-autorest/autorest/date v0.3.0/go.mod h1:BI0uouVdmngYNUzGWeSYnokU+TrmwEsOqdt8Y6sso74=
 github.com/Azure/go-autorest/autorest/mocks v0.4.1/go.mod h1:LTp+uSrOhSkaKrUy935gNZuuIPPVsHlr9DSOxSayd+k=
diff --git a/hack/verify-golint.sh b/hack/verify-golint.sh
index 1cf2c3a6e..2c5d5aff4 100755
--- a/hack/verify-golint.sh
+++ b/hack/verify-golint.sh
@@ -25,14 +25,6 @@ fi
 echo "Verifying golint"
 readonly PKG_ROOT="$(git rev-parse --show-toplevel)"
 
-LINTERS=("deadcode" "errcheck" "goconst" "gofmt" "goimports" "revive" "gosec" "gosimple" "govet" "ineffassign" "misspell" "nakedret" "staticcheck" "structcheck" "typecheck" "unconvert" "unused" "varcheck" )
-GOLANGCI_YML="${PKG_ROOT}/.golangci.yml"
-GOLANGCI_YML_TEST="${PKG_ROOT}/.golangci-test.yml"
-for LINTER in "${LINTERS[@]}"; do
-  cp -rf "${GOLANGCI_YML}" "${GOLANGCI_YML_TEST}"
-  sed -i "s|LINTER_TO_RUN|${LINTER}|g" "${GOLANGCI_YML_TEST}"
-  golangci-lint run -v --config "${GOLANGCI_YML_TEST}"
-  echo ""
-done
+golangci-lint run -v --config ${PKG_ROOT}/.golangci.yml
 
 echo "Congratulations! Lint check completed for all Go source files."
diff --git a/pkg/azureclients/privatelinkserviceclient/azure_privatelinkserviceclient.go b/pkg/azureclients/privatelinkserviceclient/azure_privatelinkserviceclient.go
index bede7e41d..4477acf34 100644
--- a/pkg/azureclients/privatelinkserviceclient/azure_privatelinkserviceclient.go
+++ b/pkg/azureclients/privatelinkserviceclient/azure_privatelinkserviceclient.go
@@ -214,7 +214,7 @@ func (c *Client) getPLS(ctx context.Context, resourceGroupName string, privateLi
 	return result, nil
 }
 
-// List gets a list of PrivateLinkServices in the resource group.
+/// List gets a list of PrivateLinkServices in the resource group.
 func (c *Client) List(ctx context.Context, resourceGroupName string) ([]network.PrivateLinkService, *retry.Error) {
 	mc := metrics.NewMetricContext("private_link_services", "list", resourceGroupName, c.subscriptionID, "")
 
diff --git a/pkg/consts/consts.go b/pkg/consts/consts.go
index b0dcc5719..01acf4141 100644
--- a/pkg/consts/consts.go
+++ b/pkg/consts/consts.go
@@ -137,6 +137,13 @@ const (
 	VMASKey = "k8svmasKey"
 	// AvailabilitySetNodesKey is the availability set nodes key
 	AvailabilitySetNodesKey = "k8sAvailabilitySetNodesKey"
+
+	// VmssFlexKey is the key when querying vmssFlexVM cache
+	VmssFlexKey = "k8sVmssFlexKey"
+
+	// GetNodeVmssFlexIDLockKey is the key for getting the lock for getNodeVmssFlexID function
+	GetNodeVmssFlexIDLockKey = "k8sGetNodeVmssFlexIDLockKey"
+
 	// AvailabilitySetNodesCacheTTLDefaultInSeconds is the TTL of the availabilitySet node cache
 	AvailabilitySetNodesCacheTTLDefaultInSeconds = 900
 	// VMSSCacheTTLDefaultInSeconds is the TTL of the vmss cache
@@ -146,6 +153,13 @@ const (
 	// VMASCacheTTLDefaultInSeconds is the TTL of the vmas cache
 	VMASCacheTTLDefaultInSeconds = 600
 
+	// VmssFlexCacheTTLDefaultInSeconds is the TTL of the vmss flex cache
+	VmssFlexCacheTTLDefaultInSeconds = 600
+	// VmssFlexVMCacheTTLInSeconds is the TTL of the vmss flex vm cache
+	VmssFlexVMCacheTTLInSeconds = 600
+	// VmssFlexVMStatusCacheTTLInSeconds is the TTL of the vmss flex vm status cache
+	VmssFlexVMStatusCacheTTLInSeconds = 600
+
 	// ZoneFetchingInterval defines the interval of performing zoneClient.GetZones
 	ZoneFetchingInterval = 30 * time.Minute
 )
diff --git a/pkg/node/node.go b/pkg/node/node.go
index f2f9eee1c..6fd475e55 100644
--- a/pkg/node/node.go
+++ b/pkg/node/node.go
@@ -67,7 +67,7 @@ func (np *IMDSNodeProvider) InstanceID(ctx context.Context, name types.NodeName)
 // InstanceType returns the type of the specified instance.
 // Note that if the instance does not exist or is no longer running, we must return ("", cloudprovider.InstanceNotFound)
 // (Implementer Note): This is used by kubelet. Kubelet will label the node. Real log from kubelet:
-// Adding node label from cloud provider: beta.kubernetes.io/instance-type=[value]
+//       Adding node label from cloud provider: beta.kubernetes.io/instance-type=[value]
 func (np *IMDSNodeProvider) InstanceType(ctx context.Context, name types.NodeName) (string, error) {
 	return np.azure.InstanceType(ctx, name)
 }
diff --git a/pkg/node/nodearm.go b/pkg/node/nodearm.go
index 5294c00ce..bc05d9d02 100644
--- a/pkg/node/nodearm.go
+++ b/pkg/node/nodearm.go
@@ -81,7 +81,7 @@ func (np *ARMNodeProvider) InstanceID(ctx context.Context, name types.NodeName)
 // InstanceType returns the type of the specified instance.
 // Note that if the instance does not exist or is no longer running, we must return ("", cloudprovider.InstanceNotFound)
 // (Implementer Note): This is used by kubelet. Kubelet will label the node. Real log from kubelet:
-// Adding node label from cloud provider: beta.kubernetes.io/instance-type=[value]
+//       Adding node label from cloud provider: beta.kubernetes.io/instance-type=[value]
 func (np *ARMNodeProvider) InstanceType(ctx context.Context, name types.NodeName) (string, error) {
 	return np.azure.InstanceType(ctx, name)
 }
diff --git a/pkg/nodeipam/ipam/doc.go b/pkg/nodeipam/ipam/doc.go
index 93be8864a..e2544692b 100644
--- a/pkg/nodeipam/ipam/doc.go
+++ b/pkg/nodeipam/ipam/doc.go
@@ -18,13 +18,13 @@ limitations under the License.
 // We currently support several kinds of IPAM allocators (these are denoted by
 // the CIDRAllocatorType):
 // - RangeAllocator is an allocator that assigns PodCIDRs to nodes and works
-// in conjunction with the RouteController to configure the network to get
-// connectivity.
+//   in conjunction with the RouteController to configure the network to get
+//   connectivity.
 // - CloudAllocator is an allocator that synchronizes PodCIDRs from IP
-// ranges assignments from the underlying cloud platform.
+//   ranges assignments from the underlying cloud platform.
 // - (Alpha only) IPAMFromCluster is an allocator that has the similar
-// functionality as the RangeAllocator but also synchronizes cluster-managed
-// ranges into the cloud platform.
+//   functionality as the RangeAllocator but also synchronizes cluster-managed
+//   ranges into the cloud platform.
 // - (Alpha only) IPAMFromCloud is the same as CloudAllocator (synchronizes
-// from cloud into the cluster.)
+//   from cloud into the cluster.)
 package ipam
diff --git a/pkg/nodemanager/nodemanager.go b/pkg/nodemanager/nodemanager.go
index d0b733a69..9f1322709 100644
--- a/pkg/nodemanager/nodemanager.go
+++ b/pkg/nodemanager/nodemanager.go
@@ -64,10 +64,10 @@ type NodeProvider interface {
 
 // labelReconcileInfo lists Node labels to reconcile, and how to reconcile them.
 // primaryKey and secondaryKey are keys of labels to reconcile.
-// - If both keys exist, but their values don't match. Use the value from the
-// primaryKey as the source of truth to reconcile.
-// - If ensureSecondaryExists is true, and the secondaryKey does not
-// exist, secondaryKey will be added with the value of the primaryKey.
+//   - If both keys exist, but their values don't match. Use the value from the
+//   primaryKey as the source of truth to reconcile.
+//   - If ensureSecondaryExists is true, and the secondaryKey does not
+//   exist, secondaryKey will be added with the value of the primaryKey.
 var labelReconcileInfo = []struct {
 	primaryKey            string
 	secondaryKey          string
diff --git a/pkg/provider/azure.go b/pkg/provider/azure.go
index e21b96481..58169cbf9 100644
--- a/pkg/provider/azure.go
+++ b/pkg/provider/azure.go
@@ -221,6 +221,12 @@ type Config struct {
 	VmssCacheTTLInSeconds int `json:"vmssCacheTTLInSeconds,omitempty" yaml:"vmssCacheTTLInSeconds,omitempty"`
 	// VmssVirtualMachinesCacheTTLInSeconds sets the cache TTL for vmssVirtualMachines
 	VmssVirtualMachinesCacheTTLInSeconds int `json:"vmssVirtualMachinesCacheTTLInSeconds,omitempty" yaml:"vmssVirtualMachinesCacheTTLInSeconds,omitempty"`
+
+	// VmssFlexCacheTTLInSeconds sets the cache TTL for VMSS Flex
+	VmssFlexCacheTTLInSeconds int `json:"vmssFlexCacheTTLInSeconds,omitempty" yaml:"vmssFlexCacheTTLInSeconds,omitempty"`
+	// VmssFlexVMCacheTTLInSeconds sets the cache TTL for vmss flex vms
+	VmssFlexVMCacheTTLInSeconds int `json:"vmssFlexVMCacheTTLInSeconds,omitempty" yaml:"vmssFlexVMCacheTTLInSeconds,omitempty"`
+
 	// VmCacheTTLInSeconds sets the cache TTL for vm
 	VMCacheTTLInSeconds int `json:"vmCacheTTLInSeconds,omitempty" yaml:"vmCacheTTLInSeconds,omitempty"`
 	// LoadBalancerCacheTTLInSeconds sets the cache TTL for load balancer
diff --git a/pkg/provider/azure_controller_vmssflex.go b/pkg/provider/azure_controller_vmssflex.go
new file mode 100644
index 000000000..eb0ae4f4a
--- /dev/null
+++ b/pkg/provider/azure_controller_vmssflex.go
@@ -0,0 +1,51 @@
+/*
+Copyright 2022 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package provider
+
+import (
+	"context"
+
+	"github.com/Azure/azure-sdk-for-go/services/compute/mgmt/2021-07-01/compute"
+	"github.com/Azure/go-autorest/autorest/azure"
+	"k8s.io/apimachinery/pkg/types"
+	azcache "sigs.k8s.io/cloud-provider-azure/pkg/cache"
+)
+
+// AttachDisk attaches a disk to vm
+func (fs *FlexScaleSet) AttachDisk(ctx context.Context, nodeName types.NodeName, diskMap map[string]*AttachDiskOptions) (*azure.Future, error) {
+	return nil, nil
+}
+
+// DetachDisk detaches a disk from VM
+func (fs *FlexScaleSet) DetachDisk(ctx context.Context, nodeName types.NodeName, diskMap map[string]string) error {
+	return nil
+}
+
+// WaitForUpdateResult waits for the response of the update request
+func (fs *FlexScaleSet) WaitForUpdateResult(ctx context.Context, future *azure.Future, resourceGroupName, source string) error {
+	return nil
+}
+
+// UpdateVM updates a vm
+func (fs *FlexScaleSet) UpdateVM(ctx context.Context, nodeName types.NodeName) error {
+	return nil
+}
+
+// GetDataDisks gets a list of data disks attached to the node.
+func (fs *FlexScaleSet) GetDataDisks(nodeName types.NodeName, crt azcache.AzureCacheReadType) ([]compute.DataDisk, *string, error) {
+	return nil, nil, nil
+}
diff --git a/pkg/provider/azure_fakes.go b/pkg/provider/azure_fakes.go
index 1ef726c56..a78281ba0 100644
--- a/pkg/provider/azure_fakes.go
+++ b/pkg/provider/azure_fakes.go
@@ -60,6 +60,16 @@ func newTestScaleSetWithState(ctrl *gomock.Controller) (*ScaleSet, error) {
 	return ss.(*ScaleSet), nil
 }
 
+func NewTestFlexScaleSet(ctrl *gomock.Controller) (*FlexScaleSet, error) {
+	cloud := GetTestCloud(ctrl)
+	fs, err := newFlexScaleSet(cloud)
+	if err != nil {
+		return nil, err
+	}
+
+	return fs.(*FlexScaleSet), nil
+}
+
 // GetTestCloud returns a fake azure cloud for unit tests in Azure related CSI drivers
 func GetTestCloud(ctrl *gomock.Controller) (az *Cloud) {
 	az = &Cloud{
diff --git a/pkg/provider/azure_instances.go b/pkg/provider/azure_instances.go
index e89630f32..fd906a92e 100644
--- a/pkg/provider/azure_instances.go
+++ b/pkg/provider/azure_instances.go
@@ -433,7 +433,7 @@ func (az *Cloud) InstanceTypeByProviderID(ctx context.Context, providerID string
 // InstanceType returns the type of the specified instance.
 // Note that if the instance does not exist or is no longer running, we must return ("", cloudprovider.InstanceNotFound)
 // (Implementer Note): This is used by kubelet. Kubelet will label the node. Real log from kubelet:
-// Adding node label from cloud provider: beta.kubernetes.io/instance-type=[value]
+//       Adding node label from cloud provider: beta.kubernetes.io/instance-type=[value]
 func (az *Cloud) InstanceType(ctx context.Context, name types.NodeName) (string, error) {
 	// Returns "" for unmanaged nodes because azure cloud provider couldn't fetch information for them.
 	unmanaged, err := az.IsNodeUnmanaged(string(name))
diff --git a/pkg/provider/azure_loadbalancer.go b/pkg/provider/azure_loadbalancer.go
index 9a528dc5d..843db923b 100644
--- a/pkg/provider/azure_loadbalancer.go
+++ b/pkg/provider/azure_loadbalancer.go
@@ -2212,7 +2212,7 @@ func (az *Cloud) getExpectedLBRules(
 	return expectedProbes, expectedRules, nil
 }
 
-// getDefaultLoadBalancingRulePropertiesFormat returns the loadbalancing rule for one port
+//getDefaultLoadBalancingRulePropertiesFormat returns the loadbalancing rule for one port
 func (az *Cloud) getExpectedLoadBalancingRulePropertiesForPort(
 	service *v1.Service,
 	lbFrontendIPConfigID string,
@@ -2268,7 +2268,7 @@ func (az *Cloud) getExpectedLoadBalancingRulePropertiesForPort(
 	return props, nil
 }
 
-// getExpectedHAModeLoadBalancingRuleProperties build load balancing rule for lb in HA mode
+//getExpectedHAModeLoadBalancingRuleProperties build load balancing rule for lb in HA mode
 func (az *Cloud) getExpectedHAModeLoadBalancingRuleProperties(
 	service *v1.Service,
 	lbFrontendIPConfigID string,
diff --git a/pkg/provider/azure_managedDiskController.go b/pkg/provider/azure_managedDiskController.go
index 740b89caf..e53eac1b2 100644
--- a/pkg/provider/azure_managedDiskController.go
+++ b/pkg/provider/azure_managedDiskController.go
@@ -37,7 +37,7 @@ import (
 	"sigs.k8s.io/cloud-provider-azure/pkg/consts"
 )
 
-// ManagedDiskController : managed disk controller struct
+//ManagedDiskController : managed disk controller struct
 type ManagedDiskController struct {
 	common *controllerCommon
 }
@@ -86,7 +86,7 @@ type ManagedDiskOptions struct {
 	SubscriptionID string
 }
 
-// CreateManagedDisk: create managed disk
+//CreateManagedDisk : create managed disk
 func (c *ManagedDiskController) CreateManagedDisk(ctx context.Context, options *ManagedDiskOptions) (string, error) {
 	var err error
 	klog.V(4).Infof("azureDisk - creating new managed Name:%s StorageAccountType:%s Size:%v", options.DiskName, options.StorageAccountType, options.SizeGB)
@@ -267,7 +267,7 @@ func (c *ManagedDiskController) CreateManagedDisk(ctx context.Context, options *
 	return diskID, nil
 }
 
-// DeleteManagedDisk : delete managed disk
+//DeleteManagedDisk : delete managed disk
 func (c *ManagedDiskController) DeleteManagedDisk(ctx context.Context, diskURI string) error {
 	resourceGroup, subsID, err := getInfoFromDiskURI(diskURI)
 	if err != nil {
diff --git a/pkg/provider/azure_routes.go b/pkg/provider/azure_routes.go
index f9b4b077c..d3d768b02 100644
--- a/pkg/provider/azure_routes.go
+++ b/pkg/provider/azure_routes.go
@@ -70,8 +70,8 @@ func (op *delayedRouteOperation) wait() error {
 // delayedRouteUpdater defines a delayed route updater, which batches all the
 // route updating operations within "interval" period.
 // Example usage:
-// op, err := updater.addRouteOperation(routeOperationAdd, route)
-// err = op.wait()
+//   op, err := updater.addRouteOperation(routeOperationAdd, route)
+//   err = op.wait()
 type delayedRouteUpdater struct {
 	az       *Cloud
 	interval time.Duration
@@ -552,7 +552,7 @@ func findFirstIPByFamily(ips []string, v6 bool) (string, error) {
 	return "", fmt.Errorf("no match found matching the ipfamily requested")
 }
 
-// strips : . /
+//strips : . /
 func cidrtoRfc1035(cidr string) string {
 	cidr = strings.ReplaceAll(cidr, ":", "")
 	cidr = strings.ReplaceAll(cidr, ".", "")
diff --git a/pkg/provider/azure_standard.go b/pkg/provider/azure_standard.go
index c3c9ed5a8..66b40f336 100644
--- a/pkg/provider/azure_standard.go
+++ b/pkg/provider/azure_standard.go
@@ -265,8 +265,8 @@ func isInternalLoadBalancer(lb *network.LoadBalancer) bool {
 // SingleStack -v4 (pre v1.16) => BackendPool name == clusterName
 // SingleStack -v6 => BackendPool name == <clusterName>-IPv6 (all cluster bootstrap uses this name)
 // DualStack
-// => IPv4 BackendPool name == clusterName
-// => IPv6 BackendPool name == <clusterName>-IPv6
+//	=> IPv4 BackendPool name == clusterName
+//  => IPv6 BackendPool name == <clusterName>-IPv6
 // This means:
 // clusters moving from IPv4 to dualstack will require no changes
 // clusters moving from IPv6 to dualstack will require no changes as the IPv4 backend pool will created with <clusterName>
@@ -434,7 +434,7 @@ outer:
 
 var polyTable = crc32.MakeTable(crc32.Koopman)
 
-// MakeCRC32 : convert string to CRC32 format
+//MakeCRC32 : convert string to CRC32 format
 func MakeCRC32(str string) string {
 	crc := crc32.New(polyTable)
 	_, _ = crc.Write([]byte(str))
@@ -1316,7 +1316,7 @@ func (as *availabilitySet) GetNodeCIDRMasksByProviderID(providerID string) (int,
 	return ipv4Mask, ipv6Mask, nil
 }
 
-// EnsureBackendPoolDeletedFromVMSets ensures the loadBalancer backendAddressPools deleted from the specified VMAS
+//EnsureBackendPoolDeletedFromVMSets ensures the loadBalancer backendAddressPools deleted from the specified VMAS
 func (as *availabilitySet) EnsureBackendPoolDeletedFromVMSets(vmasNamesMap map[string]bool, backendPoolID string) error {
 	return nil
 }
diff --git a/pkg/provider/azure_vmss.go b/pkg/provider/azure_vmss.go
index 99ff63d25..61c408aae 100644
--- a/pkg/provider/azure_vmss.go
+++ b/pkg/provider/azure_vmss.go
@@ -379,10 +379,10 @@ func (ss *ScaleSet) GetInstanceIDByNodeName(name string) (string, error) {
 
 // GetNodeNameByProviderID gets the node name by provider ID.
 // providerID example:
-// 1. vmas providerID: azure:///subscriptions/subsid/resourceGroups/rg/providers/Microsoft.Compute/virtualMachines/aks-nodepool1-27053986-0
-// 2. vmss providerID:
-// azure:///subscriptions/subsid/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/aks-agentpool-22126781-vmss/virtualMachines/1
-// /subscriptions/subsid/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/aks-agentpool-22126781-vmss/virtualMachines/k8s-agentpool-36841236-vmss_1
+// 	 1. vmas providerID: azure:///subscriptions/subsid/resourceGroups/rg/providers/Microsoft.Compute/virtualMachines/aks-nodepool1-27053986-0
+// 	 2. vmss providerID:
+//		azure:///subscriptions/subsid/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/aks-agentpool-22126781-vmss/virtualMachines/1
+//	    /subscriptions/subsid/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/aks-agentpool-22126781-vmss/virtualMachines/k8s-agentpool-36841236-vmss_1
 func (ss *ScaleSet) GetNodeNameByProviderID(providerID string) (types.NodeName, error) {
 	// NodeName is not part of providerID for vmss instances.
 	scaleSetName, err := extractScaleSetNameByProviderID(providerID)
@@ -1683,7 +1683,7 @@ func (ss *ScaleSet) GetNodeCIDRMasksByProviderID(providerID string) (int, int, e
 	return ipv4Mask, ipv6Mask, nil
 }
 
-// EnsureBackendPoolDeletedFromVMSets ensures the loadBalancer backendAddressPools deleted from the specified VMSS
+//EnsureBackendPoolDeletedFromVMSets ensures the loadBalancer backendAddressPools deleted from the specified VMSS
 func (ss *ScaleSet) EnsureBackendPoolDeletedFromVMSets(vmssNamesMap map[string]bool, backendPoolID string) error {
 	vmssUpdaters := make([]func() error, 0, len(vmssNamesMap))
 	errors := make([]error, 0, len(vmssNamesMap))
diff --git a/pkg/provider/azure_vmssflex.go b/pkg/provider/azure_vmssflex.go
new file mode 100644
index 000000000..a368fe167
--- /dev/null
+++ b/pkg/provider/azure_vmssflex.go
@@ -0,0 +1,182 @@
+/*
+Copyright 2022 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package provider
+
+import (
+	"errors"
+	"sync"
+
+	"github.com/Azure/azure-sdk-for-go/services/compute/mgmt/2021-07-01/compute"
+	"github.com/Azure/azure-sdk-for-go/services/network/mgmt/2021-08-01/network"
+	v1 "k8s.io/api/core/v1"
+	"k8s.io/apimachinery/pkg/types"
+	cloudprovider "k8s.io/cloud-provider"
+	azcache "sigs.k8s.io/cloud-provider-azure/pkg/cache"
+)
+
+var (
+	// ErrorVmssIDIsEmpty indicates the vmss id is empty.
+	ErrorVmssIDIsEmpty = errors.New("VMSS ID is empty")
+)
+
+// FlexScaleSet implements VMSet interface for Azure Flexible VMSS.
+type FlexScaleSet struct {
+	*Cloud
+
+	vmssFlexCache *azcache.TimedCache
+
+	vmssFlexVMNameToVmssID *sync.Map
+	vmssFlexVMCache        *azcache.TimedCache
+
+	// lockMap in cache refresh
+	lockMap *lockMap
+}
+
+func newFlexScaleSet(az *Cloud) (VMSet, error) {
+	fs := &FlexScaleSet{
+		Cloud:                  az,
+		vmssFlexVMNameToVmssID: &sync.Map{},
+		lockMap:                newLockMap(),
+	}
+
+	var err error
+	fs.vmssFlexCache, err = fs.newVmssFlexCache()
+	if err != nil {
+		return nil, err
+	}
+	fs.vmssFlexVMCache, err = fs.newVmssFlexVMCache()
+	if err != nil {
+		return nil, err
+	}
+
+	return fs, nil
+}
+
+// GetNodeNameByProviderID gets the node name by provider ID.
+func (fs *FlexScaleSet) GetNodeNameByProviderID(providerID string) (types.NodeName, error) {
+
+	return types.NodeName(""), nil
+}
+
+// GetPrimaryVMSetName returns the VM set name depending on the configured vmType.
+// It returns config.PrimaryScaleSetName for vmss and config.PrimaryAvailabilitySetName for standard vmType.
+func (fs *FlexScaleSet) GetPrimaryVMSetName() string {
+	return fs.Config.PrimaryScaleSetName
+}
+
+// GetNodeVMSetName returns the availability set or vmss name by the node name.
+// It will return empty string when using standalone vms.
+func (fs *FlexScaleSet) GetNodeVMSetName(node *v1.Node) (string, error) {
+	return "", nil
+}
+
+// GetAgentPoolVMSetNames returns all vmSet names according to the nodes
+func (fs *FlexScaleSet) GetAgentPoolVMSetNames(nodes []*v1.Node) (*[]string, error) {
+	return nil, nil
+}
+
+// GetVMSetNames selects all possible availability sets or scale sets
+// (depending vmType configured) for service load balancer, if the service has
+// no loadbalancer mode annotation returns the primary VMSet. If service annotation
+// for loadbalancer exists then returns the eligible VMSet. The mode selection
+// annotation would be ignored when using one SLB per cluster.
+func (fs *FlexScaleSet) GetVMSetNames(service *v1.Service, nodes []*v1.Node) (*[]string, error) {
+	return nil, nil
+}
+
+// GetInstanceIDByNodeName gets the cloud provider ID by node name.
+// It must return ("", cloudprovider.InstanceNotFound) if the instance does
+// not exist or is no longer running.
+func (fs *FlexScaleSet) GetInstanceIDByNodeName(name string) (string, error) {
+	return "", nil
+
+}
+
+// GetInstanceTypeByNodeName gets the instance type by node name.
+func (fs *FlexScaleSet) GetInstanceTypeByNodeName(name string) (string, error) {
+	return "", nil
+}
+
+// GetZoneByNodeName gets availability zone for the specified node. If the node is not running
+// with availability zone, then it returns fault domain.
+// for details, refer to https://kubernetes-sigs.github.io/cloud-provider-azure/topics/availability-zones/#node-labels
+func (fs *FlexScaleSet) GetZoneByNodeName(name string) (cloudprovider.Zone, error) {
+	return cloudprovider.Zone{}, nil
+}
+
+// GetProvisioningStateByNodeName returns the provisioningState for the specified node.
+func (fs *FlexScaleSet) GetProvisioningStateByNodeName(name string) (provisioningState string, err error) {
+	return "", nil
+}
+
+// GetPowerStatusByNodeName returns the powerState for the specified node.
+func (fs *FlexScaleSet) GetPowerStatusByNodeName(name string) (powerState string, err error) {
+	return "", nil
+}
+
+// GetPrimaryInterface gets machine primary network interface by node name.
+func (fs *FlexScaleSet) GetPrimaryInterface(nodeName string) (network.Interface, error) {
+	return network.Interface{}, nil
+}
+
+// GetIPByNodeName gets machine private IP and public IP by node name.
+func (fs *FlexScaleSet) GetIPByNodeName(name string) (string, string, error) {
+	return "", "", nil
+
+}
+
+// GetPrivateIPsByNodeName returns a slice of all private ips assigned to node (ipv6 and ipv4)
+// TODO (khenidak): This should read all nics, not just the primary
+// allowing users to split ipv4/v6 on multiple nics
+func (fs *FlexScaleSet) GetPrivateIPsByNodeName(name string) ([]string, error) {
+	ips := make([]string, 0)
+	return ips, nil
+}
+
+// GetNodeNameByIPConfigurationID gets the nodeName and vmSetName by IP configuration ID.
+func (fs *FlexScaleSet) GetNodeNameByIPConfigurationID(ipConfigurationID string) (string, string, error) {
+	return "", "", nil
+}
+
+// GetNodeCIDRMaskByProviderID returns the node CIDR subnet mask by provider ID.
+func (fs *FlexScaleSet) GetNodeCIDRMasksByProviderID(providerID string) (int, int, error) {
+	return 0, 0, nil
+}
+
+// EnsureHostInPool ensures the given VM's Primary NIC's Primary IP Configuration is
+// participating in the specified LoadBalancer Backend Pool, which returns (resourceGroup, vmasName, instanceID, vmssVM, error).
+func (fs *FlexScaleSet) EnsureHostInPool(service *v1.Service, nodeName types.NodeName, backendPoolID string, vmSetNameOfLB string) (string, string, string, *compute.VirtualMachineScaleSetVM, error) {
+	return "", "", "", nil, nil
+
+}
+
+// EnsureHostsInPool ensures the given Node's primary IP configurations are
+// participating in the specified LoadBalancer Backend Pool.
+func (fs *FlexScaleSet) EnsureHostsInPool(service *v1.Service, nodes []*v1.Node, backendPoolID string, vmSetNameOfLB string) error {
+	return nil
+}
+
+// EnsureBackendPoolDeletedFromVMSets ensures the loadBalancer backendAddressPools deleted from the specified VMSS Flex
+func (fs *FlexScaleSet) EnsureBackendPoolDeletedFromVMSets(vmssNamesMap map[string]bool, backendPoolID string) error {
+	return nil
+}
+
+// EnsureBackendPoolDeleted ensures the loadBalancer backendAddressPools deleted from the specified nodes.
+func (fs *FlexScaleSet) EnsureBackendPoolDeleted(service *v1.Service, backendPoolID, vmSetName string, backendAddressPools *[]network.BackendAddressPool, deleteFromVMSet bool) error {
+	return nil
+
+}
diff --git a/pkg/provider/azure_vmssflex_cache.go b/pkg/provider/azure_vmssflex_cache.go
new file mode 100644
index 000000000..e5b316580
--- /dev/null
+++ b/pkg/provider/azure_vmssflex_cache.go
@@ -0,0 +1,260 @@
+/*
+Copyright 2022 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package provider
+
+import (
+	"context"
+	"fmt"
+	"strings"
+	"sync"
+	"time"
+
+	"github.com/Azure/azure-sdk-for-go/services/compute/mgmt/2021-07-01/compute"
+	"github.com/Azure/go-autorest/autorest/to"
+
+	"k8s.io/apimachinery/pkg/types"
+	cloudprovider "k8s.io/cloud-provider"
+	"k8s.io/klog/v2"
+
+	azcache "sigs.k8s.io/cloud-provider-azure/pkg/cache"
+	"sigs.k8s.io/cloud-provider-azure/pkg/consts"
+)
+
+func (fs *FlexScaleSet) newVmssFlexCache() (*azcache.TimedCache, error) {
+
+	getter := func(key string) (interface{}, error) {
+		localCache := &sync.Map{}
+
+		allResourceGroups, err := fs.GetResourceGroups()
+		if err != nil {
+			return nil, err
+		}
+
+		for _, resourceGroup := range allResourceGroups.List() {
+			allScaleSets, rerr := fs.VirtualMachineScaleSetsClient.List(context.Background(), resourceGroup)
+			if rerr != nil {
+				if rerr.IsNotFound() {
+					klog.Warningf("Skip caching vmss for resource group %s due to error: %v", resourceGroup, rerr.Error())
+					continue
+				}
+				klog.Errorf("VirtualMachineScaleSetsClient.List failed: %v", rerr)
+				return nil, rerr.Error()
+			}
+
+			for i := range allScaleSets {
+				scaleSet := allScaleSets[i]
+				if scaleSet.ID == nil || *scaleSet.ID == "" {
+					klog.Warning("failed to get the ID of VMSS Flex")
+					continue
+				}
+
+				if scaleSet.OrchestrationMode == compute.OrchestrationModeFlexible {
+					localCache.Store(*scaleSet.ID, &scaleSet)
+				}
+			}
+		}
+
+		return localCache, nil
+	}
+
+	if fs.Config.VmssFlexCacheTTLInSeconds == 0 {
+		fs.Config.VmssFlexCacheTTLInSeconds = consts.VmssFlexCacheTTLDefaultInSeconds
+	}
+	return azcache.NewTimedcache(time.Duration(fs.Config.VmssFlexCacheTTLInSeconds)*time.Second, getter)
+}
+
+func (fs *FlexScaleSet) newVmssFlexVMCache() (*azcache.TimedCache, error) {
+	getter := func(key string) (interface{}, error) {
+		localCache := &sync.Map{}
+
+		ctx, cancel := getContextWithCancel()
+		defer cancel()
+
+		vms, rerr := fs.VirtualMachinesClient.ListVmssFlexVMsWithoutInstanceView(ctx, key)
+		if rerr != nil {
+			klog.Errorf("ListVmssFlexVMsWithoutInstanceView failed: %v", rerr)
+			return nil, rerr.Error()
+		}
+
+		for i := range vms {
+			vm := vms[i]
+			if vm.Name != nil {
+				localCache.Store(*vm.Name, &vm)
+				fs.vmssFlexVMNameToVmssID.Store(*vm.Name, key)
+			}
+		}
+
+		vms, rerr = fs.VirtualMachinesClient.ListVmssFlexVMsWithOnlyInstanceView(ctx, key)
+		if rerr != nil {
+			klog.Errorf("ListVmssFlexVMsWithOnlyInstanceView failed: %v", rerr)
+			return nil, rerr.Error()
+		}
+
+		for i := range vms {
+			vm := vms[i]
+			if vm.Name != nil {
+				cached, ok := localCache.Load(*vm.Name)
+				if ok {
+					cachedVM := cached.(*compute.VirtualMachine)
+					cachedVM.VirtualMachineProperties.InstanceView = vm.VirtualMachineProperties.InstanceView
+				}
+			}
+		}
+
+		return localCache, nil
+	}
+
+	if fs.Config.VmssFlexVMCacheTTLInSeconds == 0 {
+		fs.Config.VmssFlexVMCacheTTLInSeconds = consts.VmssFlexVMCacheTTLInSeconds
+	}
+	return azcache.NewTimedcache(time.Duration(fs.Config.VmssFlexVMCacheTTLInSeconds)*time.Second, getter)
+}
+
+func (fs *FlexScaleSet) getNodeVmssFlexID(nodeName string) (string, error) {
+	fs.lockMap.LockEntry(consts.GetNodeVmssFlexIDLockKey)
+	defer fs.lockMap.UnlockEntry(consts.GetNodeVmssFlexIDLockKey)
+	vmssFlexID, isCached := fs.vmssFlexVMNameToVmssID.Load(nodeName)
+	if !isCached {
+		klog.V(12).Infof("nodeName %s is not saved in vmssFlexVMnameToVmssID map, send a GET request to retrieve its VmssID", nodeName)
+		machine, err := fs.getVirtualMachine(types.NodeName(nodeName), azcache.CacheReadTypeUnsafe)
+		if err != nil {
+			return "", err
+		}
+		vmssFlexID = to.String(machine.VirtualMachineScaleSet.ID)
+		if vmssFlexID == "" {
+			return "", ErrorVmssIDIsEmpty
+		}
+		fs.vmssFlexVMNameToVmssID.Store(nodeName, vmssFlexID)
+		_, err = fs.vmssFlexVMCache.Get(fmt.Sprintf("%v", vmssFlexID), azcache.CacheReadTypeForceRefresh)
+		if err != nil {
+			return "", err
+		}
+	}
+	return fmt.Sprintf("%v", vmssFlexID), nil
+}
+
+func (fs *FlexScaleSet) getVmssFlexVM(nodeName string, crt azcache.AzureCacheReadType) (vm compute.VirtualMachine, err error) {
+	vmssFlexID, err := fs.getNodeVmssFlexID(nodeName)
+	if err != nil {
+		return vm, err
+	}
+
+	cached, err := fs.vmssFlexVMCache.Get(vmssFlexID, crt)
+	if err != nil {
+		return vm, err
+	}
+
+	vmMap := cached.(*sync.Map)
+	cachvmedVM, ok := vmMap.Load(nodeName)
+	if !ok {
+		klog.V(2).Infof("did not find node (%s) in the existing cache, which means it is deleted...", nodeName)
+		return vm, cloudprovider.InstanceNotFound
+	}
+
+	return *(cachvmedVM.(*compute.VirtualMachine)), nil
+}
+
+func (fs *FlexScaleSet) getVmssFlexByVmssFlexID(vmssFlexID string, crt azcache.AzureCacheReadType) (*compute.VirtualMachineScaleSet, error) {
+	cached, err := fs.vmssFlexCache.Get(consts.VmssFlexKey, crt)
+	if err != nil {
+		return nil, err
+	}
+
+	vmssFlexes := cached.(*sync.Map)
+	if vmssFlex, ok := vmssFlexes.Load(vmssFlexID); ok {
+		result := vmssFlex.(*compute.VirtualMachineScaleSet)
+		return result, nil
+	}
+
+	klog.V(2).Infof("Couldn't find VMSS Flex with ID %s, refreshing the cache", vmssFlexID)
+	cached, err = fs.vmssFlexCache.Get(consts.VmssFlexKey, azcache.CacheReadTypeForceRefresh)
+	if err != nil {
+		return nil, err
+	}
+
+	vmssFlexes = cached.(*sync.Map)
+	if vmssFlex, ok := vmssFlexes.Load(vmssFlexID); ok {
+		result := vmssFlex.(*compute.VirtualMachineScaleSet)
+		return result, nil
+	}
+	return nil, cloudprovider.InstanceNotFound
+}
+
+func (fs *FlexScaleSet) getVmssFlexByNodeName(nodeName string, crt azcache.AzureCacheReadType) (*compute.VirtualMachineScaleSet, error) {
+	vmssFlexID, err := fs.getNodeVmssFlexID(nodeName)
+	if err != nil {
+		return nil, err
+	}
+	vmssFlex, err := fs.getVmssFlexByVmssFlexID(vmssFlexID, crt)
+	if err != nil {
+		return nil, err
+	}
+	return vmssFlex, nil
+}
+
+func (fs *FlexScaleSet) getVmssFlexIDByName(vmssFlexName string) (string, error) {
+	cached, err := fs.vmssFlexCache.Get(consts.VmssFlexKey, azcache.CacheReadTypeDefault)
+	if err != nil {
+		return "", err
+	}
+
+	var targetVmssFlexID string
+	vmssFlexes := cached.(*sync.Map)
+	vmssFlexes.Range(func(key, value interface{}) bool {
+		vmssFlexID := key.(string)
+		name, err := getLastSegment(vmssFlexID, "/")
+		if err != nil {
+			return true
+		}
+		if strings.EqualFold(name, vmssFlexName) {
+			targetVmssFlexID = vmssFlexID
+			return false
+		}
+		return true
+	})
+	if targetVmssFlexID != "" {
+		return targetVmssFlexID, nil
+	}
+	return "", cloudprovider.InstanceNotFound
+}
+
+func (fs *FlexScaleSet) getVmssFlexByName(vmssFlexName string) (*compute.VirtualMachineScaleSet, error) {
+	cached, err := fs.vmssFlexCache.Get(consts.VmssFlexKey, azcache.CacheReadTypeDefault)
+	if err != nil {
+		return nil, err
+	}
+
+	var targetVmssFlex *compute.VirtualMachineScaleSet
+	vmssFlexes := cached.(*sync.Map)
+	vmssFlexes.Range(func(key, value interface{}) bool {
+		vmssFlexID := key.(string)
+		vmssFlex := value.(*compute.VirtualMachineScaleSet)
+		name, err := getLastSegment(vmssFlexID, "/")
+		if err != nil {
+			return true
+		}
+		if strings.EqualFold(name, vmssFlexName) {
+			targetVmssFlex = vmssFlex
+			return false
+		}
+		return true
+	})
+	if targetVmssFlex != nil {
+		return targetVmssFlex, nil
+	}
+	return nil, cloudprovider.InstanceNotFound
+}
diff --git a/pkg/provider/azure_wrap.go b/pkg/provider/azure_wrap.go
index ba166dbf3..6f82d0c12 100644
--- a/pkg/provider/azure_wrap.go
+++ b/pkg/provider/azure_wrap.go
@@ -63,9 +63,9 @@ func checkResourceExistsFromError(err *retry.Error) (bool, *retry.Error) {
 	return false, err
 }
 
-// getVirtualMachine calls 'VirtualMachinesClient.Get' with a timed cache
-// The service side has throttling control that delays responses if there are multiple requests onto certain vm
-// resource request in short period.
+/// getVirtualMachine calls 'VirtualMachinesClient.Get' with a timed cache
+/// The service side has throttling control that delays responses if there are multiple requests onto certain vm
+/// resource request in short period.
 func (az *Cloud) getVirtualMachine(nodeName types.NodeName, crt azcache.AzureCacheReadType) (vm compute.VirtualMachine, err error) {
 	vmName := string(nodeName)
 	cachedVM, err := az.vmCache.Get(vmName, crt)
diff --git a/pkg/retry/azure_retry.go b/pkg/retry/azure_retry.go
index 4093b6ba4..f2070eb3a 100644
--- a/pkg/retry/azure_retry.go
+++ b/pkg/retry/azure_retry.go
@@ -218,7 +218,7 @@ func delayForBackOff(backoff *Backoff, cancel <-chan struct{}) bool {
 	}
 }
 
-// DoFilterOutNonRetriableError decorator works with autorest.DoRetryForAttempts
+//DoFilterOutNonRetriableError decorator works with autorest.DoRetryForAttempts
 func DoFilterOutNonRetriableError(shouldRetry func(rerr *Error) bool) autorest.SendDecorator {
 	return func(s autorest.Sender) autorest.Sender {
 		return autorest.SenderFunc(func(r *http.Request) (*http.Response, error) {
diff --git a/site/content/en/blog/releases/v1.0.23.md b/site/content/en/blog/releases/v1.0.23.md
deleted file mode 100644
index 357c6b639..000000000
--- a/site/content/en/blog/releases/v1.0.23.md
+++ /dev/null
@@ -1,35 +0,0 @@
----
-title: v1.0.23
-linkTitle: v1.0.23
-date: 2022-08-02
-description: Cloud Provider Azure v1.0.23
----
-
-
-## Changes by Kind
-
-### Feature
-
-- Allow external service with floating ip disabled to create PLS. ([#2079](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2079), [@jwtty](https://github.com/jwtty))
-- Result_code in latency logs is more detailed. ([#2024](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2024), [@monianshouhou](https://github.com/monianshouhou))
-
-### Bug or Regression
-
-- Fix: decouple the LB from the VMSS before VMSS VMs ([#2042](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2042), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-- Use correct subnet name in reconcileFrontendIPConfigs() ([#2084](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2084), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-
-### Other (Cleanup or Flake)
-
-- Fix: make sure there is no duplicated rules before updating the network security group ([#2004](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2004), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-
-## Dependencies
-
-### Added
-_Nothing has changed._
-
-### Changed
-- github.com/Azure/go-autorest/autorest/adal: [v0.9.20 → v0.9.21](https://github.com/Azure/go-autorest/autorest/adal/compare/v0.9.20...v0.9.21)
-- golang.org/x/crypto: 5e0467b → 630584e
-
-### Removed
-_Nothing has changed._
diff --git a/site/content/en/blog/releases/v1.1.19.md b/site/content/en/blog/releases/v1.1.19.md
deleted file mode 100644
index 7b7e69994..000000000
--- a/site/content/en/blog/releases/v1.1.19.md
+++ /dev/null
@@ -1,36 +0,0 @@
----
-title: v1.1.19
-linkTitle: v1.1.19
-date: 2022-08-02
-description: Cloud Provider Azure v1.1.19
----
-
-
-## Changes by Kind
-
-### Feature
-
-- Allow external service with floating ip disabled to create PLS. ([#2078](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2078), [@jwtty](https://github.com/jwtty))
-- Result_code in latency logs is more detailed. ([#2025](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2025), [@monianshouhou](https://github.com/monianshouhou))
-
-### Bug or Regression
-
-- Fix PIP tags possibly not ensured issue in ensurePublicIPExists() ([#2050](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2050), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-- Fix: decouple the LB from the VMSS before VMSS VMs ([#2043](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2043), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-- Use correct subnet name in reconcileFrontendIPConfigs() ([#2083](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2083), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-
-### Other (Cleanup or Flake)
-
-- Fix: make sure there is no duplicated rules before updating the network security group ([#2005](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2005), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-
-## Dependencies
-
-### Added
-_Nothing has changed._
-
-### Changed
-- github.com/Azure/go-autorest/autorest/adal: [v0.9.20 → v0.9.21](https://github.com/Azure/go-autorest/autorest/adal/compare/v0.9.20...v0.9.21)
-- golang.org/x/crypto: e495a2d → 630584e
-
-### Removed
-_Nothing has changed._
diff --git a/site/content/en/blog/releases/v1.23.16.md b/site/content/en/blog/releases/v1.23.16.md
deleted file mode 100644
index 532b1b683..000000000
--- a/site/content/en/blog/releases/v1.23.16.md
+++ /dev/null
@@ -1,37 +0,0 @@
----
-title: v1.23.16
-linkTitle: v1.23.16
-date: 2022-08-02
-description: Cloud Provider Azure v1.23.16
----
-
-
-## Changes by Kind
-
-### Feature
-
-- Allow external service with floating ip disabled to create PLS. ([#2077](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2077), [@jwtty](https://github.com/jwtty))
-
-### Bug or Regression
-
-- Fix PIP tags possibly not ensured issue in ensurePublicIPExists() ([#2049](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2049), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-- Fix: decouple the LB from the VMSS before VMSS VMs ([#2044](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2044), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-- Use correct subnet name in reconcileFrontendIPConfigs() ([#2082](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2082), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-
-### Other (Cleanup or Flake)
-
-- Fix: make sure there is no duplicated rules before updating the network security group ([#2006](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2006), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-- Result_code in latency logs is more detailed. ([#2016](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2016), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-
-## Dependencies
-
-### Added
-_Nothing has changed._
-
-### Changed
-- github.com/Azure/go-autorest/autorest/adal: [v0.9.20 → v0.9.21](https://github.com/Azure/go-autorest/autorest/adal/compare/v0.9.20...v0.9.21)
-- github.com/go-logr/logr: [v1.2.0 → v1.2.3](https://github.com/go-logr/logr/compare/v1.2.0...v1.2.3)
-- golang.org/x/crypto: 5e0467b → 630584e
-
-### Removed
-_Nothing has changed._
diff --git a/site/content/en/blog/releases/v1.24.4.md b/site/content/en/blog/releases/v1.24.4.md
deleted file mode 100644
index 56862ee78..000000000
--- a/site/content/en/blog/releases/v1.24.4.md
+++ /dev/null
@@ -1,53 +0,0 @@
----
-title: v1.24.4
-linkTitle: v1.24.4
-date: 2022-08-02
-description: Cloud Provider Azure v1.24.4
----
-
-
-## Changes by Kind
-
-### Feature
-
-- Allow external service with floating ip disabled to create PLS. ([#2074](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2074), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-
-### Bug or Regression
-
-- Create a single transport to reuse connections, address performance issues and enable tracing. ([#2071](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2071), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-- Fix PIP tags possibly not ensured issue in ensurePublicIPExists() ([#2048](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2048), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-- Fix: decouple the LB from the VMSS before VMSS VMs ([#2045](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2045), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-- Use correct subnet name in reconcileFrontendIPConfigs() ([#2081](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2081), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-
-### Other (Cleanup or Flake)
-
-- Fix: Do not compare privateIPAllocationMethod when checking if frontend config needs update. Just check if frontend ip matches service spec to avoid unnecessary LB frontend recreation. ([#2001](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2001), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-- Result_code in latency logs is more detailed. ([#2017](https://github.com/kubernetes-sigs/cloud-provider-azure/pull/2017), [@k8s-infra-cherrypick-robot](https://github.com/k8s-infra-cherrypick-robot))
-
-## Dependencies
-
-### Added
-_Nothing has changed._
-
-### Changed
-- github.com/Azure/go-autorest/autorest: [v0.11.27 → v0.11.28](https://github.com/Azure/go-autorest/autorest/compare/v0.11.27...v0.11.28)
-- github.com/go-logr/logr: [v1.2.0 → v1.2.3](https://github.com/go-logr/logr/compare/v1.2.0...v1.2.3)
-- github.com/google/go-cmp: [v0.5.5 → v0.5.8](https://github.com/google/go-cmp/compare/v0.5.5...v0.5.8)
-- github.com/onsi/ginkgo/v2: [v2.1.3 → v2.1.4](https://github.com/onsi/ginkgo/v2/compare/v2.1.3...v2.1.4)
-- github.com/onsi/gomega: [v1.19.0 → v1.20.0](https://github.com/onsi/gomega/compare/v1.19.0...v1.20.0)
-- golang.org/x/crypto: 8634188 → 630584e
-- golang.org/x/net: 27dd868 → 2871e0c
-- golang.org/x/sys: 33da011 → 9388b58
-- google.golang.org/protobuf: v1.27.1 → v1.28.0
-- k8s.io/api: v0.24.2 → v0.24.3
-- k8s.io/apimachinery: v0.24.2 → v0.24.3
-- k8s.io/apiserver: v0.24.2 → v0.24.3
-- k8s.io/client-go: v0.24.2 → v0.24.3
-- k8s.io/cloud-provider: v0.24.2 → v0.24.3
-- k8s.io/component-base: v0.24.2 → v0.24.3
-- k8s.io/component-helpers: v0.24.2 → v0.24.3
-- k8s.io/controller-manager: v0.24.2 → v0.24.3
-- k8s.io/kubelet: v0.24.2 → v0.24.3
-
-### Removed
-_Nothing has changed._
diff --git a/site/package-lock.json b/site/package-lock.json
index d04b542f0..56cc6d6af 100644
--- a/site/package-lock.json
+++ b/site/package-lock.json
@@ -56,13 +56,13 @@
       }
     },
     "autoprefixer": {
-      "version": "10.4.8",
-      "resolved": "https://registry.npmjs.org/autoprefixer/-/autoprefixer-10.4.8.tgz",
-      "integrity": "sha512-75Jr6Q/XpTqEf6D2ltS5uMewJIx5irCU1oBYJrWjFenq/m12WRRrz6g15L1EIoYvPLXTbEry7rDOwrcYNj77xw==",
+      "version": "10.4.7",
+      "resolved": "https://registry.npmjs.org/autoprefixer/-/autoprefixer-10.4.7.tgz",
+      "integrity": "sha512-ypHju4Y2Oav95SipEcCcI5J7CGPuvz8oat7sUtYj3ClK44bldfvtvcxK6IEK++7rqB7YchDGzweZIBG+SD0ZAA==",
       "dev": true,
       "requires": {
-        "browserslist": "^4.21.3",
-        "caniuse-lite": "^1.0.30001373",
+        "browserslist": "^4.20.3",
+        "caniuse-lite": "^1.0.30001335",
         "fraction.js": "^4.2.0",
         "normalize-range": "^0.1.2",
         "picocolors": "^1.0.0",
@@ -85,21 +85,22 @@
       }
     },
     "browserslist": {
-      "version": "4.21.3",
-      "resolved": "https://registry.npmjs.org/browserslist/-/browserslist-4.21.3.tgz",
-      "integrity": "sha512-898rgRXLAyRkM1GryrrBHGkqA5hlpkV5MhtZwg9QXeiyLUYs2k00Un05aX5l2/yJIOObYKOpS2JNo8nJDE7fWQ==",
+      "version": "4.20.3",
+      "resolved": "https://registry.npmjs.org/browserslist/-/browserslist-4.20.3.tgz",
+      "integrity": "sha512-NBhymBQl1zM0Y5dQT/O+xiLP9/rzOIQdKM/eMJBAq7yBgaB6krIYLGejrwVYnSHZdqjscB1SPuAjHwxjvN6Wdg==",
       "dev": true,
       "requires": {
-        "caniuse-lite": "^1.0.30001370",
-        "electron-to-chromium": "^1.4.202",
-        "node-releases": "^2.0.6",
-        "update-browserslist-db": "^1.0.5"
+        "caniuse-lite": "^1.0.30001332",
+        "electron-to-chromium": "^1.4.118",
+        "escalade": "^3.1.1",
+        "node-releases": "^2.0.3",
+        "picocolors": "^1.0.0"
       }
     },
     "caniuse-lite": {
-      "version": "1.0.30001373",
-      "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001373.tgz",
-      "integrity": "sha512-pJYArGHrPp3TUqQzFYRmP/lwJlj8RCbVe3Gd3eJQkAV8SAC6b19XS9BjMvRdvaS8RMkaTN8ZhoHP6S1y8zzwEQ==",
+      "version": "1.0.30001335",
+      "resolved": "https://registry.npmjs.org/caniuse-lite/-/caniuse-lite-1.0.30001335.tgz",
+      "integrity": "sha512-ddP1Tgm7z2iIxu6QTtbZUv6HJxSaV/PZeSrWFZtbY4JZ69tOeNhBCl3HyRQgeNZKE5AOn1kpV7fhljigy0Ty3w==",
       "dev": true
     },
     "chokidar": {
@@ -160,9 +161,9 @@
       }
     },
     "electron-to-chromium": {
-      "version": "1.4.206",
-      "resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.4.206.tgz",
-      "integrity": "sha512-h+Fadt1gIaQ06JaIiyqPsBjJ08fV5Q7md+V8bUvQW/9OvXfL2LRICTz2EcnnCP7QzrFTS6/27MRV6Bl9Yn97zA==",
+      "version": "1.4.129",
+      "resolved": "https://registry.npmjs.org/electron-to-chromium/-/electron-to-chromium-1.4.129.tgz",
+      "integrity": "sha512-GgtN6bsDtHdtXJtlMYZWGB/uOyjZWjmRDumXTas7dGBaB9zUyCjzHet1DY2KhyHN8R0GLbzZWqm4efeddqqyRQ==",
       "dev": true
     },
     "emoji-regex": {
@@ -347,9 +348,9 @@
       }
     },
     "node-releases": {
-      "version": "2.0.6",
-      "resolved": "https://registry.npmjs.org/node-releases/-/node-releases-2.0.6.tgz",
-      "integrity": "sha512-PiVXnNuFm5+iYkLBNeq5211hvO38y63T0i2KKh2KnUs3RpzJ+JtODFjkD8yjLwnDkTYF1eKXheUwdssR+NRZdg==",
+      "version": "2.0.4",
+      "resolved": "https://registry.npmjs.org/node-releases/-/node-releases-2.0.4.tgz",
+      "integrity": "sha512-gbMzqQtTtDz/00jQzZ21PQzdI9PyLYqUSvD0p3naOhX4odFji0ZxYdnVwPTxmSwkmxhcFImpozceidSG+AgoPQ==",
       "dev": true
     },
     "normalize-path": {
@@ -361,7 +362,7 @@
     "normalize-range": {
       "version": "0.1.2",
       "resolved": "https://registry.npmjs.org/normalize-range/-/normalize-range-0.1.2.tgz",
-      "integrity": "sha512-bdok/XvKII3nUpklnV6P2hxtMNrCboOjAcyBuQnWEhO665FwrSNRxU+AqpsyvO6LgGYPspN+lu5CLtw4jPRKNA==",
+      "integrity": "sha1-LRDAa9/TEuqXd2laTShDlFa3WUI=",
       "dev": true
     },
     "path-type": {
@@ -532,16 +533,6 @@
       "integrity": "sha512-hAZsKq7Yy11Zu1DE0OzWjw7nnLZmJZYTDZZyEFHZdUhV8FkH5MCfoU1XMaxXovpyW5nq5scPqq0ZDP9Zyl04oQ==",
       "dev": true
     },
-    "update-browserslist-db": {
-      "version": "1.0.5",
-      "resolved": "https://registry.npmjs.org/update-browserslist-db/-/update-browserslist-db-1.0.5.tgz",
-      "integrity": "sha512-dteFFpCyvuDdr9S/ff1ISkKt/9YZxKjI9WlRR99c180GaztJtRa/fn18FdxGVKVsnPY7/a/FDN68mcvUmP4U7Q==",
-      "dev": true,
-      "requires": {
-        "escalade": "^3.1.1",
-        "picocolors": "^1.0.0"
-      }
-    },
     "wrap-ansi": {
       "version": "7.0.0",
       "resolved": "https://registry.npmjs.org/wrap-ansi/-/wrap-ansi-7.0.0.tgz",
diff --git a/site/package.json b/site/package.json
index 69affc14e..21f0eff29 100644
--- a/site/package.json
+++ b/site/package.json
@@ -18,7 +18,7 @@
   "homepage": "https://github.com/google/docsy-example#readme",
   "dependencies": {},
   "devDependencies": {
-    "autoprefixer": "^10.4.8",
+    "autoprefixer": "^10.4.7",
     "postcss-cli": "^10.0.0"
   }
 }
diff --git a/vendor/modules.txt b/vendor/modules.txt
index 80edb24d2..1a21cc547 100644
--- a/vendor/modules.txt
+++ b/vendor/modules.txt
@@ -21,7 +21,7 @@ github.com/Azure/go-autorest
 ## explicit; go 1.15
 github.com/Azure/go-autorest/autorest
 github.com/Azure/go-autorest/autorest/azure
-# github.com/Azure/go-autorest/autorest/adal v0.9.21
+# github.com/Azure/go-autorest/autorest/adal v0.9.20
 ## explicit; go 1.15
 github.com/Azure/go-autorest/autorest/adal
 # github.com/Azure/go-autorest/autorest/date v0.3.0


Test patch:
diff --git a/.pipelines/run-autoscaling-e2e.yml b/.pipelines/run-autoscaling-e2e.yml
index 696e8077a..9c32e45a6 100644
--- a/.pipelines/run-autoscaling-e2e.yml
+++ b/.pipelines/run-autoscaling-e2e.yml
@@ -1,13 +1,14 @@
-schedules:
-- cron: "0 17 * * *"
-  displayName: Daily AKS Autoscaling E2E Test
+pr:
   branches:
     include:
     - master
-
-trigger: none
-
-pr: none
+    - release-*
+  paths:
+    exclude:
+    - docs
+    - site
+    - helm
+    - .github
 
 pool:
   vmImage: ubuntu-18.04
@@ -59,7 +60,7 @@ stages:
             REGISTRY_PASSWORD: $(registry.password)
     - job: create_aks_cluster_and_run_tests_job
       dependsOn: [build_push_ccm_image_job, build_push_cnm_image_job]
-      timeoutInMinutes: 300
+      timeoutInMinutes: 100
       steps:
         - bash: |
             CLUSTER_TYPE="autoscaling"
diff --git a/.pipelines/run-autoscaling-multipool-e2e.yml b/.pipelines/run-autoscaling-multipool-e2e.yml
index 81adf40b0..15eb8cb23 100644
--- a/.pipelines/run-autoscaling-multipool-e2e.yml
+++ b/.pipelines/run-autoscaling-multipool-e2e.yml
@@ -1,13 +1,14 @@
-schedules:
-- cron: "0 22 * * *"
-  displayName: Daily AKS Autoscaling Multipool E2E Test
+pr:
   branches:
     include:
     - master
-
-trigger: none
-
-pr: none
+    - release-*
+  paths:
+    exclude:
+    - docs
+    - site
+    - helm
+    - .github
 
 pool:
   vmImage: ubuntu-18.04
@@ -59,7 +60,7 @@ stages:
             REGISTRY_PASSWORD: $(registry.password)
     - job: create_aks_cluster_and_run_tests_job
       dependsOn: [build_push_ccm_image_job, build_push_cnm_image_job]
-      timeoutInMinutes: 300
+      timeoutInMinutes: 100
       steps:
         - bash: |
             CLUSTER_TYPE="autoscaling-multipool"
diff --git a/.pipelines/run-basic-lb-e2e.yml b/.pipelines/run-basic-lb-e2e.yml
index fb5402938..a42f105f1 100644
--- a/.pipelines/run-basic-lb-e2e.yml
+++ b/.pipelines/run-basic-lb-e2e.yml
@@ -60,10 +60,10 @@ stages:
             REGISTRY_PASSWORD: $(registry.password)
     - job: create_aks_cluster_and_run_tests_job
       dependsOn: [build_push_ccm_image_job, build_push_cnm_image_job]
-      timeoutInMinutes: 300
+      timeoutInMinutes: 100
       steps:
         - bash: |
-            CLUSTER_TYPE="basic-lb"
+            CCLUSTER_TYPE="basic-lb"
             echo "##vso[task.setvariable variable=CLUSTER_TYPE]${CLUSTER_TYPE}"
           displayName: set cluster type
         - bash: |
diff --git a/.pipelines/scripts/run-e2e.sh b/.pipelines/scripts/run-e2e.sh
index 2ba77cb1c..9eb033234 100755
--- a/.pipelines/scripts/run-e2e.sh
+++ b/.pipelines/scripts/run-e2e.sh
@@ -18,7 +18,7 @@ set -o errexit
 set -o nounset
 set -o pipefail
 
-REPO_ROOT=$(realpath $(dirname "${BASH_SOURCE[0]}")/../..)
+REPO_ROOT=$(dirname "${BASH_SOURCE[0]}")/../..
 export GOPATH="/home/vsts/go"
 export PATH="${PATH}:${GOPATH}/bin"
 export AKS_CLUSTER_ID="/subscriptions/${AZURE_SUBSCRIPTION_ID}/resourcegroups/${RESOURCE_GROUP}/providers/Microsoft.ContainerService/managedClusters/${CLUSTER_NAME}"
@@ -27,7 +27,7 @@ az extension add -n aks-preview
 az login --service-principal -u "${AZURE_CLIENT_ID}" -p "${AZURE_CLIENT_SECRET}" --tenant "${AZURE_TENANT_ID}"
 
 get_random_location() {
-  local LOCATIONS=("eastus" "eastus2" "southcentralus" "westus3")
+  local LOCATIONS=("eastus" "eastus2" "southcentralus" "westus3" "australiaeast")
   echo "${LOCATIONS[${RANDOM} % ${#LOCATIONS[@]}]}"
 }
 
@@ -39,9 +39,6 @@ cleanup() {
 trap cleanup EXIT
 
 export AZURE_LOCATION="$(get_random_location)"
-if [[ "${CLUSTER_TYPE}" =~ "autoscaling" ]]; then
-  export AZURE_LOCATION="australiaeast"
-fi
 
 echo "Setting up customconfiguration.json"
 IMAGE_TAG="$(git rev-parse --short=7 HEAD)"
@@ -86,14 +83,5 @@ kubectl wait --for=condition=Ready node --all --timeout=5m
 kubectl get node -owide
 
 echo "Running e2e"
-
-# TODO: We should do it in autoscaling-multipool.json
-if [[ "${CLUSTER_TYPE}" == "autoscaling-multipool" ]]; then
-  az aks update --resource-group "${RESOURCE_GROUP}" --name "${CLUSTER_NAME}" --cluster-autoscaler-profile balance-similar-node-groups=true
-fi
-
 export E2E_ON_AKS_CLUSTER=true
-if [[ "${CLUSTER_TYPE}" =~ "autoscaling" ]]; then
-  export LABEL_FILTER="Feature:Autoscaling"
-fi
 make test-ccm-e2e
diff --git a/cmd/cloud-controller-manager/app/testing/testserver.go b/cmd/cloud-controller-manager/app/testing/testserver.go
index a34a22df8..8a0ac91dd 100644
--- a/cmd/cloud-controller-manager/app/testing/testserver.go
+++ b/cmd/cloud-controller-manager/app/testing/testserver.go
@@ -57,8 +57,8 @@ type Logger interface {
 // and location of the tmpdir are returned.
 //
 // Note: we return a tear-down func instead of a stop channel because the later will leak temporary
-// files that because Golang testing's call to os.Exit will not give a stop channel go routine
-// enough time to remove temporary files.
+// 		 files that because Golang testing's call to os.Exit will not give a stop channel go routine
+// 		 enough time to remove temporary files.
 func StartTestServer(t Logger, customFlags []string) (result TestServer, err error) {
 	stopCh := make(chan struct{})
 	tearDown := func() {
diff --git a/hack/test-ccm-e2e.sh b/hack/test-ccm-e2e.sh
index 4de2de608..fb34fecf8 100755
--- a/hack/test-ccm-e2e.sh
+++ b/hack/test-ccm-e2e.sh
@@ -23,7 +23,7 @@ LABEL_FILTER="${LABEL_FILTER:-${DEFAULT_LABEL_FILTER}}"
 
 if [[ -n "${E2E_ON_AKS_CLUSTER}" ]]; then
   AKS_CLUSTER_E2E_LABEL_FILTER="!Autoscaling && !Multi-Nodepool && !Single-Nodepool && !VMSS && !Spot-VM && !Kubenet && !Multi-Group && !AvailabilitySet && !PLS && !SLBOutbound && !ServiceAnnotation && !Credential && !Node && !LB && !Multi-Ports && !NSG"
-  LABEL_FILTER="${AKS_CLUSTER_E2E_LABEL_FILTER}"
+  LABEL_FILTER="${LABEL_FILTER} && ${AKS_CLUSTER_E2E_LABEL_FILTER}"
 fi
 
 source "${REPO_ROOT}/hack/ensure-ginkgo-v2.sh"
diff --git a/pkg/azureclients/armclient/azure_armclient_test.go b/pkg/azureclients/armclient/azure_armclient_test.go
index 8b3a507b2..c9123aa13 100644
--- a/pkg/azureclients/armclient/azure_armclient_test.go
+++ b/pkg/azureclients/armclient/azure_armclient_test.go
@@ -19,7 +19,7 @@ package armclient
 import (
 	"context"
 	"fmt"
-	"io"
+	"io/ioutil"
 	"net/http"
 	"net/http/httptest"
 	"net/url"
@@ -372,7 +372,7 @@ func TestGetResource(t *testing.T) {
 			response, rerr := tc.getResource(ctx, armClient, tc.expectedAPIVersion, tc.params)
 			assert.Nil(t, rerr)
 			assert.NotNil(t, response)
-			byteResponseBody, _ := io.ReadAll(response.Body)
+			byteResponseBody, _ := ioutil.ReadAll(response.Body)
 			stringResponseBody := string(byteResponseBody)
 			assert.Equal(t, "{data: testPIP}", stringResponseBody)
 			assert.Equal(t, 1, count)
diff --git a/pkg/provider/azure_test.go b/pkg/provider/azure_test.go
index 97fd85318..650d2b116 100644
--- a/pkg/provider/azure_test.go
+++ b/pkg/provider/azure_test.go
@@ -3233,12 +3233,12 @@ func TestCanCombineSharedAndPrivateRulesInSameGroup(t *testing.T) {
 		t.Fatalf("Expected security rule %q but it was not present", testRuleName23)
 	}
 
-	_, _, rule2Found = findSecurityRuleByName(*sg.SecurityRules, testRuleName3)
+	_, securityRule2, rule2Found = findSecurityRuleByName(*sg.SecurityRules, testRuleName3)
 	if !rule2Found {
 		t.Fatalf("Expected security rule %q but it was not present", testRuleName3)
 	}
 
-	_, _, rule4Found = findSecurityRuleByName(*sg.SecurityRules, expectedRuleName4)
+	_, securityRule4, rule4Found = findSecurityRuleByName(*sg.SecurityRules, expectedRuleName4)
 	if !rule4Found {
 		t.Fatalf("Expected security rule %q but it was not present", expectedRuleName4)
 	}
diff --git a/pkg/provider/azure_vmssflex_cache_test.go b/pkg/provider/azure_vmssflex_cache_test.go
new file mode 100644
index 000000000..c24a2fef0
--- /dev/null
+++ b/pkg/provider/azure_vmssflex_cache_test.go
@@ -0,0 +1,455 @@
+/*
+Copyright 2022 The Kubernetes Authors.
+
+Licensed under the Apache License, Version 2.0 (the "License");
+you may not use this file except in compliance with the License.
+You may obtain a copy of the License at
+
+    http://www.apache.org/licenses/LICENSE-2.0
+
+Unless required by applicable law or agreed to in writing, software
+distributed under the License is distributed on an "AS IS" BASIS,
+WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+See the License for the specific language governing permissions and
+limitations under the License.
+*/
+
+package provider
+
+import (
+	"fmt"
+	"net/http"
+	"testing"
+
+	"github.com/Azure/azure-sdk-for-go/services/compute/mgmt/2021-07-01/compute"
+	"github.com/Azure/go-autorest/autorest/to"
+	"github.com/golang/mock/gomock"
+	"github.com/stretchr/testify/assert"
+
+	cloudprovider "k8s.io/cloud-provider"
+
+	"sigs.k8s.io/cloud-provider-azure/pkg/azureclients/vmclient/mockvmclient"
+	"sigs.k8s.io/cloud-provider-azure/pkg/azureclients/vmssclient/mockvmssclient"
+	azcache "sigs.k8s.io/cloud-provider-azure/pkg/cache"
+	"sigs.k8s.io/cloud-provider-azure/pkg/retry"
+)
+
+var (
+	testVMWithoutInstanceView1 = compute.VirtualMachine{
+		Name: to.StringPtr("testvm1"),
+		VirtualMachineProperties: &compute.VirtualMachineProperties{
+			ProvisioningState: nil,
+			VirtualMachineScaleSet: &compute.SubResource{
+				ID: to.StringPtr("subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssflex1"),
+			},
+		},
+	}
+
+	testVMWithoutInstanceView2 = compute.VirtualMachine{
+		Name: to.StringPtr("testvm2"),
+		VirtualMachineProperties: &compute.VirtualMachineProperties{
+			ProvisioningState: nil,
+			VirtualMachineScaleSet: &compute.SubResource{
+				ID: to.StringPtr("subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssflex2"),
+			},
+		},
+	}
+	testVMListWithoutInstanceView = []compute.VirtualMachine{testVMWithoutInstanceView1, testVMWithoutInstanceView2}
+
+	testVMWithOnlyInstanceView1 = compute.VirtualMachine{
+		Name: to.StringPtr("testvm1"),
+		VirtualMachineProperties: &compute.VirtualMachineProperties{
+			InstanceView: &compute.VirtualMachineInstanceView{
+				Statuses: &[]compute.InstanceViewStatus{
+					{
+						Code: to.StringPtr("PowerState/running"),
+					},
+				},
+			},
+		},
+	}
+
+	testVMWithOnlyInstanceView2 = compute.VirtualMachine{
+		Name: to.StringPtr("testvm2"),
+		VirtualMachineProperties: &compute.VirtualMachineProperties{
+			InstanceView: &compute.VirtualMachineInstanceView{
+				Statuses: &[]compute.InstanceViewStatus{
+					{
+						Code: to.StringPtr("PowerState/running"),
+					},
+				},
+			},
+		},
+	}
+	testVMListWithOnlyInstanceView = []compute.VirtualMachine{testVMWithOnlyInstanceView1, testVMWithOnlyInstanceView2}
+
+	testVM1 = compute.VirtualMachine{
+		Name: to.StringPtr("testvm1"),
+		VirtualMachineProperties: &compute.VirtualMachineProperties{
+			ProvisioningState: nil,
+			VirtualMachineScaleSet: &compute.SubResource{
+				ID: to.StringPtr("subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssflex1"),
+			},
+			InstanceView: &compute.VirtualMachineInstanceView{
+				Statuses: &[]compute.InstanceViewStatus{
+					{
+						Code: to.StringPtr("PowerState/running"),
+					},
+				},
+			},
+		},
+	}
+
+	testVmssFlex1 = compute.VirtualMachineScaleSet{
+		ID:   to.StringPtr("subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssflex1"),
+		Name: to.StringPtr("vmssflex1"),
+		VirtualMachineScaleSetProperties: &compute.VirtualMachineScaleSetProperties{
+			VirtualMachineProfile: &compute.VirtualMachineScaleSetVMProfile{},
+			OrchestrationMode:     compute.OrchestrationModeFlexible,
+		},
+	}
+
+	testVmssFlex2 = compute.VirtualMachineScaleSet{
+		ID:   to.StringPtr("subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssflex2"),
+		Name: to.StringPtr("vmssflex2"),
+		VirtualMachineScaleSetProperties: &compute.VirtualMachineScaleSetProperties{
+			VirtualMachineProfile: &compute.VirtualMachineScaleSetVMProfile{},
+			OrchestrationMode:     compute.OrchestrationModeFlexible,
+		},
+	}
+
+	testVmssFlexList = []compute.VirtualMachineScaleSet{testVmssFlex1, testVmssFlex2}
+)
+
+func TestGetNodeVmssFlexID(t *testing.T) {
+	ctrl := gomock.NewController(t)
+	defer ctrl.Finish()
+
+	testCases := []struct {
+		description                    string
+		nodeName                       string
+		testVM                         compute.VirtualMachine
+		vmGetErr                       *retry.Error
+		testVMListWithoutInstanceView  []compute.VirtualMachine
+		testVMListWithOnlyInstanceView []compute.VirtualMachine
+		vmListErr                      error
+		expectedVmssFlexID             string
+		expectedErr                    error
+	}{
+		{
+			description:                    "getNodeVmssFlexID should return the VmssFlex ID that the node belongs to",
+			nodeName:                       "testvm1",
+			testVM:                         testVMWithoutInstanceView1,
+			vmGetErr:                       nil,
+			testVMListWithoutInstanceView:  testVMListWithoutInstanceView,
+			testVMListWithOnlyInstanceView: testVMListWithOnlyInstanceView,
+			vmListErr:                      nil,
+			expectedVmssFlexID:             "subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssflex1",
+			expectedErr:                    nil,
+		},
+		{
+			description:                    "getNodeVmssFlexID should throw InstanceNotFound error if the VM cannot be found",
+			nodeName:                       "testvm3",
+			testVM:                         compute.VirtualMachine{},
+			vmGetErr:                       &retry.Error{HTTPStatusCode: http.StatusNotFound},
+			testVMListWithoutInstanceView:  []compute.VirtualMachine{},
+			testVMListWithOnlyInstanceView: []compute.VirtualMachine{},
+			vmListErr:                      nil,
+			expectedVmssFlexID:             "",
+			expectedErr:                    cloudprovider.InstanceNotFound,
+		},
+	}
+
+	for _, tc := range testCases {
+		fs, err := NewTestFlexScaleSet(ctrl)
+		assert.NoError(t, err, "unexpected error when creating test FlexScaleSet")
+
+		mockVMClient := fs.VirtualMachinesClient.(*mockvmclient.MockInterface)
+		mockVMClient.EXPECT().Get(gomock.Any(), fs.ResourceGroup, tc.nodeName, gomock.Any()).Return(tc.testVM, tc.vmGetErr).AnyTimes()
+
+		mockVMClient.EXPECT().ListVmssFlexVMsWithoutInstanceView(gomock.Any(), gomock.Any()).Return(tc.testVMListWithoutInstanceView, tc.vmListErr).AnyTimes()
+		mockVMClient.EXPECT().ListVmssFlexVMsWithOnlyInstanceView(gomock.Any(), gomock.Any()).Return(tc.testVMListWithOnlyInstanceView, tc.vmListErr).AnyTimes()
+
+		vmssFlexID, err := fs.getNodeVmssFlexID(tc.nodeName)
+		assert.Equal(t, tc.expectedErr, err, tc.description)
+		assert.Equal(t, tc.expectedVmssFlexID, vmssFlexID, tc.description)
+	}
+}
+
+func TestGetVmssFlexVM(t *testing.T) {
+	ctrl := gomock.NewController(t)
+	defer ctrl.Finish()
+	testCases := []struct {
+		description                    string
+		nodeName                       string
+		testVM                         compute.VirtualMachine
+		vmGetErr                       *retry.Error
+		testVMListWithoutInstanceView  []compute.VirtualMachine
+		testVMListWithOnlyInstanceView []compute.VirtualMachine
+		vmListErr                      error
+		expectedVmssFlexVM             compute.VirtualMachine
+		expectedErr                    error
+	}{
+		{
+			description:                    "getVmssFlexVM should return the VmssFlex VM",
+			nodeName:                       "testvm1",
+			testVM:                         testVMWithoutInstanceView1,
+			vmGetErr:                       nil,
+			testVMListWithoutInstanceView:  testVMListWithoutInstanceView,
+			testVMListWithOnlyInstanceView: testVMListWithOnlyInstanceView,
+			vmListErr:                      nil,
+			expectedVmssFlexVM:             testVM1,
+			expectedErr:                    nil,
+		},
+		{
+			description:                    "getVmssFlexVM should throw InstanceNotFound error if the VM cannot be found",
+			nodeName:                       "testvm1",
+			testVM:                         compute.VirtualMachine{},
+			vmGetErr:                       &retry.Error{HTTPStatusCode: http.StatusNotFound},
+			testVMListWithoutInstanceView:  []compute.VirtualMachine{},
+			testVMListWithOnlyInstanceView: []compute.VirtualMachine{},
+			vmListErr:                      nil,
+			expectedVmssFlexVM:             compute.VirtualMachine{},
+			expectedErr:                    cloudprovider.InstanceNotFound,
+		},
+		{
+			description:                    "getVmssFlexVM should throw InstanceNotFound error if the VM is removed from VMSS Flex",
+			nodeName:                       "testvm1",
+			testVM:                         testVMWithoutInstanceView1,
+			vmGetErr:                       nil,
+			testVMListWithoutInstanceView:  []compute.VirtualMachine{testVMWithoutInstanceView2},
+			testVMListWithOnlyInstanceView: []compute.VirtualMachine{testVMWithOnlyInstanceView2},
+			vmListErr:                      nil,
+			expectedVmssFlexVM:             compute.VirtualMachine{},
+			expectedErr:                    cloudprovider.InstanceNotFound,
+		},
+	}
+
+	for _, tc := range testCases {
+		fs, err := NewTestFlexScaleSet(ctrl)
+		assert.NoError(t, err, "unexpected error when creating test FlexScaleSet")
+
+		mockVMClient := fs.VirtualMachinesClient.(*mockvmclient.MockInterface)
+		mockVMClient.EXPECT().Get(gomock.Any(), fs.ResourceGroup, tc.nodeName, gomock.Any()).Return(tc.testVM, tc.vmGetErr).AnyTimes()
+		mockVMClient.EXPECT().ListVmssFlexVMsWithoutInstanceView(gomock.Any(), gomock.Any()).Return(tc.testVMListWithoutInstanceView, tc.vmListErr).AnyTimes()
+		mockVMClient.EXPECT().ListVmssFlexVMsWithOnlyInstanceView(gomock.Any(), gomock.Any()).Return(tc.testVMListWithOnlyInstanceView, tc.vmListErr).AnyTimes()
+
+		vmssFlexVM, err := fs.getVmssFlexVM(tc.nodeName, azcache.CacheReadTypeDefault)
+		assert.Equal(t, tc.expectedErr, err, tc.description)
+		assert.Equal(t, tc.expectedVmssFlexVM, vmssFlexVM, tc.description)
+	}
+
+}
+
+func TestGetVmssFlexByVmssFlexID(t *testing.T) {
+	ctrl := gomock.NewController(t)
+	defer ctrl.Finish()
+
+	testCases := []struct {
+		description      string
+		vmssFlexID       string
+		testVmssFlexList []compute.VirtualMachineScaleSet
+		vmssFlexListErr  *retry.Error
+		expectedVmssFlex *compute.VirtualMachineScaleSet
+		expectedErr      error
+	}{
+		{
+			description:      "getVmssFlexByVmssFlexID should return the corresponding vmssFlex by its ID",
+			vmssFlexID:       "subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssflex1",
+			testVmssFlexList: testVmssFlexList,
+			vmssFlexListErr:  nil,
+			expectedVmssFlex: &testVmssFlex1,
+			expectedErr:      nil,
+		},
+		{
+			description:      "getVmssFlexByVmssFlexID should return cloudprovider.InstanceNotFound if there's no matching VMSS",
+			vmssFlexID:       "subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssflex1",
+			testVmssFlexList: []compute.VirtualMachineScaleSet{testVmssFlex2},
+			vmssFlexListErr:  nil,
+			expectedVmssFlex: nil,
+			expectedErr:      cloudprovider.InstanceNotFound,
+		},
+		{
+			description:      "getVmssFlexByVmssFlexID  should report an error if there's something wrong during an api call",
+			vmssFlexID:       "subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssflex1",
+			testVmssFlexList: []compute.VirtualMachineScaleSet{},
+			vmssFlexListErr:  &retry.Error{RawError: fmt.Errorf("error during vmss list")},
+			expectedVmssFlex: nil,
+			expectedErr:      fmt.Errorf("Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error during vmss list"),
+		},
+	}
+
+	for _, tc := range testCases {
+		fs, err := NewTestFlexScaleSet(ctrl)
+		assert.NoError(t, err, "unexpected error when creating test FlexScaleSet")
+
+		mockVMSSClient := fs.cloud.VirtualMachineScaleSetsClient.(*mockvmssclient.MockInterface)
+		mockVMSSClient.EXPECT().List(gomock.Any(), gomock.Any()).Return(tc.testVmssFlexList, tc.vmssFlexListErr).AnyTimes()
+
+		vmssFlex, err := fs.getVmssFlexByVmssFlexID(tc.vmssFlexID, azcache.CacheReadTypeDefault)
+		if tc.expectedErr != nil {
+			assert.EqualError(t, tc.expectedErr, err.Error(), tc.description)
+		}
+		assert.Equal(t, tc.expectedVmssFlex, vmssFlex, tc.description)
+	}
+}
+
+func TestGetVmssFlexIDByName(t *testing.T) {
+	ctrl := gomock.NewController(t)
+	defer ctrl.Finish()
+
+	testCases := []struct {
+		description        string
+		vmssFlexName       string
+		testVmssFlexList   []compute.VirtualMachineScaleSet
+		vmssFlexListErr    *retry.Error
+		expectedVmssFlexID string
+		expectedErr        error
+	}{
+		{
+			description:        "getVmssFlexIDByName should return the corresponding vmssFlex by its ID",
+			vmssFlexName:       "vmssflex1",
+			testVmssFlexList:   testVmssFlexList,
+			vmssFlexListErr:    nil,
+			expectedVmssFlexID: "subscriptions/sub/resourceGroups/rg/providers/Microsoft.Compute/virtualMachineScaleSets/vmssflex1",
+			expectedErr:        nil,
+		},
+		{
+			description:        "getVmssFlexIDByName should return cloudprovider.InstanceNotFound if there's no matching VMSS",
+			vmssFlexName:       "vmssflex1",
+			testVmssFlexList:   []compute.VirtualMachineScaleSet{testVmssFlex2},
+			vmssFlexListErr:    nil,
+			expectedVmssFlexID: "",
+			expectedErr:        cloudprovider.InstanceNotFound,
+		},
+		{
+			description:        "getVmssFlexIDByName should report an error if there's something wrong during an api call",
+			vmssFlexName:       "vmssflex1",
+			testVmssFlexList:   []compute.VirtualMachineScaleSet{},
+			vmssFlexListErr:    &retry.Error{RawError: fmt.Errorf("error during vmss list")},
+			expectedVmssFlexID: "",
+			expectedErr:        fmt.Errorf("Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error during vmss list"),
+		},
+	}
+
+	for _, tc := range testCases {
+		fs, err := NewTestFlexScaleSet(ctrl)
+		assert.NoError(t, err, "unexpected error when creating test FlexScaleSet")
+
+		mockVMSSClient := fs.cloud.VirtualMachineScaleSetsClient.(*mockvmssclient.MockInterface)
+		mockVMSSClient.EXPECT().List(gomock.Any(), gomock.Any()).Return(tc.testVmssFlexList, tc.vmssFlexListErr).AnyTimes()
+
+		vmssFlexID, err := fs.getVmssFlexIDByName(tc.vmssFlexName)
+
+		assert.Equal(t, tc.expectedVmssFlexID, vmssFlexID, tc.description)
+		if tc.expectedErr != nil {
+			assert.EqualError(t, tc.expectedErr, err.Error(), tc.description)
+		}
+	}
+
+}
+
+func TestGetVmssFlexByName(t *testing.T) {
+	ctrl := gomock.NewController(t)
+	defer ctrl.Finish()
+
+	testCases := []struct {
+		description      string
+		vmssFlexName     string
+		testVmssFlexList []compute.VirtualMachineScaleSet
+		vmssFlexListErr  *retry.Error
+		expectedVmssFlex *compute.VirtualMachineScaleSet
+		expectedErr      error
+	}{
+		{
+			description:      "getVmssFlexByName should return the corresponding vmssFlex by its ID",
+			vmssFlexName:     "vmssflex1",
+			testVmssFlexList: testVmssFlexList,
+			vmssFlexListErr:  nil,
+			expectedVmssFlex: &testVmssFlex1,
+			expectedErr:      nil,
+		},
+		{
+			description:      "getVmssFlexByName should return cloudprovider.InstanceNotFound if there's no matching VMSS",
+			vmssFlexName:     "vmssflex1",
+			testVmssFlexList: []compute.VirtualMachineScaleSet{testVmssFlex2},
+			vmssFlexListErr:  nil,
+			expectedVmssFlex: nil,
+			expectedErr:      cloudprovider.InstanceNotFound,
+		},
+		{
+			description:      "getVmssFlexByName should report an error if there's something wrong during an api call",
+			vmssFlexName:     "vmssflex1",
+			testVmssFlexList: []compute.VirtualMachineScaleSet{},
+			vmssFlexListErr:  &retry.Error{RawError: fmt.Errorf("error during vmss list")},
+			expectedVmssFlex: nil,
+			expectedErr:      fmt.Errorf("Retriable: false, RetryAfter: 0s, HTTPStatusCode: 0, RawError: error during vmss list"),
+		},
+	}
+
+	for _, tc := range testCases {
+		fs, err := NewTestFlexScaleSet(ctrl)
+		assert.NoError(t, err, "unexpected error when creating test FlexScaleSet")
+
+		mockVMSSClient := fs.cloud.VirtualMachineScaleSetsClient.(*mockvmssclient.MockInterface)
+		mockVMSSClient.EXPECT().List(gomock.Any(), gomock.Any()).Return(tc.testVmssFlexList, tc.vmssFlexListErr).AnyTimes()
+
+		vmssFlex, err := fs.getVmssFlexByName(tc.vmssFlexName)
+
+		assert.Equal(t, tc.expectedVmssFlex, vmssFlex, tc.description)
+		if tc.expectedErr != nil {
+			assert.EqualError(t, tc.expectedErr, err.Error(), tc.description)
+		}
+	}
+
+}
+
+func TestGetVmssFlexByNodeName(t *testing.T) {
+	ctrl := gomock.NewController(t)
+	defer ctrl.Finish()
+
+	testCases := []struct {
+		description                    string
+		nodeName                       string
+		testVM                         compute.VirtualMachine
+		vmGetErr                       *retry.Error
+		testVMListWithoutInstanceView  []compute.VirtualMachine
+		testVMListWithOnlyInstanceView []compute.VirtualMachine
+		vmListErr                      error
+		testVmssFlexList               []compute.VirtualMachineScaleSet
+		vmssFlexListErr                *retry.Error
+		expectedVmssFlex               *compute.VirtualMachineScaleSet
+		expectedErr                    error
+	}{
+		{
+			description:                    "getVmssFlexByName should return the VmssFlex ID that the node belongs to",
+			nodeName:                       "testvm1",
+			testVM:                         testVMWithoutInstanceView1,
+			vmGetErr:                       nil,
+			testVMListWithoutInstanceView:  testVMListWithoutInstanceView,
+			testVMListWithOnlyInstanceView: testVMListWithOnlyInstanceView,
+			vmListErr:                      nil,
+			testVmssFlexList:               testVmssFlexList,
+			vmssFlexListErr:                nil,
+			expectedVmssFlex:               &testVmssFlex1,
+			expectedErr:                    nil,
+		},
+	}
+
+	for _, tc := range testCases {
+		fs, err := NewTestFlexScaleSet(ctrl)
+		assert.NoError(t, err, "unexpected error when creating test FlexScaleSet")
+
+		mockVMClient := fs.VirtualMachinesClient.(*mockvmclient.MockInterface)
+		mockVMClient.EXPECT().Get(gomock.Any(), fs.ResourceGroup, tc.nodeName, gomock.Any()).Return(tc.testVM, tc.vmGetErr).AnyTimes()
+		mockVMClient.EXPECT().ListVmssFlexVMsWithoutInstanceView(gomock.Any(), gomock.Any()).Return(tc.testVMListWithoutInstanceView, tc.vmListErr).AnyTimes()
+		mockVMClient.EXPECT().ListVmssFlexVMsWithOnlyInstanceView(gomock.Any(), gomock.Any()).Return(tc.testVMListWithOnlyInstanceView, tc.vmListErr).AnyTimes()
+		mockVMSSClient := fs.cloud.VirtualMachineScaleSetsClient.(*mockvmssclient.MockInterface)
+		mockVMSSClient.EXPECT().List(gomock.Any(), gomock.Any()).Return(tc.testVmssFlexList, tc.vmssFlexListErr).AnyTimes()
+
+		vmssFlex, err := fs.getVmssFlexByNodeName(tc.nodeName, azcache.CacheReadTypeDefault)
+		assert.Equal(t, tc.expectedErr, err, tc.description)
+		assert.Equal(t, tc.expectedVmssFlex, vmssFlex, tc.description)
+	}
+
+}
diff --git a/tests/e2e/utils/node_utils.go b/tests/e2e/utils/node_utils.go
index fd897aec5..eae916368 100644
--- a/tests/e2e/utils/node_utils.go
+++ b/tests/e2e/utils/node_utils.go
@@ -188,7 +188,7 @@ func DeleteNodes(cs clientset.Interface, names []string) error {
 	return nil
 }
 
-// deleteNodes deletes nodes according to names
+//deleteNodes deletes nodes according to names
 func deleteNode(cs clientset.Interface, name string) error {
 	Logf("Deleting node: %s", name)
 	if err := cs.CoreV1().Nodes().Delete(context.TODO(), name, metav1.DeleteOptions{}); err != nil {