Change:
assert.False(t, isRegionValid("pl-fake-991a", fake.metadata), "expected region 'pl-fake-991' to be invalid but it was not") } <del> func TestGetCandidateZonesForDynamicVolume(t *testing.T) { tests := []struct { name          string labels        map[string]string ready         bool expectedZones sets.String }{ { name:          "master node with role label", labels:        map[string]string{labelKeyNodeRole: nodeMasterRole, v1.LabelZoneFailureDomain: "us-east-1a"}, ready:         true, expectedZones: sets.NewString(), }, { name:          "master node with master label empty", labels:        map[string]string{labelKeyNodeMaster: "", v1.LabelZoneFailureDomain: "us-east-1a"}, ready:         true, expectedZones: sets.NewString(), }, { name:          "master node with master label true", labels:        map[string]string{labelKeyNodeMaster: "true", v1.LabelZoneFailureDomain: "us-east-1a"}, ready:         true, expectedZones: sets.NewString(), }, { name:          "master node with master label false", labels:        map[string]string{labelKeyNodeMaster: "false", v1.LabelZoneFailureDomain: "us-east-1a"}, ready:         true, expectedZones: sets.NewString("us-east-1a"), }, { name:          "minion node with role label", labels:        map[string]string{labelKeyNodeRole: nodeMinionRole, v1.LabelZoneFailureDomain: "us-east-1a"}, ready:         true, expectedZones: sets.NewString("us-east-1a"), }, { name:          "minion node with minion label", labels:        map[string]string{labelKeyNodeMinion: "", v1.LabelZoneFailureDomain: "us-east-1a"}, ready:         true, expectedZones: sets.NewString("us-east-1a"), }, { name:          "minion node with compute label", labels:        map[string]string{labelKeyNodeCompute: "true", v1.LabelZoneFailureDomain: "us-east-1a"}, ready:         true, expectedZones: sets.NewString("us-east-1a"), }, { name:          "master and minion node", labels:        map[string]string{labelKeyNodeMaster: "true", labelKeyNodeCompute: "true", v1.LabelZoneFailureDomain: "us-east-1a"}, ready:         true, expectedZones: sets.NewString("us-east-1a"), }, { name:          "node not ready", labels:        map[string]string{v1.LabelZoneFailureDomain: "us-east-1a"}, ready:         false, expectedZones: sets.NewString(), }, { name:          "node has no zone", labels:        map[string]string{}, ready:         true, expectedZones: sets.NewString(), }, { name:          "node with no label", labels:        map[string]string{v1.LabelZoneFailureDomain: "us-east-1a"}, ready:         true, expectedZones: sets.NewString("us-east-1a"), }, } for i, test := range tests { t.Run(test.name, func(t *testing.T) { awsServices := newMockedFakeAWSServices(TestClusterID) c, _ := newAWSCloud(CloudConfig{}, awsServices) c.kubeClient = fake.NewSimpleClientset() nodeName := fmt.Sprintf("node-%d", i) _, err := c.kubeClient.CoreV1().Nodes().Create(&v1.Node{ ObjectMeta: metav1.ObjectMeta{ Name:   nodeName, Labels: test.labels, }, Status: genNodeStatus(test.ready), }) assert.Nil(t, err) zones, err := c.GetCandidateZonesForDynamicVolume() assert.Nil(t, err) assert.Equal(t, test.expectedZones, zones) }) } } func genNodeStatus(ready bool) v1.NodeStatus { status := v1.ConditionFalse if ready { status = v1.ConditionTrue } ret := v1.NodeStatus{ Conditions: []v1.NodeCondition{ { Type:   v1.NodeReady, Status: status, }, }, } return ret } </del> func newMockedFakeAWSServices(id string) *FakeAWSServices { s := NewFakeAWSServices(id) s.ec2 = &MockedFakeEC2{FakeEC2Impl: s.ec2.(*FakeEC2Impl)}

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: We've seen scalability issues with the current aws-ebs volume provisioner impl, which is querying possible zones from cloud provider directly (). The improvement proposed in did make api calls more efficient, but also raises people's concern about whether it will create regression or not, or is there any more reliable ways to improve. To summarize, there are 2 major concerns: zone info in Node label trustworthy? My answer is yes, since currently the zone labels (beta version only) are maintained by node controller, the information is retrieved from cloud provider directly and is always updated during node reconciliation, it can be trusted. Even if kubelet can be misconfigured by wrong zone information, node controller is going to override them. See: Once we have the GA version of zone labels pushed out, we should update node controller to maintain both zone labels. It's fair enough to assume that if user wants kubelet to report zone/region, the information should be correct. the master/minion labels trustworthy? My answer is "semantic wide they are equivalent to checking node labels from cloud provider" - just as people are not required to set master/minion labels in v1.Node, well-known labels in cloud provider is not guaranteed to be set as well. Therefore besides mainstream cluster provisioners, which maintains these sets of labels in both Kubernetes and cloud provider, the function's behavior will remain unchanged: the original impl skips nodes that are explicitly labeled with well-known master labels, same here. Please suggest. /sig aws /sig cloud-provider /kind bug /priority important-soon /cc
for item 2, an alternative is to first call a List() of all v1.Nodes, get all providerID, and make a DescribeInstance call by passing all these IDs, which might still be more efficient than running the filter, and still very heavy-weighted with large scale. Current logic can be applied to the results though. Need input from aws side.
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available .

If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
One of the quick optimizations we can do is, to remove those API calls if pod placement has already been decided because of delayed binding. I think, using delayed binding works well in practice and should eliminate calls being made.
Another thing is - since EBS volume plugin is being targeted for CSI migration, we should be super careful approving any changes into in-tree plugins which is not available in EBS CSI plugin. cc
Thanks ! yes I do agree that for topo aware storage class, when we create volume, the node is already bound, so we just get the zone information from the selected node, and this is a good to have short-term improvement. In long term, as cloud providers are going to be out of tree, we have more freedom to introduce informer to the aws library (gce is already doing that), and get zone information from cache. Since the path we are going is to make kubernetes components node role neutral, it's fair enough to say that we don't have to filter out master only zone in volume provisioner, and user should leverage topology aware storage class along with labels, taints, tolerations to achieve that. what do you think?
I believe in long term we want to default all clusters to use policy. That is the direction sig-storage wants to take(IIRC ), so IMO it may not be worth fixing general immediate binding use case for in-tree EBS driver where cloudprovider itself is deprecated (and any proposed change is non-trivial).
if the direction sig-storage is going is to always use , I agree with you that fixing a deprecated path of a deprecated EBS driver would be not worthwhile. Meanwhile, since in-tree EBS driver will still stay for a few more versions, I'm proposing for optimizing code path. Could you please provide inputs? thanks! cc
Related to
Hello! I'm the bug triage lead for the 1.16 release cycle and considering this issue is tagged for 1.16, I'd like to check its status. The code freeze is starting on August 29th EOD PST (about 5 days from now) and once the code freeze starts, only release-blocking changes will be considered to be merged. Is this issue still planned to be fixed for the 1.16 release?

I have addressed this issue via , will close this issue /close
Closing this issue. <detailsIn response to : I have addressed this issue via , will close this issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details