jiebi/CodeConvo huggingface dataset:
Code change:
} } <ins> if utilfeature.DefaultFeatureGate.Enabled(features.HonorPVReclaimPolicy) { if metav1.HasAnnotation(volume.ObjectMeta, storagehelpers.AnnMigratedTo) { // CSI migration scenario - do not depend on in-tree plugin return nil, nil } if volume.Spec.CSI != nil { // CSI volume source scenario - external provisioner is requested return nil, nil } } </ins> // The plugin that provisioned the volume was not found or the volume // was not dynamically provisioned. Try to find a plugin by spec. spec := vol.NewSpecFromPersistentVolume(volume, false)

Related discussion:
text: Client Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.5", GitCommit:"", GitTreeState:"clean", BuildDate:"2023-08-24T00:48:26Z", GoVersion:"go1.20.7", Compiler:"gc", Platform:"linux/amd64"} Kustomize Version: v5.0.1 Server Version: version.Info{Major:"1", Minor:"27", GitVersion:"v1.27.5", GitCommit:"", GitTreeState:"clean", BuildDate:"2023-08-24T00:42:11Z", GoVersion:"go1.20.7", Compiler:"gc", Platform:"linux/amd64"} Vanilla K8s setup <details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig storage maybe similar issues should go in
Opened similar issue in external-provisioner repo also There is no reply on that as well. Can you please help understand the behavior of the feature HonorPVReclaimPolicy.
If you want Persistent Volume (PV) to be deleted when the associated Persistent Volume Claim (PVC) is deleted, you donot need to use finalizers explicitly. Kubernetes handles this behavior by default. It is ok that if you use to get the desire output using Finalizer we can do it.
As per my understanding Please follow the steps : Add finalizer in metadata of as you did above need to make change in the controller which is handling PV deletion in .go file. See whether the controller administrator is working or not. I think it will work.
Finalizer is because the feature gate HonorPVReclaimPolicy for csi-provisioner is enabled. We are not adding the finalizer explicitly. From the documentation, "The newly introduced finalizers and external- are only to dynamically provisioned volumes." Not sure why is the finalizer getting set.
Code change:
} } <ins> if utilfeature.DefaultFeatureGate.Enabled(features.HonorPVReclaimPolicy) { if metav1.HasAnnotation(volume.ObjectMeta, storagehelpers.AnnMigratedTo) { // CSI migration scenario - do not depend on in-tree plugin return nil, nil } if volume.Spec.CSI != nil { // CSI volume source scenario - external provisioner is requested return nil, nil } } </ins> // The plugin that provisioned the volume was not found or the volume // was not dynamically provisioned. Try to find a plugin by spec. spec := vol.NewSpecFromPersistentVolume(volume, false)

Related discussion:
text: Also, for deleting PV it requires the annotation "" but this annotation is set in case of dynamic PVs. Should it be set explicitly for static PV?
This feature is alpha so the feature gate is not enabled by default. Did you enable the feature gate on kube-controller-manager?
Yes, i have enabled this feature.
external- is also applied to statically provisioned volumes. FYI
Hi Which version of the external provisioner is the driver using? Does the driver meet the requirements of the CSI spec on the DeleteVolume PRC? FYI: Is this problem occasional or permanently reproducible? Can you provide the provisioner logs at that time?
/triage needs-information
cc
cc
External provisioner used is : "csi-provisioner:v3.5.0" This problem is always reproducible Provisioner logs: I1123 12:22:38. 1 ] handleProtectionFinalizer Volume : &PersistentVolume{ObjectMeta:{csi-infoscale-pv -420a-4e9e-be82- 0 2023-11-23 12:21:18 +0000 UTC <nil<nilmap[] map[{"apiVersion":"v1","kind":"PersistentVolume","metadata":{"annotations":{},"name":"csi-infoscale-pv"},"spec":{"accessModes":["ReadWriteOnce"],"capacity":{"storage":"2Gi"},"csi":{"driver":"org.veritas.infoscale","fsType":"vxfs","volumeHandle":"clust40559/vrtskubedg-/vol1"},"persistentVolumeReclaimPolicy":"Delete","storageClassName":"csi-infoscale-sc-prod"}} ] [] [ external-] [{kubectl-client-side-apply Update v1 2023-11-23 12:21:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"":{}}},"f:spec":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:csi":{".
Code change:
} } <ins> if utilfeature.DefaultFeatureGate.Enabled(features.HonorPVReclaimPolicy) { if metav1.HasAnnotation(volume.ObjectMeta, storagehelpers.AnnMigratedTo) { // CSI migration scenario - do not depend on in-tree plugin return nil, nil } if volume.Spec.CSI != nil { // CSI volume source scenario - external provisioner is requested return nil, nil } } </ins> // The plugin that provisioned the volume was not found or the volume // was not dynamically provisioned. Try to find a plugin by spec. spec := vol.NewSpecFromPersistentVolume(volume, false)

Related discussion:
text: ":{},"f:driver":{},"f:fsType":{},"f:volumeHandle":{}},"f:persistentVolumeReclaimPolicy":{},"f:storageClassName":{},"f:volumeMode":{}}} } {csi-provisioner Update v1 2023-11-23 12:21:22 +0000 UTC FieldsV1 {"f:metadata":{"f:finalizers":{"v:"external-"":{}}}} } {kube-controller-manager Update v1 2023-11-23 12:21:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"":{}}},"f:spec":{"f:claimRef":{".
Code change:
} } <ins> if utilfeature.DefaultFeatureGate.Enabled(features.HonorPVReclaimPolicy) { if metav1.HasAnnotation(volume.ObjectMeta, storagehelpers.AnnMigratedTo) { // CSI migration scenario - do not depend on in-tree plugin return nil, nil } if volume.Spec.CSI != nil { // CSI volume source scenario - external provisioner is requested return nil, nil } } </ins> // The plugin that provisioned the volume was not found or the volume // was not dynamically provisioned. Try to find a plugin by spec. spec := vol.NewSpecFromPersistentVolume(volume, false)

Related discussion:
text: ":{},"f:apiVersion":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}}}} } {kube-controller-manager Update v1 2023-11-23 12:22:38 +0000 UTC FieldsV1 {"f:status":{"f:phase":{}}} status}]},Spec:PersistentVolumeSpec{Capacity:ResourceList{storage: {{ 0} {<nil} 2Gi BinarySI},},PersistentVolumeSource:PersistentVolumeSource{GCEPersistentDisk:nil,AWSElasticBlockStore:nil,HostPath:nil,Glusterfs:nil,NFS:nil,RBD:nil,ISCSI:nil,Cinder:nil,CephFS:nil,FC:nil,Flocker:nil,FlexVolume:nil,AzureFile:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Local:nil,StorageOS:nil,CSI:&CSIPersistentVolumeSource{Driver:org.veritas.infoscale,VolumeHandle:clust40559/vrtskubedg-/vol1,ReadOnly:false,FSType:vxfs,VolumeAttributes:map[string]string{},ControllerPublishSecretRef:nil,NodeStageSecretRef:nil,NodePublishSecretRef:nil,ControllerExpandSecretRef:nil,NodeExpandSecretRef:nil,},},AccessModes:[ReadWriteOnce],ClaimRef:&ObjectReference{Kind:PersistentVolumeClaim,Namespace:default,Name:csi-infoscale-pvc-static,UID:-59a5-4f02-83ac-,APIVersion:v1,ResourceVersion:,FieldPath:,},PersistentVolumeReclaimPolicy:Delete,StorageClassName:csi-infoscale-sc-prod,MountOptions:[],VolumeMode:Filesystem,NodeAffinity:nil,},Status:PersistentVolumeStatus{Phase:Released,Message:,Reason:,},} I1123 12:22:38. 1 ] delete "csi-infoscale-pv": started I1123 12:22:38. 1 ] GRPC call: /csi.v1.Controller/DeleteVolume I1123 12:22:38. 1 ] GRPC request: {"volumeid":"clust40559/vrtskubedg-/vol1"} I1123 12:22:39.
Code change:
} } <ins> if utilfeature.DefaultFeatureGate.Enabled(features.HonorPVReclaimPolicy) { if metav1.HasAnnotation(volume.ObjectMeta, storagehelpers.AnnMigratedTo) { // CSI migration scenario - do not depend on in-tree plugin return nil, nil } if volume.Spec.CSI != nil { // CSI volume source scenario - external provisioner is requested return nil, nil } } </ins> // The plugin that provisioned the volume was not found or the volume // was not dynamically provisioned. Try to find a plugin by spec. spec := vol.NewSpecFromPersistentVolume(volume, false)

Related discussion:
text: 1 ] GRPC response: {} I1123 12:22:39. 1 ] GRPC error: <nilI1123 12:22:39. 1 ] delete "csi-infoscale-pv": volume deleted I1123 12:22:39. 1 ] delete "csi-infoscale-pv": failed to remove finalizer for persistentvolume: Operation cannot be fulfilled on persistentvolumes "csi-infoscale-pv": the object has been modified; please apply your changes to the latest version and try again W1123 12:22:39. 1 ] Retrying syncing volume "csi-infoscale-pv", failure 0 E1123 12:22:39. 1 ] error syncing volume "csi-infoscale-pv": Operation cannot be fulfilled on persistentvolumes "csi-infoscale-pv": the object has been modified; please apply your changes to the latest version and try again I1123 12:22:39. 1 ] handleProtectionFinalizer Volume : &PersistentVolume{ObjectMeta:{csi-infoscale-pv -420a-4e9e-be82- 0 2023-11-23 12:21:18 +0000 UTC 2023-11-23 12:22:38 +0000 UTC 0xc0007f63d0 map[] map[{"apiVersion":"v1","kind":"PersistentVolume","metadata":{"annotations":{},"name":"csi-infoscale-pv"},"spec":{"accessModes":["ReadWriteOnce"],"capacity":{"storage":"2Gi"},"csi":{"driver":"org.veritas.infoscale","fsType":"vxfs","volumeHandle":"clust40559/vrtskubedg-/vol1"},"persistentVolumeReclaimPolicy":"Delete","storageClassName":"csi-infoscale-sc-prod"}} ] [] [external-] [{kubectl-client-side-apply Update v1 2023-11-23 12:21:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"":{}}},"f:spec":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:csi":{".
Code change:
} } <ins> if utilfeature.DefaultFeatureGate.Enabled(features.HonorPVReclaimPolicy) { if metav1.HasAnnotation(volume.ObjectMeta, storagehelpers.AnnMigratedTo) { // CSI migration scenario - do not depend on in-tree plugin return nil, nil } if volume.Spec.CSI != nil { // CSI volume source scenario - external provisioner is requested return nil, nil } } </ins> // The plugin that provisioned the volume was not found or the volume // was not dynamically provisioned. Try to find a plugin by spec. spec := vol.NewSpecFromPersistentVolume(volume, false)

Related discussion:
text: ":{},"f:apiVersion":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}}}} } {kube-controller-manager Update v1 2023-11-23 12:22:38 +0000 UTC FieldsV1 {"f:status":{"f:message":{},"f:phase":{}}} status}]},Spec:PersistentVolumeSpec{Capacity:ResourceList{storage: {{ 0} {<nil} 2Gi BinarySI},},PersistentVolumeSource:PersistentVolumeSource{GCEPersistentDisk:nil,AWSElasticBlockStore:nil,HostPath:nil,Glusterfs:nil,NFS:nil,RBD:nil,ISCSI:nil,Cinder:nil,CephFS:nil,FC:nil,Flocker:nil,FlexVolume:nil,AzureFile:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Local:nil,StorageOS:nil,CSI:&CSIPersistentVolumeSource{Driver:org.veritas.infoscale,VolumeHandle:clust40559/vrtskubedg-/vol1,ReadOnly:false,FSType:vxfs,VolumeAttributes:map[string]string{},ControllerPublishSecretRef:nil,NodeStageSecretRef:nil,NodePublishSecretRef:nil,ControllerExpandSecretRef:nil,NodeExpandSecretRef:nil,},},AccessModes:[ReadWriteOnce],ClaimRef:&ObjectReference{Kind:PersistentVolumeClaim,Namespace:default,Name:csi-infoscale-pvc-static,UID:-59a5-4f02-83ac-,APIVersion:v1,ResourceVersion:,FieldPath:,},PersistentVolumeReclaimPolicy:Delete,StorageClassName:csi-infoscale-sc-prod,MountOptions:[],VolumeMode:Filesystem,NodeAffinity:nil,},Status:PersistentVolumeStatus{Phase:Failed,Message:error getting deleter volume plugin for volume "csi-infoscale-pv": no deletable volume plugin matched,Reason:,},} I1123 12:22:40.
Code change:
} } <ins> if utilfeature.DefaultFeatureGate.Enabled(features.HonorPVReclaimPolicy) { if metav1.HasAnnotation(volume.ObjectMeta, storagehelpers.AnnMigratedTo) { // CSI migration scenario - do not depend on in-tree plugin return nil, nil } if volume.Spec.CSI != nil { // CSI volume source scenario - external provisioner is requested return nil, nil } } </ins> // The plugin that provisioned the volume was not found or the volume // was not dynamically provisioned. Try to find a plugin by spec. spec := vol.NewSpecFromPersistentVolume(volume, false)

Related discussion:
text: 1 ] handleProtectionFinalizer Volume : &PersistentVolume{ObjectMeta:{csi-infoscale-pv -420a-4e9e-be82- 0 2023-11-23 12:21:18 +0000 UTC 2023-11-23 12:22:38 +0000 UTC 0xc0007f63d0 map[] map[{"apiVersion":"v1","kind":"PersistentVolume","metadata":{"annotations":{},"name":"csi-infoscale-pv"},"spec":{"accessModes":["ReadWriteOnce"],"capacity":{"storage":"2Gi"},"csi":{"driver":"org.veritas.infoscale","fsType":"vxfs","volumeHandle":"clust40559/vrtskubedg-/vol1"},"persistentVolumeReclaimPolicy":"Delete","storageClassName":"csi-infoscale-sc-prod"}} ] [] [external-] [{kubectl-client-side-apply Update v1 2023-11-23 12:21:18 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{".":{},"":{}}},"f:spec":{"f:accessModes":{},"f:capacity":{".":{},"f:storage":{}},"f:csi":{".":{},"f:driver":{},"f:fsType":{},"f:volumeHandle":{}},"f:persistentVolumeReclaimPolicy":{},"f:storageClassName":{},"f:volumeMode":{}}} } {csi-provisioner Update v1 2023-11-23 12:21:22 +0000 UTC FieldsV1 {"f:metadata":{"f:finalizers":{"v:"external-"":{}}}} } {kube-controller-manager Update v1 2023-11-23 12:21:22 +0000 UTC FieldsV1 {"f:metadata":{"f:annotations":{"":{}}},"f:spec":{"f:claimRef":{".
Code change:
} } <ins> if utilfeature.DefaultFeatureGate.Enabled(features.HonorPVReclaimPolicy) { if metav1.HasAnnotation(volume.ObjectMeta, storagehelpers.AnnMigratedTo) { // CSI migration scenario - do not depend on in-tree plugin return nil, nil } if volume.Spec.CSI != nil { // CSI volume source scenario - external provisioner is requested return nil, nil } } </ins> // The plugin that provisioned the volume was not found or the volume // was not dynamically provisioned. Try to find a plugin by spec. spec := vol.NewSpecFromPersistentVolume(volume, false)

Related discussion:
text: ":{},"f:apiVersion":{},"f:kind":{},"f:name":{},"f:namespace":{},"f:resourceVersion":{},"f:uid":{}}}} } {kube-controller-manager Update v1 2023-11-23 12:22:38 +0000 UTC FieldsV1 {"f:status":{"f:message":{},"f:phase":{}}} status}]},Spec:PersistentVolumeSpec{Capacity:ResourceList{storage: {{ 0} {<nil} 2Gi BinarySI},},PersistentVolumeSource:PersistentVolumeSource{GCEPersistentDisk:nil,AWSElasticBlockStore:nil,HostPath:nil,Glusterfs:nil,NFS:nil,RBD:nil,ISCSI:nil,Cinder:nil,CephFS:nil,FC:nil,Flocker:nil,FlexVolume:nil,AzureFile:nil,VsphereVolume:nil,Quobyte:nil,AzureDisk:nil,PhotonPersistentDisk:nil,PortworxVolume:nil,ScaleIO:nil,Local:nil,StorageOS:nil,CSI:&CSIPersistentVolumeSource{Driver:org.veritas.infoscale,VolumeHandle:clust40559/vrtskubedg-/vol1,ReadOnly:false,FSType:vxfs,VolumeAttributes:map[string]string{},ControllerPublishSecretRef:nil,NodeStageSecretRef:nil,NodePublishSecretRef:nil,ControllerExpandSecretRef:nil,NodeExpandSecretRef:nil,},},AccessModes:[ReadWriteOnce],ClaimRef:&ObjectReference{Kind:PersistentVolumeClaim,Namespace:default,Name:csi-infoscale-pvc-static,UID:-59a5-4f02-83ac-,APIVersion:v1,ResourceVersion:,FieldPath:,},PersistentVolumeReclaimPolicy:Delete,StorageClassName:csi-infoscale-sc-prod,MountOptions:[],VolumeMode:*Filesystem,NodeAffinity:nil,},Status:PersistentVolumeStatus{Phase:Failed,Message:error getting deleter volume plugin for volume "csi-infoscale-pv": no deletable volume plugin matched,Reason:,},}
Thank you I will try to fix it. /remove-triage needs-information /assign
Code change:
} func (s *containerScope) Admit(pod *v1.Pod) lifecycle.PodAdmitResult { <del> // Exception - Policy : none if s.policy.Name() == PolicyNone { return s.admitPolicyNone(pod) } </del> for _, container := range append(pod.Spec.InitContainers, pod.Spec.Containers...) { bestHint, admit := s.calculateAffinity(pod, &container) klog.InfoS("Best TopologyHint", "bestHint", bestHint, "pod", klog.KObj(pod), "containerName", container.Name)

Related discussion:
text: Follow up from: See details there. When is , there is no need to discover the NUMA topology. /sig node /priority backlog /kind feature /help-wanted
/cc
/triage accepted
I would like to take this up also can I get more brief information about it? code review comments from asking that inside the NewManager() implementation, we need to skip NewPolicyOptions and NewNUMAInfo but when the NumaTopology comes kubelet issue also comes. I need some more information to start with this!
/cc
When there is no topology manger policy specified, NewManager should be a no-op. Just skip all checks.
Exactly. Just to add: we may need few iterations over the change to get a clean flow in the call.
/assign
Code change:
Name: "foo"}}, api.Namespace{ ObjectMeta: api.ObjectMeta{ <ins> Name: "foo"}, Status: api.NamespaceStatus{ Phase: api.NamespaceTerminating, }, }, false}, {api.Namespace{ ObjectMeta: api.ObjectMeta{ Name: "foo"}}, api.Namespace{ ObjectMeta: api.ObjectMeta{ </ins> Name: "bar"}, Status: api.NamespaceStatus{ Phase: api.NamespaceTerminating,

Related discussion:
text: There is a funny subtle behvaiour in get namespaces, in which if i call more than one time, it seems that the removal from Terminating state is accelarated. It seems this might indicate a bug, or maybe its not a bug and the eventual semantics of is a little funny sometimes. in any case, here's what we are seeing : 1) when i delete a namespace the first time usning kubectl, we get a state change to . that is correct, of course. 2) but ... if make the call a second time, then the same status line is immediately removed from . Expected behaviour : A second call to "kubectl delete ..." shouldn't have any effect on the timing of removal of a namespace from 'Terminating' state*... So i guess there is some "eventual consistency" subtle bug that is resolved by repeated calls to
suggests this may not be e bug, maybe its just the etcd eventual deleter thingy.
Looks to definitely be a bug. The 2nd delete seems to have interrupted the delete process and artifacts (like events) from the namespace still exist in etcd.
investigating...
oops, see the error, will submit a fix.
I have a fix, but need a test case, will submit PR tomorrow AM
Code change:
Rule:       admissionv1beta1.Rule{APIGroups: []string{""}, APIVersions: []string{"v1"}, Resources: []string{"pods"}}, }}, ObjectSelector:          webhook.objectSelector, <ins> NamespaceSelector:       &metav1.LabelSelector{MatchLabels: nsLabels}, </ins> FailurePolicy:           &fail, ReinvocationPolicy:      webhook.policy, AdmissionReviewVersions: []string{"v1beta1"}, }) } <ins> // Register a marker checking webhook with each set of webhook configurations markerEndpoint := webhookServer.URL + "/marker" webhooks = append(webhooks, admissionv1beta1.MutatingWebhook{ Name: "admission.integration.test.marker", ClientConfig: admissionv1beta1.WebhookClientConfig{ URL:      &markerEndpoint, CABundle: localhostCert, }, Rules: []admissionv1beta1.RuleWithOperations{{ Operations: []admissionv1beta1.OperationType{admissionv1beta1.OperationAll}, Rule:       admissionv1beta1.Rule{APIGroups: []string{""}, APIVersions: []string{"v1"}, Resources: []string{"pods"}}, }}, NamespaceSelector:       &metav1.LabelSelector{MatchLabels: markerNsLabels}, ObjectSelector:          &metav1.LabelSelector{MatchLabels: map[string]string{"marker": "true"}}, AdmissionReviewVersions: []string{"v1beta1"}, }) </ins> cfg, err := client.AdmissionregistrationV1beta1().MutatingWebhookConfigurations().Create(&admissionv1beta1.MutatingWebhookConfiguration{ ObjectMeta: metav1.ObjectMeta{Name: fmt.Sprintf("admission.integration.test-%d", i)},

Related discussion:
text: Which jobs are failing: Which test(s) are failing: Since when has it been failing: Testgrid link: Reason for failure: Anything else we need to know: /sig api-machinery /assign
/cc
want to take a look at the audit event issue first? I can dig in after that if issues remain.
yes, I'm looking at the audit event issue now
actually the failure message suggests that the missing audit log is caused by unexpected admission invocation: I think it's unrelated to auditing Note that there are two tests named "no reinvocation of webhook B when in-tree or prior webhook mutations" :( and from the audit log (), it also suggests that the test is hitting a wrong endpoint
Thanks I'll dig in.
Code change:
Expect(lineCount).To(Equal(expectedLineCount), "Line count of the written file should be %d.", expectedLineCount) <del> // Verify the pod is scheduled in the other zone. By("verifying the pod is scheduled in a different zone.") var otherZone string if cloudZones[0] == podZone { otherZone = cloudZones[1] } else { otherZone = cloudZones[0] </del> <ins> } func addTaint(c clientset.Interface, ns string, nodes []v1.Node, podZone string) (removeTaint func()) { reversePatches := make(map[string][]byte) for _, node := range nodes { oldData, err := json.Marshal(node) Expect(err).NotTo(HaveOccurred()) node.Spec.Taints = append(node.Spec.Taints, v1.Taint{ Key:    taintKeyPrefix + ns, Value:  podZone, Effect: v1.TaintEffectNoSchedule, }) newData, err := json.Marshal(node) Expect(err).NotTo(HaveOccurred()) patchBytes, err := strategicpatch.CreateTwoWayMergePatch(oldData, newData, v1.Node{}) Expect(err).NotTo(HaveOccurred()) reversePatchBytes, err := strategicpatch.CreateTwoWayMergePatch(newData, oldData, v1.Node{}) Expect(err).NotTo(HaveOccurred()) reversePatches[node.Name] = reversePatchBytes _, err = c.CoreV1().Nodes().Patch(node.Name, types.StrategicMergePatchType, patchBytes) Expect(err).ToNot(HaveOccurred()) </ins> } <del> nodeName = pod.Spec.NodeName node, err = c.CoreV1().Nodes().Get(nodeName, metav1.GetOptions{}) Expect(err).ToNot(HaveOccurred()) newPodZone := node.Labels[apis.LabelZoneFailureDomain] Expect(newPodZone).To(Equal(otherZone), "The pod should be scheduled in zone %s after all nodes in zone %s have been deleted", otherZone, podZone) </del> <ins> return func() { for nodeName, reversePatch := range reversePatches { _, err := c.CoreV1().Nodes().Patch(nodeName, types.StrategicMergePatchType, reversePatch) Expect(err).ToNot(HaveOccurred()) } } </ins> } func testRegionalDelayedBinding(c clientset.Interface, ns string) {

Related discussion:
text: <!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: What happened: Testgrid: https://k8s-
/assign
The test's assumption about test parameter is wrong. Previously it assumed the group name is the same across all zones, but they are different in regional clusters. Looking into using node taints as the permanent fix, as suggested by the TODO comment in the test.
Hm I think taint based evictions is still an alpha feature.
OK. I propose the following solution: Get the zone of Schedule the StatefulSet pod to that zone using node affinity; wait for the pod to run Remove the node affinity from StatefulSet Delete
Can it be simplified? Schedule the pod Figure out which zone that pod got scheduled in Delete the instance group of that zone
Unfortunately I don't know of a good way to look up the instance group given the zone, since there could be instance groups from multiple clusters in the same project
How about: Schedule the pod Figure out the zone Taint all the nodes in the zone Kill the pod
Taint with NoSchedule? yeah that sounds good