jiebi/CodeConvo huggingface dataset:
Code change:
Done(types.UID) Update(logger klog.Logger, oldPod, newPod *v1.Pod) Delete(pod *v1.Pod) <del> // TODO(sanposhiho): move all PreEnqueueCheck to Requeue and delete it from this parameter eventually. // Some PreEnqueueCheck include event filtering logic based on some in-tree plugins // and it affect badly to other plugins. // See https://github.com/kubernetes/kubernetes/issues/110175 </del> <ins> // Important Note: preCheck shouldn't include anything that depends on the in-tree plugins' logic. // (e.g., filter Pods based on added/updated Node's capacity, etc.) // We know currently some do, but we'll eventually remove them in favor of the scheduling queue hint. </ins> MoveAllToActiveOrBackoffQueue(logger klog.Logger, event framework.ClusterEvent, oldObj, newObj interface{}, preCheck PreEnqueueCheck) AssignedPodAdded(logger klog.Logger, pod *v1.Pod) AssignedPodUpdated(logger klog.Logger, oldPod, newPod *v1.Pod, event framework.ClusterEvent)

Related discussion:
text: So instead of "pass all unscheduled Pods to movePodsToActiveOrBackoffQueue", I'd like to come up with a solution like: It's pretty rare to move pods depending on Unschedulable Pods' events. In your particular case, maybe a proactive call would behave more efficiently. It doesn't sound correlated, is it? As the check will just pre-verify the basic checks which are orthogonal with PodTopologySpread. In other words, it's impossible that a Pod failed can become schedulable eventually. Updated: The example is a valid point... Can you share some use-cases to move Pods upon node's deletion? In the before, I don't think we have any, until the last release the introduction of feature (as in that case deleting a node would impact the number of domains).
to come up with a solution like: If we do so, also, we need to provide the way to pass some functions to decide which Pods should pass to like for out-of-tree plugins? call would behave more efficiently. Ah~, that's a good idea :) Maybe like this (similar to ) user uses WhenUnsatisfiable: DoNotSchedule. Pod is rejected by Pod Topology Spread because of the skew. e.g. a user uses maxSkew: 1 and TopologyKey is hostname, NodeA/NodeB has 10 Pods each, and NodeC has 8 Pods. deletes NodeC skew calculation result should be changed and the Pod rejected in (2) will be allowed by Pod Topology Spread. But the Pod isn't moved from unschedulableQ without flushing because of this issue. So,,, ignoring some events for performance in scheduler side (without providing a way to change this) could be inconvenient for some plugins. What I thought in the above previous comment () is to define "which kind of resource's events should be ignored" in plugin side. (similar to 's idea? ) We can add pre filter functions in like: Then we can set any pre-filter functions for each plugin. And, we can define some default behavior for PreFilterFn and use them when is empty. So, for example, if most of the plugins are considered to prefer to ignore the event for Node that doesn't match nodeAffinity/nodeSelector, we can define this behavior as the default behavior.
Code change:
Done(types.UID) Update(logger klog.Logger, oldPod, newPod *v1.Pod) Delete(pod *v1.Pod) <del> // TODO(sanposhiho): move all PreEnqueueCheck to Requeue and delete it from this parameter eventually. // Some PreEnqueueCheck include event filtering logic based on some in-tree plugins // and it affect badly to other plugins. // See https://github.com/kubernetes/kubernetes/issues/110175 </del> <ins> // Important Note: preCheck shouldn't include anything that depends on the in-tree plugins' logic. // (e.g., filter Pods based on added/updated Node's capacity, etc.) // We know currently some do, but we'll eventually remove them in favor of the scheduling queue hint. </ins> MoveAllToActiveOrBackoffQueue(logger klog.Logger, event framework.ClusterEvent, oldObj, newObj interface{}, preCheck PreEnqueueCheck) AssignedPodAdded(logger klog.Logger, pod *v1.Pod) AssignedPodUpdated(logger klog.Logger, oldPod, newPod *v1.Pod, event framework.ClusterEvent)

Related discussion:
text: EDIT: It is a bad habit of mine to always write long sentences even though I am not very good at English. Summarizing the above: I think it's not a good idea to create something that can be achieved in in-tree plugins but not in out-of-tree plugins. If we need a solid advantage to extend like this, then it has the advantage of optimizing the queue per plugin like described above.
I didn't jump in the thread since it's actively on the discussion. But, the issue is the similar one. (This time, we can solve that issue by changing scheduling queue logic since It's the k/k feature though.)
I'm not sure I fully understand your intention. The slack thread was discussing the AssignedPod update case, so it fallls into case 1 in , which we already agreed to improve in upstream.
Hi Are you working on this? I think this is quite important and if you're not working on this, I can follow.
Sorry, leave here for a while. I wanted to wait for (and then forgot to come back here. ) That proposal is quite similar to what I want to see in the conclusion of this issue; every plugin can judge if each Pod in unschedQ can be moved to schedQ/backoffQ. Do you think it makes sense to add "By which event the pod will be moved back to schedQ/backoffQ" on 's argument? I know and isn't needed in DefaultEnqueue plugin for API, but that's the last missing puzzle piece : If has , enqueue plugin can judge if the event makes it possible to schedule the Pod. For example, I believe we can move ↓ this logic to Pod Affinity plugin by implementing in Pod Affinity plugin. Pod Affinity plugin will remember Pods rejected by itself in . And in , if the given Pod was the one rejected by Pod Affinity, is AssignedPod/Created|Updated and is Pod affinity with the rejected Pod, then move the Pod back to activeQ/backoffQ. Otherwise reject the Pod moving. We can resolve (3) by implementing as well. (by exactly the way you said.↓) My original concern in this issue is that having logic for specific plugins in the scheduler core may prevent out-of-tree plugins' extendability. The scheduler itself should be pure, and shouldn't do anything special for any plugins implementation ideally.
Code change:
Done(types.UID) Update(logger klog.Logger, oldPod, newPod *v1.Pod) Delete(pod *v1.Pod) <del> // TODO(sanposhiho): move all PreEnqueueCheck to Requeue and delete it from this parameter eventually. // Some PreEnqueueCheck include event filtering logic based on some in-tree plugins // and it affect badly to other plugins. // See https://github.com/kubernetes/kubernetes/issues/110175 </del> <ins> // Important Note: preCheck shouldn't include anything that depends on the in-tree plugins' logic. // (e.g., filter Pods based on added/updated Node's capacity, etc.) // We know currently some do, but we'll eventually remove them in favor of the scheduling queue hint. </ins> MoveAllToActiveOrBackoffQueue(logger klog.Logger, event framework.ClusterEvent, oldObj, newObj interface{}, preCheck PreEnqueueCheck) AssignedPodAdded(logger klog.Logger, pod *v1.Pod) AssignedPodUpdated(logger klog.Logger, oldPod, newPod *v1.Pod, event framework.ClusterEvent)

Related discussion:
text: If we can move all such logic to each plugin, that'd be perfect.
^ is more like a new feature than just a solution to this problem. I created another issue and describe my proposal in detail. Let's have a discussion there.
It can be resolved by moving all preCheck into each plugin's QueueingHint. Let me retitle this issue so that the current status is easy to understand. /retitle scheduler: move all preCheck to QueueingHint
This issue has not been updated in over 1 year, and should be re-triaged. You can: Confirm that this issue is still relevant with (org members only) Close this issue with For more details on the triage process, see /remove-triage accepted
/triage accepted
Code change:
function create-certs { execute-cmd-on-pre-existing-master-with-retries 'sudo cat /etc/kubernetes/admin.conf' > /tmp/kubeconfig <del> # CA_CERT_BASE64, KUBELET_CERT_BASE64 and KUBELET_KEY_BASE64 might be used # in test/kubemark/iks/util.sh # If it becomes clear that the variables are not used anywhere, then we can # remove them: CA_CERT_BASE64=$(grep certificate-authority /tmp/kubeconfig | awk '{print $2}' | head -n 1) KUBELET_CERT_BASE64=$(grep client-certificate-data /tmp/kubeconfig | awk '{print $2}' | head -n 1) KUBELET_KEY_BASE64=$(grep client-key-data /tmp/kubeconfig | awk '{print $2}' | head -n 1) export CA_CERT_BASE64 export KUBELET_CERT_BASE64 export KUBELET_KEY_BASE64 </del> }

Related discussion:
text: While trying to run with a valid ibmcloud account, the following error happened I also tried to run and got the same error. Looks like a compatibility issue. ibmcloud CLI shouldn't return error Run with a valid IBM cloud account. No response I am trying to build kubemark from source <details</details<detailsIBM cloud IKS </details<details</details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig scalability
The Kubernetes project currently lacks enough contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Mark this issue or PR as rotten with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
The Kubernetes project currently lacks enough active contributors to adequately respond to all issues and PRs. This bot triages issues and PRs according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue or PR as fresh with Close this issue or PR with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle rotten [1]:
Code change:
runningPod := podPair.RunningPod apiPod := podPair.APIPod <del> if killing.Has(string(runningPod.ID)) { // The pod is already being killed. break </del> <ins> lock.Lock() exists := killing.Has(string(runningPod.ID)) if !exists { killing.Insert(string(runningPod.ID)) </ins> } <del> killing.Insert(string(runningPod.ID)) go func(apiPod *v1.Pod, runningPod *kubecontainer.Pod, ch chan types.UID) { defer func() { ch <- runningPod.ID }() glog.V(2).Infof("Killing unwanted pod %q", runningPod.Name) err := kl.killPod(apiPod, runningPod, nil, nil) if err != nil { glog.Errorf("Failed killing the pod %q: %v", runningPod.Name, err) } }(apiPod, runningPod, resultCh) </del> <ins> lock.Unlock() </ins> <del> case podID := <-resultCh: killing.Delete(string(podID)) </del> <ins> if !exists { go func(apiPod *v1.Pod, runningPod *kubecontainer.Pod) { glog.V(2).Infof("Killing unwanted pod %q", runningPod.Name) err := kl.killPod(apiPod, runningPod, nil, nil) if err != nil { glog.Errorf("Failed killing the pod %q: %v", runningPod.Name, err) } lock.Lock() killing.Delete(string(runningPod.ID)) lock.Unlock() }(apiPod, runningPod) } </ins> } } }

Related discussion:
text: There is a defer calling in this method: defer close(resultCh) It may crash the goroutine created at line 777, which then will make the whole kubelet crash. A simple fix Another simple fix
cc do you think this make sense?
You could just drop the channel and go with a mutex to update the set.
indeed.
Code change:
defer framework.DeleteRCAndPods(f.ClientSet, ns, "reserve-all-cpu") framework.ExpectNoError(err) <del> By("creating a new instance of DNS and waiting for DNS to be scheduled") label := labels.SelectorFromSet(labels.Set(map[string]string{"k8s-app": "kube-dns"})) </del> <ins> By("creating a new instance of Dashboard and waiting for Dashboard to be scheduled") label := labels.SelectorFromSet(labels.Set(map[string]string{"k8s-app": "kubernetes-dashboard"})) </ins> listOpts := api.ListOptions{LabelSelector: label} rcs, err := f.ClientSet.Core().ReplicationControllers(api.NamespaceSystem).List(listOpts) framework.ExpectNoError(err)

Related discussion:
text: In , for It("should ensure that critical pod is scheduled in case there is no resources available"), kube-dns is used as the critical pod target and being scaled out and in. However, we plan to enable the dns horizontal autoscaling feature in the near future, such as this . If this feature is turned on, the corresponding autoscaler will fight with this rescheduler e2e and maintain the desired number of replicas. Hence this test will be very likely to fail. So probably we should use another critical pod here, or either create a new critical pod that does not exist before in order to protect the ongoing functionalities.
thanks for letting me know. What is the ETA for enabling horizontal autoscaler for DNS?
I believe it would be enabled no later than the 1.5 release.
Sorry to bother, is there an ETA for this? FYI, as merged, we will upgrade the addon-manager to support rolling update for once is ready to go. After that I will start working on enabling horizontal autoscaler for . These will happen before 1.5 code freeze.
Sorry for the delay. fixes this
Thanks for helping!
Code change:
return nil } <ins> // InspectFeatureGates allows setting bools without taking a dep on a global variable func (p *Plugin) InspectFeatureGates(featureGates featuregate.FeatureGate) { p.nonPreemptingPriority = featureGates.Enabled(features.NonPreemptingPriority) p.resourceQuotaFeatureGateEnabled = featureGates.Enabled(features.ResourceQuotaScopeSelectors) } </ins> // SetExternalKubeClientSet implements the WantsInternalKubeClientSet interface. func (p *Plugin) SetExternalKubeClientSet(client kubernetes.Interface) { p.client = client

Related discussion:
text: <!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Following the documentation at the , I tried enabling quota for a namespace other than but I was still not able to create a pod with priorityClass in my namespace with following error: Error from server (Forbidden): error when creating "": pods "memory-demo" is forbidden: pods with system-cluster-critical priorityClass is not permitted in my-namespace namespace What you expected to happen: Expected pod to be created with namespace where I allowed quota. I can see that both ResourceQuota and Priority Plugins are enabled from logs. I0408 17:15:08. ] Loaded 8 mutating admission controller(s) successfully in the following order: NamespaceLifecycle,LimitRanger,ServiceAccount,TaintNodesByCondition,Priority,DefaultTolerationSeconds,DefaultStorageClass,MutatingAdmissionWebhook. I0408 17:15:08. ] Loaded 6 validating admission controller(s) successfully in the following order: LimitRanger,ServiceAccount,Priority,PersistentVolumeClaimResize,ValidatingAdmissionWebhook,ResourceQuota. How to reproduce it (as minimally and precisely as possible): Follow the steps at Anything else we need to know?: I think, the issue is with priorityPlugin where we are rejecting instead of letting resourceQuota plugin to handle the admission when is enabled. Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Others:
/sig scheduling /cc
Same happens to me. Correction, it does not allow to use the default permissions outside of kube-system
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
See also:
/remove-lifecycle stale
/assign Per offline discussion with
Same issue here.
Code change:
return nil } <ins> // InspectFeatureGates allows setting bools without taking a dep on a global variable func (p *Plugin) InspectFeatureGates(featureGates featuregate.FeatureGate) { p.nonPreemptingPriority = featureGates.Enabled(features.NonPreemptingPriority) p.resourceQuotaFeatureGateEnabled = featureGates.Enabled(features.ResourceQuotaScopeSelectors) } </ins> // SetExternalKubeClientSet implements the WantsInternalKubeClientSet interface. func (p *Plugin) SetExternalKubeClientSet(client kubernetes.Interface) { p.client = client

Related discussion:
text: We are using namespaces other than kube-system for system pods as it is getting too crowded, therefore we would like to run cluster-node-critical pods in other namespaces.
Waiting for this to be available myself. No reason my Node Exporter dataset has to run in the namespace instead of my namespace.
Code change:
var wg sync.WaitGroup var once sync.Once <del> // set up stdin stream </del> if e.stdin != nil { <del> headers.Set(api.StreamType, api.StreamTypeStdin) remoteStdin, err := conn.CreateStream(headers) if err != nil { return err } </del> // copy from client's stdin to container's stdin go func() { // if e.stdin is noninteractive, e.g. `echo abc | kubectl exec -i <pod> -- cat`, make sure

Related discussion:
text: We create the stdin stream and immediately start copying local stdin to the container. We do the same for stdout and stderr (create stream, start copying immediately). In the kubelet, we wait to receive all the streams before we start the exec and copy between the streams and the container. The spdy connection frame handling code has 5 frame workers, each with a queue of up to 50 frames. In the event that we try to push more than 50 frames to a single queue before all streams are ready, the 51st attempt to push a frame blocks until there is space available in the queue. But there won't be any space until the server starts reading from the streams, which won't happen until all the streams have been received, so we're stuck at this point. The fix is to create all the streams first, and only start copying after all the streams have been created and the server is ready to copy between the streams and the container. Originally reported here:
FYI
This seems pretty high priority.
It's an easy fix. has already tested the patch for OpenShift and it works. Are we willing to pull it in before is resolved, as that will take longer to put in place?
Here's the patch, for reference:
5 * 50 = 250 frames before you get stuck? Or did I misunderstand?
Oh, I guess all the frames go to the same worker for some reason?
anyway, I'm fine with taking this and cherry picking it. Order doesn't matter; someone has to rebase either way, right?
frames are assigned to each queue based on stream id, so if you sent 50 frames to the stdin stream, that queue fills up and the loop that processes incoming frames is blocked until something starts reading from the stream. Working on a PR now.
Gotcha, thanks. , Andy Goldstein wrote:
an update: I spent some time trying to create an e2e that would fail more often than not without this fix, but I haven't had any luck so far (I have it stream a copy of the kubectl binary from the local system to the pod). The issue that originally uncovered the problem was from running exec on a Windows system, where it reproduces consistently, but I haven't seen it fail much on Linux. I will keep trying...
Code change:
})) } <ins> // AddEphemeralContainerSync adds an EphemeralContainer to a pod and waits for it to be running. func (c *PodClient) AddEphemeralContainerSync(pod *v1.Pod, ec *v1.EphemeralContainer, timeout time.Duration) { namespace := c.f.Namespace.Name podJS, err := json.Marshal(pod) ExpectNoError(err, "error creating JSON for pod %q", format.Pod(pod)) ecPod := pod.DeepCopy() ecPod.Spec.EphemeralContainers = append(ecPod.Spec.EphemeralContainers, *ec) ecJS, err := json.Marshal(ecPod) ExpectNoError(err, "error creating JSON for pod with ephemeral container %q", format.Pod(pod)) patch, err := strategicpatch.CreateTwoWayMergePatch(podJS, ecJS, pod) ExpectNoError(err, "error creating patch to add ephemeral container %q", format.Pod(pod)) _, err = c.Patch(context.TODO(), pod.Name, types.StrategicMergePatchType, patch, metav1.PatchOptions{}, "ephemeralcontainers") ExpectNoError(err, "Failed to patch ephemeral containers in pod %q", format.Pod(pod)) ExpectNoError(e2epod.WaitForContainerRunning(c.f.ClientSet, namespace, pod.Name, ec.Name, timeout)) } </ins> // DeleteSync deletes the pod and wait for the pod to disappear for `timeout`. If the pod doesn't // disappear before the timeout, it will fail the test. func (c *PodClient) DeleteSync(name string, options metav1.DeleteOptions, timeout time.Duration) {

Related discussion:
text: Reopen the issue with . Mark the issue as fresh with . Send feedback to sig-contributor-experience at . /close
Closing this issue. <detailsIn response to : Rotten issues close after 30d of inactivity. Reopen the issue with . Mark the issue as fresh with . /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/reopen /remove-lifecycle rotten
Reopened this issue. <detailsIn response to : /reopen /remove-lifecycle rotten Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Code change:
ginkgo.Context("when querying /metrics", func() { var oldCfg *kubeletconfig.KubeletConfiguration var testPod *v1.Pod <del> var cpusNumPerNUMA, numaNodes int </del> <ins> var cpusNumPerNUMA, coresNumPerNUMA, numaNodes, threadsPerCore int </ins> ginkgo.BeforeEach(func(ctx context.Context) { var err error

Related discussion:
text: /kind failing-test <!-- If this is a flake, please add: /kind flake --/sig node started to fail since 09-13.
/cc
/assign
/cc
/triage accepted /priority important-soon it's indeed failing too often
/reopen The test is still failing: Now timing out at 120s i.e. 2 minutes as we increased the default timeout . We need to understand the root cause and get to the bottom of this.
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Snippet from the build log () corresponding from this failure We see from the above logs that: node has two numa nodes, 8 cpus per NUMA so the root cause of the failure is not because the tests are running on a node that doesn't have multi-numa support. is started and the health check fails a couple of times but eventually succeeds. pod is created with 9 cpus (cpusperNUMA +1 i.e. 8+1) which is expected to fail with Topology Affinity Error as we cannot allocate resources for such a pod from a single NUMAnode. We fail to get matching metrics. It is not clear if the check fails because we don't get kubelet metrics at all or the expected metrics but based on point 6 below it seems like it is because of latter. We can add additional logs for more clarity here. each is executed and Kubelet is restarted and health check is done. are collected from the namespace corresponding to the test pod which shows that the pod started even though it was expected to fail with Topology Affinity Error. We need to understand why the pod got successfully created even though it requested CPUs that couldn't be allocated from a single NUMAnode.