jiebi/CodeConvo huggingface dataset:
Code change:
want: []bool{true, false, false, true, false, true, false, true, false}, }, { <ins> name: "no filtering when QHint is enabled", nodeFn: func() *v1.Node { return st.MakeNode().Name("fake-node").Label("hostname", "fake-node").Capacity(cpu8).Obj() }, existingPods: []*v1.Pod{ st.MakePod().Name("p").HostPort(80).Obj(), }, pods: []*v1.Pod{ st.MakePod().Name("p1").Req(cpu4).Obj(), st.MakePod().Name("p2").Req(cpu16).Obj(), st.MakePod().Name("p3").Req(cpu4).Req(cpu8).Obj(), st.MakePod().Name("p4").NodeAffinityIn("hostname", []string{"fake-node"}).Obj(), st.MakePod().Name("p5").NodeAffinityNotIn("hostname", []string{"fake-node"}).Obj(), st.MakePod().Name("p6").Obj(), st.MakePod().Name("p7").Node("invalid-node").Obj(), st.MakePod().Name("p8").HostPort(8080).Obj(), st.MakePod().Name("p9").HostPort(80).Obj(), }, qHintEnabled: true, want:         []bool{true, true, true, true, true, true, true, true, true}, }, { </ins> name: "tainted node, pods with a single constraint", nodeFn: func() *v1.Node { node := st.MakeNode().Name("fake-node").Obj()

Related discussion:
text: If we can move all such logic to each plugin, that'd be perfect.
^ is more like a new feature than just a solution to this problem. I created another issue and describe my proposal in detail. Let's have a discussion there.
It can be resolved by moving all preCheck into each plugin's QueueingHint. Let me retitle this issue so that the current status is easy to understand. /retitle scheduler: move all preCheck to QueueingHint
This issue has not been updated in over 1 year, and should be re-triaged. You can: Confirm that this issue is still relevant with (org members only) Close this issue with For more details on the triage process, see /remove-triage accepted
/triage accepted
Code change:
return &loadbalancerList[0], nil } <del> func waitLoadbalancerActiveProvisioningStatus(client *gophercloud.ServiceClient, loadbalancerID string) error { </del> <ins> func waitLoadbalancerActiveProvisioningStatus(client *gophercloud.ServiceClient, loadbalancerID string) (string, error) { </ins> start := time.Now().Second() for { loadbalancer, err := loadbalancers.Get(client, loadbalancerID).Extract() if err != nil { <del> return err </del> <ins> return "", err </ins> } if loadbalancer.ProvisioningStatus == "ACTIVE" { <del> return nil </del> <ins> return "ACTIVE", nil } else if loadbalancer.ProvisioningStatus == "ERROR" { return "ERROR", fmt.Errorf("Loadbalancer has gone into ERROR state") </ins> } time.Sleep(1 * time.Second) if time.Now().Second()-start >= loadbalancerActiveTimeoutSeconds { <del> return fmt.Errorf("Loadbalancer failed to go into ACTIVE provisioning status within alloted time") </del> <ins> return loadbalancer.ProvisioningStatus, fmt.Errorf("Loadbalancer failed to go into ACTIVE provisioning status within alloted time") </ins> } } }

Related discussion:
text: LBaaS implementation (v1 or v2) will not support services that require to load balancer more than one port.
Fixed for LBaaS v2 by I suspect LBaaS v1 can't support multiple ports (not sure if reusing the VIP across multiple loadbalancers is a universally supported thing). I also believe LBaaS v1 is also mostly gone in favour of v2 in deployments so trying to address this in v1 is extremely low priority unless someone steps forward to ask for it. So: I think we can close this bug now, but I lack the powers to do that directly.
thank you for the update.
Code change:
if err != nil { return nil, fmt.Errorf("failed to get container info from cadvisor: %v", err) } <del> // removeTerminatedContainerInfo will also remove pod level cgroups, so save the infos into allInfos first allInfos := infos infos = removeTerminatedContainerInfo(infos) </del> <ins> filteredInfos, allInfos := filterTerminatedContainerInfoAndAssembleByPodCgroupKey(infos) </ins> // Map each container to a pod and update the PodStats with container data. podToStats := map[statsapi.PodReference]*statsapi.PodStats{} <del> for key, cinfo := range infos { </del> <ins> for key, cinfo := range filteredInfos { </ins> // On systemd using devicemapper each mount into the container has an // associated cgroup. We ignore them to ensure we do not get duplicate // entries in our summary. For details on .mount units:

Related discussion:
text: <!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --For single node with hundreds and thousands of pods and containers, the cpu usage of kubelet becomes high, one bottleneck is the cadvisor stats provider as below: the method iterate all containerInfos for every podStats cause high cpu the method should not iterate allcontainerInfos for every podStats, use map instead Kubernetes version (use ): v1.19.4 Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig node
/triage accepted
/assign
I’ve done an analysis on the code and this is what I have found, I believe we have two issues to address. There are multiple range loops being performed over v2.ContainerInfo / PodStat maps within the ListPodStatus method which can be contributing to the high cpu utilisation. The allInfos argument which is supplied to the getCadvisorContainerInfo function is a map which uses a cgroup path as the key, in order to find matching container info to podUID, processing of the keys in the map needs to be performed until a match is found. Examples of keys are as follows: /kubepods/burstable/podf9e3a173-1470-451a-9ca2-/d8426b82582fbcfae16085bc794c49769d3a9648bd5463976aca4071f9342754 /docker/2dfa17fc81e781212bcc26e189ebb724721fe097394b4de3cbeb7e069bf23290 My recommendation would be to apply the following updates: Refactor the getCadvisorContainerInfo function to allow for a filter and key type parameter to be supplied by the caller, this enables reducing the number of range loops being performed over the container info map by removing elements that are not required and the ability to set the key being used in the returned container infos map to enable fast lookup of container info.
Code change:
if err != nil { return nil, fmt.Errorf("failed to get container info from cadvisor: %v", err) } <del> // removeTerminatedContainerInfo will also remove pod level cgroups, so save the infos into allInfos first allInfos := infos infos = removeTerminatedContainerInfo(infos) </del> <ins> filteredInfos, allInfos := filterTerminatedContainerInfoAndAssembleByPodCgroupKey(infos) </ins> // Map each container to a pod and update the PodStats with container data. podToStats := map[statsapi.PodReference]*statsapi.PodStats{} <del> for key, cinfo := range infos { </del> <ins> for key, cinfo := range filteredInfos { </ins> // On systemd using devicemapper each mount into the container has an // associated cgroup. We ignore them to ensure we do not get duplicate // entries in our summary. For details on .mount units:

Related discussion:
text: My initial thoughts are to use an enumeration which has two values KEYCGROUPPATH to provide backwards compatibility and KEYPODUID to enable fast lookups via podUID Refactor ListPodStats method to be more efficient in constructing the PodStats slice and taking into account the fast lookup via podUID I think this is the best approach instead of pushing the data structure implementation for the container info upstream to the cadviser clients, this will at least initially provide a better optimisation scoped within the kubelet/stats package and reduce other possible impacts to changing the cadvisor clients Please advise.
Just realized this looks like a duplicate issue and looking as fixed by PR Both this issue and were raised by Zac
/unassign
/close
You can't close an active issue/PR unless you authored it or you are a collaborator. <detailsIn response to : /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Didn't think I could close this, leaving this with
this will be automatically closed once the PR merges
Code change:
if tc.expectCreate { expectedCreates = 1 } <ins> if tc.ranPreviously && !tc.stillActive { completionTime := tc.jobCreationTime.Add(10 * time.Second) if cjCopy.Status.LastSuccessfulTime == nil || !cjCopy.Status.LastSuccessfulTime.Time.Equal(completionTime) { t.Errorf("cj.status.lastSuccessfulTime: %s expected, got %#v", completionTime, cj.Status.LastSuccessfulTime) } } </ins> if len(jc.Jobs) != expectedCreates { t.Errorf("%s: expected %d job started, actually %v", name, expectedCreates, len(jc.Jobs)) }

Related discussion:
text: <!-- Please only use this template for submitting enhancement requests --What would you like to be : Cronjob should include the last time when a job was successfully completed. Why is this needed: Currently, a Cronjob's only includes the last time when a job was scheduled. This makes monitoring cronjobs much more complicated, because instead of simply comparing to the current timestamp, the monitoring system now has to hunt for the last job related to the cronjob and persist the state somewhere.
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
I'd be interesting in picking this up if the apps team thinks this is a good ticket - it seems like it's fairly beginner friendly.
Hi how is this going on? I could take it over if you don't have the time for this issue. :)
Yes please do! I meant to update that I actually wasn't able to pick this up, but it slipped my mind
/assign
Hi do you have any thoughts on this feature?
/unassign
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Any updates?
Reiterating the mentions to trigger a notification: <detailsIn response to : Any updates? Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Trying to understand the usage of lastSuccessfulTime. In a normally working case, lastSuccessfulTime lastScheduledTime =last job completed lastSuccessfulTime < lastScheduledTime =last job completed, but a newer one is running Is that the expectation ?
We wanted to use it in monitoring (e.g. fire an alert if is more than X ago). could also mean that the job has been failing for some time, so I don't know what would you get out of that. I guess you could also use as a proxy for "job has failed at least once"?
Code change:
if tc.expectCreate { expectedCreates = 1 } <ins> if tc.ranPreviously && !tc.stillActive { completionTime := tc.jobCreationTime.Add(10 * time.Second) if cjCopy.Status.LastSuccessfulTime == nil || !cjCopy.Status.LastSuccessfulTime.Time.Equal(completionTime) { t.Errorf("cj.status.lastSuccessfulTime: %s expected, got %#v", completionTime, cj.Status.LastSuccessfulTime) } } </ins> if len(jc.Jobs) != expectedCreates { t.Errorf("%s: expected %d job started, actually %v", name, expectedCreates, len(jc.Jobs)) }

Related discussion:
text: Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten
that's exactly what is being asked here. /close
Closing this issue. <detailsIn response to : /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
"successfully scheduled" != "successfully completed", and most of the users care more about the latter.
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign /triage accepted
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-contributor-experience at .
Code change:
if tc.expectCreate { expectedCreates = 1 } <ins> if tc.ranPreviously && !tc.stillActive { completionTime := tc.jobCreationTime.Add(10 * time.Second) if cjCopy.Status.LastSuccessfulTime == nil || !cjCopy.Status.LastSuccessfulTime.Time.Equal(completionTime) { t.Errorf("cj.status.lastSuccessfulTime: %s expected, got %#v", completionTime, cj.Status.LastSuccessfulTime) } } </ins> if len(jc.Jobs) != expectedCreates { t.Errorf("%s: expected %d job started, actually %v", name, expectedCreates, len(jc.Jobs)) }

Related discussion:
text: /lifecycle rotten
This is described in and being actively implemented in /close
Closing this issue. <detailsIn response to : This is described in and being actively implemented in /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
It doesn't look like it was actually implemented, though. Searching for only shows API changes, there don't seem to be any changes to the CronJob controller logic. Or am I missing something?
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign /remove-lifecycle rotten
Code change:
return &framework.QueuedPodInfo{ PodInfo:                 podInfo, Timestamp:               now, <del> InitialAttemptTimestamp: now, </del> <ins> InitialAttemptTimestamp: nil, </ins> UnschedulablePlugins:    sets.New(plugins...), } }

Related discussion:
text: is recording the time that a Pod is gated. We use the timestamp when the scheduler inserts the pod into the queue: The period of time when a Pod fails PreEnque (like being gated) shouldn't be accounted in the podschedulingdurationseconds. Create a Pod with scheduling gates. Wait some time before removing the gate. Observe the metric No response <details</details<detailsAny </details<details</details<details</details<details</details<details</details
/sig scheduling
/help /triage accepted
This request has been marked as needing help from a contributor. Please ensure that the issue body includes answers to the following questions: Why are we solving this issue? To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points? Does this issue have zero to low barrier of entry? How can the assignee reach out to you for help? For more details on the requirements of such an issue, please see and ensure that they are met. If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /help /triage accepted Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
would you want to take a stab on this issue?
Yup, I can give it a shot. /assign helayoty
, as we discussed, we can address this issue with different solutions: the pod timestamp to the current time , so the gated time is substituted. timestamp . Use a pointer to the InitialAttemptTimestamp and interpret as the newest timestamp so the pod will go to the back of the queue. I'll go ahead and implement solution no. 3, wdyt?
sgtm. Any concerns ?
I think it should be option 1 + 3, and pInfo.Timestamp should be pInfo.InitialAttemptTimestamp, right? as records the time based on :
I think we should revisit this issue, in terms of its assumption: So is it reasonable to always exclude time spent in PreEnqueue from ? Note that PreEnqueue != SchedulingGates, is one of the PreEnqueue's usecases.
Code change:
return &framework.QueuedPodInfo{ PodInfo:                 podInfo, Timestamp:               now, <del> InitialAttemptTimestamp: now, </del> <ins> InitialAttemptTimestamp: nil, </ins> UnschedulablePlugins:    sets.New(plugins...), } }

Related discussion:
text: In other words, some user of PreEnqueue may want to include the time spent in PreEnqueue.
is a signal that cluster providers can use to build SLOs. The time spent before passing PreEnqueue is not under control of kube-scheduler, so considering it would not allow to build an SLO. Are you aware of other users of PreEnqueue?
True, but comparing to state that metric X can or cannot be used to build SLO (which is hard IMO), I'm in favor of giving suggestion on SLI/SLO's semantic definition, for example: the official SLI definition of has a footnote clearly explaining its definition: So if a pod is gated, it's not "immediately schedulable", and hence doesn't fall into the definition of podstartup_latency.
A viable solution might be providing an extra label like to explicitly exclude the time spent on preenqueue.
That is true, but there is no way today to exclude those pods from the metric, or separate the time. Maybe we need an extra metric, but I think the new metric could be the one that includes the time guarded in PreEnqueue. It should probably be part of the KEP. The old metric is already stable, so we can't add labels. And it might already be in use for SLOs.
cc
I think we should be working on that. And treat it as part of GA requirement for PodSchedulingReadiness.
An edge(maybe not edge) case is a pod's scheduling attempts is 0 but the is a long time, so prefer not to account for the waiting time when gated. For users not using PreEnqueue, they will not be impacted, but for users using PreEnqueue, it's a break change for it increases the duration time. And we can add a new metric like to measure how long a pod stays in a scheduler.
I thought of "attempts", it's just I'm not sure it's a good idea to rely on "attempts" to define a SLI/SLO, b/c "attempts" is obtained from the in-memory object and in a HA config env, it's only updated by the primary scheduler instance. In other words, the value is inaccurate when the lease owner switches.
attempt=0 is good enough for most scenarios
do we have a conclusion?
Code change:
return &framework.QueuedPodInfo{ PodInfo:                 podInfo, Timestamp:               now, <del> InitialAttemptTimestamp: now, </del> <ins> InitialAttemptTimestamp: nil, </ins> UnschedulablePlugins:    sets.New(plugins...), } }

Related discussion:
text: Should we move on with the solution or remove this issue and the linked PR from the current release?
I would like to move forward with this proposal. ?
To move it forward, I think we should add new metric(s) to explicitly tell it's including or excluding the time spent on PreEnqueue. Given is in stable level: We will need a new metric like "podschedulingexcludingpreenqueuedurationseconds".
28 Bug Triage Shadow here. Just checking-in, if this is still on track for the 1.28 release?
We don't have a conclusion yet, but as a bug, it should qualify for changes after the freezes.
Code change:
} } <ins> func updateDS(t *testing.T, dsClient appstyped.DaemonSetInterface, dsName string, updateFunc func(*apps.DaemonSet)) *apps.DaemonSet { var ds *apps.DaemonSet if err := retry.RetryOnConflict(retry.DefaultBackoff, func() error { newDS, err := dsClient.Get(dsName, metav1.GetOptions{}) if err != nil { return err } updateFunc(newDS) ds, err = dsClient.Update(newDS) return err }); err != nil { t.Fatalf("Failed to update DaemonSet: %v", err) } return ds } </ins> func forEachFeatureGate(t *testing.T, tf func(t *testing.T)) { for _, fg := range featureGates() { func() {

Related discussion:
text: https://k8s- /sig apps /priority critical-urgent /assign PR that the test: /cc
GitHub didn't allow me to assign the following users: mortent. Note that only and repo collaborators can be assigned. For more information please see <detailsIn response to : https://k8s- /priority critical-urgent /assign /cc Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/kind flake
I'm looking into this.
The test is still flaky, but with different symptom: An example flake: https://k8s- It's likely because the collision count was updated in a previous sync loop, but the in the cache store hasn't been updated and we used a stale collision count while calculating the new hash: /assign