jiebi/CodeConvo huggingface dataset:
Code change:
} t.Run(fmt.Sprintf("feature enabled=%v, old pod %v, new pod %v", enabled, oldPodInfo.description, newPodInfo.description), func(t *testing.T) { <del> defer featuregatetesting.SetFeatureGateDuringTest(t, utilfeature.DefaultFeatureGate, features.VolumeSubpathEnvExpansion, enabled)() </del> var oldPodSpec *api.PodSpec if oldPod != nil {

Related discussion:
text: /kind cleanup /sig storage /sig node Description: Feature gate VolumeSubpathEnvExpansion was GA in 1.17 and is marked for removal in 1.19 Related Issues: GA Release:
/assign
/cc
Code change:
defer featuregatetesting.SetFeatureGateDuringTest(b, utilfeature.DefaultFeatureGate, feature, flag)() } dataItems.DataItems = append(dataItems.DataItems, runWorkload(b, tc, w)...) <ins> // Reset metrics to prevent metrics generated in current workload gets // carried over to the next workload. legacyregistry.Reset() </ins> }) } })

Related discussion:
text: In the new scheduler performance testing framework, (bool) is provided to benchmark performance of pods created in an "operator": (L11 below) However, the initialized Pods created in L5~L7 seem to be also benchmarked, although it doesn't specify . (please see the reproduce steps below). I think it's b/c once scheduler starts, the associated Prometheus artifacts are also up, and it's a single in-memory datastore. So no matter we start to collect the metric in which step, the prometheus data is already up-to-date. In the above workload config, the final data have benchmarked 1500 pods instead of 1000 pods. I guess it won't work either if running in the pod initialization step or not. Unless we manage a breakpoint at the end of each step, and calculate delta histogram for the step that specifies . Pods created in step without are not benchmarked. Apply the following diff: And then run: You will get an output sth. like: /sig scheduling
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Could you please help confirm if it's an issue?
cc
Unfortunately, does not apply for metrics gathered from . As you mentioned. The default gatherer collects all metrics produced by the scheduler. It's the scheduler itself who knows pod specs and would have to decide which pods will get ignored when updating metrics. The question is if computing metrics for a subset of pods processed by the scheduler is a valid use case.
This is probably what we want. Is something like that possible ?
I think being able to absolutely ignore certain pods for metrics is an overkill.
The use case is to ignore the pods created in steps that don't specify , so as to collect the Pod metrics create at particular step(s) (). Otherwise, the semantics of option is quite misleading. Nope, that's not the intention. We don't need to target certain pods, like using label selector or whatever. Just as I mentioned, measure pods created at particular steps.
Code change:
defer featuregatetesting.SetFeatureGateDuringTest(b, utilfeature.DefaultFeatureGate, feature, flag)() } dataItems.DataItems = append(dataItems.DataItems, runWorkload(b, tc, w)...) <ins> // Reset metrics to prevent metrics generated in current workload gets // carried over to the next workload. legacyregistry.Reset() </ins> }) } })

Related discussion:
text: I would do something like the following: if a step has collectMetrics:false, we wait for them to finish scheduling, and then we take a snapshot of the histogram. Then for the next step that has collectMetrics:true, we take the diff with the latest snapshot to report the results. This might not be perfect, as we don't control if the metrics have finished syncing when we take the snapshot. But I think that's fine. Would doing this be possible? Now, if we have steps that run in parallel, one with collectMetrics:false and another with collectMetrics:true, then this doesn't work. Note that I don't know the details of how the gatherer works.
That adheres to what I have in my mind - we can only manage this on client side. I think it's fine. As we are already fetching the "snapshot" when pods get scheduled when equals true. True. An alternative is to design the case to be like: the step with to employ Pods that are all unschedulable, and then specify a custom metric "schedulere2eschedulingdurationseconds[result="unschedulable"]" to target a particular histogram instead of all histograms (default option) inside a histogramVec. Note this can only be possible when and some subsequent changes get in place.
We might just call Histogram.Clear() method which will clear what has been collected so far at the end of phase: Assuming individual phases have no pods in common and each phase can clear metrics without affecting other phases. Though, as said, it will work only for serial case.
WDYT
I'm afraid doesn't work. That was my initial thought as well, but no luck. I guess it's b/c, theoretically, we cannot reset a metric on the server side - Promoetheus' model doesn't support this. But let me know if I miss anything.
The performance tests assume all the metrics all gathered and accessed in the same binary. As long as that's the case, works.
Code change:
defer featuregatetesting.SetFeatureGateDuringTest(b, utilfeature.DefaultFeatureGate, feature, flag)() } dataItems.DataItems = append(dataItems.DataItems, runWorkload(b, tc, w)...) <ins> // Reset metrics to prevent metrics generated in current workload gets // carried over to the next workload. legacyregistry.Reset() </ins> }) } })

Related discussion:
text: We already call in the code:
That was my understanding as well, but no luck :( Here is a diff to call in case when "concreteOp.CollectMetrics" is false: <details<summaryDiff:</summary<p</p</detailsAnd below is the output - it's still 1500 (500 init pods plus 1000 measured pods) instead of 1000 being benchmarked: <details<summaryOutput:</summary<p</p</details
Also, if you run the tests and together, the result proves doesn't work as expected:
But I do find a way to fix this: .
Code change:
"k8s.io/apimachinery/pkg/util/wait" utilfeature "k8s.io/apiserver/pkg/util/feature" "k8s.io/client-go/dynamic" <ins> "k8s.io/client-go/informers" </ins> clientset "k8s.io/client-go/kubernetes" <ins> restclient "k8s.io/client-go/rest" </ins> featuregatetesting "k8s.io/component-base/featuregate/testing" apiservertesting "k8s.io/kubernetes/cmd/kube-apiserver/app/testing" podutil "k8s.io/kubernetes/pkg/api/v1/pod" <ins> "k8s.io/kubernetes/pkg/controller/statefulset" </ins> "k8s.io/kubernetes/pkg/features" "k8s.io/kubernetes/test/integration/framework" )

Related discussion:
text: I create a ResourceQuota with two pods limit in my namespace. Then, I create a statefuleSet with 3 replicas. As expected, two pods are created successfully, and the third pod failed to create. But, the ReadyReplicas is not 2 ,but 1. The ReadyReplicas is correct 1Create a ResourceQuota with two pods limit in a new namespace. 2Create a statefulSet with 3 replicas 3Check if the ReadyReplicas of statefulSet matchs the number of stateful running pods. 1According to the logic of the statefulSet controller, new pod created in current sync is not be immediately counted into the ReadyReplicas. 2Failure to create pod will cause to return error, then skip updating status. <details</details<details</details<detailskind v1.23.3 </details<details</details<details</details<details</details
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig apps
/assign
Code change:
type NodeClient interface { // NodePrepareResources prepares several ResourceClaims // for use on the node. If an error is returned, the <del> // response is ignored. Failures for individidual claims </del> <ins> // response is ignored. Failures for individual claims </ins> // can be reported inside NodePrepareResourcesResponse. NodePrepareResources(ctx context.Context, in *NodePrepareResourcesRequest, opts ...grpc.CallOption) (*NodePrepareResourcesResponse, error) // NodeUnprepareResources is the opposite of NodePrepareResources.

Related discussion:
text: When ResourceClaimParameters or ResourceClassParameters fail validation in resource driver during PodSchedulingContext sync loop, the error is visible only in the events of the PodSchedulingContext. The error should be visible on the object that fails validation as well, ResourceClaim or ResourceClass respectively. Creating ResourceClaim with unsupported parameters' values or unsupported No response <details</details<details</details<details</details<details</details<details</details<details</details
/assign
/sig node
/triage accepted
KEP:
Code change:
priorityPairs = append(priorityPairs, priorityPair{name: priorityName, value: priorityVal}) _, err := cs.SchedulingV1().PriorityClasses().Create(context.TODO(), &schedulingv1.PriorityClass{ObjectMeta: metav1.ObjectMeta{Name: priorityName}, Value: priorityVal}, metav1.CreateOptions{}) if err != nil { <del> framework.Logf("Failed to create priority '%v/%v': %v", priorityName, priorityVal, err) framework.Logf("Reason: %v. Msg: %v", apierrors.ReasonForError(err), err) </del> <ins> framework.Logf("Failed to create priority '%v/%v'. Reason: %v. Msg: %v", priorityName, priorityVal, apierrors.ReasonForError(err), err) </ins> } framework.ExpectEqual(err == nil || apierrors.IsAlreadyExists(err), true) }

Related discussion:
text: The following PriorityClass endpoints was Promoted it 1.14 without Conformance tests, and remain untested: replace-, read-, patch-, list-, delete-collection- SchedulingV1PriorityClass According to this APIsnoop query, there are still some remaining PriorityClass endpoints which are untested. with this query you can filter untested endpoints by their category and eligiblity for conformance. e.g below shows a query to find all conformance eligible untested,stable,core endpoints ! createSchedulingV1PriorityClass deleteSchedulingV1PriorityClass Promoted to Conformance 1.19 with /sig testing /sig architecture /area conformance
Is the ultimiate goal to use conformance tests to explicitly issue all CRUD requests against PriorityClass? And, is equivalent to "update PriorityClass"?
all APIs must have Conformance tests unless they are not eligible for Conformance due to the There is other APIGroups that have both Replace and Update as endpoints. That is the prerogative of the SIG to decide.
can you please point us to similar conformance tests used for core objects such as pods and nodes?
This is a life cycle test for Create, List, Update, Patch, DeleteCollection More can be found in the test/e2e folder. Also look for examples on the Conformance at previously Promoted Conformance test Reach out if we can help with more information on our approach to Conformance testing.
I remembered about this PR. Last week a few lines of code was to cover the last of the "Lease" endpoint group for conformance. The one file contain the Conformance tests for the complete life cycle of Lease endpoints.
/assign I will work on it.
/reopen Will monitor in test grid to move to Conformance after 2 weeks soak.
Reopened this issue. <detailsIn response to : /reopen Will monitor in test grid to move to Conformance after 2 weeks soak. </details
/triage accepted
Code change:
}) }) <del> // TODO: Get rid of [DisabledForLargeClusters] tag when issue #56138 is fixed. var _ = SIGDescribe("ESIPP [Slow] [DisabledForLargeClusters]", func() { </del> <ins> var _ = SIGDescribe("ESIPP [Slow]", func() { </ins> f := framework.NewDefaultFramework("esipp") var loadBalancerCreateTimeout time.Duration

Related discussion:
text: Is this a BUG REPORT or FEATURE REQUEST?: /kind bug This is a followup from filed separately now that we're not tracking this as part of 1.9 release. The tests have been disabled with [DisabledForLargeClusters] tag for now, and this fact will be mentioned in the scalability validation report for v1.9. This issue tracks fixing the test so the tag can be removed and the test ran again in the large cluster. /assign /kind e2e-test-failure
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Is this e2e test still broken?
Sorry for the confusion here. Currently this e2e test is disabled on extra large cluster due to some scalability issues with the load balancer related components. At the moment the issues haven't been completely resolved yet.
ok, adding priority so that I know it's not a 1.13 blocker. /priority important-longterm
cc - this is why the relevant tests for are not part of the e2e suite.
This issue is still valid --- some action items on top of my head: Figure out a plan to make service controller properly handle operations in parallel. Check on the status for cloud providers (especially GCP, which the test ran against) on handling large number of nodes as LB backends. /remove-triage unresolved
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or .
Code change:
}) }) <del> // TODO: Get rid of [DisabledForLargeClusters] tag when issue #56138 is fixed. var _ = SIGDescribe("ESIPP [Slow] [DisabledForLargeClusters]", func() { </del> <ins> var _ = SIGDescribe("ESIPP [Slow]", func() { </ins> f := framework.NewDefaultFramework("esipp") var loadBalancerCreateTimeout time.Duration

Related discussion:
text: /lifecycle stale
Stale issues rot after 30d of inactivity. Mark the issue as fresh with . Rotten issues close after an additional 30d of inactivity. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle rotten
/remove-lifecycle rotten /lifecycle frozen
/reopen Reopening as is being reverted in
Reopened this issue. <detailsIn response to : /reopen Reopening as is being reverted in Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
these tests are flaking in xref
cc since these were enabled by
Yeah - though these failures doesn't seem to be scale related, they seem to be test related to me. This basically means nodes without public ips: It's pretty much the same thing as (just for a different test). should be addressing these kinds of errors. [Not a release blocker for sure]
BTW - has been opened directly for this one.
... so since is closed, Can we close this and re-enable for large clusters ? (i.e. remove the Tag in that disables this test) ? I'm not up to speed on the history but 2 related notes I came across digging into related test today... 's a couple of TODOs in the code referencing this specific issue, but your above direct issue is now closed, so we can close this as well right?... And if so, we can remove the TODO on the test... -enabling the test ... not sure wether the timing is good right now (given that is open right now , which sparked my stumbling upon this issue :))
We tried to enable all of them and that caused us a bunch of issues and those had to be reverted: A couple tests were enabled later in: and those seem fine. The remaining (mostly session-affinity ones) had some issue, which so far we didn't have time to debug and I haven't heard a good explanation why those were failing. It's possible that they were improved with , but I doubt those would really cause the failures we've seen in the past. So until this is debugged (or at least understood) I'm reluctant to enabling them.
Code change:
proxy_cpu=${KUBEMARK_HOLLOW_PROXY_MILLICPU:-$proxy_cpu} proxy_mem_per_node=${KUBEMARK_HOLLOW_PROXY_MEM_PER_NODE_KB:-50} proxy_mem=$((100 * 1024 + proxy_mem_per_node*NUM_NODES)) <ins> hollow_kubelet_params=$(eval "for param in ${HOLLOW_KUBELET_TEST_ARGS:-}; do echo -n "$param",; done") hollow_kubelet_params=${hollow_kubelet_params%?} hollow_proxy_params=$(eval "for param in ${HOLLOW_PROXY_TEST_ARGS:-}; do echo -n "$param",; done") hollow_proxy_params=${hollow_proxy_params%?} </ins> sed -i'' -e "s@{{hollow_kubelet_millicpu}}@${KUBEMARK_HOLLOW_KUBELET_MILLICPU:-40}@g" "${RESOURCE_DIRECTORY}/hollow-node.yaml" sed -i'' -e "s@{{hollow_kubelet_mem_Ki}}@${KUBEMARK_HOLLOW_KUBELET_MEM_KB:-$((100*1024))}@g" "${RESOURCE_DIRECTORY}/hollow-node.yaml"

Related discussion:
text: There are CVEs happening in different images - we already migrated Kubmeark from busybox in , but we should go one step further and migrate the base image to distroless. There are some things that would need to be changed mentioned here: /sig scalability
Can I work on this? /assign
/remove-help
- sure that would be great.
Hi based on the discussion in , the kubemark image has dependency on and maybe more other utilities. So it seems impossible to migrate the base image to distroless? Or do we need something like debian-hyperkube-base?
The mount one we may get rid of - basically we should be able to switch to FakeMounter:
Code change:
if len(o.FilenameOptions.Filenames) != 1 { return cmdutil.UsageErrorf(cmd, "--raw can only use a single local file or stdin") } <del> if strings.HasPrefix(o.FilenameOptions.Filenames[0], "http") { </del> <ins> if strings.Index(o.FilenameOptions.Filenames[0], "http://") == 0 || strings.Index(o.FilenameOptions.Filenames[0], "https://") == 0 { </ins> return cmdutil.UsageErrorf(cmd, "--raw cannot read from a url") } if o.FilenameOptions.Recursive {

Related discussion:
text: <!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: What happened: Currently when run kubectl create with --raw, it will raise "--raw cannot read from a url " if the filename has a prefix "http", which is not proper as we may have filenames like "httptest" or similar. What you expected to happen: Restrict the url check with "https://" or "http://" prefix. How to reproduce it (as minimally and precisely as possible): Run 'kubectl create --raw URI -f' with a filename which has a "http" prefix.
/sig cli
--raw should start with
the message could be improved, but I think the check is reasonable (I'd actually change it to error if the raw path doesn't start with )
This is the filename url check, not the raw path. I think you mean this
Code change:
_, err = client.CoreV1().Nodes().Patch(context.TODO(), old.Name, types.StrategicMergePatchType, patchBytes, metav1.PatchOptions{}, "status") return err } <ins> func patchPriorityClass(cs clientset.Interface, old, new *schedulingv1.PriorityClass) error { oldData, err := json.Marshal(old) if err != nil { return err } newData, err := json.Marshal(new) if err != nil { return err } patchBytes, err := strategicpatch.CreateTwoWayMergePatch(oldData, newData, &schedulingv1.PriorityClass{}) if err != nil { return fmt.Errorf("failed to create merge patch for PriorityClass %q: %v", old.Name, err) } _, err = cs.SchedulingV1().PriorityClasses().Patch(context.TODO(), old.Name, types.StrategicMergePatchType, patchBytes, metav1.PatchOptions{}) return err } </ins>

Related discussion:
text: The following PriorityClass endpoints was Promoted it 1.14 without Conformance tests, and remain untested: replace-, read-, patch-, list-, delete-collection- SchedulingV1PriorityClass According to this APIsnoop query, there are still some remaining PriorityClass endpoints which are untested. with this query you can filter untested endpoints by their category and eligiblity for conformance. e.g below shows a query to find all conformance eligible untested,stable,core endpoints ! createSchedulingV1PriorityClass deleteSchedulingV1PriorityClass Promoted to Conformance 1.19 with /sig testing /sig architecture /area conformance
Is the ultimiate goal to use conformance tests to explicitly issue all CRUD requests against PriorityClass? And, is equivalent to "update PriorityClass"?
all APIs must have Conformance tests unless they are not eligible for Conformance due to the There is other APIGroups that have both Replace and Update as endpoints. That is the prerogative of the SIG to decide.
can you please point us to similar conformance tests used for core objects such as pods and nodes?
This is a life cycle test for Create, List, Update, Patch, DeleteCollection More can be found in the test/e2e folder. Also look for examples on the Conformance at previously Promoted Conformance test Reach out if we can help with more information on our approach to Conformance testing.
I remembered about this PR. Last week a few lines of code was to cover the last of the "Lease" endpoint group for conformance. The one file contain the Conformance tests for the complete life cycle of Lease endpoints.
/assign I will work on it.
/reopen Will monitor in test grid to move to Conformance after 2 weeks soak.
Reopened this issue. <detailsIn response to : /reopen Will monitor in test grid to move to Conformance after 2 weeks soak. </details
/triage accepted
Code change:
if p.isPodBackingOff(pod) { if err := p.podBackoffQ.Add(pInfo); err != nil { klog.Errorf("Error adding pod %v to the backoff queue: %v", pod.Name, err) <ins> } else { p.unschedulableQ.delete(pod) </ins> } } else { if err := p.activeQ.Add(pInfo); err != nil { klog.Errorf("Error adding pod %v to the scheduling queue: %v", pod.Name, err) <ins> } else { p.unschedulableQ.delete(pod) </ins> } } <del> p.unschedulableQ.delete(pod) </del> } p.moveRequestCycle = p.schedulingCycle p.cond.Broadcast()

Related discussion:
text: What happened: We will now always delete the pods from unschedulable queue no matter the Add operations succeed or not in function. What you expected to happen: Delete pods from unschedulable queue only when add operations succeeds
/sig scheduling
/assign
Will the pods be to the queues again somewhere if we delete them by mistake here ? It seems that can not make sure to add them again.
I am not sure if we will add them back again somewhere. If not, not only but also need to be updated. I have already send a PR here: