jiebi/CodeConvo huggingface dataset:
Change:
What would you like to be : There is a tab in the k8s perf dashaboard for the scheduler perf benchmarks, but it is showing nothing, we should fix that: http://perf- What this dashboard should be showing is multiple runs per day of We can run it every hour, and summarize the 24 runs of each day to show 50%, 90% and 99% percentile. Why is this needed: Having daily runs of the benchmarks is important to track performance regressions. /sig scheduling /assign /priority important-soon
Ref
/cc
The ci job's config is at And one of failed logs could be found at I tried to reproduce it on my local machine(because I do not have Gloud credentials, I run the task directly in the container): However it works well on my local machine. It is so weird. It will be very appreciated if anyone could help reproduce it with Gloud credentials.
can you try and reach out to the test-infra people? ping them on the slack channel perhaps.
yes, I have tried to get help from , he is busy now. I'll try more.
While working on reproducing this in order to file a bug against test-infra folks I think I found a solution. Will send PRs shortly. /assign
/retitle sig-scalability-benchmarks#scheduler fails consistently due to build issues
/sig scalability Testgrid link: https://k8s- Error logs:
A word about the solution, the job uses the deprecated script which is not supported anymore by test-infra oncall / folks. I've rewritten the job definition to use pod utilities and it started working (will require to get merged first).
Thanks
Thanks !

Discussion:
<modelVersion>4.0.0</modelVersion> <groupId>io.k8s.cassandra</groupId> <artifactId>kubernetes-cassandra</artifactId> <del> <version>0.0.4</version> </del> <ins> <version>0.0.5</version> </ins> <build> <sourceDirectory>src</sourceDirectory> <plugins>

public List<InetAddress> getSeeds() { List<InetAddress> list = new ArrayList<InetAddress>(); <del> String host = getEnvOrDefault("KUBERNETES_API_HOST","https://kubernetes.default.svc.cluster.local"); </del> <ins> //String host = "https://kubernetes.default.svc.cluster.local"; String proto = "https://"; String host = getEnvOrDefault("KUBERNETES_PORT_443_TCP_ADDR", "kubernetes.default.svc.cluster.local"); String port = getEnvOrDefault("KUBERNETES_PORT_443_TCP_PORT", "443"); </ins> String serviceName = getEnvOrDefault("CASSANDRA_SERVICE", "cassandra"); String podNamespace = getEnvOrDefault("POD_NAMESPACE", "default"); String path = String.format("/api/v1/namespaces/%s/endpoints/", podNamespace);

SSLContext ctx = SSLContext.getInstance("SSL"); ctx.init(null, trustAll, new SecureRandom()); <del> URL url = new URL(host + path + serviceName); </del> <ins> URL url = new URL(proto + host + ":" + port + path + serviceName); </ins> logger.info("Getting endpoints from " + url); HttpsURLConnection conn = (HttpsURLConnection)url.openConnection();
Change:
What happened: containersockets, containerthreads, containermaxthreads metrics were as part of Kubernetes 1.16 release. However, I only see 0 for these metrics. What you expected to happen: These metrics should give non-zero numbers. How to reproduce it (as minimally and precisely as possible): New cluster using Kubernetes 1.16.2 + Prometheus 2.11.1. Scraped containersockets, containerthreads, containermaxthreads metrics always return 0 Anything else we need to know?: Environment: Kubernetes version (use ): 1.16.2 Cloud provider or hardware configuration: Azure, cluster provisioned using aks-engine OS (e.g: ): Ubuntu Kernel (e.g. ): Install tools: aks-engine Network plugin and version (if this is a network-related bug): Others: /sig api-machinery /sig node
/cc /remove-sig api-machinery
It seems like this fix has only been merged into 1.17+. Is it possible to backport this fix into 1.16 as well? (I still see this issue in 1.16.9)
I am also seeing this issue in 1.6.9. Any possibility of backporting this fix to 1.16?
Also seeing this in 1.16.9, is there anything preventing this change from being backported?
/cc
Though we meant to enable process metrics in 1.16, we accidentally didn't. I consider the collection of new metrics a feature, and don't think it should be backported. It will likely cause some additional resource usage on the node, which isn't appropriate for a patch release.

Discussion:
cadvisormetrics.NetworkUsageMetrics:     struct{}{}, cadvisormetrics.AcceleratorUsageMetrics: struct{}{}, cadvisormetrics.AppMetrics:              struct{}{}, <ins> cadvisormetrics.ProcessMetrics:          struct{}{}, </ins> } if usingLegacyStats { includedMetrics[cadvisormetrics.DiskUsageMetrics] = struct{}{}
Change:
/sig scheduling What would you like to be : Expose from constructed object. Why is this needed: Cluster Autoscaler is internally creating scheduler instance to perform predicates simulations. Up until now we used for bootstrapping scheduler. This allowed as to obtain the via (). We wanted to migrate to to simplify bootstrapping but now we cannot get the as it is not exposed via any public method/field of object. See WIP branch with migrated to () /sig scheduling
/assign

Discussion:
// Extenders returns a slice of extender config. This is exposed for // testing. Extenders() []algorithm.SchedulerExtender <ins> // GetPredicateMetadataProducer returns the predicate metadata producer. This is needed // for cluster autoscaler integration. // TODO(ahg-g): remove this once CA migrates to creating a Framework instead of a full scheduler. PredicateMetadataProducer() predicates.PredicateMetadataProducer </ins> } // ScheduleResult represents the result of one pod scheduled. It will contain

return g.cache.UpdateNodeInfoSnapshot(g.nodeInfoSnapshot) } <ins> // GetPredicateMetadataProducer returns the predicate metadata producer. This is needed // for cluster autoscaler integration. func (g *genericScheduler) PredicateMetadataProducer() predicates.PredicateMetadataProducer { return g.predicateMetaProducer } </ins> // Schedule tries to schedule the given pod to one of the nodes in the node list. // If it succeeds, it will return the name of the node. // If it fails, it will return a FitError error with reasons.

err    error } <ins> func (es mockScheduler) PredicateMetadataProducer() predicates.PredicateMetadataProducer { return nil } </ins> func (es mockScheduler) Schedule(ctx context.Context, state *framework.CycleState, pod *v1.Pod) (core.ScheduleResult, error) { return es.result, es.err }
Change:
Which jobs are failing: Which test(s) are failing: Since when has it been failing: Testgrid link: Reason for failure: check Anything else we need to know: /cc /milestone v1.17 /priority critical-urgent
/sig cluster-lifecycle These SIGs are my best guesses for this issue. Please comment if I am incorrect about one. I am a bot run by vllry. ‍
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Reopening to obvserve the test after the fix was merged.
we updated our jobs after the creation of the new branch. there is now: 1-17-on-1-16 needs more runs, to verify if the fix is valid.
this should be resolved yet still flaky due to sig-apps/StatefulSets and some other conditions.
/close
Closing this issue. <detailsIn response to : /close Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Discussion:
}, }, }) <del> if err == nil || !strings.Contains(err.Error(), "namespace") { t.Fatalf("expected error that namespace was missing from context") </del> <ins> // Accept "namespace" or "Namespace". if err == nil || !strings.Contains(err.Error(), "amespace") { t.Fatalf("expected error that namespace was missing from context, got: %v", err) </ins> } }
Change:
When running CSI migration tests focussing on multi-attach scearnios with GCE PD inline volumes that are read-only, the following error pops up in the logs and the second attachment takes 6 mins: For read-only inline volumes, the above warning/error around is unexpected with CSI migration enabled and needs to be root-caused/resolved. What happened: CSI migration tests for read-only inline GCE PD volumes occasionally fails with the above messages in the logs when executing tests like What you expected to happen: The tests and multi-attach scenarios to execute smoothly with no warnings in the logs. How to reproduce it (as minimally and precisely as possible): Execute with focus set to and run a few times. Anything else we need to know?: This has been observed with CSI migration enabled and the tests exercising read-only inline volume scenarios. Environment: Kubernetes version (use ): v1.16 Cloud provider or hardware configuration: GCE OS (e.g: ): Kernel (e.g. ): Install tools: kube- Network plugin and version (if this is a network-related bug): Others:
/cc
/sig storage
Looks like hard coding access modes to ReadWriteOnce during translation for GCE PD at is causing isMultiAttachForbidden to return leading down to We need to set the access mode to ReadOnlyMany for in-tree inline volumes like in GCE PD (as well as others like EBS that support multi-attach) depending on instead of always setting it to ReadWriteOnce.
I can pick up this fix /assign

Discussion:
partition = strconv.Itoa(int(pdSource.Partition)) } <del> pv := &v1.PersistentVolume{ </del> <ins> var am v1.PersistentVolumeAccessMode if pdSource.ReadOnly { am = v1.ReadOnlyMany } else { am = v1.ReadWriteOnce } return &v1.PersistentVolume{ </ins> ObjectMeta: metav1.ObjectMeta{ // A.K.A InnerVolumeSpecName required to match for Unmount Name: volume.Name,

}, }, }, <del> AccessModes: []v1.PersistentVolumeAccessMode{v1.ReadWriteOnce}, </del> <ins> AccessModes: []v1.PersistentVolumeAccessMode{am}, </ins> }, <del> } return pv, nil </del> <ins> }, nil </ins> } // TranslateInTreePVToCSI takes a PV with GCEPersistentDisk set from in-tree

} } } <ins> func TestInlineReadOnly(t *testing.T) { g := NewGCEPersistentDiskCSITranslator() pv, err := g.TranslateInTreeInlineVolumeToCSI(&v1.Volume{ VolumeSource: v1.VolumeSource{ GCEPersistentDisk: &v1.GCEPersistentDiskVolumeSource{ PDName:   "foo", ReadOnly: true, }, }, }) if err != nil { t.Fatalf("Failed to translate in tree inline volume to CSI: %v", err) } if pv == nil || pv.Spec.PersistentVolumeSource.CSI == nil { t.Fatal("PV or volume source unexpectedly nil") } if !pv.Spec.PersistentVolumeSource.CSI.ReadOnly { t.Error("PV readonly value not true") } ams := pv.Spec.AccessModes if len(ams) != 1 { t.Errorf("got am %v, expected length of 1", ams) } if ams[0] != v1.ReadOnlyMany { t.Errorf("got am %v, expected access mode of ReadOnlyMany", ams[0]) } } </ins>
Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: In kubernetes 1.15, Panic happends in kube-apiserver. Subsets[0] is nil when endpoints has no subsets The Panic happens in here: And everything returns to normal when we delete the endpoints in etcd What you expected to happen: Maybe kube-apiserver should print error,not panic How to reproduce it (as minimally and precisely as possible): The host's iowait of ETCD is high Anything else we need to know?: Environment: Kubernetes version (use ):Cloud provider or hardware configuration:NA OS (e.g: ):Kernel (e.g. ):Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig release
/remove-sig release /sig api-machinery
/assign

Discussion:
} <ins> func TestEmptySubsets(t *testing.T) { ns := metav1.NamespaceDefault om := func(name string) metav1.ObjectMeta { return metav1.ObjectMeta{Namespace: ns, Name: name} } endpoints := &corev1.EndpointsList{ Items: []corev1.Endpoints{{ ObjectMeta: om("foo"), Subsets:    nil, }}, } fakeClient := fake.NewSimpleClientset() if endpoints != nil { fakeClient = fake.NewSimpleClientset(endpoints) } epAdapter := reconcilers.NewEndpointsAdapter(fakeClient.CoreV1(), nil) reconciler := reconcilers.NewMasterCountEndpointReconciler(1, epAdapter) endpointPorts := []corev1.EndpointPort{ {Name: "foo", Port: 8080, Protocol: "TCP"}, } err := reconciler.RemoveEndpoints("foo", net.ParseIP("1.2.3.4"), endpointPorts) if err != nil { t.Errorf("unexpected error: %v", err) } } </ins> func TestCreateOrUpdateMasterService(t *testing.T) { ns := metav1.NamespaceDefault om := func(name string) metav1.ObjectMeta {

return nil, err } <del> ipList := make([]string, len(ipInfoList.Items)) for i, ip := range ipInfoList.Items { ipList[i] = ip.Subsets[0].Addresses[0].IP </del> <ins> ipList := make([]string, 0, len(ipInfoList.Items)) for _, ip := range ipInfoList.Items { if len(ip.Subsets) > 0 && len(ip.Subsets[0].Addresses) > 0 && len(ip.Subsets[0].Addresses[0].IP) > 0 { ipList = append(ipList, ip.Subsets[0].Addresses[0].IP) } </ins> } klog.V(6).Infof("Current master IPs listed in storage are %v", ipList)

}}, }, }, <ins> { testName:      "endpoint with no subset", serviceName:   "foo", ip:            "5.6.7.8", endpointPorts: []corev1.EndpointPort{{Name: "foo", Port: 8080, Protocol: "TCP"}}, endpointKeys:  []string{"1.2.3.4", "4.3.2.2", "4.3.2.3", "4.3.2.4"}, endpoints: &corev1.EndpointsList{ Items: []corev1.Endpoints{{ ObjectMeta: om("foo"), Subsets:    nil, }}, }, }, </ins> } for _, test := range stopTests { t.Run(test.testName, func(t *testing.T) {

return err } <ins> if len(e.Subsets) == 0 { // no action is needed to remove the endpoint return nil } </ins> // Remove our IP from the list of addresses new := []corev1.EndpointAddress{} for _, addr := range e.Subsets[0].Addresses {
Change:
<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: gce-master-scale-correctness Which test(s) are failing: [sig-cli] Kubectl client kubectl get output should contain custom columns for each resource Since when has it been failing: 11/6 Testgrid link: https://k8s- Reason for failure: Anything else we need to know: n/a /sig cli /sig scalability /cc
The label(s) cannot be applied. These labels are supported: <detailsIn response to : <!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --gce-master-scale-correctness [sig-cli] Kubectl client kubectl get output should contain custom columns for each resource 11/6 https://k8s- Anything else we need to know: n/a /sig scalability /cc Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Doesn't seem to be scale-related - failed for me the same way in the presubmit:
- do we run correctness test in smaller scale (so we can detect whether failures are scale related or not)? I can't find them on neither https://k8s- nor https://k8s- dashboards.
This is a new test that seem to be flaky at head: - seems that you this in Can you please take a look?
/assign
/reopen Let's wait for the test to pass /assign
Reopened this issue. <detailsIn response to : /reopen /assign Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
That seems to be creating one of every object from the etcd data test in an e2e test. The failing object looks like a node object, which is likely getting deleted by the Node controller as soon as it is created.
opened to skip nodes for this test
/reopen The job is not green
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
failed test run started from a commit prior to the fix merging ( started from ) /close
Closing this issue. <detailsIn response to : failed test started from a commit prior to the fix merging ( started from ) Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Discussion:
"LimitRange":  true, "PodPreset":   true, <ins> // ignored for being disruptive in an e2e, and getting automatically deleted by a controller "Node": true, </ins> // ignored temporarily while waiting for bug fix. "ComponentStatus":    true, "ClusterRoleBinding": true,
Change:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : Currently, during kubelet bootstrapping, while waiting for the node certificate being signed, there is no logs showing what is happening. I would like to add some logging to for easier troubleshooting. Why is this needed: When troubleshooting kubelet bootstrapping process on a node, if there is some issue with certificate signing mechanism, kubelet will wait for its certificate being signed for without a single line of log. Often, the last lines of kubelet log would look like the following, and kubelet will be in an confusing "infinite" wait. And then, some healthchecking monitor for kubelet may kick in and restart kubelet, resulting in a confusing loop. Adding some logging here will greatly help troubleshooting bootstrapping process on the node side. /sig node
The logging would be in the following func: I think we should control the frequency of logging (by remembering the timestamp of last log).

Discussion:
ctx, cancel := context.WithTimeout(context.Background(), 3600*time.Second) defer cancel() <ins> klog.V(2).Infof("Waiting for client certificate to be issued") </ins> return csr.WaitForCertificate(ctx, client, req) }
Change:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : Currently, during kubelet bootstrapping, while waiting for the node certificate being signed, there is no logs showing what is happening. I would like to add some logging to for easier troubleshooting. Why is this needed: When troubleshooting kubelet bootstrapping process on a node, if there is some issue with certificate signing mechanism, kubelet will wait for its certificate being signed for without a single line of log. Often, the last lines of kubelet log would look like the following, and kubelet will be in an confusing "infinite" wait. And then, some healthchecking monitor for kubelet may kick in and restart kubelet, resulting in a confusing loop. Adding some logging here will greatly help troubleshooting bootstrapping process on the node side. /sig node
The logging would be in the following func: I think we should control the frequency of logging (by remembering the timestamp of last log).

Discussion:
if c.Type == certificates.CertificateDenied { return false, fmt.Errorf("certificate signing request is not approved, reason: %v, message: %v", c.Reason, c.Message) } <del> if c.Type == certificates.CertificateApproved && csr.Status.Certificate != nil { return true, nil </del> <ins> if c.Type == certificates.CertificateApproved { if csr.Status.Certificate != nil { klog.V(2).Infof("certificate signing request %s is issued", csr.Name) return true, nil } klog.V(2).Infof("certificate signing request %s is approved, waiting to be issued", csr.Name) </ins> } } return false, nil
Change:
Which jobs are failing: pull-kubernetes-integration Which test(s) are failing: TestPreemptionRaces Testgrid link: /sig scheduling /help
/reopen I just saw this again
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
Seems to be green: https://k8s-
It's still flaky on both master and PR testing: master: https://k8s- () PR testing: (). /reopen
Reopened this issue. <detailsIn response to : It's still flaky on both master and PR testing: PR testing: (). </details
From the above two logs, I'm feeling that there is a "phantom" event (very likely of type Update by enforced resync) triggering the pod in the completed N<subth</subtest, to be still alive in the N+1<subth</subtest.
/triage accepted working on a fix.
It would be great if we assign a different name to the Nth round test's preemptor pod. Currently, whatever this is Nth test, the preemptor pod is always called 'preemptor-pod'.

Discussion:
} func (sched *Scheduler) updatePodInSchedulingQueue(oldObj, newObj interface{}) { <del> pod := newObj.(*v1.Pod) if sched.skipPodUpdate(pod) { </del> <ins> oldPod, newPod := oldObj.(*v1.Pod), newObj.(*v1.Pod) // Bypass update event that carries identical objects; otherwise, a duplicated // Pod may go through scheduling and cause unexpected behavior (see #96071). if oldPod.ResourceVersion == newPod.ResourceVersion { </ins> return } <del> if err := sched.SchedulingQueue.Update(oldObj.(*v1.Pod), pod); err != nil { </del> <ins> if sched.skipPodUpdate(newPod) { return } if err := sched.SchedulingQueue.Update(oldPod, newPod); err != nil { </ins> utilruntime.HandleError(fmt.Errorf("unable to update %T: %v", newObj, err)) } }
Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: I tried to with a Deployment spec that was roughly shaped like this (note that the is invalid, since it should be a list of instead of ). This caused kubectl to fail to create a three-way patch and panic. <details<summaryClick here for the stacktrace</summary</detailsWhat you expected to happen: An error is returned How to reproduce it (as minimally and precisely as possible): Create a deployment, then try to diff using the YAML provided above (make sure to replace the name and namespace) Anything else we need to know?: The panic occurs on this line because is nil: It's set to nil here: Environment: Kubernetes version (use ):Cloud provider or hardware configuration: N/A OS (e.g: ): N/A Kernel (e.g. ): N/A Install tools: N/A Network plugin and version (if this is a network-related bug): N/A Others: N/A
/sig cli
Raised a pr which may fix this bug.

Discussion:
Force           bool ServerSideApply bool ForceConflicts  bool <ins> genericclioptions.IOStreams </ins> } var _ Object = &InfoObject{}

ResourceVersion: resourceVersion, } <del> _, result, err := patcher.Patch(obj.Info.Object, modified, obj.Info.Source, obj.Info.Namespace, obj.Info.Name, nil) </del> <ins> _, result, err := patcher.Patch(obj.Info.Object, modified, obj.Info.Source, obj.Info.Namespace, obj.Info.Name, obj.ErrOut) </ins> return result, err }

Force:           force, ServerSideApply: o.ServerSideApply, ForceConflicts:  o.ForceConflicts, <ins> IOStreams:       o.Diff.IOStreams, </ins> } err = differ.Diff(obj, printer)