jiebi/CodeConvo huggingface dataset:
Code change:
cc who found it. the reason is the calls from the import here: i suspect the problem surfaced with this bug fix. cc more refs: is an anti-pattern in this case. ideally metrics registration should not trigger warnings for a simple version output. mitigation on the user side is to pipe stderr to a null device (as long as klog is writing to stderr) /kind bug /priority important-longterm /area kubelet /wg component-standard
/sig node
should be fixed before 1.17.0, doesn't block beta.2
cc
Could this be caused by the following in init() of ? It was marked I wonder when deprecatedRequestLatency can be dropped.
looks like was deprecated in 1.14 - Is that the only one that needs to be dropped? are there more.
Dropping restclientrequestlatencyseconds should remove the warning for --version. In the 1.14 changelog, there were quite a few other metrics deprecated. I guess we don't have to drop them now.
/assign It's an interesting issue. I will investigate it and get back to you. By the way: I tried this morning, I got 4 warnings, things getting worse as we deprecated more metrics this weekend. (the log can not point the exact name of a metric, we intended to improve it in v1.18)
it's being invoked from here: the message is not helpful, since it doesn't tell you which metric is deprecated
Yes. It's a known issue and we intended to improve it in v1.18. Now, I suggest removing the log temporarily, Fired a PR , for reasons as follows: Too many binaries have been affected.(kubelet, kube-proxy, kube-apiserver, kube-controller-manager, kube-proxy, kube-scheduler). We should consider more about this issue. v1.17.0 is fast approaching, we should make a quick fix. All components try to register a deprecated metrics will be affected. Maybe we should record deprecated/hidden metrics by a new metric instead of the log.
Summary the root cause here: According to metrics deprecated policy, if a metric's - = 1, it will become a hidden metric. Hidden means we will not register it to the Prometheus registry by default. If we try to register a hidden metric to the stability framework, the warning log will be printed out. Because some metrics registered by init() calls from the import, so...
I have tried to print the metric's name at local: kubelet: kube-proxy: kube-apiserver: kube-controller-manager: kube-scheduler:
ideally the registration should happen outside of init(), which means calling it on demand. but i will defer to the owning sigs.
In principle, yes, I agree. Otherwise, we need to be very very careful with every log in the stability framework. I think we can track it and work out by iteration. cc for comments.

Related discussion:
text: return } if shouldHide(&version, selfVersion) { <del> klog.Warningf("This metric(%s) has been deprecated for more than one release, hiding.", d.fqName) </del> <ins> // TODO(RainbowMango): Remove this log temporarily. https://github.com/kubernetes/kubernetes/issues/85369 // klog.Warningf("This metric(%s) has been deprecated for more than one release, hiding.", d.fqName) </ins> d.isHidden = true } })
Code change:
cc who found it. the reason is the calls from the import here: i suspect the problem surfaced with this bug fix. cc more refs: is an anti-pattern in this case. ideally metrics registration should not trigger warnings for a simple version output. mitigation on the user side is to pipe stderr to a null device (as long as klog is writing to stderr) /kind bug /priority important-longterm /area kubelet /wg component-standard
/sig node
should be fixed before 1.17.0, doesn't block beta.2
cc
Could this be caused by the following in init() of ? It was marked I wonder when deprecatedRequestLatency can be dropped.
looks like was deprecated in 1.14 - Is that the only one that needs to be dropped? are there more.
Dropping restclientrequestlatencyseconds should remove the warning for --version. In the 1.14 changelog, there were quite a few other metrics deprecated. I guess we don't have to drop them now.
/assign It's an interesting issue. I will investigate it and get back to you. By the way: I tried this morning, I got 4 warnings, things getting worse as we deprecated more metrics this weekend. (the log can not point the exact name of a metric, we intended to improve it in v1.18)
it's being invoked from here: the message is not helpful, since it doesn't tell you which metric is deprecated
Yes. It's a known issue and we intended to improve it in v1.18. Now, I suggest removing the log temporarily, Fired a PR , for reasons as follows: Too many binaries have been affected.(kubelet, kube-proxy, kube-apiserver, kube-controller-manager, kube-proxy, kube-scheduler). We should consider more about this issue. v1.17.0 is fast approaching, we should make a quick fix. All components try to register a deprecated metrics will be affected. Maybe we should record deprecated/hidden metrics by a new metric instead of the log.
Summary the root cause here: According to metrics deprecated policy, if a metric's - = 1, it will become a hidden metric. Hidden means we will not register it to the Prometheus registry by default. If we try to register a hidden metric to the stability framework, the warning log will be printed out. Because some metrics registered by init() calls from the import, so...
I have tried to print the metric's name at local: kubelet: kube-proxy: kube-apiserver: kube-controller-manager: kube-scheduler:
ideally the registration should happen outside of init(), which means calling it on demand. but i will defer to the owning sigs.
In principle, yes, I agree. Otherwise, we need to be very very careful with every log in the stability framework. I think we can track it and work out by iteration. cc for comments.

Related discussion:
text: return } if shouldHide(&version, selfVersion) { <del> klog.Warningf("This metric has been deprecated for more than one release, hiding.") </del> <ins> // TODO(RainbowMango): Remove this log temporarily. https://github.com/kubernetes/kubernetes/issues/85369 // klog.Warningf("This metric has been deprecated for more than one release, hiding.") </ins> r.isHidden = true } })
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: When developing unit tests related to PDBs/Deletion, mutates pod resource via adding deletion timestamp, and this directly conflicts with checking resource version. What you expected to happen: No resource version mutation. How to reproduce it (as minimally and precisely as possible): I'm attempting to add preconditions to delete call in eviction API: Anything else we need to know?: Happens locally and in CI. Environment: Kubernetes version (use ): Head of master: Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
There are no sig labels on this issue. Please add a sig label by either: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. See the . The in method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
can you provide more details about what you're observing?
You can see an example here: Here's the relevant section of the tests: There really shouldn't be anything modifying the pod after creation until the delete request, thus I don't believe we should be hitting this precondition failure. Resource Version in precondition I believe to be correct, but when we contact etcd to verify, it's increased by one. It's not clear why this is happening, it appears to be a bug.
Okay, I've sorted this out. We have a conflict with 'graceful' deletion with this approach: This updates the object to add a deletion timestamp, thus the first call to eviction will always fail if we need to set ResourceVersion.

Related discussion:
text: // TODO: remove the check, because we support no-op updates now. if graceful || pendingFinalizers || shouldUpdateFinalizers { err, ignoreNotFound, deleteImmediately, out, lastExisting = e.updateForGracefulDeletionAndFinalizers(ctx, name, key, options, preconditions, deleteValidation, obj) <ins> // Update the preconditions.ResourceVersion if set since we updated the object. if err == nil && deleteImmediately && preconditions.ResourceVersion != nil { accessor, err = meta.Accessor(out) if err != nil { return out, false, kubeerr.NewInternalError(err) } resourceVersion := accessor.GetResourceVersion() preconditions.ResourceVersion = &resourceVersion } </ins> } // !deleteImmediately covers all cases where err != nil. We keep both to be future-proof.
Code change:
Was just perusing the code and looks like we translate regional disks into the "zonal volume ID fmt", and there doesn't seem to be any tests for regional disks running so we should confirm that this works and fix it if it doesn't /assign /cc
/sig storage
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/lifecycle frozen /assign
/assign

Related discussion:
text: if err != nil { return nil, fmt.Errorf("failed to get region from zones: %v", err) } <del> volID = fmt.Sprintf(volIDZonalFmt, UnspecifiedValue, region, pv.Spec.GCEPersistentDisk.PDName) </del> <ins> volID = fmt.Sprintf(volIDRegionalFmt, UnspecifiedValue, region, pv.Spec.GCEPersistentDisk.PDName) </ins> } else { // Unspecified volID = fmt.Sprintf(volIDZonalFmt, UnspecifiedValue, UnspecifiedValue, pv.Spec.GCEPersistentDisk.PDName)
Code change:
Was just perusing the code and looks like we translate regional disks into the "zonal volume ID fmt", and there doesn't seem to be any tests for regional disks running so we should confirm that this works and fix it if it doesn't /assign /cc
/sig storage
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/lifecycle frozen /assign
/assign

Related discussion:
text: v1 "k8s.io/api/core/v1" storage "k8s.io/api/storage/v1" <ins> metav1 "k8s.io/apimachinery/pkg/apis/meta/v1" </ins> ) func NewStorageClass(params map[string]string, allowedTopologies []v1.TopologySelectorTerm) *storage.StorageClass {
Code change:
Was just perusing the code and looks like we translate regional disks into the "zonal volume ID fmt", and there doesn't seem to be any tests for regional disks running so we should confirm that this works and fix it if it doesn't /assign /cc
/sig storage
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/lifecycle frozen /assign
/assign

Related discussion:
text: t.Errorf("got am %v, expected access mode of ReadOnlyMany", ams[0]) } } <ins> func TestTranslateInTreePVToCSIVolIDFmt(t *testing.T) { g := NewGCEPersistentDiskCSITranslator() pdName := "pd-name" tests := []struct { desc               string topologyLabelKey   string topologyLabelValue string wantVolId          string }{ { desc:               "beta topology key zonal", topologyLabelKey:   v1.LabelFailureDomainBetaZone, topologyLabelValue: "us-east1-a", wantVolId:          "projects/UNSPECIFIED/zones/us-east1-a/disks/pd-name", }, { desc:               "v1 topology key zonal", topologyLabelKey:   v1.LabelTopologyZone, topologyLabelValue: "us-east1-a", wantVolId:          "projects/UNSPECIFIED/zones/us-east1-a/disks/pd-name", }, { desc:               "beta topology key regional", topologyLabelKey:   v1.LabelFailureDomainBetaZone, topologyLabelValue: "us-central1-a__us-central1-c", wantVolId:          "projects/UNSPECIFIED/regions/us-central1/disks/pd-name", }, { desc:               "v1 topology key regional", topologyLabelKey:   v1.LabelTopologyZone, topologyLabelValue: "us-central1-a__us-central1-c", wantVolId:          "projects/UNSPECIFIED/regions/us-central1/disks/pd-name", }, } for _, tc := range tests { t.Run(tc.desc, func(t *testing.T) { translatedPV, err := g.TranslateInTreePVToCSI(&v1.PersistentVolume{ ObjectMeta: metav1.ObjectMeta{ Labels: map[string]string{tc.topologyLabelKey: tc.topologyLabelValue}, }, Spec: v1.PersistentVolumeSpec{ PersistentVolumeSource: v1.PersistentVolumeSource{ GCEPersistentDisk: &v1.GCEPersistentDiskVolumeSource{ PDName: pdName, }, }, }, }) if err != nil { t.Errorf("got error translating in-tree PV to CSI: %v", err) } if got := translatedPV.Spec.PersistentVolumeSource.CSI.VolumeHandle; got != tc.wantVolId { t.Errorf("got translated volume handle: %q, want %q", got, tc.wantVolId) } }) } } </ins>
Code change:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : We should allow collecting both default scheduler metrics (Prometheus) and custom metrics easily. At a high level, these collectors should produce a json formatted result which will then be consumed by perf-dash to present in the dashboard. Prometheus metrics We should provide a metrics collector that collects and summarizes default scheduler metrics defined in . To begin with, we can manually pick a list of interesting metrics. Longer term, we could allow configuring the metric list in test definition. already had a PoC . As a reference, is how clusterloader collects scheduler latency metrics. Custom metrics The most important metric is scheduling throughput. is how clusterloader calculates the throughput. We can start with just calculating the average throughput without the percentiles. We may want other custom metrics that are either common to all tests or specific to certain tests. We should allow these custom collectors to be easily and ideally configurable. Why is this needed: Give more insights into scheduler performance. Part of /sig scheduling /assign
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Related discussion:
text: "//staging/src/k8s.io/client-go/informers/core/v1:go_default_library", "//staging/src/k8s.io/client-go/kubernetes:go_default_library", "//staging/src/k8s.io/client-go/rest:go_default_library", <ins> "//staging/src/k8s.io/component-base/metrics/legacyregistry:go_default_library", </ins> "//test/integration/util:go_default_library", <ins> "//vendor/github.com/prometheus/client_model/go:go_default_library", "//vendor/k8s.io/klog:go_default_library", </ins> ], )
Code change:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : We should allow collecting both default scheduler metrics (Prometheus) and custom metrics easily. At a high level, these collectors should produce a json formatted result which will then be consumed by perf-dash to present in the dashboard. Prometheus metrics We should provide a metrics collector that collects and summarizes default scheduler metrics defined in . To begin with, we can manually pick a list of interesting metrics. Longer term, we could allow configuring the metric list in test definition. already had a PoC . As a reference, is how clusterloader collects scheduler latency metrics. Custom metrics The most important metric is scheduling throughput. is how clusterloader calculates the throughput. We can start with just calculating the average throughput without the percentiles. We may want other custom metrics that are either common to all tests or specific to certain tests. We should allow these custom collectors to be easily and ideally configurable. Why is this needed: Give more insights into scheduler performance. Part of /sig scheduling /assign
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Related discussion:
text: configFile = "config/performance-config.yaml" ) <ins> var ( defaultMetrics = []string{ "scheduler_scheduling_algorithm_predicate_evaluation_seconds", "scheduler_scheduling_algorithm_priority_evaluation_seconds", "scheduler_binding_duration_seconds", "scheduler_e2e_scheduling_duration_seconds", } ) </ins> // testCase configures a test case to run the scheduler performance test. Users should be able to // provide this via a YAML file. //
Code change:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : We should allow collecting both default scheduler metrics (Prometheus) and custom metrics easily. At a high level, these collectors should produce a json formatted result which will then be consumed by perf-dash to present in the dashboard. Prometheus metrics We should provide a metrics collector that collects and summarizes default scheduler metrics defined in . To begin with, we can manually pick a list of interesting metrics. Longer term, we could allow configuring the metric list in test definition. already had a PoC . As a reference, is how clusterloader collects scheduler latency metrics. Custom metrics The most important metric is scheduling throughput. is how clusterloader calculates the throughput. We can start with just calculating the average throughput without the percentiles. We may want other custom metrics that are either common to all tests or specific to certain tests. We should allow these custom collectors to be easily and ideally configurable. Why is this needed: Give more insights into scheduler performance. Part of /sig scheduling /assign
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Related discussion:
text: } func BenchmarkPerfScheduling(b *testing.B) { <ins> dataItems := DataItems{Version: "v1"} </ins> tests := getSimpleTestCases(configFile) for _, test := range tests {
Code change:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : We should allow collecting both default scheduler metrics (Prometheus) and custom metrics easily. At a high level, these collectors should produce a json formatted result which will then be consumed by perf-dash to present in the dashboard. Prometheus metrics We should provide a metrics collector that collects and summarizes default scheduler metrics defined in . To begin with, we can manually pick a list of interesting metrics. Longer term, we could allow configuring the metric list in test definition. already had a PoC . As a reference, is how clusterloader collects scheduler latency metrics. Custom metrics The most important metric is scheduling throughput. is how clusterloader calculates the throughput. We can start with just calculating the average throughput without the percentiles. We may want other custom metrics that are either common to all tests or specific to certain tests. We should allow these custom collectors to be easily and ideally configurable. Why is this needed: Give more insights into scheduler performance. Part of /sig scheduling /assign
/reopen
Reopened this issue. <detailsIn response to : /reopen Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Related discussion:
text: for feature, flag := range test.FeatureGates { defer featuregatetesting.SetFeatureGateDuringTest(b, utilfeature.DefaultFeatureGate, feature, flag)() } <del> perfScheduling(test, b) </del> <ins> dataItems.DataItems = append(dataItems.DataItems, perfScheduling(test, b)...) </ins> }) } <ins> if err := dataItems2JSONFile(dataItems, b.Name()); err != nil { klog.Fatalf("%v: unable to write measured data: %v", b.Name(), err) } </ins> } <del> func perfScheduling(test testCase, b *testing.B) { </del> <ins> func perfScheduling(test testCase, b *testing.B) []DataItem { </ins> var nodeStrategy testutils.PrepareNodeStrategy = &testutils.TrivialNodePrepareStrategy{} if test.Nodes.NodeAllocatableStrategy != nil { nodeStrategy = test.Nodes.NodeAllocatableStrategy