jiebi/CodeConvo huggingface dataset:
Code change:
<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI -- (containerd) (crio) [sig-node] ResourceMetricsAPI [NodeFeature:ResourceMetrics] when querying /resource/metrics should report resource usage through the resource metrics api Ever since this commit was merged (containerd) (crio) We need to see what the commit is breaking this particular NodeFeatures test on both crio and containerd.
/sig node
/cc
/cc
/triage accepted
ah, let's use the RT's issue /close
Closing this issue. <detailsIn response to : ah, let's use the RT's issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Related discussion:
text: return } <del> ch <- metrics.NewLazyConstMetric(containerStartTimeDesc, metrics.GaugeValue, float64(s.StartTime.UnixNano())/float64(time.Second), s.Name, pod.PodRef.Name, pod.PodRef.Namespace) </del> <ins> ch <- metrics.NewLazyMetricWithTimestamp(s.StartTime.Time, metrics.NewLazyConstMetric(containerStartTimeDesc, metrics.GaugeValue, float64(s.StartTime.UnixNano())/float64(time.Second), s.Name, pod.PodRef.Name, pod.PodRef.Namespace)) </ins> } func (rc *resourceMetricsCollector) collectContainerCPUMetrics(ch chan<- metrics.Metric, pod summary.PodStats, s summary.ContainerStats) {
Code change:
<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI -- (containerd) (crio) [sig-node] ResourceMetricsAPI [NodeFeature:ResourceMetrics] when querying /resource/metrics should report resource usage through the resource metrics api Ever since this commit was merged (containerd) (crio) We need to see what the commit is breaking this particular NodeFeatures test on both crio and containerd.
/sig node
/cc
/cc
/triage accepted
ah, let's use the RT's issue /close
Closing this issue. <detailsIn response to : ah, let's use the RT's issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Related discussion:
text: container_memory_working_set_bytes{container="container_b",namespace="namespace_a",pod="pod_a"} 1000 1624396278302 # HELP container_start_time_seconds [ALPHA] Start time of the container since unix epoch in seconds # TYPE container_start_time_seconds gauge <del> container_start_time_seconds{container="container_a",namespace="namespace_a",pod="pod_a"} 1.6243962483020916e+09 container_start_time_seconds{container="container_a",namespace="namespace_b",pod="pod_b"} 1.6243956783020916e+09 container_start_time_seconds{container="container_b",namespace="namespace_a",pod="pod_a"} 1.6243961583020916e+09 </del> <ins> container_start_time_seconds{container="container_a",namespace="namespace_a",pod="pod_a"} 1.6243962483020916e+09 1624396248302 container_start_time_seconds{container="container_a",namespace="namespace_b",pod="pod_b"} 1.6243956783020916e+09 1624395678302 container_start_time_seconds{container="container_b",namespace="namespace_a",pod="pod_a"} 1.6243961583020916e+09 1624396158302 </ins> `, }, {
Code change:
<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI -- (containerd) (crio) [sig-node] ResourceMetricsAPI [NodeFeature:ResourceMetrics] when querying /resource/metrics should report resource usage through the resource metrics api Ever since this commit was merged (containerd) (crio) We need to see what the commit is breaking this particular NodeFeatures test on both crio and containerd.
/sig node
/cc
/cc
/triage accepted
ah, let's use the RT's issue /close
Closing this issue. <detailsIn response to : ah, let's use the RT's issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details

Related discussion:
text: }), "container_start_time_seconds": gstruct.MatchElements(containerID, gstruct.IgnoreExtras, gstruct.Elements{ <del> fmt.Sprintf("%s::%s::%s", f.Namespace.Name, pod0, "busybox-container"): boundedSample(time.Now().Add(-maxStatsAge).UnixNano(), time.Now().Add(2*time.Minute).UnixNano()), fmt.Sprintf("%s::%s::%s", f.Namespace.Name, pod1, "busybox-container"): boundedSample(time.Now().Add(-maxStatsAge).UnixNano(), time.Now().Add(2*time.Minute).UnixNano()), </del> <ins> fmt.Sprintf("%s::%s::%s", f.Namespace.Name, pod0, "busybox-container"): boundedSample(time.Now().Add(-maxStatsAge).Unix(), time.Now().Add(2*time.Minute).Unix()), fmt.Sprintf("%s::%s::%s", f.Namespace.Name, pod1, "busybox-container"): boundedSample(time.Now().Add(-maxStatsAge).Unix(), time.Now().Add(2*time.Minute).Unix()), </ins> }), "pod_cpu_usage_seconds_total": gstruct.MatchElements(podID, gstruct.IgnoreExtras, gstruct.Elements{
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --kubelet failed to start when specifying ,with following error: Try to start kubelet with : Kubernetes version (use ): master branch Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig node
Submitted PR to fix this issue.
/triage accepted /priority important-soon

Related discussion:
text: Subsystem:         KubeletSubsystem, Name:              AssignedConfigKey, Help:              "The node's understanding of intended config. The count is always 1.", <del> DeprecatedVersion: "1.22", </del> <ins> DeprecatedVersion: "1.22.0", </ins> StabilityLevel:    metrics.ALPHA, }, []string{ConfigSourceLabelKey, ConfigUIDLabelKey, ConfigResourceVersionLabelKey, KubeletConfigKeyLabelKey},
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --kubelet failed to start when specifying ,with following error: Try to start kubelet with : Kubernetes version (use ): master branch Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig node
Submitted PR to fix this issue.
/triage accepted /priority important-soon

Related discussion:
text: Subsystem:         KubeletSubsystem, Name:              ActiveConfigKey, Help:              "The config source the node is actively using. The count is always 1.", <del> DeprecatedVersion: "1.22", </del> <ins> DeprecatedVersion: "1.22.0", </ins> StabilityLevel:    metrics.ALPHA, }, []string{ConfigSourceLabelKey, ConfigUIDLabelKey, ConfigResourceVersionLabelKey, KubeletConfigKeyLabelKey},
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --kubelet failed to start when specifying ,with following error: Try to start kubelet with : Kubernetes version (use ): master branch Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig node
Submitted PR to fix this issue.
/triage accepted /priority important-soon

Related discussion:
text: Subsystem:         KubeletSubsystem, Name:              LastKnownGoodConfigKey, Help:              "The config source the node will fall back to when it encounters certain errors. The count is always 1.", <del> DeprecatedVersion: "1.22", </del> <ins> DeprecatedVersion: "1.22.0", </ins> StabilityLevel:    metrics.ALPHA, }, []string{ConfigSourceLabelKey, ConfigUIDLabelKey, ConfigResourceVersionLabelKey, KubeletConfigKeyLabelKey},
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --kubelet failed to start when specifying ,with following error: Try to start kubelet with : Kubernetes version (use ): master branch Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig node
Submitted PR to fix this issue.
/triage accepted /priority important-soon

Related discussion:
text: Subsystem:         KubeletSubsystem, Name:              ConfigErrorKey, Help:              "This metric is true (1) if the node is experiencing a configuration-related error, false (0) otherwise.", <del> DeprecatedVersion: "1.22", </del> <ins> DeprecatedVersion: "1.22.0", </ins> StabilityLevel:    metrics.ALPHA, }, )
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --I found that for with returns a very small value. The value should be almost the same as the one of . My minimal : Result: While the top three lines () are good, the float values in the bottom three lines () might be broken. Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
/assign
/cc /triage accepted

Related discussion:
text: "custom": resource.MustParse("0"), }, Limits: v1.ResourceList{ <del> "memory": resource.MustParse("2G"), </del> <ins> "memory": resource.MustParse("2.5Gi"), </ins> "custom": resource.MustParse("6"), }, }},
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --I found that for with returns a very small value. The value should be almost the same as the one of . My minimal : Result: While the top three lines () are good, the float values in the bottom three lines () might be broken. Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
/assign
/cc /triage accepted

Related discussion:
text: expected := `# HELP kube_pod_resource_limit [ALPHA] Resources limit for workloads on the cluster, broken down by pod. This shows the resource usage the scheduler and kubelet expect per pod for resources along with the unit for the resource if any. # TYPE kube_pod_resource_limit gauge kube_pod_resource_limit{namespace="test",node="node-one",pod="foo",priority="",resource="custom",scheduler="",unit=""} 6 <del> kube_pod_resource_limit{namespace="test",node="node-one",pod="foo",priority="",resource="memory",scheduler="",unit="bytes"} 2e+09 </del> <ins> kube_pod_resource_limit{namespace="test",node="node-one",pod="foo",priority="",resource="memory",scheduler="",unit="bytes"} 2.68435456e+09 </ins> # HELP kube_pod_resource_request [ALPHA] Resources requested by workloads on the cluster, broken down by pod. This shows the resource usage the scheduler and kubelet expect per pod for resources along with the unit for the resource if any. # TYPE kube_pod_resource_request gauge kube_pod_resource_request{namespace="test",node="node-one",pod="foo",priority="",resource="cpu",scheduler="",unit="cores"} 2
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --I found that for with returns a very small value. The value should be almost the same as the one of . My minimal : Result: While the top three lines () are good, the float values in the bottom three lines () might be broken. Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
/assign
/cc /triage accepted

Related discussion:
text: return base } <del> // multiply by the appropriate exponential scale switch q.Format { case DecimalExponent, DecimalSI: return base * math.Pow10(exponent) default: // fast path for exponents that can fit in 64 bits if exponent > 0 && exponent < 7 { return base * float64(int64(1)<<(exponent*10)) } return base * math.Pow(2, float64(exponent*10)) } </del> <ins> return base * math.Pow10(exponent) </ins> } // AsInt64 returns a representation of the current value as an int64 if a fast conversion
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --I found that for with returns a very small value. The value should be almost the same as the one of . My minimal : Result: While the top three lines () are good, the float values in the bottom three lines () might be broken. Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
/assign
/cc /triage accepted

Related discussion:
text: {decQuantity(1024, 0, BinarySI), 1024}, {decQuantity(8*1024, 0, BinarySI), 8 * 1024}, {decQuantity(7*1024*1024, 0, BinarySI), 7 * 1024 * 1024}, <del> {decQuantity(7*1024*1024, 1, BinarySI), (7 * 1024 * 1024) * 1024}, {decQuantity(7*1024*1024, 4, BinarySI), (7 * 1024 * 1024) * (1024 * 1024 * 1024 * 1024)}, {decQuantity(7*1024*1024, 8, BinarySI), (7 * 1024 * 1024) * (1024 * 1024 * 1024 * 1024 * 1024 * 1024 * 1024 * 1024)}, {decQuantity(7*1024*1024, -1, BinarySI), (7 * 1024 * 1024) / float64(1024)}, {decQuantity(7*1024*1024, -8, BinarySI), (7 * 1024 * 1024) / float64(1024*1024*1024*1024*1024*1024*1024*1024)}, </del> <ins> {decQuantity(7*1024*1024, 1, BinarySI), (7 * 1024 * 1024) * 10}, {decQuantity(7*1024*1024, 4, BinarySI), (7 * 1024 * 1024) * 10000}, {decQuantity(7*1024*1024, 8, BinarySI), (7 * 1024 * 1024) * 100000000}, {decQuantity(7*1024*1024, -1, BinarySI), (7 * 1024 * 1024) * math.Pow10(-1)}, // '* Pow10' and '/ float(10)' do not round the same way {decQuantity(7*1024*1024, -8, BinarySI), (7 * 1024 * 1024) / float64(100000000)}, </ins> {decQuantity(1024, 0, DecimalSI), 1024}, {decQuantity(8*1024, 0, DecimalSI), 8 * 1024},
Code change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --I found that for with returns a very small value. The value should be almost the same as the one of . My minimal : Result: While the top three lines () are good, the float values in the bottom three lines () might be broken. Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig api-machinery
/assign
/cc /triage accepted

Related discussion:
text: } } <ins> func TestStringQuantityAsApproximateFloat64(t *testing.T) { table := []struct { in  string out float64 }{ {"2Ki", 2048}, {"1.1Ki", 1126.4e+0}, {"1Mi", 1.048576e+06}, {"2Gi", 2.147483648e+09}, } for _, item := range table { t.Run(item.in, func(t *testing.T) { in, err := ParseQuantity(item.in) if err != nil { t.Fatal(err) } out := in.AsApproximateFloat64() if out != item.out { t.Fatalf("expected %v, got %v", item.out, out) } if in.d.Dec != nil { if i, ok := in.AsInt64(); ok { q := intQuantity(i, 0, in.Format) out := q.AsApproximateFloat64() if out != item.out { t.Fatalf("as int quantity: expected %v, got %v", item.out, out) } } } }) } } </ins> func benchmarkQuantities() []Quantity { return []Quantity{ intQuantity(1024*1024*1024, 0, BinarySI),
Code change:
While investigating some slowness in mounting/unmounting operations of PVs (in this case EFS volumes) for pods, we observed errors like these: fortunately it does recover quickly on Kubelet’s next retry: Looking through the Kubelet code I found out that the issue is not related to the umount calls themselves, but rather a race condition inside where we call umount. Because of the way golang’s function (called inside ) is implemented, it for the child process to finish after it is . Since there isn't any locking on the child process across those function calls, it's possible that the child process exits event before the Wait() call starts. In such case the wait call returns with a error (). This error is more likely to happen especially if the gap between the Start() and Wait() calls is more (which could for e,g happen if the goroutine had CPU cycles snatched away from it for some time in between those calls). A similar issue also exists in the . While this is a self-recovering issue (because of Kubelet retries), it does seem to increase the pod startup/cleanup latency upto a 1s. Open to suggestions for a fix. The simplest one occurring to me is to treat the error as a special one on which we don't actually return an error for the mount/unmount calls. /sig storage /sig scalability /assign
Another thing is when analyzing a distribution of such errors across different nodes, we observed a skew. The top 10% of the nodes were seeing more than 90% of these errors. And these are nodes that are running hotter (w.r.t the pod churn) compared to others: ! Given this, I'm wondering if those errors are higher on those nodes just because of statistical significance or other causes like more GC pauses happening on those nodes.
/triage accepted
^ I've opened this PR to fix the Mount/Unmount functions in the file. In general we might want to make a similar fix to other invocations of exec.Run() and exec.CombinedOutput() within our codebase. But that'll be a bigger change. Happy to open another issue for it.
Sorry for opening the PR against wrong repo, here's the right one -

Related discussion:
text: fsckErrorsCorrected = 1 // 'fsck' found errors but exited without correcting them fsckErrorsUncorrected = 4 <ins> // Error thrown by exec cmd.Run() when process spawned by cmd.Start() completes before cmd.Wait() is called (see - k/k issue #103753) errNoChildProcesses = "wait: no child processes" </ins> ) // Mounter provides the default implementation of mount.Interface
Code change:
While investigating some slowness in mounting/unmounting operations of PVs (in this case EFS volumes) for pods, we observed errors like these: fortunately it does recover quickly on Kubelet’s next retry: Looking through the Kubelet code I found out that the issue is not related to the umount calls themselves, but rather a race condition inside where we call umount. Because of the way golang’s function (called inside ) is implemented, it for the child process to finish after it is . Since there isn't any locking on the child process across those function calls, it's possible that the child process exits event before the Wait() call starts. In such case the wait call returns with a error (). This error is more likely to happen especially if the gap between the Start() and Wait() calls is more (which could for e,g happen if the goroutine had CPU cycles snatched away from it for some time in between those calls). A similar issue also exists in the . While this is a self-recovering issue (because of Kubelet retries), it does seem to increase the pod startup/cleanup latency upto a 1s. Open to suggestions for a fix. The simplest one occurring to me is to treat the error as a special one on which we don't actually return an error for the mount/unmount calls. /sig storage /sig scalability /assign
Another thing is when analyzing a distribution of such errors across different nodes, we observed a skew. The top 10% of the nodes were seeing more than 90% of these errors. And these are nodes that are running hotter (w.r.t the pod churn) compared to others: ! Given this, I'm wondering if those errors are higher on those nodes just because of statistical significance or other causes like more GC pauses happening on those nodes.
/triage accepted
^ I've opened this PR to fix the Mount/Unmount functions in the file. In general we might want to make a similar fix to other invocations of exec.Run() and exec.CombinedOutput() within our codebase. But that'll be a bigger change. Happy to open another issue for it.
Sorry for opening the PR against wrong repo, here's the right one -

Related discussion:
text: command := exec.Command(mountCmd, mountArgs...) output, err := command.CombinedOutput() if err != nil { <ins> if err.Error() == errNoChildProcesses { if command.ProcessState.Success() { // We don't consider errNoChildProcesses an error if the process itself succeeded (see - k/k issue #103753). return nil } // Rewrite err with the actual exit error of the process. err = &exec.ExitError{ProcessState: command.ProcessState} } </ins> klog.Errorf("Mount failed: %vnMounting command: %snMounting arguments: %snOutput: %sn", err, mountCmd, mountArgsLogStr, string(output)) return fmt.Errorf("mount failed: %vnMounting command: %snMounting arguments: %snOutput: %s", err, mountCmd, mountArgsLogStr, string(output))