jiebi/CodeConvo huggingface dataset:
Change:
<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --Which jobs are flaking: Which test(s) are flaking: Testgrid link: Reason for failure: Anything else we need to know: /sig cli /cc /milestone v1.20 /prority important-soon
after creating the CRD, before applying the CR, kubectl should wait until includes :
/assign
/assign is this issue related to ?
no, this is simply kubectl trying to apply a CR before the server has made the API available at all
Got it! and I have filed to address this issue.
Still happening:
still happening: looks like the root cause is not related to CRD: I think the message Error from server (NotFound): "myobj" not found is expected, as the CR was not actually created: /reopen
Reopened this issue. <detailsIn response to : still happening: I think the message Error from server (NotFound): "myobj" not found is expected, as the CR was not actually created: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
yes, the actual error is later: Line 177 succeeds, and the next line fails:

Discussion:
} __EOF__ <ins> # Ensure the API server has recognized and started serving the associated CR API local tries=5 for i in $(seq 1 $tries); do local output output=$(kubectl "${kube_flags[@]:?}" api-resources --api-group mygroup.example.com -oname) if kube::test::if_has_string "$output" resources.mygroup.example.com; then break fi echo "${i}: Waiting for CR API to be available" sleep "$i" done </ins> # Dry-run create the CR kubectl "${kube_flags[@]:?}" apply --dry-run=server -f hack/testdata/CRD/resource.yaml "${kube_flags[@]:?}" # Make sure that the CR doesn't exist
Change:
What happened: The performance benchmark does not finish even after 10 minutes of waiting. This is because the nodes do not have a label set but the pod affinity rules in the benchmark's pod spec expects one: What you expected to happen: The test should finish. How to reproduce it (as minimally and precisely as possible):
/sig scheduling /sig testing /kind bug

Discussion:
numPodsToSchedule: 2000 - template: desc: MixedSchedulingBasePod <ins> nodes: nodeTemplatePath: config/node-default.yaml labelNodePrepareStrategy: labelKey: "topology.kubernetes.io/zone" labelValues: ["zone1"] </ins> initPods: - podTemplatePath: config/pod-default.yaml - podTemplatePath: config/pod-with-pod-affinity.yaml
Change:
<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --Which jobs are flaking: Which test(s) are flaking: Testgrid link: Reason for failure: Anything else we need to know: /sig scheduling /priority critical-urgent /cc /milestone v1.19 This test waits for the limit range to be deleted before attempting to create the so this error is surprising. It appears there are no limit ranges present when they are listed here:
It seems this job failed just once since Jul 23, so I'm wondering is it a must for 1.19?
Given this is a conformance test I would prefer we at least know what is causing the issue
This is still flaking with this same issue. /reopen
Reopened this issue. <detailsIn response to : This is still flaking with this same issue. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/milestone v1.20
This issue has not occurred recently and seems to have already been fixed. I think we can close this issue.
Yes, was expected to get it fixed. /close
Closing this issue. <detailsIn response to : Yes, was expected to get it fixed. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
this is still flaking in the 1.20 branch : confused : as of one hour ago

Discussion:
imageutils "k8s.io/kubernetes/test/utils/image" "github.com/onsi/ginkgo" <del> "github.com/onsi/gomega" </del> ) const (

framework.ExpectNoError(err) ginkgo.By("Verifying the LimitRange was deleted") <del> gomega.Expect(wait.Poll(time.Second*5, e2eservice.RespondingTimeout, func() (bool, error) { selector := labels.SelectorFromSet(labels.Set(map[string]string{"name": limitRange.Name})) options := metav1.ListOptions{LabelSelector: selector.String()} limitRanges, err := f.ClientSet.CoreV1().LimitRanges(f.Namespace.Name).List(context.TODO(), options) </del> <ins> err = wait.Poll(time.Second*5, e2eservice.RespondingTimeout, func() (bool, error) { limitRanges, err := f.ClientSet.CoreV1().LimitRanges(f.Namespace.Name).List(context.TODO(), metav1.ListOptions{}) </ins> if err != nil { framework.Logf("Unable to retrieve LimitRanges: %v", err)

return true, nil } <del> if len(limitRanges.Items) > 0 { if limitRanges.Items[0].ObjectMeta.DeletionTimestamp == nil { framework.Logf("deletion has not yet been observed") return false, nil } return true, nil </del> <ins> for i := range limitRanges.Items { lr := limitRanges.Items[i] framework.Logf("LimitRange %v/%v has not yet been deleted", lr.Namespace, lr.Name) </ins> } return false, nil <del> })) framework.ExpectNoError(err, "kubelet never observed the termination notice") </del> <ins> }) framework.ExpectNoError(err) </ins> ginkgo.By("Creating a Pod with more than former max resources") pod = newTestPod(podName+"2", getResourceList("600m", "600Mi", "600Gi"), v1.ResourceList{})
Change:
Which test(s) are flaking: pkg/scheduler unit tests occasionally timeout at 5 minutes Reason for failure: Stats from 10 runs of the package: The average (excluding the run that timed out) is ~40 seconds. The run that timed out took at least 5 minutes before being killed. That smells like a deadlock / failure-to-progress scenario which could be a real bug. Our presubmit CI runs tolerate flakes, retrying 3 times and passing on a single success, which is masking this issue. Observed in which is turning off that toleration and exercising every unit test package 10 times to ensure consistent passes: I can reproduce this by building the unit test package with: and then running repeatedly with : this appears to hang increasing numbers of parallel runs on the deadlock until timeouts are hit: /sig scheduling
the first step is to try individual tests in that package to see if a single test is deadlocking Stressing each test individually with short timeouts (set at at least 5x the normal run time of the test) and seeing if they pass with no timeout failures after 2x the specified timeout: [x] [x] [x] [x] [x] [x] [x] [x] [x] [x] [x] [x] [x] [x] [x] [ ] takes ~.1 seconds on average, immediately starts failing with stress test timeout of 10s, fix in [x] [x] [x] [x] takes 15 seconds on average [x] [x] [x]
can consistently reproduce a hang with: I this debugging: and the hang appears to be while waiting for the create calls and resulting events to be produced:
the full goroutine dump at the time of the hang: <details</details
this is also reproducible against the release-1.18 branch
could you help take a look at the problem in ?
/priority important-soon
/assign
PR up for review Thanks a lot for the repro steps TIL
fantastic, that resolves the MultipleProfiles flake:
TestSchedulerWithVolumeBinding looks flaky as well (see )
I see. I went by the title thinking that the other one was already resolved. Let me have a look.
np, good to knock them out one at a time... the symptoms were consistent so I left it as one issue thinking there might be a shared root cause, but that doesn't appear to be the case
my mistake, I got the wrong average runtime for the volume binding test... it actually takes ~16 seconds because we artificially wait for channel timeout for ~2 seconds for each of the subtests. when I stressed it with a timeout of 1m, I got no flakes separately, we could look into improving the runtime of that test by selecting between error/bind/timeout with something like the following, but that is a different cleanup/improvement issue, not a flake fix

Discussion:
// Run scheduler. informerFactory.Start(ctx.Done()) <ins> informerFactory.WaitForCacheSync(ctx.Done()) </ins> go sched.Run(ctx) // Send pods to be scheduled.
Change:
Which test(s) are flaking: TestServiceRegistryUpdateDryRun Reason for failure: This is flaking rarely enough that it is not caught by our CI jobs, which currently tolerate up to 2 unit test failures per run (!). With that toleration removed in , this flake has been seen (). Reproducible with the following steps: the bug in the test that casts the return value even when the create fails: logging on slow etcd writes because of etcd-io/etcd a test binary for the affected package: the affected test using the tool: failures are seen, look at the logs to see the details about the error: /sig network
/triage unresolved Comment when the issue is assessed and confirmed. I am a bot run by vllry. ‍
heh this, was a funny one, we had 1/2^8 chances of using a duplicate, with 1/2^16 I've got 2000 runs without any failures
/assign /remove-triage unresolved

Discussion:
}}, }, }, rest.ValidateAllObjectFunc, &metav1.CreateOptions{}) <del> svc := obj.(*api.Service) </del> if err != nil { t.Fatalf("Expected no error: %v", err) } <ins> svc := obj.(*api.Service) </ins> // Test dry run update request external name to node port updatedSvc, created, err := storage.Update(ctx, svc.Name, rest.DefaultUpdatedObjectInfo(&api.Service{

Selector:        map[string]string{"bar": "baz"}, SessionAffinity: api.ServiceAffinityNone, Type:            api.ServiceTypeNodePort, <ins> ClusterIP:       "1.2.3.5", </ins> Ports: []api.ServicePort{{ NodePort:   30020, Port:       6502,

}}, }, }, rest.ValidateAllObjectFunc, &metav1.CreateOptions{}) <del> svc = obj.(*api.Service) </del> if err != nil { t.Fatalf("Expected no error: %v", err) } <ins> svc = obj.(*api.Service) </ins> _, _, err = storage.Update(ctx, svc.Name, rest.DefaultUpdatedObjectInfo(&api.Service{ ObjectMeta: metav1.ObjectMeta{ Name:            svc.Name,
Change:
What happened: I created a Pod with a ProjectedVolume with an empty array of , but Kubernetes transformed that value to . Some clients are happy with either value (e.g. Go) while others are not (e.g. Python). For example, the following is accepted and stored by Kubernetes, but cannot be read by the Python client: The field has been transformed to : Even a JSON patch has no effect: What you expected to happen: I expected the required (OpenAPI) field of core/v1.ProjectedVolumeSource to be returned as the empty array I submitted. How to reproduce it (as minimally and precisely as possible): Create the above Pod: See that has been transformed to : See that the officially-supported Python client is unable to deserialize the Pod: Anything else we need to know?: I recognize there's a lot of layers involved in the steps above. The ineffectiveness of the JSON patch is what makes me think the culprit is somewhere in the field spec or API somehow. /sig api-machinery Environment: Kubernetes version (use ): [official-clients]:
/remove-sig api-machinery /sig storage the issue is in the API definition, which does not indicate the sources field is optional (even though it is)
/cc

Discussion:
"type": "array" } }, <del> "required": [ "sources" ], </del> "type": "object" }, "io.k8s.api.core.v1.QuobyteVolumeSource": {

// Represents a projected volume source message ProjectedVolumeSource { // list of volume projections <ins> // +optional </ins> repeated VolumeProjection sources = 1; // Mode bits used to set permissions on created files by default.

// Represents a projected volume source type ProjectedVolumeSource struct { // list of volume projections <ins> // +optional </ins> Sources []VolumeProjection `json:"sources" protobuf:"bytes,1,rep,name=sources"` // Mode bits used to set permissions on created files by default. // Must be an octal value between 0000 and 0777 or a decimal value between 0 and 511.
Change:
Which test(s) are flaking: TestInvalidPodFiltered Reason for failure: This is flaking rarely enough that it is not caught by our CI jobs, which currently tolerate up to 2 unit test failures per run (!). With that toleration removed in , this flake has been seen (). Reproducible with the following steps: a test binary for the affected package: the affected test using the tool: failures are seen, look at the logs to see the details about the error: /sig node cc
/assign
It is a bit funny that if I change to the following(wait for 3 seconds until receive event from channel): The test would consistently fail. I am not sure what the invalid update means at , so I cannot think of an ideal fix. I would like to defer to the members from sig-node. /unassign
Isn't this asserting a negative? I don't think it can work, especially not without a wait or some way of forcing the config mux storage merger thingy to run. The multiple levels is kind of mindbending, really. I come to the same conclusion as Maybe the focus needs to be on the invalid update? I do not understand why it is invalid from the context. Tracing it backwards in time, v1.8 was the last time this passed. v1.9 changed what validation means, I think I have a solution. /assign
Thanks for going deep down the commit history and found out the problem !
Made if either of you wish to review, and confirm my understanding.

Discussion:
channel <- podUpdate expectPodUpdate(t, ch, CreatePodUpdate(kubetypes.ADD, TestSource, CreateValidPod("foo", "new"))) <del> // add an invalid update podUpdate = CreatePodUpdate(kubetypes.UPDATE, TestSource, &v1.Pod{ObjectMeta: metav1.ObjectMeta{Name: "foo"}}) </del> <ins> // add an invalid update, pod with the same name podUpdate = CreatePodUpdate(kubetypes.ADD, TestSource, CreateValidPod("foo", "new")) </ins> channel <- podUpdate expectNoPodUpdate(t, ch) }
Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Kubelet doesn't start during , journalctl shows: What you expected to happen: Kubelet correctly identifies absence of as "no swap present", kubelet service starts and can finish. How to reproduce it (as minimally and precisely as possible): Build a Linux kernel with , install kubeadm, kubelet and try . Anything else we need to know?: No registered swap devices or a kernel without swap enabled both means that there is no swap space present. Kubectl successfully checks for the former case but throws an error on the later. Failure to open means no swap. Environment: Kubernetes version (use ): Hardware configuration: OS: Kernel: Install tools:
/sig cluster-lifecycle
this is a potential improvement for the kubelet. /sig node /remove-sig cluster-lifecycle
/assign
The issue can be worked around by adding to (defined in on Debian): /etc/default/kubelet

Discussion:
if failSwapOn { // Check whether swap is enabled. The Kubelet does not support running with swap enabled. <del> swapData, err := ioutil.ReadFile("/proc/swaps") </del> <ins> swapFile := "/proc/swaps" swapData, err := ioutil.ReadFile(swapFile) </ins> if err != nil { <del> return nil, err } swapData = bytes.TrimSpace(swapData) // extra trailing n swapLines := strings.Split(string(swapData), "n") </del> <ins> if os.IsNotExist(err) { klog.Warningf("file %v does not exist, assuming that swap is disabled", swapFile) } else { return nil, err } } else { swapData = bytes.TrimSpace(swapData) // extra trailing n swapLines := strings.Split(string(swapData), "n") </ins> <del> // If there is more than one line (table headers) in /proc/swaps, swap is enabled and we should // error out unless --fail-swap-on is set to false. if len(swapLines) > 1 { return nil, fmt.Errorf("running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: %v", swapLines) </del> <ins> // If there is more than one line (table headers) in /proc/swaps, swap is enabled and we should // error out unless --fail-swap-on is set to false. if len(swapLines) > 1 { return nil, fmt.Errorf("running with swap on is not supported, please disable swap! or set --fail-swap-on flag to false. /proc/swaps contained: %v", swapLines) } </ins> } }
Change:
modified container log file removal in a way that made TestRemoveContainer flaky See Note that the flake is masked by the fact that the unit test job will retry tests up to 3 times. With , this flake is quite pronounced: A fix for the flake is in , but it's unclear to me whether it is correct that the removal order is non-deterministic. /assign /sig node /milestone v1.19 /priority critical-urgent
cc

Discussion:
err = m.removeContainer(containerID) assert.NoError(t, err) <del> // Verify container log is removed assert.Equal(t, []string{expectedContainerLogPath, expectedContainerLogPathRotated, expectedContainerLogSymlink}, fakeOS.Removes) </del> <ins> // Verify container log is removed. // We could not predict the order of `fakeOS.Removes`, so we use `assert.ElementsMatch` here. assert.ElementsMatch(t, []string{expectedContainerLogSymlink, expectedContainerLogPath, expectedContainerLogPathRotated}, fakeOS.Removes) </ins> // Verify container is removed assert.Contains(t, fakeRuntime.Called, "RemoveContainer") containers, err := fakeRuntime.ListContainers(&runtimeapi.ContainerFilter{Id: containerID})
Change:
<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --Which jobs are flaking: pull-kubernetes-e2e-azure-disk-windows Which test(s) are flaking: [Fail] Dynamic Provisioning [single-az] [It] should create a statefulset object, write and read to it, delete the pod and write and read to it again [] [] [Windows] Testgrid link: Reason for failure: Unknown Anything else we need to know: Unexpected error: <*errors.errorString | 0xc0007c26a0: { s: "Failed to find "hello worldrnhello worldrn", last result: "x00x00x00x00x00x00x00x00x00x00x00x00x00hello worldrn"", } Failed to find "hello world hello world ", last result: " hello world " occurred
/sig cloud-provider /area provider/azure
Ref: I will try to solve it.
Ref:
/assign
The PR kubernetes-sigs/azuredisk-csi-driver should fix the error. Let's observe this test case for a period of time. When it is stable, could close the issue.

Discussion:
// UnmountDevice unmounts the volume on the node func (d *azureDiskDetacher) UnmountDevice(deviceMountPath string) error { <ins> if runtime.GOOS == "windows" { // Flush data cache for windows because it does not do so automatically during unmount device exec := d.plugin.host.GetExec(d.plugin.GetPluginName()) err := util.WriteVolumeCache(deviceMountPath, exec) if err != nil { return err } } </ins> err := mount.CleanupMountPoint(deviceMountPath, d.plugin.host.GetMounter(d.plugin.GetPluginName()), false) if err == nil { klog.V(2).Infof("azureDisk - Device %s was unmounted", deviceMountPath)
Change:
<!-- Please only use this template for submitting enhancement requests --What would you like to be : There should be a way to override the default gcc compliers mentioned in the file with custom gcc compilers for different platforms. Also, cross compilation should be supported on platforms other than amd64. Why is this needed: Right now, there seems to be no way to override the platform specific cc flags that kubernetes sets. We should be able to override those values. For example, bottlerocket sets customized CC flags for different platforms using this . We should be able to override the default values in kubernetes using environment variables or build time flags and build kubernetes with customized gcc variants that bottlerocket uses. Also, we need an option to set gcc compilers for amd64 platform, so that amd64 targets can be built on different platforms.
/sig release
Sure, that will require cross building the build toolset itself, which nobody has taken up yet.

Discussion:
case "${platform}" in "linux/arm") export CGO_ENABLED=1 <del> export CC=arm-linux-gnueabihf-gcc </del> <ins> export CC=${KUBE_LINUX_ARM_CC:-arm-linux-gnueabihf-gcc} </ins> ;; "linux/arm64") export CGO_ENABLED=1 <del> export CC=aarch64-linux-gnu-gcc </del> <ins> export CC=${KUBE_LINUX_ARM64_CC:-aarch64-linux-gnu-gcc} </ins> ;; "linux/ppc64le") export CGO_ENABLED=1 <del> export CC=powerpc64le-linux-gnu-gcc </del> <ins> export CC=${KUBE_LINUX_PPC64LE_CC:-powerpc64le-linux-gnu-gcc} </ins> ;; "linux/s390x") export CGO_ENABLED=1 <del> export CC=s390x-linux-gnu-gcc </del> <ins> export CC=${KUBE_LINUX_S390X_CC:-s390x-linux-gnu-gcc} </ins> ;; esac fi
Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Kube-controller-manager panic when wrong Azure VMSS name is configured: This is because the error is not validated first in the following code path: Azure cloud provider should check whether err happens and whether vmss is nil here. What you expected to happen: Azure cloud provider should report errors on such case, so that customers could get the root raw error and then fix the configuration. How to reproduce it (as minimally and precisely as possible): Create Kuberenetes cluster with basic load balancer, configure a wrong VMSS name and then create/delete LoadBalancer typed services. Anything else we need to know?: The issue also happens on older versions, so the fixes need to cherry-pick on other branches. /area provider/azure /sig cloud-provider /milestone v1.20 Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/assign

Discussion:
for vmssName := range vmssNamesMap { vmss, err := ss.getVMSS(vmssName, azcache.CacheReadTypeDefault) <ins> if err != nil { return err } </ins> // When vmss is being deleted, CreateOrUpdate API would report "the vmss is being deleted" error. // Since it is being deleted, we shouldn't send more CreateOrUpdate requests for it.

klog.V(3).Infof("ensureVMSSInPool: found vmss %s being deleted, skipping", vmssName) continue } <del> if err != nil { return err } </del> if vmss.VirtualMachineProfile.NetworkProfile.NetworkInterfaceConfigurations == nil { klog.V(4).Infof("EnsureHostInPool: cannot obtain the primary network interface configuration, of vmss %s", vmssName) continue