jiebi/CodeConvo huggingface dataset:
Code change:
) const ( <del> // When these values are updated, also update cmd/kubelet/app/options/options.go // A copy of these values exist in e2e/framework/util.go. </del> <ins> // When these values are updated, also update cmd/kubelet/app/options/container_runtime.go // A copy of these values exist in test/utils/image/manifest.go </ins> currentPodInfraContainerImageName    = "gcr.io/google_containers/pause" <del> currentPodInfraContainerImageVersion = "3.0" </del> <ins> currentPodInfraContainerImageVersion = "3.1" </ins> ) // GetServerArchitecture fetches the architecture of the cluster's apiserver.

Related discussion:
text: Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: pause does not reap zombie processes using the default pause container provided by What you expected to happen: should include the up to date pause code to reap zombie processes when using docker = 1.13.1 found here How to reproduce it (as minimally and precisely as possible): a.out will leave a zombie process that is not cleaned up Anything else we need to know?: Building the pause container from master solves this issue Related: Environment: Kubernetes version (use ): 1.7.0+ Cloud provider or hardware configuration**: N/A OS (e.g. from /etc/os-release): CentOS 7 Kernel (e.g. ): 4.10.13+ Install tools: Others: docker = 1.13.1
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
i get a simliar problem when i run crond service in pod container, every day it makes some [anacron] zombie progresses UID PID PPID C STIME TTY TIME CMD root 1 0 0 Nov14 ? 00:00:00 /pause root 7 0 0 Nov14 ? 00:00:00 /usr/local/bin/dumb-init /usr/lo root 13 7 0 Nov14 ? 00:00:00 nginx: master process /usr/local nobody 21 13 0 Nov14 ? 00:00:01 nginx: worker process nobody 22 13 0 Nov14 ? 00:00:07 nginx: worker process nobody 23 13 0 Nov14 ? 00:00:07 nginx: worker process nobody 24 13 0 Nov14 ? 00:00:05 nginx: worker process root 77 1 0 Nov14 ? 00:00:00 [anacron] root 1 0 00:01 ? 00:00:00 [anacron] <defunctroot 1 0 01:01 ? 00:00:00 [anacron] <defunctroot 1 0 02:01 ?
Code change:
) const ( <del> // When these values are updated, also update cmd/kubelet/app/options/options.go // A copy of these values exist in e2e/framework/util.go. </del> <ins> // When these values are updated, also update cmd/kubelet/app/options/container_runtime.go // A copy of these values exist in test/utils/image/manifest.go </ins> currentPodInfraContainerImageName    = "gcr.io/google_containers/pause" <del> currentPodInfraContainerImageVersion = "3.0" </del> <ins> currentPodInfraContainerImageVersion = "3.1" </ins> ) // GetServerArchitecture fetches the architecture of the cluster's apiserver.

Related discussion:
text: 00:00:00 [anacron] <defunctroot 1 0 03:01 ? 00:00:00 [anacron] <defunctroot 0 0 11:49 ? 00:00:00 /bin/bash root 0 1 19:15 ? 00:00:00 /bin/bash root 0 19:15 ? 00:00:00 ps -ef !
Do we know why this was never resolved / was never deployed?
I just fought through this issue with a rabbitmq helm chart. I had to custom build a pause container from master to fix it. The rabbitmq helm chart uses for its liveness checks. It's a thin wrapper around a call to the rabbitmq-server daemon, which is not PID 1 (/pause is PID 1). The rabbitmq-server process forks some children to do the work and then they exit. Since the pause-amd64:3.0 is not reaping, those processes stick around as zombies. The liveness check runs every 5 seconds. After about 4 hours, there are about 27K zombies and the node begins failing to launch new containers because it's starved for resources. The pause container built from current master fixes that behavior, no zombies at all in the container.
The plan is to build a new version of pause-amd64 and then update the image referenced by Kubernetes. This means the default image will change in the next Kubernetes release. We can make the new image available once the freeze lifts and we can merge .
Looks like this made it into 1.10
Code change:
} idx = len(ary) - idx } <ins> if idx < 0 || idx >= len(ary) || idx > len(cur) { return fmt.Errorf("Unable to access invalid index: %d", idx) } </ins> copy(ary[0:idx], cur[0:idx]) ary[idx] = val copy(ary[idx+1:], cur[idx:])

Related discussion:
text: /kind bug What happened: Wrote a that returned a faulty json patch which specified an our of range index (e.G. when there is only one container). This resulted in the apiserver crashing. Stacktrace: What you expected to happen: No crash /sig-api-machinery
/sig api-machinery
This maybe a bug in . I will follow this question.
This bug had been fixed in PR: you can checkout it.
Thanks for your comment but I don't think the linked PR fixed the issue, as it got merged almost a year ago and I am running Kube 1.10.1 which does contain that patch but still panics. Just from quickly looking at the code I the linked PR I believe there are two things wrong: The check if the index is valid only happens when a negative index was given should be
Once the PR of is merged I will update kubernetes.
/kind bug
Code change:
return m.wrappedMounter.ExistsPath(pathname) } <ins> func (m *execMounter) EvalHostSymlinks(pathname string) (string, error) { return m.wrappedMounter.EvalHostSymlinks(pathname) } </ins> func (m *execMounter) PrepareSafeSubpath(subPath Subpath) (newHostPath string, cleanupAction func(), err error) { return m.wrappedMounter.PrepareSafeSubpath(subPath) }

Related discussion:
text: /kind bug Using , if a block device is symlinked to the discovery directory e.g. , then MapVolume will fail with since the containerized kubelet cannot see the path on host. is mounted by containerized kubelet, so if before we try to map the device we set devicePath to the evaluation of symlink and end up with a devicePath of , containerized kubelet will see the device and successfully map the device. Are there any issues with this solution? This is on Kubernetes 1.10
/sig storage
ah yes containerized kubelet has many issues when dealing with symlinks. Check out Mounter.GetMode(), it calls a nsenter EvalSymlinks which should be able to do what you need. But actually, one thing to consider, is do you need to do the losetup on the host, or does inside the container work?
Yes it appears to work from inside the container, on the host I run which disappears after I delete the pod using the device. My understanding of running losetup in MapVolume is not that great but I suppose that as long as that /dev/loop0 is created we are fine. Thanks for the quick response, I'll create a PR using that function and tag you
Does it create it on the host or in the containerized kubelet? What happens when containerized kubelet is restarted? The purpose of losetup is to do "refcounting" on the block device to prevent possible naming collisions when devices are attached/detached under unhealthy conditions. Ie, if a device showed up as /dev/sdb and you gave that to a pod, and then for some reason it become unhealthy and disappeared (while the pod is still running), another device (maybe even another volume type) could get attached to the node and reuse /dev/sdb. Now the pod has access to a device it's not supposed to. With filesystem volumes, the refcounting is done with mount points, but for block device, loopback is the equivalent.
It is created on the host (and also visible in the container), and stays around through restart. I understand now, thank you.
Actually after I restart it becomes /vdb instead of /dev/vdb in the losetup output, idk what that is about.
Code change:
return m.wrappedMounter.ExistsPath(pathname) } <ins> func (m *execMounter) EvalHostSymlinks(pathname string) (string, error) { return m.wrappedMounter.EvalHostSymlinks(pathname) } </ins> func (m *execMounter) PrepareSafeSubpath(subPath Subpath) (newHostPath string, cleanupAction func(), err error) { return m.wrappedMounter.PrepareSafeSubpath(subPath) }

Related discussion:
text: There is no such thing as /vdb and /dev/vdb is still intact of course. Seems inconsequential since it is disappearing after all pods using the device are deleted, i.e. it's still serving as a refcounter
Code change:
cp kubernetes/server/kubernetes/server/bin/kubelet  kubernetes/server/kubernetes/server/bin/kube-proxy binaries/minion <ins> cp kubernetes/server/kubernetes/server/bin/kubectl binaries/ </ins> rm -rf flannel* kubernetes* etcd* echo "Done! All your commands locate in ./binaries dir"

Related discussion:
text: Though I have successfully installed kubernetes via :smiley: , still want to point out this doc missing the step to install . Thus, if user just follow the guide in the doc without having much knowledge about kubernetes, he/she would get error "It looks as if you don't have a compiled kubectl binary" during the step of . Manually install kubectl on at least master node should be necessary (suggest to let user also install on minion node for convenience), any way, the installation step is .
I forget to mention that.I will update the doc this tuned
I update the scripts in pr , you can try it now. I copied the kubectl binary from the kubernetes release and put it in , so there is no extra step for user to manually download . Actually, in the first version of ubuntu k8s deployment, I copied the to every node, but I think it is an overhead, so I removed it.I think this is just enough.
Doc update : Move the kubectl into the PATH, it may need sudo privilege so I decide not do for the user.
Have the same issue, please update the doc ASAP.
Great job :+1:
Also ran into this bug after recklessly running ( instead of ) :)
Code change:
targetNamePath := sessionPath + "/iscsi_session/" + sessionName + "/targetname" targetName, err := io.ReadFile(targetNamePath) if err != nil { <del> return nil, err </del> <ins> klog.Infof("Failed to process session %s, assuming this session is unavailable: %s", sessionName, err) continue </ins> } // Ignore hosts that don't matchthe target we were looking for.

Related discussion:
text: If one target in multipath volume gets down and a pod that uses the volume is deleted and quickly recreated (e.g. by a Deployment), the new pod never starts. How to reproduce it (as minimally and precisely as possible): Deployment with single replica with one multipath iSCSI volume one target (e.g. switch the machine off). Multipath will mark appropriate path as failed: the deployment pod. will quickly start a new pod for the Deployment. new pod will be (or ) forever, What happened: Kubelet tries to WaitForAttach / SetUp volume for the new pod. The volume is still attached. Kubelet goes through all targets and tries to find out if it's logged in or not. Eventually it tries to read of the failed path and that returns error: What you expected to happen: Kubelet assumes that the path is failed / not logged in and tries to recover it. If it fails it continues with multipath with 2 available devices and 1 failed one. IMO, simple instead of will do its job just fine. /assign /sig storage cc for iSCSI expertise
Fix:
Code change:
framework.ExpectNoError(err, "failed to count the required APIServices") framework.Logf("APIService %s has been deleted.", apiServiceName) <del> ginkgo.By("Confirm that the group path of " + apiServiceName + " was removed from root paths") groupPath := "/apis/" + apiServiceGroupName err = wait.PollUntilContextTimeout(ctx, apiServiceRetryPeriod, apiServiceRetryTimeout, true, func(ctx context.Context) (done bool, err error) { rootPaths := metav1.RootPaths{} statusContent, err = restClient.Get(). AbsPath("/"). SetHeader("Accept", "application/json").DoRaw(ctx) if err != nil { return false, err } err = json.Unmarshal(statusContent, &rootPaths) if err != nil { return false, err } return !slices.Contains(rootPaths.Paths, groupPath), nil }) framework.ExpectNoError(err, "Expected to not find %s from root paths", groupPath) </del> cleanupSampleAPIServer(ctx, client, aggrclient, n, apiServiceName) }

Related discussion:
text: reported that conformance testing with the latest version of sonobuoy against a version 1.29.0-alpha.3 cluster fails on the support the 1.17 Sample API Server using the current Aggregator test: , related to I found the reason why the test is working fine in Kubernetes CI jobs but fails with sonobuoy is because the service account used by sonobuoy "sonobuoy-serviceaccount" doesn't have read permission of the non-resource url "/", which is used by the test to confirm the removal of group path: The permission of "sonobuoy-serviceaccount" is as below: There may be two ways to fix it: the RBAC rule of sonobuoy to include "get" of "/". However, could other test runners encounter this issue? Is there a doc listing the required permissions for running e2e tests? the e2e to skip checking the removal of the group path if the service account doesn't have permission to access the root path. Which way do we usually prefer? Running e2e with sonobuoy should succeed. a v1.29.0-alpha.3 cluster. No response <details</details<details</details<details</details<details</details<details</details<details</details
implemented this solution.
/sig api-machinery
thank you for opening this issue and working on a solution. I'm probably not the right person to decide on which solution is preferred. From my perspective, either solution is fine.
Note: Sonobuoy is not owned by any Kubernetes SIG. At most it is between SIG Architecture and the CNCF to choose whether to recommend it for running tests. SIG Testing owns the e2e image that sonobuoy can be used to schedule but not sonobuoy. IMHO this is a bug in running the tests via a pod, which is not how we develop the tests (they're run outside the cluster, by a cluster admin account typically). I would say 1) is the correct answer. xref: /sig architecture /sig testing /triage accepted cc
thanks for your comment. I guess 1) means non of the existing sonobuoy versions can pass K8s conformance test against a cluster 1.29.0-alpha3, as the RBAC rule is hardcoded.
Code change:
framework.ExpectNoError(err, "failed to count the required APIServices") framework.Logf("APIService %s has been deleted.", apiServiceName) <del> ginkgo.By("Confirm that the group path of " + apiServiceName + " was removed from root paths") groupPath := "/apis/" + apiServiceGroupName err = wait.PollUntilContextTimeout(ctx, apiServiceRetryPeriod, apiServiceRetryTimeout, true, func(ctx context.Context) (done bool, err error) { rootPaths := metav1.RootPaths{} statusContent, err = restClient.Get(). AbsPath("/"). SetHeader("Accept", "application/json").DoRaw(ctx) if err != nil { return false, err } err = json.Unmarshal(statusContent, &rootPaths) if err != nil { return false, err } return !slices.Contains(rootPaths.Paths, groupPath), nil }) framework.ExpectNoError(err, "Expected to not find %s from root paths", groupPath) </del> cleanupSampleAPIServer(ctx, client, aggrclient, n, apiServiceName) }

Related discussion:
text: I will file an issue to sonobuoy first.
question here is what are the set of client permissions required to run the conformance suite? perhaps a level of access or a more granular recommendation should be documented. what is your take on the root path requirement as approver for: EDIT: slack thread:
I agree, we already require e.g. privileged pods for a long time in conformance tests and we should make it clear that we expect a highly privileged client.
+1 for what is saying, that we should expect that the user running the tests has required permissions. These tests are not built for being run unprivileged.
i wanted to PR the admin note in our conformance docs under sig-arch, but then i saw this: so broke this guarantee. sounds like SIG Arch needs to comment.
This statement surprises me. Root discovery is one of the least-privileged permissions you can have on a cluster. For years this was even open for unauthorized users on the internet (if your api endpoint has been exposed). In other words, just add the permission. While thinking about it, it is surprising that the user in question does not have this access already. Don't we allow that for every authorized user?
In contrast, I hope that lots of privileged operations are tested by conformance today. If we don't test things like admission webhooks for example, we have much bigger problems. And admission webhooks are basically equivalent to cluster-admin.
that is outdated, we use hostNetwork pods one some of the tests IIRC
looks like we don't. before this ticket, i was under the impression that the conformance suite already requires admin and that all testers (like Sonobuoy) use admin level credentials. it is non-privileged (e.g., does not require root on nodes, access to raw network interfaces, or cluster admin permissions) looks like we may have to change that sentence completely.
Would have expected it here: The PR in question explicitly fixes the result returned by . If that is part of conformance (IMO it should), every conformance test needs access it obviously. Whether we require admin, or we add it manually, either would work.
Code change:
framework.ExpectNoError(err, "failed to count the required APIServices") framework.Logf("APIService %s has been deleted.", apiServiceName) <del> ginkgo.By("Confirm that the group path of " + apiServiceName + " was removed from root paths") groupPath := "/apis/" + apiServiceGroupName err = wait.PollUntilContextTimeout(ctx, apiServiceRetryPeriod, apiServiceRetryTimeout, true, func(ctx context.Context) (done bool, err error) { rootPaths := metav1.RootPaths{} statusContent, err = restClient.Get(). AbsPath("/"). SetHeader("Accept", "application/json").DoRaw(ctx) if err != nil { return false, err } err = json.Unmarshal(statusContent, &rootPaths) if err != nil { return false, err } return !slices.Contains(rootPaths.Paths, groupPath), nil }) framework.ExpectNoError(err, "Expected to not find %s from root paths", groupPath) </del> cleanupSampleAPIServer(ctx, client, aggrclient, n, apiServiceName) }

Related discussion:
text: adding to milestone and making release blocking ... we should not release 1.29 in a state that the most commonly used conformance tool will fail
IMO we should clarify the state and go fix the tool, but we should keep the test and we should NOT add skip. Conformance docs are not a release artifact, and sonobuoy release lifecycle is not a conformance artifact. If we can't get sonobuoy working in a reasonable time, we already know the tests can be run by scheduling the image (which is ours) directly with kubectl instead and we can document that in the upstream conformance docs. We need to fix the for this image: ,%20master%20(dev) And it should probably be to release-blocking in the future.
it looks like the e2e change made in was not consciously made part of conformance, it was just trying to functionally test the bugfix being made. For 1.29, I think we should revert the change to and make an integration test that validates the fix instead. We can revisit whether content of is part of conformance or not for 1.30
I opened to revert the e2e change and move the functional test to an integration test
Filed to try to get the image job passing again.
We probably need another issue to track 1.30 follow-up?
If no one has done it, I could create another issue and follow up with the items that have been discussed here: outdated privilege requirement in commonly used conformance tools like sonobuoy aware of the required priviledge to avoid similar situations in the future. permission of GET "/" to authorized users via the role for root discovery.
Code change:
return true, errors.New("not implemented") } <ins> func (*NsenterMounter) EvalHostSymlinks(pathname string) (string, error) { return "", errors.New("not implemented") } </ins> func (*NsenterMounter) SafeMakeDir(pathname string, base string, perm os.FileMode) error { return nil }

Related discussion:
text: /kind bug Using , if a block device is symlinked to the discovery directory e.g. , then MapVolume will fail with since the containerized kubelet cannot see the path on host. is mounted by containerized kubelet, so if before we try to map the device we set devicePath to the evaluation of symlink and end up with a devicePath of , containerized kubelet will see the device and successfully map the device. Are there any issues with this solution? This is on Kubernetes 1.10
/sig storage
ah yes containerized kubelet has many issues when dealing with symlinks. Check out Mounter.GetMode(), it calls a nsenter EvalSymlinks which should be able to do what you need. But actually, one thing to consider, is do you need to do the losetup on the host, or does inside the container work?
Yes it appears to work from inside the container, on the host I run which disappears after I delete the pod using the device. My understanding of running losetup in MapVolume is not that great but I suppose that as long as that /dev/loop0 is created we are fine. Thanks for the quick response, I'll create a PR using that function and tag you
Does it create it on the host or in the containerized kubelet? What happens when containerized kubelet is restarted? The purpose of losetup is to do "refcounting" on the block device to prevent possible naming collisions when devices are attached/detached under unhealthy conditions. Ie, if a device showed up as /dev/sdb and you gave that to a pod, and then for some reason it become unhealthy and disappeared (while the pod is still running), another device (maybe even another volume type) could get attached to the node and reuse /dev/sdb. Now the pod has access to a device it's not supposed to. With filesystem volumes, the refcounting is done with mount points, but for block device, loopback is the equivalent.
It is created on the host (and also visible in the container), and stays around through restart. I understand now, thank you.
Actually after I restart it becomes /vdb instead of /dev/vdb in the losetup output, idk what that is about.
Code change:
return true, errors.New("not implemented") } <ins> func (*NsenterMounter) EvalHostSymlinks(pathname string) (string, error) { return "", errors.New("not implemented") } </ins> func (*NsenterMounter) SafeMakeDir(pathname string, base string, perm os.FileMode) error { return nil }

Related discussion:
text: There is no such thing as /vdb and /dev/vdb is still intact of course. Seems inconsequential since it is disappearing after all pods using the device are deleted, i.e. it's still serving as a refcounter
Code change:
set -o pipefail ### Hardcoded constants <del> DEFAULT_CNI_VERSION='v0.8.7' DEFAULT_CNI_HASH='8f2cbee3b5f94d59f919054dccfe99a8e3db5473b553d91da8af4763e811138533e05df4dbeab16b3f774852b4184a7994968f5e036a3f531ad1ac4620d10ede' </del> <ins> DEFAULT_CNI_VERSION='v0.9.1' DEFAULT_CNI_HASH='b5a59660053a5f1a33b5dd5624d9ed61864482d9dc8e5b79c9b3afc3d6f62c9830e1c30f9ccba6ee76f5fb1ff0504e58984420cc0680b26cb643f1cb07afbd1c' </ins> DEFAULT_NPD_VERSION='v0.8.7' DEFAULT_NPD_HASH='853576423077bf72e7bd8e96cd782cf272f7391379f8121650c1448531c0d3a0991dfbd0784a1157423976026806ceb14ca8fb35bac1249127dbf00af45b7eea' DEFAULT_CRICTL_VERSION='v1.21.0'

Related discussion:
text: The latest release of CNI plugins is , but we still have . I'll attempt to create a PR for this based on the .
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network
- assigning to you :)