jiebi/CodeConvo huggingface dataset:
Code change:
assign bool) (assigned mountDevice, alreadyAttached bool, err error) { deviceMappings := map[mountDevice]EBSVolumeID{} <ins> volumeStatus := map[EBSVolumeID]string{} // for better logging of volume status </ins> for _, blockDevice := range info.BlockDeviceMappings { name := aws.StringValue(blockDevice.DeviceName) if strings.HasPrefix(name, "/dev/sd") {

Related discussion:
text: So, is there another way how to check that a volume is really detached? Or should we promote the error flow described above to official fix and add bunch of comments around so we won't remove / tweak the flow in future? What bad can happen when we call AWS on a volume that's detaching? AWS at least once reported that the volume is detached, so the volume probably is not be stuck.
I think it is only bad if the volume was previously stuck. But feels acceptable for me if it is being detached and the previous call in at least confirmed once. Confirming internally. But I suspect there isn't any. As long as the volume is not stuck (unmounted properly), we should be able to attach a detaching volume especially when it is at least confirmed being detached once from call in . Even if the first time it doesn't work, retry should help recover the situation. How do you think?
According pointed, AWS recommends two actions to manage the eventual consistency model (k8s already does 1, but AFAIK not 2, except when there's an exponential backoff among calls: 1) Run the appropriate call before modifying a resource 2) Add wait time between subsequent commands, even if returns a green response (this does infer that might return stale data) Currently, k8s assumes a state based on a single green response, e.g., is called repeatedly with an exponential backoff until it returns a given volume state. What if it required at least two positive responses instead? What are the chances that would return stale data twice? I'd assume that two calls returning a detached volume (with exponential backoff in between, which enforces point 2 above) would be reliable enough. The downside is that this would increase the API usage, which is already high.
do you know if the native AWS calls may be more stale-data-proof?
Looking at implementation, it tries with constant backoff of 15 seconds (which I consider pretty large) and it returns when the first returns that the volume is in use, so basically the same as what Kubernetes does.
From what I saw, sometimes it happens that reports a volume as detached, subsequent reports that the volume is not attached, succeeds and only after that shows that the volume is (from the old device name) instead of . So Kubernetes got at least two confirmations that the volume is detached.
Code change:
assign bool) (assigned mountDevice, alreadyAttached bool, err error) { deviceMappings := map[mountDevice]EBSVolumeID{} <ins> volumeStatus := map[EBSVolumeID]string{} // for better logging of volume status </ins> for _, blockDevice := range info.BlockDeviceMappings { name := aws.StringValue(blockDevice.DeviceName) if strings.HasPrefix(name, "/dev/sd") {

Related discussion:
text: What would be the correct delay to wait for second (third, ...) confirmation that the volume is really detached? IMO it would hurt our performance a lot.
The Wait function simply calls DescribeXX under the hood and uses some hard coded timeout. So it's similar to what kubernetes does like said.
Code change:
if effectiveSc.RunAsUser != nil { if *effectiveSc.RunAsUser == 0 { <del> return fmt.Errorf("container's runAsUser breaks non-root policy") </del> <ins> return fmt.Errorf("container's runAsUser breaks non-root policy (pod: %q, container: %s)", format.Pod(pod), container.Name) </ins> } return nil } switch { case uid != nil && *uid == 0: <del> return fmt.Errorf("container has runAsNonRoot and image will run as root") </del> <ins> return fmt.Errorf("container has runAsNonRoot and image will run as root (pod: %q, container: %s)", format.Pod(pod), container.Name) </ins> case uid == nil && len(username) > 0: <del> return fmt.Errorf("container has runAsNonRoot and image has non-numeric user (%s), cannot verify user is non-root", username) </del> <ins> return fmt.Errorf("container has runAsNonRoot and image has non-numeric user (%s), cannot verify user is non-root (pod: %q, container: %s)", username, format.Pod(pod), container.Name) </ins> default: return nil }

Related discussion:
text: when deploying a container that has runAsNonRoot but runs as root this error appears, but for pods with lots of containers it can be confusing which container failed, especially if there are 3-rd party webhooks that inject containers please add the container name to the message, making it
/sig api-machinery
/assign I can try this fix
I just submitted a pr, can you review it for me, is it as you expected?
thx, looks good!
/remove-sig api-machiner /sig node /area kubelet
Those labels are not set on the issue: <detailsIn response to : /remove-sig api-machiner /sig node /area kubelet Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-sig api-machinery
Code change:
policies.  An API call must be allowed by some policy in order to proceed. The following implementations are available, and are selected by flag: <del> - `--authoriation_mode=AlwaysDeny` - `--authoriation_mode=AlwaysAllow` - `--authoriation_mode=ABAC` </del> <ins> - `--authorization_mode=AlwaysDeny` - `--authorization_mode=AlwaysAllow` - `--authorization_mode=ABAC` </ins> `AlwaysDeny` blocks all requests (used in tests). `AlwaysAllow` allows all requests; use if you don't need authorization.

Related discussion:
text: In CPU usage went up significantly compared to It consumes more than one core even when there's no significant logging load on the node. Two changes between those version: CC
Ruby by default is single-core, so anything over that is likely introduced by the changes. But we'd need to investigate to make sure.
Can you please disable monitoring and see whether this is the problem?
I meant can you ...
Was going to do this, yes
Built container manually without prometheus plugin, still consumes 1.7 cpu
Can you please find the difference then?
Used new image with the old configuration (), still 1.7 cores It seems like a problem with the new image for some reason. FYI
What can consume 1.7 cores with the same configuration on a different machine? How to debug this?
Does this mean all components are same, ruby version, plugins and configuration settings? What are the differences between old and new image? OS or other parts are also different?
There are at least the following changer Ubuntu Trusty -Debian Jessie td-agent package -manually installed embedded ruby -ruby 2.1 embedded gems -fixed version gems from Gemfile embedded libjemalloc - I understand that it's a lot of changes and I'm going to narrow the problem down, the main question is where to look and how to debug to speed this process up?
If this is not fixed before Monday we will revert . cc
Only and should be reverted, not
No, we want to revert migration to the new base image. The changes to extract configuration into env var/config map should preserve. This mean that the revert might not be straightforward.
I think our backup option is to switch back to using td-agent package, not reverting whole change, because is also about security and we don't want to revert that I will check whether that helps
Thanks for investigating this! I have a couple comments: The full package diff can be found here: The first thing I would look at is whether the ruby version makes a difference. I believe td-agent was using 2.3 The main benefit of is actually not using td-agent, since the td-agent package includes lots of stuff we don't use (e.g. rails). Maybe if we need to fallback we could manually remove those extra packages. I'll poke around in the container and see if anything jumps out.
Code change:
policies.  An API call must be allowed by some policy in order to proceed. The following implementations are available, and are selected by flag: <del> - `--authoriation_mode=AlwaysDeny` - `--authoriation_mode=AlwaysAllow` - `--authoriation_mode=ABAC` </del> <ins> - `--authorization_mode=AlwaysDeny` - `--authorization_mode=AlwaysAllow` - `--authorization_mode=ABAC` </ins> `AlwaysDeny` blocks all requests (used in tests). `AlwaysAllow` allows all requests; use if you don't need authorization.

Related discussion:
text: Thanks for the comments, they totally make sense. Also, thanks for the diff. Thanks!
Actually td-agent was using ruby 2.1:
A manual search through commenting out parts of the config file revealed that removing the systemd config solves the issue: I'll play around with the systemd libraries and see if I can figure out what's going on here.
Still investigating, but I think this may be a regression in the fluent-plugin-systemd library between version 0.0.5 (old image) and 0.0.7 (new image). I realized the diff I linked above was not in sync with the version that was actually pushed (). That diff is here, and accurately reflects the change in the systemd library:
Confirmed this is a regression in (0.0.6 does not have the problem). Fortunately 0.0.7 only has a single commit, I'll file an issue on the plugin. In the meantime, I'll send a PR to pin fluent-plugin-systemd to 0.0.6.
Thanks a huge lot for investigating and fixing this!
Reopening to track the version bump.
Good investigation. Latest systemd plugin uses and it seems to cause busy loop like problem.
Hi I just released that should fix this issue.
Thanks! And thanks to who already included it into our image :)
Yeah thanks your too fast for me :)
Thank you a lot for the investigation.
Code change:
patch := getDaemonSetImagePatch(ds.Spec.Template.Spec.Containers[0].Name, RedisImage) ds, err = c.AppsV1().DaemonSets(ns).Patch(dsName, types.StrategicMergePatchType, []byte(patch)) Expect(err).NotTo(HaveOccurred()) <del> templateGeneration++ </del> By("Check that daemon pods images are updated.") err = wait.PollImmediate(dsRetryPeriod, dsRetryTimeout, checkDaemonPodsImageAndAvailability(c, ds, RedisImage, 1)) Expect(err).NotTo(HaveOccurred()) <del> By(fmt.Sprintf("Make sure all daemon pods have correct template generation %d", templateGeneration)) err = checkDaemonPodsTemplateGeneration(c, ns, label, fmt.Sprint(templateGeneration)) Expect(err).NotTo(HaveOccurred()) </del> By("Check that daemon pods are still running on every node of the cluster.") err = wait.PollImmediate(dsRetryPeriod, dsRetryTimeout, checkRunningOnAllNodes(f, ds)) Expect(err).NotTo(HaveOccurred(), "error waiting for daemon pod to start")

Related discussion:
text: Please see : https://k8s- fails just after this line: Failure was introduced in: But this was not noticed until it started running in [Conformance] tests. You can see the same problem in other dashboards too: https://k8s- https://k8s-
cc /sig apps
/priority critical-urgent
/kind bug /priority failing-test
/milestone v1.10
Is this just an oversight of not removing the template generation when the API update was done?
/cc
triaged in progress
i have never touched this test before ... so i just threw in a WIP, feel free to use/discard it. there seems to be lot of methods in there that is no longer used, may be we can rip those out too. (see )
xref
[MILESTONENOTIFIER] Milestone Issue Needs Attention Action Required: This issue has not been updated since Mar 5. Please provide an update. Note: This issue is marked as , and must be updated every 1 day during code freeze. Example update: <details<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Never automatically move issue out of a release milestone; continually escalate to contributor and SIG through all available channels. : Fixes a bug discovered during the current release. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details
Code change:
f := framework.NewDefaultFramework("metrics-grabber") var c, ec clientset.Interface var grabber *e2emetrics.Grabber <ins> var masterRegistered bool </ins> ginkgo.BeforeEach(func() { var err error c = f.ClientSet ec = f.KubemarkExternalClusterClientSet <ins> // Check if master Node is registered nodes, err := c.CoreV1().Nodes().List(context.TODO(), metav1.ListOptions{}) </ins> framework.ExpectNoError(err) <ins> for _, node := range nodes.Items { if strings.HasSuffix(node.Name, "master") { masterRegistered = true } } </ins> gomega.Eventually(func() error { grabber, err = e2emetrics.NewMetricsGrabber(c, ec, true, true, true, true, true) if err != nil { return fmt.Errorf("failed to create metrics grabber: %v", err) } <del> if !grabber.HasControlPlanePods() { </del> <ins> if masterRegistered && !grabber.HasControlPlanePods() { </ins> return fmt.Errorf("unable to get find control plane pods") } return nil

Related discussion:
text: MetricsGrabber tests failing when scheduler & controller-manager not visible. In managed offerings, like GKE, scheduler and controller-manager are not visible causing BeforeSuite of MetricsGrabber to fail. Test is failing then control plane pods can't be found (as on GKE): HasControlPlanePods was in /assign
/sig instrumentation
Just removing this check allows tests to run without problems (some skipped, one passing).
/triage accepted
Code change:
proxyPort           int socket              proxySocket timeout             time.Duration <ins> activeClients       *clientCache </ins> nodePort            int loadBalancerStatus  api.LoadBalancerStatus sessionAffinityType api.ServiceAffinity

Related discussion:
text: Depends on UDP clients have a timeout (so we don't leak connections forever), but the test does not cover this. To do this, we probably need to track active connections, and plumb that all the way down to the goroutines spawned by the ProxyLoop routines. This is probably something we want to do anyway to be able to monitor.
Code change:
nt.removeZone(zone) } klog.V(5).Infof("Removed node %v in group %v from NodeTree", n.Name, zone) <del> nt.NumNodes-- </del> <ins> nt.numNodes-- </ins> return nil } }

Related discussion:
text: Description: , part of the is exporting a field which allows to modify its value externally. This also means that this field is not thread-safe. Solution: Expose a method that returns this value in a thread-safe way and do not export the field. /kind bug /sig scheduling
Code change:
<del> approvers: - luxas - timothysc - fabriziopandini reviewers: - luxas - timothysc - fabriziopandini labels: - kind/documentation </del>

Related discussion:
text: Instead, have a presubmit test that verifies that the docs can be generated. This will reduce the number of conflicts & rebases, therefore increase speed through the queue. Confirmed with that nothing depends on these docs at the moment. fyi, some manual process might have to change once this is in.
Yes, we want to generate the docs into , anyway. cc
Excellent. One closer step to YAML generation, I hope. :)
was interested in working on this.
Want to get started on this?
is this still a goal of ours?
I've seen a thread this week-end about that. I think docs have moved but we still need to generate them there.
can this issue be closed then?
related to ?
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. Prevent issues from auto-closing with an comment. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale /lifecycle frozen There's still cruft in the docs directory.
/remove-help I don't think this meets . Please add back via if I'm incorrect
/sig docs /sig release /remove-area test-infra /area build-release /area kubeadm /area kubectl All of the following dirs/files have the following content. Do we need them? This file is autogenerated, but we've stopped checking such files into the repository to reduce the need for rebases. Please run hack/generate- to populate this file. docs/admin: .md files for kube components and kubeadm commands man/man1: .1 files for same user-guide/kubectl: .md files for kubectl commands yaml/kubectl: .yaml files for kubectl commands
/help Now that we have /good-first-issue, I think this does meet the criteria for help wanted. Probably need to talk to someone from sig-docs and figure out if/how these are used.
This request has been marked as needing help from a contributor. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command.
Code change:
<del> approvers: - luxas - timothysc - fabriziopandini reviewers: - luxas - timothysc - fabriziopandini labels: - kind/documentation </del>

Related discussion:
text: <detailsIn response to : /help Now that we have /good-first-issue, I think this does meet the criteria for help wanted. Probably need to talk to someone from sig-docs and figure out if/how these are used. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/remove-priority important-soon /priority important-longterm This hasn't been committed to for any release cycle in a while. I hope someone does commit to it, but I personally cannot, so I'm bumping down to the "hopefully next time" priority. /kind cleanup in addition to docs This impacts merge velocity of any kubectl/kubeadm changes because the docs/OWNERS file is small. We're going to work around by adding OWNERS files for the relevant sigs to the relevant dirs. But if these files don't have to exist in the first place, that'd be even better.
as discovered today this would not work as OWNERS in sub-folders do not solve the problem with the /docs/.generated_docs file which is in the parent folder. i've been looking into a solution to remove the placeholders completely and make verify-docs only consist of kube::util::gen-docs. this will: ensure that godoc errors and such still block presubmits the contributors are not blocked having to commit placeholder files
/lifecycle active
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close. If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale /lifecycle active as PR is still in flight.
Code change:
import ( "fmt" <ins> appsv1 "k8s.io/api/apps/v1" </ins> appsv1beta2 "k8s.io/api/apps/v1beta2" extensionsv1beta1 "k8s.io/api/extensions/v1beta1" apiequality "k8s.io/apimachinery/pkg/api/equality"

Related discussion:
text: /priority failing-test /priority critical-urgent /sig apps This test case has been failing for a while and affects a number of jobs: This is affecting multiple jobs on the testgrid dashboard, and prevents us from cutting 1.9.0-alpha.2 (kubernetes/sig-release). Is there work ongoing to bring this job back to green?
All test failures look similar: . ~Is it due to network issue instead of DaemonSet?~
This seems related to the new DaemonSet. The connection refused error is happening because kube-apiserver is crashing:
needs an update
Ah, well... I guess it's halfway working as intended. We put that there to make the tests fail if we forgot to update it when adding a new version. We had hoped it would fail during presubmit tests though. :(
Sending a fix now
I wonder why the test is not failing on submit queue tests. Do i need to fix deployment and replicaset?
The rest of the group is not merged, and the v1 PR takes the selector mutability into account.
Since this bug crashes apiserver I'd expect it to affect many other e2e tests.
Noted, we need to fix deployment and replicaset when the v1 PR is merged.
Not quite sure how this test isn't failing before but is now consistently failing. I'd expect the commit that triggers this to block the submit queue. The e2e test is still accessing DaemonSet through extensions/v1beta1 endpoints. From the above apiserver log it's hard to tell what updates DaemonSet via apps/v1 before the code panics. I tried to reproduce it locally, and found that garbage collector PUT apps/v1 DaemonSet:
When apps/v1 gets introduced, it may be ordered before apps/beta1 alphabetically. IIRC, there is a code change on the apps/v1 PR that changes the order of API versions such that apps/v1 is ranked before apps/v1beta2 and apps/v1beta2 before apps/v1beta1. Perhaps this problem can be fixed when the PR merges.
Since crashes apiserver and makes debugging harder, which defeats the purpose of using it, we should return error on unsupported versions instead.
The first run with this change is still in progress, but this test already passed: https://k8s- Thanks for the quick fix
Code change:
}, { evictionPriority: 0, <del> pod: diskConsumingPod("guaranteed-disk", 299 /* Mb */, nil, v1.ResourceRequirements{ </del> <ins> // Only require 99% accuracy (297/300 Mb) because on some OS distributions, the file itself (excluding contents), consumes disk space. pod: diskConsumingPod("guaranteed-disk", 297 /* Mb */, nil, v1.ResourceRequirements{ </ins> Requests: v1.ResourceList{ v1.ResourceEphemeralStorage: resource.MustParse("300Mi"), },

Related discussion:
text: Is this a BUG REPORT or FEATURE REQUEST?: Bug Report /kind bug What happened: improved disk eviction testing to make it more precise. For example, it expects a pod which uses 299 Mb of disk not to be evicted if it sets its limit to 300 Mi. As shown on , this test passes for all OS distributions except CoreOS. I have ruled out an issue with the storage driver and backing filesystem since both COS and CoreOS use overlay backed by extfs. What you expected to happen: I expected the pod using 299Mb not to be evicted when its limit is 300 Mb. I expected not to see this event: How to reproduce it (as minimally and precisely as possible): The container runs this command to use 299Mb: Each iteration writes 1Mb to disk, producing a total of 299 Mb of disk usage. The pod has set to 300Mi. The test has the LocalStorageCapacityIsolation feature gate enabled. Anything else we need to know?: This may be a discrepancy in the (which we use to measure container disk usage) or , which is used by the test to write exact quantities of disk.
/sig node
cc can you take a look at this after thanksgiving?
I bumped the coreos image... going to see if it clears up the flake
It appears the root filesystem is being included in the disk space calculation . Do you think the coreos image might be slightly larger than the other images under test?
One more code snippet that is interesting:
thats the container's root filesystem
correct. the feature appears to add and compare to the quota: The reproducible steps do not take into account the rootfs space. If the quota is set to 300Mi, then the under loop count would have to take into account the rootfs space to not exceed the limit. I do not see anywhere in the tests or the reproduce steps this part of the calculation.
The refers only to the writable layer of the container, and excludes base (image) layers. With the provided command, the only files that are in the container's rootfs are the ones written by the command, and should amount to 299Mb. Additionally, the container does not write any logs (through stdout), so that value should be 0.