jiebi/CodeConvo huggingface dataset:
Code change:
// Check that container has restarted ginkgo.By("Waiting for container to restart") restarts := int32(0) <del> err = wait.PollImmediate(10*time.Second, 2*time.Minute, func() (bool, error) { </del> <ins> err = wait.PollImmediate(10*time.Second, framework.PodStartTimeout, func() (bool, error) { </ins> pod, err := f.ClientSet.CoreV1().Pods(f.Namespace.Name).Get(context.TODO(), pod.Name, metav1.GetOptions{}) if err != nil { return false, err

Related discussion:
text: So if the container keeps restarting, the liveness probe could've been failing continuously within 2min, due to one of the following: (? not sure about this one) Command can't be executed (for example if container can't start) configmap file was never updated. Within 2min, the pod only restarted 5 times. The liveness probe's period is 2s, initial delay is 1s, and failure threshold is 1, so a restart due to liveness probe failure is triggered every 3s. The image is busybox. The command is just , so it's unlikely for the container to have crashed. There were no liveness probe unhealthy events after the test fixes the probe, and yet it continued to restart. No more events after probe is fixed.
Could we check if the configmap value was updated for sure to rule out that scenario ? I have not tried to run an end to end test locally :) so am not sure how to debug it
Will do! I ran the test in a GKE Ubuntu cluster running 1.19.6 and couldn't reproduce the failure. I'm now trying to run a cluster on GCE with Ubuntu nodes running k8s master, but haven't been successful after trying a number of techniques. If anyone happens to know how to do this let me know; I'll keep asking around otherwise.
I got a repro on a GKE cluster backed by COS. After the test failed the container ends up in Running state eventually, and the configmap file outputs the expected modified value. The fix is to increase the test timeout. PR coming up.
Code change:
local params="${API_SERVER_TEST_LOG_LEVEL:-"--v=2"} ${APISERVER_TEST_ARGS:-} ${CLOUD_CONFIG_OPT}" params+=" --address=127.0.0.1" params+=" --allow-privileged=true" <del> params+=" --authorization-policy-file=/etc/srv/kubernetes/abac-authz-policy.jsonl" </del> params+=" --cloud-provider=gce" params+=" --client-ca-file=/etc/srv/kubernetes/ca.crt" params+=" --etcd-servers=http://127.0.0.1:2379"

Related discussion:
text: Typically, the dns and dashboard is in crash loop. Ref cc
if you check the file, does it contain lines like ?
Previous comment is ref: from I'll copy that to container-linux as well.
Code change:
go_rules_dependencies() go_register_toolchains( <del> go_version = "1.8.3", </del> <ins> go_version = "1.9.1", </ins> ) docker_repositories()

Related discussion:
text: Is this a BUG REPORT or FEATURE REQUEST?: /kind feature This is a followup on , but for Go 1.9 We should do the same kind of testing now to possibly catch bugs in either Go or Kubernetes when doing this kind of dependency version bump. Ideally we'd have automated tests against Go tip (), but we don't have that at the moment, so we have to do it kind of manually. We should also decide which k8s version we should target. k8s v1.8 or v1.9? What Go versions have been used so far? Kubernetes v1.7.0 - go1.8.3 Kubernetes v1.6.0 - go1.7.5 Kubernetes v1.5.0 - go1.7.4 Kubernetes v1.4.0 - go1.6.3 Kubernetes v1.3.0 - go1.6.2 Kubernetes v1.2.0 - go1.4.2 Seems like the trend has been to stay on the same Go version for two releases and then bump. That'd mean we'd switch to Go1.9 in k8s v1.9 Everyone happy with that? Or is there any reason for us to go start using go1.9 in k8s v1.8? In any case, we should indeed start testing go1.9 out now... cc
Let me know if you find any bugs
FWIW pretty tight to code-freeze I would prefer punting out a cycle fwiw.
I don't think we're that close to code freeze given that we have 5 (of 8) weeks left in this coding cycle But I'm happy to defer this to v1.9 if others think that. I guess the main reason for a bump would be scalability goals, but I'm not sure we have the need to expand above 5000 (and that Go would be the bottleneck in that case) for v1.8
made the Go benchmarks 9% faster with 15% larger binaries. I'm interested to see how that affects our numbers.
mid-stack inlining was not included in Go 1.9. It made the compiler too slow. It's being targeted to Go 1.10 now instead, after some compiler optimization work.
Oh, thanks for the correction!
Code change:
go_rules_dependencies() go_register_toolchains( <del> go_version = "1.8.3", </del> <ins> go_version = "1.9.1", </ins> ) docker_repositories()

Related discussion:
text: Regardless, we need to run the scalability tests against the RC so we can report issues earlier-- the last upgrade was particularly painful because of an HTTP regression that we caught way too late.
yeah. I filed for that now
yeah, my recollection is that the Go releases usually occur too late in our release cadence and end up getting bumped to the next Kubernetes release. (Also we inevitably find something wrong, though testing RCs would help with that. Though I'm not sure anyone has cycles to test RCs and fix all of the issues that do come up.)
Based on our current time frame, it looks like go1.9 support will have to slip to k8s 1.9. is open to dig into and start things testing, but there's a number of things that have to move.. I'll try to document those this time around to help with future go version bumps. So far: bazel build rules needed to be updated to support go1.9 () bazel version possibly needs to be updated? some of our custom bazel rules broke
When done, we should update this:
/priority important-soon
Updating the dependency is blocked pending bazel 0.6.0 - .
Can we upgrade to 0.6.0 now? Does bazel 0.6.0 have multiarch support btw?
We can't update to depend on bazel 0.6.0+ until we can figure out how to support different versions of the bazelbuild and kubekins-e2e images in PR jobs in prow based on what branch is being tested. cc
Noting that we're working on branching jobs, is testing one option here:
[MILESTONENOTIFIER] Milestone Issue Current <details open<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Escalate to the issue owners and SIG owner; move out of milestone after several unsuccessful escalation attempts. : New functionality.
Code change:
go_rules_dependencies() go_register_toolchains( <del> go_version = "1.8.3", </del> <ins> go_version = "1.9.1", </ins> ) docker_repositories()

Related discussion:
text: </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details
I approved this for v1.9, this is something we definitely want in order to keep up with Go (and for improved performance)
we're starting to use the forked images now /cc this should be more feasible very soon Edit: test-infra jobs have also been using bazel 0.6 (now 0.6.1) / a recent rules_go to soak things a bit
We've identified a failing test as a result of the WIP PR to upgrade the codebase to go1.9: The admission webhook test fails with certificate errors under go1.9.1, but runs successfully under go1.8.3. cc: who wanted to be alerted to potential bugs
- please see for a possible fix
Code change:
metadataVMName: "node2", expected:       false, }, <del> { nodeName:       "vmss000001", metadataVMName: "vmss_1", expected:       true, }, { nodeName:       "vmss_2", metadataVMName: "vmss000000", expected:       false, }, { nodeName:       "vmss123456", metadataVMName: "vmss_$123", expected:       false, expectedErrMsg: fmt.Errorf("failed to parse VMSS instanceID %q: strconv.ParseInt: parsing %q: invalid syntax", "$123", "$123"), }, </del> } for _, test := range testcases {

Related discussion:
text: <!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: If computer name prefix does not match VMSS name, then Azure cloud provider will use VMSS API instead of using metadata service data. This leads to throttling, because every affected kubelet is making API requests. What you expected to happen: Azure cloud provider should use metadata service data instead of calling VMSS API when instance is current. How to reproduce it (as minimally and precisely as possible): Set computer name prefix of the VMSS to something different than default. You should observe Azure provider making requests to the VMSS API rather than using metadata service, which leads to throttling if you have multiple kubelets with this bug. Anything else we need to know?: This is bug was introduced by - returns false even if instance is current, because it expects node name to match VMSS name + instance id, however node name is derived from the computer-name-prefix which can be different than VMSS name. Node name should be matched against , not against . Environment: Kubernetes version (use ): 1.18.x and higher Cloud provider or hardware configuration: azure /sig cloud-provider
/sig cloud-provider
/area provider/azure
CC
/remove-sig cloud-provider /sig cloud-provider (trying to remove the label)
/sig cloud-provider
Thanks for the reporting. Unfortunately, computer name prefix is not included in IMDS, so it can't be used without VMSS API for constructing the NodeName. Would check what could we do to workaround the issue.
Can you memorize the computer name prefix in the first request to the Azure API and then use it for the isCurrentInstance check?
opened to fix the issue. Would you like to have a look?
I can confirm that your change should fix the problem I reported. Do you know if your fix will be to 1.18.x?
is working on the cherry pick
Thank you for quick fix!
Code change:
} redacted := string(redactedBytes) <ins> dataOmitted := string(dataOmittedBytes) </ins> if len(mutatingConfig.Clusters) != 2 { t.Errorf("unexpected clusters: %v", mutatingConfig.Clusters) } if !reflect.DeepEqual(startingConfig.Clusters[unchangingCluster], mutatingConfig.Clusters[unchangingCluster]) { t.Errorf("expected %v, got %v", startingConfig.Clusters[unchangingCluster], mutatingConfig.Clusters[unchangingCluster]) } <del> if string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData) != redacted { t.Errorf("expected %v, got %v", redacted, string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData)) </del> <ins> if string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData) != dataOmitted { t.Errorf("expected %v, got %v", dataOmitted, string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData)) </ins> } if len(mutatingConfig.AuthInfos) != 2 {

Related discussion:
text: Is this a BUG REPORT or FEATURE REQUEST?: What happened: shows: The use of "REDACTED" for a CA here is misleading, since it's a public certificate that is not a secret. What you expected to happen: I expected to see the full CA in pem format (like when you cat ) or something like "SUPPRESSED" or "TRUNCATED" How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Nope Environment: Kubernetes version (use ):Cloud provider or hardware configuration:OS (e.g. from /etc/os-release):Kernel (e.g. ):
(for thoughts)
Reiterating the mentions to trigger a notification: <detailsIn response to : (for thoughts) Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
use you can see the complete certificate authority data
Hey Thank you - I have tracking that . The point of this issue is that at best it's a bad UX; at worst it's a miseducation on pki. "REDACTED" isn't the right word here
If there are multiple clusters displaying certificates, the configuration will be too long, so this switch is turned off by default. But we can add an example to help use. thanks
Hey I don't personally view truncation the responsibility of the CLI... users can pipe things to a pager if they really care. That being said, I'm not concerned about the length. The issue is purely about the word choice of "REDACTED". The term means: The use of the term "REDACTED" in the output implies that a CA public key is remove for security when (as you've pointed out) it is removed for truncation. Truncation and redaction are fundamentally different things, and k8s is misusing the word redacted for this use case.
Not quite a bug, but I think you're right a better term could be used there as it is a public key, not a private key. /remove-kind bug
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close.
Code change:
} redacted := string(redactedBytes) <ins> dataOmitted := string(dataOmittedBytes) </ins> if len(mutatingConfig.Clusters) != 2 { t.Errorf("unexpected clusters: %v", mutatingConfig.Clusters) } if !reflect.DeepEqual(startingConfig.Clusters[unchangingCluster], mutatingConfig.Clusters[unchangingCluster]) { t.Errorf("expected %v, got %v", startingConfig.Clusters[unchangingCluster], mutatingConfig.Clusters[unchangingCluster]) } <del> if string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData) != redacted { t.Errorf("expected %v, got %v", redacted, string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData)) </del> <ins> if string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData) != dataOmitted { t.Errorf("expected %v, got %v", dataOmitted, string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData)) </ins> } if len(mutatingConfig.AuthInfos) != 2 {

Related discussion:
text: If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale I'm not sure what I need to do here. This was marked as a feature. Do you need something more from me?
We are waiting for someone to fix this.
/remove-good-first-issue I don't think this fits our definition of a :
So I have to keep replying to the comment bot every 30 days so it doesn't get auto-closed? That feels... odd
thanks for the education :) unfortunately other issues are higher priority and no one has stepped up to fix this. The autocloser is a form of bug bankruptcy that keeps our issue load in check. The contribex special interest group is definitely open to feedback on the bot.
The biggest hurdle in this is that Kubernetes is using some base64 trickery to display the string (basically setting the certificate data to the decoded bytes from ). We can change this to or but not (it has to be a valid base64 encoded string). I tried exploring multiple options for the text, but I couldn't find a suitable one. Given that we already print the user tokens and other credentials this is a bit weird. Should we redact those too? Is it okay if we print the CA certificate instead of truncating it?
I can't help but laugh at: I'm not wed to any particular name. I also have no problem displaying the entire certificate. The crux of the issue is that a CA public key is not secret or sensitive. It's a public key
I agree. The issue is with the constraints imposed by how this is implemented currently. I tried to come up with a suitable string (given the constraint) and couldn't find anything acceptable... The only suitable option that worked for me is . I personally prefer to print the CA certificate since we are already printing user tokens and client certs that are actual secrets.
I agree. Who do we need to get in agreement that printing the full CA is the best path forward?
the data is not omitted for security purposes, but to make the output more readable. see REDACTED is misleading... it should probably be "elided" or "..."
you can run to view full content
that's the crux of this issue - REDACTED is misleading.
Code change:
} redacted := string(redactedBytes) <ins> dataOmitted := string(dataOmittedBytes) </ins> if len(mutatingConfig.Clusters) != 2 { t.Errorf("unexpected clusters: %v", mutatingConfig.Clusters) } if !reflect.DeepEqual(startingConfig.Clusters[unchangingCluster], mutatingConfig.Clusters[unchangingCluster]) { t.Errorf("expected %v, got %v", startingConfig.Clusters[unchangingCluster], mutatingConfig.Clusters[unchangingCluster]) } <del> if string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData) != redacted { t.Errorf("expected %v, got %v", redacted, string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData)) </del> <ins> if string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData) != dataOmitted { t.Errorf("expected %v, got %v", dataOmitted, string(mutatingConfig.Clusters[changingCluster].CertificateAuthorityData)) </ins> } if len(mutatingConfig.AuthInfos) != 2 {

Related discussion:
text: However, pointed out that other "long" fields are still completely printed in plain text such as client certs, so I think we need to revisit the "readability" argument. What do you think?
they are not... client cert/key are elided in the same way for the same reason: something like "++data+omitted++" could be better If the intent of config view without --raw specified is to be readable (which it currently is), printing many extremely long fields isn't great
Hmm what about: I'm pretty sure that would give us a valid b64 string (8+12=20%4=0) and it would subtly hint that the data isn't a secret.
Only valid characters are . And adding a prefix from the certificate only complicates things. is a suitable usable option. I'll submit a PR to continue the discussion there.
Code change:
return fmt.Sprintf("%s/%d/%s", ds.UID, ds.Status.ObservedGeneration, nodeName) } <del> // getPodsWithoutNode returns list of pods assigned to not existing nodes. func getPodsWithoutNode(runningNodesList []*v1.Node, nodeToDaemonPods map[string][]*v1.Pod) []string { </del> <ins> // getUnscheduledPodsWithoutNode returns list of unscheduled pods assigned to not existing nodes. // Returned pods can't be deleted by PodGCController so they should be deleted by DaemonSetController. func getUnscheduledPodsWithoutNode(runningNodesList []*v1.Node, nodeToDaemonPods map[string][]*v1.Pod) []string { </ins> var results []string isNodeRunning := make(map[string]bool) for _, node := range runningNodesList {

Related discussion:
text: What happened: Occasionally we get a Daemonset stuck with less Current replicas than Desired, causing a Daemonset sync tends to fix it, e.g (add a new node). This happens to all our Daemonsets. I'm using fluentd as an example. There are no Daemonsets pods in pending nor are there any errors when describing the Daemonset What you expected to happen: The Daemonset Pod to be scheduled How to reproduce it (as minimally and precisely as possible): Reproducing this state is very tricky, I've had the most success by terminating nodes in our ASG causing a new one to come up. However I think I've identified why this is happening. The issue appears to be related to daemonset Pod deletion, a Pod gets deleted causing to trigger, this lowers the expectations by as expected. The delete pod event then triggers a sync of the daemonset, in which it finds which involves fetching pods from the informer cache This is somehow returning the deleted pod. You can see in the logs here thinks that it needs to delete the pod that was deleted. It is also setting the expectation back to after the delete pod event previously lowered it by 1. At this point you're stuck as the function can't get past this line As the expectation can not be satisfied as syncNodes set it back to and there's nothing it to get it back to . So I believe this is what's causing us to get less replicas than expected, the function is stuck due to the expectation so it can't perform a full sync to schedule the new pods. There's also not much to trigger a new sync once the expectation has expired until a new node comes up or one goes down. Environment: Kubernetes version (use ):Cloud provider or hardware configuration: AWS OS (e.g: ):Kernel (e.g. ):
/sig apps
I investigated this as much as I could without getting some external input, it feels like it's some kind of race condition in the informer cache. If anyone has ideas of if places to dig deeper that would be handy.
Reiterating the mentions to trigger a notification: <detailsIn response to : I investigated this as much as I could without getting some external input, it feels like it's some kind of race condition in the informer cache. If anyone has ideas of if places to dig deeper that would be handy.
Code change:
return fmt.Sprintf("%s/%d/%s", ds.UID, ds.Status.ObservedGeneration, nodeName) } <del> // getPodsWithoutNode returns list of pods assigned to not existing nodes. func getPodsWithoutNode(runningNodesList []*v1.Node, nodeToDaemonPods map[string][]*v1.Pod) []string { </del> <ins> // getUnscheduledPodsWithoutNode returns list of unscheduled pods assigned to not existing nodes. // Returned pods can't be deleted by PodGCController so they should be deleted by DaemonSetController. func getUnscheduledPodsWithoutNode(runningNodesList []*v1.Node, nodeToDaemonPods map[string][]*v1.Pod) []string { </ins> var results []string isNodeRunning := make(map[string]bool) for _, node := range runningNodesList {

Related discussion:
text: Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
This problem is definitely related to node termination, it's trying to delete the already deleted pod from the node that's been terminated. The problem seems to go away if is set, I think although this kind of race condition may have always been possible, it's being frequently caused via Which is finding the non-existent pod in the pod cache and seeing that it is assigned to a non-running node, resulting in a request to delete the non-existent pod. cc
Reiterating the mentions to trigger a notification: <detailsIn response to : This problem is definitely related to node termination, it's trying to delete the already deleted pod from the node that's been terminated. Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign could you please take a look at this? It seems that it is related to the code you in .
Thanks for reporting the bug. Bug was introduced in which deleting pods without nodes. There is race condition between removing pod from cache and listing pods for DaemonSet. Bug occurs during following scenario: is deleted. for node is deleted by PodGCController. handler is executed by informer () set is to queue by delete handler () is processed and pods are taken from cache before deleted pod is removed. for deleted pod is not found so we add deleted pod to podsToDelete list (). are set to del:1() succeeds as pod is not found(). are never met so we need to wait till expectations expire. As fix I will send a change to so only pods that can't be deleted by PodGCController are deleted by DaemonSet controller.