jiebi/CodeConvo huggingface dataset:
Change:
pod.Labels = jig.Labels _, err = cs.CoreV1().Pods(ns).Create(pod) framework.ExpectNoError(err) <del> framework.ExpectNoError(f.WaitForPodRunning(pod.Name)) </del> <ins> framework.ExpectNoError(f.WaitForPodReady(pod.Name)) </ins> defer func() { framework.Logf("Cleaning up the echo server pod") err := cs.CoreV1().Pods(ns).Delete(serverPodName, nil)

Discussion:
Per PR the test fails to error properly. One line of particular interest from that PR: If you wonder why it does not fail on docker, that's because the test does not report the error correctly but fails the same way. This issue is intended to track the discussion and resolution of making this test error properly so that it is useful to CI Signal /sig networking /cc
The label(s) cannot be appled. These labels are supported: <detailsIn response to : Per PR the test fails to error properly. One line of particular interest from that PR: If you wonder why it does not fail on docker, that's because the test does not report the error correctly but fails the same way. ` /cc </details
I'm following up on this in
/sig network
Change:
if err != nil { glog.Fatalf("Error creating storage factory: %v", err) } <del> endpointConfig, err := c.ExtraConfig.StorageFactory.NewConfig(kapi.Resource(DefaultStorageEndpoint)) </del> <ins> endpointConfig, err := c.ExtraConfig.StorageFactory.NewConfig(kapi.Resource("endpoints")) </ins> if err != nil { glog.Fatalf("Error getting storage config: %v", err) }

Discussion:
/kind bug The lease reconciler creates the kubernetes endpoint with a prefix of kube-apiserver-endpoint, however that is not the prefix that the rest of kube uses. The end result is that the kubernetes service doesn't have an endpoint, and kubectl won't list the kubernetes endpoint. The lease reconciler also creates lease files with a really long lease time instead of the default 15 seconds.
/cc
cc xref
kube-apiserver-endpoint is used for the lease itself, from what I remember. I don't think that's an issue by itself.
Oh but it does look like the endpointRegistry for the lease reconciler is misconfigured to use that
i'll look into it
what is the name that should be used?
services/endpoints afaik -
Yeah, making that change locally fixed it for me
But I'm not seeing the masterleases/apiServerIPInfo data in etcd, or I'm issuing the wrong command to find it
I had that same problem. The lease timeout is set way high, but for some reason that resulted in the lease files being created for only a very short time. I'm not sure why. Playing with the ttl amount got the lease files to stay around....forever. I've got a PR I'm about to push to fix both issues.
FYI has a PR for this now
cc
Change:
//  * If the status code and body don't make sense together: *UnexpectedStatusError //  * http.Client.Do errors are returned directly. func (r *Request) Do() Result { <ins> start := time.Now() defer func() { metrics.RequestLatency.WithLabelValues(r.verb, r.finalURLTemplate()).Observe(metrics.SinceInMicroseconds(start)) }() </ins> body, err := r.DoRaw() if err != nil { return Result{err: err}

Discussion:
Currently we do not measure latency for REST operations. This is important for any component that performs non-watch operations (scheduler, controllers, kubelet). This is also prerequisite to actually make sure that all API calls take less than 1 sec ( ).
The apiserver is already , including for REST operations. You can scrape the metrics by hitting the /metrics HTTP endpoint.
Sorry, I was not clear in the description. I'm more interested on the client side and with more details so that I can measure things like: how long it takes to write bindings in scheduler how long it takes to update status in kubelet I'd think about it as a counterpart for apiserver stats.
Change:
// Test handleConnection, using go-routine because it needs to be able to write to unbuffered pf.errorChan pf.handleConnection(localConnection, ForwardedPort{Local: 1111, Remote: 2222}) <ins> assert.Equal(t, 0, remoteConnection.streamCount, "stream count should be zero") </ins> assert.Equal(t, "", remoteDataReceived.String()) assert.Equal(t, "", localConnection.receiveBuffer.String()) assert.Equal(t, "Handling connection for 1111n", out.String())

Discussion:
The port forwarding client leaks memory when a port forwarding session is reused over time. The port forwarder client should not leak memory. forwarding a port with kubectl. and close many connections to the forwarded port. The amount of memory leaked for each connection is quite small, so many connections are needed to observe the leak. This is related to , which was reported for the kubelet and fixed in . However, the same problem happens to the client. The client needs to call on the , otherwise the streams created by will be kept in memory until the port forwarder itself is closed, even if the streams themselves closed long before. <details</details<detailsN/A </details<details</details<details</details<details</details<details</details
/sig api-machinery
/cc /triage accepted
/cc
Change:
By("Cleaning up cluster role binding") deleteClusterRoleBinding(config) <del> By("Removing the test directory") </del> <ins> By("Removing the test discovery directory") </ins> removeCmd := fmt.Sprintf("rm -r %s", path.Join(hostBase, discoveryDir)) err := framework.IssueSSHCommand(removeCmd, framework.TestContext.Provider, config.node0) Expect(err).NotTo(HaveOccurred())

Discussion:
The local PV provisioner is being updated to discover only real mount points (and not directories), that resolves some other related issues. The related e2e tests using the local PV provisioner need to be updated accordingly.
/assign
/sig storage
FYI:
Change:
t.Fatal(err) } if !exists { <del> t.Errorf("No expectations found for DaemonSet %q", oldDSKey) </del> <ins> t.Fatalf("No expectations found for DaemonSet %q", oldDSKey) </ins> } if dsExp.Fulfilled() { t.Errorf("There should be unfulfiled expectation for creating new pods for DaemonSet %q", oldDSKey) } <del> if dsc.queue.Len() != 0 { t.Fatal("Unexpected item in the queue") </del> <ins> // process updates DS, update adds to queue waitForQueueLength(1, "updated DS") ok = dsc.processNextWorkItem() if !ok { t.Fatal("queue is shutting down") </ins> } <ins> // process does not re-update the DS expectStableQueueLength(0) </ins> err = client.AppsV1().DaemonSets(oldDS.Namespace).Delete(context.Background(), oldDS.Name, metav1.DeleteOptions{}) if err != nil { t.Fatal(err) } <del> err = wait.PollImmediate(100*time.Millisecond, informerSyncTimeout, func() (bool, error) { klog.V(8).Infof("Waiting for queue to have 1 item, currently has: %d", dsc.queue.Len()) return dsc.queue.Len() == 1, nil }) if err != nil { t.Fatalf("Deleting DS didn't result in new item in the queue: %v", err) } </del> <ins> waitForQueueLength(1, "deleted DS") </ins> _, exists, err = dsc.expectations.GetExpectations(oldDSKey) if err != nil {

Discussion:
Which jobs are failing: TestExpectationsOnRecreate unit test, in along with expectations bugfix during 1.19 and cherry-picked to 1.18/1.17 Which test(s) are failing: Reason for failure: Anything else we need to know: These failures are masked in CI because bazel test runs with flake retries (), so the test can fail 2/3 times and the job can still pass (cc
adding to the milestone for triage, in case this is a real bug and not just a test issue /milestone v1.19
cc
/assign
looks like the panic is due to the test continuing in the !exists case (rather than exiting with a Fatal): however, that still means the test is failing in three distinct places:
it looks like the same failure existed in 1.18, so this probably isn't release blocking, but it does need to be resolved in order for us to stop ignoring unit test flakes in CI runs
it actually looks like this test was during the 1.19 dev cycle and cherry-picked back to 1.18/1.17, which is why release-1.18 is failing :-/
figured it out... the controller sync is updating the DS, which is triggering an update event and adding an extra enqueue which the test did not account for good news is that it appears to be a test-only issue
possible test deflake in
Change:
} } } <ins> func TestInterpretContainerStatus(t *testing.T) { tests := []struct { status          *api.ContainerStatus expectedState   string expectedMessage string expectErr       bool name            string }{ { status: &api.ContainerStatus{ State: api.ContainerState{ Running: &api.ContainerStateRunning{}, }, Ready: true, }, expectedState:   "Running", expectedMessage: "", name:            "basic", }, { status: &api.ContainerStatus{ State: api.ContainerState{ Running: &api.ContainerStateRunning{}, }, Ready: false, }, expectedState:   "Running *not ready*", expectedMessage: "", name:            "basic not ready", }, { status: &api.ContainerStatus{ State: api.ContainerState{ Waiting: &api.ContainerStateWaiting{}, }, Ready: false, }, expectedState:   "Waiting", expectedMessage: "", name:            "waiting not ready", }, { status: &api.ContainerStatus{ State: api.ContainerState{ Waiting: &api.ContainerStateWaiting{}, }, Ready: true, }, expectedState:   "Waiting", expectedMessage: "", name:            "waiting", }, { status: &api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{ ExitCode: 3, }, }, Ready: false, }, expectedState:   "Terminated", expectedMessage: "exit code 3", name:            "terminated not ready", }, { status: &api.ContainerStatus{ State: api.ContainerState{ Termination: &api.ContainerStateTerminated{ ExitCode: 5, Reason:   "test reason", }, }, Ready: true, }, expectedState:   "Terminated", expectedMessage: "exit code 5, reason: test reason", name:            "terminated", }, } for _, test := range tests { // TODO: test timestamp printing. state, _, msg, err := interpretContainerStatus(test.status) if test.expectErr && err == nil { t.Errorf("unexpected non-error (%s)", test.name) continue } if !test.expectErr && err != nil { t.Errorf("unexpected error: %v (%s)", err, test.name) continue } if state != test.expectedState { t.Errorf("expected: %s, got: %s", test.expectedState, state) } if msg != test.expectedMessage { t.Errorf("expected: %s, got: %s", test.expectedMessage, msg) } } } </ins>

Discussion:
When debugging , I found the output of 'kubectl get pods' was misleading, The status of the pod was reported as , but a further suggested the pod was not ready: shall warn the user about the fact that the pod is not ready.
Change:
return &os, nil } <ins> // NewFakeOpenStackCloud creates and returns an instance of Openstack cloudprovider. // Mainly for use in tests that require instantiating Openstack without having // to go through cloudprovider interface. func NewFakeOpenStackCloud(cfg Config) (*OpenStack, error) { provider, err := openstack.NewClient(cfg.Global.AuthURL) if err != nil { return nil, err } emptyDuration := MyDuration{} if cfg.Metadata.RequestTimeout == emptyDuration { cfg.Metadata.RequestTimeout.Duration = time.Duration(defaultTimeOut) } provider.HTTPClient.Timeout = cfg.Metadata.RequestTimeout.Duration os := OpenStack{ provider:     provider, region:       cfg.Global.Region, lbOpts:       cfg.LoadBalancer, bsOpts:       cfg.BlockStorage, routeOpts:    cfg.Route, metadataOpts: cfg.Metadata, } return &os, nil } </ins> // Initialize passes a Kubernetes clientBuilder interface to the cloud provider func (os *OpenStack) Initialize(clientBuilder cloudprovider.ControllerClientBuilder, stop <-chan struct{}) { }

Discussion:
We merged to allow configurable volume limits for Cinder. But proposed adding such a limit to as well, among other things this will enable us to support node specific limits more easily - /sig openstack /sig storage /sig scheduling
/assign
cc
/kind feature
This will require updating in-tree Openstack cloudprovider code. are we onboard with that?
What would be the extent of the modification? It must not break current compatibility, and the in-tree provider is scheduled for removal this year. I'm ok with it as long as existing users aren't broken by it.
it should be a change into which would add a new configurable field. It would be backward compatible but would allow admins to configure different limits on a node if they must.
/milestone v1.14 We're entering and this looks relevant to the 1.14 release. Please if I am incorrect
Change:
} // Overwrite the default for storage data format. s.Etcd.DefaultStorageMediaType = "application/vnd.kubernetes.protobuf" <ins> // register all admission plugins RegisterAllAdmissionPlugins(s.Admission.Plugins) </ins> // Set the default for admission plugins names s.Admission.PluginNames = []string{"AlwaysAdmit"} return &s

Discussion:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: The current help message from kube-apiserver is wrong. It shows only the 'NamespaceLifecycle' plug-in is available, and the default value is . Users cannot know what are the available plug-ins, and they get confused where comes from. What you expected to happen: The correct output should be this: How to reproduce it (as minimally and precisely as possible): Run using master or at least 1.7.2. Anything else we need to know?: This is a regression introduced when refactoring code into generic server code. Environment: Kubernetes version (use ): 1.7.2+ ? Cloud provider or hardware configuration**: Any OS (e.g. from /etc/os-release): Any Kernel (e.g. ): Any Install tools: Any Others: None
There are no sig labels on this issue. Please by: a sig: e.g., to notify the contributor experience sig, OR the label manually: e.g., to apply the label Note: Method 1 will trigger an email to the group. You can find the group list and label list . The in the method 1 has to be replaced with one of these: bugs, feature-requests, pr-reviews, test-failures, proposals
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Yes, it is really exist. If no guys works on this, i will pick it up.
Thanks for confirming this. I'm already working on it. Will submit pr in minutes.
cool. :)
Change:
err := l.volume.CleanupResource(ctx) errs = append(errs, err) <ins> if l.priorityClass != nil { framework.Logf("Deleting PriorityClass %v", l.priorityClass.Name) err := l.cs.SchedulingV1().PriorityClasses().Delete(ctx, l.priorityClass.Name, metav1.DeleteOptions{}) errs = append(errs, err) } </ins> framework.ExpectNoError(errors.NewAggregate(errs), "while cleaning up resource") l.migrationCheck.validateMigrationVolumeOpCounts(ctx) }

Discussion:
For 1.27 we should graduate the ReadWriteOncePod feature to beta. This is needed to rollout the feature to GA. In beta this feature will support preemption of pods using PVCs. See:
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig storage /sig scheduling
Change:
# GCE Load-Balancer Controller (GLBC) Cluster Addon This cluster addon is composed of: <del> * A [Google L7 LoadBalancer Controller](https://github.com/kubernetes/contrib/tree/master/Ingress/controllers/gce) </del> <ins> * A [Google L7 LoadBalancer Controller](https://github.com/kubernetes/contrib/tree/master/ingress/controllers/gce) </ins> * A [404 default backend](https://github.com/kubernetes/contrib/tree/master/404-server) Service + RC It relies on the [Ingress resource](../../../../docs/user-guide/ingress.md) only available in Kubernetes version 1.1 and beyond.

Discussion:
The has to the Google L7 load balancer controller.
+1 on merging that PR, looks good.
LGTM'd the pr
Change:
ginkgo.Context("With kubeconfig updated to static CPU Manager policy run the Topology Manager tests", func() { runTopologyManagerTests(f) }) })

Discussion:
<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: ci-kubernetes-node-kubelet-serial node-kubelet-serial job Which test(s) are failing: Hard to tell, but everything at one point or another. Since when has it been failing: A month or two, by recollection. would indicate several years... ci-kubernetes-node-kubelet-serial | failing_days | 992 Testgrid link: Reason for failure: Hard to tell, some is timeout, but I think timeouts due to test failures. Anything else we need to know: Nothing consistent about the failures that I can tell besides timeouts. This is the only place these serial tests are running.
/sig node
/cc
I will take a look at what happens with the serial lane.
So the kubelet failed to restart because of I think it can be related to the topology manager scope PR, I will dig into the code tomorrow. FYI
It actually looks like it was introduced by: With the same issue reported here: And Alexey has a fix for it here (I haven't reviewed it):
Thanks for the update.
Some of these failures are definitely due to , which is causing tests to fail since pods aren't terminating in time. See e.g.
/assign
Under the latest run, I saw that the kubelet failed to restart because of The problem that one of the topology manager tests failed in the middle and did not remove the state file, I can not guarantee that it a single problem, but let's start from it.
Change:
type LRUExpireCache struct { cache *lru.Cache <del> lock  sync.RWMutex </del> <ins> lock  sync.Mutex </ins> } func NewLRUExpireCache(maxSize int) *LRUExpireCache {

Discussion:
Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): No What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): "LRUExpireCache" Is this a BUG REPORT or FEATURE REQUEST? (choose one): BUG REPORT What happened: requires write lock since needs to manipulate its list to track recently used item. Currently it uses read lock so it may introduce race condition. What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else do we need to know:
This was fixed in
Change:
capacity := c.options.PVC.Spec.Resources.Requests[v1.ResourceName(v1.ResourceStorage)] <del> zonesWithNodes, err := cloud.GetCandidateZonesForDynamicVolume() </del> <ins> zonesWithNodes, err := getCandidateZones(cloud, node) </ins> if err != nil { <del> return "", 0, nil, "", fmt.Errorf("error querying for all zones: %v", err) </del> <ins> return "", 0, nil, "", fmt.Errorf("error finding candidate zone for pvc: %v", err) </ins> } volumeOptions, err := populateVolumeOptions(c.plugin.GetPluginName(), c.options.PVC.Name, capacity, tags, c.options.Parameters, node, allowedTopologies, zonesWithNodes)

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: We have pretty large clusters in aws and in production, we found an inefficiency in provisioning dynamic EBS volumes, where aws-ebs volume provisioner tries to look for available zone from aws directly. With scale, this can trigger massive from ec2, and affect cluster stability. This is also related to What you expected to happen: To simply get zones with available instances, we can get information from API server for efficiency. There might be a delay between node gets shutting down and such change is reflected in api server, but in case node is not ready or gets terminated, there will be higher level reconciliation triggered so we are safe. How to reproduce it (as minimally and precisely as possible): We only expect this to be hit when the cluster is huge (or aws account is huge) so not easy to reproduce. /assign /sig aws /sig cloud-provider
/assign
Change:
// if m.Current() == nil, then we try again and get a valid // client server.backdate = 0 <del> server.expectUserAgent = "FirstClient" </del> <ins> server.SetExpectUserAgent("FirstClient") </ins> if ok, err := r.RotateCerts(); !ok || err != nil { t.Fatalf("unexpected rotation err: %t %v", ok, err) }

Discussion:
<!-- Please only use this template for submitting reports about failing tests in Kubernetes CI jobs --Which jobs are failing: pull-kubernetes-bazel-test Which test(s) are failing: Test_buildClientCertificateManager Testgrid link: Reason for failure: Anything else we need to know:
/remove-kind failing-test /kind flake /sig scheduling
/sig node
Change:
"os" "path" "strconv" <del> "strings" </del> "time" "github.com/golang/glog"

Discussion:
Forked from To make debian-based node compatible with docker 1.11.X, I updated both /proc/sys/kernel/keys/rootmaxkey and /proc/sys/kernel/keys/rootmaxbytes. Also other kubernetes components do the similar thing. For example, kube_proxy: We should clean up those configurations into one common place.
cc/
Is this looking for an owner? It seems the ask is to generalize these types of settings in a module.
Yeah. We update quite a few kernel settings in kubelet itself. We should move the ones mentioned in this issue to kubelet if it is applicable to other distros too. want to own this?
Sure, I'll start poking.
Test results so far for a few distributions' default settings Operating SystemValue --- --- Debian Jessie 8.7.5200 --- --- --- Ubuntu 14. --- --- --- Ubuntu 16. --- --- --- CentOS --- --- --- Fedora --- --- --- Redhat Atomic --- --- --- CoreOS stable1000000 --- --- undefined --- undefined
Change:
command := exec.Command("mount", mountArgs...) output, err := command.CombinedOutput() if err != nil { <del> glog.Errorf("Mount failed: %vnMounting arguments: %s %s %s %vnOutput: %sn", </del> <ins> return fmt.Errorf("Mount failed: %vnMounting arguments: %s %s %s %vnOutput: %sn", </ins> err, source, target, fstype, options, string(output)) } return err

Discussion:
We have to spend times on figuring out what are the mount error code means from issues and . In , the error message is and is great for diagnosing mount problem. The console message better be returned rather than just error code.
Change:
go_test( name = "go_default_test", srcs = [ <ins> "cacher_whitebox_test.go", </ins> "selection_predicate_test.go", "time_budget_test.go", "util_test.go",

Discussion:
We noticed many goroutines stalled at in a large cluster (~200 nodes, ~1100 pods). It's because if the WatchServer timeout when the channel is full, will be blocked on the write to the even though is , because WatchServer stops consuming the channel after the timeout.
Great finding - thanks for debugging
Change:
"k8s.io/kubernetes/test/e2e/framework" e2edaemonset "k8s.io/kubernetes/test/e2e/framework/daemonset" e2edeployment "k8s.io/kubernetes/test/e2e/framework/deployment" <del> e2ekubesystem "k8s.io/kubernetes/test/e2e/framework/kubesystem" </del> e2enetwork "k8s.io/kubernetes/test/e2e/framework/network" e2enode "k8s.io/kubernetes/test/e2e/framework/node" e2epod "k8s.io/kubernetes/test/e2e/framework/pod"

Discussion:
master-informing: ! 2023-09-07 03:13:59 +0000 UTC No response /sig network
Checking this failure This is running ingress-gce version, logs in failure is It seems the healthceck interval is not updated correctly, by reading the test tagging the people with more expertise in this area /cc
/cc
it seems the test restarts the kube-controller-manager but we landed recently the switch from intree to outtree cloud providers, so this logic now runs in the cloud-controller-manager, can this be the problem, that we have to restart the cloud-controller-manager now instead of the kcm?
/triage accepted This flake started after and kept failing after 09-07.
yeah, the logic that handles the loadbalancer shifted to that cloud controller manager with that PR