jiebi/CodeConvo huggingface dataset:
Change:
if err := s.Features.ApplyTo(genericConfig); err != nil { return nil, nil, err } <ins> if err := utilwait.PollImmediate(etcdRetryInterval, etcdRetryLimit*etcdRetryInterval, preflight.EtcdConnection{ServerList: s.Etcd.StorageConfig.ServerList}.CheckEtcdServers); err != nil { return nil, nil, fmt.Errorf("error waiting for etcd connection: %v", err) } </ins> // Use protobufs for self-communication. // Since not every generic apiserver has to support protobufs, we

Discussion:
See: This code should make several attempts before ing. Alternatively, we could consider not listening for connections until this code has succeeded.
Feel free to correct me if I'm wrong on any of this since I've never looked at kubernetes code base before. The code for the and the both appear to make multiple attempts before returning an error which leads to a Fatal.
, I think we had waited for to start at . Do we need to wait more or make it configurable?
I would like to work on this if nobody else is.
No need to ask for permission! Just say here that you're working on it, that should be sufficient to prevent duplicate work. :)
Can I have a go at this?
I have a nearly completed solution for this already. Just working on testing/test cases. WIll hopefully sumbit a PR shortly.
Ok.
I wondered why we care about this. After all, kubelet restarts failing pods. For posterity, after talking with the reason is that the apiserver may fail so fast that the kubelet begins to back off on restarts of a failing pod. That backoff may be too aggressive. (Maybe it causes spurious alerts, or maybe it allows too much downtime.) So, let's fix the apiserver. That's fine, but clearly we're not the only user that might encounter this. You may be running a server you don't want to (or can't) modify, and you might wish you could tweak the kubelet's backoff heuristics for your pod. Alternately, maybe the kubelet could choose not to enter the backoff if apiserver exited with a different code. anything like this already in place?
That's a wonderful explanation that hadn't occurred to me. If there isn't a feature request for this already I'll work on something and reference this issue.
AFAIK, the backoff mechanism is there to protect kubelet from being overwhelmed by failed user containers. Trusting the containers with returning a specific exit code sounds like a big "no" to me. Making this configurable per pod might be ok, but it's still not good in general to allow aggressive restarts.
The referred PR () only fixes the issue for kube-apiserver. We still need a generic fix in genericapiserver (Ref ).
Change:
--healthz_port="${KUBELET_HEALTHZ_PORT}" 1>&2 & KUBELET_PID=$! <del> kube::util::wait_for_url "http://127.0.0.1:${KUBELET_HEALTHZ_PORT}/healthz" "kubelet: " 0.2 25 </del> <ins> kube::util::wait_for_url "http://127.0.0.1:${KUBELET_HEALTHZ_PORT}/healthz" "kubelet" </ins> # Start kube-apiserver kube::log::status "Starting kube-apiserver"

Discussion:
Example:
Is kubelet healthz coupled with the docker daemon being accessible? ?
That particular error is coming from cAdvisor and should not be causing the test to block. The Kubelet does wait up to 5min for Docker to come up though. Do we use a real Docker in these tests?
Shippable is running on different os dist? ContainerOsVersion:Ubuntu precise?
Yes today kubelet healthz coupled with docker daemon. We even periodically healthcheck docker and kubelet, and restart them even there are issues.
Another example:
bumping priority because we have to get these flaky tests fixed.
This has been marked P0 for 2 months but hasn't had any updates. Is this still happening on Shippable? If so, can we just increase a timeout in the test?
This is ridiculous, I'll fix it myself.
Change:
if err != nil { return err } <del> otelDialer := func(ctx context.Context, addr string) (net.Conn, error) { return egressDialer(ctx, "tcp", addr) </del> <ins> if egressDialer != nil { otelDialer := func(ctx context.Context, addr string) (net.Conn, error) { return egressDialer(ctx, "tcp", addr) } opts = append(opts, otlptracegrpc.WithDialOption(grpc.WithContextDialer(otelDialer))) </ins> } <del> opts = append(opts, otlptracegrpc.WithDialOption(grpc.WithContextDialer(otelDialer))) </del> } resourceOpts := []resource.Option{

Discussion:
While trying to enable API Server Tracing while using k3s, the API Server component crashes. The only way to avoid this crash is to disable egress selectors in k3s entirely. I originally opened an issue in the k3s repo (see ), and we determined that the issue seems to be an assumption that a dialer will always be returned if there's no error, but in this case there's no grpc egress configured so is returning and causing a panic at API Server Tracing to function properly. Please see the linked k3s issue for a repro case; However, I believe the minimal setup for this would be to configure cluster egress without defining a grpc egress selector and then attempting to enable API server tracing. panic stack trace -- contents of egress selector config file in direct mode- contents of egress selector config in 'agent' mode - contents of api server tracing config file -- <details25.2+k3s1 </details<detailsDigital Ocean Droplet </details<details</details<details</details<details</details<details</details
/sig instrumentation
/assign /triage accepted Thanks
Change:
} t.Logf("Created service: %s", svcNodePort.Name) } <ins> // Regresion test for https://issues.k8s.io/115316 func TestServiceCIDR28bits(t *testing.T) { serviceCIDR := "10.0.0.0/28" client, _, tearDownFn := framework.StartTestServer(t, framework.TestServerSetup{ ModifyServerRunOptions: func(opts *options.ServerRunOptions) { opts.ServiceClusterIPRanges = serviceCIDR }, }) defer tearDownFn() // Wait until the default "kubernetes" service is created. if err := wait.Poll(250*time.Millisecond, time.Minute, func() (bool, error) { _, err := client.CoreV1().Services(metav1.NamespaceDefault).Get(context.TODO(), "kubernetes", metav1.GetOptions{}) if err != nil { return false, err } return true, nil }); err != nil { t.Fatalf("creating kubernetes service timed out") } ns := framework.CreateNamespaceOrDie(client, "test-regression", t) defer framework.DeleteNamespaceOrDie(client, ns, t) service := &v1.Service{ ObjectMeta: metav1.ObjectMeta{ Name: "test-1234", }, Spec: v1.ServiceSpec{ Type: v1.ServiceTypeClusterIP, Ports: []v1.ServicePort{{ Port: int32(80), }}, Selector: map[string]string{ "foo": "bar", }, }, } _, err := client.CoreV1().Services(ns.Name).Create(context.TODO(), service, metav1.CreateOptions{}) if err != nil { t.Fatalf("Error creating test service: %v", err) } } </ins>

Discussion:
kube-apiserver observed a panic: kube-apiserver should not panic Not sure yet -- will add more details as I investigate. No response v1.25.4 N/A N/A N/A N/A N/A
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig network /cc
max is greater than offset?
/assign
Change:
func makePodSpec(readinessProbe, livenessProbe *api.Probe) *api.Pod { pod := &api.Pod{ <del> ObjectMeta: api.ObjectMeta{Name: "nginx-" + string(util.NewUUID())}, </del> <ins> ObjectMeta: api.ObjectMeta{Name: "test-webserver-" + string(util.NewUUID())}, </ins> Spec: api.PodSpec{ Containers: []api.Container{ { <del> Name:           "nginx", Image:          "nginx", </del> <ins> Name:           "test-webserver", Image:          "gcr.io/google_containers/test-webserver", </ins> LivenessProbe:  livenessProbe, ReadinessProbe: readinessProbe, },

Discussion:
If the container_probe test runs on a node without the nginx image, it sometimes takes 90s to get the pod in running. Most of this time goes in pulling the image, so I'm assuming it's registry related hiccup. Using might make it more reliable. Example failure: Note the following event in the logs:
a) we should not be pulling from docker in our tests - too hit-or-miss b) we should replace nginx (132 MB) with (4.5 MB) whenever possible. , Prashanth B wrote:
Change:
ginkgo.By("waiting for RC to be added") eventFound := false <del> ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second) </del> <ins> ctx, cancel := context.WithTimeout(context.Background(), 60*time.Second) </ins> defer cancel() _, err = watchUntilWithoutRetry(ctx, retryWatcher, func(watchEvent watch.Event) (bool, error) { if watchEvent.Type != watch.Added {

Discussion:
<!-- Please only use this template for submitting reports about continuously failing tests or jobs in Kubernetes CI --Which jobs are failing: - Triage link for more jobs that are affected: Which test(s) are failing: Since when has it been failing: Since October 23 Testgrid link: Reason for failure: Anything else we need to know: Example spyglass link: /sig apps /cc /priority important-soon
/cc missed this in the main body
/assign /triage accepted
/remove-priority important-soon /priority critical-urgent top flake for pull-kubernetes-conformance-kind-ipv6-parallel, 2nd top flake for per /area conformance this is a conformance test /remove-kind failing-test /kind flake this is not failing continuously, but is instead flaking
/assign
I increased the timeout while waiting for available Replicas.
Test grid green since Merge 5 Nov 2020 /cc
Change:
if getUnschedulablePod(q, affinityPod) != nil { t.Error("affinityPod is still in the unschedulableQ.") } <ins> q.lock.RLock() </ins> if _, exists, _ := q.activeQ.Get(newPodInfoNoTimestamp(affinityPod)); !exists { t.Error("affinityPod is not moved to activeQ.") } <ins> q.lock.RUnlock() </ins> // Check that the other pod is still in the unschedulableQ. if getUnschedulablePod(q, &unschedulablePod) == nil { t.Error("unschedulablePod is not in the unschedulableQ.")

Discussion:
<!-- Please only use this template for submitting reports about flaky tests or jobs (pass or fail with no underlying change in code) in Kubernetes CI --Which jobs are flaking: pull-kubernetes-bazel-test Which test(s) are flaking: pkg/scheduler/internal/queue Testgrid link: Reason for failure: Anything else we need to know: /sig scheduling /assign
I just sent a pr for it according to the error message, but I'm not very sure if it will make things all right.
/assign
Change:
"k8s.io/apimachinery/pkg/labels" "k8s.io/apimachinery/pkg/runtime" "k8s.io/apimachinery/pkg/util/validation/field" <del> "k8s.io/apiserver/pkg/features" </del> "k8s.io/apiserver/pkg/registry/generic" "k8s.io/apiserver/pkg/storage" "k8s.io/apiserver/pkg/storage/names" <del> utilfeature "k8s.io/apiserver/pkg/util/feature" </del> ) // strategy implements behavior for CustomResources.

Discussion:
This is the tracking issue of promoting CRD validation rules to GA: Listed the blocking issues/features below: [x] [x] [x] [x] Issues be aware of but should not be a GA blocker [ ] [ ] [ ] [ ] [ ] /sig api-machinery /triage accepted
/assign
/cc
Change:
// syncResourceQuota runs a complete sync of current status func (rm *ResourceQuotaManager) syncResourceQuota(quota api.ResourceQuota) (err error) { <ins> // quota is dirty if any part of spec hard limits differs from the status hard limits dirty := !api.Semantic.DeepEqual(quota.Spec.Hard, quota.Status.Hard) </ins> // dirty tracks if the usage status differs from the previous sync, // if so, we send a new usage with latest status // if this is our first sync, it will be dirty by default, since we need track usage <del> dirty := quota.Status.Hard == nil || quota.Status.Used == nil </del> <ins> dirty = dirty || (quota.Status.Hard == nil || quota.Status.Used == nil) </ins> // Create a usage object that is based on the quota resource version usage := api.ResourceQuota{

Discussion:
Resource quota update does not take effect until create/delete another ResourceQuota. I am not sure it is an issue or design.
I am 99% sure that the issue is the update is changing the Spec and the Kubectl client is showing the Status. The Status will update at the next quota sync interval to align with the desired Spec. Can you describe what KUBERNETES environment you are using so I can verify? In particular, how is kube-controller-manager started? You can confirm this by doing kubectl get quota -o json And you should see the spec update occurred and if you wait 1 minute you will see the status take effect. A few options to improve user experience: Improve latency of updating status with spec. This has some potential side effects that I need to think through with how the current controller is structured. Change kubectl output to show spec if it differs from status so it's clear that the system will ultimately drive to its desired state. This seems like a good thing to do in general, will experiment with some options to present this information, but I am open to your input. I think even a single line noting that the enforced usage limits are different than the desired state but will reconcile soon. Sent from my iPhone
Sure to append config of kube-controller-manager: Tried the cmd, the spec updated as you said, however, the status refuse to take effect after more than 5 mins. I am not sure if it is caused by my version v0.13.2.
I see the issue, will send a PR. It appears the quota status was only getting marked as dirty for update if the usage changed, and not the underlying limit itself. So I suspect if you removed a pod (assuming you were tracking total number of pods), you would see the status kept updated to the desired spec.
The status changed correctly after resizing a rc and wait a while (8s).
I want to update the Kubectl output to show the desired value if it differs from the status value. I will look to do that in the next few days. I think that would eliminate all confusion. Sent from my iPhone
Change:
err := decorated2.Do(func() { defer close(executed) if !delegated { <del> t.Fatalf("expected delegated execution") </del> <ins> t.Errorf("expected delegated execution") </ins> } }) if err == nil {

Discussion:
Can we get this fixed ASAP? Thanks!
yep, thanks for reporting , Brendan Burns wrote:
how reproduceable is this? I have some changes that I think will resolve it but I couldn't reproduce locally even when all my cpu's were pretty tied up and using a high GOMAXPROCS.
flakey label process?
the labelling is due to me failing @ github's label interface. Bulk editing labels removes them if any of the selected things have the label sigh Thanks for the quick fix.
Change:
}) // Rc <del> output := framework.RunKubectlOrDie("describe", "rc", "redis-master", nsFlag) </del> requiredStrings := [][]string{ {"Name:", "redis-master"}, {"Namespace:", ns},

Discussion:
https://k8s- replica set controller maybe?
The rc seems to be updated almost a second after the failure. Maybe the test needs to retry
And the real log for the failure is
/assign flakes PRs at least once a day The can filter down to focus on a single test:
For when you have some time.
Reassign to you
Output of has been change in 1.6. That's why upgrade tests are failing. But I'm not sure why it is not failing 100% from the triage board. Do you know why? Output of 1.5 or earlier: Output in 1.6 or later:
The timeout is now. Is it reasonable to give it a little more time?
That timeout waits for the pods to run based on the API server. The rc controller is using its own cache so it may lag a bit before it actually updates the rc status. The test needs to retry for some time if the output we get back from describe does not match.
In other word, should also have its own retries.
SGTM. Going to send a fix today.
FYI, this test continues to flake PRs at least once per day:
should it show up in
Hm, this seems to fail in the upgrade tests. this failure seems to be related to an image name change as you already pointed out before. Mind finding what changed the test to use that image and fix the image name?
It is actually the key name has been changed in the key-value pair, not the value. As I pointed out in before expected key: now expected key:
Hm, this is unfortunate. These kubectl tests should really be unit tests.
the 24 hr test history is largely focused on failures right now, and this is highlighting flakes specifically (aka which tests are both passing and failing at the same commit the most often). Please follow the reports I'm sending to kubernetes-dev and related sigs. Also you can generate this info yourself using PS: also try out for finding failures (I'm sending out a separate email highlighting the biggest sources of failure). PPS: we are in the process of creating infrastructure to automate creating these reports, but automation never works as well as a fellow human asking for attention.
We still need to patch the release-1.4 and release-1.5 branch to make the upgrade tests passing.

I will send PRs later this week.
We didn't fix
The change mentioned in was introduced in 1.6. The change covers describer and kubectl e2e test. After discussing with we think we are not going to backport the changes to 1.4 and 1.5, instead we are going to skip the tests in version skew cases. WDYT?
should fix this issue
looks like the PR is resolved, should we close?
Please wait until fix to be cherrypick into 1.4.
Sure! Which PR is this?
Is it possible to cherrypick to 1.4 by just changing the milestone to 1.4?
no you need to create a new PR with the target branch of 1.4
Ready to close?
Yup. /close
Change:
"//staging/src/k8s.io/apimachinery/pkg/apis/meta/v1:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/runtime:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/runtime/schema:go_default_library", <ins> "//staging/src/k8s.io/apimachinery/pkg/types:go_default_library", </ins> "//staging/src/k8s.io/apimachinery/pkg/util/intstr:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/util/rand:go_default_library", "//staging/src/k8s.io/apimachinery/pkg/util/wait:go_default_library",

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: The endpointSliceTracker creates a set of resource versions for each service, the resource versions in the set could be deleted when endpointslices are deleted, but the set and its key in the map is never deleted, leading to memory leak. What you expected to happen: The set of resource versions should be delete from endpointSliceTracker when the service is deleted. Environment: Kubernetes version (use ): 1.18.5
There are no sig labels on this issue. Please add an appropriate label by using one of the following commands: - Please see the for a listing of the SIGs, working groups, and committees available. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/cc
Change:
logger.V(4).Info(log("calling NodeUnprepareResource rpc"), "request", req) defer logger.V(4).Info(log("done calling NodeUnprepareResources rpc"), "response", resp, "err", err) <del> if r.nodeClientCreator == nil { return nil, errors.New("failed to call NodeUnprepareResources. nodeClientCreator is nil") } nodeClient, nodeClientOld, closer, err := r.nodeClientCreator(r.addr) </del> <ins> conn, err := r.plugin.getOrCreateGRPCConn() </ins> if err != nil { return nil, err } <del> defer closer.Close() </del> <ins> nodeClient := drapb.NewNodeClient(conn) nodeClientOld := drapbv1alpha2.NewNodeClient(conn) </ins> ctx, cancel := context.WithTimeout(ctx, PluginClientTimeout) defer cancel()

Discussion:
See comment for details:
/sig node
/area kubelet
/assign
The Kubernetes project currently lacks enough contributors to adequately respond to all issues. This bot triages un-triaged issues according to the following rules: After 90d of inactivity, is applied After 30d of inactivity since was applied, is applied After 30d of inactivity since was applied, the issue is closed You can: Mark this issue as fresh with Close this issue with Offer to help out with [Issue Triage][1] Please send feedback to sig-contributor-experience at . /lifecycle stale [1]:
/remove-lifecycle stale
/triage accepted
Change:
func NewCmdDescribe(f cmdutil.Factory, out, cmdErr io.Writer) *cobra.Command { options := &resource.FilenameOptions{} <del> describerSettings := &printers.DescriberSettings{} </del> <ins> describerSettings := &printers.DescriberSettings{ ShowEvents: true, } </ins> // TODO: this should come from the factory, and may need to be loaded from the server, and so is probably //   going to have to be removed

Discussion:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug /sig cli What happened: Currently xxxOptions does not get bound as default value for some flags, which may bring in potential risks/bugs for some commands. Such as , Here the default value for flag will always be an empty string, regardless of the actual value of . Already discussed this potential bug with in Slack. This issue is used to track those cleanup items. [x] flag value bindings for kubectl attach/convert/delete/drain/edit/exec commands ( ) [x] flag value bindings for kubectl apply commands ( ) [x] flag value bindings for kubectl label/patch/taint/top commands ( ) [x] flag value bindings for kubectl create/get/set commands () [x] kubectl: flag value bindings for common utils ( )
/assign
/cc
This is a really good bit of cleanup and I'm glad to see the pulls.
/priority important-longterm
/reopen
[MILESTONENOTIFIER] Milestone Issue Needs Approval Action required: This issue must have the label applied by a SIG maintainer. If the label is not applied within 7 days, the issue will be moved out of the v1.11 milestone. <details<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Escalate to the issue owners; move out of the milestone after 1 attempt. : Fixes a bug discovered during the current release. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details
/close All the fixes have been merged.
Thanks
Change:
assert.NoError(t, err, test.description) assert.Equal(t, test.expected, real.FailureDomain, test.description) <ins> assert.Equal(t, strings.ToLower(cloud.Location), real.Region, test.description) </ins> } }

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: Azure cloud provider gets the availability zone from IMDS directly and the zone is usually in lower cases in most regions. But there're chances this is not true in some regions, hence customer's workloads depend on availability zone may not work. What you expected to happen: The availability zone is always in lower cases. How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): Cloud provider or hardware configuration: OS (e.g: ): Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /kind bug /sig cloud-provider /area provider/azure
/assign
Fixed by .