jiebi/CodeConvo huggingface dataset:
Change:
c.Headers[kubectlCommandHeader] = strings.Join(cmdStrs, " ") } } <ins> // CancelRequest is propagated to the Delegate RoundTripper within // if the wrapped RoundTripper implements this function. func (c *CommandHeaderRoundTripper) CancelRequest(req *http.Request) { type canceler interface { CancelRequest(*http.Request) } // If possible, call "CancelRequest" on the wrapped Delegate RoundTripper. if cr, ok := c.Delegate.(canceler); ok { cr.CancelRequest(req) } } </ins>

Discussion:
No errors printed. Not sure. This is the same thing as , was fixed by Happened in a CI build Kubernetes version (use ): kubectl version 1.22.0 from image. Cloud provider or hardware configuration: OS (e.g: ): linux Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /sig cli
/triage accepted /assign
We'll need to cherry pick this to 1.22.
Looks like the cancellation was indeed from a timeout. Sean will have a fix but do you have more insight into the timeout?
The build I linked above is using a feature of GitLab Kubernetes Agent called , which allows to access Kubernetes API from a CI job. An agent is proxying requests into the cluster. When I was investigating timeouts I discovered that they happen because of in the agent. Throttling happened because kubectl was making way too many requests and the code was not caching the authentication results. After fixing this, I started to investigate why kubectl is making so many requests in the first place and discovered that the image that was used () has a read-only file system and discovery client from client-go cannot persist the discovery information to disc and hence it performs discovery 3 (!) times for a simple request. This results in 125 API calls. I tried to fix that in by caching the discovery client, but it doesn't help. The solution is to cache discovery info in memory because even if a single discovery client is used, it just discards the data and re-fetches it if it's not on disk. Please have a look at that PR.
The code that reported the error is The interface only has one function: The as a function of a appears to be a convention in most (but not all) Kubernetes .
For more context: the is implemented in the . This error message is printed when this cached discovery client encounters a timeout, and tries to cancel the request. The is wrapped by the , and it it unable to propagate the because the does not implement .
I'm experiencing this today, seeing this error on my two master nodes. The whole cluster is down. I was performing activities which were using the hard drive quite a bit. Not sure if that helps with debugging. Might be time to reboot the system hosting the node vms.

But I think I'll experiment some more being that it is a dev cluster. A reboot of each of the master nodes and things are coming back up. Worker nodes were fine.
Change:
}, }, requestInfo: &request.RequestInfo{ <ins> Verb:              "list", </ins> APIGroup:          "", APIVersion:        "v1", Resource:          "pods",

Discussion:
I've observed, that some metrics has and labels. (create) requests are "single resource", therefor they should be classified as . Def: customresourcehandler uses , which is based on the RequstInfo only: the RequestInfo is based only on the requests: method and url example: e.g: Since this RequestInfo has Name and Namespace empty, it is classified as . In a similar way it could be explained why: There's a mismatch in scope classification between built-in and custom resources: The key difference - the check if either "name" was provided OR it is a : Where the metrics.CleanScope ONLY checks, if name is not-empty. Tested: = v1.20 There were other scope related issues <1.20 and the distinction is not obvious. <details</details<details</details<details</details<details</details<details</details
/sig api-machinery
/triage accepted
Change:
type RollingUpdateDeployment struct { // The maximum number of pods that can be unavailable during the update. // Value can be an absolute number (ex: 5) or a percentage of total pods at the start of update (ex: 10%). <del> // Absolute number is calculated from percentage by rounding up. </del> <ins> // Absolute number is calculated from percentage by rounding down. </ins> // This can not be 0 if MaxSurge is 0. // By default, a fixed value of 1 is used. // Example: when this is set to 30%, the old RC can be scaled down by 30%

Discussion:
API documentation says is rounded up, but it is actually rounded down: The code shows it is rounded down, not up: and
Yes, the documentation is obsolete. We changed maxUnavailable to round down so that we wouldn't accidentally infer 100% unavailability for non-100 percentages. For example, maxSurge=25% and maxUanavailable=25% with a Deployment of 1 replica meant that we could scale down 1 before thus leading in 100% maxUnavailable (implicit).
Change:
} counter.Inc() <del> deprecatedCounter, err := deprecatedPodLogsUsage.GetMetricWithLabelValues(usageType) if err != nil { utilruntime.HandleError(err) return } deprecatedCounter.Inc() </del> <ins> deprecatedPodLogsUsage.WithLabelValues(usageType).Inc() </ins> } // NewGetOptions creates a new options object

Discussion:
Multiple jobs started to fail after the tag to 1.28.0-alpha.0 ! The reasons is that the apiserver is panicing when trying to access the pod logs, something very common in all e2e tests, see this occurrence per example The problem is caused by trying to update some metrics deprecated in 1.27 This change was introduced here I'm not very familiar with this subsystem, it seems that the logic to handle the deprecation in the metrics is not correct, as it is panicing when is accessed, or we should simple just not try to update them I really can't say what is the best course of action, we should not have anything in the codebase that modify the behavior depending on the external tags, but my feeling is we should not try to go for quick fix just to get the pipeline green. /priority critical-urgent /release blocking /sig instrumentation
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign /cc
/milestone v1.27

Yesterday. No response /sig release /priority critical-urgent
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Change:
// GetZone returns the Zone containing the current availability zone and locality region that the program is running in. // If the node is not running with availability zones, then it will fall back to fault domain. func (az *Cloud) GetZone(ctx context.Context) (cloudprovider.Zone, error) { <del> metadata, err := az.metadata.GetMetadata() if err != nil { return cloudprovider.Zone{}, err } </del> <ins> if az.UseInstanceMetadata { metadata, err := az.metadata.GetMetadata() if err != nil { return cloudprovider.Zone{}, err } </ins> <del> if metadata.Compute == nil { return cloudprovider.Zone{}, fmt.Errorf("failure of getting compute information from instance metadata") } </del> <ins> if metadata.Compute == nil { return cloudprovider.Zone{}, fmt.Errorf("failure of getting compute information from instance metadata") } </ins> <del> zone := "" if metadata.Compute.Zone != "" { zoneID, err := strconv.Atoi(metadata.Compute.Zone) if err != nil { return cloudprovider.Zone{}, fmt.Errorf("failed to parse zone ID %q: %v", metadata.Compute.Zone, err) </del> <ins> zone := "" if metadata.Compute.Zone != "" { zoneID, err := strconv.Atoi(metadata.Compute.Zone) if err != nil { return cloudprovider.Zone{}, fmt.Errorf("failed to parse zone ID %q: %v", metadata.Compute.Zone, err) } zone = az.makeZone(zoneID) } else { klog.V(3).Infof("Availability zone is not enabled for the node, falling back to fault domain") zone = metadata.Compute.FaultDomain </ins> } <del> zone = az.makeZone(zoneID) } else { klog.V(3).Infof("Availability zone is not enabled for the node, falling back to fault domain") zone = metadata.Compute.FaultDomain } return cloudprovider.Zone{ FailureDomain: zone, Region:        az.Location, }, nil </del> <ins> return cloudprovider.Zone{ FailureDomain: zone, Region:        az.Location, }, nil } // if UseInstanceMetadata is false, get Zone name by calling ARM hostname, err := os.Hostname() if err != nil { return cloudprovider.Zone{}, fmt.Errorf("failure getting hostname from kernel") } return az.vmSet.GetZoneByNodeName(strings.ToLower(hostname)) </ins> } // GetZoneByProviderID implements Zones.GetZoneByProviderID

Discussion:
What happened: Currently starting Kubectl failing on Azure Stack. What you expected to happen: Kubectl should start on Azure Stack environment. How to reproduce it (as minimally and precisely as possible): Try to deploy Kubernets on Azure Stack environment.. Anything else we need to know?: Environment: Kubernetes version (use kubectl version): master Cloud provider or hardware configuration: OS (e.g: cat /etc/os-release): Kernel (e.g. uname -a): Install tools: Others: /sig azure /assign
is it ? is a tool ...
my bad its kubelet. I update the title.
then could you provide the kubelet log? I am not sure kubelet on Azure stack is running as a container or process, you could get the logs by: And how do you deploy k8s on azure stack? cc
Change:
}, }, }, <del> Behavior: behavior, </del> <ins> Behavior: &autoscalingv2.HorizontalPodAutoscalerBehavior{ ScaleDown: &autoscalingv2.HPAScalingRules{ StabilizationWindowSeconds: &downscaleStabilizationSeconds, }, }, </ins> }, } hpa, errHPA := rc.clientSet.AutoscalingV2().HorizontalPodAutoscalers(rc.nsName).Create(context.TODO(), hpa, metav1.CreateOptions{})

Discussion:
gce-cos-autoscaling-hpa-cpu Kubernetes e2e suite.[sig-autoscaling] [Feature:HPA] [Serial] [Slow] Horizontal pod autoscaling (non-default behavior) with long upscale stabilization window should scale up only after the stabilization period Since it was in (first run at 2022-06-14 13:13 CEST failed, all subsequent runs failed too). https://k8s- Times out when waiting for scale down (), logs from (): I'm not sure why exactly, I have 2 ideas: It looks like the consumer briefly consumes 440 mCPU, which could result in a higher recommendation being generated HPA stabilises initial size of the consumer Increasing timeout should fix the issue No response_ /sig autoscaling
This issue is currently awaiting triage. If a SIG or subproject determines this is a relevant issue, they will accept it by applying the label and provide further guidance. The label can be by org members by writing in a comment. <detailsInstructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
FYI
This was already reverted (and later the test got fixed and merged). /close
Closing this issue. <detailsIn response to : This was already reverted (and later the test got fixed and merged). Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Change:
simpleStorage.requestedLabelSelector = nil simpleStorage.requestedFieldSelector = nil simpleStorage.requestedResourceVersion = "5" // Prove this is set in all cases <ins> simpleStorage.requestedResourceNamespace = "" </ins> dest.RawQuery = item.rawQuery resp, err := http.Get(dest.String()) if err != nil {

Discussion:
is always using the cross namespace context. This needs to be fixed to support scoped watches.
Change:
cloud:                 az, } <del> // BlobDiskController: contains the function needed to // create/attach/detach/delete blob based (unmanaged disks) blobController, err := newBlobDiskController(common) if err != nil { return fmt.Errorf("AzureDisk -  failed to init Blob Disk Controller with error (%s)", err.Error()) } // ManagedDiskController: contains the functions needed to // create/attach/detach/delete managed disks managedController, err := newManagedDiskController(common) if err != nil { return fmt.Errorf("AzureDisk -  failed to init Managed  Disk Controller with error (%s)", err.Error()) } az.BlobDiskController = blobController az.ManagedDiskController = managedController </del> <ins> az.BlobDiskController = &BlobDiskController{common: common} az.ManagedDiskController = &ManagedDiskController{common: common} </ins> az.controllerCommon = common return nil

Discussion:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: A storageAccount API call blocks kubelet from becoming Ready when there isn't any outbound IP, 1. eventually (after 10mins), the kubelet registers again... there is a call to the ARM API done by which blocks for 10mins, before letting kubelet complete with its starting logic and making the node as details: What you expected to happen: How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Environment: Kubernetes version (use ): v1.11.5 Cloud provider or hardware configuration: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others: /sig azure /assign
Change:
v, err := conversion.EnforcePtr(out) if err != nil { <del> panic("unable to convert output object to pointer") </del> <ins> return fmt.Errorf("unable to convert output object to pointer: %v", err) </ins> } key = path.Join(s.pathPrefix, key)

Discussion:
Remove panic in , please return error instead following .
/sig api-machinery /good-first-issue
This request has been marked as suitable for new contributors. Please ensure the request meets the requirements listed . If this request no longer meets these requirements, the label can be removed by commenting with the command. <detailsIn response to : /sig api-machinery /good-first-issue Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/assign
Please also add some log if returns error
/remove-kind bug /kind cleanup
Change:
gomega.Expect(pod.Status.Phase).To(gomega.Equal(v1.PodRunning)) allPods = append(allPods, pod) } <del> ginkgo.By("Ensuring cpu doesn't exceed limit by >5%") </del> <ins> ginkgo.By("Ensuring cpu doesn't exceed limit by >8%") </ins> for _, p := range allPods { ginkgo.By("Gathering node summary stats") nodeStats, err := e2ekubelet.GetStatsSummary(ctx, f.ClientSet, p.Spec.NodeName)

Discussion:
Failed error: Since migration to containerd 1.7.x No response The expected cpu usage is set to 0.5, but the actual cpu usage could be around 0.52-0.6, hence exceeding the limit more than the threshold of 5%. /sig windows
The label(s) cannot be applied, because the repository doesn't have them. <detailsIn response to : /sig sig-windows Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
/sig windows
cc:
/triage accepted I've reached out to some folks to see if the variance in the job limit is expected.
Jobs can potentially exceed the limit for a variety of reasons. If they do they get put into a deficit, so over time they would appear to be at the correct limit. Since we run for a short period of time we this is likely what we bumping into a scenario where the job happen to go over the limit. I would be in favor of softening the in this case since it appears we are just slightly over it occasionally. We can add a comment in the code that mention that it will even out over time.
Hi thank you for agreeing to soften the 5% check. Just check in to see the timeline for getting the limit updated? If needed, I can help with making the changes in a PR.
if you want to take a go at make this update, it would be appreciated!
/assign
Change:
err = apierrors.NewConflict(api.Resource("namespaces"), namespace.Name, fmt.Errorf("The system is ensuring all content is removed from this namespace.  Upon completion, this namespace will automatically be purged by the system.")) return nil, err } <del> return r.Store.Delete(ctx, name, nil) </del> <ins> return r.Store.Delete(ctx, name, options) </ins> } func (r *StatusREST) New() runtime.Object {

Discussion:
Honestly I'm not sure if this is a bug or a feature, but didn't expect it. Filing this to track.
I think it's a bug. Nothing in the code base depends on it that I can see (edit: namespace deletion does, but only via an in-server call which I'd like to see fixed), and we don't let other delete-related fields mutate during update (deleteGracePeriodSeconds)
Change:
supportedSubsystems := map[subsystem]bool{ &cgroupfs.MemoryGroup{}: true, &cgroupfs.CpuGroup{}:    true, <del> &cgroupfs.PidsGroup{}:   true, </del> <ins> &cgroupfs.PidsGroup{}:   false, </ins> } // not all hosts support hugetlb cgroup, and in the absent of hugetlb, we will fail silently by reporting no capacity. supportedSubsystems[&cgroupfs.HugetlbGroup{}] = false

Discussion:
What happened: We're updating the cluster from 1.12.1 to 1.14.3. When kubelet goes up for the first time, the following warnings related to cgroups pids controller appear: If we restart, kubelet returns fatal error with a similar motive: It makes sense since our kernel version doesn't support cgroups pids controller, then we disabled cgroups pids related feature gates: and expecting that kubelet would not require pids subsystems, but the behavior persisted. Looking at , it looks like pids subsystem will always be required even with feature gates disabled. This change was introduced with in . What you expected to happen: We expected that the cgroup pids subsystem to don't be required if feature gates related to pids are disabled. How to reproduce it (as minimally and precisely as possible): Try to start any kubelet in 1.14.x using a kernel older than 4.3. Environment: Kubernetes version (use ): Client Version: version.Info{Major:"1", Minor:"14", GitVersion:"v1.14.3", GitCommit:"", GitTreeState:"clean", BuildDate:"2019-06-06T01:44:30Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"} OS (e.g: ): Oracle Linux Server release 7.6 Kernel (e.g. ): Linux 4.1.12-124.26.12.el7uek.x8664 SMP Wed May 8 22:25:03 PDT 2019 x8664 x8664 x86_64 GNU/Linux Install tools: puppet
/sig node
Thanks for looking into this! Do you want to create a pr? We can then merge this into master and also cherry-pick it back into 1.14. Alternatively, if you don't have capacity for a pr I'm happy to take a stab.
Just created:
/assign
cc
Hi, We are facing this issue, with kubernetes v1.15.0, is this fixed?
- see the version information of when the commit landed in the commit itself:
Change:
gomega.Expect(err).To(gomega.BeNil(), "Failed to delete pod %v in namespace %v", config.Prefix+"-server", config.Namespace) } <del> func runVolumeTesterPod(client clientset.Interface, config TestConfig, podSuffix string, privileged bool, fsGroup *int64, tests []Test) (*v1.Pod, error) { </del> <ins> func runVolumeTesterPod(client clientset.Interface, config TestConfig, podSuffix string, privileged bool, fsGroup *int64, tests []Test, slow bool) (*v1.Pod, error) { </ins> ginkgo.By(fmt.Sprint("starting ", config.Prefix, "-", podSuffix)) var gracePeriod int64 = 1 var command string

Discussion:
Which jobs are flaking: gce-cos-master-alpha-features Which test(s) are flaking: [sig-storage] CSI Volumes [Driver: ][Serial] [Testpattern: Dynamic PV (default fs)] provisioning should provision storage with snapshot data source [Feature:VolumeSnapshotDataSource] Testgrid link: Reason for failure: It takes 6 minutes to start a pod with PVC that has been restored from a snapshot, while the test waits only 5 minutes: csi-attacher sidecar says: Perhaps the provisioner should wait until the volume is ready to be used? That may not help with this test timeout, as PVC is provisioned after the Pod is scheduled and the total time could be the same. All this is caused by my PR to test block snapshots, It reworked the tests from using . If snapshots are slow, should all persistent storage tests use / ?
/sig storage
Generally, cutting a snapshot is quick, but uploading can take longer. Does provisioner wait until the snapshot is "ready" before returning success?
cc
(But yes, we should increase the timeout because overall time will still be longer no matter which order we take)
The provisioner does not wait. If snapshot is not readyToUse, creating a volume from it will fail. I'll have to check to see if there's logic to wait in the e2e test. Regarding GCE PD, at restore time, do we need to download the snapshot from object store? If that's the case, it could take a while to download as well.
volume controller should keep trying until snapshot is ready. Looks like attacher tried to attach while disk is not ready. It is strange because I think attach only happen after volume is successfully provisioned.
AFAIK, most cloud providers' snapshot ATM will be uploaded to some remote when taking, and will be downloaded when restoring. GCE PD is one of them.
It's not taking snapshots what slows the test, it's restoring it - returns quickly, but the returned volume is not really usable and attach fails for ~4 minutes until the snapshot is fully restored somewhere in GCE. Now that I am thinking about it, maybe it's better if the driver returns the volume early, so it does not end up orphaned when user deletes corresponding PVC and the provisioner is restarted at the same time.

Anyway, I'll extend the timeout in volume checks. /assign
Which brings me to idea: what about other storage backends? Some of them may take even longer that 10 minutes to restore a snapshot. Or more than 5 minutes to provision an empty volume and run a pod. There should be some configuration of timeouts (timeout multiplier?) for
I opened to investigate from GCE side if this behavior is expected.