Change:
No errors printed. Not sure. This is the same thing as , was fixed by Happened in a CI build Kubernetes version (use ): kubectl version 1.22.0 from image. Cloud provider or hardware configuration: OS (e.g: ): linux Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others: /sig cli
/triage accepted /assign
We'll need to cherry pick this to 1.22.
Looks like the cancellation was indeed from a timeout. Sean will have a fix but do you have more insight into the timeout?
The build I linked above is using a feature of GitLab Kubernetes Agent called , which allows to access Kubernetes API from a CI job. An agent is proxying requests into the cluster. When I was investigating timeouts I discovered that they happen because of in the agent. Throttling happened because kubectl was making way too many requests and the code was not caching the authentication results. After fixing this, I started to investigate why kubectl is making so many requests in the first place and discovered that the image that was used () has a read-only file system and discovery client from client-go cannot persist the discovery information to disc and hence it performs discovery 3 (!) times for a simple request. This results in 125 API calls. I tried to fix that in by caching the discovery client, but it doesn't help. The solution is to cache discovery info in memory because even if a single discovery client is used, it just discards the data and re-fetches it if it's not on disk. Please have a look at that PR.
The code that reported the error is The interface only has one function: The as a function of a appears to be a convention in most (but not all) Kubernetes .
For more context: the is implemented in the . This error message is printed when this cached discovery client encounters a timeout, and tries to cancel the request. The is wrapped by the , and it it unable to propagate the because the does not implement .
I'm experiencing this today, seeing this error on my two master nodes. The whole cluster is down. I was performing activities which were using the hard drive quite a bit. Not sure if that helps with debugging. Might be time to reboot the system hosting the node vms. But I think I'll experiment some more being that it is a dev cluster. A reboot of each of the master nodes and things are coming back up. Worker nodes were fine.

Discussion:
c.Headers[kubectlCommandHeader] = strings.Join(cmdStrs, " ") } } <ins> // CancelRequest is propagated to the Delegate RoundTripper within // if the wrapped RoundTripper implements this function. func (c *CommandHeaderRoundTripper) CancelRequest(req *http.Request) { type canceler interface { CancelRequest(*http.Request) } // If possible, call "CancelRequest" on the wrapped Delegate RoundTripper. if cr, ok := c.Delegate.(canceler); ok { cr.CancelRequest(req) } } </ins>

package genericclioptions import ( <ins> "net/http" </ins> "testing" "github.com/spf13/cobra"

} return currCmd } <ins> // Tests that the CancelRequest function is propogated to the wrapped Delegate // RoundTripper; but only if the Delegate implements the CancelRequest function. func TestCancelRequest(t *testing.T) { tests := map[string]struct { delegate  http.RoundTripper cancelled bool }{ "CancelRequest propagated to delegate": { delegate:  &cancellableRoundTripper{}, cancelled: true, }, "CancelRequest not propagated to delegate": { delegate:  &nonCancellableRoundTripper{}, cancelled: false, }, } for name, tc := range tests { t.Run(name, func(t *testing.T) { rt := &CommandHeaderRoundTripper{ Delegate: tc.delegate, } req := http.Request{} rt.CancelRequest(&req) if tc.cancelled != req.Close { t.Errorf("expected RoundTripper cancel (%v), got (%v)", tc.cancelled, req.Close) } }) } } // Test RoundTripper with CancelRequest function. type cancellableRoundTripper struct{} func (rtc *cancellableRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) { return nil, nil } func (rtc *cancellableRoundTripper) CancelRequest(req *http.Request) { req.Close = true } // Test RoundTripper without CancelRequest function. type nonCancellableRoundTripper struct{} func (rtc *nonCancellableRoundTripper) RoundTrip(req *http.Request) (*http.Response, error) { return nil, nil } </ins>