Change:
<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.--Is this a request for help? (If yes, you should use our troubleshooting guide and community support channels, see ): No What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.): kubectl "does not allow access" Is this a BUG REPORT or FEATURE REQUEST? (choose one): Bug report <!-- If this is a BUG REPORT, please: Fill in as much of the template below as you can. If you leave out information, we can't help you as well. If this is a FEATURE REQUEST, please: Describe in detail the feature/behavior/change you'd like to see. In both cases, be ready for followup questions, and please respond in a timely manner. If we can't reproduce a bug or think a feature already exists, we might close your issue. If we're wrong, PLEASE feel free to reopen it and explain why. --Kubernetes version (use ): Environment: Cloud provider or hardware configuration: GCP GKE 1.4.5 OS (e.g. from /etc/os-release): Not sure, using gci Kernel (e.g. ): Not sure, using gci Install tools: GKE web UI Others: N/A What happened: In juggling multiple gcloud accounts with GKE clusters I accidentally used with the wrong application default credentials and it cached bad creds in After pointing gcloud at the right application default credentials and re-running , it continued trying and failing to use the bad creds cached in ; at this point I was stuck and confused for a few hours To repair, I had to manually edit to remove , and then started working again (or else wait ~1h for the access token to expire and refresh itself) What you expected to happen: After pointing gcloud at the right application default credentials should just work, or at least provide an error msg to the user that can help them figure out that they need to revoke the bad access token and how to do it How to reproduce it (as minimally and precisely as possible): Good scenario: just works Bad scenario: user makes one mistake and gets stuck
+1 Removing helped, thanks.
I can look into fixing this if no-one else has already made a start on it?
feel free to take it, thanks!
Ok, I think I have an understanding of what is happening call to is storing credentials in a json file in a known location. a subsequent credential-requiring kubectl command, the GCP plugin is being loaded, and the application default credentials are being exchanged for a token. That token is then cached in the .kube/config file. So where do things go wrong? As far as I can tell, if I switch to a cluster-2 without refreshing the application default credentials from it's separate account, those existing credentials will still be exchanged successfully for an oauth2 token and cached against cluster-2. Those tokens will then fail against a call to cluster-2 - because they were intended for cluster-1. The only sure-fire fix I can think of would be to ensure the application default credentials client ID is stored in users[].user.authprovider. Doing this would at least enable the ability to verify that the default credentials were valid for that particular cluster, and if not, prompt the user with a message to correctly aquire the application default credentials for it. Lets call this option 1. Of course, there is a problem here with determining what the Client ID is in the first place. I think this would require that a call to would store the client ID in the generated config. This would require exposing the client id from oauth2/google. Otherwise, perhaps storing the client ID the first time around would be enough - assuming the user at least manages to authenticate successfully against the cluster in the first instance. Then only oauth2/google would require a change, and not the gcloud command. The other option (option 2) would be, upon failure, check if is true and if-so, prompt the user that their application default credentials may be invalid for the target cluster, and to either prompt or clear the currently cached credentials in .kube/config. This would probably mean extending the auth provider interface such that a function may be provided that can be called upon failure that enables the auth provider plugin to do some checking of the state and to provide a reasonable message as to what might be wrong with the credentials that have been provided. I feel like option 2 is probably the better option in the long term. Different auth plugins are likely to have different failure modes. Providing a way to communicate back to the user about possible edge cases seems a little more flexible. Any thoughts? edit: On closer inspection this seems possible by wrapping the oauth2.Transport call in WrapTransport. I would then be able to check for an authorization failure and provide a reasonable error message based on the existing configuration.
Just come across this, its particularly strange to have to edit the config file, when there is a folder in the folder.
Is it really fixed? I still can reproduce it on k8s 1.14 with description provided in initial comment.
I also reproduced this, unfortunately. Tried to use kubectl to access a cluster from an unauthorised account. Then, switched to the authorised account but the invalid authentication was cached. To resolve it, I set the to an expired date.
Believe I'm hitting this issue as well
I just ran into this as well, gcloud 311.0.0, kubectl 1.18.6.
I've got the same issue. I need to change the expiry time to 1 year back as pointed out.
Solution that usually works for me So it will use different credentials when switching contexts
You'll need  installed ().

Discussion:
} func (g *gcpAuthProvider) WrapTransport(rt http.RoundTripper) http.RoundTripper { <del> return &conditionalTransport{&oauth2.Transport{Source: g.tokenSource, Base: rt}} </del> <ins> return &conditionalTransport{&oauth2.Transport{Source: g.tokenSource, Base: rt}, g.persister} </ins> } func (g *gcpAuthProvider) Login() error { return nil }

type conditionalTransport struct { oauthTransport *oauth2.Transport <ins> persister      restclient.AuthProviderConfigPersister </ins> } func (t *conditionalTransport) RoundTrip(req *http.Request) (*http.Response, error) { if len(req.Header.Get("Authorization")) != 0 { return t.oauthTransport.Base.RoundTrip(req) } <del> return t.oauthTransport.RoundTrip(req) </del> <ins> res, err := t.oauthTransport.RoundTrip(req) if err != nil { return nil, err } if res.StatusCode == 401 { glog.V(4).Infof("The credentials that were supplied are invalid for the target cluster") emptyCache := make(map[string]string) t.persister.Persist(emptyCache) } return res, nil </ins> }

import ( "fmt" <ins> "net/http" </ins> "os" "os/exec" "reflect"

t.Errorf("got cache %v, want %v", got, cache) } } <ins> type MockTransport struct { res *http.Response } func (t *MockTransport) RoundTrip(req *http.Request) (*http.Response, error) { return t.res, nil } func TestClearingCredentials(t *testing.T) { fakeExpiry := time.Now().Add(time.Hour) cache := map[string]string{ "access-token": "fakeToken", "expiry":       fakeExpiry.String(), } cts := cachedTokenSource{ source:      nil, accessToken: cache["access-token"], expiry:      fakeExpiry, persister:   nil, cache:       nil, } tests := []struct { name  string res   http.Response cache map[string]string }{ { "Unauthorized", http.Response{StatusCode: 401}, make(map[string]string), }, { "Authorized", http.Response{StatusCode: 200}, cache, }, } persister := &fakePersister{} req := http.Request{Header: http.Header{}} for _, tc := range tests { authProvider := gcpAuthProvider{&cts, persister} fakeTransport := MockTransport{&tc.res} transport := (authProvider.WrapTransport(&fakeTransport)) persister.Persist(cache) transport.RoundTrip(&req) if got := persister.read(); !reflect.DeepEqual(got, tc.cache) { t.Errorf("got cache %v, want %v", got, tc.cache) } } } </ins>