jiebi/CodeConvo huggingface dataset:
Code change:
return iface.Name, nil } <del> func setUpAllInterfaces() error { interfaces, err := net.Interfaces() if err != nil { return err } for _, netIf := range interfaces { setUpInterface(netIf.Name) // ignore errors } return nil } </del> func setUpInterface(ifName string) error { glog.V(3).Infof("Enabling hairpin on interface %s", ifName) ifPath := path.Join(sysfsNetPath, ifName)

Related discussion:
text: I think the only way to see if there is some detail I am missing is to try it and see if anything pops up that would make this not work. , TonyAdo wrote:
I need to check which part is responsible of what, but what if the hairpin package was just a utility package that should be called by the plugin if it needs to set this up? As long as it is single and simple function call, doing so would avoid adding extra logic here and fix the issue. This is valid only if the aim is to have a "linux bridge plugin", an "OVS plugin", etc.
actually kubelet will call after get called, I think to make the network plugin can call hairpin package is not that easy, of course, I agree that network plugin should setup hairpin mode if it really needs, kubelet should not set hairpin mode unilaterally. Besides, kubelet is not going to provide an "linux bridge plugin" or "OVS plugin".
then the validity conditions of my comment are not met, so I see any simpler way. Thanks for checking! :)
still what do you think about disable hairpin completely and leave this to network plugin?
or if there is a custom network plugin, we don't setup hairpin mode, only do this in default.
Completely? It will break my use case (and anyone using Apache Kafka).
Only by default is what I understood from the previous comments, and LGTM.
Really sorry about that, but I really want to know are you using a kubelet network plugin in your use case? If no, I think we will still setup hairpin, as in default we will do this.
I don't use any plugin, so it is ok if hairpin is set up by default.
ok, I think this will not break your use case.
It's not really the network plugin that needs hairpinmode though, it's the pure-iptables-based kube-proxy code. It's possible the proxying code could change again in the future in such a way that hairpinmode wouldn't be needed any more. So it seems wrong to put that code into the plugin. makes it so that we set hairpinmode on the veth if it's attached to a bridge, but just silently do nothing if it's not.
Code change:
return iface.Name, nil } <del> func setUpAllInterfaces() error { interfaces, err := net.Interfaces() if err != nil { return err } for _, netIf := range interfaces { setUpInterface(netIf.Name) // ignore errors } return nil } </del> func setUpInterface(ifName string) error { glog.V(3).Infof("Enabling hairpin on interface %s", ifName) ifPath := path.Join(sysfsNetPath, ifName)

Related discussion:
text: (Yes, we currently set hairpinmode even if you're using the userspace proxier, but AFAIK it's not needed in that case.)
IMHO... The proxy has the responsibility of service objects, while the kubelet is responsible of pod objects, including their networking. Also, the responsibility of hairpin must be at the same place as the code that creates the network interface, otherwise you have code managing a single object (the pod interface) spread over multiple programs, which will eventually lead to complex problems and a system harder to understand even for the end-user. Thus, it doesn't seems wrong to me that the kubelet does whatever is required to make the pod work in the environment of the cluster. While its true that the hairpin requirement is only for the proxy in iptables mode, a switch to this mode can happen at any time in the life of the cluster. And when it happens, its probably because the userland proxy becomes unresponsive, which means in a situation where the administrator is in a hurry to find a solution. Currently, you just change the proxy mode and restart the proxies. There's little chance the administrator will think about hairpin or restarting the kubelet in this case, so the kubelet should be reacting to this; which means the proxy and the kubelet of a given host should be able to communicate (AFAIK, this doesn't exist now). And all this is currently only to avoid setting up hairpin on the pod's port of standard Linux bridge.
Feel sure I'm missing something but... The CNI bridge plugin now has a flag, so kubenet can use that. Having done that, we can set false when the plugin name is non-blank.
Is this fixed now?
I apologise for leaving a somewhat vague answer, but I have been in that code recently. I don't think it's fixed.
, Bryan Boreham wrote:
Thanks for the PR. CNI Bridge plugin is expecting . Not . Hence the bridge plugin did not recognize it.
EDIT 2016-12-09: The chain below was broken by , so the issue can be closed again (remain closed) I suggest this should be re-opened. The chain of calls is: sets to , e.g. when using CNI calls something goes wrong and it sets the mode on everything. An example of "something goes wrong":
Code change:
} } <del> // RecommendedConfigOverrideFlags is a convenience method to return recommended flag names prefixed with a string of your choosing func RecommendedConfigOverrideFlags(prefix string) ConfigOverrideFlags { return ConfigOverrideFlags{ AuthOverrideFlags:    RecommendedAuthOverrideFlags(prefix), ClusterOverrideFlags: RecommendedClusterOverrideFlags(prefix), ContextOverrideFlags: RecommendedContextOverrideFlags(prefix), CurrentContext: FlagInfo{prefix + FlagContext, "", "", "The name of the kubeconfig context to use"}, Timeout:        FlagInfo{prefix + FlagTimeout, "", "0", "The length of time to wait before giving up on a single server request. Non-zero values should contain a corresponding time unit (e.g. 1s, 2m, 3h). A value of zero means don't timeout requests."}, } } </del> // RecommendedContextOverrideFlags is a convenience method to return recommended flag names prefixed with a string of your choosing func RecommendedContextOverrideFlags(prefix string) ContextOverrideFlags { return ContextOverrideFlags{

Related discussion:
text: Found in openshift, but I'm pretty sure the codepaths are the same. That means the token is getting ignored and in-cluster config is being used in preference to flags.
Is a similar issue?
Yep, kind of regression.
Sigh.
Yes, most likely the same cause.
I'm on it.
Unit test :)
Probably will want to cherrypick this into 1.5.
WIP in
Code change:
"//vendor/golang.org/x/oauth2/google:go_default_library", "//vendor/google.golang.org/api/compute/v1:go_default_library", "//vendor/google.golang.org/api/dns/v1:go_default_library", <ins> "//vendor/google.golang.org/api/googleapi:go_default_library", </ins> "//vendor/gopkg.in/gcfg.v1:go_default_library", ], )

Related discussion:
text: /kind bug What happened: walteraa [6:55 AM] Hi, The controller manager of the federation using Google cloud DNS is crashing, the log showed is that: the hyperkube command is that: I have created this in the Google DNS Someone knows why the DNS zone query is getting the 404 error?? :disappointed: What you expected to happen: It should have created the missing domain. How to reproduce it (as minimally and precisely as possible): As per above. Anything else we need to know?: Environment: Kubernetes version (use ): 1.7.4 Cloud provider or hardware configuration**: OpenStack (irrelevant) OS (e.g. from /etc/os-release): irrelevant Kernel (e.g. ): irrelevant Install tools: irrelevant Others: irrelevant
The bug is related to this: The above code assumes that the provider code returns no error but an empty list if the domain does not exist, but instead the provider returns a 404. Need to fix the provider and/or the above code.
/sig federation
I am unlikely to have time to work on this soon, so if anyone else wants to, go right ahead.
/assign
An additional information: I have created a which is based on the credential JSON file created through instruction. After that, I have configured the Federation controller manager to mount this secret as a volume in the container and set the environment variable to the JSON file inside this volume. Before doing that, I was getting an authentication error so the google DNS provider couldn't be instantiated, and after doing the instructions that I mentioned before, this authentication error was solved but now I'm getting 404 error when Google DNS provider tries access the specified managed zone. The file that I have generated follow this structure:
Code change:
return fmt.Errorf("waiting for kubelet timed out") } <del> func RestartApiserver(c discovery.ServerVersionInterface) error { </del> <ins> func RestartApiserver(cs clientset.Interface) error { </ins> // TODO: Make it work for all providers. if !ProviderIs("gce", "gke", "aws") { return fmt.Errorf("unsupported provider: %s", TestContext.Provider) } if ProviderIs("gce", "aws") { <del> return sshRestartMaster() </del> <ins> initialRestartCount, err := getApiserverRestartCount(cs) if err != nil { return fmt.Errorf("failed to get apiserver's restart count: %v", err) } if err := sshRestartMaster(); err != nil { return fmt.Errorf("failed to restart apiserver: %v", err) } return waitForApiserverRestarted(cs, initialRestartCount) </ins> } // GKE doesn't allow ssh access, so use a same-version master // upgrade to teardown/recreate master. <del> v, err := c.ServerVersion() </del> <ins> v, err := cs.Discovery().ServerVersion() </ins> if err != nil { return err }

Related discussion:
text: This test is failing in both gce serial suite: http://k8s- /sig network /priority failing-test /priority critical-urgent /kind bug /status approved-for-milestone cc /assign PTAL
xref
/milestone v1.10
this also fails in http://k8s-
Failing to connect to Services at all. I can't even ping pod IPs:
So I can ping pods same-node and cross-node right up until it kills the apiserver. Rebooting fixes a given node. The apiserver demonstrably comes back - I can query it for info...
- I could use help - have to shift attention soon.
FWIW, I am testing on a Calico-enabled node
I'll take a look at this as well.
Weird, now even after a reboot, this condition is sticky. I have a cluster in this state, if you want to poke.
ugh, now it is resolved.
I just repro'ed without the e2e - have a cluster, proved ping works, killed apiserver ping fails.
Interesting - docker shows the pods not running, but apiserver says they are alive.
Update: docker sees no running pods. API server thinks kubelet is running them. kubelet is getting "" from apiserver, even after kubelet restart.
Update and recap: Bring up a cluster Run some pods Prove that pods are running and services work pkill -9 apiserver Wait. Watch kubelets "lose" pods that were assigned to them Observe that pods are not running and services do not work Observe that apiserver says the pods are running pkill -9 kubelet Observe that it is still broken kubectl delete one pod Observe that kubelet is fixed for that one pod, but not others Q: Why is kubelet not getting a full listing when it restarts?
What exactly does it mean for it to be fixed for a deleted pod?
Kubelet started running the pod that was deleted - it got the update for that one.
The cluster I was testing was a week or 2 old. I built a cluster from HEAD and I can not repro - said the same. Is this test running against head of master?
I also ran the test manually against my new cluster and it passed
FWIW I was also unable to reproduce this yesterday on a cluster from HEAD.
Code change:
return fmt.Errorf("waiting for kubelet timed out") } <del> func RestartApiserver(c discovery.ServerVersionInterface) error { </del> <ins> func RestartApiserver(cs clientset.Interface) error { </ins> // TODO: Make it work for all providers. if !ProviderIs("gce", "gke", "aws") { return fmt.Errorf("unsupported provider: %s", TestContext.Provider) } if ProviderIs("gce", "aws") { <del> return sshRestartMaster() </del> <ins> initialRestartCount, err := getApiserverRestartCount(cs) if err != nil { return fmt.Errorf("failed to get apiserver's restart count: %v", err) } if err := sshRestartMaster(); err != nil { return fmt.Errorf("failed to restart apiserver: %v", err) } return waitForApiserverRestarted(cs, initialRestartCount) </ins> } // GKE doesn't allow ssh access, so use a same-version master // upgrade to teardown/recreate master. <del> v, err := c.ServerVersion() </del> <ins> v, err := cs.Discovery().ServerVersion() </ins> if err != nil { return err }

Related discussion:
text: Did some investigations and had some findings for this specific failure: Traced down a bit on the history, the first occurrence was https://k8s- From the , seems suspicious because it modified . Also found reported a relevant symptom: The fix () for that was merged a day ago and may explained why we can't repro from HEAD anymore.
nice, is this issue reproducible without the fix ()?
Sorry should have had confirmed this, bringing up a cluster now.
Quick update: I was able to repro the same failure () from HEAD with reverted. Though this test is now flaky on some other errors: https://k8s- https://k8s- They all seem to failed while communicating with apiserver.
Thanks ! So fixed the issue and I am curious about what introduced the issue. Do you happen to know when did this test start to fail? Just trying to find out what went wrong in between. We have some other possibly related issues so finding out the root cause of this one would be super helpful.
/assign
The first occurrence (for that exact failure) I noticed was https://k8s-, which happened on 2018-02-15 09:49 PST.
You all rock! I see a couple greens on it now, but still mostly red?
The test is now flaky mostly on creating an exec pod after restarting apiserver. Some sample failures: To elaborate a bit more, a snippet of the test procedure is: ssh to master and (sending a SIGTERM) Keep polling on /healthz on apiserver until 200 is returned. Create an exec pod for later steps (and failed on various erros --- EOF, connection refused, apiserver terminating). According to above test failures, my suspicion is that the test is assuming the apiserver is restarted too soon. It is possible that apiserver captured SIGTERM and did a graceful termination, so that polling on /healthz may succeed even in the termination phase. I'm not surprised trying to create a pod via a terminating apiserver will hit those errors. Could anyone confirme the SIGTERM theory?
Spoke with folks, sounds like after SIGTERM apiserver does wait a bit for remaining requests to finish. I will take a stab to refine the test. cc
Sent .
Test is now passing on gce but failing on gke.
Code change:
return fmt.Errorf("waiting for kubelet timed out") } <del> func RestartApiserver(c discovery.ServerVersionInterface) error { </del> <ins> func RestartApiserver(cs clientset.Interface) error { </ins> // TODO: Make it work for all providers. if !ProviderIs("gce", "gke", "aws") { return fmt.Errorf("unsupported provider: %s", TestContext.Provider) } if ProviderIs("gce", "aws") { <del> return sshRestartMaster() </del> <ins> initialRestartCount, err := getApiserverRestartCount(cs) if err != nil { return fmt.Errorf("failed to get apiserver's restart count: %v", err) } if err := sshRestartMaster(); err != nil { return fmt.Errorf("failed to restart apiserver: %v", err) } return waitForApiserverRestarted(cs, initialRestartCount) </ins> } // GKE doesn't allow ssh access, so use a same-version master // upgrade to teardown/recreate master. <del> v, err := c.ServerVersion() </del> <ins> v, err := cs.Discovery().ServerVersion() </ins> if err != nil { return err }

Related discussion:
text: Apparently we don't have a way to trigger apiserver restart on gke at this moment. Sending a PR to skip this test on gke.
[MILESTONENOTIFIER] Milestone Issue Needs Attention Action required: During code freeze, issues in the milestone should be in progress. If this issue is not being actively worked on, please remove it from the milestone. If it is being worked on, please add the label so it can be tracked with other in-flight issues. Note: This issue is marked as , and must be updated every 1 day during code freeze. Example update: <details<summaryIssue Labels</summary- : Issue will be escalated to these SIGs if needed. : Never automatically move issue out of a release milestone; continually escalate to contributor and SIG through all available channels. : Fixes a bug discovered during the current release. </details<details<summaryHelp</summary<ul<li<a href=""Additional instructions</a</li<li<a href=""Commands for setting labels</a</li</ul</details
Commented too quick. On gke we do a same-version master upgrade. Will send another fix :/
Code change:
return nil } <del> var redactedBytes []byte </del> <ins> var ( redactedBytes    []byte dataOmittedBytes []byte ) </ins> // Flatten redacts raw data entries from the config object for a human-readable view. func ShortenConfig(config *Config) {

Related discussion:
text: Is this a BUG REPORT or FEATURE REQUEST?: What happened: shows: The use of "REDACTED" for a CA here is misleading, since it's a public certificate that is not a secret. What you expected to happen: I expected to see the full CA in pem format (like when you cat ) or something like "SUPPRESSED" or "TRUNCATED" How to reproduce it (as minimally and precisely as possible): Anything else we need to know?: Nope Environment: Kubernetes version (use ):Cloud provider or hardware configuration:OS (e.g. from /etc/os-release):Kernel (e.g. ):
(for thoughts)
Reiterating the mentions to trigger a notification: <detailsIn response to : (for thoughts) Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
use you can see the complete certificate authority data
Hey Thank you - I have tracking that . The point of this issue is that at best it's a bad UX; at worst it's a miseducation on pki. "REDACTED" isn't the right word here
If there are multiple clusters displaying certificates, the configuration will be too long, so this switch is turned off by default. But we can add an example to help use. thanks
Hey I don't personally view truncation the responsibility of the CLI... users can pipe things to a pager if they really care. That being said, I'm not concerned about the length. The issue is purely about the word choice of "REDACTED". The term means: The use of the term "REDACTED" in the output implies that a CA public key is remove for security when (as you've pointed out) it is removed for truncation. Truncation and redaction are fundamentally different things, and k8s is misusing the word redacted for this use case.
Not quite a bug, but I think you're right a better term could be used there as it is a public key, not a private key. /remove-kind bug
Issues go stale after 90d of inactivity. Mark the issue as fresh with . Stale issues rot after an additional 30d of inactivity and eventually close.
Code change:
return nil } <del> var redactedBytes []byte </del> <ins> var ( redactedBytes    []byte dataOmittedBytes []byte ) </ins> // Flatten redacts raw data entries from the config object for a human-readable view. func ShortenConfig(config *Config) {

Related discussion:
text: If this issue is safe to close now please do so with . Send feedback to sig-testing, kubernetes/test-infra and/or . /lifecycle stale
/remove-lifecycle stale I'm not sure what I need to do here. This was marked as a feature. Do you need something more from me?
We are waiting for someone to fix this.
/remove-good-first-issue I don't think this fits our definition of a :
So I have to keep replying to the comment bot every 30 days so it doesn't get auto-closed? That feels... odd
thanks for the education :) unfortunately other issues are higher priority and no one has stepped up to fix this. The autocloser is a form of bug bankruptcy that keeps our issue load in check. The contribex special interest group is definitely open to feedback on the bot.
The biggest hurdle in this is that Kubernetes is using some base64 trickery to display the string (basically setting the certificate data to the decoded bytes from ). We can change this to or but not (it has to be a valid base64 encoded string). I tried exploring multiple options for the text, but I couldn't find a suitable one. Given that we already print the user tokens and other credentials this is a bit weird. Should we redact those too? Is it okay if we print the CA certificate instead of truncating it?
I can't help but laugh at: I'm not wed to any particular name. I also have no problem displaying the entire certificate. The crux of the issue is that a CA public key is not secret or sensitive. It's a public key
I agree. The issue is with the constraints imposed by how this is implemented currently. I tried to come up with a suitable string (given the constraint) and couldn't find anything acceptable... The only suitable option that worked for me is . I personally prefer to print the CA certificate since we are already printing user tokens and client certs that are actual secrets.
I agree. Who do we need to get in agreement that printing the full CA is the best path forward?
the data is not omitted for security purposes, but to make the output more readable. see REDACTED is misleading... it should probably be "elided" or "..."
you can run to view full content
that's the crux of this issue - REDACTED is misleading.
Code change:
return nil } <del> var redactedBytes []byte </del> <ins> var ( redactedBytes    []byte dataOmittedBytes []byte ) </ins> // Flatten redacts raw data entries from the config object for a human-readable view. func ShortenConfig(config *Config) {

Related discussion:
text: However, pointed out that other "long" fields are still completely printed in plain text such as client certs, so I think we need to revisit the "readability" argument. What do you think?
they are not... client cert/key are elided in the same way for the same reason: something like "++data+omitted++" could be better If the intent of config view without --raw specified is to be readable (which it currently is), printing many extremely long fields isn't great
Hmm what about: I'm pretty sure that would give us a valid b64 string (8+12=20%4=0) and it would subtly hint that the data isn't a secret.
Only valid characters are . And adding a prefix from the certificate only complicates things. is a suitable usable option. I'll submit a PR to continue the discussion there.