Change:
From another issue, said: DockerManager.SyncPod() calls the network Plugin and then sets hairpin mode unilaterally. This is wrong. We should either pass the bool down to here or else we should get rid of the default no-op plugin and use the non-existence of a plugin as a flag to control this. This should be a relatively easy one for people who want to contribute but don't know where to start.
I volunteer myself to do this. Actually I have done something about kubernetes network plugin so guess I can do this. I will start to do this when I am on the office tomorrow.
Awesome. I'd advise starting with a sketch so we can all make sure it is a good approach, before jumping in with both feet. , TonyAdo wrote:
ok, I will start with a sketch as soon as possible.
I thought about this for a while: if we pass the bool down to here, of course we can check this flag, if it is set, we should set veth hairpin mode, but problem is that if user set this flag, but still use a custom network plugin just like you mentioned the ovs plugin, I am afraid we will still enable hairpin mode in all interfaces on this host, since we cannot find a peer interface. Is that correct?
Yeah, I think the better fix is to use the existence of a network plugin as an indicator. It sort of makes --configure-cbr0 redundant , TonyAdo wrote:
ok, I will try to get rid of the default no-op plugin and use the non-existence of a plugin as a flag to control this.
I decided to use the existence of a network plugin as an indicator, also I will get ride of the no-op network plugin, I am not going to add new flag to kubelet structure, if the user don't specify network plugin name, the networkPlugin within kubelet will be which just means non-existence of a plugin, and we have to check the existence in the docker manager to avoid nil pointer exception also for setting, I think if a custom network plugin exists, kubelet should not do this. How about this? Should I open a PR?
fyi, saw a hairpin pr today that might be related. ymmv.
what do you think about this?
What you described sounds right to me. I think the only way to see if there is some detail I am missing is to try it and see if anything pops up that would make this not work. , TonyAdo wrote:
I need to check which part is responsible of what, but what if the hairpin package was just a utility package that should be called by the plugin if it needs to set this up? As long as it is single and simple function call, doing so would avoid adding extra logic here and fix the issue. This is valid only if the aim is to have a "linux bridge plugin", an "OVS plugin", etc.
actually kubelet will call after get called, I think to make the network plugin can call hairpin package is not that easy, of course, I agree that network plugin should setup hairpin mode if it really needs, kubelet should not set hairpin mode unilaterally. Besides, kubelet is not going to provide an "linux bridge plugin" or "OVS plugin".
then the validity conditions of my comment are not met, so I see any simpler way. Thanks for checking! :)
still what do you think about disable hairpin completely and leave this to network plugin?
or if there is a custom network plugin, we don't setup hairpin mode, only do this in default.
Completely? It will break my use case (and anyone using Apache Kafka).
Only by default is what I understood from the previous comments, and LGTM.
Really sorry about that, but I really want to know are you using a kubelet network plugin in your use case? If no, I think we will still setup hairpin, as in default we will do this.
I don't use any plugin, so it is ok if hairpin is set up by default.
ok, I think this will not break your use case.
It's not really the network plugin that needs hairpinmode though, it's the pure-iptables-based kube-proxy code. It's possible the proxying code could change again in the future in such a way that hairpinmode wouldn't be needed any more. So it seems wrong to put that code into the plugin. makes it so that we set hairpinmode on the veth if it's attached to a bridge, but just silently do nothing if it's not. (Yes, we currently set hairpinmode even if you're using the userspace proxier, but AFAIK it's not needed in that case.)
IMHO... The proxy has the responsibility of service objects, while the kubelet is responsible of pod objects, including their networking. Also, the responsibility of hairpin must be at the same place as the code that creates the network interface, otherwise you have code managing a single object (the pod interface) spread over multiple programs, which will eventually lead to complex problems and a system harder to understand even for the end-user. Thus, it doesn't seems wrong to me that the kubelet does whatever is required to make the pod work in the environment of the cluster. While its true that the hairpin requirement is only for the proxy in iptables mode, a switch to this mode can happen at any time in the life of the cluster. And when it happens, its probably because the userland proxy becomes unresponsive, which means in a situation where the administrator is in a hurry to find a solution. Currently, you just change the proxy mode and restart the proxies. There's little chance the administrator will think about hairpin or restarting the kubelet in this case, so the kubelet should be reacting to this; which means the proxy and the kubelet of a given host should be able to communicate (AFAIK, this doesn't exist now). And all this is currently only to avoid setting up hairpin on the pod's port of standard Linux bridge.
Feel sure I'm missing something but... The CNI bridge plugin now has a flag, so kubenet can use that. Having done that, we can set false when the plugin name is non-blank.
Is this fixed now?
I apologise for leaving a somewhat vague answer, but I have been in that code recently. I don't think it's fixed.
, Bryan Boreham wrote:
Thanks for the PR. CNI Bridge plugin is expecting . Not . Hence the bridge plugin did not recognize it.
EDIT 2016-12-09: The chain below was broken by , so the issue can be closed again (remain closed) I suggest this should be re-opened. The chain of calls is: sets to , e.g. when using CNI calls something goes wrong and it sets the mode on everything. An example of "something goes wrong":

Discussion:
kube-apiserver,busybox kube-controller-manager,busybox kube-scheduler,busybox <del> kube-proxy,gcr.io/google_containers/debian-iptables-amd64:v3 </del> <ins> kube-proxy,gcr.io/google_containers/debian-iptables-amd64:v4 </ins> );; "arm") local targets=( kube-apiserver,armel/busybox kube-controller-manager,armel/busybox kube-scheduler,armel/busybox <del> kube-proxy,gcr.io/google_containers/debian-iptables-arm:v3 </del> <ins> kube-proxy,gcr.io/google_containers/debian-iptables-arm:v4 </ins> );; "arm64") local targets=( kube-apiserver,aarch64/busybox kube-controller-manager,aarch64/busybox kube-scheduler,aarch64/busybox <del> kube-proxy,gcr.io/google_containers/debian-iptables-arm64:v3 </del> <ins> kube-proxy,gcr.io/google_containers/debian-iptables-arm64:v4 </ins> );; "ppc64le") local targets=( kube-apiserver,ppc64le/busybox kube-controller-manager,ppc64le/busybox kube-scheduler,ppc64le/busybox <del> kube-proxy,gcr.io/google_containers/debian-iptables-ppc64le:v3 </del> <ins> kube-proxy,gcr.io/google_containers/debian-iptables-ppc64le:v4 </ins> );; esac

fs.StringVar(&s.RktStage1Image, "rkt-stage1-image", s.RktStage1Image, "image to use as stage1. Local paths and http/https URLs are supported. If empty, the 'stage1.aci' in the same directory as '--rkt-path' will be used.") fs.MarkDeprecated("rkt-stage1-image", "Will be removed in a future version. The default stage1 image will be specified by the rkt configurations, see https://github.com/coreos/rkt/blob/master/Documentation/configuration.md for more details.") fs.BoolVar(&s.ConfigureCBR0, "configure-cbr0", s.ConfigureCBR0, "If true, kubelet will configure cbr0 based on Node.Spec.PodCIDR.") <ins> fs.MarkDeprecated("configure-cbr0", "Will be removed in a future version. Please use kubenet or other network plugins.") </ins> fs.StringVar(&s.HairpinMode, "hairpin-mode", s.HairpinMode, "How should the kubelet setup hairpin NAT. This allows endpoints of a Service to loadbalance back to themselves if they should try to access their own Service. Valid values are "promiscuous-bridge", "hairpin-veth" and "none".") fs.BoolVar(&s.BabysitDaemons, "babysit-daemons", s.BabysitDaemons, "If true, the node has babysitter process monitoring docker and kubelet.") fs.MarkDeprecated("babysit-daemons", "Will be removed in a future version.")

fs.Int32Var(&s.KubeAPIBurst, "kube-api-burst", s.KubeAPIBurst, "Burst to use while talking with kubernetes apiserver") fs.BoolVar(&s.SerializeImagePulls, "serialize-image-pulls", s.SerializeImagePulls, "Pull images one at a time. We recommend *not* changing the default value on nodes that run docker daemon with version < 1.9 or an Aufs storage backend. Issue #10959 has more details. [default=true]") fs.BoolVar(&s.ExperimentalFlannelOverlay, "experimental-flannel-overlay", s.ExperimentalFlannelOverlay, "Experimental support for starting the kubelet with the default overlay network (flannel). Assumes flanneld is already running in client mode. [default=false]") <ins> fs.MarkDeprecated("experimental-flannel-overlay", "Will be removed in a future version.") </ins> fs.DurationVar(&s.OutOfDiskTransitionFrequency.Duration, "outofdisk-transition-frequency", s.OutOfDiskTransitionFrequency.Duration, "Duration for which the kubelet has to wait before transitioning out of out-of-disk node condition status. Default: 5m0s") fs.StringVar(&s.NodeIP, "node-ip", s.NodeIP, "IP address of the node. If set, kubelet will use this IP address for the node") fs.BoolVar(&s.EnableCustomMetrics, "enable-custom-metrics", s.EnableCustomMetrics, "Support for gathering custom metrics.")

"addIf": "%s", "isGateway": true, "ipMasq": false, <del> "hairpin": "%t", </del> <ins> "hairpinMode": %t, </ins> "ipam": { "type": "host-local", "subnet": "%s",