Change:
// excludeCIDRs is a list of CIDR's which the ipvs proxier should not touch // when cleaning up ipvs services. ExcludeCIDRs []string <ins> // strict ARP configure arp_ignore and arp_announce to avoid answering ARP queries // from kube-ipvs0 interface StrictARP bool </ins> } // KubeProxyConntrackConfiguration contains conntrack settings for

Discussion:
/kind bug Kubernetes version: 1.13.1. kube-proxy mode: IPVS. SDN: flannel. We are used services like this: where IP 192.168.10.201 is from our internal /24 network (in particular, our Kube nodes have IPs 192.168.10.1, 192.168.10.2 etc). Note that there is no conflicts between real IPs and virtual external ones. Before updating from 1.12.3 to 1.13.1, it worked correctly. In partucular these IPs was accessible from other non-Kubernetes hosts from 192.168.10.0/24 network. After update kube-proxy to 1.13.1, this feature stops working: non-Kubernetes hosts cannot resolve service IPs via ARP: On Kubernetes hosts it still working, because not requires ARP resolving. If we reset sysctl parameter arpignore on Kube hosts, all works normally even from non-Kube hosts. Quck debugging shows that problem is in the commit . Please make this behaviour opt-out. (Now we resets arpignore to 0 via cron every 1 min.)
/sig network
Having the same problem on Kubernetes 1.13.2 btw Thanks for a workaround!
Same problem, fix this please!!!
This is similar to We've had a discussion there on how to best implement a solution but haven't decided yet I'm in favor of flag which defaults to the previous behavior to avoid breaking existing setups (alternatives are cleaner but involve creating an additional arptable rule per service which I think we should avoid) what do you think?
OK

What happened: After PR: , we noticed communications issue in our pods. We use the Lyft CNI plugin () which relies on two interfaces in pod: an IPVLAN interface connecting to the AWS VPC which has a VPC IP an Unnumbered point-to-point interface (veth) connecting to the host and that does not have an IP (it "borrows" the one from the IPVLAN interface) On the host side, there is a veth interface which also doesn't have an IP (it borrows one from the main interface). With the latest versions of kube-proxy communication between host and pods break What you expected to happen: Communication still works How to reproduce it (as minimally and precisely as possible): Use the Lyft CNI plugin Deploy kube-proxy Try to ping the host from a pod (or a pod from the host) Anything else we need to know?: The new sysctls create 2 issues: containers started after kube-proxy inherit so they will not answer ARP queries on the veth interface the veth on the host side also does not have an IP so it will no answer ARP queries from the pods I know that our setup is pretty uncommon but assuming that the veth on the host side will have an IP seems dangerous. Some discussions on the PTP plugin for instance suggest not using IP for the host interface: containernetworking/plugins Possible fixes: create a flag to enable ARP filtering (I think to avoid breaking existing setups, we should disable it by default but I'm fine with enabling it by default as long as we document the solution) use arptables to filter ARP queries for clusterIPs (): a bit more work because we need to do it for all clusterIPs (there may be a more efficient way to do it but I could not make it work by filtering on the kube-ipvs interface) and this requires the arptables binary. We could test for the existence of the binary and log a warning if it is missing (similar to behavior with conntrack) Environment: Kubernetes version (use ): kube-proxy OS (e.g. from /etc/os-release): Kernel (e.g. ): /area ipvs /sig network /kind bug /assign ping
I can say only that solution with arptables working fine with MetalLB.
Oh, really? Glad you confirm that.

But I would like to make sure that we won't create too many arptables rules. Can ipset works with arptables?
It doesn't look like arptables works with ipset
Unfortunately doesn't support ipsets, but new do. (not fully implemented yet) Eg we can create ipset for arp: But current implementation is not allowing to use it for arp rules for now: Will return: I guess because there is still no type specified for (destination ip) and (source ip) BTW: nft does not allow to create set of integer components: But there is a way for specify multiple IPs in one rule, eg: Where is IPv4 addresses in decimal format More info here:
Another option would be to use the and filter all queries but this flag is not in kube-proxy today
Bad idea, it will handle but not - which can be really differ
can you explain more detail why Lyft CNI can not assign IPs to the right interfaces for avoid this kind of issues?
Yes you are right. bad idea It could definitely add IPs to these interfaces but it's not the choice that was made. More details here: ("Unnumbered point-to-point interface"). I could probably PR a fix there. I'm a bit worried that some other CNI plugins could break here though
We are about to deploy some 1.13 clusters and can't use kube-proxy 1.13 as is We could start with a custom build but I assume others will be impacted The simplest way forward for now is probably to use a flag to control this behavior. I think the safest is probably to disable this by default to avoid the issue for people upgrading but if you think it is really important we can enable it by default. I know we'd like to avoid adding more flags but other solutions seem a bit more complicated and will most likely result in one iptables rule per service. What do you think ?
Just FYI, we have another CNI plugin that is affected by this.
We addressed the issue by assigning a link-local address 169.254.1.1/32 to the host-side veth in . So this issue is not critical at least for our case.
Good news everyone, new nft supports setting ip address for arp filters : that's makes possible for using sets