Change:
<!-- This form is for bug reports and feature requests ONLY! If you're looking for help check and the . If the matter is security related, please disclose it privately via --Is this a BUG REPORT or FEATURE REQUEST?: /kind bug /sig network /area ipvs /area ipv6 What happened: with IPVS mode creates a interface for binding service IPs. kube-proxy periodically addresses it doesn't know about from the interface. It tries to unbind the IPv6 address over and over. The error is a little vague, but see below. The IP parsing or address deletion code may be a suspect. What you expected to happen: Either ignore the IPv6 address on the interface (its not hurting anything) or remove it correctly. How to reproduce it (as minimally and precisely as possible): Its perfectly natural for a new interface to have an inet6 address by default. If for some reason your interface doesn't have a similar address, you can add one to repro. Anything else we need to know?: Environment: Kubernetes version (use ): v1.12.1 Cloud provider or hardware configuration: AWS, GCP, DigitalOcean, bare-metal, Azure OS (e.g. from /etc/os-release): Container Linux 1855.4.0 Kernel (e.g. ): 4.14.67-coreos Install tools: Typhoon
The error message from is a little vague. Hop on a node to help out with its deletion. This gives a clue about where goes wrong. With IPv4 addresses, leaving off the is permitted (though with a scary warning). Having the wrong prefix size produces the same error. With more verbose logging, we see the mask was stripped before calling 's AddrDel, so the bug lies in kube-proxy's construction of . The mask is dropped . should be (, net.IPNet). However, it looks like much of kube-proxy ipvs deals in string IPs (e.g. 1.2.3.4, 1002:ab8::2:1) so propagating a or adding a mask could be a large change. One might also point to called as the source - it assumes a mask for IPv4 and a mask for IPv6. Neither is neccessarily right for arbitrary addresses that might end up bound on a interface, but for IPv4 happens to work for Kubernetes IPv4 service IPs. for the IPv6 address does not. We could construct a netlink.IPNet without the helper, but by the time we reach , kube-proxy has already thrown away the mask information.
Any relation with this you think? Doesn't seem so since this is in and the other is ipvs targets...
Hm, I think adapting the sync'er for IPv6 might be a different fix. I was working on a fix for the IPv6 cleanup on the interface, lemme see if I can get that to a state it can be posted.
I can only reproduce this fault if I do; This will automatically assign a link-local generated address to the interface and the problem occurs. But the should not be up. By default if I start v1.12.0 it is down. Is some program, e.g network-manager, brings the interface up? Anyway I think it is possible to suspress auto-generated link-local ipv6 addresses. I will try to find the setting.
After creation; should fix this problem. I will try to get time to do a PR.

Discussion:
func (proxier *Proxier) getLegacyBindAddr(activeBindAddrs map[string]bool, currentBindAddrs []string) map[string]bool { legacyAddrs := make(map[string]bool) <ins> isIpv6 := utilnet.IsIPv6(proxier.nodeIP) </ins> for _, addr := range currentBindAddrs { <ins> addrIsIpv6 := utilnet.IsIPv6(net.ParseIP(addr)) if addrIsIpv6 && !isIpv6 || !addrIsIpv6 && isIpv6 { continue } </ins> if _, ok := activeBindAddrs[addr]; !ok { legacyAddrs[addr] = true }

fp.netlinkHandle.EnsureDummyDevice(DefaultDummyDevice) activeBindAddrs := map[string]bool{"1.1.1.1": true, "2.2.2.2": true, "3.3.3.3": true, "4.4.4.4": true} <del> currentBindAddrs := []string{"1.1.1.1", "2.2.2.2", "3.3.3.3", "4.4.4.4", "5.5.5.5", "6.6.6.6"} </del> <ins> // This is ipv4-only so ipv6 addresses should be ignored currentBindAddrs := []string{"1.1.1.1", "2.2.2.2", "3.3.3.3", "4.4.4.4", "5.5.5.5", "6.6.6.6", "fd80::1:2:3", "fd80::1:2:4"} </ins> for i := range currentBindAddrs { fp.netlinkHandle.EnsureAddressBind(currentBindAddrs[i], DefaultDummyDevice) }

} } <del> // Addresses 5.5.5.5 and 6.6.6.6 should not be bound any more </del> <ins> // Addresses 5.5.5.5 and 6.6.6.6 should not be bound any more, but the ipv6 addresses should remain </ins> remainingAddrs, _ := fp.netlinkHandle.ListBindAddress(DefaultDummyDevice) <del> if len(remainingAddrs) != 4 { t.Errorf("Expected number of remaining bound addrs after cleanup to be %v. Got %v", 4, len(remainingAddrs)) </del> <ins> if len(remainingAddrs) != 6 { t.Errorf("Expected number of remaining bound addrs after cleanup to be %v. Got %v", 6, len(remainingAddrs)) </ins> } <del> // check that address "1.1.1.1", "2.2.2.2", "3.3.3.3", "4.4.4.4" are still bound </del> <ins> // check that address "1.1.1.1", "2.2.2.2", "3.3.3.3", "4.4.4.4" are bound, ignore ipv6 addresses </ins> remainingAddrsMap := make(map[string]bool) <del> for i := range remainingAddrs { remainingAddrsMap[remainingAddrs[i]] = true </del> <ins> for _, a := range remainingAddrs { if net.ParseIP(a).To4() == nil { continue } remainingAddrsMap[a] = true } if !reflect.DeepEqual(activeBindAddrs, remainingAddrsMap) { t.Errorf("Expected remainingAddrsMap %v, got %v", activeBindAddrs, remainingAddrsMap) } } func TestCleanLegacyService6(t *testing.T) { ipt := iptablestest.NewFake() ipvs := ipvstest.NewFake() ipset := ipsettest.NewFake(testIPSetVersion) fp := NewFakeProxier(ipt, ipvs, ipset, nil, []string{"3000::/64", "4000::/64"}) fp.nodeIP = net.ParseIP("::1") // All ipvs services that were processed in the latest sync loop. activeServices := map[string]bool{"ipvs0": true, "ipvs1": true} // All ipvs services in the system. currentServices := map[string]*utilipvs.VirtualServer{ // Created by kube-proxy. "ipvs0": { Address:   net.ParseIP("1000::1"), Protocol:  string(v1.ProtocolUDP), Port:      53, Scheduler: "rr", Flags:     utilipvs.FlagHashed, }, // Created by kube-proxy. "ipvs1": { Address:   net.ParseIP("1000::2"), Protocol:  string(v1.ProtocolUDP), Port:      54, Scheduler: "rr", Flags:     utilipvs.FlagHashed, }, // Created by an external party. "ipvs2": { Address:   net.ParseIP("3000::1"), Protocol:  string(v1.ProtocolUDP), Port:      55, Scheduler: "rr", Flags:     utilipvs.FlagHashed, }, // Created by an external party. "ipvs3": { Address:   net.ParseIP("4000::1"), Protocol:  string(v1.ProtocolUDP), Port:      56, Scheduler: "rr", Flags:     utilipvs.FlagHashed, }, // Created by an external party. "ipvs4": { Address:   net.ParseIP("5000::1"), Protocol:  string(v1.ProtocolUDP), Port:      57, Scheduler: "rr", Flags:     utilipvs.FlagHashed, }, // Created by kube-proxy, but now stale. "ipvs5": { Address:   net.ParseIP("1000::6"), Protocol:  string(v1.ProtocolUDP), Port:      58, Scheduler: "rr", Flags:     utilipvs.FlagHashed, }, } for v := range currentServices { fp.ipvs.AddVirtualServer(currentServices[v]) } fp.netlinkHandle.EnsureDummyDevice(DefaultDummyDevice) activeBindAddrs := map[string]bool{"1000::1": true, "1000::2": true, "3000::1": true, "4000::1": true} // This is ipv6-only so ipv4 addresses should be ignored currentBindAddrs := []string{"1000::1", "1000::2", "3000::1", "4000::1", "5000::1", "1000::6", "1.1.1.1", "2.2.2.2"} for i := range currentBindAddrs { fp.netlinkHandle.EnsureAddressBind(currentBindAddrs[i], DefaultDummyDevice) } fp.cleanLegacyService(activeServices, currentServices, map[string]bool{"5000::1": true, "1000::6": true}) // ipvs4 and ipvs5 should have been cleaned. remainingVirtualServers, _ := fp.ipvs.GetVirtualServers() if len(remainingVirtualServers) != 4 { t.Errorf("Expected number of remaining IPVS services after cleanup to be %v. Got %v", 4, len(remainingVirtualServers)) } for _, vs := range remainingVirtualServers { // Checking that ipvs4 and ipvs5 were removed. if vs.Port == 57 { t.Errorf("Expected ipvs4 to be removed after cleanup. It still remains") } if vs.Port == 58 { t.Errorf("Expected ipvs5 to be removed after cleanup. It still remains") } } // Addresses 5000::1 and 1000::6 should not be bound any more, but the ipv4 addresses should remain remainingAddrs, _ := fp.netlinkHandle.ListBindAddress(DefaultDummyDevice) if len(remainingAddrs) != 6 { t.Errorf("Expected number of remaining bound addrs after cleanup to be %v. Got %v", 6, len(remainingAddrs)) } // check that address "1000::1", "1000::2", "3000::1", "4000::1" are still bound, ignore ipv4 addresses remainingAddrsMap := make(map[string]bool) for _, a := range remainingAddrs { if net.ParseIP(a).To4() != nil { continue } remainingAddrsMap[a] = true </ins> } if !reflect.DeepEqual(activeBindAddrs, remainingAddrsMap) { t.Errorf("Expected remainingAddrsMap %v, got %v", activeBindAddrs, remainingAddrsMap)