Change:
<!-- Please use this template while reporting a bug and provide as much info as possible. Not doing so may result in your bug not being addressed in a timely manner. Thanks! If the matter is security related, please disclose it privately via --What happened: kube-proxy don’t clean ipvs rules with . What you expected to happen: kube-proxy clean ipvs rules with by default. How to reproduce it (as minimally and precisely as possible): Run kube-proxy in ipvs mode, and clean rules with . Anything else we need to know?: Based on , should cleanup all rules by default, including ipvs rules. And also retains functionality. Environment: Kubernetes version (use ): v1.20.0-bate.0 Cloud provider or hardware configuration: OS (e.g: ): centos7 Kernel (e.g. ): Install tools: Network plugin and version (if this is a network-related bug): Others:
/sig network
it seems that this flag entered deprecation, so I think that recovering functionaly is not the right approach you were involved in the discussion with this flag in , can you take a look at this?
thanks for your reply. The was in , to avoid removing IPVS rules which are not created by kube-proxy. According to , It seems that deleting ipvs rules by default is expected, but it is not so. I think that keeping --cleanup-ipvs is meaningful, unless kube-proxy can only delete the IPVS rules created by itself.
/assign
sorr for the late answer , but the flag must have the same functionality than , actually I can see that it has implemented. In case that the is not working we should fix it, but not recover the deprecated flag
Agree
sorry if I'm missing something. I just try to make the function of as described, not to recover the flag. The problem I see is: The can't flush IPVS. Because the IPVS flush process is still controlled by . Beforce , it was assigned by flag, but now it is not assigned, always false. currently does not have any functionality, just stored by . So, should we keep the function of temporarily by assigning it to ,or just make always flushing IPVS regardless of the ? I prefer the former because the flag is not completely abandoned :)
it seems that is what the flag description says, you have to use to flush the ipvs entries if that is the case it seems that we are ok, right? no changes needed
In fact, can't flush ipvs entries. The reason is what I described above.
I´m sorry, I think I got it know, the flag is not exposed, but the option is there and initialized to true, this is what I see so, it seems that flush IPVS by default
yes, should flush IPVS by default, but it doesn’t actually
... and finally I got it, so sorry :bow: the ProxyServer is not being instantiated with the flag, despite is set :facepalm: you are more familiar with the IPVS implementation than me, I personally think that, if the flag deprecation has started, we should make that always flush IPVS and totally remove ... it seems aligned with the other proxy implementations
/triage accepted
Actually, after doing some code archaeology, this was the expectation per And the flag clearly mentions that --cleanup must flush IPVS entries too I think that we can go ahead and remove the --cleanup-ipvs flag and make --cleanup to flush ipvs too, right now is a mess and is not working at all , are you going to submit a PR?
thanks for digging up the history on this. That lines up with my recollection and the KEP around kube-proxy's clean up logic
OK, I will create a PR to remove the flag

Discussion:
WriteConfigTo string // CleanupAndExit, when true, makes the proxy server clean up iptables and ipvs rules, then exit. CleanupAndExit bool <del> // CleanupIPVS, when true, makes the proxy server clean up ipvs rules before running. CleanupIPVS bool </del> // WindowsService should be set to true if kube-proxy is running as a service on Windows. // Its corresponding flag only gets registered in Windows builds WindowsService bool

"A string slice of values which specify the addresses to use for NodePorts. Values may be valid IP blocks (e.g. 1.2.3.0/24, 1.2.3.4/32). The default empty string slice ([]) means to use all local addresses.") fs.BoolVar(&o.CleanupAndExit, "cleanup", o.CleanupAndExit, "If true cleanup iptables and ipvs rules and exit.") <del> fs.BoolVar(&o.CleanupIPVS, "cleanup-ipvs", o.CleanupIPVS, "If true and --cleanup is specified, kube-proxy will also flush IPVS rules, in addition to normal cleanup.") fs.MarkDeprecated("cleanup-ipvs", "In a future release, running --cleanup will always flush IPVS rules") </del> fs.Var(utilflag.IPVar{Val: &o.config.BindAddress}, "bind-address", "The IP address for the proxy server to serve on (set to '0.0.0.0' for all IPv4 interfaces and '::' for all IPv6 interfaces)") fs.Var(utilflag.IPPortVar{Val: &o.config.HealthzBindAddress}, "healthz-bind-address", "The IP address with port for the health check server to serve on (set to '0.0.0.0:10256' for all IPv4 interfaces and '[::]:10256' for all IPv6 interfaces). Set empty to disable.")

config:      new(kubeproxyconfig.KubeProxyConfiguration), healthzPort: ports.ProxyHealthzPort, metricsPort: ports.ProxyStatusPort, <del> CleanupIPVS: true, </del> errCh:       make(chan error), } }

Conntracker            Conntracker // if nil, ignored ProxyMode              string NodeRef                *v1.ObjectReference <del> CleanupIPVS            bool </del> MetricsBindAddress     string BindAddressHardFail    bool EnableProfiling        bool

for _, ipt := range ipts { encounteredError = userspace.CleanupLeftovers(ipt) || encounteredError encounteredError = iptables.CleanupLeftovers(ipt) || encounteredError <del> encounteredError = ipvs.CleanupLeftovers(s.IpvsInterface, ipt, s.IpsetInterface, s.CleanupIPVS) || encounteredError </del> <ins> encounteredError = ipvs.CleanupLeftovers(s.IpvsInterface, ipt, s.IpsetInterface) || encounteredError </ins> } if encounteredError { return errors.New("encountered an error while tearing down rules")

} // CleanupLeftovers clean up all ipvs and iptables rules created by ipvs Proxier. <del> func CleanupLeftovers(ipvs utilipvs.Interface, ipt utiliptables.Interface, ipset utilipset.Interface, cleanupIPVS bool) (encounteredError bool) { if cleanupIPVS { // Return immediately when ipvs interface is nil - Probably initialization failed in somewhere. if ipvs == nil { return true } encounteredError = false </del> <ins> func CleanupLeftovers(ipvs utilipvs.Interface, ipt utiliptables.Interface, ipset utilipset.Interface) (encounteredError bool) { // Clear all ipvs rules if ipvs != nil { </ins> err := ipvs.Flush() if err != nil { klog.Errorf("Error flushing IPVS rules: %v", err)

fp.syncProxyRules() // test cleanup left over <del> if CleanupLeftovers(ipvs, ipt, ipset, true) { </del> <ins> if CleanupLeftovers(ipvs, ipt, ipset) { </ins> t.Errorf("Cleanup leftovers failed") } }