Change:
fs.BoolVar(&o.CleanupAndExit, "cleanup-iptables", o.CleanupAndExit, "If true cleanup iptables and ipvs rules and exit.") fs.MarkDeprecated("cleanup-iptables", "This flag is replaced by --cleanup.") fs.BoolVar(&o.CleanupAndExit, "cleanup", o.CleanupAndExit, "If true cleanup iptables and ipvs rules and exit.") <del> fs.BoolVar(&o.CleanupIPVS, "cleanup-ipvs", o.CleanupIPVS, "If true make kube-proxy cleanup ipvs rules before running.  Default is true") </del> <ins> fs.BoolVar(&o.CleanupIPVS, "cleanup-ipvs", o.CleanupIPVS, "If true and --cleanup is specified, kube-proxy will also flush IPVS rules, in addition to normal cleanup.") </ins> // All flags below here are deprecated and will eventually be removed.

Discussion:
What would you like to be : Remove any logic in kube-proxy that tries to automatically clean up proxy rules based on the proxy mode. Only attempt to clean up if is set. Why is this needed: This is a continuation of a discussion from There's a lot of complexity that comes with switching the proxy mode on kube-proxy because it tries to delete proxy rules created from other proxy modes. This has a lot of side-effects and we should be encouraging users to reboot thier nodes along with switching proxy modes. One example of this can be found
/sig network
I can take this on - does this issue have/need a KEP?
I think not but will defer to
I think a micro-KEP is appropriate. Specifically to talk about compatibility with regards to flags and to talk about docs (not sure which, but we have to write this down :)
also to talk about --cleanup (I think it should probably clean up its own mode, not the others -- or maybe I am wrong, please think about the right thing here :)
/assign (Yay, I can do this now!)
/priority important-soon

What happened: When restarting kube-proxy in iptables mode, there will be several seconds of timeouts because it flushes the KUBE-SERVICES nat chain and several others. What you expected to happen: Restarting kube-proxy should not impact traffic in any way. How to reproduce it (as minimally and precisely as possible): an HTTP server behind a ClusterIP service an http client in a loop making requests to the HTTP server -15 the kube-proxy process on the client node timeout for a few seconds until kube-proxy re-syncs the iptables rules Anything else we need to know?: Running with makes it pretty clear what's happening: it says it's cleaning up inactive rules, but these are crucial to the iptables proxier. You can trivially reproduce by just running: Environment: Kubernetes version (use ): 1.10.11 Cloud provider or hardware configuration: amazon OS (e.g: ): custom Kernel (e.g. ): 4.14.77-hs623.el6.x86_64 Install tools: custom Others: iptables 1.6.2 We're running kube-proxy 1.10.11, but I've confirmed this issue with kube-proxy 1.13.4 too
Reiterating the mentions to trigger a notification: <detailsIn response to : Instructions for interacting with me using PR comments are available . If you have questions or suggestions related to my behavior, please file an issue against the repository. </details
Possibly a bug in the proxy. It flushes iptabes if the proxier is able to use IPVS which seems like the wrong behavior to me if there are shared tables/chains between the IPVS and iptables proxier. One workaround I can think of right now is to unload the ipvs kernel module to skip this check.
/assign
oh, that's an interesting workaround, I'll give that a shot, thanks!
np, I'll try to work on the bug fix in the meantime!
/triage unresolved
note that unloading the ipvs module is not actually enough of a workaround because kube-proxy will load the kernel module if it sees that it is available. If I unload the ipvs module and then rename to something else in then it does the right thing. Dug slightly further: the way it probes for modules is by actually running modprobe.

So the simplest hack that allows us to keep ipvs loaded for other things on the system is to put a modprobe script on its PATH that just always exits 1 for the ip_vs modules. (We can stop mounting the modules dir into the kube-proxy container in our kubernetes clusters, but we also run kube-proxy on non-kubernetes nodes via the init system, so the PATH hack works well enough there). This should hold us over fine until we get to a version with your fix in it.
Good to know, thanks for sharing!
/assign
Quick update on this issue from today's SIG Network call: we're going to try to get rid of the automatic proxy clean up altogether for v1.14.1 since this is considered a bug. is working on the KEP & implementation.