Change:
flags+=" --logtostderr" flags+=" --system-log-monitors=${km_config},${dm_config}" flags+=" --apiserver-override=https://${KUBERNETES_MASTER_NAME}?inClusterConfig=false&auth=/var/lib/node-problem-detector/kubeconfig" <ins> local -r npd_port=${NODE_PROBLEM_DETECTOR_PORT:-20256} flags+=" --port=${npd_port}" </ins> # Write the systemd service file for node problem detector. cat <<EOF >/etc/systemd/system/node-problem-detector.service

Discussion:
Is this a BUG REPORT or FEATURE REQUEST?: /kind bug What happened: Since k8s v1.7, kube-proxy starts listening on 0.0.0.0: to serve default /healthz traffic from L4 external/internal loadbalancer (). When node-problem-detector runs in  mode, it is a real daemon on node. By default, it will bind 127.0.0.1: for the use of node problem detector server (). This breaks kube-proxy's default healthz server: What you expected to happen: kube-proxy and node-problem-detector should use different ports. How to reproduce it (as minimally and precisely as possible): Create a 1.7 cluster with node-problem-detector enabled with standalone mode. Anything else we need to know?: /assign /sig network cc Environment: Kubernetes version (use ): Cloud provider or hardware configuration*: OS (e.g. from /etc/os-release): Kernel (e.g. ): Install tools: Others:
/cc
I am facing this issue on Kubernetes 1.8.9 (After upgrading via Kops from 1.7.11). Is the fix available for 1.8.9?
This should not longer be an issue for 1.7.2+ cluster. Are you observing the "bind: address already in use" error from kube-proxy logs? Could you confirm what component is occupying the same port?
Yes. In the describe nodes, I can see an event with message Failed to start node healthz on 0.0.0.0:: listen tcp 0.0.0.0:: bind: address already in use. K8s Version: v1.8.9 Installed with Kops 1.8.1 on AWS OS Image: Debian GNU/Linux 9 (stretch) Kernel Version: 4.9.0-6-amd64 Docker Run Time: docker://1.13.1
Could you confirm what component is occupying port ? Have you tried on the node?
didn't find much in netstat -l For port , I could only see this. tcp6 0 0 [::]: [::]: LISTEN
Sorry should have included the program flag, try ?
This shows that is listening by kube-proxy But, the node events and kube proxy logs shows the same error log.

This is happening for metrics-server as well. There are lot of retries on the kube-proxy pod though. Maybe it is trying to collide with it's own process. This is happening on only one of the nodes in our cluster. Tried restarting / deleting the node but no use.
Just another observation. This is happening not just for the metrics server / healthz, it's happening for all the node ports that are exposed.
Sounds like you are having a different issue other than just port collision. It would be better to file a new bug (with sig-node?).
Facing a similar issue..
I am facing the same issue here. I have a cluster of 10 nodes, 1 master, 1 etcd. Services on 6 nodes (out of 10) can not "reach"/"be reached from" other containers. However, the other 4 nodes work perfectly and if I put pods (using nodeSelector) on them they can "reach"/"be reached from" other containers placed on the other nodes (4 nodes). I checked the describe node, kube-proxy and calico logs for the 6 nodes: describe node: kube-proxy: calico-node: Does anybody know what causes this issue and how I should resolve this?
That is not the same issue. The issue reported here was specifically for the port collision between kube-proxy and node-problem-detector on port , which was a configuration mistake. What your encountered seems more like (opened by
thanks! I sent my comments (updated) in .
I am suffering from the same problem Kubernetes 1.10.3 on AWS using kops
the same problem, I am using EKS (Kubernetes 1.10.3)