Change:
cAdvisor provides potentially sensitive data and there's currently no way to block access with anything other than iptables.
What type of cluster are you running (GKE, or what instructions did you use to setup your cluster?) A short term solution might be to try adding flag to kubelet and restarting the kubelet service and see if the sensitive information is no longer leaked. Doing this may the UI and heapster which expect to be able to poll the cadvisor port. I think probably knows if there is a longer term solution planned, such as: only serve on specified interface. if this is localhost, then it breaks the ui, but maybe that is okay. maybe has thought about this. require authentication/authorization to connect to cAdvisor port.
Self deployed cluster, the kubelet is started via
We are working on improving the kubelet APIs which will subsume cadvisor APIs. Until then setting cadvisor port to '0' is probably the way to go.
Is there an issue and target date for doing this?
Filed . The current plan is to complete this for v1.1. cc
Do you mean the read-only kubelet port ( by default IIRC)? If so, then we really need to disable this & provide authorisation options on the kubelet to be able to retrieve stats fro the kubelet via the normal port with a restricted token/cert/auth options. We've discussed this for OpenShift (openshift/origin) too.
Sorry I didn't realise that cadvisor was bound to port 4194 as well. Thought this was all embedded inside the kubelet process without exposing any ports.
Any reason why we start up cadvisor http server at all? Seems unnecessary now kubernetes embeds cadvisor & exposes via & ?
does not expose all the containers (, , etc). is working on introducing a better stats API in kubelet () that should help us shut down cadvisor's http server in kubelet.
Couldn't we just add cadvisor as a handler in the kubelet anyway? No need for separate http server & would mean access to raw data is available if anyone wants it.
Indeed that is an option. But we need a better stats API in kubelet anyways for reasons mentioned in .
Absolutely agree but thinking quick win.
+1 on a quick win. We just disabled it by default instead: Which reminds me it would be great to have so we could make it easy to turn on/off stuff like this w/o rewriting the entire command-line in a service file.
cc
it's not 100% clear to me whether or not made it safe to use , could you please clarify? On a fresh k8s 1.6.4 cluster still exposes cAdvisor to the world: the file you referenced was deleted as part of coreos/coreos-overlay the documentation @ (CoreOS docs) doesn't mention cAdvisor should be disabled either. Another reason for my question to Vish.
Ahh. That's a bug that the new component config based scheme introduced. being is a valid setting.
I think we can change this. The KubeletConfiguration type is still technically alpha, so it should be ok to make this field a pointer and keep the current default when it's not provided (but allow 0). This won't affect anyone who presently omits the flag and expects port 4194. Theoretically people could be explicitly passing 0 and expecting 4194... I guess... but that's sort of an insane case, and I'd consider that behavior a bug.
Thanks for clarifying guys :+1: The original problem remains though: enabling the cAdvisor port makes the kubelet listen on all interfaces.
Yeah. We can get cAdvisor to bind to the interface that kubelet uses. cc , Antoine Cotten <:
/assign

Discussion:
} if kubeDeps.CAdvisorInterface == nil { <del> kubeDeps.CAdvisorInterface, err = cadvisor.New(uint(s.CAdvisorPort), s.ContainerRuntime, s.RootDirectory) </del> <ins> kubeDeps.CAdvisorInterface, err = cadvisor.New(s.Address, uint(s.CAdvisorPort), s.ContainerRuntime, s.RootDirectory) </ins> if err != nil { return err }

import ( "flag" "fmt" <ins> "net" </ins> "net/http" <ins> "strconv" </ins> "time" "github.com/golang/glog"

} // New creates a cAdvisor and exports its API on the specified port if port > 0. <del> func New(port uint, runtime string, rootPath string) (Interface, error) { </del> <ins> func New(address string, port uint, runtime string, rootPath string) (Interface, error) { </ins> sysFs := sysfs.NewRealSysFs() // Create and start the cAdvisor container manager.

Manager:  m, } <del> err = cadvisorClient.exportHTTP(port) </del> <ins> err = cadvisorClient.exportHTTP(address, port) </ins> if err != nil { return nil, err }

return cc.Manager.Start() } <del> func (cc *cadvisorClient) exportHTTP(port uint) error { </del> <ins> func (cc *cadvisorClient) exportHTTP(address string, port uint) error { </ins> // Register the handlers regardless as this registers the prometheus // collector properly. mux := http.NewServeMux()

// Only start the http server if port > 0 if port > 0 { serv := &http.Server{ <del> Addr:    fmt.Sprintf(":%d", port), </del> <ins> Addr:    net.JoinHostPort(address, strconv.Itoa(int(port))), </ins> Handler: mux, }

var _ Interface = new(cadvisorUnsupported) <del> func New(port uint, runtime string, rootPath string) (Interface, error) { </del> <ins> func New(address string, port uint, runtime string, rootPath string) (Interface, error) { </ins> return &cadvisorUnsupported{}, nil }

var _ Interface = new(cadvisorClient) // New creates a cAdvisor and exports its API on the specified port if port > 0. <del> func New(port uint, runtime string, rootPath string) (Interface, error) { </del> <ins> func New(address string, port uint, runtime string, rootPath string) (Interface, error) { </ins> return &cadvisorClient{}, nil }

} // Setup cadvisor to check the container environment <del> c, err := cadvisor.New(0 /*don't start the http server*/, "docker", "/var/lib/kubelet") </del> <ins> c, err := cadvisor.New("", 0 /*don't start the http server*/, "docker", "/var/lib/kubelet") </ins> if err != nil { return printError("Container Runtime Check: %s Could not start cadvisor %v", failed, err) }