Issue 1:
FEATURE REQUEST: Per destination bandwidth shaping

While it's now possible to add a per-pod bandwidth limit with annotations kubernetes.io/{ingress,egress}-bandwidth this limit is set on the pod regardless of the trafic's origin/destination.

In our use-case we would like to host containers for other companies in our Kubernetes cluster. While we would like the local<->local bandwidth to be unlimited, we want to limit the internet-egress bandwidth of theses pods to avoid paying egress charges on them.

Another option would be per-deployment internet-egress accounting but I couldn't find anything on this.

What keywords did you search in Kubernetes issues before filing this one? (If you have found any duplicates, you should instead reply there.):

bandwidth:

#11965

#2856

shaper:

#11988

#23740

Kubernetes version (use kubectl version): 1.3.5

Environment:

Cloud provider or hardware configuration: Google GKE

OS (e.g. from /etc/os-release): Debian Wheezy

Kernel (e.g. uname -a): 3.16.0-4-amd64

Issue 2:
Hi. I'm writing this partially because I want to start a discussion and to leave some trails for others potentially affected.

I started my journey with Kubernetes just a few days ago. I downloaded the 1.0.6 release (now I know this a only a pre-release and I should've downloaded the 1.0.4 release but this doesn't seem to be related to my particular issue).

I brought up a tiny k8s cluster with the following config:

export KUBERNETES_PROVIDER=aws
export KUBE_AWS_ZONE=eu-west-1a
export NUM_MINIONS=3
export MINION_SIZE=t2.micro
export AWS_S3_REGION=eu-west-1
export AWS_S3_BUCKET=xxx
export KUBE_AWS_INSTANCE_PREFIX=xxx

My cluster was created properly and after a bit of playing I left it in a good shape.

The next day I wanted to investigate some DNS related issue on k8s nodes so I logged in on one of the minions. I noticed *** System restart required *** in the MOTD. I ignored it for a moment and investigated the DNS matters (some nslookups, nothing dangerous). Then I rebooted the machine.

Once the machine was up again I found out that kubectl reported it as NotReady. I started digging and investigating. It took me some time to find out that the Docker service on the node was not running:

$ sudo docker info
FATA[0001] Get http:///var/run/docker.sock/v1.18/info: read unix /var/run/docker.sock: connection reset by peer. Are you trying to connect to a TLS-enabled daemon without TLS?

$ sudo journalctl -u docker -f
-- Logs begin at Fri 2015-09-18 08:06:53 UTC. --
Sep 18 08:09:49 ip-172-20-0-149 systemd[1]: docker.service failed.
Sep 18 08:09:51 ip-172-20-0-149 systemd[1]: docker.service holdoff time over, scheduling restart.
Sep 18 08:09:51 ip-172-20-0-149 systemd[1]: Started Docker Application Container Engine.
Sep 18 08:09:51 ip-172-20-0-149 systemd[1]: Starting Docker Application Container Engine...
Sep 18 08:09:51 ip-172-20-0-149 docker[3397]: time="2015-09-18T08:09:51Z" level=info msg="+job serveapi(fd://)"
Sep 18 08:09:51 ip-172-20-0-149 docker[3397]: time="2015-09-18T08:09:51Z" level=info msg="Listening for HTTP on fd ()"
Sep 18 08:09:51 ip-172-20-0-149 docker[3397]: time="2015-09-18T08:09:51Z" level=fatal msg="Shutting down daemon due to errors: error intializing graphdriver: driver not supported"
Sep 18 08:09:51 ip-172-20-0-149 systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Sep 18 08:09:51 ip-172-20-0-149 systemd[1]: Unit docker.service entered failed state.
Sep 18 08:09:51 ip-172-20-0-149 systemd[1]: docker.service failed.

Further investigation led me to a conclusion that the problem arose from the fact that before the reboot the kernel was upgraded to 3.19.0-28, which no longer by default supports AUFS module which Docker uses as its storage driver.

Looks like the solution is to install the linux-image-extra-virtual, as concluded from moby/moby#10859 and other sources, and reboot the node. After reboot the node got back to the Ready state.

Moreover, I guess an upgrade to a newer Docker version, using the official install command curl -sSL https://get.docker.com/ | sh would solve the problem as the most recent install script already installs linux-image-extra-virtual as per moby/moby#10860, although I'm not sure how Docker was installed on the k8s node, so I don't know if it's safe to upgrade it this way.

So I have a number of questions:

should k8s provide a workaround / fix for this particular issue?

or is it left to a cluster admin as it's an issue of a particular provider/env/configuration?

why is k8s using Docker v1.6.0 which is pretty old?

is it safe to use k8s in production on a setup in which Docker uses AUFS storage driver which seems to be deprecated by the Ubuntu team and thus dropped from the main kernel image?

Note that I haven't done anything unusual, just installed fresh k8s on AWS using the default Ubuntu release. Hence I believe it's important to fix it somehow in k8s.

Possible duplicate: #9779.