Issue 1:
This is blocking #7580 (test reboot).

I originally ran into this trying to run the guestbook test after the nodes had all rebooted; here is the log for the failed guestbook test.

I added more logging to help understand what is happening. I added the following logging after this line:

Logf("Call to makeRequestToGuestbook(client, cmd='%s', arg='%s', ns='%s') failed.", cmd, arg, ns)
Logf("\t response: expected '%s', got: '%s'", expectedResponse, res)
Logf("\t error: expected '<nil>', got: '%v'", err)

The first time it's called I get an error that says no endpoints available for "frontend", but then after that, I get the following repeated errors:

INFO: Call to makeRequestToGuestbook(client, cmd='get', arg='', ns='e2e-tests-kubectl-7d744f0d-775f-4fd0-93b2-88135adbc11e') failed.
INFO: response: expected '{"data": ""}', got: '<br />
<b>Fatal error</b>: Uncaught exception 'Predis\Connection\ConnectionException' with message 'php_network_getaddresses: getaddrinfo failed: Name or service not known [tcp://redis-slave:6379]' in /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php:141
Stack trace:
#0 /vendor/predis/predis/lib/Predis/Connection/StreamConnection.php(96): Predis\Connection\AbstractConnection-&gt;onConnectionError('php_network_get...', 0)
#1 /vendor/predis/predis/lib/Predis/Connection/StreamConnection.php(70): Predis\Connection\StreamConnection-&gt;tcpStreamInitializer(Object(Predis\Connection\ConnectionParameters))
#2 /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php(96): Predis\Connection\StreamConnection-&gt;createResource()
#3 /vendor/predis/predis/lib/Predis/Connection/StreamConnection.php(144): Predis\Connection\AbstractConnection-&gt;connect()
#4 /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php(181): Predis\Connection\StreamConnection-&gt;connect()
#5 /vendor/predis/predis/lib/Predis/Connection/StreamConnectio in <b>/vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php</b> on line <b>141</b><br />
'
INFO: error: expected '<nil>', got: '<nil>'

Just to make that more readable, the error before the stack trace is: Uncaught exception 'Predis\Connection\ConnectionException' with message 'php_network_getaddresses: getaddrinfo failed: Name or service not known [tcp://redis-slave:6379]' in /vendor/predis/predis/lib/Predis/Connection/AbstractConnection.php:141

Any tips or help greatly appreciated. I'm not sure where in the tcp / services / php / redis stack this is going wrong.

Apologies in advance for the large cc:

+cc @roberthbailey @zmerlynn @quinton-hoole @ixdy

Issue 2:
I believe I'm seeing an issue where secrets are not available after server reboot.

The steps to reproduce are below. Note that the following steps have been run on v.0.16.2:

Create the following simple secret:

# From kubernetes/examples/secrets/secret.yaml
apiVersion: v1beta3
kind: Secret
metadata:
name: test-secret
data:
data-1: dmFsdWUtMQ0K
data-2: dmFsdWUtMg0KDQo=

Then create the following pod:

apiVersion: v1beta3
kind: Pod
metadata:
name: test
spec:
containers:
- name: test
image: centos:centos7.0.1406
command: [ "bash", "-c", "while [ true ]; do cat /etc/test-secret/data-1; sleep 5; done" ]
volumeMounts:
- mountPath: /etc/test-secret
name: test-secret
readOnly: true
volumes:
- name: test-secret
secret:
secretName: test-secret

After starting the pod you can see that it is correctly slowing the data-1 value from the secret

kubectl log test
2015-05-08T16:06:37.433196624Z value-1

Now reboot the server running the pod. After reboot I see the following in the log output:

kubectl log test
2015-05-08T16:08:53.172253741Z cat: /etc/test-secret/data-1: No such file or directory

That seems to indicate to me that the pod is re-starting after the reboot but that it is unable to get the secret. Note that after the reboot manually stopping ( kubectl stop pod test ) and re-starting the pod shows the expected results.

My expectation of the behavior is that if the server a pod is running on reboots that the pod will re-start with the same secret it had when it started previously. If the secret had been updated since the pod had been originally started I would expect the original secret to be available.