Repository: kubernetes
Issue: Disable restart of containers in multi-container pods 
Labels: sig/node,kind/feature,lifecycle/frozen,needs-triage

<!-- Thanks for filing an issue! Before hitting the button, please answer these questions.-->

**Is this a request for help?** (If yes, you should use our troubleshooting guide and community support channels, see http://kubernetes.io/docs/troubleshooting/.): No


**What keywords did you search in Kubernetes issues before filing this one?** (If you have found any duplicates, you should instead reply there.): multi, restart, liveness, restart

---

**Is this a BUG REPORT or FEATURE REQUEST?** (choose one): Feature request

<!--
If this is a BUG REPORT, please:
  - Fill in as much of the template below as you can.  If you leave out
    information, we can't help you as well.

If this is a FEATURE REQUEST, please:
  - Describe *in detail* the feature/behavior/change you'd like to see.

Pods managed by a ReplicationController only allows a restart policy of "Always".

When a container in a multi-container pod managed by a ReplicationController crashes, the crashed container is restarted in the existing pod without notifying or restarting the other pods. I'd like to have a way to specify that *any* container crash in the pod should take down and replace the entire pod or at least restart all the other containers.
Our use case is as follows:
Pods consists of a worker container exposing gRPC APIs (let's call it container A) and a utility container (container B). Container B is connected to a Kubernetes HTTP liveness probe. When that probe is called, A makes a series of gRPC calls to B to determine it's health. If B returns an unhealthy result from the probe, Kubernetes restarts B in the existing container leaving A untouched and in an unhealthy state. Kubernetes should not just restart B, but delete the entire pod and have the ReplicationController bring up a new pod.

As a note, the worker containers are provided by other teams and source code, docker base image, etc are not available to the team operating the Kubernetes cluster so we cannot change the worker containers. We would also like the dynamic pairing of containers pods allows for to change out the utility containers depending on environment (dev, stg,prod) as the criteria for e.g. being healthy can change depending on environment.

In both cases, be ready for followup questions, and please respond in a timely
manner.  If we can't reproduce a bug or think a feature already exists, we
might close your issue.  If we're wrong, PLEASE feel free to reopen it and
explain why.
-->

**Kubernetes version** (use `kubectl version`): 1.3.6
**Environment**:
- **Cloud provider or hardware configuration**: AWS EC2
- **OS** (e.g. from /etc/os-release): CentOS Linux 7 (Core)
- **Kernel** (e.g. `uname -a`): Linux ip-10-44-68-102 3.10.0-327.10.1.el7.x86_64 #1 SMP Tue Feb 16 17:03:50 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
- **Install tools**: custom
- **Others**:


**What happened**: A container in a multi-container pod returned 500 from an HTTP liveness probe. Only that container in the pod was restarted


**What you expected to happen**: When a container in a multi-container pod indicates that the pod is unhealthy, the entire pod should be restarted.


**How to reproduce it** (as minimally and precisely as possible):
Create a replication controller with a template of a pod containing two containers.
Attach an HTTP liveness probe to a REST endpoint returning 500 from one of the containers.
Observe that only the connected container is restarted.
Observe that the pod is not recreated and that the non-probe-connected container has not been restarted


**Anything else we need to know**: