Repository: kubernetes
Issue: Deployment, ReplicaSet, StatefulSet, DaemonSet require restartPolicy=Always but don't NEED to
Labels: priority/backlog,sig/node,kind/feature,sig/apps,needs-triage

Since forever, these APIs all require "restartPolicy: Always".  There's not really a REASON for that, as far as I can see.

For example, suppose we allowed "Never" or "OnFailure".  If a Pod were to "finish", a Deployment, ReplicaSet, or StatefulSet will all start a new pod on some node, and a DaemonSet will start a new pod on the same node.  The primary difference would be that the WHOLE POD restarts - including initContainers.  That seems like it could be useful.

This seems mostly like deleting code:

```diff
diff --git a/pkg/apis/apps/validation/validation.go b/pkg/apis/apps/validation/validation.go
index db2e25ba5ef..2352b5f26bc 100644
--- a/pkg/apis/apps/validation/validation.go
+++ b/pkg/apis/apps/validation/validation.go
@@ -154,9 +155,6 @@ func ValidateStatefulSetSpec(spec *apps.StatefulSetSpec, fldPath *field.Path, op
 		allErrs = append(allErrs, ValidatePodTemplateSpecForStatefulSet(&spec.Template, selector, fldPath.Child("template"), opts)...)
 	}
 
-	if spec.Template.Spec.RestartPolicy != api.RestartPolicyAlways {
-		allErrs = append(allErrs, field.NotSupported(fldPath.Child("template", "spec", "restartPolicy"), spec.Template.Spec.RestartPolicy, []string{string(api.RestartPolicyAlways)}))
-	}
 	if spec.Template.Spec.ActiveDeadlineSeconds != nil {
 		allErrs = append(allErrs, field.Forbidden(fldPath.Child("template", "spec", "activeDeadlineSeconds"), "activeDeadlineSeconds in StatefulSet is not Supported"))
 	}
@@ -367,10 +365,6 @@ func ValidateDaemonSetSpec(spec *apps.DaemonSetSpec, fldPath *field.Path, opts a
 	allErrs = append(allErrs, apivalidation.ValidatePodTemplateSpec(&spec.Template, fldPath.Child("template"), opts)...)
 	// Daemons typically run on more than one node, so mark Read-Write persistent disks as invalid.
 	allErrs = append(allErrs, apivalidation.ValidateReadOnlyPersistentDisks(spec.Template.Spec.Volumes, fldPath.Child("template", "spec", "volumes"))...)
-	// RestartPolicy has already been first-order validated as per ValidatePodTemplateSpec().
-	if spec.Template.Spec.RestartPolicy != api.RestartPolicyAlways {
-		allErrs = append(allErrs, field.NotSupported(fldPath.Child("template", "spec", "restartPolicy"), spec.Template.Spec.RestartPolicy, []string{string(api.RestartPolicyAlways)}))
-	}
 	if spec.Template.Spec.ActiveDeadlineSeconds != nil {
 		allErrs = append(allErrs, field.Forbidden(fldPath.Child("template", "spec", "activeDeadlineSeconds"), "activeDeadlineSeconds in DaemonSet is not Supported"))
 	}
@@ -753,10 +747,6 @@ func ValidatePodTemplateSpecForReplicaSet(template *api.PodTemplateSpec, selecto
 		if replicas > 1 {
 			allErrs = append(allErrs, apivalidation.ValidateReadOnlyPersistentDisks(template.Spec.Volumes, fldPath.Child("spec", "volumes"))...)
 		}
-		// RestartPolicy has already been first-order validated as per ValidatePodTemplateSpec().
-		if template.Spec.RestartPolicy != api.RestartPolicyAlways {
-			allErrs = append(allErrs, field.NotSupported(fldPath.Child("spec", "restartPolicy"), template.Spec.RestartPolicy, []string{string(api.RestartPolicyAlways)}))
-		}
 		if template.Spec.ActiveDeadlineSeconds != nil {
 			allErrs = append(allErrs, field.Forbidden(fldPath.Child("spec", "activeDeadlineSeconds"), "activeDeadlineSeconds in ReplicaSet is not Supported"))
 		}
```

...and then fixing tests and probably adding some new ones.  Unfortunately, StatefulSet validation has this in `ValidatePodTemplateSpecForStatefulSet()`:

```
                // TODO: Add validation for PodSpec, currently this will check volumes, which we know will
                // fail. We should really check that the union of the given volumes and volumeClaims match
                // volume mounts in the containers.
                // allErrs = append(allErrs, apivalidation.ValidatePodTemplateSpec(template, fldPath)...)
                allErrs = append(allErrs, unversionedvalidation.ValidateLabels(template.Labels, fldPath.Child("labels"))...)
                allErrs = append(allErrs, apivalidation.ValidateAnnotations(template.Annotations, fldPath.Child("annotations"))...)
```

Which means that the template is not really being validated right now.  Sigh.

It may be that a "keystone" container (https://github.com/kubernetes/enhancements/issues/3676) would be required to really make this work.  I wanted to file this while it was in front of me.