WIP: Persistent disks proposal

This is a draft proposal for expanding GCEPersistentDisk to a fuller PersistentDisk framework.


----- Original Message -----
> > +that don't belong to them.  Security is implemented via an access control
> > list whereby an administrator specifies which usernames can access which
> > namespaces.
> > +
> > +## Behavior
> > +
> > +* PersistentDisk.Status.Mounts must be updated as disks are attached and
> > detached from hosts.
> > +* NodePersistentDisks are never detached and remain in
> > PersistentDisk.Status.Mounts until the disk is deleted.
> > +
> > +## Volumes Framework
> > +
> > +See [Tim Hockin's volumes
> > framework](https://github.com/GoogleCloudPlatform/kubernetes/pull/2598)
> > +
> > +## <a name=schema></a>Schema
> > +
> > +* Add new top-level PersistentDisk REST object
> > +    * Disks have identity. Disks outlive pods. Their IDs must be
> > persistent for future re-mounting.
> > +    * Disks requests are not expected to be POSTed to the API server.
> 
> Pods are 1:1 with volumes.  A ReplController w/ 5 replicas would make either
> 5 separate disks (1:1 to a pod) 

There is no way to do this with a replication controller by design.  You need nominal services or role assignment to be able to make the pod template vary across each replica.  Otherwise, each pod is going to have exactly the same volume template, so there's no way to ensure that the pod gets a unique volume.

> or share the 1 disk across many pods (1:1 to
> the PodTemplate).  That is why I think it can remain in the Volume section
> of a pod definition.  It is easy enough to change it to a reference (by way
> of selectors or a specific name), but it seems like an extra step than is
> necessary for a developer.
> 
> 
> 1.  Pod w/ storage request is posted to API server
> 2.  The REST implementation  (where 'business logic' exists, validation,
> saving, doing stuff) :
> 2a.  Validates
> 2b.  Creates the pod in etcd
> 2c.  Creates the volume, as necesary, in etcd

What happens if the volume can't be created?  What if there are no available volumes?  What if the user who created the pod is not allowed to create volumes?  What if the pod user asks for a volume that is too large, but after the pod is created the volume can't be created, so the pod can't be started?  Remember that the volume creation can't be blocking (there may be no storage available).  There are some advantages to doing it in this flow, but a lot of race conditions that occur when you try to manage two resources from one that in general we say we shouldn't be doing.  The only advantage here is that you have an implicit volume ref, rather than an explicit one.  But status / etc on the pod still needs to show the actual volume (the ref) so you end up needing a ref somehow.

The only use case we *must* support is that I can create a template of an app that has replication controllers and a volume, and there should be no manual intervention required by the developer for the pods to eventually start running attached to a new unique volume created on the fly.

> 2.  Various watchers do their thing
> 3.   On pod delete, use the retention policy to delete the disk or not



> 
> ---
> Reply to this email directly or view it on GitHub:
> https://github.com/GoogleCloudPlatform/kubernetes/pull/2609/files#r22109823