Initial framework for external volumes. 

Lays the framework for external volume mounts, preparing for 
persistent disks and other volumes.

Currently, it only supports host directory mounts.

Modifies the API to support host directory mounts from Volumes
instead of VolumeMounts.

On Tue, Jul 15, 2014 at 6:29 PM, Clayton Coleman
<notifications@github.com> wrote:
>
> In pkg/kubelet/kubelet.go:
>
> > @@ -69,6 +70,7 @@ type Kubelet struct {
> >   HTTPCheckFrequency time.Duration
> >   pullLock           sync.Mutex
> >   HealthChecker      HealthChecker
> > + extVolumes   map[string]volumes.Interface
>
> I believe you could miss in the delete window - have a cached copy from time t0, get new
> copy with deleted version at t1, begin deleting volume, crash at t2. You'd need to have t0
> and t1 copies on restart in order to apply the delete (maybe that's what you meant by
> checkpointing). Seems like it might be easier to make allocated volumes follow the same
> rules as pods themselves - be able to be idempotent and garbage collected. But I may be
> missing a potential problem specific to volumes.

etcd at t0 has pods p0, p1, p2

kubelet caches those pod specs and starts them

etcd at t1 has pods p1, p2 -- p0 has been removed

kubelet sets p0 to state REMOVING

kubelet removes part of p0, but crashes

kubelet comes back

kubelet loads the checkpoint, including p0

kubelet looks at etcd and finds no p0 -- reconciler says "remove it"

kubelet starts removing p0, finding half of it already gone

profit


Where's the race?  Yes, it's probably possible to make volumes be
idempotent.  Let's see.  Danny is specifically trying to add GCE
persistent disk support as a proof of concept.


etcd at t0 has pods p0, p1, p2

pods p0 and p1 both ask for a persistent disk named "foobar"

kubelet attaches foobar to the VM

kubelet mounts foobar on /var/run/kubelet/pd/foobar

kubelet creates a private space for p0 at /var/run/kubelet/pods/p0

kubelet bind mounts /var/run/kubelet/pd/foobar onto
/var/run/kubelet/pods/p0/vol/foobar

kubelet creates a private space for p1 at /var/run/kubelet/pods/p1

kubelet bind mounts /var/run/kubelet/pd/foobar onto
/var/run/kubelet/pods/p1/vol/foobar

etcd at t1 has pods p1, p2 -- p0 has been removed

kubelet removes part of p0, but crashes

kubelet comes back

kubelet looks at docker and finds p0

kubelet looks at etcd and finds no p0 -- reconciler says "remove it"

kubelet has no memory of p0

kubelet looks at /var/run/kubelet/pods/p0/vol and finds foobar

kubelet unmounts foobar from /var/run/kubelet/pods/p0/vol

should kubelet unmount /var/run/kubelet/pd/foobar ?  It doesn't know
yet whether foobar is in use by anyone else because we have no
checkpoint.

We can figure out the refcount as we re-assemble the in-memory state
from the machine.  If we are lucky, p1 still exists and says "foobar
is kind=PD"  But what if not?

kubelet has to figure out what "kind" of mount foobar really is so
that it can trigger the volume-kind code.  I guess it has to keep a
static mapping of directory names to "driver" modules so it can figure
out that /var/run/kubelet/pd/ goes to PDVolume struct.

It's certainly not impossible.  It does feel sort of like driving
screws with a hammer, though.

And if you squint, it's sort of a checkpoint anyway, isn't it?  We've
squirreled away info onto the disk that lets us recreate the state of
the world.  we just did it in a roundabout way.

Or were you thinking of something simpler, and I've just made a mess
of it.  Seriously, it's possible.  I am SO familiar with how we do
this internally that I am afraid I can't see any solution that doesn't
look like what I know.  Please show me a simpler way, and we'll do
that instead :)