progrium/dokku — pull request #433 by nickl- (2014-01-27 03:03:10+00:00)
Title: POC: Use buildstep as data volume for app containers

A proof of concept of what I quickly hacked together on a development server which we can discuss here and should you agree with the premise, design a properly solution.

#### tl;dr Proof of concept not to be merged

I was a little taken back by the size of the app containers but it makes sense to heavy load the buildstep foundation image for performance purposes. The base image size sets in at 875.5 MB pushing simple rails apps over the 1GB in size per application. Since the base would remain the same it is only the changes in the app containers which we need to persist really.

```
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
app/rails4 latest d8407a99c5b7 8 minutes ago 1.014 GB
progrium/buildstep latest 56587e295631 2 days ago 875.5 MB
```

Looking at where the usage are spent:
```
/usr = 723 MB
/etc = 1 MB
/var = 62 MB
/sys = 98 MB
/.gem = 1 MB
/lib = 16 MB
/sbin = 6 MB
/build = 18 MB
/run = 490 Bytes
/bin = 5 MB
```

So the vast majority of the total size resides under `/usr`, so I exported `progrium/buildstep:latest` to another image `progrium/buildstep:usr` and exposed the data volume share for `/usr` by running a named container with command true. As per [the documentation for hata volume containers](http://docs.docker.io/en/latest/use/working_with_volumes)

```
$ docker run -v /usr -name buildstep-usr progrium/buildstep:usr true
```

Then I deleted the `/usr` mount point from `progrium/buildstep:latest` and added a `FromVolume` to the image:

```
$ docker commit -run '{"VolumesFrom":"buildstep-usr"}' 99d238df9459 progrium/buildstep:latest
```

This allows you the run the image without bothering with the volumes and you will always have the /usr mount. I was hoping this would automagically just work transparently but it does not transfer for subsequent commits so I had to modify the dokku source code see 18979a5 Unfortunately, it turns out that the cleanup phase is a little overzealous when removing all 'Exit" containers it would remove the data volume container as well so we need to fix that 215eb5c

Now with the mount point persisting across commits and the data volume container surviving cleanup everything seems to work as suspected and we these are the results:

```
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
app/rails4 latest 4cee8b26748f 28 minutes ago 256.3 MB
progrium/buildstep latest 6458a4464058 About an hour ago 117.5 MB
progrium/buildstep usr 14161ba89cf4 2 hours ago 875.5 MB
```

As this is only a proof of concept the `/usr` mount will suffice but we will still be able to squeeze even more space into data volumes by strapping in `/lib`, `/bin`, `/sbin`, `/etc` without much effort as the VolumesFrom directive simply includes all volumes from the named container.

Caveats:
* the mount point to `/usr` or more specifically all data volume shares are not read-only not sure if this may pose a problem. Can you think of a workaround?

Advantages:
* working with the smaller containers are much quicker so we gain a performance boost
* requires less disk space for app containers but that was the point
* special precautions are taken for the mount points which may help with updating buildstep, etc

> If you remove containers that mount volumes, including the initial DATA container, or the middleman, the volumes will not be deleted until there are no containers still referencing those volumes. This allows you to upgrade, or effectively migrate data volumes between containers.

Request for comments:
What are your thought? Any concerns? Do you foresee any problems? Additional considerations we should entertain? Is this something you want? Can you pledge your assistance to get this specked out and implemented (please note availability and expertise)?