coreos/rocket — pull request #297 by sgotti (2014-12-19 14:27:34+00:00)
Title: Add an ACI Renderer library.

Hi,

this big RFC PR tries to add a new ACI Renderer library (as proposed in appc/spec#16).
I tried to split it in multiple self consistent patches as I'm touching various subsystems. Other then adding new tar functions (some already proposed in #276, if its merged I'll rebase this PR) (the first two patches are very verbose as they adds a new godep) the new features are:

* cas: on aci import, additionaly index the image manifest, its ImageID, and import time. This uses a diskv index to get all the manifests for the same App Name composed (there are probably better ways to do this but by now I want to discuss all of these things) by the App name hash concatenated with the ordered labels hash. I'm assuming that two ACIs are the same if they have the same Name and same labels (for example, during some testings, I will probably create an ACI multiple time and overwrite it every time because I'm not going to release it and I don't need to update its version every time)
* cas: A GetACI function that, given an app name and optional labels and imageID, returns the image manifest and imageID of the best matching image (the best matching image is the latest imported one of all the images that satisfies the requested App name and labels or ImageId) .
* acirenderer: a library that, given an app name and optional labels and ImageID, creates the dependencies tree and renders it in the requested directory.

I'm using and testing all these changes with https://github.com/sgotti/acido in the "acirenderer" branch.

I have some doubt that I'd like to clarify but probably opening new issues on appc/spec will be better. The most important is this:

The spec in the "Dependency matching" chapter says that every dep should be discovered using the aci discovery process and then, just the downloaded image should be verified with the requested labels or matching ImageID.
In appc/spec#16 this process is splitted in two phases: ACI fetching and ACI rendering.
So GetAci will be called by the Aci Renderer after the ACI fetching phase. In this ACI rendering phase the ACI is requested to the store and the store can find more than one ACI that matches (if no specific ImageID is requested) the requested App name and labels.

For example: if you are requesting to the store an App name without any label, all the ACIs with that App Name will match your request. Which one should be returned?

A more concrete example can be the "latest" pattern: If I'm requesting to the store an App without any version label, how can I satisfy the "latest" pattern?
*Fetch phase:
In this phase, as I'm not passing a specific version label, the aci discovery will download something like https://example.com/appname-latest-linux-amd64.aci. On the http server it can be a link to an aci called appname-2.0.1-linux-amd64.aci and in its manifest the version label will be "2.0.1".
*Rendering phase:
I'm asking to the store an ACI with appname and without any version label.
The store can have other ACIs previously downloaded or imported for this appname with different version (for example "1.0.0" "strangereleasename" etc...). As the version label is just a string without any SemVer or lexical ordering rule there isn't an ordering in these labels.

By now the unique idea I had and that was implemented in this PR was to save the time when an ACI was imported in the store and choose the latest.

I think that the splitting between the fetching phase and the rendering phase is good. Someone can, for example, also not fetch the image from a repository but just import an aci manually in the store. Or there can be an offline mode (the repository is not available for any reason).

Perhaps the spec should be clarified?

Thanks!