apache/cloudstack — pull request #16 by lsimons (2014-09-04 13:39:27+00:00)
Title: CLOUDSTACK-7143: Refactoring of the systemvm build process

E-mail thread:
http://mail-archives.apache.org/mod_mbox/cloudstack-dev/201407.mbox/%3C7A6CF878-7A28-4D4A-BCD2-0C264F8C90B7%40schubergphilis.com%3E

This started out as wanting the systemvm build to take
systemvm/patches/debian/{debian,vpn} from the local machine/branch,
rather than downloading from the apache git master [1]. In working out
how on earth to get veewee to do that cleanly (hint: you can’t, hence
resorting to shar usage) I got quite frustrated with the image rebuild
times.

It so happens that veewee has a --skip-to-postinstall instruction which
is _quite_ useful while debugging these scripts. To get that working
requires the post install steps to be retryable/convergent. Of course,
our existing scripts weren’t set up for that. So I had to add a bunch
of tests whether changes had applied already. Which implied a pretty
significant refactor.

Summarizing this kind of thing is always hard...it’s many little
things...the interesting stuff is at the end/bottom, in particular
the two main improvements

https://github.com/schubergphilis/cloudstack/commit/142d087f6a97f6ac70a858a35d2fe8b638c58cbb
When working on the systemvm in isolation, or using vagrant or
similar tools, it can be useful to inject a custom SSH key before
merging a management server systemvm.iso into it. This option
allows that. It should _not_ have effect on management-server-
managed vms which always get their SSH keys	injected.

https://github.com/schubergphilis/cloudstack/commit/e2240eaed18000d4d94dbf6a6e40612db1aeda34
The current build downloads its script from master by fetching a
cloudstack tarball. Besides being an unneeded load on the apache
git server, this is a problem when working on a branch and
wanting to inject a different set of scripts. It also makes it
pretty likely that the injected copy of the script will not match
what a production release wants, so there is very little chance of
not needing to overwrite the scripts.

Ideally we would just rsync over some files. However, veewee does
not provide an option to do that. In order to keep a 'cleanly
veewee-only' build possible, and work with any recent veewee
version, in this change we restor to using shar
(http://en.wikipedia.org/wiki/Shar) to produce an archive which
can execute as a script, which we feed to veewee to execute.

In order to avoid having to re-do this cleanup twice, I also ended up
merging the systemvm and systemvm64 template definitions, factoring out
their small differences by inspecting the os architecture.

https://github.com/schubergphilis/cloudstack/commit/f570b3921cd52672f841fc5f99cdd96f9737d629
https://github.com/schubergphilis/cloudstack/commit/50e91217f90fc952182dccac02a5af06ac33fb45

Everything else…well it pretty much falls into two categories:
* general code cleanup without functional changes
* general code defensiveness to survive various jenkins build
scenarios

All in all it should help with ongoing maintenance, I think.

Most of these commits are now a while old but I wanted to wait with
sending this upstream until we had sufficiently tested the systemvms
built with this changed approach locally.