Do not stringify None in templates

Emit "null" if the object is None.

Change-Id: Ic39c00b2b2de0fd0fedc4acd172c16b8c9788bc0
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Rename host_domain config option to default_domain

Change-Id: Ie3e2f1d656a72c62321b03f89956191dc4bcd7f6
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Allow default_domain to be set for any archetype

Drop a redundant check from add_network_device, as create_host() does
it.

Change-Id: Iac79c482b5ac0fcf2f2a9327c22b43604022480b
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Refactor cluster creation

Move the parsing of parameters common to clusters and metaclusters to a
common function.

Change-Id: I29b2ad5de0f067d80d54438c9b9358c01e13bd4b
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Allow configuring the default personality for clusters

add_host requires the default personality to be configured explicitly,
it no longer defaults to 'generic'.

Change-Id: I3ade78df01234f307f87ea90b586a2fa222edf81
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Don't clone all branches of template-king

Clone just one branch (by default, 'prod'), and call it 'prod'.

Change-Id: I2c8bae3adfd645b096c896df59771069f596e44f
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Make fake_macdata POSIX compliant

Scripts using /bin/sh should not use non-POSIX features.

Change-Id: I56ad937f6695ba8e8a53eda84dbe317ce21f0498
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Testsuite: Drop dependency on ancient panc version

Fake multiple panc versions by setting up some symlinks instead.

Use the vars=... argument of config.get() for forcing the substitution
of the compiler version, since that works correctly even if the config
file specifies multiple substitutions.

Change-Id: I8e726985e8aa39b7b86d635db30f86891d237d72
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
aq.py: Improve status reporting

Use a Condition() to ensure the show_request is sent after the real
command, instead of busy wait/retry. Note that this change does not
guarantee that the server will actually receive and process the real
command before show_request, but it increases the chance that we will
not have to retry the show_request call.

Add linear backoff to the waiting time if we do have to retry.

Change-Id: Iad1c393d72b25808492ff253479d8317a8e0e2fe
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Testsuite: Make Linux versions configurable

Make the Linux model versions we use in the tests configurable, so we
won't need to touch a large number of files if we need to bump the
versions.

Change-Id: Id874f2b7b9ee02532fa630cc70c647e6d384fac3
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Testsuite clarifications

Personalities always have generated templates. The 'badpersonality'
personality will fail to compile not because of missing templates, but
because of missing required parameters.

Change-Id: If745dffb54db1582ea6ed32f8cf212e1fc79b13e
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Testsuite: Compile aurora hosts

Add aurora hosts to the unittest domain by default, meaning they will
get compiled.

Change-Id: I23bf34e97dc6628aedf13515a31045fdd16ac0b2
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Use fake panc location in unittest.dump

Domains/sandboxes we want to compile should be created run time, where
we can compute the correct path for panc.jar. Fix the domain of Aurora
nodes we do want to compile.

Change-Id: I906c9aebd238fa100af284894d9f03eb61bf046d
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Testsuite: Cleanup test_start.py

Use a helper for calling external commands. Pass commands as lists
instead of tuples, for better visual experience.

Change-Id: I5f1f39eac096293b9ca3de0c803e61ca1173a5e8
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Fix metacluster change test

The formatting code failed if either the new or the old cluster was not
a member of a metacluster.

Change-Id: Idf926758a3e4746e3a60a48df5f9cf1dda4e95e5
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Testsuite fix

Remove dependency on the order of the items in the protobuf output,
since it is not guaranteed to be stable.

Change-Id: I66c433fa67863eff85c8d3577f3a7e23bf2bb10d
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Testsuite fix

When calling aq_compile.py, make sure it uses the same panc.jar version
as the rest of the testsuite, since unittest.conf may point to somewhere
else than aqd.conf.defaults.

Change-Id: I8ee3a6e760af12d808e09e2f45c578083afb1007
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Add dummy templates for running the testsuite

Add a minimal set of templates that will allow the testsuite to run.
There are still a couple of failures, but the majority of the tests
pass.

Note that the default is still to use the production templates, you have
to edit unittest.conf to use the bundled ones.

Change-Id: I18551380be83e665b4fcd3726ebc15bac2f42998
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Factor out merge_into_trash()

Change-Id: Id1fbfad3127701397ddbba8a2df5798aee32cc8e
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Make run_command() honour the loglevel argument

Change-Id: I05474857e403c481686c5d3afb8079b4fc97153e
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>