Update the parameters of the panc task in build.xml

The parameters of the panc task have changed in panc 9.4. This change
updates build.xml to use the new calling convention. It also contains
changes to allow turning on JSON profiles in the future.

Change-Id: I16eb3303f13fdeb77ec9e926e6a50ebec5396eb5
Reviewed-by: Nathan Dimmock <Nathan.Dimmock@morganstanley.com>
Drop "xml" component from the name of the build directory

It's unneccessary, and it will become misleading if/when we switch to
JSON profiles.

Change-Id: I1d1b6674dcfb37d004e876016b78af8ea7475d5a
Reviewed-By: Zoltan Sebestyen <Zoltan.Sebestyen@morganstanley.com>
Move non-object plenaries under %(quattordir)s/cfg

This puts both the object and non-object plenaries under one common
directory, paving the road of using git to manage that directory in the
future.

Change-Id: I162dd5cab734a294450cda0eb044a64eef9888ca
Reviewed-By: Zoltan Sebestyen <Zoltan.Sebestyen@morganstanley.com>
Rename builddir to cfgdir

There's a directory called build, but that's used for something else, so
calling the cfg directory as builddir was a bit misleading.

Change-Id: Ie987a3ba0b6c61cafcfc54e80c2516d9023514b8
Reviewed-By: Zoltan Sebestyen <Zoltan.Sebestyen@morganstanley.com>
Add logging to refresh_windows_hosts

Log hosts that have been added or removed.

Change-Id: Id20836404a27656e6aee5e56133a877d4d640ae0
Addresses-Issue: Jira/AQUILON-565
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Fix update_personality help message

Change-Id: Ic17f89b4614c219141e3534eca92d148563b6e5f
Addresses-Issue: Jira/AQUILON-1464
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Unify lock keys

Get rid of the get_write_key() and get_remove_key() methods, use
get_key() everywhere. This alone means sometimes we're locking more than
we should, but further commits will fix that.

Change-Id: I5092dc9ef01a6b9c79e14b30badeee65aa8228dd
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Implement more granular locking

Introduce PlenaryKey, to be used for non-object plenaries. The exception
is machine templates, where instead of locking all VMs when a cluster is
changed, we're locking the cluster if a VM is changed, because that
lookup is cheaper.

Change-Id: I2574b7dd8b445338a9c3ad55559c99bd1bf0c80b
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Lock only the affected domains during "aq manage"

Change-Id: Ic66eabf17e9f051564aa8546abd7b07dc386ebe5
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Various locking- and compile-related cleanups

Simplify bind_feature - there are not that many personalities that the
progress report would worth the extra complexity.

Use Plenary.get_key() where possible instead of constructing a
CompileKey() directly, as this will allow customizing the lock keys
later.

There's also a bunch of mostly cosmetic changes.

Change-Id: I52eed3bee3f09ba48168aa72486f81da7dd4bec9
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Cleanup TemplateDomain.compile() return value

Nothing uses the returned value, so there's no point returning anything.
Use the logger to tell the user if there's nothing to compile.

Change-Id: I549f78572d965632918abb4029b46b358a4c9220
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Compiling a domain should lock dependant plenaries

Make "aq compile" lock service and personality plenaries that are used
by the domain/sandbox.

Change-Id: Ic279a3b32c14ee685448d5f8d60c90fea85801a5
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Improve "aq compile --personality ..." locking

Use the plenary classes to derive the lock key, so it will include the
dependant plenaries as well.

Change-Id: I7ea1a13e928e46447896b17bfb300ae0ff803eda
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Chooser: simplify plenary handling

Call .stash() only on the final collection instead of one-by-one.

Change-Id: Iaf6ccabf803db9be0bd9fa1f199557ef8ae51486
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Chooser: manage changed server plenaries

Keep track of the plenaries of the servers that need recompiling,
instead of just their FQDNs.

Change-Id: I1298a1dab4da657563aa3aad0511129942f00d02
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Service chooser cleanups

Get rid of a couple of unneeded fields. Use format() where it makes the
code simpler. Use more modern language constructs.

Change-Id: I1c0e26662d161ed361d6f46ab6f3a6082cbbc456
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Refine locking when compiling clusters

Use the plenary objects to determine the compile key needed. This means
compiling a cluster no longer locks the whole domain.

Change-Id: I9690cd7342bd0f1af4017fc86a5df8c71f4fb29c
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Use shared locks for non-object plenaries

This allows e.g. compiling hosts in different domains simultaneously,
even if the hosts have the same personality or share service instances.

Change-Id: Ic57af766df8c84c4f08e325fff2c59f544803f43
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Domain compile optimization

Always create the 'clusters' subdirectory, because that's cheaper than
checking the DB if there are any clusters in the domain.

Change-Id: I709be7423fe615d8251333fd9f6c23aec30c0659
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>
Testsuite: leave unittest02 in compiled state

Further tests in test_manage.py depend on unittest02 being compiled.
This got broken in commit 9d1b37417407701b65326ef8ea0abca521e13328, but
the breakage was not detected due to "aq reconfigure --list" always
compiling the whole domain. Now we want to change that, so this test
case needs fixing.

Change-Id: I644d74965da2bf13d47d3181509b6881f0d97069
Reviewed-by: Dave Reeve <Dave.Reeve@morganstanley.com>