Add a nodepool command

Moves the daemon command to nodepoold.

Refactor config handling a bit in NodePool to make the config
objects just contain information by default (though things
such as database handles and managers may get added to them
later as needed).

Start with the list and image-list commands.

Change-Id: If2ba7bca7ab4ef922787176af87ad5de31ae4b3e
Add image-update command

Change-Id: I8052bcfb3bc2d03c5e50b601d9dde7f3e7d2c43b
Add alien-list command

It lists servers that exist in the provider accounts about which
nodepool has no knowledge.  Useful for identifying resource leaks.

Change-Id: Iaf71d6320d6ec7691f301208e09974cad2177ad5
Add alien-image-list command

To list images about which nodepool has no knowledge.

Change-Id: I6c861abaf6b53a8e50d730590eaa3509ca864f61
Add a hold command

To put a node into the HOLD state.

Change-Id: I2f173ce928f4ff8399c6fdc00d03a09fa3b20136
Add a delete command

To delete a node.

Change-Id: Ifd2639b1bfd069807c64d95717822f469a6601fb
Add image-delete command

To delete an image.

Change-Id: Ied9ac09073780c63201aaf2fcd6fa1e995c1c6ab
Fix HOLD state

When responding to a build complete event, don't do anything to
the node if it is in the HOLD state.

Change-Id: I37e458198bfcd08472d07ca9c206c1b4551f3341
Fix fake provider

We now expect a metadata attribute on objects returned by
novaclient.

Change-Id: Id3bf754e2e745cc3fdee35bda532928e7d7b347a
Add default location for config file

Change-Id: Ib8ae5a70c9e260302ff16f20c676aa8707acd58c
Merge "Add default location for config file"
Merge "Fix HOLD state"
Ignore hacking warnings

Some of us have hacking installed global, which means nodepool flake8
produces a lot of spurious warnings. Suppress them.

Change-Id: Ie869a92fa423dc022c5c37c102f5a9071ccaf1b0
Make node SSH timeout configurable.

* nodepool/nodepool.py: Make the initial ssh timeout configurable
(retain default of 60 seconds). Looking at the logs there is a very high
occurence of SSH timeout from our providers. Making this timeout
configurable will allow us to adjust the timeout if necessary.
Additionally, fix the comparison between old and new provider configs,
some items were not compared when they should be.

Change-Id: I51df708cb24e93e87c2fedf36d1f9de2131c76bd
Merge "Make node SSH timeout configurable."
Inspect the Gearman queue for immediate demand

Use information from Gearman to determine the immediate load
requirements of the system and spin up as many nodes as required
to meet the demand.  Use the existing information about the
min and max servers to determine the ratio of servers to spin
up from each provider.

Replaces the several fake server scripts with one script that
implements statsd, zmq, and gearman to ease testing.

Change-Id: Ic0dedc7ef2760ff664912f771377e02967ad5633
Fix parsing gearman status

The current code has some parse errors with more complex gearman
function names (which can show up due to the way Jenkins constructs
maven jobs).

Also, switch the calculation to examine only queued jobs (total -
running) instead of trying to calculate a worker shortage (total -
workers).  The latter doesn't deal well with multiple jobs that
require workers of the same image (it incorrectly behaves as if
they are independent).  By only examining queued workers, the actual
relationship between multiple jobs that require workers from the
same image is manifested by the fact that if, together, all such
jobs exceed the demand, we will see jobs sitting in the queue.

In other words, the overall picture is now that nodepool should
have at least enough ready+building nodes to accomodate the number
of jobs for a given worker/image that gearman is waiting to run.

Change-Id: Ibc2990ed2c7aea37bd4c94e5387c80ef840afa83
Fix stats after min_ready change

The movement of the min_ready value broke stats reporting, this
corrects it.

Change-Id: I4cf052e7357263de992233802c8947d638a1227f
Add the ability to ignore offline targets

If a jenkins is in shutdown mode or is offline, ignore that jenkins
for the purposes of launching nodes.  Node updates (used/complete)
for that jenkins will still be processed.

This should allow another jenkins to gracefully accept the increased
load if one goes offline.

Also, log the IP address when spinning up a node.

Change-Id: I3a8720dd5aaf154ca91cdc36136decad52eb6afa
Make image updates independent

If we're missing an image, only rebuild that one image.

Change-Id: Id27b1351469e65ebe0fc29f2e962ddf04aded5d9