Add methods to Host operations to fake hypervisor

This patch implements some features present in os-hosts extension that are not
implemented in fake hypervisor:
    - Power Actions: Shutdown/Reboot/Startup a host
    - Change host status: enable or disable
    - Set/Unset the host into maintenance mode
With this patch we're able to get full api samples for this extension

Change-Id: I3b76e3070e326b39a67b9d4d25b5e3d91a3c102f
adjust rootwrap filters for recent file injection changes

Removed filters for guestmount, fusermount and cp.
Adjusted comment paths as per the recent refactoring.

Change-Id: I643554b89862225d16456e47daf6c11828f602d2
Allow multi_host compute nodes to share dhcp ip

This adds a new flag: share_dhcp_address which if enabled in
multihost mode will allow all compute nodes to share an ip on
guest network. The code will isolate the address using iptables
and ebtables so it is only visible to the vms.

This patch has two benefits:

a) we don't have to use an ip address from every network for each
compute node. This is especially valuable in vlan mode where the
networks are generally small

b) we can improve security by blocking all access to the ip on the
guest network from outside the compute node. While we could do similar
blocking using a different ip for each node, it makes dhcp setup
much more complicated if a vm is migrated to another node.

Implements blueprint shared-dhcp-ip

Change-Id: Iaf84c0ad2848921122866956105eb44c074450dc
Use flags() helper method to override config in tests

Setting config option values using setattr() is dangerous because the
value remains set even after the particular test finishes, so you can
end up with other tests relying on it and being unable to run tests
in isolation.

We have a helper method for doing this, just use it.

Change-Id: I872d7a4f70eceb9ff64f56901cb45552f237c733
Move imagecache code from nova.virt.libvirt.utils

The imagecache related code in libvirt.utils uses the base_dir_name
config option.

If we move the imagecache utils into imagecache, then we can move
the base_dir_name option into that module since there are no other
users of it.

Having all the imagecache code in the imagecache module seems like
a sensible thing to do anyway.

blueprint: scope-config-opts
Change-Id: I1e154aa4de1628d40964207bdcbe5b3b55076442
Move ec2 config opts to nova.api.ec2.cloud

Only ec2_dmz_host and ec2_port are used outside of ec2.cloud and in
both cases pipelib is the other module that uses them.

Move all the ec2 options to ec2.cloud and make pipelib explicitly
depend on the ones it needs.

blueprint: scope-config-opts
Change-Id: Ib187ec3421a25bdd86b13dfeae1dc9a07873eb43
Remove default_image config option

The value of this option hasn't been used outside of the tests since
the very early days of the project. There's no particular reason
to have a config option for this test value, so just delete it.

blueprint: scope-config-opts
Change-Id: I00a08d4dc539f92f4fa4bd8c3c8756e5029d193f
Merge "Move imagecache code from nova.virt.libvirt.utils"
Merge "Move ec2 config opts to nova.api.ec2.cloud"
Merge "Use flags() helper method to override config in tests"
Merge "Remove default_image config option"
Verify doc/api_samples files along with the templates

This also includes a bunch of updates to the actual samples that have
become stale as things have changed in the tree and naughty developers
haven't updated the docs. This new bit of logic in the samples test
should help prevent that in the future.

In order to verify the static doc/api_samples files, this adds a new
method to the base test, which allows the subclasses to refine their
subs into pattern matches after the server response is verified, and
before the doc/api_samples file is verified.

Change-Id: Ie46b24a65145c407d46974b63a97a84911223547
Include 'hosts' and 'metadetails' in aggregate.

When converting an Aggregate sqlalchemy model to primitive types using
jsonutils.to_primitive(), the hosts and metadata were being left out.
This is because they are implemented as properties on the model and must
be explicitly listed as extra keys to be read.

We also want to make sure that the _hosts and _metadata are eager
loaded.  This patch ensures this gets done by adding it to the
_get_aggregate_query() helper, and making sure it is used everywhere
that is appropriate.  Also, on creation of the aggregate, just manually
set _hosts and _metadata to empty lists, since we know they're empty at
this point.

These changes caused a side effect to the os-aggreages API extension.
The result from creating an aggregate started including 'hosts' and
'metadata'.  There's a hack in there to maintain the same API result.
These don't add any value to this API call anyway, since we know they're
blank at this point.

Part of blueprint no-db-compute.

Change-Id: I4048ea6f477f7a695701eb3ba57f97afca0c0e41
Update migration so it supports PostgreSQL.

Updates the 'deleted' columns so that they are treated as
Booleans instead of integers. This makes it so that
144_add_node_to_migrations.py works on both MySQL and PostgreSQL.

Fixes LP Bug #1084208.

Change-Id: Ibff1c45117e47126a255b2a1df5e915c881cb5c1
Merge "Add methods to Host operations to fake hypervisor"
Merge "Update migration so it supports PostgreSQL."
Convert datetimes for conductor instance_update.

This patch ensures that jsonutils.to_primitive() is used on the updates
being sent to conductor's instance_update method.  This is primarily so
that we convert datetimes to a string using a very specific format
defined in the timeutils module.  Then, in the conductor manager,
convert these fields back into datetime objects using timeutils.

The conversion to a string was already happening implicitly if you were
using the kombu rpc driver.  The reason is that kombu uses anyjson,
which we hook in to and will automatically use our own to_primitive()
to fix serialization problems for kombu.  Doing it explicitly is better,
and is required for the other rpc drivers to work.

This is to address a problem with sqlite where it would raise an
exception when we try to do an update to a datetime field with a string.

Fix bug 1084240.

Change-Id: I2d703a92df1e85f0f3a7a793b301fe16cbd81ff5
Merge "Allow multi_host compute nodes to share dhcp ip"
Fixes HyperV compute "resume" tests

Fixes Bug #1083018

Adds support for the extra parameter added in the compute driver's
resume method. The added network_info parameter is currently not used
and can safely be set to None.

Change-Id: I887d48f39601323433384833f8b8d33132858b59
Merge "Verify doc/api_samples files along with the templates"