Add get_key_pair to compute API.

This will be used to implement a GET keypair API call.

Partial fix for LP Bug #925731.

Change-Id: Ie9976c0168842c7491d80be6213814192a2024df
Change IPtablesManager to preserve packet:byte counts.

Modified IPtablesManager.apply() method to save/restore chain and
rule packet:byte counts by using the '-c' flag with iptables-save
and iptables-restore calls.  Currently they are zeroed every time
we change something in the table.  This will allow users to better
analyze usage for instances over an extended period of time, for
example, for billing purposes.

Change all applicable iptables, libvirt and Xen tests to account
for the changes made to support the packet:byte counts.

This work uncovered two bugs in the existing implementation
found during my testing, specifically:

1. Fix IptablesManager to clean-up non-wrapped chains correctly,
   instead of leaving them in the kernel's table.  We now keep a
   list of chains and rules we need to remove, and double-check
   in apply() that they are filtered-out.

2. Fix IptablesManager to honor "top=True" iptables rules by only
   adding non-top rules after we've gone through all the top rules
   first.

Implements first work item of blueprint libvirt-network-usage.

Fixes bug 1037127 and bug 1037137.

Change-Id: Ia5a11aabbfb45b6c16c8d94757eeaa2041785b60
Collect instance capabilities from compute nodes

This change collects a list of supported instances from compute nodes.
Each supported instance is a tuple, containing the triple
(architecture, hypervisor_type, vm_mode).  This list of tuples describes
all of the instances supported by the compute node, which can then be
used by the ComputeFilter to ensure proper placement of an instance in
a heterogeneous compute node cloud.

Change-Id: I059ff55c165b372c83e63390b40a2c165df1f717
Make ComputeFilter verify compute-related instance properties

This change adds checks in the ComputeFilter to verify compute hosts can
support the (architecture, hypervisor_type, vm_mode) triple specified
in the instance properties, improving support for heterogeneous clouds.

Images can be marked with the architecture, hypervisor, and virtual
machine mode they require, e.g.

glance image-update --property architecture=x86_64 \
  --property hypervisor_type=kvm <image-uuid>
glance image-update --property architecture=i686 \
  --property hypervisor_type=xen --property vm_mode=xen <image-uuid>
glance image-update --property architecture=x86_64 \
  --property hypervisor_type=xen --property vm_mode=hvm <image-uuid>

These properties are included in the request_spec part of the run_instance
RPC message and made available to the ComputeFilter.  The ComputeFilter
will only pass compute nodes that satisfy the specified properties.
If no properties are specified, the ComputeFilter behavior is unchanged.

Adding these checks to the compute filter seems consistent with its
definition [1]:

"ComputeFilter - checks that the capabilities provided by the compute
service satisfy the extra specifications, associated with the instance
type."

[1] https://github.com/openstack/nova/blob/master/doc/source/devref/filter_scheduler.rst

DocImpact

Change-Id: I993f2e222ec036f1045f3cc5cc1851e730962729
Changes default behavior of ec2

Changes the default behaviour of create instance min/max count logic
in ec2 to match the default behavior of OS API

Fixes bug 989799

Change-Id: I1aa42c7d47674ce68442367fab487152b3e7fa67
Update disk config to check for 'server' in req.

Updates the disk config extension to check for 'server' in the
request body before trying to use it. This avoids a potential
KeyError exception. Given that we already validate the 'server'
dict in servers.py and raise a proper exception there this seems
reasonable to me... no reason to duplicate all of the validation
logic.

The reason this change was able to make it into Nova to begin
with is that the Extensions loader orders things differently
depending on the OS/python version being used. I tested this
by running SchedulerHintsTestCase.test_create_missing_server
on both Fedora 16 (passes) and Fedora 17 (fails). The problem
only manifests itself when the disk config extension gets called
before the scheduler_hints extension. To fix this issue
I've updated the scheduler_hints extensions in this patchset
to simply pass when 'server' isn't defined which like above seems
reasonable because we already validate this in servers.py.

As part of this fix I'm also moving test_create_missing_server out
of the Scheduler Hints tests and into a new ServersAllExtensionsTestCase
class in test_servers.py which explicitly tests for things with *all*
extensions enabled. As part of the move I also updated the test to
look for HTTP 422 Unprocessable Entity which is what we throw when
'server' doesn't exist in a POST to servers (the correct exception).

Fixes LP Bug #1037201.

Change-Id: I912fc1234b2e7a76ecc54140f4a31c2e32fdfa98
Allow nova to guess device if not passed to attach

partial fix for bug 1004328

Only the xen hypervisor actually respects the device name that
is passed in attach_volume. For other hypervisors it makes much
more sense to automatically generate a unique name.

This patch generates a non-conflicting device name if one is not
passed to attach_volume. It also validates the passed in volume
name to make sure another device isn't already attached there.

A corresponding change to novaclient and horizon will greatly
improve the user experience of attaching a volume.

It moves some common code out of metadata/base so that it can
be used to get a list of block devices. The code was functionally
tested as well and block device name generation works properly.

This adds a new method to the rpcapi to validate a device name. It
also adds server_id to the volumes extension, since it was omitted
by mistake.

The next step is to modify the libvirt driver to match the serial
number of the device to the volume uuid so that the volume can
always be found at /dev/disk/by-id/virtio-<uuid>.

DocImpact

Change-Id: I0b9454fc50a5c93b4aea38545dcee98f68d7e511
Merge "Changes default behavior of ec2"
Adds Hyper-V support in nova-compute (with new network_info model), including unit tests

blueprint hyper-v-revival

Features included in ths commit:

Spawn (including CoW image option support)
Destroy
Info
List
Pause
Unpause
Suspend
Resume
Reboot
Power On
Power Off
Snapshot
Volume Attach
Volume Detach
Boot from Volume
Live Migration

Supported platforms:

Windows Server / Hyper-V Server 2008 R2
Windows Server / Hyper-V Server 2012

Unit tests:

Unit tests for all the listed features are included.
Tests can be execute on Linux as well.

nova.conf relevant flags:

Compute driver:

compute_driver=nova.virt.hyperv.driver.HyperVDriver

External vswitch to be used:

vswitch_name=an_external_vswitch

Path where the VHDs are going to be stored

instances_path=C:\Hyper-V\instances

Live migration support for hosts with etherogeneus CPUs

limit_cpu_features=true

Change-Id: Ic40adcd2d78b0ca6792d77940810f5a44de8cc37
Merge "Collect instance capabilities from compute nodes"
Merge "Make ComputeFilter verify compute-related instance properties"
Merge "Allow nova to guess device if not passed to attach"
Merge "Update disk config to check for 'server' in req."
Merge "Add get_key_pair to compute API."
Merge "Adds Hyper-V support in nova-compute (with new network_info model), including unit tests"