use REDIRECT to forward local metadata request

VMs can not get metadata when metadata_host is set to 127.0.0.1 because
packet's destination address will be changed into 127.0.0.1 by iptables.
However, linux kernel will reject packets to 127.0.0.0/8 if packets do not
come from that range. We can use REDIRECT to resolve this problem because
REDIRECT will rewrite the destination address to our own host.

Fixes bug 1015853.

Change-Id: I162fc57510ef9c8e84130b2d11eb53e10dcc5cb0
Add PowerVM compute driver and unit tests.

part of bp:powervm-compute-driver

Change-Id: I9e55964b5260417d15b5def814ee305fce28c40b
Revert to prior method of executing a libvirt hard_reboot.

Fixes bug 1036826.

Using reset for hard_reboot is not reliable, even where it is supported
by libvirt. Hard reboots are one of the only ways to recover a VM in a
broken state. The reset command assumes the domain is running in some
capacity and will fail if it is not.

Hard reset is the current sledgehammer for fixing issues and it really
needs to stay that way.

Change-Id: I84705b72d79cf71adad066b18267fdfb199bc9cb
Restore libvirt block storage connections on reboot.

Fixes bug 1036902.

There are a number of cases where block storage connections are not
properly restored, impacting libvirt in particular. The most common
case is a VM which has block storage attached via iSCSI, whereby the
physical system is rebooted. When the system comes back up and
starts nova-compute, the iSCSI connections are NOT recreated for the
instances slated to be resumed (assuming
resume_guests_state_on_host_boot is set).

The patch changes the compute manager to pass block_storage_info via
driver.reboot() and driver.resume_state_on_host_boot(). The fix is
actually only present in the libvirt driver. However, all the other
drivers were updated to accept the additional, optional function
arg.

With the changes in place, iSCSI connections for libvirt are
re-established after a hypervisor reboot with
resume_guests_state_on_host_boot=True and on every hard_reboot.
The latter is intended so that users have a last ditch option for
recovering their VMs without administrative involvement.

Change-Id: Idf5d53f21991a359bec6ce26ae9fe3bd61800ce3
Adding port attribute in network parameter of boot.

Example is (network:[{‘port’:<uuid>}] )
The specified port will be used.
A port who have already device_id can not be used to be boot.
This fix is for isolating functionalities between Quantum and Nova.
There are no need update nova side if port model will be changed
in future quantum model.  This function is for QuantumV2 api use only.
Added port attribute in requested_network attribute. Fixes bug 1031096.

Change-Id: Id2c86228edb0c22f536f8b36a955c87870e9d98b
Port pre_block_migration to new image caching

This resolves bug 1036419, which was a missed call to _cache_image
when that method was refactored away.

Change-Id: I68f6aced5382e378dc707fd003d27dbfdd24aded
xenapi: remove unnecessary json decoding of injected_files

The API is responsible for decoding JSON an XML content. Also, the
code decoded results weren't used since the code below it would use
the original data anyway.

Also, add a unit test for injected_files in the xenapi driver.

Change-Id: Ie96b301b0948f92dd90060eacba0db9ae940b952
Defined IMPL in global ipv6 namespace.

* fix LP1014798

Change-Id: Ife2cc064128ee99a7ba1352f10c373812c69dd13
Add get_key_pair to compute API.

This will be used to implement a GET keypair API call.

Partial fix for LP Bug #925731.

Change-Id: Ie9976c0168842c7491d80be6213814192a2024df
Merge "Adding port attribute in network parameter of boot."
Merge "Defined IMPL in global ipv6 namespace."
Merge "Port pre_block_migration to new image caching"
Merge "Add PowerVM compute driver and unit tests."
Change IPtablesManager to preserve packet:byte counts.

Modified IPtablesManager.apply() method to save/restore chain and
rule packet:byte counts by using the '-c' flag with iptables-save
and iptables-restore calls.  Currently they are zeroed every time
we change something in the table.  This will allow users to better
analyze usage for instances over an extended period of time, for
example, for billing purposes.

Change all applicable iptables, libvirt and Xen tests to account
for the changes made to support the packet:byte counts.

This work uncovered two bugs in the existing implementation
found during my testing, specifically:

1. Fix IptablesManager to clean-up non-wrapped chains correctly,
   instead of leaving them in the kernel's table.  We now keep a
   list of chains and rules we need to remove, and double-check
   in apply() that they are filtered-out.

2. Fix IptablesManager to honor "top=True" iptables rules by only
   adding non-top rules after we've gone through all the top rules
   first.

Implements first work item of blueprint libvirt-network-usage.

Fixes bug 1037127 and bug 1037137.

Change-Id: Ia5a11aabbfb45b6c16c8d94757eeaa2041785b60
Move volume size validation to api layer.

As per the discussion on this patch:

  https://review.openstack.org/10540

Change-Id: I5d3031a08aa1e2d17b5fb233ebaec938c66fa88b
Collect instance capabilities from compute nodes

This change collects a list of supported instances from compute nodes.
Each supported instance is a tuple, containing the triple
(architecture, hypervisor_type, vm_mode).  This list of tuples describes
all of the instances supported by the compute node, which can then be
used by the ComputeFilter to ensure proper placement of an instance in
a heterogeneous compute node cloud.

Change-Id: I059ff55c165b372c83e63390b40a2c165df1f717
Make ComputeFilter verify compute-related instance properties

This change adds checks in the ComputeFilter to verify compute hosts can
support the (architecture, hypervisor_type, vm_mode) triple specified
in the instance properties, improving support for heterogeneous clouds.

Images can be marked with the architecture, hypervisor, and virtual
machine mode they require, e.g.

glance image-update --property architecture=x86_64 \
  --property hypervisor_type=kvm <image-uuid>
glance image-update --property architecture=i686 \
  --property hypervisor_type=xen --property vm_mode=xen <image-uuid>
glance image-update --property architecture=x86_64 \
  --property hypervisor_type=xen --property vm_mode=hvm <image-uuid>

These properties are included in the request_spec part of the run_instance
RPC message and made available to the ComputeFilter.  The ComputeFilter
will only pass compute nodes that satisfy the specified properties.
If no properties are specified, the ComputeFilter behavior is unchanged.

Adding these checks to the compute filter seems consistent with its
definition [1]:

"ComputeFilter - checks that the capabilities provided by the compute
service satisfy the extra specifications, associated with the instance
type."

[1] https://github.com/openstack/nova/blob/master/doc/source/devref/filter_scheduler.rst

DocImpact

Change-Id: I993f2e222ec036f1045f3cc5cc1851e730962729
Changes default behavior of ec2

Changes the default behaviour of create instance min/max count logic
in ec2 to match the default behavior of OS API

Fixes bug 989799

Change-Id: I1aa42c7d47674ce68442367fab487152b3e7fa67
Update disk config to check for 'server' in req.

Updates the disk config extension to check for 'server' in the
request body before trying to use it. This avoids a potential
KeyError exception. Given that we already validate the 'server'
dict in servers.py and raise a proper exception there this seems
reasonable to me... no reason to duplicate all of the validation
logic.

The reason this change was able to make it into Nova to begin
with is that the Extensions loader orders things differently
depending on the OS/python version being used. I tested this
by running SchedulerHintsTestCase.test_create_missing_server
on both Fedora 16 (passes) and Fedora 17 (fails). The problem
only manifests itself when the disk config extension gets called
before the scheduler_hints extension. To fix this issue
I've updated the scheduler_hints extensions in this patchset
to simply pass when 'server' isn't defined which like above seems
reasonable because we already validate this in servers.py.

As part of this fix I'm also moving test_create_missing_server out
of the Scheduler Hints tests and into a new ServersAllExtensionsTestCase
class in test_servers.py which explicitly tests for things with *all*
extensions enabled. As part of the move I also updated the test to
look for HTTP 422 Unprocessable Entity which is what we throw when
'server' doesn't exist in a POST to servers (the correct exception).

Fixes LP Bug #1037201.

Change-Id: I912fc1234b2e7a76ecc54140f4a31c2e32fdfa98
Allow nova to guess device if not passed to attach

partial fix for bug 1004328

Only the xen hypervisor actually respects the device name that
is passed in attach_volume. For other hypervisors it makes much
more sense to automatically generate a unique name.

This patch generates a non-conflicting device name if one is not
passed to attach_volume. It also validates the passed in volume
name to make sure another device isn't already attached there.

A corresponding change to novaclient and horizon will greatly
improve the user experience of attaching a volume.

It moves some common code out of metadata/base so that it can
be used to get a list of block devices. The code was functionally
tested as well and block device name generation works properly.

This adds a new method to the rpcapi to validate a device name. It
also adds server_id to the volumes extension, since it was omitted
by mistake.

The next step is to modify the libvirt driver to match the serial
number of the device to the volume uuid so that the volume can
always be found at /dev/disk/by-id/virtio-<uuid>.

DocImpact

Change-Id: I0b9454fc50a5c93b4aea38545dcee98f68d7e511