EC2-API: Fix ambiguous ipAddress/dnsName issue

Currently, the public ipAddress and dnsName take in either the
fixed or floating IP address as a value. The current fix ensures
that the public dnsName and the public ipAddress both get only
the floating ip address. A look into Amazon's EC2 API Reference
tells us that if there is no public IP associated with an instance,
an empty value is returned for <dnsName> and the key <ipAddress>
itself is never returned. This was tested with euca2ools.
Also, the key <publicDnsName> has been removed since it is
the same as <dnsName>

Fixes: bug #1096468

Change-Id: I3b3e3b3f815df41d1976444d33604363e64a7709
Make nbd reservation thread-safe

Avoid the situation where two local threads choose the same nbd number
for injecting files into the instance. If this happened quickly enough
nova was left with one qemu-nbd hanging and blocking the device forever.

Fixes bug 1207422

Change-Id: I18864b19ebd30669534e45ca7a50e12f61207302
Enable no_parent and file_only security

The websockify package was recently upgraded to version 0.5.1, which added two
parameters that provide additional security:
* no_parent - returns 403 response if contents out of the web root are
requested. Required to disable directory traversal.
* file_only - returns 404 response if non-file contents are requested. Required
to disable directory listing.
This version is available in PyPi.

Nova should upgrade it's required version for websockify, and set these
parameters while creating and starting the NovaWebSocketProxy to ensure
better security.

DocImpact
Fixes bug #1195700

Change-Id: I9c4b28eb1b384048f10d365ed204b916afd371bb
VMware: fix rescue/unrescue instance

Fixes bug 1199955

The problem is due to the way in which the VM's are named on vCenter.
That is, in Nova a VM has a name and a UUID. In vCenter the VM only
has a name. The VM name on vCenter is Nova's instance UUID. When the
rescue/unrescue operation takes place a new VM is spawned/destroyed.
The Nova VM has the name '<name>-rescue'. The problem is that the
new rescue name was not passed correctly to the vCenter, that is,
it should have been '<uuid>-rescue' but it was just '<uuid>'. This
caused the opertaion to fail as there is already a VM with that name
on vCenter. A new name should have been passed which is '<uuid>-rescue'.

Change-Id: Icfe6e5021b29026788bec8f3526f94a8ec0a65e2
Don't need to init testr in run_tests.sh

In run_tests.sh, function init_testr will initialize testr if the
directory .testrepository is not existed. Actually, testr will do
the check before run the test:

In Python package testrepository, setuptools_command.py:Testr.run

68 def run(self):
69     """Set up testr repo, then run testr"""
70     if not os.path.isdir(".testrepository"):
71         self._run_testr("init")

So, init_testr can be removed safely.

Fixes Bug #1211098

Change-Id: I4bd2ef3dc107281892732efced268eb3b7c90cbb
Removed code duplication in test_get_server_*_by_id

blueprint api-compute-servers-tests

Change-Id: I3aa8580941984029baf3d39e30b5c0e9b491bc24
Upgrade to Hacking 0.7

There was a bug in hacking 0.6 that broke H202, assertRaises Exception
too broad, so switch to Hacking 0.7 and fix the one H202 bug.

Change-Id: Ia3fb22d1524022479d7068cb84a3bdf7ca19f408
Fix HTTP response for PortInUse during boot (v3 API)

If an in use port is passed as a network parameter in the request, Nova API
should raise an HTTPConflict (409) error. This was fixed in commit
48b31c9a1f5d2a45d881e70c6223411c5abbcb72 for the V2 API but not the V3 API.

Fixes bug 1212098

Change-Id: I94429f2c04cf81cb88a8739f65a49aa7ca0d49bd
Raises masked AssertionError in _test_network_api

The _test_network_api method was catching an AssertionError to do some
more checking but failed to raise the original error if the checking
didn't hit any matches. This patch fixes the test to raise the original
assertion error.

Closes-Bug: #1212017

Change-Id: I33147d18e33cd4babfc3f5cb572007bc7e4c07e6
Implement front end rate-limiting for Cinder volume

Cinder adds Quality-of-Service control interface to allow admin to
specify QoS parameters as well as the place where these parameters
should be consumed (Front-end: Nova Compute, Back-end: Cinder
Volume, or Both) for volume types.  KVM and other hypervisor is
known to be able to do disk rate-limit control (one form of QoS)
for virtual disks, this patch enables Nova Compute to extract
rate-limit parameters for volumes while attaching, currently
libvirt only.

implement bp volume-rate-limiting

Change-Id: Ibcc8f15ab744600e6f814b46935f8022d78074fc
Allow Cinder to specify file format for NFS/GlusterFS

These drivers now have the option of passing a QCOW2 file rather
than a raw file as the volume to attach to the VM.

Required by blueprint qemu-assisted-snapshots

Change-Id: I23929538c7dad65035db5c38ff6e52348925fa27
Fix untranslated log messages in libvirt driver

Fixes bug 1213040

Change-Id: Ib3b7a46c312b88778cc6e872a2c19061a5774125
VMware: Added check for datastore state before selection

The driver only checked for the datastore type. The datastore could
be in an inaccessible state. In such cases creating an instance to
such datastore will fail. The fix to filter out inaccessible data-
stores.

The datastores have a property called accessible. The fix is to use
this property to filter out inaccessible datastores.

fix bug #1194078

Change-Id: I372fc3adc0e89c48d86f1f70164263ed35184c5d
Fix ratelimiting

Current master does not respect ratelimiting, since parsing of the
api-paste.ini is faulty. api-paste.ini limited user limiting by
setting a line as follows:
user:<user-id>:(GET, *, ".*", 4, minute)

This was passed to the Limiter as kwargs with "user" as a key.
Thus multiple user limiting is not possible as well as extracting
the id of the user was bound to fail, since we checked on the key
with startswith("user:")

An example config in the api-paste.ini has to look as follows:
limits = (POST, "*", .*, 10, MINUTE)
limits.<user-id1>:(GET, "*", .*, 4, minute)
limits.<user-id2>:(GET, "*", .*, 2, minute)

The code has been modified to be able to handle such configs

Fixes bug: 1210869

Change-Id: Iaec669b282a567d42fbe21a6f64e7bc99161053b
libvirt: allow passing 'os_type' property to glance

Currently, libvirt driver doesn't pass the 'os_type' property to
glance. Thus all snapshots created don't have this property. Other
feature depend on 'os_type'(such as mkfs for ephemeral devices)
doesn't work.

This patch fixes this by passing the 'os_type' attribute to glance
as image properties. None value in metadata properties would not be
passed to glance, thus this patch has no side effect on situation
when instance has no 'os_type' attribute set.

Bug 1214372

Change-Id: I2bab617f6c1e8866650ca7dc5178b640738c9ae6
fix volume swap exception cases

Currently when the target instance id doesn't match with the instance_id of the
volume to be replaced VolumeUnattached is raised, and InvalidVolume would be a
fittest response.
In the other hand, currently, when the volme to be replaced is not attached at
all InvalidVolume is raised, and VolumeUnnatached would be a fittest response.

Change-Id: I661b97762af70765ab8111313d24cf1a175f7e6c
[v3] Show detail of an quota in API os-quota-sets

nova-manage shows `in_use`, `reserved` including `limit` of a quota.
So create a new API named `detail` to do the same with nova-manage.

The new URL is like :/v3/os-quota-sets/1234/detail.

Fix bug: #1183870

DocImpact

Change-Id: Ic53a68acf0f3f3184c7a900360343bb576560f46
Adds missing V3 API scheduler hints testcase

Adds a testcase for the V3 API scheduler hints extension
which was missed in the initial port

Partially implements blueprint v3-api-unittests

Change-Id: I21fe5f25c806310395bbcd41e6ca84d2c0c02569
Added test case in test_migrations for migration 208

This patch is to add test case in test_migrations.py for migration 208.
Migration 208 was brought by commit 74b024f75403c168b5c69ee67a16d1053965e5db
(Change-Id: I72371260159c9fdffcdb57209095fc103b6dd79e)

Change-Id: I153abf260716c8af61d7ccf554d91ba80f559e60
Signed-off-by: Shane Wang <shane.wang@intel.com>
Make xenapi capabilities['enabled'] use service enabled

filters.compute_filter is enabled by default along with libvirt, but the
capabilities['enabled'] section of it is only supported by XenAPI.
This used to work by adding a parameter to capabilities, but since
the capabilities RPC is being removed, use conductor.service_update to tell
the scheduler which host is disabled.

service_update is already the way to tell the scheduler about other
disabled hosts, so to keep backwards compatibility keep host_enabled but
use service_update underneath.

Part of bp no-compute-fanout-to-scheduler

Change-Id: Ica1919cfe9a2402b2107c557c10d47e01a6c430b