Fix logic for aggregate_metadata_get_by_host_with_key test case

After deleting a3 metadata, it is the host that once has such metadata
should be tested.

Change-Id: I79c40aa977911cd327d478ad8d9f147e19325017
Change parameters of add_timestamp in ComputeDriverCPUMonitor class

Change parameter 'cls' to 'self' in wrapper. 'cls' may mislead developpers
that this decorator returns a classmethod. Use functools.wraps to follow
the common pattern of decorator in nova.

closes-bug: #1288281

Change-Id: I6acbcba7297607dc6d07e8668ad07e6ca81d4456
Add UNSHELVING and RESCUING into IoOPSFilter consideration state

IoOpsFilter will take instance in following state
task_states.RESIZE_MIGRATING, task_states.REBUILDING, task_states.RESIZE_PREP,
task_states.IMAGE_SNAPSHOT, task_states.IMAGE_BACKUP
into consideration ,if the instance on this host exceed the maximum
number of instance allowed to do I/O the scheduler will fail on the host

UNSHELVING and RESCUING are also I/O related so IoOps need to include them

Change-Id: I141470254c824e7e57e6734ae0e016c432b85c00
Closes-Bug: #1295558
Remove unnecessary passing of task_state to check_instance_state

Remove cases where task_state=[None] was passed to check_instance_state
when that is essentially the default value anyway

Change-Id: I49b6449b9ae43a5cfcf5a1ccac5ee9a64d2b3f3c
Make sure leases are maintained until release

Dnsmasq asks nova-dhcpbridge for existing leases on reboot, but
we don't return addresses that have been deallocated but not yet
released. This modifies nova to return leased but not allocated
addresses. It also delays removing the vif until we disassociate
the ip so we still have access to the mac address.

Resolves-bug: 1300355

Change-Id: Id78d66aad5005f052a1b49002de0ef13ef5a298d
libvirt: optimize pause mode support

Commit bfb28fcf9031af4c695177663702ce05edbbfa4d added a method
call that checks if the libvirt driver support the
VIR_DOMAIN_START_PAUSED flag. This patch optimizes that by reading
the value at startup.

Change-Id: I0fc1edec55df609ab25b9999f4a0d077ddcf0be7
Add lock on API layer delete floating IP

There are potential race condition when multiple nova-api
service is running. We need to prevent the concurrent access to
the same floating ip, though db layer function is ok for floating
ip, the quota will be released more than one time and it will
lead to potential problem later

So the solution: check whether the db is *really* updated with the
value we want, which means the second operation will fail and
return None, so network api layer can handle it

Change-Id: I028938a4c387d21a8880ae168b161a763e7361ff
Closes-Bug: #1268569
Add specific doc build option to tox

Add doc venv to align with other OpenStack projects (such as keystone,
heat and nova-specs) who are using 'docs' for there doc builds.

Change-Id: I37771db339d864b2b938d1c02d15d38b07093493
VMware: remove unused variable

The flag was never used since it was added in commit
24d609447399e468937f8a278854690864ffabcd

Change-Id: Ia034ccff1220783c2dd8c37c76fce24063f12682
Remove unused nova.crypto.compute_md5()

Remove nova.crypto.compute_md5() as it is:
 * unused
  * last user removed 3 years ago with commit bab3060
 * untested
 * incorrectly documented
  * return type is not a tuple, but a string
 * redundant
  * copied from boto which is a nova requirement
 * burdened with license text more verbose than associated code

Change-Id: Ia033d3359098fc0af8de00248dc14b4fe280e844
Support multi-version pydevd

We need to support multi-version pydevd
in order to supporting pycharm3 and pydev3 remote debug.
If we want to remote debug by pydev3 or pycharm3,
"from pydev import pydev" is not work,
need to "import pydevd" directly.

Change-Id: I975c859c46f81d7fe1ca3dfebe05040153471800
Add VolumeMapping object for use in EC2

The EC2 code has a table mapping volume UUIDs to EC2 IDs.  This patch
creates the Nova object for this so that we can remove direct use of
these db.api calls from the EC2 code.

Change-Id: I80e5d9c8e33c1f0c5ef8a9f7c54935c8c329ed25
ec2: create volume mapping using nova object

Update the EC2 code that creates a mapping in the db between the
volume UUID and an EC2 ID to use the nova object for this.

Change-Id: I2b02c9f639519a7835ffcf27be704b4810334e6d
ec2utils: Use VolumeMapping object

Update ec2utils to use the VolumeMapping object instead of direct db
API calls for getting the ec2 ID to UUID mappings.

Change-Id: Id653fc16fffc1c416379087228a0b053b8868cac
Add warning to periodic_task with interval 0

The behaviour of Oslo's @periodic_task decorator is that
a repeat interval of 0 means that the method will be
called regularly.  Several Nova methods which use that
decorator also have code for returning early (ie no-op)
if the interval is 0.  This is inconsistent for users, and
means there is timing-related code where it doesn't belong.

This patch adds a warning when a user takes a value of 0
from config and uses it as an interval in a @periodic_task.

Ideally all code will use Oslo's convention that 0 means
run at the default rate and a negative interval will mean
"don't run".  After this warning has been in place for a full
release cycle that can be the case.

Change-Id: Ia227f4c4e69ecf361ab02d1d17a3010303650104
DocImpact: Warns of upcoming change to behaviour when
           *_interval is set to 0
Closes-Bug: #1272830
Partial-Bug: #1276203
Rename NotAuthorized exception to Forbidden

The NotAuthorized NovaException has an internal code of 403 which is
actually Forbidden, so rename it appropriately.

This patch doesn't change the external behavior, the status code in
responses will still be 403 but the exception is just named properly.

This is also necessary to create an actual Unauthorized NovaException
with code 401 for use in some Neutron API bug fixes for more granular
error handling from python-neutronclient.

Related-Bug: #1298075

Change-Id: I691fac2e2c797f47c04da7965d7b1c8685c74edb
Delete in-process snapshot when deleting instance

Try to delete the image created to hold the snapshot when the snapshot
creation fails because either the instance doesn't exist or it is in the
deleting task_state.

Closes-Bug: #1296590
Change-Id: I422c4f10c348d189fee217b55e0766e73eb5b16c
Update block_device_info to contain swap and ephemeral disks

An ephemeral or swap disk is attached to an instance on boot
as follows: nova boot --flavor FLAVOR --image IMAGE_ID
--swap 512 --ephemeral size=2 INSTANCE.  When a hard reboot is
performed on the instance, nova fails to recreate the
appropriate libvirt XML definition, containing the ephemeral
disk.  This is because the correct block_device_info dict that
is passed to the compute manager's reboot_instance method does
not contain swap or ephemeral disk key values that are necessary
to recreate those disks.  In addition to nova boot, the correct
block_device_info dict is also needed by nova rebuild, reboot,
resize, and migrate to recreate those disks.  This patch updates
_get_instance_volume_block_device_info (renamed
_get_instance_block_device_info) to return the swap and ephemeral
disk key values, in addition to the block_device_mapping it
already returns today.

Change-Id: Iec329d1c12a48ea90ba9d57decd0996fde6544f0
Closes-Bug: #1305423
Remove unnecessary wrapper for 5 compute APIs

There are 5 APIs in compute layer:
check_instance_shared_storage
change_instance_metadata
add_fixed_ip_to_instance
remove_fixed_ip_from_instance
reset_network
has unnecessary wrapper, this patch removed them

reverts_task_state is a wrapper to restore instance task state
when some thing wrong in executing the task,
change_instance_metadata, add_fixed_ip_to_instance
remove_fixed_ip_from_instance, reset_network doesn't change the state,
so no need to wrap it

wrap_instance_fault is a wrapper to catch the exception related to
instances, check_instance_shared_storage is not related to it

Change-Id: I2c4779c01854c2c8bac40e7379ad2fc4d0a7478a
Add x-openstack-request-id to nova v3 responses

x-openstack-request-id is the common header name for request ID going
forward. Nova, cinder, and neutron return this header. Using the
request_id middleware is a convenient way to ensure this header is in
the response. We will also be using the middleware to generate the
request ID.

The ID will be generated by the middleware and is inserted into the
request environment. The current code generates the request ID in the
constructor of RequestContext; with this change, the ID is generated
beforehand and passed in to the RequestContext constructor. Not much
different here: a request ID is generated and is available for use in
the handling of the request.

On the response end, the middleware is again used, this time to
attach the x-openstack-request-id header, using the value of the
generated request ID.

For v3, we will be using the request_id middleware provided in oslo.
For v2, nova-specific middleware (compute_request_id.py) is used.
Therefore, v3 responses will have the header x-openstack-request-id,
and v2 responses will have the header x-compute-request-id (no
change). It is necessary to move the existing code out into
middleware, so that the old header is not attached to v3 responses.

UpgradeImpact: api-paste.ini is modified

DocImpact: v3 responses of the API will only include x-openstack-request-id

Implements: blueprint cross-service-request-id
Change-Id: I5e370fd3de5ee2f8a8d13553015d88910ff5ea87