versions API: ignore request with a body

Update the OS API versions controller so it ignores requests
with a body. Previously an incoming (unauthenticated)
request to the versions API would log a TypeError trace
in the nova-api.log file with the following message:

TypeError: index() got an unexpected keyword argument 'body'

Change-Id: Icc3ccfc537b13627b8d5ba43ae3ba91e34e08c9e
Closes-bug: #1306727
VMware: prevent image snapshot if no root disk defined

If there is no root disk defined then the snapshot should
throw an exception.

Change-Id: I0549b8ca9ba41b734277600cb66a74b27ca72d88
Related-bug: #1297605
(cherry picked from commit e0638e29cda39c609ecf518e5adacf70e158d71a)
Fixes Hyper-V iSCSI target login method

Hyper-V is trying to log in to portals/targets every time a volume
is attached, even if the portal/target was already connected. This
is not an issue with most volume backends, but it causes the attach
to fail when using Storwize as a Cinder backend.

This patch fixes the issue by checking if the target or portal is
already connected, and simply perform an update in this case,
instead of attempting to create a new connection.

Also, a retry loop is introduced for the case when the iSCSI target
login fails. As in this loop it is checked whether or not the
operation completed successfully, this method does not rely anymore
only on a sleep interval to ensure that the connection to the target
has been made. Even so, on V2 volume utilities, the sleep interval
is still needed to avoid having a new connection attempt before the
last one finished.

The method which gets the according mounted disk must raise an
exception when the device number is -1, which signifies that the
disk has not been mounted properly.

Co-Authored-By: Jay Bryant <jsbryant@us.ibm.com>

Closes-bug: #1221525

Change-Id: I92d2a586704fa3e857f46432bb571c81e86d183b
(cherry picked from commit ea91ad2b2317cb88fb14c0cdaf31187eca93ff57)
Include pending task in log message on skip sync_power_state

To better understand why sync_power_state is being skipped include the
pending task name in the log message.

Change-Id: I832a3d5820efab301e12610b28ca2a1e5cfd946a
(cherry picked from commit affbbca5af2506aafb743211c901f7f0ca4e131a)
VMware: Fix memory leaks caused by caches

Using suds objects as keys for the DatastoreBrowser cache is incorrect
because they don't implement __eq__ and __hash__ for the VIM types.
This always results in cache miss and the cache grows with every spawn()
operation.

This patch fix this by using the 'value' property (which is string) of
the MoRef as key.

Closes-Bug: 1316433
(cherry picked from commit bceb3f96b06ff8a048598724494299cf111bcaf8)

Conflicts:

	nova/tests/virt/vmwareapi/test_imagecache.py
	nova/tests/virt/vmwareapi/test_vmops.py

Change-Id: I2bcaf87e733d51055566aee41bb0a7e254027ba9
VCDriver - Ignore host in Maintenance mode in stats update

The VCDriver during stats update checks only for connected hosts.
However hosts which are in connected state to the Cluster can
be set to Maintenance mode.
The cluster does not boot instances to hosts which are in
maintenance mode.
Hence the VCDriver should check for hosts which are in
maintenance mode and ignore them for stats update.
VISDK API returns True for the property inMaintenanceMode for
Hosts which are in maintenance mode

Change-Id: Ieaf7530ed2d98c59c5450e8cb75e209823193d49
Closes-Bug: #1316674
(cherry picked from commit 4cb139fb99814f04b9e4595d7e2efa34e7992826)
libvirt: Refresh volume connection_info after volume snapshot

The following patch is related to the guest assisted snapshot
functionality.  When you take a snapshot of a volume
(e.g. GlusterFS) attached to a running instance, a new snapshot
file is created, i.e. volume-<uuid>.<snapshot-uuid>.  The
instance uses this file as the active volume.  If you shutdown
and restart the instance, nova will reattach the base volume
(volume-<uuid>) to the instance instead of the snapshot volume
(volume-<uuid>.<snapshot-uuid>).  The expected behavior is to
have the snapshot volume reattach to the instance.  This is
caused by stale data being returned from the database when
_get_instance_volume_block_device_info is called during
_power_on.  To fix this bug, this patch calls
refresh_connection_info to update the database in both
volume_snapshot_create and _volume_snapshot_delete methods of the
libvirt driver.

Conflicts:
	nova/tests/virt/libvirt/test_libvirt.py

Change-Id: I0f340a3f879580e7981d97863bc299e33d71aa84
Closes-Bug: #1304695
Neutronv2 api does not support neutron without port quota

If quota on ports are disabled in neutron (see below), neutronv2 api
set port quota to None and quota check raises a TypeError.

  # neutron server /etc/neutron/neutron.conf
  [quotas]
  quota_items=network,subnet,router,floatingip

This change set port quota to -1 (unlimited) if quota on ports are
disabled.

Change-Id: I89c530dd57a7beb35695be124ca8511148378458
Closes-Bug: #1333315
(cherry picked from commit 04add2ed0d8b0341ea347ac98834f75cf5b2be1d)
Fix security group list when not defined for an instance

If an instance has an empty security group list an exception
is thrown due to iteration attempt of None.

To fix this, this patch modifies get_instance_security_group
to return an empty list instead of None.

Change-Id: I7efb3c3a58d7e20501d126c6d194479ca1d14166
Closes-Bug: #1291489
(cherry picked from commit c32986966dfd034c3d706b2e9ab2820a2c3cfc3e)
VMware: Fix fake raising the wrong exception in _remove_file

fake was raising exception.FileNotFound error if a file was not found.
However, an actual delete on a non-existent file raises the error
returned by vSphere, which is error_util.FileNotFoundException. This
breaks tests which exercise error paths expecting the real error to be
raised.

Fortunately this change doesn't break any existing tests. However, I'm
not sure that's a good thing.

This change is required to enable a test in change
I3df3d614656e511c909b6c1837582c0d34bf84c6.

Change-Id: I2025bffa887582eaa9e9072d0400f90ca97d1898
(cherry picked from commit 529b5ab2e195181687c7cc30ece8ddbf160bdeab)
VMware: Fix race in spawn() when resizing cached image

spawn() guards against multiple threads simultaneously attempting to
cache the same image, but it wasn't guarding against them
simultanously trying to create a resized copy in the cache. Attempting
to create a large number of images simultaneously of an uncached image
would result in a race to create the resized image. This resulted in 2
classes of failed instance:

1. Instances whose disk was a linked clone of a copy which had been
   subsequently overwritten. These were corrupt.
2. Instances whose spawn() failed in ExtendVirtualDisk_Task due to a
   locked image.

This patch creates a Nova-local lock for the resized image. The image
is in a per-Nova directory on the datastore, so inter-Nova locking is
not a concern. The lock guards both testing for the existence of the
image, and its creation. Therefore when multiple processes race, only
1 will create the resized copy, and all others will find and use it.
In normal usage this will add the overhead of an additional
uncontended local lock creation and deletion in spawn().

In wrapping this code in a lock, we also make certain that any failure
to create the resized image is appropriately cleaned up. Otherwise
subsequent users will attempt to use a corrupt copy.

Conflicts:
	nova/virt/vmwareapi/vmops.py

Required changes:
  datastore.build_path -> ds_util.build_datastore_path
  No datastore object
  vm_util.copy_virtual_disk -> _copy_virtual_disk
  self.fake_image_uuid -> 'fake_image_uuid'
  removed use of _LE() for log message

This change includes a test which depends on change
I2025bffa887582eaa9e9072d0400f90ca97d1898.

Change-Id: I3df3d614656e511c909b6c1837582c0d34bf84c6
Closes-bug: 1333587
(cherry picked from commit 994cdb234b2b16d97f0276c6356db65817944ee2)
Fixes rbd backend image size

The original fix for bug 1219658 introduced a factor of 1024 error
in the resulting rbd image size.

Fix bug #1219658

Change-Id: I3ec53b3617d52f75784ebb3b0dad92ca815f8876
Co-authored-by: Nikola Dipanov <ndipanov@redhat.com>
Signed-off-by: Haomai Wang <haomai@unitedstack.com>
(cherry picked from commit f27ca3134743b45e55afd6c73abbb83c87ba8489)
shelve doesn't work on nova-cells environment

The compute_api shelve/unshelve methods expect an Instance object,
but cell is still passing the sqlalchemy form.
So adding shelve/unshelve to the list of methods that will receive an
Instance object when using nova-cell.

When shelving instance with nova-cell environment,
instance object needs to contain 'metadata' and 'info_cache'.
So fetching the same from databse and adding it to
instance object before calling shelve/unshelve methods.

Conflicts:
    nova/cells/messaging.py

Closes-Bug: #1338451
Change-Id: Ia31b33c9a1d347f15d3e0a67fa31ef6b25a7407a
(cherry picked from commit 9d4b49c542e2076c8a572d1e8c6d50e255efe087)
Handle service creation race by service workers

This handle a race where a sibling has already created the service
record in the database.  This fix is similar to that of commit
f6c341b4.

Change-Id: I7975fc632eee34390c5d7dc275b52363ad45475e
Closes-bug: 1326901
(cherry picked from commit 37247aba813fbc408901a2f1b1ad88e2acb796a1)
Updated from global requirements

Change-Id: If1677541081467bb464b0fd265ce4340a67d8862
Turn periodic tasks off in all unit tests

We shouldn't rely on periodic tasks to randomly run in unit tests,
instead if we need them to run we should explicitly call them out.

Change-Id: I35850c47288e50d3146771dc7a9a54d62e36c3cf
Related-Bug: 1311778
(cherry picked from commit f3f07f4401ff8be25598493c088e4e100b387fe4)
Attach/detach interface to paused instance with affect live flag

Currently attach/detach interface to paused instance is only with
VIR_DOMAIN_AFFECT_CONFIG flags. After instance unpause, there isn't
any effect on the instance, unless restart the instance. So add
flag VIR_DOMAIN_AFFECT_LIVE when attach/detach interface.

Conflicts:
	nova/tests/virt/libvirt/test_libvirt.py

Change-Id: Iedfc7f6fc06e72d8e3eecebede1274582c8043fd
Closes-Bug: #1299331
(cherry picked from commit a868fcedf8e46070cae6aa8e59e61934fa23db1c)
Do not process events for instances without host

In some cases Neutron might send events such as 'VIF unplugged'
for instances which are either being deleted or shelved. When
that happens there will be a failure in dispatching the event
to the appropriate compute node - as there is no host for the
instance.

As multiple neutron events can be stashed in a single call
it is important to avoid that this kind of errors will prevent
processing of other events in the same call.

This patch does not process events for instances without a host,
marking them as failed.

When the above condition occurs, the create event request will
return a 207 response code. For specific events, a 422
unprocessable entity code will be set.

This patch also preserve the characteristic that events are
returned in the response in the same order they were found in
the request.

Change-Id: I18062b81e50c722ec96b4296ac39384493683ae3
Closes-Bug: #1333654
(cherry picked from commit 4f8ccd7b95c27180a1cfe689e3c6f46bde5f803b)
Do not pass instances without host to compute API

Even if the server external events extension filters out events
whose related instance does not have a host, the corresponding
instance is still sent to the compute API module.
As this might result in KeyError, instance without host should
be filtered out before calling the compute API module.

Change-Id: If5229ec3059076dbc9f4abb6625504e8864c265e
Closes-Bug: #1348584
(cherry picked from commit 5e4a5f0d8c62ca6e94ae6db16e9fbe0428805158)
Set python hash seed to 0 in tox.ini

New tox (>=1.7.0) sets a random python hash seed by default. This is
generally good for testing because it will help keep projects working
regardless of the hash seed, but nova unittests don't currently pass
with a random hash seed so set it to the python default seed.

This change will allow us to use new tox again and remove the
restriction on tox<=1.6.1 to run unittests.

Redundant setenvs for specific test envs are removed to prevent them
from overriding the PYTHONHASHSEED value.

Note this change will need to be backported to the stable branches to
keep unittests there working with new tox as well.

Change-Id: Ib54364877a251db48c54dfdc43c503281ea1f04a
(cherry picked from commit 4f3e3cd1883ea5aca237bcc88b64128c743a6862)