versions API: ignore request with a body

Update the OS API versions controller so it ignores requests
with a body. Previously an incoming (unauthenticated)
request to the versions API would log a TypeError trace
in the nova-api.log file with the following message:

TypeError: index() got an unexpected keyword argument 'body'

Change-Id: Icc3ccfc537b13627b8d5ba43ae3ba91e34e08c9e
Closes-bug: #1306727
VMware: prevent image snapshot if no root disk defined

If there is no root disk defined then the snapshot should
throw an exception.

Change-Id: I0549b8ca9ba41b734277600cb66a74b27ca72d88
Related-bug: #1297605
(cherry picked from commit e0638e29cda39c609ecf518e5adacf70e158d71a)
Fixes Hyper-V iSCSI target login method

Hyper-V is trying to log in to portals/targets every time a volume
is attached, even if the portal/target was already connected. This
is not an issue with most volume backends, but it causes the attach
to fail when using Storwize as a Cinder backend.

This patch fixes the issue by checking if the target or portal is
already connected, and simply perform an update in this case,
instead of attempting to create a new connection.

Also, a retry loop is introduced for the case when the iSCSI target
login fails. As in this loop it is checked whether or not the
operation completed successfully, this method does not rely anymore
only on a sleep interval to ensure that the connection to the target
has been made. Even so, on V2 volume utilities, the sleep interval
is still needed to avoid having a new connection attempt before the
last one finished.

The method which gets the according mounted disk must raise an
exception when the device number is -1, which signifies that the
disk has not been mounted properly.

Co-Authored-By: Jay Bryant <jsbryant@us.ibm.com>

Closes-bug: #1221525

Change-Id: I92d2a586704fa3e857f46432bb571c81e86d183b
(cherry picked from commit ea91ad2b2317cb88fb14c0cdaf31187eca93ff57)
Include pending task in log message on skip sync_power_state

To better understand why sync_power_state is being skipped include the
pending task name in the log message.

Change-Id: I832a3d5820efab301e12610b28ca2a1e5cfd946a
(cherry picked from commit affbbca5af2506aafb743211c901f7f0ca4e131a)
VMware: Fix memory leaks caused by caches

Using suds objects as keys for the DatastoreBrowser cache is incorrect
because they don't implement __eq__ and __hash__ for the VIM types.
This always results in cache miss and the cache grows with every spawn()
operation.

This patch fix this by using the 'value' property (which is string) of
the MoRef as key.

Closes-Bug: 1316433
(cherry picked from commit bceb3f96b06ff8a048598724494299cf111bcaf8)

Conflicts:

	nova/tests/virt/vmwareapi/test_imagecache.py
	nova/tests/virt/vmwareapi/test_vmops.py

Change-Id: I2bcaf87e733d51055566aee41bb0a7e254027ba9
VCDriver - Ignore host in Maintenance mode in stats update

The VCDriver during stats update checks only for connected hosts.
However hosts which are in connected state to the Cluster can
be set to Maintenance mode.
The cluster does not boot instances to hosts which are in
maintenance mode.
Hence the VCDriver should check for hosts which are in
maintenance mode and ignore them for stats update.
VISDK API returns True for the property inMaintenanceMode for
Hosts which are in maintenance mode

Change-Id: Ieaf7530ed2d98c59c5450e8cb75e209823193d49
Closes-Bug: #1316674
(cherry picked from commit 4cb139fb99814f04b9e4595d7e2efa34e7992826)
Set the volume access mode during volume attach

cinderclient supports an optional keyword argument named 'mode'
for volumes.attach(), which allows you to set the access mode of
a volume at attach time. The keyword argument is not passed in
by nova when it attaches a read-only volume, making the volume
access mode default to 'rw' instead of the desired 'ro'. This
patch finds the access mode of the volume and passes it to
cinderclient, so the correct access mode is used at attach
time.

Change-Id: Ib62f454292e60114f96debbad96c4a356dc32a21
Closes-Bug: #1307088
(cherry picked from commit de05c2994212e83eba3cedf2544cdc549d9f8184)
libvirt: Refresh volume connection_info after volume snapshot

The following patch is related to the guest assisted snapshot
functionality.  When you take a snapshot of a volume
(e.g. GlusterFS) attached to a running instance, a new snapshot
file is created, i.e. volume-<uuid>.<snapshot-uuid>.  The
instance uses this file as the active volume.  If you shutdown
and restart the instance, nova will reattach the base volume
(volume-<uuid>) to the instance instead of the snapshot volume
(volume-<uuid>.<snapshot-uuid>).  The expected behavior is to
have the snapshot volume reattach to the instance.  This is
caused by stale data being returned from the database when
_get_instance_volume_block_device_info is called during
_power_on.  To fix this bug, this patch calls
refresh_connection_info to update the database in both
volume_snapshot_create and _volume_snapshot_delete methods of the
libvirt driver.

Conflicts:
	nova/tests/virt/libvirt/test_libvirt.py

Change-Id: I0f340a3f879580e7981d97863bc299e33d71aa84
Closes-Bug: #1304695
Use correct project/user for quotas

This fixes the compute manager to ensure the proper project_id/user_id
is used when making quota calls by converting most of the compute
manager code to use the new objects. This does not change the RPC
messages to the manager, yet.

Change-Id: I50601d12edef735f3c01ad73445e0826d05efc51
Closes-bug: 1271541
(cherry picked from commit 0a717d7085bd2bca7e5408a5c20e8ae7cd055756)
Neutronv2 api does not support neutron without port quota

If quota on ports are disabled in neutron (see below), neutronv2 api
set port quota to None and quota check raises a TypeError.

  # neutron server /etc/neutron/neutron.conf
  [quotas]
  quota_items=network,subnet,router,floatingip

This change set port quota to -1 (unlimited) if quota on ports are
disabled.

Change-Id: I89c530dd57a7beb35695be124ca8511148378458
Closes-Bug: #1333315
(cherry picked from commit 04add2ed0d8b0341ea347ac98834f75cf5b2be1d)
Avoid referencing stale instance/network_info dicts in firewall

This makes the virt.firewall code cleaner in terms of referencing
the cached instance and network_info code it stores. Before this
patch, concurrent instance operations could modify these two dicts
so that while we're iterating instances, the network_info dict
is suddenly missing information we need.

The right fix for this is to use instance objects and their
associated info_cache objects, but that's a larger fix and one
not as well-suited to backporting to previous releases which
suffer from this as well.

The approach taken here is that we store the instance and
network_info cache together in the same dict that we can pop()
from atomically (this is not really necessary, but helps to
prevent introducing more of these cases). When we iterate over
the contents, we iterate over a copy of the keys, being careful
not to let a suddenly-missing key break us, and passing the
details all the way down the stack instead of having deeper calls
hit the cache dicts again.

Conflicts:
	nova/virt/firewall.py

Change-Id: Ib1cfd68fca7789ca4a3602c25e6b8a956303fff9
Closes-bug: #1182131
(cherry picked from commit 72dd81343e73baf838bc58d413413f0d57018f15)
Make sure domain exists before referencing it

Before calling domain.destroy() in the exception handler for a failed
create attempt, make sure domain was created prior to when the
exception was thrown.

Conflicts:
        nova/virt/libvirt/driver.py

Change-Id: I3b7c5afce7f3bacce1b444c245b035334962ff7c
Closes-Bug: #1326998
(cherry picked from commit 5a98983e87e5020626db2e319f7b00ae547ae8e1)
libvirt: Make nwfilter driver use right filterref

This fixes Bug #1220856 which occurs with Libvirt NWFilterFirewall
Driver.  While creating a VM with multiple NICs including one connects
to DHCP serving network and another to DHCP non-serving network,
NWFilterFirewall Driver will use the same base filter that allows DHCP
for all NICs. This leads a libvirt launch error due to lack of
'dhcp_server' parameter which is needed to define allow-DHCP filters.

This patch makes NWFilterFirewall Driver use right base filter for
each NIC depends on 'dhcp_server' config.

Closes-bug: #1220856
Change-Id: I7f9a7c281f152985478b2ec295f0644ba475fd76
(cherry picked from commit d2874463010b129e62a568357cfa1df3ab4805e6)
Fix security group list when not defined for an instance

If an instance has an empty security group list an exception
is thrown due to iteration attempt of None.

To fix this, this patch modifies get_instance_security_group
to return an empty list instead of None.

Change-Id: I7efb3c3a58d7e20501d126c6d194479ca1d14166
Closes-Bug: #1291489
(cherry picked from commit c32986966dfd034c3d706b2e9ab2820a2c3cfc3e)
Merge "Avoid referencing stale instance/network_info dicts in firewall" into stable/icehouse
Merge "Make sure domain exists before referencing it" into stable/icehouse
VMware: Fix fake raising the wrong exception in _remove_file

fake was raising exception.FileNotFound error if a file was not found.
However, an actual delete on a non-existent file raises the error
returned by vSphere, which is error_util.FileNotFoundException. This
breaks tests which exercise error paths expecting the real error to be
raised.

Fortunately this change doesn't break any existing tests. However, I'm
not sure that's a good thing.

This change is required to enable a test in change
I3df3d614656e511c909b6c1837582c0d34bf84c6.

Change-Id: I2025bffa887582eaa9e9072d0400f90ca97d1898
(cherry picked from commit 529b5ab2e195181687c7cc30ece8ddbf160bdeab)
Avoid re-adding iptables rules for instances that have disappeared

The remove_filters_for_instance() method fails silently if the
instance's chain is gone (i.e. it's been deleted). If this
happens while we're refreshing security group rules, we will
not notice this case and re-add stale rules for an old instance,
breaking our firewall for new instances.

This adds a quick check after we've captured the lock to see if
the associated chain exists, and bails if it doesn't.

Change-Id: Ic75988939f82de49735d85fe99a9eecd4baf45c9
Related-bug: #1182131
(cherry picked from commit 6aa368b99249d01f8fd7183c15d11986ad6a6fb7)
VMware: Fix race in spawn() when resizing cached image

spawn() guards against multiple threads simultaneously attempting to
cache the same image, but it wasn't guarding against them
simultanously trying to create a resized copy in the cache. Attempting
to create a large number of images simultaneously of an uncached image
would result in a race to create the resized image. This resulted in 2
classes of failed instance:

1. Instances whose disk was a linked clone of a copy which had been
   subsequently overwritten. These were corrupt.
2. Instances whose spawn() failed in ExtendVirtualDisk_Task due to a
   locked image.

This patch creates a Nova-local lock for the resized image. The image
is in a per-Nova directory on the datastore, so inter-Nova locking is
not a concern. The lock guards both testing for the existence of the
image, and its creation. Therefore when multiple processes race, only
1 will create the resized copy, and all others will find and use it.
In normal usage this will add the overhead of an additional
uncontended local lock creation and deletion in spawn().

In wrapping this code in a lock, we also make certain that any failure
to create the resized image is appropriately cleaned up. Otherwise
subsequent users will attempt to use a corrupt copy.

Conflicts:
	nova/virt/vmwareapi/vmops.py

Required changes:
  datastore.build_path -> ds_util.build_datastore_path
  No datastore object
  vm_util.copy_virtual_disk -> _copy_virtual_disk
  self.fake_image_uuid -> 'fake_image_uuid'
  removed use of _LE() for log message

This change includes a test which depends on change
I2025bffa887582eaa9e9072d0400f90ca97d1898.

Change-Id: I3df3d614656e511c909b6c1837582c0d34bf84c6
Closes-bug: 1333587
(cherry picked from commit 994cdb234b2b16d97f0276c6356db65817944ee2)
Fixes rbd backend image size

The original fix for bug 1219658 introduced a factor of 1024 error
in the resulting rbd image size.

Fix bug #1219658

Change-Id: I3ec53b3617d52f75784ebb3b0dad92ca815f8876
Co-authored-by: Nikola Dipanov <ndipanov@redhat.com>
Signed-off-by: Haomai Wang <haomai@unitedstack.com>
(cherry picked from commit f27ca3134743b45e55afd6c73abbb83c87ba8489)