Pass volume_api to get_encryption_metadata

The module encryptors was creating a volume API instance in the module
scope which caused all the modules importing it to depend on
cinderclient.

This was affecting scheduler and cert services which at some point import
the compute manager module only to access their config options. It makes
no sense to force scheduler and cert services to require cinderclient.

This patch makes the callers of get_encryption_metadata to pass the
volume api object to this method to prevent this dependency.

Closes-Bug: #1246103
Change-Id: I9eb4ae3754fa2a5ac646560a62477d6ed672e272
(cherry picked from commit a3aeace9afd5c533f040452f33482fdb55d93927)
Check for None when cleaning PCI dev usage

The PCI manager needs to check for instance UUIDs mapped to
None when cleaning up it's internal claims/allocations maps.

Also adds a check so the claims dict doesn't map to None in
the first place.  There was already a check like this when
setting a value in the allocations dict.

Closes-Bug: #1238374

Change-Id: I7ac010f7d50ad85c145e536cfd48e07c8f4602d3
(cherry picked from commit c28c87db4c2b20e0f6c1375d2d4433e06c4e1743)
VMware: Handle cases when there are no hosts in cluster

When there are no hosts defined in a cluster the host value will be an empty
string. For example:

    obj_context: (RetrieveResult){
       objects[] =
          (ObjectContent){
             obj =
                (obj){
                   value = "domain-c185"
                   _type = "ClusterComputeResource"
                }
             propSet[] =
                (DynamicProperty){
                   name = "host"
                   val = ""
                },
          },
     }

This causes an data access exception.

The code will now raise an NoValidHost exception.

Fixes bug: 1197041

Change-Id: I75b1f069e0b27eebd63c280e36533354c04e246a
(cherry picked from commit 89e18d8c64b11630d2f21563c95b01452057a897)
VMware: fix VM resize bug

The following operations would fail due to invalid references, either by name
or UUID:
    - resize
    - resize-confirm
    - resize-revert

Closes-bug: #1199954

(cherry picked from commit c586d635387e9baa3c0857afb56d05137fcddd7c)

Conflicts:

	nova/tests/virt/vmwareapi/test_vmwareapi.py

Change-Id: I39b804ed3c0c57455547d9ece0ef779cf73a5926
Require python-cinderclient >= 1.0.6

Havana Nova should require python-cinderclient 1.0.6, which includes
support for features implemented in Havana and already being used by
Nova.  (APIs for assisted snapshots, and volume migration.)

(Landed in master in commit 831c7e6d.)

Change-Id: Ie2efb90a6e8b985f9bb8f9604eedd84e0d8a8a45
Closes-Bug: #1230925
Prevent rescheduling on block device failure

Due to a race condition - it is possible for more instances to race for
the same volume. In such a scenario, the one that fails will get
rescheduled, and in the process detach the volume of a successful
instance.

To prevent this, this patch makes nova not reschedule on block device
failures. This is actually reasonable behaviour as block device failures
are rarely related to the compute host itself and so rescheduling is not
usually useful.

This is a stable/havana only fix! This same issue is addressed on the
master branch by Iefab71047996b7cc08107794d5bc628c11680a70.

Closes-bug: 1195947

Change-Id: I6b68965ac65cdb0e1da3b44e83428f056b1693aa
Fix OLE error for HyperV

When run live-migration on HyperV, an OLE error
will occur. This is because we use wrong WQL
sentence when we try to query table
'Msvm_ResourceAllocationSettingData'.

This commit fixes the wrong WQL sentence.

Closes-Bug: #1246592

Change-Id: Iabb4096002a5be1fb930b077fbf5ef734893c258
(cherry picked from commit e74ceb68e1642d773f2ad0c655e6354f7a7fb362)
Moved quota headroom calculations into quota_reserve

Moved headroom calculations into quota_reserve and modified headroom
calculations to take into account -ve quota limits (unlimited) on
cores and ram.

Updated test cases to generate OverQuota exceptions with new headroom
parameter.

Added new tests for 'unlimited' quotas to test_quota.py in
 QuotaIntegrationTestCase and DbQuotaDriverTestCase classes.

Add Exception kwargs checking for OverQuota and QuotaError exceptions.

Change-Id: I855b0cad495182c5c4ee42b2acf5eedd198557b5
Closes-Bug: #1224453 min_count ignored for instance create
(cherry picked from commit 30fa37e7776831d6f8022f52d3d92f62189fb702)
VMware: Detach volume should not delete vmdk

During volume detach a delete_virtual_disk_spec is used to remove
the device from the running instance. This spec also "destroy" the
underlying vmdk file, this is not right, we should not delete vmdk
file when detach it from a VM instance.

The fix was remove the fileOperation field when detach vmdk volume
so as to make sure the volume will not be destroyed; but for iscsi
we still need to destroy the volume.

Also I changed the function name from delete_virtual_disk_spec to
detach_virtual_disk_spec which is more accurate.

Change-Id: Ibd218d6a8cfeede4f5ca74f28bc0c3d0c185bb14
Closes-Bug: #1241350
(cherry picked from commit 0405d7b937fdeece46efc1ba90171163f3286a56)
Avoid clobbering {system_,}metadata dicts passed to instance update

Right now, the dicts for metadata items passed to instance_update
methods will get clobbered as part of the save. This patch makes
a copy before doing the update to avoid this.

Change-Id: I546692779a7467c5cb309059bb46a9bc9b62d5f1
(cherry picked from commit bbe2c5f08f52b1080a857a7e59c227587aa89717)
Sync local from oslo.

local has a broken TLS symbol - strong_store - in Nova today, fixed in oslo
some time ago in Ib544be1485823f6c619312fdee5a04031f48bbb4

Change-Id: If4dd973acc23921dbc2bc69bb76225deb2802dad
Closes-Bug: #1251920
(cherry picked from commit 9b44d9d75d3767c6b7d1af4f237da8c1cf16266a)
Add missing argument max_size in libvirt driver

max_size argument was added in commit 3cdfe894ab58f7b91bf7fb690fc5bc724e44066f,
but missing  in _create_ephemeral and _create_swap method, which cause
exception raised when caching ephemeral and swap disk files of the instance.

Add test for _create_ephemeral used with max_size parameter
and test _create_swap method

Closes-Bug: 1251152
Closes-Bug: 1250032
(cherry picked from commit 384f55bcb32251698cc2ce839a0286ef99e74984)

Conflicts:
	nova/tests/virt/libvirt/test_libvirt.py
	nova/virt/libvirt/driver.py

Change-Id: I46032ad32f317c923692b8b3d40cedd0dce06cad
Clean BDM when snapshoting volume-backed instances

When snapshoting a volume-backed instance, its block device mapping is
copied entirely into the image properties of the resulting snapshot.

This may lead to some errors when booting from that snapshot because the
fields 'id', 'insance_uuid' and 'connection_info' must be generated for
each instance instead of reusing the ones from the original instance.
For example, sharing the connection_info will cause the new instance to
try to connect to the original volume instead of creating a new one.

This patch cleans all the database-specific plus the connection_info
fields from the block device mapping when creating the snapshot.

Change-Id: I4190e9519d641c61b00b5d86c3885f2ae4fa86f3
Partial-Bug: #1246327
(cherry picked from commit eddd8a14cd2393ce7bdb0ec1c7366177371192b2)
Process image BDM earlier to avoid duplicates

If the block device mapping from the image properties are not handled at
the same time as the ones defined from the API, we might make the false
assumption that the instance has no root and failing or, in the case
we're booting from an image that only defines a BDM and has no disk,
creating a local disk BDM as root and ending up with two root devices
which is forbidden.

This patch moves the handling of the image block device mappings to the
same method were we check the ones provided throught the API. This
allows nova to decide wether the instance needs a local disk from an
image if no root device is defined in any of the block device mappings.

Change-Id: Ide95357895ab4dd1338ab5ee3ec25294af1d010b
Closes-Bug: #1246327
(cherry picked from commit 482dfeb113f1e0ff814aa1fc980d67e0c8b06d76)
Include image block device maps in info

In cases for example soft/hard reboot, we were just getting volume and
snapshot block device maps. For block devices with a source type image,
Nova would fall back on a local disk location and fail.

This change allows us to include block devices with a source of an
image, so that we correctly get the right the location of the block
device.

Closes-Bug: #1245719
Change-Id: I1f726293e85183d4b84c05b635a7c606a092992f
(cherry picked from commit 85d0ace169be513c30b09e13a35fa7d912f5b380)
Fix libvirt evacuate instance on shared storage fails

When evacuate instance which has config drive enabled on shared storage,
disk.config file can't be created because of permision error, chowns
disk.config if it already exists.

Change-Id: I20bafc3482ead30297ad5402a09e97141816f4d1
Closes-Bug: 1247526
(cherry picked from commit 3dfa2f9fd4a3084ae3fb8a7ba5d8a7458ae1c3bc)
Fix up spelling mistake

Fixes bug 1253510

This was causing a KeyError when e2label didn't work

Change-Id: I0646409cdffa4d6d5d8b2c7a9e3c300022d1c9b8
(cherry picked from commit 22d1c2458623a1cfbb91a50c13f18329c863f70b)
VMware: fix snapshot failure when host in maintenance mode

The root cause is due to a bug in the VC's handling of the
VirtualDiskManager.DeleteVirtualDisk_Task API, which allows the picking
of any host in a datacenter with access to the datastore participating
in the disk deletion picked be to perform the operation, even when the
host is in maintenance mode and hence will always reject the call when
sent.

The fix uses an alternative API (FileManager.DeleteDatastoreFile_Task)
to delete the vmdk and -flat vmdk files separately. This API does not
suffer from the above-mentioned failure mode.

Closes-Bug: #1229994

(cherry picked from commit 7910385825ccfa705785af360fcd5717656e3557)

Conflicts:
	nova/virt/vmwareapi/fake.py
	nova/virt/vmwareapi/vmops.py

Change-Id: I786365847673e5192a21b654cba951b2e7a6f291
Merge "Check for None when cleaning PCI dev usage" into stable/havana
Merge "Pass volume_api to get_encryption_metadata" into stable/havana