Fixes Invalid tag name error when using k:v tagname

This patch fixes the invalid tag name error when k:v type
tagname appears in response of api during xml serialization,
the tagname which contains colon couldn't be supported by
default since it is reserved by xml namespace, this patch
solve it by adding the k of k:v tagname into namespace.

Fixes bug: #1213927

Change-Id: Ib20efb378425cbe5deef45fbac561955cbe0f7bc
(cherry picked from commit 80b11279ba9853a9ab743ec6987c3229fff0b5e4)
hyperv: Initialize target_iqn in attach_volume

The target_iqn is referenced before it is declared in the
case that an exception is encountered in _login_storage_target
or _get_mounted_disk_from_lun.  This causes the exception path
to cause an additional, misleading, traceback to be printed
after an exception is encountered.

This commit initializes target_iqn at the beginning of
attach_volume to avoid the traceback.

Closes-Bug: 1233837
Change-Id: I1cdfb9dfa054ee4f4f6918d7bc49081ccaebab9d
(cherry picked from commit 9baa186ebb578205c513c94fc621e254f4c13a08)
Cells: Fix instance deletes

A regression was introduced with object conversion that caused all
instances to be locally deleted in the API cell when cells is enabled.
This would happen before the child cell was told to delete the instance.

Additionally, if an instance record was created in a child cell but
failed to schedule to a host in such a way that the API cell has
no knowledge of the 'host', the instance would be deleted in the API
cell only.

Also, it appears that we would cast deletes to child cells twice due to
logic duplicated in cells_api.py that exists in api.py.

Closes-Bug: #1240247

Change-Id: I94cfddd9cef2d63b2cb31c0f2dae8e211a4418d9
(cherry picked from commit 0493c803ae9612f87ed028e1a39e880aead5bdcb)
Pass volume_api to get_encryption_metadata

The module encryptors was creating a volume API instance in the module
scope which caused all the modules importing it to depend on
cinderclient.

This was affecting scheduler and cert services which at some point import
the compute manager module only to access their config options. It makes
no sense to force scheduler and cert services to require cinderclient.

This patch makes the callers of get_encryption_metadata to pass the
volume api object to this method to prevent this dependency.

Closes-Bug: #1246103
Change-Id: I9eb4ae3754fa2a5ac646560a62477d6ed672e272
(cherry picked from commit a3aeace9afd5c533f040452f33482fdb55d93927)
Check for None when cleaning PCI dev usage

The PCI manager needs to check for instance UUIDs mapped to
None when cleaning up it's internal claims/allocations maps.

Also adds a check so the claims dict doesn't map to None in
the first place.  There was already a check like this when
setting a value in the allocations dict.

Closes-Bug: #1238374

Change-Id: I7ac010f7d50ad85c145e536cfd48e07c8f4602d3
(cherry picked from commit c28c87db4c2b20e0f6c1375d2d4433e06c4e1743)
VMware: Handle cases when there are no hosts in cluster

When there are no hosts defined in a cluster the host value will be an empty
string. For example:

    obj_context: (RetrieveResult){
       objects[] =
          (ObjectContent){
             obj =
                (obj){
                   value = "domain-c185"
                   _type = "ClusterComputeResource"
                }
             propSet[] =
                (DynamicProperty){
                   name = "host"
                   val = ""
                },
          },
     }

This causes an data access exception.

The code will now raise an NoValidHost exception.

Fixes bug: 1197041

Change-Id: I75b1f069e0b27eebd63c280e36533354c04e246a
(cherry picked from commit 89e18d8c64b11630d2f21563c95b01452057a897)
VMware: fix VM resize bug

The following operations would fail due to invalid references, either by name
or UUID:
    - resize
    - resize-confirm
    - resize-revert

Closes-bug: #1199954

(cherry picked from commit c586d635387e9baa3c0857afb56d05137fcddd7c)

Conflicts:

	nova/tests/virt/vmwareapi/test_vmwareapi.py

Change-Id: I39b804ed3c0c57455547d9ece0ef779cf73a5926
Require python-cinderclient >= 1.0.6

Havana Nova should require python-cinderclient 1.0.6, which includes
support for features implemented in Havana and already being used by
Nova.  (APIs for assisted snapshots, and volume migration.)

(Landed in master in commit 831c7e6d.)

Change-Id: Ie2efb90a6e8b985f9bb8f9604eedd84e0d8a8a45
Closes-Bug: #1230925
Prevent rescheduling on block device failure

Due to a race condition - it is possible for more instances to race for
the same volume. In such a scenario, the one that fails will get
rescheduled, and in the process detach the volume of a successful
instance.

To prevent this, this patch makes nova not reschedule on block device
failures. This is actually reasonable behaviour as block device failures
are rarely related to the compute host itself and so rescheduling is not
usually useful.

This is a stable/havana only fix! This same issue is addressed on the
master branch by Iefab71047996b7cc08107794d5bc628c11680a70.

Closes-bug: 1195947

Change-Id: I6b68965ac65cdb0e1da3b44e83428f056b1693aa
Fix OLE error for HyperV

When run live-migration on HyperV, an OLE error
will occur. This is because we use wrong WQL
sentence when we try to query table
'Msvm_ResourceAllocationSettingData'.

This commit fixes the wrong WQL sentence.

Closes-Bug: #1246592

Change-Id: Iabb4096002a5be1fb930b077fbf5ef734893c258
(cherry picked from commit e74ceb68e1642d773f2ad0c655e6354f7a7fb362)
Moved quota headroom calculations into quota_reserve

Moved headroom calculations into quota_reserve and modified headroom
calculations to take into account -ve quota limits (unlimited) on
cores and ram.

Updated test cases to generate OverQuota exceptions with new headroom
parameter.

Added new tests for 'unlimited' quotas to test_quota.py in
 QuotaIntegrationTestCase and DbQuotaDriverTestCase classes.

Add Exception kwargs checking for OverQuota and QuotaError exceptions.

Change-Id: I855b0cad495182c5c4ee42b2acf5eedd198557b5
Closes-Bug: #1224453 min_count ignored for instance create
(cherry picked from commit 30fa37e7776831d6f8022f52d3d92f62189fb702)
VMware: Detach volume should not delete vmdk

During volume detach a delete_virtual_disk_spec is used to remove
the device from the running instance. This spec also "destroy" the
underlying vmdk file, this is not right, we should not delete vmdk
file when detach it from a VM instance.

The fix was remove the fileOperation field when detach vmdk volume
so as to make sure the volume will not be destroyed; but for iscsi
we still need to destroy the volume.

Also I changed the function name from delete_virtual_disk_spec to
detach_virtual_disk_spec which is more accurate.

Change-Id: Ibd218d6a8cfeede4f5ca74f28bc0c3d0c185bb14
Closes-Bug: #1241350
(cherry picked from commit 0405d7b937fdeece46efc1ba90171163f3286a56)
Merge "Fixes Invalid tag name error when using k:v tagname" into stable/havana
Merge "Cells: Fix instance deletes" into stable/havana
Merge "hyperv: Initialize target_iqn in attach_volume" into stable/havana
Avoid clobbering {system_,}metadata dicts passed to instance update

Right now, the dicts for metadata items passed to instance_update
methods will get clobbered as part of the save. This patch makes
a copy before doing the update to avoid this.

Change-Id: I546692779a7467c5cb309059bb46a9bc9b62d5f1
(cherry picked from commit bbe2c5f08f52b1080a857a7e59c227587aa89717)
Sync local from oslo.

local has a broken TLS symbol - strong_store - in Nova today, fixed in oslo
some time ago in Ib544be1485823f6c619312fdee5a04031f48bbb4

Change-Id: If4dd973acc23921dbc2bc69bb76225deb2802dad
Closes-Bug: #1251920
(cherry picked from commit 9b44d9d75d3767c6b7d1af4f237da8c1cf16266a)
Add missing argument max_size in libvirt driver

max_size argument was added in commit 3cdfe894ab58f7b91bf7fb690fc5bc724e44066f,
but missing  in _create_ephemeral and _create_swap method, which cause
exception raised when caching ephemeral and swap disk files of the instance.

Add test for _create_ephemeral used with max_size parameter
and test _create_swap method

Closes-Bug: 1251152
Closes-Bug: 1250032
(cherry picked from commit 384f55bcb32251698cc2ce839a0286ef99e74984)

Conflicts:
	nova/tests/virt/libvirt/test_libvirt.py
	nova/virt/libvirt/driver.py

Change-Id: I46032ad32f317c923692b8b3d40cedd0dce06cad
Clean BDM when snapshoting volume-backed instances

When snapshoting a volume-backed instance, its block device mapping is
copied entirely into the image properties of the resulting snapshot.

This may lead to some errors when booting from that snapshot because the
fields 'id', 'insance_uuid' and 'connection_info' must be generated for
each instance instead of reusing the ones from the original instance.
For example, sharing the connection_info will cause the new instance to
try to connect to the original volume instead of creating a new one.

This patch cleans all the database-specific plus the connection_info
fields from the block device mapping when creating the snapshot.

Change-Id: I4190e9519d641c61b00b5d86c3885f2ae4fa86f3
Partial-Bug: #1246327
(cherry picked from commit eddd8a14cd2393ce7bdb0ec1c7366177371192b2)
Process image BDM earlier to avoid duplicates

If the block device mapping from the image properties are not handled at
the same time as the ones defined from the API, we might make the false
assumption that the instance has no root and failing or, in the case
we're booting from an image that only defines a BDM and has no disk,
creating a local disk BDM as root and ending up with two root devices
which is forbidden.

This patch moves the handling of the image block device mappings to the
same method were we check the ones provided throught the API. This
allows nova to decide wether the instance needs a local disk from an
image if no root device is defined in any of the block device mappings.

Change-Id: Ide95357895ab4dd1338ab5ee3ec25294af1d010b
Closes-Bug: #1246327
(cherry picked from commit 482dfeb113f1e0ff814aa1fc980d67e0c8b06d76)