VMWare Driver - Ignore datastore in maintenance mode

A datastore can be part of a datastore cluster. Any
datastore which is part of a cluster can be put in maintenance
mode. Instances cannot reside on a datastore if it is in
maintenance mode.
This fix ignores a datastore in the following scenarios
1. Stats update - Ignore a datastore during stats update.
2. Ignore a datastore during instance spawn.

Change-Id: Id0cb3f5fdb4d0bc4884cf5405fadc433d4a2b6ba
Closes-Bug: #1330065
Fix fake_update in test_update_missing_server

In test_update_missing_server, fake_update should be used as update()
method but it is used as create() instead. In addition, the test comment
also is wrong.
This patch fixes them.

Change-Id: Ia4704413dd7355010546dba7096ed34225bb4b0f
Merge BadRequest tests of "attach interfaces" API

In test_attach_interfaces, there are BadRequest tests of "attach interfaces".
Most parts of them are duplicated.
This patch merges them for the readability and clarifying their purposes.

Change-Id: Ib9dd2840be46dec6b70fe9e3fda35de88f84b4a9
libvirt: fix _disk_resize to make sure converted image will be restored

During the process of resizing disk if an image is in qcow2 with
partition less the process converts the image to raw.
After the extend we should to restore the original format in all cases
not only if 'use_cow_images' is configured to True.

Change-Id: I792e0fb986a6c5cf9ac477cef0949c8a40099faa
Closes-Bug: #1298976
libvirt: skip disk resize when resize_instance is False

The method finish_migration is called with an extra
parameter 'resize_instance'. It should be used to know
if it is necessary or not to resize the disks and so
call the method 'disk_resize'.

Change-Id: Ia5a9ad7994fa9abab7cf47d36468c46d97065991
Closes-Bug: #1298981
Split EC2 ID validator to validator per resource type.

Now there is only one method to validate EC2 id: validate_ec2_id().
It's used to validate both the Instance ID as well as Volume ID for
valid EC2 ID format. However exception class raised in both cases
is "InvalidInstanceIDMalformed" and EC2 error code is
InvalidInstanceID.Malformed.

This patch addresses this problem by adding two new methods:
* validate_instance_id()
* validate_volume_id()
These methods use validate_ec2_id() and raise appropriate
exception as well as appropriate EC2 error code based on
AWS EC2 Error Code documentation:
http://docs.aws.amazon.com/AWSEC2/latest/APIReference/api-error-codes.html

Change-Id: I55591b8b643bb316e5001e645d71c13094106e96
Closes-Bug: #1199308
Fix 202 responses to contain valid content.

Using a webob.exc to generate 202s makes the response return plain
text in the body, which breaks things that expect to be able to
parse data according to the content type. Using a webob.Response
object makes sure the body of the 202 response is empty.

Change-Id: Ifca3081f573407808a2196e71ac1db87180db6ec
Resolves-bug: #1356058
Fix Trusted Filter to work with Mt. Wilson `vtime`

The Trusted Filter expects that `vtime` is returned from the
attestation server in an ISO8601 format, but the Mt. Wilson
attestation server can return it in a locale appropriate string format
instead. There's no way to configure Mt.  Wilson to do the right thing
so we should just try to parse the returned `vtime` as a string
formatted date.

Closes-Bug: #1353029

Change-Id: Ic7351e0463c014321bdb4fcfeba90ac51460b325
Add unit tests for libvirt domain creation

Tests for:
nova.virt.libvirt.driver.LibvirtDriver.get_guest_config
nova.virt.libvirt.driver.LibvirtDriver._create_domain

Partially implements: bp libvirt-lxc-user-namespaces

Change-Id: I4aada9ac06cd423b04a4c6136165c65e64e378aa
Use flavor in confirm-resize to drop claim

Current the confirm_resize() clean the stashed instance_type information
before invoking the drop_resize_claim, and the drop_resize_claim() try to
fetch the old instance_type in system metadata, which will be wrong.

We change the _cleanup_stored_instance_types() to pass back the
to_be_dropped instance_type, and use that information for drop_resize_claim.

Change-Id: Iba36acc75d3840104746596977b33bcd4e010f2e
Raise Not Implemented error from V2 diagnostics API

In case any driver does not implement functionality of providing VM
diagnostics, HTTPNotImplemented error will be raised from driver.
Nova V2 API should catch the same and pass it with proper error message
for better debugging.

This patch catch & raise  HTTPNotImplemented exception for
V2 "os-server-diagnostics" API.

Adding UT also for the same.

Change-Id: I485d3487d01eba07e3c34f6ef9b5c707af54ebb7
VMware: Remove references to ebs_root from spawn()

spawn() contained an important boolean called ebs_root. This was True
for any spawn which contained a block device mapping. This no longer
has any meaning, and was confusing. This change removes references to
ebs_root, instead using block_device_info directly.

N.B. The block device mapping behaviour of the VMware driver is still
not correct, but fixing it is not the purpose of this patch. This
patch simply makes the existing behaviour easier to see.

N.B. test_spawn_mask_block_device_info_password in test_vmops.py
assumes internal details of spawn in order to cause a failure after
emitting a log message. In moving the log message we need to update
the dance this test does to ensure that spawn fails after the log
message, but not before.

This change has been split out of
https://review.openstack.org/#/c/87002/, which was written by Shawn
Hartsock.

partial blueprint vmware-spawn-refactor

Co-authored-by: Shawn Hartsock <hartsocks@acm.org>
Change-Id: I11b355a6b53194bc9f85f542b54c9fbb9c061e04
Move _is_mapping to more central location

The logic to identify volumes is currently a nested function
in _default_block_device_names, named _is_mapping. It should
be moved to a more general location so others could utilize it
and allow it to be properly unit tested. The following patch
moves _is_mapping to nova/virt/block_device.py and renames it
to is_block_device_mapping.

Change-Id: I560abc4b57ca5bd195282af7cd1ab9bbf7600b67
Closes-Bug: #1351810
Image caching tests: use list comprehension

Use a list comprehension for creating a list of instance objects
for the tests.

Change-Id: If25d024eeeb1503d7a7dc2dbc895b2bf3152c7db
Change 'admin_password' into v2 style for servers extension

This patch changes 'admin_password' to 'adminPass' that same
with v2 api. And also change the related unittest.

The differences between v2 and v3 are described on the wiki page
https://wiki.openstack.org/wiki/NovaAPIv2tov3.

Partially implements blueprint v2-on-v3-api

Change-Id: Ie5675f98bb15c55a191a207171e2ffe3be8e6deb
Change 'image_ref'/'flavor_ref' into v2 style for servers

This patch changes 'image_ref'/'flavor_ref' to 'imageRef'/'flavorRef'
that same with v2 api. And also change the related unittest.
The differences between v2 and v3 are described on the wiki page

https://wiki.openstack.org/wiki/NovaAPIv2tov3.

Partially implements blueprint v2-on-v3-api

Change-Id: Ic17334e0a9de89488d5e0c8b75950027d51a3cd4
Move fake_quotas and fake_get_quotas into a class

fake_quotas and fake_get_quotas are used in ExtendedQuotasTest only,
and this patch moves them into the class for cleanup.
In addition, this patch moves the mox settings into setUp() method
for the same purpose.

Change-Id: I9920d6207bb27537e6d4e738d69d59f7ed255240
Add missing create() method on SecurityGroupRule object

The nova.objects.SecurityGroupRule class was missing the
create() method, so we add it here, and while doing so,
ensure that the id attribute is marked as read-only.

Partial-Blueprint: compute-manager-objects-juno
Change-Id: I4584e33d9d163a7dce01b01d205f2f43824e544f
Libvirt: Do not raise ENOENT exception

If the disk disappears during update_status(), only need to print
a warning, and do not need to raise an exception.

Change-Id: I87800c855c7b349f69163e33031404ad25e2a9ca
Closes-Bug: 1358609
Add some v2 agents API tests

On most APIs, the unit test coverage of v3 is better than the one of v2.
This patch ports v3 agents API tests to v2 tests for improving v2 agent
API tests.

Change-Id: I045c8fa64eb4cd2bf5852d4656d1067ec12a8564