Make hypervisor_version an int in fakeVirt driver

This is a follow up to 9e770e62135fe9c2c8ac0121a5a79245b25a7847, which
missed the change in a duplicate copy of host_status.

Add regression test to test_virt_drivers.

This requires changes to the API samples, because they were wrong. virt
drivers use convert_version_to_int which converts a version string to a
4 digit number ("1.0" becomes to 1000)

Change-Id: I28ce23509e3c9feae183a49a8fc5bf3c7c601295
Closes-Bug: #1285035
Fix security group race condition while creating rule

Previously, it was possible for someone to create a security group rule
where the rule references another security group. To do this nova would
first create the security group rule and then look up the referenced group
(group_id's) name in order to return that via the api. During this time
it's possible for someone to delete this security group rule and the
referenced group before the call returned resulting in a 404 error being
raised. This patch addresses this issue by looking up the group name
first and then creating the security group rule in order to avoid this
from occuring.

Note: this patch also adds a test to cover the case where an invalid group
id was passed though this does not really add any real additional coverage
to the change largely because the code is currently using global stubs. That
said, the previous coverage should hopefully be sufficient

Change-Id: If58ffa5629ba5166f260379ac47922974de31be0
Related-bug: 1262566
VM in rescue state must have a restricted set of actions

Right now it is possible to pause, suspend and stop a VM in state RESCUED,
so after the state is changed, it's not possible to trigger unrescue anymore
since the original state is lost.

This patch remove vm_states.RESCUED as valid state from stop,
pause and suspend actions.

The vm_states devref is also updated to reflect this change including the
current reboot flow.( vm_states.RESCUED cannot be rebooted as per
today code)

DocImpact
Closes-Bug: #1319182
Co-Authored-By: Cyril Roelandt <cyril.roelandt@enovance.com>
Change-Id: I531dea5a5499bf93c24bea37850d562134dee281
xenapi: virt rescue method now supports objects

Commit a37a96fd13b933123e80ad24e37fc37d73c16059 ensured
that the compute API and manager are now getting objects
for the rescue method.

This patch adds support to the virt driver code.

Change-Id: I06e5032526f791dfc0c523f2302d2edff5023f78
xenapi: virt unrescue method now supports objects

Commit 65ad230af5ece020822fa963dd30fd641a54ae91 ensured
that the compute API and manager are now getting objects
for the unrescue method.

This patch adds support to the virt driver code.

Change-Id: Ie6b587d3f89202a6987291bfaea88290241651f2
Use VIF details dictionary to get physical_network

Modify mlnx_direct plug and unplug to retrieve physical_network
from VIF details dictionary if network meta dictionary does not contain
physical_network. This will serve ML2 case following the guidelines to
populate vif_details dictionary with attributes required for port plugging.

Change-Id: I0d97bc875be2fff18087c78accef3ec81c059c4b
Closes-Bug: 1304872
xenapi: Do not retry snapshot upload on 500

If Glance returns a 500 response on an initial attempt to upload
a snapshot image, it will set the image status to killed.  Any retry
attempts will fail with a 409 response.

Hence, Nova should not attempt to retry the upload in this case, which
will eliminate many pointless retries.

Closes-Bug: 1349933

Change-Id: I1a8f2416923a368b02cf8963c747ebbb24d749a1
Fixes hyper-v volume attach when host is AD member

The domain name gets added to the initiator name used by the host
if it's an Active Directory member.

Currently the method which gets the initiator name does not take
this into account when retrieving the default initiator name.

Trying to use a wrong initiator name will lead to an exception when
trying to log in to the according iSCSI target.

This patch simply appends the domain name (when the host is an AD
member) to the initiator name used to log in to iSCSI targets.

Closes-Bug: #1328870

Change-Id: Ifbe762c685e46081059a01043431b2c4ac5473fc
Replace assertTrue(not *) with assertFalse(*)

Replacements were done in tests to have
clearer messages in case of failure

Change-Id: I7eedc62e0d570ee5818c4aa9b28ad8163c63b775
Add supported file system type check at virt layer

Currently nova doesn't check whether the file system type is supported
by virt layer(hypervisor) before use it. This patch adds current
support file system type definition then create a default check
function at virt layer. Following patches will do the real check
in different driver such as libvirt.

Change-Id: Ie4d876a48b36c1a53b171dd521bdeef868a31486
Partial-Bug: #1293880
Reduce unit test times for glance

The class TestGlanceApiServers did not inherit
test.NoDBTestCase.

Change-Id: Ieafe300d793685f5b146999383695733eba3da19
Replace assertTrue/False with assertIn/NotIn

The following replacements were done in tests to have
clearer messages in case of failure:
- assertTrue(* in *) with assertIn
- assertTrue(* not in *) with assertNotIn
- assertFalse(* in *) with assertNotIn

assertTrue/False with custom error messages weren't
replaced because assertIn/NotIn doesn't have argument
'message' and we'll lose error messages if we replace it.

Also assertTrue/False with operator 'any' were
replaced with more simpler asserts assertIn/NotIn.

Change-Id: If5e78db21d5302259a4d69fd82c62baf39a5d8f8
Add API schema for v2.1/v3 hosts API

By defining the API schema, it is possible to separate the validation
code from the API method. The API method can be more simple.
In addition, a response of API validation error can be consistent for
whole of Nova APIs.

Partially implements blueprint v3-api-schema

Change-Id: Ie252dce67f8e9d3292d8dad767b6113f7affcffd
Add API schema for v2.1/v3 flavors_extraspecs API

By defining the API schema, it is possible to separate the validation
code from the API method. The API method can be more simple.
In addition, a response of API validation error can be consistent for
the whole Nova API.

Partially implements blueprint v3-api-schema

Change-Id: I71f89b59dd4e542d2befa971182cc587b6fec269
Baremetal driver: remove unused states

NULL and INIT were not used since commit c20110d15be37948ddd9ef5f38001328aabf5b1d

Change-Id: I25c34872b100807d691e7e15f7b70be7ad1912b9
VMware: remove unused parameter 'network_info'

The parameter 'network_info' is not used for destroying an instance and
doesn't need to be passed through the whole stack.

Change-Id: I0a8b912ff7693478917e849e17093d1541adeeeb
VMware: cleanup the constructors of the compute drivers

The 'read_only' parameter was never used by the compute drivers.
The 'scheme' parameter should be passed to the base constructor.

Change-Id: I3abd40a50896664ee274f057ded8ec2460679c82
Move rebuild to conductor and add find host logic

[*] Move rebuild instance to conductor task api will help
    in consolidating the logic for both evacuate instance and
    rebuild instance. Rebuild instance and evacuate instance
    uses the conductor task api for rebuild instance workflow.
[*] Add logic required to support rebuild without host.
    (used by evacuate)
[*] Refactor rebuild action to pass the instance host as argument,
    instead of None.

Evacuate and Rebuild will behave the same way as today.
Modifications for the new feature in API will be implemented in
new patches.

Partial Implements: blueprint find-host-and-evacuate-instance

Change-Id: I4b1a9c2227574db7179441d531a49145f213ea87
Co-Authored-By: Juan M. Olle <juan.m.olle@intel.com>
Co-Authored-By: Anuj Mathur <anujm@thoughtworks.com>
Co-Authored-By: Navneet Kumar <navneetk@thoughtworks.com>
Co-Authored-By: Claxton Correya <claxton@gmail.com>
Add APIv3 support to make host optional on evacuate.

DocImpact: The evacuate target host is now optional. If
'host' field is not sent in the request, the scheduler
will determine the target host. This will include
nova client changes ( on the proper commit ) to support
this new optional parameter.

Implements: blueprint find-host-and-evacuate-instance

Change-Id: I907ea4ef39cfad50a48cb80bceba217fdf482f6e
Co-Authored-By: Juan M. Olle <juan.m.olle@intel.com>
Co-Authored-By: Andres Buraschi <andres.buraschi@intel.com>
Co-Authored-By: Anuj Mathur <anujm@thoughtworks.com>
Co-Authored-By: Navneet Kumar <navneetk@thoughtworks.com>
Co-Authored-By: Claxton Correya <claxton@gmail.com>
Libvirt: Added suffix to configdrive_path required for rescue

It was observed that during nova rescue, nova failed to create
a rescue disk for disk.config. The reason being that .rescue
suffix was missing from the configdrive path. It was working
till Havana but the functionality broke in Icehouse. This commit
fixes the suffix problem.

Unittests have been added to verify following scenarios:
    1. Make sure that .rescue disks are created when not using
        config drive
    2. Make sure that .rescue disks are created when using
        config drive

Closes-Bug: #1334024

Change-Id: I87449ffddd047cb84b7b881757ea4c29927b95da