Fixes Hyper-V volume discovery exception message

On some editions of Windows / Hyper-V server the SAN policy is set
by default to "Online All", bringing online any disk, local or shared,
attached to the host.

Since only offline disks can be attached as passthrough disks to a Hyper-V
VM, this prevents Cinder volumes from being attached to instances,
resulting in an exception that can be particularly hard to troubleshoot
without a clear knowledge of the context.

This patch includes some details in the exception message for
troubleshooting purposes.

Change-Id: I6d0b49812f8ca965ca53be34ef009997c8e79728
Closes-Bug: #1361419
Remove kombu as a dependency for Nova

It was required before oslo.messaging.

Change-Id: I3159b8509ac5b4020cd584c894eb131bb16df97c
Faster get_attrname in nova/objects/base.py

get_attrname function is called very very often,
and it's string concatenation method is not fast.

* Changing the string concatenation to a 2 times
  faster + operator instead of the % operator.

Change-Id: I326fd6dda8dd41a90757c69867afd62063231362
Refactor Ironic driver tests as per review comment

During the Ironic driver merge into Nova, there were some
suggestions on refactoring the tests as per this review
(https://review.openstack.org/#/c/111425/19/nova/...
...tests/virt/ironic/test_driver.py)

This patch implements these suggestions.

Change-Id: I05e8b3d01bffebe2f68bbd50fc775e1ef48616cc
Only load necessary instance info for use in sync power state

During sync power state, instance is refreshed from the db. If the
instance gets deleted, this can result in InstanceInfoCacheNotFound
being thrown due to info_cache being loaded and therefore refreshed.

Sync power state does not make use of any extra instance attrs, so
this problem can be avoided by not loading any extra attrs in the
first place.

Two callers of _sync_instance_power_state() are updated to not load
extra attrs. 1) The periodic sync and 2) the lifecycle event handler.

Change-Id: I150bd95da65263981608dd9a033232bd142e737f
Closes-Bug: #1367845
Rename private functions in db.sqla.api

instance_get_all_by_filters() called 3 helper functions: exact_filter,
regex_filter and tag_filter. None of these functions had any other
callers.

This change prefixes them with _ to identify them as private.
Additionally, it removes the model arguments to the functions as these
always relate to an Instance and/or its InstanceMetadata.
Consequently, the functions are renamed to identify them as specific
to an Instance query. exact_filter is moved from the top of the module
to be next to the other 2.

Change-Id: I671315122438022b13d6b315b577ef2a5cf42545
Don't modify columns_to_join formal parameter in _manual_join_columns

The _manual_join_columns method was modifying the columns_to_join list
passed to it which, if not copied by the original caller, can introduce
bugs in the calling code if the original columns_to_join list is re-used
later, e.g. making nova objects with an expected_attrs list that has
been mutated by the DB API call.

This change makes a copy of the original list and passes back the new
list for the DB API methods to use, and adds some documentation to
hopefully clarify intent and usage.

A test is added to demonstrate the expected behavior of the utility
method.

Closes-Bug: #1393491

Change-Id: I49511f608291b66642ced00a3c13833e18fab3d7
Remove unused network_api.get_instance_uuids_by_ip_filter()

Last usage was removed in ae781ee97947c33d6d43e4c21df4f338c875bf1c

Change-Id: I33dc14085ca0c887778c7379f0498714e8d30425
Remove old Baremetal Host Manager

Baremetal was deprecated in Juno and its driver was removed in
46ed619b9ab1d61582f36155eea0d4a88f31fd50

This cleans out the parts that's still left in the scheduler.

Change-Id: I86f583f7d0747a164dae4530e25f8b01c2144673
Improve error return code of floating_ips in v2/v2.1 api

1.change exception.InvalidID from 404 to 400 when passing a invalid
id.

2.handle exceptions when calling get_instance_id_by_floating_address()
  FloatingIpNotFoundForAddress -> HTTPNotFound
  FloatingIpMultipleFoundForAddress ->HTTPConflict

3.change HTTPUnprocessableEntity to HTTPConflict when remove a floating
ip which is not associated the instance

Change-Id: I0c9cd5b50a2d3cf024e5c4c3506ae2d0e6394c88
Add delete not found unit testcase for floating_ip api

This patch adds unit test case for the delete api of
floating_ip extension.
V2 and V21 have difference return code when delete
a invalid when given a invalid id of floating ip.

This patch addes both nova-network api and neutronv2 api
test cases.

Partially implements blueprint v2-on-v3-api
Change-Id: I40a5b133c98e01e8f5c613b765b8d821beec03a6
Set vm state error when raising unexpected exception in live migrate

The instance stuck at migrating task_state when unexpected exception.
This is confuse for user, this patch set the vm_state to error.

Change-Id: Ib1b97452bc5e777c66c4d368f71156dbe1e116b7
Partial-Bug: 1397153
Add shelve and unshelve info into devref doc

The diagram on http://docs.openstack.org/developer/nova/devref/vmstates.html
does not currently show the "SHELVED_OFFLOADED" state and the
associated "shelve", "shelveOffload" and "unshelve" state transitions.

Change-Id: I50eb07a84336160b8120ca294598576eeb982d28
Closes-Bug: #1394215
Error code for creating secgroup default rule

Trying to create secgroup group default rules
that already exists gives HttpBadRequest error,
but it should raise HttpConflict.

So updated the code to HttpConflict error.
Closes-Bug: #1374320

Change-Id: I93c6eb7fb96e7109404206367ed81495a4724296
Remove the volume api related useless policy rules

There are some old policy rules used by old nova volume, those are
useless anymore, this patch clean them up.

Change-Id: Ibcdba00652808158019600f8ad75e80c85f9423e
Closes-Bug: #1398686
VMware: allow selection of vSAN datastores

With support for handling of stream-optimized disk images added, the
vmware driver is now able to deploy to vSAN datastores, so add support
for the the 'vsan' datastore type, and update the datastore selection
logic to take this type into account when when spawning a
stream-optimized image.

Implements: blueprint vmware-vsan-support
Co-authored-by: Radoslav Gerganov <rgerganov@vmware.com>

Change-Id: Id43ad4adcad212a8e0d8ab3830af62d4afda1442
Libvirt: SMB volume driver

Currently, there are Libvirt volume drivers that support
network-attached file systems such as Gluster or NFS. This patch
adds a new volume driver in order to support attaching volumes
hosted on SMB shares.

Co-Authored-By: Gabriel Samfira <gsamfira@cloudbasesolutions.com>

DocImpact

Change-Id: I1db3d2a6d8ee94932348c63cc03698fdefff0b5c
Implements: blueprint libvirt-smbfs-volume-support
Compute: Catch binding failed exception while init host

While compute starts it will init all instances,
if an exception is raised from one instance
(e.g NovaException during plug_vifs), then the
compute process exits unexpectedly because of
this unhandled exception.
This commit changes the NovaException to more
appropriate VirtualInterfacePlugException and
catches it during init host, as well as the
instance is set to error state, with this change
the compute process can be started normally even
if this VirtualInterfacePlugException is raised.

Closes-bug: #1324041

Change-Id: Ia584dba66affb86787e3069df19bd17b89cb5c49
Clear libvirt test on LibvirtDriverTestCase

We already create a class variable for connection,
no need to create each one for every test cases.

Change-Id: I635f5993888b21dc9b497e588057af5f794f55b3
Compute Add build_instance hook in compute manager

The build_instance hook provides a hook close to the actual spawn
of an instance. It distinguishes itself from the create_instance
hook because, in the case of a successful build, the instance will
exist on the hypervisor when the build_instance post hook is
called, instead of simply just being scheduled as is the case with
the create_instance post hook. This is useful as some details may
not be available until after scheduling.

Co-Authored-By: Corey Wright <corey.wright@rackspace.com>
Change-Id: Ic12abf6315bbc6b010b8222ef296ec65cb4b4f95