Fix migration 147 downgrading with no aggregates

Add some extra bulletproofing to downgrading migration 147 where
there may have been no aggregates.

Change-Id: I6e4aabf4fff462e9ad53083621077ce0e0a80a43
fix missing host when unshelving

An instance has the hostname of the host on which it is running.
The hostname is deleted when shelving. However, the hostname is not
recreated when unshelving.

This patch sets the hostname when unshelving.

Change-Id: I2e431718198321c46b9335e6fb7ab7be3943fab6
Closes-Bug: #1237868
(cherry picked from commit 9532e4ed969e47826801b14aff40dd9e356f2677)
Make non-admin users can unshelve a server

If non-admin users try to unshelve a server, nova-compute fails due to
"User does not have admin privileges" while getting hypervisor_hostname.
This patch changes the user context to admin's one temporarily to get the
hypervisor_hostname.

Closes-Bug: 1245312
Change-Id: I84dce1f7694d09c1281da13c6008bb3aa5940f85
(cherry picked from commit 13fa2caaad796ee66c432e0d19fd6f759234a964)
Fix docstring for disk_cachemodes

The docstring for the disk_cachemodes option implied a syntax that is
incorrect for a ListOpt.  Change the example to match exactly what you
would put in the config file for it to work.

Change-Id: Ie999d2799dba6c47facdc771e1cf6081d28d6b85
Closes-Bug: #1106423
(cherry picked from commit 1df9b26b51ff3a153b93d79e25bef454e08a8e38)
Fix difference between mysql & psql of flavor-show

If you create a new flavor using a deleted flavorid,
the result of flavor-show is different between mysql and psql.

This issue is caused by the different processing for index in each db.

We need to specify the sort-type to avoid the issue.

Change-Id: Ib1029e80c1b981e1ec86d954b63f83650c9b1cc1
Closes-Bug: #1288636
(cherry picked from commit 4bbb4d88595cfd5f75528d8789be94d0ec9fbe62)
Update aggregate should not allow duplicated names

It is not allow to create 2 aggregates with the same name
but if update an existing one with a name already used
by other aggregate that check should be made.
This patch doesn't allow to change the name to an existing
one.

Change-Id: Icfe315594beaec94229e5a4a4c1bb80fc366b66e
Closes-Bug: #1288296
(cherry picked from commit 2b2decfd1b5b0d68813bc7e5b5e4349a1ab592aa)
Add context as parameter for two libvirt APIs

When finish_revert_migration is called, the caller of
finish_revert_migration already includes context as a parameter,
but finish_revert_migration did not reuse this parameter and still
re-generate the context inside finish_revert_migration, we should
add context as a new parameter for finish_revert_migration so that
the functions inside of it can reuse context when needed.

For the function _create_domain_and_network in libvirt/driver.py,
it set context as an optional parameter, but context isn't really
an optional parameter for this method. We should always pass a
context down because it might be needed somewhere inside
_create_domain_and_network().

The patch includes two parts:
1) Add context as a new parameter for finish_revert_migration
2) Set context as a required parameter for _create_domain_and_network

Closes-Bug: #1251261
(cherry picked from commit 81fc3967bca0f154d72d5de7875b646e1b6c47c8)

Conflicts:
	nova/tests/compute/test_compute_mgr.py
	nova/virt/vmwareapi/vmops.py

Change-Id: I72d131fc8dce903400e3080d51ba0fc0a1320751
Make cells 'flavorid' for resizes

Cells should use 'flavorid' (not 'id') for resizes.

bug 1241222

Change-Id: I57b997425f183f25d3ca032c7d8c35c1f49e3afc
(cherry picked from commit 034df6e875d6c0b590f3cfb802574c2f982fd534)
libvirt: Custom disk_bus setting is being lost on hard_reboot

Currently, a non-default disk_bus setting of and image will not be used,
when powering on an instance (or _hard_reboot).

hard_reboot() should use image_meta, if it exist, when constructing
the disk_info

Closes-Bug: #1255449

(cherry picked from commit 0fe3001d639df81ae4e5f77bcfc32bc1bffb014e)

Conflicts:
	nova/virt/libvirt/driver.py

Change-Id: I477f69c47515ad22165421b4d23ac865c7869873
Add declaration of 'refresh_instance_security_rules' to virt driver

The method 'refresh_instance_security_rules' was not defined
in the virt driver ComputeDriver class. In cases where the
underlying virt driver does not support security groups
an exception would be encountered indication that the driver
class does not have the specific attribute.

In addition to adding the method the patch set also catches the
exception NotImplementedError and logs a warning.

Closes-bug: #1269448
(cherry picked from commit 5a98d7b61fa6adb1557e69f2f234406be0e2739c)

Conflicts:
	nova/tests/virt/hyperv/test_hypervapi.py

Change-Id: Ia36b0dfb54a3ede86467d87f0650a0902adb6d46
Fix security group list when not defined for an instance

If an instance has an empty security group list an exception
is thrown due to iteration attempt of None.

To fix this, this patch modifies get_instance_security_group
to return an empty list instead of None.

Conflicts:
	nova/tests/api/openstack/compute/contrib/test_neutron_security_groups.py
	nova/tests/network/security_group/test_neutron_driver.py

Change-Id: I7efb3c3a58d7e20501d126c6d194479ca1d14166
Closes-Bug: #1291489
(cherry picked from commit c32986966dfd034c3d706b2e9ab2820a2c3cfc3e)
Always filter out multicast from reflection

Instances will try IPv6 neighbour discovery via multicast
even in an IPv4-only cloud and can throw errors
if they see inbound packets from their own MAC address

Closes-bug: #1229625

Change-Id: I3539e788fe07519d87ce7c3800c5d38b7bd99d3b
(cherry picked from commit 9c044d2c94812e18cf84927fbf719cd073fe6c4f)
Fixes hyper-v volume attach when host is AD member

The domain name gets added to the initiator name used by the host
if it's an Active Directory member.

Currently the method which gets the initiator name does not take
this into account when retrieving the default initiator name.

Trying to use a wrong initiator name will lead to an exception when
trying to log in to the according iSCSI target.

This patch simply appends the domain name (when the host is an AD
member) to the initiator name used to log in to iSCSI targets.

Closes-Bug: #1328870
(cherry-picked from commit 0d9419f2b36561bf50bbe0f720c8dd8ad4e96660)

Change-Id: Ifbe762c685e46081059a01043431b2c4ac5473fc
make libvirt driver get_connection thread-safe

libvirt driver's get_connection is not thread safe in the
presence of a libvirtd restart during concurrent incoming
requests.

With existing code each will in turn call get_connection,
find the connection is broken, try to create new one, block
for a while and yield to the next thread to do the same.
You get as many connections as there are incoming requests
and only the last one is used finally. If enough are incoming
these connections can exhaust the client pool configured
for libvirtd.
One fix is to hold a lock while creating the connection.
Note that has_min_version calls _conn which calls get_connection
and thus the direct call to _has_min_version()

Also added the exception text if it fails to register an event
handler for lifecycle events.

Change-Id: I090765802bfe443440f16722bc7c43b6280fe56a
Fixes: bug #1240905
(cherry picked from commit b2e64e379835f57128e66f507438130eda716814)
domainEventRegisterAny called too often

Changeid I090765802bfe443440f16722bc7c43b6280fe56a fixes
bug #1240905 but introduced a bug whereby domainEventRegisterAny
is called on every call to _get_connection.

Conflicts:
        nova/virt/libvirt/driver.py

NOTE(mriedem): The conflict is due to the set_host_enabled
code not being in stable/havana and we don't want to backport
that for the threading fixes in this series.

Change-Id: Ifafc6a4c172b1521d62988ccbb3a2f51a95e323f
Fixes: bug #1250135
(cherry picked from commit 755f6194e29c8e3daf6fd64a59dc2be4c81b1f0d)
Use of logging in native thread causes deadlock connecting to libvirtd

When connecting to libvirtd daemon we were calling the _connect
method in a native thread. The use of logging in this method hit this
bug in eventlet https://bitbucket.org/eventlet/eventlet/issue/137/
causing nova-compute to hang.

Closes-Bug: #1252409

Conflicts:
        nova/virt/libvirt/driver.py

NOTE(mriedem): The conflict is due to set_host_enabled
being used in the original change in Icehouse and the
option rename for libvirt_nonblocking.

Change-Id: I0d22c3ed1b3288aeb1afae130f11a4f38dc6440a
(cherry picked from commit e4b0d8e9446ad3ad3a514b9cf65722f6ed95888d)
Fix bug with not implemented virConnect.registerCloseCallback

The unit test test_connection_to_primitive fails many times due to
"virConnect instance has no attribute 'registerCloseCallback'".

registerCloseCallback has been added since original(community's)
python-libvirt v1.0.0, and v1.2.0 also contains the method.
However ubuntu cloud's v1.0.1 does not contain the method, and current
version check of python-libvirt does not work.
This patch tries to operate the method and catch TypeError exception if
the method does not exist instead of the version check.

Conflicts:
        nova/tests/virt/libvirt/test_libvirt.py

NOTE(mriedem): The conflict is due to set_host_enabled
calls which aren't in stable/havana.

Co-Authored-By: Sahid Orentino Ferdjaoui <sahid.ferdjaoui@cloudwatt.com>
Closes-Bug: #1266711
Change-Id: I4ec9ff9a684639ae5b146f400c90115c83afcda7
(cherry picked from commit 85068cc9f68aa704895ffe5ac185bb9cf7d05e2d)
Merge "fix missing host when unshelving" into stable/havana
Merge "make libvirt driver get_connection thread-safe" into stable/havana
Merge "domainEventRegisterAny called too often" into stable/havana