Fix power manager hangs while executing ipmitool

In nova/virt/baremetal, IPMI power manager uses ipmitool with '-F' option to
specify a file containing the remote server password.  When the passowrd was
empty, ipmitool ignores that file and asks user to set password in stdin.
This makes the thread stop.

This patch changes power manager to write '\0' to the password file in the
case of empty password, to force the password on impitool.

Closes-bug: #1237802

Change-Id: I44f56129a8ce2af4dda8dea35eb9c60d9518d54b
Include name/level in unit test log messages

Use a more informative format than just the raw message string in log
messages captured by fixtures.FakeLogger during unit testing.  For
example, this changes cryptic unit test output such as

    pythonlogging:'': {{{
    214 -> 215...
    done
    215 -> 216...
    done
    }}}

into:

    pythonlogging:'': {{{
    INFO [migrate.versioning.api] 214 -> 215...
    INFO [migrate.versioning.api] done
    INFO [migrate.versioning.api] 215 -> 216...
    INFO [migrate.versioning.api] done
    }}}

which instantly makes it clearer that these numbers come from database
migrations.

Change-Id: I56b6fa03892e6d865ccb036fdf75d2bb499b6c93
Issue brctl/delif only if the bridge exists

Add a check for existence of the bridge before we try cleaning
up as we see a lot of spurious exceptions in the logs especially
when rebuild_instance is in progress and terminate_instance is
issued

Closes-Bug: #1243488

Change-Id: Ib0e2e4d9506e3a8f63201d28a3fc2c910c77d3bd
Add test for class InsertFromSelect

InsertFromSelect is used to generate SQL statement like
"INSERT INTO table1 SELECt * FROM table2".

Change-Id: I0fec93463be1e9930b24862724b7a6d1c8cc52af
Ensure deleted instances' status is always DELETED

Currently when deleting failed instances that were never assigned a
host, their vm_state is left in 'error' instead of 'deleted'. This is a
problem when invoking the Index Instances REST API with a Changes-Since
query parameter since that API returns both deleted and non-deleted
instances, but currently the failed deleted instances are incorrectly
marked as status:ERROR instead of status:DELETED.

This patch modifies the server ViewBuilder to ensure the status is
correctly set.

Change-Id: I614714676bab4986564b236d74f2c0d77400e48c
Closes-bug: #1243301
Rename _get_vm_state to _get_vm_status

The method is actually used to get the server's "status", so rename it
to better reflect function.

Change-Id: Ie4b0a1c76c81f6d08250a1cfd594429ef9eaf5ae
Remove unused local variables in test case

Non-functional change. Just remove the variables that had been assigned
but were never used for clarity of the code base.

Change-Id: I4c204067cff161a975df2277d6ef9d2081863bd1
Check return code of command instead of checking stderr

Command "ip link show" return a no 0 return code when passing
a no existing ip interface so we should use the return code
instead of checking stderr because stderr can contain messages
from the system even though the command succeeded.

Change-Id: Ic43e72dab3c5b6e266240b7ff7309ca3b44f8ed5
Closes-Bug: #1248699
Enable non-ascii characters in flavor names

The current implementation provided under commit e6b42d made it so that
flavor names did not have special characters (e.g. *&^%$ etc. except for
-._ and space) however it also limited names to only ascii letters.

This patch makes it so that special characters are still not allowed,
except the ones currently allowed, but it allows the use of non-ascii
characters so that flavor names can be created in locales other than
English.

Change-Id: I056451a5e8b7a49f86e0a11d0011393dc769db34
Closes-bug: #1241625
Make baremetal_deploy_helper understand ephemeral disks

Teach baremetal_deploy_helper how to run sfdisk with ephemeral disks.
baremetal_deploy_helper calls default mkfs on the epheral disk.

Change-Id: I92d473c34e9a076f5dbcd54be538410b30b10204
Partial-Bug: #1238311
Co-Authored-By: Robert Collins <rbtcollins@hp.com>
libvirt: Allow delete to complete when a volume disconnect fails

If an instance which was booted from a volume fails to spawn, the
instance is left in an ERROR state.  If the failure is in the networking
stage Nova will not have attached the volume via the Cinder driver,
and so the volume remains in an available state.  In this state the
volume may be deleted or assigned to another instance.
During the subsequent delete an exception may be thrown from
the cinder driver which prevents the instance from completing the
deletion process leaving the instance stuck in Error and unable to
be deleted.

Volume exceptions should be logged and ignored when the instance
is being deleted.

Closes-Bug #1222979

Change-Id: Icb3796b0ddba25cf344953a649b2e762fab6d782
Fixes RequestContext initialization failure.

RequestContext initialization failed in nova because of the following
error:
"TypeError: 'in <string>' requires string as left operand, not NoneType"

It must traverses in tuple not in string when find the "volume" service_catalog.

Change-Id: Ie0de50fcafbeabf4698696add610d4e6804f45eb
Closes-Bug: #1248424
handle empty network info in instance cache

The nw_info value [] was being incorrectly equated with None. This caused
infinite recursion when api._get_instance_nw_info returned [] and was wrapped
with @refresh_cache (e.g., _get_instance_nw_info in
nova.network.neutronv2.api.API).

Fixes bug #1251792.

Change-Id: I3034fbb20e790f2d6f22c50b74a9f9dcdf9081aa
Periodic task _poll_unconfirmed_resizes can now use slave db

This allows the _poll_unconfirmed_resizes periodic task to use the
slave db for reads if the deployment has a slave configured. For
slaveless deployments behavior remains unchanged.

This commit advances the MigrationList object to version 1.1

Change-Id: I10bf729e92dc599819ef74a97a834c7f2df5b162
Partially-implements: blueprint periodic-tasks-to-db-slave
Address infinite loop in nova compute when getting network info

Move the refresh_cache decorator to get_instance_nw_info.
In fact, _get_instance_nw_info is called by the decorator
itself, potentially causing an infinite loop in case nw_info
is not of the expected value. This is also inline with
the method's purpose, as stated by the docstring. At the
same time, ensure that delete_port_for_instance and
allocate_for_instance continue to refresh the cache by
using the decorated version.

Also, add a couple of debug traces that are friendly to
the developer.

Partial-bug: 1235435
Related-bug: 1251792

Change-Id: I06f8634ea241d05ac8fbcc290adf0cb23829f3e4
xenapi: move session into new client module

This is the start of building a set of objects on top of the XenAPI
library to make a clear separation between XenAPI client and
business logic.

This is all the first steps towards pull out the client lib into oslo
incubator, so we can share code between Nova and Cinder.

DocImpact
While there is a change to the sample config file, there is no change in
the content of the config file.

Part of blueprint xenapi-driver-refactor
Change-Id: I463923a9ee289ff4b86fcf26d8c656609685857e
Fix performance of Server List with Neutron for Admins

https://review.openstack.org/#/c/30048/ optimized the
SecurityGroupsOutputController for the case where the server
list only contains one server, but in all other cases the current
code calls the Neutron driver in a way that makes it retrieve
all ports and security groups visible to the tenant.

For users with a Neutron admin role this retrieves all ports
and all SecGroups in the system, which on a large system is a
major performance issue and often leads to client timeouts.
Normally these users have further qualified their query to a
specific tenant or host, or are maybe just trying to get their
own list of servers.

If instead we pass the pre-filtered list of servers to Neutron
it can query for ports associated with just the device IDs of
the servers.  We then use that list of ports to build a
specific list of security groups IDs to query for.

This approach means that we can remove the initial
optimization for a single server case.

Closes-Bug: 1228384

Change-Id: I6f72533056e4f336f7578cc883fd1a125c2048a9
Sync Log Levels from OSLO

Copy changes from just 1 commit to reduce size of n-cpu
logs reported by infra folks

cbfded9c Default iso8601 logging to WARN

Change-Id: I71a51374c58001380c52d9c577d257f98b880d87
Extend the scheduler HostState for metrics from compute_node

Changes the HostState to be more generic and extensible to store other
metrics from compute node for scheduling purpose.

This is part of the blueprint utilization-aware-scheduling.

Change-Id: Ia472e9dcdaf12371ffad94e074cecd51886a1c04
Merge "libvirt: Allow delete to complete when a volume disconnect fails"