Bad except clauses order causes wrong text in http response

InstanceInvalidState is an ancestor class of InstanceIsLocked, so
we need to catch InstanceIsLocked first. Fix the test case to
ensure we get the right http response back

Change-Id: Id22b284d9b7da8db4161ad8703bea1702cb0b82e
Fix power manager hangs while executing ipmitool

In nova/virt/baremetal, IPMI power manager uses ipmitool with '-F' option to
specify a file containing the remote server password.  When the passowrd was
empty, ipmitool ignores that file and asks user to set password in stdin.
This makes the thread stop.

This patch changes power manager to write '\0' to the password file in the
case of empty password, to force the password on impitool.

Closes-bug: #1237802

Change-Id: I44f56129a8ce2af4dda8dea35eb9c60d9518d54b
Include name/level in unit test log messages

Use a more informative format than just the raw message string in log
messages captured by fixtures.FakeLogger during unit testing.  For
example, this changes cryptic unit test output such as

    pythonlogging:'': {{{
    214 -> 215...
    done
    215 -> 216...
    done
    }}}

into:

    pythonlogging:'': {{{
    INFO [migrate.versioning.api] 214 -> 215...
    INFO [migrate.versioning.api] done
    INFO [migrate.versioning.api] 215 -> 216...
    INFO [migrate.versioning.api] done
    }}}

which instantly makes it clearer that these numbers come from database
migrations.

Change-Id: I56b6fa03892e6d865ccb036fdf75d2bb499b6c93
Ensure deleted instances' status is always DELETED

Currently when deleting failed instances that were never assigned a
host, their vm_state is left in 'error' instead of 'deleted'. This is a
problem when invoking the Index Instances REST API with a Changes-Since
query parameter since that API returns both deleted and non-deleted
instances, but currently the failed deleted instances are incorrectly
marked as status:ERROR instead of status:DELETED.

This patch modifies the server ViewBuilder to ensure the status is
correctly set.

Change-Id: I614714676bab4986564b236d74f2c0d77400e48c
Closes-bug: #1243301
Rename _get_vm_state to _get_vm_status

The method is actually used to get the server's "status", so rename it
to better reflect function.

Change-Id: Ie4b0a1c76c81f6d08250a1cfd594429ef9eaf5ae
Remove unused local variables in test case

Non-functional change. Just remove the variables that had been assigned
but were never used for clarity of the code base.

Change-Id: I4c204067cff161a975df2277d6ef9d2081863bd1
Enable non-ascii characters in flavor names

The current implementation provided under commit e6b42d made it so that
flavor names did not have special characters (e.g. *&^%$ etc. except for
-._ and space) however it also limited names to only ascii letters.

This patch makes it so that special characters are still not allowed,
except the ones currently allowed, but it allows the use of non-ascii
characters so that flavor names can be created in locales other than
English.

Change-Id: I056451a5e8b7a49f86e0a11d0011393dc769db34
Closes-bug: #1241625
Make baremetal_deploy_helper understand ephemeral disks

Teach baremetal_deploy_helper how to run sfdisk with ephemeral disks.
baremetal_deploy_helper calls default mkfs on the epheral disk.

Change-Id: I92d473c34e9a076f5dbcd54be538410b30b10204
Partial-Bug: #1238311
Co-Authored-By: Robert Collins <rbtcollins@hp.com>
Make baremetal support metadata for ephemeral block-device-mapping

Make baremetal driver set instance['default_ephemeral_device'] to
/dev/sda1 since we are using the same device for root(sda3) and
ephemeral(sda1)

Change-Id: Ib2076198eacb0ea61706043b42d673c35d26f5d1
Partial-Bug: #1239524
Users with admin role in Nova should not re-auth with Neutron

A recent change to the way Nova creates a Neutron clients
https://review.openstack.org/#/c/52954/4 changed the conditions
under which it re-authenticates using the neutron admin credentials
from "if admin" to "if admin or context.is_admin".

This means that any user with admin role in Nova now interacts
with Neutron as a different tenant.  Not only does this cause an
unnecessary re-authentication (The user may/should also have an admin
role in Neutron) it means that they can no longer allocate and assign
a floating IP to their instance via Nova (as the floating ip will now
always be allocated in the context of neutron_admin_tenant).

This change removes the context.is_admin check to revert to the
previous behaviour where the used of admin creds is determined just
on the explicit use of "admin=True" parameter to get_client().

Change-Id: Ib1720420c778960bc90c5b7d703de936ebb7d6b5
Closes-Bug: 1250763
Added monitor (e.g. CPU) to monitor and collect data

This patch gives a monitor interface for users to add their own monitors to
monitor on compute nodes on the compute manager.

The compute manager loads the monitors and gets the metrics, and sends to
the scheduler for utilization based scheduling.
The patch also gives a sample to collect CPU utilization data by libvirt
driver. The patch brings a new API get_host_cpu_stats() in class
ComputeDriver, and libvirt has implemented that in the patch.
For other virt drivers, it has not been implemented yet (TODO).

This is part of the blueprint utilization-aware-scheduling and the
blueprint generic-host-state-for-scheduler.

Change-Id: I4cd4f972c5c1cf6242a77aaeb9514c8a0d4faa0d
handle empty network info in instance cache

The nw_info value [] was being incorrectly equated with None. This caused
infinite recursion when api._get_instance_nw_info returned [] and was wrapped
with @refresh_cache (e.g., _get_instance_nw_info in
nova.network.neutronv2.api.API).

Fixes bug #1251792.

Change-Id: I3034fbb20e790f2d6f22c50b74a9f9dcdf9081aa
Fix libvirt evacuate instance on shared storage fails

When evacuate instance which has config drive enabled on shared storage,
disk.config file can't be created because of permision error, chowns
disk.config if it already exists.

Change-Id: I20bafc3482ead30297ad5402a09e97141816f4d1
Closes-Bug: 1247526
Periodic task _poll_unconfirmed_resizes can now use slave db

This allows the _poll_unconfirmed_resizes periodic task to use the
slave db for reads if the deployment has a slave configured. For
slaveless deployments behavior remains unchanged.

This commit advances the MigrationList object to version 1.1

Change-Id: I10bf729e92dc599819ef74a97a834c7f2df5b162
Partially-implements: blueprint periodic-tasks-to-db-slave
Fix up spelling mistake

Fixes bug 1253510

This was causing a KeyError when e2label didn't work

Change-Id: I0646409cdffa4d6d5d8b2c7a9e3c300022d1c9b8
Use model_query() instead of session.query in db.instance_destroy

Unify db.instance_destroy() method to use model_query() as others
methods use.

Side effect we won't try to soft delete already soft deleted instance

Change-Id: I36d5bb42b16901c260ff4276e31a6aec46ffa8a4
Sync the DB2 communication error code change from olso

Closes-Bug: #1244092

oslo patch set: 3acd57c2825a8bf6660c3124f12794414622eb26

Change-Id: I62e3a6d1137d53c4aa72b18db5bd9f7e41694436
Fix missing format specifier in ImagePropertiesFilter log message

Commit a52259e introduced some changes to debug log messages in the
ImagePropertiesFilter but with that came a bug due to a missing format
specifier.

Closes-Bug: #1253875

Change-Id: Ic60b7ba179bd43b80c500be084a1809b6b0b32ca
Add v3 api samples for shelve

Adds api sample tests and api samples for shelve

Partially implements blueprint v3-api-unittests
Partially implements blueprint v3-api-specification

Change-Id: Ib485ab042a340c4672ad01fb35d065fd52889b07
Address infinite loop in nova compute when getting network info

Move the refresh_cache decorator to get_instance_nw_info.
In fact, _get_instance_nw_info is called by the decorator
itself, potentially causing an infinite loop in case nw_info
is not of the expected value. This is also inline with
the method's purpose, as stated by the docstring. At the
same time, ensure that delete_port_for_instance and
allocate_for_instance continue to refresh the cache by
using the decorated version.

Also, add a couple of debug traces that are friendly to
the developer.

Partial-bug: 1235435
Related-bug: 1251792

Change-Id: I06f8634ea241d05ac8fbcc290adf0cb23829f3e4