Bad except clauses order causes wrong text in http response

InstanceInvalidState is an ancestor class of InstanceIsLocked, so
we need to catch InstanceIsLocked first. Fix the test case to
ensure we get the right http response back

Change-Id: Id22b284d9b7da8db4161ad8703bea1702cb0b82e
Fix power manager hangs while executing ipmitool

In nova/virt/baremetal, IPMI power manager uses ipmitool with '-F' option to
specify a file containing the remote server password.  When the passowrd was
empty, ipmitool ignores that file and asks user to set password in stdin.
This makes the thread stop.

This patch changes power manager to write '\0' to the password file in the
case of empty password, to force the password on impitool.

Closes-bug: #1237802

Change-Id: I44f56129a8ce2af4dda8dea35eb9c60d9518d54b
Include name/level in unit test log messages

Use a more informative format than just the raw message string in log
messages captured by fixtures.FakeLogger during unit testing.  For
example, this changes cryptic unit test output such as

    pythonlogging:'': {{{
    214 -> 215...
    done
    215 -> 216...
    done
    }}}

into:

    pythonlogging:'': {{{
    INFO [migrate.versioning.api] 214 -> 215...
    INFO [migrate.versioning.api] done
    INFO [migrate.versioning.api] 215 -> 216...
    INFO [migrate.versioning.api] done
    }}}

which instantly makes it clearer that these numbers come from database
migrations.

Change-Id: I56b6fa03892e6d865ccb036fdf75d2bb499b6c93
Stylistic improvement of models.ComputeNodeStat

The patch is aimed at improving the readability of models.ComputeNodeStat.

Here, we rename the 'stats' attribute of the class to 'compute_node' (due to
the fact that the corresponding relation actually maps to ComputeNode class,
not to ComputeNodeStat). We also rewrite the relationship() method call to
make it stylistically conforming with the analogous 'service' relationship
in the ComputeNode class: we add the 'foreign_keys' parameter, wrap the
'backref' parameter into a method call and remove the auxiliary 'primary_join'
variable.

Change-Id: I2b1f27b4c1d06f364af01fc82d3490fd3bbf4ac0
Closes-Bug: #1215390
all_tenants=0 should not return instances from all tenants

Fixes set up of server list filters so specifying all_tenants=0
or all_tenants=False when requesting a list of servers does not
result in instances from all tenants being returned.

Change-Id: I72575e4e2cf9019cfd5bf3b07ada53cadf2001e2
Closes-Bug: #1240347
Move get_all_tenants policy enforcement to API

Cleanup to move the enforcement of the get_all_tenants policy
enforcement to the API level. The all_tenants search option in
compute/api.py is only used to trigger policy enforcement while
the logic required by all_tenants (not specifying a project_id or
user_id filter) already resides in the servers API. The flag does
not need to be passed to the db level as it is ignored at that
level anyway.

Partially implements blueprint nova-v3-api

Change-Id: I5200c8912596ffbd32c5f7c6d74939782ff70260
tenant_id implies all_tenants for servers list in V3 API

Makes tenant_id imply all_tenants is also enabled unless
explicitly disabled when a request is made for a list of
servers. This only occurs if the requestor is an admin. The
change is only made for the V3 API as although this behavior is
more intuitive it is too late to change it for the V2 API.

Change-Id: Ia84e62c8dc80ac3a2f16aee3620af31987cce570
Partial-Bug: 1185290
Enforce compute:update policy in V3 API

Now that compute/api.py:update() is no longer being called for instance
updates the policy check is no longer enforced.  This adds policy
checking back in at the V3 API level so that the compute:update policy will
continue to function as it did before.

This was fixed in I365ad8ac2845d914deefb841c314a50f4869dba0 for the V2
API.

Change-Id: I9caa3e10fe029569c6fd467e7b86103dd6590f9b
Closes-Bug: 1242955
Sync log from oslo

5fef5a8105 allow keeping of existing loggers with fileConfig
fb8075a19d Add amqp=WARN,qpid=WARN to default_log_levels
4654a5d247 Don't override default value for eventlet.wsgi.server logging
41ddd7a975 _get_log_file_path explictly return, when logfile/logdire unset
e12e6b45f8 Make openstack.common.log Python 3 compatible
572cfecb4d Make Messages unicode before hitting logging
3243cdfbe9 python3: handle module moves in log

DocImpact

Change-Id: I7bd940a9a2dd4af3d876f2acc650f18675ca177b
Issue brctl/delif only if the bridge exists

Add a check for existence of the bridge before we try cleaning
up as we see a lot of spurious exceptions in the logs especially
when rebuild_instance is in progress and terminate_instance is
issued

Closes-Bug: #1243488

Change-Id: Ib0e2e4d9506e3a8f63201d28a3fc2c910c77d3bd
Clean BDM when snapshoting volume-backed instances

When snapshoting a volume-backed instance, its block device mapping is
copied entirely into the image properties of the resulting snapshot.

This may lead to some errors when booting from that snapshot because the
fields 'id', 'insance_uuid' and 'connection_info' must be generated for
each instance instead of reusing the ones from the original instance.
For example, sharing the connection_info will cause the new instance to
try to connect to the original volume instead of creating a new one.

This patch cleans all the database-specific plus the connection_info
fields from the block device mapping when creating the snapshot.

Change-Id: I4190e9519d641c61b00b5d86c3885f2ae4fa86f3
Partial-Bug: #1246327
Process image BDM earlier to avoid duplicates

If the block device mapping from the image properties are not handled at
the same time as the ones defined from the API, we might make the false
assumption that the instance has no root and failing or, in the case
we're booting from an image that only defines a BDM and has no disk,
creating a local disk BDM as root and ending up with two root devices
which is forbidden.

This patch moves the handling of the image block device mappings to the
same method were we check the ones provided throught the API. This
allows nova to decide wether the instance needs a local disk from an
image if no root device is defined in any of the block device mappings.

Change-Id: Ide95357895ab4dd1338ab5ee3ec25294af1d010b
Closes-Bug: #1246327
Add test for class InsertFromSelect

InsertFromSelect is used to generate SQL statement like
"INSERT INTO table1 SELECt * FROM table2".

Change-Id: I0fec93463be1e9930b24862724b7a6d1c8cc52af
Refactor UnexpectedTaskStateError for handling of deleting instances

To avoid having to repeatedly unfold UnexpectedTaskStateErrors and
do seperate handling of the 'deleting' task state from other states,
everywhere this exception is handled, introduce a separate exception
UnexpectedDeletingTaskStateError that derives from the above.

This way, most handling of InstanceNotFound to handle instances gone
during some action can be made to also include this new exception.

Change-Id: Ia833c2b2361f32c7e33c84ec14d3af6231119a99
Related-Bug: #1223843
Catch exception while building due to instance being deleted

In run_instance() a instance_update() call in _start_building() can
result in a UnexpectedDeletingTaskStateError if the instance gets
deleted while being built.Since this is an expected situation, we
don't want this exception written in the logs or worse, the instance
be set to error state. Instead catch this and raise a proper
BuildAbortException that gets handled above this call.

Change-Id: Ib9d3dd22b33b104f1593d534876a5387df81d11c
Closes-Bug: #1223843
Ensure deleted instances' status is always DELETED

Currently when deleting failed instances that were never assigned a
host, their vm_state is left in 'error' instead of 'deleted'. This is a
problem when invoking the Index Instances REST API with a Changes-Since
query parameter since that API returns both deleted and non-deleted
instances, but currently the failed deleted instances are incorrectly
marked as status:ERROR instead of status:DELETED.

This patch modifies the server ViewBuilder to ensure the status is
correctly set.

Change-Id: I614714676bab4986564b236d74f2c0d77400e48c
Closes-bug: #1243301
Rename _get_vm_state to _get_vm_status

The method is actually used to get the server's "status", so rename it
to better reflect function.

Change-Id: Ie4b0a1c76c81f6d08250a1cfd594429ef9eaf5ae
Remove unused local variables in test case

Non-functional change. Just remove the variables that had been assigned
but were never used for clarity of the code base.

Change-Id: I4c204067cff161a975df2277d6ef9d2081863bd1
Checking existence of index before dropping

Migrations modifying indexes drop and recreate the index.
If an index does not exist dropping would fail the migration.
As part of this fix, checking if index exists before dropping it.

Fixes bug 1243553

Change-Id: Ic75744bdccfbc516d36a9711ad6997fd86c7a722
Check return code of command instead of checking stderr

Command "ip link show" return a no 0 return code when passing
a no existing ip interface so we should use the return code
instead of checking stderr because stderr can contain messages
from the system even though the command succeeded.

Change-Id: Ic43e72dab3c5b6e266240b7ff7309ca3b44f8ed5
Closes-Bug: #1248699