Add expected_errors for extension console_output v3

Partially implements bp v3-api-extension-versioning

Change-Id: Ibb25c3e853f83c2dae35a5ba34e193a639b53ab1
Add expected_errors for extension keypair and availablity_zone

Partially implements bp v3-api-extension-versioning

Change-Id: Icb13630d5e4b8806ab95ccf464c9d80e9a9f160b
Fix some Instance object class usage errors

This fixes a few places where Instance methods call
directly to Instance._from_db_object() instead of
using 'self' or 'cls' so that they can properly be
subclassed.

Related to blueprint compute-api-objects

Change-Id: I8e47901bd387048211d186f180991e8f780883c4
powervm: pass on unimplemented aggregate operations

The powervm driver does not implement the add/remove/undo aggregate
operations defined in the base ComputeDriver class which can cause
issues when running Tempest with the powervm driver. This patch simply
overrides the methods in the powervm driver and does a pass like in the
libvirt driver.

Fixes bug 1206528

Change-Id: Ieac227f321339a4069b015303b6e713ab9fad7b7
Fix and Gate on H303 (no wildcard imports)

Wildcard imports make reading code unnecessarily confusing because they
make it harder to see where a functions comes from.  We had two types of
wildcard imports in the code. Unneeded ones in test files that are just
removed, and some that we actually want which are kept using the '#
noqa' comment to tell flake8 to skip specific checks (such as H303)
for that line.

Change-Id: Id4705011579659fd74a4aaa05ac541e9694c483e
Sync config.generator from Oslo

Includes:
7448e7e Fix hostname conflict in config generator

Fixes bug 1205411

Change-Id: I13988ef73db05bbdab355400c19e1ad136b59fc0
Add expected_errors for extensions quota_sets and hypervisors

Partially implements bp v3-api-extension-versioning

Change-Id: Ic9b2bebc5974a47beebee6bd6eba192ee0ede74b
Cleanup copy/paste in test_quota_sets

Tests in test_quota_sets extensively use quota_set data that's
copy/pasted everywhere, so introduce a method that generates this
quota_set data and make use of it in the tests.

blueprint nova-tests-code-duplication

Change-Id: Icac3e4b5f48592c63a8376c84a744f3dac0cec36
Refactor EC2 API error handling code

Implements blueprint ec2-error-codes

Redundant logging code was united in new __init__.py:ec2_error_ex() helper
function and all standard EC2 error responses are now uniformly logged on
DEBUG log level because they are just responses to invalid client requests,
not nova errors.  Unexpected client/server errors are logged on ERROR/CRITICAL
level and their exception name is used as error code - no more useless
UnknownError. Error descriptions are supressed for unexpected server errors as
they might contain sensitive information. New unit tests are provided.

The code for consturcting EC2 error response which was duplicated in
__init__.py:ec2_error and faults.py:Fault was moved to
faults.py:ec2_error_response.

__init__.py:exception_to_ec2code helper function now always returns exception
name as an EC2 error code since there are no exceptions with ec2_code
attribute yet. These will be added in the follwing patch as required by
referenced blueprint.

This code cleanup serves as a base for future fix of current terrible EC2
error behavior.

Change-Id: Ie88b089dfe536a56f5a46e9c92ff90c6a1fef793
maint: remove unused exceptions

* nova/exception (InstanceSystemMetadataNotFound):
Unreferenced since commit 0d417fa73
* nova/exception (CellError): Never referenced.

Change-Id: I30449748fe1156adf7da03aaace565e63bdac144
Refactor compute API's delete to properly do local soft_deletes

Right now, if you try to soft_delete something that is on a downed
host, the _local_delete code will actually hard-delete it, which
isn't what you would expect. The compute init code already handles
the case of soft-deleted instances while it was down, so we just
need to alter the frontend behavior.

Note that tests for this are added in the next patch, which was
written first and because there would be a ton of conflicts
generated by inserting them here.

Change-Id: I09f33bbcd7799b2bc1289b6100b291a7e6b26d9c
Make delete/soft_delete use objects

This makes the delete and soft_delete paths use objects.

Related to blueprint compute-api-objects

Change-Id: I74bf4ad99e28d4c349c2f739d8d515e432913194
Convert pause/unpause to use objects

Switch pause/unpause to use objects.  This also converts the compute API
tests to be unit tests.

Related to blueprint compute-api-objects

Change-Id: I439e06e7c9778b18e8a736dcbc76db534bcdb835
Fix change detection logic in conductor

The change detection logic in conductor attempts to determine what
changes an object action had on the object itself in order to
communicate them back to the caller. This logic previously assumed
that if an attribute was present on the updated object that it was
also present on the original. However, that will not be the case
in situations where a method sets a new attribute (such as
instance create, which will set many of them).

This change to the logic avoids triggering a lazy-load of the
attribute on the original object while trying to determine if
it has changed. It corrects behavior and doesn't change the interface
at all, so no RPC version changes are needed.

Related to blueprint unified-object-model

Change-Id: I8b01185ba81f3126f494db95c70c23dfa77ad959
Add ObjectActionFailed exception and make Instance use it

The obj_load_attr() code in Instance slipped through review with
just a "raise Exception()" for the case where the load cannot
complete. This adds a new exception for the case, fixes the
Instance.obj_load_attr() method, and adds a test to verify it.

Related to blueprint compute-api-objects

Change-Id: I73c966deb3c498019370eb83ade64bff2c48aec0
Add obj_attr_is_set() method to NovaObject

We have lots of places where we need to test if an object's attribute
has been set yet, and we do that rather manually. This adds a
method to the base object to make that more convienient and consistent.

Related to blueprint compute-api-objects

Change-Id: Ic5ff4a80b10dec6a5a772c21405de7ff1d331e72
Update mailmap

Change-Id: I6b6530841f242829d394998a6dfd85b34e04f232
Fix multi availability zone issue part 1

Current aggregate APIs allow user to add multi
AZs to a host, but Openstack support single AZ
of a host. When booting an instance, nova uses
the first AZ assigned to a host,even if user
assigned multi AZs to the host and user chose
the second AZ to boot an instance. So multi AZs
aren't supported by current Openstack design,
we should fix aggregate APIs to align the design
. This patch is for limit user to add multi AZs
using aggregate-add-host API.

The part 2 limits user to add multi AZs using aggregate
update and aggregate metadata update API, here is the
link:
I18ad1ac22c6aee298ccc75c978b85241ae1ac616

Fixes: bug #1196893

Change-Id: I788782a9f21ec2672551f75123753175bb268586
Code dedup in class QuotaReserveSqlAlchemyTestCase

Moved common parts to setUp()
   -quotas
   -deltas
   -usages
Move reservations update and usages init to separate functions

bp nova-tests-code-duplication

Change-Id: I03efed0d1f40ba7670c919efed4fdafaf59576f8
xenapi: Rename imageupload image

The imageupload driver concept has helped detangle some of the image managment
code in vm_utils.

It would be nice to extend that benefit to download as well, with an
additional benefit of being able to share more code between the upload and
download paths.

This is the first in a series of patches that aims to to do just that. The
first step is change the name so that it's not just 'imageupload' but
'image'es in general that the driver is managing.

DocImpact

Change-Id: Ia18b82568147a9a40b0da67d74bf20b12d08b9a9