xenapi: remove propagate xenapi_use_agent key

The compute.api class will already propagate then xenapi_use_agent key
when required, so we can remove the xenapi specific code that tries to
propagate this value. The code did not correctly access the system
metadata, would have failed to propagate the key anyway.

Fixes bug 1204933
Change-Id: Ib291a265318d80705129e565155d4250fd704b75
Add expected_errors for extension console_output v3

Partially implements bp v3-api-extension-versioning

Change-Id: Ibb25c3e853f83c2dae35a5ba34e193a639b53ab1
Add expected_errors for extension keypair and availablity_zone

Partially implements bp v3-api-extension-versioning

Change-Id: Icb13630d5e4b8806ab95ccf464c9d80e9a9f160b
xenapi: remove pv detection

It was assumed that pygrub is there in domU, so that the RAW and unknown
disks were attached to domU and investigated with pygrub to find out if
PV could be used. This patch removes this functionality. Also some
cleanup is made around the ImageType mess.

From now RAW and unknown disks will not be investigated for PV-ness and
will boot as HVM.

Fixes bug 1196570

Change-Id: If84b93cee36938257b7ad0fa44b3b22878520506
Fix some Instance object class usage errors

This fixes a few places where Instance methods call
directly to Instance._from_db_object() instead of
using 'self' or 'cls' so that they can properly be
subclassed.

Related to blueprint compute-api-objects

Change-Id: I8e47901bd387048211d186f180991e8f780883c4
powervm: pass on unimplemented aggregate operations

The powervm driver does not implement the add/remove/undo aggregate
operations defined in the base ComputeDriver class which can cause
issues when running Tempest with the powervm driver. This patch simply
overrides the methods in the powervm driver and does a pass like in the
libvirt driver.

Fixes bug 1206528

Change-Id: Ieac227f321339a4069b015303b6e713ab9fad7b7
Fix and Gate on H303 (no wildcard imports)

Wildcard imports make reading code unnecessarily confusing because they
make it harder to see where a functions comes from.  We had two types of
wildcard imports in the code. Unneeded ones in test files that are just
removed, and some that we actually want which are kept using the '#
noqa' comment to tell flake8 to skip specific checks (such as H303)
for that line.

Change-Id: Id4705011579659fd74a4aaa05ac541e9694c483e
xenapi: no image upload retry on certain errors

There are a few upload errors that glance plugin should
not retry upload on, as it will fail again. This patch changes
upload to not retry on the following conditions in addition
to the 401 error that it already handles:

- CONFLICT
- REQUEST_ENTITY_TOO_LARGE
- PRECONDITION_FAILED
- FORBIDDEN

Fixes bug 1203726

Change-Id: I538a4f9b133169b2e398e7dfcb3af58fef204f03
Sync config.generator from Oslo

Includes:
7448e7e Fix hostname conflict in config generator

Fixes bug 1205411

Change-Id: I13988ef73db05bbdab355400c19e1ad136b59fc0
Add expected_errors for extensions quota_sets and hypervisors

Partially implements bp v3-api-extension-versioning

Change-Id: Ic9b2bebc5974a47beebee6bd6eba192ee0ede74b
Cleanup copy/paste in test_quota_sets

Tests in test_quota_sets extensively use quota_set data that's
copy/pasted everywhere, so introduce a method that generates this
quota_set data and make use of it in the tests.

blueprint nova-tests-code-duplication

Change-Id: Icac3e4b5f48592c63a8376c84a744f3dac0cec36
Refactor EC2 API error handling code

Implements blueprint ec2-error-codes

Redundant logging code was united in new __init__.py:ec2_error_ex() helper
function and all standard EC2 error responses are now uniformly logged on
DEBUG log level because they are just responses to invalid client requests,
not nova errors.  Unexpected client/server errors are logged on ERROR/CRITICAL
level and their exception name is used as error code - no more useless
UnknownError. Error descriptions are supressed for unexpected server errors as
they might contain sensitive information. New unit tests are provided.

The code for consturcting EC2 error response which was duplicated in
__init__.py:ec2_error and faults.py:Fault was moved to
faults.py:ec2_error_response.

__init__.py:exception_to_ec2code helper function now always returns exception
name as an EC2 error code since there are no exceptions with ec2_code
attribute yet. These will be added in the follwing patch as required by
referenced blueprint.

This code cleanup serves as a base for future fix of current terrible EC2
error behavior.

Change-Id: Ie88b089dfe536a56f5a46e9c92ff90c6a1fef793
maint: remove unused exceptions

* nova/exception (InstanceSystemMetadataNotFound):
Unreferenced since commit 0d417fa73
* nova/exception (CellError): Never referenced.

Change-Id: I30449748fe1156adf7da03aaace565e63bdac144
Refactor compute API's delete to properly do local soft_deletes

Right now, if you try to soft_delete something that is on a downed
host, the _local_delete code will actually hard-delete it, which
isn't what you would expect. The compute init code already handles
the case of soft-deleted instances while it was down, so we just
need to alter the frontend behavior.

Note that tests for this are added in the next patch, which was
written first and because there would be a ton of conflicts
generated by inserting them here.

Change-Id: I09f33bbcd7799b2bc1289b6100b291a7e6b26d9c
Make delete/soft_delete use objects

This makes the delete and soft_delete paths use objects.

Related to blueprint compute-api-objects

Change-Id: I74bf4ad99e28d4c349c2f739d8d515e432913194
Convert pause/unpause to use objects

Switch pause/unpause to use objects.  This also converts the compute API
tests to be unit tests.

Related to blueprint compute-api-objects

Change-Id: I439e06e7c9778b18e8a736dcbc76db534bcdb835
Fix change detection logic in conductor

The change detection logic in conductor attempts to determine what
changes an object action had on the object itself in order to
communicate them back to the caller. This logic previously assumed
that if an attribute was present on the updated object that it was
also present on the original. However, that will not be the case
in situations where a method sets a new attribute (such as
instance create, which will set many of them).

This change to the logic avoids triggering a lazy-load of the
attribute on the original object while trying to determine if
it has changed. It corrects behavior and doesn't change the interface
at all, so no RPC version changes are needed.

Related to blueprint unified-object-model

Change-Id: I8b01185ba81f3126f494db95c70c23dfa77ad959
Add ObjectActionFailed exception and make Instance use it

The obj_load_attr() code in Instance slipped through review with
just a "raise Exception()" for the case where the load cannot
complete. This adds a new exception for the case, fixes the
Instance.obj_load_attr() method, and adds a test to verify it.

Related to blueprint compute-api-objects

Change-Id: I73c966deb3c498019370eb83ade64bff2c48aec0
Add obj_attr_is_set() method to NovaObject

We have lots of places where we need to test if an object's attribute
has been set yet, and we do that rather manually. This adds a
method to the base object to make that more convienient and consistent.

Related to blueprint compute-api-objects

Change-Id: Ic5ff4a80b10dec6a5a772c21405de7ff1d331e72
Update mailmap

Change-Id: I6b6530841f242829d394998a6dfd85b34e04f232