xenapi: remove propagate xenapi_use_agent key

The compute.api class will already propagate then xenapi_use_agent key
when required, so we can remove the xenapi specific code that tries to
propagate this value. The code did not correctly access the system
metadata, would have failed to propagate the key anyway.

Fixes bug 1204933
Change-Id: Ib291a265318d80705129e565155d4250fd704b75
xenapi: remove pv detection

It was assumed that pygrub is there in domU, so that the RAW and unknown
disks were attached to domU and investigated with pygrub to find out if
PV could be used. This patch removes this functionality. Also some
cleanup is made around the ImageType mess.

From now RAW and unknown disks will not be investigated for PV-ness and
will boot as HVM.

Fixes bug 1196570

Change-Id: If84b93cee36938257b7ad0fa44b3b22878520506
xenapi: no image upload retry on certain errors

There are a few upload errors that glance plugin should
not retry upload on, as it will fail again. This patch changes
upload to not retry on the following conditions in addition
to the 401 error that it already handles:

- CONFLICT
- REQUEST_ENTITY_TOO_LARGE
- PRECONDITION_FAILED
- FORBIDDEN

Fixes bug 1203726

Change-Id: I538a4f9b133169b2e398e7dfcb3af58fef204f03
maint: remove unused exceptions

* nova/exception (InstanceSystemMetadataNotFound):
Unreferenced since commit 0d417fa73
* nova/exception (CellError): Never referenced.

Change-Id: I30449748fe1156adf7da03aaace565e63bdac144
Convert pause/unpause to use objects

Switch pause/unpause to use objects.  This also converts the compute API
tests to be unit tests.

Related to blueprint compute-api-objects

Change-Id: I439e06e7c9778b18e8a736dcbc76db534bcdb835
Fix change detection logic in conductor

The change detection logic in conductor attempts to determine what
changes an object action had on the object itself in order to
communicate them back to the caller. This logic previously assumed
that if an attribute was present on the updated object that it was
also present on the original. However, that will not be the case
in situations where a method sets a new attribute (such as
instance create, which will set many of them).

This change to the logic avoids triggering a lazy-load of the
attribute on the original object while trying to determine if
it has changed. It corrects behavior and doesn't change the interface
at all, so no RPC version changes are needed.

Related to blueprint unified-object-model

Change-Id: I8b01185ba81f3126f494db95c70c23dfa77ad959
Add ObjectActionFailed exception and make Instance use it

The obj_load_attr() code in Instance slipped through review with
just a "raise Exception()" for the case where the load cannot
complete. This adds a new exception for the case, fixes the
Instance.obj_load_attr() method, and adds a test to verify it.

Related to blueprint compute-api-objects

Change-Id: I73c966deb3c498019370eb83ade64bff2c48aec0
Add obj_attr_is_set() method to NovaObject

We have lots of places where we need to test if an object's attribute
has been set yet, and we do that rather manually. This adds a
method to the base object to make that more convienient and consistent.

Related to blueprint compute-api-objects

Change-Id: Ic5ff4a80b10dec6a5a772c21405de7ff1d331e72
Update mailmap

Change-Id: I6b6530841f242829d394998a6dfd85b34e04f232
Fix instance_usage_audit_log v3 follow REST principles

Use query parameter 'before' instead of show.
BTW, add expected_errors decorator.

Partially implements bp v3-api-extension-versioning

DocImpact

Change-Id: If1168c01e6342675370acdf278e1c0317aff93d8
Raise 404 when instance not found in admin_actions API

The pause/unpause/suspend/resume/reset_network admin action APIs do not
handle the case where the instance in the request is not found and they
raise HTTPUnprocessableEntity (code 422) instead. There are several
other operations in the same API which handle the 404 case and translate
it correctly, this patch cleans up the rest.

Closes-Bug: #1204999

Change-Id: If54d6be99db55b4f99da11fe75c14bf21685e809
remove improper usage of 'assert'

The assert will not raise AssertionError when run Python with '-O' flag.
And at the same time, any program should have a same result with or
without '-O' flag. So we could use assert to check some rare condition,
but we couldn't use it in logic flow or except it must raise an
Assertionerror, for example:

this is not bad:

def func(fname):
    assert os.path.exist(fname)
    f = open(fname)
    ...

but this will have different results with or without '-O' flag

def func(num):
    try:
        assert num is 5
    except: Assertionerror:
        log.error("your number isn't 5!")

The second case is what I fixes in this patch.

fixes bug #1199354
Change-Id: I20e83ecfc181d213f357aa74e0af3a110526904c
Clean up db.sqla.api.instance_* methods

Remove session parameter from public db methods:
instance_get_all_by_filters
instance_metadata_get
instance_metadata_update
instance_system_metadata_get
instance_system_metadata_update

They were never called with session parameter and they
are fully covered by tests so we are able just to remove
session parameter.

Clean up instance_metadata_get and instance_system_metadata_get
Convert rows to dict in one line instead of 4.

blueprint db-session-cleanup

Change-Id: Ib2f50cc4b5a7a2e283d891141a88c7b1ff9f7d43
Clean up session in db.sqla.api.get_ec2 methods

Remove session parameter from public methods

  get_ec2_volume_id_by_uuid
  get_volume_uuid_by_ec2_id
  get_ec2_snapshot_id_by_uuid
  get_snapshot_uuid_by_ec2_id
  get_ec2_instance_id_by_uuid
  get_instance_uuid_by_ec2_id

Because they are never called with session parameter
we could just remove it.

blueprint db-session-cleanup

Change-Id: Iecced3cf144de4f58a1169ed2d8f6940e229bedb
Add expected_errors for extension baremetal v3

Partially implements bp v3-api-extension-versioning

Change-Id: I294aa1cb4e19aba755d8c2c78fd997c7e447204e
Add expected_errors for extension Console v3

Partially implements bp v3-api-extension-versioning

Change-Id: Ie4c20eb494c68b0d75114872ae7b227da286d609
Add expected_errors for extensions simple_tenant_usage v3

Partially implements bp v3-api-extension-versioning

Change-Id: I69d17279f4c8efe13289458fc4aa583b9168fcc7
Fix Instance objects with empty security groups

This makes sure that a DB instance with an empty list of security
groups does not result in an object with an unset attribute, thus
triggering a lazy load on access (and ultimately an exception).

Related to blueprint compute-api-objects

Change-Id: Ib33b05853e9d067cbac5d3e87824a68e9935b822
Allow user and admin lock of an instance

The existing instance lock capability is currently restricted
to being an admin-only action in the default policy. This
introduces additional functionality that allows the lock to be
used by users (to prevent accidental changes to an instance for
example), but also allows the lock to taken by an administrator
with appropriate privileges such that it overrides any lock held
by the user (for example to block actions on a rouge instance).

When unlocking an instance, an additional check is made to see if
the lock is held by user. If it isn't, an additional check is
made against the unlock_override policy.

Additionally added the current instance lock status to the
v3 extended status extension.

Blueprint mandatory-vm-lock
Flags: DocImpact

Change-Id: I5040276c2d9fd42a71c9d8d4529331992abd7538
Merge "Update mailmap"