Fixed typos in nova.objects.base docstrings

Change-Id: I1d485a2ca2fca58262bc023e5c9f273c6632e4a6
Remove useless host parameter in virt

For host api, there are useless host params for
host_power_action and set_host_enable virt API,
this patch removes the useless host params.

Change-Id: I556278fc426a800f108da9129e42175da32dac49
Deduplicate some INFO and AUDIT level messages

I'm trying to debug a problem where the last thing in the logs is
"Starting instance...".  Unfortunately this occurs twice in the same
source file, so it's hard to figure out a priori which log statement
it came from.  Audit and info messages don't disambiguate their
function/line number, all you get is the logger name like:

2014-04-06 19:51:01.871 8288 AUDIT
  nova.compute.manager [req-...] [instance: ...] Starting instance...

I think it doesn't hurt to to make sure all logs at this level are
unique, even if upstream were to log the function (see [1]).  Most
duplicate LOG.info and LOG.audit level messages are in separate files
so we can disambiguate by the name, but there are a few that overlap.

For reference, the less-then-perfect but useful check I used was:
 $ grep -r 'LOG.audit' * | sed 's/:\s*/ /' | sort -k1 | uniq -f1 -c

[1] https://review.openstack.org/#/c/85638/

Change-Id: If1ff573ffed1d12fcadfbdfd3365aae68a5fb977
Add 'instance-usage-audit-log' plugin for V2.1

This patch adds 'instance-usage-audit-log' plugin for V2.1 API.
Also implement the API sample tests and share the unit tests
between V2 & V2.1.

Partially implements blueprint v2-on-v3-api

Change-Id: I152d00fb4b12942515d5db54a7ec381279c31ccd
Share console-auth-tokens tests between V2 & V2.1

As V2 and V2.1 'os-console-auth-tokens' extensions are same,
unit tests can be shared between those.

This patch share the tests between V2 & V2.1

Partially implements blueprint v2-on-v3-api

Change-Id: I9c1ff61ff73fee551a60a0ff684dd73a1bbad72e
Compute: Add start notification for resume

The resume operation did not have a 'start' and 'end'. This has
been updated. Notifications should be sent at the start and end
of each opertaion. The support should be consistant throughout
the driver.

Change-Id: I3354d1d9c01b13bc2fed70ee06875f6d49ff1d16
Virt: create an object InstanceInfo

The virt driver method get_info returns a distionary. This patch
introduces an object for the InstanceInfo.

Change-Id: Ia4e4639ec1a593380b48f8f6e4962dd22ab8fc0b
Virt: make use of the InstanceInfo object

This ensures that all drivers and the compute manager make use of
the instance info object and do not pass around random dictionaries.

Complete the blueprint instance-info-objects

Change-Id: Idab57d33e8612fcf40f54f73d5f73d813629bddf
Convert v3 console plugin to v2.1

This patch changes v3 console plugin to v2.1 and makes v2
unit tests share between v2 and v2.1.

The differences between v2 and v3 are described on the wiki page
https://wiki.openstack.org/wiki/NovaAPIv2tov3.

This patch also fix below issue in V2.1 console plugin-
 -  Remove try catch of 'InstanceNotFound' exception from V2.1
    console->index. This returns empty dict if non-existent server
    is specified. DB does not raise 'InstanceNotFound' exception.

Partially implements blueprint v2-on-v3-api

Change-Id: I306712f47457843c94e6441c4b9db7bf477c394c
libvirt: set NUMA memory allocation policy for instances

When NUMA placement is in effect, Nova configures KVM guest vCPUs
to be placed on certain host NUMA nodes. It also provides guest
NUMA topology of vCPUs and RAM. The kernel will normally satisfy
memory allocations from the host NUMA node that the guest vCPU
is running on, but this is only a preference. Nova needs to set
a strict policy to guarantee guest RAM is always allocated from
the desired host NUMA node. This change ensures that by setting
the XML

  <numatune>
    <memory mode='strict' nodeset='0-1'/>
    <memnode cellid='0' mode='strict' nodeset='0'/>
    <memnode cellid='1' mode='strict' nodeset='1'/>
  </numatune>

Closes-bug: #1385308
Change-Id: I02c301ae648235dc48527559a5d3c3d45de859ac
Fix live migration api stuck when migrate to old nova node

Commit bc45c56f102cdef58840e02b609a89f5278e8cce disable instance
live migration between different verison node. If the two nodes
with different rpc version, will raise exception
LiveMigrationWithOldNovaNotSafe. But this exception didn't catch
as expected exception, that make the instance status stuck at
migrating. This patch add LiveMigrationWithOldNovaNotSafe as
expected exception.

Change-Id: I1dcee9181fd0ef293628b30766112f00792796ab
Closes-Bug: #1377644
Fix circular reference error when live migration failed

When unexpected exception raised in live migration, the MigrationError
carry another original exception in it. But when oslo.message
serialize the exception, the circular reference error happended. This
patch just pass the exception as unicode string into MigrationError.

Change-Id: I4e449baae74bd9a15490ae7accbd2103bae90d57
Related-Bug: #1377644
Shelve_offload() should give guests a chance to shutdown

Change Iec3dfd17725440958aac395ebc471e51afd6522e added clean_shutdown
support to shelve operations, but ignored shelve_offload as its normally called
either as a second stage to shelve().   However BFV instances call
shelve_offload() directly, so it too should have clean_shutdown support.

Change-Id: Ida612a70ad3b8a7b7b9454579870569c4129ee48
Closes-Bug: 1384637
Partially-Implements: blueprint user-defined-shutdown
Create instance_extra items atomically with the instance itself

This cleans up how we create instance_extra items by making them
go as part of the db.instance_create() call and be created according
to the instance.extra backref. It should basically eliminate the need
for .create() methods on the individual objects. Since this also makes
the Instance object create pci_requests the way it was doing for
numa_topology, this updates (and freshens) the test for that to make
sure that pci_requests get created properly as well.

Related to blueprint flavor-from-sysmeta-to-blob

Change-Id: I01c490c3cb731133caa8c2c93824458a694e9630
GET servers API sorting compute/instance/DB updates

This change is to support updating the v2 and v3 /servers and
/servers/detail APIs to support multiple sort keys and sort directions
(using the 'sort_key' and 'sort_dir' parameters); these parameters can
be specified multiple times to create a list of sort keys and
directions.

Contains the changes that will be used by the REST API layer:
* Compute API updates on the get_all function to support sort keys
  and directions
* InstanceList updates on the get_by_filters function. The new
  'sort_keys' and 'sort_dirs' kwargs are added but the existing
  'sort_key' and 'sort_dir' kwargs must remain for backward
  compatibility.
* DB API has a new 'instance_get_all_by_filters_sort' function that
  supports multiple keys/directions. The existing
  'instance_get_all_by_filters' remains for backward compatibility
  and calls the new sort function with the single sort values
  wrapped in a list.

Note that the defaulting of the sort keys and directions is done in
the dependent patch set in the new 'process_sort_params' function
(invoked in db.sqlalchemy.api); by default, the sort keys are
'created_at' and 'id' in the 'desc' direction.

The REST API changes that consume these updated will be pushed in
another change set.

Change-Id: Iec2bba18c8894302e3ba35af69379ea9c1da04fc
Partially implements: blueprint nova-pagination
Generalize dependent object backporting

This adds some infrastructure to the base NovaObject to handle backporting
subobjects based on version rules. Here, we introduce a list of version
mapping for each sub-object and logic in the base obj_make_compatible()
to honor those.

Note that this includes a set of rules for security_group_rule because
there is a test that ends up with a call to obj_make_compatible() that
complains if it's not there.

Related to blueprint kilo-objects

Change-Id: I9582a8519ceb7e131550aa97ef16995da6dd1ff2
Don't make a no-op DB call

drivers_uuid can be empty, but we are still doing a DB call. This is causing
a sqlalchemy SAWarning.

SAWarning: The IN-predicate on "instances.uuid" was invoked with an
empty sequence. This results in a contradiction, which nonetheless can
be expensive to evaluate.  Consider alternative strategies for improved
performance.

Change-Id: Ib8c9b85e84800c9e2ddcdf204851f1b51101926c
Closes-Bug: #1383617
Allow passing columns_to_join to instance_get_all_by_host_and_node()

This allows us to specify which columns should be loaded during this get
call. It is needed for the next patch which adds support for extra items
in the expected_attrs lists.

Change-Id: Id7d8fae9b0d3c7cf1f0f7debbe5d851696759c2c
object-ify flavors api and compute/api side of RPC

This change converts flavors to use objects instead of direct database
access. The rpcapi primitivizes objects before they go over the wire.

The flavors.add_flavor_access() and flavors.remove_flavor_access()
functions were also removed as flavor object add_access() and
remove_access() should be used instead.

Related to blueprint kilo-objects

Change-Id: I86432720b600e479d8d8b7f7531a2c85ab364319
Updated from global requirements

Change-Id: Ibf3abb2cf0ca3fc18fb8fc7931657bed7ed51924