Make _downsize_quota_delta() use stashed instance types

This change changes the _downsize_quota_delta() method to take instance
as a param instead of migration_ref, and use it to extract the old and new
instance_types instead of looking them up in the database. This is needed
for this case since the method can be called from compute manager during
the periodic task to confirm pending migrations, and thus access to the
database may not be allowed.

Fixes bug #1158897

Change-Id: I647c51bc4d30e6dfd62abe7b65a9109d6aa2bb6f
Initialize compute manager before loading driver.

The compute virt API uses the conductor API instance on the compute
manager.  Make sure the compute manager has been fully initialized
before loading the driver and passing it the virtapi instance.  This
prevents the possibility of the driver calling back into the compute
manager and using something not initialized yet, such as the conductor
API in the case of this bug.

This patch also moves initialization of a member variable up above
super(), where the rest of the initialization is.  There doesn't appear
to be a reason for needing to have it at the end.

Fix bug 1156490.

Change-Id: I19684f24d590201d135336107425c2be2f74c83e
(cherry picked from commit 9075069098e32b47bd5011e2653a23b61c18d4a3)
Sync lockutils from oslo-incubator stable/grizzly.

Changes include:

3cb71ff Locking edge case when lock_path does not exist
21925b6 Fix locking issues in Windows
547ab34 Fix Copyright Headers - Rename LLC to Foundation

Fix bug 1158179.

Change-Id: I8231a6d37b70c36ba2ce245407b02698faa55aaf
Import latest openstack.common.setup

Import latest openstack.common.setup from oslo-incubator.
Fixes bug 1158319 (Include Co-authored-by entries in AUTHORS), but also:
* stop using --stat for ChangeLog
* failing to get version from git
* returncode for Popen is valid only after communicate call

Change-Id: Ib7a391560ec885b5478208cd441ce2c10e9d8cac
Merge "Initialize compute manager before loading driver." into milestone-proposed
Merge "Sync lockutils from oslo-incubator stable/grizzly." into milestone-proposed
Merge "Import latest openstack.common.setup" into milestone-proposed
Fixes passing arbitrary conductor_api argument

Fixes bug #1158679

Change-Id: Iabe1f58b6139f5619e328a986143e8591b3a02cc
(cherry picked from commit 67628c56caf9d84588a92448880ecdb33eea08b4)
Add caching for ec2 mapping ids.

Requests to map ec2 ids can be slow when there are a large number
of instances. These shouldn't ever change so cache them in memory
for a week. The one week timeout is simply to prevent memory from
growing uncontrollably.

Fixes bug 1157437

Change-Id: I3f1c68959a933a7aaa50f122c34a716264c8722d
(cherry picked from commit 76844433f69a7c29ed4566ad34d7e9740feaf660)
Remove unnecessary DB call to find EC2 AZs.

Update get_availability_zone_by_host, so it doesn't need a list of services,
and remove unneeded service_get_all_by_host calls. This removes N DB calls in
the EC2 api when doing a DescribeInstances, where N is number if instances
that are being described.

Partially fix bug 1157437

Change-Id: I0050ba736828faad94549bc6e55f4ca1d05dcbe7
(cherry picked from commit 7ab98e935a63454d0736e6f8a7525a8e73cf6c04)
Pass project id in quantum driver secgroup list

The quantum driver is always returning security groups from every tenant
a user has access to, even when the "project" filter is supplied.

Make sure we pass along the project value when we call
quantum.list_security_groups() so it's properly filtered.

Fixes bug 1155381.

Change-Id: I682c66a1f3f9db18b5f9924a37b45c759ff259f7
(cherry picked from commit 775e3b02b2afbf101db22b87a1c3b189d68532e1)
Update instance network info cache to include vif_type.

vif_type is a new param in legacy_nw_info added in grizzly, and when upgrading
from folsom to grizzly, the existing instance's network info cache in the db doesn't
contain this param, which is needed by vif driver to plug vif to instance. Nova
compute will try to plug the vif when it starts the instance, so we need to update
the existing instance's network info cache before pluging instance's VIF.

fix bug #1152426

Change-Id: I1b839bf791b402b933354d9c17c5713fde21ab09
(cherry picked from commit 45e65d8c0301da689de1afcbc9f45756e71097ab)
Don't log traceback on rpc timeout.

Don't log a traceback in the rpc layer after a timeout() occurs while
waiting on a response to a call().  Let the caller get the exception and
decide whether it should be logged or not.

Fix bug 1158920.

Change-Id: I0387ab48899627cbc3eb1d7187067472ddefc1bf
(cherry picked from commit 132a0c1fd1eb127e393e5794ffa6d4a3a4950567)
nova-manage vm list fails looking 'instance_type'

The recent set of changes to remove the join on instance_types and get
data from instance_system_metadata missed out the use in
"nova-manage vm list"

fixes bug:  #1158352

Change-Id: I5b06a4fef34331abe40e7429b3abda47070bcbbc
(cherry picked from commit b8f9815c5ab61466009c0447f54abc4c309e4e3f)
translate cinder BadRequest exception

if we attempt to create a volume with values that are invalid (like
a non numeric value for size) we properly get a BadRequest returned
from cinder. However we inproperly pass that cinderclient exception
all the way to the top of the request process, causing a stack trace
to appear in the logs because this is an exception type Nova doesn't
understand.

This situation is triggered by Tempest on every run, and while the
return is successful, the stack trace might spook people.

Fix this by increasing the scope of the existing
_reraise_translated_volume_exception() method. Longer term it would
make sense to have a decorator that handles all possible exceptions
we'd expect from cinder client and turn them into Nova exceptions.

This reduces the number of stack traces in nova-api by 6 on a
successful Tempest run.

Fixes bug #1158505

Step towards blueprint no-stacktraces-in-logs

Change-Id: Ifcfbd5eb11fe9f038f648ca5291499290b7126b1
(cherry picked from commit 3801a4d2f4c59dbfda49131ddde22fcb3976d651)
Fixes Nova API /os-hosts missing element "zone"

Now the HostIndexTemplate of Nova /os-hosts API is missing element "zone",
which will cause the XML output of list hosts missing the zone attribute.

Fixes bug: 1159055

Change-Id: I9940efcfd364c35b00cde4398d531d7ff56dc2b9
(cherry picked from commit c5097ab6682ee1b3021bb628e5083108e3b368d0)
Merge "nova-manage vm list fails looking 'instance_type'" into milestone-proposed
Merge "Add caching for ec2 mapping ids." into milestone-proposed
Merge "translate cinder BadRequest exception" into milestone-proposed
Merge "Fixes Nova API /os-hosts missing element "zone"" into milestone-proposed