Prevent nova services' coverage data from combining into nova-api's

Now if combining a coverage report once, nova services' coverage
data are combined into nova-api's.
This renames nova-api coverage datafile to .nova-coverage.api and
creates a new coverage instance to combine when reporting.

Fixes bug: 1152475

Change-Id: I90b6535d490cf0123a127f8752986edd782ef37e
Pass project id in quantum driver secgroup list

The quantum driver is always returning security groups from every tenant
a user has access to, even when the "project" filter is supplied.

Make sure we pass along the project value when we call
quantum.list_security_groups() so it's properly filtered.

Fixes bug 1155381.

Change-Id: I682c66a1f3f9db18b5f9924a37b45c759ff259f7
List InstanceNotFound as a client exception

get_instance_nw_info() in nova.network.manager should list
InstanceNotFound as a client exception for rpc. This to prevent the
rpc layer from logging a traceback when this exception is raised.
This is an expected type of exception that should be left to the
client to handle.

Resolves bug 1157411.

Change-Id: I6735fb8a386f2256ac8af0ce7fdb36ef7a00b6ce
Add caching for ec2 mapping ids.

Requests to map ec2 ids can be slow when there are a large number
of instances. These shouldn't ever change so cache them in memory
for a week. The one week timeout is simply to prevent memory from
growing uncontrollably.

Fixes bug 1157437

Change-Id: I3f1c68959a933a7aaa50f122c34a716264c8722d
Fix Hyper V instance conflicts

Fixes bug 1152368

If the hyperv driver attempts to boot an instance and a previous
instance with the same name was not properly removed, the boot will
fail due to an error creating root.vhd.  This only happens when
using CoW images.  This fix deletes the instance directory before
creating the new instance so that there will never be an old image
to interfere with creation of the new one.

This is most likely to be an issue in development and test
environments that toggle between versions of nova with no migration
path.

Change-Id: I21eab01590c426234b4b9a55abec4b2d2df0e168
xenapi: rpmbuild fixes

Fixes bug 1157695

This fix removes the parted dependency from the rpm specification file,
and also gets rid of the assumption, that the build-rpm.sh is executed
at a specific directory.

Change-Id: I1dcf9096a783a88bc09af0f4f4d5541a7e505748
Change arguments to volume_detach()

Most compute API calls accept an instance object.  To be compatible with
cells, volume_detach() should as well.  Also change it to accept the
volume object, instead of just a volume ID.  This lets us proxy the
volume object to child cells a bit more easily.

This removes 2 skipTests that we have related to volumes and cells.

Fixes bug 1157408

There was also a check that if instance_get_by_uuid() returned no
instance, it VolumeUnattached should be raised.  However, this check
would never work as InstanceNotFound would have been raised.  This
patch also corrects that.

Change-Id: Ide35ad5d8e147f6bd21b5840b901363dab0eae38
Update instance network info cache to include vif_type.

vif_type is a new param in legacy_nw_info added in grizzly, and when upgrading
from folsom to grizzly, the existing instance's network info cache in the db doesn't
contain this param, which is needed by vif driver to plug vif to instance. Nova
compute will try to plug the vif when it starts the instance, so we need to update
the existing instance's network info cache before pluging instance's VIF.

fix bug #1152426

Change-Id: I1b839bf791b402b933354d9c17c5713fde21ab09
nova-manage vm list fails looking 'instance_type'

The recent set of changes to remove the join on instance_types and get
data from instance_system_metadata missed out the use in
"nova-manage vm list"

fixes bug:  #1158352

Change-Id: I5b06a4fef34331abe40e7429b3abda47070bcbbc
Remove outdated try except block in ec2 code

We now have 'ec2_id -> glance_id mapping' so this check can be removed

Change-Id: Ib10ab5faea154bae9d129c0dc1edc24c2a99ffb7
Remove unnecessary DB call to find EC2 AZs.

Update get_availability_zone_by_host, so it doesn't need a list of services,
and remove unneeded service_get_all_by_host calls. This removes N DB calls in
the EC2 api when doing a DescribeInstances, where N is number if instances
that are being described.

Partially fix bug 1157437

Change-Id: I0050ba736828faad94549bc6e55f4ca1d05dcbe7
Make nova.virt.fake.FakeDriver useable in integration testing

With this change nova.virt.fake.FakeDriver can be used for scale testing
using devstack. By setting 'compute_driver = nova.virt.fake.FakeDriver'
in nova.conf and disabiling quota and other limits, we can load up nova
to think it is running 1000's of instances.  This particular scenario is
analagous to how the baremetal driver will work (one machine managing
100's to 1000's of instances).

Make nova.virt.fake.FakeDriver use CONF.host for hostname.

Change-Id: I731ca2968176b4e39dd2fe7fc99f479cc568398b
Merge "nova-manage vm list fails looking 'instance_type'"
Merge "Add caching for ec2 mapping ids."
Merge "Make nova.virt.fake.FakeDriver useable in integration testing"
Fixes passing arbitrary conductor_api argument

Fixes bug #1158679

Change-Id: Iabe1f58b6139f5619e328a986143e8591b3a02cc
xenapi: fix console for rescued instance

Check if the instance is in vm_state RESCUED and if so
change the name-label to reflect that. Name label is used
to get vm ref and currently the original instance is being
referenced even when in rescue mode. If rescue vm is not ready
yet, InstanceNotReady is raised.

Added tests cases for xenapi get_vnc_console for instance
and rescue mode.

Fixes bug #1157389

Change-Id: I5438103d517c40c1c87ddecd5f88b1487bfc0339
Add a comment to placeholder migrations.

We've had a review that tried to reuse migration 162 rather than
skipping to 172.  Maybe a comment will prevent that in the future.

Change-Id: Ib8b73260df593d13a81ce65c85a302a54668b95e
Initialize compute manager before loading driver.

The compute virt API uses the conductor API instance on the compute
manager.  Make sure the compute manager has been fully initialized
before loading the driver and passing it the virtapi instance.  This
prevents the possibility of the driver calling back into the compute
manager and using something not initialized yet, such as the conductor
API in the case of this bug.

This patch also moves initialization of a member variable up above
super(), where the rest of the initialization is.  There doesn't appear
to be a reason for needing to have it at the end.

Fix bug 1156490.

Change-Id: I19684f24d590201d135336107425c2be2f74c83e
translate cinder BadRequest exception

if we attempt to create a volume with values that are invalid (like
a non numeric value for size) we properly get a BadRequest returned
from cinder. However we inproperly pass that cinderclient exception
all the way to the top of the request process, causing a stack trace
to appear in the logs because this is an exception type Nova doesn't
understand.

This situation is triggered by Tempest on every run, and while the
return is successful, the stack trace might spook people.

Fix this by increasing the scope of the existing
_reraise_translated_volume_exception() method. Longer term it would
make sense to have a decorator that handles all possible exceptions
we'd expect from cinder client and turn them into Nova exceptions.

This reduces the number of stack traces in nova-api by 6 on a
successful Tempest run.

Fixes bug #1158505

Step towards blueprint no-stacktraces-in-logs

Change-Id: Ifcfbd5eb11fe9f038f648ca5291499290b7126b1