maint: remove an unused import in libvirt.driver

This import was recently reintroduced, due to
confusing pylint output. Therefore remove it again
and adjust things so pylint won't warn in future.

Change-Id: I25f293f171c651c25d27e8d55c9240fd808590ba
General host aggregates part 2

Partially implements blueprint general-host-aggregates: Scheduler Filter
* Add AggregateInstanceExtraSpecsFilter
* change db.aggregate_get_by_host to return a list of aggregates
  instead of the first aggregate
* Add optional key filter to db.aggregate_get_by_host along with test
* Add db.aggregate_metadata_get_by_host to get host aggregate metadata
* add optional key filter to db.aggregate_metadata_get_by_host
* Add AggregateTypeAffinityFilter

Change-Id: I4a3061276cc3b0c5c695eaf973b773183b3c4f4b
Moves security group functionality into extension

partially implements blueprint disable-server-extensions

This patch adds an output extension for security group output
and modifies the servers code to only allow security groups
to be specified if the os-security-groups extension is
enabled.

Duplicated security group functionality was removed from the
createserverext extension, it is now a duplicate of the servers
endpoint. The boot_from_volume extension was also changed
to inherit the servers functionality so that security_groups
will continue to work. The block_device_mapping code will be
modified in a subsequent patch.

Tests were added for new functionality, and existing tests were
modified slightly. The disk_config extension was modified to be
a bit more lenient so the data for it doesn't need to be mocked
out in other tests.

Change-Id: I59bd985d68b85ce6926cde85fbc38cfad4e75f4c
Key block_device_mapping off of volume extension

partially implements blueprint disable-server-extensions

Instead of creating a child controller for boot from volume,
key the post parameter of of the os-volumes extension.

Change-Id: I0dd233c2ae12a09d6fc59c49042c2f8b83f3fdba
Key user_data in create server off of extension

Implements part of blueprint disable-server-extensions

Adds a new extension descriptor for user_data and makes sure that
user_data is only accepted in the create server request if the
extension is enabled.

Change-Id: I7aef0ebe569da841adbf380d5695c9029fb3b002
Create unique volumes_dir for testing.

Updates several Nova volume test classes so that we create a
unique volumes_dir for testing. This avoids polluting the source
tree with fake volume UUID's when running tests.

Note: I opted to go utils.tempdir in test_volume (which seemed
cleaner). I did not however use utils.tempdir in the other
test modules since it would have required a good be more
reformatting effort...

Change-Id: I24a53c28c52156750cf65942639af0b15fbeefc1
Key availability_zone in create server off of ext.

Implements part of blueprint disable-server-extensions

Adds a new extension descriptor for availability_zone and makes sure
that availability_zone is only accepted in the create server request
if the extension is enabled.

Change-Id: I326b26c4741ffa6a8b594be26d0cd22d032a6162
Key min_count, max_count, ret_res_id off of ext.

Implements part of blueprint disable-server-extensions

Adds a new extension descriptor callsed os-multiple-create. Modifies
the min_count, max_count, and return_reservation_id parameters to
server create to only be allowed if the extension is enabled.

Change-Id: Iaca009588d6c824e0852a060787d2a4cb9614278
Don't accept scheduler_hints if not enabled

partially implements blueprint disable-server-extensions

moves the filling out of the code behind a conditional to
ensure this data is only passed if the extension is actually
enabled

Change-Id: If866eb87c9e6189b6948cb37f0cb8f5e26c8cced
Make FaultWrapper handle exception code = None.

Updates the FaultWrapper middleware so that it properly handles
exceptions which contain a code variable set to None. Previously
you'd get TypeError exception like this in the api.log file:

 TypeError: %d format: a number is required, not NoneType

This was due to the fact that we tried to format a string with
an integer value which was set to None.

I hit this today when using Qpid which does in fact throw exceptions
which contain 'code' variables set to None.

Fixes LP Bug #1035159.

Change-Id: I7193031b1f5f9bf84cdb476f8f1268efc50eadf0
Add error log for live migration.

Change-Id: I740c0def3b1f759aec3d4ae5fe3264484c704807
Signed-off-by: Takaaki Suzuki <suzuki@midokura.com>
Revert "Remove unused add_network_to_project() method"

This reverts commit d8e39cb775a872c46d067bee8febc40dee799369.

Change-Id: I22f0692814ff029b4960ab32cbee63bbf4922106
Compute restart causes period of network 'blackout'

Fixes bug 1034401

When a compute service is restarted each instance running on the
host has its iptables rules built and applied sequentially during
the host init stage. The impact of this, especially on a host
running many instances, can be observed as a period where some
instances are not accessible as the existing iptables rules have
been torn down and not yet re-applied.

The presented work-around for this is a configurable/flagged deferred
mode that prevents the application of the iptables rules until all
instances on the host had been initialised then the rules for all
instances are applied all at once preventing a 'blackout' period.

Change-Id: I0da90d07e54225fb63f3884897fb00a6027cd537
maint: compare singletons with 'is' not '=='

Looks like newer pep8 will get checks for these:
https://github.com/jcrocholl/pep8/commit/b04975dc
Using 'is' is slightly faster anyway :)

Change-Id: Id14951791991cca919b54ed747ff99fb66aa3352
Merge "Add error log for live migration."
Sync some cleanups from openstack.common

Changes to openstack.common.notifier:

 - fix bug lp:1019348,update openstack-common to support pep8 1.3.

Changes to openstack.common.setup:

 - Fix errors reported by pyflakes

Changes to openstack.common.timeutils:

 - Remove unused imports

Change-Id: Iac46d11a996857c84e06a8dbd3ecfcd774142ef8
Sync with latest version of openstack.common.cfg

Changes since last sync:

 - Modifies _is_opt_registered fcn to check for duplicate opts
 - allow empty config values

Change-Id: I5150404a0ff97f37d1d1f1b778736fa385d33860
Making security group refresh more specific

Fixes bug 1029495

The trigger_members_refresh method in compute.api.py specifies
a group id in the call to refresh_security_group_members. This
is just the last group id seen and ignores the fact that a
refresh may impact members of multiple groups.

This is masked by the fact that on the host the group id is
ignored and all instances have their security rules refreshed
regardless of if they are part of the changed group or not.

This change modifies the logic surrounding refreshes so we send
a refresh request for each instance which is affected by a
security group change, this ensures we aren't spending time
refreshing unaffected instances and also removes the possibility
of refreshing an instance multiple times if it is a member of
more than one group.

Also changed to be instance-centric is the refresh carried out
when a rule is added/removed to a security group.

Change-Id: Iec98e9aed818fdc4ecc88c8dcdd4ee5fa9386e00
Send host not ComputeNode into uptime RPC call.

Updates the hypervisor uptime API extension to send the
correct hostname to the get_host_uptime in the compute API.

Fixes LP Bug #1035315.
Change-Id: I4840f16bd3c60063e08e2de8313ef5ed3510555e
Make sure ec2 mapping raises proper exceptions

The get commands in the db layer were constructing InstanceNotFound,
VolumeNotFound, and SnapshotNotFound with imporoper kwargs. This
fixes them to raise properly and adds a test to verify that the
exception formatting succeeded.

Fixes bug 1035375

Change-Id: I05a5bb57c047ca2b2f086c70ad12a640f51b2d3a