sync oslo log updates

this helps address the issue where syslog gets overloaded and we
loose the ability to figure out what's going on in clients like
nova-manage

Change-Id: Iedde5ed4e8e23e204be4fa7a19e9514acf2ebf86
Remove unused nova.db.api:instance_get_all_by_reservation

Change-Id: I55192773eaa18eb81ffb2b5ec3567b09116b0d1e
Unpin routes dependency version

Fixes bug #1133258

It looks like we should be fine with 1.13 and have no reason to expect
future versions will break us.

Cinder and Quantum both require >=1.12.3 too.

Change-Id: Ice0649d98e2a46505503072c07509fe274f195b6
Unpin PasteDeploy dependency version

Fixes bug #1133262

There are no newer releases of PasteDeploy than 1.5.0 and we have no
reason to believe newer versions will break us. Indeed, if an
incompatible version is released, we will want to know about it quickly.

Change-Id: I331b33b16ea50202886b19bf5c4060d9fa46d5ec
Make allocate_for_instance() return only info about ports allocated

Previously, the compute manager was filtering the results based on the
port id that the user requested. However, if they do not pass one (which
is valid) then we can not do the filter and we fail. This makes the
allocation function do the filtering, which should not affect the normal
use case where it is only called once, since all the ports returned were
allocated at that time.

The manager filter behavior is thus removed and a test is added to
verify that the allocate method does the right thing the second time
around.

This fixes the fallout found after fixing the actual cause of bug 1131264

Change-Id: I8f3af96051268503596007491af365e8ce28f5b3
Make sure no duplicate forward rules can exist.

Some failure conditions can cause floating forwards to not be properly
removed. We know that duplicate floating rules will never work, so
even if we have had other failures there is no need to create duplicate
forward rules. This patch fixes the issue by removing all existing
forwards for a floating ip before adding new ones. It adds a method
to IptablesTable to remove rules by regex and includes tests for new
method and the specific case of a duplicate forward.

Fixes bug 1133527

Change-Id: Iaf5df8406fc49e92bac328cd81ffda1969b77d68
Improve hackings docstring detection

Previously: def foo():\n    a = '''This is not a docstring'''
would be counted as a docstring even though it isn't.

Fix this and add docstring tests.

Fix bug 1100912

Change-Id: Id4d4aa382713eea340720e166e418262ba68eaaf
Improve output msgs for _compare_result

In case of errors, the messages are now more detailed.
Fixes bug 1133621

Change-Id: I7747d7c4dcaf2199dc55139ee0235f6adca1ad36
Make compute manager revert crashed migrations on init_host()

This should provide a path to cleanup partial migrations by reverting
them back to the original state instead of leaving them in limbo. Note
that libvirt needs a change to make this work, in the case where the
unconfirmed instance's directory is still present and needs to be
removed before the rollback.

The libvirt, xenapi and powervm drivers are modified here, and tests
are added to confirm their compliance with the proper behavior.

Addresses bug 1131588

Change-Id: I85bc0f6e9cda10aa85328199d107a3ff6e240b96
Add instance_type_get() to virt api

...and remove the use of instance['extra_specs'] from the libvirt and
baremetal virt drivers. Also remove the hack in instance_update()
which places them there in the first place.

Fixes bug 1133572

Change-Id: I39e9fabb28b48dc52ec47f58d76b0bf2c6ee0204
Don't use instance['instance_type'] for scheduler filters in migration

Instead, pull a fresh copy from the database, which has the proper
extra_specs cleanup which everyone downstream of us expects.

Fixes bug 1136430

Change-Id: Ifcb9feff8291096eed7c198b8e7a35c3c38ca93e
Fix deprecated network api

The rpc-only network api is deprecated in Grizzly, but it should still
be able to function.

Change summary:
* Remove update_cache kwyword argument to get_instance_network_info
* Add conductor_api keyword argument to get_instance_network_info
* Add blanket **kwargs for future compat to get_instance_network_info
* Pass instance['uuid'] as instance_id kwarg in _get_instance_nw_info
* Add conductor_api keyword argument to remove_fixed_ip_from_instance
  and add_fixed_ip_to_instance
* Add blanket **kwargs for future compat to remove_fixed_ip_from_instance
  and add_fixed_ip_to_instance
* Pass empty rxtx_factor in remove_fixed_ip_from_instance and
  add_fixed_ip_to_instance for rpcapi compat
* Add blanket **kwargs for future compat to deallocate_for_instance
* Add conductor_api and security_groups to allocate_for_instance
* Add blanket **kwargs for future compat to allocate_for_instance
* Pass instance['uuid'] as instance_id kwarg in allocate_for_instance

Fix Bug 1134512

Change-Id: I6791df1bf1ac2456c5e83ec5ae47b0226409cf95
Regenerate nova.conf.sample

Fix generate_sample.sh script so that we include options from the bin/
scripts too. In order to make this work, sync a zmq-receive fix from
oslo-incubator to delay parsing its config file.

Change-Id: Ia3f9083b2ebc16425dbad8a2a55c2d45f1ff2be3
Tone down logging while waiting for conductor.

When nova-compute starts, it sits in a loop waiting for nova-conductor
to be responsive, since it is required for nova-compute to be able to
complete its startup process.  If a ping request times out, don't leave
a traceback mess in the log.  This is an expected type of failure that
is handled.  Just leave it as a warning in case the problem really is
that nova-conductor hasn't been started.

Fix bug 1137994.

Change-Id: I0754fbd6faf35a09b8733d3ced77871268abf1e8
instance_info_cache_update creates wrongly

If instance_info_cache_update() finds that the entry doesn't exist, it
attempts to create an entry, but it didn't use all of the data. This
patch fixes that.

Also if the info_cache entry was found to be deleted, the old code would
attempt to create a new entry.  This would have failed with
DBDuplicateEntry..  Instead, raise InstanceInfoCacheNotFound, which is
more appropriate. (It doesn't seem we actually ever hit this case.)

Fixes bug 1131594

Change-Id: Id008faf5c4a48ab6a04e75796b7e61dea102acc9
Sync timeutils and memorycache from oslo.

Change-Id: Ifd57eac428c28e527a62cfa46f43c710ba5f46e6
Remove pylint errors for undefined GroupException members

Pylint complains about 'SecurityGroupBase' has no 'raise_group_already_exists'
member as these methods are called from SecurityGroupBase but do not exist in
this class. This is fine as this class is not invoked directly. This patch
adds these prototypes which raise NotImplementedError and changes the
inheritance order as to not raise NotImplementedError.

Fixes bug 1138269

Change-Id: I2d7b723a482dc8ca845e52e2a8f44c6fd31fa5db
Fix IBM copyright strings

Update IBM copyright strings to one consistant format

Change-Id: If4aef948bac27fe337991f36ea1bf02d4078e8bf
validate security_groups on server create

security_groups were getting passed all the way into the database
without ever checking for their existance. This allowed for crudding
up the db with bad data, as well as succeeding when an admin typoed
a security group, and getting no active security group.

Also, security_group is actually an array, which is not implied by
it being a singular variable. Make it a plural variable to cause
less confusion in the future.

And given that we multiply change security_groups from None =>
['default'], get rid of a utility function that serves no purpose
to handle the None or string cases.

To test this correctly, stub out the db call, so we manage to get
all the way down to the bottom of the stack sanely. Also ensure that
the negative case gets us the expected HTTPBadRequest.

Partial fix for nova bugs needed to reenable test_servers_negative
in Tempest. Fixes bug #1046870

Change-Id: I5b477f7bc1992abb47e014f137b8010cc76b2113
Merge "Add instance_type_get() to virt api"