Sync latest oslo.db code into neutron

Changes that were ported from oslo:

b4f72b2 Don't raise MySQL 2013 'Lost connection' errors
271adfb Format sql in db.sqlalchemy.session docstring
0334cb3 Handle exception messages with six.text_type
eff69ce Drop dependency on log from oslo db code
7a11a04 Automatic retry db.api query if db connection lost
11f2add Clean up docstring in db.sqlalchemy.session
1b5147f Only enable MySQL TRADITIONAL mode if we're running against MySQL
39e1c5c Move db tests base.py to common code
986dafd Fix parsing of UC errors in sqlite 3.7.16+/3.8.2+
9a203e6 Use dialect rather than a particular DB API driver
1779029 Move helper DB functions to db.sqlalchemy.utils
bcf6d5e Small edits on help strings
ae01e9a Transition from migrate to alembic
70ebb19 Fix mocking of utcnow() for model datetime cols
7aa94df Add a db check for CHARSET=utf8
aff0171 Remove "vim: tabstop=4 shiftwidth=4 softtabstop=4" from headers
fa0f36f Fix database connection string is secret
8575d87 Removed copyright from empty files
d08d27f Fix the obsolete exception message
8b2b0b7 Use hacking import_exceptions for gettextutils._
9bc593e Add docstring for exception handlers of session
855644a Removal of _REPOSITORY global variable.
ea6caf9 Remove string.lowercase usage
a33989e Remove eventlet tpool from common db.api
e40903b Database hook enabling traditional mode at MySQL
f2115a0 Replace xrange in for loop with range
c802fa6 SQLAlchemy error patterns improved
1c1f199 Remove unused import
6d0a6c3 Correct invalid docstrings
135dd00 Remove start index 0 in range()
28f8fd5 Make _extra_keys a property of ModelBase
45658e2 Fix violations of H302:import only modules
bb4d7a2 Enables db2 server disconnects to be handled pessimistically
915f8ab db.sqlalchemy.session add [sql].idle_timeout
e6494c2 Use six.iteritems to make dict work on Python2/3
48cfb7b Drop dependency on processutils from oslo db code
4c47d3e Fix locking in migration tests
c2ee282 Incorporating MIT licensed code
c5a1088 Typos fix in db and periodic_task module
fb0e86a Use six.moves.configparser instead of ConfigParser
1dd4971 fix typo in db session docstring
8a01dd8 The ability to run tests at various backend
0fe4e28 Use log.warning() instead of log.warn() in oslo.db
12bcdb7 Remove vim header
4c22556 Use py3kcompat urlutils functions instead of urlparse
ca7a2ab Don't use deprecated module commands
6603e8f Remove sqlalchemy-migrate 0.7.3 patching
274c7e2 Drop dependency on lockutils from oslo db code
97d8cf4 Remove lazy loading of database backend
2251cb5 Do not name variables as builtins
3acd57c Add db2 communication error code when check the db connection
c2dcf6e Add [sql].connection as deprecated opt for db
001729d Modify SQLA session due to dispose of eventlet
c2dcf6e Add [sql].connection as deprecated opt for db
001729d Modify SQLA session due to dispose of eventlet
4de827a Clean up db.sqla.Models.extra_keys interface
347f29e Use functools.wrap() instead of custom implementation
771d843 Move base migration test classes to common code
9721129 exception: remove
56ff3b3 Use single meta when change column type
3f2f70e Helper function to sanitize db url credentials
df3f2ba BaseException.message is deprecated since Python 2.6
c76be5b Add function drop_unique_constraint()
d4d8126 Change sqlalchemy/utils.py mode back to 644
cf41936 Move sqlalchemy migration from Nova
5758360 Raise ValueError if sort_dir is unknown
31c1995 python3: Add python3 compatibility support
3972c3f Migrate sqlalchemy utils from Nova
1a2df89 Enable H302 hacking check
3f503fa Add a monkey-patching util for sqlalchemy-migrate
7ba5f4b Don't use mixture of cfg.Opt() deprecated args

Change-Id: Ifddb91dc2c6b80668a07267f92f13074b974776b
Use six.moves.urllib.parse instead of urlparse

To keep Python 3.x compatibility, use six.moves.urllib.parse to
replace urlparse.

Closes-Bug: #1279611
Change-Id: I712035926f449e1bc5ab1fe85927369b826cb9cc
Update help message of flag 'enable_isolated_metadata'

Thanks to the commit c73b54e50b62c489f04432bdbc5bee678b18226e,
the way of DHCP agent determines how a subnet is isolated evolves.
But the flag help message wasn't updtated accordingly to this evolution.

Change-Id: If07d8e0802524f9babdee0c8cc07d40cb0803afb
Closes-bug: #1269722
Embrane Tempest Compliance

This changeset tracks the changes needed by the Embrane's Neutron Plugin
in order to consistently  pass tempest tests.

Changes:

- Some db transactions were too long and were causing lock timeout
exception. Removed useless transactions (waiting on non-db tasks to complete)
to fix the problem.

- The operation filter was useless, and breaking the tests. Most of the
logic which guarantees the appliance correct state when an operation in executed
is now in the internal library used for the heleos APIs.
The filter was therefore removed (as well as the corresponding exception).

- Fixed "sync" mode. The behavior was incorrect due to the queue timeout.
Furthermore, parallel requests were not waiting on the correct thread.

- Added missing methods for floating IPs (not all the scenarios were covered).

- Minor bug fixes caught during the tests.

Change-Id: If081b50b4629158016ba421b94612a4cfac82257
Closes-Bug:1269098
Improve unit test coverage for Cisco plugin model code

Closes-Bug: #1190620

This fix improves unit test coverage for:
quantum/plugins/cisco/models/virt_phy_sw_v2.py
Test coverage is improved from about 78% to 99%.

One change included in this fix is removal of some code in
the _invoke_plugin() method in virt_phy_sw_v2.py which looks
like it's attempting to handle the case where the number of
arguments being passed to _invoke_plugin() exceeds the number
of arguments expected for the target plugin method. This
section of code does not get executed for any existing
calls to _invoke_plugin(), and it doesn't appear that
this logic would work (except when the target plugin method
includes a **kwargs expansion).

Change-Id: Ibceb7a462a213f3ba693bcbe94e77d97b2e1440a
Should specify expect_errors=False for success response

In API unit tests, expect_errors=False should be specified
when expecting 2xx API response.

Closes-Bug: #1282401
Change-Id: Ib2d57aab16f73f234b3cb6ecfeecf2779bf17223
Fix test_db_plugin.test_delete_port

Also add PortNotFound check to test_delete_port_public_network.

Change-Id: Iec461364345b200e2d17ba26a6c5362eec6d93ef
Closes-Bug: #1282946
Raise max header size to accommodate large tokens

The max header is exceeded in the following scenario
- Auth tokens built with a keystone v3 API catalog
- A catalog with approximately 8 or more endpoints defined

Change-Id: Ie815e457c0b25ab828e780b5d90233ba0ceff61f
Closes-Bug: #1251026
BigSwitch: Use backend floating IP endpoint

Adds new floating IP REST calls for backend
controllers that support it. Adds backend
capability discovery mechanism.

Implements: blueprint bsn-floating-ip-endpoints
Change-Id: I2301d62a05d256867255865556625603918e84cf
Merge "Update help message of flag 'enable_isolated_metadata'"
Merge "Raise max header size to accommodate large tokens"
Merge "Sync latest oslo.db code into neutron"
Make metaplugin be used with a router service plugin

"l3_plugin_list" configuration parameter of the metaplugin is permitted
blank now.
If "l3_plugin_list" is blank, router extension and extensions which extend
the router extension don't be included in "supported-extension-aliases" of
the metaplugin.
This makes the metaplugin be able to be used with a router service plugin.
Note that if "l3_plugin_list" is not blank, a router service plugin must
not be specified, otherwise the error of the bug report still occurs.

This patch removes some router extension related meaningless codes also.
(e.g.  external-net extension belongs to L2 functionality and be handled
 by core plugins properly.)

Closes-bug: 1266347
DocImpact

Change-Id: I0454bc0a4bd7eda5dad18b0538fb7baebe0b9f91
Merge "Improve unit test coverage for Cisco plugin model code"
Merge "Use six.moves.urllib.parse instead of urlparse"
Rename Neutron core/service plugins for VMware NSX

Short aliases are introduced for the NSX plugin(s). This patch
also switches to using the new plugin names/naming scheme in
the unit tests.

Partial-implements blueprint: nicira-plugin-renaming

Change-Id: If1f1431b098bf77b9c85e0afb3d7848083d9d451
Replace binding:capabilities with binding:vif_details

In addition to binding:vif_type, the neutron core plugin needs to
supply various information to nova's VIF driver, such as VIF security
details and PCI details when SR-IOV is being used. This information is
read-only, requires admin privileges, and is not intended for normal
users. Rather than add separate mechanisms throughout the stack for
each such requirement, the binding:capabilities port attibute, which
is a dictionary and is not currently not used by nova, is renamed to
binding:vif_details to serve as a general-purpose mechanism for
supplying binding-specific details to the VIF driver.

This patch does not remove or replace the CAP_PORT_FILTER boolean
previously used in binding:capabilities. A separate patch should
implement the specific key/value pairs carried by binding:vif_details
to implement VIF security. Another patch will implement the key/value
pairs needed for SR-IOV.

The ML2 plugin now allows the bound mechanism driver to supply the
binding:vif_details dictionary content, instead of just the
CAP_PORT_FILTER boolean previously carried by the binding:capabilities
attribute.

DocImpact: Need to update portbinding extension API, but no impact on
user or administrator documentation.

Implements: blueprint vif-details
Related-Bug: 1112912
Change-Id: I34be746fcfa73c70f72b4f9add8eff3ac88c723f
options: consolidate options definitions

Some config options(interface_driver, use_namespaces) are defined
multiple times in ad-hoc way.  It causes DuplicateOptError exception
when using those module at the same time.  Right now the exception is
avoided in ad-hoc way by each executable.  Those duplicated
definitions should be consolidated and treated in uniformed way.

This is the blocker for blueprint: l3-agent-consolidation

neutron.services.loadbalancer.drivers.haproxy.agent periodic_interval
conflicts with neutron.service one. Since there is no way to fix it
without changing existing behavior/default value, it is untouched for now.

Closes-bug: #1279769
Change-Id: Ifed79b7ee0033644854499416f8a2b22a20416fe
Merge "Make metaplugin be used with a router service plugin"
Merge "Fix test_db_plugin.test_delete_port"