L3 Agent restart causes network outage

When a L3 agent controlling multiple qrouter namespaces
restarts, it destroys all qrouter namespaces even if
some of them are still in use.  As a result, network
traffic could be stopped on the VMs that use the
networks associated with these namespaces.

So what is needed is for the L3 agent to preserve those
qrouter namespaces a L3 agent instance recognizes and to
destroy those it does not know about.

Closes-Bug: #1175695

Change-Id: Idae77886bd195d773878c3d212ccfd56269216fb
Extending quota support for neutron LBaaS entities

Note: This change is a continuation of abandoned
      change https://review.openstack.org/#/c/58720/
      Previous change was abandoned due to rebase problem.

Extending quota mechanism to support neutron
LBaaS entities. Adding quota for vips, pools, members
and health monitors.

This is one of four changes related to the BP.
This one is for neutron project.
Another one is for python-neutronclient package,
another one for tempest,
and another one for horizon/openstack-dashboard project

See blueprint neutron-quota-extension for another two changes.

Change-Id: I64a1d3647a4eb21833266346377416638667a931
Implements: blueprint neutron-quota-extension
Remove unused RPC calls from n1kv plugin code

Change-Id: I75a70786827a5fd6c321b560f1f2ef02a69aa85d
Closes-Bug: #1275166
Removes an incorrect and unnecessary return

The current return statement creates a new object that is
not used anywhere and does not provide a functionality

Change-Id: Id53f6fbc8cc6fb38419e5616a352279f1a9b917f
Closes-Bug: #1276367
Add and update subnet properties in Cisco N1kv plugin

Add dns nameservers and dhcp info in subnet create.
Update subnet properties.

Change-Id: I70e3f0261f6ba5433bbf08d33e2a34cdbb20fed2
Closes-Bug: #1276876
Don't document non-existing flag '--hide-elapsed'

Closes-Bug: 1277051
Change-Id: I4435645025dabcecf3dc1b7168ce17f10d6df534
Refactor to remove _recycle_ip

Since _recycle_ip is now just a pass-through to _delete_ip_allocation
it can be removed.

Change-Id: Ifba3da902de599f748038a33ef3bd98ff77c22b9
Closes-Bug: #1269501
LBaaS: move agent based driver files into a separate dir

Replace agent based driver files from haproxy driver dir
Same for unit tests files

Finalizes blueprint lbaas-common-agent-driver

Change-Id: Ibff85a9c2e1f1c59d72616836d56a697dce72c59
Sync latest oslo.db code into neutron

Changes that were ported from oslo:

b4f72b2 Don't raise MySQL 2013 'Lost connection' errors
271adfb Format sql in db.sqlalchemy.session docstring
0334cb3 Handle exception messages with six.text_type
eff69ce Drop dependency on log from oslo db code
7a11a04 Automatic retry db.api query if db connection lost
11f2add Clean up docstring in db.sqlalchemy.session
1b5147f Only enable MySQL TRADITIONAL mode if we're running against MySQL
39e1c5c Move db tests base.py to common code
986dafd Fix parsing of UC errors in sqlite 3.7.16+/3.8.2+
9a203e6 Use dialect rather than a particular DB API driver
1779029 Move helper DB functions to db.sqlalchemy.utils
bcf6d5e Small edits on help strings
ae01e9a Transition from migrate to alembic
70ebb19 Fix mocking of utcnow() for model datetime cols
7aa94df Add a db check for CHARSET=utf8
aff0171 Remove "vim: tabstop=4 shiftwidth=4 softtabstop=4" from headers
fa0f36f Fix database connection string is secret
8575d87 Removed copyright from empty files
d08d27f Fix the obsolete exception message
8b2b0b7 Use hacking import_exceptions for gettextutils._
9bc593e Add docstring for exception handlers of session
855644a Removal of _REPOSITORY global variable.
ea6caf9 Remove string.lowercase usage
a33989e Remove eventlet tpool from common db.api
e40903b Database hook enabling traditional mode at MySQL
f2115a0 Replace xrange in for loop with range
c802fa6 SQLAlchemy error patterns improved
1c1f199 Remove unused import
6d0a6c3 Correct invalid docstrings
135dd00 Remove start index 0 in range()
28f8fd5 Make _extra_keys a property of ModelBase
45658e2 Fix violations of H302:import only modules
bb4d7a2 Enables db2 server disconnects to be handled pessimistically
915f8ab db.sqlalchemy.session add [sql].idle_timeout
e6494c2 Use six.iteritems to make dict work on Python2/3
48cfb7b Drop dependency on processutils from oslo db code
4c47d3e Fix locking in migration tests
c2ee282 Incorporating MIT licensed code
c5a1088 Typos fix in db and periodic_task module
fb0e86a Use six.moves.configparser instead of ConfigParser
1dd4971 fix typo in db session docstring
8a01dd8 The ability to run tests at various backend
0fe4e28 Use log.warning() instead of log.warn() in oslo.db
12bcdb7 Remove vim header
4c22556 Use py3kcompat urlutils functions instead of urlparse
ca7a2ab Don't use deprecated module commands
6603e8f Remove sqlalchemy-migrate 0.7.3 patching
274c7e2 Drop dependency on lockutils from oslo db code
97d8cf4 Remove lazy loading of database backend
2251cb5 Do not name variables as builtins
3acd57c Add db2 communication error code when check the db connection
c2dcf6e Add [sql].connection as deprecated opt for db
001729d Modify SQLA session due to dispose of eventlet
c2dcf6e Add [sql].connection as deprecated opt for db
001729d Modify SQLA session due to dispose of eventlet
4de827a Clean up db.sqla.Models.extra_keys interface
347f29e Use functools.wrap() instead of custom implementation
771d843 Move base migration test classes to common code
9721129 exception: remove
56ff3b3 Use single meta when change column type
3f2f70e Helper function to sanitize db url credentials
df3f2ba BaseException.message is deprecated since Python 2.6
c76be5b Add function drop_unique_constraint()
d4d8126 Change sqlalchemy/utils.py mode back to 644
cf41936 Move sqlalchemy migration from Nova
5758360 Raise ValueError if sort_dir is unknown
31c1995 python3: Add python3 compatibility support
3972c3f Migrate sqlalchemy utils from Nova
1a2df89 Enable H302 hacking check
3f503fa Add a monkey-patching util for sqlalchemy-migrate
7ba5f4b Don't use mixture of cfg.Opt() deprecated args

Change-Id: Ifddb91dc2c6b80668a07267f92f13074b974776b
add router_id to response for CRU on fw/vip objs

Closes-Bug: #1231770
Change-Id: I3763d2af855d2ad28ff89b528264ba981e71c08a
Fix ValueError in ip_lib.IpRouteCommand.get_gateway()

As metric is not necessarily the 5th word of the gateway line, the
method should search the string 'metric' in the line and pick the next
word as the metric value.

Change-Id: I2663ddbae82f80b912b364c07f9ab92c5b90b718
Closes-Bug: #1267790
Add additional unit tests for the ML2 plugin

This commit adds extended unit test coverage for the ML2 plugin.py file.

Partial-Bug: 1236127

Change-Id: I2f4465c84810965679a585007cd9005ba8a7da71
Use six.moves.urllib.parse instead of urlparse

To keep Python 3.x compatibility, use six.moves.urllib.parse to
replace urlparse.

Closes-Bug: #1279611
Change-Id: I712035926f449e1bc5ab1fe85927369b826cb9cc
Ensure that session is rolled back on bulk creates

During bulk creates, the session is began explicitely;
ensure that it gets rolled back before re-raising in
order to avoid triggering InvalidRequestError
exceptions when the session is reused.

Partial-bug: 1244757

Change-Id: I797faeec2ca5374620db905599ab6e6f04fdcabd
Avoid processing ports which are not yet ready

This patch changes get_vif_port_set in order to not return
OVS ports for which the ofport is not yet assigned, thus avoiding
a regex match failure in get_vif_port_by_id.
Because of this failure, treat_vif_port is unable to wire
the port.
As get_vif_port_by_id is also used elsewhere in the agent, it has
been enhanced in order to tolerate situations in which ofport might
have not yet been assigned.

The ofport field is added to the list of those monitored by the
SimpleInterfaceMonitor. This will guarantee an event is generated
when the ofport is assigned to a port. Otherwise there is a risk
a port would be never processed if it was not yet ready the first
time is was detected. This change won't trigger any extra processing
on the agent side.

Finally, this patch avoids fetching device details from the plugin
for ports which have disappeared from the OVS bridge. This is a
little optimization which might be beneficial for short lived ports.

Change-Id: Icf7f0c7d6fe5239a358567cc9dc9db8ec11c15be
Partial-Bug: #1253896
remove unused imports

Change-Id: I39c0de99fac423858da8c76f6cfff27b21a65601
Closes-Bug:1279644
Fix request timeout errors during calls to NSX controller

Sometimes two correlated exception traces are observed in
the server log for the Neutron Server backed by NSX:
RequestTimeout (The nsx request has timed out) and
OperationalError (Lock wait timeout exceeded). This is
generally described by Guru Salvatore Orlando as the,
and I quote, the "infamous eventlet-mysql deadlock".

This patch tries to address the issue by adding a
cooperative yield in the nsx client code (it’s a good idea
to call sleep(0) occasionally in any case) and also by
avoiding the unnecessary spawning of another Greenthread
within a call that is already executed in Greenthred
itself.

Closes-bug: #1267101
Related-bug: #1279497

Change-Id: I8e298468fb730f11a66fbd4211121ee7d3e2a548
Remove DEBUG:....nsx_cluster:Attribute is empty or null

They serve no purpose, as if an important attribute is empty
or null and it shouldn't we'll know about it, and that's what
counts.

Change-Id: I7783cb6df76362d1b9b25b23f89a3a5d6df15406
Closes-bug: 1280500
Merge "Avoid processing ports which are not yet ready"
Skip a test for nicira service plugin

test_router_add_interface_subnet_with_port_from_other_tenant
is causing intermittent failures in unit tests because of
issues related with sql session autoflush.

This patch skips this test, since it is already covered
by another test case in the same module. This should prevent
job failures while the relevant bug is addressed.

Change-Id: Iad33dada61d17fad14e569e241f4079465b5f792
Related-Bug: 1280035