Remove SELECT FOR UPDATE use in ML2 type driver release_segment

SELECT FOR UPDATE expression, which is triggered with the use of the
SQLAlchemy Query object's with_lockmode('update') method, is
detrimental to performance and scalability of the database
performance code in Neutron due to the lock contention it produces.

SELECT FOR UPDATE can be entirely avoided in release_segment methods
with the use of single-shot UPDATE and DELETE expressions, and this
patch clears a number of uses of SELECT FOR UPDATE by consolidating
multiple SQL expressions into one.

Partial-Bug: #1330562
Change-Id: I29ffcafc8d4d73ac1cb50c94df5da85514d47a3f
Big Switch: Lock consistency table for REST calls

Adds a new class to hold an SQL table lock throughout
a REST call to prevent servers in an HA deployment from
using a stale consistency hash value.

Also passes the current context down to the server manager
for every request since the consistency hash will need to
be read from the database instead of from a variable on the
server pool object.

Closes-Bug: #1328331
Change-Id: I5f8402c076d7732742c0ae4d9b9d6833d42a0b7b
Updated neutron.conf to reflect new RPC options

After oslo.messaging port, some option became unavailable to set, some
of them changed description or default values. Make sure we don't leave
anything related to old RPC layer there.

New entries are generated by calling the following from inside
oslo-incubator:

$ python openstack/common/config/generator.py -l oslo.messaging

blueprint oslo-messaging

Change-Id: I1523309cf0cd92a5d5f0b265f49979b3e47c2afa
Pass newly created router to _update_router_gw_info

pass it to _update_router_gw_info as last parameter to save
retrieving it again from the session.

Change-Id: I7fbbd675839c2874578c62dfed390d86e3b4801a
Synced log module and its dependencies from olso-incubator

Old version of 'log' module depended on oslo-incubator RPC layer
implementation. This sync is intended to copy the following fix that
makes log module use oslo.messaging.notify.log_handler if available:

  * 109e325e: Use oslo.messaging to publish log errors

blueprint oslo-messaging

Change-Id: Ieb26df4e9fbb2aa4d319c79be7510006fc8461ab
Revert "Revert "ovs-agent: Ensure integration bridge is created""

This reverts commit ccaad827b2bdcdcf4b538c5d5367f14cfc22c08f
that reverted e5cdad90f97d3a54a493eca19e7a3ff643426de1, which
was not the real cause of the problem the revert intended to fix.
Once this patch is back, change-id
I6381e3fee49910127c420dd2e3205c64cdb9e185 should fix the actual bug.

Change-Id: I4116bbb61aaf279d6d63a474134d47d97fe806f7
OVS agent: Correct bridge setup ordering

This patch fixes three issues that combined to break tunnel networks.

The first issue was the the failure mode was being set on the
integration bridge after the canary flow was installed. This
immediately wiped out the canary flow.

The second problem was that the main loop would sync tunnel networks
right before resetting the tunnel bridge when the canary flow was
missing. This meant that it would sync all of the tunnels via RPC and
then wipe them out.

The final issue was that after resetting the tunnel bridge in the
main loop, the tunnel_sync variable was not set to True, so the
tunnels would never be resynchronized.

This patch addresses the three issues in the following ways:
1. Set the failure mode on the bridge before the canary flow is
installed.
2. Run the OVS restart logic before the tunnel synchronization.
3. If the restart logic is triggered, set tunnel_sync to True to
trigger synchronization.

Closes-Bug: #1292105
Change-Id: I6381e3fee49910127c420dd2e3205c64cdb9e185
Remove __init__ method from TunnelCallback mixin

Removed an __init__ method from a mixin class that
made mixing with other classes fragile and inflexible.
This replaces it with an explicit setup method.

This allows the ML2 RPCCallbacks class to correctly
inherit the common RpcCallback class.

Closes-Bug: #1332041
Change-Id: I36cb7dcf57147468f252d61f846b2b28dd77c8ff
validate flat networks physical name

Modified flat driver to validate the physical network
name - we make sure that empty names are deducted.

unit test changes:
- added two tests to make sure that we detect empty
  physnet names
- fix bug in existing unit-tests
-- flat_networks should be a list of string and not string
- use conf to setup flat_network

Change-Id: Ib7cfbd3c97d2e95c82044a2cba683c917ea54a84
Partial-bug: #1325664
Introduce bulk calls for get device details

Allow to get multiple devices details instead of just one
This change introduces a new method in the rpc api.

blueprint bulk-get-device-details

Change-Id: I8497256d7f4f2fb48b5cb792e35aaedf63f129fc
Fixed dhcp & gateway ip conflict in PLUMgrid plugin

   * Adjust dhcp IP if conflicts with gateway IP
   * Added unit test case

Closes-Bug:1333442
Change-Id: Iaa4e63faf28b783e6b5dbc50035ca50ccde9951a
ofagent: Use port desc to monitor ports on br-int

also, use ofport.name as device identifiers.
sprinkle some TODO comments.

Implements: blueprint ofagent-port-monitor
Change-Id: I4995964f37729f954fec71c4a2e61e463a430b1a
LBaaS new object model logging no-op driver

Implements: bp/lbaas-objmodel-driver-changes
Change-Id: I39dccb93c6f9bd3264afbf0acc55a56fc6fd05a1
Merge "Remove __init__ method from TunnelCallback mixin"
Merge "Introduce bulk calls for get device details"
Exit rpc_loop when SIGTERM is recieved in ovs-agent

Previously ovs-agent exited with exit code 1 after SIGTERM was received.
SIGTERM should shutdown agent gracefully and exit code should be 0.

Closes-Bug: #1334264
Change-Id: I23e81cee5ae9e9fdabbe1377420b0902f47be8a7
Use patch ports to interconnect integration/physical bridges

Use by default patch ports to interconnect the integration to
physical bridges for the openvswitch agent in order to increase
performance.

For ovs/kernel without ovs patch port support on kernel side, veths
can be used through the new option OVS.use_veth_interconnection.

Patch port support can be tested using:

    neutron-sanity-check --ovs_patch

DocImpact
Closes-Bug: #1285335
Change-Id: I47106e57bf5b23891d7bf43815959e0d10156fba
Add config for performance gate job

This rally scenario is intended to be run in OpenStack CI
by rally-dsvm-neutron-neutron job.

Implements: blueprint neutron-performance-gate-jobs
Closes-Bug: #1334434
Change-Id: If204f3dfc32933792d09fdc929c248fc3f17b0bc
Fix re-creation of the pool directory

Whenever vip is deleted the complete pool directory
is deleted, but since pool is still alive the get_stats
function recreates the directory. The fix prevents the
re-creation of the directory. Respective unit test case
is also modified

Change-Id: I577e8e3a51db361210aa83a6c7f9b4f61210e924
Closes-Bug: #1317871
Merge "Use patch ports to interconnect integration/physical bridges"