Record and log reason for dhcp agent resync

A dhcp resync can be triggered at a number of points, but the actual
resync is done asynchronously by a helper thread.  This means by the
time the resync happens, it's hard to establish what actually caused
it.

There have been a number of problems in production systems that cause
excessive resyncs.  One is a ipv6/dnsmasq issue (rhbz#1077487) and
another is db corruption with duplicate entries [1].  The resync
triggers a whole lot of logs itself, so it becomes very unclear how to
establish any causality.

This change keeps track of what triggered the resync with some helpful
information.  We add a schedule_resync() function to to replace a
explicit set of needs_resync which tracks why it was called.

[1] The logs will contain output like "DBDuplicateEntry
(IntegrityError) (1062, "Duplicate entry
'6d799c6a-7a09-4c1e-bb63-7d30fd052c8a-d3e3ac5b-9962-428a-a9f8-6b2' for
key 'PRIMARY'") ..." in this case

Closes-Bug: #1294445
Change-Id: I9b1c6202f5a6bbad8589a8b64b2a38c9d9edb43b
ofagent: Add a missing push_vlan action

Fix the flow for _provision_local_vlan_inbound_for_tunnel
to push-vlan explicitly.  While the old code happened to
work with older versions of OVS, it was spec-wise incorrect because
it failed to meet the prerequisite of the following set-field.
The latest version of OVS correctly rejects such a flow.

Closes-Bug: #1308927
Change-Id: I66221eec0cb4083d178d7d5651360ee1874e3d1b
Big Switch: Don't use MagicMocks unnecessarily

Replace the mocks purely being used to block external
calls with functions that just return and don't record
calls.

Closes-Bug: #1315574
Change-Id: I79533135b5cb456d941ddffa62f5365970146a7e
Add 'secret' property for 'connection' option

Neutron cli registers 'connection' option without
'secret=True' property. This commit makes sure
'connection' option has 'secret' property when
it is registered.

Change-Id: I0b0a5aca44a8eeee1c12513f30b9559f1d008442
Closes-Bug: #1319651
remove token from notifier middleware

notifier middleware is capturing token and sending it to MQ. this
is not advisable so we should filter it out.

Closes-Bug: #1321080
Change-Id: Ia1bfa1bd24989681db1d2f385defc12e69a01f8d
Use list copy for events in nova notifier

There is no need to call range() and len() and .pop() for such a
simple task as list copy.

Change-Id: I4d25e0887ba0a7a5c7f49d195e90a7d2818caac2
Add NVP advanced service check before deleting a router

Since NVP advanced services(LB/FW/VPN) are all routed inserted,
it is necessary to check service in use before deleting a router.
    *If there is one service inserted into the router, it would raise
RouterInUseByXXService error when deleting the router.

Change-Id: I9844f53918cad17cafde0783fe6b69cff246791d
Close-Bug: #1295494
NSX: replace strong references to the plugin with weakref ones

This is done in accordance to a refactoring done in change 92793.

Partial-bug: #1234857

Change-Id: Iaf2bc93182c01e62bb755b4e3f104f5a926b73fc
Make linux.utils.execute log error on return codes

Previously, the execute method in neutron logs everything as debug which hides
a lot of extremely fatal errors like unable to apply security group rules!
This patch changes this code so that we log all non 0 returns as error.

Change-Id: I7328e62269212ccd9c7950ff064a3e337de56918
Closes-bug: 1323832
Related-bug: 1322945
Remove unnecessary MagicMocks in cisco unit tests

Two patches that just return static data never have
assertions made on the MagicMocks generated. This
replaces the magicmocks with lambdas to make the code
easier to read and to get a minor performance gain.

Change-Id: I064c57a66a0f7a8e3080f39cf48bd4b67cf304e3
Partial-Bug: #1316401
Remove function replacement with mock patch

In the hyperv unit tests, an rpc method is
manually replaced with a MagicMock using setattr.
This prevents it from being cleaned up by mock.patch.stopall.
This patch replaces it with a short-lived patch call
in a with statement.

Change-Id: I7b10a8115c474977e3acd16dc49d6f3ae67b0c3b
Partial-Bug: #1316401
Improve readability of MagicMock use in RYU test

Replaces an unnecessarily complex setup of a magicmock
in an RYU test module.

Change-Id: If2e99ff76114d896a6480a884a531f901198caa5
Improve usage of MagicMocks in ML2 and L3 tests

Removes usage of a MagicMock in a case where it's not needed.
Replaces a manual function replacement with a mock.patch call
so it's correctly cleaned up by mock.patch.stopall.

Partial-Bug: #1316401
Change-Id: I8f2f7e9eca2aeaa3b65be1b451e96ed2146950f4
NSX: Fix request_id in api_client to increment

Previously, the NSX request_id in the api_client would always be
0 because the current request id was stored in the class which is
always initialized on each request. This patch fixes that by storing
the request_id as a class variable.

Change-Id: I9ff4d3eb09899fe6d14b462dddf9daddbf77873b
Closes-bug: 1325100
NSX: fix tenant_id passed as security_profile_id

Previously we were passing the tenant_id as the security_profile_id
to NSX so these ids would be switched around in the system backend.
This does not affect any operations as this is just extra metadata
in nsx to help an operator debug.

Change-Id: I1afdcc3621fef8fdd7d4eb3c57bff749a71d0d33
Closes-bug: 1325118
Merge "Make linux.utils.execute log error on return codes"
NSX: fix invalid docstring

Updated the text of docstring.

Change-Id: Ic82fe4719c8b04d01cfb5ebb69411f2f35e7f646
Radware LBaaS driver is able to flip to a secondary backend node

Change-Id: Ifbfef493d5339f61dcf58dddcc8e3830aaf06bf1
Closes-Bug: #1324131
LBaaS add missing rootwrap filter for route

If one runs the lbaas agent from packages and does not have the l3-agent
installed on the same box as the lbaas agent it will fail to add the
default gw route. This is because it's missing the rootwrap filter for
route which is only present in l3.filters.

Change-Id: I59cd0a549a1f1d1564d139d42daf55d02898bf76
Closes-bug: 1325406
Neutron does not follow the RFC 3442 spec for DHCP

When setting a gateway and additional host routes in neutron subnet, the
gateway is only sent to clients via the router dhcp option, dhcp clients
conforming to rfc3442 will ignore router option if
classless-static-routes are available. This patch ensures setting both
the router option and the classless-static-routes including the gateway

Change-Id: Ia00b9385025020f848872309ae42ddac08528f53
Closes-Bug: #1317935