Record and log reason for dhcp agent resync

A dhcp resync can be triggered at a number of points, but the actual
resync is done asynchronously by a helper thread.  This means by the
time the resync happens, it's hard to establish what actually caused
it.

There have been a number of problems in production systems that cause
excessive resyncs.  One is a ipv6/dnsmasq issue (rhbz#1077487) and
another is db corruption with duplicate entries [1].  The resync
triggers a whole lot of logs itself, so it becomes very unclear how to
establish any causality.

This change keeps track of what triggered the resync with some helpful
information.  We add a schedule_resync() function to to replace a
explicit set of needs_resync which tracks why it was called.

[1] The logs will contain output like "DBDuplicateEntry
(IntegrityError) (1062, "Duplicate entry
'6d799c6a-7a09-4c1e-bb63-7d30fd052c8a-d3e3ac5b-9962-428a-a9f8-6b2' for
key 'PRIMARY'") ..." in this case

Closes-Bug: #1294445
Change-Id: I9b1c6202f5a6bbad8589a8b64b2a38c9d9edb43b
ofagent: Add a missing push_vlan action

Fix the flow for _provision_local_vlan_inbound_for_tunnel
to push-vlan explicitly.  While the old code happened to
work with older versions of OVS, it was spec-wise incorrect because
it failed to meet the prerequisite of the following set-field.
The latest version of OVS correctly rejects such a flow.

Closes-Bug: #1308927
Change-Id: I66221eec0cb4083d178d7d5651360ee1874e3d1b
Big Switch: Don't use MagicMocks unnecessarily

Replace the mocks purely being used to block external
calls with functions that just return and don't record
calls.

Closes-Bug: #1315574
Change-Id: I79533135b5cb456d941ddffa62f5365970146a7e
Add 'secret' property for 'connection' option

Neutron cli registers 'connection' option without
'secret=True' property. This commit makes sure
'connection' option has 'secret' property when
it is registered.

Change-Id: I0b0a5aca44a8eeee1c12513f30b9559f1d008442
Closes-Bug: #1319651
remove token from notifier middleware

notifier middleware is capturing token and sending it to MQ. this
is not advisable so we should filter it out.

Closes-Bug: #1321080
Change-Id: Ia1bfa1bd24989681db1d2f385defc12e69a01f8d
Use list copy for events in nova notifier

There is no need to call range() and len() and .pop() for such a
simple task as list copy.

Change-Id: I4d25e0887ba0a7a5c7f49d195e90a7d2818caac2
Add NVP advanced service check before deleting a router

Since NVP advanced services(LB/FW/VPN) are all routed inserted,
it is necessary to check service in use before deleting a router.
    *If there is one service inserted into the router, it would raise
RouterInUseByXXService error when deleting the router.

Change-Id: I9844f53918cad17cafde0783fe6b69cff246791d
Close-Bug: #1295494
Improve readability of MagicMock use in RYU test

Replaces an unnecessarily complex setup of a magicmock
in an RYU test module.

Change-Id: If2e99ff76114d896a6480a884a531f901198caa5
Improve usage of MagicMocks in ML2 and L3 tests

Removes usage of a MagicMock in a case where it's not needed.
Replaces a manual function replacement with a mock.patch call
so it's correctly cleaned up by mock.patch.stopall.

Partial-Bug: #1316401
Change-Id: I8f2f7e9eca2aeaa3b65be1b451e96ed2146950f4
NSX: Fix request_id in api_client to increment

Previously, the NSX request_id in the api_client would always be
0 because the current request id was stored in the class which is
always initialized on each request. This patch fixes that by storing
the request_id as a class variable.

Change-Id: I9ff4d3eb09899fe6d14b462dddf9daddbf77873b
Closes-bug: 1325100
NSX: fix tenant_id passed as security_profile_id

Previously we were passing the tenant_id as the security_profile_id
to NSX so these ids would be switched around in the system backend.
This does not affect any operations as this is just extra metadata
in nsx to help an operator debug.

Change-Id: I1afdcc3621fef8fdd7d4eb3c57bff749a71d0d33
Closes-bug: 1325118
NSX: fix invalid docstring

Updated the text of docstring.

Change-Id: Ic82fe4719c8b04d01cfb5ebb69411f2f35e7f646
Radware LBaaS driver is able to flip to a secondary backend node

Change-Id: Ifbfef493d5339f61dcf58dddcc8e3830aaf06bf1
Closes-Bug: #1324131
LBaaS add missing rootwrap filter for route

If one runs the lbaas agent from packages and does not have the l3-agent
installed on the same box as the lbaas agent it will fail to add the
default gw route. This is because it's missing the rootwrap filter for
route which is only present in l3.filters.

Change-Id: I59cd0a549a1f1d1564d139d42daf55d02898bf76
Closes-bug: 1325406
Neutron does not follow the RFC 3442 spec for DHCP

When setting a gateway and additional host routes in neutron subnet, the
gateway is only sent to clients via the router dhcp option, dhcp clients
conforming to rfc3442 will ignore router option if
classless-static-routes are available. This patch ensures setting both
the router option and the classless-static-routes including the gateway

Change-Id: Ia00b9385025020f848872309ae42ddac08528f53
Closes-Bug: #1317935
BSN: Set hash header to empty instead of False

Sets the consistency hash header to empty instead
of False since 'False' is handled like a string on
the backend and requires special-casing to detect.

Closes-Bug: #1325771
Change-Id: Iee1651574c01a32e78167a9bbed4e0433abbdec2
Log firewall status on delete in case of status inconsistency

This should help to troubleshoot related gate failres.

Change-Id: I92d4d886cafed5faeb2a6d0b224f4b5dc8522100
Related-Bug: #1314313
NSX: bump http_timeout to 30 seconds

We've seen that sometimes NSX takes longer then 10 seconds to return
in production setups under high bursts. Setting this value to 30 seconds
seems more appropriate and solves the issue at load.

Suggestion by: Jacob Cherkas

Change-Id: If604d5d52d15cc1165e19d20b38774dfecec275f
Closes-bug: 1325055
Improve iptables_manager _modify_rules() method

As the number of ports per default security group increases, the
number of iptables entries on the Compute Node grows.  Because of
this, there is a gradual increase in the time taken to apply chains
and rules.

Currently we are using list comprehensions to find if a new chain or
rule matches an existing one.  Instead, walk through the list in
reverse to find a matching entry.

Added a new method, _find_last_entry(), to return the entry we are
searching for.

Change-Id: I3585479ffa00be556b8b21dc9dbd6b36ad37f4de
Closes-Bug: #1302272
Related-Bug: #1253993
Merge "LBaaS add missing rootwrap filter for route"