Record and log reason for dhcp agent resync

A dhcp resync can be triggered at a number of points, but the actual
resync is done asynchronously by a helper thread.  This means by the
time the resync happens, it's hard to establish what actually caused
it.

There have been a number of problems in production systems that cause
excessive resyncs.  One is a ipv6/dnsmasq issue (rhbz#1077487) and
another is db corruption with duplicate entries [1].  The resync
triggers a whole lot of logs itself, so it becomes very unclear how to
establish any causality.

This change keeps track of what triggered the resync with some helpful
information.  We add a schedule_resync() function to to replace a
explicit set of needs_resync which tracks why it was called.

[1] The logs will contain output like "DBDuplicateEntry
(IntegrityError) (1062, "Duplicate entry
'6d799c6a-7a09-4c1e-bb63-7d30fd052c8a-d3e3ac5b-9962-428a-a9f8-6b2' for
key 'PRIMARY'") ..." in this case

Closes-Bug: #1294445
Change-Id: I9b1c6202f5a6bbad8589a8b64b2a38c9d9edb43b
Big Switch: Don't use MagicMocks unnecessarily

Replace the mocks purely being used to block external
calls with functions that just return and don't record
calls.

Closes-Bug: #1315574
Change-Id: I79533135b5cb456d941ddffa62f5365970146a7e
remove token from notifier middleware

notifier middleware is capturing token and sending it to MQ. this
is not advisable so we should filter it out.

Closes-Bug: #1321080
Change-Id: Ia1bfa1bd24989681db1d2f385defc12e69a01f8d
Use list copy for events in nova notifier

There is no need to call range() and len() and .pop() for such a
simple task as list copy.

Change-Id: I4d25e0887ba0a7a5c7f49d195e90a7d2818caac2
Add NVP advanced service check before deleting a router

Since NVP advanced services(LB/FW/VPN) are all routed inserted,
it is necessary to check service in use before deleting a router.
    *If there is one service inserted into the router, it would raise
RouterInUseByXXService error when deleting the router.

Change-Id: I9844f53918cad17cafde0783fe6b69cff246791d
Close-Bug: #1295494
Improve readability of MagicMock use in RYU test

Replaces an unnecessarily complex setup of a magicmock
in an RYU test module.

Change-Id: If2e99ff76114d896a6480a884a531f901198caa5
Improve usage of MagicMocks in ML2 and L3 tests

Removes usage of a MagicMock in a case where it's not needed.
Replaces a manual function replacement with a mock.patch call
so it's correctly cleaned up by mock.patch.stopall.

Partial-Bug: #1316401
Change-Id: I8f2f7e9eca2aeaa3b65be1b451e96ed2146950f4
NSX: Fix request_id in api_client to increment

Previously, the NSX request_id in the api_client would always be
0 because the current request id was stored in the class which is
always initialized on each request. This patch fixes that by storing
the request_id as a class variable.

Change-Id: I9ff4d3eb09899fe6d14b462dddf9daddbf77873b
Closes-bug: 1325100
NSX: fix tenant_id passed as security_profile_id

Previously we were passing the tenant_id as the security_profile_id
to NSX so these ids would be switched around in the system backend.
This does not affect any operations as this is just extra metadata
in nsx to help an operator debug.

Change-Id: I1afdcc3621fef8fdd7d4eb3c57bff749a71d0d33
Closes-bug: 1325118
NSX: fix invalid docstring

Updated the text of docstring.

Change-Id: Ic82fe4719c8b04d01cfb5ebb69411f2f35e7f646
Radware LBaaS driver is able to flip to a secondary backend node

Change-Id: Ifbfef493d5339f61dcf58dddcc8e3830aaf06bf1
Closes-Bug: #1324131
Neutron does not follow the RFC 3442 spec for DHCP

When setting a gateway and additional host routes in neutron subnet, the
gateway is only sent to clients via the router dhcp option, dhcp clients
conforming to rfc3442 will ignore router option if
classless-static-routes are available. This patch ensures setting both
the router option and the classless-static-routes including the gateway

Change-Id: Ia00b9385025020f848872309ae42ddac08528f53
Closes-Bug: #1317935
BSN: Set hash header to empty instead of False

Sets the consistency hash header to empty instead
of False since 'False' is handled like a string on
the backend and requires special-casing to detect.

Closes-Bug: #1325771
Change-Id: Iee1651574c01a32e78167a9bbed4e0433abbdec2
Log firewall status on delete in case of status inconsistency

This should help to troubleshoot related gate failres.

Change-Id: I92d4d886cafed5faeb2a6d0b224f4b5dc8522100
Related-Bug: #1314313
NSX: bump http_timeout to 30 seconds

We've seen that sometimes NSX takes longer then 10 seconds to return
in production setups under high bursts. Setting this value to 30 seconds
seems more appropriate and solves the issue at load.

Suggestion by: Jacob Cherkas

Change-Id: If604d5d52d15cc1165e19d20b38774dfecec275f
Closes-bug: 1325055
Improve iptables_manager _modify_rules() method

As the number of ports per default security group increases, the
number of iptables entries on the Compute Node grows.  Because of
this, there is a gradual increase in the time taken to apply chains
and rules.

Currently we are using list comprehensions to find if a new chain or
rule matches an existing one.  Instead, walk through the list in
reverse to find a matching entry.

Added a new method, _find_last_entry(), to return the entry we are
searching for.

Change-Id: I3585479ffa00be556b8b21dc9dbd6b36ad37f4de
Closes-Bug: #1302272
Related-Bug: #1253993
Big Switch: fix capabilities retrieval code

References the raw REST response in the capabilities
parsing code so json.loads doesn't get an object
that may already be decoded and fail.

Closes-Bug: #1326173
Change-Id: Ia3179b7499f35a6ab2e9ce1631ab15ed27d64647
Improve module-level mocks in midonet tests

Remove unnecessary module-level mocks and replace
them with test-specific mocks so they don't persist
through the entire test suite.

Partial-Bug: #1316401
Change-Id: I49039749bb700ec0a88639f4772701ba3b920352
Removes unnecessary Embrane module-level mocks

The embrane unit tests install a magicmock into
the system modules at the module level which
persists across all unit tests and isn't required.
This patch removes the unnecessary and limits the
life of the module mocks where they are necessary.

Partial-Bug: #1316401
Change-Id: Iaa38ee0c6821d46a6d78eefef39006df2c1c47e5
Check DB scheme prior to migration to Ml2

When using migration tool from LB/OVS plugin to Ml2, there is no
guarantee current scheme is supported by migration tool. This patch
checks version stored in DB by alembic and compares whether version
is supported.

Closes-bug: #1307720

Change-Id: I4519a0e5a0f3027675958a68d1f9e0440b177229