Restore ability to run functional tests with run_tests.sh

Change f20b1d42 which split the running of functional tests from
the running of unit tests caused running:
  ./run_tests.sh neutron.tests.functional
to fail as the OS_TEST_PATH variable will be defaulted to
neutron.tests.unit which means that the functional tests will not
be discovered in the above case.

This patch trys to detect an argument passed in the form of a
neutron.tests string and correctly sets the OS_TEST_PATH variable
based on it.

Change-Id: I8ccdc7f10d3c8478281aeaf8d02175d0eeb8d6e9
Record and log reason for dhcp agent resync

A dhcp resync can be triggered at a number of points, but the actual
resync is done asynchronously by a helper thread.  This means by the
time the resync happens, it's hard to establish what actually caused
it.

There have been a number of problems in production systems that cause
excessive resyncs.  One is a ipv6/dnsmasq issue (rhbz#1077487) and
another is db corruption with duplicate entries [1].  The resync
triggers a whole lot of logs itself, so it becomes very unclear how to
establish any causality.

This change keeps track of what triggered the resync with some helpful
information.  We add a schedule_resync() function to to replace a
explicit set of needs_resync which tracks why it was called.

[1] The logs will contain output like "DBDuplicateEntry
(IntegrityError) (1062, "Duplicate entry
'6d799c6a-7a09-4c1e-bb63-7d30fd052c8a-d3e3ac5b-9962-428a-a9f8-6b2' for
key 'PRIMARY'") ..." in this case

Closes-Bug: #1294445
Change-Id: I9b1c6202f5a6bbad8589a8b64b2a38c9d9edb43b
ofagent: Add a missing push_vlan action

Fix the flow for _provision_local_vlan_inbound_for_tunnel
to push-vlan explicitly.  While the old code happened to
work with older versions of OVS, it was spec-wise incorrect because
it failed to meet the prerequisite of the following set-field.
The latest version of OVS correctly rejects such a flow.

Closes-Bug: #1308927
Change-Id: I66221eec0cb4083d178d7d5651360ee1874e3d1b
Big Switch: Don't use MagicMocks unnecessarily

Replace the mocks purely being used to block external
calls with functions that just return and don't record
calls.

Closes-Bug: #1315574
Change-Id: I79533135b5cb456d941ddffa62f5365970146a7e
Add 'secret' property for 'connection' option

Neutron cli registers 'connection' option without
'secret=True' property. This commit makes sure
'connection' option has 'secret' property when
it is registered.

Change-Id: I0b0a5aca44a8eeee1c12513f30b9559f1d008442
Closes-Bug: #1319651
remove token from notifier middleware

notifier middleware is capturing token and sending it to MQ. this
is not advisable so we should filter it out.

Closes-Bug: #1321080
Change-Id: Ia1bfa1bd24989681db1d2f385defc12e69a01f8d
Use list copy for events in nova notifier

There is no need to call range() and len() and .pop() for such a
simple task as list copy.

Change-Id: I4d25e0887ba0a7a5c7f49d195e90a7d2818caac2
Add NVP advanced service check before deleting a router

Since NVP advanced services(LB/FW/VPN) are all routed inserted,
it is necessary to check service in use before deleting a router.
    *If there is one service inserted into the router, it would raise
RouterInUseByXXService error when deleting the router.

Change-Id: I9844f53918cad17cafde0783fe6b69cff246791d
Close-Bug: #1295494
NSX: replace strong references to the plugin with weakref ones

This is done in accordance to a refactoring done in change 92793.

Partial-bug: #1234857

Change-Id: Iaf2bc93182c01e62bb755b4e3f104f5a926b73fc
l2-pop : removing a TODO for the delete port use case

since this commit : d9bcd597c67900472709cd11604afe1616e0af4e
port info are accessible during the delete_post_commit

Change-Id: I2c560b78c59fc178e668ae73fb994f1ef4b68dc4
Support 'infinite' dhcp_lease_duration

Process a dhcp_lease_duration value of -1 as 'infinite'
when setting the dnsmasq dhcp-range values to support
cases where it is undesirable for instance dhcp leases
to expire.

DocImpact

Closes-Bug: #1315430
Change-Id: I1cc3cfebfec355014e6d5b5cbd656a1300c43c0b
NSX: fix bug for flat provider network

The flat provider network would cause an exception when writing to
database. This is due to the fact that the DB expected an integer
and received an object instead.

Change-Id: Ib9d38711c0c2ef16d8bf74bfae44864a1bc272b1
Closes-bug: #1324120
Fix KeyError exception while updating dhcp port

This is caused by a missing network_id in the port body.
This patch adds it so that a warning message can be traced
correctly. Wording is slightly tweaked to ensure it applies
to the right context.

Closes-bug: #1324194

Change-Id: I4b67b5c3584aa33278eb0e9e879ca338ba0ca8b0
Updated from global requirements

Change-Id: I91778ac6d505f8808a359765555a1619c2f34d3c
Revert "fix openvswitch requirement check"

This reverts commit be340d1bab015c47650687f97393c6c9015fb537.

The fix did not fix anything.
See comments at
https://review.openstack.org/#/c/95592/

Change-Id: I2926acf4b0a7020a7ad3237e6d03d5b78bfcb6c4
Reprogram flows when ovs-vswitchd restarts

When OVS is restarted, by default it will not reprogram flows which were
programmed. For the case of the OVS agent, this means a restart will cause
all traffic to be switched using the NORMAL action. This is undesirable for
a number of reasons, including obvious security reasons.

This change provides a way for the agent to check if a restart of ovs-vswitchd
has happened in the main agent loop. If a restart of ovs-vswitchd is detected,
the agent will run through the setup of the bridges on the host and reprogram
flows for all the ports connected.

DocImpact
This changes adds a new table (table 23) to the integration bridge, with a
single 'drop' flow. This is used to monitor OVS restarts and to reprogram
flows from the agent.

Change-Id: If9e07465c43115838de23e12a4e0087c9218cea2
Closes-Bug: #1290486
Merge "Support 'infinite' dhcp_lease_duration"
Merge "Revert "fix openvswitch requirement check""
FWaaS plugin doesn't need to handle firewall rule del ops

If firewall rule is attached to firewall policy, it would raise
FirewallRuleInUse excpetion in DB ops, else it is a pure DB delete ops.
So it is useless to handle delete_firewall_rule ops in fwaas plugin.
Closes-Bug: #1322076

Change-Id: I55a000d206f232c79b41230f526007f684db8f4f
Make linux.utils.execute log error on return codes

Previously, the execute method in neutron logs everything as debug which hides
a lot of extremely fatal errors like unable to apply security group rules!
This patch changes this code so that we log all non 0 returns as error.

Change-Id: I7328e62269212ccd9c7950ff064a3e337de56918
Closes-bug: 1323832
Related-bug: 1322945