Delete DHCP port without DHCP server on a net node

A DHCP-network was deleted from one host using neutron
dhcp-agent-network-remove and then added to another host
using neutron dhcp-agent-network-add command. While the
dhcp-agent-network-remove command was in progress, the
host crashed.  As a result, the removal of the DHCP-network
was partially done. The network was disassociated from
the agent in mysql. However, the agent never made the
release_dhcp_port RPC call to delete the port -- even
after the agent restarted. The end result is that there
are two DHCP ports for the same network. One of these
is found on the host that is no longer hosting the
dhcp-server.

This fix make the DHCP agent invoke the release_dhcp_port
RPC call on a stale network whose dnsmasq process is not
running (not active). Before this change, the RPC call is
made on a stale network only when the dnsmasq process is
running.

Closes-Bug: #1244860
Change-Id: Ie0bafdac698810b5455550c306c6a75ddf91d9bb
Restore ability to run functional tests with run_tests.sh

Change f20b1d42 which split the running of functional tests from
the running of unit tests caused running:
  ./run_tests.sh neutron.tests.functional
to fail as the OS_TEST_PATH variable will be defaulted to
neutron.tests.unit which means that the functional tests will not
be discovered in the above case.

This patch trys to detect an argument passed in the form of a
neutron.tests string and correctly sets the OS_TEST_PATH variable
based on it.

Change-Id: I8ccdc7f10d3c8478281aeaf8d02175d0eeb8d6e9
Record and log reason for dhcp agent resync

A dhcp resync can be triggered at a number of points, but the actual
resync is done asynchronously by a helper thread.  This means by the
time the resync happens, it's hard to establish what actually caused
it.

There have been a number of problems in production systems that cause
excessive resyncs.  One is a ipv6/dnsmasq issue (rhbz#1077487) and
another is db corruption with duplicate entries [1].  The resync
triggers a whole lot of logs itself, so it becomes very unclear how to
establish any causality.

This change keeps track of what triggered the resync with some helpful
information.  We add a schedule_resync() function to to replace a
explicit set of needs_resync which tracks why it was called.

[1] The logs will contain output like "DBDuplicateEntry
(IntegrityError) (1062, "Duplicate entry
'6d799c6a-7a09-4c1e-bb63-7d30fd052c8a-d3e3ac5b-9962-428a-a9f8-6b2' for
key 'PRIMARY'") ..." in this case

Closes-Bug: #1294445
Change-Id: I9b1c6202f5a6bbad8589a8b64b2a38c9d9edb43b
ofagent: Add a missing push_vlan action

Fix the flow for _provision_local_vlan_inbound_for_tunnel
to push-vlan explicitly.  While the old code happened to
work with older versions of OVS, it was spec-wise incorrect because
it failed to meet the prerequisite of the following set-field.
The latest version of OVS correctly rejects such a flow.

Closes-Bug: #1308927
Change-Id: I66221eec0cb4083d178d7d5651360ee1874e3d1b
Big Switch: Don't use MagicMocks unnecessarily

Replace the mocks purely being used to block external
calls with functions that just return and don't record
calls.

Closes-Bug: #1315574
Change-Id: I79533135b5cb456d941ddffa62f5365970146a7e
Fix some typos in neutron/db and IBM SDN-VE plugin

For example, in neutron/db/agents_db.py, db_plugin_base_v2 should be changed to db_base_plugin_v2.
There is no bug or blueprint related to this fix.

Change-Id: I0d348d92c9c32f172d1e505cf660aed4d064a440
Add 'secret' property for 'connection' option

Neutron cli registers 'connection' option without
'secret=True' property. This commit makes sure
'connection' option has 'secret' property when
it is registered.

Change-Id: I0b0a5aca44a8eeee1c12513f30b9559f1d008442
Closes-Bug: #1319651
OFAgent: Avoid processing ports which are not yet ready

Port the following patch to OFAgent.
commit: 2702baed390d094b0eac07d0ae167ed236868d00
https://review.openstack.org/#/c/65838/

Closes-Bug: 1307745

Change-Id: Ie1fe0944fb541ba45ffb5b03747d5da289cede51
Log iptables rules when they fail to apply

Log the set of rules that causes iptables-restore
to fail in the Linux agent iptables manager.

If a specific rule is identified as the cause, only
that rule and a few surrounding it will be logged to
reduce the output.

Closes-Bug: #1306399
Change-Id: I8e94c1faae75760e439c5abe3d9b723548398105
remove token from notifier middleware

notifier middleware is capturing token and sending it to MQ. this
is not advisable so we should filter it out.

Closes-Bug: #1321080
Change-Id: Ia1bfa1bd24989681db1d2f385defc12e69a01f8d
Set onlink routes for all subnets on an external network

The addition of the on-link routes gives us some freedom to allocate a
router's IP address from any one of multiple subnets on one external
network.  Different routers can get their IPs from different subnets and
they still have direct on-link connectivity to each other.  For example,
one router with its primary IP from 10.0.0.0/24 and another from
192.168.0.0/24 can communicate directly.  It is important that each
router has on-link routes to *all* of the subnets.

Any router can host floating ips from any of the subnets regardless of
which subnet the primary IP address comes from.

This is an alternative to the "Multiple floating IP pools" section in
the administration guide.  It is a simpler alternative that avoids
having to create multiple external networks.  It is also more flexible
because routers will no longer be restricted to getting floating IPs
from the pool to which they happen to be connected.

DocImpact
Document the procedure for adding subnets to the external network.
Potentially remove the existing procedure for "Multiple floating IP
pools" from the docs.

Change-Id: I2c283f5be0cbb6b5d350cafc1b636c300b796a7b
Closes-Bug: #1312467
OFAgent: Fixing lost vlan ids on interfaces

Port the following patch to OFAgent.
commit: 60cb0911712ad11688b4d09e5c01ac39c49f5aea
https://review.openstack.org/#/c/66375/

Closes-Bug: #1309286

Change-Id: I04f99ef7928cd07959b29eb64b5744df397979b8
Use list copy for events in nova notifier

There is no need to call range() and len() and .pop() for such a
simple task as list copy.

Change-Id: I4d25e0887ba0a7a5c7f49d195e90a7d2818caac2
Fix for multiple misspelled words

In Neutron code, a few misspelled words have been found.
This commit fix them all to this date.

Change-Id: Ic11b8ec0adf83a6675f7295e557f436051aa26cd
Closes-Bug: 1323344
Remove redundant default=None for config options

By default the cfg modules set the default value as None. There
is no reason to set this.

Change-Id: I58e09e01760b573612c2ee169a8dd1784fd36a82
Disallow 'timeout' in health_monitor to be negative

It should limit the value of 'timeout' in 'health_monitor', the 'timeout'
shouldn't to be negative.

Change-Id: I26ae1995c9cc362df7c7dda4c53fb8adda298d13
Closes-bug: #1320062
Add NVP advanced service check before deleting a router

Since NVP advanced services(LB/FW/VPN) are all routed inserted,
it is necessary to check service in use before deleting a router.
    *If there is one service inserted into the router, it would raise
RouterInUseByXXService error when deleting the router.

Change-Id: I9844f53918cad17cafde0783fe6b69cff246791d
Close-Bug: #1295494
Imported Translations from Transifex

Change-Id: I9d14a60d8915ebfec13b6635c43c70458f9b571a
Install SNAT rules for ipv4 only

Change-Id: I37bd770aa0d54a985ac2bec708c571785084e0ec
Closes-Bug: #1309195
Fixes bugs for requests sent to SDN-VE controller

This patchset fixes a couple of bugs wrt processing requests
before sending them to the backend controller and adjusts the
requests based on the controller requirements. It also corrects
typos, add quotas and changes the default for a configuration
parameter.

Change-Id: I4b64c2b49ff4854949afc1e54cba1057f376b058
Closes-Bug: #1311260
DocImpact: The default tenant type is chganged from OF to OVERLAY