Delete DHCP port without DHCP server on a net node

A DHCP-network was deleted from one host using neutron
dhcp-agent-network-remove and then added to another host
using neutron dhcp-agent-network-add command. While the
dhcp-agent-network-remove command was in progress, the
host crashed.  As a result, the removal of the DHCP-network
was partially done. The network was disassociated from
the agent in mysql. However, the agent never made the
release_dhcp_port RPC call to delete the port -- even
after the agent restarted. The end result is that there
are two DHCP ports for the same network. One of these
is found on the host that is no longer hosting the
dhcp-server.

This fix make the DHCP agent invoke the release_dhcp_port
RPC call on a stale network whose dnsmasq process is not
running (not active). Before this change, the RPC call is
made on a stale network only when the dnsmasq process is
running.

Closes-Bug: #1244860
Change-Id: Ie0bafdac698810b5455550c306c6a75ddf91d9bb
Restore ability to run functional tests with run_tests.sh

Change f20b1d42 which split the running of functional tests from
the running of unit tests caused running:
  ./run_tests.sh neutron.tests.functional
to fail as the OS_TEST_PATH variable will be defaulted to
neutron.tests.unit which means that the functional tests will not
be discovered in the above case.

This patch trys to detect an argument passed in the form of a
neutron.tests string and correctly sets the OS_TEST_PATH variable
based on it.

Change-Id: I8ccdc7f10d3c8478281aeaf8d02175d0eeb8d6e9
Add physical_network to binding:vif_details dictionary

During port binding binding:vif_details dictionary is
updated with physical_network to be used by mlnx nova vif driver.
binding:profile update will be removed once nova side fix is merged
to keep consistent solution.

Change-Id: If9525929a1546e8d05f7923570d30dfc2eed8898
Closes-Bug: 1304872
Record and log reason for dhcp agent resync

A dhcp resync can be triggered at a number of points, but the actual
resync is done asynchronously by a helper thread.  This means by the
time the resync happens, it's hard to establish what actually caused
it.

There have been a number of problems in production systems that cause
excessive resyncs.  One is a ipv6/dnsmasq issue (rhbz#1077487) and
another is db corruption with duplicate entries [1].  The resync
triggers a whole lot of logs itself, so it becomes very unclear how to
establish any causality.

This change keeps track of what triggered the resync with some helpful
information.  We add a schedule_resync() function to to replace a
explicit set of needs_resync which tracks why it was called.

[1] The logs will contain output like "DBDuplicateEntry
(IntegrityError) (1062, "Duplicate entry
'6d799c6a-7a09-4c1e-bb63-7d30fd052c8a-d3e3ac5b-9962-428a-a9f8-6b2' for
key 'PRIMARY'") ..." in this case

Closes-Bug: #1294445
Change-Id: I9b1c6202f5a6bbad8589a8b64b2a38c9d9edb43b
ofagent: Add a missing push_vlan action

Fix the flow for _provision_local_vlan_inbound_for_tunnel
to push-vlan explicitly.  While the old code happened to
work with older versions of OVS, it was spec-wise incorrect because
it failed to meet the prerequisite of the following set-field.
The latest version of OVS correctly rejects such a flow.

Closes-Bug: #1308927
Change-Id: I66221eec0cb4083d178d7d5651360ee1874e3d1b
Restore GARP by default for floating IPs

Restore the default value of the send_arp_for_ha configuration
parameter to 3. This parameter was set to 0 by default to
prevent kernel crashes due to bug 1273386.

Change-Id: Idbdc6ff687734776ede45127878af6ad9b8b3ca1
Closes-Bug: 1317834
Fix some typos in neutron/db and IBM SDN-VE plugin

For example, in neutron/db/agents_db.py, db_plugin_base_v2 should be changed to db_base_plugin_v2.
There is no bug or blueprint related to this fix.

Change-Id: I0d348d92c9c32f172d1e505cf660aed4d064a440
NSX: Do not extend fault map for network gateway ext

Most of the exceptions defined for this extension already extend
an exception-mapped class. This patch ensures this is true for
every exception and then removes the fault map extension code.

Change-Id: Ifdd9b6d3892cff2d076bdab3a2a1252923e3f85d
Closes-Bug: 1319423
Add 'secret' property for 'connection' option

Neutron cli registers 'connection' option without
'secret=True' property. This commit makes sure
'connection' option has 'secret' property when
it is registered.

Change-Id: I0b0a5aca44a8eeee1c12513f30b9559f1d008442
Closes-Bug: #1319651
NSX: Do a single query for all gateway devices

When creating a network gateway, do a single query for all
gateway devices, rather than a distinct query for each device.

Change-Id: I9e66539eb36ae75d3dcdd8a9073d2a2f69447e98
Closes-Bug: 1320278
Reorg table ml2_port_bindings when db migration

For DB2, there are restrictions on table access
after ALTER TABLE statements containing
REORG-recommended operations.
Table will be placed in "reorg pending" state
after some continuous alter table operations.
Need run reorg table to make the table available.
More info refer to:
http://publib.boulder.ibm.com/infocenter/db2luw/v9/index.jsp
?topic=%2Fcom.ibm.db2.udb.admin.doc%2Fdoc%2Fc0023297.htm

Change-Id: I7fdd61bb4d636d7a3260c0b23152f1f3083cdcfd
Fixes: bug #1318498
Add missing translation support

Update a number of files to add missing translation support.

The patch adds a new hacking check - N320. This ensures that
all log messages, except debug ones, have translations.

A '# noqa' indicates that the validation will not be done on
the specific log message. This should be used in cases where
the translations do not need to be done, for example, the log
message is logging raw data.

Change-Id: I3e1fdd04d87b09dff50950b5e85f5cacfb29afdc
Closes-bug: #1314994
OFAgent: Avoid processing ports which are not yet ready

Port the following patch to OFAgent.
commit: 2702baed390d094b0eac07d0ae167ed236868d00
https://review.openstack.org/#/c/65838/

Closes-Bug: 1307745

Change-Id: Ie1fe0944fb541ba45ffb5b03747d5da289cede51
NSX: allow net-migration only in combined mode

To allow migration of networks from the agent based model
to the agentless one, both modes need to be available; prior
to this change the extension was available in 'agentless' mode
too; this may lead to leftovers during the migration step.
This patch makes the extension available in 'combined' mode
only, which was the intended behavior.

Closes-bug: #1313985

Change-Id: I491ee3eaf25fc30c0ab4092ee198802a6b06d91d
Implement local ARP responder onto OVS agent

With ML2 plugin and l2-pop mechanism driver, it's possible to locally
answer to the ARP request of the VM and avoid ARP broadcasting emulation
on overlay which is costly.

When this functionality is enabled, the OVS flows logic evolves to [1].
This functionality was introduce in 2.1 OVS branch [2].

A README is added to describe l2-pop mechanism driver and the agents
particularities.

[1] https://wiki.openstack.org/wiki/Ovs-flow-logic#OVS_flows_logic_with_local_ARP_responder
[2] http://git.openvswitch.org/cgi-bin/gitweb.cgi?p=openvswitch;a=commitdiff;h=f6c8a6b163af343c66aea54953553d84863835f7

DocImpact: New OVS agent flag 'arp_responder' set to false by default
Closes-Bug: #1237427
Change-Id: Ic28610faf2df6566d8d876fcd7aed333647970e2
Imported Translations from Transifex

Change-Id: I270dd8e25975d51b012976c669a2ea7fb626b22d
Merge "Implement local ARP responder onto OVS agent"
Provide way to reserve dhcp port during failovers

This change provides a way to save the dhcp port when failing
over a network from one dhcp agent to another.  When a
dhcp-agent-network-remove is issued, the dhcp port device_id is
marked as reserved which causes it to not be deleted. When a
subsequent dhcp-agent-network-add is issued, the reserved port
is used and the device_id is corrected.  This is desirable
in order to maintain the dhcp port ip address so that dns doesn't
get impacted. Unit test added.

Change-Id: I531d7ffab074b01adfe186d2c3df43ca978359cd
Closes-Bug: #1288923
Remove hard dependency on novaclient

The nova notification patch introduces a hard dependency on
novaclient when it is a runtime-configurable dependency. The
import from novaclient should be conditional on the
appropriate nova notification options being enabled in the
config.

Change-Id: I2ef4bfa4d53afc7e8c800ad8e2a8737e117af238
Closes-Bug: #1321352
Log iptables rules when they fail to apply

Log the set of rules that causes iptables-restore
to fail in the Linux agent iptables manager.

If a specific rule is identified as the cause, only
that rule and a few surrounding it will be logged to
reduce the output.

Closes-Bug: #1306399
Change-Id: I8e94c1faae75760e439c5abe3d9b723548398105