Delete duplicate internal devices in router namespace

When neutron router-interface-delete <router> <subnet> is ran
during L3-agent restart, the agent may fail to delete the old
internal device. After the restart, when the command "neutron
router-interface-add <router> <subnet>" is ran again, the
router ends up having two internal devices configured with the
same IP address.

Closes-Bug: #1244853
Change-Id: I0d7e2db6aa7dae26d0fc3fe1b1527762cb1e3b23
Bugfix and refactoring for ovs_lib flow methods

Remove hardcoded flow parameters from
'_build_flow_expr_str' method, so we can
define any flows we want and can rely on 'ovs-ofctl'
command to verify flow arguments correctness.
When building flow string inside _build_flow_expr_str
use the following approach:
1. Build prefix and remove prefix params from flow_dict.
2. Build postfix (actions) and remove 'actions' from
flow dict.
3. Inside the loop build flow array from everything
what's left in flow_dict.
4. Append postfix (actions) to the flow array.
5. 'Join' flow array into flow string.

Change _build_flow_expr_str() to be a function
instead of an object method because 'self'
parameter  wasn't used.

Remove 'add_or_mod_flow_str' method because
we have to use separate logic when bulding flow
strings for 'add_flow' and 'mod_flow' methods.

Add more unit tests for OVSBridge class.

Closes-Bug: #1255058
Closes-Bug: #1240572

Change-Id: Ic89221d006a626aa2fc40314a9acffc0ea6fd61c
cisco: Do not change supported_extension_aliases directly

cisco network plugin extends supported_extension_aliases in __init__
but supported_exntension_aliases is a class attribute and it is not
reset to the original even after each unit test finished.
To avoid this this patch copies supported_extension_aliases to
an instance attribute and extends it to ensure the class variable
is not changed. This reduces unnecessary logs in unit tests.

Change-Id: I3a7313f5ca2d10b1ae6ea961d9d05611aee055c0
Closes-Bug: #1292742
Fix misleading error message about failed dhcp notifications

This is achieved by adjusting the log traces and ensuring
that the right log errors are emitted based on the status
of the network.

Also, the patch drastically simplifies the structure of the
notification method and unit tests to increase coverage.

Closes-bug: #1289130

Change-Id: I7cc78bba81c516380fc93a68aa7b295312a88e29
Ignore PortNotFound exceptions on lockless delete

Modifies the delete_ports_by_device_id method to
ignore PortNotFound exceptions because it has no
protection against concurrent operations deleting
the same ports that it's trying to delete.

Closes-Bug: #1293657
Change-Id: Icbcded149364a0e231ae811a440a691518bf20ad
LBaaS: small cleanup in agent device driver interface

remove unnecessary 'context' parameter

Change-Id: Iaa3896f89760817af9d72e653252830f8a33e390
Closes-Bug: #1294096
Fixes Hyper-V agent security groups enable issue

Fixes the weight of the applied allow rules by ignoring
the weight of the reject rules.
Fixes the override allow rules issue by fixing the
ACL filtering condition.

Change-Id: I38ddd7142d0fa45f308460153d29580f276ce07e
Closes-Bug: #1294368
Enable to select an RPC handling plugin under Metaplugin

Multiple plugins under metaplugin become 'q-plugin' topic
consumers and a request from an agent is handled by one of
them randomly. Fortunatly most of RPC callbacks are common
for plugins but a problem occurs if an RPC is not supported
by the received plugin.

This is one of risks when using metaplugin. Fundamental fix
of this problem (such as RPC delegation handling of metaplugin)
is difficult since each plugin needs to modify.
But when only one plugin has plugin specific RPCs and other
RPCs are independet of plugins, if the plugin can be selected
for RPC handling, the problem does not happen. Typical use
case of metaplugin such as combination of an agent-based
plugin and a controller-based plugin often applies to this
condition.

This patch adds 'rpc_flavor' configuration parameter to
select an RPC handling plugin. If 'rpc_flavor' is specified,
only the specified plugin becomes 'q-plugin' topic consumer.
If 'rpc_flavor' is not specified, the behavior is same as
previous one.

Change-Id: If133b054bba53829cebe63c1e0ebe6099eb1fd95
Closes-bug: #1267330
DocImpact
Add update binding:profile with physical_network

Add temporary solution in order to support multiple physical networks
by mlnx ML2 MechanismDriver.
Due to non merged patches in nova that should support propagating
physical_network retrieved from port binding:profile attribute
to VIF/Network object.
The code will be removed once relevant nova patches are merged.
The code is disabled by default and should be enabled via
ml2_conf_mlnx.ini config file.

Change-Id: I815f9e28774efd47bccd1c57481e6ba89075792b
Closes-bug: #1291209
BigSwitch ML2: Include bound_segment in port

Includes bound segment in port create/update
requests to disambiguate when the parent network
has multiple segments.

Closes-Bug: #1294355
Change-Id: I59ef38aa5de46c1229384e0c4a9f0ee01c1bc678
Sync excutils from oslo

In order to fix undesired error logs in Neutron (bug 1288188)
fixed save_and_reraise_exception() should be synced from oslo.
Oslo commit: 33a2cee6a690ecfdb2cecfe8f01b0b1dacb5bd17

Also sync gettextutils module as excutils depends on it
Latest commit in oslo: fd33d1eaa039913d8c82b94c511a3eab0c3d5789

Closes-Bug: #1294537
Related-Bug: #1288188
Change-Id: I62ab3e4e22aa000f3a8d1be26ef9e1cfb1714959
Fixes the Hyper-V agent individual ports metrics

Replaces aggregated metric values with separated values for each port.

Co-Authored-By: Adrian Vladu <avladu@cloudbasesolutions.com>
Change-Id: Ie946dff984ef53f014c6c57f8d1d5bb9c6e7596d
Closes-Bug: #1289007
Cisco VPN device driver post-merge cleanup

During review for I-3 there were some minor comments regarding log
level on message and asserts for mock calls.

In addition, some debug logging enhancements were made between the
service and device driver to better indicate the source of the
shared "vpnservice_updated" RPC.

Lastly, unit tests were updated, based on a newer Cisco CSR image,
which had REST API fixes and behavior changes.

Change-Id: I22277462270df0b2cd642ec576e8652c9df146b5
Closes-bug: 1288387
Fix usage of save_and_reraise_exception

Set reraise=False for the cases where the purpose of exception
handler is to reraise proper type of exception

Change-Id: Id6595ba1e160b9033d519ded16e0fbd2f91ccb5d
Closes-Bug: #1288188
add HEAD sentinel file that contains migration revision

This change wraps the command to generate a migration with code to
update a file called HEAD. The HEAD file will contain the revision ID
of the head of the migraton timeline. Additionally, check migrations has
been altered to verify the contents of this file against the timeline
head.

This file helps the OpenStack gate detect potential migration branches
without running alembic via git merge conflicts.

Closes-Bug:1288427
Change-Id: If382c57baea061753d3f4fcd6faec1a31fbfb7ed
tests/unit: clean up notification driver

Some tests doesn't cleanup notification driver. So some tests can
run against unintended notification driver depending on execution order

Change-Id: I7dbed47153be66684827b2693bbc995dddfb4c1b
Related-Bug: #1281481
Partial-Bug: #1284549
Imported Translations from Transifex

Change-Id: I402019fd2f41d73427f2aeb50872b1448e62f31f
ML2: Remove validate_port_binding() and unbind_port()

The API implemented by ML2 mechanism drivers included three methods
related to port binding that were called within DB transactions, but
that could potentially involve communication with controllers or
devices that should not be done within transactions. A subsequent
patch will move the calls to bind_port() outside of tranactions. This
patch eliminates the other two methods from the MechanismDriver API.

The validate_port_binding() method was previously called on the bound
mechanism driver to check whether an existing binding was still valid,
so that the port could be rebound if something changed. But since nova
has no way to handle changes to binding:vif_type or
binding:vif_details after a port is initially plugged, this turned out
not to be useful, so the method has been removed from the
MechanismDriver API. Now, once a port is successfully bound, the
binding remains until the port is deleted or any of it's
binding:host_id, binding:vnic_type, or binding:profile attribute
values are changed.

The unbind_port() method was previously called on the bound mechanism
driver as an existing binding was removed. This method was not used by
any existing mechanism drivers, and was redundant with the
update_port_precommit() and update_port_postcommit() methods that are
called on all mechanism drivers when an existing binding is removed,
so this method has also been removed from the driver API.

Eliminating the unbind_port() call allows the binding details to be
made available via the PortContext in delete_port_postcommit() calls,
completing the resolution of bug 1276395.

Closes-bug: 1276395
Partial-bug: 1276391
Change-Id: I70fb65b478373c4f07f5273baa097fc50e5ba2ef
Get rid of additional db contention on fetching VIP

This fix gets rid of fetching related record from ports table
that could potentially be blocked by concurrent operation
This is to prevent it from updating the
vip table while a port SQL lock is in place, which
vip entries have a reference to.

Partial-Bug: 1283522
Change-Id: Ic64cc268ff4817a31971734a44671c744047231b
Merge "tests/unit: clean up notification driver"