Delete DHCP port without DHCP server on a net node

A DHCP-network was deleted from one host using neutron
dhcp-agent-network-remove and then added to another host
using neutron dhcp-agent-network-add command. While the
dhcp-agent-network-remove command was in progress, the
host crashed.  As a result, the removal of the DHCP-network
was partially done. The network was disassociated from
the agent in mysql. However, the agent never made the
release_dhcp_port RPC call to delete the port -- even
after the agent restarted. The end result is that there
are two DHCP ports for the same network. One of these
is found on the host that is no longer hosting the
dhcp-server.

This fix make the DHCP agent invoke the release_dhcp_port
RPC call on a stale network whose dnsmasq process is not
running (not active). Before this change, the RPC call is
made on a stale network only when the dnsmasq process is
running.

Closes-Bug: #1244860
Change-Id: Ie0bafdac698810b5455550c306c6a75ddf91d9bb
Drop service* tables only if they exist

Tables servicedefinitions and servicetypes are not used by neutron but
created using db migration tool. If quantum-server in grizzly was
started without using migration tool then tables mentioned above are not
created. During migration from grizzly to havana alembic attempts to
drop these unused tables. This patch uses "IF EXISTS" clause when
dropping mentioned tables.

Closes-bug: #1273355
Change-Id: I33fcb3e1dc96ce37ec9c00987cb5a3a953ca691d
Restore ability to run functional tests with run_tests.sh

Change f20b1d42 which split the running of functional tests from
the running of unit tests caused running:
  ./run_tests.sh neutron.tests.functional
to fail as the OS_TEST_PATH variable will be defaulted to
neutron.tests.unit which means that the functional tests will not
be discovered in the above case.

This patch trys to detect an argument passed in the form of a
neutron.tests string and correctly sets the OS_TEST_PATH variable
based on it.

Change-Id: I8ccdc7f10d3c8478281aeaf8d02175d0eeb8d6e9
Improve ODL ML2 Exception Handling

Raise a MechanismDriverError when authentication with ODL fails.
Re-raise HTTPError in "sync_resources" for exceptions whose status code is not 404.

Closes-Bug: #1311758
Change-Id: I99d2c4f270007c369024525ba4e6188d232a0b13
Database exception causes UnboundLocalError in linuxbridge-agent

When database exception raises from update_device_down,
the linuxbridge-agent doesn't deal with them in a proper way
that causes accessing not-existed local variables.

A straightforward workaround is to set it as None by default
and verify its value then.

Change-Id: I32a9467876a619a7d0ad53ff3d5d95ff977e54f4
Closes-Bug: #1311971
Add physical_network to binding:vif_details dictionary

During port binding binding:vif_details dictionary is
updated with physical_network to be used by mlnx nova vif driver.
binding:profile update will be removed once nova side fix is merged
to keep consistent solution.

Change-Id: If9525929a1546e8d05f7923570d30dfc2eed8898
Closes-Bug: 1304872
Unit test cases for quota_db.py

This commit adds unit test cases with 100% coverage for the quotas
database implementation in neutron/db/quota_db.py

Closes-Bug: #1312310

Change-Id: Icc08132369e88a2f83fe57f524232e94ec4db702
Allow test_l3_agent unit test to run individually

When the test_l3_agent.TestL3AgentEventHandler test case is run
after test discovery it passes because discovery imports files
that registers all the config options. For the test case to run
individually we need to explicitly register the config options
used in the test.

Change-Id: Ic31503ad7be8601ff843dbdc8edf5856ce8b0e39
Closes-bug: #1313812
Reference new get_engine() method from wsgi.py

wsgi.py referenced get_engine in neutron.openstack.common.db.sqlalchemy.session.
This method was removed from that file in commit 53609f29f3c8fcadc545afb891189253c07b33c3
change-id I0e1d86878d3eb924b01e04dced0f90b4e57757d8. This patch references the
replacement method added to neutron.db.api.

Closes-Bug: #1314850
Change-Id: Ifd4db5f49e2aaebc67eccb8bdcef8eea54983112
Add support to dynamically upload drivers in PLUMgrid plugin

PLUMgrid plugin will be able to dynamycally upload any of its available drivers.
It will simplify any CI testing by changing the configuration file for the plugin
instead of changing the code directly.

Change-Id: I56da881688cfdf8f9a1b655c1080c39ffc0133a5
Closes-bug: #1315137
Performance improvement of router routes operations

This patch fixes inefficiency of updating extra routes.
* remove the code repeated by every routes in validation check.
* remove searching a record to delete per record.

Note: Unit tests are covered by existing ones. So no unit test added.

Change-Id: Ia335ac21c65148063d330e4533a15a73962c43d8
Closes-bug: #1312482
NSX: pass the right argument during metadata setup

The NSX cluster was erroneously passed instead of context.
This patch addressed that and tweaks UT to verify the right
parameter is passed instead.

Change-Id: Ife660b3a23705342043d121c7dcecad43e22abc1
Closes-bug: #1315538
lbaas on a network without gateway

Not only check the gateway_ip in subnet, but also
check the host_routes. If there is a default route in host_routes
then the next hop of the default route is also a gateway_ip.

Closes-Bug: 1300017
DocImpact

Change-Id: Id3ae04bfa6cefb1c030857894f0594828b8e856d
Fix H302 violations

H302 violation is reported by flake8 when importing separated objects from
modules instead of importing the whole module.
e.g.   from package.module import function
       function()
is changed to
       from package import module
       module.function()

Change-Id: Ifbf31b52316d3cade40743752a49ce700f384a21
Closes-Bug: #1291032
Adding tenant-id while creating Radware ADC service

Adding tenant_id as a parameter when creating Radware ADC service

Changing unit tests to test the udated REST call resource
with tenant as a parameter

Change-Id: I7a3e9bfb492e847195031bb72ef15dae8769e91f
Closes-Bug: #1315815
L3 RPC loop could delete a router on concurrent update

routers_updated does not acquire any lock just updates
a set for future rpc loop processing.

The self.updated_routers can be changed by concurrent update
notification. If this change happens at the time around the
self.plugin_rpc.get_routers call, the additional routers
- by mistake - is considered as admin_state_up=false routers, which
 are safe to delete.

Creating a local copy of the updated_routers and preserve
the fresh updated_routers entries for the next _rpc_loop
operations.

Change-Id: Icc7377f9c29e248c3b34562465e859b15ecc2ec3
Closes-Bug: #1315467
Partial-Bug: #1253896
Big Switch: Check source_address attribute exists

Check that the source_address attribute exists on
HTTPSConnection objects before referencing it since
it's not present on python 2.6 deployments.

Closes-Bug: #1316382
Change-Id: Id82e70f124cba73c33737099027b5c61aea713bb
Record and log reason for dhcp agent resync

A dhcp resync can be triggered at a number of points, but the actual
resync is done asynchronously by a helper thread.  This means by the
time the resync happens, it's hard to establish what actually caused
it.

There have been a number of problems in production systems that cause
excessive resyncs.  One is a ipv6/dnsmasq issue (rhbz#1077487) and
another is db corruption with duplicate entries [1].  The resync
triggers a whole lot of logs itself, so it becomes very unclear how to
establish any causality.

This change keeps track of what triggered the resync with some helpful
information.  We add a schedule_resync() function to to replace a
explicit set of needs_resync which tracks why it was called.

[1] The logs will contain output like "DBDuplicateEntry
(IntegrityError) (1062, "Duplicate entry
'6d799c6a-7a09-4c1e-bb63-7d30fd052c8a-d3e3ac5b-9962-428a-a9f8-6b2' for
key 'PRIMARY'") ..." in this case

Closes-Bug: #1294445
Change-Id: I9b1c6202f5a6bbad8589a8b64b2a38c9d9edb43b
Fix importing module in test_netscaler_driver

During H302 was introduced new bug importing package instead of module.
This patch is reverting back change on this file from
https://review.openstack.org/#/c/89628/5

Closes-Bug: #1316486
Change-Id: I7830fba6687655b133073ace75f32004e72ad650
Allow DHCPv6 solicit from VM

DHCPv6 solicit UDP package is currently dropped because
there is no security group rule to allow it like IPv4.

This fix allows UDP DHCPv6 solicit from client port 546 to
server port 547. This fix also drops DHCP reply from VM.

Change-Id: I4941d401576e2c8bad37859d2cba227afde9e764
Closes-Bug: 1316515