Work toward Python 3.4 support and testing

Change-Id: Ieae7819e0489ae19b7fcfb63497e06527c918b5c
Set vif_details to reflect enable_security_group

While plugging vif, VIFDriver in Nova follows "ovs_hybrid_plug" and
"port_filter" in "binding:vif_detail" which is passed from Neutron, but
those are always true.  This patch make ML2 OVS mech driver set those
param depends on enable_security_group flag.  It enables users to avoid
ovs_hybrid plugging.

This patch also fixes the same issue in the following plugins/drivers:
  * NEC Plugin
  * BigSwitch Plugin
  * Ryu Plugin
  * ML2 Plugin - OFAgent Mech Driver

Closes-Bug: #1336624
Change-Id: I2b7fb526a6f1b730ad65289307b24fd28b996e1b
Create DHCP port for IPv6 subnet

There is a bug in dhcp agent code that when first port is created
in an IPv6 subnet, DHCP port is not automatically created.

This fix resolves this problem by removing the IP version check
in configure_dhcp_for_network method.

Change-Id: If3f405d367a7099d9f33d72d11ffcb7a393abe23
Closes-Bug: #1367500
Check for VPN Objects when deleting interfaces

When we delete Router interfaces/gateway,
we need to check if any VPN services are
associated with that router.

Closes-Bug:1261598

Change-Id: I7df2b8b130b47ec070d0b0a36b1a62df40532760
mock.assert_called_once() is not a valid method

mock.assert_called_once() is a no-op that tests nothing. Instead
mock.assert_called_once_with() should be used (or use
assertEqual(1, mock_obj.call_count) if you don't want to check
parameters).

Borrowed HACKING rule from Davanum Srinivas's nova patch to
prevent it from appearing again.

Change-Id: Idac1d3c89c07e13c9a209663f4e557fcb7547821
Closes-Bug: #1365751
Closes-Bug: #1300265
NSX: Make conn_idle_timeout configurable

If there is a loadbalancer that sits between NSX and neutron usually it has
a default http_timeout on long lived http_connections. This can cause
connections that have not been used from neutron to nsx to be reset after
sometime. This patch makes the amount time we wait to reconnect the connection
configurable so one can set it to match the balancer timeout to avoid having
to try the connection if the loadbalaner has already timed it out.

closes-bug #1382000

Change-Id: Ifca76ca6c24a2002b46b1935fbb4fc63f15c5af1
Cisco N1kv: Fix update network profile for add tenants

This patch makes sure that while updating network profile to add new
tenants, it does not delete the older profile bindings and maintains
them, while adds only the new tenant ids to the bindings.

Change-Id: I862eb1c400e022334a2f6a4078425448bb144843
Closes-bug: #1379609
Updated service.py and its dependencies

This is to avoid service module dependency on importutils that are now
moved to oslo.utils.

The following changes are included:

 * neutron/openstack/common/eventlet_backdoor.py
  5d40e14 Remove code that moved to oslo.i18n
  90ae24b Remove redundant default=None for config options
  fcf517d Update oslo log messages with translation domains

 * neutron/openstack/common/loopingcall.py
  5d40e14 Remove code that moved to oslo.i18n
  e377393 Changes calcuation of variable delay
  ab5d5f1 Use timestamp in loopingcall
  bc48099 Log the function name of looping call
  fb4e863 Remove deprecated LoopingCall
  fcf517d Update oslo log messages with translation domains

 * neutron/openstack/common/service.py
  5d40e14 Remove code that moved to oslo.i18n
  6ede600 rpc, notifier: remove deprecated modules

 * neutron/openstack/common/systemd.py
  17c4e21 Fix docstring indentation in systemd

 * neutron/openstack/common/threadgroup.py
  5a1a016 Make stop_timers() method public
  fdc8883 Add graceful stop function to ThreadGroup.stop
  2d06d6c Simple typo correction
  4d18b57 threadgroup: use threading rather than greenthread

Change-Id: I4887545f861a93223e2c7cbcdd39efe991bff547
Updated cache module and its dependencies

This is to avoid cache module dependency on timeutils that are now moved
to oslo.utils.

The following changes are included:

 * neutron/openstack/common/cache/_backends/memory.py
  6ff6b4b Switch oslo-incubator to use oslo.utils and remove old modules
  2bedce3 Fix MemoryBackend not purging item from _keys_expired on delete

 * neutron/openstack/common/cache/backends.py
  39625e1 Set pbr 'warnerrors' option for doc build

 * neutron/openstack/common/cache/cache.py
  9c683be fix small typo

 * neutron/openstack/common/lockutils.py
  5d40e14 Remove code that moved to oslo.i18n
  7209975 Always log the releasing, even under failure
  bbb266c Clarify logging in lockutils
  942e1aa Use file locks by default again
  ac995be Fix E126 pep8 errors
  15b8352 Remove oslo.log from lockutils

Change-Id: I02cb4b2bc4b7bcba948e67cffdb8bd0219c89a29
Improve performance of security group DB query

The _select_ips_for_remote_group method was joining the
IP allocation, port, allowed address pair, and security group tables
together in a single query. Additionally, it was loading all of
the port columns and using none of them. This resulted in a
very expensive query with no benefit.

This patch eliminates the unnecessary use of the port table by joining
the IP allocation table directly to the security groups and allowed
address pairs tables. In local testing of the method, this sped it up
by an order of magnitude.

Closes-Bug: #1373851
Change-Id: I12899413004838d2d22b691f1e2f3b18f7ec2c27
Remove redundant initialization and check from DVR RPC mixin

The initialization, as well as the recurrent checks on whether the agent
is able to handle DVR requests, are not necessary; this is because the
dvr_agent is going to be set at the time the L2 agent starts receiving
RPC messages.

Change-Id: I9caad965d0ff507e2ed93d6432997ad86139ac04
Remove superfluous except/re-raise

This exception code catches all exceptions, and then always raises them
again.

More interestingly, it uses excutils.save_and_reraise_exception
incorrectly (outside a `with` statement), which triggers the pylint test
designed to catch exactly this.

Change-Id: Iab9d69944cbae5257682ac65ff2b8fba3ef7017e
Modify docstring on send_delete_port_request in N1kv plugin

N1kv plugin: Modify docstring on _send_delete_port_request method

Change-Id: I64d34878ffd8f6db703e4c1d9849032fef9bae96
Closes-Bug: #1381277
Moves the HA resource creations outside of transaction

Currently the HA resources are created in the
_create_router_db which includes calls to
the plugin and generates RPC calls. Even if the
resource creations are outside of any transaction
from the _create_router_db point of view, this
method is  called in a transaction from the
create_router method.
This patch moves the resource creations to the
create_router method outside the transaction.
The failures are handled as previously with
a try/expect.

Change-Id: If8fcfd012f8e992175e49bbefb2ae667881a620a
Closes-bug: #1374461
Ensure ofagent unit tests handles random hashseeds

Several tests in test_ofa_neutron_agent.py and test_ofa_flows.py fail when
running the unit tests with hashseed 2701526934.

This patch fixes the failing unit tests by specifying that assert_has_calls does
not need to assert any order.

Change-Id: I48d6601130f7de1f6eeb7333a17eddd82e7acbdf
Partial-Bug: 1348818
Ensure test_agent_manager handles random hashseeds

Several tests in test_agent_manager.py fail when tox is run using the
hashseed 2701526934, this is down to the nature of using dictionaries
and sets in Python causing some function calls and function arguments
to be out of order. This patch fixes this by either specifying that
assert_has_calls does not need to assert the order, just that the calls
were made, or by letting the unit test get affected in the same way as
the code, e.g. add variables to a dict so they are ordered by the python
hashing algorithms in the same way as they would be in the real code.

Change-Id: If83d1f33c187eab45c2a65fd50fd70cce011c9e7
Partial-Bug: 1348818
Fix race condition on processing DVR floating IPs

Fip namespace and agent gateway port can be shared by multiple dvr routers.
This change uses a set as the control variable for these shared resources
and ensures that Test and Set operation on the control variable are
performed atomically so that race conditions do not occur among
multiple threads processing floating IPs.
Limitation: The scope of this change is limited to addressing the race
condition described in the bug report. It may not address other issues
such as pre-existing issue with handling of DVR floatingips on agent
restart.

closes-bug: #1381238

Change-Id: I6dc2b7bad6e8ddbaa86c1f7a1e2028aeacc3afef
l2population_rpc: docstring improvements

Closes-Bug: #1382319
Change-Id: I72284fbcb16ea2d94898adf357f226775a20d02a
tox.ini: Avoid using bash where unnecessary

Switch to sh, which is hopefully more ubiquitously available than bash.

A recent change (commit 085a35d657cf0fa41a402f2af66c4beaa0f60db2)
introduced bash dependency for "tox -e pep8".  It broke my environment,
where bash is not available.  This change aims to restore it.
As far as I undestand, the change in question doesn't actually need
the specific shell dialect.  So switch to sh, which is expected to be
available for any POSIX-like systems, would improve the situation.

Closes-Bug: #1382327
Change-Id: Ie023d2265d9949a3e3721f03e865e4acfb460de2
Merge "Updated service.py and its dependencies"