Add extraroute_db support for Cisco N1kv Plugin

Adding extraroute_db support for router
scheduling functionality in Cisco N1kv Plugin.

Change-Id: I2b9082fc820114a9b10148be08021e5d4b5e5043
Closes-Bug: #1287437
Remove legacy quantum config path

partially implements bp quantum-clean-up
Change-Id: Ifd876320a82991fb532988b9cdcecefde36ff378
L3 agent fetches the external network id once

Rather than fetching the id of the external network each time that
_process_routers is called, get it once and remember it.  If the agent
is ever requested to connect to a different ext-net then it will fetch
the current ext-net to double check for the unlikely event that the
ext-net has changed.  If it has then it will remember the new ext-net.

This is only applicable in the case where there is only one ext-net
that has not been configured explicitly in the config file.  That was
the only case that would cause an RPC message in the first place.

Change-Id: I40bbbf6233131ea5d40122ef9495fd3cb7dc823a
Closes-Bug: #1269567
Reset the policy after loading extensions

The loading of extensions might impact how rules are interpreted in the
policies; for instance, the external-net extension specifies how to
convert the router:external field of a network (to a boolean). So we
need to make sure that the policy is recreated afterwards.

We also need to fix a unit test that assumes that the policy is loaded.

Change-Id: Ifde13323b1435fc84c6cb24ab6a71de5d67f6b9f
Closes-Bug: #1280738
Fix class name typo in test_db_rpc_base

Change-Id: I23fd11fe4fdd2988463f2cf592d884cba72cdb41
Closes-Bug: #1281921
Remove pyudev dependency

pyudev was only used by the linuxbridge-agent to get the list
of virtual network devices. This can be got from /sys instead.
This patch fixes the problem where testr could not import the
lb-agent module because pyudev was not in requirements.txt.

Change-Id: I0a78c91e97de4413f2ecf6fb56d2ff61b36baa78
Closes-Bug: 1269040
ML2 mechanism driver access to binding details

The following properties are added to the PortContext object passed to
ML2 mechanism drivers for port operations:

* bound_driver - name of current bound driver
* original_bound_driver - name previously bound driver in an update
* original_bound_segment - network segment used in previous binding

Some issues with the existing ML2 port binding unit tests were also
fixed.

The remainder of the fix for bug 1276395, making these binding details
available to mechanism drivers when a port is deleted, will be
addressed as part of the fix for bug 1276391.

Partial-Bug: #1276395
Change-Id: I9ecff4a4e044920ed2dde709c89aeb9bc773220d
Developer documentation

* Turns TESTING into a rst file, that we include in the developer
  documentation, for instructions on how to run the unit tests.
* Link to a Vagrant project that sets up Neutron inside a VM.
* Adds a section for how to debug with Nose
* Add new section for Neutron Internals
* Neutron L2 Agent documentation - currently only OVS
* Make the Security Group API extension an example of how an API
  extension is implemented

Implements bp developer-documentation

Change-Id: I9b452abc9da3b1a41ae65cff727967de0eab12fe
Fix DetachedInstanceError for Agent instance

The stacktrace observed seems to hint to a
situation where a session is not properly
rolled back when duplicate agent entries are
found.

Since it is ok to deal with duplicates, the
addition of agents to the DB must be done
one by one, otherwise we go against the
ATOMICITY property of a transaction of all
or none. It's either that or I am barking at
the wrong tree.

This patch also added UT to cover affected
code.

Closes-bug: 1282421
Related-bug: 1282253

Change-Id: Ic5f77808cc10f885d67f5a9cf3b3836283692850
ovs-agent: use hexadecimal IP address in tunnel port name

The remote IP address is used to form the tunnel port name when
the ovs-agent is used with the ML2 plugin. With some OVS/Linux
combinations there is a 15 character limit on port names, so a
name like 'gre-192.168.10.10' does not work. This fix uses the
shorter hexadecimal representation of the IP address instead.

Change-Id: I58caf54219130159525b5a6492eaa451062c2ff5
Closes-Bug: 1281098
N1kv: Fixes fields argument not None

N1kv plugin includes 'fields' in call to get_<resource> function in
superclass. This can trigger errors if other fields are used in the
subsequent processing in the N1kv plugin's own get_<resource>
function.

Change-Id: I63687a830579a178c6daaa2b8f71bef7eb53d584
Closes-bug: #1285289
Improves Arista's ML2 driver's sync performance

In large scale deployments a full sync between Neutron and EOS can take minutes.
In order to cut that time, this patch batches multimle EOS CLI commands and
sends them to EOS instead of sending each command separately. For example, if a
tenant has 10 networks, instead of making 10 RPC calls to EOS to create those 10
networks, this patch builds a commands to create those 10 networks and makes
just one RPC call to EOS which cuts down sync times significantly. All the _bulk()
methods are added to batch such requests.

Another optimization is to timestamp when the Region data was modified (This
includes any tenant creation, their networks, VMs and ports). The sync gets the
timestamp from EOS and only if the timestamps do not match, the driver performs
a full sync.

Closes-Bug: 1279619
Change-Id: I7d17604a7088d7dbb6e3dbb0afdb8e6759c1f67d
Support advanced NVP IPsec VPN Service

The patch adds NVP advanced IPsec VPN Service support for NVP with VCNS:
    * NVP IPsec VPN is an  advanced Service depending on NVP
      advanced service router
    * NVP IPsec VPN Service will finally call VCNS IPsec VPN bulk
      reconfiguration to map to VPN DB logic

Implements: blueprint nvp-vpnaas-plugin
Change-Id: Ic8a96f5defc2b868c6f18fb4966b04079d3c781a
Remove unused 'as e' in exception blocks

Change-Id: I0bd26e1657a4e0ec40405d03d43aee9e10a13636
Closes-bug: #1287031
Use database session from the context in N1kv plugin

Avoid inconsistencies by using context.session for database
transactions wherever possible in the Cisco N1KV plugin.

Change-Id: Ic0784cbbf50beae6bb3b124c959ae90f3affb604
Closes-Bug: #1285473
Implement Mellanox ML2 MechanismDriver

This commit adds support for currently provided Mellanox Plugin
embedded switch functionality as part of the VPI (Ethernet/InfiniBand)
HCA as an ML2 MechanismDriver.
MechanismDriver adds support for VNIC_DIRECT and VNIC_MACVTAP vnic types.
MechanismDriver provides configurable default vif_type for neutron port created
with default VNIC_NORMAL vnic type till nova api support for vnic_type is available.

Implements blueprint mlnx-ml2-support

Change-Id: I16ad318f095b7af879e1b99dcc7f5f9e92facd2b
Embrane Plugin fails alembic migrations

Alembic migration from revision f44ab9871cd6 to 2eeaf963a447 is failing on
Embrane Plugin because the floatingips table doesn't exist.

The problem happens because the plugin is actually inheriting from OVS's Plugin,
and therefore it doesn't take part of the correct migration path at installation time.

As far as I have investigated, adding the support on ext_gw_mode and l3_support is enough to solve the problem.

Change-Id: Ide740df34d06d87fab50cf6468c5d753cb46f112
Closes-bug: 1287330
NSX: make sync backend run more often

This patch bumps the state_sync_interval from 120 seconds to 10 seconds
so that resource's operation status are synced to the db quicker. This cuts
the amount of time that tempest takes to run by half.

Closes-bug: 1285338
Co-Authored-By: Salvatore Orlando <salv.orlando@gmail.com>
Change-Id: I494a6f95c2321befc3c0bfedc719e18a1826d9d5
NSX: nicira_models should import model_base directly

Change-Id: I49bdf72434384899d12aa6c528f8d31125a3b34e
Closes-bug: #1282354
Merge "Support advanced NVP IPsec VPN Service"