Add extraroute_db support for Cisco N1kv Plugin

Adding extraroute_db support for router
scheduling functionality in Cisco N1kv Plugin.

Change-Id: I2b9082fc820114a9b10148be08021e5d4b5e5043
Closes-Bug: #1287437
Remove legacy quantum config path

partially implements bp quantum-clean-up
Change-Id: Ifd876320a82991fb532988b9cdcecefde36ff378
L3 agent fetches the external network id once

Rather than fetching the id of the external network each time that
_process_routers is called, get it once and remember it.  If the agent
is ever requested to connect to a different ext-net then it will fetch
the current ext-net to double check for the unlikely event that the
ext-net has changed.  If it has then it will remember the new ext-net.

This is only applicable in the case where there is only one ext-net
that has not been configured explicitly in the config file.  That was
the only case that would cause an RPC message in the first place.

Change-Id: I40bbbf6233131ea5d40122ef9495fd3cb7dc823a
Closes-Bug: #1269567
Reset the policy after loading extensions

The loading of extensions might impact how rules are interpreted in the
policies; for instance, the external-net extension specifies how to
convert the router:external field of a network (to a boolean). So we
need to make sure that the policy is recreated afterwards.

We also need to fix a unit test that assumes that the policy is loaded.

Change-Id: Ifde13323b1435fc84c6cb24ab6a71de5d67f6b9f
Closes-Bug: #1280738
Remove pyudev dependency

pyudev was only used by the linuxbridge-agent to get the list
of virtual network devices. This can be got from /sys instead.
This patch fixes the problem where testr could not import the
lb-agent module because pyudev was not in requirements.txt.

Change-Id: I0a78c91e97de4413f2ecf6fb56d2ff61b36baa78
Closes-Bug: 1269040
ML2 mechanism driver access to binding details

The following properties are added to the PortContext object passed to
ML2 mechanism drivers for port operations:

* bound_driver - name of current bound driver
* original_bound_driver - name previously bound driver in an update
* original_bound_segment - network segment used in previous binding

Some issues with the existing ML2 port binding unit tests were also
fixed.

The remainder of the fix for bug 1276395, making these binding details
available to mechanism drivers when a port is deleted, will be
addressed as part of the fix for bug 1276391.

Partial-Bug: #1276395
Change-Id: I9ecff4a4e044920ed2dde709c89aeb9bc773220d
Developer documentation

* Turns TESTING into a rst file, that we include in the developer
  documentation, for instructions on how to run the unit tests.
* Link to a Vagrant project that sets up Neutron inside a VM.
* Adds a section for how to debug with Nose
* Add new section for Neutron Internals
* Neutron L2 Agent documentation - currently only OVS
* Make the Security Group API extension an example of how an API
  extension is implemented

Implements bp developer-documentation

Change-Id: I9b452abc9da3b1a41ae65cff727967de0eab12fe
Fix DetachedInstanceError for Agent instance

The stacktrace observed seems to hint to a
situation where a session is not properly
rolled back when duplicate agent entries are
found.

Since it is ok to deal with duplicates, the
addition of agents to the DB must be done
one by one, otherwise we go against the
ATOMICITY property of a transaction of all
or none. It's either that or I am barking at
the wrong tree.

This patch also added UT to cover affected
code.

Closes-bug: 1282421
Related-bug: 1282253

Change-Id: Ic5f77808cc10f885d67f5a9cf3b3836283692850
ovs-agent: use hexadecimal IP address in tunnel port name

The remote IP address is used to form the tunnel port name when
the ovs-agent is used with the ML2 plugin. With some OVS/Linux
combinations there is a 15 character limit on port names, so a
name like 'gre-192.168.10.10' does not work. This fix uses the
shorter hexadecimal representation of the IP address instead.

Change-Id: I58caf54219130159525b5a6492eaa451062c2ff5
Closes-Bug: 1281098
N1kv: Fixes fields argument not None

N1kv plugin includes 'fields' in call to get_<resource> function in
superclass. This can trigger errors if other fields are used in the
subsequent processing in the N1kv plugin's own get_<resource>
function.

Change-Id: I63687a830579a178c6daaa2b8f71bef7eb53d584
Closes-bug: #1285289
Use database session from the context in N1kv plugin

Avoid inconsistencies by using context.session for database
transactions wherever possible in the Cisco N1KV plugin.

Change-Id: Ic0784cbbf50beae6bb3b124c959ae90f3affb604
Closes-Bug: #1285473
Implement Mellanox ML2 MechanismDriver

This commit adds support for currently provided Mellanox Plugin
embedded switch functionality as part of the VPI (Ethernet/InfiniBand)
HCA as an ML2 MechanismDriver.
MechanismDriver adds support for VNIC_DIRECT and VNIC_MACVTAP vnic types.
MechanismDriver provides configurable default vif_type for neutron port created
with default VNIC_NORMAL vnic type till nova api support for vnic_type is available.

Implements blueprint mlnx-ml2-support

Change-Id: I16ad318f095b7af879e1b99dcc7f5f9e92facd2b
NSX: make sync backend run more often

This patch bumps the state_sync_interval from 120 seconds to 10 seconds
so that resource's operation status are synced to the db quicker. This cuts
the amount of time that tempest takes to run by half.

Closes-bug: 1285338
Co-Authored-By: Salvatore Orlando <salv.orlando@gmail.com>
Change-Id: I494a6f95c2321befc3c0bfedc719e18a1826d9d5
NSX: nicira_models should import model_base directly

Change-Id: I49bdf72434384899d12aa6c528f8d31125a3b34e
Closes-bug: #1282354
Remove unused method update_fixed_ip_lease_expiration

This should have been log removed here:
	Ifcb4f093c92904ceb896438987d53e692eb7fb26

Change-Id: I9395fea531365b82a0cd16f29d6392199e2d9c22
Closes-bug: #1287407
NEC plugin: PFC packet fitler support

It also enhances the following points of packet filter code:
- Allow to clear filter fields to wildcard by specifying None
  in PUT method
- Return None for wildcard fields in an API response
- Determine eth_type based on protocol field on DB layer
- Support OFC driver-specific API validation and update support
- Refactor packet filter code and better validations

Implements blueprint nec-packet-filter-pfc
Change-Id: I2b208c66a9f119ef6d2ef78079d9a7e5965202e7
Adds multiple RPC worker processes to neutron server

blueprint multiple-rpc-workers

Co-Authored-By: Terry Wilson<twilson@redhat.com>
Change-Id: I51f2a52add6c11af905e6f1e6e45d31731ebbb5d
Different class names for VPNaaS migrations

In migrations 52ff27f7567a_support_for_vpnaas.py and
338d7508968c_vpnaas_peer_address_.py different class names are set:
neutron.services.vpn.plugin.VPNDriverPlugin and ne
utron.services.vpn.plugin.VPNPlugin.

Closes-bug: #1285641

Change-Id: Ieda8d3a8a022179867010b6e47d09cf7f43f5da9
Adds Hyper-V Security Groups implementation

Implements the security groups API in the Hyper-V agent.

To enable security groups on the Hyper-V agent, its config file
must contain the following option:

[SECURITYGROUP]
firewall_driver=neutron.plugins.hyperv.agent.security_groups_driver.HyperVSecurityGroupsDriver

Change-Id: I7556001557cd013c10b7f883dbf371afa8d09626
Implements: blueprint hyperv-security-groups
Minor refactoring for Hyper-V utils and tests

A separate commit in the blueprint addressed by this patch
introduces the "_filter_acls" and "_create_acl" methods
which can be used in "enable_port_metrics_collection"
as well to reduce code duplication.

This commit eliminates also some code duplication
in test_hyperv_utilsv2.py.

Implements: blueprint hyperv-security-groups

Change-Id: I48fb5389b6049641ca2649990e81e94e4c45ef7f