ofagent: Fix an argument mismatch bug in commit 9d13ea88

recently merged commit 9d13ea884bff749b3975acb5eb5630e5aca4a665
had an argument mismatch bug which causes the following crash.

    TypeError: _get_ports() takes exactly 2 arguments (1 given)

this commit fixes it.

Closes-Bug: #1336175
Change-Id: I46bb1fc1aebf06274bb226014e4a9bc5a83f98ac
Security groups extension for PLUMgrid plugin

This patch includes immplementation of security groups
extension in PLUMgrid plugin.

APIs include CRUD operations for security groups
and security group rules.

Added unit tests.

Implements: blueprint plumgrid-plugin-security-groups
Change-Id: I23fea3fe6c3abbe310e883e00706b93863db3bfb
Add -s option for neutron metering rules

While adding iptables rule, cidr is added as destination for both ingress
and egress directions. Modified code to add -s for egress and -d for ingress.

Closes-bug: 1310589

Change-Id: Id9ca10855e6527d4bec689f8f9bcd6f681221954
Freeze models for healing migration

Take a snapshot of all models from the code base at the time when the
healing migration merges. The healing migration needs this frozen view
of the models to be available (even as the models change in the future)
to compare with the current DB schema. The healing migration will use
this comparison to heal the schema.

partially implement bp: db-migration-refactor

Change-Id: I438147c7961b1ecad47f6146cc7fc9396aee7bc5
ML2: Bind ports outside transactions

The ML2 plugin now calls the bind_port() operation on the registered
mechanism drivers outside of any enclosing DB transaction. Ports are
created or updated in one transaction, then a binding is established
if possible, and finally a second transaction commits the binding
result.

With [re]binding moved outside the DB transaction that triggered it,
it is now possible that multiple threads or processes will
concurrently try to bind the same port, or that the port will be
updated between transactions. Concurrent attempts to bind the same
port are allowed to proceed, which results are used is resolved in the
second transaction, and binding is retried if necessary.

Improvements to the Cisco Nexus driver and unit tests from Rich Curran
needed due to the binding changes are also included.

Closes-Bug: 1276391
Closes-Bug: 1335226
Change-Id: I65dafc330d6e812dad0667d2383858504d0ba299
Merge "Add -s option for neutron metering rules"
Database healing migration

Add script that will add all tables for all plugins and make db schema
unconditional.

partially implement bp: db-migration-refactor

Closes-bug: #1277379
Closes-bug: #1304741
Closes-bug: #1298456
Closes-bug: #1298461
Closes-bug: #1239974
Closes-bug: #1336177
Closes-bug: #1337185

Change-Id: Ie49088a74bc5a87466f46989ce14d935e27567d1
L3 agent prefers RPC messages over full sync

When the L3 agent starts up and runs the sync task it doesn't process
any incoming RPC events until the sync task is complete.

This change combines the work from _rpc_loop and _sync_routers_task in
to a single loop called _process_routers_loop.  This loop spawns
threads that pull from a priority queue.  The queue ensures that RPC
messages are handled before _process_routers_loop.  The latter is
generally maintenance tasks triggered by the agent rather than user
triggered tasks.

Synchronization between RPC and sync routers loops is no longer
necessary since they both feed in to a single queue.  There were
places where it was necessary to reorder some things to allow for the
lack of synchronization.  For example, it is necessary to list
namespaces before fetching the full list of routers to ensure that it
doesn't delete a new namespace that gets created after listing
namespaces.  The lack of the need for synchronization between loops is
probably the main strength of this patch.

With multiple worker threads, need to handle the case where an RPC
message came in while a thread was working on a router.  Another
thread should not handle the same router that is already in progress.
Adds a mechanism to signal to the working thread that an update came
in for the router it is working on.  The original thread will repeat
processing the router when it is finished to get the update.
Multiple rapid updates to the same router will be consolidated.
Essentially, there is still synchronization of work for a given router
but not between routers.  Much better than before.

blueprint l3-agent-responsiveness
Closes-Bug: #1289066
Change-Id: I39afe86c66f864d71adf865d7bd1c9db35511505
Merge "ML2: Bind ports outside transactions"
Merge "Freeze models for healing migration"
Merge "L3 agent prefers RPC messages over full sync"
Merge "ofagent: Fix an argument mismatch bug in commit 9d13ea88"
Merge "Database healing migration"
Merge "Security groups extension for PLUMgrid plugin"
Add L3 Extension for Distributed Routers

This patch introduces the model and extension
framework for implementing distributed virtual
routing on top of Open vSwitch.

A new admin-only 'distributed' (as opposed to a
'centralized' or legacy-mode) attribute is added
to the API router resource. It is possible to convert
an existing (centralized) router to a distributed
one; the opposite conversion, even though allowed by
the API, may not be honored by the underlying
plugin implementation and an appropriate error will
be reported.

When creating a router (regardless of the user role),
Neutron will rely on a system wide configuration, whose
default currently allows to create 'centralized' routers.

Tests are added for basic unit coverage; when the first
building blocks for neutron-testing-refactor
are complete, functional testing will be added.
This is because we should be moving away from how
extension tests have been done up until now.

Partially-implements: blueprint neutron-ovs-dvr

DocImpact

Change-Id: I7227fbe2718eba6665a5afb5dcaaaa77b341091f
Authored-by:    Swaminathan Vasudevan <swaminathan.vasudevan@hp.com>
Co-Authored-By: Armando Migliaccio <armamig@gmail.com>
Modify L3 Agent for Distributed Routers

This patch is an enhancement to the existing L3 Agent.
This allows the L3 Agent to support distributed routers
by enhancing the router object to function across
multiple nodes.

Utilized two new types of namespaces:

- FIP to handle multiple VM fips and routers per node
- SNAT to handle centralized SNAT per router

Rules and tables are enhanced and added to support routing
across distributed routers without going to a centralized
router.

Finally, a new configuration param 'agent_mode' is introduced
and it controls what the L3 agent can do: the available values
are: 'legacy', 'dvr', 'dvr_snat' (more details inline).

The l3-scheduler uses the newly introduced agent_mode to
determine what L3 agent to select during the scheduling
process.

Partially-Implements: blueprint neutron-ovs-dvr

DocImpact

Change-Id: Icead821bb74372b15aac2e5cefe8ad7f08c037ab
Co-Authored-By: Rajeev Grover <rajeev.grover@hp.com>
L2 Model additions to support DVR

This patch introduces the models, the DB migrations
and the config options required by the L2 layer to
support DVR east/west traffic.

These changes will be used by the control-plane made
of ML2, L2pop and L2 agent.

Two new configuration options have been introduced:
'dvr_base_mac' is used to set DVR MAC addresses apart
from tenant ones (every distributed router will have
ports being created on compute hosts) and
'enable_distributed_routing' is used to enable dvr
support in the L2 agent. This gives the capability of
rolling out the dvr functionality in stages.

Partially-implements: blueprint neutron-ovs-dvr

DocImpact

Change-Id: Iab6505f239d2c4c9bcbf4e32a292d7b4b5320c8e
Authored-by: Vivekanandan Narasimhan <vivekanandan.narasimhan@hp.com>
Co-Authored-By: Armando Migliaccio <armamig@gmail.com>
RPC additions to support DVR

This patch introduces the RPC contract changes
required for both the server (plugin) and agent
to propagate and retrieve additional information
about Distributed Routers, like MAC addresses
and Port Bindings.

Partially-implements: blueprint neutron-ovs-dvr

Change-Id: I04a2ee5fceea79d2786c799178f8dd1675925a39
Authored-by: Vivekanandan Narasimhan <vivekanandan.narasimhan@hp.com>
Co-Authored-By: Armando Migliaccio <armamig@gmail.com>
ML2 additions to support DVR

This patch introduces the changes necessary to
support DVR at Layer 2 with ML2, and L2pop.

Partially-implements: blueprint neutron-ovs-dvr

Change-Id: I709d877e7df83728868e68e023e6a37bb2e11704
Add L3 Scheduler Changes for Distributed Routers

This patch implements the L3 Scheduler changes for the
Distributed Virtual Routers.

Partially-implements: blueprint neutron-ovs-dvr

Change-Id: I407c3d639ebdf885b1418bceac7cfc251e7eba1f
Co-Authored-By: Carl Baldwin <carl.baldwin@hp.com>
Co-Authored-By: Armando Migliaccio <armamig@gmail.com>