Improve DHCP agent performance

Previously when starting the dhcp agent the sync_state() process would
be extremely expensive as it would query quantum server for each network.
In order to improve performance a get_active_networks_info() was added
so this information could be retrieved in one query rather than doing a
query for each active network.

The second part of this patch optimizes the logic to avoid calling
get_dhcp_port(). Previously, this method was called once for each network
which makes a call to get_subnets() and get_ports() unnecessarily as
the dhcp agent can determine itself if it needs to update a port or create a
port for dhcp.

This patch also threads the inital sync process and maintains backwards
compatibility with the previous rpc api.

There was also a trivial change to the nvp_plugin where filters are assumed to
be a dict.

implements blueprint improve-dhcp-agent-performance

Change-Id: I3b631057f595250dad76516faa9b421789f60953
Quiet down a large log file heavy hitter.

This line of code serialized the python structure for the device and throws it
at the log file.  The string is about 90K characters, isn't very useful and we
found that it takes a very large amount of space in the log file causing them
to grow very quickly and fill our log disk.  This change extracts the device
name (e.g.  "tapcdc07af9-8a") and uses that in the log message.

Change-Id: I240eb811445752407583f0fb322ce061acb293b9
Fixes: Bug #1200321
Fix issue with pip installing oslo.config-1.2.0

Fixes bug #1194807

Firstly, we update the oslo.config dep to 1.2.0a3 because of the issue
with namespace packages (bug #1194742).

But the main issue here is that if you currently do:

  $> pip install -r quantum/requirements.txt

then you end up with the oslo.config 1.1.1 code installed. This is
because oslo.config>=1.1.0 gets pulled in as a transitive dep and pip
gets confused. You can reproduce with e.g.

  $> pip install \
       http://.../oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3 \
       python-keystoneclient
  $> pip freeze | grep oslo.config
  oslo.config-1.2.0a3
  $> python -c 'from oslo.config.cfg import DeprecatedOpt'
  Traceback (most recent call last):
    File "<string>", line 1, in <module>
  ImportError: cannot import name DeprecatedOpt

This is because of a bug with pip where it sees oslo.config-1.2.0a3 and
oslo.config as two unrelated things. It should strip the version part of
the egg= fragment before using it as a package name, but it doesn't.

However, we can simply use the -f/--find-links pip option in our
requirements.txt to add the tarball URL to the list of URLs considered
and also add the oslo.config>=1.2.0a3 dependency:

  $> pip install \
       -f http://.../oslo.config-1.2.0a3.tar.gz#egg=oslo.config-1.2.0a3 \
       'oslo.config>=1.2.0a3' \
       python-keystoneclient
  $> pip freeze | grep oslo.config
  oslo.config-1.2.0a3
  $> python -c 'from oslo.config.cfg import DeprecatedOpt'

This is actually exactly the semantics we want and we go to great
lengths in pbr to get these semantics while using a single tarball URL.
The only downside to this --find-links strategy is that we gain an extra
line in our requirements.txt ... but it does work around the pip bug.

Change-Id: I6f3eb5fd2c75615d9a1cae172aed859b36b27d4c
Merge "Quiet down a large log file heavy hitter."
Merge "Fix issue with pip installing oslo.config-1.2.0"
Refactor unit tests for NEC Plugin

blueprint nec-plugin-test-coverage

This commit refactors the base class of unit tests for NEC Plugin to
make writing new tests easier.

Add fake OFC Manager and change unit tests which use OFC Manager mock
to using this fake.  With this fake, we don't need to set return_value
and side_effect to the mock instance of OFC Manager in each test case.
This fake keeps all resources on OFC, and returns existence of
resources properly.  This fake is attached to OFC Manager mock instance
by setting all side_effects.  We can also set it to raise an exception.

Make NEC Plugin configurable in each test case.  Now we can configure
NEC Plugin by setting string in class value "_nec_ini" which will be
written into a temporal file and loaded by NEC plugin.

Move setup for callbacks into NecPluginV2TestCase() so that other unit
tests can use callbacks.

Change-Id: I3a52423bb5813547345677dc49abfea6563dd6cd
Merge "Improve DHCP agent performance"