IpsetManager refactoring

Refactor the IpsetManager to move all the low level
knowledge about the ipset behaviour and performance
considerations from the firewall driver to the manager,
reduce redundant function names, and change missleading
variables talking about ipset chains, where they
should say ipset sets.

No logical changes to behaviour, just responsibilities
moved from one class to another.

Unit testing is move from iptables_firewall to ipset_manager
to test the new responsibilities of the class.

Implements: blueprint ipset-manager-refactor

Change-Id: I93000a37a71cd22753b32edbd0a5f3c9cb8b0bde
Fix AttributeError during startup of ovs agent in DVR mode

Make sure the agent starts processing incoming requests only after the entire
initialization is complete. This is done by making explicit when the rpc loop
is supposed to start, i.e. right at the end of the init process.

This fix was necessary because the agent starts processing rpc messages even
though it has not completed the entire initialization of bridges and data
structures; this is usually okay, but in case of DVR, this leads to a
situation where during the the first run, the agent asks the server to be
assigned a MAC address; this in turn leads the server to fanout the generated
MAC to the running agents, the requesting one included; because of the
incomplete setup, the above mentioned error occurs. During subsequent
restarts, the problem no longer appears.

Closes-bug: #1395196

Change-Id: I792697b94fef39971693cf8aff715c270601cecb
Improve performance of get_active_networks_info

RPC 'get_active_networks_info' currently uses nested loop to fill
the network structure with subnet and port info. Speed up this
operation by using itertools.groupby.

Change-Id: If765dd8f2f8010b79eb168179a64dccf940e9cbb
Closes-Bug: #1390356
Avoid unnecessary explicit str() conversion around exceptions

There are number of places like

except Exception as exc:
            LOG.error("Failed to get network: %s", str(exc))

where str() is not needed since %s substitution already does
the same conversion. Also LOG.error could be replaced with
LOG.exception, so argument exc won't be needed at all.

Closes-bug: #1398839

Change-Id: I73cc6e1ce55ade08e7706b99a5ab075f7059a4ef
Advanced services support in neutron-db-manage

Partially-Implements: blueprint services-split

Change-Id: Ib8cd77e5e777be83c44b69edc0a6d2d2ba5d2ab1
Move contrib directory to base test directory

This is needed to have post hook apply to both functional and api jobs.

This is aimed at fixing the output for the dsvm-api job in preparation
of the retargatable testing blueprint.

Change-Id: I39e2a7521a7b545a124b1cd42dc79da8986eff93
Update heal_script for alembic 0.7.1

Alembic 0.7.1 contains checks of foreign keys so
method check_foreign_keys is not needed anymore.
Heal script should be updated to make it possible to use alembic
methods.

Closes-bug: #1398417

Change-Id: I7d0ba7f00567ae5a4dd0669ed86d6e8d554ca4ec
Backward compatibility for advanced services

Patch implements translation from class paths to neutron to class paths
to neutron_<adv_service>. It's achieved by defining entry point in
setup.cfg which is translated by stevedore.

There will be needed patches in advanced services tree calling
get_provider_driver_class() function before importing class.

This patch specifically fixes loading service plugins and
drivers for service plugin. Patches for agents are still needed in
neutron repo and adv services repos.

Alternative and better solution would be implementing new DriverType
to oslo.config, which will have callback to
get_provider_driver_class()-like function.

Co-Authored-By: Ihar Hrachyshka <ihrachys@redhat.com>
Change-Id: I76af175c4387326a4e5ff95c2f15d8b866dedab3
Partial-Bug: 1401895
Merge "Advanced services support in neutron-db-manage"
Merge "Backward compatibility for advanced services"
tests: drop unit tests that only check default configuration values

Those tests do not add much value and only introduce burden when a
default value should be changed.

Change-Id: I32049399b16338996d6bef342c004d5de2446cf1
Merge "Improve performance of get_active_networks_info"
Merge "Fix AttributeError during startup of ovs agent in DVR mode"
Merge "Avoid unnecessary explicit str() conversion around exceptions"
Merge "Update heal_script for alembic 0.7.1"
Merge "Move contrib directory to base test directory"
Move DB TestModelsMigrations from unit to functional

The tests to check that DB migrations and models are in sync depends
on the mysql and postgresql backends being available with a specific
DB user and database created. This violates the principles for unit
tests and therefore these tests should be moved to functional tests.

For these tests to work in the functional job in the gate, the
backends must be installed and the DB user and database created.
We do this via the functional gate hook.

Closes-bug: #1372981

Change-Id: I7b89feef3f19ca07dbfb05acfaa30529a5bf683d
Merge "tests: drop unit tests that only check default configuration values"
Merge "Move DB TestModelsMigrations from unit to functional"
Imported Translations from Transifex

For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I1c8a871becb19341c55d515cb9ed604e2d7ea723