Remove default dictionary from function def

Removes an unnecessary dictionary created in the default
parameters definition for the execute function in ip_lib.

Partial-Bug: #1327473
Change-Id: I9b60e87d1043aa67a14675dad2185395df940301
Ignore top-level hidden dirs/files by default

Rather than waiting for each contributor to attempt to submit
a change to .gitignore trying to add the $editor_of_the_week
hidden directory just to be rejected, let's just ignore hidden
files/dirs by default in the top-level neutron directory.

Change-Id: I145bec958a3db7423a4e820a310b138ed189d2cc
Refactor l2_pop code to pass mac/ip info more readably

Previous code used a 2 element array to represent a mac/ip address
pair.  Code assumed that element 0 was mac and 1 was ip.  This made
the code difficult to read and difficult to maintain.  An attempt was
made to insert a third value that failed miserably because of the
position dependence and other code that assumed not only positions but
also the number of elements.

Using a namedtuple seems to be the best way to get better
maintainability.  Named tuples can be compared with regular tuples.
The json serializer still uses an array to represent it so the on-wire
representation of the object has not changed.  A short snip of code
was required to restore the namedtuple from the RPC message.

Change-Id: I7f8c93b0e12ee0179bb23dfbb3a3d814615b1c2e
Closes-Bug: #1352801
Validate local_ip for OVS agent is actual ip address

tunnel_sync() writes this value into ovsdb and value is not validated. In
case non-ip value is written into ovsdb row must be deleted manually.

Change-Id: I57eee81e6821eb2cb26b3d07a39bc94b81243570
Closes-bug: #1284684
Big Switch: Send notification after port update

The Big Switch plugin was not sending port update
notifications for security group changes after the
port was updated. This patch corrects that behavior
by calling the port_update method on the RPC notifier
when necessary after a port update.

Closes-Bug: #1376527
Change-Id: Ic2fd872725c1da8e2dc394473b7feb407d21268f
Drop and recreate FK if adding new PK to routerl3bindings

Since MySQL 5.6 FK columns cannot be altered. As a workaround we can
drop the FK before ALTER COLUMN and then recreate it.

Change-Id: Icd1b3075cf29a6b0c477a4ddea2e6ebee91baef8
Closes-Bug: 1384555
Remove useless return

This change removes a never executed return after a raise.

Change-Id: I6e6cc1580469f6cc676492c8c27b3e45dd444031
Add rootwrap filters for ofagent

neutron-ofagent-agent currently relies on the fact the rootwrap
filters for neutron-openvswitch-agent covers what it needs.
as they are independent agents and their requirements are
getting more different, introduce a dedicated rootwrap filters
for ofagent.

Closes-Bug: #1392560
Change-Id: Iba205260a238431432caf8d9697268ceeef85eca
linuxbridge-agent: make vxlan unicast check more efficent

A list with the full range of vxlan VNIs (1 to 2**24) was using
over 300MB of memory. This patch uses xrange instead of range.

Closes-Bug: #1393362
Change-Id: I21ccab758e7911712690fd5e732f64361e809264
Prevent an iteration through ports on IPv6 slaac

A recent change[1] allowed subnets to be deleted even if they
had active IPv6 SLAAC allocations on them. The updated check
was inefficient because it would check every port in the subnet
even if the subnet was a SLAAC subnet. This patch just shortcuts
out that check.

1. I281f5a1553248e09174dc49d0a42aef4b5c44bee

Change-Id: I2c35495b3642c644e4758f28ccddcc076139ec3b
NSX: Validate gateway device list against DB

This patch adds a check for validating the devices for a network
gateway are defined in the neutron DB before the request for
creating the network gateway is sent to the NSX backend.

Change-Id: Ic2245f965da332dbaee4ab5a869ce47f06a9e6ce
Partial-Bug: #1319581
BSN: Optimistic locking strategy for consistency

Summary:
  Adds an optimistic locking strategy for the Big Switch
  server manager so multiple Neutron servers wanting to
  communicate with the backend do not receive the consistency
  hash for use simultaneously.

  The bsn-rest-call semaphore is removed because serialization
  is now provided by the new locking scheme.

  A new DB engine is added because the consistency hashes
  need a life-cycle with rollbacks and other DB operations
  than cannot impact or be impacted by database operations
  happening on the regular Neutron objects.

  Unit tests are included for each of the new branches
  introduced.

Problem Statement:
  Requests to the Big Switch controllers must contain the
  consistency hash value received from the previous update.
  Otherwise, an inconsistency error will be triggered which
  will force a synchronization. Essentially, a new backend
  call must be prevented from reading from the consistency
  hash table in the DB until the previous call has updated
  the table with the hash from the server response.

  This can be addressed by a semaphore around the rest_call
  function for the single server use case and by a table lock
  on the consistency table for multiple Neutron servers.
  However, both solutions are inadequate because a single
  Neutron server does not scale and a table lock is not
  supported by common SQL HA deployments (e.g. Galera).

  This issue was previously addressed by deploying servers
  in an active-standby configuration. However, that only
  prevented the problem for HTTP API calls. All Neutron
  servers would respond to RPC messages, some of which would
  result in a port update and possible backend call which
  would trigger a conflict if it happened at the same time
  as a backend call from another server. These unnecessary
  syncs are unsustainable as the topology increases beyond
  ~3k VMs.

  Any solution needs to be back-portable to Icehouse so new
  database tables, new requirements, etc. are all out of the
  question.

Solution:
  This patch stores the lock for the consistency hash as a part
  of the DB record. The guaruntees the database offers around
  atomic insertion and constrained atomic updates offer the
  primitives necessary to ensure that only one process/thread
  can lock the record at once.

  The read_for_update method is modified to not return the hash
  in the database until an identifier is inserted into the
  current record or added as a new record. By using an UPDATE
  query with a WHERE clause restricting to the current state,
  only one of many concurrent callers to the DB will successfully
  update the rows. If a caller sees that it didn't update any
  rows, it will start the process over of trying to get the
  lock.

  If a caller observes that the same ID has the lock for
  more than 60 seconds, it will assume the holder has
  died and will attempt to take the lock. This is also done
  in a concurrency-safe UPDATE call since there may be many
  other callers may attempt to do the same thing. If it
  fails and the lock was taken by someone else, the process
  will start over.

  Some pseudo-code resembling the logic:
    read_current_lock
    if no_record:
      insert_lock
      sleep_and_retry if constraint_violation else return
    if current_is_locked and not timer_exceeded:
      sleep_and_retry
    if update_record_with_lock:
      return
    else:
      sleep_and_retry

Closes-Bug: #1374261
Change-Id: Ifa5a7c9749952bc2785a9bf3fed69ad55bf21acc
Enable default SNAT from networks connected to a router indirectly

Make outgoing packets to an external interface SNATed regardless
of source address of the packets. As a result of deep review,
any problem was not found with this change.

Change-Id: I71a1288633bb6af2951d571540bbb9ec5e1270e2
Closes-bug: #1386041
Drop RpcProxy usage from neutron.agent.rpc.PluginApi

This patch removes the usage of the RpcProxy compatibility class from
the neutron.agent.rpc.PluginApi class.  The equivalent use of
oslo.messaging APIs have been put in place instead.  This simple
conversion had a pretty wide impact on unit tests, as well.

The security groups API was converted in this patch as well.  It was
necessary because the security group class is used as a mixin, so it
must be implemented the same way.  Unfortunately, the way this is used
as a mix-in is not consistent, so for now it's only conditionally
converted.

Finally, some other miscellaneous plugin specific interfaces were
converted as well.  Again, these were methods mixed-in for certain
plugins.

Note that there's one very minor functional difference in this patch.
The previous code set the base version to be '1.1'.  The right pattern
is for this to be set to '1.0'.  This version is the default version
specified by the client, telling the server that it must implement at
least this version to satisfy the request.  The default should be
'1.0' and methods that require higher than that should specify it.
From looking at other parts of the code, '1.0' vs '1.1' is not
actually important, as '1.1' was actually the addition of some
security group methods defined elsewhere.  The correction is more
about establishing the right pattern to follow.

Change-Id: I391c01e79943ef179d815ea602253720925ccce1
Drop RpcProxy usage from FWaaS code

This patch removes the usage of the RpcProxy compatibility class from
the FWaaS code.  The equivalent direct usage of oslo.messaging APIs
are now used instead.

Part of blueprint drop-rpc-compat.

Change-Id: I1bab4dcd376d4e9684fee1edc97fd29a61ba38de
Imported Translations from Transifex

For more information about this automatic import see:
https://wiki.openstack.org/wiki/Translations/Infrastructure

Change-Id: I23309c925c4be60fffc5919c2c9c6cf096022001
Update i18n translation for neutron.common/debug log msg's

Validate that hacking rules apply to directories:
    - neutron/common
    - neutron/hacking
    - neutron/debug
    - neutron/locale

Change-Id: I177f5fa5fb7cfdcf332949933aa4bd755a90bb94
Partial-bug: #1320867
Update i18n translation for neutron.notifiers log msg's

Validate that hacking rules apply to directories:
 - neutron/notifiers
 - neutron/openstack (this is already 'clean')

Change-Id: I37b8c45e20d2f59e4e9f5d80d5dc87f19a7a8b3a
Partial-bug: #1320867
Update i18n translation for neutron.server/scheduler log msg's

Validate that hacking rules apply to directories:
- neutron/scheduler
- neutron/server

Change-Id: I1353f5c8b01f85e4995ee1fb23a066506ed98e3f
Partial-bug: #1320867
Check for default sec-group made case insensitive

Currently creating another security group with name
"default" is not allowed, however we can create
another security group with name "DEFAULT"
(or any other CASE pattern).

When trying to boot a VM without specifying
a security group it should always pick the
"default" security group. However, if another
security group of the name DEFAULT is present,
the VM gets associated with the wrong
security group (i.e DEFAULT and not default).

So, updated the code so that a security group
with name DEFAULT (or any other case pattern)
is not allowed.
Closes-Bug: #1384505

Change-Id: I3bc4e48cf723bcf74e098d7ef66df0333f6a8686