ovs-ctl: Raise the limit on the number of open file descriptors.

Since the removal of dispatcher thread, OVS creates 'n-handler-threads'
file descriptors for each bridge port.  To allow more bridge ports
be supported, this commit raises the limit on the number of open file
descriptors from 7500 to 65535.

Bug #1254038.

Signed-off-by: Alex Wang <alexw@nicira.com>
cmap: New module for cuckoo hash table.

This implements an "optimistic concurrent cuckoo hash", a single-writer,
multiple-reader hash table data structure.  The point of this data
structure is performance, so this commit message focuses on performance.

I tested the performance of cmap with the test-cmap utility included in
this commit.  It takes three parameters for benchmarking:

    - n, the number of elements to insert.

    - n_threads, the number of threads to use for searching and
      mutating the hash table.

    - mutations, the percentage of operations that should modify the
      hash table, from 0% to 100%.

e.g. "test-cmap 1000000 16 1" inserts one million elements, uses 16
threads, and 1% of the operations modify the hash table.

Any given run does the following for both hmap and cmap
implementations:

    - Inserts n elements into a hash table.

    - Iterates over all of the elements.

    - Spawns n_threads threads, each of which searches for each of the
      elements in the hash table, once, and removes the specified
      percentage of them.

    - Removes each of the (remaining) elements and destroys the hash
      table.

and reports the time taken by each step,

The tables below report results for various parameters with a draft version
of this library.  The tests were not formally rerun for the final version,
but the intermediate changes should only have improved performance, and
this seemed to be the case in some informal testing.

n_threads=16 was used each time, on a 16-core x86-64 machine.  The compiler
used was Clang 3.5.  (GCC yields different numbers but similar relative
results.)

The results show:

    - Insertion is generally 3x to 5x faster in an hmap.

    - Iteration is generally about 3x faster in a cmap.

    - Search and mutation is 4x faster with .1% mutations and the
      advantage grows as the fraction of mutations grows.  This is
      because a cmap does not require locking for read operations,
      even in the presence of a writer.

      With no mutations, however, no locking is required in the hmap
      case, and the hmap is somewhat faster.  This is because raw hmap
      search is somewhat simpler and faster than raw cmap search.

    - Destruction is faster, usually by less than 2x, in an hmap.

n=10,000,000:

        .1% mutations    1% mutations    10% mutations     no mutations
          cmap  hmap     cmap   hmap      cmap   hmap       cmap  hmap
insert:   6132  2182     6136   2178      6111   2174       6124  2180
iterate:   370  1203      378   1201       370   1200        371  1202
search:   1375  8692     2393  28197     18402  80379       1281  1097
destroy:  1382  1187     1197   1034       324    241       1405  1205

n=1,000,000:

        .1% mutations    1% mutations    10% mutations     no mutations
          cmap  hmap     cmap   hmap      cmap   hmap       cmap  hmap
insert:    311    25      310     60       311     59        310    60
iterate:    25    62       25     64        25     57         25    60
search:    115   637      197   2266      1803   7284        101    67
destroy:   103    64       90     59        25     13        104    66

n=100,000:

        .1% mutations    1% mutations    10% mutations     no mutations
          cmap  hmap     cmap   hmap      cmap   hmap       cmap  hmap
insert:     25     6       26      5        25      5         25     5
iterate:     1     3        1      3         1      3          2     3
search:     12    57       27    219       164    735         10     5
destroy:     5     3        6      3         2      1          6     4

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
datapath: 16bit inner_network_header field in struct ovs_gso_cb

The motivation for this is to create a 16bit hole in struct ovs_gso_cb
which may be used for the inner_protocol field which is needed
for the proposed implementation of compatibility for MPLS GSO segmentation.

This should be safe as inner_network_header is now an offset to
the inner_mac_header rather than skb->head.

As pointed out by Thomas Graf simply making both inner offsets 16bis is not
safe as there have been cases of overflow with "with collapsed TCP frames
on IB when the headroom grew beyond 64K. See commit 50bceae9bd ``tcp:
Reallocate headroom if it would overflow csum_start'' for additional
details."

This patch is based on suggestions by Thomas Graf and Jesse Gross.

Cc: Thomas Graf <tgraf@suug.ch>
Cc: Jesse Gross <jesse@nicira.com>
Signed-off-by: Simon Horman <horms@verge.net.au>
Signed-off-by: Jesse Gross <jesse@nicira.com>
ofproto: Remove per-flow miss hash table from upcall handler.

The upcall handler keeps a hash table which hashes flow to a list
of corresponding packets.  This used to be necessary as packets with
the same flow had similar actions and calculating actions used to be
a performance bottleneck.  Now that userspace action calculation
performance has improved, there is no need for this hash map.

This patch removes this hash map and each packet has its own upcall.

Signed-off-by: Ryan Wilson <wryan@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
ofproto: Fix comments

Acked-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
netdev-bsd: Fix a whitespace

Acked-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
FAQ: Add an entry about datapath-id

Acked-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
rhel: Remove creation of fake bond interface

fake-iface, option to add-bond, was added for compatibility
reasons and need not be used otherwise.

Signed-off-by: Anoob Soman <anoob.soman@citrix.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
ovs-vsctl: Add an example of how to limit the flows in a flow table.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Gurucharan Shetty <gshetty@nicira.com>
revalidator: Re-fix a flow duplication bug.

Commit 73a3c4757e59 (revalidator: Prevent handling the same flow twice.)
fixed a bug where duplicated flows could be deleted twice. Commit
7d1700980b5d (ofproto-dpif-upcall: Remove the flow_dumper thread.)
partially re-introduced this bug.

The bug would cause the logs to show messages such as
"failed to flow_get (No such file or directory) skb_priority(0),..."
"failed to flow_del (No such file or directory) skb_priority(0),..."

This patch fixes the issue again.

Signed-off-by: Joe Stringer <joestringer@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
tests: Fix an ofproto test.

This was previously logging that there was no such function, but passing
the test anyway.

Signed-off-by: Joe Stringer <joestringer@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
cmap: Fix memory ordering for counter_changed().

Release memory ordering only affects visibility of stores, and is not
allowed on a memory read.  Some compilers enforce this, making this code
fail to compile.

Reported-by: Alex Wang <alexw@nicira.com>
Reported-by: Kmindg G <kmindg@gmail.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
dpif-netdev: Fix use-after-free in port_unref().

When the last rxq is closed (which releases the rxq's internal reference
to its netdev) the next call to netdev_n_rxq() accesses freed memory.

Found by valgrind.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Reported-by: Jarno Rajahalme <jrajahalme@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
dpif-netdev: Fix memory leak.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
dpif-netdev: Use cmap for ports.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
ofproto: Add reference count for Openflow groups.

When adding support for OpenFlow group and bucket stats, a group entry is added
to the xlate_cache. If the main thread removes the group from an ofproto, we
need to guarantee that the group remains accessible to users of
xlate_group_actions and the xlate_cache, as the xlate_cache will not be cleaned
up until the corresponding datapath flows are revalidated.

Before, modify_group could change the bucket list for a group. With group
entries in the xlate_cache, this could leave already-freed bucket pointers in
the cache. Modify_group now recreates the group and replaces the old group in
ofproto's ofgroup hash map. Thus, any subsequent group lookup will find the new
group while the old group and buckets will still exist until the xlate_cache
entries unref the group.

Since ofgroup's properties were only written in-place by modify_group and this
is no longer done, this eliminates the need for ofgroup's lock.

Signed-off-by: Ryan Wilson <wryan@nicira.com>
Signed-off-by: Andy Zhou <azhou@nicira.com>
ofproto: Add support for Openflow group and bucket stats.

Signed-off-by: Ryan Wilson <wryan@nicira.com>
Acked-by: Andy Zhou <azhou@nicira.com>
Signed-off-by: Andy Zhou <azhou@nicira.com>
tests: Add tests for Openflow group stats

Signed-off-by: Andy Zhou <azhou@nicira.com>
Signed-off-by: Ryan Wilson <wryan@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
Signed-off-by: Andy Zhou <azhou@nicira.com>
ofproto-dpif-xlate: Fix a bug.

Commit b256dc525c8 (ofproto-dpif-xlate: Cache xlate_actions() effects.)
caches the variables needed for refreshing mac-learning table in
xlate_normal().  Wherein, the cache entry always records reference to
the original 'ofproto'.

When patch port is used to connect two 'ofproto's, packet goes through the
patch port will have two mac-learning cache entries created for each
'ofproto'.  So, each entry should reference to the corresponding 'ofproto'.
However, due to the bug mentioned above, all cache entries will refer to the
same 'ofproto'.  Subsequently, the mac-learning tables can be corrupted, which
causes connection loss.

This commit fixes the bug by making each cache entry refer to the correct
'ofproto'.

Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Joe Stringer <joestringer@nicira.com>
TODO: Remove "OpenFlow Group Bucket Stats" from TODO list

Signed-off-by: Ryan Wilson <wryan@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>