tunnel: Support all combinations of flow-based and specific tunnel matches.

There are 12 possible ways to specify a tunnel (2 * 2 * 3 == 12):

    - Specific in_key or flow-based (2 choices).

    - Specific ip_dst or flow-based (2 choices).

    - Specific ip_src, wildcarded, or flow-based (3 choices).

Until now, only 6 of the 12 possibilities have been supported.  We
have had a couple of requests to add another.  This commit adds all the
possibilities, so that we won't have to add the other 6 one by one.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Requested-by: Thomas Morin <thomas.morin@orange.com>
Acked-by: pritesh <pritesh.kothari@cisco.com>
meta-flow: Fix setting MFF_IP_FRAG.

mf_set_flow_value() was not setting 'flow->nw_frag' properly.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
netdev-dummy: Fix reconnecting.

The netdev-dummy unit test ran into the reconnect condition on Jarno's
machine. With his test environment, we found and fixed the bugs in
handling reconnect.

Co-authored-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
netdev-dummy: Reduce reconnect back off timeout

netdev-dummy will mostly be used for testing and debugging over fairly
reliable connection. Reduce reconnect back off timeout in case the first
connect attempt failed.

Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
ofproto: Lock for vlan splinters only if have them.

Reading the hmap count for determining if it is empty or not is thread
safe, so avoid locking when not necessary.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
ofproto: Optimize the case of a repeated learn action execution.

When the target flow exists already as intended, only the 'modified'
time needs to be updated.  Allow modification without taking the
'ofproto_mutex' by always using the 'rule->mutex' when accessing the
'modified' time.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
ofproto: Move 'rule->used' to the provider.

Rule's 'used' timestamp is updated at the same time with the other
stats.  So far the 'used' has been updated without proper protection,
which may lead to 'tearing' in 32-bit architectures, resulting in an
incorrect 'used' timestamp.  While in practice this is highly
improbable, it is still better to handle this correctly.

This is resolved by moving the 'used' field to the provider's stats,
so that whatever protection is used for updating packet and byte
counts, can be also used for both reading and writing of the 'used'
timestamp.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
datapath: Fix race.

ovs_vport_cmd_dump() did rcu_read_lock() only after getting the
datapath, which could have been deleted in between.  Resolved by
taking rcu_read_lock() before the get_dp() call.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: Pravin B Shelar <pshelar@nicira.com>
ovs-thread: Add support for pthread adaptive mutex

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
ofproto-dpif: Use adaptive mutex for stats.

Stats critical sections are held for very short time, so it makes
sense to spin on them for a while rather than sleeping immediately.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
util: Make xreadlink a static function.

It is only used in util.c

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
string: strerror_r() for windows.

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
util: follow_symlinks() for windows.

Start with not supporting symbolic links for windows.
This is useful for an upcoming commit.

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
lockfile: Implementation for Windows platform.

Start with a simple implementation that does not support
symbolic links.

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
util: Pre-allocate buffer for ovs_lasterror_to_string().

This lets us call ovs_lasterror_to_string() and not having
to do an extra call of LocalFree() on the returned string.

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
tests: Add support for automatically running Ryu tests against OVS.

The Ryu controller comes with an extensive library of OpenFlow tests, but
it doesn't seem so easy to me to run all of them against a development
version of Open vSwitch.  This commit introduces a Makefile target so that
one can run all the Ryu tests with a simple "make check-ryu".

This commit adds documentation for the new target to INSTALL.  It also
moves the documentation for the "check-oftest" target and the
--enable-coverage configure option into INSTALL.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Reviewed-by: Simon Horman <horms@verge.net.au>
datapath: fix dp check in ovs_dp_reset_user_features

This fixes crash when userspace does "ovs-dpctl add-dp dev" where dev is
existing non-dp netdevice.

Introduced by:
commit 94358dcffbec33cbcfd425e925139fd7e9d6153f
"openvswitch: Drop user features if old user space attempted to create datapath"

Signed-off-by: Jiri Pirko <jiri@resnulli.us>
Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Remove 5-tuple optimization.

The 5-tuple optimization becomes unnecessary with a later per-NUMA
node stats patch.  Remove it first to make the changes easier to
grasp.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Per NUMA node flow stats.

Keep kernel flow stats for each NUMA node rather than each (logical)
CPU.  This avoids using the per-CPU allocator and removes most of the
kernel-side OVS locking overhead otherwise on the top of perf reports
and allows OVS to scale better with higher number of threads.

With 9 handlers and 4 revalidators netperf TCP_CRR test flow setup
rate doubles on a server with two hyper-threaded physical CPUs (16
logical cores each) compared to the current OVS master.  Tested with
non-trivial flow table with a TCP port match rule forcing all new
connections with unique port numbers to OVS userspace.  The IP
addresses are still wildcarded, so the kernel flows are not considered
as exact match 5-tuple flows.  This type of flows can be expected to
appear in large numbers as the result of more effective wildcarding
made possible by improvements in OVS userspace flow classifier.

Perf results for this test (master):

Events: 305K cycles
+   8.43%     ovs-vswitchd  [kernel.kallsyms]   [k] mutex_spin_on_owner
+   5.64%     ovs-vswitchd  [kernel.kallsyms]   [k] __ticket_spin_lock
+   4.75%     ovs-vswitchd  ovs-vswitchd        [.] find_match_wc
+   3.32%     ovs-vswitchd  libpthread-2.15.so  [.] pthread_mutex_lock
+   2.61%     ovs-vswitchd  [kernel.kallsyms]   [k] pcpu_alloc_area
+   2.19%     ovs-vswitchd  ovs-vswitchd        [.] flow_hash_in_minimask_range
+   2.03%          swapper  [kernel.kallsyms]   [k] intel_idle
+   1.84%     ovs-vswitchd  libpthread-2.15.so  [.] pthread_mutex_unlock
+   1.64%     ovs-vswitchd  ovs-vswitchd        [.] classifier_lookup
+   1.58%     ovs-vswitchd  libc-2.15.so        [.] 0x7f4e6
+   1.07%     ovs-vswitchd  [kernel.kallsyms]   [k] memset
+   1.03%          netperf  [kernel.kallsyms]   [k] __ticket_spin_lock
+   0.92%          swapper  [kernel.kallsyms]   [k] __ticket_spin_lock
...

And after this patch:

Events: 356K cycles
+   6.85%     ovs-vswitchd  ovs-vswitchd        [.] find_match_wc
+   4.63%     ovs-vswitchd  libpthread-2.15.so  [.] pthread_mutex_lock
+   3.06%     ovs-vswitchd  [kernel.kallsyms]   [k] __ticket_spin_lock
+   2.81%     ovs-vswitchd  ovs-vswitchd        [.] flow_hash_in_minimask_range
+   2.51%     ovs-vswitchd  libpthread-2.15.so  [.] pthread_mutex_unlock
+   2.27%     ovs-vswitchd  ovs-vswitchd        [.] classifier_lookup
+   1.84%     ovs-vswitchd  libc-2.15.so        [.] 0x15d30f
+   1.74%     ovs-vswitchd  [kernel.kallsyms]   [k] mutex_spin_on_owner
+   1.47%          swapper  [kernel.kallsyms]   [k] intel_idle
+   1.34%     ovs-vswitchd  ovs-vswitchd        [.] flow_hash_in_minimask
+   1.33%     ovs-vswitchd  ovs-vswitchd        [.] rule_actions_unref
+   1.16%     ovs-vswitchd  ovs-vswitchd        [.] hindex_node_with_hash
+   1.16%     ovs-vswitchd  ovs-vswitchd        [.] do_xlate_actions
+   1.09%     ovs-vswitchd  ovs-vswitchd        [.] ofproto_rule_ref
+   1.01%          netperf  [kernel.kallsyms]   [k] __ticket_spin_lock
...

There is a small increase in kernel spinlock overhead due to the same
spinlock being shared between multiple cores of the same physical CPU,
but that is barely visible in the netperf TCP_CRR test performance
(maybe ~1% performance drop, hard to tell exactly due to variance in
the test results), when testing for kernel module throughput (with no
userspace activity, handful of kernel flows).

On flow setup, a single stats instance is allocated (for the NUMA node
0).  As CPUs from multiple NUMA nodes start updating stats, new
NUMA-node specific stats instances are allocated.  This allocation on
the packet processing code path is made to never block or look for
emergency memory pools, minimizing the allocation latency.  If the
allocation fails, the existing preallocated stats instance is used.
Also, if only CPUs from one NUMA-node are updating the preallocated
stats instance, no additional stats instances are allocated.  This
eliminates the need to pre-allocate stats instances that will not be
used, also relieving the stats reader from the burden of reading stats
that are never used.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: Jesse Gross <jesse@nicira.com>
Signed-off-by: Pravin B Shelar <pshelar@nicira.com>
OPENFLOW-1.1+: Remove "rework tag order" and MPLS BoS matching from OF1.3.

OpenFlow 1.3.4 is going to revert the tag order change made by OpenFlow
1.3.

MPLS BoS matching is implemented.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Joe Stringer <joestringer@nicira.com>