dpif: Allow execute to modify the packet.

Allowing the packet to be modified by execution allows less data
copying for userspace action execution.  Some users of the
dpif_execute already expect that the packet may be modified.  This
patch makes this behavior uniform and makes the userspace datapath and
the execution helpers modify the packet as it is being executed.
Userspace action now steals the packet if given permission, as the
packet is normally not needed after it.  The only exception is the
sample action, and this is accounted for my keeping track of any
actions that could be following the userspace action.

The packet in dpif_upcall is changed from a pointer to a struct,
allowing the packet to be honest about it's headroom.  After this
change the packet can safely be pushed on over the precarious 4 byte
limit earlier allowed by the netlink data preceding the packet.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
bridge: Remove superfluous "inline" from ofp12_controller_role_to_str().

Putting "static inline" on a function definition in a .c file does not help
the compiler and does suppress warnings for unused functions that one would
want, should the function ever become unused.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Joe Stringer <joestringer@nicira.com>
connmgr: Fix attempt to take mutex recursively when exiting fail-open.

If one configured a controller which does not exist, waited for the switch
to enter fail-open mode, and then deleted the controller, then
ofproto_set_controllers() would take ofproto_mutex and call
update_fail_open(), which would call fail_open_destroy(), which would call
fail_open_recover(), which would call ofproto_delete_flow(), which requires
ofproto_mutex not to be held since it eventually try to take it.  This
caused OVS to abort.

This fixes the problem by releasing ofproto_mutex earlier, since nothing
seems to require it being held so long (a comment in
connmgr_set_controllers() says that this is likely to be the case).

Better annotations would have found this problem at compile time.  A later
patch adds them.

Reported-by: ZhengLingyun <konghuarukhr@163.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
datapath: remove duplicated include from flow_table.c

Remove duplicated include.

Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Backport for architecture hash library.

The architecture-specific hash library automatically selects either
jhash or CRC32 if it is available on the current processor.

Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: use CRC32 accelerated flow hash if available

Currently OVS uses jhash2() for calculating flow hashes in its
internal flow_hash() function. The performance of the flow_hash()
function is critical, as the input data can be hundreds of bytes
long.

OVS is largely deployed in x86_64 based datacenters.  Therefore,
we argue that the performance critical fast path of OVS should
exploit underlying CPU features in order to reduce the per packet
processing costs. We replace jhash2 with the hash implementation
provided by the kernel hash lib, which exploits the crc32l
instruction to achieve high performance

Our patch greatly reduces the hash footprint from ~200 cycles of
jhash2() to around ~90 cycles in case of ovs_flow_hash_crc()
(measured with rdtsc over maximum length flow keys on an i7 Intel
CPU).

Additionally, we wrote a microbenchmark to stress the flow table
performance. The benchmark inserts random flows into the flow
hash and then performs lookups. Our hash deployed on a CRC32
capable CPU reduces the lookup for 1000 flows, 100 masks from
~10,100us to ~6,700us, for example.

Thus, simply use the newly introduced arch_fast_hash2() as a
drop-in replacement.

Signed-off-by: Francesco Fusco <ffusco@redhat.com>
Signed-off-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: Thomas Graf <tgraf@redhat.com>
Signed-off-by: Jesse Gross <jesse@nicira.com>
ofproto/trace: Fix memory leak on error path.

Propagate error response from parse_ofp_exact_flow() to the caller and
properly free() it afterward.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
datapath: Backport skb_zerocopy() functions.

These functions will be factored out and exported upstream. On
kernels 3.5 and newer the backport will provide zero copy support
but older kernels will work as before (due to lack of skb->head_frag).

Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Allow user space to announce ability to accept unaligned Netlink messages

Signed-off-by: Thomas Graf <tgraf@suug.ch>
Reviewed-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Drop user features if old user space attempted to create datapath

Drop user features if an outdated user space instance that does not
understand the concept of user_features attempted to create a new
datapath.

Signed-off-by: Thomas Graf <tgraf@suug.ch>
Reviewed-by: Daniel Borkmann <dborkman@redhat.com>
Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Pass datapath into userspace queue functions

Allows removing the net and dp_ifindex argument and simplify the
code.

Signed-off-by: Thomas Graf <tgraf@suug.ch>
Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Use skb_zerocopy() for upcall

Use of skb_zerocopy() can avoid the expensive call to memcpy()
when copying the packet data into the Netlink skb. Completes
checksum through skb_checksum_help() if not already done in
GSO segmentation.

Zerocopy is only performed if user space supported unaligned
Netlink messages. memory mapped netlink i/o is preferred over
zerocopy if it is set up.

Cost of upcall is significantly reduced from:
+   7.48%       vhost-8471  [k] memcpy
+   5.57%     ovs-vswitchd  [k] memcpy
+   2.81%       vhost-8471  [k] csum_partial_copy_generic

to:
+   5.72%     ovs-vswitchd  [k] memcpy
+   3.32%       vhost-5153  [k] memcpy
+   0.68%       vhost-5153  [k] skb_zerocopy

(megaflows disabled)

Signed-off-by: Thomas Graf <tgraf@suug.ch>
Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Compute checksum in skb_gso_segment() if needed

The copy & csum optimization is no longer present with zerocopy
enabled. Compute the checksum in skb_gso_segment() directly by
dropping the HW CSUM capability from the features passed in.

Signed-off-by: Thomas Graf <tgraf@suug.ch>
Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Add missing #include for skb page accessors.

Reported-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Backport skb_frag_ functions

Define accessors skb_frag_* functions whch were introduced in 3.2

Signed-off-by: Pravin B Shelar <pshelar@nicira.com>
datapath: Backport skb_has_frag_list().

Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Backport __skb_fill_page_desc

Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Add hash library to .gitignore.

Signed-off-by: Jesse Gross <jesse@nicira.com>
datapath: Handle different definitions of page in skb_frag_t.

Signed-off-by: Jesse Gross <jesse@nicira.com>
FAQ: Describe weak and strong ES models.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Andy Zhou <azhou@nicira.com>