ofproto: New function ofproto_refresh_rule().

This function checks for a rule in the classifier:
* If the rule exists, reset its modified time.
* If an equivalent rule exists, reset that rule's modified time.
* If no rule exists, re-install the rule and reset its modified time.
* Finally, return the rule that was modified.

This function will be used to ensure that hard timeouts for learnt rules
are refreshed if traffic consistently hits a rule with a learn action in
it. The first user will be the next commit.

Signed-off-by: Joe Stringer <joestringer@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
---
v2: Acked.
v1: Ensure rule->modified is updated correctly.
RFC: First post.
ofproto-dpif-xlate: Cache xlate_actions() effects.

This patch adds a new object called 'struct xlate_cache' which can be
set in 'struct xlate_in', and passed to xlate_actions() to cache the
modules affected by this flow translation. Subsequently, the caller can
pass the xcache to xlate_push_stats() to credit stats and perform side
effects for a lower cost than full flow translation.

These changes are aimed currently at long-lived flows, decreasing the
average dump duration for such flows by 50-80%. This allows more flows
to be supported in the datapath at a given time. Applying these changes
to short-lived flows is left for a later commit.

Signed-off-by: Joe Stringer <joestringer@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
---
v2: Acked.
v1: Add caching for fin_timeout action.
    Expire netflows on xlate_cache_clear().
    Account to bonds using a copy of 'flow' rather than hash.
    Always build XC_NORMAL entry (previously only if may_learn is true)
    Rename xlate_from_cache()->xlate_push_stats()
    Add may_learn parameter to xlate_push_stats()
    Tidy up xlate_actions__() mirror/netflow code.
    Fold in style fixups.
RFC: First post.
revalidator: Only revalidate high-throughput flows.

Previously we would revalidate all flows if the "need_revalidate" flag
was raised. This patch modifies the logic to delete low throughput flows
rather than revalidate them. High-throughput flows are unaffected by
this change. This patch identifies the flows based on the mean time
between packets since the last dump.

This change is primarily targeted at situations where:
* Flow dump duration is high (~1 second)
* Revalidation is triggered. (eg, by bridge reconfiguration or learning)

After the need_revalidate flag is set, next time a new flow dump session
starts, revalidators will begin revalidating the flows. This full
revalidation is more expensive, which significantly increases the flow
dump duration. At the end of this dump session, the datapath flow
management algorithms kick in for the next dump:

* If flow dump duration becomes too long, the flow limit is decreased.
* The number of flows in the datapath then exceeds the flow_limit.
* As the flow_limit is exceeded, max_idle is temporarily set to 100ms.
* Revalidators delete all flows that haven't seen traffic recently.

The effect of this is that many low-throughput flows are deleted after
revalidation, even if they are valid. The revalidation is unnecessary
for flows that would be deleted anyway, so this patch skips the
revalidation step for those flows.

Note that this patch will only perform this optimization if the flow has
already been dumped at least once, and only if the time since the last
dump is sufficiently long. This gives the flow a chance to become
high-throughput.

Signed-off-by: Joe Stringer <joestringer@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
---
v2: Acked.
v1: Determine "high-throughput" by packets rather than bytes.
    Calculate the mean time between packets for comparison, rather than
      comparing the number of packets since the last dump.
RFC: First post.
Merge pull request #3 from joestringer/submit/xlate_cache_v2

Cache the modules affected by xlate_actions().
lib/ofp-util: Restore the check for minus sign in port number strings.

Commit 33ab38d9 (meta-flow: Simplify mf_from_ofp_port_string())
inadvertently removed a check for minus sign at the beginning of a
port number string introduced by commit 05dddba (meta-flow: Don't
allow negative port numbers).  This check is still needed, so put it
back, but to ofputil_port_from_string() this time.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Acked-by: Justin Pettit <jpettit@nicira.com>
Reviewed-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
lib/flow: Introduce miniflow_extract().

miniflow_extract() extracts packet headers directly to a miniflow,
which is a compressed form of the struct flow.  This does not require
a large struct to be cleared to begin with, and accesses less memory.
These performance benefits should allow this to be used in the DPDK
datapath.

miniflow_extract() takes a miniflow as an input/output parameter.  On
input the buffer for values to be extracted must be properly
initialized.  On output the map contains ones for all the fields that
have been extracted.

Some struct flow fields are reordered to make miniflow_extract to
progress in the logical order.

Some explicit "inline" keywords are necessary for GCC to optimize this
properly.  Also, macros are used for same reason instead of inline
functions for pushing data to the miniflow.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Reviewed-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
lib/flow: Add miniflow accessors and miniflow_get_tcp_flags().

Add inlined generic accessors for miniflow integer type fields, and a
new miniflow_get_tcp_flags() usinge these.  These will be used in a
later patch.

Some definitions also used in lib/packets.h had to be moved there to
resolve circular include dependencies.  Similarly, some inline
functions using struct flow are now in lib/flow.h.  IMO this is
cleaner, since now the lib/flow.h need not be included from
lib/packets.h.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Reviewed-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
lib/flow: Possibly faster miniflow_hash_in_minimask()

Upcoming patches add classifier lookups using miniflows, this is
heavily used for it.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Reviewed-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
classifier: Support miniflow as a key.

Support struct miniflow as a key for datapath flow lookup.

The new classifier interface classifier_lookup_miniflow_first() takes
a miniflow as a key and stops at the first match with no regard to
flow prioritites.  This works only if the classifier has no
conflicting rules (as is the case with the userspace datapath
classifier).

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Reviewed-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
dpif-netdev: Use miniflow as a flow key.

Use miniflow as a flow key in the userspace datapath classifier.  The
miniflow is expanded for upcalls, but for existing datapath flows, the
key need not be expanded.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Reviewed-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
dpif-linux: Implement the API functions to allow multiple handler threads read upcall.

Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
dpif-linux: Pass 'struct dpif_linux *' to internal static functions.

This commit reformats the dpif-linux module so that all internal
static functions take 'struct dpif_linux *' as input argument.
This will allow the adding of thread-safety annotations.

Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
dpif-linux: Add thread-safety annotations.

Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
ofproto-dpif-upcall: Remove the dispatcher thread.

With the foundation laid in previous commits, this commit
removes the 'dispatcher' thread by allowing 'handler'
threads to read upcalls directly from dpif.

This commit significantly simplifies the flow miss handling
code and brings slight improvement to flow setup rate.

Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
ofproto-dpif-upcall: Don't use stack garbage

Catched by "learning action - self-modifying flow with hard_timeout"
test case.

The bug introduced by commit b256dc52.
("ofproto-dpif-xlate: Cache xlate_actions() effects.")

Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
Signed-off-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
ofproto-dpif: Rule lookup starts from table zero for non-recirc datapath

Currently, all packet lookup starts from internal table for possible
matching of post recirculation rules. This is not necessary for
datapath that does not support recirculation.

This patch adds the ability to steering rule lookup starting table
based on whether datapath supports recirculation.

Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
odp-util: Always generate key/mask pair in netlink for recirc_id

Currently netlink flow (and mask) recirc_id attribute is only
serialized when the recirc_id value is non-zero. For this logic
to work correctly, the interpretation of the missing recirc_id
depends on whether the datapath supports recirculation.

This patch remove the ambiguity of the meaning of missing recirc_id
attribute in netlink message.  When recirc_id is non-zero, or when
it is not a wildcard match, both key and mask attributes are
serialized.  On the other hand, when recirc_id is zero, and being
wildcarded, they are not serialized.  A missing recirc_id key and
mask attribute thus should always be interpreted as wildcard,
same as other flow fields.

Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
openvswitch.h: rename hash action definition

Rename hash_bias to hash_basis to make it consistent with similar
usages.

Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Jesse Gross <jesse@nicira.com>
datapath: remove unneeded declaration of new_vport().

This patch removes the new_vport() forward declaration in datapath.c
as it is not needed.

Signed-off-by: Rami Rosen <ramirose@gmail.com>
Signed-off-by: Jesse Gross <jesse@nicira.com>