ofproto-dpif-upcall: Fix logic error in handler/revalidator threads
creation and deletion.

Commit 1f8675481e (ofproto-dpif-upcall: Fix ovs-vswitchd crash.)
directly copied the udpif_set_threads() logic to udpif_stop_threads()
and udpif_start_threads().  In fact, this was erroneous and caused
unittest failures.

This commit fixes the above issue by correcting the checks in
udpif_stop_threads() and udpif_start_threads(), and adding necessary
checks in udpif_set_threads().

Acked-by: Ethan Jackson <ethan@nicira.com>
Signed-off-by: Alex Wang <alexw@nicira.com>
ovs-rcu: Name the ovsrcu_postpone_thread to 'urcu'.

The ovs-rcu module adds a new thread for checking the grace period.
Since the thread name is not set, it will inherit the name of the
thread that creates it.  This makes the 'top' output quite confusing.

This commit names the thread to 'urcu' for clarity.

Acked-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: Alex Wang <alexw@nicira.com>
tests: Define a variable "IS_WIN32" for tests.

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
stream: Introduce [p]windows_[p]stream_class.

On Linux, we heavily use --remote=punix:* to listen for
connections through unix domain sockets. We also use, unix:*
to connect to a daemon that is listening on unix domain sockets.
Many times, we create default unix domain sockets for listening
and many utilities connect to these sockets by default.

Windows does not have unix domain sockets. So far, we could just use
ptcp:* and tcp:* for listening and initiating connections respectively.
The drawback here is that one has to provide a specific TCP port.

For unit tests, it looks useful to let kernel choose that port.
As such, we can let that chosen kernel port be stored in the
file specified with punix:* and unix:*. For this purpose, introduce
a new [p]windows_[p]stream_class. Since it is just a wrapper around
[p]tcp_[p]stream_class, add it to stream-tcp.c.

commit cb54a8c (unixctl: Add support for Windows.) used the above concept
for only control channel connections (i.e., --unixctl for daemons and its
interaction with ovs-appctl). This commit adds the same support for
all unix domain sockets.  Now that we have a separate class
[p]stream_class for hiding kernel assigned TCP port inside a file meant for
unix domain sockets in windows, make unixctl use it.

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
daemon-windows: Rename service variables.

Sa far, we are using variable 'detach' to indicate whether the option
"--service" has been set. We were using variable 'detached' to indicate that
the daemon is being called from the Windows services manager.

An upcoming commit introduces command line option "--detach" for daemons
running on Windows. This will cause confusion with variable names.
Therefore, rename the variables.

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
daemon-windows: Implement --detach option for Windows.

When "--detach" is specified, a daemon will create a new
process with the same command line options as the parent.
Additionally, an undocumented command line option "--pipe-handle"
is passed to child. Once the child is ready to handle external
commands, it communicates with the parent that it is ready using
the pipe handle. The parent exits. This lets us run the daemons
in background. This will also help the unit tests because currently
most of the unit tests pass the '--detach' option to the daemons.

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
daemon-windows: Create pidfiles with option --pidfile.

In Windows, we cannot delete a file that has been opened.
We use this feature to "lock" the pidfile.

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
testsuite.at: pwd for windows.

On MinGW, "pwd -W" gives the present working directory
in the form of windows path (i.e C:/temp instead of /c/temp).
When we pass the directory path to daemons as arguments,
we should be passing it in the form of windows path.

Suggested-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
testsuite.at: kill for windows.

We use kill to cleanup processes from pidfiles.
Windows has a 'taskkill' which does something similar.
We can check if the process with a PID exists with
'tasklist'. Both tasklist and taskkill return 0 for
both success and failure. So, we will have to grep
to see if there is a o/p.

A typical o/p of tasklist is:
$ tasklist | grep ovs
ovsdb-server.exe              3228 RDP-Tcp#0                  2      6,132 K
ovs-vswitchd.exe              2080 RDP-Tcp#0                  2      5,808 K

$ tasklist //fi "PID eq 3228"

Image Name                     PID Session Name        Session#    Mem Usage
========================= ======== ================ =========== ============
ovsdb-server.exe              3228 RDP-Tcp#0                  2      6,132 K

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
testsuite.at: Workaround for carriage returns on windows.

In unit tests, we compare text written in logs or stdout/stderr
to figure out the success or failure of tests. In Windows,
since new line is represented by CR+LF, autoconf tests run in
MinGW environment fail.

Asking diff to ignore trailing carriage returns is one way
to solve the problem

Suggested-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
TODO: Add the project list from the hackathon.

I've had a couple of requests for an updated project list, so this commit
adds it to the tree.

Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Joe Stringer <joestringer@nicira.com>
daemon-windows: Recognize --no-chdir option for windows.

The option won't have any effect on the running of the daemon.
Recognizing the option lets us avoid if else conditions in unit
tests.

Signed-off-by: Gurucharan Shetty <gshetty@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
run-ryu: Use unix socket rather than patch ports

My understanding of the implementation of patch ports is that they
are rather special, being handled as a special case inside
compose_output_action__() and do not exist in the datapath.

A side effect of the implementation of patch ports (though perhaps not the
portion mentioned above) is that the OFPUTIL_PC_PORT_DOWN bit may not be
set via a port mod message. In particular, the call to
netdev_turn_flags_on() in update_port_config() fails.

There is a test provided by Ryu that test this via port mod and thus fails.

While that test could be modified or the results ignored it seems to me
that it would be best if ryu-check used ports which were more fully
featured and not special cases.

Thus this patch moves run-ryu to use unix socket backed ports rather than
patch ports.

I believe a more significant problem with the use of patch ports
is that they will require (more) special case code in order to correctly
handle recirculation. As Ryu provides many tests that exercise
recirculation for MPLS it would be nice if they could be used to exercise
recirculation for MPLS (which I have provided patches for separately[1])
without the need to add more special-case code for that purpose.

I believe that patch ports are also incompatible with recirculation for
bonding, which has already been merged, though I have not verified that
and it is not strictly related to this patch as I do not believe that Ryu
provides any tests to exercise that case.

The key problem with patch ports in the context of recirculation is that
the ofproto and in_port may change during translation.  And this
information is lost by the time that execution occurs.

Furthermore the new in_port will not exist in the datapath as it is a
patch port. That particular problem may be addressed by executing the
actions in user-space, I have posted patches to provide infrastructure
for that[1].

Overall it is not clear to me that the complexity of supporting
recirculation for patch-ports would have sufficient pay-off.

[1] [PATCH v3 00/16] Flow-Based Recirculation for MPLS

Signed-off-by: Simon Horman <horms@verge.net.au>
Signed-off-by: Ben Pfaff <blp@nicira.com>
lib/util: Input validation in str_to_uint

This function returns true when 's' is negative or greater than UINT_MAX. Also,
the representation of 'int' and 'unsigned int' is implementation dependent, so
converting [INT_MAX..UINT_MAX] values with str_to_int is fragile.
Instead, we should convert straight to 'long long' and do a boundary check
before returning the converted value.
This patch also move the function to the .c file as it's not-trivial now, and
deletes the other str_to_u* functions as they are not used.

Signed-off-by: Zoltan Kiss <zoltan.kiss@citrix.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
ofproto-dpif: Improve code clarity and comments on recirc changes to rule_dpif_lookup()

This patch improves the code readability and comments on the
recirculation related changes to rule_dpif_lookup() base on off-line
discussions with Jarno.  There is no behavior changes.

Signed-off-by: Andy Zhou <azhou@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
revalidator: Prevent handling the same flow twice.

When the datapath flow table is modified while a flow dump operation is
in progress, it is possible for the same flow to be dumped twice. In
such cases, revalidators may perform redundant work, or attempt to
delete the same flow twice.

This was causing intermittent testsuite failures for test #670 -
"ofproto-dpif, active-backup bonding" where a flow (that had not
previously been dumped) was dumped, revalidated and deleted twice.

The logs show errors such as:
"failed to flow_get (No such file or directory) skb_priority(0),..."
"failed to flow_del (No such file or directory) skb_priority(0),..."

This patch adds a 'flow_exists' field to 'struct udpif_key' to track
whether the flow is (in progress) to be deleted. After doing a ukey
lookup, we check whether ukey->mark or ukey->flow indicates that the
flow has already been handled. If it has already been handled, we skip
handling the flow again.

We also defer ukey cleanup for flows that fail revalidation, so that the
ukey will still exist if the same flow is dumped twice. This allows the
above logic to work in this case.

Signed-off-by: Joe Stringer <joestringer@nicira.com>
Acked-by: Alex Wang <alexw@nicira.com>
bridge: When ports disappear from a datapath, add them back.

Before commit 2a73b1d73d4bdb (bridge: Reconfigure in single pass.), if a
port disappeared, for one reason or another, from a datapath, the next
bridge reconfiguration pass would notice and, if the port was still
configured in the database, add the port back to the datapath.  That
commit, however, removed the logic from bridge_refresh_ofp_port() that
did that and failed to add the same logic to the replacement function
bridge_delete_or_reconfigure_ports().  This commit fixes the problem.

To see this problem on a Linux kernel system:

ovs-vsctl add-br br0                             # 1
tunctl -t tap                                    # 2
ovs-vsctl add-port br0 tap                       # 3
ovs-dpctl show                                   # 4
tunctl -d tap                                    # 5
ovs-dpctl show                                   # 6
tunctl -t tap                                    # 7
ovs-vsctl del-port tap -- add-port br0 tap       # 8
ovs-dpctl show                                   # 9

Steps 1-4 create a bridge and a tap and add it to the bridge and
demonstrate that the tap is part of the datapath.  Step 5 and 6 delete
the tap and demonstrate that it has therefore disappeared from the
datapath.  Step 7 recreates a tap with the same name, and step 8
forces ovs-vswitchd to reconfigure.  Step 9 shows the effect of the
fix: without the fix, the new tap is not added back to the datapath;
with this fix, it is.

Special thanks to Gurucharan Shetty <gshetty@nicira.com> for finding a
simple reproduction case and then bisecting to find the commit that
introduced the problem.

Bug #1238467.
Reported-by: Ronald Lee <ronaldlee@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>
ofproto-dpif-upcall: Remove the flow_dumper thread.

Previously, we had a separate flow_dumper thread that fetched flows from
the datapath to distribute to revalidator threads. This patch takes the
logic for dumping and pushes it into the revalidator threads, resulting
in simpler code with similar performance to the current code.

One thread, the "leader", is responsible for beginning and ending each
flow dump, maintaining the flow_limit, and checking whether the
revalidator threads need to exit. All revalidator threads dump,
revalidate, delete datapath flows and garbage collect ukeys.

Co-authored-by: Joe Stringer <joestringer@nicira.com>
Signed-off-by: Joe Stringer <joestringer@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
hmap_random_node: Improve distribution

Improve random distribution for an hmap with a small number of nodes
with the expense of the increased cpu cost.
It would be a fair trade-off because the situation is rather common
for bond, which is currently the only consumer of this API in tree.

Consider 2 items, 4 buckets, no collision.

    bucket 0   item 0
    bucket 1
    bucket 2
    bucket 3   item 1

The old algorithm picks item 0 if rand % 4 == 0.  (25%)
Otherwise it picks item 1.  (75%)

This change makes them 50%.

Acked-by: Ben Pfaff <blp@nicira.com>
Signed-off-by: YAMAMOTO Takashi <yamamoto@valinux.co.jp>
ofproto: RCU postpone rule destruction.

This allows rules to be used without taking references while RCU
protected.

The last step of destroying an ofproto also needs to be postponed, as
the rule destruction requires the class structure to be available at
the postponed destruction callback.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Acked-by: Ethan Jackson <ethan@nicira.com>