dpif-netdev: Garbage collect the exact match cache periodically.

On current master, the exact match cache entry can keep reference to
'struct dp_netdev_flow' even after the flow is removed from the flow
table.  This means the free of allocated memory of the flow is delayed
until the exact match cache entry is cleared or replaced.

If the allocated memory is ahead of chunks of freed memory on heap,
the delay will prevent the reclaim of those freed chunks, causing
falsely high memory utilization.

To fix the issue, this commit makes the owning thread conduct periodic
garbage collection on the exact match cache and clear dead entries.

Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>

---
PATCH -> V2:
- Adopt Jarno's suggestion and conduct slow sweep to avoid introducing
  jitter.
cmap: Shrink cmap when load factor is below 20%.

This commit adds check in cmap_remove() and shrinks the cmap by half
if the load factor is below 20%.  This is to reduce the memory
utilization of cmap and to avoid the allocated cmap memory occupying
the top of heap memory, preventing the trim of heap.

Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Ben Pfaff <blp@nicira.com>
xen.spec: Fix xenserver build failure.

This commit conducts similar operations as commit 2fd84d5e5
(rhel: Include pkgconfig and update header install in
Fedora/RHEL spec.) to xenserver spec file to prevent the
following rpmbuild failure.

  RPM build errors:
       Installed (but unpackaged) file(s) found:
           /usr/include/openflow/nicira-ext.h
           /usr/include/openflow/openflow-1.0.h
           /usr/include/openflow/openflow-1.1.h
           /usr/include/openflow/openflow-1.2.h
           /usr/include/openflow/openflow-1.3.h
           /usr/include/openflow/openflow-1.4.h
           /usr/include/openflow/openflow-1.5.h
           /usr/include/openflow/openflow-common.h
           /usr/include/openflow/openflow.h
           /usr/include/openvswitch/types.h
           /usr/lib/pkgconfig/libofproto.pc
           /usr/lib/pkgconfig/libopenvswitch.pc
           /usr/lib/pkgconfig/libovsdb.pc
           /usr/lib/pkgconfig/libsflow.pc

VMware-BZ: #1362092

Reported-by: Edwin Chiu <echiu@nicira.com>
Signed-off-by: Alex Wang <alexw@nicira.com>
Acked-by: Gurucharan Shetty <gshetty@nicira.com>
datapath-windows: do not complete an IRP when pending status returned

The driver completes a pending IRP, the IRP is being marked free by the
I/O manager which causes a system crash when the IRP is completed at
the second time.
The driver should not complete a pending IRP in the context of the
Device I/O control thread.

*** Fatal System Error: 0x00000044
                       (0xFFFFFA800CF897D0,0x0000000000000F7A,0x0000000000000000,0x0000000000000000)

Break instruction exception - code 80000003 (first chance)

BugCheck 44, {fffffa800cf897d0, f7a, 0, 0}

Probably caused by : OVSExt.sys ( OVSExt!OvsCompleteIrpRequest+3a )

Followup: MachineOwner
---------

nt!DbgBreakPointWithStatus:
fffff800`5a100930 cc              int     3
11: kd> !analyze -v
*******************************************************************************
*                                                                             *
*                        Bugcheck Analysis                                    *
*                                                                             *
*******************************************************************************

MULTIPLE_IRP_COMPLETE_REQUESTS (44)
A driver has requested that an IRP be completed (IoCompleteRequest()), but
the packet has already been completed.  This is a tough bug to find because
the easiest case, a driver actually attempted to complete its own packet
twice, is generally not what happened.  Rather, two separate drivers each
believe that they own the packet, and each attempts to complete it.  The
first actually works, and the second fails.  Tracking down which drivers
in the system actually did this is difficult, generally because the trails
of the first driver have been covered by the second.  However, the driver
stack for the current request can be found by examining the DeviceObject
fields in each of the stack locations.
Arguments:
Arg1: fffffa800cf897d0, Address of the IRP
Arg2: 0000000000000f7a
Arg3: 0000000000000000
Arg4: 0000000000000000

Signed-off-by: Eitan Eliahu <eliahue@vmware.com>
Acked-by: Nithin Raju <nithin@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
netlink-socket: Set socket pid number in NL message on Windows.

The pid must be set in the NL header as the driver checks it against the pid in
the instance paired with the socket.

Signed-off-by: Eitan Eliahu <eliahue@vmware.com>
Acked-by: Nithin Raju <nithin@vmware.com>
Signed-off-by: Ben Pfaff <blp@nicira.com>
tunnel-push-pop.at: Make test more deterministic.

Using the '-march=native' broke this test in a i7 laptop, most likely
due to a different hash implementation, which resulted in different
datapath port number assignment accross different compilations.

Break up the ports setup so that the datapath port number assignment
is more deterministic.

Signed-off-by: Jarno Rajahalme <jrajahalme@nicira.com>
Acked-by: Pravin B Shelar <pshelar@nicira.com>
dpctl: Fix crash.

ovs-dpctl crashed due to uninitialized router classifier. To
fix this issue move ovs router initialization to route table
module.

Reported-by: Madhu Challa <challa@noironetworks.com>
Signed-off-by: Pravin B Shelar <pshelar@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
dpif-netdev: Remove redundant hash action handling.

odp_execute_actions() already handles hash execution part.

Signed-off-by: Pravin B Shelar <pshelar@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
tests: Add command to purge revalidators of flows.

This patch adds a new 'ovs-appctl revalidator/purge' command which
flushes all flows from all datapaths, and updates the revalidator
udpif_key cache at the same time.

Update the ofproto-dpif fragment tests which may fail when ukeys are
created from handler threads.

Signed-off-by: Joe Stringer <joestringer@nicira.com>
Acked-by: Jarno Rajahalme <jrajahalme@nicira.com>
ofp-util.c: Enable packet-in messages for OpenFlow 1.4 and 1.5.

Signed-off-by: Jean Tourrilhes <jt@hpl.hp.com>
[blp@nicira.com added simple test]
Signed-off-by: Ben Pfaff <blp@nicira.com>
Change Travis CI notification email to ljakab@ac.upc.edu
userspace: add support for pop_eth and push_eth actions

These actions will allow L2->L3 and L3->L2 switching, and are supposed
to be added to flows installed in the datapath transparently by
ovs-vswitchd.

Signed-off-by: Lorand Jakab <lojakab@cisco.com>
userspace: add layer 3 flow and switching support

This commit relaxes the assumption that all packets have an Ethernet
header, and adds support for layer 3 flows.  For each packet received on
the Linux kernel datapath the l2 and l3 members of struct ofpbuf are
intialized appropriately, and some functions now expect this (notably
flow_extract()), in order to differentiate between layer 2 and layer 3
packets.  struct flow has now a new 'base_layer' member, because we
cannot assume that a flow has no Ethernet header when eth_src and
eth_dst are 0.  For layer 3 packets, the protocol type is still stored
in the eth_type member.

Switching L2->L3 and L3->L2 are both implemented by adding the pop_eth
and push_eth actions respectively when a transition is detected.  The
push_eth action puts 0s on both source and destination MACs.  These
addresses can be modified with mod_dl_dst and mod_dl_src actions.

Added new prerequisite MFP_ETHERNET for fields MFF_VLAN_TCI,
MFF_DL_VLAN, MFF_VLAN_VID and MFF_DL_VLAN_PCP.

Signed-off-by: Lorand Jakab <lojakab@cisco.com>
datapath: add layer 3 flow/port support

Implementation of the pop_eth and push_eth actions in the kernel, and
layer 3 flow support.

Signed-off-by: Lorand Jakab <lojakab@cisco.com>
userspace: add layer 3 support to packet metadata

This is needed for sending a packet back to the datapath after a miss upcall
was processed.  The presence of a layer 2 packet is signaled by adding
OVS_KEY_ATTR_ETHERNET to the packet metadata sent with the ovs_packet netlink
message.  Layer 3 packets need to be accompanied by OVS_KEY_ATTR_ETHERTYPE to
indicate network protocol.

Signed-off-by: Lorand Jakab <lojakab@cisco.com>
datapath: add layer 3 support to ovs_packet_cmd_execute()

Packet metadata for layer 3 packets is expected to contain
OVS_KEY_ATTR_ETHERTYPE to indicate the network protocol.

Signed-off-by: Lorand Jakab <lojakab@cisco.com>