alibaba/ali_kernel — pull request #37 by taoma-tm (2013-11-28 07:54:01+00:00)
Title: tcp friends

TCP/IP loopback socket pair stack bypass, based on an idea by, and
rough upstream patch from, David Miller called
"friends", the data structure modifcations and connection scheme are
reused with extensive data-path changes.

A new sysctl, net.ipv4.tcp_friends, is added:
0: disable friends and use the stock data path.
1: enable friends and bypass the stack data path, the default.

Note, when friends is enabled any loopback interpose, e.g. tcpdump,
will only see the TCP/IP packets during connection establishment and
finish, all data bypasses the stack and instead is delivered to the
destination socket directly.

Testing done on a 4 socket 2.2GHz "Quad-Core AMD Opteron(tm) Processor
8354 CPU" based system, netperf results for a single connection show
increased TCP_STREAM throughput, increased TCP_RR and TCP_CRR transaction
rate for most message sizes vs baseline and comparable to AF_UNIX.

Significant increase (up to 5x) in aggregate throughput for multiple
netperf runs (STREAM 32KB I/O x N) is seen.

Some results:

Default netperf: netperf
netperf -t STREAM_STREAM
netperf -t STREAM_STREAM -- -s 51882 -m 16384 -M 87380
netperf

Baseline AF_UNIX AF_UNIX Friends
Mbits/S Mbits/S Mbits/S Mbits/S
6860 714 8% 9444 138% 1323% 10576 154% 1481% 112%

Note, for the AF_UNIX (STREAM_STREAM) test 2 results are listed, 1st
with no options but as the defaults for AF_UNIX sockets are much lower
performaning a 2nd set of runs with a socket buffer size and send/recv
buffer sizes equivalent to AF_INET (TCP_STREAM) are done.

Note, all subsequent AF_UNIX (STREAM_STREAM, STREAM_RR) tests are done
with "-s 51882" such that the same total effective socket buffering is
used as for the AF_INET runs defaults (16384+NNNNN/2).

STREAM 32KB I/O x N: netperf -l 100 -t TCP_STREAM -- -m 32K -M 32K
netperf -l 100 -t STREAM_STREAM -- -s 51882 -m 32K -M 32K
netperf -l 100 -t TCP_STREAM -- -m 32K -M 32K

Baseline AF_UNIX Friends
N COC Mbits/S Mbits/S Mbits/S
1 - 8616 9416 109% 11116 129% 118%
2 - 15419 17076 111% 20267 131% 119%
16 2 59497 303029 509% 347349 584% 115%
32 4 54223 273637 505% 272891 503% 100%
256 32 58244 85476 147% 273696 470% 320%
512 64 58745 87402 149% 260837 444% 298%
1600 200 83161 158915 191% 383947 462% 242%

COC = Cpu Over Commit ratio (16 core platform)

Stream: netperf -l 100 -t TCP_STREAM
netperf -l 100 -t STREAM_STREAM -- -s 51882
netperf -l 100 -t TCP_STREAM

netperf Baseline AF_UNIX Friends
-m/-M N Mbits/S Mbits/S Mbits/S
64 1020 445 44% 515 50% 116%
1K 4881 4340 89% 5070 104% 117%
8K 5933 8387 141% 9770 165% 116%
32K 8168 9538 117% 11067 135% 116%
64K 9116 9774 107% 11515 126% 118%
128K 9053 10044 111% 13082 145% 130%
256K 9642 10351 107% 13470 140% 130%
512K 10050 10142 101% 13327 133% 131%
1M 8640 9843 114% 12201 141% 124%
16M 7179 9619 134% 11316 158% 118%

Rr: netperf -l 100 -t TCP_RR
netperf -l 100 -t STREAM_RR -- -s 51882 -m 16384 -M 87380
netperf -l 100 -t TCP_RR

netperf Baseline AF_UNIX Friends
-r N,N Trans./S Trans./S Trans./S
64 47913 99681 208% 98225 205% 99%
1K 44045 92327 210% 91608 208% 99%
8K 26732 33201 124% 33025 124% 99%
32K 10903 11972 110% 13574 124% 113%
64K 7113 6718 94% 7176 101% 107%
128K 4191 3431 82% 3695 88% 108%
256K 2324 1937 83% 2147 92% 111%
512K 958 1056 110% 1202 125% 114%
1M 404 508 126% 497 123% 98%
16M 26.1 34.1 131% 32.9 126% 96%

Crr: netperf -l 100 -t TCP_CRR
netperf -l 100 -t TCP_CRR

netperf Baseline AF_UNIX Friends
-r N Trans./S Trans./S Trans./S
64 14690 - 18191 124% -
1K 14258 - 17492 123% -
8K 11535 - 14012 121% -
32K 7035 - 8974 128% -
64K 4312 - 5654 131% -
128K 2252 - 3179 141% -
256K 1237 - 2008 162% -
512K 17.5* - 1079 ? -
1M 4.93* - 458 ? -
16M 8.29* - 32.5 ? -

Note, "-" denotes test not supported for transport.
"*" denotes test results reported without statistical confidence.
"?" denotes results not comparable.

SPLICE 32KB I/O:

Source
Sink Baseline Friends
FSFS Mbits/S Mbits/S
---- 8042 10810 134%
Z--- 7071 9773 138%
--N- 8039 10820 135%
Z-N- 7902 9796 124%
-S-- 17326 37496 216%
ZS-- 9008 9573 106%
-SN- 16154 36269 225%
ZSN- 9531 9640 101%
---S 8466 9933 117%
Z--S 8000 9453 118%
--NS 12783 11379 89%
Z-NS 11055 9489 86%
-S-S 12741 24380 191%
ZS-S 8097 10242 126%
-SNS 16657 30954 186%
ZSNS 12108 12763 105%

Note, "Z" source File /dev/zero, "-" source user memory
"N" sink File /dev/null, "-" sink user memory
"S" Splice on, "-" Splice off

Note from Tao Ma:
1. rebase to 2.6.32-358.14.1
2. fix a bug when syncookies are used.

Signed-off-by: Bruce \"Brutus\" Curtis
Signed-off-by: Li Yu
Signed-off-by: Zhu Yanhai
Signed-off-by: Tao Ma