TCP BBR + ShadowsocksR + fast.com の速度を macOS + テザリング環境で測定


fast.com への25MBファイルのダウンロード速度を、モバイルのテザリング回線環境上の macOS + curl 環境より測定。

| テザリング回線のみ|BBR有効SSR |BBR無効(cubic) SSR|
|---|---|---|
| 1分27秒| 13秒 |1分47秒|

BBRアルゴリズムが速度向上に寄与しているわけではなく、MVNO回線のトラフィックシェイピングの挙動が解ることにより、速度向上が起きている模様。この利用方法では、TCPの公平性に悪影響を与えてしまう行為になる可能性があり、一般良識の範囲内で試すなど、定常的な利用は控えた方が良いでしょう。

# 環境

## SSRサーバ

Vultr VPS 東京リージョン(512MBメモリサーバ)

## SSR サーバのディストリビューションとカーネル

`Ubuntu 18.04 LTS`

```
$ uname -a
Linux ssr 4.15.0-47-generic #50-Ubuntu SMP Wed Mar 13 10:44:52 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
```

## 測定場所と時間

IIJmio タイプDをAndroidのテザリングで東京の新橋で月曜21時台に測定。

## 測定クライアント OS

`macOS 10.14.1`

## 測定 SSR クライアント

`ShadowsocksX-NG 1.4.3-R8 (3)`

## SSR サーバの設定

```
{
        "server":"0.0.0.0",
        "server_ipv6":"[::]",
        "server_port":10023,
        "local_address":"127.0.0.1",
        "local_port":1080,
        "password":"********",
        "timeout":300,
        "method":"chacha20-ietf",
        "protocol":"auth_aes128_md5",
        "protocol_param":"",
        "obfs":"tls1.2_ticket_auth",
        "obfs_param":"",
        "redirect":"",
        "dns_ipv6":false,
        "fast_open":true,
        "workers":1
}
```

## 測定URL

ブラウザで開いたときにに fast.com へリクエストしたCDNのURL、25MBのファイル。

- https://ipv4-c026-tyo001-ix.1.oca.nflxvideo.net/speedtest/range/0-26214400\?c\=jp\&n\=2497\&v\=3\&e\=1555939238\&t\=_bq6LeisT4EjUdtyFLHF4HoFb-U

なお上記URLはテンポラリーな物で、しばらくたつと 403 になる。

# 測定

curlで複数回測定。ほとんど同じようなの速度結果だったので、一つを掲載している。`X-Session-Info` のアドレスは一部マスク。またダウンロード内容は、リクエストごとにレスポンスのファイルが異なるわけではなく、毎回同一なファイルであることを確認。

## SSRを使わない、通常のテザリング回線

ダウンロード速度は1分27秒ほど。X-Session-Infoは繋いでいるクライアントIPの模様(後ろ二つはマスクしてます)。

`X-Session-Info: addr=210.149.*.*`

```
$ time curl --verbose https://ipv4-c026-tyo001-ix.1.oca.nflxvideo.net/speedtest/range/0-26214400\?c\=jp\&n\=2497\&v\=3\&e\=1555939238\&t\=_bq6LeisT4EjUdtyFLHF4HoFb-U  > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 23.246.46.140...
* TCP_NODELAY set
* Connected to ipv4-c026-tyo001-ix.1.oca.nflxvideo.net (23.246.46.140) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [245 bytes data]
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [108 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2798 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [300 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [37 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=California; L=Los Gatos; O=Netflix, Inc.; OU=Content Delivery; CN=*.1.oca.nflxvideo.net
*  start date: Apr 10 00:00:00 2019 GMT
*  expire date: May 12 12:00:00 2019 GMT
*  subjectAltName: host "ipv4-c026-tyo001-ix.1.oca.nflxvideo.net" matched cert's "*.1.oca.nflxvideo.net"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
> GET /speedtest/range/0-26214400?c=jp&n=2497&v=3&e=1555939238&t=_bq6LeisT4EjUdtyFLHF4HoFb-U HTTP/1.1
> Host: ipv4-c026-tyo001-ix.1.oca.nflxvideo.net
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx
< Date: Mon, 22 Apr 2019 12:34:51 GMT
< Content-Type: application/octet-stream
< Content-Length: 26214400
< Last-Modified: Fri, 22 Mar 2019 06:12:59 GMT
< Connection: keep-alive
< Timing-Allow-Origin: *
< Cache-Control: no-store
< Pragma: no-cache
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: X-TCP-Info,X-Session-Info
< X-TCP-Info: h0=0;h1=0;h2=0;h3=0;h4=0;
< X-Session-Info: addr=210.149.*.*;port=58039;argp=
<
{ [16384 bytes data]
100 25.0M  100 25.0M    0     0   291k      0  0:01:27  0:01:27 --:--:--  371k
* Connection #0 to host ipv4-c026-tyo001-ix.1.oca.nflxvideo.net left intact
noglob curl --verbose  > /dev/null  1.65s user 0.70s system 2% cpu 1:27.99 total
```

## BBR が有効なカーネル + SSR

設定しているカーネルパラメータは以下。

```
$ sudo sysctl -p
net.ipv6.conf.all.accept_ra = 2
net.core.default_qdisc = fq
net.ipv4.tcp_congestion_control = bbr
fs.file-max = 1024000
net.core.rmem_max = 67108864
net.core.wmem_max = 67108864
net.core.rmem_default = 65536
net.core.wmem_default = 65536
net.core.netdev_max_backlog = 4096
net.core.somaxconn = 4096
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 1200
net.ipv4.ip_local_port_range = 10000 65000
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.tcp_max_tw_buckets = 5000
net.ipv4.tcp_rmem = 4096 87380 67108864
net.ipv4.tcp_wmem = 4096 65536 67108864
net.ipv4.tcp_mtu_probing = 1
net.ipv4.ip_forward = 1
net.ipv4.tcp_fastopen = 3
```

ダウンロード速度は13秒ほど。

`X-Session-Info: addr=45.32.*.*` なので、テザリング直回線(SSRを通してない)と異なっている模様。

```
$ time curl --socks5 127.0.0.1:1086 --verbose https://ipv4-c026-tyo001-ix.1.oca.nflxvideo.net/speedtest/range/0-26214400\?c\=jp\&n\=2497\&v\=3\&e\=1555939238\&t\=_bq6LeisT4EjUdtyFLHF4HoFb-U  > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 127.0.0.1...
* TCP_NODELAY set
* SOCKS5 communication to ipv4-c026-tyo001-ix.1.oca.nflxvideo.net:443
* SOCKS5 connect to IPv4 23.246.46.140 (locally resolved)
* SOCKS5 request granted.
* Connected to 127.0.0.1 (127.0.0.1) port 1086 (#0)
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [245 bytes data]
* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [108 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2798 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [300 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [37 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=California; L=Los Gatos; O=Netflix, Inc.; OU=Content Delivery; CN=*.1.oca.nflxvideo.net
*  start date: Apr 10 00:00:00 2019 GMT
*  expire date: May 12 12:00:00 2019 GMT
*  subjectAltName: host "ipv4-c026-tyo001-ix.1.oca.nflxvideo.net" matched cert's "*.1.oca.nflxvideo.net"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
> GET /speedtest/range/0-26214400?c=jp&n=2497&v=3&e=1555939238&t=_bq6LeisT4EjUdtyFLHF4HoFb-U HTTP/1.1
> Host: ipv4-c026-tyo001-ix.1.oca.nflxvideo.net
> User-Agent: curl/7.54.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Server: nginx
< Date: Mon, 22 Apr 2019 12:34:30 GMT
< Content-Type: application/octet-stream
< Content-Length: 26214400
< Last-Modified: Fri, 22 Mar 2019 06:12:59 GMT
< Connection: keep-alive
< Timing-Allow-Origin: *
< Cache-Control: no-store
< Pragma: no-cache
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: X-TCP-Info,X-Session-Info
< X-TCP-Info: h0=0;h1=0;h2=0;h3=0;h4=0;
< X-Session-Info: addr=45.32.*.*;port=49785;argp=
<
{ [16384 bytes data]
100 25.0M  100 25.0M    0     0  2040k      0  0:00:12  0:00:12 --:--:-- 2873k
* Connection #0 to host 127.0.0.1 left intact
noglob curl --socks5 127.0.0.1:1086 --verbose  > /dev/null  1.03s user 0.09s system 8% cpu 12.573 total
```

## BBR が有効でないカーネル + SSR

設定しているカーネルパラメータは以下。

```
$ sudo sysctl -p
net.ipv6.conf.all.accept_ra = 2
net.core.default_qdisc = pfifo_fast
net.ipv4.tcp_congestion_control = cubic
fs.file-max = 1024000
net.core.rmem_max = 67108864
net.core.wmem_max = 67108864
net.core.rmem_default = 65536
net.core.wmem_default = 65536
net.core.netdev_max_backlog = 4096
net.core.somaxconn = 4096
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 1200
net.ipv4.ip_local_port_range = 10000 65000
net.ipv4.tcp_max_syn_backlog = 4096
net.ipv4.tcp_max_tw_buckets = 5000
net.ipv4.tcp_rmem = 4096 87380 67108864
net.ipv4.tcp_wmem = 4096 65536 67108864
net.ipv4.tcp_mtu_probing = 1
net.ipv4.ip_forward = 1
net.ipv4.tcp_fastopen = 3
```

ダウンロード速度は1分47秒ほど。

`X-Session-Info: addr=45.32.*.*` なので、さきほどのSSRを通してる環境と一緒。

```
$ time curl --socks5 127.0.0.1:1086 --verbose https://ipv4-c026-tyo001-ix.1.oca.nflxvideo.net/speedtest/range/0-26214400\?c\=jp\&n\=2497\&v\=3\&e\=1555939238\&t\=_bq6LeisT4EjUdtyFLHF4HoFb-U  > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0*   Trying 127.0.0.1...
* TCP_NODELAY set
* SOCKS5 communication to ipv4-c026-tyo001-ix.1.oca.nflxvideo.net:443
* SOCKS5 connect to IPv4 23.246.46.140 (locally resolved)
* SOCKS5 request granted.
* Connected to 127.0.0.1 (127.0.0.1) port 1086 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* Cipher selection: ALL:!EXPORT:!EXPORT40:!EXPORT56:!aNULL:!LOW:!RC4:@STRENGTH
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/cert.pem
  CApath: none
* TLSv1.2 (OUT), TLS handshake, Client hello (1):
} [245 bytes data]
  0     0    0     0    0     0      0      0 --:--:--  0:00:01 --:--:--     0* TLSv1.2 (IN), TLS handshake, Server hello (2):
{ [108 bytes data]
* TLSv1.2 (IN), TLS handshake, Certificate (11):
{ [2798 bytes data]
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
{ [300 bytes data]
* TLSv1.2 (IN), TLS handshake, Server finished (14):
{ [4 bytes data]
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
} [37 bytes data]
* TLSv1.2 (OUT), TLS change cipher, Client hello (1):
} [1 bytes data]
* TLSv1.2 (OUT), TLS handshake, Finished (20):
} [16 bytes data]
* TLSv1.2 (IN), TLS change cipher, Client hello (1):
{ [1 bytes data]
* TLSv1.2 (IN), TLS handshake, Finished (20):
{ [16 bytes data]
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use http/1.1
* Server certificate:
*  subject: C=US; ST=California; L=Los Gatos; O=Netflix, Inc.; OU=Content Delivery; CN=*.1.oca.nflxvideo.net
*  start date: Apr 10 00:00:00 2019 GMT
*  expire date: May 12 12:00:00 2019 GMT
*  subjectAltName: host "ipv4-c026-tyo001-ix.1.oca.nflxvideo.net" matched cert's "*.1.oca.nflxvideo.net"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert SHA2 Secure Server CA
*  SSL certificate verify ok.
> GET /speedtest/range/0-26214400?c=jp&n=2497&v=3&e=1555939238&t=_bq6LeisT4EjUdtyFLHF4HoFb-U HTTP/1.1
> Host: ipv4-c026-tyo001-ix.1.oca.nflxvideo.net
> User-Agent: curl/7.54.0
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: nginx
< Date: Mon, 22 Apr 2019 12:44:50 GMT
< Content-Type: application/octet-stream
< Content-Length: 26214400
< Last-Modified: Fri, 22 Mar 2019 06:12:59 GMT
< Connection: keep-alive
< Timing-Allow-Origin: *
< Cache-Control: no-store
< Pragma: no-cache
< Access-Control-Allow-Origin: *
< Access-Control-Expose-Headers: X-TCP-Info,X-Session-Info
< X-TCP-Info: h0=0;h1=0;h2=0;h3=0;h4=0;
< X-Session-Info: addr=45.32.*.*;port=62591;argp=
< 
{ [16384 bytes data]
100 25.0M  100 25.0M    0     0   237k      0  0:01:47  0:01:47 --:--:--  248k
* Connection #0 to host 127.0.0.1 left intact
noglob curl --socks5 127.0.0.1:1086 --verbose  > /dev/null  1.65s user 0.27s system 1% cpu 1:47.63 total
```