Iperf & Seedbox speed testing

Many
 folks are unaware of the dramatic effect disk speeds have on there
network speeds, since the bulk of your data comes from or goes to disk,
and mass storage is the slowest link in the chain, it is the single most
 important factor in how fast you can download media home, download
torrent payloads, or even upload to others.

The chain is:

```
DISK <==> Cache/Memory <==> Network

```

When tuning a system these are the three components that garner the
most attention: how can I get things on to and off of my disk quickly;
how best to optimize my cache; and then finally how can I insure my
network is running at its peak.

In a previous posting
 we looked at how best diagnose route and peering issues using the linux
 command mtr. But mtr looks at road conditions, what is likely slowing
you down. It doesn't address how fast you can actually go. We can do
that with Iperf, and significantly iperf removes your disk from the mix.
 MTR lays out the map, the roads you are taking, any potholes, traffic
jams, bad interchanges, each turn and curve along the way. Iperf looks
at how fast your car can go down that route, pure speed, in real
numbers. MTR lays out the track, iperf runs the race.

*This is a tutorial, it is recommended you follow along using your server and your home machine to run the commands in parallel.*

Iperf, or its latest incarnation, iperf3, is a client/server tool.
You run iperf on your server, then from a client connect and see how
fast your server either receives or can send you packets. It is really
quite simple.

First we need to install iperf3 on your server, as always if you have
 superuser this is simple, if not a bit harder, with sudo (everything
here is debian or ubuntu):

```
 sudo apt-get install iperf3

```

If you don't have root you'll need to manually install it, here is what you need to do:

```
mkdir -p ~/bin ~/lib
wget --no-check-certificate -O ~/bin/iperf3 https://iperf.fr/download/ubuntu/iperf3\_3.1.3
chmod +x bin/iperf3
export PATH=$PATH:~/bin
wget --no-check-certificate -O ~/lib/libiperf.so.0 https://iperf.fr/download/ubuntu/libiperf.so.0\_3.1.3
echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:~/lib" >>~/.bashrc
source ~/.bashrc

```

Running things on the client is now easier than tying your shoes:

```
 iperf -p 10666 -s

```

10666 is the port that we will be using, the server will be listen
for connections on. It is optional, but since the default port might be
taken (5201), I've included how to set your own.

Now you need to install the client, a client for virtually any platform can be found over at iperf.fr, in particular https://iperf.fr/iperf-download.php

I'm going to assume you have windows, but it is largely the same for
any non-Unix client, you download the zip file, unzip it into its own
directory, then open a DOS or command window in that directory. With
Windows, iperf3 uses the cygwin runtime in the form of DLL(s).

Ok, with our server set up, and the client installed, a quick aside
for those unanointed in parlance of networking. The two most common
internet protocols are TCP (Transmission Control Protocol) and UDP (User
 Datagram Protocol). TCP is like a phone call, a circuit is created
between the caller and the person being called, they say something
you'll hear it unless the whole line drops, it is reliable. UDP is more
like sending a letter, you drop a packet on the wire with an address and
 best effort is made to get it delivered, it is inherently unreliable.
We'll be using both in our tests. Thing to understand is UDP it is
cheaper, and its unreliability can be useful.

Ok, lets start with the simplest case:

```
iperf3 -p 10666 -c server.hostname.or.ip

```

This will SEND packets to the server for 10 seconds, and tell you how fast the packets went, the output will look like:

```
 E:\mtrs\iperf-3.1.3-win64>iperf3 -p 10666 -c iperftest.chmuranet.com
 Connecting to host iperftest.chmuranet.com, port 10666
 [  4] local 192.168.1.19 port 10358 connected to .... port 10666
 [ ID] Interval           Transfer     Bandwidth
 [  4]   0.00-1.00   sec  1.00 MBytes  8.39 Mbits/sec
 [  4]   1.00-2.00   sec  2.25 MBytes  18.9 Mbits/sec
 [  4]   2.00-3.00   sec  2.25 MBytes  18.9 Mbits/sec
 [  4]   3.00-4.00   sec  2.25 MBytes  18.9 Mbits/sec
 [  4]   4.00-5.00   sec  2.25 MBytes  18.9 Mbits/sec
 [  4]   5.00-6.00   sec  2.25 MBytes  18.9 Mbits/sec
 [  4]   6.00-7.00   sec  2.25 MBytes  18.9 Mbits/sec
 [  4]   7.00-8.00   sec  2.25 MBytes  18.9 Mbits/sec
 [  4]   8.00-9.00   sec  2.25 MBytes  18.9 Mbits/sec
 [  4]   9.00-10.00  sec  2.25 MBytes  18.9 Mbits/sec
 - - - - - - - - - - - - - - - - - - - - - - - - -
 [ ID] Interval           Transfer     Bandwidth
 [  4]   0.00-10.00  sec  21.2 MBytes  17.8 Mbits/sec                  sender
 [  4]   0.00-10.00  sec  21.2 MBytes  17.8 Mbits/sec                  receiver

```

The last set of numbers (sender / receiver) is our summary line,
average speed. This is in megabits per second, so my upload speed is
2.23MB/s.

This though is just a test of how fast I can talk to my server, which
 isn't nearly as interesting as how fast my server can talk to me, to do
 that we need to tell iperf to receive. We add the **-R** parameter

```
 E:\mtrs\iperf-3.1.3-win64>iperf3 -p 10666 -c iperftest.chmuranet.com  -R

 [  4]   0.00-1.00   sec  1.62 MBytes  13.6 Mbits/sec
 [  4]   1.00-2.00   sec  1.65 MBytes  13.8 Mbits/sec
 [  4]   2.00-3.00   sec  1.07 MBytes  8.99 Mbits/sec
 [  4]   3.00-4.00   sec   814 KBytes  6.67 Mbits/sec
 [  4]   4.00-5.00   sec   510 KBytes  4.18 Mbits/sec
 [  4]   5.00-6.00   sec   446 KBytes  3.66 Mbits/sec
 [  4]   6.00-7.00   sec   205 KBytes  1.68 Mbits/sec
 [  4]   7.00-8.00   sec  99.8 KBytes   818 Kbits/sec
 [  4]   8.00-9.00   sec   141 KBytes  1.16 Mbits/sec
 [  4]   9.00-10.00  sec   345 KBytes  2.83 Mbits/sec
 - - - - - - - - - - - - - - - - - - - - - - - - -
 [ ID] Interval           Transfer     Bandwidth       Retr
 [  4]   0.00-10.00  sec  8.83 MBytes  7.41 Mbits/sec   22             sender
 [  4]   0.00-10.00  sec  6.97 MBytes  5.85 Mbits/sec                  receiver

```

Now this is interesting, my speed down is hitting Sunday night
traffic, and is significantly slower. See the number of retries (Retr)?
There is some problem on the down route, you can also see the jitter
(variations in latency or speed).

Now you should compare this to the speed your are getting on a single
 FTP connection using like Filezilla. Checking my connection, I'm
getting .5MB/s that means the disk is costing me 25% of my download
speed, not unusual. If the disk is re

To see how fast I can get things to, like segmented FTP, I'm going to
 need to use multiple simultaneous connections, lucky iperf3 can do
that too. Lets do 10 segments, with the **-P 10** parameter.

```
 E:\mtrs\iperf-3.1.3-win64>iperf3 -p 10666 -c iperftest.chmuranet.com -R -P 10

 [SUM]   0.00-10.00  sec  30.2 MBytes  25.3 Mbits/sec  239             sender
 [SUM]   0.00-10.00  sec  26.0 MBytes  21.8 Mbits/sec                  receiver

```

Just the summary lines, but you can see much faster. Lets try 30.

```
 [SUM]   0.00-10.00  sec  77.0 MBytes  64.6 Mbits/sec  947             sender
 [SUM]   0.00-10.00  sec  64.4 MBytes  54.0 Mbits/sec                  receiver

```

That scales rather nicely, I am receiving 6.75MB/s, but look at my
retransmits, that has scaled too. Server is sending at over 8MB/s.

At this point further escalation of connections, doesn't scale nearly as well.

OK, lets see what might be going on, we'll use UDP now (by adding the **-u** parameter), with 10 connections, from me to my server, :

```
 E:\mtrs\iperf-3.1.3-win64>iperf3 -p  10666 -c iperftest.chmuranet.com  -P 10 -u

 [SUM]   0.00-10.00  sec  15.6 MBytes  13.1 Mbits/sec  2.989 ms  0/1998 (0%)

```

No packet loss, of the 1998 packets sent 0% were lost, at a speed of 13.1Mbits/s - pretty good.

Now going the other way (-R):

```
 E:\mtrs\iperf-3.1.3-win64>iperf3 -p  10666 -c iperftest.chmuranet.com  -P 10 -u -R

 [SUM]   0.00-10.00  sec  12.7 MBytes  10.6 Mbits/sec  1.192 ms  9/1620 (0.56%)

```

Fewer packets sent and an overall loss of .56% of the packets
wandered off. I'm willing to bet if I did an mtr, I'd see some bad
jitter, and probably some packet loss.

Yep, checking mtr, we see where the problem is:

```
 4. ae-21-52.car1.Chicago1.Level3.net            1.9%    340   95.5  95.6  95.4  97.0   23

```

1.9% packet loss in Chicago, those ruffians.

So you can see, iperf can be an effective tool, primarily for
determining the hit you are taking because of your hard disk, but also
in analyzing route dynamics, what is costing you against your ideal.
I've covered most of the interesting parameters, the only other flag I
use is **-t** as in -t 30, so that iperf will run for
longer (30 secs vs 10sec default), which can give you a better picture.
But with all linux commands, you should take a look at the manpage
yourself: https://www.mankier.com/1/iperf3

*Disclaimer: I fiddled with the numbers to make this more
interesting, I also ran a laptop against a Chmuranet 10G server, a DC
based CPU-laden 10G server can send packets much faster then my poor
laptop can handle, this is part of the packet loss you saw in my
examples. Consider this when looking at your own server.*