Project: CubeFS (Runtime / Cloud Native Storage)
File: io.md

The results of IO performance testing using are as follows: ::: tip Note Multiple clients mount the same volume, and the process refers to the fio process. ::: Tool Settings ``` bash fio -directory={} \ -ioengine=psync \ -rw=read \ # sequential read -bs=128k \ # block size -direct=1 \ # enable direct IO -group_reporting=1 \ -fallocate=none \ -time_based=1 \ -runtime=120 \ -name=testfilec{} \ -numjobs={} \ -nrfiles=1 \ -size=10G ``` Bandwidth (MB/s) | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 319.000 | 1145.000 | 3496.000 | 2747.000 | | 2 Clients | 625.000 | 2237.000 | 6556.000 | 5300.000 | | 4 Clients | 1326.000 | 4433.000 | 8979.000 | 9713.000 | | 8 Clients | 2471.000 | 7963.000 | 11878.400 | 17510.400 | IOPS | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 2552 | 9158 | 27000 | 21000 | | 2 Clients | 5003 | 17900 | 52400 | 42400 | | 4 Clients | 10600 | 35500 | 71800 | 77700 | | 8 Clients | 19800 | 63700 | 94700 | 140000 | Latency (Microsecond) | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 391.350 | 436.170 | 571.200 | 2910.960 | | 2 Clients | 404.030 | 459.330 | 602.270 | 3011.920 | | 4 Clients | 374.450 | 445.550 | 892.390 | 2948.990 | | 8 Clients | 404.530 | 503.590 | 1353.910 | 4160.620 | Tool Settings ``` bash fio -directory={} \ -ioengine=psync \ -rw=write \ # sequential write -bs=128k \ # block size -direct=1 \ # enable direct IO -group_reporting=1 \ -fallocate=none \ -name=testfilec{} \ -numjobs={} \ -nrfiles=1 \ -size=10G ``` Bandwidth (MB/s) | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 119.000 | 473.000 | 1618.000 | 2903.000 | | 2 Clients | 203.000 | 886.000 | 2917.000 | 5465.000 | | 4 Clients | 397.000 | 1691.000 | 4708.000 | 7256.000 | | 8 Clients | 685.000 | 2648.000 | 6257.000 | 7166.000 | IOPS | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 948 | 3783 | 12900 | 23200 | | 2 Clients | 1625 | 7087 | 23300 | 43700 | | 4 Clients | 3179 | 13500 | 37700 | 58000 | | 8 Clients | 5482 | 21200 | 50100 | 57300 | Latency (Microsecond) | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 1053.240 | 1051.450 | 1228.230 | 2745.800 | | 2 Clients | 1229.270 | 1109.490 | 1359.350 | 2893.780 | | 4 Clients | 1248.990 | 1164.050 | 1642.660 |

| | 8 Clients | 1316.560 | 1357.940 | 2378.950 | 8040.250 | Tool Settings ``` bash fio -directory={} \ -ioengine=psync \ -rw=randread \ # random read -bs=4k \ # block size -direct=1 \ # enable direct IO -group_reporting=1 \ -fallocate=none \ -time_based=1 \ -runtime=120 \ -name=testfilec{} \ -numjobs={} \ -nrfiles=1 \ -size=10G ``` Bandwidth (MB/s) | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 15.500 | 76.300 | 307.000 | 496.000 | | 2 Clients | 32.600 | 161.000 | 587.000 | 926.000 | | 4 Clients | 74.400 | 340.000 | 1088.000 | 1775.000 | | 8 Clients | 157.000 | 628.000 | 1723.000 | 2975.000 | IOPS | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 3979 | 19500 | 78700 | 127000 | | 2 Clients | 8345 | 41300 | 150000 | 237000 | | 4 Clients | 19000 | 86000 | 278000 | 454000 | | 8 Clients | 40200 | 161000 | 441000 | 762000 | Latency (Microsecond) | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 250.720 | 203.960 | 202.480 | 502.940 | | 2 Clients | 250.990 | 204.100 | 219.750 | 558.010 | | 4 Clients | 211.240 | 180.720 | 226.840 | 551.470 | | 8 Clients | 192.660 | 196.560 | 288.090 | 691.920 | Tool Settings ``` bash fio -directory={} \ -ioengine=psync \ -rw=randwrite \ # random write -bs=4k \ # block size -direct=1 \ # enable direct IO -group_reporting=1 \ -fallocate=none \ -time_based=1 \ -runtime=120 \ -name=testfilec{} \ -numjobs={} \ -nrfiles=1 \ -size=10G ``` Bandwidth (MB/s) | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 7.743 | 43.500 | 164.000 | 429.000 | | 2 Clients | 19.700 | 84.300 | 307.000 | 679.000 | | 4 Clients | 41.900 | 167.000 | 480.000 | 877.000 | | 8 Clients | 82.600 | 305.000 | 700.000 | 830.000 | IOPS | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 1982 | 11100 | 42100 | 110000 | | 2 Clients | 5050 | 21600 | 78600 | 174000 | | 4 Clients | 10700 | 42800 | 123000 | 225000 | | 8 Clients | 1100 | 78100 | 179000 | 212000 | Latency (Microsecond) | | 1 Process | 4 Processes | 16 Processes | 64 Processes | |--|--|-|--|--| | 1 Client | 503.760 | 358.190 | 379.110 | 580.970 | | 2 Clients | 400.150 | 374.010 | 412.900 | 751.020 | | 4 Clients | 371.620 | 370.520 | 516.930 | 1139.920 | | 8 Clients | 380.650 | 403.510 | 718.900 | 2409.250 |