IBM Cloud: BlockStorage性能評価

# 1. はじめに

IBM Cloud Classic Infrastructureでは、`10 IOPS/GB x 4000 GB = 40000 IOPS`のBlock Storage(iSCSI)が利用可能である。

本稿では、一世代前のBMSサーバー(Intel Xeon-Skylake (4110-SILVER))およびOS(Windows Server 2016)であっても、NICが10Gbpsさえあれば、問題なくこの性能が出ることを確認したい。

なお、このIOPS性能は、[ここ](https://cloud.ibm.com/docs/BlockStorage?topic=BlockStorage-block-storage-faqs#iopsmeasure)にも以下のように説明されているように、16KBブロックで、read/writeが50%のランダムアクセスであることを前提としている。また、あくまでストレージそのものが提供するIOPS性能であり、例えばストレージにアクセスするサーバーのOS設定やNICスピードがボトルネックになる可能性がある。

>IOPS is measured based on a load profile of 16-KB blocks with random 50 percent read and 50 percent writes. Workloads that differ from this profile can experience inferior performance. 

# 2. HW/Storage構成

- サーバー環境(public側のNICはセキュリティーの都合上、停止）
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/c7a14a32-74c5-533c-aed7-a27cc6b94982.png)

- ストレージ環境(40000 IOPS Block Storage)
![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/74f1a901-bac4-758d-986f-0d9dfa55f207.png)

- OS設定

```txt:systeminfo
C:\Users\Administrator>systeminfo

Host Name:                 BMSBLK01
OS Name:                   Microsoft Windows Server 2016 Standard
OS Version:                10.0.14393 N/A Build 14393
OS Manufacturer:           Microsoft Corporation
OS Configuration:          Standalone Server
OS Build Type:             Multiprocessor Free
Registered Owner:          NetworkLayer
Registered Organization:   NetworkLayer
Product ID:                00377-60000-00000-AA934
Original Install Date:     3/18/2024, 12:44:32 PM
System Boot Time:          3/19/2024, 12:31:05 PM
System Manufacturer:       Supermicro
System Model:              SYS-6029U-E1CR4T
System Type:               x64-based PC
Processor(s):              2 Processor(s) Installed.
                           [01]: Intel64 Family 6 Model 85 Stepping 4 GenuineIntel ~700 Mhz
                           [02]: Intel64 Family 6 Model 85 Stepping 4 GenuineIntel ~700 Mhz
BIOS Version:              American Megatrends Inc. 4.0, 8/11/2023
Windows Directory:         C:\windows
System Directory:          C:\windows\system32
Boot Device:               \Device\HarddiskVolume1
System Locale:             en-us;English (United States)
Input Locale:              en-us;English (United States)
Time Zone:                 (UTC+09:00) Osaka, Sapporo, Tokyo
Total Physical Memory:     65,187 MB
Available Physical Memory: 63,003 MB
Virtual Memory: Max Size:  74,915 MB
Virtual Memory: Available: 72,808 MB
Virtual Memory: In Use:    2,107 MB
Page File Location(s):     C:\pagefile.sys
Domain:                    WORKGROUP
Logon Server:              \\BMSBLK01
Hotfix(s):                 13 Hotfix(s) Installed.
                           [01]: KB4514354
                           [02]: KB3192137
                           [03]: KB4013418
                           [04]: KB4091664
                           [05]: KB4132216
                           [06]: KB4346087
                           [07]: KB4494175
                           [08]: KB4516115
                           [09]: KB4520724
                           [10]: KB4521858
                           [11]: KB4589210
                           [12]: KB5035962
                           [13]: KB4343887
Network Card(s):           6 NIC(s) Installed.
                           [01]: Intel(R) Ethernet Controller X710/X557-AT 10GBASE-T
                                 Connection Name: PublicNetwork-B
                                 Status:          Media disconnected
                           [02]: Intel(R) Ethernet Controller X710/X557-AT 10GBASE-T
                                 Connection Name: PrivateNetwork-A
                                 DHCP Enabled:    No
                                 IP address(es)
                           [03]: Intel(R) Ethernet Controller X710/X557-AT 10GBASE-T
                                 Connection Name: PrivateNetwork-B
                                 DHCP Enabled:    Yes
                                 DHCP Server:     N/A
                                 IP address(es)
                           [04]: Intel(R) Ethernet Controller X710/X557-AT 10GBASE-T
                                 Connection Name: PublicNetwork-A
                                 Status:          Media disconnected
                           [05]: Microsoft Network Adapter Multiplexor Driver
                                 Connection Name: PublicNetwork-Teamed
                                 Status:          Media disconnected
                           [06]: Microsoft Network Adapter Multiplexor Driver
                                 Connection Name: PrivateNetwork-Teamed
                                 DHCP Enabled:    No
                                 IP address(es)
                                 [01]: 10.192.27.6
                                 [02]: fe80::7852:47a:cd6f:6a08
Hyper-V Requirements:      VM Monitor Mode Extensions: Yes
                           Virtualization Enabled In Firmware: Yes
                           Second Level Address Translation: Yes
                           Data Execution Prevention Available: Yes
```
```txt:NIC情報
PS C:\Users\Administrator> Get-NetAdapter

Name                      InterfaceDescription                    ifIndex Status       MacAddress             LinkSpeed
----                      --------------------                    ------- ------       ----------             ---------
PublicNetwork-Teamed      Microsoft Network Adapter Multiplexo...      13 Disconnected 3C-EC-EF-1C-4B-4F          0 bps
PublicNetwork-B           Intel(R) Ethernet Controller X710/X5...       7 Disconnected 3C-EC-EF-1C-4B-51        10 Gbps
PrivateNetwork-B          Intel(R) Ethernet Controller X710/...#3      12 Up           3C-EC-EF-1C-4B-50        10 Gbps
PublicNetwork-A           Intel(R) Ethernet Controller X710/...#4       4 Disconnected 3C-EC-EF-1C-4B-4F        10 Gbps
PrivateNetwork-Teamed     Microsoft Network Adapter Multiple...#2       6 Up           3C-EC-EF-1C-4B-4E        20 Gbps
PrivateNetwork-A          Intel(R) Ethernet Controller X710/...#2       9 Up           3C-EC-EF-1C-4B-4E        10 Gbps
```


- Windows Firewall
    - デフォルトのまま、停止した状態である。（有効化しても大きな差は見られなかったが）![image.png](https://qiita-image-store.s3.ap-northeast-1.amazonaws.com/0/62127/ee91f1c0-2120-60c0-c3dc-9815fd868320.png)

- iSCSI構成
    - [IBM Cloud docsによる構成方法](https://cloud.ibm.com/docs/BlockStorage?topic=BlockStorage-mountingWindows&interface=ui)に従ってBlock StorageをMPIO冗長化パスでマウント。特別な追加構成はしていない。

- JumboFrameを有効にするなどの設定はしていない。

# 3. 計測ツール

- [fio](https://github.com/axboe/fio/releases)を利用。
- 本計測のタイミングで最新の[fio-3.36-x64.msi](https://github.com/axboe/fio/releases/download/fio-3.36/fio-3.36-x64.msi)を利用。

# 4. 性能計測

## 4-1. ブロックサイズ＝4K

```
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=4k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 4096B-4096B, (W) 4096B-4096B, (T) 4096B-4096B, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=1): [m(1)][100.0%][r=78.9MiB/s,w=80.6MiB/s][r=20.2k,w=20.6k IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=4608: Tue Mar 19 14:21:55 2024
  read: IOPS=20.5k, BW=80.1MiB/s (84.0MB/s)(9608MiB/120002msec)
    slat (usec): min=4, max=492, avg=14.37, stdev=11.85
    clat (usec): min=31, max=46403, avg=1282.48, stdev=1150.95
     lat (usec): min=272, max=46418, avg=1296.85, stdev=1150.35
    clat percentiles (usec):
     |  1.00th=[  562],  5.00th=[  701], 10.00th=[  783], 20.00th=[  889],
     | 30.00th=[  971], 40.00th=[ 1045], 50.00th=[ 1106], 60.00th=[ 1188],
     | 70.00th=[ 1270], 80.00th=[ 1369], 90.00th=[ 1549], 95.00th=[ 1778],
     | 99.00th=[ 9372], 99.50th=[10552], 99.90th=[11207], 99.95th=[11469],
     | 99.99th=[11994]
   bw (  KiB/s): min=68673, max=95090, per=100.00%, avg=82075.05, stdev=1655.15, samples=238
   iops        : min=17168, max=23772, avg=20518.49, stdev=413.79, samples=238
  write: IOPS=20.5k, BW=80.0MiB/s (83.9MB/s)(9601MiB/120002msec); 0 zone resets
    slat (usec): min=5, max=572, avg=21.27, stdev=13.40
    clat (usec): min=59, max=13843, avg=1367.62, stdev=1173.36
     lat (usec): min=331, max=13858, avg=1388.90, stdev=1172.33
    clat percentiles (usec):
     |  1.00th=[  586],  5.00th=[  734], 10.00th=[  832], 20.00th=[  963],
     | 30.00th=[ 1057], 40.00th=[ 1139], 50.00th=[ 1205], 60.00th=[ 1270],
     | 70.00th=[ 1352], 80.00th=[ 1450], 90.00th=[ 1631], 95.00th=[ 1844],
     | 99.00th=[ 9634], 99.50th=[10945], 99.90th=[11731], 99.95th=[11994],
     | 99.99th=[12387]
   bw (  KiB/s): min=68593, max=95098, per=100.00%, avg=82014.73, stdev=1673.36, samples=238
   iops        : min=17148, max=23774, avg=20503.42, stdev=418.34, samples=238
  lat (usec)   : 50=0.01%, 100=0.01%, 250=0.01%, 500=0.27%, 750=6.56%
  lat (usec)   : 1000=22.29%
  lat (msec)   : 2=67.61%, 4=1.29%, 10=1.11%, 20=0.88%, 50=0.01%
  cpu          : usr=7.50%, sys=62.50%, ctx=0, majf=0, minf=0
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=5.8%, 32=81.4%, >=64=12.6%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=94.8%, 8=2.3%, 16=1.5%, 32=0.8%, 64=0.6%, >=64=0.0%
     issued rwts: total=2459575,2457919,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=80.1MiB/s (84.0MB/s), 80.1MiB/s-80.1MiB/s (84.0MB/s-84.0MB/s), io=9608MiB (10.1GB), run=120002-120002msec
  WRITE: bw=80.0MiB/s (83.9MB/s), 80.0MiB/s-80.0MiB/s (83.9MB/s-83.9MB/s), io=9601MiB (10.1GB), run=120002-120002msec
```

## 4-2. ブロックサイズ＝16K

```
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=16k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 16.0KiB-16.0KiB, (W) 16.0KiB-16.0KiB, (T) 16.0KiB-16.0KiB, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=1): [m(1)][100.0%][r=314MiB/s,w=320MiB/s][r=20.1k,w=20.5k IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=1212: Tue Mar 19 12:37:34 2024
  read: IOPS=20.5k, BW=320MiB/s (336MB/s)(37.5GiB/120001msec)
    slat (usec): min=4, max=389, avg=14.37, stdev=10.80
    clat (usec): min=137, max=14469, avg=1263.95, stdev=948.27
     lat (usec): min=278, max=14484, avg=1278.32, stdev=947.96
    clat percentiles (usec):
     |  1.00th=[  586],  5.00th=[  742], 10.00th=[  824], 20.00th=[  930],
     | 30.00th=[ 1004], 40.00th=[ 1074], 50.00th=[ 1139], 60.00th=[ 1205],
     | 70.00th=[ 1270], 80.00th=[ 1385], 90.00th=[ 1565], 95.00th=[ 1795],
     | 99.00th=[ 6325], 99.50th=[10421], 99.90th=[11207], 99.95th=[11338],
     | 99.99th=[11731]
   bw (  KiB/s): min=313809, max=353045, per=100.00%, avg=328459.27, stdev=5064.91, samples=238
   iops        : min=19613, max=22065, avg=20528.25, stdev=316.52, samples=238
  write: IOPS=20.5k, BW=320MiB/s (336MB/s)(37.5GiB/120001msec); 0 zone resets
    slat (usec): min=6, max=421, avg=22.98, stdev=13.28
    clat (usec): min=108, max=14596, avg=1419.20, stdev=1018.25
     lat (usec): min=371, max=14614, avg=1442.19, stdev=1017.53
    clat percentiles (usec):
     |  1.00th=[  635],  5.00th=[  816], 10.00th=[  922], 20.00th=[ 1045],
     | 30.00th=[ 1139], 40.00th=[ 1205], 50.00th=[ 1287], 60.00th=[ 1352],
     | 70.00th=[ 1434], 80.00th=[ 1549], 90.00th=[ 1762], 95.00th=[ 2040],
     | 99.00th=[ 6915], 99.50th=[10945], 99.90th=[11994], 99.95th=[12256],
     | 99.99th=[12649]
   bw (  KiB/s): min=312396, max=346170, per=100.00%, avg=328204.25, stdev=4995.11, samples=238
   iops        : min=19524, max=21635, avg=20512.34, stdev=312.14, samples=238
  lat (usec)   : 250=0.01%, 500=0.20%, 750=4.08%, 1000=18.31%
  lat (msec)   : 2=73.06%, 4=2.99%, 10=0.72%, 20=0.64%
  cpu          : usr=10.00%, sys=61.67%, ctx=0, majf=0, minf=0
  IO depths    : 1=0.0%, 2=0.1%, 4=0.1%, 8=0.2%, 16=4.6%, 32=79.4%, >=64=15.8%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=94.8%, 8=2.5%, 16=1.5%, 32=0.7%, 64=0.5%, >=64=0.0%
     issued rwts: total=2460316,2458563,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=320MiB/s (336MB/s), 320MiB/s-320MiB/s (336MB/s-336MB/s), io=37.5GiB (40.3GB), run=120001-120001msec
  WRITE: bw=320MiB/s (336MB/s), 320MiB/s-320MiB/s (336MB/s-336MB/s), io=37.5GiB (40.3GB), run=120001-120001msec
```

## 4-3. ブロックサイズ＝32K

```
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=32k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 32.0KiB-32.0KiB, (W) 32.0KiB-32.0KiB, (T) 32.0KiB-32.0KiB, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=1): [m(1)][100.0%][r=658MiB/s,w=646MiB/s][r=21.0k,w=20.7k IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=6300: Wed Mar 20 09:19:57 2024
  read: IOPS=20.3k, BW=635MiB/s (665MB/s)(74.4GiB/120002msec)
    slat (usec): min=4, max=766, avg=11.72, stdev= 9.50
    clat (usec): min=348, max=23848, avg=1425.23, stdev=709.11
     lat (usec): min=358, max=23887, avg=1436.95, stdev=709.18
    clat percentiles (usec):
     |  1.00th=[  709],  5.00th=[  848], 10.00th=[  930], 20.00th=[ 1037],
     | 30.00th=[ 1106], 40.00th=[ 1188], 50.00th=[ 1254], 60.00th=[ 1352],
     | 70.00th=[ 1483], 80.00th=[ 1663], 90.00th=[ 2024], 95.00th=[ 2474],
     | 99.00th=[ 3884], 99.50th=[ 5145], 99.90th=[10552], 99.95th=[11076],
     | 99.99th=[11600]
   bw (  KiB/s): min=376432, max=741835, per=100.00%, avg=650553.12, stdev=41538.21, samples=235
   iops        : min=11763, max=23182, avg=20329.33, stdev=1298.08, samples=235
  write: IOPS=20.3k, BW=634MiB/s (665MB/s)(74.3GiB/120002msec); 0 zone resets
    slat (usec): min=5, max=1314, avg=18.50, stdev=11.47
    clat (usec): min=344, max=24120, avg=1502.18, stdev=734.82
     lat (usec): min=380, max=24138, avg=1520.68, stdev=734.69
    clat percentiles (usec):
     |  1.00th=[  734],  5.00th=[  881], 10.00th=[  979], 20.00th=[ 1090],
     | 30.00th=[ 1188], 40.00th=[ 1270], 50.00th=[ 1352], 60.00th=[ 1450],
     | 70.00th=[ 1565], 80.00th=[ 1762], 90.00th=[ 2114], 95.00th=[ 2540],
     | 99.00th=[ 3982], 99.50th=[ 5211], 99.90th=[11076], 99.95th=[11731],
     | 99.99th=[12649]
   bw (  KiB/s): min=374380, max=742541, per=100.00%, avg=649893.56, stdev=41961.83, samples=235
   iops        : min=11699, max=23204, avg=20308.71, stdev=1311.26, samples=235
  lat (usec)   : 500=0.01%, 750=1.52%, 1000=12.40%
  lat (msec)   : 2=74.72%, 4=10.40%, 10=0.81%, 20=0.14%, 50=0.01%
  cpu          : usr=9.17%, sys=50.00%, ctx=0, majf=0, minf=0
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.1%, 16=0.9%, 32=77.6%, >=64=21.3%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=94.1%, 8=3.5%, 16=1.7%, 32=0.6%, 64=0.1%, >=64=0.0%
     issued rwts: total=2437066,2434644,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=635MiB/s (665MB/s), 635MiB/s-635MiB/s (665MB/s-665MB/s), io=74.4GiB (79.9GB), run=120002-120002msec
  WRITE: bw=634MiB/s (665MB/s), 634MiB/s-634MiB/s (665MB/s-665MB/s), io=74.3GiB (79.8GB), run=120002-120002msec
```

## 4-4. ブロックサイズ＝64K

```
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=64k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 64.0KiB-64.0KiB, (W) 64.0KiB-64.0KiB, (T) 64.0KiB-64.0KiB, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=1): [m(1)][100.0%][r=549MiB/s,w=543MiB/s][r=8787,w=8682 IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=4468: Tue Mar 19 14:29:06 2024
  read: IOPS=8677, BW=542MiB/s (569MB/s)(63.6GiB/120003msec)
    slat (usec): min=7, max=527, avg=20.45, stdev=11.56
    clat (usec): min=488, max=16740, avg=3601.53, stdev=1277.24
     lat (usec): min=503, max=16756, avg=3621.98, stdev=1278.90
    clat percentiles (usec):
     |  1.00th=[ 1975],  5.00th=[ 2311], 10.00th=[ 2507], 20.00th=[ 2737],
     | 30.00th=[ 2933], 40.00th=[ 3130], 50.00th=[ 3326], 60.00th=[ 3523],
     | 70.00th=[ 3818], 80.00th=[ 4228], 90.00th=[ 4948], 95.00th=[ 5800],
     | 99.00th=[ 8717], 99.50th=[10421], 99.90th=[13435], 99.95th=[14222],
     | 99.99th=[15401]
   bw (  KiB/s): min=431488, max=741707, per=100.00%, avg=555877.32, stdev=57805.44, samples=238
   iops        : min= 6742, max=11589, avg=8685.26, stdev=903.20, samples=238
  write: IOPS=8671, BW=542MiB/s (568MB/s)(63.5GiB/120003msec); 0 zone resets
    slat (usec): min=8, max=500, avg=25.30, stdev=13.35
    clat (usec): min=551, max=17566, avg=3677.17, stdev=1308.41
     lat (usec): min=575, max=17582, avg=3702.47, stdev=1310.26
    clat percentiles (usec):
     |  1.00th=[ 2024],  5.00th=[ 2376], 10.00th=[ 2540], 20.00th=[ 2802],
     | 30.00th=[ 2999], 40.00th=[ 3163], 50.00th=[ 3392], 60.00th=[ 3621],
     | 70.00th=[ 3884], 80.00th=[ 4293], 90.00th=[ 5080], 95.00th=[ 5932],
     | 99.00th=[ 8979], 99.50th=[10683], 99.90th=[13566], 99.95th=[14353],
     | 99.99th=[15533]
   bw (  KiB/s): min=419014, max=754276, per=100.00%, avg=555510.02, stdev=58003.07, samples=238
   iops        : min= 6547, max=11785, avg=8679.45, stdev=906.30, samples=238
  lat (usec)   : 500=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=1.01%, 4=73.00%, 10=25.34%, 20=0.64%
  cpu          : usr=7.50%, sys=34.17%, ctx=0, majf=0, minf=0
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.1%, 32=50.1%, >=64=49.9%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=97.9%, 8=1.9%, 16=0.2%, 32=0.1%, 64=0.1%, >=64=0.0%
     issued rwts: total=1041329,1040585,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=542MiB/s (569MB/s), 542MiB/s-542MiB/s (569MB/s-569MB/s), io=63.6GiB (68.2GB), run=120003-120003msec
  WRITE: bw=542MiB/s (568MB/s), 542MiB/s-542MiB/s (568MB/s-568MB/s), io=63.5GiB (68.2GB), run=120003-120003msec
```

## 4-5. ブロックサイズ＝128K

```
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=128k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 128KiB-128KiB, (W) 128KiB-128KiB, (T) 128KiB-128KiB, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=0): [f(1)][100.0%][r=722MiB/s,w=734MiB/s][r=5774,w=5875 IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=6296: Tue Mar 19 14:32:26 2024
  read: IOPS=5850, BW=731MiB/s (767MB/s)(85.7GiB/120007msec)
    slat (usec): min=7, max=376, avg=30.32, stdev=16.32
    clat (usec): min=612, max=286636, avg=3865.61, stdev=5240.34
     lat (usec): min=657, max=286656, avg=3895.93, stdev=5240.64
    clat percentiles (msec):
     |  1.00th=[    3],  5.00th=[    3], 10.00th=[    3], 20.00th=[    3],
     | 30.00th=[    4], 40.00th=[    4], 50.00th=[    4], 60.00th=[    4],
     | 70.00th=[    5], 80.00th=[    5], 90.00th=[    5], 95.00th=[    6],
     | 99.00th=[    9], 99.50th=[   10], 99.90th=[   17], 99.95th=[   37],
     | 99.99th=[  239]
   bw (  KiB/s): min=31367, max=995014, per=100.00%, avg=753353.26, stdev=153117.92, samples=237
   iops        : min=  245, max= 7773, avg=5885.07, stdev=1196.23, samples=237
  write: IOPS=5842, BW=730MiB/s (766MB/s)(85.6GiB/120007msec); 0 zone resets
    slat (usec): min=9, max=752, avg=48.69, stdev=21.44
    clat (usec): min=1346, max=1150.9k, avg=6910.57, stdev=13019.43
     lat (usec): min=1392, max=1151.0k, avg=6959.26, stdev=13019.81
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    5], 20.00th=[    6],
     | 30.00th=[    6], 40.00th=[    6], 50.00th=[    7], 60.00th=[    7],
     | 70.00th=[    7], 80.00th=[    8], 90.00th=[    9], 95.00th=[   10],
     | 99.00th=[   15], 99.50th=[   18], 99.90th=[   41], 99.95th=[  241],
     | 99.99th=[  300]
   bw (  KiB/s): min=13154, max=991422, per=100.00%, avg=752364.50, stdev=153225.64, samples=237
   iops        : min=  102, max= 7745, avg=5877.38, stdev=1197.10, samples=237
  lat (usec)   : 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.38%, 4=35.36%, 10=61.65%, 20=2.40%, 50=0.14%
  lat (msec)   : 100=0.01%, 250=0.05%, 500=0.01%, 2000=0.01%
  cpu          : usr=11.67%, sys=33.33%, ctx=0, majf=0, minf=0
  IO depths    : 1=0.0%, 2=0.0%, 4=0.0%, 8=0.0%, 16=0.1%, 32=46.3%, >=64=53.7%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=97.9%, 8=1.7%, 16=0.3%, 32=0.1%, 64=0.1%, >=64=0.0%
     issued rwts: total=702102,701160,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=731MiB/s (767MB/s), 731MiB/s-731MiB/s (767MB/s-767MB/s), io=85.7GiB (92.0GB), run=120007-120007msec
  WRITE: bw=730MiB/s (766MB/s), 730MiB/s-730MiB/s (766MB/s-766MB/s), io=85.6GiB (91.9GB), run=120007-120007msec
```

## 4-6. ブロックサイズ＝256K

```
D:\>fio.exe --randrepeat=1 --ioengine=windowsaio --direct=1 --sync=1 --name=fiotest --filename=fio-test-file.fio --overwrite=1 --iodepth=64 --size=5GB --readwrite=randrw --rwmixwrite=50 --rwmixread=50 --bs=256k --time_based --thread --runtime=120 --ramp_time=15
fiotest: (g=0): rw=randrw, bs=(R) 256KiB-256KiB, (W) 256KiB-256KiB, (T) 256KiB-256KiB, ioengine=windowsaio, iodepth=64
fio-3.36
Starting 1 thread
Jobs: 1 (f=1): [m(1)][100.0%][r=588MiB/s,w=604MiB/s][r=2352,w=2417 IOPS][eta 00m:00s]
fiotest: (groupid=0, jobs=1): err= 0: pid=6844: Tue Mar 19 13:00:27 2024
  read: IOPS=2764, BW=691MiB/s (725MB/s)(81.0GiB/120012msec)
    slat (usec): min=9, max=7987, avg=55.79, stdev=179.18
    clat (usec): min=3, max=249740, avg=8276.41, stdev=5380.75
     lat (usec): min=847, max=249752, avg=8332.20, stdev=5379.98
    clat percentiles (msec):
     |  1.00th=[    4],  5.00th=[    5], 10.00th=[    6], 20.00th=[    7],
     | 30.00th=[    7], 40.00th=[    8], 50.00th=[    8], 60.00th=[    9],
     | 70.00th=[    9], 80.00th=[   10], 90.00th=[   12], 95.00th=[   14],
     | 99.00th=[   19], 99.50th=[   22], 99.90th=[   33], 99.95th=[   47],
     | 99.99th=[  245]
   bw (  KiB/s): min=300544, max=964535, per=100.00%, avg=708659.40, stdev=118761.43, samples=234
   iops        : min= 1174, max= 3767, avg=2767.88, stdev=463.87, samples=234
  write: IOPS=2760, BW=690MiB/s (724MB/s)(80.9GiB/120012msec); 0 zone resets
    slat (usec): min=13, max=11597, avg=91.76, stdev=214.39
    clat (usec): min=1113, max=268285, avg=13994.41, stdev=7325.54
     lat (usec): min=1533, max=268401, avg=14086.16, stdev=7322.05
    clat percentiles (msec):
     |  1.00th=[    6],  5.00th=[    9], 10.00th=[   10], 20.00th=[   11],
     | 30.00th=[   12], 40.00th=[   13], 50.00th=[   14], 60.00th=[   14],
     | 70.00th=[   15], 80.00th=[   17], 90.00th=[   19], 95.00th=[   22],
     | 99.00th=[   32], 99.50th=[   38], 99.90th=[   52], 99.95th=[  245],
     | 99.99th=[  257]
   bw (  KiB/s): min=265234, max=1014929, per=100.00%, avg=707990.54, stdev=123579.24, samples=234
   iops        : min= 1036, max= 3964, avg=2765.26, stdev=482.73, samples=234
  lat (usec)   : 4=0.01%, 10=0.01%, 250=0.01%, 750=0.01%, 1000=0.01%
  lat (msec)   : 2=0.04%, 4=1.01%, 10=46.54%, 20=48.67%, 50=3.67%
  lat (msec)   : 100=0.03%, 250=0.03%, 500=0.01%
  cpu          : usr=5.83%, sys=10.83%, ctx=0, majf=0, minf=0
  IO depths    : 1=0.1%, 2=0.1%, 4=0.1%, 8=0.2%, 16=1.5%, 32=57.9%, >=64=40.4%
     submit    : 0=0.0%, 4=100.0%, 8=0.0%, 16=0.0%, 32=0.0%, 64=0.0%, >=64=0.0%
     complete  : 0=0.0%, 4=96.8%, 8=1.9%, 16=0.9%, 32=0.3%, 64=0.1%, >=64=0.0%
     issued rwts: total=331763,331313,0,0 short=0,0,0,0 dropped=0,0,0,0
     latency   : target=0, window=0, percentile=100.00%, depth=64

Run status group 0 (all jobs):
   READ: bw=691MiB/s (725MB/s), 691MiB/s-691MiB/s (725MB/s-725MB/s), io=81.0GiB (87.0GB), run=120012-120012msec
  WRITE: bw=690MiB/s (724MB/s), 690MiB/s-690MiB/s (724MB/s-724MB/s), io=80.9GiB (86.9GB), run=120012-120012msec
```

# 5. まとめ

以上の結果より、4K~32Kブロックにおいて、特にチューニングなしで40000 IOPS性能(read 20000 IOPS, write 20000 IOPS)が出ていることが確認できた。