zfsonlinux/zfs — pull request #1465 by b333z (2013-05-17 12:48:25+00:00)
Title: Balance vdev mirror reads off pending queue length

When selecting which vdev mirror child to read from,
redirect io's to child vdevs with the smallest pending
queue lengths.

During the vdev selection process if a readable vdev is found
with a queue length of 0 then the vdev will be used and the
above process will be cut short.

It is hoped that this will cause all available vdevs to be
utilised while ensuring the more capable devices take on
more of the workload.

This is based of an algorithm suggested by @GregorKopka and @behlendorf and should close #1461.

Needs more testing and peer review before suitable for any sort of use.

Here are the result of some initial testing on 3 way mirror with 2 hd's and a ram drive showing more reads being directed to the ram0 vdev:

```
# zfs set primarycache=none tank1
# zfs set secondarycache=none tank1

# zpool status
pool: tank1
state: ONLINE
status: The pool is formatted using a legacy on-disk format. The pool can
still be used, but some features are unavailable.
action: Upgrade the pool using 'zpool upgrade'. Once this is done, the
pool will no longer be accessible on software that does not support
feature flags.
scan: resilvered 612K in 0h0m with 0 errors on Fri May 17 22:14:44 2013
config:

NAME STATE READ WRITE CKSUM
tank1 ONLINE 0 0 0
mirror-0 ONLINE 0 0 0
vda ONLINE 0 0 0
vdb ONLINE 0 0 0
ram0 ONLINE 0 0 0

# zpool offline tank1 ram0
# dd if=/dev/ram0 of=/dev/null bs=1024k count=300
300+0 records in
300+0 records out
314572800 bytes (315 MB) copied, 6.60732 s, 47.6 MB/s
# zpool online tank1 ram0
# zpool offline tank1 vda
# dd if=/dev/vda of=/dev/null bs=1024k count=300
300+0 records in
300+0 records out
314572800 bytes (315 MB) copied, 45.1447 s, 7.0 MB/s
# zpool online tank1 vda

# while [ 1 ]; do dd of=/dev/null if=/tank1/testfile1 bs=1024k; done &
# zpool iostat -v tank1 1

capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
tank1 775M 241M 5.57K 0 98.4M 0
mirror 775M 241M 5.57K 0 98.4M 0
vda - - 1.36K 0 27.0M 0
vdb - - 1.27K 0 26.8M 0
ram0 - - 2.64K 0 58.2M 0
---------- ----- ----- ----- ----- ----- -----

capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
tank1 775M 241M 6.39K 0 98.2M 0
mirror 775M 241M 6.39K 0 98.2M 0
vda - - 1.49K 0 27.1M 0
vdb - - 1.41K 0 26.7M 0
ram0 - - 3.11K 0 60.3M 0
---------- ----- ----- ----- ----- ----- -----

capacity operations bandwidth
pool alloc free read write read write
---------- ----- ----- ----- ----- ----- -----
tank1 775M 241M 3.97K 0 99.6M 0
mirror 775M 241M 3.97K 0 99.6M 0
vda - - 1.04K 0 25.9M 0
vdb - - 1016 0 26.6M 0
ram0 - - 1.88K 0 55.6M 0
---------- ----- ----- ----- ----- ----- -----

```