zfsonlinux/zfs — Issue #425: Incorrect ashift value may cause faulted pools
Author: gunnarbeutner
Created: 2011-10-20 07:33:55+00:00
State: open

While investigating another issue I've come across this:

root@zt:~# zpool create -f -o ashift=16 tank sdb
root@zt:~# dd if=/dev/zero of=/tank/test bs=1M &
[1] 3405
root@zt:~# sleep 5; echo b > /proc/sysrq-trigger

after the reboot:

root@zt:~# zpool status
pool: tank
state: FAULTED
status: One or more devices could not be used because the label is missing
or invalid. There are insufficient replicas for the pool to continue
functioning.
action: Destroy and re-create the pool from
a backup source.
see: http://www.sun.com/msg/ZFS-8000-5E
scan: none requested
config:

NAME STATE READ WRITE CKSUM
tank FAULTED 0 0 0 corrupted data
sdb UNAVAIL 0 0 0 corrupted data
root@zt:~#

I'm able to reproduce this same behavior with ashift values as low as 14. The disk in this test is an iSCSI LUN (with 64kB sectors) that's passed-through to the VM by VMware as a disk with 512-byte sectors.

There are two things I'm currently wondering about:

a) Can this possibly happen with lower ashift values (e.g. ashift=12 with 512-bytes-per-sector disks) and is this because ZFS assumes that single-sector-writes are atomic (I'm really just guessing here).

b) Depending on how (a) turns out, is this suggestion (from zpool(8)) such a great idea:

For optimal performance, the pool sector size should be greater than or equal to the
sector size of the underlying disks. Since the property cannot be changed after pool
creation, if in a given pool, you ever want to use drives that report 4KiB sectors,
you must set ashift=12 at pool creation time.