zfsonlinux/zfs — pull request #2413 by behlendorf (2014-06-20 23:23:38+00:00)
Title: Refactor mmap(2) writepages (WIP)

The idea here is to pull the zfs_range_lock() out of writepage()
function and in to the writepages() function. In theory, this
should reduce our overhead, improve performance, and simplify
the code. In practice, the reworked code is still complicated
and the performance may be worse.

Moving the zfs_range_lock() outside writepages makes ensuring the
data-integrity semantics difficult. We cannot rely on the generic
write_cache_pages() function in WB_SYNC_ALL mode because it can
deadlock as follows.

--- Process 1 --- --- Process 2 ---
zfs_range_lock sync_page
zfs_get_data wait_on_page_bit
zil_commit write_cache_pages
zfs_putpage zfs_putpage
zpl_writepages zpl_writepages

That means we need to implement our own logic which is similar
to that in write_cache_pages() to ensure pages which are already
in writeback and are redirtied do not get skipped. This patch
accomplishes that by tagging the pages with the TOWRITE tag but
the convergence logic is overly broad. If other processes are
calling msync() over the same file range the pages may be written
more often that needed. That said, it is semantically correct.

This needs to be carefully profiled and benchmarked under mmap(2)
workloads to determine if it's going to work well. I've been
using an fio workload which does small 4k IOs from 32 threads.
To make the workload more realistic the sync_file_range option
was added to cause msync() to be called every 32 writes.

--- FIO workload ---
[global]
bs=4k
ioengine=mmap
iodepth=1
size=1g
direct=0
runtime=60
directory=/tank/fio
filename=mmap.test.file
numjobs=32
sync_file_range=write:32

[seq-read]
rw=read
stonewall

[rand-read]
rw=randread
stonewall

[seq-write]
rw=write
stonewall

[rand-write]
rw=randwrite
stonewall

Original-patch-by: Richard Yao
Signed-off-by: Brian Behlendorf