LU-1669 llite: Replace write mutex with range lock

Testing has shown the ll_inode_inode's lli_write_mutex to be a limiting
factor with single shared file write performance, when using many
writing threads on a single machine. Even if each thread is writing to a
unique portion of the file, the lli_write_mutex will prevent no more
than a single thread to ever write to the file simultaneously.

This change attempts to remove this bottle neck, by replacing this mutex
with a range lock. This should allow multiple threads to write to a
single file simultaneously iff the threads are writing to unique regions
of the file.

Performance testing shows this change to garner a significant
performance boost to write bandwidth. Using FIO on a single machine with
32 GB of RAM, write performance tests were run with and without this
change applied; the results are below:

    +---------+-----------+---------+--------+--------+--------+
    |     fio v2.0.13     |        Write Bandwidth (KB/s)      |
    +---------+-----------+---------+--------+--------+--------+
    | # Tasks | GB / Task | Test 1  | Test 2 | Test 3 | Test 4 |
    +---------+-----------+---------+--------+--------+--------+
    |    1    |    64     |  452446 | 454623 | 457653 | 463737 |
    |    2    |    32     |  850318 | 565373 | 602498 | 733027 |
    |    4    |    16     | 1058900 | 463546 | 529107 | 976284 |
    |    8    |     8     | 1026300 | 468190 | 576451 | 963404 |
    |   16    |     4     | 1065500 | 503160 | 462902 | 830065 |
    |   32    |     2     | 1068600 | 462228 | 466963 | 749733 |
    |   64    |     1     |  991830 | 556618 | 557863 | 710912 |
    +---------+-----------+---------+--------+--------+--------+

 * Test 1: Lustre client running 04ec54f. File per process write
           workload. This test was used as a baseline for what we
           _could_ achieve in the single shared file tests if the
           bottle necks were removed.

 * Test 2: Lustre client running 04ec54f. Single shared file
           workload, each task writing to a unique region.

 * Test 3: Lustre client running 04ec54f + I0023132b. Single shared
           file workload, each task writing to a unique region.

 * Test 4: Lustre client running 04ec54f + I0023132b + this patch.
           Single shared file workload, each task writing to a unique
           region.

Although the performance boost displayed by this patch in "Test 4" is
progress, there is still room for improvement. Specifically, the next
easy to fix bottle neck appears to be the usage of ll_inode_size_lock
and ll_inode_size_unlock from within vvp_io_commit_write. Addressing
that area of contention is left to be done in a follow up patch.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I71e060c190065d87a20dc8df3104f898883d0583
LU-1669 llite: Reduce contention on lli_size_sem

Testing shows there is still some low hanging fruit to be had in terms
of increasing the write performance of a single shared file workload,
using many threads on a single machine. With many threads concurrently
writing to a single file, the ll_size_sem becomes a limiting factor to
write throughput.

This change attempts to remove this bottle neck by changing the
ll_size_sem (which is essentially a mutex) to be a rw_semaphore. This
allows multiple threads to read the inode size concurrently, and the
write lock only needs to be taken when the inode size changes (which is
an infrequent event for many single shared file workloads). Thus
allowing much better concurrency and parallelization out of the
vvp_io_commit_write function.

Performance testing shows this change to garner a sizeable performance
boost to write bandwidth. Using FIO on a single machine with 32 GB of
RAM, write performance tests were run with and without this change
applied; the results are below:

    +---------+-----------+---------+--------+--------+--------+--------+
    |     fio v2.0.13     |            Write Bandwidth (KB/s)           |
    +---------+-----------+---------+--------+--------+--------+--------+
    | # Tasks | GB / Task | Test 1  | Test 2 | Test 3 | Test 4 | Test 5 |
    +---------+-----------+---------+--------+--------+--------+--------+
    |    1    |    64     |  452446 | 454623 | 457653 | 463737 | 441509 |
    |    2    |    32     |  850318 | 565373 | 602498 | 733027 | 703742 |
    |    4    |    16     | 1058900 | 463546 | 529107 | 976284 | 986097 |
    |    8    |     8     | 1026300 | 468190 | 576451 | 963404 | 968389 |
    |   16    |     4     | 1065500 | 503160 | 462902 | 830065 | 959462 |
    |   32    |     2     | 1068600 | 462228 | 466963 | 749733 | 890119 |
    |   64    |     1     |  991830 | 556618 | 557863 | 710912 | 847302 |
    +---------+-----------+---------+--------+--------+--------+--------+

 * Test 1: Lustre client running 04ec54f. File per process write
           workload. This test was used as a baseline for what we
           _could_ achieve in the single shared file tests if the
           bottle necks were removed.

 * Test 2: Lustre client running 04ec54f. Single shared file
           workload, each task writing to a unique region.

 * Test 3: Lustre client running 04ec54f + I0023132b. Single shared
           file workload, each task writing to a unique region.

 * Test 4: Lustre client running 04ec54f + I0023132b + I71e060c1.
           Single shared file workload, each task writing to a unique
           region.

 * Test 5: Lustre client 04ec54f + I0023132b + I71e060c1 + this patch.
           Single shared file workload, each task writing to a unique
           region.

As you can see, this patch and the other two patches in this series
(I0023132b and I71e060c1) increase the write bandwidth for this workload
by factors ranging from ~1.5 to ~2, depending on the number of threads.
This patch alone increases the write bandwidth by about 20% at the
highest thread count workload tested (64 threads).

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I7558ba03a1609ec4509dae38a87ced104217a791