LU-335 ldiskfs: SuSE 11 SP1 support for Lustre 2.X

Lustre 2.X has been lagging for SuSE server side support. This
work brings the code up to date to support the SuSE 11 SP1 kernel
2.6.32.36-0.5. To reduce the maintenance overhead we basically
add a patch that modifies the SuSE ext4 code to match very
closely to the RHEL6.1 ext4 code base.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I554679c9b0d400eeda1d5d30157c5d86e68793f9
Reviewed-on: http://review.whamcloud.com/575
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-506 kernel: FC15 - generic_permission() need 4 arguments.

generic_permission() add a new parameter flags to determine
if in rcu-walk case. it will return -ECHILD when rcu-walk
request cannot be satisfied. check_acl callback also changed
for this intention. So we don't need check it separately.

Change-Id: I15e0660f47c5db414dbb920f68177f310a185e4d
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1843
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
LU-506 kernel: FC15 - super_operations add evict_inode method.

super_operations add a new method evict_inode(), it hybird of
delete_inode() and clear_inode().

Change-Id: Ifa6de3bca16294386f33404b5a42ff749e1d0665
Signed-off-by: Yang Sheng <ys@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1844
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Hudson
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-506 kernel: FC15 - fix GCC 'set-but-unused' warnings

Eliminate a number of unused variables in the code that
caused GCC-4.6.0 to warn about "set-but-unused variables".
In some cases the variables are only used in
conditionally-compiled code, so they are used in some way
just to quiet the compiler.

Signed-off-by: Yang Sheng <ys@whamcloud.com>
Change-Id: I8b9b744c787fff4057722420866adea86d8240f5
Reviewed-on: http://review.whamcloud.com/1330
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-935 quota: break early when b/i_unit_sz exceeded upper limit

While expanding b/i_unit_sz in dquot_create_oqaq(), we'd break the loop
early when the b/i_unit_sz exceeded upper limit, otherwise, qaq_b/iunit_sz
could be overflow and result in endless loop.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I0bf069e9259627426d7a87ec42844eaed7a733b4
Reviewed-on: http://review.whamcloud.com/1890
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-417 llite: report non-zero blocks on writing client

Writing client may not report accurate allocated block numbers when
dirty pages has not been writting back to OSTs, some "cp" or "tar" may
skip the file because it thinks it is completely sparse.

This patch makes writing client consider dirty pages when reporting
allocated blocks, lest the file be treated as a completely sparse one.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I985d50a44ea1e917bf8e1cba3b5cb770eec35c3f
Reviewed-on: http://review.whamcloud.com/1647
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
New tag 2.1.53

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Change-Id: I0d5a2dc7af8e866f59b5d1dcf5b7ccc6e4268567
LU-424 tests: conf-sanity test 55, 56, 58 failure fixes

The MGS service was not started in conf-sanity test 55,
56 and 58 with separate MGS and MDT configuration. This
patch fix the issue.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I0c01955b4baf81535959cdbf38bf84e7acb04ddf
Reviewed-on: http://review.whamcloud.com/1870
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-635 tests: conf-sanity 27 28 29 30 35 43 failures

Several test in conf-sanity would fail with a error
This command must be run on the MGS. This was due
to seperating the MGS and MDS. I tracked down the fix
to doing a pdsh for several lctl commands to MGS
instead of the MDS.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I721472b15821e2dbcc636292a8e82c1a1b5e0149
Reviewed-on: http://review.whamcloud.com/1869
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-571 ldlm: add parallel ast flow control

Commit {hash: 8c83e7d75989ef527e43a824a0dbe46bffabd07d} removed the
parallel AST limit on the server. However, if there are too many locks
to revoke, it will have to allocate too many RPCs.

Return to having an upper limit on the number of AST RPCs in flight by
adding a flow control algorithm that allows a configurable upper limit on
the number of RPCs in flight.

Change-Id: Ifb68991acf7a33119b334447aec50f7717ed546e
Signed-off-by: Jinshan Xiong <jay@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1608
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-884 clio: client in memory checksum

Use page_mkwrite() method from latest kernels to correctly implement
RPC checksum functionality. Also OBD_FL_MMAP is removed because it
won't be used any more.

Change-Id: I6ec5aae14f56c95b1ac6936d21b5a273582fa4e8
Signed-off-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1609
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-625 test: sanityn.sh test file size should be dynamic

For large OSTCOUNT, a small write can't take extent locks on all OSTs.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ic3fb8e88444767db70545292c4b81b9fe9f1f813
Reviewed-on: http://review.whamcloud.com/1901
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-81 deadlock of changelog adding vs. changelog cancelling

This is a workaround for the deadlock of changelog adding vs.
changelog cancelling. Changelog adding always start transaction
before acquiring the catlog lock(lgh_lock), whereas, changelog
cancelling do start transaction after holding the catlog lock.

We start transaction earlier to avoid above deadlock.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I9647b9a559f68a27dc0d4b4885857d3cf73b5b8e
Reviewed-on: http://review.whamcloud.com/1260
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-728 tests: pdsh HOSTLIST test suite support

For large systems it becomes tedious to list all the nodes
or devices. The application pdsh uses a HOSTLIST form to
make listing nodes more compact. This patch takes data in
the HOSTLIST format and creates a expanded list for the
shell scripts in the test suite to use. Also this gives the
option for the pdsh commands to use the nodes list in
HOSTLIST format directly. Its is also possible to use the
HOSTLIST format for devices as well as nodes.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ia7a6c95ab4e701a17fc2b80d8443a5ff79da3f3c
Reviewed-on: http://review.whamcloud.com/1462
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-882 quota: Quota code compares unsigned < 0

Port from b23858.

In check_cur_qunit(), it checks "if (limit + record < 0)", however,
the limit is unsigned, so this check will be always false, and when
limit is smaller than -record, following "limit += record" will make
limit a unreasonable large value.

This patch also fixed a similar defect in dqacq_handler().

Signed-off-by: Vladimir Saveliev <vladimir.saveliev@oracle.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Iea02143dae5542f1a9f9cc823a684a18031b8a03
Reviewed-on: http://review.whamcloud.com/1889
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-867 ptlrpc: Resolve section mismatch

The __init and __exit attributes appear to have been over used for
the ptlrpc module.  They should only be attached to ptlrpc_init and
ptlrpc_exit respectively and those functions should call the required
cleanup functions.  By using the attributes too broadly the following
section mismatch warning with be hit.  This indentifies cases where
your calling an __exit function from an __init function which is in
the wrong section.

WARNING: ~/src/git/lustre/lustre/ptlrpc/ptlrpc.o(.init.text+0x354):

Section mismatch in reference from the function ptlrpc_init() to the
function .exit.text:sptlrpc_fini() The function __init ptlrpc_init()
references a function __exit sptlrpc_fini().

This is often seen when error handling in the init function uses
functionality in the exit path.  The fix is often to remove the
__exit annotation of sptlrpc_fini() so it may be used outside an
exit section.

Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I94a4cb5fb7deea41fce5ee939470dc2e40908f98
Reviewed-on: http://review.whamcloud.com/1653
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-617 recovery: setattr from open breaks recovery

The setattr from open(open(O_TRUNC)) is now serialized with
'cl_setattr_lock' on client and goes to a dedicate portal, which is
different with other reint operations, consequently, setattr RPC
can be parallel with other reint RPCs, and that result in the race of
updating last_transno/last_xid on server.

This patch removed the 'cl_setattr_lock' stuff to make all the reint
operations serialized by 'cl_rpc_lock', and the code on server side
which assumes client is holding DLM lock when setattr from open is also
removed, since it's not true.

The MDS_SETATTR_PORTAL service is preserved to keep the compatibility
with old client, and the MDS_SETATTR_FROM_OPEN is also preserved, since
we are using this flag to check write access for open(O_TRUNC), and
it probably can be used for some optimization purpose in future.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I45f83f8f05022ff0d31f8e7784381821c835785d
Reviewed-on: http://review.whamcloud.com/1654
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Mikhail Pershin <tappro@whamcloud.com>
LU-32 osd: keep root node BH ref of IAM container

IAM in ldiskfs-osd will always consume some slots in bh_lru (see:
fs/buffer.c), if we keep buffer_head reference on root node,
we can save one slot in bh_lru and could be somehow helpful for
overall performance, I did some tests, LRU hits rate increased
5%-10% while creating files if we always keep this reference.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I954f26932462169c9bfc6bbfe1a57b5348624179
Reviewed-on: http://review.whamcloud.com/1826
Tested-by: Hudson
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-924 test: sync client data before reboot server

Lustre doesn't write client data synchronously (to avoid flooding sync
writes when there are many clients connecting, see exp_need_sync), so
if the server reboots before client data reachs disk, the client data
will be lost and client will be evicted after recovery, which is not
what we expected.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I21bcafe1f2630285c9108c0528467eb177e3449b
Reviewed-on: http://review.whamcloud.com/1888
Tested-by: Hudson
Reviewed-by: Chris Gearing <chris@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-847 quota: client retrieve quota usage directly

Current 'lfs quota' sends getquota RPC to MDS, and MDS is responsible
for retrieving disk usage from all targets, this scheme will be
changed to client retrieving disk usage from all targets directly.

This patch addresses the compatibility issue as well: If the getquota
returned by MDS has QIF_SPACE, client just trust the disk usage
returned by MDS, otherwise, client has to issue RPCs to all OSTs to
collect disk usage by itself.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ia94d3b3ba280d5b31d2d3c508412d662f4e95321
Reviewed-on: http://review.whamcloud.com/1382
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>