LU-3529 lod: create striped directory

1. Add "lfs setdirstripe -i -c" to create striped
directory.

2. client send create request to the master MDT, which
will allocate FIDs and create slaves. for all of slaves.

3. Client needs to revalidate slaves during intent getattr
and open request.

4. lmv_stripe_md will include attributes(size, nlink etc)
from all of stripe, which will be protected by UPDATE lock.
client needs to merge these attributes when update inode.

5. send create request to the MDT where the file is located,
which can help creating master stripe of striped directory.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I7ac560e39dcb415e310dc5e6ade531d76227ffae
Reviewed-on: http://review.whamcloud.com/7196
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
LU-3531 llite: fix "lfs getdirstripe" to show stripe info

Fix "lfs getdirstripe", so it can show layout information
of striped directory

[root@testnode tests]# ../utils/lfs getdirstripe /mnt/lustre/test1
/mnt/lustre/test1
lmv_stripe_count: 2
lmv_stripe_offset: 0
mdtidx		 FID[seq:oid:ver]
     0		 [0x280000400:0x1:0x0]
     1		 [0x2c0000400:0x1:0x0]

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I586f78ee2e0c35d8c3ed10726d5f5e12a4b543e7
Reviewed-on: http://review.whamcloud.com/7228
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4472 mdc: Fix mdc_page_locate ASSERT

Storing hash 0 at same as hash 1 should be okay, don't ASSERT in this
case.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I9b9488e5374d22dcfc9e7f2c969da3b02778097a
Reviewed-on: http://review.whamcloud.com/8821
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3593 lfsck: repair inconsistent layout EA

The layout EA storing on the MDT-object records not only the file
layout but also some information which indicates the layout owner,
such as lov_mds_md.lmm_oi. They are generated from MDT-object FID,
with them we can know which file the layout EA belongs to. In the
LFSCK phase II, we need to verify whether such information in the
layout EA is correct or not by re-caculating from the MDT-object
FID. If inconsistency is found, trust the MDT-object FID rather
than the FID information in the layout EA, and repair the later.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I3c31e19e9fabe66fe7ffdba2fe8569795ae49b4a
Reviewed-on: http://review.whamcloud.com/7456
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4482 grant: don't use cache data in osd_statfs()

osd_statfs() shouldn't cache statfs data anymore: the statfs data
is already cached in ofd layer, put another cache in osd layer
looks redundant, and what more important is: grant mechanism relies
on dt_statfs() returning fresh statfs data, caching statfs data in
osd layer would just break grant.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I89b6384cc59d77b1edb0412f24b5c8e823532170
Reviewed-on: http://review.whamcloud.com/8911
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
LU-4422 tests: disable sanity-quota test_6 temporary

To avoid other patches to be failed for LU-4422 under DNE.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ia48f7bb14733f6bdc5ef34bde1ce91b5762f9192
Reviewed-on: http://review.whamcloud.com/9203
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
LU-4488 build: fix compilation with --enable-invariants

Fix the build which was broken since the following commit:

    commit 0a259bd7dbac76d75b89a389bc317720153aa452
    Author: Jinshan Xiong <jinshan.xiong@intel.com>
    Date:   Mon Sep 30 15:00:38 2013 -0700

    LU-3321 clio: collapse layer of cl_page

    Move radix tree to osc layer to for performance improvement.

    Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
    Change-Id: I93e3cb8352f7be41c23465b12945874316aa1809
    Reviewed-on: http://review.whamcloud.com/7892
    Tested-by: Jenkins
    Tested-by: Maloo <hpdd-maloo@intel.com>
    Reviewed-by: Lai Siyao <lai.siyao@intel.com>
    Reviewed-by: Bobi Jam <bobijam@gmail.com>
    Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Ie8543513852d98d0aa82bca0f227d286cdf8ebd2
Reviewed-on: http://review.whamcloud.com/8853
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4529 quota: call qsd_op_end() after trasn stop

qsd_op_end() shouldn't be called before the transaction stopped,
because qsd_op_end() is a quite heavy operation which could
probably allocate memory with standard allocator flag (__GFP_IO),
and allocating memory could result in dirty flush on other
filesystems, that will lead to opening transaction on different
journal and trigger the assert in jbd2_journal_start():
J_ASSERT(handle->h_transaction->t_journal == journal) at the end.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I4ea3ff011fa7e44460b9912050e90b174813e01a
Reviewed-on: http://review.whamcloud.com/8968
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
LU-4597 clio: clear nowait flag agl lock re-enqueue

The LDLM_FL_BLOCK_NOWAIT flag should be cleared when re-enqueue
the agl lock as normal glimpse, otherwise, it won't get size back
if there is conflicting locks on other client.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ifd311606e824d6574bfbf3256841061e8867214a
Reviewed-on: http://review.whamcloud.com/9249
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Ned Bass <bass6@llnl.gov>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4102 utils: add dry-run to ll_recover_lost_found_objs

Add the dry-run (-n) option to ll_recover_lost_found_objs.  This
allows scanning an OST filesystem without modifying it.

It is now possible to test both the "lost+found/" directory, as well
as the "O/" directory to verify the LMA and FID xattrs on existing
objects that are in the normal filesystem hierarchy.

Fix verbose (-v) option to print all of the inodes being checked.
This option previously did nothing.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I8e2b427046c6acf5cf7429f41ccd57496c500c1e
Reviewed-on: http://review.whamcloud.com/8061
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4356 kernel: kernel update [SLES11 SP2 3.0.101-0.7]

update target and config files for new version

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Ia5a9472029e7134b3f6b997506371f5e5e624797
Reviewed-on: http://review.whamcloud.com/8951
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4515 tests: set fail_loc only once per node

Since fail_loc and fail_val are common for all services on a node,
it is only necessary to set it once per node instead of once per
facet.  That avoids a bunch of extra remote commands and spew in
the test output.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I9a69eb325fa80a90d929ab9a258cce21973ebbe5
Reviewed-on: http://review.whamcloud.com/8978
Tested-by: Jenkins
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4413 ptlrpc: don't try to recover no_recov connection

If a connection has been stopped with ptlrpc_pinger_del_import() and
marked obd_no_recov, don't reconnect in ptlrpc_disconnect_import() if
the import is already disconnected.  Otherwise, without the pinger it
will just wait there indefinitely for the reconnection that will never
happen.

Put the obd_no_recov check inside ptlrpc_import_in_recovery() so that
any threads waiting on the connection to recover would also be broken
out of their sleep if obd_no_recov is set.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Icd8041be0ce344add8d67b026353df1b1e0cab07
Reviewed-on: http://review.whamcloud.com/8996
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3943 utils: fix lfs df -i summary line

If the number of free objects on the OSTs is fewer than on the MDT,
use the number of free OST objects in the filesystem summary, so
it matches "df -i" (limited internally by ll_statfs_internal()).

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Icfaaba3f9c39f12174e6681e9fb68c1f7a2540e5
Reviewed-on: http://review.whamcloud.com/8614
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-1538 tests: delete test files from /tmp after use

Delete files created for tests in /tmp after testing has finished.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I8c3fb62f844cb50d82eba81f274d86c73e3d2e08
Reviewed-on: http://review.whamcloud.com/8615
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4423 obdclass: fix return value check in capa_hmac()

In case of error, the function crypto_alloc_hash() returns ERR_PTR()
and never returns NULL. The NULL test in the return value check
should be replaced with IS_ERR().

Signed-off-by: Wei Yongjun <yongjun_wei@trendmicro.com.cn>
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>

Change-Id: I4889387752d1eb5400649cd5f4da172d64c054e2
Reviewed-on: http://review.whamcloud.com/8681
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4598 quota: fix s-q test_30

After LU-4139 landed, the block grace time isn't accurate as before,
the s-q test_30 should now write more bytes to make sure the spare
quota allocated on slave be used up.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Iee14721051b9e41074a13f11afb11a7b286352c2
Reviewed-on: http://review.whamcloud.com/9177
Tested-by: Jenkins
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
LU-3963 libcfs: remove cfs_hash_long

Replace the name with Linux defined hash_long.
The similar patch has already been submitted upstream.

Signed-off-by: Peng Tao <bergwolf@gmail.com>
Change-Id: Ia96bb703284ed4843c4433a1a50539d9c68ed6d1
Reviewed-on: http://review.whamcloud.com/9268
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-4629 llite: fix suspicious dereference (merge issue)

Suspicious dereference of pointer 'lfd' before NULL check at line 286

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I64c652279abb8fa1e720d23d645f74f07e5237ca
Reviewed-on: http://review.whamcloud.com/9273
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4629 gss: fix few issues found by Klocwork Insight tool

Array 'message_buf' of size 500 may use index value(s) -1

Object 'enc_key.data' was freed at line 164 after being freed
by calling 'free' at line 150. Also there are 3 similar errors
on line(s) 164.

Suspicious dereference of pointer 'vmsg' before NULL check at
line 187. Also there are 2 similar errors on line(s) 196, 205.

Suspicious dereference of pointer 'rmsg' before NULL check at
line 191. Also there are 2 similar errors on line(s) 200, 209.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I50905ea99d904123df30ba7078b180b44b8a6e06
Reviewed-on: http://review.whamcloud.com/9274
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>