LU-4852 osc: osc_extent_truncate()) ASSERTION( !ext->oe_urgent ) failed

The bug was caused by race between truncate & fsync.
osc_extent_wait() doesn't takes into account oe_trunc_pending
during setting oe_urgent. The race arises after
osc_object_unlock().
osc_extent_wait() should ignore extents with oe_trunc_pending
while waiting for OES_INV. osc_cache_truncate_end() will set
oe_urgent and call osc_io_unplug_async()

Change-Id: I6f4c8420faa67bf4b14b1c5c2bc37fa282a7b072
Xyratex-bug-id: LELUS-239
Signed-off-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Reviewed-on: http://review.whamcloud.com/10204
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4676 hsm: Fix return value error of ct_run()

ct_run() returns 0 after find out that fs_name is invalid.
lhsmtool_posix will exists with 0 after that error happens.
That might mislead scripts calling lhsmtool_posix.

Lustre-change: http://review.whamcloud.com/9412
Lustre-commit: d5ad6ff819e635c6ba337586cf70698b7728ff7f

Change-Id: Id182e989011e02104b5960c6692995825d939ad6
Signed-off-by: Li Xi <lixi@ddn.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10234
LU-4830 tests: only deactivate MDTs of Lustre FSNAME

Problem has been discovered using a very specific copytool Lustre
root/archive usage. All MDCs, including archive one, were
deactivated.

Lustre-commit: 09cc4025440f7e4f9f887f4a2a9bdceef0da7689
Lustre-change: http://review.whamcloud.com/9843

Change-Id: Ifab4bd5a7d6a88ae0e0b95d5a6774dbfed8822e3
Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Reviewed-on: http://review.whamcloud.com/10232
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2524 test: Modify tdir to be single directory

Currently, the tdir variable is a directory with a subdirectory.
This requires the '-p' option when calling mkdir. We've made tdir
be a single directory so calls to mkdir and test_mkdir do not
require the '-p' option in most cases.

tdir was changed from d0.${TESTSUITE}/d${base} to
d${testnum}.${TESTSUITE} and tfile was changed from
f.${TESTSUITE}.${testnum} to f${testnum}.${TESTSUITE}. Now tdir and
tfile are consistent in their format and the call to remove files
and directories at the beginning of many test scripts will remove
these files and directories.

Once this patch lands, we can remove the "-p" option from many
of the calls to mkdir.

Lustre-commit: a98d44e0439ad5b20d0d7f273b29e6ea990312f0
Lustre-change: http://review.whamcloud.com/8123

LU-2524 tests: run sanity test_51ba in test_51b dir

Run the test_51ba directory cleanup in the same directory as the
test_51b subtest created its subdirectories.

Lustre-commit: 47c581b8155f865ade5c9d067f85930146361c2a
Lustre-change: http://review.whamcloud.com/9021

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Ib49d7102a49ff6b5f3ec539a5b2f2f5186231a04
Reviewed-on: http://review.whamcloud.com/8123
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10349
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4573 tests: check all MDTs for open files

sanityn.sh test_76 verifying mdt.*.exports.*.open_files content
did not check all of the MDTs. It was broken by commit 370de927fc5,
which created all directories striped across all MDTs by default.

There is actually no need to special-case striped or remote
directories in this test, and it should be possible to just check
for open files on all MDTs, and this also simplifies the test.

Lustre-change: http://review.whamcloud.com/10107
Lustre-commit: fed7a99441c2dd078a671cf5b3dcaf27996bf448

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I2b7b434052946c8d84bffef1fc7358cc29b00b75
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10499
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4102 doc: recommend newer e2fsprogs version

Recommend newer e2fspogs version for 2.5.2 release so that it gets
the latest fixes.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: Ica1e47b4242228334f9d673ee5159471f4e1ca8c
Reviewed-on: http://review.whamcloud.com/10561
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4780 lnet: NI shutdown may loop forever

lnet_shutdown_lndnis() may enter endless loop if there is a busy NI,
this is injected by LNet SMP improvements. It's fixed in this patch.

Lustre-change: http://review.whamcloud.com/9706
Lustre-commit: 3efa31eacc612261ea3019fbd7243e573714042c

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Change-Id: If6bf6a8362d8808cd924d6fe79dbe43151f41f30
Reviewed-on: http://review.whamcloud.com/10489
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
LU-5100 llite: set dir LOV xattr length variable

the LOV xattr of directory could be either lov_user_md_v1
(size is 32) or lov_user_md_v3 (size is 48), then the actual
size of the LOV xattr should be return.

this patch also changes lod_verify_stripe to make it to accept
lov stripe from user and it's "pattern" is "LOV_PATTERN_RAID0".

Change-Id: I1ccba6da19aef0b7464c43b1b50871df44ce0eb7
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Reviewed-on: http://review.whamcloud.com/10453
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10575
LU-5133 tests: Add version check in sanity/238

Fix for LU-4659 was not included until after 2.5.57 and after 2.5.1.
Ensure MDS is running one of these valid branches before allowing the
test to run.

Lustre-change: http://review.whamcloud.com/10559

Test-Parameters: alwaysuploadlogs envdefinitions=ONLY=238 \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=sanity

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I25787874866b38f290b21baf76a09a2aec3dc7f4
Reviewed-on: http://review.whamcloud.com/10560
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3386 lproc: improve osc/mdc "imports" connect data

Improve /proc/fs/lustre/{osc,mdc}/*/import files to print
the struct obd_connect_data data fields as "connect_data:"
in addition to the "connect_flags:" field.
Lustre-commit: 62660c799be886e557ae0426126fc3e4b61927d4
Lustre-change: http://review.whamcloud.com/6588
Also, this patch includes the fix in
http://review.whamcloud.com/#/c/10344

Signed-off-by: Liu Ying <emoly.liu@intel.com>
Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: Iabf2231341f3d674960626fb7230e2ba5cf4f113
Reviewed-on: http://review.whamcloud.com/9264
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5132 tests: Add version check to sanity/160c

Account for what versions of Lustre this test should work for.
This test was added in http://review.whamcloud.com/9723 and that
was included in 2.5.58 and prior to 2.5.2 in the b2_5 branch.

Lustre-change: http://review.whamcloud.com/10557

Test-Parameters: alwaysuploadlogs envdefinitions=ONLY=160c \
ossjob=lustre-b2_4 mdsjob=lustre-b2_4 ossbuildno=73 mdsbuildno=73 \
testlist=sanity

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I560a9900de983b06963d49946e9188649dd5e7fc
Reviewed-on: http://review.whamcloud.com/10558
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5047 tests: correct cleanup files in sanity.sh

Don't remove all files from Lustre share. Create files that will be
cleaned up at the end tests session in check_and_cleanup_lustre().

Fix cleanup for test_54c.

Luster-Commit: c42bb35bc32f472d69276475de3cd186ca5f5247
Lustre-Review: http://review.whamcloud.com/10291

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I23624b9d14e7de3423b82b4a5b8508c4820c55d4
Reviewed-on: http://review.whamcloud.com/10291
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10633
LU-4887 tests: sanity-scrub interoperability tests with master

In the master branch, the default behaviour for lfsck_start without
"-t" specified will trigger all supported LFSCK components. And the
master branch uses new "-t scrub" for control OI scrub. The changes
will cause the old (2.5.58 or older) sanity-scrub test scripts fail
at test_9/test_11.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I134f8ec7d3f9e5c60abb76d728727871a960a095
Reviewed-on: http://review.whamcloud.com/10641
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
LU-4569 hsm: Prevent copytool from importing existing file.

Requesting import of an already existing FID can lead to data
corruption since both inodes in lustre would refer to the same
file in the backend.

Lustre-change: http://review.whamcloud.com/10185
Lustre-commit: 8dfee22dd5a896144fb913f5f9f0e8f5c0e4ed86

Change-Id: I92d951e3e0796c9bde8bd744f39f242e2aa233eb
Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10479
Reviewed-by: John L. Hammond <john.hammond@intel.com>
LU-2272 statahead: ll_intent_drop_lock() called in spinlock

ll_intent_drop_lock() may sleep, which should not be called inside
spinlock.

This is back-ported from the following patch:
Lustre-change: http://review.whamcloud.com/9665

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes \
mdtcount=1 testlist=lustre-rsync-test,lustre-rsync-test

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: Id658c0bd3b9dc0a918995c66842b0c3fabe8a7ae
Signed-off-by: Jian Yu <jian.yu@intel.com>
Reviewed-on: http://review.whamcloud.com/10674
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5116 ptlrpc: race at req processing

Race between ptlrpc_resend_req() and ptlrpc_check_set().
1 thread do ptlrpc_check_set()->after_reply()
2 thread do ptlrpc_resend_req()
The result is request with rq_resend = 1 and MSG_REPLY flag.
When this request will came to server it will cause client eviction.
The patch skip ptlrpc_resend_req logic if rq_replied is set,
and clear rq_resend flag at reply_in_callback() when client got
reply.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Xyratex-bug-id: MRP-1888
Change-Id: If47b9a1f559ab16e4b416332ada7c73a8758d46c
Reviewed-on: http://review.whamcloud.com/10471
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-on: http://review.whamcloud.com/10562
Reviewed-by: James Nunez <james.a.nunez@intel.com>