LU-4438 utils: handle ct_setup() errors

Problem has been encountered causing sanity-hsm/test_402 errors
because copytool threads were found still alive after MDCs
deactivation. This appears to be due to no ct_setup() error
handling with continuing+unnecessary startup actions to occur.
Also, now that copytool will exit earlier, HSMTOOL_NOERROR variable
has been added to allow errors in copytool_setup().

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: I108ea679faf2731f2a64fb08d2f8fba9db01c71f
Reviewed-on: http://review.whamcloud.com/9853
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4957 ofd: convert FID to OSTID before put it on wire

For the compatible purpose, OFD has to convert the FID(IDIF)
to ost_id before reply to MDT(in ofd_create_hdl), because the
old MDT(<2.6) can not deal with IDIF, but only ostid.

Do not LASSERT for comparing OFD precreate fids with local
used FID, instead it will just return -ESTALE.

Change-Id: Ic35cca7a78bff2e4ca24f507a2c32666ab032579
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10580
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
LU-5151 lnet: return +ve for blocked lnet messsage

returned value of lnet_post_send_locked and
lnet_post_routed_recv_locked are changed to -ve by:
http://review.whamcloud.com/#/c/9369/

this is wrong because callers rely on +ve to identify blocked
message which is not a failure.

To respect linux kernel coding style and not use positive error
code, this patch adds two macros as non-error returned values of
these functions:
  LNET_CREDIT_OK	has credit for message
  LNET_CREDIT_WAIT	no credit and message is blocked

both these functions will return these two values instead of 0
and EAGAIN

Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Id809e4fa6e22780a0bd9ad9c0f0ef96b2c9aa879
Reviewed-on: http://review.whamcloud.com/10625
Tested-by: Jenkins
Reviewed-by: Chris Horn <hornc@cray.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4214 target: LWP failover should create new export

Upon reconnection from new NID LWP client need to create new
export and destroy old one like that is done for MDS connection

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Id21a742a978f629e2324bc568a1a50c2922fe3cc
Reviewed-on: http://review.whamcloud.com/8465
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
LU-4793 clio: Reduce memory overhead of per-page allocation

A page in clio used to occupy 584 bytes, which will use size-1024
slab cache. This patch reduces the per-page overhead to 512 bytes
so it can use size-512 instead.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I54d7ed7b64d5987ddaafbf16edafa5cdf491b0a1
Reviewed-on: http://review.whamcloud.com/10070
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
LU-4629 utils: fix NULL pointer dereference

'uuid' and 'lnet' can be NULL. So check for this case.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I06ead7470649054ca5bd0e0395f5e817c08bc2db
Reviewed-on: http://review.whamcloud.com/10136
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5053 ptlrpc: Add schedule point to ptlrpc_check_set()

Most ptlrpc sets are believed to be small and bounded in length.  However
at the very least the ptlrpcd reuses the ptlrpc sets at its primary work
queue.  This work queue can easily have work added faster than the ptlrpcd
thread can process the work.  The unbounded work can lead to the ptlrpcd
monopolizing a CPU for hundreds of seconds.  Obviously a well-behaved
kernel function should obey the scheduler and share the processor.

We address that problem by inserting a cond_resched() at the top of the
main loop of ptlrpc_check_set().

Some have suggested putting the cond_resched() lower in the loop.  However,
the only current way to bound the number of loops that we exceed our
allocated run time is to put the call at the top of the loop.  Putting it
lower would allow an unknown number (and since it is unknown, it might be
excessively large at times) of cycles through the loop before a
resched is allowed.

Change-Id: If46a6e1af1d0fdae24af623045aaf902a04fc589
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/10358
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
LU-5122 lfsck: maintain the MDT devices list properly

The layout LFSCK needs to know which MDT/OST devices have taken part
in the LFSCK, it maintains two lists for that. When the layout LFSCK
starts/stops, it should handle the two lists separately and properly.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I26103ddc7796dbb1706f5c4da3737df297214e68
Reviewed-on: http://review.whamcloud.com/10507
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5117 lfsck: naming rules for orphan

The rules for naming orphan under /ROOT/.lustre/lost+found/MDTxxxx/
are as following:

1) The orphan's name should start with the "FID" to simplify the
   sorting by FID.

2) The name needs to contain the orphan type to indicate why the
   orphan is created.

3) To resolve the name conflict, there is the conflict version
   in the name.

4) There can be infix in the name to indicate more information,
   such as its original name, the original stripe offset, etc.

So the name will be like:
${FID}-${infix}-${type}-${conflict_version}

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I6eaa0daff39729f401a4f45eae4c1efdd527275a
Reviewed-on: http://review.whamcloud.com/10482
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4629 osd-ldiskfs: Fix use after free

Object 'dir' was dereferenced at line 3343 after being freed
by calling 'kfree' at line 3332.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Id8d216868798622f12773b621acf46e81988b954
Reviewed-on: http://review.whamcloud.com/10609
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
LU-4629 osd-ldiskfs: Fix null pointer dereference

Null pointer 'ooc' that comes from line 1036 may be dereferenced
at line 1083.

Pointer 'it' checked for NULL at line 1036 may be passed to function
and may be dereferenced there by passing argument 2 to function
'osd_scrub_wakeup' at line 1089. Also there is one similar error on
line 1089.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: Iaffa27e48063fbff8b09f390760c24a1da8dd6b3
Reviewed-on: http://review.whamcloud.com/10610
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4629 obdclass: Fix uninitialized variables

'sd.page_link' is used uninitialized in this function.
'ss.page_link' is used uninitialized in this function.
'sl.page_link' is used uninitialized in this function.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I5f571fe2b5dc9007720601292484dbf24a348b83
Reviewed-on: http://review.whamcloud.com/10613
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-5039 mdd: do orphan cleanup asynchronously

Orphan cleanup shouldn't block mount process, it can be done
asynchronously.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: Ib282dbece0cf789b33105aaeaeef04bd6f1dcf70
Reviewed-on: http://review.whamcloud.com/10584
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2234 man: Add zfs options to mkfs.lustre man page

Document zfs options in mkfs.lustre(8).  Also update index and fsname
options to match current requirements.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I06e018c41b29d8f9ca8d1f2ef595140bed16096e
Reviewed-on: http://review.whamcloud.com/10629
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Cliff White <cliff.white@intel.com>
LU-5133 tests: Add version check in sanity/238

Fix for LU-4659 was not included until after 2.5.57 and after 2.5.1.
Ensure MDS is running one of these valid branches before allowing the
test to run.

Test-Parameters: alwaysuploadlogs envdefinitions=ONLY=238 \
ossjob=lustre-b2_5 mdsjob=lustre-b2_5 ossbuildno=42 mdsbuildno=42 \
testlist=sanity

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: If0601c048e6641b2cfcd27b7a6f62d8662440018
Reviewed-on: http://review.whamcloud.com/10559
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5132 tests: Add version check to sanity/160c

Account for what versions of Lustre this test should work for.
This test was added in http://review.whamcloud.com/9723 and that
was included in 2.5.58 and prior to 2.5.2 in the b2_5 branch.

Test-Parameters: alwaysuploadlogs envdefinitions=ONLY=160c \
ossjob=lustre-b2_5 mdsjob=lustre-b2_5 ossbuildno=42 mdsbuildno=42 \
testlist=sanity

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I12a8fbfe1ba09b97e0346f148884d5355eb704e4
Reviewed-on: http://review.whamcloud.com/10557
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>