LU-5208 tests: inject failure on the proper OST

The old test scripts assumed that the file created by the MDS2 will
be striped to the OST2 and OST1, but such assumption is wrong. When
the OSTs reside on different OSS nodes, the sanity-lfsck test_18c
will get failure.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I755370e6a70384b5a70e046b1acc806e63817f6e
Reviewed-on: http://review.whamcloud.com/11275
Tested-by: Jenkins
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-4970 lfsck: flush async updating before exit

Otherwise, the test scripts may get some internal status by race.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I4af07cf91f6b6c77d5cab67fc0df21b27174ee4c
Reviewed-on: http://review.whamcloud.com/11276
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
LU-5075 test: keep LFSCK fail_loc until recovery completed

In sanity-lfsck test_8, the old script may over wrote the fail_loc
before the recovery completed, that will cause the paused LFSCK to
be restarted automatically, then the subsequent manual command for
starting the LFSCK will get -EALREADY. That is unexpected.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I8500ecdccf354e83ec70be8dea88943cafa47d81
Reviewed-on: http://review.whamcloud.com/11288
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5466 lfsck: typo in lfsck_del_target

To handle MDT or OST target differently.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I8ede89935ca89774261efa466f753f78a09a50c2
Reviewed-on: http://review.whamcloud.com/11407
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5195: hsm: delete HSM records not found on disk

After a MDS crash, it is possible the file containing an hsm record is
not present anymore. When the MDS restarts, we get traces like this:

(mdt_hsm_cdt_actions.c:104:cdt_llog_process()) tas01-MDT0000:
    failed to process HSM_ACTIONS llog (rc=-2)
(mdt_hsm_cdt_actions.c:104:cdt_llog_process())
    Skipped 600 previous similar messages
(llog_cat.c:192:llog_cat_id2handle()) tas01-MDD0000:
    error opening log id 0x1c:1:0: rc = -2
(llog_cat.c:192:llog_cat_id2handle()) Skipped 600 previous similar messages
(llog_cat.c:556:llog_cat_process_cb()) tas01-MDD0000:
    cannot find handle for llog 0x1c:1: -2
(llog_cat.c:556:llog_cat_process_cb()) Skipped 600 previous similar messages
(mdt_hsm_cdt_actions.c:104:cdt_llog_process()) tas01-MDT0000:
    failed to process HSM_ACTIONS llog (rc=-2)
(mdt_hsm_cdt_actions.c:104:cdt_llog_process())
    Skipped 600 previous similar messages
(llog_cat.c:192:llog_cat_id2handle()) tas01-MDD0000:
    error opening log id 0x1c:1:0: rc = -2
(llog_cat.c:192:llog_cat_id2handle()) Skipped 600 previous similar messages

No HSM operation can happen, and the only way to clean is to unmount
the MDT and delete hsm_actions.

If the record can't be found, let the MDS delete it instead.

Change-Id: I61f625d4c18750c8044909ff56d53042cf0b6d86
Signed-off-by: frank zago <fzago@cray.com>
Reviewed-on: http://review.whamcloud.com/11419
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Ryan Haasken <haasken@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2182 llapi: implementation of new llapi_layout API

Add a new set of llapi routines for interacting with file layouts that
hide the details of the wire-protocol data structures from the user.
Define an opaque struct llapi_layout to abstract the layout of a
lustre file and generic accessor functions to read and write it.  The
following documented functions are added the liblustreapi public
interface with accompanying man pages:

  llapi_layout_alloc - allocate a new layout
  llapi_layout_free - free memory allocated for a layout
  llapi_layout_file_create - create new file with given layout
  llapi_layout_file_open - open or create a file with given layout
  llapi_layout_get_by_path - get file layout given a path
  llapi_layout_get_by_fd - get file layout given a file descriptor
  llapi_layout_get_by_fid - get file layout given a Lustre FID
  llapi_layout_ost_index_get - get OST index associated with a stripe
  llapi_layout_ost_index_set - set OST index associated with a stripe
  llapi_layout_pattern_get - get RAID pattern of a layout
  llapi_layout_pattern_set - set RAID pattern of a layout
  llapi_layout_pool_name_get - get pool name of a layout
  llapi_layout_pool_name_set - set pool name of a layout
  llapi_layout_stripe_count_get - get stripe count of a layout
  llapi_layout_stripe_count_set - set stripe count of a layout
  llapi_layout_stripe_size_get - get stripe size of a layout
  llapi_layout_stripe_size_set - set stripe size of a layout

The layouts are read and written using fgetxattr() and fsetxattr()
instead of ioctl() to make it easier for architectures like Blue Gene
to function-ship the system calls.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I35fb51055b6438ef3090f43c28a4083a66eaa907
Reviewed-on: http://review.whamcloud.com/5302
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4808 tests: fix check for sanity tests prerequisites

Check of prerequisites for tests that manipulate Lustre servers.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I17d0c90356a62976e1b5478588925862b588d7a2
Reviewed-on: http://review.whamcloud.com/10295
Tested-by: Jenkins
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3963 libcfs: remove invalid cfs_hlist_for_each_* checks

For newer kernels cfs_hlist_for_each_entry* changed the
number of arguments so in this case lustre still needs
wrappers. The HPDD script to test decrepit wrappers
incorrectly complains about cfs_hlist_for_* functions
that are required. Added some more wrappers to deal with
kernel api change that were missing before.

Change-Id: Idda94c147f2e3e9c138ee50c242604494041c244
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/11005
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: frank zago <fzago@cray.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3963 libcfs: convert lod, mdt, and gss to linux list api

Move from the cfs_[h]list api to the native linux api for
the lod, mdt, and gss part of the ptlrpc layers.

Change-Id: Ieff231f3220a850521713a6f1c997b7e09130a4c
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/10387
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: frank zago <fzago@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5339 tests: Un-duplicate test_24 and skip if MDS is 2.5.2 or older

Un-duplicate test_24 and disable the test if MDS version
is 2.5.2 or older.

Change-Id: I3564c44a21bf149baf09294d42edd1529179736a
Signed-off-by: Wei Liu <wei3.liu@intel.com>
Reviewed-on: http://review.whamcloud.com/11262
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5455 ptlrpc: fix magic return value of ptlrpc_init_portals

Previously, when running 'modprobe lustre', it hit the following
error message which is becaue of network initialisation failure:
modprobe: ERROR: could not insert 'lustre': Input/output error

However, error code is there, just let it return to caller,
after this patch, error message will be something like:
modprobe: ERROR: could not insert 'lustre': Network is down

Signed-off-by: Wang Shilong <wshilong@ddn.com>
Change-Id: Ib0c893052b4cd4f2e9b427fb4ce91da356065351
Reviewed-on: http://review.whamcloud.com/11337
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>