LU-3561 tests: improve sanity-hsm.sh to support remote agent

In the current sanity-hsm.sh, only one HSM copytool is started on
the local Lustre client node.

In order to test multiple remote HSM agents/copytools, this patch
introduces a new facet type named "AGT" and the following associated
variables:

- AGTCOUNT: number of agents
- AGTDEV{N}: target HSM mount point (root path of the backend)
- agt{N}_HOST: hostname of the agent agt{N}
- SINGLEAGT: facet of the single agent

By default, only single copytool is started on a remote client node.
If there was no remote client, then the copytool will be started on
the local client.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I6e85d6948da894bd50148b0504347159ae23a79b
Reviewed-on: http://review.whamcloud.com/7110
Tested-by: Hudson
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3647 hsm: Add support to drop all pages for ll_data_version

This will be used by HSM release to get data version and drop all
caching pages from all clients, before sending IT_RELEASE close
REQ to MDT.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I670521881d77dfa584e9124d21eb65b048cc82e9
Reviewed-on: http://review.whamcloud.com/6794
Tested-by: Hudson
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3097 build: fix 'no effect' errors

Fix 'no effect' issues found by Coverity version 6.5.1:
Unsigned compared against 0 (NO_EFFECT)
This greater-than-or-equal-to-zero comparison of an unsigned value
is always true.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: Ic832e990c428ba674dcb8ba0f04c2bbf78fa8ee9
Reviewed-on: http://review.whamcloud.com/7166
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3508 test: small fix for sanity.sh test_101c

Filter out those "found no match" proc entries in sanity.sh test_101c
and do some code cleanup.

Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: Ie56c058318c52c8809a21ed242e9bb797b7d421a
Reviewed-on: http://review.whamcloud.com/6854
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3763 utils: set multipath devices recursively

Make set_blockdev_tunables() set tunables recursively to reach
slaves' slaves.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: Ic56e433758c91d3cbf5063f0d5e652b57c607a1e
Reviewed-on: http://review.whamcloud.com/7337
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3703 tests: skip getfattr part of sanity test 234

Test fails due to a known bug in the upstream getfattr command in
SLES clients.  Skip the failing part of the test to work around
this problem.

If the upstream fix already present in source and in RHEL releases ever
lands in SLES releases this mod can be removed.

Signed-off-by: Bob Glossman <bob.glossman@intel.com>
Change-Id: I153713dc0dd43178d16f3c5bdf79c7ce07b6bbef
Reviewed-on: http://review.whamcloud.com/7306
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3720 test: use actual stripe count in sanity.sh 130d

The test is expecting each OST to get a specific amount if data based
in the size and stripe count, but if an OST is full, more data will be
written to each of the other OSTs.
To fix that, use the actual number of stripes allocated instead of the
requested number of stripes.

Signed-off-by: Emoly Liu <emoly.liu@intel.com>
Change-Id: Iadb0daf16d7c121d2104f1fe49afa7f762a7f52a
Reviewed-on: http://review.whamcloud.com/7303
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: James Nunez <james.a.nunez@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2253 tests: get proper free indoes

In the DNE environment, usually we want to get free indoes on the
master MDT but not the whole filesystem.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I3bbae177ba556eaa7a7cdb582448c94d24c57bfb
Reviewed-on: http://review.whamcloud.com/7375
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2675 cleanup: define sparse annotations for libcfs

Add empty definitions for the sparse checker annotations (__user,
__kernel, __safe, ...) defined by the kernel. This will allow these
annotations to be used in files common to kernel modules and libcfs.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I26a7d3fad0bc07535db4dfaa25e40bb10ff4e951
Reviewed-on: http://review.whamcloud.com/6969
Tested-by: Hudson
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2484 obd: add md_stats to MDC and LMV devices

Enable md_stats for MDC and LMV devices and fix the definition of
EXP_MD_COUNTER_INCREMENT() so that they will be tallied. These stats
track usage of the md_ops methods (from the OBD layer, not to be
confused with md_device methods) and are exported through the files
/proc/fs/lustre/{lmv,mdc}/*/md_stats.

Rename m_sync to m_fsync making the counter name (fsync) more
intuitive. Prune the minimally useful operations from set of md_ops to
be counted. The operations counted are close, create, enqueue,
getattr, intent_lock, link, rename, setattr, fsync, readpage, unlink,
setxattr, egtxattr, intent_getattr_async, and revalidate_lock.

Add assertions to lprocfs_counter_{add,sub}() to ensure that the
counter index lies in the appropriate range for stats. Remove the
corresponding assertions from OBD_COUNTER_INCREMENT(),
EXP_COUNTER_INCREMENT(), and EXP_MD_COUNTER_INCREMENT(). Add macros
NUM_OBD_STATS and NUM_MD_STATS for use when allocating stats.

In mdt_export_stats_init() allocate and initialize the nid_stats
member of struct exp_nid_stats using only the MDT private
counters. This does not affect the contents of
/proc/fs/lustre/mdt/*/exports/*/stats but does save memory.

Add a flag to struct obd_device to indicate if EXP_COUNTER_INCREMENT()
should increment stats in the nid_stats member (as in the case of OFD)
or not (MDT).

Remove the unused member exp_md_stats from struct obd_export.

Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Idccd30e13adbc11d10b544cdfbe1d23123176e94
Reviewed-on: http://review.whamcloud.com/4827
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3787 test: Missing "$" on variable

In test 90 of replay-single, the argument of
getstripe is $dir/file when it really should be
$dir/$file. Getstripe fails since "file" does
not exist.

The patch adds the "$" to the argument of getstripe.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: I253c358d2912b57b54024f02a0a1ca54094369a7
Reviewed-on: http://review.whamcloud.com/7403
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3768 tunefs: make tunefs.lustre work correctly for MGS

Since LDD_F_NEED_INDEX flag is unset for a stand-alone MGS during
mkfs.lustre, when running tunefs.lustre on that MGS, it always says
"'----index' only valid for MDT,OST". To fix that, this patch is to
- print that error message only for mkfs.lustre
- fix the typo '----index' in the error message.

Signed-off-by: Liu Ying <emoly.liu@intel.com>
Change-Id: I4fa00dc554521c03671602ff9b2bbaca02c6a581
Reviewed-on: http://review.whamcloud.com/7353
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2800 autoconf: remove LC_SECURITY_PLUG test

Only the SLES 10 kernel took a vfsmount for various vfs operations
to support AppArmor before the path variants were available.
We can remove the test and the dead code.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I5d4b1c1c9c4994582c51c39ef3b7375b2e0997d3
Reviewed-on: http://review.whamcloud.com/5360
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3497 build: Use alt. path for ZFS development headers

In the move to dkms the ability to use ZFS userland developement
headers from the ZFS source tree instead of /usr/include was
removed. This patch restores the ability to build lustre using
ZFS on any build box that lacks having the ZFS developement rpms
installed.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I4307d07072f655df86f542686ece71e11cc97045
Reviewed-on: http://review.whamcloud.com/6750
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3549 llapi: add printf attribute to llapi_{printf,error}

Add printf attributes to llapi_printf and llapi_error for compile time
checking that the format string agrees with the argument list, and
const qualify the format parameter of both functions. Fixup the
mismatched calls to these functions.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: Ic8df148abb0bfe8ce72d05d76cb7ce0f1832c28f
Reviewed-on: http://review.whamcloud.com/6867
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-3581 osc: Lustre returns EINTR from writes when SA_RESTART is set

When Lustre is in a read or write system call and receives a
SIGALRM, it will return EINTR if interrupted in osc_enter_cache.
This prevents the system call from being restarted if
SA_RESTART is set in the handler.

This patch changes behavior in this location to return ERESTARTSYS
when a signal arrives during the call to l_wait_event.

Signed-off-by: Patrick Farrell <paf@cray.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I95423c5d572974352173f3c5c94d980385bfef29
Reviewed-on: http://review.whamcloud.com/7002
Reviewed-by: Rahul Deshmukh <rahul_deshmukh@xyratex.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Cory Spitz <spitzcor@cray.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-3789 mgs: Add deprecation warning for "lctl conf_param"

lctl set_param -P has been added with LU-3155. Old mechanism
(lctl conf_param) should be deprecated.

This patch adds warnings since 2.7.53 and disable lctl set_param
option since 3.0.53

Xyratex-bug-id: MRP-272
Signed-off-by: Artem Blagodarenko <artem_blagodarenko@xyratex.com>
Change-Id: I37479a333869de1d10b74b623bd4d921774664e8
Reviewed-on: http://review.whamcloud.com/7436
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
LU-2800 autoconf: remove LC_FS_RENAME_DOES_D_MOVE test

FS_RENAME_DOES_D_MOVE has been upstream since v2.6.19.
We can remove the test and the dead code.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Change-Id: I71fc1e7c2216861dff2ce31f45c8ecaea64e7cfa
Reviewed-on: http://review.whamcloud.com/5348
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2800 autoconf: update LC_INODE_PERMISION_2ARGS test

inode_operations->permission has changed over time:
v2.6.27..v2.6.37 had a 2 arg version
v2.6.28..v3.0 had a 3 arg version
v3.1.. has a 2 arg version again

We move the test so it makes more sense in the list.

Signed-off-by: Jeff Mahoney <jeffm@suse.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I0e9b2a35e934f8efb3bd932303cdee859d7a6b44
Reviewed-on: http://review.whamcloud.com/5363
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3841 lod: handle released defined layouts

In lod_use_defined_striping() covert layout magic to non _DEF form
before passing to lov_mds_md_size() and check that the supplied buffer
is sufficiently large only after accounting for released layouts.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ic5fdebaf0bc4a9327d6f5bba1be048bb033b1d3a
Reviewed-on: http://review.whamcloud.com/7467
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>