LU-5707 lfsck: store namespace LFSCK statistics info in new EA

For Lustre-2.6 or older release, the namespace LFSCK statistics info
was stored as XATTR_NAME_LFSCK_NAMESPACE EA, but in Lustre-2.7, the
namespace LFSCK will introduce more statistics information that will
cause the XATTR_NAME_LFSCK_NAMESPACE EA to be extended. If it still
uses the old XATTR_NAME_LFSCK_NAMESPACE EA, then when downgrade, the
old LFSCK will get -ERANGE when load the new trace file from disk,
and then the LFSCK cannot be started after downgrade.

To avoid such trouble, Lustre-2.7 will use new EA to store the
namespace LFSCK statistics info: XATTR_NAME_LFSCK_NAMESPACE_V2,
and keep a dummy XATTR_NAME_LFSCK_NAMESPACE EA in the trace file
to be compatible with old LFSCK.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I55b5adb962434013b00e3938a67b671010ecc206
Reviewed-on: http://review.whamcloud.com/12321
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
LU-4839 tests: Give copytool more time to start

Copytool can take some time to start, and if the HSM archive directory
is on a busy NFS server, it can take a bit of time for the initial
opens to occur.  This allows those actions more time to complete which
should give this test a better chance of passing correctly.

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes \
mdtfilesystemtype=zfs mdsfilesystemtype=zfs ostfilesystemtype=zfs \
testlist=sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm

Test-Parameters: alwaysuploadlogs envdefinitions=SLOW=yes,ONLY=60 \
mdtfilesystemtype=ldiskfs mdsfilesystemtype=ldiskfs ostfilesystemtype=ldiskfs \
mdtcount=4 testlist=sanity-hsm,sanity-hsm,sanity-hsm,sanity-hsm

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I28bc57b92c34b4eee07ba34a2d976f2c39dc70dc
Reviewed-on: http://review.whamcloud.com/12682
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Michael MacDonald <michael.macdonald@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5731 osp: flush async updates for osp_sync

Current osp_sync() only considers the async requests that are
handled by the osp_sync_thread, but ignores the async updates
that are handled directly by the background ptlrpcd threads.
Usually, such async updates are for LFSCK remote repairing.
This patch will flush all of them when dt_sync() is called.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I0e6d54120acbd8ab82cf776222277ae3b805812d
Reviewed-on: http://review.whamcloud.com/12359
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5568 lnet: fix kernel crash when network failed to start

When loading Lustre modules without proper network configuration,
it always hit the following kernel panic:
LNetError: 105-4: Error -100 starting up LNI tcp
LNetError: 2145:0:(api-ni.c:823:lnet_unprepare())
 ASSERTION( list_empty(&the_lnet.ln_nis) ) failed:
LNetError: 2145:0:(api-ni.c:823:lnet_unprepare()) LBUG
Pid: 2145, comm: modprobe
x0aCall Trace:
[<ffffffffa044f853>] libcfs_debug_dumpstack+0x53/0x80 [libcfs]
[<ffffffffa044fdf5>] lbug_with_loc+0x45/0xc0 [libcfs]
[<ffffffffa04f3267>] lnet_unprepare+0x297/0x340 [lnet]
[<ffffffffa04f3b5c>] LNetNIInit+0x25c/0x3e0 [lnet]
[<ffffffff81061bc6>] ? put_online_cpus+0x56/0x80
[<ffffffffa0983000>] ? init_module+0x0/0x1000 [ptlrpc]
[<ffffffffa081310c>] ptlrpc_ni_init+0x2c/0x1a0 [ptlrpc]
[<ffffffffa0983000>] ? init_module+0x0/0x1000 [ptlrpc]
[<ffffffffa0813291>] ptlrpc_init_portals+0x11/0xf0 [ptlrpc]
[<ffffffffa0983000>] ? init_module+0x0/0x1000 [ptlrpc]
[<ffffffffa09831c4>] init_module+0x1c4/0x1000 [ptlrpc]
[<ffffffff810020e2>] do_one_initcall+0xe2/0x190
[<ffffffff810ca7fb>] load_module+0x129b/0x1a90
[<ffffffff812da590>] ? ddebug_dyndbg_module_param_cb+0x0/0x60
[<ffffffff810c7133>] ? copy_module_from_fd.isra.43+0x53/0x150
[<ffffffff810cb1a6>] SyS_finit_module+0xa6/0xd0
[<ffffffff815f2119>] system_call_fastpath+0x16/0x1b
...
This is because in lnet_startup_lndnis(), we may add list items to
@the_lnet.ln_nis and @the_lnet.ln_nis_cpt before it failed. But in
lnet_startup_lndis() failure path,it did not cleanup list thus
causing assertion in lnet_unprepare().

Fix the assertion by cleaning up using lnet_shutdown_lndnis()
if the startup fails.

In a future enahancement the ni startup API will be modified to
cleanup after itself in case of failure.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ia344fd7c0f24c87b654554dda9e57bf5525edc85
Reviewed-on: http://review.whamcloud.com/12512
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5832 utils: Fix buffer overflow in bound string copy

The function 'strncpy' may incorrectly check buffer boundaries
and may overflow buffer 'info->name' of fixed size (256). Also
there is one similar error on line 1135.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I512ab6678fbf1d02bac2eb290fd13c22fca9dc2b
Reviewed-on: http://review.whamcloud.com/12516
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5892 lfsck: remove improper LASSERT in lfsck_needs_scan_dir

Inside the lfsck_needs_scan_dir(), when the internal variable @fid
becomes the input @obj's parent FID, the internal variable @depth
may be still zero, so the original "LASSERT(depth > 0);" is improper
under such case. Then remove it.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I64f10be682c51c6ac5cc1af3497eb569281fcd21
Reviewed-on: http://review.whamcloud.com/12670
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5641 tests: ensure user daemon is in group bin on mds

The previous fix for this problem only fixed groups on client.
That worked as long as we were only testing with el7 client,
but was an incomplete solution for el7 client/servers.
Need to apply the same fix to mds too to keep things consistent.

Signed-off-by: Bob Gossman <bob.glossman@intel.com>
Change-Id: I411970c591a72b0393ed892f15da1f5d6340df8c
Reviewed-on: http://review.whamcloud.com/12762
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5848 debug: more debug log for dt_sync

Add some D_CACHE logs at the entry/exit for osp_sync()/osd_sync().

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iaa7fbfbbadb9312528b5092d64615b277de6b679
Reviewed-on: http://review.whamcloud.com/12573
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>