LU-4970 tests: wait async LFSCK updates to be done

There may be some async LFSCK updates in processing when the
test scripts check the reparation result. So make the scripts
to wait/retry until the async updates has been done or timeout.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I71a1986de432631ec7158a7e90797c46ae672812
Reviewed-on: http://review.whamcloud.com/11590
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
LU-4788 lfsck: replace cfs_list_t with list_head

Do not use cfs_list_xxx any more in LFSCK.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I74bd14fe37274b06762ce5f01aec6e1288837000
Reviewed-on: http://review.whamcloud.com/10602
Tested-by: Jenkins
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
LU-4788 lfsck: LFSCK code framework adjustment (1)

The LFSCK wrap functions are only used by the LFSCK engines.
So move these functions from lfsck_lib.c to lfsck_engine.c.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Ic192c71101c1718fe893b64f8e4ff74f4992914b
Reviewed-on: http://review.whamcloud.com/10493
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
LU-5395 lfsck: misc patch to prevent lfsck hung

1) When the LFSCK rebuilt the crashed LAST_ID files, it will
   notify the MDS to sync lastid information via disconnecting
   the connection. OFD should hold the export reference before
   disconnecting to allow to send RPC reply message.

2) When the layout LFSCK scans on the OST, it needs to handle
   the IDIF objects specially (use fid_idif_id() to get the
   OST object ID) to avoid to regard the LAST_ID file as
   corrupted by wrong.

3) The LFSCK should check the ostid_to_fid() return value for
   corrupted OSTID and/or index.

4) If the LAST_ID file is not crashed, then do not update the
   LAST_ID file.

5) NOT change the lu_buf::lb_len once the lu_buf::lb_buf is
   allocated to prevent accessing released or non-allocated
   RAM space by wrong.

6) Other small fixes and code cleanup.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I84726ddcf0b8fa6b334163fb13d9bae273033d20
Reviewed-on: http://review.whamcloud.com/11304
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5395 lfsck: deadlock between LFSCK and destroy

There is potential deadlock race condition between object
destroy and layout LFSCK. Consider the following scenario:

1) The LFSCK thread obtained the parent object firstly, at
   that time, the parent object has not been destroyed yet.

2) One RPC service thread destroyed the parent and all its
   children objects. Because the LFSCK is referencing the
   parent object, then the parent object will be marked as
   dying in RAM. On the other hand, the parent object is
   referencing all its children objects, then all children
   objects will be marked as dying in RAM also.

3) The LFSCK thread tries to find some child object with
   the parent object referenced. Then it will find that the
   child object is dying. According to the object visibility
   rules: the object with dying flag cannot be returned to
   others. So the LFSCK thread has to wait until the dying
   object has been purged from RAM, then it can allocate a
   new object (with the same FID) in RAM. Unfortunately, the
   LFSCK thread itself is referencing the parent object, and
   cause the parent object cannot be purged, then cause the
   child object cannot be purged also. So the LFSCK thread
   will fall into deadlock.

We introduce non-blocked version lu_object_find() to allow
the LFSCK thread to return failure immediately (instead of
wait) when it finds dying (child) object, then the LFSCK
thread can check whether the parent object is dying or not.
So avoid above deadlock.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I7f465259011ad5fb92ef1b4dba0ff9f46d134352
Reviewed-on: http://review.whamcloud.com/11373
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4788 lfsck: take ldlm lock before modifying visible object

Before the LFSCK modifying on the namespace visible object,
it needs to acquire related ibits lock to pervent the client
to cache stale information. The .lustre/lost+found/ and its
sub-directories also needs related ldlm locks.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I439e02e1b7b24e87e7e6e25c5084f1c98116e7f7
Reviewed-on: http://review.whamcloud.com/10986
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
LU-4975 doxygen: add comments for lproc_osp.c

- Fix up GPL header block to reference proper GPLv2 license URL.
- Remove mention of contacting Sun.
- Add introductionary comment block for the lproc_osp.c file.
- Add function comments blocks to all functions in the lproc_osp.c
  file

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: If037ab1777b0640e8cd6b66f0a78ca54339907ff
Reviewed-on: http://review.whamcloud.com/10893
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-3963 libcfs: Delete empty source files for linux layer

Some of the files in libcfs/libcfs/linux are actually
empty. Remove them.

Change-Id: Ic49d3bea7777488ab51fdcdc1f93ed516654c665
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/11725
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4928 utils: LLAPI helpers for file leases

Introduced llapi_lease_{get,check,put} functions to abstract the ioctl
implementation to manipulate file leases.

Signed-off-by: Henri Doreau <henri.doreau@cea.fr>
Change-Id: I956408defe550c1b432526e142825ea736d0e285
Reviewed-on: http://review.whamcloud.com/10022
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
LU-2675 lustre: remove linux/lustre_common.h

Remove lustre/include/linux/lustre_common.h and several unnecessary
calls to cfs_cleanup_group_info().

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I149673dea6559b02a5de1c0a160836d67ea96119
Reviewed-on: http://review.whamcloud.com/11495
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5540 lnet: fix crash due to NULL networks string

If there is an invalid networks or ip2nets lnet_parse_networks()
gets called with a NULL 'network' string parameter

lnet_parse_networks() needs to sanitize its input string now that
it's being called from multiple places.  Instead, check for
a NULL string everytime the function is called, which reduces the
probability of errors with other code modifications.

Signed-off-by: Amir Shehata <amir.shehata@intel.com>
Change-Id: Ifcc1f6f74a3e0e804cb65e3d1b83f85a24f44d9b
Reviewed-on: http://review.whamcloud.com/11626
Tested-by: Jenkins
Reviewed-by: Isaac Huang <he.huang@intel.com>
Reviewed-by: Doug Oucharek <doug.s.oucharek@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>