LU-4923 lfsck: detailed statistics for namespace LFSCK

Some enhancement for namespace LFSCK:

1) For the MDT device upgraded from Lustre-1.x and still not enable
   the 'dirdata' feature, then send warning message when mount.

2) More detailed statistics for how many FID-in-dirent entries have
   been repaired and how many linkEA entries have been repaired.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I6a2468b1b8ac4dee91d5d3a26872214391fd3e3b
Reviewed-on: http://review.whamcloud.com/10030
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4725 mdt: child-parent lock ordering in rename

change rename so that it always has parent-child lock ordering,
otherwise it may deadlock with other operations.

Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: If676da82ca50a20a4bb3aadef0f81c9c5ed3cbcb
Xyratex-bug-id: MRP-1700
Reviewed-on: http://review.whamcloud.com/9538
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4955 lfsck: reset param when the LFSCK is reset

If "--reset" or "-r" is specified when start the LFSCK manually,
then all non-specified parameters will be set as default values.

If the former LFSCK instance completed, then when re-trigger the
LFSCK, it will restart from the beginning, and all non-specified
parameters will be set as default values.

Change the "window" option in lfsck_start to "window_size".

Update the lctl manpage for LFSCK.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I5b1d7dd2507ab2f50e152f5a839bf9544a9dc255
Reviewed-on: http://review.whamcloud.com/10091
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
LU-4972 lfsck: skip .lustre and children for namespace check

The sub-directories/files under .lustre are either for partial
repaired objects (such as .lustre/lost+found/) or for special
non-real objects (such as .lustre/obf). It is unnecessary to
do namespace LFSCK for them.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I39e8382c226b2ee19a41e6f47af38a5367f3bdd8
Reviewed-on: http://review.whamcloud.com/10143
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
LU-4609 ofd: auto resume LFSCK after the recovery

To prevent the LFSCK to recreate some objects which should be done
via some replayable RPCs, we will postpone the LFSCK auto assuming
after the server restart until the recovery finished.

Such serialization also avoid some race between LFSCK and recovery
to misguide the LFSCK to regard the system as inconsistent.

Another fix is that:
During the OST recovery, the client write RPC may create the missed
OST-object which will cause the real last_id to be greater than the
last_id value stored in the LAST_ID file temporarily. Normally, the
LAST_ID file will be synced between MDT and OST during the recovery,
but we should not assume that the MDT-OST recovery will succeed. If
it failed, then subsequent LFSCK on the OST may regard such LAST_ID
file as crashed, then it will mark the OST as read-only and rebuild
the LAST_ID files.

To avoid such case, before the write RPC to create the lost object,
it needs to update the LAST_ID file firstly.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I963f3c1b70c7cad0c943f2485417d2e783768bf1
Reviewed-on: http://review.whamcloud.com/10010
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-4528 llog: don't write llog in 3-steps

The llog record my be written by passing just buffer
and writting its header and tail separately. If any write fails
then that cause partially written buffer and corrupted llog. There is
special undo procedure to clean things up after that but that demands
twice more credits for normal operations and for undo case which
is critical in case of wide striping.

Such 3-steps write can be avoided by preparing correct llog record
before calling llog_write().

Patch does the following:
- remove all cases with passing separate buffer for llog_write(),
llog_add(), llog_cat_add() and change API according with that.
- special allocation case for lustre_cfg if it is going to be written
to the llog, it allocates llog header/tail and initialize it.
- uniform error checks after lustre_cfg_new() call to check for NULL
result always instead of ERR_PTR()
- remove 3-steps write in llog_osd code completely
- write llog padding record in single step too
- remove duplicated code from llog_cat_new_log(). Previously it
assigned the next index for the record, changed bitmap and called
llog_write with new index, so that was considered as 'modification'
of record but not append. Meanwhile the llog_osd_write_rec() does
the same intenally for append request. Now llog_cat_new_log() just
calls llog_write() with -1 index (append), avoiding all header
changes outside that call.
- remove numcookie parameter from llog_write_rec() because it is
always 1 anyway
- remove cookie parameter from llog_write() because it is not used,
note, the cookie remains as parameter of llog_write_rec() for llog
catalog purposes.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ibf231021ece9d7cb8a3c4288e12077e11f691661
Reviewed-on: http://review.whamcloud.com/10108
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-1778 llite: fix inconsistencies of root squash feature

Root squash exhibits inconsistent behaviour on a client when
enabled. If a file is not cached on the client, then root will get
a permission denied error when accessing the file. When
the file has recently been accessed by a regular user and is
still in cache, root will be able to access the file without error
because the permission check is only done by the client that
isn't aware of root squash.

While the only real security benefit from root squash is to deny
clients access to files owned by root itself, it also makes sense
to treat file access on the client in a consistent manner
regardless of whether the file is in cache or not.

This patch adds root squash settings to llite so that client is able
to apply root squashing when it is relevant.

Configuration of MDT root squash settings will automatically be
applied to llite config log as well.

Update cfs_str2num_check() routine by removing any modification
of the specified string parameter. Since string can come from ls_str
field of a lstr structure, this avoids inconsistent ls_len field.

Signed-off-by: Gregoire Pichon <gregoire.pichon@bull.net>
Change-Id: Ie52d04424d0937af9fed777a4e4912c90405c19b
Reviewed-on: http://review.whamcloud.com/5700
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>