LU-3594 lfsck: repair inconsistent owner and multiple referenced cases

Sometimes, the OST-object owner information is inconsistent with the
MDT-object owner information because of incompleted chown/chgrp, or
other system crash. Under such case, the MDT-object owner information
is trusted over the OST-object's. Because the chown/chgrp processing
order is: client => MDT => OST, it is possible that the OST-object
owner information is stale rather than the MDT-object's. Also, the
MDT-object's owner information is visible to users and can be directly
repaired by the system administrator, while the OST-object's owner
information is only used internally by quota. So the LFSCK will update
the OST-object owner information according to the MDT-object's owner.

If both MDT-object1 and MDT-object2 claim the OST-object1 as one
of its child OST-object, but the OST-object1 only recognizes the
MDT-object1, then the LFSCK will create new a OST-object and fix
the MDT-object2's layout information to reference the new created
OST-object.

Replace is_remote_th() with is_only_remote_trans(), then drop the
compat patch http://review.whamcloud.com/9361

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I6b148180b5a2d68650b291250c03aac651e5f6e9
Reviewed-on: http://review.whamcloud.com/7524
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3336 lfsck: use rbtree to record OST-object accessing

To find out orphan OST-objects, the LFSCK on OST side maintains
two bitmaps in RAM for the OST-object accessed during the LFSCK.
After the first cycle system scanning, the LFSCK got the bitmap
for the known OST-objects, and got another bitmap for which OST
objects have been referenced by MDT-objects. Then the LFSCK can
know which OST-objects are not referenced by any MDT-object via
comparing the two bitmaps.

Above two bitmaps are organized via a single rbtree. The rbtree
is maintained by LFSCK on the OST side. For every LFSCK scanned
OST-object, it will be recorded in the known-bitmap, for every
OST-object accessed by any RPC during the scanning, it will be
recoreded in the accessed-bitmap.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If399584a8617e7c368e48922a3582294ac98d5f4
Reviewed-on: http://review.whamcloud.com/7743
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3950 lfsck: control all LFSCK nodes via single command (2)

The single command should work for not only layout LFSCK, but also for
other LFSCK components, such as namespace LFSCK, OI scrub on each node
and DNE LFSCK in the future.

Introduce another lfsck_start option "-o" for enable orphan handling.
Currently it is used for orphan OST-objects handling. When enable it,
the layout LFSCK will be triggered on all servers by default.

Code cleanup and more log information.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Iaed9ee61d3d0fced32f9dd6b2a7f6663de6d2dc7
Reviewed-on: http://review.whamcloud.com/9257
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3531 mdt: delete striped directory

Add delete striped directory, it includes

1. enable sync log between MDTs, so slave objects will
be delete by unlink log, which is similar as deleting ost
object.

2. retrieve layout information of striped directory on MDT,
then lock all of the slave objects before unlink.

3. remove a few unnecessary cfs_size_round, because update_size
and update_buf_size already do size_around inside.

4. add sanity 300 for striped dir test

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ib461156bbff9e416ac9d2500a0b9491427542340
Reviewed-on: http://review.whamcloud.com/7445
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
New tag 2.5.56

Change-Id: I9f095e4102fc46f7ed9829a6056ed83c0ac3f692
Signed-off-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4471 mdd: mdd_unlink: do trans_start after sanity check

Currently, mdd_trans_start is called before
mdd_unlink_sanity_check. This means a remote directory
which has files in it can be removed on MDT0 before the
sanity check on MDT1 finds the files and errors, which
orphans the files on MDT1. This patch moves the sanity
check before mdd_trans_create and mdd_trans_start.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I08882b682b9f0016577214821efec4759ee5c184
Reviewed-on: http://review.whamcloud.com/8827
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
LU-4460 mount: fix lmd_parse() to handle comma-separated NIDs

This patch reverts commit 3917e62018878dfffac59ceed70f20b0419945d3,
which cannot handle the upgrade situation that old mountdata already
contains comma-separated NIDs. The correct way to fix the original
issue is to parse comma-separated NIDs in lmd_parse().

The patch also updates disk2_4-ldiskfs.tar.bz2 to make the mountdata
of ost contain comma-separated NIDs so as to verify the patch under
upgrade situation.

Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes testlist=conf-sanity

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: If179618c9c89dc2168f748aeba59384ea31197ff
Reviewed-on: http://review.whamcloud.com/8918
Tested-by: Maloo <hpdd-maloo@intel.com>
Tested-by: Jenkins
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>