LU-5130 lod: Do not merge attr for striped directory

Does not need to merge attributes in lod_attr_get(),
because client will merge attributes itself for striped
directory (see lmv_merge_attr()), and also there are no
logic in MDD will reply on directory nlink/size/time.

Besides lod_attr_get is in the path of object initialization,
mdd_object_start()->mdd_la_get()->lod_attr_get(), if it tries
to get attribute from other MDTs, and if other MDTs are not
ready yet, it will cause unnecessary failures for object
initialization.

Change-Id: I56395a40fcd82142f99228a0776b5b7338c633ac
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10841
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5260 llite: removing ll_cl_context when error occurs

The bug is imported from commit f1b9122c of LU-5108.
It should have removed ll_cl_context from the list if failed to take
lli_write_mutex at:

	if ((iot == CIT_WRITE) &&
	    !(cio->cui_fd->fd_flags & LL_FILE_GROUP_LOCKED)) {
		if (mutex_lock_interruptible(&lli->lli_write_mutex))
			GOTO(out, result = -ERESTARTSYS);

Revise it to add ll_context right before cl_io_loop.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I6532bb8919c13692bd0ceb1ba0d755c1417e5095
Reviewed-on: http://review.whamcloud.com/10955
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Patrick Farrell <paf@cray.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5095 hsm: Allow receiving messages to be non-blocking

When using the HSM API for a copy tool, it's impossible to
properly exit an application because one thread might be
stuck in llapi_hsm_copytool_recv(), which in turn blocks
forever on a read.

So this patch expands the existing Lustre userspace HSM API
to be able to apply flags to the read side of the
communication pipe, so it can be opened with O_NONBLOCK,
and gives access to the pipe file descriptor used. (read side)

So now a program can do a poll/select, and then call
llapi_hsm_copytool_recv without blocking.

Notes: The flags passed in from the copy tool to
llapi_hsm_copytool_register are applied to the read side of
the pipe using fcntl, because we do not want to apply
the same flags to the read and write sides of the pipe.

No functionality is added for setting flags on the write
side of the pipe, since the kernel expects particular
behavior from that side of the pipe, and it seems unwise
to let user space modify it.

Additionally, this patch changes llapi_hsm_copytool_recv to
not return -EAGAIN when it finds a message for an archive
not serviced by that copytool.

Instead of returning -EAGAIN to the copy tool, which just
restarts the receive operation, this patch just loops inside
the recv function.

This is because this -EAGAIN conflicts with -EAGAIN
returned from the call to libcfs_ukuc_msg_get
which can occur when O_NONBLOCK is set.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: I3e611d3c259de9a8c30f3939fb5e48ab88210c2e
Reviewed-on: http://review.whamcloud.com/10393
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Henri Doreau <henri.doreau@cea.fr>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5223 lmv: build master LMV EA dynamically build via readdir

When creating a striped directory, the master object saves the slave
objects (or shards) as internal sub-directories. The sub-directory's
name is composed of ${shard_FID}:${shard_idx}. With the name, we can
easily to know what the shard is and where it should be.

On the other hand, we need to store some information related with the
striped directory, such as magic, hash type, shards count, and so on.
That is the LMV EA (header). We do NOT store the FID of each shard in
the LMV EA. Instead, when we need the shards' FIDs (such as readdir()
on client-side), we can build the entrie LMV EA on the MDT (in RAM) by
iterating the sub-directory entries that are contained in the master
object of the striped directroy.

Above mechanism can simplify the striped directory create operation.
For very large striped directory, logging the FIDs array in the LMV
EA will be trouble. It also simplify the LFSCK for verifying striped
directory, because it reduces the inconsistency sources.

Another fixing is about the lmv_master_fid in master LMV EA header,
it is redundant information, and may become one of the inconsistency
sources. So replace it with two __u64 padding fields.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: If7ddec892b75ff8f6e14d9f3f7a3ae0fa065239e
Reviewed-on: http://review.whamcloud.com/10751
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4249 quota: race in qsd_upd_thread()

qsd_upd_thread() uses list_for_each_entry_safe() to process
list items one by one, however, it has to drop lock while
processing each item, that'll race with other list processing
thread.

The proper way is to check list head each time when it acquired
lock.

Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I55dfe3f65cc214c5d0fd8a9afdda795af94bbdce
Reviewed-on: http://review.whamcloud.com/10988
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Jenkins
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
LU-5289 mdc: cleanup intent if mdc_finish_enqueue() fails

In mdc_enqueue() clear the lock handle, lock mode, and request stored
in the lookup intent if mdc_finish_enqueue() fails.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: I233588ab13df86be509a9763e00697d29dee1c7c
Reviewed-on: http://review.whamcloud.com/10963
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
LU-5291 vvp: Make sure ft_flags is valid

In ll_fault0, the 'fault' struct is mostly cleared before the call to
cl_io_loop, but ft_flags is not reset. It is ordinarily set by
the call to filemap_fault in vvp_io_kernel_fault, but if Lustre
returns before calling filemap_fault, it still has the old value of
ft_flags.

ll_fault0 will then consume the ft_flags field. If it has the
VM_FAULT_RETRY bit set, it will be used as ll_fault0() and
ll_fault()'s return value.

This is a problem when VM_FAULT_RETRY is in ft_flags:
When fault/filemap_fault return with that flag set, they have already
released the mmap semaphore, and do_page_fault does not need to
release it.
Incorrectly returning this flag from ll_fault means mmap_sem
is not upped in the kernel's do_page_fault().

In addition to clearing ft_flags, this patch does not use it unless
it is valid.  It's potentially misleading to return ft_flags in
"fault_ret" if ft_flags has not been set by filemap_fault.

This adds clarity, but does not change the current behavior:
When not valid, ft_flags is replaced by fault_ret, which is zero,
as is ft_flags when not set by filemap_fault.

Signed-off-by: Patrick Farrell <paf@cray.com>
Change-Id: If3a7ad87ee94aff5ecc65429c79e0e595281546d
Reviewed-on: http://review.whamcloud.com/10956
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>