LU-2855 kuc: error management in KUC broadcast delivery

In KUC broadcast groups, the message delivery fails if
a userland process terminates without closing its kuc
channel properly.
This patch improves the behavior of KUC broadcast groups,
to make it more adapted for broadcast usage: the message
delivery is successful if at least 1 userland process is
present in this group and receives
the message successfully.

Signed-off-by: Thomas Leibovici <thomas.leibovici@cea.fr>
Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: I2a121c7cfcaa6c2ee5ac48b721668bf2f254d848
Reviewed-on: http://review.whamcloud.com/5521
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2896 mgs: fix handling non IR targets.

non IR targets don't know about extra command as have single
"register target" so we need to treat - none command
as register new target

Xyratex-bug-id: MRP-880
Signed-off-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Change-Id: I22a8fd034772c9355d2d56a166fdf3766edec719
Reviewed-on: http://review.whamcloud.com/5574
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-903 ldlm: inode references moved to resource

There is a race condition while get_attr after cancel_lru_locks
and sysctl drop_caches. ll_clear_inode clears l_ast_data for
ldlm lock and this lock can't be canceled because "inode == NULL".
ll_mdc_blocking_ast finds such lock. As result DCACHE_LUSTRE_INVALID
is not set and lookup returns wrong inode.

This patch moves inode structure reference to
"ldlm_lock::l_resource::lr_lvb_inode". This prevents from different
inode references for same resource's lock.

Xyratex-bug-id: MRP-338
Reviewed-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Signed-off-by: Artem Blagodarenko <artem_blagodarenko@xyratex.com>
Change-Id: I4105b2aec38c90d3f5a20d1498a563192a74de55
Reviewed-on: http://review.whamcloud.com/2627
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2523 tests: disable racer setstripe until fixed

Disable the use of "lfs setstripe" in racer file_create.sh until
the LU-2523 and LU-2789 are fixed, so that racer can be added to
the "review" test workload instead of only in the "full" workload.

Test-Parameters: testlist=racer,racer,racer
Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I8b0814ef7864efd817ff585f715183fbf73ebbe5
Reviewed-on: http://review.whamcloud.com/5424
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2714 hsm: limit MDT-side allocations for HSM RPCs

Limit the amount of memory an MDT will allocate for a single HSM RPC
to 1 MB and add some sanity checking to the HSM handlers that use
variable length buffers.  In hur_len() compute the size of an HSM
request in a portable way.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ie03b85a8524cb377bf43446be429cc60c2fe39a7
Reviewed-on: http://review.whamcloud.com/5507
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
LU-2628 tests: disable test_40 of replay-single

This test case assumes that IO to OSTs could go on even when the
connect to MDT is lost, this is not true any more because clients
have to verify the layout is correct before operating OST objects.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I08eaad7b97da7ee152c066426f24bc1d15db5738
Reviewed-on: http://review.whamcloud.com/5056
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-2724 ptlrpc: skip NULL obd_svc_stats in lprocfs_rd_import()

In lprocfs_rd_import() don't print trivial obd_svc_stats if they were
not allocated for this import. Add a general proc file read check to
sanity.

Signed-off-by: John L. Hammond <john.hammond@intel.com>
Change-Id: Ic94056eb4f02d71491cdaf948cbe27b82de2153d
Reviewed-on: http://review.whamcloud.com/5234
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2858 mdt: check .lustre before reint operation.

1. Check .lustre in MDT layer to make sure any attempt to change
.lustre will return EPERMT.

2. send parent fid for remote open, so mdt can check whether it
will create lov object for the fid under OBF.

3. add a few test cases in sanity 154(OBF checking) to check .lustre
and remote fid

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ia708856e026a959ca30b06fdd3acd907e5d1b913
Reviewed-on: http://review.whamcloud.com/5555
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
LU-2922 ldiskfs: depend on e2fsprogs-1.42.6.wc2

This patch updates lustre-ldiskfs.spec.in to depend on
a newer e2fsprogs-1.42.6.wc2 release instead of the
old 1.41.12.2.ora1 version.

Signed-off-by: Jian Yu <jian.yu@intel.com>
Change-Id: I58828fa5427e5f50cfb3b285f980e946e7eb62e7
Reviewed-on: http://review.whamcloud.com/5623
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-2148 kernel: Kernel update for latest FC18 kernel

Add fc18 support for build system

Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: Iaeb24b5e44f969eb23a55d115b866c926b25bd55
Reviewed-on: http://review.whamcloud.com/5194
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Tested-by: Hudson
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2840 tests: Clean the environment for test.

Use unique directory for sanityn test_21. Avoid it
impacted by previous test.

Signed-off-by: yang sheng <yang.sheng@intel.com>
Change-Id: I36a06605732bb0048c25c622d4c8f1192d761a88
Reviewed-on: http://review.whamcloud.com/5556
Tested-by: Hudson
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2738 mdt: add proc entry to enable lfs mkdir for non-admin

Add enable_remote_dir_gid to enable lfs mkdir for non-admin user,
1. enable_remote_dir_gid = group, only users, whose gid == group
   can create remote dir.
2. enable_remote_dir_gid = -1, all users can create remote dir.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I6605016aa89ef3e2763ca14f94a763685fc689b6
Reviewed-on: http://review.whamcloud.com/5442
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: John Hammond <johnlockwoodhammond@gmail.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-2190 utils: Map ldd_params to individual props

At mkfs/tunefs/mount time the configuration utilities will access
a .ldd_params configuraton string.  This string consists of space
delimited key values pairs which must be saved somewhere.

For ldiskfs it makes sense to store this string on disk as part
of the binary lustre_disk_data structure.  But the logical place
for ZFS is to use the user dataset properties where all the other
lustre_disk_data fields are stored.

This patch updates the ZFS portions of the utilities to take the
.ldd_params string and break it apart in to individual key value
pairs which can be stored as user dataset properties.  This is
done by simply appending 'lustre:' to the provided param key.

The original .ldd_params field can then be reconstructed by
fetching all the dataset properties prefixed by 'lustre:' and
appending them with their keys.

The advantage in this approach is new parameters can be added
without the need to update utilties.

Change-Id: Ida1b0e845b27154b209cc807726ae6ed3d22d189
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Signed-off-by: Prakash Surya <surya1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/5220
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
LU-2859 osc: unplug IO queue async to avoid stack overflow

Otherwise, there is a stack overflow problem with the following
stacktrace:

18 cfs_trace_unlock_tcd at ffffffffa034336c [libcfs]
19 libcfs_debug_vmsg2 at ffffffffa0354038 [libcfs]
20 libcfs_debug_msg at ffffffffa03545f1 [libcfs]
21 osc_key_init at ffffffffa0a90e17 [osc]
22 keys_fill at ffffffffa06d8c3f [obdclass]
23 lu_context_init at ffffffffa06dcc6b [obdclass]
24 lu_env_init at ffffffffa06dce3e [obdclass]
25 cl_env_new at ffffffffa06e40bd [obdclass]
26 cl_env_get at ffffffffa06e4acb [obdclass]
27 lov_sub_get at ffffffffa0b3777d [lov]
28 lov_page_subio at ffffffffa0b37c5d [lov]
29 lov_page_own at ffffffffa0b311af [lov]
30 cl_page_own0 at ffffffffa06e8b5b [obdclass]
31 cl_page_own_try at ffffffffa06e8db3 [obdclass]
32 discard_pagevec at ffffffffa0a926a9 [osc]
33 osc_lru_shrink at ffffffffa0a935d9 [osc]
34 osc_lru_del at ffffffffa0a94aa6 [osc]
35 osc_page_delete at ffffffffa0a951b4 [osc]
36 cl_page_delete0 at ffffffffa06e99e5 [obdclass]
37 cl_page_delete at ffffffffa06e9e62 [obdclass]
38 ll_releasepage at ffffffffa0bfe41b [lustre]
39 try_to_release_page at ffffffff81110070
40 shrink_page_list.clone.0 at ffffffff8112a501
41 shrink_inactive_list at ffffffff8112a8cb
42 shrink_zone at ffffffff8112b5df
43 zone_reclaim at ffffffff8112c384
44 get_page_from_freelist at ffffffff81122834
45 __alloc_pages_nodemask at ffffffff81123ab1
46 kmem_getpages at ffffffff8115e2f2
47 cache_grow at ffffffff8115e95f
48 cache_alloc_refill at ffffffff8115ebb2
49 __kmalloc at ffffffff8115f8d9
50 cfs_alloc at ffffffffa0344c40 [libcfs]
51 ptlrpc_request_alloc_internal at ffffffffa085d407 [ptlrpc]
52 ptlrpc_request_alloc_pool at ffffffffa085d66e [ptlrpc]
53 osc_brw_prep_request at ffffffffa0a8451b [osc]
54 osc_build_rpc at ffffffffa0a8a513 [osc]
55 osc_io_unplug0 at ffffffffa0aa642d [osc]
56 osc_io_unplug at ffffffffa0aa7ce1 [osc]
57 osc_enter_cache at ffffffffa0aa8473 [osc]
58 osc_queue_async_io at ffffffffa0aae916 [osc]
59 osc_page_cache_add at ffffffffa0a94fc9 [osc]
60 cl_page_cache_add at ffffffffa06e61d7 [obdclass]
61 lov_page_cache_add at ffffffffa0b31325 [lov]
62 cl_page_cache_add at ffffffffa06e61d7 [obdclass]
63 vvp_io_commit_write at ffffffffa0c1161d [lustre]
64 cl_io_commit_write at ffffffffa06f5b1d [obdclass]
65 ll_commit_write at ffffffffa0be68be [lustre]
66 ll_write_end at ffffffffa0bfe4e0 [lustre]
67 generic_file_buffered_write at ffffffff81111684
68 __generic_file_aio_write at ffffffff81112f70
69 generic_file_aio_write at ffffffff8111320f
70 vvp_io_write_start at ffffffffa0c11f3c [lustre]
71 cl_io_start at ffffffffa06f244a [obdclass]
72 cl_io_loop at ffffffffa06f6d54 [obdclass]
73 ll_file_io_generic at ffffffffa0bbda7b [lustre]
74 ll_file_aio_write at ffffffffa0bbdce2 [lustre]
75 ll_file_write at ffffffffa0bbeaac [lustre]
76 vfs_write at ffffffff81177b98
77 sys_write at ffffffff811785a1

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Id61b481fe2036a4d7adb7140c39e50fe61c264ba
Reviewed-on: http://review.whamcloud.com/5526
Tested-by: Hudson
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2730 mdt: fix erroneous LASSERT in mdt_reint_opcode

Only set return code err_serious(-EFAULT) when neccessary and do not
run unknown opcode through err_serious.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Iea616d8afd676ee5ccda52cf09b398198f38f992
Reviewed-on: http://review.whamcloud.com/5416
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-2694 test: fix is_empty_fs in t-f

The original is_empty_fs is incorrect, which can cause unexpected
error when running lfsck.sh, because lfsck sets the max error
level based on whether filesystem is empty.

Test-Parameters: testlist=lfsck
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I2269cc41744e3c9fe228898323a1508a03616efe
Reviewed-on: http://review.whamcloud.com/5576
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2694 test: improve lfsck.sh

The test directory of lfsck.sh contains some files referencing
same object, which could cause error when removing the directory
on test cleanup.

Test-Parameters: testlist=lfsck
Signed-off-by: Niu Yawei <yawei.niu@intel.com>
Change-Id: I334ff2b7b5f77498eed940f009e4bc18728bb5da
Reviewed-on: http://review.whamcloud.com/5606
Tested-by: Hudson
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2804 mdd: move ACL mode handling to MDD

Move ACL mode handling from OSD to MDD, so both ldiskfs and zfs
can be set mode correctly, and also by this, it can avoid to
transfer the local mask to the remote MDT for fixing the mode.

Move sanity 103 out of sanity SLOW list, so it can be run in the
normal review/landing check, so to avoid regression about ACL.
And also it only take 15 seconds in my local VM run, which probably
not be put to the SLOW list at all.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I73146425baa7d8b712ce46e18955ecaa2a3fd9a4
Reviewed-on: http://review.whamcloud.com/5421
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2899 lod: get ldo_stripenr correctly

Current code relies on lod_statfs_and_check() to count the number of
activated LOD targets, while lod::ldo_stripenr derivation happens
before calling lod_statfs_and_check(), and that makes
lod::ldo_stripenr not accurate.

This patch make sure lod_statfs_and_check() called before updating
::ldo_stripenr. And if there is [de]activation happens on OST target,
client needs wait 2*lod_qos_maxage seconds to get accurate
ld_active_tgt_count number.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I37bebc69f876dd68da609fb5180bc6db36f01e84
Reviewed-on: http://review.whamcloud.com/5573
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-1199 build: Cleanup ldiskfs kernel config defines

Make it more clear what the purpose of the "CONFIG_LDISKFS_*"
defines are with a comment, and separate them from options
that do not necessarily originate in the kernel source. All of
these options are needed for ldiskfs and some are needed by
osd-ldiskfs in the lustre file system. To handle these shared
options a extra autoconf header file is create containing only
the values of interest to osd-ldiskfs.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I8e71b6cf471c4317b20fdce14d66f8b2883a226e