LU-2214 lod: fix tricky iterator methods

instead of bypassing LOD layer in the iterator methods,
just get own iterator structure in lod, which keep references
to the object and the iterator of the layer below.

this also let LOD to have different iterators in different
objects which is required for DNE.

to verify the approach lfsck goes through LOD now.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I62935319a686f4b06b2cdf5ea4002a800c0c430d
Reviewed-on: http://review.whamcloud.com/4370
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: wangdi <di.wang@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2219 ptlrpc: so_hpreq_handler is set twice for the ost_io svc

ptlrpc_service_conf.psc_ops.so_hpreq_handler is set twice for
the ost_io service in ost_setup(); the second assignment
overwrites the first to NULL, so ost_io threads would never
handle RPCs as high-priority ones.

While we are at it, remove some superfluous assignments of
so_hpreq_handler to NULL for statically allocated
ptlrpc_service_conf structs when initializing other ptlrpc
services, and rename some relevant functions.

Signed-off-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Change-Id: Ia728a3d7f20511fcb58b259126b05055d5860455
Xyratex-bug-id: MRP-724
Reviewed-on: http://review.whamcloud.com/4368
Tested-by: Hudson
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2173 lod: QoS code to give up if no good OSP found

on any iteration. this code was removed by mistake in
commit 03b988a (LU-2093).

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: Ifa0d3a5ceeaaf84d3ec49e39bd2f337414a216ce
Reviewed-on: http://review.whamcloud.com/4300
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Tested-by: Oleg Drokin <green@whamcloud.com>
LU-2179 osc: truncate partial page correctly

If a partial page is being truncated, the corresponding osc extent
should be held until the truncate finished.

Debug patch for osc_extent_wait() and don't wait for completion
of RPC it's not even sent in truncate.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I96a5ec1fdbb3133c735ebdfdd0330a45a2a8ab1a
Reviewed-on: http://review.whamcloud.com/4317
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2216 mdt: remove obsolete DNE code

1. remove split checking and cross-ref code from DNE.
2. remove IAM code on ldiskfs and utils.
3. remove cmm directory.

Change-Id: I0c81d753462863706e8918393369dde94a45030c
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4353
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-2224 osd-zfs: Fix osd_commit_async() locking

The ZFS osd_commit_async() function never properly acquires the
tx->tx_sync_lock() mutex to protext the tx_state_t.  However,
the mutex is correctly dropped so we just add the obviously
missing mutex_enter().

Change-Id: Iae426feaeb5885034515d6bf0ccb9509ed098bb0
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/4383
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-2213 scrub: stop LFSCK before osd_shutdown

The osd_shutdown will clean all the otable-based iteration,
but up layer LFSCK depends on the otable-based iteration.

So we need to stop the LFSCK before osd_shutdown called.

Signed-off-by: Fan Yong <yong.fan@whamcloud.com>
Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I97625d54766122314630aff0069d9e14d23b9840
Reviewed-on: http://review.whamcloud.com/4217
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
LU-1756 kernel: clean up lustre_compat25.h

1. unused functions:
   mapping_has_pages(), ll_call_writepage(), __set_page_ll_data()
   ll_invalidate_inode_pages(), __set_page_ll_data()
   CheckWriteback(), KIOBUF_GET_BLOCKS()
2. rename ll_vfs_create to vfs_create
3. remove kdev_t related macros
4. move cfs_cleanup_group_info() to lustre_common.h
5. remove kiobuf
6. move ll_inode_blksize() to lustre_common.h
7. drop LL_RENAME_DOES_D_MOVE

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: Ic5e29e399e70ccd04cbe1448f3c6cfc3a258289b
Reviewed-on: http://review.whamcloud.com/3686
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2019 llite: update i_flags in ll_iocontrol properly

When client has lsm, we still need to update cache i_flags.
Otherwise i_flags is out of sync.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I7fcb84da82129238f327885a0fc5827fcac90a8d
Reviewed-on: http://review.whamcloud.com/4078
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1526 utils: Supply default MDT index

To prepare for DNE indexing has become a requirement
for MDTs and with the latest lustre you can't mount
a MDT that was not formated with a index. While mount
has this requirement mkfs.lustre has a bug that allows
you to format a MDS without a index and not even warn
the user. At the same time mkfs.lustre has to handle
the case were a user will not supply a index since it
was not required in earlier lustre releases. This patch
address this problem by supplying a default index of
zero for the MDT if no index is supplied to mkfs.lustre
and warns the user they must supply a index in the
future.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I45932321885856d97b10630a0667e8338822b199
Reviewed-on: http://review.whamcloud.com/4293
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-1857 build: fix 'Unbounded source buffer' errors

Fix 'unbounded source buffer' defects found by Coverity version 6.0.3:
Unbounded source buffer (STRING_SIZE)
Passing string of unknown size to a function that expects
a string of a particular size.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I18e51f04e62241b5c5dad7ae963d8070d6954dd4
Reviewed-on: http://review.whamcloud.com/3904
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1889 build: fix false 'uninitialized scalar variable' errs

Fix false 'uninitialized scalar variable' errors found by Coverity
version 6.0.3:
Uninitialized scalar variable (UNINIT)
Using uninitialized value, element or field when calling function.

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I83a7dd3ae4a027bf0ebced572245bc4fff35e119
Reviewed-on: http://review.whamcloud.com/3939
Tested-by: Hudson
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-812 llite: 3.0+ kernel fsync should call write

Since 3.0, kernel pushes i_mutex and fsync to fs fsync
callback. So Lustre should check and do the same. Otherwise
there might be data corruption and sanity 63b will fail.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I2f2f6792276eaf6783bffb813f3c3e5405be0450
Reviewed-on: http://review.whamcloud.com/4091
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1337 llite: ll_inode_permission should check RCU walk

For >3.1 kernels, RCU flag is folded into mask field.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: Icc6751493e7359646cb6bd84b3ac05de167e4d88
Reviewed-on: http://review.whamcloud.com/4039
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Liu Xuezhao <xuezhao.liu@emc.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
New tag 2.3.54

Change-Id: I0c6415d7924ee83c11a5e383915d06fca41ccf2a
Signed-off-by: Oleg Drokin <green@whamcloud.com>
LU-1822 llite:  Remove deprecated truncate handler

Remove the ll_truncate handler. This handler was only being used
to display a debug message about the truncated object. That line
was moved to a different location, and the handler removed.
This handler is an issue in kernels after 2.6.34 when running the
patchless client. In that version of the kernel the kernel will log a
kernel warning if its called and the inode has a handler for truncate.
The truncate logic was updated some time ago to be more
consistent with the new sequence of events.

Xyratex-bug-id: MRP-597
Reviewed-by: Alexander Zarochentsev <Alexander_Zarochentsev@xyratex.com>
Reviewed-by: Iurii Golovach <iurii_golovach@xyratex.com>
Signed-off-by: Michael McKay <michael_mckay@xyratex.com>
Change-Id: I77b372a2825fd2bdc4b215ee20a979f03dc7d64b
Reviewed-on: http://review.whamcloud.com/3860
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Iurii Golovach <iurii.golovach@gmail.com>