new tag 2.3.55

Change-Id: I92940a37023baccdbd36fa9ca4c669b67cad8c95
LU-2129 llog: protect llog write against concurrent read

llog_write_rec contains case when buffer passed separately from
header and tail, resulting 3 writes instead of single one, so
concurrent read may read partial record.
Another similar case is llog_pad, which writes header and tail
with required gap to align records at the end of 8K buffer.

Use dt_read/write locks on llog object to protect writes against
read. Such '3-writes' case happens only with configs llogs.

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Idfb94706b2cf8ad22776c1a277b862863f94c2f6
Reviewed-on: http://review.whamcloud.com/4303
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Restore build after a tag crossed imaginary line. tag 2.3.56

We need to find a better way to control such can-kicking excercises.

Change-Id: I25d8a2fbf7d5905c8f002a7c38fc44ef3124dda5
LU-1956 test: Disable test_156 for ZFS

Since osd-zfs does not collect or report caching stats, do not run 156
against it.  Mark test as skipped.  Created LU-2261 as enhancement to
add caching stats to osd-zfs.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Id4a0b92cc775a0751fa023d6027240e8511aa312
Reviewed-on: http://review.whamcloud.com/4448
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1337 llite: take trunc_sem outside cl_io_loop for mkwrite

In truncate path it acquires lli_trunc_sem and then dlm lock, so
in page_mkwrite we should take it in the same sequence to avoid
dead lock.

This is a fix for f78805dbf041bb42b250fe754dff7b864856203f .

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: Idb5574bffdaf00461befd43b867cdd9b3e549efe
Reviewed-on: http://review.whamcloud.com/4485
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2284 quota: debug patch for test_7d

Try to collect D_TRACE & D_QUOTA log to see why the setquota failed
with -EINVAL.

This patch improve the test script a bit: fail the test when setquota
failed.

Test-Parameters: testlist=sanity-quota
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ic866dcefcb174bd0128e63f5777824608db52962
Reviewed-on: http://review.whamcloud.com/4510
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
LU-2220 doc: add CLIO notes

Add notes on the CLIO subsystem originally composed by Nikita Danilov
to the lustre/doc directory.  Update document to improve accuracy,
language, and formatting.

Original document URL:
http://wiki.lustre.org/images/3/37/CLIO-TOI-notes.pdf

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: I18ac65bc437cf095ac7a897ebd2f8882318d27ad
Reviewed-on: http://review.whamcloud.com/4377
Tested-by: Hudson
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Christopher J. Morrone <chris.morrone.llnl@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Ian Colle <Ian.Colle@intel.com>
LU-2156 misc: too noisy messages are suppressed

- number of messages leading to confusion and adding no
  useful information to the customers are suppressed.

- remove KEY_REVIMP_UPD - it's not used with OFD

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I9679b786354a2f4b52c5c13e0e2ec98a5a5b86a9
Reviewed-on: http://review.whamcloud.com/4261
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Prakash Surya <surya1@llnl.gov>
LU-2322 quota: OBD_IDX_READ should be handled during recovery

OBD_IDX_READ should be handled during recovery.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I509be43b062943e7601fdfc3a38178acb2d150b0
Reviewed-on: http://review.whamcloud.com/4580
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
LU-1337 llite: kernel 3.2 protects inode->i_nlink

Kernel 3.2 protects inode->i_nlink from direct modification.
Filesystems can only read i_nlink directly and shall use the
(set|clear|inc|drop)_nlink for modification.
See kernel commit a78ef704a8dd430225955f0709b22d4a6ba21deb.

This patch adds LC_HAVE_PROTECT_I_NLINK checking and implements
set_nlink for old kernel, clear/inc/drop_nlink exists after 2.6.18

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: Ie958cb308291ecc48d409a1282fed7ea3549a561
Reviewed-on: http://review.whamcloud.com/3577
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1623 mdt: Atomically update MDT export connection flags

MDT processing of connect requests currently updates the export
connection flags in two steps: client/server feature matching is
performed first, then much later various security-related bits are
removed as needed.  Certain error paths may leave the export flags
partially initialized.

A problem arises if multiple connect requests from the same client are
handled out of order, as may occur due to network disruptions. If the
last such request to be handled has a lower connection count than one
that already completed, it will be aborted with -EALREADY after having
modified the connection flags in the export.  However, the
security-related flags are left with incorrect values, as the
top-level connect handler skips setting these in the error path.
Replies to subsequent client requests may then contain unexpected
security information, causing the client to crash.

Similar issues may exist with other target types having non-atomic
export flag updates, and these should be addressed in follow-up
patches.

This patch makes the following changes:

- To avoid the problem described above, update the export connection
  flags atomically, and only in the successful case.

- To make this important atomic operation more conspicuous, move it
  from mdt_init_sec_level() to the top-level handler mdt_connect().

- Add a comment to mdt_connect_internal(), and delete a disabled code
  block from it.

- Correct debug message in target_handle_connect() to match code.

Signed-off-by: Ned Bass <bass6@llnl.gov>
LLNL-bug-id: bz1711
Change-Id: Ic00c4679cc9b813bdb47cf148bef8f62c0ef8ddb
Reviewed-on: http://review.whamcloud.com/4406
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1606 api: Do not include lustre_idl.h from lustreapi.h

lustre_idl.h has become increasingly difficult to use from
user-space.  Normal users of the lustre api should not
be looking into lustre wire protocol anyway, so this change
eliminates the include of lustre_idl.h.

After removing lustre_idl.h, it became obvious that a number
of programs have been picking up normal user-space headers
through a very windy path of includes.  With the include
of lustre_idl.h gone, they no longer compiled, so we also
add the missing explicit includes.

It became clear that copytool.c explicity requires
libcfs/libcfs.h, and lustre_rsync.c require lustre_idl.h.
But I believe that it is far better to have those includes
explicitly stated, so it is obvious that those programs
are peeking into lustre's internals.  In the future we
can work on creating new lustre API calls that provide the
information that they need without side-stepping abstraction
layers.

Change-Id: I8ce55ffe0e1766515061f3dbc63c2372d5713f58
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/4505
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Richard Henwood <richard.henwood@intel.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Yu Jian <yujian@whamcloud.com>
LU-2345 tests: remove $FSTYPE from tests

$FSTYPE is deprecated, it should be replaced by facet_fstype.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I6671249c88a53b01badebd7172fb3307c8f4f48b
Reviewed-on: http://review.whamcloud.com/4596
Tested-by: Hudson
Reviewed-by: Li Wei <wei.g.li@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
LU-591 mdd: link ea code cleanup

Change-Id: I7504a2a3ae540471ef37d75e46ca0d7cf9d8a9f5
Xyratex-bug-id: MRP-139
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Nathan Rutman <Nathan_Rutman@xyratex.com>
Signed-off-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Reviewed-on: http://review.whamcloud.com/1230
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2244 lov: remove unused bits from lov, osc

- precreation logic in OSC
- QoS code in LOV
- fake requests in ptlrpc

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I7a3e3ec6cf254a9fcd53ae7eab6c2d23b7520234
Reviewed-on: http://review.whamcloud.com/4399
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
LU-2031 obdclass: check if client is connected to the target

When the client is notified by IR, it may not connect to the OST
yet, verify it before writing the new NID to the import.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: Ia144b695443a9850ecccd0c8946de029492b4446
Reviewed-on: http://review.whamcloud.com/4525
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1756 kernel: cleanup lustre_compat25.h

1. unused functions:
   ll_bdi_wb_cnt()
2. call bdi_init/bdi_destroy directly
3. call add_to_page_cache_lru directly
4. fix a typo of ll_pagevec_init
5. remove QUOTA_OK/NO_QUOTA. They are defined in libcfs

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I9953dabbc758e6fb7a3d0649bdc64ba1455efb99
Reviewed-on: http://review.whamcloud.com/3689
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
LU-2139 osc: Move cl_client_lru to cl_client_cache

The cl_client_lru structure was renamed to cl_client_cache. This is
intended as a purely cosmetic change motivated by the future inclusion
of other, non LRU, information in that structure. For example, the plan
is to include "unstable" page information in this structure; which is
unrelated to the LRU list of clean pages, but still part of a client's
cache.

Signed-off-by: Prakash Surya <surya1@llnl.gov>
Change-Id: I286a1bbb2fb0d7dc52f007407bb823f960ae7c62
Reviewed-on: http://review.whamcloud.com/4483
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Revert "LU-1756 kernel: cleanup lustre_compat25.h"

This is causing build failures on SLES11SP2

This reverts commit 472fe13982952c9ccad969a996c054f9cf643847.
LU-1944 ldlm: Add another net_latency to recoverry timer.

Add another net_latency to this request deadline, i.e.
we need add 2 net_latency to extend the recovery timer,
one for balance rq_deadline (see ptl_send_rpc), one for
resend the req to server.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: I6cb6e57358f4fe32ae2ac8d982af6ce03b68014b
Reviewed-on: http://review.whamcloud.com/4351
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>