LU-793 ptlrpc: allow client to reconnect with RPC in progress

Since not letting clients with pending RPCs to reconnect caused a lot
of problems, this patch allows such a reconnection and would only
refuse to process requests with the same xids that are already being
processed.

Bulk requests are aborted upon reconnection by comparing connection
count of request and export.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: Ic4b23a71fd288df02d1040d98867373ae06b60f6
Reviewed-on: http://review.whamcloud.com/4960
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4194 ldlm: set l_lvb_type coherent when layout is returned

In case layout has been packed into server reply when not
requested, lock l_lvb_type must be set accordingly.

Signed-off-by: Bruno Faccini <bruno.faccini@intel.com>
Change-Id: Iaf54c9ba27785e529f4f2fb967d2fad4fc1dfbcb
Reviewed-on: http://review.whamcloud.com/8270
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
LU-4194 ldlm: Make OBD_[ALLOC|FREE]_LARGE use consistent

struct ldlm_lock's l_lvb_data field is freed in ldlm_lock_put()
using OBD_FREE.  However, some other code paths can attach
a buffer to l_lvb_data that was allocated using OBD_ALLOC_LARGE.
This can lead to a kfree() of a vmalloc()ed buffer, which can
trigger a kernel Oops.

Change-Id: Ic75a67530862eeb4d065c14bbbac80939bff5731
Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Reviewed-on: http://review.whamcloud.com/8298
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
LU-4208 osd-zfs: hold pool config lock to register property

- Hold the DSL pool configuration lock when calling
  dsl_prop_register().  Failure to do so will panic the node if
  assertions are enabled in ZFS.  This change requires a build of ZFS
  on Linux that exports symbols dsl_pool_config_enter and
  dsl_pool_config_exit, which was done in commit 40a806d [1], and will
  appear in ZFS release 0.6.3.

- Fix up variable declaration alignment in osd_mount().

- Add check for exported symbols in autoconf

[1] https://github.com/zfsonlinux/zfs/commit/40a806d

Signed-off-by: Ned Bass <bass6@llnl.gov>
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: Ib2df05301ca73847d49ca2b183d72b9e2d957c10
Reviewed-on: http://review.whamcloud.com/8172
Tested-by: Jenkins
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Li Wei <wei.g.li@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4245 ldlm: do not fill empty lvbo buffer

Don't fill empty lvbo buffer if server does not prepare it.

Signed-off-by: Bobi Jam <bobijam.xu@intel.com>
Change-Id: I22a2ddd016dcc43c00b2b3d6be5d18921bf26d67
Reviewed-on: http://review.whamcloud.com/8285
Tested-by: Jenkins
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4316 mdt: Do not grant Layout lock for remote file

Do not grant layout lock for remote file, because only
the MDT, where the file/dir resides, should grant the
LAYOUT lock.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: I174d54f3ce5b7c69cb35aa63f4322b743ea677e8
Reviewed-on: http://review.whamcloud.com/8406
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
LU-3319 procfs: provide framework for seq_file handling

In linux 3.10 kernels and above the proc file system interface
changed. To handle this change we need to move from the current
method of using proc_read and proc_write function pointers to
using the seq_file. This patch provides the base to enable moving
all the lustre proc handling gradually over to this new method
of proc data handling.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I06d941398bf248bffab851f9f25bea6bbe438a30
Reviewed-on: http://review.whamcloud.com/7135
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2880 ldiskfs: rename patches for consistency

Rename the ext4_data_in_dirent.patch, ext4_pdirop.patch, and
ext4_extra_isize.patch files to be ext4-data-in-dirent.patch,
ext4-extra-isize.patch and ext4-pdirop.patch respectively.
This is more consistent with all of the other patches in the
series, and avoids missing these files with tab completion.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I92125f09284e23ab08d5e9f8658e68416bde025f
Reviewed-on: http://review.whamcloud.com/6411
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4049 build: fix 'NULL pointer dereference' errors

Fix 'NULL pointer dereference' defects found by Coverity version
6.6.1:
Dereference after null check (FORWARD_NULL)
For instance, Passing null pointer to a function which dereferences
it.
Dereference before null check (REVERSE_INULL)
Null-checking variable suggests that it may be null, but it has
already been dereferenced on all paths leading to the check.
Dereference null return value (NULL_RETURNS)

Signed-off-by: Sebastien Buisson <sebastien.buisson@bull.net>
Change-Id: I1db4e80fd2c048cc20459e8da4cb61a97b3171cb
Reviewed-on: http://review.whamcloud.com/7827
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4015 ldiskfs: access external journal using device path

Currently ldiskfs when configured with an external journal
uses the device number to access it. The problem with this
approach is that the device number can change across
reboots. This patch adds a new mount option -o journal_path
to tell ldiskfs where the external journal is instead.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I43fff89c2bebeff4f0305cdc24cefa534fb9fe67
Reviewed-on: http://review.whamcloud.com/7777
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
LU-2800 autoconf: clean up sysctl table handling

Before linux kernel version 2.6.24 came out lustre
like other subsystems registering with sysctl had
to assign unquie numbers for ctl_name in struct
ctl_table. After 2.6.24 this requirement changed
and lustre started to use CTL_UNNUMBERED as the
default value. Since we no longer support 2.6.24
and earlier kernels we can remove this test and
simply the code. We also simplifed the handling
of the strategy field in struct ctl_table. The
final fix is that several LND drivers never handled
that case of 2.6.33 and above kernels which removed
the ctl_name and strategy fields altogether. This
patch addresses this so LND drivers can be handled
better on distros using newer kernels.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: I0f5b64552e7d54abf4b73bfe285109573725edaf
Reviewed-on: http://review.whamcloud.com/8102
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-3867 utils: llapi_create_volatile_idx uses 0 instead idx

If llapi_create_volatile_idx() is passed an MDT index other than
-1, it always uses 0 as the MDT index rather than the one provided.

This patch removes the use of LUSTRE_VOLATILE_IDX and uses
LUSTRE_VOLATILE_HDR with the MDT index instead, uses the provided
MDT index in the volatile file name and uses base 16 when reading
the MDT index from the file name using strtoul.

Signed-off-by: James Nunez <james.a.nunez@intel.com>
Signed-off-by: Bobbie Lind <bobbie.j.lind@intel.com>
Change-Id: I27e2beb8524683aba07c48297614dd02d7957af4
Reviewed-on: http://review.whamcloud.com/7613
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-2800 autoconf: block device release changed in newer kernels.

For kernels at 3.10 and above the release method for block
devices no longer return a integer. This impacts the lloop
block device we use on the lustre client side.

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Ib63e2897a48989f1cf5185a1c88a5ddbf989c0a9
Reviewed-on: http://review.whamcloud.com/5380
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4069 build: cleanup from GOTO(label, -ERRNO)

Cleanup the code from GOTO(label, -ERRNO) and other bad GOTOs.

Signed-off-by: Dmitry Eremin <dmitry.eremin@intel.com>
Change-Id: I58df89810c112a6856da83c56c9f29dc22ffdf99
Reviewed-on: http://review.whamcloud.com/8080
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Jenkins
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-4083 mdt: Take lov_mutex in mdt_reint_unlink and _rename

Take the mot_lov_mutex lock around mdo_unlink and mdo_rename
to synchronize around striping.

Test-Parameters: testlist=racer
Signed-off-by: James Nunez <james.a.nunez@intel.com>
Change-Id: Iaa00b81fd7cfb25ce5f3dcea3c2d6289d133134f
Reviewed-on: http://review.whamcloud.com/7919
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3726 tests: make sanity-hsm compatible with MDSCOUNT >= 2.

Add generic functions to perform set_param and check_param
on all MDTs, and persistent set_param on the mgs.
CL_USER variable replaced by CL_USERS array (1 value per MDT).
MDT0 variable replaced by MDT array (1 value per MDT).

Test-Parameters: mdtcount=2 mdscount=2 testlist=sanity-hsm
Signed-off-by: Thomas Leibovici <thomas.leibovici@cea.fr>
Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Change-Id: I0d8610a74427423dda25a6defd85f5c52c535850
Reviewed-on: http://review.whamcloud.com/7437
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: jacques-Charles Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-36 mdd: cache lu_attr to avoid duplicate fetch

- we called mdd_la_get for so many times in MDD, it's not good for
  performance of high contention pdir operations.
- mdd_attr_check is not strictly correct under pdir operations,
  there is still chance to decrease ctime because we don't have
  exclusive lock on parent while checking ctime
- mdd_attr_check_set_internal and mdd_attr_check_set_internal_locked
  can be merged into one function: mdd_update_time, because set
  cmtime is the only use-case of them.

Change-Id: I59500835473ae6d42aa9bd2dcf9be644b7b7ec16
Signed-off-by: Liang Zhen <liang.zhen@intel.com>
Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Reviewed-on: http://review.whamcloud.com/6897
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3489 osd-zfs: varsize index support

LFSCK needs indices with keys/values of different size.
add a support for this to osd-zfs. key/value size can't
vary within a single index object so far.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I6faa730ce1f778563c1fcf43d3b50c5b62c899d9
Reviewed-on: http://review.whamcloud.com/7152
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3230 osp: unstick precreates on unmount

Allow osp_precreate_reserve to propagate error, Add debug code to
illuminate what the error code is if not EIO.

Test-Parameters: mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs
Test-Parameters: mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs mdscount=3
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I1c15375d5e50d5a47a72b6c3c8e27512f51c3f9c
Reviewed-on: http://review.whamcloud.com/7995
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
LU-4277 lod: handle os_state as a flag, check READONLY

Handle the osfs->os_state values as flags instead of as values.
Otherwise, if an OSD has OS_STATE_DEGRADED and another flag set
it will not match to skip the OST during object allocation.

Similarly, there was no check for the OS_STATE_READONLY flag in
the create path, which should be used to always skip an OST for
object creation until it is cleared.

These flags are not yet set by osd-zfs, only by osd-ldiskfs so far.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I0ef59a9fd631a6942d963e37cfce2bfe183ebbe5
Reviewed-on: http://review.whamcloud.com/8378
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>