LU-4367 ptlrpc: add OBD_CONNECT_UNLINK_CLOSE flag

Add OBD_CONNECT_UNLINK_CLOSE flag for interop, once this is supported,
client packs file handle in unlink RPC, and MDT will close file before
unlink.

Signed-off-by: Lai Siyao <lai.siyao@intel.com>
Change-Id: If2f89a5268b95386e01cdc2f8613fd5c4cbd1d2c
Reviewed-on: http://review.whamcloud.com/10426
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3326 tests: sync after file deleted in run_*.sh

In run_*.sh, call sync to flush any dirty pages to OST
if MDT has not sent the object destroy request to it,
which will shrink the available disk space gotten before
running the tests and cause -EIO error.

Signed-off-by: Hongchao Zhang <hongchao.zhang@intel.com>
Change-Id: Ia246122a5c3471b7cbea9cdaa7fad2ea4db856c5
Test-Parameters: alwaysuploadlogs \
envdefinitions=SLOW=yes,ENABLE_QUOTA=yes,REQFAIL=100,DURATION=28800 \
clientdistro=el6 ossdistro=el6 mdsdistro=el6 clientarch=x86_64 \
ossarch=x86_64 mdsarch=x86_64 clientcount=4 osscount=2 mdscount=2 \
mdssizegb=10 ostsizegb=10 austeroptions=-R failover=true iscsi=1 \
testlist=recovery-mds-scale
Reviewed-on: http://review.whamcloud.com/8071
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5234 llite: restore newer kernel support for ll_read_dir

During the merger of the patch for LU-4906 the support for
newer kernels was removed in the new code. This patch
restores this functionality so RHEL7 and SuSE12 clients can
function again.

Change-Id: Ieea4fb23cd644b2d6ce998cfdba8e0e15e481fc5
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Reviewed-on: http://review.whamcloud.com/10761
Tested-by: Jenkins
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Yang Sheng <yang.sheng@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-3386 lproc: fix "imports" output format

There was no comma being printed in the import flags listing since
commit 62660c799b because the "first" marker wasn't being set false.

Fix the output format to put a space around the '[' and ']' list
delimiters.

Make the "unknown" connect flag be a single word.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Change-Id: Iee725b503ec771773d456b3436480c6e52500c1e
Reviewed-on: http://review.whamcloud.com/10344
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-1095 ofd: Silence 'slow create' warnings

These messages were originally added to analyze a performance issue.
That issue is now much better understand and no longer needs to
be logged so verbosely.  They probably should have been disabled
prior to Lustre 2.4 being released but this was overlooked.

Change-Id: I395de36cdccc82afd4ed7b9c424d12d17b881f54
Signed-off-by: Brian Behlendorf <behlendorf1@llnl.gov>
Reviewed-on: http://review.whamcloud.com/10394
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-2675 libcfs: remove unused open flag conversions

Remove the unused cfs_oflag2univ() and cfs_univ2oflag() conversion
routines.  These were intended to convert O_* open flags to and from
platform-specific flags at the user interface, but are not used
anywhere in the code.  The conversion of the client-specific open
flags to the Lustre protocol flags is done in mds_pack_open_flags()
and there is no need to ever convert back to client-specific flags.

Signed-off-by: Andreas Dilger <andreas.dilger@intel.com>
Change-Id: I1f5f7fb7eefa2f8dff161f02b37bee6a9d500c1e
Reviewed-on: http://review.whamcloud.com/10742
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5224 lnet: build failed with MLNX_OFED-2.2

Build lustre client can fail on the SLES11 SP3 with
MLNX_OFED-2.2 against the kernel-3.0.76-0.11, which will
cause the "'pm_qos_req' has incomplete type" error.This patch
put "#include <linux/compat-2.6.h>" in front of
"#include <net/sock.h>" to solve this problem.

This error come with the configure option
"--with-o2ib=/usr/src/ofa_kernel/default/", it affect the
search path of the compiler, which will search the header
files in this directory before the kernel header files
directory.

This patch change this way because "net/sock.h"(exactly is
"linux/netdevice.h") include header file "linux/pm_qos_params.h",
the pm_qos_params.h is exist both in ofa_kernel include
directory and kernel include directory, so we need to put the
linux/compat-2.6.h in front of net/sock.h to solve the
compatible problem.

Signed-off-by: Wu Libin <lwu@ddn.com>
Change-Id: I98df2a5faaecbd44528d3bd8a7ba796cc1d14cdc
Reviewed-on: http://review.whamcloud.com/10749
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Dmitry Eremin <dmitry.eremin@intel.com>
Tested-by: Jenkins
Reviewed-by: Minh Diep <minh.diep@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Revert "LU-4398 mdt: acquire an open lock for write or execute"

This is causing significant READ performance degradation in mdts: LU-5197

This reverts commit 708d85a652a77f85153790e6cca1b7a2b91947cf.

Change-Id: Ia591bed8a6f098ebad5bb8460c659516bcf24d97
Reviewed-on: http://review.whamcloud.com/10790
Tested-by: Jenkins
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5203 zfs: Update spl/zfs version to 0.6.3

Update SPL and ZFS version to official 0.6.3.
This also fixes the "search path" for libzfs.so.

Test-Parameters: mdsfilesystemtype=zfs  mdtfilesystemtype=zfs ostfilesystemtype=zfs testgroup=review-zfs
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I2530a7a1f7850e31aa99892f5a3e826baf69d20b
Reviewed-on: http://review.whamcloud.com/8979
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5062 llite: Solve a race to access lli_has_smd in read case

In vvp_io_read_lock(), it used to decide if to add read lock by
checking lli_has_smd. Accessing lli_has_smd is racy when an empty
file is turned into raid0, therefore, it may result in read requests
are issued without corresponding lock.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I70cb1888b42e0e9928f8346359e2b1f554fe4e8f
Reviewed-on: http://review.whamcloud.com/10760
Tested-by: Jenkins
Reviewed-by: Patrick Farrell <paf@cray.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5223 lod: generate linkEA for shard of striped directory

Otherwise, it will misguide namespace LFSCK to regard it as the case
of linkEA corrupted or upgraded from 1.8 device, then cause improper
reparing or status reporting.

On the other hand, the linkEA for slave object of striped directory
will also be used for fid2path in subsequent patches.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: I877b091eb330df0283de2ea21f2796387023da30
Reviewed-on: http://review.whamcloud.com/10765
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
LU-5045 build: update OFED download site to https

OFED download site has changed to use https.

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: Icb26444db5a3c239f595efb01cb03b1c1dd3e6de
Reviewed-on: http://review.whamcloud.com/10755
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Jenkins
Reviewed-by: Brian J. Murrell <brian.murrell@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5218 llog: keep llog ctxt indices constant

The llog context id table cannot be shrinked easily because that
will cause index shifting and incompatibility between old client
and new server and vice versa.

Patch moves llog_ctxt_id table to the lustre_idl.h because this is
wire protocol data, these values are added to the wirecheck and
checks are added in llog server code for wire data.

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I003120223504d95164e4392b01c096e3e8b0fef0
Reviewed-on: http://review.whamcloud.com/10758
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Tested-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5249 osd: check if FLD is fully intialized

Check if FLD is fully initialized in OSD, before lookup
sequence in the local FLDB.

Change-Id: I07cbf293e617935588ce638f710868a05bdd0943
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10806
Tested-by: Jenkins
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5123 procfs: u64 helper does not respect multiplier

lprocfs_write_frac_u64_helper should apply the multiplier argument to
the value if units are not specified by the user.  It has been
overriding the multiplier with '1' regardless.

Signed-off-by: Stephen Champion <schamp@sgi.com>
Change-Id: Iaacbb6923122479ba1ba4a5ee0088c670245d081
Reviewed-on: http://review.whamcloud.com/10535
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: James Simmons <uja.ornl@gmail.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-5233 lod: Remove ldo_def_striping_set check

Only check ldo_stripenr/ldo_stripe_offset before create
striped directory, and ldo_def_striping_set is used to
mark whether the directory has default stripe, which can
be inherited to its children.

Reset ldo_stripenr to 0, if initialization fails, otherwise
it will leave some invalid object in cache.

Change-Id: Iba6985861fb1443bd4e7fcce3cd8c523b31a5940
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10772
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Jenkins
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5177 mdt: fix object leak and use after free

The mdt_intent_layout() and mdt_open_by_fid_lock() may exit without
object put and causing object leakage.

The mdt_md_create() passed possibly freed object to the
mdt_create_pack_capa()

Signed-off-by: Mikhail Pershin <mike.pershin@intel.com>
Change-Id: I634052c58ee8595871af987755fda5a9f2c942e1
Reviewed-on: http://review.whamcloud.com/10750
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: wangdi <di.wang@intel.com>
Reviewed-by: John L. Hammond <john.hammond@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
LU-5188 osp: return 1 if osp_sync_xxx_job issue RPC

Return 1 if osp_sync_new_xxx_job() issue RPC, so
sp_sync_process_record() can decrease opd_syn_rpc_in_flight
and opd_syn_rpc_in_progress correctly if RPC is not
being sent, otherwise the opd_sync_thread will not be
stopped, caused LBUG (see LU-5244)

osp_sync_thread()) ASSERTION( count < 10 ) failed:

The problem is introduced by

http://review.whamcloud.com/10706

Change-Id: Iaf6dd6c4e1938a0c5d44a40c303eda1695b72105
Signed-off-by: Wang Di <di.wang@intel.com>
Reviewed-on: http://review.whamcloud.com/10828
Tested-by: Jenkins
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-5240 obdclass: Increase minimum LU Cache for ZFS

If the lu cache is too small threads will thrash for data including
the OI tables and other important metadata.

Test-Parameters: mdsfilesystemtype=zfs mdtfilesystemtype=zfs ostfilesystemtype=zfs testgroup=review-zfs
Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I1ce7ef9a160c8b028be0d3c9671a0b347cbe3331
Reviewed-on: http://review.whamcloud.com/10802
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>
Tested-by: Jenkins
LU-5227 build: build dkms using lbuild

To include dkms build in lbuild
1. before build lustre, generate lustre-dkms.spec with configure
2. use rpmbuild to build the src
3. rebuild dkms using the src

Signed-off-by: Minh Diep <minh.diep@intel.com>
Change-Id: Iac71d20110fe0f84b3bfffb071f3e3b84c7f5f17
Reviewed-on: http://review.whamcloud.com/10747
Tested-by: Jenkins
Reviewed-by: Faccini Bruno <bruno.faccini@intel.com>
Reviewed-by: Bob Glossman <bob.glossman@intel.com>
Tested-by: Maloo <hpdd-maloo@intel.com>
Reviewed-by: Oleg Drokin <oleg.drokin@intel.com>