LU-1187 tests: chmod +x for dir_remote.sh

chmod +x for dir_remote.sh

Signed-off-by: wang di <di.wang@whamcloud.com>
Change-Id: I77d670594c128d4f18903bd247a49df7f8ead970
Reviewed-on: http://review.whamcloud.com/4997
Tested-by: Hudson
Reviewed-by: Jian Yu <jian.yu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2441 mdd: Implement volatile file

Add a way to create files which are created unlinked, so
they are automatically removed at last close

User interface is:
- llapi_create_volatile_idx()
- llapi_create_volatile()

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: Ia33a6832d31b83191e4d364da4350cedbe6eef39
Reviewed-on: http://review.whamcloud.com/4836
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
LU-398 ptlrpc: Add the NRS framework and FIFO policy

The Network Request Scheduler (NRS) allows a user to control the way
in which RPCs are dispatched from PTLRPC services. This can be used to
various effects, most importantly improved fairness across the
cluster, better I/O performance under certain workloads, and allowing
for QoS semantics to be used at the filesystem level. Central to the
operation of NRS are NRS policies, each of which implements handling
of RPCs in a different way, aiming to achieve a particular goal.

This patch adds the core NRS framework and the NRS FIFO policy, which
is a logical wrapper around previous, non-NRS functionality, and is
used as the default policy for handling all types of RPCs.

Signed-off-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Co-authored-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I7fcd7885cc89b653d9bc482da533aae2f129bdf9
Oracle-bug-id: b=13634
Xyratex-bug-id: MRP-73
Reviewed-on: http://review.whamcloud.com/4411
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Liang Zhen <liang.zhen@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2183 osp: cleanup osp-on-ost

With new quota design and FIDonOST introduced, a new connection
from OST/MDTn to MDT0 is required for sending quota and sequence
RPCs. This connection was implemented by adding a new special
osp device named osp-on-ost, which introduced complexity to osp
code, and many hacks of checking the osp device name in both
client and server side code.

In this patch, we removed the osp-on-ost, and introduced a new
device type lwp (Light Weight Proxy) to manage the light weight
connection established from OSTs/MDTs to MDT0.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: Ic1ba25f941a4f971a542dc4be8a81e28ee0f477b
Reviewed-on: http://review.whamcloud.com/4923
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
New tag 2.3.60

Also another can kicking for LU-2449, hopefully last this time.

Change-Id: I1455e48f1b557bc8787ef234e1894b0f9d7f4843
Signed-off-by: Oleg Drokin <green@whamcloud.com>
LU-2062 hsm: llapi hsm calls for CopyTool

llapi hsm calls for CopyTool
- llapi_hsm_copytool_start()
- llapi_hsm_copytool_fini()
- llapi_hsm_copytool_recv()
- llapi_hsm_copytool_free()
- llapi_hsm_copy_start()
- llapi_hsm_copy_end()
- llapi_hsm_progress()
- llapi_hsm_import()

and related MDT functions
- mdt_hsm_ct_register()
- mdt_hsm_ct_unregister()
- mdt_hsm_progress()

The new RPC are:
- MDS_HSM_CT_REGISTER
- MDS_HSM_CT_UNREGISTER
- MDS_HSM_PROGRESS

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: I93617ac6e66a3324344b57f1a22d7c851d715262
Reviewed-on: http://review.whamcloud.com/4736
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1187 lod: Add remote object for DNE

LOD will do FLD lookup to check whether the object
is for OST or remote MDT object.

Because there are remote object invlove, it might
return some error other than ENOMEM for object allocation.

Signed-off-by: wang di <di.wang@intel.com>
Change-Id: Ie226b3c475524fc7e705665c6200e4c66d50acf0
Reviewed-on: http://review.whamcloud.com/4924
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1187 lod: move seq client init from MDT to LOD

Move seq client init from MDT to LOD, so to unify
config log for adding OSP and MD OSP.

Change-Id: I93f91c0dd6f3568196a99eecf422e0b409abda31
Signed-off-by: Wang Di <di.wang@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4337
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-2638 ldiskfs: update dotdot entry for rename

On master, when rename a directory, its old dotdot entry will
be removed firstly, then insert the new dotdot entry, and try
to append FID-in-dirent. But the space for dotdot entry maybe
not enough to hold the new dotdot with FID-in-dirent, such as
the MDT device restored from file-level backup, or the device
is upgraded from 1.8. If we do not move data in the directory
block, the FID-in-dirent will overwrite the dx_root and cause
the directory crashed. Currently, we do not want to introduce
complex logic to handle directory data moving, instead, under
such case, ignore the FID-in-dirent for the new dotdot entry,
and just insert the new dotdot entry.

Signed-off-by: Fan Yong <fan.yong@intel.com>
Change-Id: Id38f5136fb2bc607344c37ca10a32e63ab427e3d
Reviewed-on: http://review.whamcloud.com/5179
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
LU-1338 hsm: HSM flags feature

This extends lustre_hsm_attrs struct in MDD to store HSM per-file
information. It adds infrastructure (function, RPCs, ...) to read and
modify it, from MDT code or from user-space on clients through llapi
and lfs command.

This patch implements:
- lfs hsm_state
- lfs hsm_set
- lfs hsm_clear

matching llapi functions:
- llapi_hsm_state_get()
- llapi_hsm_state_set()

matching mdt functions:
- mdt_hsm_state_get()
- mdt_hsm_state_set()

and MDS RPC:
- MDS_HSM_STATE_GET
- MDS_HSM_STATE_SET

Signed-off-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: Ic8f82ddc9a56206307c2e5be2523fb7ce42b8638
Reviewed-on: http://review.whamcloud.com/3035
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-2061 hsm: get list of HSM actions

This patch implements a MDS RPC to get
the list of registered and running hsm actions on files

It implements:
- lfs hsm_action

The matching llapi call:
- llapi_hsm_current_action()

The matching MDS function:
- mdt_hsm_action()

and MDS RPC:
- MDS_HSM_ACTION

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: If4b00949ca8c94c8572cc81a7f7b8db32914542c
Reviewed-on: http://review.whamcloud.com/5029
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-2061 hsm: send HSM resquest to CDT

This patch implements the user interface to ask for hsm requests

It implements:
- lfs hsm_archive
- lfs hsm_restore
- lfs hsm_cancel
- lfs hsm_remove
- lfs hsm_release

The matching llapi call:
- llapi_hsm_user_request_alloc()
- llapi_hsm_request()

related MDS functions:
- mdt_hsm_request()

and the MDS RPC:
- MDS_HSM_REQUEST

Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Change-Id: I635f6e0518ec12c359dbb528c3a61216650d952d
Reviewed-on: http://review.whamcloud.com/5030
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>