LU-2378 lma: move HSM & SOM attributes to dedicated xattrs

Reduce LMA to its minimum and move HSM & SOM attributes to dedicated
on-disk extented attributes. This is more consistent with the MDS
layering (HSM, LMA & SOM are managed in different layers) and avoids
locking + read/modify/write in many cases.

As far as interoperability is concerned, the old code works just fine
if it reads a smaller LMA size and the new code retries with a
larger buffer on ERANGE error.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: Id4ee465ea69820229f7214d800821864330a8aec
Reviewed-on: http://review.whamcloud.com/4746
Reviewed-by: Aurelien Degremont <aurelien.degremont@cea.fr>
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-2194 tests: Wait for reconnect in recovery-small/19

Fully wait for clients to reconnect before checking if evicted.

Signed-off-by: Nathaniel Clark <nathaniel.l.clark@intel.com>
Change-Id: I332d9b43344eb2c7c883fc28adfbbfb4eeb8a0d0
Reviewed-on: http://review.whamcloud.com/4652
Tested-by: Hudson
Reviewed-by: Vitaly Fertman <vitaly_fertman@xyratex.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jian Yu <jian.yu@intel.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-744 obdclass: revise cl_page refcount

By holding a refcount for radix tree, we can remove a couple of
spin lock and atomic operation for cl_page.

Signed-off-by: Jinshan Xiong <jinshan.xiong@intel.com>
Change-Id: I9440d6b86a63d00ce716cb11676101acca07d535
Reviewed-on: http://review.whamcloud.com/4617
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2016 mdd: add layout swap between 2 objects

This patch add a new method in mdd to swap the layouts between
2 lustre objects.
The 2 objects have to be of the same type.

Change-Id: I26dfef2745eac67168aeceac196453c5126148fd
Signed-off-by: JC Lafoucriere <jacques-charles.lafoucriere@cea.fr>
Reviewed-on: http://review.whamcloud.com/4189
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-398 libcfs: Add libcfs heap, a binary heap implementation

The heap can be used to build and maintain sorted arrays and lists,
and prioritized queues of large numbers of elements, with minimal
insertion and removal time. The first user for the data structure are
NRS policies, which use it to maintain prioritized queues of RPCs at
PTLRPC services.

There is no 'search' operation, but the data type aims to be useful
in cases where performing searches on the data set is not required,
and instead the lowest priority element is usually removed from the
data set for consumption.

Original-author: Eric Barton <eeb@whamcloud.com>
Original-author: Liang Zhen <liang@whamcloud.com>
Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Nikitas Angelinas <nikitas_angelinas@xyratex.com>
Change-Id: Ifd293d0a8201e45bd6030d7a16ce0be691f79cb9
Oracle-bug-id: b=13634
Xyratex-bug-id: MRP-73
Reviewed-on: http://review.whamcloud.com/4412
Tested-by: Hudson
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1924 build: configure can not find libgssapi_krb5.so

./configure --enable-gss cannot find on Debian amd64/x86_64
wheezy the file libgssapi_krb5.so although it
exists in the directory /usr/lib/x86_x64-linux-gnu.
Probably some other amd64/x86_64 Linux distributions
are effected as well.

Signed-off-by: Thomas Stibor <thomas@stibor.net>
Change-Id: Ife12e11224db4ef64adbcedb31cc1c07cf1c9b2e
Reviewed-on: http://review.whamcloud.com/4378
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1887 ptlrpc: grant shrink rpc format is special

Grant shrink rpc is packed in OST_SET_INFO request, but its format is
different from others.

Change-Id: I8b3398d534fbae683722f942403e031c68023b16
Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/4277
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1484 kernel: fix build error with 2.6.18 kernel

When crossing building 2.6.18 kernel, I always got following
errors:
lustre-release/libcfs/include/libcfs/params_tree.h:107:2:
	   error: #error proc_dir_entry->deleted is conflicted
	   with proc_dir_entry->pde_users

It is because after commit 76bf16d1, HAVE_PROCFS_USERS and
HAVE_PROCFS_DELETED can both be true on old kernels with
backported code.

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I2b420f401f221c619b9643acee2ff54057af0fc9
Reviewed-on: http://review.whamcloud.com/4589
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Bobi Jam <bobijam@gmail.com>
LU-1756 kernel: cleanup lustre_compat25.h

1. unused functions:
   KIOBUF_GET_BLOCKS()
2. container_of() always defined
3. place linux/mpage.h at where it is needed

Signed-off-by: Peng Tao <tao.peng@emc.com>
Change-Id: I3369572fbd8aef2a5ad4881ec3f0f67c11772881
Reviewed-on: http://review.whamcloud.com/3687
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Liu Xuezhao <xuezhao.liu@emc.com>
LU-2363 lod: Fix statfs entries in lod procfs

Factor toplevel lod procfs out of lod_pools_{init,fini}.  Support
lod statfs entries using osd methods.

Change-Id: I27f8c1441079ad455a03f16abb05491fd655d2e3
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Reviewed-on: http://review.whamcloud.com/4637
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <kemannthey@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-1632 fid: remove fid_delete in delete_inode procedure

Remove fid_delete in delete_inode procedure, which is
completely obsolete in 2.x infrastructure , especially
when lmv layer is not being created during upgrade,
because it will cause the client fid space to be zero
out whenever an inode is being deleted.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: I4002945603790f8dda3e33847160c13afb7b41a4
Reviewed-on: http://review.whamcloud.com/3434
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1287 mountconf: write failover nid config correctly

Once OST starts before MDT, MDT needs steal OSC config from client
config, but both target nids and failover nids are stored as the
same type of config LCFG_ADD_UUID. To handle this correctly, all
LCFG_ADD_UUID before LCFG_SETUP is treated as target nids, while
after as failover nids. NB this implies that records should be
written into config file in fixed order.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I42d014911edad1bc53bb1a66def160ee0da3e670
Reviewed-on: http://review.whamcloud.com/3107
Reviewed-by: Sebastien Buisson <sebastien.buisson@bull.net>
Reviewed-by: Mike Pershin <mike.pershin@intel.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2358 procfs: Implement /proc/fs/lustre/mgs/MGS/fstype as symlink

Add symlinks:
  /proc/fs/lustre/mgs/MGS/osd    -> ../../osd-ldiskfs/lustre-MDT0000
  /proc/fs/lustre/mgs/MGS/fstype -> osd/fstype
  /proc/fs/lustre/mgs/MGS/mntdev -> osd/mntdev
Remove unused lprocfs_rd_fstype().

Change-Id: Ic5cce7cf31962c7d6ea97c5d3117fba81539cb68
Signed-off-by: John L. Hammond <jhammond@tacc.utexas.edu>
Reviewed-on: http://review.whamcloud.com/4635
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Emoly Liu <emoly.liu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
LU-2346 quota: set default grace time

Set default grace time as MAX_IQ_TIME or MAX_DQ_TIME.

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I8f57e5483cdf0948532d3185fdf43c6efec73233
Reviewed-on: http://review.whamcloud.com/4598
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
LU-2361 quota: keep slave's glb idx consistent with master's

Issue glimpse callback and bump global version each time quota
settings are modified in the global index. This way, the global
recovered if required.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: I91c11b8bce68c8f96289588e612295ece7664c3f
Reviewed-on: http://review.whamcloud.com/4632
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Hudson
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2388 statahead: don't statahead if it's stopped

Statahead may be stopped but lli->lli_sai is not NULL(inflight
statahead RPCs), so check lli->lli_opendir_key before accessing
statahead data.

Signed-off-by: Lai Siyao <laisiyao@whamcloud.com>
Change-Id: I4bcd4da0dc2913697473bf85ff6e502df43ce5a2
Reviewed-on: http://review.whamcloud.com/4718
Tested-by: Hudson
Reviewed-by: Fan Yong <fan.yong@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Peng Tao <bergwolf@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-2371 ptlrpc: get new xid for resend on EINPROGRESS

Allocate new xid for ptlrpc resend on EINPROGRESS to avoid MDT's reply
recontruction code which will cause EINPROGRESS to always been replied
on subsequent resent.
Also fix non-intent creation path to set rq_no_retry_einprogress.

Signed-off-by: Johann Lombardi <johann.lombardi@intel.com>
Change-Id: Id423142e4440952777b74a834e2b4fa38c338280
Reviewed-on: http://review.whamcloud.com/4719
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Fan Yong <fan.yong@intel.com>
LU-2302 scripts: fix lfs_migrate with non-English locale

Parsing of stat output assumes an English locale. Make sure
this is always the case.

Signed-off-by: Daniel Kobras <d.kobras@science-computing.de>
Change-Id: I66acd6eb49cac749db701b97a5a375dc3b141a01
Reviewed-on: http://review.whamcloud.com/4697
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
LU-2302 scripts: null-terminated file lists in lfs_migrate

New option -0 allows to use a null-terminated list of file names
on stdin. Changed internal lfs find to use null-terminates lists.

Signed-off-by: Daniel Kobras <d.kobras@science-computing.de>
Change-Id: Iccd928a766968a7513833035684d41a1060c4a4d
Reviewed-on: http://review.whamcloud.com/4700
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>
LU-2302 scripts: prevent lfs_migrate data disclosure

Tighten umask to make sure temporary rsync files are created
with strict permissions that don't grant access to unprivileged
users.

Signed-off-by: Daniel Kobras <d.kobras@science-computing.de>
Change-Id: Ie662edfd615a24ce08cbddb9347f197de41d4d27
Reviewed-on: http://review.whamcloud.com/4699
Tested-by: Hudson
Reviewed-by: Andreas Dilger <andreas.dilger@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Keith Mannthey <keith.mannthey@intel.com>