LU-1302 llog: llog over OSD primitives

set of low-level functions implementing llog functionality over
OSD device

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I6b623a821d39ecca90c27a7a84b969aada11c0cd
Reviewed-on: http://review.whamcloud.com/4093
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
LU-1302 llog: llog ioctl to use env

Update llog_ioctl.c to use lu_env

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I8f5476cccacae2d4922ac9af87d334929c0eba6c
Reviewed-on: http://review.whamcloud.com/4094
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
LU-1302 mgs: mgs uses llog over OSD

- llog over OSD in MGS. Switch llog_test to work over OSD too.
- local_oid_store_init() don't create names for llog because all
  llogs have names in O/d/n
- Enable all llog tests functionality

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I60cc5ec7094a2640d7fc0a0d87519c56b1383557
Reviewed-on: http://review.whamcloud.com/4104
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1842 quota: lquota entry

quota entry management.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: I57e6a736680b25c737859be2cd9434997b2ccdbf
Reviewed-on: http://review.whamcloud.com/4101
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1842 quota: new sanity-quota tests

Added new sanity-quota.sh for the new quota architecture, meanwhile,
the old sanity-quota.sh is reserved for the interoprability with
old server.

Removed the quotacheck tests in conf-sanity, since quotacheck is
deprecated in the new quota architecture.

Test-Parameters: envdefinitions=SLOW=yes testlist=sanity-quota

Signed-off-by: Niu Yawei <niu@whamcloud.com>
Change-Id: Ia3e3248a167e9f312a782aefea7af0aeced52220
Reviewed-on: http://review.whamcloud.com/4040
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Johann Lombardi <johann.lombardi@intel.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1842 ldlm: add support for quota LVB

This patch defines new LVB types for quota.
A new request buffer is also added to glimpse AST to allow the server
to pass additional information about the callback to the client.
This additional buffer will only be used in conjunction with quota
locks for the time being.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: If5842bfff79d24e7d11ef33c47edb96fe6f11e76
Reviewed-on: http://review.whamcloud.com/4107
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1867 tests: disable replay-single/89

as it's been coloring tests red quite often and seem to be very
localized issued I suggest to disable it till the root cause or
the test is fixed properly.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I85feb214d6d23077cfdde47a6a439b806c5d0b6e
Reviewed-on: http://review.whamcloud.com/4105
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1303 lod: add lprocfs support to lod

and initialize it at device startup time

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Id8f5e40f18685f3ffbf50405932754f594d7e60f
Reviewed-on: http://review.whamcloud.com/4046
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-1303 lod: methods to manage pools

external components (like configuration) call these
methods to initialize and manage in-core list of
pools in lod. so that lod itself doesn't need to
implement store/load functionality.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I6b8bbc55b9acf116cc8c06e756ad26b327b5d4e8
Reviewed-on: http://review.whamcloud.com/4047
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1303 osp: OSP logging functionality

OSP writes unlink and setattr llogs and tracks corresponding commits
at OST side, then cancel llog records locally

Add missed obd_device methods and import event handling

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I63ecc975e483a7f9caac9554cc15c3a4fb4a6e6d
Reviewed-on: http://review.whamcloud.com/4095
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1303 osp: add OSP lprocfs functions

OSP lprocfs functions

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: Id85447311847c7e75ed6b0505734f5e8eb909ed5
Reviewed-on: http://review.whamcloud.com/4100
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1303 tests: Support for MDS-initiated OST_DESTROYs

This patch makes sure the tests work with MDSs that destroy OST
objects asynchronously on behalf of clients.

Change-Id: I3d365766f1af8a305cf723546e868ff3d6b2501d
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/2982
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
LU-1303 lod: functions to manipulate sub-devices

lod needs to be connected directly to all OST. configuration
subsystem brings OSP for every OST in the cluster, then
LOD connects to them and use.

the patch fixes cfs_foreach_bit() so that it continues scan
from the next bit.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I6a8b98ed8964fba1410552c94c408b004eae00a4
Reviewed-on: http://review.whamcloud.com/4048
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1303 lod: object methods

in this patch - just passing control down to the stack.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I0f4f0cbc2e08cf5d3db9771d88593beb53df927c
Reviewed-on: http://review.whamcloud.com/4049
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1303 lod: destroy and setattr to use striping

destroy and setattr methods fetch striping, instantiate
involved objects and then call corresponding methods.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I33eea2717d10eb5c34b5a324ef35a83f1430242b
Reviewed-on: http://review.whamcloud.com/4050
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-1303 lod: transfer default striping from parent/fs

lod_ah_init() transfers default striping from the provided
parent if present or from filesystem defaults.
loaded/parsed default striping is cached in the parent.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I575f2c2bddbf464c1eb93953b845ed563cfd081d
Reviewed-on: http://review.whamcloud.com/4054
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1303 lod: introcude QoS structures

like LOV before LOD will be maintaining OST->OSS mapping to be
able to distribute load among OSSs.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I132382cb09b50aec1edcd667cde1851c9ee043dc
Reviewed-on: http://review.whamcloud.com/4055
Tested-by: Hudson
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-1303 lod: directories to inherit default striping

on creation. the patch adds sanity check on default striping
being set by an user.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I4142a2189b00158454c7525939a9c6019a5e874a
Reviewed-on: http://review.whamcloud.com/4056
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-1303 lod: object create to create underlying objects

and generate LOV EA for just created striping. the policies
choosing specific OSPs to use in the striping is in the
subsequent patch.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I2675dd3e2ba4455b2b0daaf4aa7af30b910e9a75
Reviewed-on: http://review.whamcloud.com/4057
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1303 lod: QoS allocation policy

basically a copy of lov_qos.c implemented with OSD API.

the patch introduces a special lu-helpers allowing objects
with zero fid - used by LOD/OSP as fid is being created late.
this is a requirement of current orphan cleanup protocol.
probably will be replace with natural fids generated outside
of OSP/transaction at some point.

Signed-off-by: Alex Zhuravlev <alexey.zhuravlev@intel.com>
Change-Id: I9f41d8a770eb7b8ca04b729bbf0866e5d27fff8a
Reviewed-on: http://review.whamcloud.com/4058
Reviewed-by: wangdi <di.wang@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Mike Pershin <tappro@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>