LU-80 mds: use md_size supplied by client, repack reply

- mdt uses only client easize to pack reply buffer with the same
buffer sizes as client has.
- introduce reply growing, when packing reply the proper MD size
might be unknown so clients data is being used, but after request
processing the proper size may be bigger than client expects and
reply buffer must be re-packed.
- if server data doesn't fit in buffer then it allocates bigger
buffer instead of using reply message buffer.
- rename mdt_shrink_reply into mdt_fix_reply. It does shrink and/or
grow when needed. Upon growing the bigger MD is copied into new
reply.
- server unpacks ea attr size always in mdt_reint_record for all
operation, so it is available always in rr_eadatasize.

Port from: ORI-80
Author: Mikhail Pershin <tappro@whamcloud.com>

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: Ieae6a1d4d07fdf7643ca6900d02d0dd962a07f6c
Reviewed-on: http://review.whamcloud.com/1808
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1095 debug: Standardize, suppress mount/umount messages

Standardize mount/umount console message to include profile name,
and optionally suppress them with the 'quiet' mount option.  We
have been using private namespaces for testing and mounting then
umounting the FS as needed for each job.  In this context these
messages end up causing alot of syslog noise.

Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: I7514f6016c337a358e5e31146644810dff292d02
Reviewed-on: http://review.whamcloud.com/2199
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1095 debug: Common client/server message standardization

Enhance and standardize several common messages.  In particular
when a peer is involved ensure peers nid is in the message, and
on the server include the obd name in the message.

Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: Iaea477e7dab240866a10c1863886d21d674e293d
Reviewed-on: http://review.whamcloud.com/2200
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1095 debug: Improve messages for fake requests

Update the console filter to correctly handle fake requests and
squelched the lov_update_create_set() message for the
-ETIMEDOUT/-ENOTCONN case.

 LustreError: 7872:0:(lov_request.c:693:lov_update_create_set()) error
 creating fid 0x104c5e0b sub-object on OST idx 53/2: rc = -107

Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: I5f37f585566b053d515665fcddbcc8a3e653d89a
Reviewed-on: http://review.whamcloud.com/2203
Tested-by: Hudson
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1095 debug: Report remaining recovery time consistently

Consistency is good, always report the remaining recovery time
in the mm:ss format.  This patch get's the last 3 remaining
instances where it is simply reported as a total number of seconds.

Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: If5599d8c24b1cd862ab89670553fcd24672cadbc
Reviewed-on: http://review.whamcloud.com/2204
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1214 ptlrpc: splits server-side connection/bulkIO/recovery

Splits/removes these server-side handling from client:
1. Server-side connection and disconnection
2. Splits obdecho client and server
3. Server-side bulk I/O
4. Server-side recovery handling

Uses "HAVE_SERVER_SUPPORT" to protect those server-side codes.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: I14422a36ffa6fa46604682c9eed6d27dd56a4888
Reviewed-on: http://review.whamcloud.com/2360
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1247 obdfilter: fix invalid check of precrate objects

MDT precreate objects when it has objects count less than the
oscc->oscc_grow_count / 2. oscc->oscc_grow_count can be equal
to OST_MAX_PRECREATE, so MDT (last_id - next_id) is less than the
(OST_MAX_PRECREAT * 3 / 2). This patch fix the wrong condition at
filter_handle_precreate() when delete orphans request happend.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Andrew Perepechko <andrew_perepechko@xyratex.com>
Reviewed-by: Andriy Skulysh <andriy_skulysh@xyratex.com>
Xyratex-bug-id: MRP-440
Change-Id: I5bac3ee7261fe7462254bd80565d770041c3c206
Reviewed-on: http://review.whamcloud.com/2365
Reviewed-by: Andriy Skulysh <Andriy_Skulysh@xyratex.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1280 ldiskfs: remove LASSERTF from ext3_ext_new_extent_cb()

The LASSERTF() in ext3_ext_new_extent_cb() was injected for
debugging purpose to make sure the race really happened but
was forgotten to be removed from the original patch in
http://review.whamcloud.com/1618 .

Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: I978b8ab88cc4413c7ac00db838f7578f8011b192
Reviewed-on: http://review.whamcloud.com/2452
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
LU-709 build: clean up LC_STRUCT_INTENT_FILE and LC_SUNRPC_CACHE

1. open_intent.file was added since v2.6.14
2. cache_head.ref was added since v2.6.16

Signed-off-by: Peng Tao <peng_tao@emc.com>
Change-Id: Ie4047431f56cf71ef7a17115c2cd1a3812256419
Reviewed-on: http://review.whamcloud.com/2314
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-709 build: clean up HAVE_CAN_SLEEP_ARG and HAVE_F_OP_FLOCK

1. flock_lock_file_wait only has two arguments ever since v2.6.12
2. file_operations.flock exists since v2.6.12

Signed-off-by: Peng Tao <peng_tao@emc.com>
Change-Id: I92406f1042dab71ade780b02500fba4404a56a2b
Reviewed-on: http://review.whamcloud.com/2315
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-911 osd: fix th_sync and OSD_EXEC_OP issues

This issues occurs during the porting from Orion to the master
- th_sync should accumulate values with '|=' to be not dropped
  occasionally to the async state from sync
- remove duplicated code
- remove ASSERTION in OSD_EXEC_OP as it was previously done but
  returned again from orion by mistake

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I6ee7eb8a086eae6e0a96364a90a192e6ac95903b
Reviewed-on: http://review.whamcloud.com/2530
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-911 osd: use struct obd_statfs instead of kstatfs_t

OSD API and users to use struct obd_statfs.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I6bd98a632db9cd9f6ee4309c29d878a61bd7a228
Reviewed-on: http://review.whamcloud.com/1837
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-911 ost: ost_handle() to refill environment on every requests

OST threads hold environment (lu_env) for request handling. when
a last instance of some device (say ofd or osd) gone, all keys
registered by the device are released, but enviroments are kept.
then at device instantiation the environments aren't repopulated.
this patch fixes this.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ib691f22955d40335535fcf3013d62badb24f7f49
Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/1850
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1209 tests: add debug info to 133d

Add more information in 133d to help understanding the
failure.

Signed-off-by: Wang Di <di.wang@whamcloud.com>
Change-Id: Ic84120fa5adcab9490abce732993c7c90d3e2881
Reviewed-on: http://review.whamcloud.com/2309
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-549 llite: Improve statfs performance if selinux is disabled

Even if selinux is disabled, client still tries to get selinux
attributes from MDS. As xattrs are not yet cached, this significantly
slows down xattr heavy operations like ls -l. This patch forces
to return -EOPNOTSUPP on the client side if selinux is disabled.
It speeds up ls -l 25% for cold-cache case and 50% for hot-cache
case.

Signed-off-by: Yevheniy Demchenko <zheka@uvt.cz>
Change-Id: I5e416093bba4126e5fcad62d8c0a2963c1866386
Reviewed-on: http://review.whamcloud.com/2503
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1249 debug: Auto correct improper debug buffer size setting

Use the minimum required value when the debug buffer size setting
value is too small, and use the maximum acceptable value when it is
too large.

Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: I89def7762f2ec9da3a25d28f7ffa9aede390eb85
Reviewed-on: http://review.whamcloud.com/2489
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1319 mdt: increment MDT getattr stats

Move increment of MDT getattr stat from mdt_getattr() to
mdt_getattr_internal() so we don't miss other call paths
that may service getattr requests.

Signed-off-by: Ned Bass <bass6@llnl.gov>
Change-Id: I45ae7a73eaf81fbe56775725d07ec275b4261f1f
Reviewed-on: http://review.whamcloud.com/2526
Reviewed-by: Yang Sheng <ys@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1312 kernel: crash at boot time in isci driver

Restore SG_ALL to default value to avoid crash isci.

Signed-off-by: yang sheng <ys@whamcloud.com>
Change-Id: Ie20bbf669c64514fc95f1f2b274a689d65cd0c2a
Reviewed-on: http://review.whamcloud.com/2596
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-577 tests: FAIL replay-single test_70b rundbench load

Test 70b for replay-single assumes that lustre is mounted on
/mnt/lustre which is not the case for us. This patch passes
the proper MOUNT. The test also was not using the standard
DIR/tdir setup which had generated data files not being
cleaned up. Increased the sleep period to match dbench's
warm up period. This gives dbench a change to start up when
using many clients. Set the pdsh FANOUT environment variable
because by default pdsh launches in blocks of 32 nodes. This
way pdsh will lauch all node jobs at the same time

Signed-off-by: James Simmons <uja.ornl@gmail.com>
Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: Iefdfe4bf47d322e782e138cf6ae17240b3004d8b
Reviewed-on: http://review.whamcloud.com/252
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1095 debug: CWARN to CDEBUG for mds_notify() event

Both of these warnings represent correct behavior the administrator
does not need to know about, or more importantly do anything about.
As such I am moving both of these warnings to CDEBUG(D_CONFIG).

  Lustre: 8099:0:(mds_lov.c:1167:mds_notify()) MDS lc1-MDT0000:
  add target lc1-OST0023_UUID

  Lustre: lc1-MDT0000: in recovery, not resetting orphans on
  lc1-OST0007_UUID

Signed-off-by: Christopher J. Morrone <morrone2@llnl.gov>
Change-Id: I66a98d87e3d5de7205420c74db4f6d9bcaaf31a7
Reviewed-on: http://review.whamcloud.com/2202
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>