LU-1305 osd: osd-zfs to use correct objset name

before utils were fixed, we have to add '/' to objset name.
now this is not required.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ice60fa6c30ede531965ec0032f7134c6c412e4c2
Reviewed-on: http://review.whamcloud.com/3199
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-1095 debug: no console message for long symlink

Some minor fixups from running local sanity.sh tests:
- don't print a console message for long-but-legal symlinks,
  which could be hit during normal usage as well

Signed-off-by: Andreas Dilger <adilger@whamcloud.com>
Change-Id: I24e3e8ea9d8db0ca9c59597f3170f2b18359500c
Reviewed-on: http://review.whamcloud.com/3130
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Li Wei <liwei@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1203 mdt: recognize old rootsquash and nosquash_nid params

Change mdt_process_config() to make it capable of recognizing
old "mdt.rootsquash" and "mdt.nosquash_nid" parameters.

The new parameters are "mdt.root_squash" and "mdt.nosquash_nids".

Signed-off-by: Yu Jian <yujian@whamcloud.com>
Change-Id: I9fc71e2a997a3b0835a3adad2ccfde6d2c64ca56
Reviewed-on: http://review.whamcloud.com/2680
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-56 ldlm: SMP improvement for ldlm_lock_cancel

ldlm_del_waiting_lock() is always called twice in ldlm_lock_cancel
even the ldlm_lock is not on expiring waiter list, this is bad for
performance because it needs to grab a global lock and disable
softirq.
This patch add a new bit l_waited to ldlm_lock, it's set only if
ldlm_add_waiting_lock() is ever called, and we can bypass
ldlm_del_waiting_lock() in ldlm_lock_cancel() if w/o this bit.

I also adopted patch(32562) on BZ 20509 because it's necessary for
this improvement.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ie48d527b8da187a88646aa070be3aa4beb304b1d
Reviewed-on: http://review.whamcloud.com/3141
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1479 build: enable srp initiator module with lbuild

kernel-ib doesn't include ib_srp module if RPMs built by lbuild.
It's required for Infiniband based storage.

Signed-off-by: Shuichi Ihara <sihara@ddn.com>
Change-Id: I16e940592f93abc9150f391316b8f55eb60c663a
Reviewed-on: http://review.whamcloud.com/3049
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Minh Diep <mdiep@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1448 llite: Prevent NULL pointer dereference on disabled OSC

When a file system is mounted with a disabled OSC reading the import
information from the proc file system can result in a NULL pointer
dereference. The Lustre import on a disabled OSC with remain
in the LUSTRE_IMP_NEW state and imp_connection will remain NULL.

Signed-off-by: Jeremy Filizetti <jeremy.filizetti@gmail.com>
Signed-off-by: Bobi Jam <bobijam@whamcloud.com>
Change-Id: Ia6df51a36efbcd5a7fc7668bb23455b253ae4855
Reviewed-on: http://review.whamcloud.com/2995
Reviewed-by: Liang Zhen <liang@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1528 test: run_one_logged should return error when test fail

run_one_logged need to return error code even FAIL_ON_ERROR=false

Signed-off-by: Minh Diep <mdiep@whamcloud.com>
Change-Id: I1ac59ddfb58b25e7292aacc8c5a31fc218371520
Reviewed-on: http://review.whamcloud.com/3112
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Wei Liu <sarah@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1561 tests: Fix ALWAYS_EXCEPT in mds-survey.sh

Modifying the ALWAYS_EXCEPT assignment in mds-survey.sh does not take
effect, because the test filters have already been built.  This patch
moves the build_test_filter() call below the ALWAYS_EXCEPT assignment.
Also moved downward is the check_and_setup_lustre() call, which
usually goes after environment checks.

Test-Parameters: testlist=mds-survey
Change-Id: I547310e4fccfaaa4fbe7010e14baefa2ab9bbdd4
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/3183
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1305 osd: dmu helpers

few simple functions to wrap native dmu funtions.
to be replaced with direct calls at some point.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: Ibf9331b5668b1e7db22670caf395391bca002061
Reviewed-on: http://review.whamcloud.com/2971
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
LU-1415 tests: Basic support for ZFS-based servers

This patch extends Test Framework to support formatting and mounting
ZFS-based servers.

Current global FSTYPE is no longer flexible enough, as different
facets can use different types of back ends.  Such "mixed-fstype"
configurations are firstly resulted from the incremental landing of
Orion changes.  For example, when we have OFD and ZFS OSD, although
OSTs will be able to use ZFS, MGS and MDTs can still only use LDiskFS.
Secondly, to reduce the test matrix, we may also want to have both
OSTs using LDiskFS and OSTs using ZFS in the same test cluster.
Hence, this patch makes back end file system type a per-facet
attribute.  The flexible configuration variables in local.sh are
reserved for users; all other scripts shall use facet_fstype()
instead.

The major differences between using an LDiskFS-based target and a
ZFS-based one are on the mkfs.lustre(8) command line:

  - ZFS-based targets shall have "--backfstype=zfs", while
    LDiskFS-based ones shall have "--backfstype=ldiskfs".

  - LDiskFS-specific "--mkfsoptions" arguments shall not be given to
    ZFS-based targets, and vice versa.

  - ZFS-based targets have different device specifications.  See
    mkfs.lustre(8).

In addition, we will make "--index" mandatory.  Naturally, format
options have to be generated for each facet dynamically, based on its
back end type, index, etc.  This patch changes mkfs_opts() to do
exactly that.  All test scripts shall use mkfs_opts() instead of
reading related environment variables directly.

Similarly, mount options should also be generated on a per-facet, or
at least per-fstype, basis.  Nevertheless, this patch takes a shortcut
by keeping current per-facet-type {MGS,MDS,OST}_MOUNT_OPTSs and
generating the "loop" mount option dynamically in mount_facet().  I
believe this solution introduces fewer changes comparing to a pure
per-facet one and is sufficient for current use cases.

This patch is based on Brian Behlendorf's work under ORI-155.  See
http://review.whamcloud.com/1417.

Change-Id: Ifcce9b10179dd1b4992a30d10df13ea10bc34548
Whamcloud-bug-id: ORI-155
Test-Parameters: testgroup=full
Test-Parameters: testgroup=full envdefinitions=USE_OFD=yes,LOAD_MODULES_REMOTE=true
Signed-off-by: Li Wei <liwei@whamcloud.com>
Reviewed-on: http://review.whamcloud.com/2907
Tested-by: Hudson
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-1577 lnet: should export lnet_net2ni

We don't want to export lnet_net2ni_locked because it requires
an extra parameter CPT number, which is internal to LNet and
confusing to user, also, no LND is using it/should use it, however,
LND does need lnet_net2ni() which is an inline function and it is
referring lnet_net2ni_locked(), which means it forces us to export
lnet_net2ni_locked.
The way to fix should be simple, we can just dis-inline lnet_net2ni()
and export itself instead of exporting lnet_net2ni_locked.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I251d42d47770211ae6238ee56972fb9aacd63599
Reviewed-on: http://review.whamcloud.com/3207
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1305 osd: Makefiles for osd-zfs

Makefiles ...

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I1fdf9eecf98306b0cfe0bbc0611df3a977f48b10
Reviewed-on: http://review.whamcloud.com/2972
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
LU-1305 build: support for osd-zfs

changes to enable building of spl, zfs and osd-zfs.

Signed-off-by: Alex Zhuravlev <bzzz@whamcloud.com>
Change-Id: I01dcc211b6943b3ed19eaf8756397a97095bbc59
Reviewed-on: http://review.whamcloud.com/2885
Tested-by: Hudson
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-1201 checksum: add libcfs crypto hash

Add libcfs crypto hash and cleanup all lustre hash checksumming.
Now lustre hash calculations base on linux kernel crypto api for kernel,
and base on libcfs crypto implementation for userlevel. So any improvement
at linux kernel for hashes would improve lustre.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Alexander Zarochentsev <alexander_zarochentsev@xyratex.com>
Xyratex-bug-id: MRP-337, MRP-471
Change-Id: I2fbf0f4d0c8ce7e7c3c7ea411c6ccd9dcfc7e03a
Reviewed-on: http://review.whamcloud.com/3098
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1182 ldiskfs-osd: space accounting support

Add space accounting support to ldiskfs OSD.

This patch also sets initial attributes in do_create().
mdd_attr_set_internal() from mdd_object_initialize() is kept until
EDQUOT is returned in lquota itself.
Attributes of new inodes are now initialized in osd_object_create().
All LA_MODE bits are now passed to ldiskfs_create_inode().
(original patch from LiWei, see ORI-46)

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I77a621c76343c2633810bb3cef9859ee30b7b23a
Reviewed-on: http://review.whamcloud.com/3160
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-56 lnet: SMP improvements for LNet selftest

LNet selftest is using a global WI threads pool to handle all RPCs,
it has performance problem on fat cores machine because all threads
will contend on global queues protected by a single spinlock.
This patch will fix this by creating WI scheduler for each CPT,
RPCs will be dispatched to WI schedulers on different CPTs, and there
is no contention between threads in different WI schedulers.

Another major change in this patch is create percpt data for LST
service. In current implementation each service has a global data
structure to store buffer list and RPC list etch, and all operations
on them are protected by per-service lock, again, this could be a
serious performance issue if the service is busy enough. Having
percpt service data would resolve this issue because threads running
in one CPT will only require lock lock and access local data.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I8035faf2e87d8e424a8c2fac903bf3b241668e00
Reviewed-on: http://review.whamcloud.com/2805
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Reviewed-by: Lai Siyao <laisiyao@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-1182 ofd: add space accounting support

Add space accounting support to OFD. This is done by calling into the
lquota library which looks up space usage via the accounting objects.
Report success on quotacheck, quotaon & quotaoff for the time being,
otherwise space accounting can't be tested with current MDT stack
which requires quotacheck to be run for space accounting to be
enabled.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: I442f3d1bcc9e7c95bb3f505f7a00665419d0f517
Reviewed-on: http://review.whamcloud.com/3186
Tested-by: Hudson
Reviewed-by: Niu Yawei <niu@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-56 lnet: tuning wildcard portals rotor

By default, PUT messages arrived on wildcard Portals always prefer
to match buffers on current CPT, this patch added a few more
options for dispatching message match which's called portals rotor:

- OFF: Turn off message rotor
- ON: round-robin dispatch all PUT messages to different CPTS
- RR_RT: round-robin dispatch routed PUT message to different CPTs
- HASH_RT: dispatch routed PUT message to CPTs by hashing source NID

User can change portals rotor by /proc interface at runtime.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ie2f03255d9b00b005ca5365315fff350d45a6933
Reviewed-on: http://review.whamcloud.com/3193
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>