LU-1201 checksum: add libcfs crypto hash

Add libcfs crypto hash and cleanup all lustre hash checksumming.
Now lustre hash calculations base on linux kernel crypto api for kernel,
and base on libcfs crypto implementation for userlevel. So any improvement
at linux kernel for hashes would improve lustre.

Signed-off-by: Alexander Boyko <alexander_boyko@xyratex.com>
Reviewed-by: Alexey Lyashkov <alexey_lyashkov@xyratex.com>
Reviewed-by: Alexander Zarochentsev <alexander_zarochentsev@xyratex.com>
Xyratex-bug-id: MRP-337, MRP-471
Change-Id: I71e7d4fbf45ee40c0c15e235eb71fb4aa86c17d1
Reviewed-on: http://review.whamcloud.com/2276
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1406 ofd: grant support

Add grants functionality

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I660508681e1e33b01dfbdb44ac32705575ee85e1
Reviewed-on: http://review.whamcloud.com/2871
Tested-by: Hudson
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Johann Lombardi <johann@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1406 ofd: transaction and last_rcvd functions

Add ofd transaction handlers and last_rcvd update function

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I14467658ea36d5b71eec7ce183190509a049eb09
Reviewed-on: http://review.whamcloud.com/2886
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1406 ofd: add object methods

OFD objects methods except IO

Signed-off-by: Mikhail Pershin <tappro@whamcloud.com>
Change-Id: I17d1e9fd462f8022b516f16df71e40be0879efea
Reviewed-on: http://review.whamcloud.com/2887
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
LU-56 ptlrpc: clean up ptlrpc svc initializing APIs

Both ptlrpc_init_svc and ptlrpc_init_svc_conf have long parameter list
and difficult for user, also, it doesn't make too much sense to keep
both of them. This patch will merge them to a new one
ptlrpc_register_service, and improve the structured parameter
ptlrpc_service_conf and make it more clear for user.
This is the first patch for ptlrpc service SMP improvements, more
patches depending on this will come soon.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I7b2932d4235fb418bee8e9d65f8b3e4f62d2849e
Reviewed-on: http://review.whamcloud.com/2878
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: wangdi <di.wang@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-56 lnet: new internal object lnet_peer_table

Instead of attaching all peers under the big global data of LNet, we
created lnet_peer_table which is just a simple container of peers.

We also made another small change in this patch, when the last
refcount of a peer is released, the dead peer wouldn't be released
anymore, instead it will be attached on deathrow list of peer-table
and released while destroying the peer-table. Peers on deathrow list
can be re-used if we need create a new peer.

Both changes in this patch are just preparations for other LNet SMP
improvements. For example, we will create instances of
lnet_peer_table for each CPT in following patches.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I60e6f2e564f9ab9c86278c9a08e24aaac114e89c
Reviewed-on: http://review.whamcloud.com/2923
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-56 lnet: cleanup for LNet Event Queue

This patch is just a cleanup patch for LNet Event Queue (EQ),
it's an intermediate patch for upcoming LNet SMP improvement
for EQ.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I2761c0c4118fa709a921b4060a7bd5ec6700a3f0
Reviewed-on: http://review.whamcloud.com/2924
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-56 lnet: allow to create EQ with zero eq_size

Current LNet doesn't allow to create EQ with zero eq_size, it's
an unnecessary limit if the EQ has callback, because EQ callback
is guaranteed to get every event. EQ with no-zero eq_size is only
useful for LNetEQPoll, there is no Lustre use-case that is using
both EQ callback and LNetEQPoll on a same EQ.

Also, even with upcoming LNet SMP improvements, there has to be
a lock to protect enqueue/dequeue operations of event, which might
hurt performance because of lock contention.

This patch will allow user to create EQ with eq_size being zero,
which means events will only be delivered by callback and user
will get nothing by calling LNetEQPoll.

In multiple thread environment, we would suggest user to create EQ
only with callback (eq_size=0) if the EQ might get high rate events,
LNetEQPoll can still work fine if eq_size is non-zero and nothing
will be changed at all. User is still able to create EQ with both
callback and non-zero eq_size, and call LNetEQPoll to get event,
although it's deprecated.

Summary: still can support all current use-cases, although set
eq_size to zero and only rely on callback will be the better way.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: Ia984f2f65eb5fd064a36dfd7b399e46378013c57
Reviewed-on: http://review.whamcloud.com/2925
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Hudson
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-933 mdc: allow disabling the mdc_rpc_lock

It is desirable to allow disabling the client mdc_{get,put}_rpc_lock()
in order to allow clients to send multiple filesystem-modifying RPCs
at the same time. While this would break MDS recovery (due to
insufficient transaction slots in the MDS last_rcvd file) it would
allow a smaller number of clients to generate a much higher RPC load
on the MDS. This is ideal for MDS/RPC load testing purposes, and can
also be used to help evaluate the potential benefits of implementing
the multi-slot last_rcvd feature.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I39be390ed37f0327bf7eb8015d528195b8568e0b
Reviewed-on: http://review.whamcloud.com/2084
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Andreas Dilger <adilger@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-56 lnet: move "match" functions to lib-ptl.c

This is still an intermediate patch for LNet SMP improvements,
it covered a few things:
- create a new file lib-ptl.c, all functions about portals
  are moved into this file.
- always precreate match hash-table for all portals, it's a little
  wasty for unique-portal, but it will save a lot of problems for
  upcoming patches
- instead of storing all portals in contiguous buffer, we allocate
  memory for each portal in turn, this change is also for upcoming
  patches.

Signed-off-by: Liang Zhen <liang@whamcloud.com>
Change-Id: I19e61c7f3a01f1c90a9f3f78d48d81dc00cd037d
Reviewed-on: http://review.whamcloud.com/2926
Tested-by: Hudson
Reviewed-by: Doug Oucharek <doug@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Bobi Jam <bobijam@whamcloud.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
LU-1461 vmalloc usage needs linux/vmalloc.h include

It seems the ext4-vmalloc patch forgot to include linux/vmalloc.h
in mballoc.c

Signed-off-by: Oleg Drokin <green@whamcloud.com>
Test Requirements: fortestonly
Test-Parameters: ostcount=2

Change-Id: I994b75353ce13bac41f45e6bf494191b43304e3c
Reviewed-on: http://review.whamcloud.com/3036
Tested-by: Hudson
Tested-by: Oleg Drokin <green@whamcloud.com>
Reviewed-by: Yu Jian <yujian@whamcloud.com>
Reviewed-by: Alex Zhuravlev <bzzz@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
LU-1373 ptlrpc: add flow control extension to ptlrpc req set

This patch allows new requests to be added to a request set while
this latter has already requests in flight. This is done by adding a
callback function invoked by the request set to generate RPCs.
The request set will fire a new RPC each time one completes, keeping
the number of RPCs in flight equals to set->set_max_inflight.

Lock callbacks can thus be sent by the service thread again. This
avoids doing disk I/O from the ptlrpcd context.

Signed-off-by: Johann Lombardi <johann@whamcloud.com>
Change-Id: If95922fa8da1dfa7ce7c98d6bfe35e9a5f5bb34f
Reviewed-on: http://review.whamcloud.com/2650
Tested-by: Hudson
Reviewed-by: Fan Yong <yong.fan@whamcloud.com>
Reviewed-by: Jinshan Xiong <jinshan.xiong@whamcloud.com>
Tested-by: Maloo <whamcloud.maloo@gmail.com>
Reviewed-by: Oleg Drokin <green@whamcloud.com>
Revert "LU-1201 checksum: add libcfs crypto hash"

This reverts commit 84a3fd67356c8073a917ea6abd63928055e38156
that broke build for external ofed on rhel5 kernels.

Signed-off-by: Oleg Drokin <green@whamcloud.com>