add IOD mlogfacs, and pull out plfs_init.

1. add four mlogfacs for IOD.
2. allow plfs_init to be called by IOD.

Signed-off-by: Liu Xuezhao <xuezhao.liu@emc.com>
Change-Id: Ie287a204374ac3e02c4d73b3315b31bd8942f34a
Reviewed-by: Zhang Jingwang <jingwang.zhang@emc.com>
Tested-by: Zhang Jingwang <jingwang.zhang@emc.com>
readx/writex

This patch set is written for list IO in HPC.

patch set 9:
    include <sys/uio.h> in plfs.h.in, so APP can save it.

patch set 8:
    rebase to head of master @50dc7dd7ec548b0e009f19e928d74782acf4cddc

patch set 7:
    Don't write more data than total size of iovec in writex
    Don't merge readTask if they are not sequensial in memory

patch set 6:
    rename to container_free_or_cache_temporary_index

patch set 5:
    change WriteFile::writeIndex private
    rename container_disposal_index to container_free_temporary_index

patch set 4:
    refined based on John's comments following README.coding

patch set 2:
    replace ?: to std::min
    refined writex based on John's idea - one writev syscall only

patch set 3:
    refine readx to use thread pool

Change-Id: Ib7d564e4cee2d5bbabefb083a95df999593fda32
Reviewed-by: Zhang Jingwang <jingwang.zhang@emc.com>
Tested-by: Zhang Jingwang <jingwang.zhang@emc.com>
IOD PLFS interface

This patch set includes PLFS changes for IOD usage.

patch set 11:
    ignore ENOENT for rmdir

patch set 10:
    rebase to readx/writex patch set 9
    ignor EEXIST error for mkdir/rmdir in container mode

patch set 9:
    add a macro PLFSRC_PARSE_BUFSZ to drop some hard-coded values

patch set 8:
    change to plfs_query_shard API carrying a switch to retrieve
    back location info

patch set 7:
    rebased readx/writex patch set 7

patch set 6:
    change ssize_t to int in plfs_query_range.
    rebased readx/writex patch set 6

patch set 5:
    add some comments and revised valid range stashing logic

patch set 4:
    rebased readx/writex patch set 5

patch set 3:
    rebased readx/writex patch set 4

patch set 2:
    add plfs_query_range API

patch set 1:
    IOD and PLFS share one configration file. Two IOD options are added
    and PLFS will just ignore them.
    Priority: ENV(PLFSRC) > $HOME/.plfsrc > /etc/plfsrc

Change-Id: Ida63c2c6e93be57316e5034023a6fcd1e0eda278
Tested-by: Zhang Jingwang <jingwang.zhang@emc.com>
Reviewed-by: Zhang Jingwang <jingwang.zhang@emc.com>
add recursive supoort to plfs_rmdir

if user passes in a non-zero recursive flag, then remove directory and
its contents recursively.

patch set 3:
    revise unlinkop::op_r() to only operate on directory or container

patch set 2:
    rebased to current head of PLFS master @50dc7dd7ec548b0e009f19e928d74782acf4cddc

Change-Id: If1e3462108eee0355877c819a113b84d2bc8739b
Reviewed-by: Zhang Jingwang <jingwang.zhang@emc.com>
Tested-by: Zhang Jingwang <jingwang.zhang@emc.com>
fix a possible bug when merging tasks

Change-Id: I8b95474a8c19a421f3ab48e422b523a065d4279d
Reviewed-by: Zhang Jingwang <jingwang.zhang@emc.com>
Tested-by: Zhang Jingwang <jingwang.zhang@emc.com>
Fix some compilation problems

Change-Id: I45ce8879ff99a39380c3ff7b894f0865578ce7f2
Reviewed-by: Zhang Jingwang <jingwang.zhang@emc.com>
Tested-by: Zhang Jingwang <jingwang.zhang@emc.com>
Rebase the readx and writex

Change-Id: I7ab833cb066fcf98a8334b6f05c823db469c2421
Reviewed-by: Zhang Jingwang <jingwang.zhang@emc.com>
Tested-by: Zhang Jingwang <jingwang.zhang@emc.com>
Add log facs for IOD

So that the users need not to run ./makefacs.pl themselves.

Change-Id: I0c26a9486e404d312a6437d7aff35869d36a01cb
Reviewed-by: Zhang Jingwang <jingwang.zhang@emc.com>
Tested-by: Zhang Jingwang <jingwang.zhang@emc.com>
Add checksum support to PLFS

Finished the write path. Applications can call the plfs_writec()
to start a write with checksum.

Disable compress_contiguous because of checksum support.

Change-Id: I59a65e2499a344d7a465dc480fc5836af6a9b051
Clean up the index related code

Make the HostEntry a POD data structure to represent the raw data
in memory or index files.

The ContainerEntry represents the index entry in the global index
tree.

Change-Id: I217039315105fa5729480270b143f45e2854ebe7
Finish the new Index tree and keep the legacy read path

Add remap layer and checksum to the Index tree to form the new
Index tree. And ignore the checksum in the legacy read code
path to be compatible with old code.

Truncation is handled by recalculating the checksum.

Change-Id: Id74e248d2c687ef64379f161801821c5e5283c31
Finish read with checksum

If the user don't want to get the checksum, we will get the checksum
internally and verify it for testing purpose.

Change-Id: Ieb318077f2e4bf988697efd6a142159540c4917a
Add checksum support to plfs_readx and plfs_writex

Add unit tests and fix problems found.

Change-Id: Iedfc20bac9dab32450d13e0f9ba0d7eb5d1fa298
Use S_ISVTX to control the checksum feature for PLFS

So that checksum can be disabled or enabled by the user.

Change-Id: I4a4b95476b73ab1798fa5f35b6eddbd1f8072525
Fix a problem found in IOD unit testing.