Open up 0.9 development

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheepfs: add framework for http interface

Adding http.c in sheepfs dir as the framework and implement "address" file
to store http address.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheepfs: implement the http interface of sheepfs by using libcurl

Use libcurl to call the interface of http for object-storage and implement
the read/write operation.

Signed-off-by: Robin Dong <sanbai@taobao.com>
[ replace sd_err by sheepfs_pr ]
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
test: add test-case for http interface of sheepfs

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
tests/func: fix Copies output due to format change

also update 081.out for a wrong update by commit 71c86fa

Signed-off-by: Liu Yuan <namei.unix@gmail.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
tests/func: remove --nosalt for openssl

Remove it fix a segfault on ubuntu-13.04. Though I don't know why.

Signed-off-by: Liu Yuan <namei.unix@gmail.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheepfs: add check for libcurl in configure.ac

Check header and library for libcurl in configure.ac

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheepfs: add support for "rmdir" and "unlink"

Add support for "rmdir" and "unlink" so users can remove all directories
of accounts/containers and files of objects after using it.
This "remove" operation only remove local entry in sheepfs, the objects
are still in remote object-service.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
tests/084: add sheepfs http rm test

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
dog: add 'vdi object map' command

This command will show inode internal 'vdi' array states, is useful to check if
cloned VDI's inode is sane or not.

usage:
yliu@ubuntu-precise:~/sheepdog$ dog/dog vdi object map clone1
Index       VID
00000000   c8f989
00000001   c8f989
00000002   c8f989
00000003   7c2b25
00000004   7c2b25
00000005   7c2b25
...

yliu@ubuntu-precise:~/sheepdog$ dog/dog vdi object map -i 4 clone1
Index       VID
00000004   7c2b25

With this patch, original 'dog vdi object' becomes 'dog vdi object location'

Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep/http: check http driver in http_init()

If users start sheep daemon like:

sheep -r host=127.0.0.1,port=8000 -c local

the sheep will startup normally, but when someone access
127.0.0.1:8000 by swift interface, it will get message of
"405 Method Not Allowed" without any explaination in sheep.log

So we use 'swift' as default http driver if users doesn't point out
in command line.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
dog: fix compile warning

It will report warning:

node.c: In function ‘node_stat’:
node.c:278: warning: ‘last.r.peer_total_rx’ may be used uninitialized in this function
node.c:278: warning: ‘last.r.peer_total_tx’ may be used uninitialized in this function
node.c:278: warning: ‘last.r.peer_total_nr’ may be used uninitialized in this function
node.c:278: warning: ‘last.r.gway_total_rx’ may be used uninitialized in this function
node.c:278: warning: ‘last.r.gway_total_tx’ may be used uninitialized in this function
node.c:278: warning: ‘last.r.gway_total_nr’ may be used uninitialized in this function

when compiling sheepdog. The reason is we forget to initialize argument 'last'.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheepfs: disable http interface if http is not enabled

Signed-off-by: Liu Yuan <namei.unix@gmail.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: remove fd leak in vdi deletion process

The commit 5e460b4d6444 introduced fd leak in the vdi deletion
process. This patch removes it.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheepfs: fix complie error when http isn't enabled

Forgot to remove -lcurl in my last patch, remove it.

Signed-off-by: Liu Yuan <namei.unix@gmail.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
tests/func/058: fix supurious failure

Reviewed-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
lib: change xmalloc() to xvalloc() in sd_inode.c

The default_read_from_path() returned "Invalid argument" when we running
sheepdog cluster in test environment with sheep option "-D". The reason
is that the allocating memory for reading and writing btree-object is not
aligned. Therefore we should use xvalloc() instead of xmalloc() to allocate
aligned memory.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: go to panic if using unaligned memory in direct-io mode

When startup sheep with option "-D", the plain_store will check alignment of
read/write buffer, but if the buffer is not aligned, it will do nothing if
it dose not choose "--enable-debug" for configuration. Then sheep will open
file with "O_DIRECT" and call pread() with unaligned buffer, it will cause
error io and remove the disk from md which is totally wrong behavior.
So, we should make the check strong enough: sheep will panic directlly if
finding unaligned buffer.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: cleaning vdi deletion process, round 2

This patch cleans the vdi deletion process of sheep. Current vdi
deletion code creates one work for one deletion. It seems that there
is no reason to create so many work. This patch lets the code to
create only one work for vdi deletion request.

With this change, this patch improves simplicity of the code. For
example, the queue of struct deletion_info and related code are
removed.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: fix some compile errors

Both MIN/MAX and roundup are provided by glibc and would cause compile error
when we #include <sys/param.h>. Just use GNU version.

Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>