sheep: make the vid deletion proceduer correct order

Current VDI deletion process deletes VDI in this order:
1. delete objects which belong to the VDI
2. notify deletion of the VDI for removing object list cache
3. clear inode object with zero

But this should be like this:
1. delete objects which belong to the VDI
2. clear inode object with zero
3. notify deletion of the VDI for removing object list cache

Because in the process of removing object list cache, existing of the VDI is
checked with the name of the VDI is zero-cleared or not
(objlist_deletion_work()). So objlist_deletion_work() would have a possibility
of leaving the object list cache incorrectly. This patch makes the process
correct order.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: initialize vdi bitmap after completion of reading inode object

Current init_objlist_and_vdi_bitmap() sets a bit of corresponding VDI
before initializing the VDI. This should be done after the
initialization for defensive purpose.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: refine help messages

Signed-off-by: Liu Yuan <namei.unix@gmail.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
dog: add 'vdi cache purge' option

This command is intended to be used when

1. cache of some vdi isn't clean on node A, e.g, due to sheep crash
2. This vdi is run on other node B.

After some time, A comes back without manually 'rm cache/', the cache about this
vdi is stale. For this case, we provide a manual option for users to purge the
stale cache.

usage:
 $ dog vdi cache purge image # purge cache of image
 $ dog vdi cache purge # purge cache of all the images

Signed-off-by: Liu Yuan <namei.unix@gmail.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
dog: check the result of dog_exec_req() correctly

dog_exec_req doesn't return a result code after the commit 5940548.
We have to check rsp->result instead of its return value.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
dog: add more result check of dog_collie_exec()

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: don't remove vdi object from object list cache

We don't remove a vdi object from the underlying store even after the
vdi is deleted.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
modify configure.ac to build sheepdog on rhel5 environment

In our product machine we use rhel5 on 2.6.32-linux-kernel, but sheepdog can't build
on rhel5. It need some modifications to do this.

1. old version autoconf does not support AM_SILENT_RULES() so replace it with check for shell.
2. use SIZEOF_SHORT/INT/LONG instead of AC_TYPE_INT16/32/64.
3. check header files such as sys/eventfd.h sys/signalfd.h etc.
4. use SIZEOF_LONG instead of __SIZEOF_LONG__

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
modify implemention of functions to build sheepdog on rhel5 environment

For using timerfd, eventfd, signalfd, it need to use syscall() instead of glibc's
standard fucntions on rhel5.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
let logger work on rhel5 environment

In gcc-4.1.2, it dose not support attribute like "constructor(priority)" (it only support
contstructor without 'priority'), so change log_format_register from macro to function and
call it in init_log_formatter() directly.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
event: refresh event info after unregistering

The current code causes a segfault with the following scenario:

 1. Both worker_thread_request_done() event and client_handler() event
    become ready with epoll_wait() in do_event_loop().

 2. do_event_loop() calls worker_thread_request_done(), and it calls
    clear_client_info().  This can happen when the client connection
    is already closed (e.g. in put_request(), tx_main()).

 3. clear_client_info() calls unregister_event(), and it frees the
    client_handler() event.

 3. do_event_loop() tries to call client_handler() but the event is
    already freed and no longer valid.

To fix the problem, this patch calls event_force_refresh() in
unregister_event() and make do_event_loop() call epoll_wait() again.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
add helper macros to insert/search/iterate rbtree node

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
remove redundant list_node

We can iterate over a rbtree without a linked list.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
dog: fix send_light_req

This patch fixes segfault of many dog commands due to commit a4d01de

Signed-off-by: Liu Yuan <namei.unix@gmail.com>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
fix compiler warning in object_cache.c

When building sheepdog it reports warning message like:

	object_cache.c: In function ‘object_cache_flush_and_delete’:
	object_cache.c:1039: warning: comparison is always false due to limited range of data type
	object_cache.c: In function ‘load_cache_object’:
	object_cache.c:1252: warning: comparison is always false due to limited range of data type
	object_cache.c: In function ‘load_cache’:
	object_cache.c:1290: warning: comparison is always false due to limited range of data type

The 'idx' is the result of strtoul and it should be 'unsgiend long' instead of 'uint32_t' and
it can't reach ULLONG_MAX but ULONG_MAX.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
sheep: detect invalid response code in sd_strerror()

Commit a4d01de did wrong treatment of data structure of request. And
it caused segfault in sd_strerror() because of wrong index of the desc
array. This patch let the function catch such a bug and return string
representation of unknown response code.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
use fcntl to set FD_CLOSEXEC instead of type "e" in popen()

In glibc < 2.9 environment (rhel5), it dose not support type "e" in popen(), so
we should replace it by calling fcntl(xxx, FD_CLOSEXEC).

If pass NULL into pclose(), it would make program coredump, therefore add check
for argument.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
fix error-io in sheepfs when using ext4 filesystem

When using ext4 filesystem on sheepfs volumes, it will print kern message:

	[171972.794292] io-error-guard: catch 1 continuous bio error.
	[171977.699795] io-error-guard: catch 2 continuous bio errors.
	[171977.707797] Aborting journal on device loop0-8.
	[171978.162038] EXT4-fs error (device loop0): ext4_journal_start_sb: Detected aborted journal
	[171978.170034] EXT4-fs (loop0): Remounting filesystem read-only

The reason is sheepfs process FLUSH opcode incorrectly. If the sheep daemon dose not
support FLUSH, sheepfs should explicitly set sheepfs_object_cache to 'false' and return
success, not return error.

Signed-off-by: Robin Dong <sanbai@taobao.com>
Signed-off-by: Liu Yuan <namei.unix@gmail.com>
tests/functional: let check clean directories of passed tests in default

Some tests consume lots of disk space and cause disk full (at least on
my box with 64GB SSD). This patch lets the check script clean
directories of passed tests in default. In addition, this patch adds a
new option -keep-passed. When this option is passed, the script keeps
directories of passed tests. This would be useful when we need to
check correctness of test scriptes themselves.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: generalize the option parsing

Signed-off-by: Liu Yuan <namei.unix@gmail.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>