sheep: free memory used for exceptional nodes

It seems that current clear_exceptional_node_lists() leaks memory used
for representing delayed and failed nodes.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: let sheep return correct exit status when locking a base directory fails

Currently, lock_and_daemon() always returns 0 as its exit
status. But this behavior is confusing because users cannot detect
errors during locking a base directory via the exit status. This patch
modifies lock_and_daemon() for returning correct exit status.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
configure: remove -Wdeclaration-after-statement from the parameter for GCC

Current sheepdog employes gnu99 style. But
-Wdeclaration-after-statement is still passed to GCC so interleaving
declarations and statements is not allowed. This patch remove it.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
[don't merge] tests: valgrind option for lower memory machine

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
[tmp] tests: don't use /dev/urandom for better performance

/dev/urandom is too slow.  We don't need good random values to fill vdis.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
work: move some globals into work.c

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
work: rename init_wqueue_eventfd and init_work_queue

init_wqueue_eventfd() is not appropriate for a global name because we
don't have to know that the work queue uses eventfd outside of work.c.
This renames init_wqueue_eventfd() to init_work_queue(), and
init_work_queue() to create_work_queue().

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
work: remove sheep_priv.h and trace.h from include files

This removes sheep specific header files to move work.c to lib.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
move work queue codes from sheep to lib

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
work: initialize nr_nodes in init_work_queue()

With this patch, collie can create an appropriate size of worker
threads for dynamic work queues at the start time.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
collie: add a function to wait for all the running works

This is in preparation to use work queue in collie.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: add operation to get object sha1 digest

Currently, we compare entire object contents to check whether there is
no difference among them.  However, this wastes a lot of network
bandwidth and some operations like vdi check or object recovery take
too long time to complete.

This adds an operation SD_OP_GET_HASH to get the sha1 digest of the
object, and enables us to do faster vdi check and recovery.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
collie: use hash for collie check

This patch checks the hash values of the target objects and skips
check and repair when there is no difference between them.  This also
uses work queue to boost the vdi check performance.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
tests: fix vdi check output

Now 'collie vdi check' fixes vdi in parallel, so we have to sort the
result of vdi check to get a unique output.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: recover from local when target node has same content

This checks the hash value of the target object before reading the
entire data, and recovers from local if there is a same object in the
stale directory.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>