work: move some globals into work.c

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
work: rename init_wqueue_eventfd and init_work_queue

init_wqueue_eventfd() is not appropriate for a global name because we
don't have to know that the work queue uses eventfd outside of work.c.
This renames init_wqueue_eventfd() to init_work_queue(), and
init_work_queue() to create_work_queue().

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
work: remove sheep_priv.h and trace.h from include files

This removes sheep specific header files to move work.c to lib.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
move work queue codes from sheep to lib

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
work: initialize nr_nodes in init_work_queue()

With this patch, collie can create an appropriate size of worker
threads for dynamic work queues at the start time.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
collie: add a function to wait for all the running works

This is in preparation to use work queue in collie.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: add operation to get object sha1 digest

Currently, we compare entire object contents to check whether there is
no difference among them.  However, this wastes a lot of network
bandwidth and some operations like vdi check or object recovery take
too long time to complete.

This adds an operation SD_OP_GET_HASH to get the sha1 digest of the
object, and enables us to do faster vdi check and recovery.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
collie: use hash for collie check

This patch checks the hash values of the target objects and skips
check and repair when there is no difference between them.  This also
uses work queue to boost the vdi check performance.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
tests: fix vdi check output

Now 'collie vdi check' fixes vdi in parallel, so we have to sort the
result of vdi check to get a unique output.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: recover from local when target node has same content

This checks the hash value of the target object before reading the
entire data, and recovers from local if there is a same object in the
stale directory.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: make failed_nodes and delayed_nodes static variables

This also guards them with the main_thread macro because they are not
thread-safe.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep/group: allow sending SD_OP_GET_EPOCH to delayed nodes

We cannot check whether the node is a delayed one or not in
epoch_log_read_remote() because we cannot access delayed node list in
the worker thread.  It is not a problem to send SD_OP_GET_EPOCH to
delayed nodes, so let's simply allow it.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
[don't merge] tests: valgrind option for lower memory machine

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
[tmp] tests: don't use /dev/urandom for better performance

/dev/urandom is too slow.  We don't need good random values to fill vdis.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
collie: allow calling queue_work() again in done()

Currently, if we call queue_work() in the second event_loop(),
work_queue_wait() cannot wait for completion of the work.  This fixes
the problem.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>