sheep: teach journal file to be md friendly

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
tests/047: add md test support

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: teach journal file to handle epoch and config write

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: use new journal API for config file

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: rename journal_file.c as journal.c

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: journaling epoch file write

Also bundle two writes into one.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: handle broken epoch file at get_latest_epoch

Without journaling enabled, we might have empty epoch file or
partial write file.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Reviewed-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
include: move node_to_str() and str_to_node() from sheep/cluster.h to include/sheep.h

shephered uses node_to_str() and str_to_node(), so they should be
moved to the header which can be included by other directories of
sheep/.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
include: move enum cluster_join_result from sheep/cluster.h to include/internal_proto.h

enum cluster_join_result is also used by shepherd. So this patch
moves it from sheep/ to include/.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
lib: sockaddr_in_to_str(), convert sockaddr_in to string representation

This patch adds new function sockaddr_in_to_str() to lib/net.c. This
is for converting variables typed struct sockaddr_in to string
representation.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
lib: add wrappers do_writev2() and writev2() for typical usage of writev()

This patch adds 2 new wrappers: do_write2v() and writev2() for
enshorting typical usage of writev().

Typical usage of writev() is sending an array of struct iovec with 2
elements, one for a header and another for a message body.
With do_writev2(), declaring the array in caller side can be removed.

writev2() is a wrapper for do_writev2(). Typical headers have fixed
length, so writev2() obtains their length with sizeof() (so writev2()
is defined as a macro function).

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
lib: add a mechanism for prioritizing events

I posted the patch which implements a prioritizing mechanism in
event_loop() of libsheepdog [1]. This patch implements a similar
but simpler one.

The previous patch changed type of event handlers for detecting
requirements of refresh, etc by their return values. The new one
doesn't require changing the types. If event handlers want event loop
to refresh, they only need to call a new API: event_force_refresh().

In addition, event_loop() doesn't cause sort of events based on their
priority. If programs which employ libsheepdog.a want to use the event
prioritize mechanism, they have to call event_loop_prio() instead of
event_loop() in their main event loops.

Because of these features, overhead of implementation and runtime
performance is minimized.

[1] http://lists.wpkg.org/pipermail/sheepdog/2012-December/007426.html

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
shepherd: a new cluster manager specialized for sheepdog

This patch adds shepherd, a new cluster manager for sheepdog. shepherd
is not a general cluster manager software. The design is heavily
depends on the internal knowledge of sheepdog. So the usage will be
limited but it can be optimized only for sheepdog.

Currently, shepherd cluster manager is designed as master-slave. So it
can be a SPOF. But I'm planning to let it be redundant by utilizing
corosync. If it can be improved, it can be an alternative of the
group management part of ZooKeeper, and it doesn't depend on JVM.

This patch adds the new directory shepherd/ for it, and the
executable binary will be produced as shepherd/shepherd.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: add a cluster driver for shepherd

This patch adds a cluster driver for shepherd. The source file is
sheep/cluster/shepherd.c.

Supported options are:
- hostname or IP address which shepherd uses
- port number which shepherd binds

example usage of the cluster driver:
$ sheep -c shepherd:127.0.0.1:2501 /store/sheep

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
collie: rename SUBCMD_FLAG_NEED_THIRD_ARG

THIRD is very misleading and this is a prepare patch.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
collie: introduce do_generic_subcommand()

This helper will check required argument if SUBCMD_FLAG_NEED_ARG is set and
call update_node_list if SUBCMD_FLAG_NEED_NODELIST is set.

Argument check is fatal to fix segfault by later patch.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
collie: fix segfault when required argument is not set

Currently, command like 'collie node md plug' without arugment will cause
segfault because the lack of the argument check.

Use do_generic_subcommand() helper to fix this problem.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>