logger: rotate log by size, default is 500MB

a simple patch to split log:  when log size exceed 500MB, a new
sheep.log file will be created, and old log will be renamed with
specified suffix(actually, it's the time when split happened).

the main purpose of this patch is to prevent log became too big,
which is hard to open and read.

Signed-off-by: Andy Chen <sanjie.cyg@taobao.com>
Signed-off-by: Yibin Shen <zituan@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: process gateway request only in fast path

For request with flag SD_FLAG_CMD_IO_LOCAL, we handle
it in normal path.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
sheep: split cpg_queue into two

We need to split cpg_queue because:

1) IO requests shouldn't block node change events because succeeding
   event will replace the previous one to mitigate the overhead of
   unnecessary IOs from recovery.
2) IO requests may not block some notify events such as SHUTDOWN because
   some nodes are shutdown-ed maybe mistaken as nodes to be left by the
   blocked nodes.

This patch splits the queue with the following characteristics
 - local requests and IO requests are placed on request queue
 - confchg and notify event are placed on event queue

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
remove coroutine code

It was overkill to use coroutine for group_handler race conditions.

Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: refactor process_request_queue()

- move away request checking logic from request handling
  process. This will improve the readibility a bit.

- add a new field 'local_cow_oid' in struct request to avoid
  redundant calls of is_access_local()

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: remove unnecessary guard check

- setup_access_to_local_objects() does the job to find
  the if the targeted object is local, so we don't need
  to check it again.

- SD_FLAG_CMD_RECOVERY and SD_OP_READ_OBJ should be paired
  by programmers, since it is just internal API, we should
  trust 'em.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: use io workers to run local request

There is only one cpg worker, so it should be dedicated to
event queue.

So we have the following worker strategy:

1 cpg worker for event queue
  - node & notify event
4 IO workers for request queue
  - local requests & sheep p2p IO requests
4 gateway workers for request queue
  - VM guest requests

- factor out consistency check code

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: remove cpg* leftover

Now sheep support more than just corosync cluster driver, so
we'd better remove cpg* naming from sheep core code.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
object cache: make sync flush as default

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: make gateway and io workers configurable

Most of the time one VM would issue multiple requests in one
go, so it would be useful to let users to decide how many workers
are useful if we have more than several VMs in single sheep node.

In a large set nodes of cluster, every single node will get multiple
concurrent recovery IO requests, so it would be useful to have io_workers
as configurable too.

default 4 workers for both and maximum UINT32_MAX.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
trace: add GNU liscense to the source files

It seems that every .c file has this disclaimer inside, so follow
the convention.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
collie: make some optimization for parse_objs()

Currently, parse_objs traverse all the nodes to get the object,
and then call the callback function, it's OK for do_print_obj(),
but it's a waste for get_data_oid(), when the cluster gets larger
with more nodes, it makes collie slower the get the object id.

I add a return value for the callback function obj_parse_func_t,
if the function returns 1, the loop breaks when the first time
it succeeds, if 0, it will traverse the whole node list.

Signed-off-by: levin li <xingke.lwp@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
collie: show nodes at all epoches for collie cluster info

When we check the cluster status using 'collie cluster info',
it can only output the nodes at the latest eight epoch at most,
It's better to output as many as possible, as it's good for us
to check the cluster status.

Signed-off-by: levin li <xingke.lwp@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
collie: add an operation to track an object's locations at each epoch

When I was debuging the recovery issue(commit: 470533b), I found it
conveninet to have such a tool to track an object's trace at each epoch
to check whether an object is placed at the right place, so I add this
operation for collie at such case.

usage:
$collie/collie vdi track debian -i 3

output:

obj b1f0b000000003 locations at epoch 1, copies = 3
---------------------------------------------------
127.0.0.1:7004
127.0.0.1:7000
127.0.0.1:7001

obj b1f0b000000003 locations at epoch 2, copies = 3
---------------------------------------------------
127.0.0.1:7007
127.0.0.1:7004
127.0.0.1:7000

obj b1f0b000000003 locations at epoch 3, copies = 3
---------------------------------------------------
127.0.0.1:7007
127.0.0.1:7008
127.0.0.1:7004

Signed-off-by: levin li <xingke.lwp@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
object cache: incorrect lock may lead to update lost

Currently, we use a dirty_rb tree to record which cache object
have been updated.

Two kinds of threads will operate this dirty tree concurrently:
 a. mulitple io-workers: write something to cache objects
 b. one flush-worker: flush all updates to cluster

In the following scenario, update will be lost:

flush-worker                                  one io-worker
[object_cache_push]                         [object_cache_rw]
 |-(1) get a cache object from dirty tree                 |
 |-(2) read the data file of this object                  |
 |                    modify data file of this object (3)-|
 |-(4) forward_write_obj_req()                            |
 |                      add this object to dirty tree (5)-|
 |-(6) rb_erase: remove this object from dirty tree       |

Note: io-worker generate *new update* in step (3), but flush-worker
remove this cache object from dirty tree in step (6).

I use two dirty trees to fix this bug and avoid heavy lock between
flush-worker and io-wroker threads.

There is only one *active* dirty tree for io-workers in any time.
After io-worker modify something, it operate this active dirty tree.

When flush-worker want to flush all updates to cluster, it:
 1. mark another tree as *active* dirty tree.
 2. get update info from *inactive* dirty tree.
 3. if something wrong occur in flushing process,
    merge two dirty trees into active drity tree.

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: fix read_copy_from_cluster()

We should pass nr_vnodes to the obj_to_sheep()

- rename read_copy_from_replica to get better consistent naming

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>