sheep: support specifying the initial count of virtual nodes

With this patch, you can set the weight of each node with the -v
option.

Example:

  $ ./sheep/sheep /store/0 -p 7000 -z 0 -v 1000
  $ ./sheep/sheep /store/1 -p 7001 -z 1 -v 2000
  $ ./sheep/sheep /store/2 -p 7002 -z 2 -v 3000
  $ ./collie/collie cluster format -c 1
  using backend simple store
  $ ./collie/collie node list
  M   Id   Host:Port         V-Nodes       Zone
  -    0   10.68.14.1:7000        1000          0
  -    1   10.68.14.1:7001        2000          1
  -    2   10.68.14.1:7002        3000          2
  $ ./collie/collie vdi create image 600M -P
  $ ./collie/collie node info
  Id      Size    Used    Use%
   0      188 GB  100 MB    0%
   1      188 GB  200 MB    0%
   2      188 GB  304 MB    0%
  Total   564 GB  604 MB    0%

  Total virtual image size  600 MB

If you specify zero to the number of virtual nodes, no data will be
stored to the node.

Reviewed-by: Liu Yuan <tailai.ly@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
varies printf type typo

I found several printf related type errors in 32-bit linux environment.
Here's the trivial patch to fix them.

- fix a warning (Liu Yuan)

Signed-off-by: huxinwei <huxinwei@huawei.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: fix the memmove bug in del_cpg_node

The size is not right, maybe lead to error

Signed-off-by: HaiTing Yao <wujue.yht@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: Improve the verbose error message in log.

Yet another trivial patch.
I just copied over the comments for these results.
It helps me to figure out the situation more easiler.

Signed-off-by: huxinwei <huxinwei@huawei.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
farm: Fix to put_sha1_file

I found an panic in sheep today.

Mar 02 23:57:28 trunk_file_write(315) try delete stale snapshot object 8071d67a00000000...
Mar 02 23:57:28 put_sha1_file(92) No such file or directory

It happens when the sheep is killed after omap_file_init but before
omap_file_final, and some of the staled snapshot have been unlinked.
Then sheep will always panic every time afterward.

The patch is a proposal to make put_sha1_file tolerant not-exist-file.

Signed-off-by: huxinwei <huxinwei@huawei.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
farm: fix lookup_trunk_entry

It should return NULL when 1. not found and 2. create = 0.
Cleanup the log function

The log_sigsegv function in logger.c actually serves two purposes.
1, the logger itself exit on illegal memory access.
2, the sheep exit, which makes the kernel deliver a SIGSEGV to the logger process

I think it'll be a good idea to make distinguish between these 2.

Signed-off-by: Xinwei Hu <huxinwei@huawei.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: remove unnecessary code

This code is never visited and acted as a safe guard, but the
uppper layer code should take responsibility of initializing
hdr->copies to avoid this redundant check in the hot path.
simple store: fix preallocation object size

We should allocate different sizes for vdi and data object, instead
of fixed size for both.
farm: fix preallocation object size
sheep: use pointer to avoid extra copy

forward_write_obj_req() is kind of hot code path, so this kind of
extra copy is nasty.
sheep: drive object cache to work
sheep: add red black tree library

Red black tree is more scalable than hash lists if we
cannot predict the size of the data set.

This is excerpted from Linux kernel.
sheep: object cache proper

Object cache caches data and vdi objects on the local node. It is at 
higher level than backend store. This extra cache layer translate gateway 
requests into local requests, largely reducing the network traffic and highly
improve the IO performance.

Dirty objects will be flushed to cluster storage by 'sync' request from
guest OS.

- use red-black tree to track dirty objects
- use file lock to avoid RW race on object granularity
- use hash lists to maintain vdi space.
- each vid has its own independent object cache
sheep: teach sheep to use object cache

We only intrude IO code for gateway requests. Object IO path from
recovery logic is intact.
sheep: add flush_vdi operation

This is supposed to be initiated by Guest OS, but our collie
friend might also like it.

Flush operation is operated on vdi basis, that is, when one guest
flush its own dirty data, other guests are not affected.

- use forward_write_obj_req() to flush dirty objects
sheep: let object cache better interact with recovery logic

If the targeted object of the gateway request is cached already,
we can simply operate on it despite of node changes.
collie: flush the vdi before searching the object

'collie vdi object' need to read latest vdi object from backend
store to assist it to locate the object. Since the vdi object
is also cached in object cache, we need to flush it beforehand to
cluster.
sheep: add object list cache implemented by rb-tree

Added object list cache implemented by red-black tree, 
everytime we creates a new object, we write the oid of the
object into the object list cache.
sheep: object list cache applied to farm and simple store

Applied the object list cache to get_obj_list() which
walk through the rb-tree cache and return the cached list, 
instead of calling sd_store->get_objlist(),
make it much faster to get the object list.