Refactor zookeeper driver

This patch tries to refactor zookeeper driver.

It have changed so much code that it's difficult to
explain how it work in detail.

There were several problems in the old zookeeper driver:
 1. Each zookeeper message contains whole member list,
    it make network become heavy and slow.
 2. Before sending each message, get_nodes function was call.
    We used this function to get the number of members by comparing
    each znode in /sheepdog/queue. This function will become
    more and more slower as znodes increasing in /sheepdog/queue.
 3. The processing method of LEAVE EVENT is too lengthy. When a sheep
    droped from the cluster, zookeeper will remove the corresponding
    znodes in /sheepdog/member automatically, and each sheep will be
    notified by the registered watcher fcuntion. Normally, sheep should
    process this leave event when it got it, but it didn't, it created
    a new leave event message and send it to /sheepdog/queue, untill all
    sheep received this new message again.
 4. There are so many unnecessary zookeeper locks. These locks greatly
    reduce zookeeper driver's performance.

This patch tries to fix all problems memtioned above:
 1. Reduce message size, remove member list from message content
    and save it in two places:
    a) In /sheepdog/member. When a sheep joins into cluster, it put it's
       information into /sheepdog/member/<node_to_str(myself)>, and get
       other sheep's information from there.
    b) In eache sheep's private memory. Each sheep will maintain an array
       to contain the newest member list so that it can access this content
       as much as quickly, and needn't to call get_nodes function when it send
       a new message.
 2. Optimize the processing method of LEAVE EVENT. Now, sheep will process
    leave event once it received it.
 3. Remove unncecessary zookeeper locks.

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Optimize the size of buffer send to zookeeper

We send the whole buf[MAX_EVENT_BUF_SIZE] to zookeeper regardless the
real content is so small.

Now, we only send the real content in the buffer, not waste anymore.

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
retry again when zoo_* api return ZCONNECTIONLOSS/ZOPERATIONTIMEOUT error

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Fix two bug:

1. Change pthread lock in zk_dispatch/zk_block so that
it will not block IO request between sheep.
2. If one node pointed to by queue_pos was send by leaver,
and it have blocked whole cluster, we should ignore it.

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Use atomic builtins to replace pthread mutex locks

There were two pthread mutex locks in zookeeper dirver:
1. pthread mutex lock in zk_dispatch/zk_lock, this lock was used to
prevent zk_lock poping data before zk_dispatch have pushed it.
2. pthread mutex lock in zk_queue_pop/add_event, this lock was used to
protected leave event list.

This patch use GNU atomic builtins(__sync_xxx) to replace these two
pthread mutex locks. It makes zookeeper driver more lighter and
and faster.

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Rewatch znode in /sheepdog/member after it changed

If we not rewatch znode in /sheepdog/member after it changed, we
will lose all events of it in the future, such as DELETE event.

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
If previous zookeeper session exists, shutdown sheep

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Add code to handle sequence number overflow

The range of sequence number in zookeeper is: [-2^31 ~ (2^31-1)],
We store it in queue_pos variable, and use queue_pos++ to refer
next znode in /sheepdog/queue.

When sheep creates the first znode in /sheepdog/queue, zookeeper
initials the sequence number with 0. Each time we create a new
znode, it's value plus 1. So it will overflow and become a negative
in the future. In fact, it's value just like a circle.

This patch try to handle this problem.

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Fix bug: zk_leave doesn't work

Now we just need to delete znode in /sheepdog/member,
and zookeeper will notify LEAVE event to all other
sheeps automatically.

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Fix bug: leave event lost in zookeeper driver

If one sheep joined and left too quickly,
it may lead to leave event lost.

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>
Remove zk_lock from zk_join

Use ip address and port number to sort member list instead of sequence
number from /sheepdog/queue.

Benefit from these changes, the order of member list no longer depends
on the order of joining, then need not to take following steps into
one transaction:
- get sequence number from /sheepdog/queue
- create znode in /sheepdog/member/
- send join message to cluster

As a result, we can remove lock from zk_join, and move the creation of
znode in /sheepdog/member/ into zk_dispatch.

I use binary tree to store member list so that it can make the sort more
quickly. Methods with node_btree_xxx prefix are used to operate this
binary tree.

==*Note*==:
When sheep startups, it fetchs member list by reading /sheepdog/member/*,
if the result is empty, sheep will consider itself as *master*.

Now we have removed lock from zk_join，if we start multiple sheeps
simultaneously, one problem arises: there may exist more than one *master*,
this is bad.

To prevent this problem, we can start multiple sheeps like this:
- start the fist sheep alone, and sleep 2 seconds:
  $ sheep -d /store/0 -z 0 -p 7000 -c zookeeper:localhost:2181
  $ sleep 2

- start other sheeps simultaneously(need not to sleep between them):
  $ for i in {1..100}; do sheep -d /store/$i -z $i -p $((7000 + $i)) \
   -c zookeeper:localhost:2181

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>