zookeeper: fix cluster hang by giving priority to process LEAVE event

As cluster request may retry infinitely when some sheeps left, than
cluster_op_done could not to be called forever, so it will cause cluster
hang problem.

By giving priority to process LEAVE event when there is unfinished BLOCK
event, we can fix this issue, but also comply with the rule which is very
important for distributed system I think:

All sheeps should process all events in the same order.

Signed-off-by: Yunkai Zhang <qiushu.zyk@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
net: remove request throttling

Request throttling is introduced to tackle OOM prolems when requests were very
easily accumulated because of very limited worker threads. Now with short
threads construct, we will process any request ASAP and requests won't occupy
as many memory as before in a short period. So it's time to remove this
throttling because they are never reached in reality.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
net: clean up clear_client()

- rename it to clear_client_info()
- move dprintf in it instead of call dprintf every time after clear_client()

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
net: clean up clear_client_info() usage

We should call clear_client_info() as soon as we find the connection is dead.

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
net: clean up client_tx_handler()

- add some comments

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
net: refactor client_rx_handler()

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
net: refactor client_tx_handler()

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: rename sdnet.c to request.c

for sheep itself, sd as a file name means nothing. We can have a more consistent
file naming with request.c

Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
change nohalt flag into a switch to control which mode to run

This removes the --nohalt switch, and adds a mode switch

	--mode safe|quorum|unsafe

safe mode: halt cluster when nr_nodes < nr_copies (default)
quorum mode: halt cluster when nr_nodes < nr_copies / 2 + 1
unsafe mode: never halt the cluster.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
add early test for xattr support

We want to detect that early and prevent startup. Else the daemon starts and
simply crash with panic() later.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: don't call farm_init() for gateway-only node

Gateway-only node doesn't need backend at all, and we can restart it as gateway
node (has backend) again.
add mode option to man page collie.8

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
fix collie.8 man page

   - replace -H option with -m mode option

Signed-off-by: Jens Weber <jweber@tek2b.org>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
use have_enough_zones() instead of inline expresion

To make it work with new quorum mode.

Signed-off-by: Dietmar Maurer <dietmar@proxmox.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: notify VDI deletion to all nodes when deleting a VDI

After deleting a VDI, we should notify the VDI deletion event
to all the other nodes to make them clear up the object list
cache.

Signed-off-by: levin li <xingke.lwp@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
object list cache: put all the cache entry into a list

Compared to rb-tree, putting the entry into a list makes
it easy to traverse and reclaim.

Signed-off-by: levin li <xingke.lwp@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
object list cache: reclaim object list cache when receiving a deletion event.

Before reclaiming the cache belonging to the VDI just deleted, we should test
whether the VDI is exist, because after some node delete it and before the
notification is sent to all the node, another node may issus a VDI creation
event and reused the VDI id again, in which case we should reclaim the cached
entry.

Signed-off-by: levin li <xingke.lwp@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
farm: comment why we can't remove objlist entry while deleting object

Signed-off-by: levin li <xingke.lwp@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: add name for SD_OP_NOTIFY_VDI_DEL

Signed-off-by: levin li <xingke.lwp@taobao.com>
Signed-off-by: Liu Yuan <tailai.ly@taobao.com>
sheep: refactor sd_check_join_cb

By handling the trivial master selfelection separately we can pass down the
join_message structure to the various check helpers for the main path and
remove the get_cluster_status helper.

Signed-off-by: Christoph Hellwig <hch@lst.de>
Signed-off-by: MORITA Kazutaka <morita.kazutaka@lab.ntt.co.jp>