sheep: fix a build error

Reported-by: Marcin Mirosław <marcin@mejor.pl>
Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
optimize epoch_log structure to reduce network and memory

Current epoch_log contains a long nodes array to sync nodes and
epoch in the cluster. It is simple, but there is a potential
performance issue because each epoch log occupies nearly 500
KBytes. If the cluster members change frequently, epoch is lifted
frequently, more and more unused data is transfered between
client and server. If we don't find a way, the performance will
go from bad to worse.

Although the max node number is 6144, we only use a few of them.
Therefore, the first solution is using a zero-length array,
client (dog) and server (sheep) will negotiate an appropriate
supported node number. This way will spend much less memory and
will run much faster than before.

Signed-off-by: Ruoyu <liangry@ucweb.com>
Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
sheepdog proto: a new field type in vdi request for locking

The iSCSI multipath feature is implemented as an expansion of the vdi
locking. This patch introduces a type of locking. Existing ordinal
lock (used by QEMU) is typed as "normal", and new one for iSCSI
multipath is typed as "shared". This patch adds a new field type to
vdi request struct and let sheep know which type of lock is required
by client.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
sheep, dog: introduce a new state for expressing coherence of inode objects in clients

The new shared lock is different from existing normal lock. Under the
scheme of normal lock, vdi is simply locked or unlocked. But the new
shared lock requires additional states. Below is a brief description
of the states and its usage:

The main problem of enabling iSCSI multipath for sheepdog's VDI is
keeping coherence of tgtd's internal inode data structure. Assume tgt
A and B have logical units which point same sheepdog VDI. After
initialization of the logical units, A and B have copies of inode
object in their address space (let's X denote this object). When an
initiator sends write request to A and the request causes COW, A
creates a new object and updates the inode object X (let's X' denote
the updated one).

After the update by A, A can die with hardware faults. In such a case,
the initiator detects a connection error and switch its target to B if
iSCSI multipath is configured correctly. But the problem can arise at
this point. Because B doesn't know the update caused by A and its
internal inode object is still X, not X'. B creates requests based on
X, so it can read obsolete data which is already hidden by X' or
update X' in wrong manner.

For solving the above problem, this patch implements a mechanism for
keeping inode objects consistent in clients' address spaces. The above
problem is very similar to cache coherence problem of shared memory
multicore processors, so the mechanism mocks a simple MSI
protocol. When VDI is locked with shared mode for iSCSI multipath,
sheep's internal data structure for representing locking has 3 states
(Modified, Shared, and Invalidated) per participating node (in the
above example, A and B).

The above states are updated via gateway read and write of inode
objects (read makes states of every node shared, write makes writer's
state modified and others invalidated). If its state is invalidated,
client is forced to reload inode like a case of snapshot. This
coherence protocol is implemented in the next patch.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
sheep: implement coherence protocol of inode objects in address spaces of clients

The MSI states introduced by the previous patch should be updated via
gateway read and write of inode objects (read makes states of every
node shared, write makes writer's state modified and others
invalidated). If its state is invalidated, client is forced to reload
inode like a case of snapshot (a new return code
SD_RES_INODE_INVALIDATED is used for this purpose). This patch
implements the coherence protocol.

The update of the MSI states are done by atomic broadcast provided by
cluster managers. Therefore the update is costly operation, but it
happens rarely so it wouldn't be problematic.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>
sheep: handle node leave event for updating inode object shared state

When a node leaves from sheepdog cluster, it should be removed from a
list of participants of shared VDI.

Signed-off-by: Hitoshi Mitake <mitake.hitoshi@lab.ntt.co.jp>