sanlock: shutdown reply

A wait option (-w 1) can be added to sanlock shutdown.
The shutdown command will wait for a result from the
daemon indicating success (exiting), or failure (not
exiting because lockspaces exist).

Signed-off-by: David Teigland <teigland@redhat.com>
sanlock: fix get_hosts off by one

When a host_id is specified, the information for
host_id+1 is returned.

Signed-off-by: David Teigland <teigland@redhat.com>
sanlock: lockspace host events

Hosts can set/get events between each other using a common lockspace.
. set_event takes two opaque 64 bit values ("event" and "data"), and
  a target host_id with optional host generation.
. reg_event creates an fd that a program can use in poll(2), and
. get_event on this fd returns the event values that have been set
  along with the source host_id/generation.
. end_event releases the registered fd.

Each set_event call replaces generation/event/data values from the
previous set_event call, but host_id's from each are maintained.

set_event can be used to send the same event/data values to multiple
hosts at once using multiple set_event calls (generation is not
applicable in this case.)

set_event with different values within the same window of time results
in -EBUSY (unless the REPLACE flag is used).  Otherwise, the second
set_event may result in the earlier target host(s) seeing later values
not intended for them, and missing the earlier values from the first
set_event.

Event values are passed through fields in the lockspace delta leases.
Event notification is done via the host_id bitmaps in lockspace delta
leases.  Setting an event (or making a resource request) for a target host
results in the target host's bit being set in the sending host's bitmap.
The bit remains set for set_bitmap_seconds (60 sec with 10 sec io
timeout.) After this, the bit is cleared and the target host will no
longer be notified.  The target host will typically generate three events
for each set_event, once from each delta lease renewal (every 20 seconds)
over the course of the set_bitmap_seconds.

Signed-off-by: David Teigland <teigland@redhat.com>
sanlock: set_config api

Add an interface to configure aspects of a lockspace.
Include two initial config options:
USED: the lockspace is used.  This flag has the same
effect as a process holding a resource lease that
will not exit.  UNUSED: clears the USED flag.
reset: daemon and program for resetting hosts

The sanlk-resetd daemon gets events from specified lockspaces.
Events are defined to cause the sanlk-resetd to:
- use wdmd/watchdog to reset the host (event 1)
- use /proc/sysrq-trigger to reboot the host (event 4)

The sanlk-resetd daemon can be used alone, and events passed
to it from another host by:
sanlock client set_event -s lockspace_name -i host_id -e 1

The sanlk-reset program can also be used to communicate with
sanlk-resetd on another host.  It will set the event(s), then
monitor the host status until the target host is dead.

Signed-off-by: David Teigland <teigland@redhat.com>
sanlock: disk endian handling

Signed-off-by: David Teigland <teigland@redhat.com>
sanlock: persistent and orphan resource leases

If a resource lease has the PERSISTENT flag set when it is
acquired, then sanlock will not release the lease if the
process holding the lease exits.  Instead, the lease is
moved to the orphans list.  From the perspective of other
hosts, and on disk, there is no difference between a held
lease and an orphan lease.  On the origin host, another
process can acquire an orphan lease by using the
ACQUIRE_ORPHAN flag, or release an orphan lease using
the REL_ORPHAN flag.  All orphan leases in a lockspace can
be released using REL_ORPHAN and setting the lockspace name
and an empty resource name.

If the lockspace config flag SANLK_CONFIG_USED_BY_ORPHANS
is set, then an orphan resource in a lockspace will have
the same effect on the lockspace as the USED flag or clients
holding resources that have not exited.  Other things on the
host may still be using resources that are protected by the
orphaned resource lease, so it is not safe to release the
lockspace, which would allow another host to acquire the
orphan lease.

If SANLK_CONFIG_USED_BY_ORPHANS is not set, then orphans
will have no effect on the removal of a lockspace.  When
the lockspace is removed, the orphans will just be left.

Signed-off-by: David Teigland <teigland@redhat.com>
fence_sanlock: use persistent and orphan features

The resource lock needs to be persistent so that it
is not simply released by sanlock if the daemon exits.

The lockspace needs to not be removed as long as orphan
leases exist, otherwise the orphan could be acquired by
another host if the lockspace delta lease is released.

Signed-off-by: David Teigland <teigland@redhat.com>
reset: remove DEBUG

Signed-off-by: David Teigland <teigland@redhat.com>
init: add service file for sanlk-resetd

Signed-off-by: David Teigland <teigland@redhat.com>
init: misc changes

- use requires for sanlk-resetd because it's probably better
- remove syslog from sanlk-resetd because it seems systemd
  doesn't like that any more
- remove controlgroup from sanlock and wdmd services files
  because systemd doesn't seem to like that any more

Signed-off-by: David Teigland <teigland@redhat.com>
python: add support for lockspace events

Add support for sending events to other hosts and listening for
lockspace events.

I added thin wrappers for sanlock_reg_event, sanlock_get_event,
sanlock_end_event, and sanlock_set_event.

Where sanlock C API has unused arguments, the Python bindings do not use
them, since we can add them later as optional arguments in a backward
compatible manner. This ensures that new code depending on this argument
will fail when working with a version that does not provide it.

sanlock_get_event returns one event per call. The Python version is
easier to use, returning list of events in one call, similar to
get_hosts.

Signed-off-by: Nir Soffer <nsoffer@redhat.com>