ssa.c: Log rclose error

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
ssa.c: Reconnect in the case of an rsocket error

This feature is part of SSA resiliency. It addresses non-core node
resiliency. The feature allows rsocket connection to be reestablished
with an upstream node.

The node recognizes connection failure using rsocket keepalives.
If node reaches the max. number of reconnection attempts, it gives up
and tries rejoin to the distribution tree.

Reconnection policy: This patch implements a constant reconnection policy.
A node reconnects to upstream node with a fixed/constant timeout
in between each reconnection attempt and limits of max. number of
attempts before giving up.

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
[acm, distrib]: Add reconnect_max_count and reconnect_timeout configuration parameters

reconnect_max_count specifies max. number of reconnections to the
upstream node. If the number is reached, the node rejoins to
distribution tree and asks for a new parent.
reconnect_max_count < 0, reconnection is disabled.
reconnect_max_count = 0, rejoin without reconnection.
Default value is 10.

reconnect_timeout specifies timeout (in sec.) between reconnections.
reconnect_timeout < 0, reconnection is disabled.
Default value is 10.

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
ssa.c: Add random shift in reconnection process

The random shift helps to avoid massive nodes reconnection at
the same time.

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
ssa.c: Stop reconnection if port is not ACTIVE
ssa.c: Set invalid value into timer's file descriptor after close.

Pointed-out-by: Hal Rosenstock <hal@mellanox.com>
ssa.c: Clear event and revent on timer's file descriptor, if
setimerfd_settime failed

Pointed-out-by: Hal Rosenstock <hal@mellanox.com>
ssa.c:Go to next descriptor rather than next iteration in timer event
processing.

Pointed-out-by: Hal Rosenstock <hal@mellanox.com>
ssa.c: Stick to existing code style.

Pointed-out-by: Hal Rosenstock <hal@mellanox.com>