ssa.c: Use -1 rather than 0 as socket id for "fake" ACMs

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
core.c: Use DB_EPOCH_INVALID rather than 0 in core_extract_db

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
[acm acme].c: Add keepalive configuration parameter to ACM

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
ssa.c: Log rclose error

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
ssa.c: Reconnect in the case of an rsocket error

This feature is part of SSA resiliency. It addresses non-core node
resiliency. The feature allows rsocket connection to be reestablished
with an upstream node.

The node recognizes connection failure using rsocket keepalives.
If node reaches the max. number of reconnection attempts, it gives up
and tries rejoin to the distribution tree.

Reconnection policy: This patch implements a constant reconnection policy.
A node reconnects to upstream node with a fixed/constant timeout
in between each reconnection attempt and limits of max. number of
attempts before giving up.

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
[acm, distrib]: Add reconnect_max_count and reconnect_timeout configuration parameters

reconnect_max_count specifies max. number of reconnections to the
upstream node. If the number is reached, the node rejoins to
distribution tree and asks for a new parent.
reconnect_max_count < 0, reconnection is disabled.
reconnect_max_count = 0, rejoin without reconnection.
Default value is 10.

reconnect_timeout specifies timeout (in sec.) between reconnections.
reconnect_timeout < 0, reconnection is disabled.
Default value is 10.

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
ssa.c: Add random shift in reconnection process

The random shift helps to avoid massive nodes reconnection at
the same time.

Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Hal Rosenstock <hal@mellanox.com>
ssa_path_record.c: Use destroy and build rather than rebuild in ssa_pr_reinit_context

 ssa_pr_rebuild_indexes compares index's epoch and SMDB's epoch.
 In case of SM failover/handover new SMDB could have the same
 epoch like last pushed SMDB from previous core.
core.c: Reset ssa_db_diff dirty flag if SMDB was not changed

Signed-off-by: Ilya Nelkenbaum <ilyan@mellanox.com>
ssa.c: Fix handling port error in downstream thread

In case of port error device event, in addition to closing
smdb, prdb listen sockets and all downstream open connections,
proper file descriptors should be cleaned, so that downstream
thread rpoll will not listen to them.

Signed-off-by: Ilya Nelkenbaum <ilyan@mellanox.com>
core.c: Raise 'first_extraction' flag in case of SM state change

Signed-off-by: Ilya Nelkenbaum <ilyan@mellanox.com>
ssa.c: Eliminate closing listen rsockets

If SMDB / PRDB listen rsockets are closed on port error
event, in case of port bouncing or handover back, later
on those sockets won't be opened properly (due to some
librdma library limitation) so no distribution node will
be available to connect to it.

Signed-off-by: Ilya Nelkenbaum <ilyan@mellanox.com>