rpc: Ignore and log errors during cancel

Cancelling the RPC acceptors can sometimes result in an error about
a bad file descriptor.

As this is the shutdown sequence we need to continue nevertheless,
ignore these errors, log a warning and proceed.

Fixes #4352.
rpc: Add acceptors only when listening succeeded

Rebased-From: 6afa493
Rebased-By: Wladimir J. van der Laan <laanwj@gmail.com>