Add a setting to configure rejoin on reconnect behaviour

This leaks `chans` and `keys`