Gracefully exit before cleanup and reconnect after some delay