Correctly handle failing to connect to the given session in client mode