Handle refused connection more immediately