Don't ignore ssh port preference

Thanks azizLIGHT for discovering this.