Let ssh figure out port/user if not specified so we don't override .ssh/config