If no port is specified, try to guess it from wildcards