don't add the colon if no port exists