Repository: aio-libs/yarl

Issue:
Incorrect handling of empty host in absolute URL
### Describe the bug

Absolute URLs are permitted to have empty hosts in RFC 3986.

Relevant grammar rules:
```
   host          = IP-literal / IPv4address / reg-name
   reg-name      = *( unreserved / pct-encoded / sub-delims )
```

Thus, a URL like `a://:1` conforms to the standard.
However, yarl rejects this URL.

`urllib3`, CPython `urllib`, `rfc3986`, `furl`, and `hyperlink` all correctly handle this situation.

### To Reproduce

Try running the following snippet:
```python3
>>> yarl.URL("a://:1")
```

### Expected behavior

The parse should have succeeded, resulting in `URL('a://:1')`.

### Logs/tracebacks

```python-traceback
This is the output from the snippet:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/site-packages/yarl/_url.py", line 163, in __new__
    raise ValueError("Invalid URL: host is required for absolute urls")
ValueError: Invalid URL: host is required for absolute urls
```
```


### Python Version

```console
$ python --version
Python 3.10.9
```


### multidict Version

```console
$ python -m pip show multidict
Name: multidict
Version: 6.0.4
Summary: multidict implementation
Home-page: https://github.com/aio-libs/multidict
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache 2
Location: /home/bkallus/fuzzing/url_differential_fuzzing/url_fuzz_env/lib/python3.10/site-packages
Requires:
Required-by: yarl
```


### yarl Version

```console
$ python -m pip show yarl
Name: yarl
Version: 1.8.2
Summary: Yet another URL library
Home-page: https://github.com/aio-libs/yarl/
Author: Andrew Svetlov
Author-email: andrew.svetlov@gmail.com
License: Apache-2.0
Location: /home/bkallus/fuzzing/url_differential_fuzzing/url_fuzz_env/lib/python3.10/site-packages
Requires: idna, multidict
Required-by:
```


### OS

Arch Linux

### Additional context

_No response_

### Code of Conduct

- [X] I agree to follow the aio-libs Code of Conduct

Discussion:
codecov[bot]: ## [Codecov](https://app.codecov.io/gh/aio-libs/yarl/pull/1136?dropdown=coverage&src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) Report
All modified and coverable lines are covered by tests :white_check_mark:
> Project coverage is 95.33%. Comparing base [(`1bd0937`)](https://app.codecov.io/gh/aio-libs/yarl/commit/1bd09379a9bf7f82a9db49c9520e4c5d3f7b800e?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) to head [(`0e15d69`)](https://app.codecov.io/gh/aio-libs/yarl/commit/0e15d69e87a0dca66e0dcd4d83a9dbf59b89c8e1?dropdown=coverage&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs).
> Report is 1 commits behind head on master.

<details><summary>Additional details and impacted files</summary>


```diff
@@            Coverage Diff             @@
##           master    #1136      +/-   ##
==========================================
+ Coverage   95.20%   95.33%   +0.12%     
==========================================
  Files          30       30              
  Lines        4709     4733      +24     
  Branches      417      420       +3     
==========================================
+ Hits         4483     4512      +29     
+ Misses        200      195       -5     
  Partials       26       26              
```

| [Flag](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flags&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | Coverage Δ | |
|---|---|---|
| [CI-GHA](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `95.28% <100.00%> (+0.13%)` | :arrow_up: |
| [MyPy](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `40.89% <52.63%> (+0.11%)` | :arrow_up: |
| [OS-Linux](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.34% <100.00%> (-0.05%)` | :arrow_down: |
| [OS-Windows](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.49% <100.00%> (+<0.01%)` | :arrow_up: |
| [OS-macOS](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.10% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.10.11](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.00% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.10.14](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.23% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.11.9](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.23% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.12.5](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.23% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.13.0-rc.1](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.23% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.8.10](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `98.94% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.8.18](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.18% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.9.13](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `98.94% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-3.9.19](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.18% <100.00%> (+<0.01%)` | :arrow_up: |
| [Py-pypy7.3.11](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `?` | |
| [Py-pypy7.3.16](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `?` | |
| [Py-pypy7.3.17](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `?` | |
| [VM-macos-latest](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.10% <100.00%> (+<0.01%)` | :arrow_up: |
| [VM-ubuntu-latest](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.34% <100.00%> (-0.05%)` | :arrow_down: |
| [VM-windows-latest](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.49% <100.00%> (+<0.01%)` | :arrow_up: |
| [pytest](https://app.codecov.io/gh/aio-libs/yarl/pull/1136/flags?src=pr&el=flag&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs) | `99.34% <100.00%> (-0.05%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs#carryforward-flags-in-the-pull-request-comment) to find out more.

</details>

[:umbrella: View full report in Codecov by Sentry](https://app.codecov.io/gh/aio-libs/yarl/pull/1136?dropdown=coverage&src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs).   
:loudspeaker: Have feedback on the report? [Share it here](https://about.codecov.io/codecov-pr-comment-feedback/?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=aio-libs).


Patch:
diff --git a/CHANGES/1136.bugfix.rst b/CHANGES/1136.bugfix.rst
new file mode 100644
index 0000000..b7cb174
--- /dev/null
+++ b/CHANGES/1136.bugfix.rst
@@ -0,0 +1,1 @@
+Allow empty host for URL schemes other than the special schemes listed in the WHATWG URL spec -- by :user:`bdraco`.
diff --git a/CHANGES/1137.misc.rst b/CHANGES/1137.misc.rst
new file mode 100644
index 0000000..a5abe16
--- /dev/null
+++ b/CHANGES/1137.misc.rst
@@ -0,0 +1,1 @@
+Improved performance of normalizing paths -- by :user:`bdraco`.
diff --git a/yarl/_url.py b/yarl/_url.py
index b735622..22f41da 100644
--- a/yarl/_url.py
+++ b/yarl/_url.py
@@ -34,10 +34,14 @@ from multidict import MultiDict, MultiDictProxy, istr
 from ._helpers import cached_property
 from ._quoting import _Quoter, _Unquoter
 
-DEFAULT_PORTS = {"http": 80, "https": 443, "ws": 80, "wss": 443}
+DEFAULT_PORTS = {"http": 80, "https": 443, "ws": 80, "wss": 443, "ftp": 21}
 USES_AUTHORITY = frozenset(uses_netloc)
 USES_RELATIVE = frozenset(uses_relative)
 
+# Special schemes https://url.spec.whatwg.org/#special-scheme
+# are not allowed to have an empty host https://url.spec.whatwg.org/#url-representation
+SCHEME_REQUIRES_HOST = frozenset(("http", "https", "ws", "wss", "ftp"))
+
 sentinel = object()
 
 SimpleQuery = Union[str, int, float]
@@ -255,7 +259,14 @@ class URL:
             else:
                 username, password, host, port = cls._split_netloc(val[1])
                 if host is None:
-                    raise ValueError("Invalid URL: host is required for absolute urls")
+                    if scheme in SCHEME_REQUIRES_HOST:
+                        msg = (
+                            "Invalid URL: host is required for "
+                            f"absolute urls with the {scheme} scheme"
+                        )
+                        raise ValueError(msg)
+                    else:
+                        host = ""
                 host = cls._encode_host(host)
                 raw_user = None if username is None else cls._REQUOTER(username)
                 raw_password = None if password is None else cls._REQUOTER(password)
@@ -899,9 +910,12 @@ class URL:
     @classmethod
     def _normalize_path(cls, path: str) -> str:
         # Drop '.' and '..' from str path
+        if "." not in path:
+            # No need to normalize if there are no '.' or '..' segments
+            return path
 
         prefix = ""
-        if path.startswith("/"):
+        if path and path[0] == "/":
             # preserve the "/" root element of absolute paths, copying it to the
             # normalised output as per sections 5.2.4 and 6.2.2.3 of rfc3986.
             prefix = "/"


Test patch:
diff --git a/tests/test_normalize_path.py b/tests/test_normalize_path.py
index defc4d8..20d8981 100644
--- a/tests/test_normalize_path.py
+++ b/tests/test_normalize_path.py
@@ -8,6 +8,7 @@ PATHS = [
     ("/", "/"),
     ("//", "//"),
     ("///", "///"),
+    ("path", "path"),
     # Single-dot
     ("path/to", "path/to"),
     ("././path/to", "path/to"),
@@ -15,6 +16,7 @@ PATHS = [
     ("path/././to", "path/to"),
     ("path/to/.", "path/to/"),
     ("path/to/./.", "path/to/"),
+    ("/path/to/.", "/path/to/"),
     # Double-dots
     ("../path/to", "path/to"),
     ("path/../to", "to"),
diff --git a/tests/test_url.py b/tests/test_url.py
index 5f6e033..085cf78 100644
--- a/tests/test_url.py
+++ b/tests/test_url.py
@@ -993,6 +993,35 @@ def test_joinpath_path_starting_from_slash_is_forbidden():
         assert url.joinpath("/to/others")
 
 
+PATHS = [
+    # No dots
+    ("", ""),
+    ("path", "path"),
+    # Single-dot
+    ("path/to", "path/to"),
+    ("././path/to", "path/to"),
+    ("path/./to", "path/to"),
+    ("path/././to", "path/to"),
+    ("path/to/.", "path/to/"),
+    ("path/to/./.", "path/to/"),
+    # Double-dots
+    ("../path/to", "path/to"),
+    ("path/../to", "to"),
+    ("path/../../to", "to"),
+    # Non-ASCII characters
+    ("μονοπάτι/../../να/ᴜɴɪ/ᴄᴏᴅᴇ", "να/ᴜɴɪ/ᴄᴏᴅᴇ"),
+    ("μονοπάτι/../../να/𝕦𝕟𝕚/𝕔𝕠𝕕𝕖/.", "να/𝕦𝕟𝕚/𝕔𝕠𝕕𝕖/"),
+]
+
+
+@pytest.mark.parametrize("original,expected", PATHS)
+def test_join_path_normalized(original: str, expected: str) -> None:
+    """Test that joinpath normalizes paths."""
+    base_url = URL("http://example.com")
+    new_url = base_url.joinpath(original)
+    assert new_url.path == f"/{expected}"
+
+
 # with_path
 
 
diff --git a/tests/test_url_parsing.py b/tests/test_url_parsing.py
index 5ce419a..4fe9518 100644
--- a/tests/test_url_parsing.py
+++ b/tests/test_url_parsing.py
@@ -210,19 +210,14 @@ class TestPort:
         assert u.query_string == ""
         assert u.fragment == ""
 
-    @pytest.mark.xfail(
-        # FIXME: remove "no cover" pragmas upon xfail marker deletion
-        reason="https://github.com/aio-libs/yarl/issues/821",
-        raises=ValueError,
-    )
     def test_no_host(self):
-        u = URL("//:80")
-        assert u.scheme == ""  # pragma: no cover
-        assert u.host == ""  # pragma: no cover
-        assert u.port == 80  # pragma: no cover
-        assert u.path == "/"  # pragma: no cover
-        assert u.query_string == ""  # pragma: no cover
-        assert u.fragment == ""  # pragma: no cover
+        u = URL("//:77")
+        assert u.scheme == ""
+        assert u.host == ""
+        assert u.port == 77
+        assert u.path == "/"
+        assert u.query_string == ""
+        assert u.fragment == ""
 
     def test_double_port(self):
         with pytest.raises(ValueError):
@@ -457,9 +452,19 @@ class TestFragment:
 
 
 class TestStripEmptyParts:
-    def test_all_empty(self):
+    def test_all_empty_http(self):
         with pytest.raises(ValueError):
-            URL("//@:?#")
+            URL("http://@:?#")
+
+    def test_all_empty(self):
+        u = URL("//@:?#")
+        assert u.scheme == ""
+        assert u.user is None
+        assert u.password is None
+        assert u.host == ""
+        assert u.path == ""
+        assert u.query_string == ""
+        assert u.fragment == ""
 
     def test_path_only(self):
         u = URL("///path")
@@ -580,3 +585,22 @@ class TestStripEmptyParts:
         assert u.path == ""
         assert u.query_string == ""
         assert u.fragment == ""
+
+
+@pytest.mark.parametrize(
+    ("scheme"),
+    [
+        ("http"),
+        ("https"),
+        ("ws"),
+        ("wss"),
+        ("ftp"),
+    ],
+)
+def test_schemes_that_require_host(scheme: str) -> None:
+    """Verify that schemes that require a host raise with empty host."""
+    expect = (
+        "Invalid URL: host is required for " f"absolute urls with the {scheme} scheme"
+    )
+    with pytest.raises(ValueError, match=expect):
+        URL(f"{scheme}://:1")