Repository: pantsbuild/pants

Issue:
Document/fix quirks for file:// URLs for tool downloads
**Describe the bug**

`file://` URLs are supported, and mentioned in: https://github.com/pantsbuild/pants/blob/ed272f57be30912234f805cfd156df937eb1a6d7/src/python/pants/core/util_rules/external_tool.py#L279-L293
But they have some quirks that we should document or fix:
1. They currently require an ignored token for the hostname.
2. They are absolute, so will need to be used with `%(buildroot)s` templating in most cases.

Together, this might mean that to resolve a file from a relative position within the repository, you need something like:
```
[download-pex-bin]
# The `ignored` token is parsed as the hostname and discarded.
url_template = "file://ignored/%(buildroot)s/pants_files/pex"
```

**Pants version**

`2.8.0.dev2`


Discussion:
The implementation is here: https://github.com/pantsbuild/pants/blob/ed272f57be30912234f805cfd156df937eb1a6d7/src/rust/engine/src/nodes.rs#L954-L956 ... arguably the required hostname is enough of a bug that it would be worth breaking backwards compatibility to use the entire path after the scheme.

Resolving patch:
diff --git a/src/python/pants/core/util_rules/external_tool.py b/src/python/pants/core/util_rules/external_tool.py
--- a/src/python/pants/core/util_rules/external_tool.py
+++ b/src/python/pants/core/util_rules/external_tool.py
@@ -18,6 +18,7 @@
 from pants.engine.platform import Platform
 from pants.engine.rules import Get, collect_rules, rule
 from pants.option.subsystem import Subsystem
+from pants.util.docutil import doc_url
 from pants.util.logging import LogLevel
 from pants.util.meta import classproperty
 
@@ -294,8 +295,10 @@ def register_options(cls, register):
             help=(
                 "URL to download the tool, either as a single binary file or a compressed file "
                 "(e.g. zip file). You can change this to point to your own hosted file, e.g. to "
-                "work with proxies or for access via the filesystem through a file:// URL.\n\nUse "
-                "`{version}` to have the value from --version substituted, and `{platform}` to "
+                "work with proxies or for access via the filesystem through a `file:$abspath` URL (e.g. "
+                "`file:/this/is/absolute`, possibly by [templating the buildroot in a "
+                f"config file]({doc_url('options#config-file-entries')})).\n\n"
+                "Use `{version}` to have the value from --version substituted, and `{platform}` to "
                 "have a value from --url-platform-mapping substituted in, depending on the "
                 "current platform. For example, "
                 "https://github.com/.../protoc-{version}-{platform}.zip."


Test patch:
diff --git a/src/python/pants/engine/fs_test.py b/src/python/pants/engine/fs_test.py
--- a/src/python/pants/engine/fs_test.py
+++ b/src/python/pants/engine/fs_test.py
@@ -872,7 +872,7 @@ def test_download_file(downloads_rule_runner: RuleRunner) -> None:
         roland.write_text("European Burmese")
         snapshot = downloads_rule_runner.request(
             Snapshot,
-            [DownloadFile(f"file://ignored.see.13054/{roland}", ROLAND_FILE_DIGEST)],
+            [DownloadFile(f"file:{roland}", ROLAND_FILE_DIGEST)],
         )
 
     assert snapshot.files == ("roland",)
@@ -889,7 +889,7 @@ def test_download_caches(downloads_rule_runner: RuleRunner) -> None:
         roland.write_text("European Burmese")
         snapshot = downloads_rule_runner.request(
             Snapshot,
-            [DownloadFile(f"file://ignored.see.13054/{roland}", ROLAND_FILE_DIGEST)],
+            [DownloadFile(f"file:{roland}", ROLAND_FILE_DIGEST)],
         )
 
     assert snapshot.files == ("roland",)