Repository: bazelbuild/bazelisk (go)

Issue:
Support arbitrary URL layouts
I'm in a situation where I have to download custom Bazel builds, but the only file hosting service I have access to imposes a weird URL layout on me. As a result, I cannot use `BAZELISK_BASE_URL` due to its assumptions on how the repository is laid out.

Could I suggest adding a new `BAZELISK_URL_FMT` variable or similar to generalize the way URLs are computed? It could take something like this:

```BAZELISK_URL_FMT=https://server.example.com/my/weird/<ARCH>/layout/bazel-<RELEASE>-foo/weirder/<OS>```

This new variable should conflict with at least `BAZELISK_BASE_URL` and result in an invalid configuration file error, because the two are incompatible.

Happy to take a stab at implementing this if the idea sounds reasonable.

Pull request description:
Implement support for BAZELISK_FORMAT_URL
This new configuration setting provides a format-like string to compute the URL from which to fetch Bazel.  Takes precedence over BAZELISK_BASE_URL as this is a more general concept.

Fixes #423.

Patch:
diff --git a/README.md b/README.md
index 1d059b5..6ae96fd 100644
--- a/README.md
+++ b/README.md
@@ -77,11 +77,21 @@ By default Bazelisk retrieves Bazel releases, release candidates and binaries bu
 
 As mentioned in the previous section, the `<FORK>/<VERSION>` version format allows you to use your own Bazel fork hosted on GitHub:
 
-If you want to create a fork with your own releases, you have to follow the naming conventions that we use in `bazelbuild/bazel` for the binary file names.
+If you want to create a fork with your own releases, you should follow the naming conventions that we use in `bazelbuild/bazel` for the binary file names as this results in predictable URLs that are similar to the official ones.
 The URL format looks like `https://github.com/<FORK>/bazel/releases/download/<VERSION>/<FILENAME>`.
 
 You can also override the URL by setting the environment variable `$BAZELISK_BASE_URL`. Bazelisk will then append `/<VERSION>/<FILENAME>` to the base URL instead of using the official release server. Bazelisk will read file [`~/.netrc`](https://everything.curl.dev/usingcurl/netrc) for credentials for Basic authentication.
 
+If for any reason none of this works, you can also override the URL format altogether by setting the environment variable `$BAZELISK_FORMAT_URL`. This variable takes a format-like string with placeholders and performs the following replacements to compute the download URL:
+
+- `%e`: Extension suffix, such as the empty string or `.exe`.
+- `%h`: Value of `BAZELISK_VERIFY_SHA256`, respecting uppercase/lowercase characters.
+- `%m`: Machine architecture name, such as `arm64` or `x86_64`.
+- `%o`: Operating system name, such as `darwin` or `linux`.
+- `%v`: Bazel version as determined by Bazelisk.
+- `%%`: Literal `%` for escaping purposes.
+- All other characters after `%` are reserved for future use and result in a processing error.
+
 ## Ensuring that your developers use Bazelisk rather than Bazel
 
 Bazel installers typically provide Bazel's [shell wrapper script] as the `bazel` on the PATH.
@@ -100,7 +110,9 @@ require users update their bazel.
 [shell wrapper script]: https://github.com/bazelbuild/bazel/blob/master/scripts/packages/bazel.sh
 ## Other features
 
-The Go version of Bazelisk offers two new flags.
+The Go version of Bazelisk offers three new flags.
+
+### --strict
 
 `--strict` expands to the set of incompatible flags which may be enabled for the given version of Bazel.
 
@@ -108,17 +120,34 @@ The Go version of Bazelisk offers two new flags.
 bazelisk --strict build //...
 ```
 
+### --migrate
+
 `--migrate` will run Bazel multiple times to help you identify compatibility issues.
 If the code fails with `--strict`, the flag `--migrate` will run Bazel with each one of the flag separately, and print a report at the end.
 This will show you which flags can safely enabled, and which flags require a migration.
 
+
+### --bisect
+
+`--bisect` flag allows you to bisect Bazel versions to find which version introduced a build failure. You can specify the range of versions to bisect with `--bisect=<GOOD>..<BAD>`, where GOOD is the last known working Bazel version and BAD is the first known non-working Bazel version. Bazelisk uses [GitHub's compare API](https://docs.github.com/en/rest/commits/commits#compare-two-commits) to get the list of commits to bisect. When GOOD is not an ancestor of BAD, GOOD is reset to their merge base commit.
+
+```shell
+bazelisk --bisect=6.0.0..HEAD test //foo:bar_test
+```
+
+Note that, Bazelisk uses prebuilt Bazel binaries at commits on the main and release branches, therefore you cannot bisect your local commits.
+
+### Useful environment variables for --migrate and --bisect
+
 You can set `BAZELISK_INCOMPATIBLE_FLAGS` to set a list of incompatible flags (separated by `,`) to be tested, otherwise Bazelisk tests all flags starting with `--incompatible_`.
 
 You can set `BAZELISK_GITHUB_TOKEN` to set a GitHub access token to use for API requests to avoid rate limiting when on shared networks.
 
-You can set `BAZELISK_SHUTDOWN` to run `shutdown` between builds when migrating if you suspect this affects your results.
+You can set `BAZELISK_SHUTDOWN` to run `shutdown` between builds when migrating or bisecting if you suspect this affects your results.
+
+You can set `BAZELISK_CLEAN` to run `clean --expunge` between builds when migrating or bisecting if you suspect this affects your results.
 
-You can set `BAZELISK_CLEAN` to run `clean --expunge` between builds when migrating if you suspect this affects your results.
+## tools/bazel
 
 If `tools/bazel` exists in your workspace root and is executable, Bazelisk will run this file, instead of the Bazel version it downloaded.
 It will set the environment variable `BAZEL_REAL` to the path of the downloaded Bazel binary.
diff --git a/core/BUILD b/core/BUILD
index 2f5ec60..b941e3d 100644
--- a/core/BUILD
+++ b/core/BUILD
@@ -19,6 +19,9 @@ go_library(
 
 go_test(
     name = "go_default_test",
-    srcs = ["core_test.go"],
+    srcs = [
+        "core_test.go",
+        "repositories_test.go",
+    ],
     embed = [":go_default_library"],
 )
diff --git a/core/core.go b/core/core.go
index 139c17b..ef4a66f 100644
--- a/core/core.go
+++ b/core/core.go
@@ -6,10 +6,12 @@ package core
 import (
 	"bufio"
 	"crypto/sha256"
+	"encoding/json"
 	"fmt"
 	"io"
 	"io/ioutil"
 	"log"
+	"net/http"
 	"os"
 	"os/exec"
 	"os/signal"
@@ -89,24 +91,7 @@ func RunBazeliskWithArgsFunc(argsFunc ArgsFunc, repos *Repositories) (int, error
 	// If we aren't using a local Bazel binary, we'll have to parse the version string and
 	// download the version that the user wants.
 	if !filepath.IsAbs(bazelPath) {
-		bazelFork, bazelVersion, err := parseBazelForkAndVersion(bazelVersionString)
-		if err != nil {
-			return -1, fmt.Errorf("could not parse Bazel fork and version: %v", err)
-		}
-
-		var downloader DownloadFunc
-		resolvedBazelVersion, downloader, err = repos.ResolveVersion(bazeliskHome, bazelFork, bazelVersion)
-		if err != nil {
-			return -1, fmt.Errorf("could not resolve the version '%s' to an actual version number: %v", bazelVersion, err)
-		}
-
-		bazelForkOrURL := dirForURL(GetEnvOrConfig(BaseURLEnv))
-		if len(bazelForkOrURL) == 0 {
-			bazelForkOrURL = bazelFork
-		}
-
-		baseDirectory := filepath.Join(bazeliskHome, "downloads", bazelForkOrURL)
-		bazelPath, err = downloadBazelIfNecessary(resolvedBazelVersion, baseDirectory, repos, downloader)
+		bazelPath, err = downloadBazel(bazelVersionString, bazeliskHome, repos)
 		if err != nil {
 			return -1, fmt.Errorf("could not download Bazel: %v", err)
 		}
@@ -130,7 +115,7 @@ func RunBazeliskWithArgsFunc(argsFunc ArgsFunc, repos *Repositories) (int, error
 		return 0, nil
 	}
 
-	// --strict and --migrate must be the first argument.
+	// --strict and --migrate and --bisect must be the first argument.
 	if len(args) > 0 && (args[0] == "--strict" || args[0] == "--migrate") {
 		cmd, err := getBazelCommand(args)
 		if err != nil {
@@ -140,7 +125,6 @@ func RunBazeliskWithArgsFunc(argsFunc ArgsFunc, repos *Repositories) (int, error
 		if err != nil {
 			return -1, fmt.Errorf("could not get the list of incompatible flags: %v", err)
 		}
-
 		if args[0] == "--migrate" {
 			migrate(bazelPath, args[1:], newFlags)
 		} else {
@@ -148,6 +132,18 @@ func RunBazeliskWithArgsFunc(argsFunc ArgsFunc, repos *Repositories) (int, error
 			// that should be enabled for the given Bazel version.
 			args = insertArgs(args[1:], newFlags)
 		}
+	} else if len(args) > 0 && strings.HasPrefix(args[0], "--bisect") {
+		// When --bisect is present, we run the bisect logic.
+		if !strings.HasPrefix(args[0], "--bisect=") {
+			return -1, fmt.Errorf("Error: --bisect must have a value. Expected format: '--bisect=<good bazel commit>..<bad bazel commit>'")
+		}
+		value := args[0][len("--bisect="):]
+		commits := strings.Split(value, "..")
+		if len(commits) == 2 {
+			bisect(commits[0], commits[1], args[1:], bazeliskHome, repos)
+		} else {
+			return -1, fmt.Errorf("Error: Invalid format for --bisect. Expected format: '--bisect=<good bazel commit>..<bad bazel commit>'")
+		}
 	}
 
 	// print bazelisk version information if "version" is the first argument
@@ -404,6 +400,27 @@ func parseBazelForkAndVersion(bazelForkAndVersion string) (string, string, error
 	return bazelFork, bazelVersion, nil
 }
 
+func downloadBazel(bazelVersionString string, bazeliskHome string, repos *Repositories) (string, error) {
+	bazelFork, bazelVersion, err := parseBazelForkAndVersion(bazelVersionString)
+	if err != nil {
+		return "", fmt.Errorf("could not parse Bazel fork and version: %v", err)
+	}
+
+	resolvedBazelVersion, downloader, err := repos.ResolveVersion(bazeliskHome, bazelFork, bazelVersion)
+	if err != nil {
+		return "", fmt.Errorf("could not resolve the version '%s' to an actual version number: %v", bazelVersion, err)
+	}
+
+	bazelForkOrURL := dirForURL(GetEnvOrConfig(BaseURLEnv))
+	if len(bazelForkOrURL) == 0 {
+		bazelForkOrURL = bazelFork
+	}
+
+	baseDirectory := filepath.Join(bazeliskHome, "downloads", bazelForkOrURL)
+	bazelPath, err := downloadBazelIfNecessary(resolvedBazelVersion, baseDirectory, repos, downloader)
+	return bazelPath, err
+}
+
 func downloadBazelIfNecessary(version string, baseDirectory string, repos *Repositories, downloader DownloadFunc) (string, error) {
 	pathSegment, err := platforms.DetermineBazelFilename(version, false)
 	if err != nil {
@@ -422,8 +439,14 @@ func downloadBazelIfNecessary(version string, baseDirectory string, repos *Repos
 	}
 
 	var tmpDestPath string
-	if url := GetEnvOrConfig(BaseURLEnv); url != "" {
-		tmpDestPath, err = repos.DownloadFromBaseURL(url, version, destDir, tmpDestFile)
+	baseURL := GetEnvOrConfig(BaseURLEnv)
+	formatURL := GetEnvOrConfig(FormatURLEnv)
+	if baseURL != "" && formatURL != "" {
+		return "", fmt.Errorf("cannot set %s and %s at once", BaseURLEnv, FormatURLEnv)
+	} else if formatURL != "" {
+		tmpDestPath, err = repos.DownloadFromFormatURL(formatURL, version, destDir, tmpDestFile)
+	} else if baseURL != "" {
+		tmpDestPath, err = repos.DownloadFromBaseURL(baseURL, version, destDir, tmpDestFile)
 	} else {
 		tmpDestPath, err = downloader(destDir, tmpDestFile)
 	}
@@ -717,6 +740,174 @@ func cleanIfNeeded(bazelPath string, startupOptions []string) {
 	}
 }
 
+type ParentCommit struct {
+	SHA string `json:"sha"`
+}
+
+type Commit struct {
+	SHA string `json:"sha"`
+	PARENTS []ParentCommit `json:"parents"`
+}
+
+type CompareResponse struct {
+	Commits []Commit `json:"commits"`
+	BaseCommit Commit `json:"base_commit"`
+	MergeBaseCommit Commit `json:"merge_base_commit"`
+}
+
+func sendRequest(url string) (*http.Response, error) {
+	client := &http.Client{}
+
+	req, err := http.NewRequest("GET", url, nil)
+	if err != nil {
+		return nil, err
+	}
+
+	githubToken := GetEnvOrConfig("BAZELISK_GITHUB_TOKEN")
+	if len(githubToken) != 0 {
+		req.Header.Set("Authorization", fmt.Sprintf("token %s", githubToken))
+	}
+
+	return client.Do(req)
+}
+
+func getBazelCommitsBetween(goodCommit string, badCommit string) (string, []string, error) {
+	commitList := make([]string, 0)
+	page := 1
+	perPage := 250 // 250 is the maximum number of commits per page
+
+	for {
+		url := fmt.Sprintf("https://api.github.com/repos/bazelbuild/bazel/compare/%s...%s?page=%d&per_page=%d", goodCommit, badCommit, page, perPage)
+
+		response, err := sendRequest(url)
+		if err != nil {
+			return goodCommit, nil, fmt.Errorf("Error fetching commit data: %v", err)
+		}
+		defer response.Body.Close()
+
+		body, err := ioutil.ReadAll(response.Body)
+		if err != nil {
+			return goodCommit, nil, fmt.Errorf("Error reading response body: %v", err)
+		}
+
+		if response.StatusCode == http.StatusNotFound {
+			return goodCommit, nil, fmt.Errorf("repository or commit not found: %s", string(body))
+		} else if response.StatusCode == 403 {
+			return goodCommit, nil, fmt.Errorf("github API rate limit hit, consider setting BAZELISK_GITHUB_TOKEN: %s", string(body))
+		} else if response.StatusCode != http.StatusOK {
+			return goodCommit, nil, fmt.Errorf("unexpected response status code %d: %s", response.StatusCode, string(body))
+		}
+
+		var compareResponse CompareResponse
+		err = json.Unmarshal(body, &compareResponse)
+		if err != nil {
+			return goodCommit, nil, fmt.Errorf("Error unmarshaling JSON: %v", err)
+		}
+
+		if len(compareResponse.Commits) == 0 {
+			break
+		}
+
+		mergeBaseCommit := compareResponse.MergeBaseCommit.SHA
+		if mergeBaseCommit != compareResponse.BaseCommit.SHA {
+			fmt.Printf("The good Bazel commit is not an ancestor of the bad Bazel commit, overriding the good Bazel commit to the merge base commit %s\n", mergeBaseCommit)
+			goodCommit = mergeBaseCommit
+		}
+
+		for _, commit := range compareResponse.Commits {
+			// If it has only one parent commit, add it to the list, otherwise it's a merge commit and we ignore it
+			if len(commit.PARENTS) == 1 {
+				commitList = append(commitList, commit.SHA)
+			}
+		}
+
+		// Check if there are more commits to fetch
+		if len(compareResponse.Commits) < perPage {
+			break
+		}
+
+		page++
+	}
+
+	if len(commitList) == 0 {
+		return goodCommit, nil, fmt.Errorf("no commits found between (%s, %s], the good commit should be first, maybe try with --bisect=%s..%s ?", goodCommit, badCommit, badCommit, goodCommit)
+	}
+	fmt.Printf("Found %d commits between (%s, %s]\n", len(commitList), goodCommit, badCommit)
+	return goodCommit, commitList, nil
+}
+
+func bisect(goodCommit string, badCommit string, args []string, bazeliskHome string, repos *Repositories) {
+
+	// 1. Get the list of commits between goodCommit and badCommit
+	fmt.Printf("\n\n--- Getting the list of commits between %s and %s\n\n", goodCommit, badCommit)
+	goodCommit, commitList, err := getBazelCommitsBetween(goodCommit, badCommit)
+	if err != nil {
+		log.Fatalf("Failed to get commits: %v", err)
+		os.Exit(1)
+	}
+
+	// 2. Check if goodCommit is actually good
+	fmt.Printf("\n\n--- Verifying if the given good Bazel commit (%s) is actually good\n\n", goodCommit)
+	bazelExitCode, err := testWithBazelAtCommit(goodCommit, args, bazeliskHome, repos)
+	if err != nil {
+		log.Fatalf("could not run Bazel: %v", err)
+		os.Exit(1)
+	}
+	if bazelExitCode != 0 {
+		fmt.Printf("Failure: Given good bazel commit is already broken.\n")
+		os.Exit(1)
+	}
+
+	// 3. Bisect commits
+	fmt.Printf("\n\n--- Start bisecting\n\n")
+	left := 0
+	right := len(commitList)
+	for left < right {
+		mid := (left + right) / 2
+		midCommit := commitList[mid]
+		fmt.Printf("\n\n--- Testing with Bazel built at %s, %d commits remaining...\n\n", midCommit, right -left)
+		bazelExitCode, err := testWithBazelAtCommit(midCommit, args, bazeliskHome, repos)
+		if err != nil {
+			log.Fatalf("could not run Bazel: %v", err)
+			os.Exit(1)
+		}
+		if bazelExitCode == 0 {
+			fmt.Printf("\n\n--- Succeeded at %s\n\n", midCommit)
+			left = mid + 1
+		} else {
+			fmt.Printf("\n\n--- Failed at %s\n\n", midCommit)
+			right = mid
+		}
+	}
+
+	// 4. Print the result
+	fmt.Printf("\n\n--- Bisect Result\n\n")
+	if right == len(commitList) {
+		fmt.Printf("first bad commit not found, every commit succeeded.\n")
+	} else {
+		firstBadCommit := commitList[right]
+		fmt.Printf("first bad commit is https://github.com/bazelbuild/bazel/commit/%s\n", firstBadCommit)
+	}
+
+	os.Exit(0)
+}
+
+func testWithBazelAtCommit(bazelCommit string, args []string, bazeliskHome string, repos *Repositories) (int, error) {
+	bazelPath, err := downloadBazel(bazelCommit, bazeliskHome, repos)
+	if err != nil {
+		return 1, fmt.Errorf("could not download Bazel: %v", err)
+	}
+	startupOptions := parseStartupOptions(args)
+	shutdownIfNeeded(bazelPath, startupOptions)
+	cleanIfNeeded(bazelPath, startupOptions)
+	fmt.Printf("bazel %s\n", strings.Join(args, " "))
+	bazelExitCode, err := runBazel(bazelPath, args, nil)
+	if err != nil {
+		return -1, fmt.Errorf("could not run Bazel: %v", err)
+	}
+	return bazelExitCode, nil
+}
+
 // migrate will run Bazel with each flag separately and report which ones are failing.
 func migrate(bazelPath string, baseArgs []string, flags []string) {
 	var startupOptions = parseStartupOptions(baseArgs)
diff --git a/core/repositories.go b/core/repositories.go
index f0ab732..7b5b4be 100644
--- a/core/repositories.go
+++ b/core/repositories.go
@@ -13,6 +13,9 @@ import (
 const (
 	// BaseURLEnv is the name of the environment variable that stores the base URL for downloads.
 	BaseURLEnv = "BAZELISK_BASE_URL"
+
+	// FormatURLEnv is the name of the environment variable that stores the format string to generate URLs for downloads.
+	FormatURLEnv = "BAZELISK_FORMAT_URL"
 )
 
 // DownloadFunc downloads a specific Bazel binary to the given location and returns the absolute path.
@@ -232,6 +235,65 @@ func (r *Repositories) DownloadFromBaseURL(baseURL, version, destDir, destFile s
 	return httputil.DownloadBinary(url, destDir, destFile)
 }
 
+func BuildURLFromFormat(formatURL, version string) (string, error) {
+	osName, err := platforms.DetermineOperatingSystem()
+	if err != nil {
+		return "", err
+	}
+
+	machineName, err := platforms.DetermineArchitecture(osName, version)
+	if err != nil {
+		return "", err
+	}
+
+	var b strings.Builder
+	b.Grow(len(formatURL) * 2) // Approximation.
+	for i := 0; i < len(formatURL); i++ {
+		ch := formatURL[i]
+		if ch == '%' {
+			i++
+			if i == len(formatURL) {
+				return "", errors.New("trailing %")
+			}
+
+			ch = formatURL[i]
+			switch ch {
+			case 'e':
+				b.WriteString(platforms.DetermineExecutableFilenameSuffix())
+			case 'h':
+				b.WriteString(GetEnvOrConfig("BAZELISK_VERIFY_SHA256"))
+			case 'm':
+				b.WriteString(machineName)
+			case 'o':
+				b.WriteString(osName)
+			case 'v':
+				b.WriteString(version)
+			case '%':
+				b.WriteByte('%')
+			default:
+				return "", fmt.Errorf("unknown placeholder %%%c", ch)
+			}
+		} else {
+			b.WriteByte(ch)
+		}
+	}
+	return b.String(), nil
+}
+
+// DownloadFromFormatURL can download Bazel binaries from a specific URL while ignoring the predefined repositories.
+func (r *Repositories) DownloadFromFormatURL(formatURL, version, destDir, destFile string) (string, error) {
+	if formatURL == "" {
+		return "", fmt.Errorf("%s is not set", FormatURLEnv)
+	}
+
+	url, err := BuildURLFromFormat(formatURL, version)
+	if err != nil {
+		return "", err
+	}
+
+	return httputil.DownloadBinary(url, destDir, destFile)
+}
+
 // CreateRepositories creates a new Repositories instance with the given repositories. Any nil repository will be replaced by a dummy repository that raises an error whenever a download is attempted.
 func CreateRepositories(releases ReleaseRepo, candidates CandidateRepo, fork ForkRepo, commits CommitRepo, rolling RollingRepo, supportsBaseURL bool) *Repositories {
 	repos := &Repositories{supportsBaseURL: supportsBaseURL}
diff --git a/go.mod b/go.mod
index fb48e45..d22d050 100644
--- a/go.mod
+++ b/go.mod
@@ -3,7 +3,7 @@ module github.com/bazelbuild/bazelisk
 go 1.18
 
 require (
-	github.com/bazelbuild/rules_go v0.38.1
+	github.com/bazelbuild/rules_go v0.39.1
 	github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d
 	github.com/hashicorp/go-version v1.6.0
 	github.com/mitchellh/go-homedir v1.1.0
diff --git a/go.sum b/go.sum
index bf171a5..c003c62 100644
--- a/go.sum
+++ b/go.sum
@@ -1,5 +1,5 @@
-github.com/bazelbuild/rules_go v0.38.1 h1:YGNsLhWe18Ielebav7cClP3GMwBxBE+xEArLHtmXDx8=
-github.com/bazelbuild/rules_go v0.38.1/go.mod h1:TMHmtfpvyfsxaqfL9WnahCsXMWDMICTw7XeK9yVb+YU=
+github.com/bazelbuild/rules_go v0.39.1 h1:wkJLUDx59dntWMghuL8++GteoU1To6sRoKJXuyFtmf8=
+github.com/bazelbuild/rules_go v0.39.1/go.mod h1:TMHmtfpvyfsxaqfL9WnahCsXMWDMICTw7XeK9yVb+YU=
 github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d h1:xDfNPAt8lFiC1UJrqV3uuy861HCTo708pDMbjHHdCas=
 github.com/bgentry/go-netrc v0.0.0-20140422174119-9fd32a8b3d3d/go.mod h1:6QX/PXZ00z/TKoufEY6K/a0k6AhaJrQKdFe6OfVXsa4=
 github.com/hashicorp/go-version v1.6.0 h1:feTTfFNnjP967rlCxM/I9g701jU+RN74YKx2mOkIeek=
diff --git a/httputil/BUILD b/httputil/BUILD
index f168cff..d48f290 100644
--- a/httputil/BUILD
+++ b/httputil/BUILD
@@ -10,12 +10,12 @@ go_library(
         "fake.go",
         "httputil.go",
     ],
+    importpath = "github.com/bazelbuild/bazelisk/httputil",
+    visibility = ["//visibility:public"],
     deps = [
+        "@com_github_bgentry_go_netrc//:go_default_library",
         "@com_github_mitchellh_go_homedir//:go_default_library",
-        "@com_github_bgentry_go_netrc//:go_default_library"
     ],
-    importpath = "github.com/bazelbuild/bazelisk/httputil",
-    visibility = ["//visibility:public"],
 )
 
 go_test(
diff --git a/platforms/BUILD b/platforms/BUILD
index dcb655f..8ab333f 100644
--- a/platforms/BUILD
+++ b/platforms/BUILD
@@ -16,3 +16,9 @@ go_test(
     srcs = ["platforms_test.go"],
     embed = [":go_default_library"],
 )
+
+go_test(
+    name = "go_default_test",
+    srcs = ["platforms_test.go"],
+    embed = [":go_default_library"],
+)
diff --git a/platforms/platforms.go b/platforms/platforms.go
index 3c0fcbe..2e918dc 100644
--- a/platforms/platforms.go
+++ b/platforms/platforms.go
@@ -54,8 +54,7 @@ func DetermineExecutableFilenameSuffix() string {
 	return filenameSuffix
 }
 
-// DetermineBazelFilename returns the correct file name of a local Bazel binary.
-func DetermineBazelFilename(version string, includeSuffix bool) (string, error) {
+func DetermineArchitecture(osName, version string) (string, error) {
 	var machineName string
 	switch runtime.GOARCH {
 	case "amd64":
@@ -66,16 +65,32 @@ func DetermineBazelFilename(version string, includeSuffix bool) (string, error)
 		return "", fmt.Errorf("unsupported machine architecture \"%s\", must be arm64 or x86_64", runtime.GOARCH)
 	}
 
-	var osName string
+	if osName == "darwin" {
+		machineName = DarwinFallback(machineName, version)
+	}
+
+	return machineName, nil
+}
+
+func DetermineOperatingSystem() (string, error) {
 	switch runtime.GOOS {
 	case "darwin", "linux", "windows":
-		osName = runtime.GOOS
+		return runtime.GOOS, nil
 	default:
 		return "", fmt.Errorf("unsupported operating system \"%s\", must be Linux, macOS or Windows", runtime.GOOS)
 	}
+}
 
-	if osName == "darwin" {
-		machineName = DarwinFallback(machineName, version)
+// DetermineBazelFilename returns the correct file name of a local Bazel binary.
+func DetermineBazelFilename(version string, includeSuffix bool) (string, error) {
+	osName, err := DetermineOperatingSystem()
+	if err != nil {
+		return "", err
+	}
+
+	machineName, err := DetermineArchitecture(osName, version)
+	if err != nil {
+		return "", err
 	}
 
 	var filenameSuffix string
diff --git a/repositories/gcs.go b/repositories/gcs.go
index 542e781..db6f0b8 100644
--- a/repositories/gcs.go
+++ b/repositories/gcs.go
@@ -9,6 +9,7 @@ import (
 	"fmt"
 	"log"
 	"strings"
+	"time"
 
 	"github.com/bazelbuild/bazelisk/core"
 	"github.com/bazelbuild/bazelisk/httputil"
@@ -77,7 +78,22 @@ func listDirectoriesInReleaseBucket(prefix string) ([]string, bool, error) {
 		if nextPageToken != "" {
 			url = fmt.Sprintf("%s&pageToken=%s", baseURL, nextPageToken)
 		}
-		content, _, err := httputil.ReadRemoteFile(url, "")
+
+		var content []byte
+		var err error
+		// Theoretically, this should always work, but we've seen transient
+		// errors on Bazel CI, so we retry a few times to work around this.
+		// https://github.com/bazelbuild/continuous-integration/issues/1627
+		waitTime := 100 * time.Microsecond
+		for attempt := 0; attempt < 5; attempt++ {
+			content, _, err = httputil.ReadRemoteFile(url, "")
+			if err == nil {
+				break
+			}
+			time.Sleep(waitTime)
+			waitTime *= 2
+		}
+
 		if err != nil {
 			return nil, false, fmt.Errorf("could not list GCS objects at %s: %v", url, err)
 		}


Test patch:
diff --git a/bazelisk_test.sh b/bazelisk_test.sh
index 66c5b7b..2ce2c90 100755
--- a/bazelisk_test.sh
+++ b/bazelisk_test.sh
@@ -197,7 +197,20 @@ function test_bazel_version_from_file() {
       (echo "FAIL: Expected to find 'Build label: 5.0.0' in the output of 'bazelisk version'"; exit 1)
 }
 
-function test_bazel_version_from_url() {
+function test_bazel_version_from_format_url() {
+  setup
+
+  echo "0.19.0" > .bazelversion
+
+  BAZELISK_FORMAT_URL="https://github.com/bazelbuild/bazel/releases/download/%v/bazel-%v-%o-%m%e" \
+      BAZELISK_HOME="$BAZELISK_HOME" \
+          bazelisk version 2>&1 | tee log
+
+  grep "Build label: 0.19.0" log || \
+      (echo "FAIL: Expected to find 'Build label: 0.19.0' in the output of 'bazelisk version'"; exit 1)
+}
+
+function test_bazel_version_from_base_url() {
   setup
 
   echo "0.19.0" > .bazelversion
@@ -427,8 +440,12 @@ if [[ $BAZELISK_VERSION == "GO" ]]; then
   test_bazel_last_rc
   echo
 
-  echo "# test_bazel_version_from_url"
-  test_bazel_version_from_url
+  echo "# test_bazel_version_from_format_url"
+  test_bazel_version_from_format_url
+  echo
+
+  echo "# test_bazel_version_from_base_url"
+  test_bazel_version_from_base_url
   echo
 
   echo "# test_bazel_version_prefer_environment_to_bazeliskrc"
diff --git a/core/repositories_test.go b/core/repositories_test.go
new file mode 100644
index 0000000..f644244
--- /dev/null
+++ b/core/repositories_test.go
@@ -0,0 +1,78 @@
+package core
+
+import (
+	"errors"
+	"fmt"
+	"os"
+	"testing"
+
+	"github.com/bazelbuild/bazelisk/platforms"
+)
+
+func TestBuildURLFromFormat(t *testing.T) {
+	osName, err := platforms.DetermineOperatingSystem()
+	if err != nil {
+		t.Fatalf("Cannot get operating system name: %v", err)
+	}
+
+	version := "6.0.0"
+
+	machineName, err := platforms.DetermineArchitecture(osName, version)
+	if err != nil {
+		t.Fatalf("Cannot get machine architecture name: %v", err)
+	}
+
+	suffix := platforms.DetermineExecutableFilenameSuffix()
+
+	previousSha256, hadSha256 := os.LookupEnv("BAZELISK_VERIFY_SHA256")
+	sha256 := "SomeSha256ValueThatIsIrrelevant"
+	if err := os.Setenv("BAZELISK_VERIFY_SHA256", sha256); err != nil {
+		t.Fatalf("Failed to set BAZELISK_VERIFY_SHA256")
+	}
+	defer func() {
+		if hadSha256 {
+			os.Setenv("BAZELISK_VERIFY_SHA256", previousSha256)
+		} else {
+			os.Unsetenv("BAZELISK_VERIFY_SHA256")
+		}
+	}()
+
+	type test struct {
+		format  string
+		want    string
+		wantErr error
+	}
+
+	tests := []test{
+		{format: "", want: ""},
+		{format: "no/placeholders", want: "no/placeholders"},
+
+		{format: "%", wantErr: errors.New("trailing %")},
+		{format: "%%", want: "%"},
+		{format: "%%%%", want: "%%"},
+		{format: "invalid/trailing/%", wantErr: errors.New("trailing %")},
+		{format: "escaped%%placeholder", want: "escaped%placeholder"},
+
+		{format: "foo-%e-bar", want: fmt.Sprintf("foo-%s-bar", suffix)},
+		{format: "foo-%h-bar", want: fmt.Sprintf("foo-%s-bar", sha256)},
+		{format: "foo-%m-bar", want: fmt.Sprintf("foo-%s-bar", machineName)},
+		{format: "foo-%o-bar", want: fmt.Sprintf("foo-%s-bar", osName)},
+		{format: "foo-%v-bar", want: fmt.Sprintf("foo-%s-bar", version)},
+
+		{format: "repeated %v %m %v", want: fmt.Sprintf("repeated %s %s %s", version, machineName, version)},
+
+		{format: "https://real.example.com/%e/%m/%o/%v#%%20trailing", want: fmt.Sprintf("https://real.example.com/%s/%s/%s/%s#%%20trailing", suffix, machineName, osName, version)},
+	}
+
+	for _, tc := range tests {
+		got, err := BuildURLFromFormat(tc.format, version)
+		if fmt.Sprintf("%v", err) != fmt.Sprintf("%v", tc.wantErr) {
+			if got != "" {
+				t.Errorf("format '%s': got non-empty '%s' on error", tc.format, got)
+			}
+			t.Errorf("format '%s': got error %v, want error %v", tc.format, err, tc.wantErr)
+		} else if got != tc.want {
+			t.Errorf("format '%s': got %s, want %s", tc.format, got, tc.want)
+		}
+	}
+}