Change:
We're passing --retry: but by default that only retries 500 errors, not any error: Originally posted by in <!-- TRIAGEBOTSTART --<!-- TRIAGEBOTASSIGNSTART --<!-- TRIAGEBOTASSIGNDATASTARTTRIAGEBOTASSIGNDATAEND --<!-- TRIAGEBOTASSIGNEND --<!-- TRIAGEBOTEND --
This is a regression from when I moved the downloads from python to rust (about a year ago now I think).
Mentoring instructions: Add the option to the invocations, in and This doesn't affect the Windows Powershell fallback, since we already retry on any error. There are many other invocations across the repo, but they seem to not be causing problems.
claim
This is quite new flag shipped with recent versions of curl, which is not supported on many machines/platforms(including our CI runners). We can consider using as I also suggest at
I'd like to take a look at this. Looking through the PR history, we would like basic retry logic written in Rust versus using a curl flag like to retry during the bootstrap process. If it's a 404, then do not retry, but any other errors should be retried. I assume there is a http library already vendored in, so no new dependencies should be .
claim
Thanks! This shouldn't be needed. For now, retrying on "connection refused" () should be fine.
I'm a little confused as to why was closed then. What more is there beyond adding the retry refused connections flag?
I forgot that our CI runners depend on very old curl versions which doesn't support the flag I suggested. Sorry, you are right. As jyn also said() in that PR, we can handle this in Rust
are we ok with adding the dependency to facilitate sending requests to the payload URL? I'm very hesitant to add another crate dependency to the core payload. But without it, we would need to manually built a request via the standard library tcp library. Which is a little awkward. doesn't actually have support for sending requests. I suppose, in the case we decide to use it, another library would need to be as well?
What about using curl arguments to read the status code and handle it in rust logic? It might be a bit ugly, but adding a dependency solely for this purpose seems like overkill to me.
I agree adding a new dep seems extreme. I can try parsing the curl exit code, great idea.
claim
claim
Hello Is this still relevant?
please don't ping me about rust-lang/ issues. is the current maintainer.
We didn't change any curl flags since then; it should be still relevant.
release-assignment

Discussion:
eprint("removing", temp_path) os.unlink(temp_path) <ins> def curl_version(): m = re.match(bytes("^curl ([0-9]+).([0-9]+)", "utf8"), require(["curl", "-V"])) if m is None: return (0, 0) return (int(m[1]), int(m[2])) </ins> def download(path, url, probably_big, verbose): for _ in range(4):

# If curl is not present on Win32, we should not sys.exit #   but raise `CalledProcessError` or `OSError` instead require(["curl", "--version"], exception=platform_is_win32()) <del> run(["curl", option, </del> <ins> extra_flags = [] if curl_version() > (7, 70): extra_flags = [ "--retry-all-errors" ] run(["curl", option] + extra_flags + [ </ins> "-L", # Follow redirect. "-y", "30", "-Y", "10",    # timeout if speed is < 10 bytes/sec for > 30 seconds "--connect-timeout", "30",  # timeout if cannot connect within 30 seconds "-o", path, <ins> "--continue-at", "-", </ins> "--retry", "3", "-SRf", url], verbose=verbose, exception=True, # Will raise RuntimeError on failure

config.try_run(cmd) } <ins> fn extract_curl_version(out: &[u8]) -> semver::Version { let out = String::from_utf8_lossy(out); // The output should look like this: "curl <major>.<minor>.<patch> ..." out.lines() .next() .and_then(|line| line.split(" ").nth(1)) .and_then(|version| semver::Version::parse(version).ok()) .unwrap_or(semver::Version::new(1, 0, 0)) } fn curl_version() -> semver::Version { let mut curl = Command::new("curl"); curl.arg("-V"); let Ok(out) = curl.output() else { return semver::Version::new(1, 0, 0) }; let out = out.stdout; extract_curl_version(&out) } </ins> /// Generic helpers that are useful anywhere in bootstrap. impl Config { pub fn is_verbose(&self) -> bool {

"30", // timeout if cannot connect within 30 seconds "-o", tempfile.to_str().unwrap(), <ins> "--continue-at", "-", </ins> "--retry", "3", "-SRf",

} else { curl.arg("--progress-bar"); } <ins> // --retry-all-errors was added in 7.71.0, don't use it if curl is old. if curl_version() >= semver::Version::new(7, 71, 0) { curl.arg("--retry-all-errors"); } </ins> curl.arg(url); if !self.check_run(&mut curl) { if self.build.contains("windows-msvc") {