Change:
Hello. I have been trying to compile a rather big project for wasm32-unknown-emscripten using cargo on Windows. Unfortunately, the linking process fails when trying to run because the command line invoked is too long for Windows' limitations (about 8k characters). After manually counting, the command indeed is too big with in my case about 11k characters, as it includes the absolute path of every single .o and .rlib files. My project, while including piston as a dependency, does not have an incredibly large amount of them, which makes me believe this could be an important issue in the long run (this does not apply to Linux however as the maximum character limit usually is much higher). After a little bit of digging, emscripten seems to provide a workaround for this problem by accepting parameters from files. However it has to be handled by the user. Thanks in advance for your help.
why not use a folder like c:a so that it works?
Well yes, that would work in my specific case. But do you really want that in a final product?
Maybe is related.
can you provide the full error message? It looks like this may be a bug in emscripten, not in rustc.
I uploaded the project I tried to compile to this repo contains the entire output edit: i'm adding the verbose output too edit: done
Ah ok thanks for the extra info So for a bit of background, rustc does indeed have for reinvoking the linker if necessary when the command line is deemed too large. What's happening here, though, is that the error is happening at the wrong time. The current detection relies on the spawning of the process itself to fail (aka the OS tells us about the failure) but it looks like at least is succeeding. Some spawned process actually prints out but it's not clear to me whether that's or .
Is there any additional test you'd want me to run to try to figure this out?
If you'd like to debug further that'd be great. We'd first need to understand what's actually failing here ( or something in emscripten). Once that's known we could figure out if it's reasonable for us to catch the error and do something about it.
So I edited to print a dummy message when it gets called. As the message did not show up, I came to the conclusion that cmd itself refuses to run the command. I tried to run it with a shorter command and the message showed up. This is surprising because that means rust would have failed to survive cmd's refusal. I'm not an OS expert so I'm not sure what rawoserror actually returns, but here cmd's return code is 1.
Ok so sounds like we successfully spawn a process and then fails to spawn its own subprocess. That's going to be much trickier to handle an catch, but it at least sounds like what's happening here at least.
A dirty way to catch it would be to simply check the error description. At least it is self explanatory so there won't be any false positive.
Another way I just thought of would be to try to summon without cmd, maybe trying to run it using something similar to C's . Maybe this would allow better handling.
Assuming and em++ are working in a similar matter, seems to provide a solution. We can simply write all the command parameters to an .rsp file, and then pass that file to the
Honestly, I believe doing this approach by default would be the best solution. The performance overhead is negligible when it's only about linking, and this would cover all future problems. It's simply a matter of deleting . I can not try it right now but if anyone is interested to remove it and see if it works.
Writing the arguments to file looks like a good solution here, yeah.
I'm running into this issue as well when trying to use the cargo-web crate. Any ETA on this being fixed in nightly?
If nobody does it before I do, I'll make file parameters default behavior, check if nothing breaks and make a pull request by Sunday.
I've posted a PR for this at

Discussion:
use std::ffi::{OsStr, OsString}; use std::fmt; use std::io; <ins> use std::mem; </ins> use std::process::{self, Output}; <ins> #[derive(Clone)] </ins> pub struct Command { <del> program: OsString, </del> <ins> program: Program, </ins> args: Vec<OsString>, env: Vec<(OsString, OsString)>, } <ins> #[derive(Clone)] enum Program { Normal(OsString), CmdBatScript(OsString), } </ins> impl Command { pub fn new<P: AsRef<OsStr>>(program: P) -> Command { <del> Command::_new(program.as_ref()) </del> <ins> Command::_new(Program::Normal(program.as_ref().to_owned())) } pub fn bat_script<P: AsRef<OsStr>>(program: P) -> Command { Command::_new(Program::CmdBatScript(program.as_ref().to_owned())) </ins> } <del> fn _new(program: &OsStr) -> Command { </del> <ins> fn _new(program: Program) -> Command { </ins> Command { <del> program: program.to_owned(), </del> <ins> program, </ins> args: Vec::new(), env: Vec::new(), }

} pub fn command(&self) -> process::Command { <del> let mut ret = process::Command::new(&self.program); </del> <ins> let mut ret = match self.program { Program::Normal(ref p) => process::Command::new(p), Program::CmdBatScript(ref p) => { let mut c = process::Command::new("cmd"); c.arg("/c").arg(p); c } }; </ins> ret.args(&self.args); ret.envs(self.env.clone()); return ret

// extensions <del> pub fn get_program(&self) -> &OsStr { &self.program </del> <ins> pub fn take_args(&mut self) -> Vec<OsString> { mem::replace(&mut self.args, Vec::new()) </ins> } <del> pub fn get_args(&self) -> &[OsString] { &self.args } </del> <ins> /// Returns a `true` if we're pretty sure that this'll blow OS spawn limits, /// or `false` if we should attempt to spawn and see what the OS says. pub fn very_likely_to_exceed_some_spawn_limit(&self) -> bool { // We mostly only care about Windows in this method, on Unix the limits // can be gargantuan anyway so we're pretty unlikely to hit them if cfg!(unix) { return false } </ins> <del> pub fn get_env(&self) -> &[(OsString, OsString)] { &self.env </del> <ins> // Ok so on Windows to spawn a process is 32,768 characters in its // command line [1]. Unfortunately we don't actually have access to that // as it's calculated just before spawning. Instead we perform a // poor-man's guess as to how long our command line will be. We're // assuming here that we don't have to escape every character... // // Turns out though that `cmd.exe` has even smaller limits, 8192 // characters [2]. Linkers can often be batch scripts (for example // Emscripten, Gecko's current build system) which means that we're // running through batch scripts. These linkers often just forward // arguments elsewhere (and maybe tack on more), so if we blow 8192 // bytes we'll typically cause them to blow as well. // // Basically as a result just perform an inflated estimate of what our // command line will look like and test if it's > 8192 (we actually // test against 6k to artificially inflate our estimate). If all else // fails we'll fall back to the normal unix logic of testing the OS // error code if we fail to spawn and automatically re-spawning the // linker with smaller arguments. // // [1]: https://msdn.microsoft.com/en-us/library/windows/desktop/ms682425(v=vs.85).aspx // [2]: https://blogs.msdn.microsoft.com/oldnewthing/20031210-00/?p=41553 let estimated_command_line_len = self.args.iter().map(|a| a.len()).sum::<usize>(); estimated_command_line_len > 1024 * 6 </ins> } }

use std::env; use std::ffi::OsString; use std::fmt; <del> use std::fs::{self, File}; use std::io::{self, Write, BufWriter}; </del> <ins> use std::fs; use std::io; </ins> use std::path::{Path, PathBuf}; use std::process::{Output, Stdio}; use std::str;

let cmd = |linker: &Path| { if let Some(linker) = linker.to_str() { if cfg!(windows) && linker.ends_with(".bat") { <del> let mut cmd = Command::new("cmd"); cmd.arg("/c").arg(linker); return cmd </del> <ins> return Command::bat_script(linker) </ins> } } Command::new(linker)

// that contains all the arguments. The theory is that this is then // accepted on all linkers and the linker will read all its options out of // there instead of looking at the command line. <del> match cmd.command().stdout(Stdio::piped()).stderr(Stdio::piped()).spawn() { Ok(child) => return child.wait_with_output(), Err(ref e) if command_line_too_big(e) => {} Err(e) => return Err(e) </del> <ins> if !cmd.very_likely_to_exceed_some_spawn_limit() { match cmd.command().stdout(Stdio::piped()).stderr(Stdio::piped()).spawn() { Ok(child) => return child.wait_with_output(), Err(ref e) if command_line_too_big(e) => {} Err(e) => return Err(e) } </ins> } <del> let file = tmpdir.join("linker-arguments"); let mut cmd2 = Command::new(cmd.get_program()); cmd2.arg(format!("@{}", file.display())); for &(ref k, ref v) in cmd.get_env() { cmd2.env(k, v); } let mut f = BufWriter::new(File::create(&file)?); for arg in cmd.get_args() { writeln!(f, "{}", Escape { </del> <ins> let mut cmd2 = cmd.clone(); let mut args = String::new(); for arg in cmd2.take_args() { args.push_str(&Escape { </ins> arg: arg.to_str().unwrap(), is_like_msvc: sess.target.target.options.is_like_msvc, <del> })?; </del> <ins> }.to_string()); args.push_str("n"); </ins> } <del> f.into_inner()?; </del> <ins> let file = tmpdir.join("linker-arguments"); fs::write(&file, args.as_bytes())?; cmd2.arg(format!("@{}", file.display())); </ins> return cmd2.output(); #[cfg(unix)]

<ins> -include ../tools.mk all: $(RUSTC) foo.rs -g cp foo.bat $(TMPDIR)/ OUT_DIR="$(TMPDIR)" RUSTC="$(RUSTC_ORIGINAL)" $(call RUN,foo) </ins>

<ins> // Copyright 2018 The Rust Project Developers. See the COPYRIGHT // file at the top-level directory of this distribution and at // http://rust-lang.org/COPYRIGHT. // // Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or // http://www.apache.org/licenses/LICENSE-2.0> or the MIT license // <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your // option. This file may not be copied, modified, or distributed // except according to those terms. // Like the `long-linker-command-lines` test this test attempts to blow // a command line limit for running the linker. Unlike that test, however, // this test is testing `cmd.exe` specifically rather than the OS. // // Unfortunately `cmd.exe` has a 8192 limit which is relatively small // in the grand scheme of things and anyone sripting rustc's linker // is probably using a `*.bat` script and is likely to hit this limit. // // This test uses a `foo.bat` script as the linker which just simply // delegates back to this program. The compiler should use a lower // limit for arguments before passing everything via `@`, which // means that everything should still succeed here. use std::env; use std::fs::{self, File}; use std::io::{BufWriter, Write, Read}; use std::path::PathBuf; use std::process::Command; fn main() { if !cfg!(windows) { return } let tmpdir = PathBuf::from(env::var_os("OUT_DIR").unwrap()); let ok = tmpdir.join("ok"); let not_ok = tmpdir.join("not_ok"); if env::var("YOU_ARE_A_LINKER").is_ok() { match env::args().find(|a| a.contains("@")) { Some(file) => { fs::copy(&file[1..], &ok).unwrap(); } None => { File::create(&not_ok).unwrap(); } } return } let rustc = env::var_os("RUSTC").unwrap_or("rustc".into()); let me = env::current_exe().unwrap(); let bat = me.parent() .unwrap() .join("foo.bat"); let bat_linker = format!("linker={}", bat.display()); for i in (1..).map(|i| i * 10) { println!("attempt: {}", i); let file = tmpdir.join("bar.rs"); let mut f = BufWriter::new(File::create(&file).unwrap()); let mut lib_name = String::new(); for _ in 0..i { lib_name.push_str("foo"); } for j in 0..i { writeln!(f, "#[link(name = "{}{}")]", lib_name, j).unwrap(); } writeln!(f, "extern {{}}nfn main() {{}}").unwrap(); f.into_inner().unwrap(); drop(fs::remove_file(&ok)); drop(fs::remove_file(&not_ok)); let status = Command::new(&rustc) .arg(&file) .arg("-C").arg(&bat_linker) .arg("--out-dir").arg(&tmpdir) .env("YOU_ARE_A_LINKER", "1") .env("MY_LINKER", &me) .status() .unwrap(); if !status.success() { panic!("rustc didn't succeed: {}", status); } if !ok.exists() { assert!(not_ok.exists()); continue } let mut contents = String::new(); File::open(&ok).unwrap().read_to_string(&mut contents).unwrap(); for j in 0..i { assert!(contents.contains(&format!("{}{}", lib_name, j))); } break } } </ins>