Change:
I went to this page: and I discovered that the download is 203 MB. This was surprising to me, so I looked at other languages: Language Link ------------------------------- Go Perl Julia Python PHP D Nim https://nim- Ruby So Rust is 70% larger than Go. Or to put another way, Rust is larger than Julia, Python, PHP, D, Nim and Ruby combined. Can anything be done about this or is the large size unavoidable?
IIUC will reduce the size, potentially by a lot
Also the comparison might want to include Haskell, whose installer is 268 MB ().
These comparisons are probably missing various aspects, e.g. Haskell includes the full haskell platform with a list of packages as well as a runtime. Rust might include other things... We should compare with our past selves first and foremost.
where are you getting that 205 MB number from? When I try to download it tells me 252 MB so the problem is even worse than you say it is... Maybe linux vs windows? Some points for where the potential bloat could come from: licenses are stored multiple times for each package. Costs a few KB but I guess it can't be avoided. It would probably be useful to have a compression format that can do content based deduplication, but I guess there is none out there, is what we need to use. save analysis of all std crates is being stored. Using json, the save analysis format is extremely verbose. It contains repetitive stuff like references to the same source file over and over. It could probably be improved a lot by a) interning stuff like source file names and b) a binary format. As IIRC serde is used already for json output and input, adopting bincode would be a first easy step then followed by interning (interning can be done transparently from bincode). The save-analysis format is unstable and probably always will be, given that replacement is on its way with rust-analyzer. If there is ever a desire to stabilize, one can switch back to json. As for the point that we are using deflate compression, sure it helps but a native compression would help even more. rust's copy of LLVM is 76 MiB while julia's copy of LLVM is 48 MB (both uncompressed). Why is that the case? Rust has LLVM 8.0 and julia 6.0 but that can't be the reason, has LLVM grown this much in size between two versions? there is a documentation directory that contains rustdoc/mdbook of various components. Each component has its very own copy of fonts (each almost half of a MB large). It's repeated multiple times. I'm sure much can be saved by using a shared copy of those fonts.
At least for Linux a lot of dependencies are linked statically (even libstdc++) to make it work on old Linux releases.
no need to wait for nightlies, the built artifacts are already being uploaded to public servers. This gives us the following size impact of : Numbers collected via commands like . So for rustc there is a nice sweet reduction but std has a massive size increase. why is this the case? Maybe some cache not being emptied?
also includes the compiler crates, and with there's duplication for these as the same code is included in the rlibs and 's dylib.
Hi, I have a question. What needs those rlib files in rust-std? Can I remove those rlib files from rust-std and have most crates build as normal? (I'm repackaging rust-nightly for Arch Linux.) It's 109MiB larger now and it'll take me four more minutes to download (what's worse, I can't watch online videos meanwhile)....
There are some generic bloat problems that Rust has like which increase binary size which probably play into Rust's large platform size, too.
Before a lot of symbols were placed into shared libs and duplication wasn't an issue. Many of those shared libs were replaced by static ones () which has benefits (better performance, simplification of Rust internals) and downsides (duplicated symbols, bigger size). Using archives instead of reduces download size of std by 30 MiB at the expense of extraction time. It can help with slow connections. Rust itself and the crates. Some of the libs aren't necessary for every use case but they are relatively small. Big libs which are the issue here cannot be removed. It doesn't mean nothing can be improved. Multiple dependencies are built multiple times, it's clearly visible with . In std from commit there are 2 with size 2 MiB each. Most of the duplicates are much smaller but also harder to find because they don't have their own libs and can be found only inside other libs. EDIT: cc
thank you for the detailed response. However this part is concerning: that seems to be "cutting at the branches rather than cutting at the root". To compare Go 1.12.6 is 118 MB: Then Rust is 294 MB: thats nearly 2.5 times larger. I know every language is different but this seems too much.
It's not the solution by any means, I just wanted to make people aware of better compressed archives. seems too much. Yes, it's concerning for sake of the completeness there is another big compiler. Official is 325 MiB and for it's 262 MiB. It also means Rust is now bigged than GHC. That said nightly to nightly it's over 100 MiB download size regression and there should be an effort on reducing it but there is no immediate solution.
The two biggest are (85M) and (42M) -- are you sure those cannot be removed? I suppose nightly/unstable users may need those, but for stable ISTM we only need the artifacts from and (which includes the proc-macro shim).
We could potentially replace the s with just s, which would still allow to use those crates, we'd just need a way to force linking against (which contains the actual machine code from those crates). I've previously suggested this in
I'm sorry for being dense, but what is the reason for having two copies of the compiler? That is, each distribution contains the duplicates and (among many others).
are host libraries installed by the component, and are target libraries installed by the component. The latter may be used for cross compilation too. But yes, that duplication is unfortunate.
Can you explain more what purpose they serve for cross compilation? Targets which do not have host builds do not include a second copy of the compiler. Is it to support projects that link the compiler directly with ? I'm not sure I see how the target's is relevant for cross compiling.
Yes, you could cross compile something using rustc libs -- unstable, of course. But if goes through, we won't include this in anymore, probably just another optional component.

Discussion:
dist::Rustc, dist::DebuggerScripts, dist::Std, <ins> dist::RustcDev, </ins> dist::Analysis, dist::Src, dist::PlainSourceTarball,

} } <ins> fn skip_host_target_lib(builder: &Builder<'_>, compiler: Compiler) -> bool { // The only true set of target libraries came from the build triple, so // let's reduce redundant work by only producing archives from that host. if compiler.host != builder.config.build { builder.info("tskipping, not a build host"); true } else { false } } /// Copy stamped files into an image's `target/lib` directory. fn copy_target_libs(builder: &Builder<'_>, target: &str, image: &Path, stamp: &Path) { let dst = image.join("lib/rustlib").join(target).join("lib"); t!(fs::create_dir_all(&dst)); for (path, host) in builder.read_stamp_file(stamp) { if !host || builder.config.build == target { builder.copy(&path, &dst.join(path.file_name().unwrap())); } } } </ins> #[derive(Debug, PartialOrd, Ord, Copy, Clone, Hash, PartialEq, Eq)] pub struct Std { pub compiler: Compiler,

let target = self.target; let name = pkgname(builder, "rust-std"); <del> // The only true set of target libraries came from the build triple, so // let's reduce redundant work by only producing archives from that host. if compiler.host != builder.config.build { builder.info("tskipping, not a build host"); return distdir(builder).join(format!("{}-{}.tar.gz", name, target)); </del> <ins> let archive = distdir(builder).join(format!("{}-{}.tar.gz", name, target)); if skip_host_target_lib(builder, compiler) { return archive; </ins> } <del> // We want to package up as many target libraries as possible // for the `rust-std` package, so if this is a host target we // depend on librustc and otherwise we just depend on libtest. if builder.hosts.iter().any(|t| t == target) { builder.ensure(compile::Rustc { compiler, target }); } else { builder.ensure(compile::Std { compiler, target }); } </del> <ins> builder.ensure(compile::Std { compiler, target }); </ins> let image = tmpdir(builder).join(format!("{}-{}-image", name, target)); let _ = fs::remove_dir_all(&image); <del> let dst = image.join("lib/rustlib").join(target); t!(fs::create_dir_all(&dst)); let mut src = builder.sysroot_libdir(compiler, target).to_path_buf(); src.pop(); // Remove the trailing /lib folder from the sysroot_libdir builder.cp_filtered(&src, &dst, &|path| { if let Some(name) = path.file_name().and_then(|s| s.to_str()) { if name == builder.config.rust_codegen_backends_dir.as_str() { return false } if name == "bin" { return false } if name.contains("LLVM") { return false } } true }); </del> <ins> let compiler_to_use = builder.compiler_for(compiler.stage, compiler.host, target); let stamp = compile::libstd_stamp(builder, compiler_to_use, target); copy_target_libs(builder, &target, &image, &stamp); </ins> let mut cmd = rust_installer(builder); cmd.arg("generate")

let _time = timeit(builder); builder.run(&mut cmd); builder.remove_dir(&image); <del> distdir(builder).join(format!("{}-{}.tar.gz", name, target)) </del> <ins> archive } } #[derive(Debug, PartialOrd, Ord, Copy, Clone, Hash, PartialEq, Eq)] pub struct RustcDev { pub compiler: Compiler, pub target: Interned<String>, } impl Step for RustcDev { type Output = PathBuf; const DEFAULT: bool = true; const ONLY_HOSTS: bool = true; fn should_run(run: ShouldRun<'_>) -> ShouldRun<'_> { run.path("rustc-dev") } fn make_run(run: RunConfig<'_>) { run.builder.ensure(RustcDev { compiler: run.builder.compiler_for( run.builder.top_stage, run.builder.config.build, run.target, ), target: run.target, }); } fn run(self, builder: &Builder<'_>) -> PathBuf { let compiler = self.compiler; let target = self.target; let name = pkgname(builder, "rustc-dev"); let archive = distdir(builder).join(format!("{}-{}.tar.gz", name, target)); if skip_host_target_lib(builder, compiler) { return archive; } builder.ensure(compile::Rustc { compiler, target }); let image = tmpdir(builder).join(format!("{}-{}-image", name, target)); let _ = fs::remove_dir_all(&image); let compiler_to_use = builder.compiler_for(compiler.stage, compiler.host, target); let stamp = compile::librustc_stamp(builder, compiler_to_use, target); copy_target_libs(builder, &target, &image, &stamp); let mut cmd = rust_installer(builder); cmd.arg("generate") .arg("--product-name=Rust") .arg("--rel-manifest-dir=rustlib") .arg("--success-message=Rust-is-ready-to-develop.") .arg("--image-dir").arg(&image) .arg("--work-dir").arg(&tmpdir(builder)) .arg("--output-dir").arg(&distdir(builder)) .arg(format!("--package-name={}-{}", name, target)) .arg(format!("--component-name=rustc-dev-{}", target)) .arg("--legacy-manifest-dirs=rustlib,cargo"); builder.info(&format!("Dist rustc-dev stage{} ({} -> {})", compiler.stage, &compiler.host, target)); let _time = timeit(builder); builder.run(&mut cmd); builder.remove_dir(&image); archive </ins> } }

fn add_packages_to(&mut self, manifest: &mut Manifest) { let mut package = |name, targets| self.package(name, &mut manifest.pkg, targets); package("rustc", HOSTS); <ins> package("rustc-dev", HOSTS); </ins> package("cargo", HOSTS); package("rust-mingw", MINGW); package("rust-std", TARGETS);

"rls-preview", "rust-src", "llvm-tools-preview", "lldb-preview", "rust-analysis", "miri-preview" ]); <ins> // The compiler libraries are not stable for end users, but `rustc-dev` was only recently // split out of `rust-std`. We'll include it by default as a transition for nightly users. if self.rust_release == "nightly" { self.extend_profile("default", &mut manifest.profiles, &["rustc-dev"]); self.extend_profile("complete", &mut manifest.profiles, &["rustc-dev"]); } </ins> } fn add_renames_to(&self, manifest: &mut Manifest) {

components.push(host_component("rust-mingw")); } <ins> // The compiler libraries are not stable for end users, but `rustc-dev` was only recently // split out of `rust-std`. We'll include it by default as a transition for nightly users, // but ship it as an optional component on the beta and stable channels. if self.rust_release == "nightly" { components.push(host_component("rustc-dev")); } else { extensions.push(host_component("rustc-dev")); } </ins> // Tools are always present in the manifest, // but might be marked as unavailable if they weren't built. extensions.extend(vec![

.filter(|&&target| target != host) .map(|target| Component::from_str("rust-std", target)) ); <ins> extensions.extend( HOSTS.iter() .filter(|&&target| target != host) .map(|target| Component::from_str("rustc-dev", target)) ); </ins> extensions.push(Component::from_str("rust-src", "*")); // If the components/extensions don't actually exist for this

dst.insert(profile_name.to_owned(), pkgs.iter().map(|s| (*s).to_owned()).collect()); } <ins> fn extend_profile(&mut self, profile_name: &str, dst: &mut BTreeMap<String, Vec<String>>, pkgs: &[&str]) { dst.get_mut(profile_name).expect("existing profile") .extend(pkgs.iter().map(|s| (*s).to_owned())); } </ins> fn package(&mut self, pkgname: &str, dst: &mut BTreeMap<String, Package>,