Change:
I was trying to determine if sccache could cache crates built as dependencies of different root crates. When I looked into it I found that the rlibs were not bit-identical despite being built from the same source. Most of the differences were in the debug info because the current directory winds up encoded there. I tried using which removed most of the differences but not all. A small difference in the metadata remains, which from looking at the diff of the serialized metadata said was the . do you know why this would be? I built this twice in different directories using the script. The diff of the rlib from each build (I generated that using .) Presumably this would block and also
Lots of stuff goes into the SVH. It might well be that something directory related accidentally goes into it. The SVH is computed here:
cc This is similar to the GCC build-path issue where other programs (and in the past, GCC's DWATproducer) likes to embed CFLAGS into the output. Suggested options: out -Zremap-path-prefix away from that SVH calculation - but other programs that also embed RUSTFLAGS or similar, would also need similar logic the envvar or something similar cargo support a constant flag (whose value does not depend on build-path) that figures out the right -Zremap-path-prefix for rustc so they don't need to appear in RUSTFLAGS. you'd also need to implement (1) but at least the caveat (about other programs) doesn't apply.
addresses (1), which is kind of orthogonal to (2) and (3) because the SVH will always be part of the output binary and it must change if the result of the mapping changes.
With the latest Rust nightly: which includes the fix from , the outputs from are still not the same:
I think we may still be hashing arguments?
Yes, it seems we do: Good catch! Not sure if we have to hash those at this point. The tracking system should catch resulting changes later on, if we don't have any holes in the tracking. This needs some review.
Edit: never mind - seems fixed now that I've rebased to master. I found that (after implementing it) still affects the SVH, even if the mapping does nothing: will generate different metadata hashes if I change the mapping, even though it doesn't affect anything and nothing else changes.

Discussion:
lint_cap: Option<lint::Level> [TRACKED], describe_lints: bool [UNTRACKED], output_types: OutputTypes [TRACKED], <del> // FIXME(mw): We track this for now but it actually doesn't make too //            much sense: The search path can stay the same while the //            things discovered there might have changed on disk. search_paths: SearchPaths [TRACKED], </del> <ins> search_paths: SearchPaths [UNTRACKED], </ins> libs: Vec<(String, Option<String>, Option<cstore::NativeLibraryKind>)> [TRACKED], maybe_sysroot: Option<PathBuf> [TRACKED],

// version of `debugging_opts.borrowck`, which is just a plain string. borrowck_mode: BorrowckMode [UNTRACKED], cg: CodegenOptions [TRACKED], <del> // FIXME(mw): We track this for now but it actually doesn't make too //            much sense: The value of this option can stay the same //            while the files they refer to might have changed on disk. externs: Externs [TRACKED], </del> <ins> externs: Externs [UNTRACKED], </ins> crate_name: Option<String> [TRACKED], // An optional name to use as the crate for std during std injection, // written `extern crate std = "name"`. Default to "std". Used by

mod dep_tracking { use lint; use middle::cstore; <del> use session::search_paths::{PathKind, SearchPaths}; </del> use std::collections::BTreeMap; use std::hash::Hash; use std::path::PathBuf; use std::collections::hash_map::DefaultHasher; use super::{Passes, CrateType, OptLevel, DebugInfoLevel, Lto, <del> OutputTypes, Externs, ErrorOutputType, Sanitizer, Epoch}; </del> <ins> OutputTypes, ErrorOutputType, Sanitizer, Epoch}; </ins> use syntax::feature_gate::UnstableFeatures; use rustc_back::{PanicStrategy, RelroLevel};

impl_dep_tracking_hash_via_hash!(Lto); impl_dep_tracking_hash_via_hash!(DebugInfoLevel); impl_dep_tracking_hash_via_hash!(UnstableFeatures); <del> impl_dep_tracking_hash_via_hash!(Externs); </del> impl_dep_tracking_hash_via_hash!(OutputTypes); impl_dep_tracking_hash_via_hash!(cstore::NativeLibraryKind); impl_dep_tracking_hash_via_hash!(Sanitizer);

impl_dep_tracking_hash_for_sortable_vec_of!((String, Option<String>, Option<cstore::NativeLibraryKind>)); impl_dep_tracking_hash_for_sortable_vec_of!((String, u64)); <del> impl DepTrackingHash for SearchPaths { fn hash(&self, hasher: &mut DefaultHasher, _: ErrorOutputType) { let mut elems: Vec<_> = self .iter(PathKind::All) .collect(); elems.sort(); Hash::hash(&elems, hasher); } } </del> impl<T1, T2> DepTrackingHash for (T1, T2) where T1: DepTrackingHash,

} #[test] <del> fn test_externs_tracking_hash_different_values() { let mut v1 = super::basic_options(); let mut v2 = super::basic_options(); let mut v3 = super::basic_options(); v1.externs = Externs::new(mk_map(vec![ (String::from("a"), mk_set(vec![String::from("b"), String::from("c")])), (String::from("d"), mk_set(vec![String::from("e"), String::from("f")])), ])); v2.externs = Externs::new(mk_map(vec![ (String::from("a"), mk_set(vec![String::from("b"), String::from("c")])), (String::from("X"), mk_set(vec![String::from("e"), String::from("f")])), ])); v3.externs = Externs::new(mk_map(vec![ (String::from("a"), mk_set(vec![String::from("b"), String::from("c")])), (String::from("d"), mk_set(vec![String::from("X"), String::from("f")])), ])); assert!(v1.dep_tracking_hash() != v2.dep_tracking_hash()); assert!(v1.dep_tracking_hash() != v3.dep_tracking_hash()); assert!(v2.dep_tracking_hash() != v3.dep_tracking_hash()); // Check clone assert_eq!(v1.dep_tracking_hash(), v1.clone().dep_tracking_hash()); assert_eq!(v2.dep_tracking_hash(), v2.clone().dep_tracking_hash()); assert_eq!(v3.dep_tracking_hash(), v3.clone().dep_tracking_hash()); } #[test] </del> fn test_externs_tracking_hash_different_construction_order() { let mut v1 = super::basic_options(); let mut v2 = super::basic_options();

} #[test] <del> fn test_search_paths_tracking_hash_different_values() { let mut v1 = super::basic_options(); let mut v2 = super::basic_options(); let mut v3 = super::basic_options(); let mut v4 = super::basic_options(); let mut v5 = super::basic_options(); // Reference v1.search_paths.add_path("native=abc", super::ErrorOutputType::Json(false)); v1.search_paths.add_path("crate=def", super::ErrorOutputType::Json(false)); v1.search_paths.add_path("dependency=ghi", super::ErrorOutputType::Json(false)); v1.search_paths.add_path("framework=jkl", super::ErrorOutputType::Json(false)); v1.search_paths.add_path("all=mno", super::ErrorOutputType::Json(false)); // Native changed v2.search_paths.add_path("native=XXX", super::ErrorOutputType::Json(false)); v2.search_paths.add_path("crate=def", super::ErrorOutputType::Json(false)); v2.search_paths.add_path("dependency=ghi", super::ErrorOutputType::Json(false)); v2.search_paths.add_path("framework=jkl", super::ErrorOutputType::Json(false)); v2.search_paths.add_path("all=mno", super::ErrorOutputType::Json(false)); // Crate changed v2.search_paths.add_path("native=abc", super::ErrorOutputType::Json(false)); v2.search_paths.add_path("crate=XXX", super::ErrorOutputType::Json(false)); v2.search_paths.add_path("dependency=ghi", super::ErrorOutputType::Json(false)); v2.search_paths.add_path("framework=jkl", super::ErrorOutputType::Json(false)); v2.search_paths.add_path("all=mno", super::ErrorOutputType::Json(false)); // Dependency changed v3.search_paths.add_path("native=abc", super::ErrorOutputType::Json(false)); v3.search_paths.add_path("crate=def", super::ErrorOutputType::Json(false)); v3.search_paths.add_path("dependency=XXX", super::ErrorOutputType::Json(false)); v3.search_paths.add_path("framework=jkl", super::ErrorOutputType::Json(false)); v3.search_paths.add_path("all=mno", super::ErrorOutputType::Json(false)); // Framework changed v4.search_paths.add_path("native=abc", super::ErrorOutputType::Json(false)); v4.search_paths.add_path("crate=def", super::ErrorOutputType::Json(false)); v4.search_paths.add_path("dependency=ghi", super::ErrorOutputType::Json(false)); v4.search_paths.add_path("framework=XXX", super::ErrorOutputType::Json(false)); v4.search_paths.add_path("all=mno", super::ErrorOutputType::Json(false)); // All changed v5.search_paths.add_path("native=abc", super::ErrorOutputType::Json(false)); v5.search_paths.add_path("crate=def", super::ErrorOutputType::Json(false)); v5.search_paths.add_path("dependency=ghi", super::ErrorOutputType::Json(false)); v5.search_paths.add_path("framework=jkl", super::ErrorOutputType::Json(false)); v5.search_paths.add_path("all=XXX", super::ErrorOutputType::Json(false)); assert!(v1.dep_tracking_hash() != v2.dep_tracking_hash()); assert!(v1.dep_tracking_hash() != v3.dep_tracking_hash()); assert!(v1.dep_tracking_hash() != v4.dep_tracking_hash()); assert!(v1.dep_tracking_hash() != v5.dep_tracking_hash()); // Check clone assert_eq!(v1.dep_tracking_hash(), v1.clone().dep_tracking_hash()); assert_eq!(v2.dep_tracking_hash(), v2.clone().dep_tracking_hash()); assert_eq!(v3.dep_tracking_hash(), v3.clone().dep_tracking_hash()); assert_eq!(v4.dep_tracking_hash(), v4.clone().dep_tracking_hash()); assert_eq!(v5.dep_tracking_hash(), v5.clone().dep_tracking_hash()); } #[test] </del> fn test_search_paths_tracking_hash_different_order() { let mut v1 = super::basic_options(); let mut v2 = super::basic_options();

-include ../tools.mk <del> all: </del> <ins> all:   smoke  debug  opt  link_paths  remap_paths  different_source_dirs  extern_flags smoke: rm -rf $(TMPDIR) && mkdir $(TMPDIR) </ins> $(RUSTC) reproducible-build-aux.rs $(RUSTC) reproducible-build.rs -o"$(TMPDIR)/reproducible-build1" $(RUSTC) reproducible-build.rs -o"$(TMPDIR)/reproducible-build2" <ins> $(B2) </ins> nm "$(TMPDIR)/reproducible-build1" | sort > "$(TMPDIR)/reproducible-build1.nm" nm "$(TMPDIR)/reproducible-build2" | sort > "$(TMPDIR)/reproducible-build2.nm" cmp "$(TMPDIR)/reproducible-build1.nm" "$(TMPDIR)/reproducible-build2.nm" || exit 1 <ins> debug: rm -rf $(TMPDIR) && mkdir $(TMPDIR) </ins> $(RUSTC) reproducible-build-aux.rs -g <del> $(RUSTC) reproducible-build.rs -g -o"$(TMPDIR)/reproducible-build1-debug" $(RUSTC) reproducible-build.rs -g -o"$(TMPDIR)/reproducible-build2-debug" nm "$(TMPDIR)/reproducible-build1-debug" | sort > "$(TMPDIR)/reproducible-build1-debug.nm" nm "$(TMPDIR)/reproducible-build2-debug" | sort > "$(TMPDIR)/reproducible-build2-debug.nm" </del> <ins> $(RUSTC) reproducible-build.rs -o"$(TMPDIR)/reproducible-build1" -g $(RUSTC) reproducible-build.rs -o"$(TMPDIR)/reproducible-build2" -g nm "$(TMPDIR)/reproducible-build1" | sort > "$(TMPDIR)/reproducible-build1-debug.nm" nm "$(TMPDIR)/reproducible-build2" | sort > "$(TMPDIR)/reproducible-build2-debug.nm" </ins> cmp "$(TMPDIR)/reproducible-build1-debug.nm" "$(TMPDIR)/reproducible-build2-debug.nm" || exit 1 <ins> opt: rm -rf $(TMPDIR) && mkdir $(TMPDIR) </ins> $(RUSTC) reproducible-build-aux.rs -O <del> $(RUSTC) reproducible-build.rs -O -o"$(TMPDIR)/reproducible-build1-opt" $(RUSTC) reproducible-build.rs -O -o"$(TMPDIR)/reproducible-build2-opt" nm "$(TMPDIR)/reproducible-build1-opt" | sort > "$(TMPDIR)/reproducible-build1-opt.nm" nm "$(TMPDIR)/reproducible-build2-opt" | sort > "$(TMPDIR)/reproducible-build2-opt.nm" </del> <ins> $(RUSTC) reproducible-build.rs -o"$(TMPDIR)/reproducible-build1" -O $(RUSTC) reproducible-build.rs -o"$(TMPDIR)/reproducible-build2" -O nm "$(TMPDIR)/reproducible-build1" | sort > "$(TMPDIR)/reproducible-build1-opt.nm" nm "$(TMPDIR)/reproducible-build2" | sort > "$(TMPDIR)/reproducible-build2-opt.nm" </ins> cmp "$(TMPDIR)/reproducible-build1-opt.nm" "$(TMPDIR)/reproducible-build2-opt.nm" || exit 1 <ins> link_paths: rm -rf $(TMPDIR) && mkdir $(TMPDIR) $(RUSTC) reproducible-build-aux.rs $(RUSTC) reproducible-build.rs --crate-type rlib -L /b cp $(TMPDIR)/libreproducible_build.rlib $(TMPDIR)/libfoo.rlib $(RUSTC) reproducible-build.rs --crate-type rlib -L /a cmp "$(TMPDIR)/libreproducible_build.rlib" "$(TMPDIR)/libfoo.rlib" || exit 1 remap_paths: rm -rf $(TMPDIR) && mkdir $(TMPDIR) $(RUSTC) reproducible-build-aux.rs $(RUSTC) reproducible-build.rs --crate-type rlib --remap-path-prefix=/a=/c cp $(TMPDIR)/libreproducible_build.rlib $(TMPDIR)/libfoo.rlib $(RUSTC) reproducible-build.rs --crate-type rlib --remap-path-prefix=/b=/c cmp "$(TMPDIR)/libreproducible_build.rlib" "$(TMPDIR)/libfoo.rlib" || exit 1 different_source_dirs: rm -rf $(TMPDIR) && mkdir $(TMPDIR) $(RUSTC) reproducible-build-aux.rs mkdir $(TMPDIR)/test cp reproducible-build.rs $(TMPDIR)/test $(RUSTC) reproducible-build.rs --crate-type rlib --remap-path-prefix=$$PWD=/b cp $(TMPDIR)/libreproducible_build.rlib $(TMPDIR)/libfoo.rlib (cd $(TMPDIR)/test && $(RUSTC) reproducible-build.rs  --remap-path-prefix=$(TMPDIR)/test=/b  --crate-type rlib) cmp "$(TMPDIR)/libreproducible_build.rlib" "$(TMPDIR)/libfoo.rlib" || exit 1 extern_flags: rm -rf $(TMPDIR) && mkdir $(TMPDIR) $(RUSTC) reproducible-build-aux.rs $(RUSTC) reproducible-build.rs  --extern reproducible_build_aux=$(TMPDIR)/libreproducible_build_aux.rlib  --crate-type rlib cp $(TMPDIR)/libreproducible_build_aux.rlib $(TMPDIR)/libbar.rlib cp $(TMPDIR)/libreproducible_build.rlib $(TMPDIR)/libfoo.rlib $(RUSTC) reproducible-build.rs  --extern reproducible_build_aux=$(TMPDIR)/libbar.rlib  --crate-type rlib cmp "$(TMPDIR)/libreproducible_build.rlib" "$(TMPDIR)/libfoo.rlib" || exit 1 </ins>

// - Trait object shims // - Fn Pointer shims <del> #![allow(dead_code)] </del> <ins> #![allow(dead_code, warnings)] </ins> extern crate reproducible_build_aux;