Change:
A while ago we stopped getting full re-use when re-compiling an unchanged code base in the rust-icci tests: https://travis- https://travis- https://travis- The regression seems to have been introduced in this commit range, which includes the new graph reduction algorithm: cc
Hmm, I am trying to reproduce this and so far I'm not succeeding. I see full re-use for at the given commit, at least, when using either the latest nightly or my local build (which is at the commit ). I'll take a look at the others.
One thing is that I am not seeing as many modules as the logs report. Perhaps I am not using the right set of switches.
The problem seems to be specific to crates that contain only generic items. In these cases, we get an empty serialized dep-graph. I think that, in the new representation, if there are literally no edges then we wind up never created the work-product nodes from last time, and hence we conclude they are dirty. Roughly speaking the problem is that the second rustc run gets no re-use here:
Fix here:

Discussion:
pub struct SerializedDepGraph { pub edges: Vec<SerializedEdgeSet>, <ins> /// These are output nodes that have no incoming edges. We track /// these separately so that when we reload all edges, we don't /// lose track of these nodes. pub bootstrap_outputs: Vec<DepNode<DefPathIndex>>, </ins> /// These are hashes of two things: /// - the HIR nodes in this crate /// - the metadata nodes from dependent crates we use

} } <ins> // Recreate bootstrap outputs, which are outputs that have no incoming edges (and hence cannot // be dirty). for bootstrap_output in &serialized_dep_graph.bootstrap_outputs { if let Some(n) = retraced.map(bootstrap_output) { if let DepNode::WorkProduct(ref wp) = n { clean_work_products.insert(wp.clone()); } tcx.dep_graph.with_task(n, || ()); // create the node with no inputs } } </ins> // Subtle. Sometimes we have intermediate nodes that we can't recreate in the new graph. // This is pretty unusual but it arises in a scenario like this: //

use rustc::dep_graph::{DepGraphQuery, DepNode}; use rustc::hir::def_id::DefId; use rustc_data_structures::fx::FxHashMap; <del> use rustc_data_structures::graph::Graph; </del> <ins> use rustc_data_structures::graph::{Graph, NodeIndex}; </ins> use super::hash::*; use ich::Fingerprint;

// of the graph down. pub reduced_graph: Graph<&'query DepNode<DefId>, ()>, <ins> // These are output nodes that have no incoming edges. We have to // track these specially because, when we load the data back up // again, we want to make sure and recreate these nodes (we want // to recreate the nodes where all incoming edges are clean; but // since we ordinarily just serialize edges, we wind up just // forgetting that bootstrap outputs even exist in that case.) pub bootstrap_outputs: Vec<&'query DepNode<DefId>>, </ins> // For the inputs (hir/foreign-metadata), we include hashes. pub hashes: FxHashMap<&'query DepNode<DefId>, Fingerprint>, }

// Reduce the graph to the most important nodes. let compress::Reduction { graph, input_nodes } = <del> compress::reduce_graph(&query.graph, HashContext::is_hashable, is_output); </del> <ins> compress::reduce_graph(&query.graph, HashContext::is_hashable, |n| is_output(n)); </ins> let mut hashes = FxHashMap(); for input_index in input_nodes {

.or_insert_with(|| hcx.hash(input).unwrap()); } <ins> let bootstrap_outputs: Vec<&'q DepNode<DefId>> = (0 .. graph.len_nodes()) .map(NodeIndex) .filter(|&n| graph.incoming_edges(n).next().is_none()) .map(|n| *graph.node_data(n)) .filter(|n| is_output(n)) .collect(); </ins> Predecessors { reduced_graph: graph, <ins> bootstrap_outputs: bootstrap_outputs, </ins> hashes: hashes, } }

} // Create the serialized dep-graph. <ins> let bootstrap_outputs = preds.bootstrap_outputs.iter() .map(|n| builder.map(n)) .collect(); </ins> let edges = edges.into_iter() .map(|(k, v)| SerializedEdgeSet { source: k, targets: v }) .collect(); let graph = SerializedDepGraph { <del> edges: edges, </del> <ins> bootstrap_outputs, edges, </ins> hashes: preds.hashes .iter() .map(|(&dep_node, &hash)| {

}; if tcx.sess.opts.debugging_opts.incremental_info { <ins> println!("incremental: {} nodes in reduced dep-graph", preds.reduced_graph.len_nodes()); </ins> println!("incremental: {} edges in serialized dep-graph", graph.edges.len()); println!("incremental: {} hashes in serialized dep-graph", graph.hashes.len()); }

<ins> // Copyright 2014 The Rust Project Developers. See the COPYRIGHT // file at the top-level directory of this distribution and at // http://rust-lang.org/COPYRIGHT. // // Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or // http://www.apache.org/licenses/LICENSE-2.0> or the MIT license // <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your // option. This file may not be copied, modified, or distributed // except according to those terms. // revisions:rpass1 rpass2 // compile-flags: -Z query-dep-graph #![rustc_partition_reused(module="__rustc_fallback_codegen_unit", cfg="rpass2")] #![feature(rustc_attrs)] #![crate_type="rlib"] pub fn foo<T>() { } </ins>

<ins> // Copyright 2014 The Rust Project Developers. See the COPYRIGHT // file at the top-level directory of this distribution and at // http://rust-lang.org/COPYRIGHT. // // Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or // http://www.apache.org/licenses/LICENSE-2.0> or the MIT license // <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your // option. This file may not be copied, modified, or distributed // except according to those terms. // Regression test for #39828. If you make use of a module that // consists only of generics, no code is generated, just a dummy // module. The reduced graph consists of a single node (for that // module) with no inputs. Since we only serialize edges, when we // reload, we would consider that node dirty since it is not recreated // (it is not the target of any edges). // revisions:rpass1 rpass2 // aux-build:generic.rs extern crate generic; fn main() { } </ins>