Change:
When compiletest fails, it prints an error summary like this at the end: However, there is one very important bit of information that I cannot gleam from this information: what are the host and target architectures that were tested! Figuring this out requires scrolling up quite a bit to where this test beings (and carefully not scrolling up too far because then one might look at the wrong test). See for an example where this confused people. I have also been confused by this multiple times in the past, e.g. using the name of the CI runner to incorrectly deduce what the architecture(s) are (and not realizing that some of our runners run tests for quite a few different targets). So I think in case of a failure, that summary at the end should repeat the important information that is printed at the top of the test run:
Cc
And also it would be nice if when it ignored a test because it was the wrong platform, it showed some indication of that, because otherwise it looks like it ran the test and it passed. Perhaps show a question mark ()? (We already use for tests that were ignored because they were already run and passed.)
AFAIK should already be used for all kinds of "ignored", including directives.
Turns out these messages are printed by bootstrap, not compiletest: However, detecting failures and exiting is done a few layers deeper: In case of failure, never returns or unwinds (except if fail-fast is off). I am not sure how to best adjust this to print an error before exiting (or to always print an error after the process finished, whether it succeeded or failed). any advice? If we just want this to happen when fail-fast is off, that would be easy, but I am not sure if that ever applies.

Discussion:
cmd.arg("--src-base").arg(builder.src.join("src/test").join(suite)); cmd.arg("--build-base").arg(testdir(builder, compiler.host).join(suite)); cmd.arg("--stage-id").arg(format!("stage{}-{}", compiler.stage, target)); <ins> cmd.arg("--suite").arg(suite); </ins> cmd.arg("--mode").arg(mode); cmd.arg("--target").arg(target.rustc_target_arg()); cmd.arg("--host").arg(&*compiler.host.triple);

/// The test mode, compile-fail, run-fail, ui pub mode: Mode, <ins> /// The test suite (essentially which directory is running, but without the /// directory prefix such as src/test) pub suite: String, </ins> /// The debugger to use in debuginfo mode. Unset otherwise. pub debugger: Option<Debugger>,

let args = &[ "compiletest", "--mode=ui", <ins> "--suite=ui", </ins> "--compile-lib-path=", "--run-lib-path=", "--rustc-path=",

"compile-fail | run-fail | run-pass-valgrind | pretty | debug-info | codegen | rustdoc  codegen-units | incremental | run-make | ui | js-doc-test | mir-opt | assembly", ) <ins> .reqopt( "", "suite", "which suite of compile tests to run. used for nicer error reporting.", "SUITE", ) </ins> .optopt( "", "pass",

build_base: opt_path(matches, "build-base"), stage_id: matches.opt_str("stage-id").unwrap(), mode: matches.opt_str("mode").unwrap().parse().expect("invalid mode"), <ins> suite: matches.opt_str("suite").unwrap(), </ins> debugger: None, run_ignored, filter: matches.free.first().cloned(),

configs.extend(configure_lldb(&config)); } } else { <del> configs.push(config); </del> <ins> configs.push(config.clone()); </ins> }; let mut tests = Vec::new();

let res = test::run_tests_console(&opts, tests); match res { Ok(true) => {} <del> Ok(false) => panic!("Some tests failed"), </del> <ins> Ok(false) => { // We want to report that the tests failed, but we also want to give // some indication of just what tests we were running. Especially on // CI, where there can be cross-compiled tests for a lot of // architectures, without this critical information it can be quite // easy to miss which tests failed, and as such fail to reproduce // the failure locally. eprintln!( "Some tests failed in compiletest suite={}{} mode={} host={} target={}", config.suite, config.compare_mode.map(|c| format!(" compare_mode={:?}", c)).unwrap_or_default(), config.mode, config.host, config.target ); std::process::exit(1); } </ins> Err(e) => { // We don't know if tests passed or not, but if there was an error // during testing we don't want to just suceeed (we may not have // tested something), so fail. <ins> // // This should realistically "never" happen, so don't try to make // this a pretty error message. </ins> panic!("I/O failure during tests: {:?}", e); } }