Code change:
I was sloppy when I implemented and did not include a test. So, surprise, a follow-on commit a few days later broke it. As self-punishment for neglecting to include a regression test, I am assigning myself the task of reviewing our test suite to make sure that every feature-gate has a test. Here is a transcribed list of feature gates, based on , that I have annotated according to whether the feature is Accepted/Removed (and thus needs no tests), Active/Deprecated but has tests already (marked with an , or Active/Deprecated but has no tests that I saw via a cursory grep/skim. - , - , - , [x] - ("asm", "1.0.0", Active), - , [x] - ("nonasciiidents", "1.0.0", Active), [x] - ("threadlocal", "1.0.0", Active), [x] - ("linkargs", "1.0.0", Active), - , [x] - ("pluginregistrar", "1.0.0", Active), but it is not clear whether this is testing the gate or the gate. From the error message it seems like the gate is firing. [x] - ("logsyntax", "1.0.0", Active), [x] - ("tracemacros", "1.0.0", Active), [x] - ("concatidents", "1.0.0", Active), [x] - ("unsafedestructor", "1.0.0", Active), [x] - ("intrinsics", "1.0.0", Active), [x] - ("langitems", "1.0.0", Active), [x] - ("simd", "1.0.0", Active), - , [x] - ("quote", "1.0.0", Active), [x] - ("linkllvmintrinsics", "1.0.0", Active), [x] - ("linkage", "1.0.0", Active), - , - , [ ] - ("rustcdiagnosticmacros", "1.0.0", Active), [ ] - ("unboxedclosures", "1.0.0", Active), - , [ ] - ("advancedslicepatterns", "1.0.0", Active), - , - , [ ] - ("visibleprivatetypes", "1.0.0", Active), - , [x] - ("boxsyntax", "1.0.0", Active), gated-box- [ ] - ("onunimplemented", "1.0.0", Active), [x] - ("simdffi", "1.0.0", Active), gated-simd- - , - , [ ] - ("plugin", "1.0.0", Active), but it is not clear whether this is testing the gate or the gate [ ] - ("start", "1.0.0", Active), [ ] - ("main", "1.0.0", Active), - , - , [ ] - ("oldorphancheck", "1.0.0", Deprecated), [ ] - ("oldimplcheck", "1.0.0", Deprecated), [ ] - ("optinbuiltintraits", "1.0.0", Active), [x] - ("intuint", "1.0.0", Active), [x] - ("macroreexport", "1.0.0", Active), gated-macro- - , - , [x] - ("stagedapi", "1.0.0", Active), [ ] - ("unmarkedapi", "1.0.0", Active), [x] - ("nostd", "1.0.0", Active), [x] - ("boxpatterns", "1.0.0", Active), gated-box- [x] - ("unsafenodropflag", "1.0.0", Active), [x] - ("customattribute", "1.0.0", Active), [x] - ("customderive", "1.0.0", Active), single-derive- [ ] - ("rustcattrs", "1.0.0", Active), [x] - ("staticassert", "1.0.0", Active), [x] - ("allowinternal_unstable", "1.0.0", Active),
(but hey, if other people want to add such missing tests in the meantime, while I am asleep, I will not mind. :)
In case its not clear: If you want to help, just find a case above that does not have a check-mark and also does not have a pull request number attached beneath it. (And I guess write your name so that other people do not waste time adding a test for the same feature gate).
The test for is located in
I am not surprised that I "overlooked" that ... the expected error message prefix does not say anything about a feature gate. Let me just confirm that the whole message does ... Update: Okay, the help output for that test () does mention the feature gate: So I have now checked off that box, though I wonder if I should add this help message to the expected test output there.
I was wondering the same thing about the help message. It is not very clear what that test is doing, the only hint was the filename. From reading some help messages were removed, would there be any concern for adding help messages to the expected test output in this case? It could be worth adding a comment in there similar to the other gated tests: In addition, would it be worth going as far as to rename to (and doing the same for any other gated tests that do not have a prefix in their filename)?
In relation to the feature gate list above I'm going to start off by looking into the following: update: removed items that had tests, and , - - -
ah yes, is (correctly) why the help mag is missing .... hmm .... that complicates things
I've done a pass through of the unchecked feature gates and have found tests for the following: () () () () () () Could these be marked as complete?
thanks!
Ok, so I've had a look through everything, here's my findings: I have some questions below for the three outstanding feature gates (at the very bottom of this comment) and was wondering if you could help confirm the status of those feature gate tests? My previous listed feature gates that had existing tests. I've since found a few more to add to that list. Here is the full list of feature gates with existing tests. New findings are in bold: : : : : : : : : : The following feature gates were removed after this issue was initially created. Since they are no longer around they can be considered complete or crossed out of the task list above: : see : see The following feature gates appear to have duplicate tests introduced in , the initial tests were in and : - The following are not in the above task list as they have been since the issue was initially created: : see : see The following are remaining feature gates needing tests (excluding the feature gates I had questions for, see below). I'm cooking up a PR for these: - - - The following items are ones that I had trouble completing because I have questions and need some feedback: - The was introduced in , and tests for were replaced with tests for missing stability attributes: . In addition, it seems that there is no way to trigger unless there is a bug in a library or the compiler itself. Are the tests in missing- enough to consider complete? These appear to be the we're looking for relating to , could someone confirm this? This appears to be the feature gate test, but was removed in () Should we reinstate the test in this issue or let it be handled by which is in the works?
For visibleprivatetypes, i found src/test/run-pass/visible-private-types-feature- Not sure if item this should be ticked as well.
There appear to be a lot of new unstable features since this was touched. It seems hopeless to ever catch up on this issue without an automatic tidy script verifying the tests exist. do you want this issue to continue? Should we try to augment tidy to check the existence of compile-fail tests mentioning unstable features?
I agree that it seems like we'll never be able to close this without having some sort of automation (e.g. a tidy script, as you say) to verify that the tests exist. I don't have any immediate idea about the best way to write such a script. I guess it could just be based on some naming convention for the test itself; i.e. it would be just a heuristic check, not a true verification of it.
I think it could be written like so: after , search everything in for . If this makes sense I can mentor.
but requiring the test to explicitly deny the feature defeats the point, doesn't it? That is, the goal here is to catch code that makes use of a feature without explicitly opting into that feature. .. Though I suppose you might be thinking that if there is enough infrastructure present to support some kind of deny(feature) attribute, then that is a sign that the necessary gating is in place, regardless of what the default lint settings actually are?
Good point. I think yes, I was considering that just confirming that the compiler has code that correctly can deny the feature is good enough, but there is some room for error by making the deny explicit. The attribute also wouldn't actually confirm that the compile failed for the correct reason. We could also just require tests to annotate in some other way that yes, it is testing that a particular feature is denied. As an extension to my previous suggestion, the test runner could take the test case that is annotated to be a 'deny feature' test, run it once with then run it again with no feature attribute, and compare the results.
Similarly to the above, but to avoid creating another custom test runner. The featureck lint could require that and contain pairs of feature tests that are identical, one with and one without.
With fixed, maybe this issue can be closed as resolved?
Thanks

Related discussion:
text: <ins> // Copyright 2015 The Rust Project Developers. See the COPYRIGHT // file at the top-level directory of this distribution and at // http://rust-lang.org/COPYRIGHT. // // Licensed under the Apache License, Version 2.0 <LICENSE-APACHE or // http://www.apache.org/licenses/LICENSE-2.0> or the MIT license // <LICENSE-MIT or http://opensource.org/licenses/MIT>, at your // option. This file may not be copied, modified, or distributed // except according to those terms. // Test that `quote`-related macro are gated by `quote` feature gate. // (To sanity-check the code, uncomment this.) // #![feature(quote)] // FIXME the error message that is current emitted seems pretty bad. #![feature(rustc_private)] #![allow(dead_code, unused_imports, unused_variables)] #[macro_use] extern crate syntax; use syntax::ast; use syntax::codemap::Span; use syntax::parse; struct ParseSess; impl ParseSess { fn cfg(&self) -> ast::CrateConfig { loop { } } fn parse_sess<'a>(&'a self) -> &'a parse::ParseSess { loop { } } fn call_site(&self) -> Span { loop { } } fn ident_of(&self, st: &str) -> ast::Ident { loop { } } fn name_of(&self, st: &str) -> ast::Name { loop { } } } pub fn main() { let ecx = &ParseSess; let x = quote_tokens!(ecx, 3);   //~ ERROR macro undefined: 'quote_tokens!' let x = quote_expr!(ecx, 3);     //~ ERROR macro undefined: 'quote_expr!' let x = quote_ty!(ecx, 3);       //~ ERROR macro undefined: 'quote_ty!' let x = quote_method!(ecx, 3);   //~ ERROR macro undefined: 'quote_method!' let x = quote_item!(ecx, 3);     //~ ERROR macro undefined: 'quote_item!' let x = quote_pat!(ecx, 3);      //~ ERROR macro undefined: 'quote_pat!' let x = quote_arm!(ecx, 3);      //~ ERROR macro undefined: 'quote_arm!' let x = quote_stmt!(ecx, 3);     //~ ERROR macro undefined: 'quote_stmt!' let x = quote_matcher!(ecx, 3);  //~ ERROR macro undefined: 'quote_matcher!' let x = quote_attr!(ecx, 3);     //~ ERROR macro undefined: 'quote_attr!' } </ins>