Repository: eslint/eslint (JavaScript)

Issue:
`fixable` property only necessary when `meta` is present
**What did you expect to happen?**

Per the docs:

> Important: Without the fixable property, ESLint does not apply fixes even if the rule implements fix functions. Omit the fixable property if the rule is not fixable.

Based on this, I would expect a rule without any `meta` at all to also not be fixable.

**What actually happened? Please include the actual, raw output from ESLint.**

A rule without `meta` is successfully fixed when it has a fixer.

The docs are true, however, if there is a `meta` object of some sort (including an empty object); in such a case, an error will be reported that `fixable` is necessary if the rule attempts to add a fixer. But if there is no `meta` object at all, `fixable` is not necessary for fixes to be applied for the rule.

I am not expecting breaking changes to start preventing fixers without `meta`; I'd just expect the docs to mention that rules still without the recommended `meta` property at all are currently still fixable.


Discussion:
@brettz9 Can you fill out an issue template? It's not clear to me what the desired outcome of this issue is. Thanks!
I've updated to include those two headings relevant to a doc bug.
Thank you for clarifying. Agreed, and PRs to improve documentation are always welcome!
> I am not expecting breaking changes to start preventing fixers without meta;

agreed. but shouldn't it?

IMO, the fixer shouldnt work or throw an error if meta is not present. 
cause in rule authoring docs, it is mentioned that we can omit `docs` but it is not mentioned about the `meta`. (or maybe I am missing.)

thoughts? 
otherwise I think `bug` label can be removed then ?
Re: it not mentioning `meta`, yeah, the [Working with Rules](https://eslint.org/docs/developer-guide/working-with-rules) page doesn't mention that `meta` can be omitted.

As far as whether it should be a breaking change to require `meta.fixable` always, I'd personally even be in favor of always requiring `meta` (specifically so as to require `meta.type` even for non-fixable rules--though that is not required now and would perhaps also need an "other" type; without getting too off track, in my eslint [badge-maker](https://github.com/brettz9/eslint-formatter-badger), I felt the need to let users override the built-in `type`'s as they are not very specific (e.g., as to whether the "problem" is for security, preventing obtrusiveness, etc); I could perhaps use `docs.category` for that instead, but these are not enumerated in the docs nor encouraged for third-party use).

However, I have come across quite a number of plugins using the legacy format or not using `meta` if they are, so I would expect people might worry about breaking too many old plugins. Might be good to mention it may become required in a future version, both to encourage `meta` (and `meta.type`) in case a breaking change is ever seen as feasible and desirable.
Hey guys I am looking to contribute to ESLint. I would like to work on this issue however based on what I read, I am not sure if this requires just a documentation fix. Can anyone appropriately guide me to work on it. I have already set up ESLint on my machine.
This is marked as accepted for a documentation update, though I do think this is unclear behavior.  
Hi.  I'm interested in taking this one if no one minds.  Seems a pretty straight forward documentation update.
Is anyone working on this issue? If not, I would not mind working on it.
> I have come across quite a number of plugins using the legacy format or not using `meta`

@brettz9 do you happen to know of any examples that currently rely on this buggy behavior by providing fixers without including `meta`?

I see three options to resolve this:

1. Fix the bug and require `meta`.
1. Document the existing behavior with a warning that we'll fix it in a future major version.
1. Document the existing behavior and live with it even though it's unintended.

Which path I prefer would depend on how much fixing the buggy behavior would break.
Authors I've encountered have thankfully been open to updates, e.g., recently with https://github.com/SonarSource/eslint-plugin-sonarjs/pull/161 . (In the cases where the lack of `meta.fixable` is due to using the old format where there is just a function exported and no `create` as well as no `meta` (i.e., when the old format is in use), authors might see this less as "buggy" and more as merely "deprecated" since those rules predated `meta` entirely, but I get your gist, esp. with the docs urging this now.)

I am waiting on one PR, https://github.com/Turbo87/eslint-plugin-chai-expect/pull/99 , to introduce the new rule format, so I know at least one plugins still using the old format, but in this case, the rules do not happen to be fixable.

If you were only going to fix the case of requiring `meta.fixable` when `create` was present, I would think you might avoid a major bump (assuming the docs when the [new rule format](https://eslint.org/blog/2016/07/eslint-new-rule-format) was introduced had from the beginning added mention of this need).

However, I would think no. 1 on your list is not an option for a major bump when the fixing occurs with the deprecated rule format since that format (which did not even allow `meta` at all despite supporting fixing) is still mentioned as supported (though deprecated) at https://eslint.org/docs/developer-guide/working-with-rules (beginning of page).

While I haven't done any exhaustive study (e.g., of https://github.com/dustinspecker/awesome-eslint/ or those on npm), I would think therefore that with no. 1 excluded, no. 2 could be chosen over no. 3 as:

1. At least the big ones I have used or needed to use with other projects, "standard", "prettier", "promise", "import", "jsdoc", "sonarjs" and other ones of interest to me (e.g., "mocha", "mocha-cleanup", various chai ones) are pretty much all on board already.
2. It's not too big of a hassle to just add a `create` method and if needed add a `fixable`.
3. The blog post introducing the new rule format (at https://eslint.org/blog/2016/07/eslint-new-rule-format ) mentions "knowing that a rule is fixable ahead of time allows us to optimize the autofixing process as well as call this out in documentation" so it should be nice to be able to enforce this benefit (even if there may be some plugins or custom rules in the wild that have yet to make the change).

IMO, it would be nice to also require `meta.type` (and `meta.docs` (`url` and/or `description`) too if the latter were worded less as being ESLint-project-specific). Even for non-fixable rules, a `type` can be used by formatters, so that tools could more readily make use of them. Authors I have encountered have been open to adding the former or have at least not objected for the few PRs I have still waiting merging (I haven't made PRs for `docs`). Don't want to distract with this suggestion, but just mentioning if a breaking change is being made anyways and there were interest.

Because we don't guarantee fixes to be applied, I think we can fix this behavior to work as it was originally intended. The result of using a rule with a legacy fixer is that it will only warn and not fix, which I wouldn't consider a breaking change.

The bigger question is why `RuleTester` isn't testing for this already. That's one of our primary ways of enforcing rule standards and I'm surprised there's not a check in there.

Here's what I would suggest as a path forward:

1. Make an announcement about this explaining that it is a bug and we know it will affect people.
2. Update `RuleTester` to check that `meta.fixable` is present whenever a fix is provided. Release this first without changing how rules actually work.
3. Fix the bug with how rule fixes are being applied in a subsequent release.
If there is consensus on these steps, I could start working on `RuleTester`.
3 of the other 4 TSC members left a :+1: on @nzakas’s comment, and nobody’s objected, so I’d say you’re good to go.
I think the documentation is still incorrect, even if there's `meta` without `fixable`:

> Important: Without the fixable property, ESLint does not apply fixes even if the rule implements fix functions. Omit the fixable property if the rule is not fixable.

> Important: Unless the rule exports the meta.fixable property, ESLint does not apply fixes even if the rule implements fix functions.

It sounds like ESLint will just not apply fixes, but it actually [throws](https://github.com/eslint/eslint/blob/c8f9c8210cf4b9da8f07922093d7b219abad9f10/lib/linter/linter.js#L919) when this happens. This was an intended change (#6043), so perhaps we should update the docs to be more accurate about what will actually happen.

If we want the same behavior in the case when rule that provides fixes doesn't have `meta` (which is what this issue is about), then ESLint should throw in that case as well. Maybe we should add this as step 4 and implement it in the next major version?
> The bigger question is why `RuleTester` isn't testing for this already. That's one of our primary ways of enforcing rule standards and I'm surprised there's not a check in there.

It was decided to do these checks and throw in the linter (#5970), so there was no need to update `RuleTester`.

However, it seems that this scenario (new-format rule without `meta`) was missed from the start. Even prior to the change to throw, fixes were being applied for rules that don't have `meta` in order to support fixable rules that have the legacy-format (rules that export function).

If we still want to allow fixable rules that have the legacy format (I guess we should?), maybe we could add some info about the original format [here](https://github.com/eslint/eslint/blob/c8f9c8210cf4b9da8f07922093d7b219abad9f10/lib/linter/rules.js#L26), and then use it in the ~rule tester (step 2) and~ linter (subsequent steps).

edit: `RuleTester` can see the original (unnormalized) rule.
> If we want the same behavior in the case when rule that provides fixes doesn't have meta (which is what this issue is about), then ESLint should throw in that case as well. Maybe we should add this as step 4 and implement it in the next major version?

Nice catch! I think in this case, we should just make step 3 the final fix that throws an error and implement that in a major release. I don't see a ton of benefit to doing a silent-fail of fixes first and then throwing an error later.
So, the steps would be:

1. Make an announcement.
2. Throw in `RuleTester` (might break plugin builds).
3. Throw in `Linter` (might affect end users if plugin hasn't fixed the problem) - major release.

step 2 is ready in #13489. Should there be some time between step 1 and step 2?

Also to confirm: this doesn't apply to legacy-format rules (those that export a function and thus don't have `meta`)? 
> Also to confirm: this doesn't apply to legacy-format rules (those that export a function and thus don't have meta)?

Per @nzakas in https://github.com/eslint/eslint/issues/13349#issuecomment-646729029

> ...The result of using a rule with a legacy fixer is that it will only warn and not fix, which I wouldn't consider a breaking change.
> The bigger question is why RuleTester isn't testing for this already...

While this is speaking of only warning, it seems to me from this indication of (no longer) fixing, that the idea is to start treating legacy format rules the same, i.e., that all your steps should apply to both the legacy format (functions) and to missing `meta` on objects (as with an object missing `meta.fixable`) and also that another step should be added, presumably preceding your current step 3, that avoids fixing for legacy (function) format.
I interpreted [this comment](https://github.com/eslint/eslint/issues/13349#issuecomment-646729029) differently since it wasn't _originally intended_ to disallow fixable legacy-format rules.

I'm not sure whether we should now break legacy-format rules that are following the spec for that format ([working-with-rules-deprecated](https://github.com/eslint/eslint/blob/master/docs/developer-guide/working-with-rules-deprecated.md)).

@nzakas can you clarify this? I think we all agree to disallow new-format rules that produce fixes but don't export `meta`. The question is what to do with legacy-format rules. 


As clarified in today's TSC meeting, [steps 1-3](https://github.com/eslint/eslint/issues/13349#issuecomment-658417721) will apply to legacy (function) format as well.

Meaning that, as of `v8.0.0`, only rules that export `meta.fixable` (which implies that it's a new-format rule, and that it exports `meta`) can be fixable. In `v8.0.0`, an error will be thrown during linting whenever a rule that doesn't satisfy this condition produces a fix.
> and also that another step should be added, presumably preceding your current step 3, that avoids fixing for legacy (function) format.

I think this isn't necessary, similar to how it seems unnecessary for new-format rules that don't export `meta`.
Added "breaking" label for the remaining step 3.



Patch:
diff --git a/lib/linter/linter.js b/lib/linter/linter.js
index bdc6c1b1d016..f6cfdb4963fa 100644
--- a/lib/linter/linter.js
+++ b/lib/linter/linter.js
@@ -919,8 +919,8 @@ function runRules(sourceCode, configuredRules, ruleMapper, parserOptions, parser
                         }
                         const problem = reportTranslator(...args);
 
-                        if (problem.fix && rule.meta && !rule.meta.fixable) {
-                            throw new Error("Fixable rules should export a `meta.fixable` property.");
+                        if (problem.fix && !(rule.meta && rule.meta.fixable)) {
+                            throw new Error("Fixable rules must set the `meta.fixable` property to \"code\" or \"whitespace\".");
                         }
                         lintingProblems.push(problem);
                     }


Test patch:
diff --git a/lib/rule-tester/rule-tester.js b/lib/rule-tester/rule-tester.js
index b08303c62b26..3627a96eb0b8 100644
--- a/lib/rule-tester/rule-tester.js
+++ b/lib/rule-tester/rule-tester.js
@@ -865,16 +865,6 @@ class RuleTester {
                 );
             }
 
-            // Rules that produce fixes must have `meta.fixable` property.
-            if (result.output !== item.code) {
-                assert.ok(
-                    hasOwnProperty(rule, "meta"),
-                    "Fixable rules should export a `meta.fixable` property."
-                );
-
-                // Linter throws if a rule that produced a fix has `meta` but doesn't have `meta.fixable`.
-            }
-
             assertASTDidntChange(result.beforeAST, result.afterAST);
         }
 
diff --git a/tests/fixtures/rules/make-syntax-error-rule.js b/tests/fixtures/rules/make-syntax-error-rule.js
index 528b4b0f4de9..fce60a56c4c2 100644
--- a/tests/fixtures/rules/make-syntax-error-rule.js
+++ b/tests/fixtures/rules/make-syntax-error-rule.js
@@ -1,14 +1,19 @@
-module.exports = function(context) {
-    return {
-        Program: function(node) {
-            context.report({
-                node: node,
-                message: "ERROR",
-                fix: function(fixer) {
-                    return fixer.insertTextAfter(node, "this is a syntax error.");
-                }
-            });
-        }
-    };
+module.exports = {
+    meta: {
+        schema: [],
+        fixable: "code"
+    },
+    create(context) {
+        return {
+            Program: function(node) {
+                context.report({
+                    node: node,
+                    message: "ERROR",
+                    fix: function(fixer) {
+                        return fixer.insertTextAfter(node, "this is a syntax error.");
+                    }
+                });
+            }
+        };
+    }
 };
-module.exports.schema = [];
diff --git a/tests/fixtures/testers/rule-tester/fixes-one-problem.js b/tests/fixtures/testers/rule-tester/fixes-one-problem.js
index 9ed9ca71cbdc..e692cae33864 100644
--- a/tests/fixtures/testers/rule-tester/fixes-one-problem.js
+++ b/tests/fixtures/testers/rule-tester/fixes-one-problem.js
@@ -5,19 +5,24 @@
 
 "use strict";
 
-module.exports = context => {
-    return {
-        Program(node) {
-            context.report({
-                node,
-                message: "No programs allowed."
-            });
+module.exports = {
+    meta: {
+        fixable: "code"
+    },
+    create(context) {
+        return {
+            Program(node) {
+                context.report({
+                    node,
+                    message: "No programs allowed."
+                });
 
-            context.report({
-                node,
-                message: "Seriously, no programs allowed.",
-                fix: fixer => fixer.remove(node)
-            });
+                context.report({
+                    node,
+                    message: "Seriously, no programs allowed.",
+                    fix: fixer => fixer.remove(node)
+                });
+            }
         }
     }
 };
diff --git a/tests/lib/linter/linter.js b/tests/lib/linter/linter.js
index 976bd7657551..0eb0970216f0 100644
--- a/tests/lib/linter/linter.js
+++ b/tests/lib/linter/linter.js
@@ -4934,17 +4934,24 @@ var a = "test2";
             it("should use postprocessed problem ranges when applying autofixes", () => {
                 const code = "foo bar baz";
 
-                linter.defineRule("capitalize-identifiers", context => ({
-                    Identifier(node) {
-                        if (node.name !== node.name.toUpperCase()) {
-                            context.report({
-                                node,
-                                message: "Capitalize this identifier",
-                                fix: fixer => fixer.replaceText(node, node.name.toUpperCase())
-                            });
-                        }
+                linter.defineRule("capitalize-identifiers", {
+                    meta: {
+                        fixable: "code"
+                    },
+                    create(context) {
+                        return {
+                            Identifier(node) {
+                                if (node.name !== node.name.toUpperCase()) {
+                                    context.report({
+                                        node,
+                                        message: "Capitalize this identifier",
+                                        fix: fixer => fixer.replaceText(node, node.name.toUpperCase())
+                                    });
+                                }
+                            }
+                        };
                     }
-                }));
+                });
 
                 const fixResult = linter.verifyAndFix(
                     code,
@@ -5033,15 +5040,23 @@ var a = "test2";
         });
 
         it("stops fixing after 10 passes", () => {
-            linter.defineRule("add-spaces", context => ({
-                Program(node) {
-                    context.report({
-                        node,
-                        message: "Add a space before this node.",
-                        fix: fixer => fixer.insertTextBefore(node, " ")
-                    });
+
+            linter.defineRule("add-spaces", {
+                meta: {
+                    fixable: "whitespace"
+                },
+                create(context) {
+                    return {
+                        Program(node) {
+                            context.report({
+                                node,
+                                message: "Add a space before this node.",
+                                fix: fixer => fixer.insertTextBefore(node, " ")
+                            });
+                        }
+                    };
                 }
-            }));
+            });
 
             const fixResult = linter.verifyAndFix("a", { rules: { "add-spaces": "error" } });
 
@@ -5065,10 +5080,10 @@ var a = "test2";
 
             assert.throws(() => {
                 linter.verify("0", { rules: { "test-rule": "error" } });
-            }, /Fixable rules should export a `meta\.fixable` property.\nOccurred while linting <input>:1$/u);
+            }, /Fixable rules must set the `meta\.fixable` property to "code" or "whitespace".\nOccurred while linting <input>:1$/u);
         });
 
-        it("should not throw an error if fix is passed and there is no metadata", () => {
+        it("should throw an error if fix is passed and there is no metadata", () => {
             linter.defineRule("test-rule", {
                 create: context => ({
                     Program(node) {
@@ -5077,7 +5092,21 @@ var a = "test2";
                 })
             });
 
-            linter.verify("0", { rules: { "test-rule": "error" } });
+            assert.throws(() => {
+                linter.verify("0", { rules: { "test-rule": "error" } });
+            }, /Fixable rules must set the `meta\.fixable` property/u);
+        });
+
+        it("should throw an error if fix is passed from a legacy-format rule", () => {
+            linter.defineRule("test-rule", context => ({
+                Program(node) {
+                    context.report(node, "hello world", {}, () => ({ range: [1, 1], text: "" }));
+                }
+            }));
+
+            assert.throws(() => {
+                linter.verify("0", { rules: { "test-rule": "error" } });
+            }, /Fixable rules must set the `meta\.fixable` property/u);
         });
     });
 
diff --git a/tests/lib/rule-tester/rule-tester.js b/tests/lib/rule-tester/rule-tester.js
index 3aaf6f91f3f4..9c32e0d31cf4 100644
--- a/tests/lib/rule-tester/rule-tester.js
+++ b/tests/lib/rule-tester/rule-tester.js
@@ -1289,7 +1289,7 @@ describe("RuleTester", () => {
                     { code: "var foo = bar;", output: "5", errors: 1 }
                 ]
             });
-        }, "Fixable rules should export a `meta.fixable` property.");
+        }, /Fixable rules must set the `meta\.fixable` property/u);
     });
     it("should throw an error if a legacy-format rule produces fixes", () => {
 
@@ -1313,7 +1313,7 @@ describe("RuleTester", () => {
                     { code: "var foo = bar;", output: "5", errors: 1 }
                 ]
             });
-        }, "Fixable rules should export a `meta.fixable` property.");
+        }, /Fixable rules must set the `meta\.fixable` property/u);
     });
 
     describe("suggestions", () => {
@@ -1923,17 +1923,24 @@ describe("RuleTester", () => {
         assert.throw(() => {
             ruleTester.run(
                 "foo",
-                context => ({
-                    Identifier(node) {
-                        context.report({
-                            node,
-                            message: "make a syntax error",
-                            fix(fixer) {
-                                return fixer.replaceText(node, "one two");
+                {
+                    meta: {
+                        fixable: "code"
+                    },
+                    create(context) {
+                        return {
+                            Identifier(node) {
+                                context.report({
+                                    node,
+                                    message: "make a syntax error",
+                                    fix(fixer) {
+                                        return fixer.replaceText(node, "one two");
+                                    }
+                                });
                             }
-                        });
+                        };
                     }
-                }),
+                },
                 {
                     valid: ["one()"],
                     invalid: []