Repository: not-an-aardvark/eslint-plugin-eslint-plugin
Pull request #165 (2021-07-11 17:58:40)

New: Add `catchNoFixerButFixableProperty` option (default false) to catch non-fixable rules that enable the fixable property in `require-meta-fixable` rule
Added behind an option due to risk of false positives.

Open to better names for the option.

bmish: I think the solution to this is to put it behind an option (default off) since it may be too aggressive for some people. Will update.

Patch:
diff --git a/README.md b/README.md
index a78fc8c..33ccc35 100644
--- a/README.md
+++ b/README.md
@@ -57,6 +57,7 @@ Name | ✔️ | 🛠 | 💡 | Description
 [no-only-tests](https://github.com/not-an-aardvark/eslint-plugin-eslint-plugin/blob/master/docs/rules/no-only-tests.md) |  |  | 💡 | disallow the test case property `only`
 [no-unused-placeholders](https://github.com/not-an-aardvark/eslint-plugin-eslint-plugin/blob/master/docs/rules/no-unused-placeholders.md) | ✔️ |  |  | disallow unused placeholders in rule report messages
 [no-useless-token-range](https://github.com/not-an-aardvark/eslint-plugin-eslint-plugin/blob/master/docs/rules/no-useless-token-range.md) | ✔️ | 🛠 |  | disallow unnecessary calls to `sourceCode.getFirstToken()` and `sourceCode.getLastToken()`
+[prefer-message-ids](https://github.com/not-an-aardvark/eslint-plugin-eslint-plugin/blob/master/docs/rules/prefer-message-ids.md) |  |  |  | require using `messageId` instead of `message` to report rule violations
 [prefer-object-rule](https://github.com/not-an-aardvark/eslint-plugin-eslint-plugin/blob/master/docs/rules/prefer-object-rule.md) |  | 🛠 |  | disallow rule exports where the export is a function
 [prefer-output-null](https://github.com/not-an-aardvark/eslint-plugin-eslint-plugin/blob/master/docs/rules/prefer-output-null.md) |  | 🛠 |  | disallow invalid RuleTester test cases where the `output` matches the `code`
 [prefer-placeholders](https://github.com/not-an-aardvark/eslint-plugin-eslint-plugin/blob/master/docs/rules/prefer-placeholders.md) |  |  |  | require using placeholders for dynamic report messages
diff --git a/docs/rules/prefer-message-ids.md b/docs/rules/prefer-message-ids.md
new file mode 100644
index 0000000..1c2ce5f
--- /dev/null
+++ b/docs/rules/prefer-message-ids.md
@@ -0,0 +1,57 @@
+# Require using `messageId` instead of `message` to report rule violations (prefer-message-ids)
+
+When reporting a rule violation, it's preferred to provide the violation message with the `messageId` property instead of the `message` property. Message IDs provide the following benefits:
+
+* Rule violation messages can be stored in a central `meta.messages` object for convenient management
+* Rule violation messages do not need to be repeated in both the rule file and rule test file
+
+## Rule Details
+
+This rule catches usages of the `message` property when reporting a rule violation.
+
+Examples of **incorrect** code for this rule:
+
+```js
+/* eslint eslint-plugin/prefer-message-ids: error */
+
+module.exports = {
+  create (context) {
+    return {
+      CallExpression (node) {
+        context.report({
+          node,
+          message: 'Some error message.',
+        });
+      },
+    };
+  },
+};
+```
+
+Examples of **correct** code for this rule:
+
+```js
+/* eslint eslint-plugin/prefer-message-ids: error */
+
+module.exports = {
+  meta: {
+    messages: {
+      someMessageId: 'Some error message',
+    },
+  },
+  create (context) {
+    return {
+      CallExpression (node) {
+        context.report({
+          node,
+          messageId: 'someMessageId',
+        });
+      },
+    };
+  },
+};
+```
+
+## Further Reading
+
+* [messageIds API](https://eslint.org/docs/developer-guide/working-with-rules#messageids)
diff --git a/docs/rules/require-meta-fixable.md b/docs/rules/require-meta-fixable.md
index 02ac112..ef0ba9e 100644
--- a/docs/rules/require-meta-fixable.md
+++ b/docs/rules/require-meta-fixable.md
@@ -44,6 +44,17 @@ module.exports = {
 };
 ```
 
+```js
+/* eslint eslint-plugin/require-meta-fixable: ["error", { catchNoFixerButFixableProperty: true }] */
+
+module.exports = {
+  meta: { fixable: 'code' }, // property enabled but no fixer detected
+  create (context) {
+    context.report({ node, message: 'foo' });
+  },
+};
+```
+
 Examples of **correct** code for this rule:
 
 ```js
@@ -77,6 +88,12 @@ module.exports = {
 };
 ```
 
+## Options
+
+This rule takes an optional object containing:
+
+* `boolean` — `catchNoFixerButFixableProperty` — default `false` - Whether the rule should attempt to detect rules that do not have a fixer but enable the `meta.fixable` property. This option is off by default because it increases the chance of false positives since fixers can't always be detected when helper functions are used.
+
 ## Further Reading
 
 * [ESLint's autofix API](http://eslint.org/docs/developer-guide/working-with-rules#applying-fixes)
diff --git a/lib/rules/consistent-output.js b/lib/rules/consistent-output.js
index aee28ad..b62b6df 100644
--- a/lib/rules/consistent-output.js
+++ b/lib/rules/consistent-output.js
@@ -26,6 +26,9 @@ module.exports = {
         enum: ['always', 'consistent'],
       },
     ],
+    messages: {
+      missingOutput: 'This test case should have an output assertion.',
+    },
   },
 
   create (context) {
@@ -48,7 +51,7 @@ module.exports = {
             casesWithoutOutput.forEach(testCase => {
               context.report({
                 node: testCase,
-                message: 'This test case should have an output assertion.',
+                messageId: 'missingOutput',
               });
             });
           }
diff --git a/lib/rules/meta-property-ordering.js b/lib/rules/meta-property-ordering.js
index 3f407d1..0c4a46d 100644
--- a/lib/rules/meta-property-ordering.js
+++ b/lib/rules/meta-property-ordering.js
@@ -23,13 +23,15 @@ module.exports = {
       type: 'array',
       elements: { type: 'string' },
     }],
+    messages: {
+      inconsistentOrder: 'The meta properties should be placed in a consistent order: [{{order}}].',
+    },
   },
 
   create (context) {
     const sourceCode = context.getSourceCode();
     const info = getRuleInfo(sourceCode);
 
-    const message = 'The meta properties should be placed in a consistent order: [{{order}}].';
     const order = context.options[0] || ['type', 'docs', 'fixable', 'schema', 'messages'];
 
     const orderMap = new Map(order.map((name, i) => [name, i]));
@@ -67,7 +69,7 @@ module.exports = {
         for (const violatingProp of violatingProps) {
           context.report({
             node: violatingProp,
-            message,
+            messageId: 'inconsistentOrder',
             data: {
               order: knownProps.map(getKeyName).join(', '),
             },
diff --git a/lib/rules/no-deprecated-context-methods.js b/lib/rules/no-deprecated-context-methods.js
index 1ceeca6..7677152 100644
--- a/lib/rules/no-deprecated-context-methods.js
+++ b/lib/rules/no-deprecated-context-methods.js
@@ -44,6 +44,9 @@ module.exports = {
     },
     fixable: 'code',
     schema: [],
+    messages: {
+      newFormat: 'Use `{{contextName}}.getSourceCode().{{replacement}}` instead of `{{contextName}}.{{original}}`.',
+    },
   },
 
   create (context) {
@@ -66,7 +69,7 @@ module.exports = {
             contextId =>
               context.report({
                 node: contextId.parent,
-                message: 'Use `{{contextName}}.getSourceCode().{{replacement}}` instead of `{{contextName}}.{{original}}`.',
+                messageId: 'newFormat',
                 data: {
                   contextName: contextId.name,
                   original: contextId.parent.property.name,
diff --git a/lib/rules/no-deprecated-report-api.js b/lib/rules/no-deprecated-report-api.js
index e53d9df..d20fbb3 100644
--- a/lib/rules/no-deprecated-report-api.js
+++ b/lib/rules/no-deprecated-report-api.js
@@ -21,6 +21,9 @@ module.exports = {
     },
     fixable: 'code', // or "code" or "whitespace"
     schema: [],
+    messages: {
+      useNewAPI: 'Use the new-style context.report() API.',
+    },
   },
 
   create (context) {
@@ -44,7 +47,7 @@ module.exports = {
         ) {
           context.report({
             node: node.callee.property,
-            message: 'Use the new-style context.report() API.',
+            messageId: 'useNewAPI',
             fix (fixer) {
               const openingParen = sourceCode.getTokenBefore(node.arguments[0]);
               const closingParen = sourceCode.getLastToken(node);
diff --git a/lib/rules/no-missing-placeholders.js b/lib/rules/no-missing-placeholders.js
index e855e2b..77aca1b 100644
--- a/lib/rules/no-missing-placeholders.js
+++ b/lib/rules/no-missing-placeholders.js
@@ -22,6 +22,9 @@ module.exports = {
     },
     fixable: null,
     schema: [],
+    messages: {
+      placeholderDoesNotExist: 'The placeholder {{{{missingKey}}}} does not exist.',
+    },
   },
 
   create (context) {
@@ -66,7 +69,7 @@ module.exports = {
               if (!matchingProperty) {
                 context.report({
                   node: reportInfo.message,
-                  message: 'The placeholder {{{{missingKey}}}} does not exist.',
+                  messageId: 'placeholderDoesNotExist',
                   data: { missingKey: match[1] },
                 });
               }
diff --git a/lib/rules/no-unused-placeholders.js b/lib/rules/no-unused-placeholders.js
index 911e6d4..83c38a4 100644
--- a/lib/rules/no-unused-placeholders.js
+++ b/lib/rules/no-unused-placeholders.js
@@ -22,6 +22,9 @@ module.exports = {
     },
     fixable: null,
     schema: [],
+    messages: {
+      placeholderUnused: 'The placeholder {{{{unusedKey}}}} is unused.',
+    },
   },
 
   create (context) {
@@ -69,7 +72,7 @@ module.exports = {
               if (!placeholdersInMessage.has(key)) {
                 context.report({
                   node: reportInfo.message,
-                  message: 'The placeholder {{{{unusedKey}}}} is unused.',
+                  messageId: 'placeholderUnused',
                   data: { unusedKey: key },
                 });
               }
diff --git a/lib/rules/no-useless-token-range.js b/lib/rules/no-useless-token-range.js
index 718db95..0a8a35d 100644
--- a/lib/rules/no-useless-token-range.js
+++ b/lib/rules/no-useless-token-range.js
@@ -21,6 +21,9 @@ module.exports = {
     },
     fixable: 'code',
     schema: [],
+    messages: {
+      useReplacement: "Use '{{replacementText}}' instead.",
+    },
   },
 
   create (context) {
@@ -125,7 +128,7 @@ module.exports = {
               sourceCode.text.slice(identifier.parent.parent.range[1], fullRangeAccess.range[1]);
             context.report({
               node: identifier.parent.parent,
-              message: "Use '{{replacementText}}' instead.",
+              messageId: 'useReplacement',
               data: { replacementText },
               fix (fixer) {
                 return fixer.replaceText(identifier.parent.parent, sourceCode.getText(identifier.parent.parent.arguments[0]));
diff --git a/lib/rules/prefer-message-ids.js b/lib/rules/prefer-message-ids.js
new file mode 100644
index 0000000..2c35aee
--- /dev/null
+++ b/lib/rules/prefer-message-ids.js
@@ -0,0 +1,54 @@
+'use strict';
+
+const utils = require('../utils');
+
+// ------------------------------------------------------------------------------
+// Rule Definition
+// ------------------------------------------------------------------------------
+
+module.exports = {
+  meta: {
+    type: 'problem',
+    docs: {
+      description: 'require using `messageId` instead of `message` to report rule violations',
+      category: 'Rules',
+      recommended: false,
+    },
+    fixable: null,
+    schema: [],
+    messages: {
+      foundMessage: 'Use `messageId` instead of `message`.',
+    },
+  },
+
+  create (context) {
+    let contextIdentifiers;
+
+    // ----------------------------------------------------------------------
+    // Public
+    // ----------------------------------------------------------------------
+
+    return {
+      Program (ast) {
+        contextIdentifiers = utils.getContextIdentifiers(context, ast);
+      },
+      CallExpression (node) {
+        if (
+          node.callee.type === 'MemberExpression' &&
+             contextIdentifiers.has(node.callee.object) &&
+             node.callee.property.type === 'Identifier' && node.callee.property.name === 'report'
+        ) {
+          const reportInfo = utils.getReportInfo(node.arguments, context);
+          if (!reportInfo || !reportInfo.message) {
+            return;
+          }
+
+          context.report({
+            node: reportInfo.message.parent,
+            messageId: 'foundMessage',
+          });
+        }
+      },
+    };
+  },
+};
diff --git a/lib/rules/prefer-output-null.js b/lib/rules/prefer-output-null.js
index c5c0b97..f622f00 100644
--- a/lib/rules/prefer-output-null.js
+++ b/lib/rules/prefer-output-null.js
@@ -21,6 +21,9 @@ module.exports = {
     },
     fixable: 'code',
     schema: [],
+    messages: {
+      useOutputNull: 'Use `output: null` to assert that a test case is not autofixed.',
+    },
   },
 
   create (context) {
@@ -28,7 +31,6 @@ module.exports = {
     // Public
     // ----------------------------------------------------------------------
 
-    const message = 'Use `output: null` to assert that a test case is not autofixed.';
     const sourceCode = context.getSourceCode();
 
     return {
@@ -54,7 +56,7 @@ module.exports = {
             if (output && sourceCode.getText(output.value) === sourceCode.getText(code.value)) {
               context.report({
                 node: output,
-                message,
+                messageId: 'useOutputNull',
                 fix: fixer => fixer.replaceText(output.value, 'null'),
               });
             }
diff --git a/lib/rules/prefer-placeholders.js b/lib/rules/prefer-placeholders.js
index d76f80e..21cf23a 100644
--- a/lib/rules/prefer-placeholders.js
+++ b/lib/rules/prefer-placeholders.js
@@ -22,6 +22,9 @@ module.exports = {
     },
     fixable: null,
     schema: [],
+    messages: {
+      usePlaceholders: 'Use report message placeholders instead of string concatenation.',
+    },
   },
 
   create (context) {
@@ -80,7 +83,7 @@ module.exports = {
           ) {
             context.report({
               node: messageNode,
-              message: 'Use report message placeholders instead of string concatenation.',
+              messageId: 'usePlaceholders',
             });
           }
         }
diff --git a/lib/rules/prefer-replace-text.js b/lib/rules/prefer-replace-text.js
index f26d7a2..9275883 100644
--- a/lib/rules/prefer-replace-text.js
+++ b/lib/rules/prefer-replace-text.js
@@ -21,11 +21,13 @@ module.exports = {
     },
     fixable: null,
     schema: [],
+    messages: {
+      useReplaceText: 'Use replaceText instead of replaceTextRange.',
+    },
   },
 
   create (context) {
     const sourceCode = context.getSourceCode();
-    const message = 'Use replaceText instead of replaceTextRange.';
     let funcInfo = {
       upper: null,
       codePath: null,
@@ -74,7 +76,7 @@ module.exports = {
           if (isIdenticalNodeRange) {
             context.report({
               node,
-              message,
+              messageId: 'useReplaceText',
             });
           }
         }
diff --git a/lib/rules/report-message-format.js b/lib/rules/report-message-format.js
index 59eba72..343f46a 100644
--- a/lib/rules/report-message-format.js
+++ b/lib/rules/report-message-format.js
@@ -24,6 +24,9 @@ module.exports = {
     schema: [
       { type: 'string' },
     ],
+    messages: {
+      noMatch: "Report message does not match the pattern '{{pattern}}'.",
+    },
   },
 
   create (context) {
@@ -44,7 +47,7 @@ module.exports = {
       ) {
         context.report({
           node: message,
-          message: "Report message does not match the pattern '{{pattern}}'.",
+          messageId: 'noMatch',
           data: { pattern: context.options[0] || '' },
         });
       }
diff --git a/lib/rules/require-meta-fixable.js b/lib/rules/require-meta-fixable.js
index 9503b2d..3223ded 100644
--- a/lib/rules/require-meta-fixable.js
+++ b/lib/rules/require-meta-fixable.js
@@ -20,14 +20,28 @@ module.exports = {
       category: 'Rules',
       recommended: true,
     },
-    schema: [],
+    schema: [
+      {
+        type: 'object',
+        properties: {
+          catchNoFixerButFixableProperty: {
+            type: 'boolean',
+            default: false,
+          },
+        },
+        additionalProperties: false,
+      },
+    ],
     messages: {
       invalid: '`meta.fixable` must be either `code`, `whitespace`, or `null`.',
       missing: '`meta.fixable` must be either `code` or `whitespace` for fixable rules.',
+      noFixerButFixableValue: '`meta.fixable` is enabled but no fixer detected.',
     },
   },
 
   create (context) {
+    const catchNoFixerButFixableProperty = context.options[0] && context.options[0].catchNoFixerButFixableProperty;
+
     const sourceCode = context.getSourceCode();
     const ruleInfo = utils.getRuleInfo(sourceCode);
     let contextIdentifiers;
@@ -82,6 +96,9 @@ module.exports = {
           if (usesFixFunctions && !['code', 'whitespace'].includes(staticValue.value)) {
             // Rule is fixable but `fixable` property does not have a fixable value.
             context.report({ node: metaFixableProp.value, messageId: 'missing' });
+          } else if (catchNoFixerButFixableProperty && !usesFixFunctions && ['code', 'whitespace'].includes(staticValue.value)) {
+            // Rule is NOT fixable but `fixable` property has a fixable value.
+            context.report({ node: metaFixableProp.value, messageId: 'noFixerButFixableValue' });
           }
         } else if (!metaFixableProp && usesFixFunctions) {
           // Rule is fixable but is missing the `fixable` property.


Test patch:
diff --git a/lib/rules/test-case-property-ordering.js b/lib/rules/test-case-property-ordering.js
index 7e81ff5..65ce61c 100644
--- a/lib/rules/test-case-property-ordering.js
+++ b/lib/rules/test-case-property-ordering.js
@@ -24,13 +24,15 @@ module.exports = {
       type: 'array',
       elements: { type: 'string' },
     }],
+    messages: {
+      inconsistentOrder: 'The properties of a test case should be placed in a consistent order: [{{order}}].',
+    },
   },
 
   create (context) {
     // ----------------------------------------------------------------------
     // Public
     // ----------------------------------------------------------------------
-    const message = 'The properties of a test case should be placed in a consistent order: [{{order}}].';
     const order = context.options[0] || [
       'filename',
       'code',
@@ -63,7 +65,7 @@ module.exports = {
 
                   context.report({
                     node: properties[i],
-                    message,
+                    messageId: 'inconsistentOrder',
                     data: { order: orderMsg.join(', ') },
                     fix (fixer) {
                       return orderMsg.map((key, index) => {
diff --git a/lib/rules/test-case-shorthand-strings.js b/lib/rules/test-case-shorthand-strings.js
index 7498f23..540571b 100644
--- a/lib/rules/test-case-shorthand-strings.js
+++ b/lib/rules/test-case-shorthand-strings.js
@@ -21,6 +21,9 @@ module.exports = {
     },
     fixable: 'code',
     schema: [{ enum: ['as-needed', 'never', 'consistent', 'consistent-as-needed'] }],
+    messages: {
+      useShorthand: 'Use {{preferred}} for this test case instead of {{actual}}.',
+    },
   },
 
   create (context) {
@@ -62,7 +65,7 @@ module.exports = {
       }[shorthandOption]).forEach(badCaseInfo => {
         context.report({
           node: badCaseInfo.node,
-          message: 'Use {{preferred}} for this test case instead of {{actual}}.',
+          messageId: 'useShorthand',
           data: {
             preferred: badCaseInfo.shorthand ? 'an object' : 'a string',
             actual: badCaseInfo.shorthand ? 'a string' : 'an object',
diff --git a/tests/lib/rules/consistent-output.js b/tests/lib/rules/consistent-output.js
index 7cbd04c..535dd9c 100644
--- a/tests/lib/rules/consistent-output.js
+++ b/tests/lib/rules/consistent-output.js
@@ -12,7 +12,7 @@
 const rule = require('../../../lib/rules/consistent-output');
 const RuleTester = require('eslint').RuleTester;
 
-const ERROR = { message: 'This test case should have an output assertion.', type: 'ObjectExpression' };
+const ERROR = { messageId: 'missingOutput', type: 'ObjectExpression' };
 
 // ------------------------------------------------------------------------------
 // Tests
diff --git a/tests/lib/rules/meta-property-ordering.js b/tests/lib/rules/meta-property-ordering.js
index c98cea6..3d5f094 100644
--- a/tests/lib/rules/meta-property-ordering.js
+++ b/tests/lib/rules/meta-property-ordering.js
@@ -15,14 +15,6 @@ const RuleTester = require('eslint').RuleTester;
 // Tests
 // ------------------------------------------------------------------------------
 
-/**
- * @param {string[]} order
- * @returns {string}
- */
-function getMessage (order) {
-  return `The meta properties should be placed in a consistent order: [${order.join(', ')}].`;
-}
-
 const ruleTester = new RuleTester({ parserOptions: { ecmaVersion: 6 } });
 ruleTester.run('test-case-property-ordering', rule, {
   valid: [
@@ -46,7 +38,7 @@ ruleTester.run('test-case-property-ordering', rule, {
 
     `
     module.exports = {
-      meta: { 
+      meta: {
         type: 'problem',
         docs: {},
         fixable: 'code',
@@ -91,7 +83,7 @@ ruleTester.run('test-case-property-ordering', rule, {
           },
           create() {},
         };`,
-      errors: [{ message: getMessage(['type', 'docs', 'fixable']) }],
+      errors: [{ messageId: 'inconsistentOrder', data: { order: ['type', 'docs', 'fixable'].join(', ') } }],
     },
     {
       code: `
@@ -106,27 +98,27 @@ ruleTester.run('test-case-property-ordering', rule, {
           create() {},
         };`,
       errors: [
-        { message: getMessage(['type', 'docs', 'fixable', 'schema']) },
-        { message: getMessage(['type', 'docs', 'fixable', 'schema']) },
+        { messageId: 'inconsistentOrder', data: { order: ['type', 'docs', 'fixable', 'schema'].join(', ') } },
+        { messageId: 'inconsistentOrder', data: { order: ['type', 'docs', 'fixable', 'schema'].join(', ') } },
       ],
     },
 
     {
       code: `
         module.exports = {
-          meta: {fixable, fooooooooo, doc, type},
+          meta: {fixable, fooooooooo, docs, type},
           create() {},
         };`,
 
       output: `
         module.exports = {
-          meta: {type, doc, fixable, fooooooooo},
+          meta: {type, docs, fixable, fooooooooo},
           create() {},
         };`,
-      options: [['type', 'doc', 'fixable']],
+      options: [['type', 'docs', 'fixable']],
       errors: [
-        { message: getMessage(['type', 'doc', 'fixable']) },
-        { message: getMessage(['type', 'doc', 'fixable']) },
+        { messageId: 'inconsistentOrder', data: { order: ['type', 'docs', 'fixable'].join(', ') } },
+        { messageId: 'inconsistentOrder', data: { order: ['type', 'docs', 'fixable'].join(', ') } },
       ],
     },
   ],
diff --git a/tests/lib/rules/no-deprecated-report-api.js b/tests/lib/rules/no-deprecated-report-api.js
index 2aa5984..d71e3df 100644
--- a/tests/lib/rules/no-deprecated-report-api.js
+++ b/tests/lib/rules/no-deprecated-report-api.js
@@ -11,7 +11,6 @@
 
 const rule = require('../../../lib/rules/no-deprecated-report-api');
 const RuleTester = require('eslint').RuleTester;
-const ERROR = { message: 'Use the new-style context.report() API.', type: 'Identifier' };
 
 // ------------------------------------------------------------------------------
 // Tests
@@ -98,7 +97,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
           }
         };
       `,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       code: `
@@ -115,7 +114,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
           }
         };
       `,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       code: `
@@ -132,7 +131,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
           }
         };
       `,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       code: `
@@ -143,7 +142,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
         };
       `,
       output: null,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       code: `
@@ -160,7 +159,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
           }
         };
       `,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       code: `
@@ -177,7 +176,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
           }
         };
       `,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       // With message string in variable.
@@ -197,7 +196,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
           }
         };
       `,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       // With message in variable but no autofix since we can't statically determine its type.
@@ -210,7 +209,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
         };
       `,
       output: null,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       code: `
@@ -221,7 +220,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
         };
       `,
       output: null,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       code: `
@@ -234,7 +233,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
           context.report({node: theNode, message: \`blah\`, data: theData, fix: theFix});
         };
       `,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       code: `
@@ -247,7 +246,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
           context.report({node: theNode, loc: 5, message: foo, data: bar});
         };
       `,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       // Location in variable as number.
@@ -263,7 +262,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
           context.report({node: theNode, loc: LOC, message: foo, data: bar});
         };
       `,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       // Location in variable as object.
@@ -279,7 +278,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
           context.report({node: theNode, loc: LOC, message: foo, data: bar});
         };
       `,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       // Location in variable but no autofix since we can't statically determine its type.
@@ -290,7 +289,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
         };
       `,
       output: null,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       code: `
@@ -301,7 +300,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
         };
       `,
       output: null,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
     {
       code: `
@@ -312,7 +311,7 @@ ruleTester.run('no-deprecated-report-api', rule, {
         };
       `,
       output: null,
-      errors: [ERROR],
+      errors: [{ messageId: 'useNewAPI', type: 'Identifier' }],
     },
   ],
 });
diff --git a/tests/lib/rules/prefer-message-ids.js b/tests/lib/rules/prefer-message-ids.js
new file mode 100644
index 0000000..81d299e
--- /dev/null
+++ b/tests/lib/rules/prefer-message-ids.js
@@ -0,0 +1,104 @@
+'use strict';
+
+// ------------------------------------------------------------------------------
+// Requirements
+// ------------------------------------------------------------------------------
+
+const rule = require('../../../lib/rules/prefer-message-ids');
+const RuleTester = require('eslint').RuleTester;
+
+// ------------------------------------------------------------------------------
+// Tests
+// ------------------------------------------------------------------------------
+
+const ruleTester = new RuleTester({ parserOptions: { ecmaVersion: 6 } });
+
+ruleTester.run('prefer-message-ids', rule, {
+  valid: [
+    `
+      module.exports = {
+        create(context) {
+          context.report({ node });
+        }
+      };
+    `,
+    `
+      module.exports = {
+        create(context) {
+          context.report({ node, messageId: 'foo' });
+        }
+      };
+    `,
+    `
+      module.exports = {
+        create(context) {
+          foo.report({ node, message: 'foo' }); // unrelated function
+        }
+      };
+    `,
+    `
+      module.exports = {
+        create(context) {
+          context.foo({ node, message: 'foo' }); // unrelated function
+        }
+      };
+    `,
+    `
+      context.report({ node, message: 'foo' }); // outside rule
+      module.exports = {
+        create(context) {
+        }
+      };
+    `,
+    `
+      // Tests are still allowed to use 'message' which is helpful for verifying that dynamically-constructed messages (i.e. from placeholders) look correct.
+      new RuleTester().run('foo', bar, {
+        invalid: [
+          { code: 'foo', errors: [{message: 'foo'}] },
+        ]
+      });
+    `,
+  ],
+
+  invalid: [
+    {
+      code: `
+        module.exports = {
+          create(context) {
+            context.report({ node, message: 'foo' });
+          }
+        };
+      `,
+      errors: [{ messageId: 'foundMessage', type: 'Property' }],
+    },
+    {
+      // With message in variable.
+      code: `
+        const MESSAGE = \`\${foo} is bad.\`;
+        module.exports = {
+          create(context) {
+            context.report({
+              node,
+              message: MESSAGE
+            });
+          }
+        };
+      `,
+      errors: [{ messageId: 'foundMessage', type: 'Property' }],
+    },
+    {
+      // With constructed message.
+      code: `
+        module.exports = {
+          create(context) {
+            context.report({
+              node,
+              message: foo + ' is bad.'
+            });
+          }
+        };
+      `,
+      errors: [{ messageId: 'foundMessage', type: 'Property' }],
+    },
+  ],
+});
diff --git a/tests/lib/rules/prefer-output-null.js b/tests/lib/rules/prefer-output-null.js
index a26daec..6eabb45 100644
--- a/tests/lib/rules/prefer-output-null.js
+++ b/tests/lib/rules/prefer-output-null.js
@@ -12,7 +12,7 @@
 const rule = require('../../../lib/rules/prefer-output-null');
 const RuleTester = require('eslint').RuleTester;
 
-const ERROR = { message: 'Use `output: null` to assert that a test case is not autofixed.' };
+const ERROR = { messageId: 'useOutputNull', type: 'Property' };
 
 // ------------------------------------------------------------------------------
 // Tests
diff --git a/tests/lib/rules/prefer-placeholders.js b/tests/lib/rules/prefer-placeholders.js
index e309382..2de61b8 100644
--- a/tests/lib/rules/prefer-placeholders.js
+++ b/tests/lib/rules/prefer-placeholders.js
@@ -17,7 +17,7 @@ const RuleTester = require('eslint').RuleTester;
 // ------------------------------------------------------------------------------
 
 const ruleTester = new RuleTester({ parserOptions: { ecmaVersion: 6 } });
-const ERROR = { message: 'Use report message placeholders instead of string concatenation.' };
+const ERROR = { messageId: 'usePlaceholders' };
 
 
 ruleTester.run('prefer-placeholders', rule, {
diff --git a/tests/lib/rules/prefer-replace-text.js b/tests/lib/rules/prefer-replace-text.js
index 3402196..10777b2 100644
--- a/tests/lib/rules/prefer-replace-text.js
+++ b/tests/lib/rules/prefer-replace-text.js
@@ -17,7 +17,7 @@ const RuleTester = require('eslint').RuleTester;
 // ------------------------------------------------------------------------------
 
 const ruleTester = new RuleTester({ parserOptions: { ecmaVersion: 6 } });
-const ERROR = { message: 'Use replaceText instead of replaceTextRange.' };
+const ERROR = { messageId: 'useReplaceText', type: 'CallExpression' };
 
 
 ruleTester.run('prefer-placeholders', rule, {
diff --git a/tests/lib/rules/require-meta-fixable.js b/tests/lib/rules/require-meta-fixable.js
index f9aa6b3..ce4761b 100644
--- a/tests/lib/rules/require-meta-fixable.js
+++ b/tests/lib/rules/require-meta-fixable.js
@@ -76,14 +76,6 @@ ruleTester.run('require-meta-fixable', rule, {
         }
       };
     `,
-    `
-      module.exports = {
-        meta: { fixable: 'code' },
-        create(context) {
-          context.report({node, message});
-        }
-      };
-    `,
     `
       module.exports = {
         meta: { fixable: null },
@@ -132,6 +124,53 @@ ruleTester.run('require-meta-fixable', rule, {
         ecmaVersion: 9,
       },
     },
+
+    // catchNoFixerButFixableProperty = false (implicitly)
+    `
+      module.exports = {
+        meta: { fixable: 'code' },
+        create(context) { context.report({node, message}); }
+      };
+    `,
+    `
+      module.exports = {
+        meta: { fixable: 'whitespace' },
+        create(context) { context.report({node, message}); }
+      };
+    `,
+    // catchNoFixerButFixableProperty = false (explicitly)
+    {
+      code: `
+        module.exports = {
+          meta: { fixable: 'code' },
+          create(context) { context.report({node, message}); }
+        };
+      `,
+      options: [{ catchNoFixerButFixableProperty: false }],
+    },
+    {
+      code: `
+        module.exports = {
+          meta: { fixable: 'whitespace' },
+          create(context) { context.report({node, message}); }
+        };
+      `,
+      options: [{ catchNoFixerButFixableProperty: false }],
+    },
+    // catchNoFixerButFixableProperty = true
+    {
+      code: `
+        module.exports = {
+          meta: { fixable: 'code' },
+          create(context) {
+            foo
+              ? context.report({ node, message })
+              : context.report({ node, message, fix });
+          }
+        };
+      `,
+      options: [{ catchNoFixerButFixableProperty: true }],
+    },
   ],
 
   invalid: [
@@ -207,5 +246,37 @@ ruleTester.run('require-meta-fixable', rule, {
       `,
       errors: [{ messageId: 'missing', type: 'Identifier' }],
     },
+
+    // catchNoFixerButFixableProperty = true
+    {
+      code: `
+        module.exports = {
+          meta: { fixable: 'code' },
+          create(context) { context.report({node, message}); }
+        };
+      `,
+      options: [{ catchNoFixerButFixableProperty: true }],
+      errors: [{ messageId: 'noFixerButFixableValue', type: 'Literal' }],
+    },
+    {
+      code: `
+        module.exports = {
+          meta: { fixable: 'whitespace' },
+          create(context) { context.report({node, message}); }
+        };
+      `,
+      options: [{ catchNoFixerButFixableProperty: true }],
+      errors: [{ messageId: 'noFixerButFixableValue', type: 'Literal' }],
+    },
+    {
+      code: `
+        module.exports = {
+          meta: { fixable: null },
+          create(context) { context.report({node, message, fix}); }
+        };
+      `,
+      options: [{ catchNoFixerButFixableProperty: true }],
+      errors: [{ messageId: 'missing', type: 'Literal' }],
+    },
   ],
 });
diff --git a/tests/lib/rules/test-case-shorthand-strings.js b/tests/lib/rules/test-case-shorthand-strings.js
index c15eefb..cf29c26 100644
--- a/tests/lib/rules/test-case-shorthand-strings.js
+++ b/tests/lib/rules/test-case-shorthand-strings.js
@@ -28,7 +28,7 @@ function getTestCases (cases) {
   `;
 }
 
-const EXPECTED_SHORTHAND_ERROR = { message: 'Use a string for this test case instead of an object.' };
+const EXPECTED_SHORTHAND_ERROR = { message: 'Use a string for this test case instead of an object.', type: 'ObjectExpression' };
 const UNEXPECTED_SHORTHAND_ERROR = { message: 'Use an object for this test case instead of a string.' };
 
 // ------------------------------------------------------------------------------