Repository: stoplightio/spectral
Pull request #461 (2019-08-19 09:22:35)

 feat: handle circular rulesets
Closes https://github.com/stoplightio/spectral/issues/479

**Checklist**

- [x] Tests added / updated
- [ ] Docs added / updated

**Does this PR introduce a breaking change?**

- [ ] Yes
- [x] No


See ticket for more info :) It has implementation details included.

philsturgeon: If circular rulesets is getting a bit tough, we can change things around a bit in a different way.

new ruleset: `spectral:openapi`

This has all of the rules in it, every single one, because now we have formats we don't need to split by version.

Then we just mark oas, oas2 and oas3 for deprecation in v4.1, and remove them in spectral v5.0. What do you think?
P0lip: > If circular rulesets is getting a bit tough, we can change things around a bit in a different way.

Actually, it's all good, no worries!
I'll just need to update a bunch of tests, but I got it working.
Just waiting for custom functions to be merged...

P0lip: @philsturgeon @marbemac 
I'd like to get the oas ruleset extends oas2 and oas3 rulesets thing done as a part of this PR.
There is on issue, though. A few rule definitions collide, i.e. `valid-example`.
Shall I just move such rules to the base oas ruleset?
philsturgeon: Yeah anything with the same name can just have oas2 or oas3 prefixed. 

--
Phil Sturgeon
@philsturgeon

> On Aug 25, 2019, at 11:54, Jakub Rożek <notifications@github.com> wrote:
> 
> @philsturgeon @marbemac
> I'd like to get the oas ruleset extends oas2 and oas3 rulesets thing done as a part of this PR.
> There is on issue, though. A few rule definitions collide, i.e. valid-example.
> Shall I just move such rules to the base oas ruleset?
> 
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub, or mute the thread.

P0lip: @philsturgeon @marbemac 
Added support for oas ruleset extends oas2 and oas3 rulesets in https://github.com/stoplightio/spectral/pull/461/commits/d47b53e7ce60abd728e80720361825a7b5f60b54

I know you will ask "Why so many changes? Shouldn't it work out of the box?", so in order to give you an answer in advance, let me explain it.

There was a subtle yet fairly significant issue/quirk with the current implementation of rulesets.
Some may say it's a feature, some may say it's a bug, but IMHO it's rather an issue that needs to be fixed (and it's fixed).
So the case is that a base ruleset treats its own rule as they were all recommended.
This might be observed quite well here
https://github.com/stoplightio/spectral/pull/461/commits/d47b53e7ce60abd728e80720361825a7b5f60b54#diff-1d22b24e26ececefd0c87be1d0569b68L543
`model-description` rule is *not* recommended, therefore we should not be running that rule when you load oas2 ruleset in the following way `spectral.loadRuleset('spectral:oas2')`.
It did work fine if you extended spectral:oas2 ruleset, i.e.
```
extends: [spectral:oas2]
rules:
```
There are also other tests showing this invalid behavior, for instance https://github.com/stoplightio/spectral/pull/461/commits/d47b53e7ce60abd728e80720361825a7b5f60b54#diff-62b31a98be4036e65207ec7971253e93R19
In general, all updated tests cover that issue.
It's not straightforward to spot. I wouldn't notice it either if we didn't bring support for `oas ruleset extends oas2 and oas3 rulesets` feature.

The diff is also large due to prettifying (lines shuffled etc). Moreover, I made assertions a bit cleaner in a few places.

I hope it's all good now, but I'd really encourage you to build the code and try it in a couple of rulesets. Also, pay attention to tests, validate whether the expected results are okay etc.
We had a few invalid assertions, well at least they 

If we get all of that right, we should be in a very good spot - we have lots of various tests now, so in general, I feel fairly confident about that particular feature now.

Anyway, as we all know, tests will detect issues, but will never detect the lack of issues :D

LMK if you have any other questions.



Patch:
diff --git a/src/cli/commands/lint.ts b/src/cli/commands/lint.ts
index ecf853fc..55e6302c 100644
--- a/src/cli/commands/lint.ts
+++ b/src/cli/commands/lint.ts
@@ -121,7 +121,7 @@ async function lint(name: string, flags: ILintConfig, command: Lint, rulesets: O
   const spectral = new Spectral({ resolver: httpAndFileResolver });
 
   try {
-    await spectral.loadRuleset(rulesets ? rulesets : ['spectral:oas2', 'spectral:oas3']);
+    await spectral.loadRuleset(rulesets ? rulesets : ['spectral:oas']);
   } catch (ex) {
     command.log(ex.message);
     process.exitCode = 2;
diff --git a/src/rulesets/mergers/rules.ts b/src/rulesets/mergers/rules.ts
index 9b2335d2..d020a91f 100644
--- a/src/rulesets/mergers/rules.ts
+++ b/src/rulesets/mergers/rules.ts
@@ -1,5 +1,6 @@
+import { DiagnosticSeverity } from '@stoplight/types';
 import { cloneDeep } from 'lodash';
-import { Rule } from '../../types';
+import { HumanReadableDiagnosticSeverity, Rule } from '../../types';
 import { FileRule, FileRuleCollection, FileRulesetSeverity } from '../../types/ruleset';
 import { DEFAULT_SEVERITY_LEVEL, getDiagnosticSeverity, getSeverityLevel } from '../severity';
 import { isValidRule } from '../validation';
@@ -20,10 +21,11 @@ export function mergeRules(
     if (rulesetSeverity !== undefined) {
       const severity = getSeverityLevel(source, name, rulesetSeverity);
       if (isValidRule(rule)) {
+        markRule(rule);
         rule.severity = severity;
         processRule(target, name, rule);
       } else {
-        processRule(target, name, severity);
+        processRule(target, name, typeof rule === 'boolean' ? rule : severity);
       }
     } else {
       processRule(target, name, rule);
@@ -79,7 +81,6 @@ function processRule(rules: FileRuleCollection, name: string, rule: FileRule | F
 
       break;
     case 'object':
-      normalizeRule(rule);
       if (Array.isArray(rule)) {
         processRule(rules, name, rule[0]);
 
@@ -91,8 +92,10 @@ function processRule(rules: FileRuleCollection, name: string, rule: FileRule | F
           updateRootRule(existingRule, null);
         }
       } else if (isValidRule(existingRule)) {
+        normalizeRule(rule, existingRule.severity);
         updateRootRule(existingRule, rule);
       } else {
+        normalizeRule(rule, getSeverityLevel(rules, name, rule));
         // new rule
         markRule(rule);
         rules[name] = rule;
@@ -104,12 +107,10 @@ function processRule(rules: FileRuleCollection, name: string, rule: FileRule | F
   }
 }
 
-function normalizeRule(rule: FileRule) {
-  if (isValidRule(rule)) {
-    if (rule.severity === undefined) {
-      rule.severity = DEFAULT_SEVERITY_LEVEL;
-    } else {
-      rule.severity = getDiagnosticSeverity(rule.severity);
-    }
+function normalizeRule(rule: Rule, severity: DiagnosticSeverity | HumanReadableDiagnosticSeverity | undefined) {
+  if (rule.severity === void 0) {
+    rule.severity = severity === void 0 ? (rule.recommended ? DEFAULT_SEVERITY_LEVEL : -1) : severity;
+  } else {
+    rule.severity = getDiagnosticSeverity(rule.severity);
   }
 }
diff --git a/src/rulesets/oas/index.json b/src/rulesets/oas/index.json
index 3d91ca7e..0e797c7b 100644
--- a/src/rulesets/oas/index.json
+++ b/src/rulesets/oas/index.json
@@ -1,4 +1,5 @@
 {
+  "extends": ["spectral:oas2", "spectral:oas3"],
   "formats": ["oas2", "oas3"],
   "functions": [
     "oasOp2xxResponse",
@@ -413,6 +414,20 @@
       "tags": [
         "api"
       ]
+    },
+    "valid-example": {
+      "description": "Examples must be valid against their defined schema.",
+      "message": "{{error}}",
+      "recommended": true,
+      "type": "validation",
+      "given": "$..[?(@.example)]",
+      "then": {
+        "function": "schemaPath",
+        "functionOptions": {
+          "field": "example",
+          "schemaPath": "$"
+        }
+      }
     }
   }
 }
diff --git a/src/rulesets/oas2/index.json b/src/rulesets/oas2/index.json
index 9729fc45..393d15dd 100644
--- a/src/rulesets/oas2/index.json
+++ b/src/rulesets/oas2/index.json
@@ -73,7 +73,7 @@
         "function": "truthy"
       }
     },
-    "operation-security-defined": {
+    "oas2-operation-security-defined": {
       "description": "Operation `security` values must match a scheme defined in the `securityDefinitions` object.",
       "recommended": true,
       "type": "validation",
@@ -90,20 +90,6 @@
         "operation"
       ]
     },
-    "valid-example": {
-      "description": "Examples must be valid against their defined schema.",
-      "message": "{{error}}",
-      "recommended": true,
-      "type": "validation",
-      "given": "$..[?(@.example)]",
-      "then": {
-        "function": "schemaPath",
-        "functionOptions": {
-          "field": "example",
-          "schemaPath": "$"
-        }
-      }
-    },
     "oas2-anyOf": {
       "description": "OpenAPI v3 keyword `anyOf` detected in OpenAPI v2 document.",
       "message": "anyOf is not available in OpenAPI v2, it was added in OpenAPI v3",
diff --git a/src/rulesets/oas3/index.json b/src/rulesets/oas3/index.json
index 0dbfaee5..6d9cb482 100644
--- a/src/rulesets/oas3/index.json
+++ b/src/rulesets/oas3/index.json
@@ -34,7 +34,7 @@
         "function": "truthy"
       }
     },
-    "operation-security-defined": {
+    "oas3-operation-security-defined": {
       "description": "Operation `security` values must match a scheme defined in the `components.securitySchemes` object.",
       "recommended": true,
       "type": "validation",
@@ -78,20 +78,6 @@
         }
       }
     },
-    "valid-example": {
-      "description": "Examples must be valid against their defined schema.",
-      "message": "{{error}}",
-      "recommended": true,
-      "type": "validation",
-      "given": "$..[?(@.example)]",
-      "then": {
-        "function": "schemaPath",
-        "functionOptions": {
-          "field": "example",
-          "schemaPath": "$"
-        }
-      }
-    },
     "oas3-schema": {
       "description": "Validate structure of OpenAPI v3 specification.",
       "message": "{{error}}",
diff --git a/src/rulesets/reader.ts b/src/rulesets/reader.ts
index e0e050d8..35f11677 100644
--- a/src/rulesets/reader.ts
+++ b/src/rulesets/reader.ts
@@ -18,101 +18,112 @@ export async function readRuleset(uris: string | string[]): Promise<IRuleset> {
     functions: {},
   };
 
-  const cache = new Cache();
-
-  for (const uri of Array.isArray(uris) ? uris : [uris]) {
-    const resolvedRuleset = await processRuleset(cache, uri, uri);
-    mergeRules(base.rules, resolvedRuleset.rules);
+  const processedRulesets = new Set<string>();
+  const processRuleset = createRulesetProcessor(processedRulesets, new Cache());
+
+  for (const uri of Array.isArray(uris) ? new Set([...uris]) : [uris]) {
+    processedRulesets.clear(); // makes sure each separate ruleset starts with clear list
+    const resolvedRuleset = await processRuleset(uri, uri);
+    if (resolvedRuleset === null) continue;
+    Object.assign(base.rules, resolvedRuleset.rules);
     Object.assign(base.functions, resolvedRuleset.functions);
   }
 
   return base;
 }
 
-async function processRuleset(
-  uriCache: ICache,
-  baseUri: string,
-  uri: string,
-  severity?: FileRulesetSeverity,
-): Promise<IRuleset> {
-  const rulesetUri = await findFile(join(baseUri, '..'), uri);
-  const { result } = await httpAndFileResolver.resolve(parse(await readParsable(rulesetUri, 'utf8')), {
-    baseUri: rulesetUri,
-    dereferenceInline: false,
-    uriCache,
-    async parseResolveResult(opts) {
-      try {
-        opts.result = parse(opts.result);
-      } catch {
-        // happens
-      }
-      return opts;
-    },
-  });
-  const ruleset = assertValidRuleset(JSON.parse(JSON.stringify(result)));
-  const rules = {};
-  const functions = {};
-  const newRuleset: IRuleset = {
-    rules,
-    functions,
-  };
-
-  const extendedRulesets = ruleset.extends;
-  const rulesetFunctions = ruleset.functions;
-
-  if (extendedRulesets !== void 0) {
-    for (const extended of Array.isArray(extendedRulesets) ? extendedRulesets : [extendedRulesets]) {
-      let extendedRuleset: IRuleset;
-      let parentSeverity: FileRulesetSeverity;
-      if (Array.isArray(extended)) {
-        parentSeverity = severity === undefined ? extended[1] : severity;
-        extendedRuleset = await processRuleset(uriCache, uri, extended[0], parentSeverity);
-      } else {
-        parentSeverity = severity === undefined ? 'recommended' : severity;
-        extendedRuleset = await processRuleset(uriCache, uri, extended, parentSeverity);
-      }
-
-      mergeRules(rules, extendedRuleset.rules, parentSeverity);
-      mergeFunctions(functions, extendedRuleset.functions, rules);
+const createRulesetProcessor = (processedRulesets: Set<string>, uriCache: ICache) => {
+  return async function processRuleset(
+    baseUri: string,
+    uri: string,
+    severity?: FileRulesetSeverity,
+  ): Promise<IRuleset | null> {
+    const rulesetUri = await findFile(join(baseUri, '..'), uri);
+    if (processedRulesets.has(rulesetUri)) {
+      return null;
     }
-  }
-
-  mergeRules(rules, ruleset.rules);
-  if (Array.isArray(ruleset.formats)) {
-    mergeFormats(rules, ruleset.formats);
-  }
-
-  if (rulesetFunctions !== void 0) {
-    const rulesetFunctionsBaseDir = join(
-      rulesetUri,
-      ruleset.functionsDir !== void 0 ? join('..', ruleset.functionsDir) : '../functions',
-    );
-    const resolvedFunctions: FunctionCollection = {};
-
-    await Promise.all(
-      rulesetFunctions.map(async fn => {
-        const fnName = Array.isArray(fn) ? fn[0] : fn;
-        const fnSchema = Array.isArray(fn) ? (fn[1] as JSONSchema7) : null;
 
+    processedRulesets.add(rulesetUri);
+    const { result } = await httpAndFileResolver.resolve(parse(await readParsable(rulesetUri, 'utf8')), {
+      baseUri: rulesetUri,
+      dereferenceInline: false,
+      uriCache,
+      async parseResolveResult(opts) {
         try {
-          const exportedFn = evaluateExport(
-            await readFile(await findFile(rulesetFunctionsBaseDir, `./${fnName}.js`), 'utf-8'),
-          ) as IFunction;
-
-          resolvedFunctions[fnName] = fnSchema !== null ? wrapIFunctionWithSchema(exportedFn, fnSchema) : exportedFn;
-
-          Reflect.defineProperty(resolvedFunctions[fnName], 'name', {
-            configurable: true,
-            value: fnName,
-          });
-        } catch (ex) {
-          console.warn(`Function '${fnName}' could not be loaded: ${ex.message}`);
+          opts.result = parse(opts.result);
+        } catch {
+          // happens
+        }
+        return opts;
+      },
+    });
+    const ruleset = assertValidRuleset(JSON.parse(JSON.stringify(result)));
+    const rules = {};
+    const functions = {};
+    const newRuleset: IRuleset = {
+      rules,
+      functions,
+    };
+
+    const extendedRulesets = ruleset.extends;
+    const rulesetFunctions = ruleset.functions;
+
+    if (extendedRulesets !== void 0) {
+      for (const extended of Array.isArray(extendedRulesets) ? extendedRulesets : [extendedRulesets]) {
+        let extendedRuleset: IRuleset | null;
+        let parentSeverity: FileRulesetSeverity;
+        if (Array.isArray(extended)) {
+          parentSeverity = severity === undefined ? extended[1] : severity;
+          extendedRuleset = await processRuleset(rulesetUri, extended[0], parentSeverity);
+        } else {
+          parentSeverity = severity === undefined ? 'recommended' : severity;
+          extendedRuleset = await processRuleset(rulesetUri, extended, parentSeverity);
         }
-      }),
-    );
 
-    mergeFunctions(functions, resolvedFunctions, rules);
-  }
+        if (extendedRuleset !== null) {
+          mergeRules(rules, extendedRuleset.rules, parentSeverity);
+          mergeFunctions(functions, extendedRuleset.functions, rules);
+        }
+      }
+    }
 
-  return newRuleset;
-}
+    mergeRules(rules, ruleset.rules, severity === undefined ? 'recommended' : severity);
+    if (Array.isArray(ruleset.formats)) {
+      mergeFormats(rules, ruleset.formats);
+    }
+
+    if (rulesetFunctions !== void 0) {
+      const rulesetFunctionsBaseDir = join(
+        rulesetUri,
+        ruleset.functionsDir !== void 0 ? join('..', ruleset.functionsDir) : '../functions',
+      );
+      const resolvedFunctions: FunctionCollection = {};
+
+      await Promise.all(
+        rulesetFunctions.map(async fn => {
+          const fnName = Array.isArray(fn) ? fn[0] : fn;
+          const fnSchema = Array.isArray(fn) ? (fn[1] as JSONSchema7) : null;
+
+          try {
+            const exportedFn = evaluateExport(
+              await readFile(await findFile(rulesetFunctionsBaseDir, `./${fnName}.js`), 'utf-8'),
+            ) as IFunction;
+
+            resolvedFunctions[fnName] = fnSchema !== null ? wrapIFunctionWithSchema(exportedFn, fnSchema) : exportedFn;
+
+            Reflect.defineProperty(resolvedFunctions[fnName], 'name', {
+              configurable: true,
+              value: fnName,
+            });
+          } catch (ex) {
+            console.warn(`Function '${fnName}' could not be loaded: ${ex.message}`);
+          }
+        }),
+      );
+
+      mergeFunctions(functions, resolvedFunctions, rules);
+    }
+
+    return newRuleset;
+  };
+};


Test patch:
diff --git a/src/__tests__/__fixtures__/custom-directory-function-ruleset.json b/src/__tests__/__fixtures__/custom-directory-function-ruleset.json
index d91a46a6..3085d3f1 100644
--- a/src/__tests__/__fixtures__/custom-directory-function-ruleset.json
+++ b/src/__tests__/__fixtures__/custom-directory-function-ruleset.json
@@ -9,6 +9,7 @@
       "message": "{{error}}",
       "given": "$",
       "type": "style",
+      "recommended": true,
       "then": {
         "function": "hasIn",
         "functionOptions": {
diff --git a/src/__tests__/__fixtures__/custom-functions-oas-ruleset.json b/src/__tests__/__fixtures__/custom-functions-oas-ruleset.json
index 8e2c9235..8c514da8 100644
--- a/src/__tests__/__fixtures__/custom-functions-oas-ruleset.json
+++ b/src/__tests__/__fixtures__/custom-functions-oas-ruleset.json
@@ -20,6 +20,7 @@
       "message": "Info must contain Stoplight",
       "given": "$.info.contact",
       "type": "style",
+      "recommended": true,
       "then": {
         "function": "truthy"
       }
diff --git a/src/__tests__/__fixtures__/custom-oas-ruleset.json b/src/__tests__/__fixtures__/custom-oas-ruleset.json
index f5f2c701..69a54100 100644
--- a/src/__tests__/__fixtures__/custom-oas-ruleset.json
+++ b/src/__tests__/__fixtures__/custom-oas-ruleset.json
@@ -5,6 +5,7 @@
       "message": "Info must contain Stoplight",
       "given": "$.info",
       "type": "style",
+      "recommended": true,
       "then": {
         "field": "title",
         "function": "pattern",
diff --git a/src/__tests__/linter.test.ts b/src/__tests__/linter.test.ts
index 943cc73a..0cce0b9c 100644
--- a/src/__tests__/linter.test.ts
+++ b/src/__tests__/linter.test.ts
@@ -535,9 +535,6 @@ responses:: !!foo
       expect.objectContaining({
         code: 'invalid-ref',
       }),
-      expect.objectContaining({
-        code: 'model-description',
-      }),
       expect.objectContaining({
         code: 'valid-example',
         message: '"foo.example" property type should be number',
diff --git a/src/__tests__/spectral.jest.test.ts b/src/__tests__/spectral.jest.test.ts
index ee02dde2..fe240f53 100644
--- a/src/__tests__/spectral.jest.test.ts
+++ b/src/__tests__/spectral.jest.test.ts
@@ -16,24 +16,26 @@ describe('Spectral', () => {
       const s = new Spectral();
       await s.loadRuleset(path.join(__dirname, '__fixtures__/custom-oas-ruleset.json'));
 
-      expect(s.rules).toEqual({
-        ...[...Object.entries(oasRuleset.rules)].reduce<Dictionary<unknown>>((oasRules, [name, rule]) => {
-          oasRules[name] = {
-            name,
-            ...rule,
-            formats: expect.arrayContaining([expect.any(String)]),
-            severity: expect.any(Number),
-            then: expect.any(Object),
-          };
+      expect(s.rules).toEqual(
+        expect.objectContaining({
+          ...[...Object.entries(oasRuleset.rules)].reduce<Dictionary<unknown>>((oasRules, [name, rule]) => {
+            oasRules[name] = {
+              name,
+              ...rule,
+              formats: expect.arrayContaining([expect.any(String)]),
+              severity: expect.any(Number),
+              then: expect.any(Object),
+            };
 
-          return oasRules;
-        }, {}),
-        'info-matches-stoplight': {
-          ...customOASRuleset.rules['info-matches-stoplight'],
-          name: 'info-matches-stoplight',
-          severity: DiagnosticSeverity.Warning,
-        },
-      });
+            return oasRules;
+          }, {}),
+          'info-matches-stoplight': {
+            ...customOASRuleset.rules['info-matches-stoplight'],
+            name: 'info-matches-stoplight',
+            severity: DiagnosticSeverity.Warning,
+          },
+        }),
+      );
     });
 
     test('should support loading rulesets over http', async () => {
@@ -65,7 +67,7 @@ describe('Spectral', () => {
         'info-matches-stoplight': {
           ...ruleset.rules['info-matches-stoplight'],
           name: 'info-matches-stoplight',
-          severity: DiagnosticSeverity.Warning,
+          severity: -1,
         },
       });
     });
diff --git a/src/__tests__/spectral.karma.test.ts b/src/__tests__/spectral.karma.test.ts
index 168e6016..89489da0 100644
--- a/src/__tests__/spectral.karma.test.ts
+++ b/src/__tests__/spectral.karma.test.ts
@@ -1,4 +1,3 @@
-import { DiagnosticSeverity } from '@stoplight/types';
 import { FetchMockSandbox } from 'fetch-mock';
 import { Spectral } from '../spectral';
 
@@ -45,7 +44,7 @@ describe('Spectral', () => {
         'info-matches-stoplight': {
           ...ruleset.rules['info-matches-stoplight'],
           name: 'info-matches-stoplight',
-          severity: DiagnosticSeverity.Warning,
+          severity: -1,
         },
       });
     });
diff --git a/src/__tests__/spectral.test.ts b/src/__tests__/spectral.test.ts
index 65fa5937..c2ddb094 100644
--- a/src/__tests__/spectral.test.ts
+++ b/src/__tests__/spectral.test.ts
@@ -16,19 +16,21 @@ describe('spectral', () => {
       await s.loadRuleset('spectral:oas2');
 
       expect(s.rules).toEqual(
-        [...Object.entries(oasRuleset.rules), ...Object.entries(oas2Ruleset.rules)].reduce<Dictionary<unknown>>(
-          (oasRules, [name, rule]) => {
-            oasRules[name] = {
-              name,
-              ...rule,
-              formats: expect.arrayContaining([expect.any(String)]),
-              severity: expect.any(Number),
-              then: expect.any(Object),
-            };
-
-            return oasRules;
-          },
-          {},
+        expect.objectContaining(
+          [...Object.entries(oasRuleset.rules), ...Object.entries(oas2Ruleset.rules)].reduce<Dictionary<unknown>>(
+            (oasRules, [name, rule]) => {
+              oasRules[name] = {
+                name,
+                ...rule,
+                formats: expect.arrayContaining([expect.any(String)]),
+                severity: expect.any(Number),
+                then: expect.any(Object),
+              };
+
+              return oasRules;
+            },
+            {},
+          ),
         ),
       );
     });
diff --git a/src/cli/commands/__tests__/__fixtures__/ruleset-valid.yaml b/src/cli/commands/__tests__/__fixtures__/ruleset-valid.yaml
index ad1217a5..6d206f7d 100644
--- a/src/cli/commands/__tests__/__fixtures__/ruleset-valid.yaml
+++ b/src/cli/commands/__tests__/__fixtures__/ruleset-valid.yaml
@@ -2,6 +2,7 @@ rules:
   info-matches-stoplight:
     message: Info must contain Stoplight
     given: $.info
+    recommended: true
     type: style
     then:
       field: title
diff --git a/src/cli/commands/__tests__/__fixtures__/spectral.yml b/src/cli/commands/__tests__/__fixtures__/spectral.yml
index ad1217a5..e015e10a 100644
--- a/src/cli/commands/__tests__/__fixtures__/spectral.yml
+++ b/src/cli/commands/__tests__/__fixtures__/spectral.yml
@@ -3,6 +3,7 @@ rules:
     message: Info must contain Stoplight
     given: $.info
     type: style
+    recommended: true
     then:
       field: title
       function: pattern
diff --git a/src/cli/commands/__tests__/lint.test.ts b/src/cli/commands/__tests__/lint.test.ts
index f2bf04c5..8c52d90c 100644
--- a/src/cli/commands/__tests__/lint.test.ts
+++ b/src/cli/commands/__tests__/lint.test.ts
@@ -236,7 +236,7 @@ describe('lint', () => {
           expect(ctx.stdout).toContain(
             '46:24  warning  operation-description   Operation `description` must be present and non-empty string',
           );
-          expect(ctx.stdout).toContain('28 problems (0 errors, 28 warnings, 0 infos)');
+          expect(ctx.stdout).toContain('22 problems (0 errors, 22 warnings, 0 infos)');
           expect(ctx.stdout).toContain('OpenAPI 2.0 (Swagger) detected');
         });
     });
@@ -255,6 +255,10 @@ describe('lint', () => {
       .command(['lint', invalidOas3SpecPath, '--quiet', '--format=json'])
       .it('outputs warnings/errors in a parseable json format', ctx => {
         expect(JSON.parse(ctx.stdout)).toEqual([
+          expect.objectContaining({
+            code: 'api-servers',
+            message: 'OpenAPI `servers` must be present and non-empty array.',
+          }),
           expect.objectContaining({
             message: 'Info object should contain `contact` object.',
             code: 'info-contact',
@@ -263,10 +267,6 @@ describe('lint', () => {
             code: 'info-description',
             message: 'OpenAPI object info `description` must be present and non-empty string.',
           }),
-          expect.objectContaining({
-            code: 'api-servers',
-            message: 'OpenAPI `servers` must be present and non-empty array.',
-          }),
         ]);
       });
   });
@@ -356,22 +356,6 @@ describe('lint', () => {
       .command(['lint', draftRefSpec, '-q', '-f=json'])
       .it('outputs errors occurring in referenced files', ctx => {
         expect(JSON.parse(ctx.stdout)).toEqual([
-          expect.objectContaining({
-            code: 'info-description',
-            message: 'OpenAPI object info `description` must be present and non-empty string.',
-            path: ['info', 'description'], // todo: relative path or absolute path? there is no such path in linted file, but there is such in spec when working on resolved file
-            range: {
-              end: {
-                character: 22,
-                line: 5,
-              },
-              start: {
-                character: 21,
-                line: 5,
-              },
-            },
-            source: expect.stringContaining('src/cli/commands/__tests__/__fixtures__/refs/info.json'),
-          }),
           expect.objectContaining({
             code: 'api-schemes',
             message: 'OpenAPI host `schemes` must be present and non-empty array.',
@@ -395,30 +379,30 @@ describe('lint', () => {
             },
             source: expect.stringContaining('src/cli/commands/__tests__/__fixtures__/refs/info.json'),
           }),
-        ]);
-      });
-
-    test
-      .stdout()
-      .command(['lint', draftNestedRefSpec, '-q', '-f=json'])
-      .it('outputs errors occurring in nested referenced files', ctx => {
-        expect(JSON.parse(ctx.stdout)).toEqual([
           expect.objectContaining({
             code: 'info-description',
             message: 'OpenAPI object info `description` must be present and non-empty string.',
-            path: ['info', 'description'],
+            path: ['info', 'description'], // todo: relative path or absolute path? there is no such path in linted file, but there is such in spec when working on resolved file
             range: {
               end: {
-                character: 18,
-                line: 2,
+                character: 22,
+                line: 5,
               },
               start: {
-                character: 17,
-                line: 2,
+                character: 21,
+                line: 5,
               },
             },
-            source: expect.stringContaining('src/cli/commands/__tests__/__fixtures__/refs/contact.json'),
+            source: expect.stringContaining('src/cli/commands/__tests__/__fixtures__/refs/info.json'),
           }),
+        ]);
+      });
+
+    test
+      .stdout()
+      .command(['lint', draftNestedRefSpec, '-q', '-f=json'])
+      .it('outputs errors occurring in nested referenced files', ctx => {
+        expect(JSON.parse(ctx.stdout)).toEqual([
           expect.objectContaining({
             code: 'api-schemes',
             message: 'OpenAPI host `schemes` must be present and non-empty array.',
@@ -442,6 +426,22 @@ describe('lint', () => {
             },
             source: expect.stringContaining('src/cli/commands/__tests__/__fixtures__/refs/contact.json'),
           }),
+          expect.objectContaining({
+            code: 'info-description',
+            message: 'OpenAPI object info `description` must be present and non-empty string.',
+            path: ['info', 'description'],
+            range: {
+              end: {
+                character: 18,
+                line: 2,
+              },
+              start: {
+                character: 17,
+                line: 2,
+              },
+            },
+            source: expect.stringContaining('src/cli/commands/__tests__/__fixtures__/refs/contact.json'),
+          }),
         ]);
       });
   });
diff --git a/src/rulesets/__tests__/__fixtures__/bar-extends-foo-ruleset.json b/src/rulesets/__tests__/__fixtures__/bar-extends-foo-ruleset.json
new file mode 100644
index 00000000..8ef08778
--- /dev/null
+++ b/src/rulesets/__tests__/__fixtures__/bar-extends-foo-ruleset.json
@@ -0,0 +1,12 @@
+{
+  "extends": ["./foo-extends-bar-ruleset.json"],
+  "rules": {
+    "bar-rule": {
+      "message": "Bar is truthy",
+      "given": "$.bar",
+      "then": {
+        "function": "truthy"
+      }
+    }
+  }
+}
diff --git a/src/rulesets/__tests__/__fixtures__/custom-functions-directory-ruleset.json b/src/rulesets/__tests__/__fixtures__/custom-functions-directory-ruleset.json
index 5a9b5d48..d7676dd6 100644
--- a/src/rulesets/__tests__/__fixtures__/custom-functions-directory-ruleset.json
+++ b/src/rulesets/__tests__/__fixtures__/custom-functions-directory-ruleset.json
@@ -8,6 +8,7 @@
     "bar-rule": {
       "message": "should be OK",
       "given": "$.info",
+      "recommended": true,
       "then": {
         "function": "bar"
       }
@@ -15,6 +16,7 @@
     "truthy-rule": {
       "message": "should be OK",
       "given": "$.x",
+      "recommended": true,
       "then": {
         "function": "truthy"
       }
diff --git a/src/rulesets/__tests__/__fixtures__/extends-oas2-with-override-ruleset.json b/src/rulesets/__tests__/__fixtures__/extends-oas2-with-override-ruleset.json
index 8e3b3a3f..f0a0db08 100644
--- a/src/rulesets/__tests__/__fixtures__/extends-oas2-with-override-ruleset.json
+++ b/src/rulesets/__tests__/__fixtures__/extends-oas2-with-override-ruleset.json
@@ -1,10 +1,11 @@
 {
   "extends": [["spectral:oas2", "all"]],
   "rules": {
-    "operation-security-defined": "off",
+    "oas2-operation-security-defined": "off",
     "operation-2xx-response": {
       "description": "should be overridden",
       "given": "$.info",
+      "recommended": true,
       "then": {
         "function": "truthy"
       }
diff --git a/src/rulesets/__tests__/__fixtures__/foo-extends-bar-ruleset.json b/src/rulesets/__tests__/__fixtures__/foo-extends-bar-ruleset.json
new file mode 100644
index 00000000..0f25ca5a
--- /dev/null
+++ b/src/rulesets/__tests__/__fixtures__/foo-extends-bar-ruleset.json
@@ -0,0 +1,12 @@
+{
+  "extends": ["./bar-extends-foo-ruleset.json"],
+  "rules": {
+    "foo-rule": {
+      "message": "Foo is falsy",
+      "given": "$.foo",
+      "then": {
+        "function": "falsy"
+      }
+    }
+  }
+}
diff --git a/src/rulesets/__tests__/__fixtures__/self-extending-ruleset.json b/src/rulesets/__tests__/__fixtures__/self-extending-ruleset.json
new file mode 100644
index 00000000..ac433f3d
--- /dev/null
+++ b/src/rulesets/__tests__/__fixtures__/self-extending-ruleset.json
@@ -0,0 +1,13 @@
+{
+  "extends": ["./self-extending-ruleset.json"],
+  "rules": {
+    "foo-rule": {
+      "message": "Foo",
+      "given": "$",
+      "recommended": true,
+      "then": {
+        "function": "falsy"
+      }
+    }
+  }
+}
diff --git a/src/rulesets/__tests__/__fixtures__/valid-flat-ruleset.json b/src/rulesets/__tests__/__fixtures__/valid-flat-ruleset.json
index 7deb30d8..b1da30fc 100644
--- a/src/rulesets/__tests__/__fixtures__/valid-flat-ruleset.json
+++ b/src/rulesets/__tests__/__fixtures__/valid-flat-ruleset.json
@@ -6,6 +6,14 @@
       "then": {
         "function": "truthy"
       }
+    },
+    "valid-rule-recommended": {
+      "message": "should be OK",
+      "given": "$.info",
+      "recommended": true,
+      "then": {
+        "function": "truthy"
+      }
     }
   }
 }
diff --git a/src/rulesets/__tests__/reader.jest.test.ts b/src/rulesets/__tests__/reader.jest.test.ts
index 771eac98..4e05b197 100644
--- a/src/rulesets/__tests__/reader.jest.test.ts
+++ b/src/rulesets/__tests__/reader.jest.test.ts
@@ -13,7 +13,7 @@ const validRequireInfo = path.join(__dirname, './__fixtures__/valid-require-info
 const github447 = path.join(__dirname, './__fixtures__/github-issue-447-fixture.yaml');
 const enabledAllRuleset = path.join(__dirname, './__fixtures__/enable-all-ruleset.json');
 const invalidRuleset = path.join(__dirname, './__fixtures__/invalid-ruleset.json');
-const extendsOas2Ruleset = path.join(__dirname, './__fixtures__/extends-oas2-ruleset.json');
+const extendsAllOas2Ruleset = path.join(__dirname, './__fixtures__/extends-oas2-ruleset.json');
 const extendsUnspecifiedOas2Ruleset = path.join(__dirname, './__fixtures__/extends-unspecified-oas2-ruleset.json');
 const extendsDisabledOas2Ruleset = path.join(__dirname, './__fixtures__/extends-disabled-oas2-ruleset.yaml');
 const extendsOas2WithOverrideRuleset = path.join(__dirname, './__fixtures__/extends-oas2-with-override-ruleset.json');
@@ -22,6 +22,8 @@ const myOpenAPIRuleset = path.join(__dirname, './__fixtures__/my-open-api-rulese
 const fooRuleset = path.join(__dirname, './__fixtures__/foo-ruleset.json');
 const customFunctionsDirectoryRuleset = path.join(__dirname, './__fixtures__/custom-functions-directory-ruleset.json');
 const rulesetWithMissingFunctions = path.join(__dirname, './__fixtures__/ruleset-with-missing-functions.json');
+const fooExtendsBarRuleset = path.join(__dirname, './__fixtures__/foo-extends-bar-ruleset.json');
+const selfExtendingRuleset = path.join(__dirname, './__fixtures__/self-extending-ruleset.json');
 const oasRuleset = require('../oas/index.json');
 const oas2Ruleset = require('../oas2/index.json');
 const oas3Ruleset = require('../oas3/index.json');
@@ -43,9 +45,16 @@ describe('Rulesets reader', () => {
       expect.objectContaining({
         rules: {
           'valid-rule': {
+            given: '$.info',
+            message: 'should be OK',
+            severity: -1,
+            then: expect.any(Object),
+          },
+          'valid-rule-recommended': {
             given: '$.info',
             message: 'should be OK',
             severity: DiagnosticSeverity.Warning,
+            recommended: true,
             then: expect.any(Object),
           },
         },
@@ -58,15 +67,22 @@ describe('Rulesets reader', () => {
       expect.objectContaining({
         rules: {
           'valid-rule': {
+            given: '$.info',
+            message: 'should be OK',
+            severity: -1,
+            then: expect.any(Object),
+          },
+          'valid-rule-recommended': {
             given: '$.info',
             message: 'should be OK',
             severity: DiagnosticSeverity.Warning,
+            recommended: true,
             then: expect.any(Object),
           },
           'require-info': {
             given: '$.info',
             message: 'should be OK',
-            severity: DiagnosticSeverity.Warning,
+            severity: -1,
             then: expect.any(Object),
           },
         },
@@ -75,37 +91,39 @@ describe('Rulesets reader', () => {
   });
 
   it('should inherit properties of extended rulesets', async () => {
-    const { rules } = await readRuleset(extendsOas2Ruleset);
+    const { rules } = await readRuleset(extendsAllOas2Ruleset);
 
     // we pick up *all* rules only from spectral:oas and spectral:oas2 and keep their severity level or set a default one
-    expect(rules).toEqual({
-      ...[...Object.entries(oasRuleset.rules), ...Object.entries(oas2Ruleset.rules)].reduce<Dictionary<unknown>>(
-        (oasRules, [name, rule]) => {
-          oasRules[name] = {
-            ...rule,
-            formats: expect.arrayContaining([expect.any(String)]),
-            ...((rule as IRule).severity === undefined && { severity: DiagnosticSeverity.Warning }),
-            then: expect.any(Object),
-          };
-
-          return oasRules;
+    expect(rules).toEqual(
+      expect.objectContaining({
+        ...[...Object.entries(oasRuleset.rules), ...Object.entries(oas2Ruleset.rules)].reduce<Dictionary<unknown>>(
+          (oasRules, [name, rule]) => {
+            oasRules[name] = {
+              ...rule,
+              formats: expect.arrayContaining([expect.any(String)]),
+              ...((rule as IRule).severity === void 0 && { severity: DiagnosticSeverity.Warning }),
+              then: expect.any(Object),
+            };
+
+            return oasRules;
+          },
+          {},
+        ),
+
+        'valid-rule': {
+          given: '$.info',
+          message: 'should be OK',
+          severity: -1,
+          then: expect.any(Object),
         },
-        {},
-      ),
-
-      'valid-rule': {
-        given: '$.info',
-        message: 'should be OK',
-        severity: DiagnosticSeverity.Warning,
-        then: expect.any(Object),
-      },
-    });
+      }),
+    );
   });
 
   it('should inherit properties of extended rulesets and disable not recommended ones', () => {
     return expect(readRuleset(extendsUnspecifiedOas2Ruleset)).resolves.toEqual(
       expect.objectContaining({
-        rules: {
+        rules: expect.objectContaining({
           ...[...Object.entries(oasRuleset.rules), ...Object.entries(oas2Ruleset.rules)].reduce<Dictionary<unknown>>(
             (rules, [name, rule]) => {
               rules[name] = {
@@ -124,10 +142,10 @@ describe('Rulesets reader', () => {
           'valid-rule': {
             given: '$.info',
             message: 'should be OK',
-            severity: DiagnosticSeverity.Warning,
+            severity: -1,
             then: expect.any(Object),
           },
-        },
+        }),
       }),
     );
   });
@@ -136,95 +154,92 @@ describe('Rulesets reader', () => {
   it('given GitHub issue #447, loads recommended oas3 and oas rules correctly', async () => {
     const { rules: readRules } = await readRuleset(github447);
 
-    expect(readRules).toEqual({
-      ...[...Object.entries(oasRuleset.rules), ...Object.entries(oas3Ruleset.rules)].reduce<Dictionary<unknown>>(
-        (rules, [name, rule]) => {
-          const formattedRule: Rule = {
-            ...(rule as Rule),
-            formats: expect.arrayContaining([expect.any(String)]),
-            ...((rule as IRule).severity === undefined && { severity: DiagnosticSeverity.Warning }),
-            ...(!(rule as IRule).recommended && { severity: -1 }),
-            then: expect.any(Object),
-          };
-
-          rules[name] = formattedRule;
-
-          if (name === 'operation-operationId') {
-            formattedRule.severity = DiagnosticSeverity.Error;
-          }
-
-          if (name === 'operation-tags') {
-            formattedRule.severity = DiagnosticSeverity.Hint;
-          }
+    expect(readRules).toEqual(
+      expect.objectContaining({
+        ...[...Object.entries(oasRuleset.rules), ...Object.entries(oas3Ruleset.rules)].reduce<Dictionary<unknown>>(
+          (rules, [name, rule]) => {
+            const formattedRule: Rule = {
+              ...(rule as Rule),
+              formats: expect.arrayContaining([expect.any(String)]),
+              ...((rule as IRule).severity === undefined && { severity: DiagnosticSeverity.Warning }),
+              ...(!(rule as IRule).recommended && { severity: -1 }),
+              then: expect.any(Object),
+            };
+
+            rules[name] = formattedRule;
+
+            if (name === 'operation-operationId') {
+              formattedRule.severity = DiagnosticSeverity.Error;
+            }
+
+            if (name === 'operation-tags') {
+              formattedRule.severity = DiagnosticSeverity.Hint;
+            }
 
-          return rules;
-        },
-        {
-          'schema-names-pascal-case': {
-            description: 'Schema names MUST be written in PascalCase',
-            given: '$.components.schemas.*~',
-            message: '{{property}} is not PascalCase: {{error}}',
-            recommended: true,
-            severity: DiagnosticSeverity.Warning,
-            then: {
-              function: 'pattern',
-              functionOptions: {
-                match: '^[A-Z][a-zA-Z0-9]*$',
+            return rules;
+          },
+          {
+            'schema-names-pascal-case': {
+              description: 'Schema names MUST be written in PascalCase',
+              given: '$.components.schemas.*~',
+              message: '{{property}} is not PascalCase: {{error}}',
+              recommended: true,
+              severity: DiagnosticSeverity.Warning,
+              then: {
+                function: 'pattern',
+                functionOptions: {
+                  match: '^[A-Z][a-zA-Z0-9]*$',
+                },
               },
+              type: 'style',
             },
-            type: 'style',
-          },
-          'operation-id-kebab-case': {
-            description: 'operationId MUST be written in kebab-case',
-            given:
-              "$.paths.*[?( @property === 'get' || @property === 'put' || @property === 'post' || @property === 'delete' || @property === 'options' || @property === 'head' || @property === 'patch' || @property === 'trace' )]",
-            message: '{{property}} is not kebab-case: {{error}}',
-            recommended: true,
-            severity: DiagnosticSeverity.Warning,
-            then: {
-              field: 'operationId',
-              function: 'pattern',
-              functionOptions: {
-                match: '^[a-z][a-z0-9\\-]*$',
+            'operation-id-kebab-case': {
+              description: 'operationId MUST be written in kebab-case',
+              given:
+                "$.paths.*[?( @property === 'get' || @property === 'put' || @property === 'post' || @property === 'delete' || @property === 'options' || @property === 'head' || @property === 'patch' || @property === 'trace' )]",
+              message: '{{property}} is not kebab-case: {{error}}',
+              recommended: true,
+              severity: DiagnosticSeverity.Warning,
+              then: {
+                field: 'operationId',
+                function: 'pattern',
+                functionOptions: {
+                  match: '^[a-z][a-z0-9\\-]*$',
+                },
               },
+              type: 'style',
             },
-            type: 'style',
           },
-        },
-      ),
-    });
+        ),
+      }),
+    );
   });
 
   it('should set severity of disabled rules to off', () => {
-    return expect(readRuleset(extendsDisabledOas2Ruleset)).resolves.toEqual(
+    return expect(readRuleset(extendsDisabledOas2Ruleset)).resolves.toHaveProperty(
+      'rules',
       expect.objectContaining({
-        rules: {
-          ...[...Object.entries(oasRuleset.rules), ...Object.entries(oas2Ruleset.rules)].reduce<Dictionary<unknown>>(
-            (rules, [name, rule]) => {
-              rules[name] = {
-                ...rule,
-                formats: expect.arrayContaining([expect.any(String)]),
-                severity: -1,
-                then: expect.any(Object),
-              };
-
-              return rules;
-            },
-            {},
-          ),
+        ...[...Object.entries(oasRuleset.rules), ...Object.entries(oas2Ruleset.rules)].reduce<Dictionary<unknown>>(
+          (rules, [name, rule]) => {
+            rules[name] = expect.objectContaining({
+              description: (rule as IRule).description,
+              severity: -1,
+            });
 
-          'operation-operationId-unique': {
-            // value of oasRuleset.rules['operation-operationId-unique']
-            description: 'Every operation must have a unique `operationId`.',
-            formats: expect.arrayContaining([expect.any(String)]),
-            recommended: true,
-            type: 'validation',
-            severity: 0,
-            given: '$',
-            then: expect.any(Object),
-            tags: ['operation'],
+            return rules;
           },
-        },
+          {},
+        ),
+
+        'operation-operationId-unique': expect.objectContaining({
+          // value of oasRuleset.rules['operation-operationId-unique']
+          description: 'Every operation must have a unique `operationId`.',
+          recommended: true,
+          type: 'validation',
+          severity: DiagnosticSeverity.Error,
+          given: '$',
+          tags: ['operation'],
+        }),
       }),
     );
   });
@@ -244,7 +259,7 @@ describe('Rulesets reader', () => {
 
   it('should persist disabled properties of extended rulesets', () => {
     return expect(readRuleset(extendsOas2WithOverrideRuleset)).resolves.toHaveProperty(
-      'rules.operation-security-defined',
+      'rules.oas2-operation-security-defined',
       {
         given: '$',
         recommended: true,
@@ -261,18 +276,18 @@ describe('Rulesets reader', () => {
   it('should prefer top-level ruleset severity level', async () => {
     const { rules: enabledRules } = await readRuleset(enabledAllRuleset);
     expect(enabledRules).toEqual(
-      [...Object.entries(oasRuleset.rules), ...Object.entries(oas2Ruleset.rules)].reduce<Dictionary<unknown>>(
-        (rules, [name, rule]) => {
-          rules[name] = {
-            ...rule,
-            formats: expect.arrayContaining([expect.any(String)]),
-            ...((rule as IRule).severity === undefined && { severity: DiagnosticSeverity.Warning }),
-            then: expect.any(Object),
-          };
+      expect.objectContaining(
+        [...Object.entries(oasRuleset.rules), ...Object.entries(oas2Ruleset.rules)].reduce<Dictionary<unknown>>(
+          (rules, [name, rule]) => {
+            rules[name] = expect.objectContaining({
+              description: (rule as IRule).description,
+              ...((rule as IRule).severity === undefined && { severity: DiagnosticSeverity.Warning }),
+            });
 
-          return rules;
-        },
-        {},
+            return rules;
+          },
+          {},
+        ),
       ),
     );
 
@@ -320,6 +335,19 @@ describe('Rulesets reader', () => {
     );
   });
 
+  it('given spectral:oas ruleset, should not pick up unrecommended rules', () => {
+    return expect(readRuleset('spectral:oas')).resolves.toEqual(
+      expect.objectContaining({
+        rules: expect.objectContaining({
+          'contact-properties': expect.objectContaining({
+            severity: -1,
+            recommended: false,
+          }),
+        }),
+      }),
+    );
+  });
+
   it('should support local rulesets', () => {
     return expect(readRuleset(extendsRelativeRuleset)).resolves.toEqual(
       expect.objectContaining({
@@ -344,7 +372,7 @@ describe('Rulesets reader', () => {
           snake_case: {
             given: '$',
             message: 'foo',
-            severity: DiagnosticSeverity.Warning,
+            severity: -1,
             then: {
               function: 'truthy',
             },
@@ -365,7 +393,7 @@ describe('Rulesets reader', () => {
       'foo-rule': expect.objectContaining({
         message: 'should be OK',
         given: '$.info',
-        severity: DiagnosticSeverity.Warning,
+        severity: -1,
         then: {
           function: 'random-id-0',
         },
@@ -418,6 +446,47 @@ describe('Rulesets reader', () => {
     });
   });
 
+  it('should handle ruleset with circular extensions', () => {
+    return expect(readRuleset(fooExtendsBarRuleset)).resolves.toEqual({
+      functions: {},
+      rules: {
+        'bar-rule': {
+          given: '$.bar',
+          message: 'Bar is truthy',
+          severity: -1, // rule was not recommended, hence the severity is set to false
+          then: {
+            function: 'truthy',
+          },
+        },
+        'foo-rule': {
+          given: '$.foo',
+          message: 'Foo is falsy',
+          severity: -1, // rule was not recommended, hence the severity is set to false
+          then: {
+            function: 'falsy',
+          },
+        },
+      },
+    });
+  });
+
+  it('should handle ruleset that extends itself', () => {
+    return expect(readRuleset(selfExtendingRuleset)).resolves.toEqual({
+      functions: {},
+      rules: {
+        'foo-rule': {
+          given: '$',
+          message: 'Foo',
+          severity: DiagnosticSeverity.Warning,
+          recommended: true,
+          then: {
+            function: 'falsy',
+          },
+        },
+      },
+    });
+  });
+
   it('given non-existent ruleset should output error', () => {
     nock('https://unpkg.com')
       .get('/oneParentRuleset')
diff --git a/src/rulesets/oas2/__tests__/valid-example.ts b/src/rulesets/oas/__tests__/valid-example.ts
similarity index 77%
rename from src/rulesets/oas2/__tests__/valid-example.ts
rename to src/rulesets/oas/__tests__/valid-example.ts
index 53563499..979f0ab2 100644
--- a/src/rulesets/oas2/__tests__/valid-example.ts
+++ b/src/rulesets/oas/__tests__/valid-example.ts
@@ -2,6 +2,10 @@ import { DiagnosticSeverity } from '@stoplight/types';
 import { RuleType, Spectral } from '../../../spectral';
 import * as ruleset from '../index.json';
 
+// @oclif/test packages requires @types/mocha, therefore we have 2 packages coming up with similar typings
+// TS is confused and prefers the mocha ones, so we need to instrument it to pick up the Jest ones
+declare var test: jest.It;
+
 describe('valid-example', () => {
   const s = new Spectral();
   s.setRules({
@@ -262,4 +266,66 @@ describe('valid-example', () => {
 
     expect(results).toHaveLength(0);
   });
+
+  test('does not report example mismatches for unknown AJV formats', async () => {
+    const results = await s.run({
+      xoxo: {
+        type: 'object',
+        properties: {
+          ip_address: {
+            type: 'integer',
+            format: 'foo',
+            example: 2886989840,
+          },
+        },
+      },
+    });
+
+    expect(results).toEqual([]);
+  });
+
+  test.each([['byte', '1'], ['int32', 2 ** 31], ['int64', 2 ** 63], ['float', 2 ** 128]])(
+    'reports invalid usage of %s format',
+    async (format, example) => {
+      const results = await s.run({
+        xoxo: {
+          type: 'object',
+          properties: {
+            ip_address: {
+              type: ['string', 'number'],
+              format,
+              example,
+            },
+          },
+        },
+      });
+
+      expect(results).toEqual([
+        expect.objectContaining({
+          code: 'valid-example',
+          message: `"ip_address.example" property format should match format "${format}"`, // hm, ip_address is likely to be more meaningful no?
+        }),
+      ]);
+    },
+  );
+
+  test.each([['byte', 'MTI3'], ['int32', 2 ** 30], ['int64', 2 ** 40], ['float', 2 ** 64], ['double', 2 ** 1028]])(
+    'does not report valid usage of %s format',
+    async (format, example) => {
+      const results = await s.run({
+        xoxo: {
+          type: 'object',
+          properties: {
+            ip_address: {
+              type: ['string', 'number'],
+              format,
+              example,
+            },
+          },
+        },
+      });
+
+      expect(results).toHaveLength(0);
+    },
+  );
 });
diff --git a/src/rulesets/oas/functions/__tests__/oasOpSecurityDefined.test.ts b/src/rulesets/oas/functions/__tests__/oasOpSecurityDefined.test.ts
index 30c973ea..27a32d3e 100644
--- a/src/rulesets/oas/functions/__tests__/oasOpSecurityDefined.test.ts
+++ b/src/rulesets/oas/functions/__tests__/oasOpSecurityDefined.test.ts
@@ -9,9 +9,9 @@ describe('oasOpSecurityDefined', () => {
     const s = new Spectral();
     s.setFunctions({ oasOpSecurityDefined });
     s.setRules({
-      'operation-security-defined': Object.assign(oas2Rules['operation-security-defined'], {
+      'oas2-operation-security-defined': Object.assign(oas2Rules['oas2-operation-security-defined'], {
         recommended: true,
-        type: RuleType[oas2Rules['operation-security-defined'].type],
+        type: RuleType[oas2Rules['oas2-operation-security-defined'].type],
       }),
     });
 
@@ -53,7 +53,7 @@ describe('oasOpSecurityDefined', () => {
 
       expect(results).toEqual([
         {
-          code: 'operation-security-defined',
+          code: 'oas2-operation-security-defined',
           message: 'Operation `security` values must match a scheme defined in the `securityDefinitions` object.',
           path: ['paths', '/path', 'get', 'security', '0'],
           range: {
@@ -76,9 +76,9 @@ describe('oasOpSecurityDefined', () => {
     const s = new Spectral();
     s.setFunctions({ oasOpSecurityDefined });
     s.setRules({
-      'operation-security-defined': Object.assign(oas3Rules['operation-security-defined'], {
+      'oas3-operation-security-defined': Object.assign(oas3Rules['oas3-operation-security-defined'], {
         recommended: true,
-        type: RuleType[oas3Rules['operation-security-defined'].type],
+        type: RuleType[oas3Rules['oas3-operation-security-defined'].type],
       }),
     });
 
@@ -122,7 +122,7 @@ describe('oasOpSecurityDefined', () => {
 
       expect(results).toEqual([
         {
-          code: 'operation-security-defined',
+          code: 'oas3-operation-security-defined',
           message:
             'Operation `security` values must match a scheme defined in the `components.securitySchemes` object.',
           path: ['paths', '/path', 'get', 'security', '0'],
diff --git a/src/rulesets/oas3/__tests__/valid-example.ts b/src/rulesets/oas3/__tests__/valid-example.ts
deleted file mode 100644
index c1135ff5..00000000
--- a/src/rulesets/oas3/__tests__/valid-example.ts
+++ /dev/null
@@ -1,197 +0,0 @@
-import { DiagnosticSeverity } from '@stoplight/types';
-import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../index.json';
-
-// @oclif/test packages requires @types/mocha, therefore we have 2 packages coming up with similar typings
-// TS is confused and prefers the mocha ones, so we need to instrument it to pick up the Jest ones
-declare var test: jest.It;
-
-describe('valid-example', () => {
-  const s = new Spectral();
-  s.setRules({
-    'valid-example': Object.assign(ruleset.rules['valid-example'], {
-      recommended: true,
-      type: RuleType[ruleset.rules['valid-example'].type],
-    }),
-  });
-
-  test('will pass when simple example is valid', async () => {
-    const results = await s.run({
-      xoxo: {
-        type: 'string',
-        example: 'doggie',
-      },
-    });
-    expect(results).toHaveLength(0);
-  });
-
-  test('will fail when simple example is invalid', async () => {
-    const results = await s.run({
-      xoxo: {
-        type: 'string',
-        example: 123,
-      },
-    });
-    expect(results).toHaveLength(1);
-  });
-
-  test('will pass when complex example is used ', async () => {
-    const results = await s.run({
-      xoxo: {
-        type: 'object',
-        properties: {
-          url: {
-            type: 'string',
-          },
-          width: {
-            type: 'integer',
-          },
-          height: {
-            type: 'integer',
-          },
-        },
-        required: ['url'],
-        example: {
-          url: 'images/38.png',
-          width: 100,
-          height: 100,
-        },
-      },
-    });
-
-    expect(results).toHaveLength(0);
-  });
-
-  test('will error with totally invalid input', async () => {
-    const results = await s.run({
-      xoxo: {
-        type: 'object',
-        properties: {
-          url: {
-            type: 'string',
-          },
-          width: {
-            type: 'integer',
-          },
-          height: {
-            type: 'integer',
-          },
-        },
-        required: ['url'],
-        example: {
-          url2: 'images/38.png',
-          width: 'coffee',
-          height: false,
-        },
-      },
-    });
-
-    expect(results).toHaveLength(1);
-  });
-
-  test('will error with totally invalid input', async () => {
-    const results = await s.run({
-      openapi: '3.0.1',
-      info: {
-        title: 'OpenAPI Petstore',
-        version: '1.0.0',
-      },
-      paths: {
-        '/pet': {
-          post: {
-            requestBody: {
-              content: {
-                '*/*': {
-                  schema: {
-                    type: 'string',
-                    format: 'email',
-                    example: 'hello',
-                  },
-                },
-              },
-              required: true,
-            },
-            responses: {
-              '405': {
-                description: 'Invalid input',
-                content: {},
-              },
-            },
-          },
-        },
-      },
-    });
-
-    expect(results).toEqual([
-      expect.objectContaining({
-        code: 'valid-example',
-        message: '"schema.example" property format should match format "email"',
-        path: ['paths', '/pet', 'post', 'requestBody', 'content', '*/*', 'schema', 'example'],
-        range: expect.any(Object),
-        severity: DiagnosticSeverity.Warning,
-      }),
-    ]);
-  });
-
-  test('does not report example mismatches for unknown AJV formats', async () => {
-    const results = await s.run({
-      xoxo: {
-        type: 'object',
-        properties: {
-          ip_address: {
-            type: 'integer',
-            format: 'foo',
-            example: 2886989840,
-          },
-        },
-      },
-    });
-
-    expect(results).toEqual([]);
-  });
-
-  test.each([['byte', '1'], ['int32', 2 ** 31], ['int64', 2 ** 63], ['float', 2 ** 128]])(
-    'reports invalid usage of %s format',
-    async (format, example) => {
-      const results = await s.run({
-        xoxo: {
-          type: 'object',
-          properties: {
-            ip_address: {
-              type: ['string', 'number'],
-              format,
-              example,
-            },
-          },
-        },
-      });
-
-      expect(results).toEqual([
-        expect.objectContaining({
-          code: 'valid-example',
-          message: `"ip_address.example" property format should match format "${format}"`, // hm, ip_address is likely to be more meaningful no?
-        }),
-      ]);
-    },
-  );
-
-  test.each([['byte', 'MTI3'], ['int32', 2 ** 30], ['int64', 2 ** 40], ['float', 2 ** 64], ['double', 2 ** 1028]])(
-    'does not report valid usage of %s format',
-    async (format, example) => {
-      const results = await s.run({
-        xoxo: {
-          type: 'object',
-          properties: {
-            ip_address: {
-              type: ['string', 'number'],
-              format,
-              example,
-            },
-          },
-        },
-      });
-
-      expect(results).toHaveLength(0);
-    },
-  );
-});