Repository: stoplightio/spectral
Pull request #232 (2019-05-29 10:33:13)

feat: different ruleset extending
### Please check if the PR fulfills these requirements
- [x] Tests for the changes have been added (for bug fixes / features)
- [ ] Docs have been added (docs will be updated in a following PR, started by @marbemac and finished by @philsturgeon)

### What kind of change does this PR introduce?

This might be a controversial change, but unfortunately, the new approach to ruleset introduced in https://github.com/stoplightio/spectral/pull/182 breaks Studio as well as general web usage.
We should avoid using `fs` as much as possible, as this is a Node.js specific module. While, we can shim it in a browser, loading files dynamically during the runtime is almost impossible.
The solution implemented here is not ideal, I am not a fan of it. I believe we might need to reconsider our approach to rulesets. I'm not saying it's totally bad, but I'm not quite sure whether it will play nicely with Studio.

How should we handle CLI? ESLint/TSLint does it quite nicely, cause they treat all configs as packages.

### What is the current behavior?

API usage in CLI works fine. Browser is broken :P

### If this is a feature change, what is the new behavior?

We took the chance to drastically improve the approach to rulesets, and merge config and rulesets together. Pretty much whatever you wanted to do in the config file (loading rulesets, skipping rules) is not available in rulesets, so just make one locally. If you name it `.spectral.yml` it'll automatically load it up in the CLI. 

### Does this PR introduce a breaking change?

Your old `spectral.yml` file will no longer mean anything. The more CLI-based config options will have to be passed in to the CLI, and the ruleset loading will be done with "extends" in your new ruleset, or the CLI --ruleset option.



P0lip: btw, would be also nice to include some browser in our CI to make sure such breaking changes cannot be merged in future.
philsturgeon: The browser testing is a good idea. Could you start off a Jira story with as much detail as you can, then send me the link.
P0lip: @philsturgeon

Yeah, will create a story shortly. Shall I create an issue on GitHub as well or is Jira ticket sufficient?

> Actually before merging - what's going on with the tests?

@marbemac 
Yeah, I said in the description that CLI is broken now.
My solution was a quick fix for our Studio usage, but it's not sufficient enough to be merged due to lack of CLI support. I could make it support CLI, but that would be yet another breaking change.
I opened this PR to discuss the possible solutions.

Here is my take on that:
In general, I'd revamp our approach to rulesets. I'd keep them separate - plugins (rules) and configs.
A plugin would come with a set of rules, but wouldn't specify their severity, enable/disable state and such.
The latter would be configured in the config file. At the moment, everything is kind of mixed and end-user has very little control and is kind of forced to follow our rules. This might be a problem in Studio in the near future as well since we will have to configure rulesets rather than having a simple config, i.e.:
```
"invalid-ref": "warn",
"valid-example": "off",
"valid-schema": "error"
```

The above is just my proposal, I didn't spend much time thinking about it. 
What we have to be very careful is about is any fs access, the one that cannot be determined during the build-time in particular. This makes any shimming almost impossible since we'd need to manually import the *all* files to make sure everything is accessible.






philsturgeon: We're only making GitHub issues for things we think our community might be interested in, and chores are usually not going to fall under that criteria. Just jira will be fine. :)
philsturgeon: What is the current status of this PR? I didn't follow the ruleset change suggestions, that looked a bit more like just toggling things on and off or changing reporting levels, and rulesets are meant to be a lot more powerful than that. Instead of just enabling/disabling our stuff, we want people to be able to write their own stuff. 
P0lip: @philsturgeon so I posted some idea in Jira, let me paste it here.

[...] Instead of having the registry I added, we could simply load the files manually in Studio and then have some interceptor in Spectral to use require rather than fs.readFile or any other method operating on fs.

I'll try to make that work right now, but I still find it as some workaround rather than the actual solution,. We don't have much time though and given the primary API consumer is likely to be Studio, it's not such an issue. Most of Spectral users are CLI users.
P0lip: As promised yesterday, I'm giving a brief overview of the changes taking place in the following PR.
There are no notable breaking changes apart from a less informative ruleset validation - previously we provided more feedback in this regard, i.e. which property causes the issue, etc. Right now, we just say ruleset invalid, but don't provide any more information. I am willing to revert this change if required. It wasn't really intended yet happened to change once I refactored parts of the code

As Marc noticed, I decided to support require-ish extensions of rulesets aiming to be primarily used by browsers and other environments than Node.js. I gave it a quick try in Studio and it pretty much worked out of the box, it seemed.
There is some weirdness going on in resolving ruleset function, but this is mostly due to the fact we run tests in a slightly different environment where Spectral is not installed as a module. I'd say it's safe to assume this is not going to be a case in 99% of projects.

> Extending by require-style path is pretty similar to eslint actually. 

Yeah, ESLint did inspire me on this matter.  Nonetheless, I kept our solution more robust and still allow files as well as URLs to be supplied.
ESLint and its convention of plugins and configs is by far more opinionated than ours, but also considerably more limited at the same time, so we are unable to bring everything in if we'd like to support external URLs, file paths and such.

Alright, now responding to suggestions.

> BREAKING: Right now we use a combination of enabled and severity values to control default rule on/off/severity state. I suggest that we replace enabled: true with recommended: true. This works, because the suggested default behavior of a ruleset (described in number 4) that extends another ruleset is to only run recommended rules (equivalent to them being "enabled" in the current architecture).

I'd say no to this idea. I find our rulesets concept already fairly confusing.
First of all, we have functions you need to import in order to make the rule function properly, but those functions are stored somewhere else. I'd rather go towards more ESLint-ish approach on this matter and have:
- plugins - in our case it'd be an actual rule with an optional function if required
- rulesets - the severity of rules, disabled/enabled state, and any other configuration supported by a particular rule. This would let us have a recommended ruleset we could have enabled by default.

> FEATURE: Suggest we expand the ways one can specify severity to include english words.

:clap:  totally agree.


> BREAKING: If we're going to extend rulesets by require style paths, suggest we move the ruleset files into a more default style location, e.g. @stoplight/spectral/rulesets/oas/index.json instead of @stoplight/spectral/rulesets/oas/ruleset.json.


:+1: this is indeed a nice addition and goes on par with the general ecosystem


> Suggest map a the built-in rulesets to shorter identifiers, similar to what eslint does:

:+1: this is also a very cool idea. I thought about that as well but hesitated to implement it. Will do.

> FEATURE: Suggest we add ability to extend a ruleset with rules off by default, only recommended rules on by default, or everything on by default. See example config file below. Can start with just the simple recommend rules by default option (equivalent of "enabled" rules in the current lingo).


This indicates the problem I mentioned above.
If we kept plugins and configs more isolated, one would have more granular control over rules.
Basically, you could import a plugin which would come with a set of rules, but they would be turned off by default. You'd need to turn them on. Obviously, a plugin could provide a recommended config, but it'd be still so much simpler to reason about than having it all mixed.
Don't get me wrong - I'm not saying our concept is totally flawed, it's not. It has plenty of good sides and I'd keep most, if not all, of its features. What I'm saying is that we could split it somehow.



> BREAKING: Suggest we define functions inside of the config file. This allows us to display docs / do things like code autocomplete inside of studio, etc without having to load and run custom function JS code. It also means that, during programmatic use, don't need to separately load rules and functions (can just load config file, which contains rules + list of functions to be loaded) - this is the breaking change.

Ditto.

> FEATURE: Suggest we add ability to define function options schema inside of the config file. Helps with docs and validation.


Ditto.

> FEATURE: Suggest we allow extended rules to be overriden via several mechanisms. These are described in the example below.

I'd say we should have only 3 ways:
- config
- ignore list - `.spectralignore` or a field in package.json, etc.
- a comment in a document, like `// spectral-disable-next-line rule` etc. We could borrow syntax from other linters. It should have at least the following capabilities:
  - disable linting on given line/file/block entirely
  - disable linting on given line/file/block against particular rule(s)
  - change the severity of particular rule(s)
  - enable rule(s)

Yaml supports comments out of the box, and so does JSONC, therefore the implementation should be possible without any huge hassle.









P0lip: I'm going to implement the 3rd suggestion in the scope of this PR.


philsturgeon: Implementing 2 and 3 in this PR sounds good. I have questions about 1, and am curious to see what conclusions you folks come to after you chat.
philsturgeon: I think most of this stuff is artifacts left over from the old config files we just nuked. Maybe we still need `ConfigFormat` and its just the wrong name now, but `ConfigCommand` is almost certainly not useful anymore.
 
```
export enum ConfigFormat {
  JSON = 'json',
  STYLISH = 'stylish',
}
export enum ConfigCommand {
  LINT = 'lint',
}
export interface ILintConfig {
  encoding: string;
  format: ConfigFormat;
  maxResults?: number;
  output?: string;
  ruleset?: string[];
  skipRule?: string[];
  verbose: boolean;
}
```
marbemac: Good point. Perhaps we should just have a default severity? Would solve it
and is nice and easy. Warn? What do you think phil?

On Thu, Jul 4, 2019 at 3:08 PM Jakub Rożek <notifications@github.com> wrote:

> *@P0lip* commented on this pull request.
> ------------------------------
>
> In src/rulesets/merger.ts
> <https://github.com/stoplightio/spectral/pull/232#discussion_r300492647>:
>
> > @@ -86,3 +99,33 @@ function processRule(rules: FileRuleCollection, name: string, rule: FileRule) {
>  function isValidRule(rule: FileRule): rule is Rule {
>    return typeof rule === 'object' && rule !== null && !Array.isArray(rule) && ('given' in rule || 'then' in rule);
>  }
> +
> +function getSeverityLevel(
> +  rules: FileRuleCollection,
> +  name: string,
> +  rule: FileRule | FileRulesetSeverity,
> +): DiagnosticSeverity | HumanReadableDiagnosticSeverity {
> +  const existingRule = rules[name];
> +
> +  if (!isValidRule(existingRule)) return 'off';
> +
> +  if (rule === 'recommended') {
> +    return getRootRule(existingRule).recommended
> +      ? getRootRule(existingRule).severity || DiagnosticSeverity.Error // not sure what a default value could be
>
> The "extends": [[someRuleset, "recommended"]] stuff is unrelated to
> severity - it should all be driven by the single recommended property.
>
> Yeah, I get it - when recommended is set, I just inherit that rule. My
> main concern is that I don't set any severity and this is kind of weird. It
> definitely was an issue in v3, but right now, once we introduce
> human-readable severity levels, I try to normalize severity to match
> DiagnosticSeverity everywhere and I'm just stuck on that. I can leave it
> empty or so, but I find this to be kind of an issue in particular when you
> set the severity to true as it means you might end up trying to inherit a
> non-existing value t.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> <https://github.com/stoplightio/spectral/pull/232?email_source=notifications&email_token=AAGO5NXH6LGE2NHZOPNYAP3P5ZKDJA5CNFSM4HQLLZ72YY3PNVWWK3TUL52HS4DFWFIHK3DMKJSXC5LFON2FEZLWNFSXPKTDN5WW2ZLOORPWSZGOB5RTZOA#discussion_r300492647>,
> or mute the thread
> <https://github.com/notifications/unsubscribe-auth/AAGO5NTHCB4QREM7REATPLTP5ZKDJANCNFSM4HQLLZ7Q>
> .
>


-- 
Marc MacLeod
http://marbemac.com

P0lip: @marbemac 
Regarding Studio - it's not working *yet* due to
```
Could not parse https://unpkg.com/@stoplight/spectral/rulesets/oas/index.json: (node: rua6otxu8na)
```
It should work just fine once we have Spectral released.
P0lip: One more thought on merging. I introduced a more deep merging strategy, but the previous approach is more shallow-ish, meaning a new merged rule simply overrides everything as is. This primarily applies to `then` property which is an object literal. 
To illustrate, given such an existing rule:
```json
{
  "example": {
    "summary": "",
    " given": "$.responses",
    "then": [
      {
        "foo": "bar"
      }
    ]
  }
}
```
and a rule with the same key that aims to override it
```json
{
  "example": {
    "summary": "",
    "given": "$.responses",
    "then": {
      "field": "$..description",
      "function": "fake"
    }
  }
}
```

we end up with the following
```json
{
  "example": {
    "summary": "",
    "given": "$.responses",
    "then": [
      {
        "foo": "bar"
      }
    ]
  }
}
```

where the value of `then` property, array, has two new properties: `field` and `function`, so in other words the final outcome equals
```ts
Array.isArray(example.then); // true
example.then[0]; // { foo: 'bar' }
example.then.field; // $..description
example.then.function; // fake
```

rather than
```json
{
  "example": {
    "summary": "",
    "given": "$.responses",
    "then": {
      "field": "$..description",
      "function": "fake"
    }
  }
}
```

```ts
example.then; // { field: '$..description', function: 'fake' }
```

I hope I described that potential subtle issue well enough. :sweat_smile: 
What do you think about the change? Shall we make it shallow? 


marbemac: I suggest we straight up replace if the rules et is re defining the rule, and don’t even bother with merging. Whatcha think?
marbemac: I suggest we straight up replace if the rules et is re defining the rule, and don’t even bother with merging. Whatcha think?
marbemac: I suggest we straight up replace if the rules et is re defining the rule, and don’t even bother with merging. Whatcha think?
philsturgeon: I too like the replacing rules > merging them. Less spooky things can happen. 
P0lip: @philsturgeon merging change implemented, so it's ready for another pass.
https://github.com/stoplightio/spectral/pull/232/files/775aa642aa3e804d6ba6395d022a3d195e880b3d#diff-2ce1f604d6e3502e183ce27481205ace
I can address the above suggestion in a separate PR if we want to merge this PR quickly.

P0lip: @marbemac @philsturgeon ready for another pass!

There are two more outstanding suggestions https://github.com/stoplightio/spectral/pull/232/files/775aa642aa3e804d6ba6395d022a3d195e880b3d#diff-ced3a759a4a53e4cf09a4df79dcf1bf2 - needs more consideration
and
https://github.com/stoplightio/spectral/pull/232/files/a1f2e8adaaf54420a62b6b326e1fe1a026ce6f67#diff-4ac0167e41ff45c110fedad5ba76c201 - this one has a broader scope, as we need to unify the naming and new ruleset convention across the entire project.

XVincentX: @philsturgeon @P0lip I do not know why but apparently TypeScript was struggling to distinguish between `index.ts` and `index.json`; I've renamed the json file to `rules.json`.

Patch:
diff --git a/package.json b/package.json
index 9ffee557..8ea2009d 100644
--- a/package.json
+++ b/package.json
@@ -41,7 +41,7 @@
   },
   "scripts": {
     "build": "tsc -p ./tsconfig.build.json",
-    "postbuild": "copyfiles -u 1 \"src/**/ruleset.json\" dist && oclif-dev manifest",
+    "postbuild": "copyfiles -u 1 \"src/rulesets/**/*.json\" dist && oclif-dev manifest",
     "build.binary": "pkg . --targets linux,macos --out-path ./binaries",
     "commit": "git-cz",
     "lint": "tslint 'src/**/*.ts'",
@@ -58,6 +58,7 @@
     "@oclif/plugin-help": "^2.0",
     "@stoplight/json": "^2.2.2",
     "@stoplight/json-ref-resolver": "^2.0.2",
+    "@stoplight/path": "^1.0.1",
     "@stoplight/types": "^9.0.2",
     "@stoplight/yaml": "^2.5",
     "ajv": "^6.7",
@@ -101,7 +102,7 @@
   },
   "lint-staged": {
     "*.{ts,tsx}": [
-      "yarn lint.fix",
+      "tslint --fix",
       "git add"
     ]
   },
diff --git a/src/cli/commands/lint.ts b/src/cli/commands/lint.ts
index cd809f24..252a5d22 100644
--- a/src/cli/commands/lint.ts
+++ b/src/cli/commands/lint.ts
@@ -1,13 +1,13 @@
 import { Command, flags as flagHelpers } from '@oclif/command';
+import { isAbsolute, resolve } from '@stoplight/path';
 import { IParserResult } from '@stoplight/types';
-import { getLocationForJsonPath } from '@stoplight/yaml';
+import { getLocationForJsonPath, parseWithPointers } from '@stoplight/yaml';
 import { writeFile } from 'fs';
 import { isNil, omitBy } from 'lodash';
-import { resolve } from 'path';
 import { promisify } from 'util';
 
 import { IRuleResult } from '../..';
-import { createEmptyConfig, getDefaultConfigFile, load as loadConfig } from '../../config/configLoader';
+import { createEmptyConfig, getDefaultRulesetFile } from '../../config/configLoader';
 import { json, stylish } from '../../formatters';
 import { readParsable } from '../../fs/reader';
 import { httpAndFileResolver } from '../../resolvers/http-and-file';
@@ -16,7 +16,7 @@ import { oas3Functions, rules as oas3Rules } from '../../rulesets/oas3';
 import { readRulesFromRulesets } from '../../rulesets/reader';
 import { Spectral } from '../../spectral';
 import { IParsedResult, RuleCollection } from '../../types';
-import { ConfigCommand, IConfig, ILintConfig } from '../../types/config';
+import { ILintConfig } from '../../types/config';
 
 const writeFileAsync = promisify(writeFile);
 export default class Lint extends Command {
@@ -76,29 +76,29 @@ linting ./openapi.yaml
 
   public async run() {
     const { args, flags } = this.parse(Lint);
-    const { config: configFileFlag } = flags;
+    const { ruleset } = flags;
+    let rules;
+
+    const cwd = process.cwd();
 
-    let config: ILintConfig = mergeConfig(createEmptyConfig(), flags);
+    const config: ILintConfig = mergeConfig(createEmptyConfig(), flags as Partial<ILintConfig>);
 
     this.quiet = flags.quiet;
 
-    const configFile = configFileFlag || getDefaultConfigFile(process.cwd()) || null;
-    if (configFile) {
+    const rulesetFile = ruleset || (await getDefaultRulesetFile(cwd)) || null;
+
+    if (rulesetFile) {
       try {
-        const loadedConfig = await loadConfig(configFile, ConfigCommand.LINT);
-        config = mergeConfig(loadedConfig, flags);
+        rules = await readRulesFromRulesets(
+          ...(Array.isArray(rulesetFile) ? rulesetFile : [rulesetFile]).map(
+            file => (isAbsolute(file) ? file : resolve(cwd, file)),
+          ),
+        );
       } catch (ex) {
-        this.error(`Cannot load provided config file. ${ex.message}.`);
+        this.log(ex.message);
+        this.error(ex);
       }
     }
-    const { ruleset } = config;
-    let rules;
-
-    if (ruleset) {
-      rules = await tryReadOrLog(this, async () => {
-        return readRulesFromRulesets(...ruleset);
-      });
-    }
 
     if (args.source) {
       try {
@@ -146,7 +146,7 @@ async function lint(name: string, flags: any, command: Lint, rules?: RuleCollect
     targetUri = resolve(name);
   }
 
-  const spec: IParserResult = await readParsable(targetUri, flags.encoding);
+  const spec: IParserResult = parseWithPointers(await readParsable(targetUri, flags.encoding));
   const spectral = new Spectral({ resolver: httpAndFileResolver });
   if (parseInt(spec.data.swagger) === 2) {
     command.log('Adding OpenAPI 2.0 (Swagger) functions');
@@ -254,10 +254,10 @@ export async function writeOutput(outputStr: string, flags: any, command: Lint)
   command.print(outputStr);
 }
 
-function mergeConfig(config: IConfig, flags: any): ILintConfig {
+function mergeConfig(config: ILintConfig, flags: Partial<ILintConfig>): ILintConfig {
   return {
-    ...config.lint,
-    ...omitBy<ILintConfig>(
+    ...config,
+    ...omitBy<Partial<ILintConfig>>(
       {
         encoding: flags.encoding,
         format: flags.format,
diff --git a/src/config/configLoader.ts b/src/config/configLoader.ts
index 25fb7f4f..b9c304b3 100644
--- a/src/config/configLoader.ts
+++ b/src/config/configLoader.ts
@@ -1,59 +1,28 @@
-import { IParserResult } from '@stoplight/types';
-import { existsSync } from 'fs';
-import { isEmpty, merge } from 'lodash';
-import * as path from 'path';
-import { readParsable } from '../fs/reader';
-import { ConfigCommand, ConfigFormat, IConfig } from '../types/config';
+import { join } from '@stoplight/path';
+import * as fs from 'fs';
+import { ILintConfig, OutputFormat } from '../types/config';
 
-const DEFAULT_CONFIG_FILE = 'spectral.yml';
+const DEFAULT_RULESET_FILE = /^\.?spectral\.(?:ya?ml|json)$/;
 
-const loadConfig = async (filePath: any): Promise<IConfig> => {
-  switch (path.extname(filePath)) {
-    case '.json':
-    case '.yaml':
-    case '.yml':
-      const { data, diagnostics }: IParserResult = await readParsable(filePath, 'utf8');
-      if (!isEmpty(diagnostics)) {
-        throw new Error('Parsing exception');
-      }
-      return data;
-    default:
-      throw new Error('Unknown file');
-  }
-};
-
-export const load = async (filePath: string, command: ConfigCommand): Promise<IConfig> => {
-  const config: IConfig = await loadConfig(filePath);
-  validate(config, command);
-  normalize(config, command);
-  return merge(createEmptyConfig(), config);
-};
-
-export const validate = (config: IConfig, command: ConfigCommand) => {
-  if (!config) throw new Error('Missing config');
-  if (!config[command]) throw new Error(`Missing ${command} command in config file`);
-};
+export const createEmptyConfig = (): ILintConfig => ({
+  encoding: 'utf8',
+  format: OutputFormat.STYLISH,
+  verbose: false,
+});
 
-export const normalize = (config: IConfig, command: ConfigCommand) => {
-  switch (command) {
-    case ConfigCommand.LINT:
-      if (config.lint.ruleset) {
-        const { ruleset } = config.lint;
-        config.lint.ruleset = Array.isArray(ruleset) ? ruleset : [ruleset];
+export const getDefaultRulesetFile = (directory: string): Promise<string | null> => {
+  return new Promise(resolve => {
+    fs.readdir(directory, (err, files) => {
+      if (err === null) {
+        for (const file of files) {
+          if (DEFAULT_RULESET_FILE.test(file)) {
+            resolve(join(directory, file));
+            return;
+          }
+        }
       }
-      break;
-  }
-};
-
-export const createEmptyConfig = (): IConfig => ({
-  lint: {
-    encoding: 'utf8',
-    format: ConfigFormat.STYLISH,
-    verbose: false,
-  },
-});
 
-export const getDefaultConfigFile = (directory: string) => {
-  const filePath = path.join(directory, DEFAULT_CONFIG_FILE);
-  return existsSync(filePath) ? filePath : null;
+      resolve(null);
+    });
+  });
 };
diff --git a/src/fs/reader.ts b/src/fs/reader.ts
index 0ba337bb..8ac67ddb 100644
--- a/src/fs/reader.ts
+++ b/src/fs/reader.ts
@@ -1,26 +1,32 @@
-import { IParserResult } from '@stoplight/types';
-import { parseWithPointers } from '@stoplight/yaml';
-import { existsSync, readFileSync } from 'fs';
-// @ts-ignore
-import * as fetch from 'node-fetch';
+import { isURL } from '@stoplight/path';
+import { readFile } from 'fs';
+const fetch = require('node-fetch');
 
 async function doRead(name: string, encoding: string) {
-  if (name.startsWith('http')) {
-    const result = await fetch(name);
-    return parseWithPointers(await result.text());
-  } else if (existsSync(name)) {
+  if (isURL(name)) {
+    const response = await fetch(name);
+    if (!response.ok) throw new Error(response.statusText);
+    return await response.text();
+  } else {
     try {
-      return parseWithPointers(readFileSync(name, encoding));
+      return await new Promise((resolve, reject) => {
+        readFile(name, encoding, (err, data) => {
+          if (err !== null) {
+            reject(err);
+          } else {
+            resolve(data);
+          }
+        });
+      });
     } catch (ex) {
       throw new Error(`Could not read ${name}: ${ex.message}`);
     }
   }
-  throw new Error(`${name} does not exist`);
 }
 
-export async function readParsable(name: string, encoding: string): Promise<IParserResult> {
+export async function readParsable(name: string, encoding: string): Promise<string> {
   try {
-    return doRead(name, encoding);
+    return await doRead(name, encoding);
   } catch (ex) {
     throw new Error(`Could not parse ${name}: ${ex.message}`);
   }
diff --git a/src/linter.ts b/src/linter.ts
index 7bdb2ffb..4e1e3952 100644
--- a/src/linter.ts
+++ b/src/linter.ts
@@ -1,11 +1,27 @@
-import { DiagnosticSeverity, JsonPath } from '@stoplight/types';
+import { DiagnosticSeverity, Dictionary, JsonPath } from '@stoplight/types';
 import { get, has } from 'lodash';
 
 const { JSONPath } = require('jsonpath-plus');
 
 import { Resolved } from './resolved';
 import { message } from './rulesets/message';
-import { IFunction, IGivenNode, IRuleResult, IRunRule, IThen } from './types';
+import { HumanReadableDiagnosticSeverity, IFunction, IGivenNode, IRuleResult, IRunRule, IThen } from './types';
+
+const SEVERITY_MAP: Dictionary<number, HumanReadableDiagnosticSeverity> = {
+  error: DiagnosticSeverity.Error,
+  warn: DiagnosticSeverity.Warning,
+  info: DiagnosticSeverity.Information,
+  hint: DiagnosticSeverity.Hint,
+  off: -1,
+};
+
+function printSeverity(severity: DiagnosticSeverity | HumanReadableDiagnosticSeverity): DiagnosticSeverity {
+  if (Number.isNaN(Number(severity))) {
+    return SEVERITY_MAP[severity];
+  }
+
+  return Number(severity);
+}
 
 // TODO(SO-23): unit test but mock whatShouldBeLinted
 export const lintNode = (
@@ -75,6 +91,9 @@ export const lintNode = (
   let results: IRuleResult[] = [];
 
   for (const target of targets) {
+    const severity = rule.severity !== undefined ? printSeverity(rule.severity) : DiagnosticSeverity.Warning;
+
+    if (severity === -1) continue;
     const targetPath = givenPath.concat(target.path);
 
     const targetResults =
@@ -91,7 +110,7 @@ export const lintNode = (
         },
       ) || [];
 
-    const severity = rule.severity !== undefined ? rule.severity : DiagnosticSeverity.Warning;
+    // NOTE: we might want to normalize that during merging (once we have it in place)
 
     results = results.concat(
       targetResults.map(result => {
diff --git a/src/meta/rule.schema.json b/src/meta/rule.schema.json
index d4d119de..d16dd32e 100644
--- a/src/meta/rule.schema.json
+++ b/src/meta/rule.schema.json
@@ -8,10 +8,10 @@
                     "type": "string"
                 },
                 "function": {
-                    "$ref": "http://stoplight.io/schemas/rule.schema.json#/definitions/T"
+                    "$ref": "#/definitions/T"
                 },
                 "functionOptions": {
-                    "$ref": "http://stoplight.io/schemas/rule.schema.json#/definitions/O"
+                    "$ref": "#/definitions/O"
                 }
             },
             "required": [
@@ -24,9 +24,21 @@
         },
         "T": {
             "type": "string"
+        },
+        "HumanReadableSeverity": {
+            "type": "string",
+            "enum": [
+                "error",
+                "warn",
+                "info",
+                "hint",
+                "off"
+            ]
         }
     },
-    "properties": {
+    "oneOf": [
+        {
+            "properties": {
         "description": {
             "type": "string"
         },
@@ -57,11 +69,11 @@
         "then": {
             "anyOf": [
                 {
-                    "$ref": "http://stoplight.io/schemas/rule.schema.json#/definitions/IThen<T,O>"
+                    "$ref": "#/definitions/IThen<T,O>"
                 },
                 {
                     "items": {
-                        "$ref": "http://stoplight.io/schemas/rule.schema.json#/definitions/IThen<T,O>"
+                        "$ref": "#/definitions/IThen<T,O>"
                     },
                     "type": "array"
                 }
@@ -89,9 +101,30 @@
             "type": "object"
         }
     },
-    "required": [
-        "given",
-        "then"
-    ],
-    "type": "object"
+            "required": [
+                "given",
+                "then"
+            ],
+            "type": "object"
+        },
+        {
+            "$ref": "#/definitions/HumanReadableSeverity"
+        },
+        {
+            "type": "boolean"
+        },
+        {
+            "type": "array",
+            "additionalItems": false,
+            "minItems": 1,
+            "items": [
+                {
+                    "$ref": "#/definitions/HumanReadableSeverity"
+                },
+                {
+                    "type": "object"
+                }
+            ]
+        }
+    ]
 }
diff --git a/src/meta/ruleset.schema.json b/src/meta/ruleset.schema.json
index e13c1431..5f65adc3 100644
--- a/src/meta/ruleset.schema.json
+++ b/src/meta/ruleset.schema.json
@@ -12,7 +12,24 @@
         "extends": {
             "type": "array",
             "items": {
-                "type": "string"
+                "oneOf": [
+                    {
+                        "type": "string"
+                    },
+                    {
+                        "type": "array",
+                        "items": [
+                            {
+                                "type": "string"
+                            },
+                            {
+                                "type": "string",
+                                "enum": ["off", "recommended", "all"]
+                            }
+                        ],
+                        "additionalItems": false
+                    }
+                ]
             }
         }
     },
diff --git a/src/resolvers/http-and-file.ts b/src/resolvers/http-and-file.ts
index 72e25082..c13952a8 100644
--- a/src/resolvers/http-and-file.ts
+++ b/src/resolvers/http-and-file.ts
@@ -1,7 +1,7 @@
 import { Resolver } from '@stoplight/json-ref-resolver';
+import { extname } from '@stoplight/path';
 import { parse } from '@stoplight/yaml';
 import * as fs from 'fs';
-import { extname } from 'path';
 
 import { httpReader } from './http';
 
diff --git a/src/rulesets/finder.ts b/src/rulesets/finder.ts
new file mode 100644
index 00000000..13c77af3
--- /dev/null
+++ b/src/rulesets/finder.ts
@@ -0,0 +1,56 @@
+import * as path from '@stoplight/path';
+import * as fs from 'fs';
+import { rulesetsMap } from './map';
+
+const SPECTRAL_SRC_ROOT = path.join(__dirname, '..');
+
+export async function findRuleset(from: string, to: string) {
+  const mapped = rulesetsMap.get(to);
+  if (mapped !== void 0) {
+    to = mapped;
+  }
+
+  if (path.isURL(to) || path.isAbsolute(to)) {
+    return to;
+  }
+
+  if (path.isURL(from)) {
+    return path.join(from, '..', to);
+  }
+
+  try {
+    const targetPath = path.join(from, '..', to);
+    if (await exists(targetPath)) {
+      return targetPath;
+    }
+  } catch {
+    // nothing very bad, let's move on
+    // it's just not a file, but could be a npm module
+  }
+
+  if (SPECTRAL_SRC_ROOT.length > 0 && SPECTRAL_SRC_ROOT !== '/') {
+    try {
+      const targetPath = path.join(SPECTRAL_SRC_ROOT, to.replace('@stoplight/spectral/', './'));
+      if (await exists(targetPath)) {
+        return targetPath;
+      }
+    } catch {
+      // same as above
+    }
+  }
+
+  try {
+    return require.resolve(to);
+  } catch {
+    // todo: put hardcoded version here? might be good if we decide to make a breaking change in future.
+    return path.join('https://unpkg.com/', to); // try to point to npm module
+  }
+}
+
+function exists(uri: string): Promise<boolean> {
+  return new Promise<boolean>(resolve => {
+    fs.access(uri, fs.constants.F_OK, err => {
+      resolve(err === null);
+    });
+  });
+}
diff --git a/src/rulesets/index.ts b/src/rulesets/index.ts
index 12e2969e..2dc0133d 100644
--- a/src/rulesets/index.ts
+++ b/src/rulesets/index.ts
@@ -1,7 +1,7 @@
 import { oas2Functions } from './oas2';
-import * as oas2Ruleset from './oas2/ruleset.json';
+import * as oas2Ruleset from './oas2/rules.json';
 import { oas3Functions } from './oas3';
-import * as oas3Ruleset from './oas3/ruleset.json';
+import * as oas3Ruleset from './oas3/rules.json';
 
 const merge = require('lodash/merge');
 
diff --git a/src/rulesets/map.ts b/src/rulesets/map.ts
new file mode 100644
index 00000000..6ef8b0c9
--- /dev/null
+++ b/src/rulesets/map.ts
@@ -0,0 +1,8 @@
+function resolveSpectralRuleset(ruleset: string) {
+  return `@stoplight/spectral/rulesets/${ruleset}/rules.json`;
+}
+export const rulesetsMap = new Map<string, string>([
+  ['spectral:oas', resolveSpectralRuleset('oas')],
+  ['spectral:oas2', resolveSpectralRuleset('oas2')],
+  ['spectral:oas3', resolveSpectralRuleset('oas3')],
+]);
diff --git a/src/rulesets/merger.ts b/src/rulesets/merger.ts
new file mode 100644
index 00000000..3bb8675d
--- /dev/null
+++ b/src/rulesets/merger.ts
@@ -0,0 +1,128 @@
+import { DiagnosticSeverity } from '@stoplight/types/dist';
+import { cloneDeep } from 'lodash';
+import { HumanReadableDiagnosticSeverity, Rule } from '../types';
+import { FileRule, FileRuleCollection, FileRulesetSeverity, IRulesetFile } from '../types/ruleset';
+
+const DEFAULT_SEVERITY_LEVEL = DiagnosticSeverity.Warning;
+
+/*
+- if rule is object, simple deep merge (or we could replace to be a bit stricter?)
+- if rule is true, use parent rule with it's default severity
+- if rule is false, use parent rule but set it's severity to "off"
+- if rule is string or number, use parent rule and set it's severity to the given string/number value
+- if rule is array, index 0 should be false/true/string/number - same severity logic as above. optional second
+*/
+export function mergeRulesets(target: IRulesetFile, src: IRulesetFile, rulesetSeverity?: FileRulesetSeverity) {
+  const { rules } = target;
+
+  for (const [name, rule] of Object.entries(src.rules)) {
+    if (rulesetSeverity !== undefined) {
+      if (isValidRule(rule)) {
+        processRule(rules, name, { ...rule, severity: getSeverityLevel(src.rules, name, rulesetSeverity) });
+      } else {
+        processRule(rules, name, rulesetSeverity);
+      }
+    } else {
+      processRule(rules, name, rule);
+    }
+  }
+
+  return target;
+}
+
+const ROOT_DESCRIPTOR = Symbol('root-descriptor');
+
+function markRule(rule: Rule) {
+  if (!(ROOT_DESCRIPTOR in rule)) {
+    Object.defineProperty(rule, ROOT_DESCRIPTOR, {
+      configurable: false,
+      enumerable: false,
+      writable: false,
+      value: copyRule(rule),
+    });
+  }
+}
+
+function updateRootRule(root: Rule, newRule: Rule | null) {
+  markRule(root);
+  Object.assign(root[ROOT_DESCRIPTOR], copyRule(newRule === null ? root : Object.assign(root, newRule)));
+}
+
+function copyRule(rule: Rule) {
+  return cloneDeep(rule);
+}
+
+function processRule(rules: FileRuleCollection, name: string, rule: FileRule | FileRulesetSeverity) {
+  const existingRule = rules[name];
+
+  switch (typeof rule) {
+    case 'boolean':
+    case 'string':
+    case 'number':
+      // what if rule does not exist (yet)? throw, store the invalid state somehow?
+      if (isValidRule(existingRule)) {
+        existingRule.severity = getSeverityLevel(rules, name, rule);
+      }
+
+      break;
+    case 'object':
+      if (Array.isArray(rule)) {
+        processRule(rules, name, rule[0]);
+
+        if (isValidRule(existingRule) && rule.length === 2 && rule[1] !== undefined) {
+          if ('functionOptions' in existingRule.then) {
+            existingRule.then.functionOptions = rule[1];
+          }
+
+          updateRootRule(existingRule, null);
+        }
+      } else if (isValidRule(existingRule)) {
+        updateRootRule(existingRule, rule);
+      } else {
+        // new rule
+        markRule(rule);
+        rules[name] = rule;
+        if (rule.severity === undefined) {
+          rule.severity = DEFAULT_SEVERITY_LEVEL;
+        }
+      }
+
+      break;
+    default:
+      throw new Error('Invalid value for a rule');
+  }
+}
+
+function isValidRule(rule: FileRule): rule is Rule {
+  return typeof rule === 'object' && rule !== null && !Array.isArray(rule) && ('given' in rule || 'then' in rule);
+}
+
+function getSeverityLevel(
+  rules: FileRuleCollection,
+  name: string,
+  rule: FileRule | FileRulesetSeverity,
+): DiagnosticSeverity | HumanReadableDiagnosticSeverity {
+  const existingRule = rules[name];
+
+  if (!isValidRule(existingRule)) return 'off';
+
+  const existingSeverity = existingRule.severity !== undefined ? existingRule.severity : DEFAULT_SEVERITY_LEVEL;
+
+  if (rule === 'recommended') {
+    return existingRule.recommended ? existingSeverity : 'off';
+  }
+
+  if (rule === 'all') {
+    return existingSeverity;
+  }
+
+  switch (typeof rule) {
+    case 'number':
+    case 'string':
+      return rule;
+    case 'boolean':
+      return rule ? existingSeverity : 'off';
+    default:
+      return 'off';
+  }
+}
diff --git a/src/rulesets/oas/index.ts b/src/rulesets/oas/index.ts
index 045411dd..be02e7af 100644
--- a/src/rulesets/oas/index.ts
+++ b/src/rulesets/oas/index.ts
@@ -1,12 +1,6 @@
-import { resolve } from 'path';
 import { FunctionCollection } from '../../types';
 import { readRulesFromRulesets } from '../reader';
 
-export const operationPath =
-  "$..paths.*[?( name() === 'get' || name() === 'put' || name() === 'post'" +
-  " || name() === 'delete' || name() === 'options' || name() === 'head'" +
-  " || name() === 'patch' || name() === 'trace' )]";
-
 export const commonOasFunctions = (): FunctionCollection => {
   return {
     oasPathParam: require('./functions/oasPathParam').oasPathParam,
@@ -18,6 +12,4 @@ export const commonOasFunctions = (): FunctionCollection => {
   };
 };
 
-export const rules = async () => {
-  return readRulesFromRulesets(resolve(__dirname, 'ruleset.json'));
-};
+export const rules = async () => readRulesFromRulesets(require.resolve('./rules.json'));
diff --git a/src/rulesets/oas2/index.ts b/src/rulesets/oas2/index.ts
index 21af9586..009a0193 100644
--- a/src/rulesets/oas2/index.ts
+++ b/src/rulesets/oas2/index.ts
@@ -1,8 +1,5 @@
-import { resolve } from 'path';
 import { readRulesFromRulesets } from '../reader';
 
 export { commonOasFunctions as oas2Functions } from '../oas';
 
-export const rules = async () => {
-  return readRulesFromRulesets(resolve(__dirname, 'ruleset.json'));
-};
+export const rules = async () => readRulesFromRulesets(require.resolve('./rules.json'));
diff --git a/src/rulesets/oas2/ruleset.json b/src/rulesets/oas2/rules.json
similarity index 99%
rename from src/rulesets/oas2/ruleset.json
rename to src/rulesets/oas2/rules.json
index 4bd9527e..24e186e1 100644
--- a/src/rulesets/oas2/ruleset.json
+++ b/src/rulesets/oas2/rules.json
@@ -1,7 +1,5 @@
 {
-  "extends": [
-    "../oas/ruleset.json"
-  ],
+  "extends": ["spectral:oas"],
   "rules": {
     "oas2-schema": {
       "summary": "Validate structure of OpenAPIv2 specification.",
diff --git a/src/rulesets/oas3/index.ts b/src/rulesets/oas3/index.ts
index 6a189b79..ddb9b671 100644
--- a/src/rulesets/oas3/index.ts
+++ b/src/rulesets/oas3/index.ts
@@ -1,8 +1,5 @@
-import { resolve } from 'path';
 import { readRulesFromRulesets } from '../reader';
 
 export { commonOasFunctions as oas3Functions } from '../oas';
 
-export const rules = async () => {
-  return readRulesFromRulesets(resolve(__dirname, 'ruleset.json'));
-};
+export const rules = async () => readRulesFromRulesets(require.resolve('./rules.json'));
diff --git a/src/rulesets/oas3/ruleset.json b/src/rulesets/oas3/rules.json
similarity index 99%
rename from src/rulesets/oas3/ruleset.json
rename to src/rulesets/oas3/rules.json
index 26020ffc..cdfde88d 100644
--- a/src/rulesets/oas3/ruleset.json
+++ b/src/rulesets/oas3/rules.json
@@ -1,7 +1,5 @@
 {
-  "extends": [
-    "../oas/ruleset.json"
-  ],
+  "extends": ["spectral:oas"],
   "rules": {
     "oas3-schema": {
       "summary": "Validate structure of OpenAPIv3 specification.",
diff --git a/src/rulesets/path.ts b/src/rulesets/path.ts
deleted file mode 100644
index 4143b7ef..00000000
--- a/src/rulesets/path.ts
+++ /dev/null
@@ -1,9 +0,0 @@
-const URL = require('url').URL;
-import { dirname, resolve } from 'path';
-
-export function resolvePath(from: string, to: string) {
-  if (from.startsWith('http')) {
-    return new URL(to, from).href;
-  }
-  return resolve(dirname(resolve(process.cwd(), from)), to);
-}
diff --git a/src/rulesets/reader.ts b/src/rulesets/reader.ts
index 4bd898f9..017de54a 100644
--- a/src/rulesets/reader.ts
+++ b/src/rulesets/reader.ts
@@ -1,53 +1,41 @@
-import { merge } from 'lodash';
+import { parse } from '@stoplight/yaml';
 import { readParsable } from '../fs/reader';
 import { RuleCollection } from '../types';
-import { IRulesetFile } from '../types/ruleset';
-import { formatAjv } from './ajv';
-import { resolvePath } from './path';
-import { validateRuleset } from './validation';
-
-export async function readRulesFromRulesets(...files: string[]): Promise<RuleCollection> {
-  const rulesets = await Promise.all(files.map(file => readRulesFromRuleset(file)));
-  return merge({}, ...rulesets);
-}
+import { IRuleset, IRulesetFile } from '../types/ruleset';
+import { findRuleset } from './finder';
+import { mergeRulesets } from './merger';
+import { assertValidRuleset } from './validation';
 
-export type Logger = {
-  log: (message?: string | undefined, ...args: any[]) => void;
-  error: (
-    input: string | Error,
-    options?:
-      | {
-          code?: string | undefined;
-          exit?: number | undefined;
-        }
-      | undefined,
-  ) => never;
-};
-
-async function readRulesFromRuleset(file: string): Promise<RuleCollection> {
-  const parsed = await readParsable(file, 'utf8');
-  const { data: ruleset } = parsed;
-  const errors = validateRuleset(ruleset);
-
-  if (errors.length) {
-    throw {
-      messages: [formatAjv(errors), `Provided ruleset '${file}' is not valid`],
-    };
-  }
+export async function readRulesFromRulesets(...uris: string[]): Promise<RuleCollection> {
+  const base: IRuleset = {
+    rules: {},
+  };
 
-  const extendz = (ruleset as IRulesetFile).extends;
-  let extendedRules = {};
-  if (extendz && extendz.length) {
-    extendedRules = await blendRuleCollections(
-      extendz.map(extend => {
-        return readRulesFromRuleset(resolvePath(file, extend));
-      }),
-    );
+  for (const uri of uris) {
+    mergeRulesets(base, await readRulesFromRuleset('', uri));
   }
 
-  return merge(extendedRules, ruleset.rules);
+  return base.rules;
 }
 
-export async function blendRuleCollections(futureCollections: Array<Promise<RuleCollection>>) {
-  return Promise.all(futureCollections).then(collections => merge({}, ...collections));
+async function readRulesFromRuleset(baseUri: string, uri: string): Promise<IRulesetFile> {
+  const ruleset = assertValidRuleset(parse(await readParsable(await findRuleset(baseUri, uri), 'utf8')));
+
+  const newRuleset: IRulesetFile = {
+    rules: {},
+  };
+
+  const extendz = ruleset.extends;
+
+  if (extendz && extendz.length) {
+    for (const extended of extendz) {
+      if (Array.isArray(extended)) {
+        mergeRulesets(newRuleset, await readRulesFromRuleset(uri, extended[0]), extended[1]);
+      } else {
+        mergeRulesets(newRuleset, await readRulesFromRuleset(uri, extended));
+      }
+    }
+  }
+
+  return mergeRulesets(newRuleset, ruleset);
 }
diff --git a/src/rulesets/validation.ts b/src/rulesets/validation.ts
index 880a5a0b..c1d7d034 100644
--- a/src/rulesets/validation.ts
+++ b/src/rulesets/validation.ts
@@ -1,15 +1,27 @@
-const AJV = require('ajv');
+import { IRulesetFile } from '../types/ruleset';
+
 import { ErrorObject } from 'ajv';
+const AJV = require('ajv');
 import * as ruleSchema from '../meta/rule.schema.json';
 import * as rulesetSchema from '../meta/ruleset.schema.json';
-import { IRuleset } from '../types/ruleset';
 
 const ajv = new AJV({ allErrors: true, jsonPointers: true });
 const validate = ajv.addSchema(ruleSchema).compile(rulesetSchema);
 
-export function validateRuleset(ruleset: IRuleset): ErrorObject[] {
-  if (validate(ruleset)) {
-    return [];
+export function assertValidRuleset(ruleset: unknown): IRulesetFile {
+  if (ruleset === null || typeof ruleset !== 'object') {
+    throw new Error('Provided ruleset is not an object');
+  }
+
+  if (!('rules' in ruleset!)) {
+    throw new Error('Ruleset must have rules property');
+  }
+
+  if (!validate(ruleset)) {
+    throw new Error(
+      (validate.errors as ErrorObject[]).map(({ message, dataPath }) => `${dataPath} ${message}`).join('\n'),
+    );
   }
-  return validate.errors || [];
+
+  return ruleset as IRulesetFile;
 }
diff --git a/src/spectral.ts b/src/spectral.ts
index 5068aece..e48e87b0 100644
--- a/src/spectral.ts
+++ b/src/spectral.ts
@@ -220,6 +220,7 @@ const formatResolverErrors = (resolved: Resolved): IRuleResult[] => {
         message: prettyPrintResolverError(error.message),
         severity: DiagnosticSeverity.Error,
         range: location.range,
+        source: resolved.spec.source,
       });
     }
 
diff --git a/src/types/config.ts b/src/types/config.ts
index 74f357d7..55efc7b6 100644
--- a/src/types/config.ts
+++ b/src/types/config.ts
@@ -1,23 +1,15 @@
-export enum ConfigFormat {
+export enum OutputFormat {
   JSON = 'json',
   STYLISH = 'stylish',
 }
 
-export enum ConfigCommand {
-  LINT = 'lint',
-}
-
 export interface ILintConfig {
   encoding: string;
-  format: ConfigFormat;
+  format: OutputFormat;
   maxResults?: number;
   output?: string;
   ruleset?: string[];
   skipRule?: string[];
   verbose: boolean;
-}
-
-export interface IConfig {
-  extends?: string;
-  lint: ILintConfig;
+  quiet?: boolean;
 }
diff --git a/src/types/rule.ts b/src/types/rule.ts
index 4638f7eb..1576d9a8 100644
--- a/src/types/rule.ts
+++ b/src/types/rule.ts
@@ -16,7 +16,7 @@ export interface IRule<T = string, O = any> {
   description?: string;
 
   // The severity of results this rule generates
-  severity?: DiagnosticSeverity;
+  severity?: DiagnosticSeverity | HumanReadableDiagnosticSeverity;
 
   // Tags attached to the rule, which can be used for organizational purposes
   tags?: string[];
@@ -96,3 +96,5 @@ export interface ISchemaPathOptions {
   field?: string;
 }
 export type SchemaPathRule = IRule<RuleFunction.SCHEMAPATH, ISchemaPathOptions>;
+
+export type HumanReadableDiagnosticSeverity = 'error' | 'warn' | 'info' | 'hint' | 'off';
diff --git a/src/types/ruleset.ts b/src/types/ruleset.ts
index 4c07a747..5bc758c4 100644
--- a/src/types/ruleset.ts
+++ b/src/types/ruleset.ts
@@ -1,10 +1,20 @@
+import { Dictionary } from '@stoplight/types';
+import { DiagnosticSeverity } from '@stoplight/types/dist';
+import { HumanReadableDiagnosticSeverity, Rule } from './rule';
 import { RuleCollection } from './spectral';
 
+export type FileRuleSeverity = DiagnosticSeverity | HumanReadableDiagnosticSeverity | boolean;
+export type FileRulesetSeverity = 'off' | 'recommended' | 'all';
+
+export type FileRule = Rule | FileRuleSeverity | [FileRuleSeverity] | [FileRuleSeverity, object];
+
+export type FileRuleCollection = Dictionary<FileRule, string>;
+
 export interface IRuleset {
   rules: RuleCollection;
 }
 
 export interface IRulesetFile {
-  extends?: string[];
-  rules: RuleCollection;
+  extends?: Array<string | [string, FileRulesetSeverity]>;
+  rules: FileRuleCollection;
 }
diff --git a/tsconfig.json b/tsconfig.json
index 1f573dbe..790ac656 100644
--- a/tsconfig.json
+++ b/tsconfig.json
@@ -1,6 +1,5 @@
 {
   "include": ["src"],
-  "exclude": ["node_modules", "**/__tests__/**"],
   "compilerOptions": {
     "moduleResolution": "node",
     "target": "es6",
diff --git a/yarn.lock b/yarn.lock
index 3f35d502..f1888e9b 100644
--- a/yarn.lock
+++ b/yarn.lock
@@ -567,6 +567,11 @@
     jsonc-parser "2.1.0"
     lodash "4.x.x"
 
+"@stoplight/path@^1.0.1":
+  version "1.0.1"
+  resolved "https://registry.yarnpkg.com/@stoplight/path/-/path-1.0.1.tgz#0350a33ccc122ae05f940d1cf19531f78660f6f5"
+  integrity sha512-P/fHhBtnmwv3rZwOMZL1BTGzGInf/GW9HVNkqAyNx22mEOmV0gfghRnhQNOkFDwyHdt9V7eJ06cjhMzoKTfM1A==
+
 "@stoplight/types@9.0.2", "@stoplight/types@^9.0.2":
   version "9.0.2"
   resolved "https://registry.yarnpkg.com/@stoplight/types/-/types-9.0.2.tgz#1978287d2cfbcb176d00300fa158d5a1daf63c00"


Test patch:
diff --git a/src/__tests__/linter.test.ts b/src/__tests__/linter.test.ts
index 787e7242..14922572 100644
--- a/src/__tests__/linter.test.ts
+++ b/src/__tests__/linter.test.ts
@@ -1,12 +1,15 @@
+import * as path from '@stoplight/path';
 import { DiagnosticSeverity } from '@stoplight/types';
 import * as fs from 'fs';
-import * as path from 'path';
+import { cloneDeep } from 'lodash';
+import { mergeRulesets } from '../rulesets/merger';
 import { oas2Functions } from '../rulesets/oas2';
-import * as oas2Ruleset from '../rulesets/oas2/ruleset.json';
+import * as oas2Ruleset from '../rulesets/oas2/rules.json';
 import { oas3Functions } from '../rulesets/oas3';
-import * as oas3Ruleset from '../rulesets/oas3/ruleset.json';
+import * as oas3Ruleset from '../rulesets/oas3/rules.json';
 import { Spectral } from '../spectral';
 import { RuleCollection } from '../types';
+import { IRulesetFile } from '../types/ruleset';
 
 const invalidSchema = fs.readFileSync(
   path.join(__dirname, './__fixtures__/petstore.invalid-schema.oas3.yaml'),
@@ -146,6 +149,63 @@ describe('linter', () => {
     expect(result[0]).toHaveProperty('severity', DiagnosticSeverity.Hint);
   });
 
+  test('should not report anything for disabled rules', async () => {
+    spectral.addFunctions(oas3Functions());
+    spectral.addRules(mergeRulesets(cloneDeep(oas3Ruleset) as IRulesetFile, {
+      rules: {
+        'valid-example': 'off',
+      },
+    }).rules as RuleCollection);
+
+    const result = await spectral.run(invalidSchema);
+
+    expect(result).toEqual([
+      expect.objectContaining({
+        code: 'invalid-ref',
+      }),
+      expect.objectContaining({
+        code: 'invalid-ref',
+      }),
+    ]);
+  });
+
+  test('should support human readable severity levels', async () => {
+    spectral.addFunctions(oas2Functions());
+
+    spectral.addRules({
+      rule1: {
+        given: '$.x',
+        severity: 'error',
+        then: {
+          function: 'truthy',
+        },
+      },
+      rule2: {
+        given: '$.y',
+        severity: 'warn',
+        then: {
+          function: 'truthy',
+        },
+      },
+    });
+
+    const result = await spectral.run({
+      x: false,
+      y: '',
+    });
+
+    expect(result).toEqual([
+      expect.objectContaining({
+        code: 'rule1',
+        severity: DiagnosticSeverity.Error,
+      }),
+      expect.objectContaining({
+        code: 'rule2',
+        severity: DiagnosticSeverity.Warning,
+      }),
+    ]);
+  });
+
   test('should include parser diagnostics', async () => {
     spectral.addRules(oas2Ruleset.rules as RuleCollection);
     spectral.addFunctions(oas2Functions());
diff --git a/src/cli/commands/__tests__/__fixtures__/config.output.yml b/src/cli/commands/__tests__/__fixtures__/config.output.yml
deleted file mode 100644
index 8f14af88..00000000
--- a/src/cli/commands/__tests__/__fixtures__/config.output.yml
+++ /dev/null
@@ -1,3 +0,0 @@
----
-lint:
-  output: results.json
diff --git a/src/cli/commands/__tests__/__fixtures__/config.ruleset.invalid.yml b/src/cli/commands/__tests__/__fixtures__/config.ruleset.invalid.yml
deleted file mode 100644
index 8b1a75f9..00000000
--- a/src/cli/commands/__tests__/__fixtures__/config.ruleset.invalid.yml
+++ /dev/null
@@ -1,4 +0,0 @@
----
-lint:
-  ruleset:
-    - src/cli/commands/__tests__/__fixtures__/ruleset-invalid.yaml
diff --git a/src/cli/commands/__tests__/__fixtures__/config.ruleset.yml b/src/cli/commands/__tests__/__fixtures__/config.ruleset.yml
deleted file mode 100644
index 82d933c3..00000000
--- a/src/cli/commands/__tests__/__fixtures__/config.ruleset.yml
+++ /dev/null
@@ -1,4 +0,0 @@
----
-lint:
-  ruleset:
-    - src/cli/commands/__tests__/__fixtures__/ruleset-valid.yaml
diff --git a/src/cli/commands/__tests__/__fixtures__/config.yml b/src/cli/commands/__tests__/__fixtures__/config.yml
deleted file mode 100644
index c408dcec..00000000
--- a/src/cli/commands/__tests__/__fixtures__/config.yml
+++ /dev/null
@@ -1,6 +0,0 @@
----
-lint:
-  encoding: utf8
-  format: json
-  maxResults: 5
-  verbose: true
diff --git a/src/cli/commands/__tests__/__fixtures__/spectral.yml b/src/cli/commands/__tests__/__fixtures__/spectral.yml
index 9d07456d..053fc1f9 100644
--- a/src/cli/commands/__tests__/__fixtures__/spectral.yml
+++ b/src/cli/commands/__tests__/__fixtures__/spectral.yml
@@ -1,5 +1,10 @@
----
-lint:
-  encoding: utf8
-  format: json
-  maxResults: 1
+rules:
+  info-matches-stoplight:
+    summary: Info must contain Stoplight
+    given: $.info
+    type: style
+    then:
+      field: title
+      function: pattern
+      functionOptions:
+        match: Stoplight
diff --git a/src/cli/commands/__tests__/lint.test.ts b/src/cli/commands/__tests__/lint.test.ts
index 5f5163ba..ea80bf2a 100644
--- a/src/cli/commands/__tests__/lint.test.ts
+++ b/src/cli/commands/__tests__/lint.test.ts
@@ -1,22 +1,18 @@
 import { test } from '@oclif/test';
+import { resolve } from '@stoplight/path';
 import * as fs from 'fs';
-import { resolve } from 'path';
 type SpyInstance = jest.SpyInstance;
 
 const invalidOas3SpecPath = resolve(__dirname, '__fixtures__/openapi-3.0-no-contact.yaml');
 const validOas3SpecPath = resolve(__dirname, '__fixtures__/openapi-3.0-valid.yaml');
 const oas2PetstoreSpecPath = resolve(__dirname, '../../../__tests__/__fixtures__/petstore.oas2.json');
-const validConfigPath = resolve(__dirname, '__fixtures__/config.yml');
-const outputConfigPath = resolve(__dirname, '__fixtures__/config.output.yml');
 const validCustomOas3SpecPath = resolve(__dirname, '__fixtures__/openapi-3.0-valid-custom.yaml');
 const invalidRulesetPath = resolve(__dirname, '__fixtures__/ruleset-invalid.yaml');
 const validRulesetPath = resolve(__dirname, '__fixtures__/ruleset-valid.yaml');
 const validNestedRulesetPath = resolve(__dirname, '__fixtures__/ruleset-extends-valid.yaml');
 const invalidNestedRulesetPath = resolve(__dirname, '__fixtures__/ruleset-extends-invalid.yaml');
-const validRulesetConfigPath = resolve(__dirname, '__fixtures__/config.ruleset.yml');
-const invalidRulesetConfigPath = resolve(__dirname, '__fixtures__/config.ruleset.invalid.yml');
-const standardOas3RulesetPath = resolve(__dirname, '../../../rulesets/oas3/ruleset.json');
-const standardOas2RulesetPath = resolve(__dirname, '../../../rulesets/oas2/ruleset.json');
+const standardOas3RulesetPath = resolve(__dirname, '../../../rulesets/oas3/rules.json');
+const standardOas2RulesetPath = resolve(__dirname, '../../../rulesets/oas2/rules.json');
 const draftRefSpec = resolve(__dirname, './__fixtures__/draft-ref.oas2.json');
 const draftNestedRefSpec = resolve(__dirname, './__fixtures__/draft-nested-ref.oas2.json');
 
@@ -142,11 +138,11 @@ describe('lint', () => {
         .command(['lint', validOas3SpecPath, '-r', invalidRulesetPath, '-r', validRulesetPath])
         .exit(2)
         .it('given one is valid other is not, outputs "invalid ruleset" error', ctx => {
-          expect(ctx.stdout).toContain(`/rules/rule-without-given-nor-them 	 should have required property 'given'`);
-          expect(ctx.stdout).toContain(`/rules/rule-without-given-nor-them 	 should have required property 'then'`);
-          expect(ctx.stdout).toContain(`/rules/rule-with-invalid-enum/severity 	 should be number`);
+          expect(ctx.stdout).toContain(`/rules/rule-without-given-nor-them should have required property 'given'`);
+          expect(ctx.stdout).toContain(`/rules/rule-without-given-nor-them should have required property 'then'`);
+          expect(ctx.stdout).toContain(`/rules/rule-with-invalid-enum/severity should be number`);
           expect(ctx.stdout).toContain(
-            `/rules/rule-with-invalid-enum/severity 	 should be equal to one of the allowed values`,
+            `/rules/rule-with-invalid-enum/severity should be equal to one of the allowed values`,
           );
         });
     });
@@ -163,11 +159,11 @@ describe('lint', () => {
         .command(['lint', validOas3SpecPath, '-r', invalidRulesetPath])
         .exit(2)
         .it('outputs "invalid ruleset" error', ctx => {
-          expect(ctx.stdout).toContain(`/rules/rule-without-given-nor-them 	 should have required property 'given'`);
-          expect(ctx.stdout).toContain(`/rules/rule-without-given-nor-them 	 should have required property 'then'`);
-          expect(ctx.stdout).toContain(`/rules/rule-with-invalid-enum/severity 	 should be number`);
+          expect(ctx.stdout).toContain(`/rules/rule-without-given-nor-them should have required property 'given'`);
+          expect(ctx.stdout).toContain(`/rules/rule-without-given-nor-them should have required property 'then'`);
+          expect(ctx.stdout).toContain(`/rules/rule-with-invalid-enum/severity should be number`);
           expect(ctx.stdout).toContain(
-            `/rules/rule-with-invalid-enum/severity 	 should be equal to one of the allowed values`,
+            `/rules/rule-with-invalid-enum/severity should be equal to one of the allowed values`,
           );
         });
 
@@ -290,19 +286,10 @@ describe('lint', () => {
       });
   });
 
-  describe('when using config file', () => {
+  describe('when using ruleset file', () => {
     test
       .stdout()
-      .command(['lint', invalidOas3SpecPath, '-c', validConfigPath])
-      .it('outputs warnings in json format', ctx => {
-        expect(ctx.stdout).toContain('"Info object should contain `contact` object."');
-        expect(ctx.stdout).toContain('"OpenAPI object info `description` must be present and non-empty string."');
-        expect(ctx.stdout).toContain('"OpenAPI `servers` must be present and non-empty array."');
-      });
-
-    test
-      .stdout()
-      .command(['lint', invalidOas3SpecPath, '-c', outputConfigPath])
+      .command(['lint', invalidOas3SpecPath, '-r', validRulesetPath])
       .it('saves results to a file', () => {
         expect(fs.writeFile).toHaveBeenCalledWith(
           'results.json',
@@ -314,28 +301,10 @@ describe('lint', () => {
 
     test
       .stdout()
-      .command(['lint', validOas3SpecPath, '-c', validRulesetConfigPath])
-      .it('outputs invalid ruleset error when invalid ruleset provided', ctx => {
-        expect(ctx.stdout).toContain(`5:10  warning  info-matches-stoplight  Info must contain Stoplight`);
-      });
-
-    test
-      .stdout()
-      .command(['lint', validOas3SpecPath, '-c', invalidRulesetConfigPath])
+      .command(['lint', validOas3SpecPath, '-r', invalidRulesetPath])
       .exit(2)
       .it('outputs invalid ruleset error when invalid ruleset provided', ctx => {
-        expect(ctx.stdout).toContain(`/rules/rule-without-given-nor-them 	 should have required property 'given'`);
-      });
-  });
-
-  describe('when using config file and command args', () => {
-    test
-      .stdout()
-      .command(['lint', invalidOas3SpecPath, '-c', validConfigPath, '--max-results', '1'])
-      .it('setting --max-results to 1 will override config value of 5', ctx => {
-        expect(ctx.stdout).toContain('"Info object should contain `contact` object."');
-        expect(ctx.stdout).not.toContain('"OpenAPI object info `description` must be present and non-empty string."');
-        expect(ctx.stdout).not.toContain('"OpenAPI `servers` must be present and non-empty array."');
+        expect(ctx.stdout).toContain(`/rules/rule-without-given-nor-them should have required property 'given'`);
       });
   });
 
@@ -358,8 +327,8 @@ describe('lint', () => {
     test
       .stdout()
       .command(['lint', invalidOas3SpecPath])
-      .it('outputs data in format from default config file', ctx => {
-        expect(ctx.stdout).toContain('"Info object should contain `contact` object."');
+      .it('respects rules from a ruleset file', ctx => {
+        expect(ctx.stdout).toContain(' 5:10  warning  info-matches-stoplight  Info must contain Stoplight');
       });
   });
 
diff --git a/src/config/__tests__/__fixtures__/config.invalid.empty.yaml b/src/config/__tests__/__fixtures__/config.invalid.empty.yaml
deleted file mode 100644
index ed97d539..00000000
--- a/src/config/__tests__/__fixtures__/config.invalid.empty.yaml
+++ /dev/null
@@ -1,1 +0,0 @@
----
diff --git a/src/config/__tests__/__fixtures__/config.invalid.json b/src/config/__tests__/__fixtures__/config.invalid.json
deleted file mode 100644
index dcb513df..00000000
--- a/src/config/__tests__/__fixtures__/config.invalid.json
+++ /dev/null
@@ -1,7 +0,0 @@
-  "lint": {
-    "encoding": "utf8",
-    "format": "json",
-    "maxResults": 5,
-    "verbose": true
-  }
-}
diff --git a/src/config/__tests__/__fixtures__/config.invalid.nolint.yaml b/src/config/__tests__/__fixtures__/config.invalid.nolint.yaml
deleted file mode 100644
index c8b76882..00000000
--- a/src/config/__tests__/__fixtures__/config.invalid.nolint.yaml
+++ /dev/null
@@ -1,5 +0,0 @@
----
-encoding: utf8
-format: json
-maxResults: 5
-verbose: true
diff --git a/src/config/__tests__/__fixtures__/config.invalid.yaml b/src/config/__tests__/__fixtures__/config.invalid.yaml
deleted file mode 100644
index 9fd62835..00000000
--- a/src/config/__tests__/__fixtures__/config.invalid.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
----
-lint:
-  encoding: utf8
-  format: json
-  maxResults: 5
-  verbose true
diff --git a/src/config/__tests__/__fixtures__/config.json b/src/config/__tests__/__fixtures__/config.json
deleted file mode 100644
index 70cabfc2..00000000
--- a/src/config/__tests__/__fixtures__/config.json
+++ /dev/null
@@ -1,8 +0,0 @@
-{
-  "lint": {
-    "encoding": "utf8",
-    "format": "json",
-    "maxResults": 5,
-    "verbose": true
-  }
-}
diff --git a/src/config/__tests__/__fixtures__/config.yaml b/src/config/__tests__/__fixtures__/config.yaml
deleted file mode 100644
index c408dcec..00000000
--- a/src/config/__tests__/__fixtures__/config.yaml
+++ /dev/null
@@ -1,6 +0,0 @@
----
-lint:
-  encoding: utf8
-  format: json
-  maxResults: 5
-  verbose: true
diff --git a/src/config/__tests__/__fixtures__/config.yml b/src/config/__tests__/__fixtures__/config.yml
deleted file mode 100644
index c408dcec..00000000
--- a/src/config/__tests__/__fixtures__/config.yml
+++ /dev/null
@@ -1,6 +0,0 @@
----
-lint:
-  encoding: utf8
-  format: json
-  maxResults: 5
-  verbose: true
diff --git a/src/config/__tests__/__fixtures__/empty.json b/src/config/__tests__/__fixtures__/empty.json
new file mode 100644
index 00000000..e69de29b
diff --git a/src/config/__tests__/__fixtures__/ruleset/config.ruleset.string.yml b/src/config/__tests__/__fixtures__/ruleset/config.ruleset.string.yml
deleted file mode 100644
index 8c6ca297..00000000
--- a/src/config/__tests__/__fixtures__/ruleset/config.ruleset.string.yml
+++ /dev/null
@@ -1,3 +0,0 @@
----
-lint:
-  ruleset: src/cli/commands/__tests__/__fixtures__/ruleset-invalid.yaml
diff --git a/src/config/__tests__/__fixtures__/ruleset/config.ruleset.yml b/src/config/__tests__/__fixtures__/ruleset/config.ruleset.yml
deleted file mode 100644
index 8b1a75f9..00000000
--- a/src/config/__tests__/__fixtures__/ruleset/config.ruleset.yml
+++ /dev/null
@@ -1,4 +0,0 @@
----
-lint:
-  ruleset:
-    - src/cli/commands/__tests__/__fixtures__/ruleset-invalid.yaml
diff --git a/src/config/__tests__/__fixtures__/spectral.json b/src/config/__tests__/__fixtures__/spectral.json
new file mode 100644
index 00000000..e69de29b
diff --git a/src/config/__tests__/__fixtures__/yaml/.spectral.yaml b/src/config/__tests__/__fixtures__/yaml/.spectral.yaml
new file mode 100644
index 00000000..e69de29b
diff --git a/src/config/__tests__/config.test.ts b/src/config/__tests__/config.test.ts
index 9c1588fb..bac29e2b 100644
--- a/src/config/__tests__/config.test.ts
+++ b/src/config/__tests__/config.test.ts
@@ -1,73 +1,19 @@
-import * as path from 'path';
-import { ConfigCommand } from '../../types/config';
-import { load } from '../configLoader';
+import * as path from '@stoplight/path';
+import { getDefaultRulesetFile } from '../configLoader';
 
-const getFixture = (filePath: string) => path.resolve(__dirname, './__fixtures__', filePath);
+const ROOT = path.join(__dirname, './__fixtures__');
 
 describe('config loading', () => {
-  let exampleConfig: any;
-  beforeEach(() => {
-    exampleConfig = {
-      lint: {
-        encoding: 'utf8',
-        format: 'json',
-        verbose: true,
-        maxResults: 5,
-      },
-    };
-  });
-  test('should load JSON config', async () => {
-    const configPath = getFixture('config.json');
-    const config = await load(configPath, ConfigCommand.LINT);
-    expect(config).toEqual(exampleConfig);
-  });
-
-  test('should load YAML config', async () => {
-    const configPath = getFixture('config.yaml');
-    const config = await load(configPath, ConfigCommand.LINT);
-    expect(config).toEqual(exampleConfig);
-  });
-
-  test('should load YML config', async () => {
-    const configPath = getFixture('config.yml');
-    const config = await load(configPath, ConfigCommand.LINT);
-    expect(config).toEqual(exampleConfig);
-  });
-
-  test('should throw error if file does not exist', async () => {
-    const configPath = getFixture('config.ghost.json');
-    await expect(load(configPath, ConfigCommand.LINT)).rejects.toThrowError(/does not exist/);
-  });
-
-  test('should throw error if JSON file is invalid', async () => {
-    const configPath = getFixture('config.invalid.json');
-    await expect(load(configPath, ConfigCommand.LINT)).rejects.toThrowError();
-  });
-
-  test('should throw error if YAML file is invalid', async () => {
-    const configPath = getFixture('config.invalid.yaml');
-    await expect(load(configPath, ConfigCommand.LINT)).rejects.toThrowError();
-  });
-
-  test('should load config with ruleset', async () => {
-    const configPath = getFixture('./ruleset/config.ruleset.yml');
-    const config = await load(configPath, ConfigCommand.LINT);
-    expect(config.lint.ruleset).toEqual(['src/cli/commands/__tests__/__fixtures__/ruleset-invalid.yaml']);
-  });
-
-  test('should load config with string ruleset', async () => {
-    const configPath = getFixture('./ruleset/config.ruleset.string.yml');
-    const config = await load(configPath, ConfigCommand.LINT);
-    expect(config.lint.ruleset).toEqual(['src/cli/commands/__tests__/__fixtures__/ruleset-invalid.yaml']);
-  });
-
-  test('should throw error if there is no lint specified', async () => {
-    const configPath = getFixture('config.invalid.nolint.yaml');
-    await expect(load(configPath, ConfigCommand.LINT)).rejects.toThrowError('Missing lint command in config file');
-  });
-
-  test('should throw error if config is empty', async () => {
-    const configPath = getFixture('config.invalid.empty.yaml');
-    await expect(load(configPath, ConfigCommand.LINT)).rejects.toThrowError('Missing config');
+  describe('getDefaultRulesetFile', () => {
+    test('should resolve with null if a default config does not exist', () => {
+      return expect(getDefaultRulesetFile(path.join(ROOT, './empty'))).resolves.toBeNull();
+    });
+
+    test('should resolve with a path to found config if there is any matching', async () => {
+      await expect(getDefaultRulesetFile(ROOT)).resolves.toEqual(path.join(ROOT, './spectral.json'));
+      await expect(getDefaultRulesetFile(path.join(ROOT, './yaml'))).resolves.toEqual(
+        path.join(ROOT, './yaml/.spectral.yaml'),
+      );
+    });
   });
 });
diff --git a/src/rulesets/__tests__/__fixtures__/extends-oas2-ruleset.json b/src/rulesets/__tests__/__fixtures__/extends-oas2-ruleset.json
new file mode 100644
index 00000000..a600a40c
--- /dev/null
+++ b/src/rulesets/__tests__/__fixtures__/extends-oas2-ruleset.json
@@ -0,0 +1,12 @@
+{
+  "extends": [["spectral:oas2", "all"]],
+  "rules": {
+    "valid-rule": {
+      "summary": "should be OK",
+      "given": "$.info",
+      "then": {
+        "function": "truthy"
+      }
+    }
+  }
+}
diff --git a/src/rulesets/__tests__/__fixtures__/extends-oas2-with-override-ruleset.json b/src/rulesets/__tests__/__fixtures__/extends-oas2-with-override-ruleset.json
new file mode 100644
index 00000000..9635c3d7
--- /dev/null
+++ b/src/rulesets/__tests__/__fixtures__/extends-oas2-with-override-ruleset.json
@@ -0,0 +1,13 @@
+{
+  "extends": [["spectral:oas2", "all"]],
+  "rules": {
+    "operation-security-defined": "off",
+    "operation-2xx-response": {
+      "summary": "should be OK",
+      "given": "$.info",
+      "then": {
+        "function": "truthy"
+      }
+    }
+  }
+}
diff --git a/src/rulesets/__tests__/__fixtures__/ruleset-invalid.json b/src/rulesets/__tests__/__fixtures__/invalid-ruleset.json
similarity index 100%
rename from src/rulesets/__tests__/__fixtures__/ruleset-invalid.json
rename to src/rulesets/__tests__/__fixtures__/invalid-ruleset.json
diff --git a/src/rulesets/__tests__/__fixtures__/valid-ruleset.json b/src/rulesets/__tests__/__fixtures__/valid-flat-ruleset.json
similarity index 98%
rename from src/rulesets/__tests__/__fixtures__/valid-ruleset.json
rename to src/rulesets/__tests__/__fixtures__/valid-flat-ruleset.json
index bb04ec9e..0f6d4b22 100644
--- a/src/rulesets/__tests__/__fixtures__/valid-ruleset.json
+++ b/src/rulesets/__tests__/__fixtures__/valid-flat-ruleset.json
@@ -8,4 +8,4 @@
       }
     }
   }
-}
\ No newline at end of file
+}
diff --git a/src/rulesets/__tests__/__fixtures__/valid-require-info-ruleset.yaml b/src/rulesets/__tests__/__fixtures__/valid-require-info-ruleset.yaml
new file mode 100644
index 00000000..b30a9243
--- /dev/null
+++ b/src/rulesets/__tests__/__fixtures__/valid-require-info-ruleset.yaml
@@ -0,0 +1,6 @@
+rules:
+  require-info:
+    summary: should be OK
+    given: "$.info"
+    then:
+      function: truthy
diff --git a/src/rulesets/__tests__/__snapshots__/validation.unit.ts.snap b/src/rulesets/__tests__/__snapshots__/validation.unit.ts.snap
deleted file mode 100644
index 662669ac..00000000
--- a/src/rulesets/__tests__/__snapshots__/validation.unit.ts.snap
+++ /dev/null
@@ -1,59 +0,0 @@
-// Jest Snapshot v1, https://goo.gl/fbAQLP
-
-exports[`Ruleset Validation given invalid ruleset should emit errors 1`] = `
-"[
-  {
-    \\"keyword\\": \\"required\\",
-    \\"dataPath\\": \\"/rules/no-given-no-then\\",
-    \\"schemaPath\\": \\"#/required\\",
-    \\"params\\": {
-      \\"missingProperty\\": \\"given\\"
-    },
-    \\"message\\": \\"should have required property 'given'\\"
-  },
-  {
-    \\"keyword\\": \\"required\\",
-    \\"dataPath\\": \\"/rules/no-given-no-then\\",
-    \\"schemaPath\\": \\"#/required\\",
-    \\"params\\": {
-      \\"missingProperty\\": \\"then\\"
-    },
-    \\"message\\": \\"should have required property 'then'\\"
-  },
-  {
-    \\"keyword\\": \\"type\\",
-    \\"dataPath\\": \\"/rules/rule-with-invalid-enum/severity\\",
-    \\"schemaPath\\": \\"#/properties/severity/type\\",
-    \\"params\\": {
-      \\"type\\": \\"number\\"
-    },
-    \\"message\\": \\"should be number\\"
-  },
-  {
-    \\"keyword\\": \\"enum\\",
-    \\"dataPath\\": \\"/rules/rule-with-invalid-enum/severity\\",
-    \\"schemaPath\\": \\"#/properties/severity/enum\\",
-    \\"params\\": {
-      \\"allowedValues\\": [
-        0,
-        1,
-        2,
-        3
-      ]
-    },
-    \\"message\\": \\"should be equal to one of the allowed values\\"
-  },
-  {
-    \\"keyword\\": \\"enum\\",
-    \\"dataPath\\": \\"/rules/rule-with-invalid-enum/type\\",
-    \\"schemaPath\\": \\"#/properties/type/enum\\",
-    \\"params\\": {
-      \\"allowedValues\\": [
-        \\"style\\",
-        \\"validation\\"
-      ]
-    },
-    \\"message\\": \\"should be equal to one of the allowed values\\"
-  }
-]"
-`;
diff --git a/src/rulesets/__tests__/finder.test.ts b/src/rulesets/__tests__/finder.test.ts
new file mode 100644
index 00000000..0eb64b09
--- /dev/null
+++ b/src/rulesets/__tests__/finder.test.ts
@@ -0,0 +1,50 @@
+import * as path from '@stoplight/path';
+import { findRuleset } from '../finder';
+
+// @oclif/test packages requires @types/mocha, therefore we have 2 packages coming up with similar typings
+// TS is confused and prefers the mocha ones, so we need to instrument it to pick up the Jest ones
+declare var it: jest.It;
+
+describe('Rulesets finder', () => {
+  it('should join relative path', () => {
+    return expect(findRuleset(__filename, './finder.test.ts')).resolves.toEqual(
+      path.join(__dirname, './finder.test.ts'),
+    );
+  });
+
+  it('should join absolute path', () => {
+    return expect(findRuleset('a/b/c', '/d')).resolves.toEqual('/d');
+  });
+
+  it('should join http relative', () => {
+    return expect(findRuleset('http://www.example.com/a/b.json', 'd.json')).resolves.toEqual(
+      'http://www.example.com/a/d.json',
+    );
+  });
+
+  it('should resolve http absolute', () => {
+    return expect(findRuleset('http://www.example.com/a/b.json', 'http://www.acme.com/b.json')).resolves.toEqual(
+      'http://www.acme.com/b.json',
+    );
+  });
+
+  it('should support spectral built-in rules', () => {
+    return expect(findRuleset('/b/c/d', '@stoplight/spectral/rulesets/oas2/rules.json')).resolves.toEqual(
+      path.join(process.cwd(), 'src/rulesets/oas2/rules.json'),
+    );
+  });
+
+  it.each(['oas', 'oas2', 'oas3'])('should support spectral built-in %s ruleset shorthand', shorthand => {
+    return expect(findRuleset('', `spectral:${shorthand}`)).resolves.toEqual(
+      path.join(process.cwd(), `src/rulesets/${shorthand}/rules.json`),
+    );
+  });
+
+  it('should load local npm module if available', () => {
+    return expect(findRuleset('', '@stoplight/json')).resolves.toEqual(require.resolve('@stoplight/json'));
+  });
+
+  it('should point to unpkg.com if npm module if unavailable', () => {
+    return expect(findRuleset('', '@stoplight/foo-bar')).resolves.toEqual('https://unpkg.com/@stoplight/foo-bar');
+  });
+});
diff --git a/src/rulesets/__tests__/merger.unit.ts b/src/rulesets/__tests__/merger.unit.ts
new file mode 100644
index 00000000..ef358fd1
--- /dev/null
+++ b/src/rulesets/__tests__/merger.unit.ts
@@ -0,0 +1,319 @@
+import { DiagnosticSeverity } from '@stoplight/types';
+import { IRule } from '../../types';
+import { mergeRulesets } from '../merger';
+
+describe('Rulesets merger', () => {
+  const baseRule: IRule = {
+    summary: 'Operation must have at least one `2xx` response.',
+    given:
+      "$..paths.*[?( @property === 'get' || @property === 'put' || @property === 'post' || @property === 'delete' || @property === 'options' || @property === 'head' || @property === 'patch' || @property === 'trace' )]",
+    then: {
+      field: 'responses',
+      function: 'oasOp2xxResponse',
+      functionOptions: {
+        foo: 'bar',
+      },
+    },
+    recommended: true,
+    severity: 'warn',
+  };
+
+  it('performs a shallow merging', () => {
+    const ruleset = {
+      rules: {
+        test: JSON.parse(JSON.stringify(baseRule)),
+      },
+    };
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: {
+          ...JSON.parse(JSON.stringify(baseRule)),
+          then: {
+            field: 'info',
+            function: 'truthy',
+            functionOptions: {},
+          },
+        },
+      },
+    });
+
+    expect(ruleset).toHaveProperty('rules.test.then', {
+      field: 'info',
+      function: 'truthy',
+      functionOptions: {},
+    });
+  });
+
+  it('is possible to disable the rule using flag', () => {
+    const ruleset = {
+      rules: {
+        test: JSON.parse(JSON.stringify(baseRule)),
+      },
+    };
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: false,
+      },
+    });
+
+    expect(ruleset).toHaveProperty('rules.test.severity', 'off');
+  });
+
+  it('supports nested severity', () => {
+    const ruleset = {
+      rules: {
+        test: JSON.parse(JSON.stringify(baseRule)),
+      },
+    };
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: false,
+      },
+    });
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: 'error',
+      },
+    });
+
+    expect(ruleset).toHaveProperty('rules.test.severity', 'error');
+  });
+
+  it('merges the same rules', () => {
+    const ruleset = {
+      rules: {
+        test: JSON.parse(JSON.stringify(baseRule)),
+      },
+    };
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: {
+          summary: 'Operation must have at least one `2xx` response.',
+          given:
+            "$..paths.*[?( @property === 'get' || @property === 'put' || @property === 'post' || @property === 'delete' || @property === 'options' || @property === 'head' || @property === 'patch' || @property === 'trace' )]",
+          then: {
+            field: 'responses',
+            function: 'oasOp2xxResponse',
+          },
+          severity: 'error',
+        },
+      },
+    });
+
+    expect(ruleset).toHaveProperty('rules.test.severity', 'error');
+  });
+
+  it('prefers the most recent severity level', () => {
+    const ruleset = {
+      rules: {
+        test: JSON.parse(JSON.stringify(baseRule)),
+      },
+    };
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: {
+          summary: 'Operation must have at least one `2xx` response.',
+          given:
+            "$..paths.*[?( @property === 'get' || @property === 'put' || @property === 'post' || @property === 'delete' || @property === 'options' || @property === 'head' || @property === 'patch' || @property === 'trace' )]",
+          then: {
+            field: 'responses',
+            function: 'oasOp2xxResponse',
+          },
+          severity: 'error',
+        },
+      },
+    });
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: false,
+      },
+    });
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: true,
+      },
+    });
+
+    expect(ruleset).toHaveProperty('rules.test.severity', 'off');
+  });
+
+  it('includes new rules', () => {
+    const ruleset = {
+      rules: {},
+    };
+
+    mergeRulesets(ruleset, {
+      rules: {
+        example: JSON.parse(JSON.stringify(baseRule)),
+      },
+    });
+
+    expect(ruleset).toEqual({
+      rules: {
+        example: baseRule,
+      },
+    });
+  });
+
+  it('supports array-ish syntax', () => {
+    const ruleset = {
+      rules: {
+        test: JSON.parse(JSON.stringify(baseRule)),
+      },
+    };
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: ['off'],
+      },
+    });
+
+    expect(ruleset).toHaveProperty('rules.test.severity', 'off');
+  });
+
+  it('supports array-ish syntax', () => {
+    const ruleset = {
+      rules: {
+        test: JSON.parse(JSON.stringify(baseRule)),
+      },
+    };
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: ['off'],
+      },
+    });
+
+    expect(ruleset).toHaveProperty('rules.test.severity', 'off');
+  });
+
+  it('does not set functionOptions if rule does not implement it', () => {
+    const ruleset = {
+      rules: {
+        test: JSON.parse(JSON.stringify(baseRule)),
+      },
+    };
+
+    delete ruleset.rules.test.then.functionOptions;
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: ['off', { baz: 'bar' }],
+      },
+    });
+
+    expect(ruleset).not.toHaveProperty('rules.test.then.functionOptions');
+  });
+
+  it('provides support for custom functionOptions via array-ish syntax', () => {
+    const ruleset = {
+      rules: {
+        test: JSON.parse(JSON.stringify(baseRule)),
+      },
+    };
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: ['off', { baz: 'bar' }],
+      },
+    });
+
+    expect(ruleset).toHaveProperty('rules.test.then.functionOptions', { baz: 'bar' });
+  });
+
+  it('provides support to disable all rules present in a given ruleset', () => {
+    const ruleset = {
+      rules: {},
+    };
+
+    mergeRulesets(
+      ruleset,
+      {
+        rules: {
+          test: JSON.parse(JSON.stringify(baseRule)),
+          test2: JSON.parse(JSON.stringify(baseRule)),
+        },
+      },
+      'off',
+    );
+
+    expect(ruleset).toHaveProperty('rules.test.severity', 'off');
+    expect(ruleset).toHaveProperty('rules.test2.severity', 'off');
+  });
+
+  it('picks up recommended rules', () => {
+    const ruleset = {
+      rules: {},
+    };
+
+    mergeRulesets(
+      ruleset,
+      {
+        rules: {
+          test: JSON.parse(JSON.stringify(baseRule)),
+          test2: {
+            ...JSON.parse(JSON.stringify(baseRule)),
+            recommended: false,
+          },
+        },
+      },
+      'recommended',
+    );
+
+    expect(ruleset).toHaveProperty('rules.test.severity', 'warn');
+    expect(ruleset).toHaveProperty('rules.test2.severity', 'off');
+  });
+
+  it('sets warning as default severity level if a rule has no severity specified', () => {
+    const ruleset = {
+      rules: {},
+    };
+
+    const baseWithoutSeverity = JSON.parse(JSON.stringify(baseRule));
+    delete baseWithoutSeverity.severity;
+
+    mergeRulesets(ruleset, {
+      rules: {
+        test: baseWithoutSeverity,
+        test2: {
+          ...JSON.parse(JSON.stringify(baseRule)),
+          severity: DiagnosticSeverity.Error,
+        },
+      },
+    });
+
+    expect(ruleset).toHaveProperty('rules.test.severity', DiagnosticSeverity.Warning);
+    expect(ruleset).toHaveProperty('rules.test2.severity', DiagnosticSeverity.Error);
+  });
+
+  it('picks up all rules', () => {
+    const ruleset = {
+      rules: {},
+    };
+
+    mergeRulesets(
+      ruleset,
+      {
+        rules: {
+          test: JSON.parse(JSON.stringify(baseRule)),
+          test2: {
+            ...JSON.parse(JSON.stringify(baseRule)),
+            recommended: false,
+          },
+        },
+      },
+      'all',
+    );
+
+    expect(ruleset).toHaveProperty('rules.test.severity', 'warn');
+    expect(ruleset).toHaveProperty('rules.test2.severity', 'warn');
+  });
+});
diff --git a/src/rulesets/__tests__/path.unit.ts b/src/rulesets/__tests__/path.unit.ts
deleted file mode 100644
index a84d078e..00000000
--- a/src/rulesets/__tests__/path.unit.ts
+++ /dev/null
@@ -1,21 +0,0 @@
-import { resolvePath } from '../path';
-
-describe('Path', () => {
-  it('should resolve relative path', () => {
-    expect(resolvePath('a/b/c', 'd')).toEqual(`${process.cwd()}/a/b/d`);
-  });
-
-  it('should resolve absolute path', () => {
-    expect(resolvePath('a/b/c', '/d')).toEqual('/d');
-  });
-
-  it('should resolve http relative', () => {
-    expect(resolvePath('http://www.example.com/a/b.json', 'd.json')).toEqual('http://www.example.com/a/d.json');
-  });
-
-  it('should resolve http absolute', () => {
-    expect(resolvePath('http://www.example.com/a/b.json', 'http://www.acme.com/b.json')).toEqual(
-      'http://www.acme.com/b.json',
-    );
-  });
-});
diff --git a/src/rulesets/__tests__/reader.test.ts b/src/rulesets/__tests__/reader.test.ts
new file mode 100644
index 00000000..080662b0
--- /dev/null
+++ b/src/rulesets/__tests__/reader.test.ts
@@ -0,0 +1,121 @@
+import { Dictionary } from '@stoplight/types';
+import { DiagnosticSeverity } from '@stoplight/types/dist';
+import * as path from 'path';
+import { IRule } from '../../types';
+import { readRulesFromRulesets } from '../reader';
+
+const validFlatRuleset = path.join(__dirname, './__fixtures__/valid-flat-ruleset.json');
+const validFlatRuleset2 = path.join(__dirname, './__fixtures__/valid-require-info-ruleset.yaml');
+const invalidRuleset = path.join(__dirname, './__fixtures__/invalid-ruleset.json');
+const extendsOas2Ruleset = path.join(__dirname, './__fixtures__/extends-oas2-ruleset.json');
+const extendsOas2WithOverrideRuleset = path.join(__dirname, './__fixtures__/extends-oas2-with-override-ruleset.json');
+const oasRuleset = require('../oas/rules.json');
+const oas2Ruleset = require('../oas2/rules.json');
+
+describe('Rulesets reader', () => {
+  it('given flat, valid ruleset file should return rules', async () => {
+    expect(await readRulesFromRulesets(validFlatRuleset)).toEqual({
+      'valid-rule': {
+        given: '$.info',
+        summary: 'should be OK',
+        severity: DiagnosticSeverity.Warning,
+        then: {
+          function: 'truthy',
+        },
+      },
+    });
+  });
+
+  it('given two flat, valid ruleset files should return rules', async () => {
+    expect(await readRulesFromRulesets(validFlatRuleset, validFlatRuleset2)).toEqual({
+      'valid-rule': {
+        given: '$.info',
+        summary: 'should be OK',
+        severity: DiagnosticSeverity.Warning,
+        then: {
+          function: 'truthy',
+        },
+      },
+      'require-info': {
+        given: '$.info',
+        summary: 'should be OK',
+        severity: DiagnosticSeverity.Warning,
+        then: {
+          function: 'truthy',
+        },
+      },
+    });
+  });
+
+  it('should inherit properties of extended rulesets', () => {
+    return expect(readRulesFromRulesets(extendsOas2Ruleset)).resolves.toEqual({
+      ...[...Object.entries(oasRuleset.rules), ...Object.entries(oas2Ruleset.rules)].reduce<Dictionary<unknown>>(
+        (rules, [name, rule]) => {
+          rules[name] = {
+            ...rule,
+            ...((rule as IRule).severity === undefined && { severity: DiagnosticSeverity.Warning }),
+          };
+
+          return rules;
+        },
+        {},
+      ),
+
+      'valid-rule': {
+        given: '$.info',
+        summary: 'should be OK',
+        severity: DiagnosticSeverity.Warning,
+        then: {
+          function: 'truthy',
+        },
+      },
+    });
+  });
+
+  it('should override properties of extended rulesets', () => {
+    return expect(readRulesFromRulesets(extendsOas2WithOverrideRuleset)).resolves.toHaveProperty(
+      'operation-2xx-response',
+      {
+        summary: 'should be OK',
+        given: '$.info',
+        recommended: true,
+        severity: DiagnosticSeverity.Warning,
+        tags: ['operation'],
+        then: {
+          function: 'truthy',
+        },
+        type: 'style',
+      },
+    );
+  });
+
+  it('should persists disabled properties of extended rulesets', () => {
+    return expect(readRulesFromRulesets(extendsOas2WithOverrideRuleset)).resolves.toHaveProperty(
+      'operation-security-defined',
+      {
+        given: '$',
+        recommended: true,
+        severity: 'off',
+        summary: 'Operation `security` values must match a scheme defined in the `securityDefinitions` object.',
+        tags: ['operation'],
+        then: {
+          function: 'oasOpSecurityDefined',
+          functionOptions: {
+            schemesPath: ['securityDefinitions'],
+          },
+        },
+        type: 'validation',
+      },
+    );
+  });
+
+  it('given non-existent ruleset should output error', () => {
+    return expect(readRulesFromRulesets('oneParentRuleset')).rejects.toThrowError(
+      'Could not parse https://unpkg.com/oneParentRuleset: Not Found',
+    );
+  });
+
+  it('given invalid ruleset should output errors', () => {
+    return expect(readRulesFromRulesets(invalidRuleset)).rejects.toThrowError(/should have required property/);
+  });
+});
diff --git a/src/rulesets/__tests__/reader.unit.ts b/src/rulesets/__tests__/reader.unit.ts
deleted file mode 100644
index 98c5319b..00000000
--- a/src/rulesets/__tests__/reader.unit.ts
+++ /dev/null
@@ -1,207 +0,0 @@
-jest.mock('../../fs/reader');
-jest.mock('../ajv');
-jest.mock('../validation');
-jest.mock('../path');
-
-import { Dictionary } from '@stoplight/types';
-import { when } from 'jest-when';
-import { IRule } from '../..';
-import { readParsable } from '../../fs/reader';
-import { IRulesetFile } from '../../types/ruleset';
-import { formatAjv } from '../ajv';
-import { resolvePath } from '../path';
-import { readRulesFromRulesets } from '../reader';
-import { validateRuleset } from '../validation';
-
-const readParsableMock: jest.Mock = readParsable as jest.Mock;
-const formatAjvMock: jest.Mock = formatAjv as jest.Mock;
-const validateRulesetMock: jest.Mock = validateRuleset as jest.Mock;
-const resolvePathMock: jest.Mock = resolvePath as jest.Mock;
-
-const simpleRule: IRule = {
-  recommended: false,
-  given: 'abc',
-  then: {
-    function: 'f',
-  },
-};
-
-describe('reader', () => {
-  afterEach(() => {
-    jest.resetAllMocks();
-  });
-
-  it('given flat, valid ruleset file should return rules', async () => {
-    validateRulesetMock.mockReturnValue([]);
-    givenRulesets({
-      'flat-ruleset.yaml': {
-        rules: {
-          'rule-1': simpleRule,
-        },
-      },
-    });
-
-    expect(await readRulesFromRulesets('flat-ruleset.yaml')).toEqual({
-      'rule-1': { given: 'abc', then: { function: 'f' }, recommended: false },
-    });
-  });
-
-  it('given two flat, valid ruleset files should return rules', async () => {
-    validateRulesetMock.mockReturnValue([]);
-    givenRulesets({
-      'flat-ruleset-a.yaml': {
-        rules: {
-          'rule-1': simpleRule,
-        },
-      },
-      'flat-ruleset-b.yaml': {
-        rules: {
-          'rule-2': simpleRule,
-        },
-      },
-    });
-
-    expect(await readRulesFromRulesets('flat-ruleset-a.yaml', 'flat-ruleset-b.yaml')).toEqual({
-      'rule-1': { given: 'abc', then: { function: 'f' }, recommended: false },
-      'rule-2': { given: 'abc', then: { function: 'f' }, recommended: false },
-    });
-  });
-
-  it('should override properties of extended rulesets', async () => {
-    validateRulesetMock.mockReturnValue([]);
-    givenRulesets({
-      flatRuleset: {
-        rules: {
-          'rule-1': simpleRule,
-        },
-      },
-      oneParentRuleset: {
-        extends: ['flatRuleset'],
-        rules: {
-          'rule-1': {
-            ...simpleRule,
-            // note that i'm enabling the rule here
-            recommended: true,
-          },
-        },
-      },
-    });
-
-    expect(await readRulesFromRulesets('oneParentRuleset')).toEqual({
-      'rule-1': { given: 'abc', then: { function: 'f' }, recommended: true },
-    });
-  });
-
-  it('should inherit properties of extended rulesets', async () => {
-    validateRulesetMock.mockReturnValue([]);
-    givenRulesets({
-      flatRuleset: {
-        rules: {
-          'rule-2': {
-            given: 'another given',
-            then: {
-              function: 'b',
-            },
-          },
-        },
-      },
-      oneParentRuleset: {
-        extends: ['flatRuleset'],
-        rules: {
-          'rule-1': simpleRule,
-        },
-      },
-    });
-
-    expect(await readRulesFromRulesets('oneParentRuleset')).toEqual({
-      'rule-1': { given: 'abc', then: { function: 'f' }, recommended: false },
-      'rule-2': { given: 'another given', then: { function: 'b' } },
-    });
-  });
-
-  it('should blend together pointers rulesets', async () => {
-    validateRulesetMock.mockReturnValue([]);
-    givenRulesets({
-      rulesetA: {
-        rules: {
-          'rule-a': {
-            given: 'given-a',
-            then: {
-              function: 'a',
-            },
-          },
-          'common-rule': {
-            given: 'common',
-            then: {
-              function: 'ca',
-            },
-          },
-        },
-      },
-      rulesetB: {
-        rules: {
-          'rule-b': {
-            given: 'given-b',
-            then: {
-              function: 'b',
-            },
-          },
-          'common-rule': {
-            given: 'common',
-            then: {
-              function: 'cb',
-            },
-          },
-        },
-      },
-      oneParentRuleset: {
-        extends: ['rulesetA', 'rulesetB'],
-        rules: {
-          'rule-1': simpleRule,
-        },
-      },
-    });
-
-    expect(await readRulesFromRulesets('oneParentRuleset')).toEqual({
-      'rule-1': { given: 'abc', then: { function: 'f' }, recommended: false },
-      'rule-a': { given: 'given-a', then: { function: 'a' } },
-      'rule-b': { given: 'given-b', then: { function: 'b' } },
-      'common-rule': { given: 'common', then: { function: 'cb' } },
-    });
-  });
-
-  it('given invalid ruleset should output errors', () => {
-    const flatRuleset = {
-      rules: {
-        'rule-1': simpleRule,
-      },
-    };
-    givenRulesets({
-      'flat-ruleset.yaml': flatRuleset,
-    });
-    validateRulesetMock.mockReturnValue(['fake errors']);
-    when(formatAjvMock)
-      .calledWith(['fake errors'])
-      .mockReturnValue('fake formatted message');
-
-    return expect(readRulesFromRulesets('flat-ruleset.yaml')).rejects.toEqual({
-      messages: ['fake formatted message', "Provided ruleset 'flat-ruleset.yaml' is not valid"],
-    });
-  });
-
-  function givenRulesets(rulesets: Dictionary<IRulesetFile, string>) {
-    Object.entries(rulesets).forEach(([key, ruleset]) => {
-      (ruleset.extends || []).forEach(extend =>
-        when(resolvePathMock)
-          .calledWith(key, extend)
-          .mockReturnValue(extend),
-      );
-
-      when(readParsableMock)
-        .calledWith(key, 'utf8')
-        .mockReturnValue({
-          data: ruleset,
-        });
-    });
-  }
-});
diff --git a/src/rulesets/__tests__/validation.test.ts b/src/rulesets/__tests__/validation.test.ts
new file mode 100644
index 00000000..51b39c7e
--- /dev/null
+++ b/src/rulesets/__tests__/validation.test.ts
@@ -0,0 +1,61 @@
+import { assertValidRuleset } from '../validation';
+const invalidRuleset = require('./__fixtures__/invalid-ruleset.json');
+const validRuleset = require('./__fixtures__/valid-flat-ruleset.json');
+
+describe('Ruleset Validation', () => {
+  it('given invalid ruleset should throw', () => {
+    expect(assertValidRuleset.bind(null, invalidRuleset)).toThrow();
+  });
+
+  it('given valid ruleset should emit no errors', () => {
+    expect(assertValidRuleset.bind(null, validRuleset)).not.toThrow();
+  });
+
+  it('recognizes severity flags', () => {
+    expect(
+      assertValidRuleset.bind(null, {
+        rules: {
+          rule: 'off',
+        },
+      }),
+    ).not.toThrow();
+  });
+
+  it('recognizes array-ish syntax', () => {
+    expect(
+      assertValidRuleset.bind(null, {
+        rules: {
+          rule: ['off'],
+        },
+      }),
+    ).not.toThrow();
+  });
+
+  it('recognizes invalid array-ish syntax', () => {
+    expect(
+      assertValidRuleset.bind(null, {
+        rules: {
+          rule: ['off', 2],
+        },
+      }),
+    ).toThrow();
+  });
+
+  it('recognizes valid array-ish extends syntax', () => {
+    expect(
+      assertValidRuleset.bind(null, {
+        extends: [['foo', 'off'], 'test'],
+        rules: {},
+      }),
+    ).not.toThrow();
+  });
+
+  it('recognizes invalid array-ish extends syntax', () => {
+    expect(
+      assertValidRuleset.bind(null, {
+        extends: [['foo', 'test']],
+        rules: {},
+      }),
+    ).toThrow();
+  });
+});
diff --git a/src/rulesets/__tests__/validation.unit.ts b/src/rulesets/__tests__/validation.unit.ts
deleted file mode 100644
index 001aae00..00000000
--- a/src/rulesets/__tests__/validation.unit.ts
+++ /dev/null
@@ -1,13 +0,0 @@
-import { validateRuleset } from '../validation';
-const invalidRuleset = require('./__fixtures__/ruleset-invalid.json');
-const validRuleset = require('./__fixtures__/valid-ruleset.json');
-
-describe('Ruleset Validation', () => {
-  it('given invalid ruleset should emit errors', () => {
-    expect(JSON.stringify(validateRuleset(invalidRuleset), null, 2)).toMatchSnapshot();
-  });
-
-  it('given valid ruleset should emit no errors', () => {
-    expect(validateRuleset(validRuleset)).toEqual([]);
-  });
-});
diff --git a/src/rulesets/oas/__tests__/contact-properties.ts b/src/rulesets/oas/__tests__/contact-properties.ts
index 32b5736a..2e4f1b22 100644
--- a/src/rulesets/oas/__tests__/contact-properties.ts
+++ b/src/rulesets/oas/__tests__/contact-properties.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('contact-properties', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/example-value-or-externalValue.ts b/src/rulesets/oas/__tests__/example-value-or-externalValue.ts
index 54422296..ecc6ee1e 100644
--- a/src/rulesets/oas/__tests__/example-value-or-externalValue.ts
+++ b/src/rulesets/oas/__tests__/example-value-or-externalValue.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('example-value-or-externalValue', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/info-contact.ts b/src/rulesets/oas/__tests__/info-contact.ts
index 7ff35f6c..75455c86 100644
--- a/src/rulesets/oas/__tests__/info-contact.ts
+++ b/src/rulesets/oas/__tests__/info-contact.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('info-contact', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/info-description.ts b/src/rulesets/oas/__tests__/info-description.ts
index bf7769d0..70ab055f 100644
--- a/src/rulesets/oas/__tests__/info-description.ts
+++ b/src/rulesets/oas/__tests__/info-description.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('info-description', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/info-license.ts b/src/rulesets/oas/__tests__/info-license.ts
index 78dc3b96..1d55bc0b 100644
--- a/src/rulesets/oas/__tests__/info-license.ts
+++ b/src/rulesets/oas/__tests__/info-license.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('info-license', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/license-url.ts b/src/rulesets/oas/__tests__/license-url.ts
index 73cf84bf..a9756eca 100644
--- a/src/rulesets/oas/__tests__/license-url.ts
+++ b/src/rulesets/oas/__tests__/license-url.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('license-url', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/no-eval-in-markdown.ts b/src/rulesets/oas/__tests__/no-eval-in-markdown.ts
index 2036caa9..db83cefd 100644
--- a/src/rulesets/oas/__tests__/no-eval-in-markdown.ts
+++ b/src/rulesets/oas/__tests__/no-eval-in-markdown.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('no-eval-in-markdown', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/no-script-tags-in-markdown.ts b/src/rulesets/oas/__tests__/no-script-tags-in-markdown.ts
index 2c79d9ad..3bc259d5 100644
--- a/src/rulesets/oas/__tests__/no-script-tags-in-markdown.ts
+++ b/src/rulesets/oas/__tests__/no-script-tags-in-markdown.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('no-script-tags-in-markdown', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/only-local-references.ts b/src/rulesets/oas/__tests__/only-local-references.ts
index 53c0d617..0b59cb75 100644
--- a/src/rulesets/oas/__tests__/only-local-references.ts
+++ b/src/rulesets/oas/__tests__/only-local-references.ts
@@ -1,6 +1,6 @@
 import { DiagnosticSeverity } from '@stoplight/types';
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('only-local-references', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/openapi-tags-alphabetical.ts b/src/rulesets/oas/__tests__/openapi-tags-alphabetical.ts
index 1ff3e3e2..76ac823a 100644
--- a/src/rulesets/oas/__tests__/openapi-tags-alphabetical.ts
+++ b/src/rulesets/oas/__tests__/openapi-tags-alphabetical.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('openapi-tags-alphabetical', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/openapi-tags.ts b/src/rulesets/oas/__tests__/openapi-tags.ts
index 8d1de5f9..db1d6bdc 100644
--- a/src/rulesets/oas/__tests__/openapi-tags.ts
+++ b/src/rulesets/oas/__tests__/openapi-tags.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('openapi-tags', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/operation-2xx-response.ts b/src/rulesets/oas/__tests__/operation-2xx-response.ts
index cda34c48..bc55657b 100644
--- a/src/rulesets/oas/__tests__/operation-2xx-response.ts
+++ b/src/rulesets/oas/__tests__/operation-2xx-response.ts
@@ -1,6 +1,6 @@
 import { commonOasFunctions } from '../';
 import { Spectral } from '../../../spectral';
-import { rules } from '../ruleset.json';
+import { rules } from '../rules.json';
 
 const { oasOp2xxResponse } = commonOasFunctions();
 
diff --git a/src/rulesets/oas/__tests__/operation-default-response.ts b/src/rulesets/oas/__tests__/operation-default-response.ts
index 7eefb02c..e51f2a01 100644
--- a/src/rulesets/oas/__tests__/operation-default-response.ts
+++ b/src/rulesets/oas/__tests__/operation-default-response.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('operation-default-response', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/operation-description.ts b/src/rulesets/oas/__tests__/operation-description.ts
index ac3f1093..7dc9ed55 100644
--- a/src/rulesets/oas/__tests__/operation-description.ts
+++ b/src/rulesets/oas/__tests__/operation-description.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('operation-description', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/operation-operationId-valid-in-url.ts b/src/rulesets/oas/__tests__/operation-operationId-valid-in-url.ts
index a535d147..bf5f2293 100644
--- a/src/rulesets/oas/__tests__/operation-operationId-valid-in-url.ts
+++ b/src/rulesets/oas/__tests__/operation-operationId-valid-in-url.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('operation-operationId-valid-in-url', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/operation-operationId.ts b/src/rulesets/oas/__tests__/operation-operationId.ts
index 5661b425..fb7c15f6 100644
--- a/src/rulesets/oas/__tests__/operation-operationId.ts
+++ b/src/rulesets/oas/__tests__/operation-operationId.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('operation-operationId', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/operation-singular-tag.ts b/src/rulesets/oas/__tests__/operation-singular-tag.ts
index 049a4ab6..43219296 100644
--- a/src/rulesets/oas/__tests__/operation-singular-tag.ts
+++ b/src/rulesets/oas/__tests__/operation-singular-tag.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('operation-singular-tag', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/operation-summary-formatted.ts b/src/rulesets/oas/__tests__/operation-summary-formatted.ts
index 1e256959..e478e8f3 100644
--- a/src/rulesets/oas/__tests__/operation-summary-formatted.ts
+++ b/src/rulesets/oas/__tests__/operation-summary-formatted.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('operation-summary-formatted', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/operation-tags.ts b/src/rulesets/oas/__tests__/operation-tags.ts
index d191908a..be0be643 100644
--- a/src/rulesets/oas/__tests__/operation-tags.ts
+++ b/src/rulesets/oas/__tests__/operation-tags.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('operation-tags', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/parameter-description.ts b/src/rulesets/oas/__tests__/parameter-description.ts
index 3bc13ade..b5d22204 100644
--- a/src/rulesets/oas/__tests__/parameter-description.ts
+++ b/src/rulesets/oas/__tests__/parameter-description.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('parameter-description', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/path-declarations-must-exist.ts b/src/rulesets/oas/__tests__/path-declarations-must-exist.ts
index 6fa9d790..ca112f6f 100644
--- a/src/rulesets/oas/__tests__/path-declarations-must-exist.ts
+++ b/src/rulesets/oas/__tests__/path-declarations-must-exist.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('path-declarations-must-exist', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/path-keys-no-trailing-slash.ts b/src/rulesets/oas/__tests__/path-keys-no-trailing-slash.ts
index 5a77df89..acbb6f0a 100644
--- a/src/rulesets/oas/__tests__/path-keys-no-trailing-slash.ts
+++ b/src/rulesets/oas/__tests__/path-keys-no-trailing-slash.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('path-keys-no-trailing-slash', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/path-not-include-query.ts b/src/rulesets/oas/__tests__/path-not-include-query.ts
index 7647a717..2844e008 100644
--- a/src/rulesets/oas/__tests__/path-not-include-query.ts
+++ b/src/rulesets/oas/__tests__/path-not-include-query.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('path-not-include-query', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/__tests__/tag-description.ts b/src/rulesets/oas/__tests__/tag-description.ts
index c31ab349..1acbaa23 100644
--- a/src/rulesets/oas/__tests__/tag-description.ts
+++ b/src/rulesets/oas/__tests__/tag-description.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('tag-description', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas/functions/oasOp2xxResponse/__tests__/index.ts b/src/rulesets/oas/functions/oasOp2xxResponse/__tests__/index.ts
index 78029620..1a4e06f2 100644
--- a/src/rulesets/oas/functions/oasOp2xxResponse/__tests__/index.ts
+++ b/src/rulesets/oas/functions/oasOp2xxResponse/__tests__/index.ts
@@ -1,6 +1,6 @@
 import { RuleType, Spectral } from '../../../../../spectral';
 import { commonOasFunctions } from '../../../../oas/index';
-import { rules } from '../../../../oas/ruleset.json';
+import { rules } from '../../../../oas/rules.json';
 const ruleset = { functions: commonOasFunctions(), rules };
 
 describe('oasOp2xxResponse', () => {
diff --git a/src/rulesets/oas/functions/oasOpFormDataConsumeCheck/__tests__/index.ts b/src/rulesets/oas/functions/oasOpFormDataConsumeCheck/__tests__/index.ts
index b5d811a1..6f14f3c0 100644
--- a/src/rulesets/oas/functions/oasOpFormDataConsumeCheck/__tests__/index.ts
+++ b/src/rulesets/oas/functions/oasOpFormDataConsumeCheck/__tests__/index.ts
@@ -1,7 +1,7 @@
 import { RuleType, Spectral } from '../../../../../index';
 import { commonOasFunctions } from '../../../index';
 
-import { rules } from '../../../ruleset.json';
+import { rules } from '../../../rules.json';
 
 const ruleset = { functions: commonOasFunctions(), rules };
 
diff --git a/src/rulesets/oas/functions/oasOpIdUnique/__tests__/index.ts b/src/rulesets/oas/functions/oasOpIdUnique/__tests__/index.ts
index bf501a03..997c3122 100644
--- a/src/rulesets/oas/functions/oasOpIdUnique/__tests__/index.ts
+++ b/src/rulesets/oas/functions/oasOpIdUnique/__tests__/index.ts
@@ -1,7 +1,7 @@
 import { RuleType, Spectral } from '../../../../../index';
 import { commonOasFunctions } from '../../../index';
 
-import { rules } from '../../../ruleset.json';
+import { rules } from '../../../rules.json';
 
 const ruleset = { functions: commonOasFunctions(), rules };
 
diff --git a/src/rulesets/oas/functions/oasOpParams/__tests__/index.test.ts b/src/rulesets/oas/functions/oasOpParams/__tests__/index.test.ts
index 59ded664..3d80321f 100644
--- a/src/rulesets/oas/functions/oasOpParams/__tests__/index.test.ts
+++ b/src/rulesets/oas/functions/oasOpParams/__tests__/index.test.ts
@@ -1,7 +1,7 @@
 import { RuleType, Spectral } from '../../../../../index';
 import { commonOasFunctions } from '../../../index';
 
-import { rules } from '../../../ruleset.json';
+import { rules } from '../../../rules.json';
 
 const ruleset = { functions: commonOasFunctions(), rules };
 
diff --git a/src/rulesets/oas/functions/oasOpSecurityDefined/__tests__/index.ts b/src/rulesets/oas/functions/oasOpSecurityDefined/__tests__/index.ts
index cd0c2891..2252f7cc 100644
--- a/src/rulesets/oas/functions/oasOpSecurityDefined/__tests__/index.ts
+++ b/src/rulesets/oas/functions/oasOpSecurityDefined/__tests__/index.ts
@@ -1,8 +1,8 @@
 import { RuleType, Spectral } from '../../../../../index';
 import { oas2Functions } from '../../../../oas2/index';
-import { rules as oas2Rules } from '../../../../oas2/ruleset.json';
+import { rules as oas2Rules } from '../../../../oas2/rules.json';
 import { oas3Functions } from '../../../../oas3/index';
-import { rules as oas3Rules } from '../../../../oas3/ruleset.json';
+import { rules as oas3Rules } from '../../../../oas3/rules.json';
 
 const oas2Ruleset = { functions: oas2Functions(), rules: oas2Rules };
 const oas3Ruleset = { functions: oas3Functions(), rules: oas3Rules };
diff --git a/src/rulesets/oas/functions/oasPathParam/__tests__/index.test.ts b/src/rulesets/oas/functions/oasPathParam/__tests__/index.test.ts
index e7eff9ce..d0b4cd4a 100644
--- a/src/rulesets/oas/functions/oasPathParam/__tests__/index.test.ts
+++ b/src/rulesets/oas/functions/oasPathParam/__tests__/index.test.ts
@@ -1,7 +1,7 @@
 import { RuleType, Spectral } from '../../../../../index';
 import { commonOasFunctions } from '../../../index';
 
-import { rules } from '../../../ruleset.json';
+import { rules } from '../../../rules.json';
 
 const ruleset = { functions: commonOasFunctions(), rules };
 
diff --git a/src/rulesets/oas/ruleset.json b/src/rulesets/oas/rules.json
similarity index 100%
rename from src/rulesets/oas/ruleset.json
rename to src/rulesets/oas/rules.json
diff --git a/src/rulesets/oas2/__tests__/api-host.ts b/src/rulesets/oas2/__tests__/api-host.ts
index a0f9ff7b..667f49ca 100644
--- a/src/rulesets/oas2/__tests__/api-host.ts
+++ b/src/rulesets/oas2/__tests__/api-host.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('api-host', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas2/__tests__/api-schemes.ts b/src/rulesets/oas2/__tests__/api-schemes.ts
index 2c9be1c7..0c4f92c2 100644
--- a/src/rulesets/oas2/__tests__/api-schemes.ts
+++ b/src/rulesets/oas2/__tests__/api-schemes.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('api-schemes', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas2/__tests__/host-not-example.ts b/src/rulesets/oas2/__tests__/host-not-example.ts
index 80149bbc..05b76277 100644
--- a/src/rulesets/oas2/__tests__/host-not-example.ts
+++ b/src/rulesets/oas2/__tests__/host-not-example.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('host-not-example', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas2/__tests__/host-trailing-slash.ts b/src/rulesets/oas2/__tests__/host-trailing-slash.ts
index 8d3e6c5a..0e140a20 100644
--- a/src/rulesets/oas2/__tests__/host-trailing-slash.ts
+++ b/src/rulesets/oas2/__tests__/host-trailing-slash.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('host-trailing-slash', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas2/__tests__/model-description.ts b/src/rulesets/oas2/__tests__/model-description.ts
index c71b30af..0f520ebc 100644
--- a/src/rulesets/oas2/__tests__/model-description.ts
+++ b/src/rulesets/oas2/__tests__/model-description.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('model-description', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas2/__tests__/oas2-schema.ts b/src/rulesets/oas2/__tests__/oas2-schema.ts
index 6d08f965..2bb773fd 100644
--- a/src/rulesets/oas2/__tests__/oas2-schema.ts
+++ b/src/rulesets/oas2/__tests__/oas2-schema.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('oas2-schema', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas2/__tests__/valid-example.ts b/src/rulesets/oas2/__tests__/valid-example.ts
index 18483098..7c8ae646 100644
--- a/src/rulesets/oas2/__tests__/valid-example.ts
+++ b/src/rulesets/oas2/__tests__/valid-example.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('valid-example', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas3/__tests__/api-servers.ts b/src/rulesets/oas3/__tests__/api-servers.ts
index f210015c..e74dfa36 100644
--- a/src/rulesets/oas3/__tests__/api-servers.ts
+++ b/src/rulesets/oas3/__tests__/api-servers.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('api-servers', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas3/__tests__/model-description.ts b/src/rulesets/oas3/__tests__/model-description.ts
index 8e5df413..9fa6d29f 100644
--- a/src/rulesets/oas3/__tests__/model-description.ts
+++ b/src/rulesets/oas3/__tests__/model-description.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('model-description', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas3/__tests__/server-not-example.ts b/src/rulesets/oas3/__tests__/server-not-example.ts
index 2ecdb5ed..d9b926ca 100644
--- a/src/rulesets/oas3/__tests__/server-not-example.ts
+++ b/src/rulesets/oas3/__tests__/server-not-example.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('server-not-example.com', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas3/__tests__/server-trailing-slash.ts b/src/rulesets/oas3/__tests__/server-trailing-slash.ts
index 4f683118..9beebbbc 100644
--- a/src/rulesets/oas3/__tests__/server-trailing-slash.ts
+++ b/src/rulesets/oas3/__tests__/server-trailing-slash.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 describe('server-trailing-slash', () => {
   const s = new Spectral();
diff --git a/src/rulesets/oas3/__tests__/valid-example.ts b/src/rulesets/oas3/__tests__/valid-example.ts
index 9fa0b5b6..c9133bbd 100644
--- a/src/rulesets/oas3/__tests__/valid-example.ts
+++ b/src/rulesets/oas3/__tests__/valid-example.ts
@@ -1,5 +1,5 @@
 import { RuleType, Spectral } from '../../../spectral';
-import * as ruleset from '../ruleset.json';
+import * as ruleset from '../rules.json';
 
 // @oclif/test packages requires @types/mocha, therefore we have 2 packages coming up with similar typings
 // TS is confused and prefers the mocha ones, so we need to instrument it to pick up the Jest ones