Repository: amperser/proselint

Issue:
Problems packaging .proselintrc
Hey :wave:

I'm running into issues packaging this for Arch due to the system config file `.proselintrc`. Since the project migrated from `setuptools` to `poetry` I use [dephell](https://github.com/dephell/dephell) to convert it back to `setuptools` during installtion, as is recommended  by Arch community. It does seem like this conversion does not produce the correct `package_data` in `setup.py` (probably due to this not being supported) resulting in `.proselintrc` not being installed at all.

While I could patch the package and do a number of work-arounds, I wonder if we could get rid of `.proselintrc` altogether by hardcoding the system defaults directly in `proselint`. Having `.proselintrc` as a Python file or a part of `proselint/tools.py` would make packaging for distros less complex and it would not hurt `pip` distribution as far as I can tell.

Not distributing the default config in file format would make it a bit harder to make a custom config from scratch, but this could be rectified by a `--dump-config` flag which would print the current config. This is how the Clang tools works, for example.

Discussion:
codecov[bot]: # [Codecov](https://codecov.io/gh/amperser/proselint/pull/1212?src=pr&el=h1&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser) Report
> Merging [#1212](https://codecov.io/gh/amperser/proselint/pull/1212?src=pr&el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser) (f745d5a) into [main](https://codecov.io/gh/amperser/proselint/commit/b9aeb0adee890aba2f661c3f200f8795bdc56a22?el=desc&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser) (b9aeb0a) will **decrease** coverage by `0.22%`.
> The diff coverage is `83.33%`.

[![Impacted file tree graph](https://codecov.io/gh/amperser/proselint/pull/1212/graphs/tree.svg?width=650&height=150&src=pr&token=8E0I9sRpot&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser)](https://codecov.io/gh/amperser/proselint/pull/1212?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser)

```diff
@@            Coverage Diff             @@
##             main    #1212      +/-   ##
==========================================
- Coverage   94.34%   94.11%   -0.23%     
==========================================
  Files          82       83       +1     
  Lines        1131     1138       +7     
==========================================
+ Hits         1067     1071       +4     
- Misses         64       67       +3     
```

| Flag | Coverage Δ | |
|---|---|---|
| macos-latest | `94.11% <83.33%> (-0.23%)` | :arrow_down: |
| py3.6 | `93.49% <83.33%> (-0.25%)` | :arrow_down: |
| py3.7 | `93.49% <83.33%> (-0.25%)` | :arrow_down: |
| py3.8 | `94.11% <83.33%> (-0.23%)` | :arrow_down: |
| py3.9 | `94.11% <83.33%> (-0.23%)` | :arrow_down: |
| pypypy3 | `?` | |
| ubuntu-latest | `94.11% <83.33%> (-0.23%)` | :arrow_down: |
| windows-latest | `94.11% <83.33%> (-0.23%)` | :arrow_down: |

Flags with carried forward coverage won't be shown. [Click here](https://docs.codecov.io/docs/carryforward-flags?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser#carryforward-flags-in-the-pull-request-comment) to find out more.

| [Impacted Files](https://codecov.io/gh/amperser/proselint/pull/1212?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser) | Coverage Δ | |
|---|---|---|
| [proselint/command\_line.py](https://codecov.io/gh/amperser/proselint/pull/1212/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser#diff-cHJvc2VsaW50L2NvbW1hbmRfbGluZS5weQ==) | `73.39% <80.00%> (+1.11%)` | :arrow_up: |
| [proselint/tools.py](https://codecov.io/gh/amperser/proselint/pull/1212/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser#diff-cHJvc2VsaW50L3Rvb2xzLnB5) | `86.09% <85.71%> (-0.86%)` | :arrow_down: |
| [proselint/config.py](https://codecov.io/gh/amperser/proselint/pull/1212/diff?src=pr&el=tree&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser#diff-cHJvc2VsaW50L2NvbmZpZy5weQ==) | `100.00% <100.00%> (ø)` | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/amperser/proselint/pull/1212?src=pr&el=continue&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/amperser/proselint/pull/1212?src=pr&el=footer&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser). Last update [b9aeb0a...f745d5a](https://codecov.io/gh/amperser/proselint/pull/1212?src=pr&el=lastupdated&utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments?utm_medium=referral&utm_source=github&utm_content=comment&utm_campaign=pr+comments&utm_term=amperser).


Patch:
diff --git a/proselint/.proselintrc b/proselint/.proselintrc
deleted file mode 100644
index 5a87592..0000000
--- a/proselint/.proselintrc
+++ /dev/null
@@ -1,85 +0,0 @@
-{
-    "max_errors": 1000,
-    "checks": {
-        "airlinese.misc"                : true,
-        "annotations.misc"              : true,
-        "archaism.misc"                 : true,
-        "cliches.hell"                  : true,
-        "cliches.misc"                  : true,
-        "consistency.spacing"           : true,
-        "consistency.spelling"          : true,
-        "corporate_speak.misc"          : true,
-        "cursing.filth"                 : true,
-        "cursing.nfl"                   : false,
-        "cursing.nword"                 : true,
-        "dates_times.am_pm"             : true,
-        "dates_times.dates"             : true,
-        "hedging.misc"                  : true,
-        "hyperbole.misc"                : true,
-        "jargon.misc"                   : true,
-        "lexical_illusions.misc"        : true,
-        "lgbtq.offensive_terms"         : true, 
-        "lgbtq.terms"                   : true,
-        "links.broken"                  : false,
-        "malapropisms.misc"             : true,
-        "misc.apologizing"              : true,
-        "misc.back_formations"          : true,
-        "misc.bureaucratese"            : true,
-        "misc.but"                      : true,
-        "misc.capitalization"           : true,
-        "misc.chatspeak"                : true,
-        "misc.commercialese"            : true,
-        "misc.composition"              : true,
-        "misc.currency"                 : true,
-        "misc.debased"                  : true,
-        "misc.false_plurals"            : true,
-        "misc.illogic"                  : true,
-        "misc.inferior_superior"        : true,
-        "misc.institution_name"	        : true,
-        "misc.latin"                    : true,
-        "misc.many_a"                   : true,
-        "misc.metaconcepts"             : true,
-        "misc.metadiscourse"            : true,
-        "misc.narcissism"               : true,
-        "misc.not_guilty"               : true,
-        "misc.phrasal_adjectives"       : true,
-        "misc.preferred_forms"          : true,
-        "misc.pretension"               : true,
-        "misc.professions"              : true,
-        "misc.punctuation"              : true,
-        "misc.scare_quotes"             : true,
-        "misc.suddenly"                 : true,
-        "misc.tense_present"            : true,
-        "misc.waxed"                    : true,
-        "misc.whence"                   : true,
-        "mixed_metaphors.misc"          : true,
-        "mondegreens.misc"              : true,
-        "needless_variants.misc"        : true,
-        "nonwords.misc"                 : true,
-        "oxymorons.misc"                : true,
-        "psychology.misc"               : true,
-        "redundancy.misc"               : true,
-        "redundancy.ras_syndrome"       : true,
-        "skunked_terms.misc"            : true,
-        "spelling.able_atable"          : true,
-        "spelling.able_ible"            : true,
-        "spelling.athletes"             : true,
-        "spelling.em_im_en_in"          : true,
-        "spelling.er_or"                : true,
-        "spelling.in_un"                : true,
-        "spelling.misc"                 : true,
-        "security.credit_card"          : true,
-        "security.password"             : true,
-        "sexism.misc"                   : true,
-        "terms.animal_adjectives"       : true,
-        "terms.denizen_labels"          : true,
-        "terms.eponymous_adjectives"    : true,
-        "terms.venery"                  : true,
-        "typography.diacritical_marks"  : true,
-        "typography.exclamation"        : true,
-        "typography.symbols"            : true,
-        "uncomparables.misc"            : true,
-        "weasel_words.misc"             : true,
-        "weasel_words.very"             : true
-    }
-}
diff --git a/proselint/command_line.py b/proselint/command_line.py
index e4ff598..3cdeb2f 100644
--- a/proselint/command_line.py
+++ b/proselint/command_line.py
@@ -1,5 +1,6 @@
 """Command line utility for proselint."""
 
+import json
 import os
 import shutil
 import subprocess
@@ -8,11 +9,12 @@ import traceback
 
 import click
 
+from .config import default
 from .tools import (close_cache_shelves, close_cache_shelves_after,
-                    errors_to_json, lint)
+                    errors_to_json, lint, load_options)
 from .version import __version__
 
-CONTEXT_SETTINGS = dict(help_option_names=['-h', '--help'])
+CONTEXT_SETTINGS = {"help_option_names": ['-h', '--help']}
 base_url = "proselint.com/"
 proselint_path = os.path.dirname(os.path.realpath(__file__))
 demo_file = os.path.join(proselint_path, "demo.md")
@@ -95,11 +97,23 @@ def print_errors(filename, errors, output_json=False, compact=False):
 @click.option('--time', '-t', is_flag=True, help="Time on a corpus.")
 @click.option('--demo', is_flag=True, help="Run over demo file.")
 @click.option('--compact', is_flag=True, help="Shorten output.")
+@click.option('--dump-config', is_flag=True, help="Prints current config.")
+@click.option('--dump-default-config', is_flag=True,
+              help="Prints default config.")
 @click.argument('paths', nargs=-1, type=click.Path())
 @close_cache_shelves_after
-def proselint(paths=None, config=None, version=None, clean=None, debug=None,
-              output_json=None, time=None, demo=None, compact=None):
+def proselint(paths=None, config=None, version=None, clean=None,
+              debug=None, output_json=None, time=None, demo=None, compact=None,
+              dump_config=None, dump_default_config=None):
     """Create the CLI for proselint, a linter for prose."""
+    if dump_default_config:
+        return print(json.dumps(default, sort_keys=True, indent=4))
+
+    config = load_options(config, default)
+    if dump_config:
+        print(json.dumps(config, sort_keys=True, indent=4))
+        return
+
     if time:
         # click.echo(timing_test())
         print("This option does not work for the time being.")
@@ -129,14 +143,13 @@ def proselint(paths=None, config=None, version=None, clean=None, debug=None,
             f = sys.stdin
         else:
             try:
-                f = click.open_file(
-                    fp, 'r', encoding="utf-8", errors="replace")
+                f = click.open_file(fp, 'r', "utf-8", "replace")
             except Exception:
                 traceback.print_exc()
                 sys.exit(2)
-        errors = lint(f, debug=debug, config_file_path=config)
+        errors = lint(f, debug, config)
         num_errors += len(errors)
-        print_errors(fp, errors, output_json, compact=compact)
+        print_errors(fp, errors, output_json, compact)
 
     # Return an exit code
     close_cache_shelves()
diff --git a/proselint/config.py b/proselint/config.py
new file mode 100644
index 0000000..7f00eb1
--- /dev/null
+++ b/proselint/config.py
@@ -0,0 +1,87 @@
+"""Proselint config - replacement for default .proselintrc since #1212."""
+
+default = {
+    "max_errors": 1000,
+    "checks": {
+        "airlinese.misc": True,
+        "annotations.misc": True,
+        "archaism.misc": True,
+        "cliches.hell": True,
+        "cliches.misc": True,
+        "consistency.spacing": True,
+        "consistency.spelling": True,
+        "corporate_speak.misc": True,
+        "cursing.filth": True,
+        "cursing.nfl": False,
+        "cursing.nword": True,
+        "dates_times.am_pm": True,
+        "dates_times.dates": True,
+        "hedging.misc": True,
+        "hyperbole.misc": True,
+        "jargon.misc": True,
+        "lexical_illusions.misc": True,
+        "lgbtq.offensive_terms": True,
+        "lgbtq.terms": True,
+        "links.broken": False,
+        "malapropisms.misc": True,
+        "misc.apologizing": True,
+        "misc.back_formations": True,
+        "misc.bureaucratese": True,
+        "misc.but": True,
+        "misc.capitalization": True,
+        "misc.chatspeak": True,
+        "misc.commercialese": True,
+        "misc.composition": True,
+        "misc.currency": True,
+        "misc.debased": True,
+        "misc.false_plurals": True,
+        "misc.illogic": True,
+        "misc.inferior_superior": True,
+        "misc.institution_name": True,
+        "misc.latin": True,
+        "misc.many_a": True,
+        "misc.metaconcepts": True,
+        "misc.metadiscourse": True,
+        "misc.narcissism": True,
+        "misc.not_guilty": True,
+        "misc.phrasal_adjectives": True,
+        "misc.preferred_forms": True,
+        "misc.pretension": True,
+        "misc.professions": True,
+        "misc.punctuation": True,
+        "misc.scare_quotes": True,
+        "misc.suddenly": True,
+        "misc.tense_present": True,
+        "misc.waxed": True,
+        "misc.whence": True,
+        "mixed_metaphors.misc": True,
+        "mondegreens.misc": True,
+        "needless_variants.misc": True,
+        "nonwords.misc": True,
+        "oxymorons.misc": True,
+        "psychology.misc": True,
+        "redundancy.misc": True,
+        "redundancy.ras_syndrome": True,
+        "skunked_terms.misc": True,
+        "spelling.able_atable": True,
+        "spelling.able_ible": True,
+        "spelling.athletes": True,
+        "spelling.em_im_en_in": True,
+        "spelling.er_or": True,
+        "spelling.in_un": True,
+        "spelling.misc": True,
+        "security.credit_card": True,
+        "security.password": True,
+        "sexism.misc": True,
+        "terms.animal_adjectives": True,
+        "terms.denizen_labels": True,
+        "terms.eponymous_adjectives": True,
+        "terms.venery": True,
+        "typography.diacritical_marks": True,
+        "typography.exclamation": True,
+        "typography.symbols": True,
+        "uncomparables.misc": True,
+        "weasel_words.misc": True,
+        "weasel_words.very": True
+    }
+}
diff --git a/proselint/tools.py b/proselint/tools.py
index f4ee7ad..ec43ecf 100644
--- a/proselint/tools.py
+++ b/proselint/tools.py
@@ -1,6 +1,5 @@
 """General-purpose tools shared across linting checks."""
 
-
 import copy
 import dbm
 import functools
@@ -13,6 +12,7 @@ import re
 import shelve
 import sys
 import traceback
+from warnings import showwarning as warn
 
 _cache_shelves = dict()
 proselint_path = os.path.dirname(os.path.realpath(__file__))
@@ -22,7 +22,7 @@ cwd = os.getcwd()
 
 def close_cache_shelves():
     """Close previously opened cache shelves."""
-    for _, cache in _cache_shelves.items():
+    for cache in _cache_shelves.values():
         cache.close()
     _cache_shelves.clear()
 
@@ -138,8 +138,7 @@ def get_checks(options):
     """Extract the checks."""
     sys.path.append(proselint_path)
     checks = []
-    check_names = [key for (key, val)
-                   in list(options["checks"].items()) if val]
+    check_names = [key for (key, val) in options["checks"].items() if val]
 
     for check_name in check_names:
         module = importlib.import_module("checks." + check_name)
@@ -163,24 +162,18 @@ def deepmerge_dicts(dict1, dict2):
     return result
 
 
-def load_options(config_file_path=None):
+def load_options(config_file_path=None, conf_default=None):
     """Read various proselintrc files, allowing user overrides."""
-    system_config_paths = (
-        '/etc/proselintrc',
-        os.path.join(proselint_path, '.proselintrc'),
-    )
-
-    system_options = {}
-    for path in system_config_paths:
-        if os.path.isfile(path):
-            system_options = json.load(open(path))
-            break
+    conf_default = conf_default or {}
+    if os.path.isfile("/etc/proselintrc"):
+        conf_default = json.load(open("/etc/proselintrc"))
 
     user_config_paths = [
-        os.path.join(cwd, '.proselintrc'),
-        os.path.join(_get_xdg_config_home(), 'proselint', 'config'),
-        os.path.join(home_dir, '.proselintrc')
+        os.path.join(cwd, '.proselintrc.json'),
+        os.path.join(_get_xdg_config_home(), 'proselint', 'config.json'),
+        os.path.join(home_dir, '.proselintrc.json')
     ]
+
     if config_file_path:
         if not os.path.isfile(config_file_path):
             raise FileNotFoundError(
@@ -192,10 +185,14 @@ def load_options(config_file_path=None):
         if os.path.isfile(path):
             user_options = json.load(open(path))
             break
+        oldpath = path.replace(".json", "")
+        if os.path.isfile(oldpath):
+            warn(f"{oldpath} was found instead of a JSON file."
+                 f" Rename to {path}.", DeprecationWarning, "", 0)
+            user_options = json.load(open(oldpath))
+            break
 
-    options = deepmerge_dicts(system_options, user_options)
-
-    return options
+    return deepmerge_dicts(conf_default, user_options)
 
 
 def errors_to_json(errors):
@@ -215,7 +212,7 @@ def errors_to_json(errors):
         })
 
     return json.dumps(
-        dict(status="success", data={"errors": out}), sort_keys=True)
+        {"status": "success", "data": {"errors": out}}, sort_keys=True)
 
 
 def line_and_column(text, position):
@@ -230,17 +227,16 @@ def line_and_column(text, position):
     return (line_no, position - position_counter)
 
 
-def lint(input_file, debug=False, config_file_path=None):
+def lint(input_file, debug=False, config=None):
     """Run the linter on the input file."""
-    options = load_options(config_file_path)
-
+    config = config or {}
     if isinstance(input_file, str):
         text = input_file
     else:
         text = input_file.read()
 
     # Get the checks.
-    checks = get_checks(options)
+    checks = get_checks(config)
 
     # Apply all the checks.
     errors = []
@@ -255,11 +251,11 @@ def lint(input_file, debug=False, config_file_path=None):
                 errors += [(check, message, line, column, start, end,
                             end - start, "warning", replacements)]
 
-        if len(errors) > options["max_errors"]:
+        if len(errors) > config["max_errors"]:
             break
 
     # Sort the errors by line and column number.
-    errors = sorted(errors[:options["max_errors"]], key=lambda e: (e[2], e[3]))
+    errors = sorted(errors[:config["max_errors"]], key=lambda e: (e[2], e[3]))
 
     return errors
 


Test patch:
diff --git a/tests/test_config_flag.py b/tests/test_config_flag.py
index e9fc265..64ad547 100644
--- a/tests/test_config_flag.py
+++ b/tests/test_config_flag.py
@@ -1,9 +1,17 @@
 """Test user option overrides using --config and load_options"""
+import json
+import os
+from unittest import TestCase
+from unittest.mock import patch
+
 from click.testing import CliRunner
 
 from proselint.command_line import proselint
+from proselint.config import default
 from proselint.tools import deepmerge_dicts, load_options
 
+runner = CliRunner()
+
 
 def test_deepmerge_dicts():
     """Test deepmerge_dicts"""
@@ -12,17 +20,23 @@ def test_deepmerge_dicts():
     assert deepmerge_dicts(d1, d2) == {'a': 2, 'b': {'c': 3, 'd': 3, 'e': 4}}
 
 
-def test_load_options_function():
+@patch("os.path.isfile")
+def test_load_options_function(isfile):
     """Test load_options by specifying a user options path"""
-    overrides = load_options("tests/test_config_flag_proselintrc")
-    assert load_options()["checks"]["uncomparables.misc"]
+
+    isfile.side_effect = "tests/test_config_flag_proselintrc".__eq__
+
+    overrides = load_options("tests/test_config_flag_proselintrc", default)
+    assert load_options(conf_default=default)["checks"]["uncomparables.misc"]
     assert not overrides["checks"]["uncomparables.misc"]
 
+    isfile.side_effect = os.path.join(os.getcwd(), ".proselintrc").__eq__
+
+    TestCase().assertRaises(FileNotFoundError, load_options)
+
 
 def test_config_flag():
     """Test the --config CLI argument"""
-    runner = CliRunner()
-
     output = runner.invoke(proselint, "--demo")
     assert "uncomparables.misc" in output.stdout
 
@@ -36,3 +50,14 @@ def test_config_flag():
 
     output = runner.invoke(proselint, "non_existent_file")
     assert output.exit_code == 2
+
+
+def test_dump_config():
+    """Test --dump-default-config and --dump-config"""
+    output = runner.invoke(proselint, "--dump-default-config")
+    assert json.loads(output.stdout) == default
+
+    output = runner.invoke(
+        proselint, "--dump-config --config tests/test_config_flag_proselintrc")
+    assert json.loads(output.stdout) == json.load(
+        open("tests/test_config_flag_proselintrc"))
diff --git a/tests/test_tools.py b/tests/test_tools.py
index b5408e2..ec110d6 100644
--- a/tests/test_tools.py
+++ b/tests/test_tools.py
@@ -1,11 +1,16 @@
 """Test the tools module."""
 
 
-from proselint.tools import lint
+from proselint.config import default
+from proselint.tools import lint as proselint, load_options
 
 from .check import Check
 
 
+def lint(text):
+    return proselint(text, config=load_options(conf_default=default))
+
+
 class TestLint(Check):
     """The test class for tools.lint."""