Repository: typeddjango/django-stubs

Issue:
Exceptions should be reported in a more clear way
# Technical improvement suggestion
Not exactly a bug something that should get some more attention: exception handling and reporting them to the user. I've been working on the configuration reading in `main.py` module and found that _handled exceptions_ end up being reported like this:
```bash
$ mypy -p django-stubs --config-file ./test-data/plugins.ini
Error constructing plugin instance of NewSemanalDjangoPlugin

test-data/plugins.ini:0: error: 'django_settings_module' is not set: no section [mypy.plugins.django-stubs]
Found 1 error in 1 file (checked 477 source files)
```
![image](https://user-images.githubusercontent.com/16212750/86977324-466ae880-c174-11ea-9d46-5329753c9725.png)

in comparison to

**mypy error with a cmd line argument**
```bash
$ mypy -p django-stubs --config-file ./tst-data/plugins.ini
usage: mypy [-h] [-v] [-V] [more options; see below]
            [-m MODULE] [-p PACKAGE] [-c PROGRAM_TEXT] [files ...]
mypy: error: Cannot find config file './tst/plugins.ini'
```
or the unhandled default
 
```bash
You must install the lxml package before you can run mypy with `--html-report`.
You can do this with `python3 -m pip install lxml`.
Traceback (most recent call last):
  File "/home/sarunas/.pyenv/versions/django-stubs-3.6.11/bin/mypy", line 11, in <module>
    sys.exit(console_entry())
  File "/home/sarunas/.pyenv/versions/3.6.11/envs/django-stubs-3.6.11/lib/python3.6/site-packages/mypy/__main__.py", line 8, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "mypy/main.py", line 89, in main
  File "mypy/build.py", line 180, in build
  File "mypy/build.py", line 217, in _build
  File "mypy/report.py", line 58, in __init__
  File "mypy/report.py", line 71, in add_report
ImportError
```

While the output is the most visually appealing, I find it very quite opaque and misleading. No source files have been checked by that point, so including the count isn't correct; it's also formatted like an issue, so my first reaction (which is basically muscle memory now) is to switch back to the source code and to have a look at the most recent change that has been made. It takes a lot to find that this is an internal problem rather than something that is related to my source files. 

I'd think that `mypy.Errors` may not be the best solution for this - even though it's a very convenient one. There are a couple of unneeded calls made until it reaches a more related `CompileError` - though it gets thrown plainly, without using its `report_internal_error` method - which seems like a good option that would include some extras for the user for debugging. Though I guess `mypy` may have to change into something more related to this project. Though I haven't seen any of them writing to `stderr` instead of `stdout`. Also, once the issues are raised, they return `None` instead of `NoReturn` - could this indicate that they are not meant to cut off the execution completely?

Though, correct if I'm wrong (I probably am, since I haven't spent days on this :) ), but both of the above seem to be used _during the parsing / linting stage_. Would it be wrong to assume that reading the configuration should be treated as the setup stage? This way, it could report the error in the same format like `mypy` does - could also reuse their stuff. This shouldn't be a significant change I reckon. 


Discussion:
@snejus I would love to merge a PR with this feature! 
Perfect! It's in the works :)

Resolving patch:
diff --git a/mypy_django_plugin/main.py b/mypy_django_plugin/main.py
--- a/mypy_django_plugin/main.py
+++ b/mypy_django_plugin/main.py
@@ -1,9 +1,8 @@
 import configparser
 from functools import partial
-from typing import Callable, Dict, List, Optional, Tuple
+from typing import Callable, Dict, List, NoReturn, Optional, Tuple, cast
 
 from django.db.models.fields.related import RelatedField
-from mypy.errors import Errors
 from mypy.nodes import MypyFile, TypeInfo
 from mypy.options import Options
 from mypy.plugin import (
@@ -52,25 +51,40 @@ def add_new_manager_base(ctx: ClassDefContext) -> None:
 
 
 def extract_django_settings_module(config_file_path: Optional[str]) -> str:
-    errors = Errors()
-    if config_file_path is None:
-        errors.report(0, None, "'django_settings_module' is not set: no mypy config file specified")
-        errors.raise_error()
+
+    def exit(error_type: int) -> NoReturn:
+        """Using mypy's argument parser, raise `SystemExit` to fail hard if validation fails.
+
+        Considering that the plugin's startup duration is around double as long as mypy's, this aims to
+        import and construct objects only when that's required - which happens once and terminates the
+        run. Considering that most of the runs are successful, there's no need for this to linger in the
+        global scope.
+        """
+        from mypy.main import CapturableArgumentParser
+
+        usage = """(config)
+        ...
+        [mypy.plugins.django_stubs]
+            django_settings_module: str (required)
+        ...
+        """.replace("\n" + 8 * " ", "\n")
+        handler = CapturableArgumentParser(prog='(django-stubs) mypy', usage=usage)
+        messages = {1: 'mypy config file is not specified or found',
+                    2: 'no section [mypy.plugins.django-stubs]',
+                    3: 'the setting is not provided'}
+        handler.error("'django_settings_module' is not set: " + messages[error_type])
 
     parser = configparser.ConfigParser()
-    parser.read(config_file_path)  # type: ignore
-
-    if not parser.has_section('mypy.plugins.django-stubs'):
-        errors.report(0, None, "'django_settings_module' is not set: no section [mypy.plugins.django-stubs]",
-                      file=config_file_path)
-        errors.raise_error()
-    if not parser.has_option('mypy.plugins.django-stubs', 'django_settings_module'):
-        errors.report(0, None, "'django_settings_module' is not set: setting is not provided",
-                      file=config_file_path)
-        errors.raise_error()
-
-    django_settings_module = parser.get('mypy.plugins.django-stubs', 'django_settings_module').strip('\'"')
-    return django_settings_module
+    try:
+        parser.read_file(open(cast(str, config_file_path), 'r'), source=config_file_path)
+    except (IsADirectoryError, OSError):
+        exit(1)
+
+    section = 'mypy.plugins.django-stubs'
+    if not parser.has_section(section):
+        exit(2)
+    settings = parser.get(section, 'django_settings_module', fallback=None) or exit(3)
+    return cast(str, settings).strip('\'"')
 
 
 class NewSemanalDjangoPlugin(Plugin):


Test patch:
diff --git a/test-plugin/test_error_handling.py b/test-plugin/test_error_handling.py
new file mode 100644
--- /dev/null
+++ b/test-plugin/test_error_handling.py
@@ -0,0 +1,73 @@
+import tempfile
+import typing
+
+import pytest
+
+from mypy_django_plugin.main import extract_django_settings_module
+
+TEMPLATE = """usage: (config)
+...
+[mypy.plugins.django_stubs]
+    django_settings_module: str (required)
+...
+(django-stubs) mypy: error: 'django_settings_module' is not set: {}
+"""
+
+
+@pytest.mark.parametrize(
+    'config_file_contents,message_part',
+    [
+        pytest.param(
+            None,
+            'mypy config file is not specified or found',
+            id='missing-file',
+        ),
+        pytest.param(
+            ['[not-really-django-stubs]'],
+            'no section [mypy.plugins.django-stubs]',
+            id='missing-section',
+        ),
+        pytest.param(
+            ['[mypy.plugins.django-stubs]',
+             '\tnot_django_not_settings_module = badbadmodule'],
+            'the setting is not provided',
+            id='missing-settings-module',
+        ),
+        pytest.param(
+            ['[mypy.plugins.django-stubs]'],
+            'the setting is not provided',
+            id='no-settings-given',
+        ),
+    ],
+)
+def test_misconfiguration_handling(capsys, config_file_contents, message_part):
+    #  type: (typing.Any, typing.List[str], str) -> None
+    """Invalid configuration raises `SystemExit` with a precise error message."""
+    with tempfile.NamedTemporaryFile(mode='w+') as config_file:
+        if not config_file_contents:
+            config_file.close()
+        else:
+            config_file.write('\n'.join(config_file_contents).expandtabs(4))
+            config_file.seek(0)
+
+        with pytest.raises(SystemExit, match='2'):
+            extract_django_settings_module(config_file.name)
+
+    error_message = TEMPLATE.format(message_part)
+    assert error_message == capsys.readouterr().err
+
+
+def test_correct_configuration() -> None:
+    """Django settings module gets extracted given valid configuration."""
+    config_file_contents = [
+        '[mypy.plugins.django-stubs]',
+        '\tsome_other_setting = setting',
+        '\tdjango_settings_module = my.module',
+    ]
+    with tempfile.NamedTemporaryFile(mode='w+') as config_file:
+        config_file.write('\n'.join(config_file_contents).expandtabs(4))
+        config_file.seek(0)
+
+        extracted = extract_django_settings_module(config_file.name)
+
+    assert extracted == 'my.module'