Repository: python/mypy

Issue:
UnicodeDecodeError on Windows with pyproject.toml
<!--
  Use this form only if mypy reports an "INTERNAL ERROR" and/or gives a traceback.
  Please include the traceback and all other messages below (use `mypy --show-traceback`).
-->

**Crash Report**

On Windows (where the default encoding for `open` is usually not UTF-8), with a `pyproject.toml` file containing the mypy configuration, mypy crashes due to being unable to decode unicode characters.

The [TOML specification](https://toml.io/en/v1.0.0#spec) requires files to be encoded in UTF-8, so it is valid to assume tools will correctly parse files containing unicode characters.

**Traceback**

```python
Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\a\domdf_python_tools\domdf_python_tools\.tox\mypy\Scripts\mypy.EXE\__main__.py", line 7, in <module>
  File "D:\a\domdf_python_tools\domdf_python_tools\.tox\mypy\lib\site-packages\mypy\__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "mypy\main.py", line 87, in main
  File "mypy\main.py", line 165, in run_build
  File "mypy\build.py", line 179, in build
  File "mypy\build.py", line 229, in _build
  File "mypy\build.py", line 475, in load_plugins
  File "mypy\build.py", line 391, in load_plugins_from_config
  File "mypy\build.py", line 511, in find_config_file_line_number
  File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 220: character maps to <undefined>
```

This is caused by the following function, particularly the call to `open` on line 511 without an encoding being specified.

https://github.com/python/mypy/blob/5c4aea39ab6a14eeef85cc849d6057bebf2147a3/mypy/build.py#L503-L523

**To Reproduce**

* Use the following `pyproject.toml` file content:
```toml
[project]
description = "Factory ⸻ A code generator 🏭"

[tool.mypy]
```

* Run mypy 0.910 on Windows

**Your Environment**

<!-- Include as many relevant details about the environment you experienced the bug in -->

- Mypy version used: 0.910
- Mypy command-line flags: None
- Mypy configuration options from `mypy.ini` (and other config files): None (empty `tool.mypy` section in `pyproject.toml`)
- Python version used: 3.6 and 3.9
- Operating system and version: Windows 10

<!--
You can freely edit this text, please remove all the lines
you believe are unnecessary.
-->



Discussion:

Resolving patch:
diff --git a/mypy/build.py b/mypy/build.py
--- a/mypy/build.py
+++ b/mypy/build.py
@@ -511,7 +511,7 @@ def find_config_file_line_number(path: str, section: str, setting_name: str) ->
     in_desired_section = False
     try:
         results = []
-        with open(path) as f:
+        with open(path, encoding="UTF-8") as f:
             for i, line in enumerate(f):
                 line = line.strip()
                 if line.startswith('[') and line.endswith(']'):


Test patch:
diff --git a/test-data/unit/cmdline.pyproject.test b/test-data/unit/cmdline.pyproject.test
--- a/test-data/unit/cmdline.pyproject.test
+++ b/test-data/unit/cmdline.pyproject.test
@@ -125,3 +125,11 @@ i: int = 0
 This isn't even syntatically valid!
 [file x/please_skipme_.py]
 Neither is this!
+
+[case testPyprojectTOMLUnicode]
+# cmd: mypy x.py
+[file pyproject.toml]
+\[project]
+description = "Factory ⸻ A code generator 🏭"
+\[tool.mypy]
+[file x.py]