Repository: psf/black

Issue:
AttributeError: 'int' object has no attribute 'lower'
**Describe the bug** 
When I try to run black on a file I get this error. even a simple file that has only ```print('hello world')```
I am runnning ```black temp.py```

**To Reproduce** Steps to reproduce the behavior:
```black temp.py```

**Environment (please complete the following information):**
python 3.8 macos
black.__version__
'19.10b0'
click.__version__
'7.1.2'

**Error** 

`(venv) $ black temp.py
Traceback (most recent call last):
  File "/venv/bin/black", line 10, in <module>
    sys.exit(patched_main())
  File "/venv/lib/python3.8/site-packages/black.py", line 4135, in patched_main
    main()
  File "/venv/lib/python3.8/site-packages/click/core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 781, in main
    with self.make_context(prog_name, args, **extra) as ctx:
  File "/venv/lib/python3.8/site-packages/click/core.py", line 700, in make_context
    self.parse_args(ctx, args)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1048, in parse_args
    value, args = param.handle_parse_result(ctx, opts, args)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1623, in handle_parse_result
    value = self.full_process_value(ctx, value)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1965, in full_process_value
    return Parameter.full_process_value(self, ctx, value)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1589, in full_process_value
    value = self.process_value(ctx, value)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1579, in process_value
    return self.type_cast_value(ctx, value)
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1568, in type_cast_value
    return _convert(value, (self.nargs != 1) + bool(self.multiple))
  File "/venv/lib/python3.8/site-packages/click/core.py", line 1565, in _convert
    return self.type(value, self, ctx)
  File "/venv/lib/python3.8/site-packages/click/types.py", line 46, in __call__
    return self.convert(value, param, ctx)
  File "/venv/lib/python3.8/site-packages/click/types.py", line 412, in convert
    value = value.lower()
AttributeError: 'int' object has no attribute 'lower'`




Discussion:
At first glance, it looks like your installation of `click` is messed up. Try `python38 -m pip install click --force-reinstall (and maybe --user)`. I'll look into it further once I get access to an computer in a moment.

Edit: please modify the pip command as needed since I just realized that you are using a virtual environment.
@ichard26 
I get the same error

`(venv) VMD-MBP-16:Zwiftpower_tools vincentdavis$ which python
/Users/vincentdavis/repo/Zwiftpower_tools/venv/bin/python`

`(venv) VMD-MBP-16:Zwiftpower_tools vincentdavis$ which black
/Users/vincentdavis/repo/Zwiftpower_tools/venv/bin/black`

`(venv) VMD-MBP-16:Zwiftpower_tools vincentdavis$ python -m pip install click --force-reinstall --upgrade
Collecting click
  Using cached https://files.pythonhosted.org/packages/d2/3d/fa76db83bf75c4f8d338c2fd15c8d33fdd7ad23a9b5e57eb6c5de26b430e/click-7.1.2-py2.py3-none-any.whl
Installing collected packages: click
  Found existing installation: click 7.1.2
    Uninstalling click-7.1.2:
      Successfully uninstalled click-7.1.2
Successfully installed click-7.1.2`
@ichard26 I cloned, created a new venv and ran the test. I get one error(see below). I tried to run this version on black on my simple hello world file and get the same error as above

```
(.venv) VMD-MBP-16:black vincentdavis$ python -m unittest               E....[2020-05-24 16:40:45,413] DEBUG: Using selector: KqueueSelector (selector_events.py:54)
.[2020-05-24 16:40:45,414] DEBUG: Using selector: KqueueSelector (selector_events.py:54)
.[2020-05-24 16:40:45,772] DEBUG: Using selector: KqueueSelector (selector_events.py:54)
.[2020-05-24 16:40:45,773] DEBUG: Using selector: KqueueSelector (selector_events.py:54)
[2020-05-24 16:40:45,775] INFO: 16 projects to run Black over (lib.py:311)
[2020-05-24 16:40:45,775] DEBUG: Using 2 parallel workers to run Black (lib.py:316)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on aioexabgp (lib.py:247)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on attrs (lib.py:247)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on bandersnatch (lib.py:247)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on channels (lib.py:247)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on django (lib.py:247)
[2020-05-24 16:40:45,775] INFO: Skipping django as it's disabled via config (lib.py:254)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on flake8-bugbear (lib.py:247)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on hypothesis (lib.py:247)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on pandas (lib.py:247)
[2020-05-24 16:40:45,775] INFO: Skipping pandas as it's disabled via config (lib.py:254)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on poetry (lib.py:247)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on ptr (lib.py:247)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on pyramid (lib.py:247)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on pytest (lib.py:247)
[2020-05-24 16:40:45,775] INFO: Skipping pytest as it's disabled via config (lib.py:254)
[2020-05-24 16:40:45,775] DEBUG: worker 0 working on sqlalchemy (lib.py:247)
[2020-05-24 16:40:45,776] DEBUG: worker 0 working on tox (lib.py:247)
[2020-05-24 16:40:45,776] DEBUG: worker 0 working on virtualenv (lib.py:247)
[2020-05-24 16:40:45,776] DEBUG: worker 0 working on warehouse (lib.py:247)
[2020-05-24 16:40:45,776] DEBUG: project_runner 0 exiting (lib.py:245)
[2020-05-24 16:40:45,776] DEBUG: project_runner 1 exiting (lib.py:245)
[2020-05-24 16:40:45,776] INFO: Analyzing results (lib.py:327)
.
======================================================================
ERROR: tests.test_black (unittest.loader._FailedTest)
----------------------------------------------------------------------
ImportError: Failed to import test module: tests.test_black
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 436, in _find_test_path
    module = self._get_module_from_name(name)
  File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/unittest/loader.py", line 377, in _get_module_from_name
    __import__(name)
  File "/Users/vincentdavis/repo/black/tests/test_black.py", line 1766, in <module>
    class BlackDTestCase(AioHTTPTestCase):
NameError: name 'AioHTTPTestCase' is not defined


----------------------------------------------------------------------
Ran 9 tests in 0.370s

FAILED (errors=1)
```

You need to install with the blackd "extra" dependencies to run the test suite. 

```
pip install -e .[d]
```

I think it's time to write a wrapper to run all our unittests and make it easier for first timers.
@cooperlees I did not have the `[d]` on the pip install command. Here are the new results.

(.venv) VMD-MBP-16:black vincentdavis$ python -m unittest
/Users/vincentdavis/repo/black/.venv/lib/python3.8/site-packages/aiohttp/connector.py:964: DeprecationWarning: The loop argument is deprecated since Python 3.8, and scheduled for removal in Python 3.10.
  hosts = await asyncio.shield(self._resolve_host(
........................--- /Users/vincentdavis/repo/black/tests/data/string_quotes.py  2020-05-24 22:36:58.082733 +0000
+++ /Users/vincentdavis/repo/black/tests/data/string_quotes.py  2020-05-25 00:27:57.472129 +0000
@@ -1,57 +1,52 @@
-''''''
-'\''
+""""""
+"'"
 '"'
 "'"
-"\""
+'"'
 "Hello"
 "Don't do that"
 'Here is a "'
-'What\'s the deal here?'
-"What's the deal \"here\"?"
-"And \"here\"?"
+"What's the deal here?"
+'What\'s the deal "here"?'
+'And "here"?'
 """Strings with "" in them"""
-'''Strings with "" in them'''
+"""Strings with "" in them"""
 '''Here's a "'''
-'''Here's a " '''
-'''Just a normal triple
-quote'''
+"""Here's a " """
+"""Just a normal triple
+quote"""
 f"just a normal {f} string"
-f'''This is a triple-quoted {f}-string'''
+f"""This is a triple-quoted {f}-string"""
 f'MOAR {" ".join([])}'
 f"MOAR {' '.join([])}"
 r"raw string ftw"
-r'Date d\'expiration:(.*)'
+r"Date d\'expiration:(.*)"
 r'Tricky "quote'
-r'Not-so-tricky \"quote'
-rf'{yay}'
-'\n\
-The \"quick\"\n\
-brown fox\n\
-jumps over\n\
-the \'lazy\' dog.\n\
-'
+r"Not-so-tricky \"quote"
+rf"{yay}"
+"\nThe \"quick\"\nbrown fox\njumps over\nthe 'lazy' dog.\n"
 re.compile(r'[\\"]')
 "x = ''; y = \"\""
 "x = '''; y = \"\""
 "x = ''''; y = \"\""
 "x = '' ''; y = \"\""
-"x = ''; y = \"\"\""
-"x = '''; y = \"\"\"\""
-"x = ''''; y = \"\"\"\"\""
-"x = '' ''; y = \"\"\"\"\""
-'unnecessary \"\"escaping'
-"unnecessary \'\'escaping"
+'x = \'\'; y = """'
+'x = \'\'\'; y = """"'
+'x = \'\'\'\'; y = """""'
+'x = \'\' \'\'; y = """""'
+'unnecessary ""escaping'
+"unnecessary ''escaping"
 '\\""'
 "\\''"
-'Lots of \\\\\\\\\'quotes\''
+"Lots of \\\\\\\\'quotes'"
 f'{y * " "} \'{z}\''
-f'{{y * " "}} \'{z}\''
+f"{{y * \" \"}} '{z}'"
 f'\'{z}\' {y * " "}'
-f'{y * x} \'{z}\''
-'\'{z}\' {y * " "}'
-'{y * x} \'{z}\''
+f"{y * x} '{z}'"
+"'{z}' {y * \" \"}"
+"{y * x} '{z}'"
 
 # output
 
 """"""
 "'"
....................................................................................................................
----------------------------------------------------------------------
Ran 140 tests in 26.340s

OK
/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/asyncio/base_events.py:646: ResourceWarning: unclosed event loop <_UnixSelectorEventLoop running=False closed=False debug=False>`

Right, so the tests pass for you. Then we're back to @ichard26's original diagnosis that you probably have some issue with click in your environment.
> Right, so the tests pass for you. Then we're back to @ichard26's original diagnosis that you probably have some issue with click in your environment.

I created a new env and pip installed black, and it fails. Sure it might be click but I am not trying to use click. :-) I did try installing several different versions of click with no luck.
something else is going on, it did work a few weeks ago.

Ideas?
Is there by any chance a `pyproject.toml` in the directory you are running _Black_ on? If so, would it be possible to see the source?
> Is there by any chance a `pyproject.toml` in the directory you are running _Black_ on? If so, would it be possible to see the source?

Wow, yes that was it. The `pyproject.toml` had only the following. Thank you!
```
[tool.black]
line-length = 120
skip-string-normalization = 0
```
Even though I am not in the folder with the `pyproject.toml` black finds that and uses it, that is not obvious. Maybe black could show it is using that configuration when it runs or a better error.

Could you change `skip-string-normalization = 0` to `skip-string-normalization = True`.

`click` accepts it that way, we should update this in the documentation. 
@vincentdavis, sorry for not responding earlier, my laptop charger decided yesterday was the day to stop working and that had thrown my plans to investigate the issue out the window. I'll see what can do in terms of documentation. FYI, if you run _Black_ in verbose mode, you will see what config file it is using:
> If you're running with `--verbose`, you will see a blue message if a file was found and used.

[source](https://github.com/psf/black#where-black-looks-for-the-file)
@SanketDG, do you know why `Click` doesn't like integers as bool? If you replace `skip-string-normalization = 0` with `skip-string-normalization = "hi"` (in the TOML file), `Click` gives a helpful message instead of throwing an exception:

```console
(black-rHKUX7ap) R:\Programming\black>black setup.py

Usage: black [OPTIONS] [SRC]...
Try 'black -h' for help.

Error: Invalid value for '-S' / '--skip-string-normalization': hi is not a valid boolean
```

It accepts `"false"` and will convert it to `False` ... is this just a bug with `Click` since using ones and zeros as boolean seems reasonable IMO.
@SanketDG , @ichard26 

- My issue is fixed and you see/found the problem.

- The hint to use --verbose was a good one.

- IMO some way to catch the error as a configuration problem would be helpful.

- IMO black should always show if a configuration file is being used as part of its run message.

Thanks for the help I will leave this to you.
@ichard26 

> do you know why `Click` doesn't like integers as bool? If you replace `skip-string-normalization = 0` with `skip-string-normalization = "hi"` (in the TOML file), `Click` gives a helpful message instead of throwing an exception:

Apparently, that's how it's written https://github.com/pallets/click/blob/97de2c97275f52adc5c6ef8bb8aa993ce997fe7e/src/click/types.py#L389

Also, https://github.com/pallets/click/issues/1553, this one's from running _Black_ too

> * IMO some way to catch the error as a configuration problem would be helpful.

This seems hard to do when `AttributeError` is just being raised. Opening an issue on `click` seems the right way, ideally a more specific exception is returned.

> * IMO black should always show if a configuration file is being used as part of its run message.

:+1: 
@SanketDG thanks for the info! What's interesting is that they accept `"1"` but not `1`, which I guess is reasonable since all input from the command line are strings. I do agree with you that a error message should be outputted then an hard to protect `AttributeError` happening. Since its happening in the `Click` core, we would have to `try and except` the `main()` call which isn't ideal. We could just verify the types in the _Black_ TOML logic, but that isn't ideal too. 

Although this isn't exactly ~~welcoming~~ promising for an issue :-)
> That sounds like an issue with how they're using Click, not with Click. ;-)
https://github.com/toml-lang/toml/pull/116

https://json-schema-everywhere.github.io/toml
So I have opened an issue on Click's repository to see if they would be interested in either having `BoolParamType` gently (so no `AttributeError`) disallow `0` and `1` as boolean or start accepting them as boolean. And now we are left with a few options to choose from:

1. Just add a note in the documentation that the values should follow the TOML specification for boolean, integer, etc
2. Add our own in-house value verification logic to handle bad types and bad values (seems like a lot of work and maintenance although we could provide specialized and helpful error messages)
3. Use a tool like https://pypi.org/project/yschema/ to do value validation (I don't like the extra dependency)
4. Just convert all of the values pulled from the `pyproject.toml` file into strings and let Click handle the conversions (recommended by a core developer of the Click project; seems good enough to me)

Opinions? I like number four the most and it seems to not **break** anything.
> 4\. Just convert all of the values pulled from the `pyproject.toml` file into strings and let Click handle the conversions

This seems the least invasive, but if I am getting it right, if `0` is passed as a string, it would get autoconverted to a boolean?
Well `Click` accepts a lot of different values as boolean. I'll list them in my PR for this change. First gotta test though.

Resolving patch:
diff --git a/src/black/__init__.py b/src/black/__init__.py
--- a/src/black/__init__.py
+++ b/src/black/__init__.py
@@ -281,7 +281,12 @@ def parse_pyproject_toml(path_config: str) -> Dict[str, Any]:
     """
     pyproject_toml = toml.load(path_config)
     config = pyproject_toml.get("tool", {}).get("black", {})
-    return {k.replace("--", "").replace("-", "_"): v for k, v in config.items()}
+    return {
+        k.replace("--", "").replace("-", "_"): str(v)
+        if not isinstance(v, (list, dict))
+        else v
+        for k, v in config.items()
+    }
 
 
 def read_pyproject_toml(


Test patch: