solving `make quality` failures
Author: stas00
Pull request: no
Labels: name: wontfix

`make quality` currently and for a while now fails with 1 warning and 1 failure:

1. isort warning:
with `isort==4.3.21` or the required by the latest stable `pylint`, or `setup.py`'s current
`git+git://github.com/timothycrosley/isort.git@e63ae06ec7d70b06df9e528357650281a3d3ec22#egg=isort`

we get:
```
make style
```
```
/home/stas/anaconda3/envs/main/lib/python3.7/site-packages/setuptools/distutils_patch.py:26: UserWarning: Distutils was imported before Setuptools. This usage is discouraged and may exhibit undesirable behaviors or errors. Please use Setuptools' objects directly or at least import Setuptools first.
  "Distutils was imported before Setuptools. This usage is discouraged "
[...]
```

If I install `isort==5.3.0` it now wants to reformat a whole bunch of imports:
```
ERROR: /mnt/nvme1/code/huggingface/transformers-unittests/examples/longform-qa/eli5_app.py Imports are incorrectly sorted and/or formatted.
ERROR: /mnt/nvme1/code/huggingface/transformers-unittests/examples/text-generation/pplm/run_pplm_discrim_train.py Imports are incorrectly sorted and/or formatted.
[...] some dozens of those
```

This version has deprecated the `--recursive` flag to `isort`, so once the code is re-formatted to appease to never-ending new rules we can:

1. require  `isort>=5.3.0` in `setup.py`'s `quality` section 
2. remove  the `--recursive` flag to `isort` in Makefile (I validated that just removing this deprecated flag won't change the configuration - it still checks the listed dirs recursively)

the only potential problem if we need to appease to `pylint`, which wants `isort==4.3.21`

----

2. and then older `flake8` can't handle `TYPE_CHECKING` - error

```
flake8 examples templates tests src utils

tests/test_tokenization_common.py:31:5: F401 'transformers.PretrainedConfig' imported but unused
tests/test_tokenization_common.py:31:5: F401 'transformers.PreTrainedModel' imported but unused
tests/test_tokenization_common.py:31:5: F401 'transformers.TFPreTrainedModel' imported but unused
src/transformers/pipelines.py:77:5: F401 '.modeling_utils.PreTrainedModel' imported but unused
src/transformers/pipelines.py:78:5: F401 '.modeling_tf_utils.TFPreTrainedModel' imported but unused
`
```

`flake8-3.8.3` doesn't complain about these.

Can we add:

```
diff --git a/setup.py b/setup.py
index 206c3e35..c33898b4 100644
--- a/setup.py
+++ b/setup.py
@@ -95,7 +95,7 @@ extras["quality"] = [
     "black",
     # "isort",
     "isort @ git+git://github.com/timothycrosley/isort.git@e63ae06ec7d70b06df9e528357650281a3d3ec22#egg=isort",
-    "flake8",
+    "flake8>=3.8.3",
 ]
 extras["dev"] = extras["testing"] + extras["quality"] + extras["ja"] + ["scikit-learn", "tensorflow", "torch"]
```





Comments:
Confirming the warning started to appear since I did a conda upgrade (for pytorch 1.6.0), never got the error here.

> Confirming the warning started to appear since I did a conda upgrade (for pytorch 1.6.0)

Thank you for validating this.

The warning is kind of like an error, since it's noisy, so there is no quick way to see if all is clean before committing.

> never got the error here.

You probably happened to have a newer `flake8`, hence suggesting a minimum requirement.


This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.


this has been resolved.