DEV/STYLE: use ruff for linting
State: closed

`flake8` is one of the slow parts of our linting (which can become especially annoying when using it as a pre-commit check), and https://github.com/charliermarsh/ruff/ is a faster replacement, and is starting to picked up by several other projects.


This PR only replaces flake8 and flake8-bugbear on code (not docs) and yesqa with ruff. Later on, we could also consider replacing other parts of our linting with ruff (eg isort, pylint, pyupgrade, or some of our custom checks)


Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
Currently, the errors that I get are:

```
$ ruff pandas
Found 25 error(s).
pandas/core/arrays/casting.py:100:1: E501 Line too long (101 > 88 characters)
pandas/core/arrays/casting.py:111:1: E501 Line too long (99 > 88 characters)
pandas/core/arrays/casting.py:144:70: W292 No newline at end of file
pandas/core/computation/pytables.py:247:25: E713 Test for membership should be `not in`
pandas/core/generic.py:10825:52: B026 Star-arg unpacking after a keyword argument is strongly discouraged. It only works when the keyword parameter is declared after all parameters supplied by the unpacked sequence, and this change of ordering can surprise and mislead readers.
pandas/io/formats/latex.py:57:1: B024 `RowStringConverter` is an abstract base class, but it has no abstract methods
pandas/io/formats/style.py:2250:24: E714 Test for object identity should be `is not`
pandas/io/parsers/python_parser.py:1001:21: E741 Ambiguous variable name: `l`
pandas/plotting/_matplotlib/core.py:458:5: B027 `MPLPlot` is an empty method in an abstract base class, but has no abstract decorator
pandas/plotting/_matplotlib/core.py:667:5: B027 `MPLPlot` is an empty method in an abstract base class, but has no abstract decorator
pandas/plotting/_matplotlib/core.py:1858:20: E741 Ambiguous variable name: `l`
pandas/tests/arithmetic/test_datetime64.py:208:9: E741 Ambiguous variable name: `l`
pandas/tests/arithmetic/test_datetime64.py:211:13: E741 Ambiguous variable name: `l`
pandas/tests/frame/test_constructors.py:2481:22: E713 Test for membership should be `not in`
pandas/tests/indexes/common.py:35:5: F401 `pandas.core.api.UInt64Index` imported but unused
pandas/tests/indexes/period/test_indexing.py:781:20: E713 Test for membership should be `not in`
pandas/tests/indexes/period/test_indexing.py:782:20: E713 Test for membership should be `not in`
pandas/tests/io/formats/test_format.py:143:13: E741 Ambiguous variable name: `l`
pandas/tests/io/test_html.py:346:59: B026 Star-arg unpacking after a keyword argument is strongly discouraged. It only works when the keyword parameter is declared after all parameters supplied by the unpacked sequence, and this change of ordering can surprise and mislead readers.
pandas/tests/plotting/test_datetimelike.py:1024:16: E741 Ambiguous variable name: `l`
pandas/tests/plotting/test_datetimelike.py:1048:16: E741 Ambiguous variable name: `l`
pandas/tests/plotting/test_datetimelike.py:1065:16: E741 Ambiguous variable name: `l`
pandas/tests/plotting/test_datetimelike.py:1089:16: E741 Ambiguous variable name: `l`
pandas/tests/test_problem.py:8:5: F841 Local variable `res` is assigned to but never used
pandas/util/_doctools.py:80:21: E741 Ambiguous variable name: `l`
```

Are that all things we are OK with to fix, or do we want to ignore those rules instead?

(it seems we are currenlty not ignoring E741 (ambiguous short variable name) and E713 (Test for membership should be `not in`), but so ruff seems to find a few more of those compared to flake8?)
I think it'd be good to fix them - is it OK if I add a commit to do that? I'd be pretty keen to get this in and remove flake8, it'd be a nice improvement to dev workflow

---

EDIT: timings from running on all files in CI:
```
title-capitalization              0.06
pandas-errors-documented          0.06
validate-min-versions-in-sync     0.07
end-of-file-fixer                 0.09
pip-to-conda                      0.09
pg8000-not-installed-CI           0.12
invalid-ea-testing                0.13
seed-check-asv                    0.13
future-annotations                0.13
no-bool-in-core-generic           0.14
cython-casting                    0.17
no-return-exception               0.21
incorrect-backticks               0.24
np-testing-array-equal            0.27
rst-backticks                     0.27
trailing-whitespace               0.33
double-quote-cython-strings       0.59
rst-directive-colons              0.63
use-pd_array-in-core              0.82
validate-errors-locations         0.85
rst-inline-touching-normal        0.87
flake8-pyi                        0.90
unwanted-patterns                 1.05
use-io-common-urlopen             1.37
cpplint                           1.67
codespell                         1.70
ruff                              3.35
flake8-rst                        3.57
check-test-naming                 3.96
absolufy-imports                  5.13
debug-statements                  5.29
isort                             6.59
cython-lint                       6.98
vulture                          10.31
sphinx-lint                      14.08
pyupgrade                        21.74
black                            59.38
```
3.35 seconds to lint the entire codebase...incredible 😱 
> (though I added a commit to fixup the remaining issues, so probably best if someone else takes a look too)

All code changes look fine to me.
> > (though I added a commit to fixup the remaining issues, so probably best if someone else takes a look too)
> 
> All code changes look fine to me.

Cool - good to merge, or shall we wait for a 3rd opinion?
+1 to move to a faster style checker. Probably comfortable merging https://github.com/pandas-dev/pandas/pull/50519 first so we don't lose the pandas style checker checks in the meantime
good points @twoertwein , thanks - we OK to get this in now?

There's a few more checks that can be enabled (e.g. `TID252` allows to get rid of `absolufy-imports`), but perhaps let's keep that to a follow-up?

Regarding removing linters from the environment file - yeah I'd be totally in favour of that, let's just touch base on this in the call and then do it
> Regarding removing linters from the environment file - yeah I'd be totally in favour of that, let's just touch base on this in the call and then do it

Just got:

```
  error    libmamba Could not solve for environment specs
      Encountered problems while solving:
        - nothing provides requested ruff 0.0.217**
      
      The environment can't be solved, aborting the operation
      
  critical libmamba Could not solve for environment specs
  Error: Failed to execute ["bash","-c","micromamba create -n test --strict-channel-priority -y --log-level warning -f /home/runner/work/pandas/pandas/environment.yml"]
```

Let's just spare ourselves with extra headaches and just not include it in the environment.yml file?
conda-forge is at 0.0.215 (https://anaconda.org/conda-forge/ruff), so you can use that version
> Regarding removing linters from the environment file - yeah I'd be totally in favour of that, let's just touch base on this in the call and then do it

If you want to discuss that, can you open an issue about it? (that can be done in advance of the call)
sure, have reverted to 0.0.215

there's https://github.com/pandas-dev/pandas/issues/46561 from sometime ago, just never got around to discussing it in calls, perhaps worth revisiting if there's time
Let's get this in - thanks @jorisvandenbossche !
And thanks you for much of the follow-up work!
This is so cool! Thank you for trying Ruff. I had no idea this was happening. You all should've been bugging me way more for fixes :)

(And sorry again for the PR spam. Just never thought I'd see Ruff in a project like Pandas :))
> Just never thought I'd see Ruff in a project like Pandas :)

And thanks a lot to _you_ for Ruff, it's a great project making the python linting experience so much better!