CLN: Use generators instead of lists in built-in Python functions
State: closed

- [x] tests passed
- [x] passes `git diff upstream/master -u -- "*.py" | flake8 --diff`

Pass generators in built-in Python functions instead of passing in a list, e.g `any([value for value in iterator])` --> `any(value for value in iterator)`

Labels:
name: Code Style
description: Code style, linting, code_checks

Comments:
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=h1) Report
> Merging [#18276](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/69472f9ae9c645a0a5932c7c65b6efbdf66fbb5f?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `87.5%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18276/graphs/tree.svg?src=pr&token=eZ4WkYLtcO&width=650&height=150)](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18276      +/-   ##
==========================================
- Coverage    91.4%   91.38%   -0.02%     
==========================================
  Files         164      164              
  Lines       49878    49878              
==========================================
- Hits        45590    45581       -9     
- Misses       4288     4297       +9
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.19% <85.41%> (ø)` | :arrow_up: |
| #single | `39.41% <22.91%> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `100% <ø> (ø)` | :arrow_up: |
| [pandas/util/\_doctools.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX2RvY3Rvb2xzLnB5) | `0% <0%> (ø)` | :arrow_up: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.8% <100%> (-0.1%)` | :arrow_down: |
| [pandas/io/parsers.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnNlcnMucHk=) | `95.59% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.42% <100%> (ø)` | :arrow_up: |
| [pandas/io/json/normalize.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2lvL2pzb24vbm9ybWFsaXplLnB5) | `96.93% <100%> (ø)` | :arrow_up: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `94.54% <100%> (ø)` | :arrow_up: |
| [pandas/io/pytables.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2lvL3B5dGFibGVzLnB5) | `92.84% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexing.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhpbmcucHk=) | `92.8% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexes/range.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9yYW5nZS5weQ==) | `95.66% <100%> (ø)` | :arrow_up: |
| ... and [11 more](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=footer). Last update [69472f9...6402974](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=h1) Report
> Merging [#18276](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/69472f9ae9c645a0a5932c7c65b6efbdf66fbb5f?src=pr&el=desc) will **decrease** coverage by `0.01%`.
> The diff coverage is `87.5%`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18276/graphs/tree.svg?height=150&width=650&token=eZ4WkYLtcO&src=pr)](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18276      +/-   ##
==========================================
- Coverage    91.4%   91.38%   -0.02%     
==========================================
  Files         164      164              
  Lines       49878    49878              
==========================================
- Hits        45590    45581       -9     
- Misses       4288     4297       +9
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.19% <85.41%> (ø)` | :arrow_up: |
| #single | `39.41% <22.91%> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `100% <ø> (ø)` | :arrow_up: |
| [pandas/util/\_doctools.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX2RvY3Rvb2xzLnB5) | `0% <0%> (ø)` | :arrow_up: |
| [pandas/core/internals.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW50ZXJuYWxzLnB5) | `94.54% <100%> (ø)` | :arrow_up: |
| [pandas/io/pytables.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2lvL3B5dGFibGVzLnB5) | `92.84% <100%> (ø)` | :arrow_up: |
| [pandas/core/sparse/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvc3BhcnNlL2ZyYW1lLnB5) | `94.78% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexes/base.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9iYXNlLnB5) | `96.42% <100%> (ø)` | :arrow_up: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.8% <100%> (-0.1%)` | :arrow_down: |
| [pandas/core/indexes/range.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9yYW5nZS5weQ==) | `95.66% <100%> (ø)` | :arrow_up: |
| [pandas/io/html.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2lvL2h0bWwucHk=) | `85.98% <100%> (ø)` | :arrow_up: |
| [pandas/core/indexing.py](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhpbmcucHk=) | `92.8% <100%> (ø)` | :arrow_up: |
| ... and [11 more](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=footer). Last update [69472f9...6402974](https://codecov.io/gh/pandas-dev/pandas/pull/18276?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@mroeschke thanks.

can you add a linting check in ``ci/lint.sh`` to search for cases like this?
does this have any perf impact ?
I ran one full asv and two asvs for `benchmarks/frame_methods.py` and got pretty inconsistent results. I can rerun if needed.

Sure I can include this check in `ci/lint.sh`. I'll be available to add it later in the week. 
thanks

> Sure I can include this check in ci/lint.sh. I'll be available to add it later in the week.


would be great
I don't thing you would expect to see any difference in asv. I think this is mainly about code style / redundancy. Certainly given that, apparently (just tried it out, didn't expect this), with small lists it can actually be a bit faster using a list comprehension instead of generator (for bigger ones there is a clear difference though):

```
In [168]: l = [True, False]*2

In [169]: %timeit any(i for i in l)
511 ns ± 7.38 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [170]: %timeit any([i for i in l])
448 ns ± 11.9 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [171]: l = [True, False]*100

In [172]: %timeit any(i for i in l)
514 ns ± 7.66 ns per loop (mean ± std. dev. of 7 runs, 1000000 loops each)

In [173]: %timeit any([i for i in l])
5.7 µs ± 39.5 ns per loop (mean ± std. dev. of 7 runs, 100000 loops each)
```

(and the cases in our code typically are small numbers, eg looping of the axes of a DataFrame (thus 2))