CLN: ASV Gil benchmark
State: closed

- Adding missing `setup` import to `attrs_caching.py`

- Now lint benchmark files that start with g and h

- Utilized `params` for `gil.py` benchmarks, flake8 and remove start imports

```
asv dev -b ^gil
· Discovering benchmarks
· Running 15 total benchmarks (1 commits * 1 environments * 15 benchmarks)
[  0.00%] ·· Building for existing-py_home_matt_anaconda_envs_pandas_dev_bin_python
[  0.00%] ·· Benchmarking existing-py_home_matt_anaconda_envs_pandas_dev_bin_python
[  6.67%] ··· Running gil.ParallelDatetimeFields.time_datetime_field_day                                                   211ms
[ 13.33%] ··· Running gil.ParallelDatetimeFields.time_datetime_field_daysinmonth                                           224ms
[ 20.00%] ··· Running gil.ParallelDatetimeFields.time_datetime_field_normalize                                             324ms
[ 26.67%] ··· Running gil.ParallelDatetimeFields.time_datetime_field_year                                                  201ms
[ 33.33%] ··· Running gil.ParallelDatetimeFields.time_datetime_to_period                                                   258ms
[ 40.00%] ··· Running gil.ParallelDatetimeFields.time_period_to_datetime                                                   325ms
[ 46.67%] ··· Running gil.ParallelFactorize.time_loop                                                                         ok
[ 46.67%] ···· 
               ========= ========
                threads          
               --------- --------
                   2      76.9ms 
                   4      156ms  
                   8      314ms  
               ========= ========

[ 53.33%] ··· Running gil.ParallelFactorize.time_parallel                                                                     ok
[ 53.33%] ···· 
               ========= ========
                threads          
               --------- --------
                   2      95.9ms 
                   4      209ms  
                   8      474ms  
               ========= ========

[ 60.00%] ··· Running gil.ParallelGroupbyMethods.time_loop                                                                    ok
[ 60.00%] ···· 
               ========= ======== ======== ======== ======== ======== ======== ======== =======
               --                                        method                                
               --------- ----------------------------------------------------------------------
                threads   count     last     max      mean     min      prod     sum      var  
               ========= ======== ======== ======== ======== ======== ======== ======== =======
                   2      92.6ms   85.2ms   79.0ms   82.7ms   82.3ms   82.0ms   82.2ms   106ms 
                   4      187ms    163ms    158ms    164ms    156ms    165ms    160ms    207ms 
                   8      452ms    406ms    396ms    410ms    398ms    405ms    401ms    493ms 
               ========= ======== ======== ======== ======== ======== ======== ======== =======

[ 66.67%] ··· Running gil.ParallelGroupbyMethods.time_parallel                                                                ok
[ 66.67%] ···· 
               ========= ======= ======= ======= ======== ======= ======= ======= =======
               --                                     method                             
               --------- ----------------------------------------------------------------
                threads   count    last    max     mean     min     prod    sum     var  
               ========= ======= ======= ======= ======== ======= ======= ======= =======
                   2      150ms   104ms   105ms   83.9ms   104ms   103ms   103ms   111ms 
                   4      305ms   235ms   227ms   227ms    227ms   231ms   219ms   271ms 
                   8      740ms   466ms   481ms   504ms    493ms   515ms   484ms   709ms 
               ========= ======= ======= ======= ======== ======= ======= ======= =======

[ 73.33%] ··· Running gil.ParallelGroups.time_get_groups                                                                      ok
[ 73.33%] ···· 
               ========= =======
                threads         
               --------- -------
                   2      1.41s 
                   4      2.94s 
                   8      5.76s 
               ========= =======

[ 80.00%] ··· Running gil.ParallelKth.time_kth_smallest                                                                    291ms
[ 86.67%] ··· Running gil.ParallelReadCSV.time_read_csv                                                                       ok
[ 86.67%] ···· 
               ========== ========
                 dtype            
               ---------- --------
                 float     554ms  
                 object    24.4ms 
                datetime   561ms  
               ========== ========

[ 93.33%] ··· Running gil.ParallelRolling.time_rolling                                                                        ok
[ 93.33%] ···· 
               ================ ========
                    method              
               ---------------- --------
                rolling_median   239ms  
                 rolling_mean    24.7ms 
                 rolling_min     27.3ms 
                 rolling_max     29.1ms 
                 rolling_var     23.7ms 
                 rolling_skew    31.3ms 
                 rolling_kurt    29.7ms 
                 rolling_std     30.0ms 
               ================ ========

[ 93.33%] ····· 
                
                For parameters: 'rolling_median'
                /home/matt/Projects/pandas-mroeschke/asv_bench/benchmarks/gil.py:194: FutureWarning: pd.rolling_median is deprecated for ndarrays and will be removed in a future version
                  rolling[method](arr, win)
                
                For parameters: 'rolling_mean'
                /home/matt/Projects/pandas-mroeschke/asv_bench/benchmarks/gil.py:194: FutureWarning: pd.rolling_mean is deprecated for ndarrays and will be removed in a future version
                  rolling[method](arr, win)
                
                For parameters: 'rolling_min'
                /home/matt/Projects/pandas-mroeschke/asv_bench/benchmarks/gil.py:194: FutureWarning: pd.rolling_min is deprecated for ndarrays and will be removed in a future version
                  rolling[method](arr, win)
                
                For parameters: 'rolling_max'
                /home/matt/Projects/pandas-mroeschke/asv_bench/benchmarks/gil.py:194: FutureWarning: pd.rolling_max is deprecated for ndarrays and will be removed in a future version
                  rolling[method](arr, win)
                
                For parameters: 'rolling_var'
                /home/matt/Projects/pandas-mroeschke/asv_bench/benchmarks/gil.py:194: FutureWarning: pd.rolling_var is deprecated for ndarrays and will be removed in a future version
                  rolling[method](arr, win)
                
                For parameters: 'rolling_skew'
                /home/matt/Projects/pandas-mroeschke/asv_bench/benchmarks/gil.py:194: FutureWarning: pd.rolling_skew is deprecated for ndarrays and will be removed in a future version
                  rolling[method](arr, win)
                
                For parameters: 'rolling_kurt'
                /home/matt/Projects/pandas-mroeschke/asv_bench/benchmarks/gil.py:194: FutureWarning: pd.rolling_kurt is deprecated for ndarrays and will be removed in a future version
                  rolling[method](arr, win)
                
                For parameters: 'rolling_std'
                /home/matt/Projects/pandas-mroeschke/asv_bench/benchmarks/gil.py:194: FutureWarning: pd.rolling_std is deprecated for ndarrays and will be removed in a future version
                  rolling[method](arr, win)

[100.00%] ··· Running gil.ParallelTake1D.time_take1d                                                                          ok
[100.00%] ···· 
               ========= ========
                 dtype           
               --------- --------
                 int64    24.3ms 
                float64   8.17ms 
               ========= ========
```


Labels:
name: Benchmark
description: Performance (ASV) benchmarks

Comments:
Is it a bit strange that you see such a slowdown with multiple threads ?
# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=h1) Report
> Merging [#18675](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/ba3a442556e3c04e316f790335427e6d983db01d?src=pr&el=desc) will **decrease** coverage by `0.02%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18675/graphs/tree.svg?token=eZ4WkYLtcO&src=pr&height=150&width=650)](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18675      +/-   ##
==========================================
- Coverage   91.59%   91.57%   -0.03%     
==========================================
  Files         153      153              
  Lines       51263    51212      -51     
==========================================
- Hits        46956    46899      -57     
- Misses       4307     4313       +6
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.43% <ø> (-0.01%)` | :arrow_down: |
| #single | `40.67% <ø> (-0.13%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/tseries/converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvY29udmVydGVyLnB5) | `0% <0%> (-100%)` | :arrow_down: |
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/plotting/\_converter.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb252ZXJ0ZXIucHk=) | `65.25% <0%> (-1.27%)` | :arrow_down: |
| [pandas/core/config\_init.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvY29uZmlnX2luaXQucHk=) | `98.34% <0%> (-0.12%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.81% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/window.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvd2luZG93LnB5) | `96.31% <0%> (ø)` | :arrow_up: |
| [pandas/tseries/frequencies.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL3RzZXJpZXMvZnJlcXVlbmNpZXMucHk=) | `94.09% <0%> (+0.07%)` | :arrow_up: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `82.49% <0%> (+0.12%)` | :arrow_up: |
| [pandas/util/\_test\_decorators.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvX3Rlc3RfZGVjb3JhdG9ycy5weQ==) | `95.23% <0%> (+1.9%)` | :arrow_up: |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=footer). Last update [ba3a442...a83286d](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

# [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=h1) Report
> Merging [#18675](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=desc) into [master](https://codecov.io/gh/pandas-dev/pandas/commit/34a8d36e45c1623d2e61671561cdbd5de36adac8?src=pr&el=desc) will **decrease** coverage by `<.01%`.
> The diff coverage is `n/a`.

[![Impacted file tree graph](https://codecov.io/gh/pandas-dev/pandas/pull/18675/graphs/tree.svg?width=650&height=150&src=pr&token=eZ4WkYLtcO)](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=tree)

```diff
@@            Coverage Diff             @@
##           master   #18675      +/-   ##
==========================================
- Coverage    91.6%    91.6%   -0.01%     
==========================================
  Files         153      153              
  Lines       51273    51306      +33     
==========================================
+ Hits        46970    46999      +29     
- Misses       4303     4307       +4
```

| Flag | Coverage Δ | |
|---|---|---|
| #multiple | `89.46% <ø> (+0.01%)` | :arrow_up: |
| #single | `40.72% <ø> (-0.07%)` | :arrow_down: |

| [Impacted Files](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=tree) | Coverage Δ | |
|---|---|---|
| [pandas/io/gbq.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL2dicS5weQ==) | `25% <0%> (-58.34%)` | :arrow_down: |
| [pandas/io/parquet.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3BhcnF1ZXQucHk=) | `64.55% <0%> (-0.83%)` | :arrow_down: |
| [pandas/core/reshape/merge.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvcmVzaGFwZS9tZXJnZS5weQ==) | `94.2% <0%> (-0.21%)` | :arrow_down: |
| [pandas/core/frame.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZnJhbWUucHk=) | `97.81% <0%> (-0.1%)` | :arrow_down: |
| [pandas/core/indexes/datetimes.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9kYXRldGltZXMucHk=) | `95.68% <0%> (ø)` | :arrow_up: |
| [pandas/core/generic.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvZ2VuZXJpYy5weQ==) | `95.9% <0%> (ø)` | :arrow_up: |
| [pandas/io/pytables.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2lvL3B5dGFibGVzLnB5) | `92.84% <0%> (ø)` | :arrow_up: |
| [pandas/plotting/\_core.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL3Bsb3R0aW5nL19jb3JlLnB5) | `82.41% <0%> (+0.03%)` | :arrow_up: |
| [pandas/core/indexes/numeric.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL2NvcmUvaW5kZXhlcy9udW1lcmljLnB5) | `97.33% <0%> (+0.07%)` | :arrow_up: |
| [pandas/util/testing.py](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree#diff-cGFuZGFzL3V0aWwvdGVzdGluZy5weQ==) | `82.34% <0%> (+0.32%)` | :arrow_up: |
| ... and [4 more](https://codecov.io/gh/pandas-dev/pandas/pull/18675/diff?src=pr&el=tree-more) | |

------

[Continue to review full report at Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=continue).
> **Legend** - [Click here to learn more](https://docs.codecov.io/docs/codecov-delta)
> `Δ = absolute <relative> (impact)`, `ø = not affected`, `? = missing data`
> Powered by [Codecov](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=footer). Last update [34a8d36...0c4f3e7](https://codecov.io/gh/pandas-dev/pandas/pull/18675?src=pr&el=lastupdated). Read the [comment docs](https://docs.codecov.io/docs/pull-request-comments).

@jorisvandenbossche This benchmark makes heavy use of the `test_parallel` decorator which creates threads using the `threading` module, but I think all these operations are CPU bound so threading shouldn't help here.

Here's an example analysis of that point: https://medium.com/practo-engineering/threading-vs-multiprocessing-in-python-7b57f224eadb
> I think all these operations are CPU bound so threading shouldn't help here.

As far as I understand, threads don't help for that due to the GIL, but here we are explicitly benchmarking the GIL-freeing of those groupby et al methods.

Trying out the example interactively also does give a speedup, so I was mainly wondering why we don't see this in the asv benchmarks:

```
N = 1000000
ngroups = 1000
np.random.seed(1234)
df = DataFrame({'key' : np.random.randint(0,ngroups,size=N),
             'data' : np.random.randn(N) })
            
def f():
    df.groupby('key')['data'].sum()

def g4():
    for i in range(4):
        f()

from pandas.util.testing import test_parallel

@test_parallel(num_threads=4)
def pg4():
    f()
```

```
In [14]: %timeit g4()
74.5 ms ± 1e+03 µs per loop (mean ± std. dev. of 7 runs, 10 loops each)

In [15]: %timeit pg4()
39.4 ms ± 2.1 ms per loop (mean ± std. dev. of 7 runs, 10 loops each)
```
Okay thanks for the explaination. The `gil.py` benchmarks on master are also showing that threads are slower so maybe it's an artifact of asv?

(`time_sum_4_notp` is faster than `time_sum`)
```
asv run -b ^gil.NoGilGroupby· Creating environments
· Discovering benchmarks
·· Uninstalling from conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt.
·· Installing into conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt...
· Running 15 total benchmarks (1 commits * 1 environments * 15 benchmarks)
[  0.00%] · For pandas commit hash 34a8d36e:
[  0.00%] ·· Building for conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt....
[  0.00%] ·· Benchmarking conda-py3.6-Cython-matplotlib-numexpr-numpy-openpyxl-pytables-pytest-scipy-sqlalchemy-xlrd-xlsxwriter-xlwt
[  6.67%] ··· Running gil.NoGilGroupby.time_count_2                                            161±3ms
[ 13.33%] ··· Running gil.NoGilGroupby.time_groups_2                                        1.32±0.02s
[ 20.00%] ··· Running gil.NoGilGroupby.time_groups_4                                             2.70s
[ 26.67%] ··· Running gil.NoGilGroupby.time_groups_8                                             5.28s
[ 33.33%] ··· Running gil.NoGilGroupby.time_last_2                                           144±0.4ms
[ 40.00%] ··· Running gil.NoGilGroupby.time_max_2                                              141±2ms
[ 46.67%] ··· Running gil.NoGilGroupby.time_mean_2                                           130±0.6ms
[ 53.33%] ··· Running gil.NoGilGroupby.time_min_2                                            141±0.6ms
[ 60.00%] ··· Running gil.NoGilGroupby.time_prod_2                                             132±1ms
[ 66.67%] ··· Running gil.NoGilGroupby.time_sum_2                                            141±0.7ms
[ 73.33%] ··· Running gil.NoGilGroupby.time_sum_4                                              296±3ms
[ 80.00%] ··· Running gil.NoGilGroupby.time_sum_4_notp                                        190±20ms
[ 86.67%] ··· Running gil.NoGilGroupby.time_sum_8                                            554±0.8ms
[ 93.33%] ··· Running gil.NoGilGroupby.time_sum_8_notp                                        296±40ms
[100.00%] ··· Running gil.NoGilGroupby.time_var_2                                              158±2ms
```
There is still one linting issue:

```
Linting asv_bench/benchmarks/

asv_bench/benchmarks/groupby.py:279:9: E731 do not assign a lambda expression, use a def

Linting asv_bench/benchmarks/*.py DONE
```
Thanks!
For a follow-up, can you 'fix' those usage of `rolling_` functions? (eg have the dict be dependent on whether `pd.DataFrame.rolling` exists and then fall back to rolling_.. etc. (can be in https://github.com/pandas-dev/pandas/pull/18725) 
See https://github.com/pandas-dev/pandas/pull/18723, then that PR does not need to worry about the benchmarks.
> See #18723, then that PR does not need to worry about the benchmarks.

So I'll roll back my changes in the ``gil.py`` file, unless otherwise notified.
> So I'll roll back my changes in the gil.py file, unless otherwise notified.

Yes, that is maybe the easiest, then you don't need to worry about the merge conflicts