PERF: pandas read_excel perf optimisations
State: closed

#### WHAT
Attempts on perf optimisations for pandas read_excel
by descending the skiprows (as an integer) to get_sheet_data depending on the engine
openpyxl, xlrd, pyxlsb or odf...

STATUS : Working in progress...

- [ ] closes #47290 (Replace xxxx with the Github issue number)
- [ ] [Tests added and passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#writing-tests) if fixing a bug or adding a new feature
- [ ] All [code checks passed](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#pre-commit).
- [ ] Added [type annotations](https://pandas.pydata.org/pandas-docs/dev/development/contributing_codebase.html#type-hints) to new arguments/methods/functions.
- [ ] Added an entry in the latest `doc/source/whatsnew/vX.X.X.rst` file if fixing a bug or adding a new feature.

#### BENCHMARKS

##### BENCH SPECS

OS: Ubuntu 20.04.4 LTS x86_64 
Host: 20TH0010FR ThinkPad P1 Gen 3 
Kernel: 5.14.0-1038-oem 
CPU: Intel i7-10750H (12) @ 5.000GHz 
GPU: NVIDIA Quadro T1000 Mobile 
GPU: Intel UHD Graphics 
Memory: 47849MiB
Python 3.8.6 - [GCC 9.4.0]

##### BENCH SCRIPT
```python
import pandas as pd
from timeit import default_timer

def bench_mark_func():
    print(f">>> {pd.__version__}")
    for ext in ["xls", "xlsx", "xlsb", "ods"]:
        print(f"\n[{ext}] no nrows, nor skiprows :")
        start = default_timer()
        for i in range(100):
            pd.read_excel(f"./fixtures/benchmark_5000.{ext}")
        print(f"[{ext}] done in {default_timer() - start}")
        print("*" * 30)

        print(f"\n[{ext}] with nrows and skiprows (reading top lines):")
        start = default_timer()
        for i in range(100):
            pd.read_excel(
                f"./fixtures/benchmark_5000.{ext}", nrows=50 * i, skiprows=100 + i
            )
        print(f"[{ext}] done in {default_timer() - start}")
        print("*" * 30)

        print(f"\n[{ext}] with nrows and skiprows (reading middle lines):")
        start = default_timer()
        for i in range(100):
            pd.read_excel(
                f"./fixtures/benchmark_5000.{ext}", nrows=50 * i, skiprows=2000 + i
            )
        print(f"[{ext}] done in {default_timer() - start}")
        print("*" * 30)

        print(f"\n[{ext}] with nrows and skiprows (reading bottom lines):")
        start = default_timer()
        for i in range(100):
            pd.read_excel(
                f"./fixtures/benchmark_5000.{ext}", nrows=50 * i, skiprows=4000 + i
            )
        print(f"[{ext}] done in {default_timer() - start}")
        print("*" * 30)

        print("==" * 30)


if __name__ == "__main__":
    bench_mark_func()
```

Fixtures are available here : [fixtures](https://github.com/Sanix-Darker/pandas/tree/toucan-pandas-excel-preview/fixtures)

##### BENCH REPORTS

##### xls format - - -

|                                             | main branch | perf branch | diff (second)
| ---------------                                |------------        |-------------      |--------------
| no nrows, nor skiprows                         |9.355613674968481   |9.48072951193899   |__-0.1251158369705081__
| nrows and skiprows (top lines)    |8.50429745996371    |8.566174849052913  |__-0.06187738908920437__
| nrows and skiprows (middle lines) |9.003354059066623   |9.093110702931881  |__-0.0897566438652575__
| nrows and skiprows (bottom lines) |9.281007815967314   |9.35020213900134   |__-0.06919432303402573__
|                                           |                                    | __AVERAGE__                                 |__-0.08648604823974892__

##### xlsx format + + +

|                                             | main branch | perf branch | diff (second)
| ---------------                                |------------        |-------------      |--------------
| no nrows, nor skiprows                         |47.444979660911486  |47.119721286930144 |__+0.3252583739813417__
| nrows and skiprows (top lines)    |25.616206042002887  |25.10350460803602  |__+0.5127014339668676__
| nrows and skiprows (middle lines) |39.18539171805605   |38.66645851393696  |__+0.5189332041190937__
| nrows and skiprows (bottom lines) |46.56151840998791   |45.87342134909704  |__+0.6880970608908683__
|                                           |                                    | __AVERAGE__                                 |__+0.5112475182395428__

##### xlsb format - - -

|                                             | main branch | perf branch | diff (second)
| ---------------                                |------------        |-------------      |--------------
| no nrows, nor skiprows                         |9.378919824026525   |9.404310946003534  |__-0.02539112197700888__
| nrows and skiprows (top lines)    |8.567789324908517   |8.607287417980842  |__-0.03949809307232499__
| nrows and skiprows (middle lines) |9.089394484995864   |9.141500656027347  |__-0.05210617103148252__
| nrows and skiprows (bottom lines) |9.342884571058676   |9.409172061015852  |__-0.06628748995717615__
|                                           |                                    | __AVERAGE__                                 |__-0.045820719009498134__

##### ods format - - -

|                                             | main branch | perf branch | diff (second)
| ---------------                                |------------        |-------------      |--------------
| no nrows, nor skiprows                         |9.38043470599223    |9.435379554051906  |__-0.05494484805967659__
| nrows and skiprows (top lines)    |8.553725612931885   |8.641130893025547  |__-0.08740528009366244__
| nrows and skiprows (middle lines) |8.839795237989165   |8.97894280392211   |__-0.13914756593294442__
| nrows and skiprows (bottom lines) |9.031556493951939   |9.168779775965959  |__-0.1372232820140198__
|                                           |                                    | __AVERAGE__                                 |__-0.10468024402507581__

#### NOTE

Where openpyxl is beeing optimized, other readers's engine are not... but with 'realy low differences'.


Labels:
name: Performance
description: Memory or execution speed performance
name: IO Excel
description: read_excel, to_excel
name: Stale

Comments:
Hi, thanks for trying this. I am not sure, if I understand your benchmarks correctly, but this looks like noise to me? You can convert your pr to draft, if you are not ready yet
This pull request is stale because it has been open for thirty days with no activity. Please [update](https://pandas.pydata.org/pandas-docs/stable/development/contributing.html#updating-your-pull-request) and respond to this comment if you're still interested in working on this.
Thanks for the pull request, but it appears to have gone stale. If interested in continuing, please merge in the main branch, address any review comments and/or failing tests, and we can reopen.