Get our efficiency back
Author: LysandreJik
Pull request: yes

jedi is unfortunately extremely slow at parsing the imports such as `from .models import *`; in order to have it be faster, we have no choice but to put the individual objects back in the main init.

These are only necessary for intellisense/static type checkers -> it doesn't impact actual importable objects.

These are maintained with a new `python utils/check_main_init.py` that runs when doing `make fixup` and that updates the main init accordingly.

Comments:
Hi 👋, thank you for opening this pull request! The pull request is converted to draft by default. The CI will be paused while the PR is in draft mode. When it is ready for review, please click the `Ready for review` button (at the bottom of the PR page). This will assign reviewers and trigger CI.

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_37884). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

# Tests of jedi speed

## TLDR

I cannot reproduce slow jedi completions locally on `main`. The `non-model-inits branch` is noticeably slower, but still barely acceptable. The `get-our-efficiency-back` branch was fast when using the script (but a bit slower than main) but very slow in interactive testing. On colab, I got very bad results no matter the branch.

## Script

The script used for testing is:

```python
import time
from IPython.terminal.interactiveshell import TerminalInteractiveShell
# import jedi                     # <= uncomment for debug logs
# jedi.set_debug_function(print)  # <= uncomment for debug logs

def time_completer():
    text = "from transformers import M"
    shell = TerminalInteractiveShell()
    # shell.Completer.use_jedi = False  # uncomment this to disable jedi and make completions fast
    start = time.perf_counter()
    _ = shell.complete(text, text, len(text))
    elapsed = time.perf_counter() - start
    return elapsed

if __name__ == "__main__":
    elapsed = time_completer()
    print(f"Fetching IPython completions took {elapsed:.2f} s")
```

## Old env with many packages

### Script

```
$ git checkout main
$ python foo.py
Fetching IPython completions took 1.24 s
$ python foo.py
Fetching IPython completions took 1.22 s
$ git checkout origin/non-model-inits
$ python foo.py
Fetching IPython completions took 6.59 s
$ python foo.py
Fetching IPython completions took 3.23 s
$ git checkout origin/get-our-efficiency-back
$ python foo.py
Fetching IPython completions took 1.80 s
$ python foo.py
Fetching IPython completions took 1.68 s
```

### Manual testing in IPython

This means: Starting IPython, typing `from transformers import M` then hitting tab

main: ~1 s
origin/non-model-inits: ~7 s
get-our-efficiency-back: ~18 s

## New env with barebones packages

```
$ conda create -n test python=3.12
$ conda activate test
$ python -m pip install -e .  # in transformers directory
$ python -m pip install jedi ipython
$ pip list | rg "jedi|ipython"
ipython                 9.2.0
ipython_pygments_lexers 1.1.1
jedi                    0.19.2
```

### Script

```
$ git checkout main
$ python foo.py
Fetching IPython completions took 0.72 s
$ python foo.py
Fetching IPython completions took 0.50 s
$ git checkout origin/non-model-inits
$ python foo.py
Fetching IPython completions took 5.54 s
$ python foo.py
Fetching IPython completions took 1.78 s
$ git checkout origin/get-our-efficiency-back
$ python foo.py
Fetching IPython completions took 1.08 s
$ python foo.py
Fetching IPython completions took 0.98 s
```

### Manual testing

main: < 1 s
origin/non-model-inits: ~5 s
get-our-efficiency-back: ~18 s

## New env with many packages

Same as "new env with barebones packages" +

```
$ python -m pip install -e .[testing]
$ python -m pip install -e .[all]
```

### Script

```
$ git checkout main
$ python foo.py
Fetching IPython completions took 0.52 s
$ python foo.py
Fetching IPython completions took 0.51 s
$ git checkout origin/non-model-inits
$ python foo.py
Fetching IPython completions took 4.67 s
$ python foo.py
Fetching IPython completions took 1.66 s
$ git checkout origin/get-our-efficiency-back
$ python foo.py
Fetching IPython completions took 1.12 s
$ python foo.py
Fetching IPython completions took 1.05 s
```

### Manual testing

main: < 1 s
origin/non-model-inits: ~5 s
get-our-efficiency-back: ~18 s

## Colab

### Manual testing

Involves first running install transformers, then typing `from transformers import M` and hitting ctrl+space (disregard the AI autocompletions!)

Default transformers install: ~42 s

`!pip install -U git+https://github.com/huggingface/transformers.git`: timed out? no completions, had to prompt twice

`!pip install -U git+https://github.com/huggingface/transformers.git@non-model-inits`: timed out? no completions, had to prompt twice

`!pip install -U git+https://github.com/huggingface/transformers.git@get-our-efficiency-back`: ~39 s

I did one more test: On this branch, I went to `transformers/__init__.py` and commented out most `from .models import` lines, leaving only a few. Then I tried the interactive test in IPython again and this time the import was nearly instantaneous. Therefore, I think that jedi is still parsing all the model files.