Start too many processes when run AutoModelForCausalLM.from_pretrained()
Author: SidneyLann
Pull request: no
Labels: name: bug

### System Info
- `transformers` version: 4.51.3
- Platform: Linux-6.12.0-58.el10.x86_64-x86_64-with-glibc2.39
- Python version: 3.12.9
- Huggingface_hub version: 0.31.2
- Safetensors version: 0.5.3
- Accelerate version: 1.5.2
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: yes
- GPU type: NVIDIA GeForce GTX 1080 Ti

- huggingface_hub version: 0.31.2
- Platform: Linux-6.12.0-58.el10.x86_64-x86_64-with-glibc2.39
- Python version: 3.12.9
- Running in iPython ?: No
- Running in notebook ?: No
- Running in Google Colab ?: No
- Running in Google Colab Enterprise ?: No
- Token path ?: /home/sidney/.cache/huggingface/token
- Has saved token ?: False
- Configured git credential helpers:
- FastAI: N/A
- Tensorflow: N/A
- Torch: 2.6.0
- Jinja2: 3.1.6
- Graphviz: N/A
- keras: N/A
- Pydot: N/A
- Pillow: 10.3.0
- hf_transfer: 0.1.8
- gradio: N/A
- tensorboard: N/A
- numpy: 1.26.4
- pydantic: 2.10.6
- aiohttp: 3.11.14
- hf_xet: N/A
- ENDPOINT: https://huggingface.co
- HF_HUB_CACHE: /home/sidney/.cache/huggingface/hub
- HF_ASSETS_CACHE: /home/sidney/.cache/huggingface/assets
- HF_TOKEN_PATH: /home/sidney/.cache/huggingface/token
- HF_STORED_TOKENS_PATH: /home/sidney/.cache/huggingface/stored_tokens
- HF_HUB_OFFLINE: False
- HF_HUB_DISABLE_TELEMETRY: False
- HF_HUB_DISABLE_PROGRESS_BARS: None
- HF_HUB_DISABLE_SYMLINKS_WARNING: False
- HF_HUB_DISABLE_EXPERIMENTAL_WARNING: False
- HF_HUB_DISABLE_IMPLICIT_TOKEN: False
- HF_HUB_ENABLE_HF_TRANSFER: False
- HF_HUB_ETAG_TIMEOUT: 10
- HF_HUB_DOWNLOAD_TIMEOUT: 10

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

![Image](https://github.com/user-attachments/assets/b42b8b43-e208-4c44-97d1-cd0225ebd446)
![Image](https://github.com/user-attachments/assets/dfcd499b-3e8f-405c-ab77-863e876e4c0b)

from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    "/d/dev/model_root/llama_31_8b",
    device_map="auto"
)

When run above code to load original llama3.1 8b model, too many processes start, this should not be normal. this causes the finetune or inferenc very slow.

### Expected behavior

should run 1 or 2 processes.

Comments:
Hi @SidneyLann, this is quite odd! I don't think I've seen it before. Do the processes disappear after loading is finished?

Hi @Rocketknight1 
  All processes close after model loaded. But they exist in all the time in training or inference and all close when training or inference is finished.

python3.10 has no this issue, but python3.10 run code below has the same issue:
from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="/d/dev/idea/model_root/llama_31_8b",
)

 Could you please help to run the code, it is simple, just load the original llama3 in local. 

The whole top command line is:

sidney 78253 65.8 1.7 9130444 573452 pts/0 Sl+ 12:28 0:02 /usr/bin/python310 /home/sidney/.local/lib/python3.10/site-packages/torch/_inductor/compile_worker/main.py --pickler=torch._inductor.compile_worker.subproc_pool.SubprocPickler --kind=fork --workers=16 --parent=78213 --read-fd=38 --write-fd=41


So what's the original source lib?  Maybe this is normal for pytorch to use multiprocess instead multithread to compile.

Maybe this is normal for pytorch to use multiprocess instead multithread to compile.