Simplify and improve model loading logic
Author: Cyrilvallez
Pull request: yes

# What does this PR do?

Quite a bit of improvements in this PR to keep making things simpler:

- completely remove `offload_state_dict`: it was added several years ago to avoid loading 2x memory on cpu between the model and the state dict -> this is no longer needed from some time as the model is loaded on meta device, then params are loaded one after the other -> removes quite a bit of convoluted logic
- remove distinction between `model_to_load` and `model` in `_load_pretrained_model` -> this was there originally and is only confusing -> removes quite a bit of logic here as well
- rename `check_quantized_param() -> bool` to `param_needs_quantization() -> bool` in the quantizers -> muuuuch more self-explanatory, the old name was quite hard to understand
- removes `logger.info` messages -> if everything went normal, no need to tell me... Only tell me what went wrong
- rework the `mismatched_shapes` tests -> they were redundant and quite badly written, with a ton of unneeded exceptions and skips
- several other improvements in terms of logic/readability, as well as performances during init

Comments:
The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_41103). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

run-slow: fbgemm_fp8, finegrained_fp8, higgs, hqq, mxfp4

This comment contains run-slow, running the specified jobs: 

models: []
quantizations: ['quantization/fbgemm_fp8', 'quantization/finegrained_fp8', 'quantization/higgs', 'quantization/hqq', 'quantization/mxfp4'] ...

run-slow: fbgemm_fp8, finegrained_fp8, higgs, hqq, mxfp4, bnb

This comment contains run-slow, running the specified jobs: 

models: []
quantizations: ['quantization/bnb', 'quantization/fbgemm_fp8', 'quantization/finegrained_fp8', 'quantization/higgs', 'quantization/hqq', 'quantization/mxfp4'] ...

run-slow: fbgemm_fp8, higgs, hqq, bnb, fp_quant

This comment contains run-slow, running the specified jobs: 

models: []
quantizations: ['quantization/bnb', 'quantization/fbgemm_fp8', 'quantization/higgs', 'quantization/hqq'] ...

slow tests on quantizers are similar as on main (though main as quite a lot of failures!)

**[For maintainers]** Suggested jobs to run (before merge)

run-slow: bamba, deberta_v2, falcon_h1, gemma3, jamba, qwen3_next, timm_wrapper, zamba, zamba2, fbgemm_fp8, finegrained_fp8, higgs, hqq, mxfp4

I am getting an error when I Load Qwen2.5-VL-72B-Instruct-AWQ:

```sh
Loading checkpoint shards:   0%|                                                                                                         | 0/11 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "/home/deepspeed/workdir/shl/EvalScope/test_0918.py", line 4, in <module>
    model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
  File "/root/miniforge3/envs/shl_py310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 281, in _wrapper
    return func(*args, **kwargs)
  File "/root/miniforge3/envs/shl_py310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 4948, in from_pretrained
    model, missing_keys, unexpected_keys, mismatched_keys, offload_index, error_msgs = cls._load_pretrained_model(
  File "/root/miniforge3/envs/shl_py310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 5372, in _load_pretrained_model
    _error_msgs, disk_offload_index = load_shard_file(args)
  File "/root/miniforge3/envs/shl_py310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 883, in load_shard_file
    disk_offload_index = _load_state_dict_into_meta_model(
  File "/root/miniforge3/envs/shl_py310/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/root/miniforge3/envs/shl_py310/lib/python3.10/site-packages/transformers/modeling_utils.py", line 778, in _load_state_dict_into_meta_model
    elif not is_quantized or not hf_quantizer.param_needs_quantization(
TypeError: HfQuantizer.param_needs_quantization() takes 3 positional arguments but 5 were given
```

with these code:

```py
import torch
from transformers import Qwen2_5_VLForConditionalGeneration

model_path="/home/deepspeed/workdir/models/Qwen/Qwen2.5-VL-72B-Instruct-AWQ"
model = Qwen2_5_VLForConditionalGeneration.from_pretrained(
                model_path,
                torch_dtype=torch.float16,
                attn_implementation='flash_attention_2',
                device_map="auto"
            )
```

I'm not sure if it's an issue with this commit~

Indeed, it slipped in this PR! We're working on the quantizers before next release! Will be solved by https://github.com/huggingface/transformers/pull/41138 🤗