[GGUF] Refactor and decouple gguf checkpoint loading logic
Author: Isotr0py
Pull request: yes

# What does this PR do?

Since there are more and more GGUF architectures have supported, the gguf loading logic (especially the `load_gguf_checkpoint` function) become complicated and couple tensor rename and tensor "reshape"/"permute" reverse.

Therefore, I think there is a need to refactor the gguf checkpoint loading logic for better maintenance.

This PR is going to refactor and cleanup the gguf model loading logic with following methods:
- Introduce a [`get_gguf_hf_weights_map`](https://github.com/vllm-project/vllm/blob/f58454968fe1c5ddf84199b341a6ed5c99f0c0cc/vllm/model_executor/model_loader/loader.py#L1172-L1204) function that has been used in vLLM to infer `gguf_weights_map` (or `GGUF_TENSOR_MAPPING` we used in `transformers` currently) from model config with `gguf` package automatically.
- Since `GGUF_TENSOR_MAPPING` can be inferred with above function, we can try to remove most of existing hardcoded mapping. And we don't need to ask contributor to add new mapping in `GGUF_TENSOR_MAPPING` anymore at most of cases.
- ~~Separate the GGUF tensors rename and "reshape"/"permute" reverse logic.~~

Since vLLM and transformers have a different model weights loading logic, it wiil cost some time for me to figure out a better method to finish these works. So I marked this PR as a draft.

## Before submitting
- [ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
- [ ] Did you read the [contributor guideline](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md#create-a-pull-request),
      Pull Request section?
- [ ] Was this discussed/approved via a Github issue or the [forum](https://discuss.huggingface.co/)? Please add a link
      to it if that's the case.
- [ ] Did you make sure to update the documentation with your changes? Here are the
      [documentation guidelines](https://github.com/huggingface/transformers/tree/main/docs), and
      [here are tips on formatting docstrings](https://github.com/huggingface/transformers/tree/main/docs#writing-source-documentation).
- [ ] Did you write any new necessary tests?


## Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

<!-- Your PR will be replied to more quickly if you can figure out the right person to tag with @

 If you know how to use git blame, that is the easiest way, otherwise, here is a rough guide of **who to tag**.
 Please tag fewer than 3 people.

Models:

- text models: @ArthurZucker
- vision models: @amyeroberts, @qubvel
- speech models: @ylacombe, @eustlb
- graph models: @clefourrier

Library:

- flax: @sanchit-gandhi
- generate: @zucchini-nlp (visual-language models) or @gante (all others)
- pipelines: @Rocketknight1
- tensorflow: @gante and @Rocketknight1
- tokenizers: @ArthurZucker
- trainer: @muellerzr and @SunMarc
- chat templates: @Rocketknight1

Integrations:

- deepspeed: HF Trainer/Accelerate: @muellerzr
- ray/raytune: @richardliaw, @amogkam
- Big Model Inference: @SunMarc
- quantization (bitsandbytes, autogpt): @SunMarc @MekkCyber

Documentation: @stevhliu

HF projects:

- accelerate: [different repo](https://github.com/huggingface/accelerate)
- datasets: [different repo](https://github.com/huggingface/datasets)
- diffusers: [different repo](https://github.com/huggingface/diffusers)
- rust tokenizers: [different repo](https://github.com/huggingface/tokenizers)

Maintained examples (not research project or legacy):

- Flax: @sanchit-gandhi
- PyTorch: See Models above and tag the person corresponding to the modality of the example.
- TensorFlow: @Rocketknight1

 -->


Comments:
@SunMarc @MekkCyber for quantization - but let me know if someone else is handling the GGUF integration and I'll ping them instead in future!

The docs for this PR live [here](https://moon-ci-docs.huggingface.co/docs/transformers/pr_34385). All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Feel free to ping @Isotr0py and me for anything related to gguf @Rocketknight1  ! 
Let me know when this is ready to be reviewed @Isotr0py 

@Rocketknight1 I'm diving into the code of gguf integration to fix some issues so feel free to ping me as well for related issues and PRs


BTW, since the GGUF tests are very slow, I used a script to do quick tests for GGUF tensors renaming (which is modified in this PR):
```python3
import torch
from huggingface_hub import hf_hub_download

from transformers import AutoConfig, AutoModelForCausalLM
from transformers.modeling_gguf_pytorch_utils import load_gguf_checkpoint


model_id = "duyntnet/Qwen1.5-MoE-A2.7B-Chat-imatrix-GGUF"
gguf_model_id = "Qwen1.5-MoE-A2.7B-Chat-IQ1_M.gguf"

config = AutoConfig.from_pretrained(model_id, gguf_file=gguf_model_id)
with torch.device("meta"):
    model = AutoModelForCausalLM.from_config(config)
    expected_keys = set(model.state_dict().keys())

    gguf_path = hf_hub_download(model_id, filename=gguf_model_id)
    exact_keys = set(load_gguf_checkpoint(gguf_path, True, model)["tensors"].keys())

assert (
    expected_keys == exact_keys
), f"Following parameters are not initialized from GGUF file: {expected_keys - exact_keys}"

```

Looks awesome ! More readable than it used to be ! Thanks for the clean up 🔥 

I have run slow tests for all architectures we supported (except falcon-40b) locally, and most of them have passed. 

Only `Nemotron` is failling because its tokenizer failed to initialize, it also failed on the main branch with same reason. So I think it's not related to this PR, and we need to fix it in a following PR.

(Or we should re-organize the GGUF tests, because it's too messy 😅)

Yes the `nemotron` integration fails because of the tokenizer, I'm working on it