KeyError: 'general.name'
Author: nitinmukesh
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.51.3
- Platform: Windows-10-10.0.26100-SP0
- Python version: 3.11.9
- Huggingface_hub version: 0.30.1
- Safetensors version: 0.5.3
- Accelerate version: 1.7.0.dev0
- Accelerate config:    not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0+cu124 (True)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA GeForce RTX 4060 Laptop GPU

### Who can help?

@SunMarc 
@MekkCyber 

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
import torch
from transformers import PreTrainedTokenizerFast, LlamaForCausalLM, CLIPTextModelWithProjection, T5EncoderModel
from diffusers import UniPCMultistepScheduler, HiDreamImagePipeline
from diffusers import AutoencoderKL, HiDreamImageTransformer2DModel
from diffusers import GGUFQuantizationConfig

scheduler = UniPCMultistepScheduler(
    flow_shift=3.0,
    prediction_type="flow_prediction",
    use_flow_sigmas=True,
)
text_encoder_2 = CLIPTextModelWithProjection.from_pretrained("calcuis/hidream-gguf", gguf_file="clip_g_hidream_fp32-f16.gguf")
text_encoder_3 = T5EncoderModel.from_pretrained("calcuis/hidream-gguf", gguf_file="t5xxl_fp32-q4_0.gguf")
tokenizer_4 = PreTrainedTokenizerFast.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
text_encoder_4 = LlamaForCausalLM.from_pretrained("calcuis/hidream-gguf", gguf_file="llama-q2_k.gguf")

```

### Expected behavior

CLIPTextModelWithProjection and T5EncoderModel not working and throws same error.
LlamaForCausalLM is working, 

```
(sddw-dev) C:\aiOWN\diffuser_webui>python HiDream_GGUF.py
import error: No module named 'triton'
Traceback (most recent call last):
  File "C:\aiOWN\diffuser_webui\HiDream_GGUF.py", line 12, in <module>
    text_encoder_2 = CLIPTextModelWithProjection.from_pretrained("calcuis/hidream-gguf", gguf_file="clip_g_hidream_fp32-f16.gguf")
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\nitin\miniconda3\envs\sddw-dev\Lib\site-packages\transformers\modeling_utils.py", line 279, in _wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\nitin\miniconda3\envs\sddw-dev\Lib\site-packages\transformers\modeling_utils.py", line 4175, in from_pretrained
    config, model_kwargs = cls.config_class.from_pretrained(
                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\nitin\miniconda3\envs\sddw-dev\Lib\site-packages\transformers\configuration_utils.py", line 550, in from_pretrained
    config_dict, kwargs = cls.get_config_dict(pretrained_model_name_or_path, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\nitin\miniconda3\envs\sddw-dev\Lib\site-packages\transformers\configuration_utils.py", line 590, in get_config_dict
    config_dict, kwargs = cls._get_config_dict(pretrained_model_name_or_path, **kwargs)
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\nitin\miniconda3\envs\sddw-dev\Lib\site-packages\transformers\configuration_utils.py", line 681, in _get_config_dict
    config_dict = load_gguf_checkpoint(resolved_config_file, return_tensors=False)["config"]
                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\nitin\miniconda3\envs\sddw-dev\Lib\site-packages\transformers\modeling_gguf_pytorch_utils.py", line 369, in load_gguf_checkpoint
    model_name = read_field(reader, "general.name")
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\nitin\miniconda3\envs\sddw-dev\Lib\site-packages\transformers\modeling_gguf_pytorch_utils.py", line 260, in read_field
    value = reader.fields[field]
            ~~~~~~~~~~~~~^^^^^^^
KeyError: 'general.name'

```

Comments:
hey @nitinmukesh , could you explain to us what is your goal with this snippet ? Diffusers have a gguf integration but it relies on the single file format, so it is not meant to be loaded like that if your goal is to load a diffusers model 

cc @Isotr0py for clip and t5

Hello @SunMarc 

Sorry I thought I should only post the code to replicate the issue, should have provided the background.

I am trying to do inference using HiDream model. Since the model is huge, I am trying to use GGUF for text encoders available here.
https://huggingface.co/calcuis/hidream-gguf/tree/main

```python
import torch
from transformers import PreTrainedTokenizerFast, LlamaForCausalLM, CLIPTextModelWithProjection, T5EncoderModel
from diffusers import UniPCMultistepScheduler, HiDreamImagePipeline
from diffusers import AutoencoderKL, HiDreamImageTransformer2DModel
from diffusers import GGUFQuantizationConfig

scheduler = UniPCMultistepScheduler(
    flow_shift=3.0,
    prediction_type="flow_prediction",
    use_flow_sigmas=True,
)
text_encoder_2 = CLIPTextModelWithProjection.from_pretrained("calcuis/hidream-gguf", gguf_file="clip_g_hidream_fp32-f16.gguf")
text_encoder_3 = T5EncoderModel.from_pretrained("calcuis/hidream-gguf", gguf_file="t5xxl_fp32-q4_0.gguf")
tokenizer_4 = PreTrainedTokenizerFast.from_pretrained("meta-llama/Meta-Llama-3.1-8B-Instruct")
text_encoder_4 = LlamaForCausalLM.from_pretrained("calcuis/hidream-gguf", gguf_file="llama-q2_k.gguf")

base_repo = "HiDream-ai/HiDream-I1-Full"
pipe = HiDreamImagePipeline.from_pretrained(
    base_repo,
    scheduler=scheduler,
    text_encoder_3=text_encoder_3,
    tokenizer_4=tokenizer_4,
    text_encoder_4=text_encoder_4,
    transformer=None,
    vae=None,
    torch_dtype=torch.bfloat16,
).to('cuda')

# pipe.enable_sequential_cpu_offload()
# pipe.enable_model_cpu_offload()

print("Generating prompt embeddings...")
prompt_embeds, negative_prompt_embeds, pooled_prompt_embeds, negative_pooled_prompt_embeds = pipe.encode_prompt(
    prompt="A cat holding a sign that says \"Hi-Dreams.ai\".",
    prompt_2=None,
    prompt_3=None,
    prompt_4=None,
)
print("Successfully generated prompt embeddings!")
print(prompt_embeds, negative_prompt_embeds, pooled_prompt_embeds, negative_pooled_prompt_embeds)

```

Regardless of the `gguf-py` package support, these gguf checkpoints don't have enough metadata to reconstruct a transformers config (at least `clip_g_hidream_fp32-f16.gguf` and `t5xxl_fp32-q4_0.gguf`, they only have architecture name and tensor count) . 

And this is also why this `KeyError` will be encountered...

cc @DN6 for the gguf support in diffusers 

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.