Error when loading gguf file
Author: ZJY0516
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.55.0
- Platform: Linux-6.8.0-53-generic-x86_64-with-glibc2.39
- Python version: 3.13.5
- Huggingface_hub version: 0.34.4
- Safetensors version: 0.6.2
- Accelerate version: not installed
- Accelerate config: not found
- DeepSpeed version: not installed
- PyTorch version (accelerator?): 2.8.0+cu128 (CUDA)
- Tensorflow version (GPU?): not installed (NA)
- Flax version (CPU?/GPU?/TPU?): not installed (NA)
- Jax version: not installed
- JaxLib version: not installed
- Using distributed or parallel set-up in script?: <fill in>
- Using GPU in script?: <fill in>
- GPU type: NVIDIA L40

### Who can help?

@SunMarc @MekkCyber

### Information

- [x] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

model_id = "google/gemma-3-27b-it"
filename = "/data/datasets/models-hf/gemma-3-27b-it-GGUF/gemma-3-27b-it-Q8_0.gguf"

tokenizer = AutoTokenizer.from_pretrained(model_id, gguf_file=filename)
model = AutoModelForCausalLM.from_pretrained(model_id, gguf_file=filename)
```


### Expected behavior

error log
```
Traceback (most recent call last):
  File "/home/zjy/code/vllm-src/test_transfomer.py", line 6, in <module>
    tokenizer = AutoTokenizer.from_pretrained(model_id, gguf_file=filename)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zjy/code/vllm-src/.venv/lib/python3.12/site-packages/transformers/models/auto/tokenization_auto.py", line 1116, in from_pretrained
    return tokenizer_class.from_pretrained(pretrained_model_name_or_path, *inputs, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zjy/code/vllm-src/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2069, in from_pretrained
    return cls._from_pretrained(
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/zjy/code/vllm-src/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_base.py", line 2315, in _from_pretrained
    tokenizer = cls(*init_inputs, **init_kwargs)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zjy/code/vllm-src/.venv/lib/python3.12/site-packages/transformers/models/gemma/tokenization_gemma_fast.py", line 100, in __init__
    super().__init__(
  File "/home/zjy/code/vllm-src/.venv/lib/python3.12/site-packages/transformers/tokenization_utils_fast.py", line 123, in __init__
    gguf_param = load_gguf_checkpoint(kwargs.get("vocab_file"))
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zjy/code/vllm-src/.venv/lib/python3.12/site-packages/transformers/modeling_gguf_pytorch_utils.py", line 367, in load_gguf_checkpoint
    reader = GGUFReader(gguf_checkpoint_path)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zjy/code/vllm-src/.venv/lib/python3.12/site-packages/gguf/gguf_reader.py", line 133, in __init__
    self.data = np.memmap(path, mode = mode)
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/zjy/code/vllm-src/.venv/lib/python3.12/site-packages/numpy/_core/memmap.py", line 235, in __new__
    os.fspath(filename),
    ^^^^^^^^^^^^^^^^^^^
TypeError: expected str, bytes or os.PathLike object, not NoneType
```


Comments:
Hi @ZJY0516 ! The correct usage is:

```python
model_id = "unsloth/gemma-3-27b-it-GGUF"
filename = "gemma-3-27b-it-Q8_0.gguf"

tokenizer = AutoTokenizer.from_pretrained(model_id, gguf_file=filename)
model = AutoModelForCausalLM.from_pretrained(model_id, gguf_file=filename)
```

> Hi [@ZJY0516](https://github.com/ZJY0516) ! The correct usage is:
> 
> model_id = "unsloth/gemma-3-27b-it-GGUF"
> filename = "gemma-3-27b-it-Q8_0.gguf"
> 
> tokenizer = AutoTokenizer.from_pretrained(model_id, gguf_file=filename)
> model = AutoModelForCausalLM.from_pretrained(model_id, gguf_file=filename)

Thanks for replying. @bvantuan 

So Transformers can't load local GGUF files directly?

Unfortunately not! You’d need to use a compatible inference library like llama.cpp or its Python bindings.