Repository: huggingface/transformers (Python)
Date: 2024-05-01 19:16:26+00:00
Task type: Bug Fix

Issue:
`model.safetensors` missing in model file not found error in default case
### System Info

System info isn't super relevant here since the confusion is really just an just an error message string. I just reproduced in a CPU instance but this is applicable whenever model loading is needed.
- `transformers` version: 4.40.1
- Platform: Linux-6.1.58+-x86_64-with-glibc2.35
- Python version: 3.10.12
- Huggingface_hub version: 0.22.2
- Safetensors version: 0.4.3
- Accelerate version: 0.29.3
- Accelerate config: 	not found
- PyTorch version (GPU?): 2.2.1+cu121 (False)
- Tensorflow version (GPU?): 2.15.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.8.2 (cpu)
- Jax version: 0.4.26
- JaxLib version: 0.4.26
- Using GPU in script?: No
- Using distributed or parallel set-up in script?: No


### Who can help?

Not sure who most recently worked on `modeling_util.py` or model loading parts. Please feel free to point me to the right person

### Information

- [ ] The official example scripts
- [X] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [X] My own task or dataset (give details below)

### Reproduction

Setup has safetensors library installed

```python
from transformers import AutoTokenizer, AutoModelForCausalLM

# this is just a cloned example GPTQ quant model. 
# The point of this is that this repo contains a `gptq_model-4bit-128g.safetensors` file (default naming by AutoGPTQ library), but not `model.safetensors`.
# Not having a `.safetensors` and/or any modeling files produces the same behavior
# Note how use_safetensors is not passed in
# With safetensors library, it will default to `use_safetensors=None` inside `.from_pretrained()` function
tokenizer = AutoTokenizer.from_pretrained("davidxmle/Llama-3-8B-Instruct-GPTQ-4-Bit-Debug")
model = AutoModelForCausalLM.from_pretrained("davidxmle/Llama-3-8B-Instruct-GPTQ-4-Bit-Debug")
```

An error will get thrown
```
davidxmle/Llama-3-8B-Instruct-GPTQ-4-Bit-Debug does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack.
```
 Error message has no mention of `.safetensors` file is an accepted model file format.


### Expected behavior

This isn't necessarily a bug but rather a confusing error message. I initially noticed this issue when I uploaded a GPTQ quant model made using the AutoGPTQ library to huggingface but some folks reported that they are getting the error above `does not appear to have a file named pytorch_model.bin, tf_model.h5, model.ckpt or flax_model.msgpack.` One would have immediately assumed `safetensors` is not an accepted file format and I must not use `safetensors` but rather use one of the models listed above, since I do have a `safetensors` file in the repo.

Upon further investigation I realized that the `use_safetensors` is an optional argument to be passed in added in https://github.com/huggingface/transformers/pull/22083. This means `use_safetensors` can either be `False` if explicitly defined as `False` or does not have safetensors package installed, `True` if explicitly defined as so, or, **most importantly `use_safetensors` will default to`None` if not specified but has safetensors installed**.

The pull request added in https://github.com/huggingface/transformers/pull/22083, does add an error message specifically for safe tensors, but it uses `elif use_safetensors:` for the error message specific for safetensors, and used `elif use_safetensors is not False` elsewhere... `elif use_safetensors:` evaluates both the default case with `None` value and explict `False` case to false, so the error message containing safetensors in the default case never reached there. I do believe this is the expected behavior since we do not want to only mention safetensors is missing in the default case. 

However, in the case where the `use_safetensors` is defaulted to `None` when the arg is not passed in and model file is missing, there is no mention of `.safetensors` is supported. This should be comprehensive list of all modeling files that are supported, including `model.safetensors`.

I have put together a super simple PR and changed the unit test to address this.


Issue discussion:

Solution patch:
diff --git a/src/transformers/modeling_utils.py b/src/transformers/modeling_utils.py
--- a/src/transformers/modeling_utils.py
+++ b/src/transformers/modeling_utils.py
@@ -3263,8 +3263,8 @@ def from_pretrained(
                     )
                 else:
                     raise EnvironmentError(
-                        f"Error no file named {_add_variant(WEIGHTS_NAME, variant)}, {TF2_WEIGHTS_NAME},"
-                        f" {TF_WEIGHTS_NAME + '.index'} or {FLAX_WEIGHTS_NAME} found in directory"
+                        f"Error no file named {_add_variant(WEIGHTS_NAME, variant)}, {_add_variant(SAFE_WEIGHTS_NAME, variant)},"
+                        f" {TF2_WEIGHTS_NAME}, {TF_WEIGHTS_NAME + '.index'} or {FLAX_WEIGHTS_NAME} found in directory"
                         f" {pretrained_model_name_or_path}."
                     )
             elif os.path.isfile(os.path.join(subfolder, pretrained_model_name_or_path)):
@@ -3410,8 +3410,8 @@ def from_pretrained(
                         else:
                             raise EnvironmentError(
                                 f"{pretrained_model_name_or_path} does not appear to have a file named"
-                                f" {_add_variant(WEIGHTS_NAME, variant)}, {TF2_WEIGHTS_NAME}, {TF_WEIGHTS_NAME} or"
-                                f" {FLAX_WEIGHTS_NAME}."
+                                f" {_add_variant(WEIGHTS_NAME, variant)}, {_add_variant(SAFE_WEIGHTS_NAME, variant)},"
+                                f" {TF2_WEIGHTS_NAME}, {TF_WEIGHTS_NAME} or {FLAX_WEIGHTS_NAME}."
                             )
                 except EnvironmentError:
                     # Raise any environment error raise by `cached_file`. It will have a helpful error message adapted


Test patch:
diff --git a/tests/test_modeling_utils.py b/tests/test_modeling_utils.py
--- a/tests/test_modeling_utils.py
+++ b/tests/test_modeling_utils.py
@@ -1001,6 +1001,26 @@ def test_use_safetensors(self):
             self.assertTrue(any(f.endswith("safetensors") for f in all_downloaded_files))
             self.assertFalse(any(f.endswith("bin") for f in all_downloaded_files))
 
+        # test no model file found when use_safetensors=None (default when safetensors package available)
+        with self.assertRaises(OSError) as missing_model_file_error:
+            BertModel.from_pretrained("hf-internal-testing/config-no-model")
+
+        self.assertTrue(
+            "does not appear to have a file named pytorch_model.bin, model.safetensors,"
+            in str(missing_model_file_error.exception)
+        )
+
+        with self.assertRaises(OSError) as missing_model_file_error:
+            with tempfile.TemporaryDirectory() as tmp_dir:
+                with open(os.path.join(tmp_dir, "config.json"), "w") as f:
+                    f.write("{}")
+                f.close()
+                BertModel.from_pretrained(tmp_dir)
+
+        self.assertTrue(
+            "Error no file named pytorch_model.bin, model.safetensors" in str(missing_model_file_error.exception)
+        )
+
     @require_safetensors
     def test_safetensors_save_and_load(self):
         model = BertModel.from_pretrained("hf-internal-testing/tiny-random-bert")