OSError: meta-llama/Llama-4-Scout-17B-16E-Instruct does not appear to have a file named X
Author: sam-h-bean
Pull request: no
Labels: name: bug

### System Info

I have transformers 4.51.0 and am trying to load the Llama 4 Scout model for training. I loaded the safetensor files to disk and am pointing to that location with the `cache_dir` arg. It seems like each time I load the model with `AutoModelForCausalLM` I get a different error. One run it is

```bash
OSError: meta-llama/Llama-4-Scout-17B-16E-Instruct does not appear to have a file named model-00008-of-00050.safetensors. Checkout 'https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct/tree/main'for available files.
```

and the next run it is

```bash
OSError: meta-llama/Llama-4-Scout-17B-16E-Instruct does not appear to have files named ('model-00002-of-00050.safetensors', 'model-00003-of-00050.safetensors', 'model-00004-of-00050.safetensors', 'model-00005-of-00050.safetensors', 'model-00006-of-00050.safetensors', 'model-00007-of-00050.safetensors', 'model-00008-of-00050.safetensors', 'model-00011-of-00050.safetensors', 'model-00012-of-00050.safetensors', 'model-00014-of-00050.safetensors', 'model-00016-of-00050.safetensors', 'model-00017-of-00050.safetensors'). Checkout 'https://huggingface.co/meta-llama/Llama-4-Scout-17B-16E-Instruct/tree/main'for available files.
```

It seems to change each time I run the training script. Wondering if there is some error with loading from the cache for this many files.

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Load the model with

```python
model = AutoModelForCausalLM.from_pretrained(
  "meta-llama/Llama-4-Scout-17B-16E-Instruct",
  torch_dtype=torch.bfloat16,
  token=HF_TOKEN,
  attn_implementation="flash_attention_2",
)
```

### Expected behavior

Model is loaded into memory

Comments:
Delete the model from ~/.cache/huggingface/hub and download again. There was an error with your download.

@aroradrishan I've tried that a few times with no luck

This does seem to be an issue of reading weights from many processes (hundreds) at once. Reading from a single 8-H100 node can load the model with `device_map='auto'` however loading from many processes in a ray cluster leads to this error. Pretty odd...

Might be `hf_transfer` ! 

+1 can't get this to work

It looks like the issue is if you don't have enough disk space all the files will not download, but there is no warning or error printed.

Same error with 9.5 TB space

Update:

Created new env with no `deepspeed` and `hf_xet` and it works.

Yeah I have been able to load the model in a python repl with no deepspeed. However when I try to load from the cache in a distributed setting I get the reported error. Using the same env and loading Mistral 24B doesn't have an issue either. So it seems to be something around this log of fetching files which might be creating some lock on the files? I have had race condition issues loading HF checkpoints in a distributed setting previously...

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.