Safetensors deserializing silently mishandles tied parameters
Author: edmcman
Pull request: no
Labels: name: bug

### System Info

- `transformers` version: 4.52.4
- Platform: Linux-6.1.123+-x86_64-with-glibc2.35
- Python version: 3.11.13
- Huggingface_hub version: 0.33.0
- Safetensors version: 0.5.3
- Accelerate version: 1.7.0
- Accelerate config: 	not found
- DeepSpeed version: not installed
- PyTorch version (GPU?): 2.6.0+cu124 (False)
- Tensorflow version (GPU?): 2.18.0 (False)
- Flax version (CPU?/GPU?/TPU?): 0.10.6 (cpu)
- Jax version: 0.5.2
- JaxLib version: 0.5.1
- Using distributed or parallel set-up in script?: no

### Who can help?

@Cyrilvallez (this is a pretty rough guess)

### Information

- [ ] The official example scripts
- [x] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [x] My own task or dataset (give details below)

### Reproduction

https://colab.research.google.com/drive/1FwWTcK9N9nDiLeDyuW9ckeLSzZfpI7wJ?usp=sharing

### Expected behavior

I expected that my model would be loaded and that the `transformers.wte.weight` would have weights loaded from the safetensors checkpoint.  Instead, it does not have weights and causes `accelerate` to crash.

At the very least, I expected a warning about the missing keys.  There is actually some code to warn about this, but in this case it did not fire because the missing key is a tied parameter.

I can't quite clearly describe or fix the bug, but it has to do with safetensors deserialization of tied parameters.  The tiee parameter's weights are not actually set from the saved checkpoint and are just the blank `meta` versions.  

Disabling safetensors avoids the problem.

Setting `tie_word_embeddings=False` oddly does emit a warning:
```
Some weights of GPTBigCodeForCausalLM were not initialized from the model checkpoint at ejschwartz/resym-fielddecoder and are newly initialized: ['transformer.wte.weight']
You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.
```

Here is the the original [issue](https://github.com/huggingface/accelerate/issues/3617) I opened in accelerate, but I think that the model accelerate is operating on is invalid.

Comments:
Most likely in core model loading so cc @cyrilvallez @arthurzucker, but feel free to ping me or someone else to take it if you're overworked!

Hey! This is not an issue with model loading, but with the actual safetensors checkpoint in `ejschwartz/resym-fielddecoder`. Tied weights are `lm_head.weight` and `transformer.wte.weight`, and the code expects the actual data to reside in `transformer.wte.weight`, however it resides in `lm_head.weight` for this checkpoint, so weights are no tied correctly. You could easily solve it by opening a PR on the hub repo, and switching the embedding from the key `lm_head.weight` to `transformer.wte.weight`. 

However, as it seems conversion to safetensors was performed automatically by our bot, I am still going to check if everything is handled correctly for tied weights there.

Alright, opened a PR on the conversion bot [here](https://huggingface.co/spaces/safetensors/convert/discussions/41) to avoid such issues in the future (issue was that this repo does not have the `transformers` library tag, so tied_weights are not correctly taken into account)

Thanks Cyril!

Could there have been/should there have been a warning while loading in
transformers about a missing parameter too?  This would at least hint that
the checkpoint itself might be corrupted!

Best,

Edward

On Thu, Jun 19, 2025, 6:01 AM Cyril Vallez - ***@***.***
***@***.***> wrote:

> *Cyrilvallez* left a comment (huggingface/transformers#38870)
> <https://github.com/huggingface/transformers/issues/38870#issuecomment-2987496340>
>
> Alright, opened a PR on the conversion bot here
> <https://huggingface.co/spaces/safetensors/convert/discussions/41> to
> avoid such issues in the future (issue was that this repo does not have the
> transformers library tag, so tied_weights are not correctly taken into
> account)
>
> —
> Reply to this email directly, view it on GitHub
> <https://github.com/huggingface/transformers/issues/38870#issuecomment-2987496340>,
> or unsubscribe
> <https://github.com/notifications/unsubscribe-auth/AAHYKZIV2GQOUZKCDP7BYTD3EKDBLAVCNFSM6AAAAAB7RGA4SWVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDSOBXGQ4TMMZUGA>
> .
> You are receiving this because you authored the thread.Message ID:
> ***@***.***>
>


Indeed, I will try to see if I can surface a warning in this specific edge case - for now it's not detected as all the weights are technically present, but they are just not tied correctly downstream at the end!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

The safetensors PR is not merged yet, and still no warning, so leaving open.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Any luck on the warnings @Cyrilvallez?

Hey @edmcman! Sorry for the delay. Unfortunately at this point I don't think the PR for the converter will be merged. Seeing how niche the issue you encounter is, I don't think it's really worth the added logic to add a warning in the end. The best way to proceed would be to update the safetensors file of your checkpoint as I explained above (just switch the name of the key in the `.saftensors` file itself)

@Cyrilvallez That's disappointing to hear, especially that the PR for the converter is unlikely to be merged.  I have been able to work around the issue personally for a long time simply by disabling safetensors.

I opened these issues to help other people (and myself in the future).  I know I personally have run into many similar problems with the meta device when using `accelerate` before, but only this instance had time to investigate it.  There is certainly no shortage of results if I google for `huggingface "NotImplementedError: Cannot copy out of meta tensor; no data!"` so I'm not sure if this is really a niche problem or if the error message is so bad no one knows what to do. If this was my project, I wouldn't want that to be an error message that is exposed to end users.

Well, `"NotImplementedError: Cannot copy out of meta tensor; no data!"` can have a lot of sources, and this one is definitely very niche

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

Hey @edmcman! Just to let you know that my hub PR was merged in the end! 🤗 Should avoid the same issues in the future!

Woohoo, we've made the world a (slightly) better place!

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.