Possible issue with safetensors loading for TF models
Author: amyeroberts
Pull request: no
Labels: name: TensorFlow

### System Info

- transformers v4.40.0.dev0 on a907a903d644f835a5563b1133b1fc961e33c3d2
- Platform: macOS-14.2.1-arm64-arm-64bit
- Python version: 3.10.9
- Huggingface_hub version: 0.21.3
- Safetensors version: 0.4.2
- Accelerate version: 0.29.0.dev0
- Accelerate config:    not found
- PyTorch version (GPU?): 2.2.2 (False)
- Tensorflow version (GPU?): 2.16.1 (False)
- Flax version (CPU?/GPU?/TPU?): 0.7.0 (cpu)
- Jax version: 0.4.12
- JaxLib version: 0.4.12


### Who can help?

@Rocketknight1 

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

Running on a907a903d644f835a5563b1133b1fc961e33c3d2, the following tests will fail: 

```
FAILED tests/generation/test_tf_utils.py::TFGenerationIntegrationTests::test_eos_token_id_int_and_list_beam_search - AssertionError: False is not true
FAILED tests/generation/test_tf_utils.py::TFGenerationIntegrationTests::test_eos_token_id_int_and_list_contrastive_search - AssertionError: False is not true
FAILED tests/generation/test_tf_utils.py::TFGenerationIntegrationTests::test_eos_token_id_int_and_list_greedy_search - AssertionError: False is not true
FAILED tests/generation/test_tf_utils.py::TFGenerationIntegrationTests::test_eos_token_id_int_and_list_top_k_top_sampling - AssertionError: False is not true
```

and 

```
tests/test_pipeline_mixin.py::TextGenerationPipelineTests::test_stop_sequence_stopping_criteria
```

to run the generation tests: 

```
pytest tests/generation/test_tf_utils.py::TFGenerationIntegrationTests::test_eos_token_id_int_and_list_beam_search tests/generation/test_tf_utils.py::TFGenerationIntegrationTests::test_eos_token_id_int_and_list_contrastive_search tests/generation/test_tf_utils.py::TFGenerationIntegrationTests::test_eos_token_id_int_and_list_greedy_search tests/generation/test_tf_utils.py::TFGenerationIntegrationTests::test_eos_token_id_int_and_list_top_k_top_sampling
```

To run the pipeline test, it's necessary to have only tensorflow and no torch installed in your environment. Then:

```
pytest tests/test_pipeline_mixin.py::TextGenerationPipelineTests::test_stop_sequence_stopping_criteria
```


This resulted from an upstream change in the model weights used for testing: https://huggingface.co/hf-internal-testing/tiny-random-gpt2/tree/main, [specifically this commit](https://huggingface.co/hf-internal-testing/tiny-random-gpt2/commit/914a3832e3d85d1fad25be459c785ca159d3c87a). 

The tests same tests did not fail for the PT model, indicating the safetensors weights conserve the previous PT behaviour, but not tensorflow. 

Both the TF and PT model weights were uploaded at the same time, and haven't been changed since, so it's not expected for them to have diverged. 

### Expected behavior

When loading in safetensors weights, both PT and TF models preserve their behaviour. 

Comments:
Once the issue has been identified, the changes made in #30118 should be reverted 

Did some testing here, and I'm pretty sure TF loading is okay and the safetensors file is just malformed in some way. I had no issues saving checkpoints from Torch as safetensors and loading them into TF. Most tellingly, if I loaded the `hf-internal-testing/tiny-random-gpt2` checkpoint in `transformers` and then saved it again as safetensors, TF could load that save without problems.

I'm going to replace the safetensors file in the repo with an updated one created using the save-load method, and hopefully we can revert your changes after that.

@Rocketknight1 thanks for the quick investigation! Glad that save/load seems to work. 

Slightly worried that we might have lots of malformed weights on the hub due to the safetensors bot 😬. As the save/load seems to produce an equivalent PT model, and fixed TF model, then re-converting the weights might be the answer. 

I still have a question about why the converted safetensors weights seem to be OK for the PT model 🤔 

I'm not sure myself! However, I suspect we'd see an issue quickly - on the other occasions when I've accidentally broken a TF model with high usage, I got an issue on Github within 48 hours, so I doubt broken safetensors conversions would slip by unnoticed, especially since the bug manifested as all of the model weights being randomly initialized instead of being loaded from the checkpoint.

My guess is the `tiny-random-gpt2` checkpoint is either quite old, or created in an unusual way. The bug seemed to be related to the naming of tensors inside the archive - TF didn't seem to be matching them up to the model weights correctly. I think we can close this for now as a weird anomaly, and reopen it if the issue crops up again!

Sounds good!