use_safetensors doesn't work for Falcon models?
Author: RonanKMcGovern
Pull request: no

### System Info

System Information
Operating System: Linux
OS Version: 5.4.0-155-generic
Machine: x86_64
Processor: x86_64

Python Environment
Python Version: 3.10.6 (main, May 29 2023, 11:10:38) [GCC 11.3.0]
Executable: /usr/bin/python

CUDA & Hardware Information
CUDA Available: True
CUDA Version: 11.8
CUDA Device Count: 1
Current CUDA Device: 0
CUDA Device Name: NVIDIA RTX A6000

Transformers Library Information
Transformers Version: 4.34.0

### Who can help?

@younesbelkada @ArthurZucker I've just tried to push an unloaded and merged model to hub using:
```
model_to_push.push_to_hub(new_model, use_auth_token=True, max_shard_size="10GB", use_safetensors=True)
```
However, the files pushed are .bin . Is there support for Falcon?

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

```

model_id = "tiiuae/falcon-7b"
adapter_to_push = "Trelis/falcon-7b-chat-llama-style-adapters"

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    device_map='cuda',
    torch_dtype=torch.bfloat16,
    cache_dir=cache_dir)

model_to_push = PeftModel.from_pretrained(
    model,
    adapter_to_push,
)

model_to_push = model_to_push.merge_and_unload() # merge adapters with the base model.

model_to_push.push_to_hub(new_model, use_auth_token=True, max_shard_size="10GB", use_safetensors=True)
```
This works but pushes bin rather than safetensors files.

Interestingly, if I try to save before pushing, like this:
```
model_to_push.save_pretrained(new_model, push_to_hub=True)
```

I get this error:
```
Cell In[22], line 1
----> 1 model_to_push.save_pretrained(new_model, push_to_hub=True)

File /usr/local/lib/python3.10/dist-packages/transformers/modeling_utils.py:2116, in PreTrainedModel.save_pretrained(self, save_directory, is_main_process, state_dict, save_function, push_to_hub, max_shard_size, safe_serialization, variant, token, save_peft_format, **kwargs)
   2114         safe_save_file(shard, os.path.join(save_directory, shard_file), metadata={"format": "pt"})
   2115     else:
-> 2116         save_function(shard, os.path.join(save_directory, shard_file))
   2118 if index is None:
   2119     path_to_weights = os.path.join(save_directory, _add_variant(WEIGHTS_NAME, variant))

File /usr/local/lib/python3.10/dist-packages/torch/serialization.py:618, in save(obj, f, pickle_module, pickle_protocol, _use_new_zipfile_serialization, _disable_byteorder_record)
    615 _check_save_filelike(f)
    617 if _use_new_zipfile_serialization:
--> 618     with _open_zipfile_writer(f) as opened_zipfile:
    619         _save(obj, opened_zipfile, pickle_module, pickle_protocol, _disable_byteorder_record)
    620         return

File /usr/local/lib/python3.10/dist-packages/torch/serialization.py:466, in _open_zipfile_writer_file.__exit__(self, *args)
    465 def __exit__(self, *args) -> None:
--> 466     self.file_like.write_end_of_file()
    467     if self.file_stream is not None:
    468         self.file_stream.close()

RuntimeError: [enforce fail at inline_container.cc:424] . unexpected pos 1199971712 vs 1199971600
```

### Expected behavior

Typically this approach results in safetensors, not bin being pushed. Also, there is typically no issues saving the model before pushing.

As a side note, inference on the peft model works well (both pre and post merge). 

Comments:
Hi @RonanKMcGovern 
Thanks for the issue
I tried to reproduce but the `Trelis/falcon-7b-chat-llama-style-adapters` does not exists on the Hub (or it is private), would it be possible to make that repo public or push random weights on the Hub? 🙏 

Thanks! Public now @younesbelkada 

OK thanks! WIll try now and let you know

Hi @RonanKMcGovern 
I just managed to push the merged weights on the hub: https://huggingface.co/ybelkada/test-falcon-st/tree/main I suspect it is a CPU OOM issue that you are getting. What is the max VRAM of your CPU? Can you perhaps try with smaller shard size?

Thanks!

Were you able to push safetensors?

Yeah could have been OOM but also could have been an issue with weights format owing to the SFT issue that was recently solved?

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.

This is working now