[TF] Save finetuned-model without huggingface-hub login
Author: goreng2
Pull request: no

### Feature request

[TF] Save finetuned-model in local without huggingface-hub login

### Motivation

in TF, We need to login for saving finetuned-model.
```
from transformers.keras_callbacks import PushToHubCallback

push_to_hub_callback = PushToHubCallback(
    output_dir="my_awesome_model",
    tokenizer=tokenizer,
)
```

But I don't want to sync in my hub yet. Firstly, I want to save my models in local and test them

I checked that works in PyTorch, But It's not in Tensorflow

### Your contribution

I think we need to add argument whether to login or not

https://github.com/huggingface/transformers/blob/0526a075c567d7508205fe6054310f3d132b3227/src/transformers/keras_callbacks.py#L267

Comments:
cc @Rocketknight1 

I also find the problem that finetuned-model doesn't sync with hub when give only 1 epoch or at the last epoch

Hi @goreng2, you're correct that right now the callback expects a HF login. This is because that callback is designed for uploading models to the hub. If you just want to save the model locally, you can try either:

1) The [ModelCheckpoint callback](https://keras.io/api/callbacks/model_checkpoint/) in Keras to save the weights every epoch if you just want to save/resume training.

2) The `model.save_pretrained()` method if you want to save the model locally after training and reload it with `from_pretrained` afterwards.

Are you specifically interested in saving the model like `save_pretrained()` every epoch? We could add that, but let us know if you think that'd be useful for you first, or the solutions above are enough!

Hi @Rocketknight1 ! Thanks for your comment.

I want to use Huggingface's `pipeline` API for inference. I think `pipeline` perhaps can receive only `.h5` model

When I tried `ModelCheckpoint callback`, It returns `ckpt` files. It can't be used in `pipeline`.
For convert `ckpt` to `.h5`, I need to write model architecture (in my case `ELECTRA`) But It's so difficult and complex to me 😥
I tried to convert `ckpt` to `pth (PyTorch)` But It doesn't work... Maybe [this code](https://github.com/huggingface/transformers/blob/main/src/transformers/models/electra/convert_electra_original_tf_checkpoint_to_pytorch.py) only works in converting TF1 to PyTorch

When I tried `model.save('my_model.h5')`, Error msg raised. Maybe Something format is not match

I don't test `model.save_pretrained()` yet, It returns `.h5`?

Ah, yes. The `.ckpt` files from `ModelCheckpoint` are only useful for saving/resuming training, and you won't be able to use them in pipelines.

The way TF models on HuggingFace work is that they're built on top of Keras models. `model.save()` and `ModelCheckpoint` are both part of Keras. However, if you want to save the model to load with other HuggingFace tools, you should use `save_pretrained()`. This is our method and doesn't exist in base Keras models. It saves the model as `.h5`, but also adds a `config.json` that will allow the `pipeline` API and other methods like `from_pretrained` to initialize the model correctly.

Try just doing this:
```
model.save_pretrained("my_model")
pipe = pipeline("text-classification", model="my_model")
```
Though of course, make sure to change `text-classification` to the task you want to do!

@Rocketknight1 
Hi, Thanks for your answer 😀

I tested it and It worked!
I got `tf_model.h5`, `config.json` and success to run `pipeline`!

But It's not perfect for inference, Because `model.save_pretrained("output_folder")` returns only 2 files that I mentioned upper.

`tokenizer.json`, `tokenizer_config.json` and so on are also needed for inference

So, How about make `model.save_pretrained("output_folder")` return other files about tokenizer?

@goreng2 Sorry for the delay! Yes, you will need to save the tokenizer to the same directory with `tokenizer.save_pretrained()` in order to load the whole directory with a pipeline.