Converting models for tensoflowjs (node)
Author: ierezell
Pull request: no
Labels: name: External

## Environment info
- `transformers` version: 4.3.3
- Platform: Linux-5.11.2-zen1-1-zen-x86_64-with-glibc2.2.5
- Python version: 3.8.8 (also 3.9.2 but tensorflowjs is not available in 3.9.2)
- PyTorch version (GPU?): 1.7.1 (True)
- Tensorflow version (GPU?): 2.4.1 (False)
- Using GPU in script?: No 
- Using distributed or parallel set-up in script?: No

### Who can help
- tensorflow: @jplu

## Information

Model I am using (Bert, XLNet ...): Any tensorflow ones

The problem arises when using:
* [X] the official example scripts: (give details below)
* [X] my own modified scripts: (give details below)

The tasks I am working on is:
* [X] my own task or dataset: (give details below)

## To reproduce

Steps to reproduce the behavior:

1. Load a tensforflow model (e.g : `model = TFAutoModel.from_pretrained("distilbert-base-uncased")`) 
2. Convert it to H5 with `model.save_pretrained(path)`
3. Try to load it in tensorflowjs and get `UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'model_config' of null`

## Expected behavior
Be able to load a model in the LayerModel format as it's the only one which allows finetuning. 

## More informations. 
I know the question was already posted in https://github.com/huggingface/transformers/issues/4073

I need to do finetuning so onnx, graphModels & cie should be avoided. 

Seems that the H5 model just need the config file which seems saved on the side with the custom HF script. 
I went to read some issues on tensorflowjs (exemple : [this](https://github.com/tensorflow/tfjs/issues/2455) or [that](https://github.com/tensorflow/tfjs/issues/931)) and the problem is that the HF model contains only the weights and not the architecture. The goal would be to adapt the `save_pretrained` function to save the architecture as well. I guess it's complex because of the `Saving the model to HDF5 format requires the model to be a Functional model or a Sequential model.` error in described bellow. 

Seems also that only H5 model can be converted to a trainable LayerModel. 

I'm willing to work on a PR or to help as i'm working on a web stack (nodejs) and I need this. 

I made a drawing of all models (that I'm aware of) to summarize loading converting : 
![image](https://user-images.githubusercontent.com/30974685/109985131-6edebe80-7cd2-11eb-8f3c-60080e98a115.png)

## Also tried : 
Use the nodejs `tf.node.loadSavedModel` which return only a saved model which I cannot use as the base structure with something like : 
```
const bert = await tf.loadLayersModel(`file://${this._bert_model_path}/model.json`)
this._clf = tf.sequential();
this._clf.add(bert); // Raise Error
this._clf.add(tf.layers.dense({ units: 768, useBias: true }));
etc...
this._clf.compile(....)
this._clf.train(...)
```

Look for other libraries to train models (libtorch : incomplete, onnx training: only in python etc..) 

Should I also write an issue on tensorflowjs ? 

Thanks in advance for you time and have a great day. 

Comments:
Hello!

Thanks for reporting this issue! Did you try to convert your H5 file to be able to use it with `tensorflowjs_converter --input_format keras path/to/my_model.h5 path/to/tfjs_target_dir`? You can also have a SavedModel version with:

1. `model = TFAutoModel.from_pretrained("distilbert-base-uncased")`
2. `model.save_pretrained(path, saved_model=True)`

H5 and SavedModel conversion process are nicely explained in https://www.tensorflow.org/js/tutorials/conversion/import_keras and https://www.tensorflow.org/js/tutorials/conversion/import_saved_model

Hi @jplu thanks for the fast reply ! 

Yes I used the `tensorflowjs_converter` to convert the H5 huggingFace model to a TFJS layerModel (.json).

If not using a h5 but a savedModel I don't know how to then use it in tensorflowJS.... all my researches seems to indicate that a savedModel can only do inference but i would like to finetune it.   

My goal is just to finetune Bert in nodejs (could be with other framework than tfjs if you know some (like pytorch node bindings or some fancy framework but I found nothing that allow training))

## Attempts : 

### HuggingFace : 
1. Load model with huggingFace : `model = TFAutoModel.from_pretrained(model_name)`
2. Save it (H5 format) : `model.save_pretrained(path_saved)`
3. convert it to tfjs.json Layer format : 
```python
dispatch_keras_h5_to_tfjs_layers_model_conversion(
    h5_path='path_saved/tf_model.h5',
    output_dir=my_tfjs_output_dir
)
```
(`dispatch_keras_h5_to_tfjs_layers_model_conversion` is the function called when using `tensorflowjs_convert`, i'm calling it from python code)
4. try to load it in nodejs : `const bert = await tf.loadLayersModel('file://${my_tfjs_output_dir}/model.json')`
5. Get the error `UnhandledPromiseRejectionWarning: TypeError: Cannot read property 'model_config' of null`

### Keras or tfjs saving (to have the `model_config`)
1. Load model with huggingFace : `model = TFAutoModel.from_pretrained(model_name)`
2. Save it (H5 format) : 
```python
tf.keras.models.save_model(
    model=model,
    filepath=path_saved,
    signatures=tf.function(model.call).get_concrete_function([
        tf.TensorSpec([1, 512], tf.int32, name="input_ids"),
        tf.TensorSpec([1, 512], tf.int32, name="attention_mask")
    ]),
    save_format='h5'
)
```
or 
`tfjs.converters.save_keras_model( model, path_saved)`
3. Get the error `Saving the model to HDF5 format requires the model to be a Functional model or a Sequential model.`

### Other formats : 
1. Load model with huggingFace : `model = TFAutoModel.from_pretrained(model_name)`
2. Save it (SavedModel format) : `model.save_pretrained(path_saved)`
4. try to load it in nodejs : `const bert = await tf.node.loadSavedModel(path_saved, ['serve'], "serving_default")`
5. Cannot train it or use it in another model with `this._clf = tf.sequential(); this._clf.add(bert);`

#### GraphModel or onnx model are for inference only 

Sorry for this (too) long reply. 

Thanks for your help (I guess many people try to attempt the same), 

Have a great day 

Indeed you cannot train or fine tune a SavedModel in other envs than Python.

Currently to load a model you are forced to have the config file because it is required as an argument to init a model. Then the H5 as well. Furthermore, the TF implementations are using subclass models which are not compliant with most of the internal TF process (such as what you are trying to do in your described process when saving a model).

Then, I suggest you to fine tune your model in Python, create a SavedModel, and then use it in JS as described in the link I shared. Sorry for the inconvenience.

Okay so SOA is training custom models in tfjs but finetunning is almost impossible. It's what I thought but I'm glad you confirmed it. 

Do you know any other solutions that would allow me to finetune a model in a webstack (nodejs / typescript) ?
I'm still hoping that one day pytorch will have node bindings... 

Thanks again for your time.  
I guess we can close the issue.

> Do you know any other solutions that would allow me to finetune a model in a webstack (nodejs / typescript) ?

No, sorry this is not really my domain. And sorry again for the inconvenience on this.