OSError: Can't load weights for 'facebook/mbart-large-cc25' when using TFMBartModel
Author: SantiagoEG
Pull request: no

## Environment info

- `transformers` version: 4.3.3
- Platform: Windows 10
- Python version: 3.8
- PyTorch version (GPU?): - No
- Tensorflow version (GPU?): 2.4.1, Yes
- Using GPU in script?: Yes (I think it is indifferent for this issue)
- Using distributed or parallel set-up in script?: No

## Issue Description
Firstly, I would to thank you this extraordinary contribution to NLP. We are starting to apply transformers to our NLP problem and we want to test the pretrained mBART model. I have tried to load the TF version of this model following your documentation: https://huggingface.co/transformers/master/model_doc/mbart.html#tfmbartmodel

Unfortunately, we are experiencing an error in _"model = TFMBartModel.from_pretrained('facebook/mbart-large-cc25')"_.

**Error Traceback:**

404 Client Error: Not Found for url: https://huggingface.co/facebook/mbart-large-cc25/resolve/main/tf_model.h5
Traceback (most recent call last):

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\transformers\modeling_tf_utils.py", line 1203, in from_pretrained
    resolved_archive_file = cached_path(

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\transformers\file_utils.py", line 1078, in cached_path
    output_path = get_from_cache(

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\transformers\file_utils.py", line 1216, in get_from_cache
    r.raise_for_status()

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\requests\models.py", line 943, in raise_for_status
    raise HTTPError(http_error_msg, response=self)

HTTPError: 404 Client Error: Not Found for url: https://huggingface.co/facebook/mbart-large-cc25/resolve/main/tf_model.h5


During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "G:\Mi unidad\D4.2\Proof of Concept simCLR for MT\test_tokenizer.py", line 46, in <module>
    bart_model = TFMBartModel.from_pretrained('facebook/mbart-large-cc25')

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\transformers\modeling_tf_utils.py", line 1219, in from_pretrained
    raise EnvironmentError(msg)

OSError: Can't load weights for 'facebook/mbart-large-cc25'. Make sure that:

- 'facebook/mbart-large-cc25' is a correct model identifier listed on 'https://huggingface.co/models'

- or 'facebook/mbart-large-cc25' is the correct path to a directory containing a file named one of tf_model.h5, pytorch_model.bin.

It seems that .h5 file with pretrained weights are not available in your repository. If it is not possible to update it, do you any way to transform pytorch .bin to TF .h5? 

Thank you in advance!!! 


Comments:
Hi! Yes, you can load the PyTorch weights into a Transformer model by adding `from_pt=True` in the `from_pretrained` method.

Thank you very much @LysandreJik!
I have tried two ways:

**Option 1 Using from_pt = True**
_bart_model = TFMBartModel.from_pretrained("facebook/mbart-large-cc25", from_pt=True)_

This worked well, but the following message appeared:
Some weights of the PyTorch model were not used when initializing the TF 2.0 model TFMBartModel: ['final_logits_bias', 'model.encoder.embed_tokens.weight', 'model.decoder.embed_tokens.weight']
- This IS expected if you are initializing TFMBartModel from a PyTorch model trained on another task or with another architecture (e.g. initializing a TFBertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing TFMBartModel from a PyTorch model that you expect to be exactly identical (e.g. initializing a TFBertForSequenceClassification model from a BertForSequenceClassification model).
All the weights of TFMBartModel were initialized from the PyTorch model.
If your task is similar to the task the model of the checkpoint was trained on, you can already use TFMBartModel for predictions without further training.

Furthermore, I had memory problems when trying this way in GPU.

**Option 2 Using MBartConfig**
_configuration = MBartConfig(name_or_path  = "facebook/mbart-large-cc25")
bart_model = TFMBartModel(configuration)_

I have checked the outputs for the same input, and they were different. So I think that Option 1 did not properly loaded all the weights. So, I recommend using Option 2! Hope this helps!



Hi! The option 2 you mention isn't loading any weights on the model itself. You're instantiating a configuration that is similar to `facebook/mbart-large-cc25`, and initializing a model with random weights following that configuration.

If you run inference twice through the model loaded with option 1, do you get different inputs?

Hi @LysandreJik,

The outputs were different between option 1 & 2, but it is obvious if they do not load the same weights. But other fact is that when I load the model with option 1 several times, the outputs are different. Meanwhile, if I load once the model and predict twice, the outputs are the same. Could be it due to dropout? 

To avoid memory problems with option 1, I am going to load the model in CPU and export the TF weights to h5 file. Then load them with GPU settings.

**[CPU] Saving pretrained model**
I have tried loading the pretrained mBART model in CPU settings and save it in TF formar with the following code:

_mbart_cpu = TFMBartModel.from_pretrained("facebook/mbart-large-cc25", from_pt=True)
mbart_cpu.save_pretrained('saved_models/')_

No errors appeared

**[GPU] Loading pretrained weights**
After exporting pretrained mBART, I tried loading it with GPU settings as follows:

_mbart_in_gpu = TFMBartModel.from_pretrained("saved_models")_

However, the following error appeared:

**Traceback (most recent call last):**

  File "G:\Mi unidad\D4.2\Proof of Concept simCLR for MT\load_in_GPU.py", line 23, in <module>
    mbart_model_2 = TFMBartModel.from_pretrained("saved_models")

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\transformers\modeling_tf_utils.py", line 1244, in from_pretrained
    missing_keys, unexpected_keys = load_tf_weights(model, resolved_archive_file)

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\transformers\modeling_tf_utils.py", line 532, in load_tf_weights
    K.batch_set_value(weight_value_tuples)

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\tensorflow\python\util\dispatch.py", line 201, in wrapper
    return target(*args, **kwargs)

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\tensorflow\python\keras\backend.py", line 3706, in batch_set_value
    x.assign(np.asarray(value, dtype=dtype(x)))

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\tensorflow\python\ops\resource_variable_ops.py", line 892, in assign
    assign_op = gen_resource_variable_ops.assign_variable_op(

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\tensorflow\python\ops\gen_resource_variable_ops.py", line 144, in assign_variable_op
    _ops.raise_from_not_ok_status(e, name)

  File "c:\users\vicen\anaconda3\envs\signon_2\lib\site-packages\tensorflow\python\framework\ops.py", line 6862, in raise_from_not_ok_status
    six.raise_from(core._status_to_exception(e.code, message), None)

  File "<string>", line 3, in raise_from

**InternalError: Failed copying input tensor from /job:localhost/replica:0/task:0/device:CPU:0 to /job:localhost/replica:0/task:0/device:GPU:0 in order to run AssignVariableOp: Dst tensor is not initialized. [Op:AssignVariableOp]**

Thank you in advance for your help!


Pinging @patrickvonplaten and @patil-suraj 

Thank you very much @LysandreJik!

Hey @SantiagoEG 

the reason for the issue is that `TFMBartModel` is not TF counterpart of `MBartModel`,  the counterpart is `TFMBartMainLayer`

as you can see here
pt `MBartModel` : https://github.com/huggingface/transformers/blob/master/src/transformers/models/mbart/modeling_mbart.py#L1096
tf `TFMBartModel`: https://github.com/huggingface/transformers/blob/master/src/transformers/models/mbart/modeling_tf_mbart.py#L1178

the structure is different, `TFMBartModel`  does not contain the shared token embeddings layer, but instead, it wraps `TFMBartMainLayer `, which is why we can’t do `TFMBartModel.from_pretrained(..., from_pt=True)`
instead, we need to load weights using `TFMBartForConditionalGeneration` and then  we can load `TFMBartModel`  using the saved `TFMBartForConditionalGeneration`

```python
tf_model = TFMBartForConditionalGeneration.from_pretrained("facebook/mbart-large-cc25", from_pt=True)
tf_model.save_pretrained("tf_model")

tf_mbart_model = TFMBartModel.from_pretrained("tf_model")

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.