m2m100 conversion failed from fairseq to hf format
Author: harveenchadha
Pull request: no

When I try to convert m2m100 model from fairseq repository the script throws an error:

```
RuntimeError: Error(s) in loading state_dict for M2M100Model:
Missing key(s) in state_dict: "encoder.embed_positions.weights", "decoder.embed_positions.weights". 
```
I manually checked the state dict and no positional encoding are present in the checkpoint. How did you convert this model to the one in hf format?

The problem arises when using:
* [script](https://github.com/harveenchadha/transformers/blob/master/src/transformers/models/m2m_100/convert_m2m100_original_checkpoint_to_pytorch.py) 
* original [model](https://dl.fbaipublicfiles.com/m2m_100/418M_last_checkpoint.pt ) I am trying to convert:  


## Environment info

- `transformers` version: 4.9.1
- Platform: Ubuntu
- Python version: 3.7
- PyTorch version (GPU?): None
- fairseq version: 1.0.0a0+9589463

@patil-suraj @patrickvonplaten 


Model I am using : M2M 100




Comments:
Hi @harveenchadha 

M2M100 uses sinusoidal positional embeds, so it does not save those in the `state_dict`, I didn't get this error when I converted the models. Is this a new version?

And to fix this, you could either add those keys in `ignore_keys` list [here](https://github.com/harveenchadha/transformers/blob/b89a964d3f46bff56e7652e572c63c13b703b7d8/src/transformers/models/m2m_100/convert_m2m100_original_checkpoint_to_pytorch.py#L24) or pass `strict=False` to `model.load_state_dict`

Sorry I am a bit new to translation models so I didn't know about sinusoidal positional embeddings.

No it is the same model that was open sourced by facebook. When I do strict=false, I am able to get a final model. But the final files generated are config.json and pytorch_model.bin

I think there should be three other files:
1. tokenizer_config.json
2. vocab.json
3. special_tokens_map.json

How to generate these files? 

the vocab files needs to be created using the fairseqs `dict.txt`. The conversion script does not convert the tokenizer. 
I will try to post the tokenizer conversion scripts in few days.

That would be really helpful. Right now I have [published](https://huggingface.co/Harveenchadha/indictrans) the model but it cannot be used as the tokenizer is not ready.

If it uses the same M2M100 tokenizer, then you could use the tokenizer from https://huggingface.co/facebook/m2m100_418M
and upload those file to your repo.

I tried this but this didn't work. Actually the tokenizer is not the same as language support is different.

if it's the same model, then the tokenizer should also be the same, no? Did you create your tokenizer or did you add any tokens?

The model arch is same but its not the same model. Original m2m 100 has support for 100 languages but this model is specific for Indic languages and support 13 languages only as it was finetuned on Indic parallel corpus. 

I am not the author of this model, so I am trying to understand the code structure.
[Here](https://github.com/AI4Bharat/indicTrans/blob/main/inference/engine.py) is the inference file.

Maybe I can load the tokenizer from here and model from HuggingFace to give it a try how this works.


Aah, I see. The tokenizer looks different than the current m2m100 tokenizer i.e it uses different normalizers, preprocessors.
In this case, we could add a new tokenizer file for this model. In `Transformers` it's possible to just add a tokenizer, for example [`barthez`](https://github.com/huggingface/transformers/tree/master/src/transformers/models/barthez) uses the BART arch but the tokenizer is different, hence only tokenizer file is added, we could do something similar here and add `IndicTransTokenizer`

This sounds really good, can you please give me a head start like what methods I need to implement from the base library to achieve this? Any document will also be fine.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

@harveenchadha I am also getting the same error loading original m2m100 checkpoint. Missing key(s) in state_dict: "encoder.embed_positions.weights", "decoder.embed_positions.weights"
How did you solve?

Hey @nikhiljaiswal !

M2M uses sinusoidal position embeds, so it's fine if those weights are missing.
To fix this, you could either add those keys in `ignore_keys` list [here](https://github.com/harveenchadha/transformers/blob/b89a964d3f46bff56e7652e572c63c13b703b7d8/src/transformers/models/m2m_100/convert_m2m100_original_checkpoint_to_pytorch.py#L24) or pass `strict=False` to `model.load_state_dict`

Thanks @patil-suraj for the response. It worked. I had another doubt. After converting, I got config.json  & pytorch_model.bin files. Now to load the model, I am using 
from transformers import M2M100ForConditionalGeneration, M2M100Tokenizer
model = M2M100ForConditionalGeneration.from_pretrained("pytorch_model.bin")

I am facing error. Is not this the way to load the model?

you should pass the path to the directory containing the `config.json` and `pytorch_model.bin` files to the `from_pretrained` method.

@patil-suraj that worked, thanks. Now when I am trying to load the finetuned m2m model  on my custom data, I am facing new error while converting the checkpoint

AttributeError: 'NoneType' object has no attribute 'encoder_layers'

@patil-suraj can you please help me with above problem?