Allow training from multiple languages for multilingual seq2seq models (varying forced_bos_token_id)
Author: nfortescue
Pull request: no

# 🚀 Feature request

Allow mBART and M2M100 to be easily fine-tuned with multiple target languages in the fine-tuning data set, probably by allowing forced_bos_token_id to be provided in the training dataset.

## Motivation

<!-- Please outline the motivation for the proposal. Is your feature request
     related to a problem? e.g., I'm always frustrated when [...]. If this is related
     to another GitHub issue, please link here too. -->

A number of multilingual models already exist in huggingface transformers (m2m100, mBART, mt5). These can do translation to and from multiple languages. However, at the moment, m2m100 and mBART can only easily be fine-tuned using training data with a single target language. 

This is because mBART and M2M100 required a "forced beginning of sequence token" [`forced_bos_token_id`](https://huggingface.co/docs/transformers/main_classes/model#transformers.generation_utils.GenerationMixin.generate.forced_bos_token_id) to be set indicating the target language. This is set on the model. Because of this, there is no obvious way to have different target language outputs to be used while training.

This has been asked about by multiple people independently in the discussion forums with no response yet. I've found two at this point (and I would have been a third):

- https://discuss.huggingface.co/t/m2m-model-finetuning-on-multiple-language-pairs/13203
- https://discuss.huggingface.co/t/how-to-force-bos-token-id-for-each-example-individually-in-mbart/8712

## Your contribution

<!-- Is there any way that you could help, e.g. by submitting a PR?
     Make sure to read the CONTRIBUTING.MD readme:
     https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md -->

I don't feel confident enough in my python or transformers expertise to contribute a pull request. However, to me it feels like this code should live in the trainer, rather than in the model. So for my own individual project I believe I have made a workaround to this by: ([Sample Colab notebook for the code below](https://colab.research.google.com/drive/11Wml-dOasQTuUYtk7dwKuU6B7bnQhwAq?usp=sharing))

- adding `forced_bos_token_id` as a column in my dataset, with one entry for each training example
- subclassing Seq2SeqTrainer and making the following changes:
  - Overriding `prediction_step(...)` with a copy and paste of the original code, and adding code to read forced_bos_token_id from inputs, and add it as an argument in `generated_tokens = self.model.generate(**generation_inputs, **gen_kwargs, forced_bos_token_id = forced_bos_token_id)`
  - Overriding `_remove_unused_columns()` to be a no-op, so the forced_bos_token_id doesn't get removed (as it isn't in the model signature, as it isn't a parameter for the model)
  - Overriding `compute_loss()` to have an `inputs.pop("forced_bos_token_id")` to prevent this unexpected input breaking the forward step

This seems to run, and I hope it is working, but I could have easily made a simple error. And all the copy and pasting of code makes this very fragile, which is why it would be nicer to have it in the transformers library.

Comments:
Hi @nfortescue ! Thanks for the detailed issue!

The example scripts are intended to be simple to follow and adapt instead of trying to cover every use case. Training a multi-lingual model is also a little more complicated than just setting `forced_bos_token_id`. For example, when you have a dataset with multiple languages it's also important to correctly sample the examples per language especially if the number of examples per language is not balanced. And there are different ways of doing this. So for this use case, it would be better to just fork the training script and modify it according to your needs.

And if you have a working example, then would be awesome if you could share it with [community-examples](https://huggingface.co/docs/transformers/master/en/community#community-notebooks). Thanks!

Also cc @sgugger 

Understood all this. My understanding is you can't do this just by forking the training script. Even doing this, you still need to  fork the Seq2SeqTrainer code (unless I've missed something). This fork to the Seq2SeqTrainer code is the part that makes this a maintenance difficulty.

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/master/CONTRIBUTING.md) are likely to be ignored.

> Hi @nfortescue ! Thanks for the detailed issue!
> 
> The example scripts are intended to be simple to follow and adapt instead of trying to cover every use case. Training a multi-lingual model is also a little more complicated than just setting `forced_bos_token_id`. For example, when you have a dataset with multiple languages it's also important to correctly sample the examples per language especially if the number of examples per language is not balanced. And there are different ways of doing this. So for this use case, it would be better to just fork the training script and modify it according to your needs.
> 
> And if you have a working example, then would be awesome if you could share it with [community-examples](https://huggingface.co/docs/transformers/master/en/community#community-notebooks). Thanks!
> 
> Also cc @sgugger

Hi, it has been more than 1.5 years since this issue was originally raised, especially with the rise in interest in multilingual models, do you think support for multilingual training can now be slated to be introduced in huggingface transformers? 
I think this would be very important to allow multilingual models to effectively stay multilingual even after finetuning.

Unfortunately, I am still relatively new and don't have sufficient experience and expertise to contribute to a pull request.

@patil-suraj @sgugger 

Hey! I think a lot of cool new models are now multilingual, and don't require this complexe scheme. And again, the examples are meant to be adapted for each person's needs, mostly you can use it on a multilingual dataset and train llama! 
If you need help for this, your question should be asked on the [forum](https://discuss.huggingface.co/) I'm sure the community will be of help!

Thanks!