T5 fine tune for seq2seq generation
Author: tuhinjubcse
Pull request: no

# ❓ Questions & Help

<!-- The GitHub issue tracker is primarly intended for bugs, feature requests,
     new models and benchmarks, and migration questions. For all other questions,
     we direct you to Stack Overflow (SO) where a whole community of PyTorch and
     Tensorflow enthusiast can help you out. Make sure to tag your question with the
     right deep learning framework as well as the huggingface-transformers tag: 
     https://stackoverflow.com/questions/tagged/huggingface-transformers 
     
     If your question wasn't answered after a period of time on Stack Overflow, you
     can always open a question on GitHub. You should then link to the SO question 
     that you posted.
     -->

## Details
<!-- Description of your issue -->

Hi
Is a script available for fine-tuning T5 base or large to to do seq2seq generative tasks like translation or dialog generation?

https://github.com/huggingface/transformers/blob/master/examples/run_generation.py

Doesn't seem to have T5


Comments:
@patrickvonplaten 

Not yet :-) For translation you could try to create a `run_train.py` script using the following resources:

- How to run `T5` for translation: https://github.com/huggingface/transformers/tree/master/examples/translation/t5
- How to train `Bart` for summarization (should be very similar to "How to train" T5 for translation): https://github.com/huggingface/transformers/blob/master/examples/summarization/bart/run_train.sh and https://github.com/huggingface/transformers/blob/master/examples/summarization/bart/run_bart_sum.py

For dialog generation - I would not recommend using T5, but rather a "decoder-only" model like gpt2. You could take a look at this script with implements a SOTA dialogue bot using DialoGPT from Microsoft: https://huggingface.co/microsoft/DialoGPT-medium#how-to-use

Thanks for answering @patrickvonplaten 
One more query: how to create the **data files** and **vocab file** for T5. 

If I am not wrong, it requires 4 data files and 1 vocab file. And in train.source and val.source files, each instance should have a prefix like "translate English to German: ". I prepare the data in the same format.
It gives this error.
TypeError: 'NoneType' object is not iterable
![image](https://user-images.githubusercontent.com/30004110/78461843-b2006e00-76cc-11ea-9be8-06171cad86e5.png)

@sshleifer 

in examples/transformer_base.py change line 105 to
```python
avg_loss = getattr(self.trainer, "avg_loss", 0.0)
```

@sshleifer Thanks for the reply. It didn't work. and have the same error. Below is the screenshot of the changed line.

<img width="937" alt="Screenshot 2020-04-12 at 9 17 24 PM" src="https://user-images.githubusercontent.com/30004110/79077852-8a308c00-7d04-11ea-92a2-30d9c6fb0c84.png">

@prabalbansal I think @sshleifer means line 107, so I added this patch in PR #3768

@hugoabonizio Thanks for the patch. It works.

@sshleifer @hugoabonizio When I use the model to predict for test set using the following command:

python '/content/transformers-master/examples/summarization/bart/run_bart_sum.py' --data_dir='/content/drive/My Drive/two_keywords/' --model_type=t5 --output_dir=/content/t5 --do_predict --model_name_or_path=t5-small

Error generated:
<img width="1010" alt="Screenshot 2020-04-13 at 6 18 47 PM" src="https://user-images.githubusercontent.com/30004110/79137728-8a3b9500-7db3-11ea-90e4-218cbc3e1e74.png">


> Thanks for answering @patrickvonplaten
> One more query: how to create the **data files** and **vocab file** for T5.
> 
> If I am not wrong, it requires 4 data files and 1 vocab file. And in train.source and val.source files, each instance should have a prefix like "translate English to German: ". I prepare the data in the same format.
> It gives this error.
> TypeError: 'NoneType' object is not iterable
> ![image](https://user-images.githubusercontent.com/30004110/78461843-b2006e00-76cc-11ea-9be8-06171cad86e5.png)

Hi, I didn't understand why you have to prepare the vocab file. I think the pertrained T5 and its default tokenizer will take care of the tokenization? Thanks for your response.

@MichaelZhouwang  yes we didn't need vocab file here.

> For dialog generation - I would not recommend using T5, but rather a "decoder-only" model like gpt2. You could take a look at this script with implements a SOTA dialogue bot using DialoGPT from Microsoft: https://huggingface.co/microsoft/DialoGPT-medium#how-to-use


Hi @patrickvonplaten,

I'm trying out T5 and BART for dialogue generation.  

I'm wondering why you say that it's better to just have a decoder.  Both FacebookAI's Blender and Google's Meena had encoders in their architectures.

What's the reason for decoder-only systems being better?

1) @Valdegg I think you are correct that it makes sense to use a seq2seq model.
2) We are also currently working on porting blenderbot from parlai, which was trained on dialogue. 3) 3) We have new forums at https://discuss.huggingface.co/ for discussing higher-level things like which model to use. 

The same question is there any example about training T5 to translate multiple sentences?