Support multitask learning
Author: ghomasHudson
Pull request: no

# 🚀 Feature request

There should be an easy way to support multitask learning.

## Motivation

It seems like many of the best performing models on the GLUE benchmark make some use of multitask learning (simultaneous training on multiple tasks).

The [T5 paper](https://arxiv.org/pdf/1910.10683.pdf) highlights multiple ways of mixing the tasks together during finetuning:
- **Examples-proportional mixing** - sample from tasks proportionally to their dataset size
- **Equal mixing** - sample uniformly from each task
- **Temperature-scaled mixing** - The generalized approach used by multilingual BERT which uses a temperature T, where the mixing rate of each task is raised to the power 1/T and renormalized. When T=1 this is equivalent to equal mixing, and becomes closer to equal mixing with increasing T.

This definitely seems like a reusable component that would allow replication of the headline results used by many models.

The `run_glue.py` example only trains using a single GLUE task at a time, so I'm assuming people have made their own modifications to allow multitask learning. It seems especially sensible that there should be a way of training the T5 model in the multitask setting as was originally intended by the authors.

Maybe this could be an extension of the Trainer or a wrapper around multiple Datasets? Or even just an example.

## Your contribution

I can certainly help with implementation, though would need guidance on the best place to add this functionality.


Comments:
Having thought about this more, it would probably make most sense as an example of a PyTorch `Sampler`which implements temperature-scaled mixing on `ConcatDataset([dataset1, dataset2,...])`.

I'll attempt it using the t5 model firstly, and contribute it back if successful.

Hi, @ghomasHudson,
This seems like a great addition to hugging face, I have been looking around as well for multi-task learning. Most libraries out there don't support an easy way to achieve this and a lot of people seem to request this feature. 

I believe there are two way to achieve multitask learning:
1) As you mentioned, treating i/p & o/p as  "text to text" and most problems can fit in this , so the only change I guess is at the data sampler level. It should be easy to support
2) Another common way is ,having multiple "heads" for different tasks, and each task has a shared bert. So, essentially bert is learning on different tasks. There is no easy way to abstract things out for this in hugging face for this yet. 

As you are trying out 1). Do you think it's worth investing time in the second approach too?
I have this weekend free, might as well raise a PR for the same .

@ghomasHudson I think these forms of *multi-task dataset mixing* can also be integrated straight into the new `nlp` library.

@avhirupc I've been focusing on 1. as it seems to be the simplest (and currently the best performing on GLUE). I think 2. would still rely on this and extend it further (choosing which head to train based on the task, and a bit of modelling work). 

Multitask learning seems such an obvious part of current NLP approaches so I'm surprised more people aren't requesting it (Maybe it's more of a research-y aim than a production one?)

My current approach I'm working on is simply using `ConcatDataset` with weights decided using Temperature-scaled mixing. Something like this:

``` python
def temperature_to_weights(dataset_lengths, temperature=2.0, maximum=None, scale=1.0):
    '''Calculate mixing rates'''
    mixing_rates = []
    for length in dataset_lengths:
        rate = length * scale
        if maximum:
            rate = min(rate,maximum)
        if temperature != 1.0
            rate = rate ** (1.0/temperature)
        mixing_rates.append(rate)
    return mixing_rates

datasets = [Dataset1(), Dataset2()]
dataset_lengths = [len(d) for d in datasets]
dataset_weights = temperature_to_weights(dataset_lengths)

# Calculate weights per sample
weights = []
for i in range(len(datasets)):
    weights += [dataset_weights[i]] * len(datasets[i])

dataloader = Dataloader(ConcatDataset(datasets),
                        sampler=WeightedRandomSampler(
                                           num_samples=min(dataset_lengths),
                                           weight=weights,
                                           replacement=False) 
                        )
```

There's still a few things I'm unclear about (e.g. what should the `num_samples` be? clearly if we sample everything it's just the same as not doing any balancing at all). Would be nice to have a `MultitaskSampler` if I can work it out.

@enzoampil I'm open to ideas about the best way to do this in terms of interfacing with the library. Should properly open an issue over at [nlp](https://github.com/huggingface/nlp).

Hi @patrickvonplaten & @thomwolf , 

Do you think working on 2. is a good enhancement to hugging face. Multitask learning is common in industry as well as research. Similar to MT-DNN (https://arxiv.org/pdf/1901.11504.pdf)

If you feel it is a good enhancement to hugging face, I could work on it and raise a PR for the same.

Summarising:
Having task-specific heads eg classification,  token classification, and an ability to train multiple tasks at once? 

Hi everybody, 

I think the `nlp` library is the best place for this as @enzoampil said.