Multi-task dataset mixing 
 It seems like many of the best performing models on the GLUE benchmark make some use of multitask learning (simultaneous training on multiple tasks).

The [T5 paper](https://arxiv.org/pdf/1910.10683.pdf) highlights multiple ways of mixing the tasks together during finetuning:
- **Examples-proportional mixing** - sample from tasks proportionally to their dataset size
- **Equal mixing** - sample uniformly from each task
- **Temperature-scaled mixing** - The generalized approach used by multilingual BERT which uses a temperature T, where the mixing rate of each task is raised to the power 1/T and renormalized. When T=1 this is equivalent to equal mixing, and becomes closer to equal mixing with increasing T.

Following this discussion https://github.com/huggingface/transformers/issues/4340 in [transformers](https://github.com/huggingface/transformers), @enzoampil suggested that the `nlp` library might be a better place for this functionality.

Some method for combining datasets could be implemented ,e.g.
```
dataset = nlp.load_multitask(['squad','imdb','cnn_dm'], temperature=2.0, ...)
```

We would need a few additions:
- Method of identifying the tasks - how can we support adding a string to each task as an identifier: e.g. 'summarisation: '?
- Method of combining the metrics - a standard approach is to use the specific metric for each task and add them together for a combined score.

It would be great to support common use cases such as pretraining on the GLUE benchmark before fine-tuning on each GLUE task in turn. 

I'm willing to write bits/most of this I just need some guidance on the interface and other library details so I can integrate it properly.

 
 From thinking upon @thomwolf 's suggestion, I've started experimenting:
```python
class MultitaskDataset(DatasetBuilder):
    def __init__(self, *args, **kwargs):
        super(MultitaskDataset, self).__init__(*args, **kwargs)
        self._datasets = kwargs.get("datasets")

    def _info(self):
        return nlp.DatasetInfo(
            description=_DESCRIPTION,
            features=nlp.Features({
                    "source": nlp.Value("string"),
                    "target": nlp.Sequence(nlp.Value("string"))
                })
        )

    def _get_common_splits(self):
        '''Finds the common splits present in all self._datasets'''
        min_set = None
        for dataset in self._datasets:
            if min_set != None:
                min_set.intersection(set(dataset.keys()))
            else:
                min_set = set(dataset.keys())
        return min_set

....

# Maybe this?:
squad = nlp.load_dataset("squad")
cnn_dm = nlp.load_dataset("cnn_dailymail","3.0.0")
multitask_dataset = nlp.load_dataset(
    'multitask_dataset',
    datasets=[squad,cnn_dailymail], 
    k=1000, 
    t=2.0
)

```

Does anyone know what methods of `MultitaskDataset` I would need to implement? Maybe `as_dataset` and `download_and_prepare`? Most of these should be just calling the methods of the sub-datasets. 

I'm assuming DatasetBuilder is better than the more specific `GeneratorBasedBuilder`, `BeamBasedBuilder`, etc....

One of the other problems is that the dataset size is unknown till you construct it (as you can pick the sub-datasets). Am hoping not to need to make changes to `nlp.load_dataset` just for this class.

I'd appreciate it if anyone more familiar with nlp's internal workings could tell me if I'm on the right track!