Dynamic Sampling Strategies for Multi-Task Reading Comprehension (2020)

Abstract: Building general reading comprehension systems, capable of solving multiple datasets at the same time, is a recent aspirational goal in the research community. Prior work has focused on model architecture or generalization to held out datasets, and largely passed over the particulars of the multi-task learning set up. We show that a simple dynamic sampling strategy, selecting instances for training proportional to the multi-task model’s current performance on a dataset relative to its single task performance, gives substantive gains over prior multi-task sampling strategies, mitigating the catastrophic forgetting that is common in multi-task learning. We also demonstrate that allowing instances of different tasks to be interleaved as much as possible between each epoch and batch has a clear benefit in multitask performance over forcing task homogeneity at the epoch or batch level. Our final model shows greatly increased performance over the best model on ORB, a recently-released multitask reading comprehension benchmark.

## introduction building multi-task reading comprehension systems has received significant attention and been a focus of active research @xcite . these approaches mostly focus on model architecture improvements or generalizability to new tasks or domains. while these contributions are important, it is also important to explore the optimal way to structure training; as we will show, training on instances from diverse datasets (tasks) means that unlike in a single-task setting, ample instances from each task distribution must be represented during training to properly capture that diversity. we explore 2 fundamental aspects of structuring multi-task training: how many instances are sampled from each task per epoch and how those instances are organized within the epoch. we investigate the importance of this structuring by training a multi-task model on the 8 datasets from orb @xcite , a recent multi-task reading comprehension benchmark. we first explore the sampling distribution over datasets at each epoch: how many instances from each dataset should be used to train. prior work has typically either made this a uniform distribution over datasets (implicitly favoring smaller datasets), a distribution proportional to the sizes of the datasets (implicitly favoring larger datasets), or some combination of the two. because these sampling strategies favor some datasets over others, they can lead to catastrophic forgetting in the non-favored datasets. we introduce a dynamic sampling strategy that selects instances from a dataset with probability proportional to the gap between its current performance on some metric (like em or f1 score) and measured single-task performance of the same model on that dataset. by adjusting the sampling distribution over the course of training according to what the model is learning, this method is able to mitigate the catastrophic forgetting that is observed with other sampling strategies. next we explore the impact of within-epoch scheduling strategies: once a set of instances has been selected for training, how should they be ordered and batched together? we explore three different strategies: partitioning, homogeneous batches, and heterogeneous batches. we observe a steady increase in performance as instances from different datasets become more and more interleaved within an epoch (less partitioned) and batches are more heterogeneous. this suggests that more variety in batches aids convergence when performing gradient descent steps as opposed to steps using homogeneous batches which only update the model with respect to one task at a time. partitioning also yields poorer performance since it does not allow the model to see the least recent tasks later in the epoch which leads to catastrophic forgetting on those tasks. we empirically evaluate these various training strategies on orb, a recent multi-task reading comprehension benchmark: we take the previous best published model and retrain it using dynamic sampling and heterogeneous batches, leading to a performance increase averaging about 12 points em and 9 points f1 per task. while we only evaluate on reading comprehension, the methods we present are quite general and can be applied to any multitask learning setting. ## sampling and scheduling strategies we explore two main dimensions along which the instances are ordered in multi-task learning: (1) instance sampling from each dataset to get a collection of examples to use for an epoch; and (2) within-epoch scheduling of those instances, determining how they should be ordered and batched. the key consideration for these various strategies is avoiding a phenomenon similar to "catastrophic forgetting" @xcite , where performance on a specific dataset in an unbalanced training set can drop dramatically when training moves on from that dataset. ## conclusions our goal was to investigate which instance sampling method and epoch scheduling strategy gives optimal performance in a multi-task reading comprehension setting. the results suggest that dynamic sampling-sampling instances from each task based on their respective metric differentialsis a fruitful direction to explore for improving performance. we also show that interleaving instances from different tasks within each epoch and forming heterogeneous batches is crucial for optimizing multi-task performance. it is also worth noting that for the duorc, narrativeqa, squad, and quoref datasets there are cases where the multi-task model outperforms the single-task model. this suggests that for specific cases, we observe an effect similar to data augmentation (like exposure to squad benefitting quoref performance as mentioned above) but this needs to be explored further. we hope that future work experiments further with dynamic sampling such as by modifying the metric (e.g., using bleu or rouge score if applicable) and/or modifying other values like number of instances per epoch based on performance metrics (not only does this effectively change learning rate, but it would also allow the model to update the sampling distribution more or less frequently).