Do Text-to-Text Multi-Task Learners Suffer from Task Conflict? (2022)

Abstract: Traditional multi-task learning architectures learn a single model across multiple tasks through a shared encoder followed by task-specific decoders. Learning these models often requires specialized training algorithms that address task-conflict in the shared parameter updates, which otherwise can lead to negative transfer. A new type of multi-task learning within NLP homogenizes multi-task architectures as a shared encoder and language model decoder, which does surprisingly well across a range of diverse tasks. Does this new architecture suffer from task-conflicts that require specialized training algorithms? We study how certain factors in the shift towards text-to-text models affects multi-task conflict and negative transfer, finding that both directional conflict and transfer are surprisingly constant across architectures.

## introduction multi-task learning (mtl) aims to obtain superior results by training a single model over multiple related tasks @xcite . despite mtl's promises, training neural networks with a multitask objective often yields worse results on a given task than task-specific models, resulting in negative transfer. why do some mtl settings work while others suffer from negative transfer? many attribute this behavior to optimization challenges in the mtl loss surface @xcite , that arise from multi-task conflict: significant differences across the gradients of different tasks which may trap sgd in poor optima @xcite . prior research has focused on methods to mitigate conflict between tasks during training to ensure consistent mtl improvements @xcite @xcite . research on conflict in mtl has largely focused on the canonical mtl setting: all tasks share an encoder which projects inputs into a shared representation space, and then task-predictions are made using a task-specific head. recently, it has become common in nlp to leverage a unified textto-text mtl framework: all tasks are framed as sequence generation problems using a single, fullyshared language model to make predictions for each task (figure 1 ; @xcite . this framing relaxes several constraints imposed in the canonical setting: text-totext models impose almost no constraints on the output space, and leave the model to infer the task via natural language prompts. despite this flexibility, text-to-text models, in combination with unsupervised pre-training, are strong multi-task learners @xcite . in this work, we are interested in the following question: how does reframing mtl with a single text-to-text model affect negative (or positive) transfer, specifically with regard to multi-task conflict in the training objective? we explore this through the following contributions: overall, our findings suggest that, despite having a fully-shared parameterization between tasks, textto-text models are surprisingly not inherently superior multi-task learners than canonical multi-task architectures and may similarly benefit from research in conflict mitigation techniques. 2 background & related work 2.1 canonical multi-task learning suppose we have k tasks, each with a dataset d k over x × y k , where x is all sequences of tokens from a fixed vocabulary and y k is a task-specific output space. our goal is to learn task-specific models f k : @xmath0, without any prior knowledge about how tasks are related. canonically, this is done by training a shared encoder g θ : @xmath1: r @xmath2, to project the input into a shared representation space before predicting into y k . 1 parameters θ are shared across all tasks, parameters ϕ k are specific to task k, and we specify f k (x) ≡ (h to train a canonical multi-task model, we first define task-specific loss functions ℓ k (ŷ, y); for instance, ℓ k may be mean-squared error for a regression task or cross-entropy for a classification task. let l k (θ) represent the average task loss ℓ k (f (x; θ), y) over (x, y) pairs from a mini-batch of d k , where θ = {θ, ϕ 1 , . . . ϕ k } for convenience. at timestep t, we update our parameters with sgd using the gradient update ∇ ϕ k l k (θ t ) for all parameters ϕ k , and canonical architectures have long been the dominant approach to deep multi-task learning in deep neural networks @xcite . classically, multi-task conflict occurs in the multitask gradient update ∇ m t θ , and negative transfer arises from difficulties in learning a proper shared encoder model g θ . ## factors 1 & 2 models we use t5 @xcite , pretrained on the c4 language modeling task, as the base of our models. a pre-trained t5 model can provide rich representations of inputs which can be passed to canonical mtl heads @xcite or a language model decoder for text-to-text learning. our canonical models simply attach task-specific canonical heads to the outputs of a shared t5 encoder, and jointly fine-tune all task-specific and shared parameters. for more details on the canonical head and loss for each task, see appendix a. to move from canonical models to models exhibiting factor 1, we replace each task-specific head with a pre-trained t5 decoder head; that is, for all tasks k we replace h ϕ k : r @xmath3: r @xmath4. note that each decoder is still parameterized with task-specific parameters ϕ k , and therefore each task is still specified by which decoder is used at test-time. we refer to this architecture as text-to-text-id: models which treat all tasks as text-to-text problems, but which still leverage independent heads for task specification. foot_5 factor 2 requires a unified output space for each task in order to remove task-specific parameters. to test the effects of factor 2 on mtl, we start from factor 1 models (text-to-text-id) and combine all task heads into a single decoder head shared across all tasks, j θ d . this yields a fully unified text-to-text architecture, equivalent to the original t5 mtl setting @xcite . factor 2 removes task-specific parameters from the model, and therefore we must specify tasks at the input. for the following experiments we utilize the default prompts and output spaces listed in appendix b; in §5 we explore how factor 3 (different task prompts and output spaces) affect our results. ## factor 3: task prompts & labels we have seen that factors 1 & 2 have little effect on transfer and task conflict, in our settings. we now turn to factor 3: the form of p k and r k . in text-totext mtl, tasks are specified via natural language prompts which can range in semantic richness from nondescript tokens, such as "sst: ", to descriptions of the task: "is the sentiment of this review positive or negative?: ". in theory, a t2t multi-task model may leverage the semantics of task descriptions to implicitly align related tasks during learning, especially when leveraging a pre-trained model which exhibits non-trivial prompt understanding. however, @xcite demonstrated that t5 could be fine-tuned with null prompts and retain competitive accuracy, suggesting that models prefer to memorize prompts regardless of their semantic information. if t2t multi-task models memorize task prompts, the form of p k and r k should have no impact on task conflict. ## discussion conclusions general purpose text-to-text architectures, typically pre-trained on large amounts of data, are increasingly popular both as a topic of academic study and in commercial nlp applications. by using natural language prompts rather than taskspecific parameters, models like t5 can encode multiple tasks in the same framework with fully shared parameters. empirically, text-to-text models often achieve comparable or superior performance compared to task-specific models. surprisingly, we find similar task conflict in both canonical and text-to-text models, with similar negative transfer. this begs the question: are text-to-text models fundamentally better multi-task learners than canonical mtl models? or is their success largely attributable to other factors, such as increased model capacity and pre-training on more data? our findings suggest the latter: text-to-text architectures are not inherently better multi-task learners than canonical models. in particular, when controlling for other factors, we have found that t5 exhibits comparable transfer characteristics to canonical mtl parameterizations, both in terms of performance on held-out data and over the course of optimization by measuring rates of gradient conflict. ## limitations a foremost limitation of our work is that we consider only a single language, english, largely due to the access of both strong text-to-text models for english as well as several english multi-task benchmarks. while we expect that our findings regarding multi-task learning are generalizable across languages, further experiments involving multi-task benchmarks and pre-trained encoders in other languages are necessary to fully support this claim. our work is additionally limited by the multitask settings we examine. while we study two very popular nlp mtl benchmarks, glue and de-canlp, both have limitations and prior work has shown wide variance in how well mtl works on different datasets. notably, the canonical output space of all tasks is fairly limited, e.g. our work overall considers regression, classification, spanlabeling, and sequence output spaces. however, other classical nlp output spaces are also worth studying, such as constrained structured prediction in named entity recognition or syntax parsing. finally, our mtl settings consider only 9 tasks to be learned jointly. a smaller universe of tasks enables us to fairly compare single-and multi-task models across several different architectures. recent work has begun to consider massively multi-task models on 100+ tasks simultaneously @xcite . while we expect that our findings will generalize to these extreme settings, expanding our experiments to larger universes of tasks would validate this claim. 8 adina williams, nikita nangia, and samuel r. bowman. 2018. a broad-coverage challenge corpus for sentence understanding through inference. derrick xin, behrooz ghorbani, ankush garg, orhan firat, and justin gilmer. 2022. do current multi-task optimization methods in deep learning even help? tianhe yu, saurabh kumar, abhishek gupta, sergey levine, karol hausman, and chelsea finn. 2020. gradient surgery for multi-task learning. victor zhong, caiming xiong, and richard socher. 2018. seq2sql: generating structured queries from natural language using reinforcement learning.