Multitask learning has been applied successfully to a range of tasks, mostly morphosyntactic. However, little is known on when MTL works and whether there are data characteristics that help to determine its success. In this paper we evaluate a range of semantic sequence labeling tasks in a MTL setup. We examine different auxiliary tasks, amongst which a novel setup, and correlate their impact to datadependent conditions. Our results show that MTL is not always effective, significant improvements are obtained only for 1 out of 5 tasks. When successful, auxiliary tasks with compact and more uniform label distributions are preferable.
Even though auxiliary co-training is beneficial for learning in general, "little is known on when multitask learning works and whether there are data characteristics that help to determine its success" REF .
When is multitask learning effective? Semantic sequence prediction under varying data conditions
arXiv: Computation and Language
Computer Science