  While multi-task learning (MTL) has been successfully applied in several
domains, it still triggers challenges. As a consequence of negative transfer,
simultaneously learning several tasks can lead to unexpectedly poor results. A
key factor contributing to this undesirable behavior is the problem of
conflicting gradients. In this paper, we propose a novel approach for MTL,
Rotograd, which homogenizes the gradient directions across all tasks by
rotating their shared representation. Our algorithm is formalized as a
Stackelberg game, which allows us to provide stability guarantees. Rotograd can
be transparently combined with task-weighting approaches (e.g., GradNorm) to
mitigate negative transfer, resulting in a robust learning process. Thorough
empirical evaluation on several architectures (e.g., ResNet) and datasets
(e.g., CIFAR) verifies our theoretical results, and shows that Rotograd
outperforms previous approaches. A Pytorch implementation can be found in
https://github.com/adrianjav/rotograd .
