Task: Question Generation
Input (English): Notably, disabling the first layer in the RTE task gives a significant boost, resulting in an absolute performance gain of 3.2%. However, effects of this operation vary across tasks, and for QNLI and MNLI, it produces a performance drop of up to -0.2%.
Output (English): How much is performance improved by disabling attention in certain heads?