Question: How much is performance improved by disabling attention in certain heads?
Answer: disabling the first layer in the RTE task gives a significant boost, resulting in an absolute performance gain of 3.2%,  this operation vary across tasks