Question: How does their model improve interpretability compared to softmax transformers?
Answer: the attention heads in the proposed adaptively sparse Transformer can specialize more and with higher confidence
Alternative answer: We introduce sparse attention into the Transformer architecture