Before revision:  In this paper, we propose to replace all but one attention head of each encoder layer with  fixed -- non-learnable -- attentive patterns that are solely based on position and do not require any external knowledge.
After revision:  In this paper, we propose to replace all but one attention head of each encoder layer with simple fixed -- non-learnable -- attentive patterns that are solely based on position and do not require any external knowledge.