Before revision:  In this paper, we theoretically study the interplay between depth and width in self-attention , and shed light on the root of the above phenomenon.
After revision:  We theoretically predict a width-dependent transition between depth-efficiency and depth-inefficiency in self-attention , and shed light on the root of the above phenomenon.