Venue: NeurIPS 2022 Conference

Reviews:

final_score: rating: 5: Borderline accept: Technically solid paper where reasons to accept outweigh reasons to reject, e.g., limited evaluation. Please use sparingly.
review_content: summary: This paper focuses on studies about the rank collapse phenomenon of transformers regarding its causes and effects. Authors first show that rank collapse has side effect on training by causing vanishing gradients of the queries and keys of a Transformer. Then the causes of this phenomenon is described and a solution is proposed to prevent it via depth dependent scaling of the residual branches. Finally, preliminary evidences are given about high efficacy of Adam compared to SGD in training Transformers.
strengths_and_weaknesses: Strengths:

+ Studying the rank collapse phenomenon of transformers is an interesting topic. It is important for training depth and large Transformer models, which is the key for popular foundation models. 

+ Thorough theoretical analysis are provided to describe the cause of this phenomenon and potential solution to prevent it. Assumptions and limitations are also described clearly. 

+ Experimental evidences are also given to validate the theoretical analysis.


Weakness:

- Experimental evidences are mainly provided for natural language processing (NLP) area. It is not clear whether the conclusions can be generalized to computer vision tasks, e.g. the most basic image classification. 

- The effectiveness of the proposed depth dependent residual branch scaling seems not linked with evaluation metrics. Does this solution help training deeper Transformer models to achieve better performance for NLP and computer vision models? 
questions: Please refer to the weakness part in the Strengths And Weaknesses section above.
limitations: The limitations and assumptions are discussed in Section 3 of the paper.
review_title: null

final_score: rating: 7: Accept: Technically solid paper, with high impact on at least one sub-area, or moderate-to-high impact on more than one areas, with good-to-excellent evaluation, resources, reproducibility, and no unaddressed ethical considerations.
review_content: summary: This paper studies the rank collapse problem of the tokens' representations at initialization in Transformer. The authors provide theoretical analysis to further understand and deal with this problem. They conclude that the rank collapse problem can cause the gradient vanishing problem during training and propose that applying an appropriate depth dependent scaling of the residual branches can prevent the rank collapse problem. Besides theoretical analysis, the authors also provide some experimental results to demonstrate the effectiveness of their proposed remedy method. 
strengths_and_weaknesses: Strengths:
1. This paper studies a novel problem, rank collapse, to further understand the optimization of Transformer. They investigate how the rank collapse hinders the training process of Transformer from the aspect of gradient vanishing, which has not yet been well studied for new architectures, such as Transformer. 
2. The authors propose a remedy method, an appropriate depth dependent scaling of the residual branches, to avoid the rank collapse problem. The experimental results demonstrate the effectiveness of this approach, as shown in Table 5 and Figure 1. 
3. This paper brings a new problem to studying the signal propagation in Transformers. This would motivate other researchers to further study and improve the optimization of Transformer models.
4. This paper introduces the problems, their analysis, and their solutions clearly. The paper is well written and easy to follow. The authors present their theoretical analysis clearly and easy to be understood. The notations and equations are easy to read. 

Weaknesses:
1. Some theoretical analyses are based on the previous work, Dong et al. 2021. This would make the novelty of this paper a little bit trivial.
2. Section 3.4 seems to have less relationship with the main topic of this paper. To my understanding, it is redundant.
3. The authors only provide some results on NLP tasks. I wonder if this problem is still severe and if we can address it in the same way for vision transformers.  
questions: 1. In line 31, you mentioned Dong et.al 2021 study the rank collapse problem in the absence of residual connections, I wonder whether it is proper or what the problem is for the requirement? 
2. What is the Depth in Figures 3 and 4? 

limitations: N/A
review_title: null

final_score: rating: 4: Borderline reject: Technically solid paper where reasons to reject, e.g., limited evaluation, outweigh reasons to accept, e.g., good evaluation. Please use sparingly.
review_content: summary: This work investigates the rank collapse problem for transformers at the initialization, showing that it results in vanishing gradient and slow training for self-attention parameters. Based on the theoretical study, the work proposed a method to alleviate this issue via adding a temperature parameter to the self-attention. The authors claim that the work opens the way for new, well-founded, and motivating approaches to improve optimization in these models.

strengths_and_weaknesses: *Strength: the paper is well-presented and easy to follow;
The work studies an interesting problem of the rank collapsing phenomenon at initialization.  The work provides rigorous analysis for explaining why this happens under several assumptions. 
The work utilizes the understanding, showing it can be useful for more efficient training of deep networks.

*Weakness: The assumption is quite strong (Assumption 3.1), and the sigma is linear. These do not hold in general in practice deep networks. Because of the significant simplification, It is unclear about the connection between theory and practice. 
To deal with the vanishing gradient problem caused by rank collapse, the author proposed to add a temperature scaling to the self-attention. It is very vague how the theory applies such a temperature would help with training. And it is unclear how to choose its size.

questions: The reviewer’s questions relate to the weakness of this work.
1. Although the authors justify Assumption 3.1 to some extent. The paper assumes linear activation, which does not hold in practice. How does the theory well connect to the practice? 
2. Some of the argument is in the setting where layers or dimensions are infinite. How do these results connect to the finite sample case?
3. The major question is about Section 3.4. It is unclear (at least not well explained) from the theory how a temperature can be helpful. And how to choose the size of the temperature parameter?

limitations: n/a
review_title: null

Meta-review: metareview: The paper presents a theoretical analysis of "rank collapse" in transformers. The analysis is backed by several experiments on a (En-De) machine translation task. The authors do not study other application domains, but results can be expected to be highly correlated, as those experiments cover encoder-decoder architectures. The authors make and test (re-)scaling recommendations (residual scaling and a temperature parameter). While not new in themselves, those recommendations are in line with their analysis, and they allow them to train transformers to strong performance with SGD. Overall, the paper represents an increment addition to the understanding of transformer optimization and is suitable for publication.
confidence: Less certain
Decision: Accept