Revisiting Over-smoothing in BERT from the Perspective of Graph
Authors: Han Shi
JIAHUI GAO
Hang Xu
Xiaodan Liang
Zhenguo Li
Lingpeng Kong
Stephen M. S. Lee
James Kwok
Venue: ICLR 2022 Spotlight (2022)
Keywords: BERT
Over-smoothing
Transformer

Abstract: Recently over-smoothing phenomenon of Transformer-based models is observed in both vision and language fields. However, no existing work has delved deeper to further investigate the main cause of this phenomenon. In this work, we make the attempt to analyze the over-smoothing problem from the perspective of graph, where such problem was first discovered and explored. Intuitively, the self-attention matrix can be seen as a normalized adjacent matrix of a corresponding graph. Based on the above connection, we provide some theoretical analysis and find that layer normalization plays a key role in the over-smoothing issue of Transformer-based models. Specifically, if the standard deviation of layer normalization is sufficiently large, the output of Transformer stacks will converge to a specific low-rank subspace and result in over-smoothing. To alleviate the over-smoothing problem, we consider hierarchical fusion strategies, which combine the representations from different layers adaptively to make the output more diverse. Extensive experiment results on various data sets illustrate the effect of our fusion method.

Published as a conference paper at ICLR 2022
REVISITING OVER-SMOOTHING IN BERT FROM THE PERSPECTIVE OF GRAPH
Han Shi'; Jiahui Hang, Xu', Xiaodan Liang,
Zhenguo Li", Lingpeng Kong', Stephen M.S. Lee", James T. Kwok'
l 'Hnng Kong University of Science and Technology, 'The University of Hong Kong,
HHuawei Noah's s Ark Lab, "Sun Yat-sen University (hshiac, jamesk (cse. ust hk, (sumiless smslee/ @hku .kk hk, 1pk0cs (xu, hang, 1i. zhenguo ) @huawei. com, xd xdliang322 com
ABSTRACT
Recently ovor-smogthing phenomenon of Transformer-basee models is observed in both vision and language fields. However, no existing work has delved deeper to further investigate the main cause of this phenomenon. In this work, we make the attempt analyze the soo-smng problem from the perspective of graph, where such problem was first discovered and explored. Intuitively, the self-attention matrix can seen as a normalized adjacent matrix of a corresponding graph. Based the above connection, we provide some theoretical analysis and find that layer normalization plays a key role in the over-smootie smothhnng issue of Transformer based models. Specifically, if the standard deviation of layer normalization is sufficiently large, the output of Transformer stacks will converge to a specific low-rank subspace and result in over-smoothing. To alleviate the over-smoothing problem, we consider hierarchical fusion strategies, which combine the representations from different layers adaptively make the output more diverse. Extensive experiment results on various data sets illustrate the effect of our fusion method.
INTRODUCTION
Over the past few years, Transformer (Vaswani et al. 2017) has been widely used in various natural language processing (NLP) tasks, including text classification (Wang et al. 2018a). text translation (Ott et 2 question answering (Rajpurkar al. 2016 2018) and text generation (Brown) et al. 2020). The recent application of Trannformer in computer vision (CV) field also demonstrate the potential capacity of Transformer architecture. For instance, Transformer variants have been successfully used for image classification (Dosovitskiy et al. 202I), object detection (Carion et al., 2020) and semantic segmentation (Strudel et al.] (021). Three fundamental descendants from Transformer include BERT (Devlin et al.] (019). RoBERTa (Liu al., 2019) and ALBERT (ann ct al. 2020), which achieve ofate-of-the-art performance on a wide range of NLP tasks.
Recently, Dong et al. (2021) observes the "token uniformity" problem, which reduces the capacity of Transfoormr--asee architectures by making all token representations identical. They claim that pure self-attention (SAN) modules cause token uniformity, but they do not discuss whether the token uniformity problem still exists in Transformer blocks. On the other hand, Gong et al.) (202T) observe the "over-smoothingg" problem for ViT (Dosovitskiy et al.) 2021). in that different input patches mapped to a similar latent representation. To prevent loss of information, they introduce additional loss functions encourage diversity and successfully improve model performance by suppressing over-smoothing. Moreover, "overthinking" phenomenon, indicating that shallow representations are better than deep representations, also be observed in (Zhou al., 2020; Kaya al., 2019). As discussed in Section 3 this phenomenon has some inherent connection with over-smoothing. In this paper, we use lover-smoothing" unify the above issues, and refer this as the phenomenon that the model performance is deteriorated because different inputs are mapped to similar representation.
As the over-smoothing problem is first studied in the graph neural network (GNN) literature (Li al., 2018: Xu et al.) 2018 Zhao & Akoglu 2020). in this paper, we attempt to explore the cause of such
"Equal contribution.

---

Published as a conference paper at ICLR 2022
problem by building relationship between Transformer blocks and graphs. Specifically, we consider the self-attention matrix as the normalized adjacency matrix of a weighted graph, whose nodes are the tokens in a sentence. Furthermore, convolutional networks (Kipf & Welling 2017). Inspired by the over-smoothing problem in GNN, study over-smoothing in BERT from theoretical view via matrix projection. As opposed Doogg iet al. (202I), where the authors claim that layer normalization is irrelevant to over-smoothing, we find that layer normalization (Ba al. 2016) plays an important role over-smoothing. Specifically, we theoretically prove that, if the standard deviation in layer normalization is sufficiently large, the outputs of the Transformer stacks will converge a low-rank subspace, resulting in over-smoothing. Empirically, we verify that the conditions hold for a certain number of samples for pre-trained and fine-tuned BERT model (Devlin et al., 2019), which consistent with our above observations.
consider the inherent connection between BERT and graph
To alleviate the ovor-smn problem, we propose hierarchical fusion strategy that adaptively fuses representations from different layers. Three fusion approaches are used: (i) Coneat Fusion, (ii) Max Fusion, and (iii) Gate Fusion. The proposed method reduces the similarity between tokens and outperforms BERT baseline on the GLUE (Wang et al. 2018a), SWAG (Zellers et al.) 2018 and
SQuAD (Rajpurkar al. 2016 2018) data sets.
In summary, the contributions this paper are as follows: (i) We develop the relationship between self-attention and graph for better understanding of over-smoothing in BERT. (ii) We provide theoretical analysis on over-smoothing the BERT model, and empirically verify the theoretical results. (ii) We propose hierarchical fusion strategies that adaptively combine different layers to alleviate over-smoothing. Extensive experimental results verify our methods' effectiveness.
2 RELATED WORK
2.1 TRANSFORMER BLOCK AND CELE-ATTNNTION
Transformer block a basic component in Transformer model (Vaswani et al., 2017). Each Trans- former block consists of a self-attention layer and feed-forward layer. Let X E Rxd be the input to Transformer block, where the number of input tokens and d the embedding size. The self-attention layer output can be written as:
Attn(X) X Σ o (XWX) Wo = X + Σ A AxWW
(1)
where h is the number of heads, a is the softmax function, and W W.K. W. W e Rdxd, (where dn, d/h // is the dimension of single-head output) weight matrices for the query, key, value, and output, respectively of the kth head. In particular, the self-attention matrix
À a X (XW*)T) = TQ
(2)
in T plays key role in the self-attention laver (Park et al.
2019
Gong et al. 2019 Kovaleva et al.. we drop the scale product I/Vda
2019). As in (Yun et al. 2020: Shi et al. 2021 Dong et al. to simplify analysis.
2021
The feed-forward layer usually has two fully-connected (FC) layers with residual connection:
FF(X)= = Attn(X) + ReLU (Attn( X) + W2 + b2,
where Wi E Rdxdd W₂ Rdexd (dw is the size of the intermediate layer) are the weight matrices, and b, by are the biases. Two layer normalization (Ba et al. 2016) operations are performed after the self-attention layer and fully cully-connected layer, respectively.
2.2 OVER- -VER-SMOOTHING
In graph neural networks, over-smoothing refers to the problem that the performance deteriorates
as representations all the nodes become similar (Li et al. 2018: Xu et al. 2018 Huang et al.
2020). Its main cause is the stacked aggregation layer using the same adjacency matrix. Recently. several approaches have been proposed to alleviate the smoothngg problem. Xu et al. (2018) propose jumping knowledge network for better structure-aware representation, which flexibly
2

---

Published as a conference paper at ICLR 2022
1.0
0.85
BERT RuBERTTa ALBERT
15.5
85.5
0.8
0.84
Hinachial Fuskin
Sirilatty
0.82
14.5
GLIE
14.0
0.80
0.4
13.52
13.0
83.5
0.78
01U
0.0
layer
Layer
Layen
(a) Token-wise cosine similarity.
(b) Token similarity and error.
(c) Hierarchical fusion.
Figure 1: wvor-snoothing in BERT models.
leverages different neighborhood ranges. ResGCN (Li et al. 2019) adapts the residual connection and dilated convolution the graph convolutional network (GCN), and successfully scales the GCN to 56 layers. Zhao & Akoglu (2020) propose PairNorm, novel normalization layer, that prevents node embeddings from becoming too similar. DropEdge (Rong et al.) 2020 Huang et al., 2020) randomly removes edges from the input graph at each training epoch, and reduces the effect of over-smoothing.
Unlike graph neural networks, over-smoothing in Transformer-basee architectures has not been discussed in detail. Dong et al. (2021) introduce the "'oken-uniformity" problem for self-attention. and show that skip connections and multi-layer perceptron can mitigate this problem. However, Gong et al. (2021) still observe over-smoothing on the Vision Transformers (Dosovitskiy et al. 2021).
DOES
EXIST IN BERT?
In this section, we first explore the existence of over-smoothing in BERT, by measuring the similarity between tokens in each Transformer layer. Specifically, we use the token-wiss wise cosine similarity
(Gong al. 2021) as our similarity measure:
1
h h hh, 12,
CosSim
n(n 1)
where n is the number tokens, h; and h2 are two representations of different tokens, and || ||2 is the Euclidean norm. Following Dong et al. (2021).
WikiBio (Lebret et al.) 2016) input to the following Transformer-bast based models tuned on the SQuAD data set (Rajpurkar et al. 2): (i) BERT (Devlin et al. 2019). (ii) RoBERTa (Liu et al. 2019) and (iit) ALBERT (Lan al. 2020 For comparison, all three models are stacked with 12 blocks. We calculate each CosSim for each data sample and show the average and standard derivation of CosSim values all WikiBio data.
In the figures, layer 0 represents original input token representation, and layer 1-12 represents the corresponding transformer layers. As shown in Figure [[a] the original token representations are different from each other, while token similarities are high in the last layer. For instance, the average token-wiss wise cosine similarity of the last layer of ALBERT and RoBERTa are both larger than 90%.
To illustrate the relationship between fover-thinking and "over-smoothing" we compare the token- wise cosine similarity at each layer with the corresponding error rate. for the corresponding error rate of layer i, we use the representations from layer as the final output and fine-tune the classifier. Following Zhou et al.) (2020). we experiment with ALBERT (Lan et al., 2020) fine-tuned on the MRPC data set (Dolan & Brocketi 2005) and use their error rate results for convenience. As shown Figure [(b) layer 10 has the lowest cosine similarity and error rate. At layers 11 and 12, the tokens have larger cosine similarities, making them harder to distinguish and resulting in the performance drop. Thus "over-thinking can be explained by "over- mootshint..in"..
A direct consequence of over-smoothing is that the performance cannot be improved when the model gets deeper, since the individual tokens are no longer distinguishable. To illustrate this, we increase the number of layers in BERT to 24 while keeping the other settings. As shown in Figure [(c) the
Our implementation is based the HuggingFace's Transformers library Wolf et al. 2020
3

---

Published as a conference paper at ICLR 2022
[CLS]
worth
[SEP]
the
820
15
affort
-5
p00
105
133 300
[040 A02
(a) Graph G. (b) Adjacency matrix A.
(c) Normalized adjacency matrix
Figure 2: Illustration of self attentonnn and the corresponding graph G. For simplicity, we drop the self-loops in G.
performance of vanilla BERT cannot improve as the model gets deeper. In contrast, the proposed hierarchical fusion (as will be discussed in Section 6) consistently outperforms the baseline, and has better and better performance as the model gets deeper. Based on these observations, we conclude that the over-smoothing problem still exists in BERT.
4 RELATIONSHIP BETWEEN SELF-ATTENTION AND GRAPH
Since meothhig is first discussed in the graph neural network literature (Li et al. 2018 Zhao Akoglu 2020), we attempt to understand its cause from graph perspective in this section.
4.1 SELF-ATTENTION vs RESGCN
Given a Transformer block, construct a weighted graph 9 with the input tokens as nodes and exp( K2) the (ih entry of its adjacency matrix A. By rewriting the self-attention matrix À in ( as A., aQQ )a, = cxp(Q. K2)/ E, c Ki), A can thus be viewed as G's normalized adjacency matrix (Von Luxburg 2007). In other words, A = D-'A where D diag(d, (d1, .2..... d,) and d, Σ3 Ai, Figure 2 shows an example for the sentence "worth the effort watch." from the SST-2 data (Socher al., 2013) processed by BERT.
Note that graph convolutional network combined with residual connections (ResGCN) (Kipf & Welling 2017) be expressed follows.
ResGCN(X) = X + ReLU 1/2 -C = = X ReXU(
(3)
which has the similar form with the self-attention layer in Eq. Tb. By comparing self-attention module with ResGCN, we have the following observations: (i) Since A..; + Aj.s in general, 9 in self-attention
a directed graph; (ii) À = D in self-attention the random walk normalization (Chung & Graham 1997). while GCN usually uses the symmetric normalization version Á D D-1/2 AD-1/D11 1/2. (iii) The attention matrices constructed different Transformer layers are different, while in typical graphs, the adjacency matrices are usually static.
4.2 UNSHARED ATTENTION MATRIX SHARED ATTENTION MATRIX
As discussed in Section over-smoothing graph neural networks is mainly due the repeated aggregation operations using the same adjacency matrix. To compare the self-attention matrices (A's) at different Transformer layers, first flatten the multi-head attention and then measure the cosine similarity between Á' 's at successive layers. Experiment is performed with BERT (Devlin ctt al., 2019). RoBERTa (Liu et al. 2019) and ALBERT (Lan et al. 2020) on the WikiBio data set (Lebret et al. 2016).
4

---

Published as a conference paper at ICLR 2022
Table 1: Performance (%) the GLUE development by the original BERT (top row) and various BERT variants with different degrees of self-attention matrix sharing. Numbers in parentheses are
the layers that share the self-attention matrix (e.g., BERT (1-12) means that the A's from layers 1-12
are shared). The last column shows the FLOPs the self-attention modules.
MNLI (m/mm)
QQP
QNLI
SST-2
COLA
STS-B
MRPC
RTE
Average
FLOPs
BERT
85.4/85.8
88.2
91.5
92.9
62.1
88.8
90.4
69.0
83.8
2.7G
BERT (11-12)
84.9/85.0
88.1
91.0
93.0
89.7
91.1
70.8
84.0
2.4G
BERT (9-12)
85.3/85.1
88.1
90.1
92.9
89.3
68.5
83.7
BERT (7-12)
84.2/84.8
88.0
90.6
92.1
62.7
89.2
90.5
68.2
83.4
1.8G
BERT (5-12)
84.0/84.3
88.0
89.7
92.8
64.1
89.0
90.3
68.2
83.4
1.5G
BERT (3-12)
82.5/82.4
87.5
88.6
91.6
57.0
87.9
88.4
65.7
81.3
1.2G
BERT (1-12)
81,3/81.7
87.3
88.5
92.0
$7.7
87,4
87.5
65.0
80.9
1.1G
Figure shows the cosine similarities obtained. As can be seen, the similarities the last few layers are high while those at the first few layers are different from each other. In other words, the attention patterns the first few layers are changing, and become stable at the upper layers.
In the following, we focus on BERT and ex- plore how many layers can share the same self- attention matrix. Note that this is different from ALBERT, which shares model parameters in- stead of attention matrices. Results shown in Table I As can be seen, sharing attention matrices among the last 8 layers (i.e., layers 5- 12) does not harm model performance. This consistent with the observation in Figure 3
1.0
0.8
Cosine Similarity
0.6
0.4
Note that sharing attention matrices not only reduces the number of parameters in the self- attention module, but also makes the model more efficient by reducing the computations dur- ing training and inference. As shown in Table BERT (5-12) reduces 44.4% FLOPs in the self- attention modules compared with the vanilla BERT, while still achieving comparable average GLUE scores.
BERT
0.2
RoBERTa
ALBERT
0.0
2
3
5
6
7
8
9
10
11
Layer
Figure Consine similarity between the attention matrices Å's at layer and its next higher layer.
OVER SVER-SMMOTTING IN BERT
In this section, we analyze the over-smoothing problem in BERT theoretically, and then verify the result empirically.
5.1 THEORETICAL ANALYSIS
Our analysis is based matrix projection. We define subspace M, in which each row vector of the element this subspace is identical.
Definition Define M = (Y E Rn = eC,C E Rixd] as a subspace in gaxd, where 1, I] E | is the number of tokens and d is the dimension of token representation.
Each Y in subspace M suffers from the over-smoothing issue since the representation of each token C, which the same with each other. We define the distance between matrix H E Rund and as dM(H) == miny H - Yl where || |l is the Frobenius norm. Next, we investigate the distance between the output layer and subspace M. We have the following Lemma.
For example, in BERT, the attention matrices À's for the last layers very similar.

---

Published as a conference paper at ICLR 2022
Ho
M
Ho
H1
H1
Figure 4: The illustration svortsng problem. Recursively, H, will converge to subspace M where representation of each token is identical.
Lemma 1. For self-attention matrix Å, any H, E Raxd and 01, 02 2 we have:
dm(HW) S sdM(H),
(4)
dm(ReLU(H) (H) - dM(HH)
(5)
H 02 B) (nd (2)
dM(AH) S V (H)
(7)
where Amax the largest eigenvalue of ÀT e) A and s the largest singular value of W.
Using Lemma have the following Theorem.
Theorem 2. For BERT block with h heads, we have
d S (H))
(8)
0 is the largest element of all singular values of all Wi, is the largest eigenvalue of all A AT (I ee A for each self-attention matrix A, and σs, 02 are the minimum standard deviation for two layer normalization operations.
where v = (1 (1++5) 52(L+ (oha)/(O102)
Proof is in Appendix A Theorem 2 shows that if y < (i.e., 0102 > (1+ Xhs)), the output of layer + will be closer M than the output of layer 1. An illustration Theorem is shown in Figure Ho is the graph corresponding to the input layer. Initially, the token representations are very different (indicated by the different colors of the nodes). Recursively, H, will converge towards and all representations
to subspace M if
the same, resulting in over-smoothing.
Remark Though we only focus on the case v < 1, over-smoothing may still exist v 2 1.
As can be seen, layer normalization plays important role for the convergence rate Interestingly. Dong et al. (2021) claim that layer normalization plays no roles for token uniformity, which seems to conflict with the conclusion in Theorem 2 However, note that the matrix rank cannot indicate similarity between tokens completely because matrix rank is discrete while similarity is continuous. For instance, given two token embeddings h, and h,, the matrix [h; hj] has rank only if h; hj- In contrast, the consine similarity between tokens is
h. by
[/, |a 11a
1/2 resulting in the target subspace M' = (Y E gn D2/2eC, C [ is dependent with adjacent matrix (Huang et al. 2020 In contrast, subspace M is independent of Á thanks its random walk normalization. Thus, Theorem 2 can be applied to the vanilla BERT even though its attention matrix Á is not similar.
As discussed in Section 4.1 GCN use the symmetric normalization version À =
5.2 EMPIRICAL VERIFICATION
Theorem illustrates that the magnitude of 0 is important for over-smoothing issue. If (1+ (1 / Ahs), the output will be closer to subspace M suffered from over-smoothing, Since s is usually small due the &g f2-penalty during training (Huang et al. 2020), compare o J. with 1 for simplicity. To verify the theoretical results, we visualize o oidz in different fine-tuned BERT models. Specifically, take the development set data of STS-B (Cer et al. 2017)
neglect the effect of and

---

Published as a conference paper at ICLR 2022
30
25-
1.5
20-
1.0
9.1
15
0.0
84
0.6
12
0.0
041
0.6
1.0
1.2
mu6
(a) STS-B.
(b) CoLA.
(c) SQuAD.
Figure 5: The estimated distribution of 0102 for different fine-tuned models.
CoLA (Warstadt et al. 2019), SQuAD (Rajpurkar et al. 2016) as input to the fine-tuned models and visualize the distribution of 0102 the last layer using kernel density estimation (Rosenblat) 1956).
Results are shown in Figure 5 As can be seen, the distributions of 0102 can be very different across data sets. For STS-B (Cer et al. 2017), 0102 of all data is larger than 1, which means that over-smoothing is serious for this data set. For CoLA (Warstadt et al. 2019) and SQuAD (Rajpurkar
al. 2016), there also exists fraction of samples satistying 0102
6
METHOD
From our proof in Appendix A we figure out that the main reason is the post-normalization scheme BERT. In comparison, to train 1000-layer GCN, et al. (2021) instead apply pre-normalization with skip connections to ensure than post-normalization for layer normalization empirically (He et al. 202T). p. this section, we preserve the nost-normalization scheme and propose hierarchical fuston strategy alleviate the over-smoothing issue. Specifically, since only deep layers suffer from the over-smoothing issue, we allow the model select representations from both shallow layers and deep layers final output.
1. However, the performance of pre-normalization is not better
6.1 HIERARCHICAL FUSION STRATEGY
Concat Fusion We first consider simple and direct layer-wise Concat Fusion approach. Considering L-layer model, first concatenate the representations H& from each layer & to generate matrix [H1, Hg...., H 1 and then apply linear mapping to generate the final representation D 0g H₃- Here } are model parameters independent with inputs. Since this scheme requires preserving feature maps from all layers, the memory cost will be huge the model gets deep.
Max Fusion Inspired by the idea the widely adopted max-pooling mechanism, we construct the final output by taking the maximum value across all layers for each dimension of the representation. Max Fusion is an adaptive fusion mechanism since the model can dynamically decide the important layer for each element the representation. Max Fusion is the most flexible strategy, since it does not require learning any additional parameters and is more efficient in terms of speed and memory.
Gate Fusion Gate mechanism is commonly used for information propagation in natural language processing field (Cho et al. 2014). To exploit the advantages from different semantic levels, we propose a vertical gate fusion module, which predicts the respective importance representations from different layers and aggregate them adaptively. Given token representations {H where denotes the token index and denotes the layer index, the final representation for token / is calculated by Ek T H where 71. 12 ........ TL softmax H o( (H)).... g9( (HE)) )) Here the number layers and the gate function g(-) is fully-connected (FC) layer, which relies on the word representation itself in respective layers to predict its importance scores. The weights of the gate function g()) are shared across different layers.
tosen-wise
Even though Concat Fusion and Max Fusion have been investigated in the graph field (Xu et al. 2018), their effectiveness for pre-trained language model have not yet been explored. Besides, since the layer-wise Concat Fusion and element-wise Max Fusion lack the ability generate token representations according to each token's specificity, we further propose the token-wise Gate Fusion for adapting fusion to the language scenario.
7

---

Published as a conference paper at ICLR 2022
Table 2: Performance (in %) of the various BERT variants the GLUE development data set.
MNLI (m/mm)
QQP
QNLI
SST-2
COLA
STS-B
MRPC
RTE
Average
BERT
85.4/85.8
88.2
91.5
92.9
62.1
88.8
90.4
69.0
83.8
BERT (concat)
85.3/85.4
87.8
91.8
93.8
65.1
89.8
91.3
71.1
84.6
BERT (max)
85.3/85.6
88.5
92.0
93.7
64.6
90.3
91.7
71.5
84.7
BERT (gate)
85,4/85.7
88.4
92.3
93.9
64.0
90.3
92.0
73.9
85.1
ALBERT
81.6/82.2
85.6
90.7
90.3
50.8
89.4
91.3
75.5
81.8
ALBERT (concat)
82.8/82.8
86.7
90.9
90.7
48.7
89.7
91.5
76.5
82.3
ALBERT (max)
82.5/82.8
86.9
91.1
90.7
50.5
89.6
92.6
77.3
82.6
ALBERT (gate)
83.0/83.7
87.0
90.9
90.4
51.3
90.0
92.4
76.2
82.7
6.2 EXPERIMENT RESULTS
The BERT model is stacked with 12 Transformer blocks (Section 2.1 with the following hyper- parameters: number of tokens n 128. number of self-attention heads 12, and hidden layer size d = 768. As for the feed-forward layer, we set the filter size dæ to 3072 as in Devlin et al. (2019).
All experiments are performed on NVIDIA Tesla V100 GPUs.
6.2.1 DATA AND SETTINGS
Pre-training For the setting in pre-training phase, mainly follows BERT paper (Devlin al. 2019). Our pre-training tasks are vanilla masked language modeling (MLM) and next sentence prediction (NSP). The pre-training datasets are English BooksCorpus (Zhu al. 2013) and Wikipedia (Devlin et al. 2019) (16G in total). The WordPiece embedding (Wu et al. 2016) and the dictionary containing 30, 000 tokens (Devlin al. 2019) are still used our paper. To pre-process text, we use the special token [CLS] as the first token of each sequence and [SEP] to separate sentences in a sequence. The pre-training is performed for epochs.
Fine-tuning In the fine-tuning phase, we perform downstream experiments the GLUE (Wang al. (2088)). SWAG (Zelers et al.) 2018) and SQuAD (Rajpurkar et al.) 2016; 2018) benchmarks. GLUE is a natural language understanding benchmark, which includes three categories tasks: (i) single-sentence tasks (CoLA and SST-2); (ii) similarity and paraphrase tasks (MRPC, QQP and STS-B); (iii) inference tasks (MNLL, QNLI and RTE). For MNLI task, experiment matched (MNLI-m) and mismatched (MNLI-mm) versions. The SWAG data set is for grounded commonsense inference, while SQuAD is task for question answering. In SQuAD) v1.1 (Rajpurkar et af. 2016), the answers are included in the context. SQuAD v2.0 (Rajpurkar et al. (018) is more challenge than SQuAD v1.0, in which some answers
both the
not included the context. Following BERT (Devlin et al. 2019), we report accuracy for MNLI, QNLI, RTE, SST-2 tasks, Fl score for QQP and MRPC, Spearman correlation for STS-B, and Matthews correlation for CoLA. For SWAG task, we use accuracy for evaluation. For SQuAD v1.1 and v2.0, we report the Exact Match (EM) and FI scores. Descriptions of the data sets and details of other hyper-parameter settings are in Appendix B and in Appendix respectively.
6.2.2 RESULTS AND ANALYSIS
Since BERT (Dellin et al. 2019) and RoBERTa (Liu al. 2019) share the same architecture and the only difference is data resource and training steps, here we mainly evaluate our proposed method on BERT and ALBERT (Lan et al.) 2020). Results on the GLUE benchmark are shown in Table 2, while results on SWAG and SQuAD are illustrated in Table For SQuAD task, in contrast to BERT which (Devlin et al.] 2019) utilize the augmented training data during fine-tuning phase, we only fine-tune our model on the standard SQuAD data set. As be seen, proposed fusion strategies also perform better than baselines on various tasks consistently.
Table 3: Performance (in %) the SWAG and
SQuAD development sets.
SWAG
SQuAD vI.1
SQuAD v2.0
acc
EM
Fl
EM
Fl
BERT
81.6
79.7
87.1
72.9
75.5
BERT (coneat)
77.0
BERT (max)
81.9
80.1
87.6
73.6
76.6
BERT (gate)
82.1
80.7
88.0
73.9
77.3
Following the previous swor-smoothing measure, visualize the token-wise cosine similarity in each layer. Here we perform visualization on the same data sets as Section [.2 and the results are shown in Figure For all three data sets, the cosine similarity has drop in the last layer compared
8

---

Published as a conference paper at ICLR 2022
1,0
1.0
1.0
BERT
DCRT
DCRT BURI (gatin)
0.8
BLRI (gale)
0.8
B:R (pata))
Siiitriit
Collre Smilort
6.0
Layer Indox
Layer
(a) STS-B.
(b) CoLA.
(c) SQuAD.
Figure 6: The token-wise similarity comparison between BERT and BERT with gate fusion. Here means the final output, which is the fusion results for our approach.
00.5
0..10
10
0.05
[SSP])
wintian
[EEP)
possos
playings
Dao
barge
dagge
ramsing
Figure 7: Visualization of importance weights of gate fusion on different layers.
with baseline. It's remarkable that the similarity drop the most obvious in STS-B (Cer et 2017 which is consistent with our empirical verification that STS-B's 0202 is the largest in Section 5.2 Since the representation of tokens from prior layers not similar with each other, our fusion method alleviates the over-smoothing issue and improve the model performance at the same time.
To study the dynamic weights fusion gate strategy, visualize the importance weight I for each token i and for each layer k. We randomly select three samples and the visualization results are illustrated in Figure Note that our gate strategy will reduce vanilla model if representation from the last layer is selected for each token. As can be seen, the weight distribution of different tokens adaptively decided, illustrating that the vanilla BERT stacks model is not the best choice for all tokens. The keywords which highly affect meaning of sentences (i.e. "women", "water", "fish") ") are willing to obtain more semantic representations from the deep layer, while for some simple words which appear frequently (i.e. "a", "is "), the features in shallow layers preferred.
CONCLUSION
In this paper, revisit the over-smoothing problem in BERT models. Since this issue has been detailed discuss in graph learning field, we firstly establish the relationship between BERT and graph for inspiration, and find out that self-attention matrix be shared among last few blocks without performance drop. Inspired by over-smoothing discussion in graph convolutional network, we provide some theoretical analysis for BERT models and figure out the importance of layer normalization. Specifically, if the standard derivation of layer normalization is sufficiently large, the output will converge towards low-rank subspace. To alleviate the over-smoothing problem, we also propose a hierarchical fusion strategy to combine representations from different layers adaptively. Extensive experiment results on various data sets illustrate the effect of our fusion methods.

---

Published as a conference paper at ICLR 2022
REFERENCES
J. Ba, J. Kiros, and G. Hinton. Layer normalization. Preprint arXiv: 107.v:1607.06450, 2016.
L. Bentivogli, I. Dagan, D. Hoa, D. Giampiccolo, and B. Magnini. The Fifth PASCAL Recognizing Textual Entailment Challenge. In TAC 2009 Workshop, 2009.
T. Brown, B. Mann, N. Ryder, M. Subbiah, Kaplan, P. Dhariwal, A. Neelakantan, Shyam, G. Sastry, A. Askell, et al. Language Models are Few-Shot Learners. Preprint arXiv:2005.144 14166, 2020.
N. Carion, F. Massa, G. Synnaeve, N. Usunier, A. Kirillov, and S. Zagoruyko. End-to-End Object Detection with Transformers. In European Conference on Computer Vision, 2020.
D. Cer, M. Diab, E. Agirre, I. Lopez-Gazpio, and L. Specia. SemEval-2017 Task 1 Semantic Textual Similarity Multilingual and Crosslingual Focused Evaluation. In International Workshop on Semantic Evaluation, 2017.
Z. Chen, H. Zhang, X. Zhang, and L. Zhao. Quora Question Pairs. University of Waterloo, 2018.
K. Cho, B. van Merriënboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Bengio. Learning Phrase Representations using RNN Decoder for Statistical Machine Translation. In Empirical Methods in Natural Language Processing, 2014.
Chung and F. Graham. Spectral Graph Theory. American Mathematical Soc., 1997.
J. Devlin, M. Chang, K. Lee, and K. Toutanova, BERT: Pre-training of Deep Bidirectional Transform- for Language Understanding. In North American Chapter of the Association for Computational Linguistics, 2019,
W. Dolan and C. Brockett. Automatically Constructing a Corpus of Sentential Paraphrases. In International Workshop on Paraphrasing, 2005.
Y. Dong, J. Cordonnier, and A. Loukas. Attention is Not All You Need: Pure Attention Loses Rank Doubly Exponentially with Depth. In International Conference on Machine Learning, 2021.
A. Dosovitskiy, L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, Unterthiner, M. Dehghani, M. Minderer, G. Heigold, S. Gelly, al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. In International Conference on Learning Representations, 2021.
Gantmakher. The Theory of Matrices, Volume 2. American Mathematical Soc., 2000.
C. Gong, D. Wang, M. Li, V. Chandra, and Q. Liu. Improve Vision Transformers Training by Suppressing Over-smoothing. Preprint arXiv:2104.12753. 2021.
L. Gong, D. He, Z. Li, T. Qin, L. Wang, and T. Liu. Efficient Training of BERT by Progressively Stacking, In International Conference on Machine Learning, 2019.
R. He, A. Ravula, B. Kanagal, and J. Ainslie, Realformer: Transformer likes residual attention. In Findings of Annual Meeting of the Association for Computational Linguistics, 2021.
W. Huang, Y. Rong, T. Xu, F. Sun, and J. Huang. Tackling Over-Smoothing for General Graph Convolutional Networks. Preprint MeXiv:2008.09866 2020.
Y. Kaya, S. Hong, and Dumitras. Shallow-Den DDep Networks:
Understanding and Mitigating Network Overthinking. In International Conference on Machine Learning, 2019.
T. Kipf and M. Welling. Semi-Supervised Classification with Graph Convolutional Networks. In International Conference on Learning Representations, 2017.
O. Kovaleva, A. Romanov, A. Rogers, and A. Rumshisky, Revealing the Dark Secrets BERT, In Empirical Methods in Natural Language Processing, 2019.
Z. Lan, M. Chen, S. Goodman, K. Gimpel. P. Sharma, and R. Soricut. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. In International Conference on Learning Representations, 2020.
10

---

Published as a conference paper at ICLR 2022
R. Lebret, D. Grangier, and M. Auli. Neural Text Generation from Structured Data with Application to the Biography Domain. In Empirical Methods in Natural Language Processing, 2016.
G. Li, M. Müller, A. Thabet, and B. Ghanem. DeepGCNs: Can GCNs Go Deep as CNNs? In Internatioaa Conference on Computer Vision, 2019.
G. Li, M. Müller, B. Ghanem, and V. Koltun. Training Graph Neural Networks with 1000 Layers. In International Conference on Machine Learning, 2021.
Q. Li, Z. Han, and X. Wu. Deeper Insights into Graph Convolutional Networks for Semi-Suuervised Learning. In AAAI conference on artificial intelligence, 2018.
Y. Liu, M. Out, N. Goyal, Du, M. Joshi, D. Chen, O. Levy, Lewis, L. Zettlemoyer, and V. Stoy- anov. RoBERTa: A Robustly Optimized BERT Pretraining Approach. Preprint 1907111907.1192, 2019.
K. Oono and T. Suzuki. Graph Neural Networks Exponentially Lose Expressive Power for Node Classification. International Conference on Learning Representations, 2020.
M. Ou, S. Edunov, D. Grangier, and M. Auli. Scaling Neural Machine Translation. In Machine Translation, 2018.
C. Park, I. Na, Y. Jo, S. Shin, Yoo, B. Kwon, J. Zhao, H. Noh, Y. Lee, and Choo. SANVis: Visual Analytics for Understanding Self-Attention Networks. In IEEE Visualization Conference, 2019.
Rajpurkar, J. Zhang, Lopyrev, and Liang- SQuAD: 100,000+ Questions for Machine Comprehension of Text. In Empirical Methods in Natural Language Processing, 2016.
Rajpurkar, R. Jia, and P. Liang Know What You Don't Know: Unanswerable Questions for SQuAD. In Annual Meeting of the Association for Computational Linguistics, 2018.
Y. Rong, W. Huang, T. Xu, and Huang. DropEdge: Towards Deep Graph Convolutional Networks on Node Classification. In International Conference Learning Representations, 2020.
M. Rosenblatt. Remarks on Some Nonparametric Estimates of Density Function. Annals of Mathematical Statistics, 1956.
H. Shi, J. Gao, X. Ren, H. Xu, X. Liang, Z. Li, and J. Kwok. SparseBERT: Rethinking the Importance Analysis in Self-attention. In International Conference on Machine Learning, 2021.
R. Socher, A. Perelygin, J. Wu, J. Chuang, C. Manning, A. Ng, and C. Potts. Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank. In Empirical Methods in Natural Language Processing, 2013.
R. Strudel, R. Garcia, Laptev, and C. Schmid. Segmenter: Transformer for Semantic Segmentation.
In International Conference on Computer Vision, 2021.
A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. Gomez, L. Kaiser, and I, Polosukhin. Attention Is All You Need. In Neural Information Processing Systems, 2017.
U. Von Luxburg. A Tutorial on Spectral Clustering. Statistics and computing, 2007.
A. Wang, A. Singh, Michael, Hill, Levy, and S. Bowman. GLUE: Multi-Task Benchmark and Analysis Platform for Natural Language Understanding. In EMNLP Workshop BlackboxNLP, 2018a.
W. Wang, M. Yan, and C. Wu. Multi-Granularity Hierarchical Attention Fusion Networks for Reading Comprehension and Question Answering. In Annual Meeting of the Association for Computational Linguistics, 2018b.
A. Warstadt, A. Singh, and S. Bowman. Neural Network Acceptability Judgments. Transactions of the Association for Computational Linguistics, 2019.

---

Published as a conference paper at ICLR 2022
A. Williams, N. Nangia, and S. Bowman. A Broad-Coveriie Challenge Corpus for Sentence Understanding through Inference. In North American Chapter of the Association for Computational Linguistics, 2018.
T. Wolf, Chaumond, L. Debut, V. Sanh, C. Delangue, A. Moi, P. Cistac, M. Funtowicz, Davison, S. Shleifer, et al. Transformers: State-of-the-Art Natural Language Processing. In Empirical Methods in Natural Language Processing: System Demonstrations, 2020.
Y. Wu, M. Schuster, Z. Chen, Q. Le, M. Norouzi, W. Macherey, M. Krikun, Y. Cao, Q. Gao, K. Macherey, et al. Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. Preprint (ee.iv::609.00 144, 2016.
K. Xu, C. Li, Y. Tian, T. Sonobe, K. Kawarab, and S. Jegelka. Representation Learning on Graphs with Jumping Knowledge Networks. In International Conference Machine Learning, 2018.
C. Yun, Y. Chang, S. Bhojanapalli, Rawat, S. Reddi, and Kumar. O(n) Connections are Expressive Enough: Universal Approximability of Sparse Transformers. In Neural Information Processing Systems, 2020.
R. Zellers, Y. Bisk, R. Schwartz, and Choi. SWAG: A Large-Scale Adversarial Dataset for Grounded Commonsense Inference. In Empirical Methods in Natural Language Processing, 2018.
L. Zhao and L. Akoglu. PairNorm: Tackling Oversmoothing in GNNs. In International Conference on Learning Representations, 2020.
W. Zhou, C. Xu, T. Ge, J. McAuley, K. Xu, and F. Wei. BERT Loses Patience: Fast and Robust Inference with Early Exit. Neural Information Processing Systems, 2020.
Y. Zhu, R. Kiros, R. Zemel, R. Salakhutdinov, R. Urtasun, A. Torralba, and S. Fidler. Aligning Books and Movies: Towards Story-like Visual Explanations by Watching Movies and Reading Books. In International Conference on Computer Vision, 2015.
12

---

Published as a conference paper at ICLR 2022
A PROOF
Lemma 1. For self-attention matrix A, any H, Raxd and 0n, 02 2 0, we have:
dM(HW) S sdM(H),
(4)
dM(ReLU(H)) S dM(HH)
(5)
dm(aiH + B) S aidm((H) +22d((B),
(6)
dm(AH) S (
(7)
where Amax is the largest eigenvalue of A (I eeT) ) and s the largest singular value of W.
Proof. Here only prove the last inequality 7 the inequity is different from the theories in GCN since A is symmetric and shared in Transformer architecture. For the first three inequalities, we refer to Oono & Suzuki (2220) and Huang et al. (2020).
Write H QuQ T for the eigin-decomposiiionn of HH where Q 91 ....... ... In. the orthogonal and $2 (ilagww.......00 with all w, O. Recall e 1/2 ...., 1]" R
Note that
dm(AH) |1(I Ee T AA = -rr((l- - ee ce")eA 'AHH A -I-ee"
Σ wiq A A ee )
Since matrix ÁT ee A is positive semidefinite, its all eigenvalues are non-negative. Let Amax be the largest eigenvalue of A AT(I- ce ).. Consider
AmaxdM(HF dm (AH) -E wiq. (Amax AI -eee) - - -)
1=1
Let E Amax (I ee ce
Note that D-1AB is a stochastic matrix, we have Åe =-=e. Thus, A ee eigenvalue and corresponding eigenvecter e. Let fi normalised eigenvector of À (I - ee T) orthogonal to e, and À be its corresponding eigenvalue. Then we have
TeT)A ) has an
e
Ee 0,
f Ef Amax 220
It follows that d (AH) Am
Discussion Assume further that Å is doubly stochastic (so that ÅTe = e) with positive entries.
Then by Perron-Frobenius theorem (Gantmakher, 2000), A T Á has a maximum eigenvalue 1 with associated eigenvector e as well. this case, the matrix ÅT AT(I ee ") Á ee has maximum eigenvalue Amaz < 1.
Theorem 2. For BERT block with h heads, we have
d (M(HH)1) I (dy(HH),
(8)
where u = (1+ (1 + (ohs)/(o1og)) s > 0 is the largest element of all singular values of all Wi, is the largest eigenvalue of all Á (I ee ) for each self-attention matrix Å, and a are the minimum standard deviation for two layer normalization operations.
Proof. From the definition of self-attention and feed-forward modules, we have
H
H
(X) = LayerNorm( X
+
A * + 1bb = (X Σ A x + 1 1b D
k=
km
FF(X) = LayerNorm( X X + 16 W2 + 1b
(X ReLLUUXW 16 W2 + 1b 1b D
13

---

Published as a conference paper at ICLR 2022
Based the Lemma
we have
dm(Attn(X)) dm((x
A XW* +1b -1bix) D
S <dm(XD
dm LN
A xxXW*D dd bLN)
k=1
h So Σ
dd X
k=1
dM + VAso (x
= (1+ dM)))
dm(FF(X)) du((X + ReLU 1b, WW2 + 1b. - DIN)
s dd
+b] W2DDNN) + dd bb,N)
ddb(b)) W2DTN)
+ {088; 8877'dmM
(1 + 200 (X))) (X).
It follows that
dM(H4)) S (1 + 21 + Vihs)oiog 22 (Hi)).
B
DATA SET
B.1
MNLI
The Multi-Gent Genre Natural Language Inference (Williams et al. 2018) is a crowdsourced ternary classification task. Given premise sentence and hypothesis sentence, the target to predict whether the last sentence is an [entailment), [contradiction), or [neutral] relationships with respect to the first one.
B.2 QQP
The Quora Question Pairs (Chen et al. 2018) is a binary classification task. Given two questions on Quora, the target determine whether these two asked questions are semantically equivalent or not.
B.3 QNLI
The Question Natural Language Inference (Wang et al. 2018b) binary classification task derived from the Stanford Question Answering Dataset (Rajpurkar et al. (016). Given sentence pairs (question, sentence), the target to predict whether the last sentence contains the correct answer to the question.
B.4 SST-2
The Stanford Sentiment Treebank (Socher et al. 2013)) is binary sentiment classification task for single sentence. All sentences extracted from movie reviews with human annotations of their sentiment.
B.5 CoLA
The Corpus of Linguistic Acceptability (Warstadt al.] 2019) is binary classification task consisting of English acceptability judgments extracted from books and journal articles. Given a single sentence, the target is determine whether the sentence is linguistically acceptable not.
14

---

Published as a conference paper at ICLR 2022
B.6 STS-B
The Semantic Textual Similarity Benchmark (Cer et al. 2017) is regression task for predicting the similarity score (from 5) between given sentence pair, whose sentence pairs are drawn from news headlines and other sources.
B.7 MRPC
The Microsoft Research Paraphrase Corpus (Dolan & Brocketi 2005) is a binary classification task. Given a sentence pair extracted from online news sources, the target is to determine whether the sentences in the pair are semantically equivalent.
B.8 RTE
The Recognizing Textual Entailment (Bentivogli al. 2009) is binary entailment classification task similar to MNLI, where [neutral] and [contradiction] relationships are classified into [not entailment].
B.9 SWAG
The Situations with Adversarial Generations Zellers et al.) 2018) is multiple-choice task consisting of 113K questions about grounded situations. Given a source sentence, the task is to select the most possible one among four choices for sentence continuity.
B.10 SQUAD v1.1
The Stanford Question Answering Dataset (SQuAD v1.1) (Rajpurkar al. 2016) is large-scale scale question and answer task consisting of 100K question and answer pairs from more than 500 articles. Given passage and the question from Wikipedia, the goal determine the and the end token of the answer text.
B.11 SQUAD v2.0
The SQuAD v2.0 task (Rajpurkar et al. 2018) is the extension of above SQuAD v1.1, which contains the 100K questions in SQuAD v and 50K unanswerable questions. The existence of unanswerable question makes this task more realistic and challenging.
C IMPLEMENTATION DETAILS
The hyper-parameters of various downstream tasks are shown in Table
Table 4: Hyper-parameters for different downstream tasks.
GLUE
SWAG
SQuAD vI.1
SQuAD v2.0
Batch size
32
16
32
48
Weight decay
[0.1, 0.01]
[0.1, 0.01]
[0.1, 0.01]
[0.1, 0.01]
Warmup proportion
0.1
0.1
0.1
0.1
Learning rate decay
linear
linear
linear
linear
Training Epochs
Learning rate
[2c-5, le-5, 1.5c-5, 3c-5, 4c-5, 5c-5]
15


Reviews:
[{"review_id": "wMq841Tlxqw", "reviewer": "Reviewer_M9jn", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "This paper tries to understand the over-smoothing phenomenon of Transformer-based models such as BERT. The analysis is from the perspective of viewing BERT and Transformer as graph neural networks. By analogy and analysis of graph neural networks, authors show some theoretical analysis and find that layer\nnormalization plays a key role in the over-smoothing issue. Specifically, if the standard deviation of layer normalization is sufficiently large, the output of Transformer stacks will converge to a specific low-rank subspace\nand results in over-smoothing. Then to alleviate the over-smoothing problem, authors\nconsider hierarchical fusion strategies, which combine the representations from\ndifferent layers adaptively to make the output more diverse.\n\nAn accept has been recommended for this paper for the interestingness of the topic and the novel perspective of theoretical study of BERT by graph neural networks.  The over-smoothing study with layernorm via graph is useful to both BERT, optimization, and graph community. This will inspire some new directions for researchers and hence an accept is recommended.\n\nAuthors however need to address several concerns in the review. \n1. If LayerNorm cause issue, Why does it help then? What do you suggest to replace it, e.g., will using prenorm by Di He et al resolve the issue?\n\n2. The concat and max fusion has appeared in the graph paper that author cited. It's interesting to see this can help BERT as well and in that sense this is novel but authors should clearly indicate where this has appeared before as well.\n\n", "summary_of_the_paper": "This paper tries to understand the over-smoothing phenomenon of Transformer-based models such as BERT. The analysis is from the perspective of viewing BERT and Transformer as graph neural networks. By analogy and analysis of graph neural networks, authors show some theoretical analysis and find that layer\nnormalization plays a key role in the over-smoothing issue. Specifically, if the standard deviation of layer normalization is sufficiently large, the output of Transformer stacks will converge to a specific low-rank subspace\nand results in over-smoothing. Then to alleviate the over-smoothing problem, authors\nconsider hierarchical fusion strategies, which combine the representations from\ndifferent layers adaptively to make the output more diverse.", "summary_of_the_review": "An accept has been recommended for this paper for the interestingness of the topic and the novel perspective of theoretical study of BERT by graph neural networks. This will inspire some new directions for researchers and hence an accept is recommended. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "qut2QbQHwM5", "reviewer": "Reviewer_8SR3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n\na) The theoretical analysis on BERT is a good contribution. The analysis methods is not completely new but applicable to analysis on transformers.\n\nCons:\n\na) Experimental results shown in  Table 1 and Table 2  do not show any significant improvement to overcome oversmothing. Most of the proposed method only give a small improvement which let me ask the question whether proposed fusion method is good effective enough for oversmoothing. Can authors provide further experiments with different datasets or applications to demonstrate the effectiveness of the proposed method?\n\nb) Lack of theoretical support for the proposed fusion method. Is it possible to theoretically demonstrate that the fusion method overcome oversmoothing? ", "summary_of_the_paper": "This paper discusses the oversmoothing in transformer models such as BERT. It provides a theoretical analysis on the existence of oversmoothing in transformers and proposes a hierarchical fusion method as a solution to oversmoothing. ", "summary_of_the_review": "The analysis of oversmoothing in transformer is a good contribution. The limited empirical evidence and lack strong improvement over oversmoothing by the proposed method are the main limitations of this paper.", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "q3l-fgWNYHy", "reviewer": "Reviewer_g9XE", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "•\tThe authors have provided sufficient deductions to establish the theory of token over-smooth in deeper transformer layers. I feel convinced that layer normalization could lead to the over-smoothing issue. One question, it is not obvious to me that how theoretically fusion strategy would help prevent token over-smoothing. Could authors discuss the question in terms of v in equation 8? How does the distribution of the layer normalization term look like in the fusion models (replicate figure 5)? A further question is whether the fusion strategies improved the over smoothing issue by reducing the layer normalization relatively. \n\n•\tThe second question is about the contribution of self-attention block and feedforward block to token over-smoothing. The theory in section 5 considered both the self-attention block and the feedforward block, and the layer normalization of both blocks seem to be equally contributed to token representation smoothing. However, the shared attention test in figure 3 seems to suggest a dominant role of the self-attention block. I’m wondering how authors interpret this. ", "summary_of_the_paper": "The current manuscript explored the token over-smoothing problem in Bert transformer blocks. Empirically, the authors showed that the token over-smooth problem is increasingly severe in deeper layers of a Bert model. The token over-smoothing is potentially harmful to task performance. As the over-smoothing problems have been discussed in graph networks extensively, the authors established a connection between transformer blocks with graph networks and looked at the over-smoothing problem of transformer blocks analogously. The authors revealed that layer normalization in the transformer blocks contributed to the over-smoothing issue both theoretically and empirically. Moreover, the authors proposed to improve the over-smoothing issue by hierarchical fusion strategies, that is utilize token representation from earlier layers. Overall, the author applied a novel approach to shed light on an insufficiently investigated problem. The paper was well written.   ", "summary_of_the_review": "Overall, I vote for accepting this paper. I like the idea of establishing an analogy between self-attention and the adjacency matrix of a graph model, and I like the theory that establishes the relationship between layer normalization and dimensionality reduction of token representation. My major concern is about the clarity of some parts of the paper and some additional improvement. I hope the authors could address my concerns in the rebuttal. ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}, {"review_id": "52WoxSzVUdF", "reviewer": "Reviewer_9nYH", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "Pros:\n1. Addressing the over-smoothing problem in BERT is a promising direction, which will further the research on the study of transformers.\n2. The theoretical analysis on the over-smoothing problem in BERT from the graph perspective is new and provides interesting findings. \n\nCons:\n1. The authors claim that they propose three layer fusion methods which have been explored in previous methods in my understanding. The authors should carefully claim that the fusion methods are \"proposed\" in this paper. A sufficient survey should be conducted on layer fusion of BERT.\n2. The over-smoothing problem is neither well theoretically analyzed nor empirically proved to be addressed via layer fusion methods. \n\nQuestions:\n1. Is it possible to theoretically prove that layer fusion can alleviate the over-smoothing problem like Section 5?\n2. Is it possible to empirically verify the layer fusion methods' ability to address the over-smoothing problem on more datasets. In fact, I notice that Fig. 6 has proved cosine similarity between BERT and BERT(self-gate) on three datasets. However, the paper should provide more experimental setting information. In my understanding, since \"the last layer is replaced with the aggregated representation\", should the cosine similarity of BERT(self-gate) at the 12th layer be lower than some of the layers before (since the last layer is the aggregated representation of all the 12 layers)?", "summary_of_the_paper": "This paper explored the over-smoothing problem in BERT from the perspective of graph. In detail, empirical analysis first has been provided for demonstrating over-smoothing exist in BERT. Then, the authors theoretically prove the observation through the comparison between self-attention and graph. The authors claim that \"layer normalization plays a key role in the over-smoothing issue, namely, if the standard deviation of layer normalization is sufficiently large, the output of Transformer stacks will converge to a specific low-rank subspace and results in over-smoothing\". To alleviate the over-smoothing issue, layer fusion methods (which have been widely explored in previous studies) have been presented and verified to be effective in downstream tasks. However, the over-smoothing problem is neither well theoretically analyzed nor empirically proved to be addressed via layer fusion methods. ", "summary_of_the_review": "A marginally above the acceptance threshold is given. Overall, this paper addresses the over-smoothing problem in BERT. Theoretical analysis on the over-smoothing problem in BERT from the perspective of graph is new and provides interesting findings. However, proposed fusion methods are not that novel and not well theoretically and empirically proved to be able to address the over-smoothing problem.  ", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper has a deep analysis of the over-smoothing phenomenon in BERT from the perspective of graph. Over-smoothing refers to token uniformity problem in BERT, different input patches mapping to similar latent representation in ViT and the problem of shallower representation better than deeper (overthinking). The authors build a relationship between Transformer blocks and graphs. Namely, self-attention matrix can be regarded as a normalized adjacency matrix of a weighted graph. They prove that if the standard deviation in layer normalization is sufficiently large, the outputs of the transformer stack will converge to a low-rank subspace, resulting in over-smoothing.

In this paper, they also provide theoretical proof why higher layers can lead to over-smoothing. Empirically , they investigate the effects of the magnitude of the two standard deviations between two consecutive layers on possible over-smoothing in diverse tasks.

In order to overcome over-smoothing, they propose a series of hierarchical fusion strategy that adaptively fuses presentation from different layers, including concatenation fusion, max fusion and self-gate fusion into post-normalization. These strategies reduce similarities between tokens and outperforms BERT baseline on a few datasets (GLUE, SWAG and SQuAD).

Overall I agree with reviewers that this is a good contribution.
Decision: Accept (Spotlight)