Characterizing Massive Activations of Attention Mechanism in Graph Neural Networks

Authors: Lorenzo Bini, Marco Sorbi, Stephane Marchand-Maillet
Primary area: learning on graphs and other geometries & topologies
Keywords: GraphML, Transformers, GraphNN, Adversarial

Abstract:
Graph Neural Networks (GNNs) have become increasingly popular for effectively modeling data with graph structures. Recently, attention mechanisms have been integrated into GNNs to improve their ability to capture complex patterns. This paper presents the first comprehensive study revealing a critical, unexplored consequence of this integration: the emergence of Massive Activations (MAs) within attention layers. We introduce a novel method for detecting and analyzing MAs, focusing on edge features in different graph transformer architectures. Our study assesses various GNN models using benchmark datasets, including ZINC, TOX21, and PROTEINS. Key contributions include (1) establishing the direct link between attention mechanisms and MAs generation in GNNs, (2) developing a robust definition and detection method for MAs based on activation ratio distributions, (3) introducing the Explicit Bias Term (EBT) as a potential countermeasure and exploring it as an adversarial framework to assess models robustness based on the presence or absence of MAs. Our findings highlight the prevalence and impact of attention-induced MAs across different architectures, such as GraphTransformer, GraphiT, and SAN. The study reveals the complex interplay between attention mechanisms, model architecture, dataset characteristics, and MAs emergence, providing crucial insights for developing more robust and reliable graph models.

Decision: reject
Reviewer scores: 3, 5, 5, 3

Reviews and discussion:
<thread>
creation date:
1730448820601

thread_type:
official_review

<thread_object>
creator: reviewer_o79q

<content>
summary:
This paper investigates the emergence of Massive Activations (MAs) in attention-based Graph Neural Networks (GNNs). It identifies the link between attention mechanisms and MAs, which can destabilize models. The study proposes a method to detect and analyze MAs, introduces the Explicit Bias Term (EBT) as a countermeasure, and evaluates various GNN architectures like GraphTransformer and GraphiT on benchmark datasets. The findings emphasize the impact of MAs on model performance and robustness, offering insights for developing more resilient graph-based models.

strengths:
1.	The paper provides a comprehensive study on the emergence of Massive Activations (MAs) in attention-based GNNs, highlighting a previously unexplored issue that affects model stability and performance.
2.	The writing is logically clear and coherent.

weaknesses:
1.	The contribution is overstated. The core observation of Massive Activations (MAs) and the proposed solution, Explicit Bias Term (EBT), are not new. The work primarily transfers concepts from prior work [1] on LLMs to attention-based GNNs.
2.	The observations of MAs on attention-based GNNs lack deeper analysis. For instance, while massive activations are suggested to serve as an important bias in the attention mechanism, the paper mainly discusses their focus on unimportant features without thorough examination.
3.	The technical motivation is not clear. The authors do not clearly explain the fundamental advantage between using an untrained model to estimate MAs and using the median activation value within the layer for estimation.
4.	All the figures and tables in the paper are not up to standard. The figures in the paper lack detailed explanations of the axes and the different lines, which may confuse readers. 
5.	The experimental results are not convincing, as the tables do not show the variance across multiple trials.

questions:
1.	What is the main contribution of this paper compared to the previous work [1]? It’s important to identify what new insights or advancements this paper offers.
2.	On page 5, line 258, the authors mention, "When MAs appear, we have found two possible phenomena." This statement, “possible phenomena,” may leave readers confused. The paper could benefit from a more detailed mechanistic explanation of these observations.
3.	On page 4, line 197, the authors state, "MAs are an inherent characteristic of the attention-based mechanism in graph transformers and related architectures, not strictly dependent on the choice of the dataset." However, observations from Figure 1 suggest that the same architecture exhibits significant variations in MAs across different datasets. This raises questions about the potential dependency of MAs on the dataset, which should be addressed more thoroughly.

[1] Sun M, Chen X, Kolter J Z, et al. Massive activations in large language models[J]. arXiv preprint arXiv:2402.17762, 2024.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730619642693

thread_type:
official_review

<thread_object>
creator: reviewer_u2zz

<content>
summary:
Graph neural networks (GNNs) are popular for effectively modeling data with graph structures. Recently, attention mechanisms have been integrated into GNNs to improve their ability to capture complex patterns. This paper presents a comprehensive study on the emergence of massive activations (MAs) within attention layers. This study assesses various GNN models on three datasets, ZINC, TOX21, and PROTEINS.

strengths:
1. This work studies the existence of massive activations (MAs) in attention layers of graph neural networks, improving the understanding of attention in graph structures.
2. This work presents various types of visualization, providing a deep insight on the existence of MAs and their distributions.
3. The observations and insights presented in this work can be beneficial to the community, considering that many people are working on improving a graph transformer structure.

weaknesses:
1. In such a benchmark study, I would expect a lot more combinations of models, datasets, and configurations to verify the generalizability of observations and insights. I don’t believe the current experiments are comprehensive enough to have conclusive claims. It would be also nice to include more recent graph transformer structures.
2. This work is not motivated well, as it is not clear why we should study MAs, and why they are problematic. How exactly do they affect the final performance on downstream tasks? The authors provide Tables 1 and 2 in this regard, but they seem to be indirect evidence.
3. The authors claim that they propose a robust detection methodology and the explicit bias term as a solution for MAs, but I’m not certain if any of these make a significant contribution. Overall, I believe the technical quality of the paper should be improved, and the insights presented in the paper should be connected to what happens in the actual models.

questions:
1. In lines 191 - 194, the authors mention that “attention is computed between pairs of adjacent nodes only,” but many graph transformers compute attention also for non-adjacent node pairs. Do the authors consider only the model structures with this constraint?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730693058936

thread_type:
official_review

<thread_object>
creator: reviewer_hp9k

<content>
summary:
This paper investigates an interesting and significant phenomenon in attention-based Graph neural networks (GNNs): Massive activations (MAs) in attention layers. The authors conduct a systematic analysis of this phenomenon in GNNs built with different Graph Transformer layers and propose a method for detecting MAs and analyzing their effects on edge features. Besides, the Explicit Bias Term (EBT) is proposed as a countermeasure, whose robustness bringing to attention-based GNNs is also validated on well-established datasets.

strengths:
1. The phenomenon investigated in this paper is significant, and the exploration of it may influence the performance of attention-based GNNs.
2. Some empirical experiments are comprehensive.

weaknesses:
1. The motivations of this paper are not clear enough. The authors claim that their focus is variants of Graph Transformers. Why conventional attention-based GNNs (e.g., GATs and corresponding variants) are not considered in this work is not mentioned in the Section of Introduction.
2. Some terms are not well defined or explained. For example, what does the edge feature stand for in this paper? 
3. The insights and observations presented in this paper are mainly based on the analysis of three test datasets, i.e., ZINC, TOX21, and PROTEINS. I’m wondering whether similar observations or insights can be obtained from a wide range of test datasets. The authors are suggested to conduct a more comprehensive analysis on more real-world datasets, e.g., ones from OGBN (for graph property prediction) and other publicly available datasets for graph-level tasks.
4. More recent attention-based GNNs should be investigated in the paper, which may provide an overview of the advances of graph attention. For example, GATv2, Graph conjoint attentions networks, and Adaptive structural fingerprint GNN should be investigated in the manuscript.
5. Some experiments can be extended. For example, the ablation studies shown in Table 2 can be extended to all three variants of Graph Transformers on all test datasets considered in this paper.

questions:
1. Why are conventional attention-based GNNs not considered in this work?
2. What does the edge feature stand for in this paper?
3. Are similar observations or insights obtained from a wide range of test datasets?
4. More recent attention-based GNNs should be investigated in the paper.
5. Some experiments, e.g., the ablation studies in Table 2 can be extended.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730695228960

thread_type:
official_review

<thread_object>
creator: reviewer_5kwj

<content>
summary:
This paper identifies the existence of Massive Activations (MAs) across various graph transformer models, emphasizing their adverse effects on model stability and performance. The paper proposes a novel mechanism called the Explicit Bias Term (EBT) that compares activation ratios between untrained and trained models to detect MAs. Subsequently, EBT is designed to stabilize activation values by integrating bias terms directly into the attention mechanism. Additionally, the study investigates the role of MAs in adversarial attacks, introducing a gradient ascent attack method to rigorously evaluate EBT’s effectiveness in mitigating MAs. Experimental results demonstrate that MAs are widely present across different GNN architectures and datasets, while EBT significantly reduces the impact of MAs and enhances model robustness.

strengths:
1.	The paper employs a rigorous approach to define Massive Activations (MAs) and experimentally confirms that MAs are a widespread phenomenon across various attention-based models and datasets.
2.	The paper proposes the Explicit Bias Term (EBT) to mitigate the influence of Massive Activations (MAs), and this method is applicable across various downstream tasks.
3.	Experiments have shown that EBT can significantly reduce the influence of MAs and has proven to be effective in enhancing model robustness against the gradient ascent attack method.

weaknesses:
1.	The paper introduces the Explicit Bias Term (EBT) as a countermeasure to reduce the impact of Massive Activations (MAs). However, it lacks sufficient theoretical justification or a detailed explanation of how EBT can effectively mitigate MAs.
2.	The paper presents the fitting results of the logarithm of activation distributions in Figure 3, but it uses the same gamma distribution for different datasets, models (GraphTransformer, GraphiT, SAN), and network layers. This approach is not sufficiently reliable, as activation distributions may have different distributions across various datasets, models, and layers, requiring different fitting distribution to accurately capture these characteristics.
3.	In Figures 1 and 2, the activation ratios of the base model appear within a stable range, while those of the trained model fluctuate with the edges. This inconsistent visualization hinders an intuitive comparison of the activation distributions between the base and trained models within the same figure.
4. The presentation of tables should be improved and contain at least two lines.

questions:
1.	Figure 3 applies the gamma distribution to approximate the negative logarithm of activation magnitudes across distinct layers and datasets, such as layer 3 of GraphTransformer on ZINC, layer 0 of GraphiT on ZINC, and layer 1 of SAN on OGBN-PROTEINS. Is this single gamma distribution capable of accurately fitting the activation distributions across these diverse scenarios?
2.	While the paper demonstrates the existence of MAs in older models like GraphiT, GraphTransformer, and SAN, is this phenomenon also observed in more recent advancements, such as SGFormer?
</content>
</thread_object>

</thread>


Metareview:
metareview:
The authors examine Massive Activations (MAs) in graph transformers, and they propose an Explicit Bias Term (EBT) to counteract the emergence of MAs.

The reviewers recognized the importance of studying MAs within the graph learning domain, found the results insightful, and regarded the proposed EBT as an effective method. 

However, they raised several significant concerns:
- The novelty and technical contribution over a similar study in the natural language domain remain unclear.
- The study focuses on graph transformers for graphs with edge features but does not address widely-used attention-based GNNs, limiting its broader relevance.
- More extensive experiments, especially with a wider range of datasets and models, are necessary to strengthen the findings.
- The theoretical analysis lacks sufficient depth.

Considering both the reviews and rebuttals, the meta-reviewer recommends rejecting the paper but encourages the authors to improve the work based on the provided feedback.