Kati8/ICLR_2023_2025 huggingface dataset:
authors: Tianzhu Ye
authors: Li Dong
authors: Yuqing Xia
authors: Yutao Sun
authors: Yi Zhu
authors: Furu Wei
reviews: 
Comments: The paper introduces a new Transformer variant, called Diff Transformer. The key idea is in the design of attention mechanism, where Diff Transformer uses two softmax attention functions to cancel out potential attention noises. Through a suite of empirical studies, Diff Transformer show promising performances compared to standard Transformer architecture.
Strengths: - The paper introduces a simple architecture tweak to mitigate the observations that Transformer models tend to incorrectly allocate excessive attentions to irrelevant contexts.
- The paper demonstrates promising empirical results of the proposed architecture through a decent suite of evaluations, ranging from language modeling capability that fits well with scaling law, in-context learning, to improving long-context capability and mitigating contextual hallucination.
- I appreciate that the authors include a variety of downstream tasks to showcase how addressing attention noises in Transformer models can lead to performance improvements.
Weaknesses: - Related work can be discussed more thoroughly in the paper. For example, [1] discusses that Transformer models tend to mis-allocate attention to irrelevant contexts potentially biased by their position within the context [1], and [2] also shows LLMs can be easily distracted by irrelevant contexts.
- While the proposed Diff Transformer shows promising performances, they have to be trained from scratch which can be computationally expensive compared to post-training approaches that mitigate attention noises in standard Transformer models, such as the calibration technique used in [1], and improved decoding approaches used in [3] and [4]. Currently, vanilla Transformer is the only baseline considered in the paper. However, I would appreciate if the authors can show how Diff Transformer compares to other more lightweight techniques that fix Transformer's attention noises.
- Apart from pretraineing Diff Transformer, there is no further instruction tuning experiments included in the current paper. I would be interested in seeing whether instruction tuned Diff Transformer also shows better performances than standard instruction tuned Transformers.


[1] Found in the Middle: Calibrating Positional Attention Bias Improves Long Context Utilization. Hsieh et al. 2024.

[2] Large Language Models Can Be Easily Distracted by Irrelevant Context. Shi et al. 2023.

[3] Trusting Your Evidence: Hallucinate Less with Context-aware Decoding. Shi et al. 2023.

[4] Lookback Lens: Detecting and Mitigating Contextual Hallucinations in Large Language Models Using Only Attention Maps. Chuang et al. 2024.
Recommendation: 8
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: This paper proposes a new model architecture called Differential Transformer (DIFF Transformer), which reduces noise and more accurately focuses on relevant context through a differential attention mechanism. In a series of language modeling experiments, DIFF Transformer outperforms the standard Transformer across various tasks and model sizes. The paper demonstrates the superiority of this method in long-context modeling, key information retrieval, hallucination mitigation, in-context learning, and reduction of activation outliers.
Strengths: 1.The proposed DIFF Transformer introduces an innovative differential attention mechanism that reduces attention noise through the difference between two independent softmax mappings. This approach performs exceptionally well in long-context processing, making a significant improvement for long-text tasks.
2.DIFF Transformer demonstrates strong performance in language modeling, in-context learning, and multi-needle retrieval tasks, particularly outperforming traditional Transformers in long-context scenarios. It greatly enhances the accuracy and robustness in practical applications like question answering and text summarization, showing broad potential for various use cases.
3.The paper provides extensive experimental validation, confirming DIFF Transformer’s stability across different tasks and model parameter settings. The hyperparameter tuning is thorough, ensuring both robust performance and reliable results.
4.The paper is well-organized, with clear illustrations and analogies that present the differential attention mechanism effectively. Detailed experimental steps make it easy to understand the innovative contributions.
Weaknesses: While the differential attention mechanism brings notable performance improvements, it also adds computational complexity. Efficiency tests indicate that DIFF Transformer’s throughput, particularly with multi-head normalization and dual softmax calculations, is slightly lower than that of traditional Transformers (by about 5%-12%). Although this impact is relatively minor in the experiments, further exploration of computational efficiency would be valuable if scaling to large-scale applications.
Recommendation: 8
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 3
reviews: 
Comments: The paper introduces Differential Transformer, a novel architecture for LLMs that enhances attention to relevant context while canceling out noise. The core innovation is a differential attention mechanism that calculates attention scores as the difference between two softmax attention maps, promoting sparse attention patterns and reducing attention noise. The paper demonstrates through extensive experiments that the DIFF Transformer outperforms the standard Transformer in various aspects, including scaling model size, training tokens, long-context modeling, key information retrieval, hallucination mitigation, in-context learning, and reduction of activation outliers.
Strengths: 1. this is a solid and well-written paper. 
2. while the technique itself is not complex, as far as I know, this is the first work to propose a new architecture using differential attention. 3. the experiments and conclusions in this paper are thorough and insightful.
Weaknesses: 1. the authors mention "promoting the emergence of sparse attention patterns" multiple times on Lines 13, 160, and 539, but do not provide statistics and quantification of the sparsity of attention distribution between DIFF Transformer and the general Transformer.
2. The authors did not discuss or conduct experimental comparisons with work related to the sparse attention. 
For example: 
> Efficient Content-Based Sparse Attention with Routing Transformers
>
> Generating Long Sequences with Sparse Transformers.
3. It would be even better if the effectiveness of the DIFF Transformer could be validated on image or speech modalities.
4. The absence of a *Related Work* section limits the paper's ability to contextualize its contributions within the broader field.
Recommendation: 8
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 4
reviews: 
Comments: The paper aims to mitigate the problem in standard softmax attention to over-allocate attention weights to irrelevant context. The proposed Diff Transformer uses the difference between two separate softmax attention scores to cancel noise assigned to irrelevant contextual tokens. Concretely, Diff Transformer first partition the query and key vectors into two groups to compute separate softmax attention scores. The final attention score is the subtraction of the two groups. One learnable parameter $\lambda$ is introduced to balance the two groups of attention scores.

Experiments were conducted on a 3B-parameter model trained on 350B data tokens. Diff Transformer outperforms Transformer on training loss and downstream benchmarks. Ablation studies investigate the importance of the RMSNorm on each attention head.
Strengths: 1. The proposed modification of attention in Diff Transformer is well-motivated

2. The experimental results are strong, with large-scale experiments up to 3B-parameter models and 350B data tokens. 

3. The pre-trained model was evaluated on multiple benchmarks, and also on long-context evaluation, retrieval-oriented tasks, many-shot in-context learning and hallucination evaluation.

4. The paper is well-written, easy to follow.
Weaknesses: Some design motivation in the model is still not clear:

1. Why the learnable $\lambda$ is re-parameterized in Eq (2)?

2. Why in Eq (3) there is a term $(1 - \lambda_{init}$ for each head?

3. Why the RMSNorm for each head is so important for Diff Transformer stability? The explainable in section 3.8 is unconvincing to me.
Recommendation: 8
Reviewer confidence: 4
Soundness/correctness: 3
Clarity: 4