Title: Unleashing Diffusion Transformers for Visual Correspondence by Modulating Massive Activations
Venue: NeurIPS 2025
Accepted: yes
Keywords: Diffusion Transformers

Massive Activations

Representation Learning
Abstract: Pre-trained stable diffusion models (SD) have shown great advances in visual correspondence. 
In this paper, we investigate the capabilities of Diffusion Transformers (DiTs) for accurate dense correspondence. Distinct from SD, DiTs exhibit a critical phenomenon in which very few feature activations exhibit significantly larger values than others, known as massive activations, leading to uninformative representations and significant performance degradation for DiTs.
The massive activations consistently concentrate at very few fixed dimensions across all image patch tokens, holding little local information. 
We analyze these dimension-concentrated massive activations and uncover that their concentration is inherently linked to the Adaptive Layer Normalization (AdaLN) in DiTs.
Building on these findings, we propose the Diffusion Transformer Feature (DiTF), a training-free AdaLN-based framework that extracts semantically discriminative features from DiTs. 
Specifically, DiTF leverages AdaLN to adaptively localize and normalize massive activations through channel-wise modulation. 
Furthermore, a channel discard strategy is introduced to mitigate the adverse effects of massive activations.
Experimental results demonstrate that our DiTF outperforms both DINO and SD-based models and establishes a new state-of-the-art performance for DiTs in different visual correspondence tasks (e.g., with +9.4\% on Spair-71k and +4.4\% on AP-10K-C.S.).


Peer reviews:
{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates Diffusion Transformers (DiT) and demonstrates that a slight test-time modification significantly boosts their performance on correspondence tasks. Firstly, the paper reveals the presence of massive activation in DiT features and shows that AdaLN alleviates this issue, leading to improved feature representation. Furthermore, the paper employs channel discarding to enhance these representations. The authors tested their method on various standard correspondence benchmarks and also show performance on two variants of DiT, namely SD3-5 and Flux.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths**\n\nI have always wondered why DiT features exhibit weak performance on correspondence tasks, and this paper provides a compelling explanation through intriguing and well-supported observations, presented in a clearly written text. The finding that massive activation in DiT harms its matching ability is particularly interesting.\n\n**Major Weaknesses**\n\nThe paper offers weak intuition regarding why features with massive activation are suboptimal for representation learning, as claimed in Line 166. The authors should at least elaborate on why this might be the case. Additionally, this argument could be considered an overstatement, given that diffusion features are typically well-learned from vast amounts of data. In my opinion, massive activation is specifically more detrimental to correspondence tasks. Typically, feature matching involves normalization followed by a dot product to construct cost volumes. My hypothesis is that since massive activation forms in the exact same channel for all features, the dot product between any features would be dominated by the square of this massive activation, leading to imprecise matching.\n\nBased on the hypothesis above, I wonder how the performance would be if channel discarding were applied before AdaLN. If my hypothesis is correct, then the crucial point is not whether pre-AdaLN or post-AdaLN is used, but rather that channel discarding is the most significant solution. If this shows similar performance to DiTF, then the methodology section should be slightly modified to de-emphasize the effect of AdaLN and instead highlight the effectiveness of channel discarding itself.\n\n**Minor Weaknesses**\n\n- In Line 73, the paper that used GAN should also be cited. \n- In Figure 5, the upper-left side of the activation visualization is partially cropped.\n- It would be great if the temporal correspondence results were highlighted in the main text, not just in the supplementary material.\n- Adding an explanation for why the geometric correspondence performance is lagging behind DIFT would be greatly appreciated. I suspect DIFT's high-resolution features might contribute to its strength in geometric correspondence.\n\nQUESTIONS:\nThis is a very interesting paper with a strong presentation, and I support its acceptance. However, the reason why massive activation is harmful for representation remains unclear. I have suggested a hypothesis, which may or may not be correct, and would appreciate the authors' comments and a test of this hypothesis. Whether my hypothesis is right or wrong, if the authors can provide another persuasive hypothesis, I would be happy to raise my score.\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the usage of pre-trained Diffusion Transformers (DiTs) for visual correspondence tasks and identifies a phenomenon of \"massive activations\" in DiT feature maps. To mitigate this, the authors propose a training-free feature extraction method (DiTF) that leverages existing AdaLN layers and introduces a simple channel discard strategy to suppress such activations. The paper demonstrates consistent improvements across several benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\nStrengths\nThe identification and quantification of massive activations in DiTs is the first attempt in the community. Ablation studies and visualizations are massive. The performances are evaluated across several datasets with consistent gains. The reference is sufficient and comprehensive, and up-to-date.\n\nWeaknesses\n1. The main concern is that the core contribution is relatively limited and heavily dependent on prior works, which undermines the novelty and significance required for a NeurIPS-level submission. It's more like a reimplementation of [20] in DiT [36] without novel insights. The entire pipeline builds upon existing DiT architectures and the concept of massive activations. The only novel contribution appears to be the channel discard strategy, which, while useful, cannot reach the bar for NeurIPS.\n\n3. Misleading attribution of contribution: Several operations, particularly the use of AdaLN for normalization and modulation, are not proposed by this paper, but are rather built-in functionalities of DiTs. While authors claim that 'DiTF employs AdaLN to adaptively localize and normalize massive activations with channel-wise modulation,' which might be misleading.\n\n4. Lack of theoretical depth: Many conclusions are derived from visual inspection or statistical correlations without formal analysis or proof. There is no theoretical justification for why the proposed modifications improve semantic representation. E.g., Massive activations hold little local information; however, the claim is made based on the low variance of the massive activations on the feature, which cannot directly lead to the conclusion.\n\n5. Inconsistency with prior literature: According to the prior work [20] “Massive Activations in LLMs”, massive activations are correlated with important representational capacity, disabling them will lead to performance degradation. However, this paper argues for suppressing them entirely (even zeroing them out) to improve the performances, which appears contradictory and is not justified. It's strongly recommended to give a further explanation or analysis.\n\n6. Since the proposed method is a universal paradigm, not specified for the task, it would be beneficial to also test it on some common CV tasks, like image classification.\n\n\nMinors: The manuscript would benefit from a thorough proofreading—e.g., in line 107, “AdaIN” appears to be a typo and should be corrected. [45] seems to be a wrong citation where Massive activations were not discussed. Incorrect assumptions: The paper simplifies AdaLN-zero to AdaLN, ignoring the important differences in their initialization schemes. This is conceptually incorrect.\n\nFinal Justification\nWhile the experimental results are decent, the paper relies too much on prior architectural elements and empirical heuristics. The channel discard strategy, being the only new technical contribution, is not substantial enough to warrant acceptance at NeurIPS. The work would benefit significantly from deeper theoretical insights or architectural innovations rather than retrofitting observations on existing models.\n\nQUESTIONS:\nIf the authors can address my main concerns in the rebuttal, I will adjust my rating accordingly:\n\n1. Perhaps I misunderstood, but could the authors clarify the distinction (if any) between the proposed AdaLN and the AdaLN that already exists in DiT models? As it stands, this appears to be reused rather than redesigned.\n\n2. Your work suppresses or even zeros out massive activations. However, the Massive Activation paper you cite argues that these activations are critical for model performance. Why do you observe the opposite? Could you elaborate on this contradiction?\n\nLIMITATIONS:\nyes", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 3, "CLARITY": 4, "IMPACT": 3, "ORIGINALITY": 4, "DATE": "2025-07-02"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThe authors identify and analyze  “massive activation” in Diffusion Transformers (DiT): certain feature's activation exhibit significantly large activations that infected fine-grained tasks such as visual correspondence. To address this, they propose DiTF (Diffusion Transformer Feature), a modified DiT framework incorporating AdaLN, an adaptive LayerNorm that learns channel scaling and shifting parameters conditioned on the activation statistics. Also they discard the weakly massive activation after the AdaLN features by zero out the corresponding dimensions. Experiments on standard benchmarks demonstrate that DiTF substantially reduces activation peaks and variance, leading to around 5–10 % higher matching accuracy in semantic correspondence. This work is the first to diagnose massive \nactivations in DiT, offering a general normalization design that enhances stability and performance in large scale diffusion Transformer models\n\nSTRENGTHS AND WEAKNESSES:\nStrengths: \n1. This paper carefully evaluates the “massive activation” phenomenon in the DiT model and provides a detailed analysis of it. \n2. The exploration of the phenomenon and the application of the proposed method follow a logical flow, and the experimental results are relatively comprehensive.\n\nWeakness: \n1. The paper’s explanation of how AdaLN specifically mitigates the “massive activation” phenomenon is not sufficiently detailed. The evaluation metrics lack direct comparisons between the same base models (e.g., SD3.5 or Flux) with and without the DiTF framework\n\nQUESTIONS:\n1. In Section 4.2, it is stated that AdaLN alleviates massive activations. However, in Figure 5, after applying DiTF to the Flux model, the dimensions of the original massive activations and the normalized (weakened) massive activations appear to be inconsistent. Could you explain why this occurs, and whether it is reasonable?\n2. As mentioned in the weaknesses, could you provide additional comparisons of the SD3.5 and Flux  models on specific tasks before and after applying DiTF? I did not find these results in the paper’s Results section. Also, which version of the SD3.5 model are you using? (SD3.5 large or Medium)\n\nLIMITATIONS:\nThis work lacks an analysis of its methodological limitations; the authors could examine potential constraints of their approach, including its impact on transferability to a broader set of tasks.", "TITLE": "Official Review", "RECOMMENDATION": 5, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-03"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the massive activation phenomenon in Diffusion Transformers (DiTs) for visual correspondence tasks. The authors identify that DiTs exhibit massive activations concentrated in specific feature dimensions across all image patch tokens, which leads to uninformative representations and performance degradation. They trace this phenomenon to the AdaLN-zero layers and propose DiTF (Diffusion Transformer Feature), a training-free framework that leverages AdaLN's channel-wise modulation to suppress massive activations and extract semantically discriminative features. The method achieves state-of-the-art performance on multiple correspondence benchmarks, with significant improvements over existing approaches.\n\nSTRENGTHS AND WEAKNESSES:\n**Strengths:**\n\n1. **Novel Discovery and Analysis**: This work provides the first systematic analysis of massive activations in DiTs, identifying unique characteristics distinct from LLMs (concentration across all spatial locations vs. specific tokens). The connection between massive activations and AdaLN-zero residual scaling factors is insightful.\n2. **Practical Training-Free Solution**: The proposed DiTF framework is elegant and immediately applicable to existing pre-trained DiTs without requiring additional training, making it highly practical for the community.\n3. **Comprehensive Experimental Validation**: The authors demonstrate consistent improvements across multiple DiT architectures (Pixart-alpha, SD3, SD3-5, Flux) and datasets (SPair-71k, AP-10K, PF-Pascal), establishing strong empirical evidence.\n4. **Strong Performance Gains**: The method achieves substantial improvements (+9.4% on SPair-71k, +4.4% on AP-10K-C.S.) and establishes new state-of-the-art results for DiT-based visual correspondence.\n\n**Weaknesses:**\n\n1. **Insufficient Theoretical Analysis**: While the paper identifies the connection between massive activations and AdaLN-zero, it lacks theoretical explanation for why DiTs exhibit massive activations across *all* image patch tokens, unlike the token-specific patterns observed in LLMs. This fundamental difference deserves deeper theoretical investigation.\n2. **Inadequate Computational Cost Analysis**: The paper compares against models with vastly different computational requirements without transparent discussion. For instance, Flux operates at billion-parameter scale while supervised methods like CATs++ use ~50M parameters. While supervised methods require labeled data training, their dramatically smaller size represents a significant efficiency advantage that should be explicitly acknowledged and discussed.\n3. **Limited Hyperparameter Analysis**: The paper introduces several hyperparameters (channel discard thresholds, condition dependencies) but provides insufficient sensitivity analysis. The robustness of the method across different threshold values and its stability across diverse scenarios remain unclear.\n4. **Missing Related Work**: The paper fails to cite \"TinyFusion: Diffusion Transformers Learned Shallow\" (Fang et al., 2024), which also identified massive activation issues in DiTs and proposed Masked RepKD as a solution. This is a critical omission as it represents the first work to address massive activations in DiTs, even if at a preliminary level.\n5. **Lack of Transparency in Limitations**: While the supplementary material includes a limitations section, the main paper does not transparently discuss these constraints, which is important for reader understanding and future research directions.\n\nQUESTIONS:\n1. **Theoretical Foundation**: Can you provide theoretical insights into why massive activations appear uniformly across all image patch tokens in DiTs, unlike the token-specific patterns in LLMs? What architectural or training differences in DiTs might explain this phenomenon?\n2. **Computational Efficiency**: Could you provide a detailed computational cost comparison between your billion-parameter DiT models and much smaller supervised baselines? How do you justify the computational overhead given the dramatic parameter size differences?\n3. **Hyperparameter Sensitivity**: How sensitive is the method to the channel discard threshold and other hyperparameters? Can you provide ablation studies showing performance across different threshold values and discuss the method's robustness?\n\nLIMITATIONS:\nThe authors should more transparently discuss several limitations in the \"main\" paper (not only in supplementary): (1) the lack of theoretical understanding for the unique spatial distribution of massive activations in DiTs, (2) the significant computational overhead compared to smaller supervised methods, (3) the need for hyperparameter tuning and its robustness implications, and (4) the method's reliance on pre-trained models which may limit applicability to novel architectures or training paradigms.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-07-05"}