Title: Mint: A Simple Test-Time Adaptation of Vision-Language Models against Common Corruptions
Venue: NeurIPS 2025
Accepted: yes
Keywords: vision-language models

test-time adaptation
Abstract: Pretrained vision-language models such as CLIP achieve strong zero-shot generalization but remain vulnerable to distribution shifts caused by input corruptions. In this work, we investigate how corruptions affect CLIP’s image embeddings and uncover a consistent phenomenon we term as embedding variance collapse, where both intra-class and inter-class variances shrink as corruption severity increases. We find that this collapse is closely tied to performance degradation, with inter-class variance strongly correlated with classification accuracy. To explain this phenomenon, we analyze how corruptions alter the structure of the embedding space. Our theoretical results suggest that the visual encoder tends to encode corruption-related signals, which dilute class-discriminative features and compress the representation geometry. We further show that maximizing inter-class variance, even when estimated from pseudo-labels, can provably enhance embedding quality. Based on this insight, we propose Mint, a simple test-time adaptation method that maximizes pseudo-label-based inter-class variance on the fly using a mean accumulator and a gradient accumulator. Mint operates effectively with small batch sizes and consistently improves performance across multiple corruption benchmarks and CLIP architectures. Our code is available at https://github.com/baowenxuan/Mint.


Peer reviews:
comments: SUMMARY:
The paper introduces Mint, a test-time adaptation technique aimed at improving model robustness against image corruptions. Mint leverages pseudo-labels to maximize inter-class variance, addressing the authors’ observation that both intra- and inter-class variances of CLIP image embeddings tend to collapse under corruptions, which is closely linked to performance degradation. Experimental results demonstrate that Mint significantly improves robustness across various corruption benchmarks, even when limited test data is available.

STRENGTHS AND WEAKNESSES:
Strengths:

* The paper is clearly written and easy to follow.

* The proposed method is based on insightful observations about variance collapse of intra- and inter-class embeddings, also supported by a theoretical rationale.

* The usage of mean and gradient accumulators is effective in low batch size cases, where only a few test samples are available.

* Mint demonstrates time efficiency, compared to many TTA methods.

* The paper includes comprehensive ablation studies and analysis that highlight the roles of different accumulators and the influence of hyperparameters.

Weaknesses:

* The paper lacks comparative results showing how SOTA methods perform under varying batch sizes. While Mint is evaluated across different batch sizes, it is unclear how it compares to SOTA methods in these settings, i.e. it is uncertain whether Mint maintains its advantage in more constrained batch size scenarios.

QUESTIONS:
Table 2 shows that increasing the batch size does not always lead to improved accuracy, which appears counterintuitive given the common assumption that more data should yield better estimates of unknown data statistics. What could be the underlying reasons for this behavior?

LIMITATIONS:
Yes
TITLE: Official Review
CLARITY: 3
IMPACT: 3
ORIGINALITY: 2
DATE: 2025-06-12

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nWork discusses variance collapse under corruptions i.e., embeddings of images tend to become more similar, regardless of whether they belong to the same class or different classes. A strong correlation between inter-class variance and classification accuracy is further shown. The proposed method uses Test time augmentation to improve model accuracy on the noise by    \n1)\tMaximizing intra-variance among features using pseudo labels.    \n2)\tUsing mean accumulator (average image embeddings for each pseudo-class) and gradient accumulator to improve TTA under limited batch.    \n3)\tOnly updating the LayerNorm parameters to improve image embedding.    \nAuthors postulate that models should show low intra-class variance (same class embedding should be closer) and high inter-class variance (different classes embedding separated) for higher accuracy.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths: \n1)\tPaper gets credit for mathematically justifying some known solution for low resolution (and similar other kinds of noises).   \n    a.\tEffect of noise severity of feature variance.   \n    b.\t“maximizing PL-inter is equivalent to jointly maximizing PL-total variance and minimizing PL-intra variance”.    \n2)\tCode is provided with supplementary, hopefully they will make the code public, as it's not going to be an easy implementation.    \n3)\tSmall batch size in TTA has a series of problems like “noisy and biased gradients” and few samples across classes, where the distance between a sample and its mean becomes zero. Thus, the work opts for a mean accumulator and a gradient accumulator to aggregate information across batches, which is the first of its kind.      \n4)\tCost-efficient as the model updates only the layer norm during test time.    \n5)\tAblation successfully establishes the effectiveness of each component.    \n\n\n## Weakness \n1.\t**The idea of Variance Collapse is not novel.** Features collapse under low resolution is a fairly well-researched topic across severities. The work simply extends the observation to other forms of noise.  Missing reference works, and the tone of writing indicates novelty. Here are some works indicating a similar property:    \n    a)\t**Feature collapse on foundation models:** LR0.FM: Low-Res Benchmark and Improving robustness for Zero-Shot Classification in Foundation Models, ICLR’25   \n    b)\t**Overlapping of positive and negative instances of class under low resolution**: Dive into the Resolution Augmentations and Metrics in Low Resolution Face Recognition: A Plain yet Effective New Baseline, AAAI’23.   \n   c)\t**Overlap of feature centers:** Recognizability Embedding Enhancement for Very Low-Resolution Face Recognition and Quality Estimation, CVPR’23.    \n\n2.\tOne of the main contributions of the paper is the theoretical justification for variance collapse with noise. But the theoretical proof is not at all an easy read, with tons of confusion in the derivation of Theorem 3.1. & Theorem 3.2   \n    a.\t**Confusion regarding layer norm** (equation number missing) Line 165. Why are authors using L2 norm as Layer norm? In traditional layer norm there is no L2 normalization,                        \n   Z =  (  (v – mean_dim) / (variance_dim) **0.5  )  * w + beta   \nAuthors have instead used two L2 normalization steps, which is not how PyTorch has implemented layer norm either (ref: https://docs.pytorch.org/docs/stable/generated/torch.nn.LayerNorm.html). The original layer norm doesn't allow L2 normalization either.    \n   b.\tA core assumption seems to be feature vector distangled as: $v_i = v_i^{cls} +  v_i^{irr} + v_i^{shift} + v_i^{noise}$  & weight as $w = w^{cls} + w^{irr} +  w^{shift} + w^{noise}$, where  $v_i^j \\bigodot w^k = 0 \\text{ if } k \\ne j $. \nHow weights are broken into these components is not clear, and why $v_i^j \\bigodot w^k = 0 \\text{ if } k \\ne j $, for example $v_i^{cls} \\bigodot w^{irr} = 0 $. If its not addition, and instead concatenation, it will raise more questions.     \n    c. Appendix: Line 576 (equation number missing) appears to be using Cauchy Cauchy-Schwarz inequality. I’m not sure why  $|| u \\bigodot w^{cls} || =  || u || $ (not sure why $w^{cls}$ is assumed to be taken from a similar distribution as that of u. Similarly, for other multiplication products as well.    \n    d.\tLine 580 (equation number missing). Under the infinite sum, average (mean) becomes an l2 normalized vector? Clarity for the equation is required, with missing steps and lack of assumptions, it's not easy to understand how Theorem 3.1, and Theorem 3.2 are derived. I can only take the author's word for it if these equations are derived correctly.     \n   e.\tB.4 Adaptation: What is conv? How is equation line 589 (equation no missing) derived?    \n   f.\t[Line 190] *“maximize the PL-inter variance by updating the parameters of LayerNorm, the parameters associated with structured distribution shifts (i.e., wshift) are necessarily suppressed”* how? Which equation 3 or 4? \n  g.\t[Line 194] Maximizing PL-inter variance will increase the weights associated with task-relevant features (i.e., wcls)? How?    \n  h.\tText writes inferences without actually pointing to what equation it's using to derive those inferences. This is not possible for me to understand.     \n   i.\t[line 176] *“GT-inter variance has strong relevance to classification accuracy, as it reflects the proportion of task-relevant features within the overall feature representation”*. How is accuracy reflected in Theorem 3.1, which primarily connects to variance to noise severity. \n\n\n3.\tAblation results requested for :   \n    a.\tWhat happens if you maximize variance inter (instead of total - intra)   \n    b.\tOther variations, updating last block? Last MLP layer?   \n\n4.\t**Fundamental flaw with logic: Why test time augmentation?**   \nMissing works include Super resolution (generate HQ sample on fly), Lr-TK0 [1] (pretraining model on synthetic diffusion generated images with noise), Robsut SAM [2] (prompts help improve accuracy)    \nIf the Idea is to have never seen the noise before, why do authors treat the model training on each noise independently [line 265]. Separating noise means the authors already know what noise looks like. Not training on it is purely a design choice. Ideally, should evaluate the **same model across all noise**, i.e., training using an amalgam of all noises. if the assumption is to not been seen the noise before.    \n[1] LR0.FM: Low-Res Benchmark and Improving robustness for Zero-Shot Classification in Foundation Models, ICLR’25 (feature collapse on foundation models)   \n[2] RobustSAM: Segment Anything Robustly on Degraded Images, CVPR’24   \n\n5. How does this lab-simulated noise translate to the real world? Can the authors show how the model would be affected when faced with real-world images, which often have a large variety of noise? \n\nMinor Complaint: \nFigure 3 does not clearly convey what it’s trying to convey. I don’t see how severity is playing any role here.\nMissing citation: Feature Augmentation based Test-Time Adaptation, WACV'25, it has a running mean (mean accumulator) for test time augmentation.\n\nQUESTIONS:\nPlease refer to the above weakness. Empirically, as the severity of noises goes up, features start to mix into one another is not novel and has been done before. The main contribution of theoretically showing how severity affects features is novel, but also needs to clearly explain the steps, which I believe severely weakened the paper. Also, the case for why using test time augmentation is not clear, given previous works which have seen these noise before, may easily outperform such a approach.\n\nLIMITATIONS:\nNo limitation as such.", "TITLE": "Official Review", "RECOMMENDATION": 3, "REVIEWER_CONFIDENCE": 4, "CLARITY": 1, "IMPACT": 1, "ORIGINALITY": 4, "DATE": "2025-06-24"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper proposes a novel test-time adaptation (TTA) method for vision-language models (VLMs) like CLIP targetting image corruption.\nThe authors found a phenomenon in which the variance of image embeddings decreases, and the embeddings become less distinguishable under image corruption, which is called variance collapse.\nThe mechanism of variance collapse and the fact that variance collapse is closely related to model performance is also theoretically explained.\nBased on this finding, a novel TTA method called Mint, which maximizes the inter-class variance estimated with pseudo labels, is proposed.\nExperimental results show that Mint outperforms prior TTA methods for VLMs under image corruption.\n\nSTRENGTHS AND WEAKNESSES:\n## Strengths\n- The paper is well-structured, and the motivation is clear.\n- The finding of variance collapse is interesting.\n- The design choice of Mint based on variance collapse is reasonable.\n\n## Weaknesses\n- W1: It is questionable whether the assumption for the theoretical analysis that the image features can be disentangled into the four components. Under image corruption, such as noise, the information is lost and cannot be recovered. Although this assumption is motivated by Wiles et al. [33], they mainly consider attributes rather than corruption.\n- W2: It would be interesting to validate that variance collapse is specific to image corruption experimentally. For example, comparing vanilla ImageNet and ImageNet-A/R/S would support the claim.\n- W3: Maximizing PL-total variance while minimizing PL-intra may be related to neural collapse [a,b]. Discussing neural collapse would improve the coverage of related works.\n\n[a] Kothapalli, Neural Collapse: A Review on Modelling Principles and Generalization, TMLR, 2023.     \n[b] Papyan et al., Prevalence of neural collapse during the terminal phase of deep learning training, PNAS, 2020.\n\nQUESTIONS:\n- Q1: Since $\\mathcal{V}^\\text{PL}\\_\\text{inter} = \\mathcal{V}^\\text{PL}\\_\\text{total} - \\mathcal{V}^\\text{PL}\\_\\text{intra}$, $\\mathcal{V}^\\text{PL}\\_\\text{intra}$ should have negative correlation with accuracy. However, L136 and Fig. 3 state that $\\mathcal{V}^\\text{PL}\\_\\text{intra}$ increases with severe corruption and has a positive correlation with performance. Is the sign flipped?\n- Q2: In Tabs. 1 and 2, how the model and dataset combinations, i.e., (ViT-B/32, CIFAR10-C), (ViT-B/16, CIFAR100-C), and (ViT-L/14, ImageNet-C), were selected?\n\nLIMITATIONS:\nYes.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 4, "CLARITY": 3, "IMPACT": 3, "ORIGINALITY": 3, "DATE": "2025-06-27"}

{"IS_META_REVIEW": false, "comments": "SUMMARY:\nThis paper investigates the degradation of CLIP's performance under input corruptions and introduces a novel phenomenon, variance collapse.\nThe authors analyze variance collapse and provide a theoretical explanation: corrupted images weaken class-discriminative features, leading to a performance drop.\nTo address this, the authors propose Mint, a test-time adaptation method that maximizes inter-class variance estimated from pseudo-labels.\nMint consists of a mean accumulator to estimate PL-inter variance and a gradient accumulator to stabilize updates.\nExtensive results show that Mint improves the classification performance of CLIP across three commonly used corruption benchmarks.\n\nSTRENGTHS AND WEAKNESSES:\n### Strengths:\n\n1. The authors provide a solid theoretical framework connecting the variance collapse to performance degradation, enhancing the interpretability of the proposed method.\n2. Experimental results show that Mint outperforms existing TTA methods on several corrupted benchmarks.\n3. The submission is well written and easy to follow.\n\n### Weaknesses:\n\n1. The decomposition in Eq.5 may be redundant. If certain classes have no samples, they contribute nothing to the computation of the inter-class variance. The average embedding of these classes can effectively be treated as the global center. In this way, the calculation of the left term of Eq.5 can be directly performed while retaining the update of Eq.6. If there is a meaningful numerical difference between the two formulations, the authors should provide quantitative evidence to justify the added complexity.\n2. The experiments are limited to corrupted benchmarks and do not report classification accuracy on clean images. Moreover, the paper omits evaluation on other standard distribution shift benchmarks commonly used for VLMs (e.g., ImageNet-A, -V, -R, -K).\n3. Is Mint sensitive to optimization space? The authors need to provide the results of optimizing the prompt or the text encoder and explain whether the image encoder must be selected for optimization.\n4. In Table 3, the authors only select some training-based methods for time comparison, but explain that they select the top-performing algorithm. In fact, training-free methods such as TDA are better than TPT in experimental results, but they were not selected. The authors should provide the time of high-performance training-free methods for reference.\n\n### Minor Weakness:\n1. The statement ‘where $K_{prior}$ is the strength of prior’ in Line 249 would make one think that $K_{prior}$ is an adaptive variable rather than a hyperparameter.\n2. There is a conflict between the statement in Lines 49-53 about ‘simultaneous reduction of GT-intra and GT-inter variances ... dilutes class-discriminative information.’ and the method that continues to minimize PL-intra variances.\n\nQUESTIONS:\nPlease refer to the above weaknesses and provide corresponding explanations.\n\nLIMITATIONS:\nYes, the authors provide the limitations and potential negative societal impact in the supplementary material.", "TITLE": "Official Review", "RECOMMENDATION": 4, "REVIEWER_CONFIDENCE": 5, "CLARITY": 3, "IMPACT": 2, "ORIGINALITY": 3, "DATE": "2025-06-30"}