Regulating Internal Alignment Flows for Robust Learning Under Spurious Correlations
Authors: Rajeev Ranjan Dwivedi
Mohammedkaif Mohammedrafiq Kalagond
Niramay M.Patel
Vinod K. Kurmi
Venue: ICLR 2026 Poster (2026)
Keywords: Fairness
Regularization
Bias Free
Primary area: alignment, fairness, safety, privacy, and societal considerations

Abstract: Deep models often exploit spurious correlations (e.g., backgrounds or dataset artifacts), hurting worst-group performance. We propose \textbf{Alignment-Gated Suppression (AGS)}, a lightweight, plug-in regularizer that intervenes inside the network during training. AGS tracks a class-conditional, confidence-weighted contribution for each neuron (more negative $\Leftrightarrow$ stronger support) and applies a percentile-based, multiplicative decay to the most extreme contributors, reducing overconfident shortcut pathways while leaving other features relatively more influential. AGS integrates with standard ERM, requires no group labels, and adds $<5\%$ training overhead. We provide analysis linking AGS to minority-margin gains, path-norm-like capacity control, and stability benefits via EMA-smoothed gating. Empirically, AGS improves worst-group accuracy and calibration vs.\ ERM and is competitive with state-of-the-art methods across spurious-correlation benchmarks (e.g., Waterbirds, CelebA, BAR, COCO), while maintaining strong average accuracy. These results suggest that regulating internal alignment flow is a simple and scalable route to robustness without group labels.

Published as a conference paper at ICLR 2026
REGULATING INTERNAL ALIGNMENT FLOWS FOR ROBUST LEARNING UNDER SPURIOUS CORRELATIONS
Rajeev Ranjan Dwivedi", Mohammedkaif Kalagond', Niramay M. Patel & Vinod K Kurmi Indian Institute of Science Education and Research Bhopal (USERB) (rajeev22, mohammedkai f23, n niramay233 vinodkk] @ felieiseeeeeeeee iesrr.rra. ac. in
ABSTRACT
Deep models often exploit spurious correlations (c.g., backgrounds or dataset arti- facts), hurting worst-group performance. We propose Alignment-Gated Suppres- sion (AGS), a lightweight, plug-in regularizer that intervenes inside the network during training. AGS tracks classcconiiiinna,, confidence- weighted contribution for each neuron (more negative + stronger support) and applies a percentile- multiplicative decay the most extreme contributors, reducing overconfident shortcut pathways while leaving other features relatively more influential. AGS integrates with standard ERM, requires no group labels, and adds 5% training overhead. We provide analysis linking AGS to minority-marghe gains, path-norm like capacity control, and stability benefits via E.A-smoothed gating. Empirically, AGS improves worst-group accuracy and calibration vs. ERM and is competitive with state-of-the-a methods across spurious-corrclatin correlation benchmarks (c.g.. Water- birds, CelebA, BAR, COCO), while maintaining strong average accuracy. These results suggest that regulating internal alignment flow is a simple and scalable route to robustness without group labels.
INTRODUCTION
Deep learning models have achieved remarkable success across a wide range of tasks, establishing them as the foundation of modern AI. However, understanding how these models achieve such results reveals critical flaws in their learning process. A critical issue is that such models often rely on spurious correlations superficial features like backgrounds, attributes, dataset artifacts rather than the core cues (You et al., 2025 Beery et al., 2018). While this shortcut learning can produce high average-case accuracy, can also cause catastrophic failures on minority or worst-case subpopulations, where these spurious cues no longer hold (Geirhos et al., 2020; Yang et al.) 2023 Hashimoto et al. 2018 Tatman 2017 Duchi et al. 2019). Such vulnerabilities make models unreliable in real-workd deployments, especially in safety- and fairness-critice applications. Addressing this challenge requires encouraging models focus their predictions on robust, transferable features rather than shortcuts, with the goal of improving worst-group accuracy and narrowing group disparities while maintaining strong overall performance (Izmailov et al. 2022).
A large body of robust learning methods attacks this problem by modifying data (e.g., group-aware reweighting Nam et al. (2020): : Kim et al. [2019); Lee et al. (0022), augmentations Kim et al. (202T) objectives that require access to group environment labels (c.g., DRO-style surrogates and invariance constraints (Sagawa et al. ))20 These approaches can be highly effective when high- quality group annotations and multiple environments available, but in many real deployments such side information missing, expensive, or unreliable Mehrabi et al. 2022). Moreover, interventions that operate outside the network (data-level) only at the loss often lack direct control over the internal contributors of specific neurons or connections that propagate spurious alignment energy forward Dwivedi al. (2024): K Kurmi et al. (2022).
We take a complementary path and intervene inside the model during training. We introduce Alignment-Gater Suppression (AGS), plug-and-play, group-agnostic regularizer that selectively suppresses spurious feature neurons using a class conditoan,i, softmax-weighted measure of on- bn-batch
'These authors contributed equally this work.

---

Published as a conference paper at ICLR 2026
alignment energy. Intuitively, for each class, estimate how much each neuron contributes to class-consistent predictions; neurons whose smoothed alignment score falls in the most negative tail are multiplicatively shrunk during training, while others remain unchanged except for mild global decay applied for stability. The mechanism integrates seamlessly with standard cross-entropy optimization, requires architectural changes, and adds less than 5% training-ting overhead.
Concretely, AGS maintains an exponential moving average (EMA) of per-neuron, per-class align- ment energy computed from mini-batches, and applies a percentile-gated decay connections whose smoothed alignment score falls below a data-driven threshold. More negative alignment score indicates stronger class-alignment, and suppressing the extreme negative tail prevents over- confident shortcut-driven pathways, leaving the remaining connections relatively more influential. This alignment-gant suppression thus weights neurons tied to spurious cues while indirectly encouraging reliance on more robust features, yielding representations that improve generalization and benefit minority groups even without group annotations.
Beyond empirical gains, our method is also clearly explained through theory. If we look at the model from the perspective of its final linear classifier (assuming the features it receives are well-behaved), the update we apply has three main effects: 1 increases margins for hard or minority groups by shrinking connections that rely on spurious cues; @ controls model capacity in way similar to path-norm regularization, by reducing the combined strength of weights and activations; improves algorithmic stability due to EMA smoothing and mild decay, which in turn connects to calibration benefits by reducing overconfident logits from misaligned neurons. These properties explain why the method tends to lift average accuracy, worst-group accuracy and lower calibration error. Em- pirically, across standard spurious-correlation benchmarks (Waterbirds, CelebA, COCO, BAR) our method consistently improves average and worst-group performance while incurring minimal training overhead. Our analyses further illustrate how internal alignment score is reallocated: neuron-level alignment distributions show contracted aigh-alignment score tails for spurious contributor.
Our contributions
follows:
① We define alignment energy, a fast, on-batch, caas-conniiional, softmax softmax-weighted mea- sure of neuron alignment, enabling training-time attribution without group labels.
We propose simple percentile-get plug aud-aan with standard training loops and incurs less than 5% overhead.
suppression rule with EMA smoothing that is
We provide theory linking alignment gaantd suppression to minority- group margin gains, capacity control through path- wurth-oormliiie shrinkage, and stability/calibratin improve- ments.
We demonstrate strong and consistent improvements in average-grout accuracy, worst- group accuracy and calibration spurious- correlattin benchmarks, accompanied by neuron-level and representation analyses.
These results suggest that directly regulating the internal flow of class-conditional alignment score is a simple, scalable, and effective route to robustness under spurious correlations particularly in the common setting where group labels are unavailable.
RELATED WORK
Shortcut learning and worst-group risk. Deep models often exploit shortcut features that correlate with labels but fail under distribution or subpopulation shift Geirhos al. (2020); [eerr et al. (2018), leading to sharp drops in worst-group accuracy Krueger et (202 Existing solutions largely act outside the network via data balancing Nam al. (2020), environment construction Bahng al.) (2020), or robust loss design Vandenhirtz et al. (2023) yet leave the model's 's internal reliance on spurious pathways unchecked. We instead regulate neurons directly, intervening at the source of shortcut learning.
Group-aware robust training. When group labels are known, methods such GroupDRO Sagawa ct al. (2020), IRM Arrovsky et (201)), V-REx Krueger et al. (202I). and CVaR-DRG Levy et al. (2020) explicitly bound or penalize worst-group risk. These approaches are powerful but brittle:
2

---

Published as a conference paper at ICLR 2026
they require high-quality annotations and often incur large computational costs. Moreover, they adjust data or objectives rather than controlling how alignment flows inside the model. Our method avoids supervision, remains lightweight, and complements such objectives by suppressing weak or misaligned connections during training.
Group-agnostic robustness. In the absence of labels, prior work reweights examples by confidence, error, or heuristics to anticipate spurious features Qraitem et al. (2023). Such strategies are indirect and unstable, since they rely on proxies group structure. By contrast, we compute neuron-level, class-conditional alignment energy from the model's s own predictions, and apply percentile-gated decay, yielding a direct, iitributiongrrundee signal that efficient and stable.
Beyond generic regularization and pruning. Classical penalties (e.g., weight decay, dropout, Jacobian constraints) are global and input-agnostic, improving average generalization but not worst- group stability Sokolić al. (2017). Pruning and sparsification, while effective for compression. typically act post hoc and fail to reshape learning dynamics that cause shortcuts. Our approach differs by offering a training-time class-aware regularizer that suppresses spurious pathways without altering architecture or requiring sparsity targets.
Relation to EvA He et al. (2255) introduced the term evidence energy and used it as a post hoc diagnostic identify spurious channels, which then hard-erased (channel deletion) with an additional last-layer retraining stage. Our approach is different in both derivation and mechanism: we derive parameter-space alignment statistic (sensitivity classifier weights) and use as an in-training, link-level regularizer that continuously contracts overconfident shortcut pathways via EMA-smoothed, class-wise percentile gating and multiplicative decay. To avoid terminology confusion, we refer to our quantity as alignment energy rather than "evidence energy."
Complementarity to existing approaches. AGS is group-agnostic and plug-and-plla integrating seamlessly with ERM and coexisting with data- objective-level robust training (c.g.. GroupDRO Sagawa et al.] 22000) IRM Arjovsky et al. (2019), V-REx Krueger et al. (202]. JTT Liu et al. 2021))
LfF|Nam al. (2020)). Whereas reweighting
invariance methods operate examples or loss terms, AGS modulates the internal conduits that propagate spurious alignments. This complementary
locus of control suggests potential gains from combining AGS with group-aware group-discovery pipelines, as AGS can regularize the model's 's reliance on emergent shortcuts even when enviromment or group labels are unavailable or noisy.
Positioning. In summary, prior work either (i) relies on labels to enforce robustness, (ii) heuris- tically reweights data without touching internal contributors, or (iii) applies uniform or post hoc regularization. We propose simple, group-agnostic neuron-level mechanism that gates shrinkage by class-conditional alignment, directly reducing shortcut reliance with negligible overhead. This fills critical gap by addressing spurious correlations their origin inside the model while remaining complementary to existing data- and loss-level approaches.
3 METHOD
3.1
NOTATION AND SETTING
We consider a standard C-class classification problem with labeled dataset = [xx, y) where y E [1, .I,..., .....,,, CJ. For an input x, let ( E R. denote the penultimate-layer representation;
use
j E [1, (l...., D) to index feature coordinates (neurons). The final prediction layer is a linear classifier
W [wi.... wel DxC that maps representations to logits =(x) and class probabilities p(a):
exp (w) (o()))
2(x) = W (z),
(x())= = [p())] h = softmax, ( (W))
(I)
Ie exp( w do(x))
We write (x) for the j-th coordinate of de (z) and Wjs for the weight from feature j to class k.
'EvA coined "evidence energy"; ; use related but distinct parameter-space quantity and different training-time mechanism.
3

---

Published as a conference paper at ICLR 2026
Our Alignment- Hignnment-Gated Gated Suppression (AGS), acts on this last linear layer by default, but the same interface applies unchanged to intermediate units (e.g., channel activations in convolutional neural networks). For clarity omit bias term. Unless stated otherwise, the base training objective is empirical risk minimization (ERM) with cross-entropy; AGS augments ERM as a plug-in regularizer and is agnostic the choice of optimizer. All "alignment" quantities introduced below are computed during the forward pass and treated as stop-gradient statistics no gradients are propagated through the gating decisions or the EMA buffers akin consistency-based approaches (Tarvainen & Valpola 2017). This design keeps the procedure numerically stable and fully compatible with standard optimizers such as SGD Robbins & Monro (1951) and Adam Kingma Ba (2015).
3.2 ALIGNMENT SCORE AND CLASS-AVERAGE ENERGY
We quantify how feature (neuron) contributes to class k on an input through the alignment score
Cjk (x) A - (x) W (x)),
(2)
and, in practice, evaluate it only for the true class = y. The leading minus sign fixes the direction of the scale: more negative means stronger alignment. Indeed, when ( and Wjy align so to increase the y-logit, the product Wjy (y(x) is positive; if the model is confident on x (large Py (x)), then (2)(2) moves further into the negative range. The softmax factor pk (z) therefore couples alignment to confidence, accentuating contributions that the model appears most sure about and de-emphasizing uncertain ones. To aggregate across examples with the same ground-truth label, we define the class-k alignment energy
Ejk- EEND.[<)
where D denotes the population of inputs with true label k. More negative Eys indicates that feature j consistently delivers confidence-weighted support for class k.
Direct comparison to EvA (He al., 2025). Because the per-example expression in Eq. (2) can appear superficially similar to EvA He et al. (025), we make the distinction explicit. EvA computes an octivattionsspas score (sensitivity of an energy/free-enery and uses it post hoc to hard-deles entire channels, followed by separate last-layer retraining stage. In contrast, our alignment score is parameter-spac statistic: it is designed to measure how strongly specific neuron-to-class connection (j,k) k) supports the current prediction and to serve as an in-training regularizer on the same weights we intervene on. This parameter-spe space viewpoint is the key enabler of our single-stage, link-level multiplicative decay rule (Eq. B)) and it naturally supports (i) confidence weighting via pę (3). (ii) online history-aware aggregation via EMA (Eq. (5), and (iii) scale-free suppression via within-class percentiles (Eq. ( For completeness. we provide derivation contrasting the activation -oooon--ppe space and purameter-space forms in Appendix
quantity to penultimate features)
Normalization and scale. Because (z) scales linearly with both Wjk and (z)), we avoid hand- tuned rescaling and instead rely on three stabilizers: 1 EMA smoothing of batch estimates ($3.4) @ within-class percentile gating that depends only on order statisties (hence scale-free) ($3.5) and ① a mild global decay (Eq. 8) to prevent drift. When BatchNorm//LyerNNon computed on nost-normalization features, so no extra per-feature normalization is required. These choices make our alignment ordering robust while keeping the magnitude unconstrained, which precisely what the percentile gate exploits downstream.
is present, alignment is
3.3 BATCH-WISE ESTIMATION AND THE ROLE BATCH SIZE
In training, we estimate (Eq. 3 from mini- batches. At step t, for batch B let B {x E B y(x) = k]. The per-class batch alignment energy
(
Σ
(g(x),
10-
B&
where the small guards against empty classes. If Bg = Ø, the class contributes 0 at this step; the running EMA ($3.4) carries information forward until i reappears.
4

---

Published as a conference paper at ICLR 2026
Effect of [B]. Since E5t is finite-sample estimator of Ejs, its variability depends on batch size and class balance. Smaller batches increase sampling noise, which can push more features into the lower (more negative) tail and thus trigger additional suppression at fixed percentile E [10, 20]. Larger batches reduce this variance, yielding steadier gates but potentially less sensitivity transient shortcut spikes. Consequently, [B] acts as a stability knob: too small risks over-pruning due to noisy tail estimates; too large may under-suppress persistent shortcuts.
3.4 EXPONENTIAL MOVING AVERAGE (EMA)
The percentile gate in 33.5 relies on stable, low-variance estimates of per-class alignment energies ).3) To smooth the noisy batch means F:(O we maintain an exponential moving average (EMA)
Bit = (1 (1- 3) E + 3 B E()
B [0.6, 1),
(5)
with 3 = 0.75 by default. When class k is absent in the current batch (B₃ Ø), we set E(i)
0,
yielding the deterministic decay E 3 E Smaller 3 increases responsiveness to new alignments; larger 3 adds inertia and is especially helpful for rare classes. We initialize Ej 0 and do not apply bias correction, since gating decisions are made by within-class rank (rather than absolute scale).
3.5 PERCENTILE-GATED SUPPRESSION
After brief warm-up of T 5 epochs to populate the EMA buffers, compute class-specific threshold by taking the within-class q-th percentile (across features) of the smoothed energies:
A Percentile, (LE12..... ,..,, EE0))
E [10, 20).
(6)
Features whose energy falls below this threshold are temporarily suppressed via a binary gate
(t)
I E/0 < x()
(t)
We then apply decoupled multiplicative decay to the classifier weights:
Wfk < (1 - a s (1 0.05 Wsk:
a [0.005, 0.15].
(8)
The per-connection factor 1 a Sik enforees selective suppression, while the mild global factor 1 0.05 curbs scale oscillations when many connections gated simultaneously. Decay is applied after the forward pass (so gating is driven by the current predictions) and before backpropagation:
(t)
(t) gradients are computed with respect the post-decay weights. We treat $jk stop-gradient mask
and do not backpropagate through it. Ties at T
(t) broken deterministically that exactly [g%] of
features are suppressed per class.
3.6 OBJECTIVE VIEW, TRAINING RECIPE, AND PRACTICALITIES
Although the mechanism operationally defined by Eq. 8 it admits a usefuul proxy objective:
J(e W) (ERM((, W) + 5k + c) W
which interprets gating as an adaptive, class-conditional quadratic penalty applied only to the currently flagged connections.
Margin view. For intuition, it is useful to imagine that the learned representation decomposes into two parts, ø érob + &spu. where drob denotes robust, group-ime cooupinnvrriat features and Øspu captures spurious or shortcut-aligned features. Although this split is not directly observable in practice, provides a lens to interpret the effect of suppression. In this view, the gate primarily attenuates @spu by reducing ||WT @epull, which heuristically increases decision margins in settings where spurious cues dominate:
Amargin
Σ
jk Wys (dspu)j-
(1)
(10)
5

---

Published as a conference paper at ICLR 2026
End-to-end pipeline. At each training step:
1. Forward pass to obtain p(x) and Φ0(2); compute per-example alignment ej s(a) (2) and
batch means (( ($3.3).
jk
2. Update EMA F0(1) via Eq.
(0) 3. After warm-up, form T (Eq. (6), compute gates Sjk (Eq. 77, and apply decay (Eq. (8).
(1)
4. Backpropagate LERM with the decayed weights and take the optimizer step.
3.7 PROPERTIES OF AGS
Under mild boundedness (10y S. Rg. ||w/ll2 S Rw Ps (x: E [0, 1)), the four properties below formalize what AGS targets, why its gates stable and budgeted, and how its decay operator suppresses spurious pathways while preserving robust ones. Throughout, the alignment for feature and class & on input x is (zx(z) = px(x) W (z)), and Est denotes the per-class exponential moving average (EMA) of on-batch alignment; the classwise gate thresholds are percentiles т computed over }.
Property (Confidence- targeting and scale invariance) For y k. czx(z) becomes more negative as the alignment (z) increases while (x(x) does not decrease-thur higher- confidence, class-aligned contributions receive larger magnitude. Moreover, any loeie-preeerving rescaling ø' = aΦ, W W/a (with a >> 0) leaves W d' = W & and hence p' consequently (z((z) = Czk (x), the ordering of [F.)), and the gated set are invariant. Lower-confidence samples (smaller Py) produce alignment closer to and thus exert less influence on gating.
Property 2 (Stable, budgeted gating via EMA and percentiles) The EMA update E(t) (1 3) E(t) + 3 E(--)) (where 3 E [0.6, 1) is the EMA parameter) is a convex combination with step-to- step drift bounded by 2(1 B) R. Ray- Per class k, most [4%] features are gated (here q the gate budget in percent), because decisions depend only on the within-class order of { {E.k}, which [t-1] + A (gap A >> 0) and batch-induce perturbations have size at most (perturbation radius), then whenever
is invariant under any strictly increasing transform. If E E(t-1) 2 T
2(1 - 3)5 < A,
feature j remains ungated at step t (gate inertia). Larger mini-batches B (we write [B] for its size) and class-balanced sampling reduce the variance of E!t) and hence the probability of flips.
Property (Multiplicative decay contracts and sparsifies) Under the update
Wjs (1 a85k) (1 ac) W,k:
Syx E (0, 1} denotes the gate indicator for kj, > is the gated shrinkage rate, and c= 0.05 is small global decay coefficient. Gated coordinates ($yk- 1) contract by
p= p= (1- a) ac)
(the per-step contraction factor), yielding geometric attenuation across consecutive gated steps. For any threshold E > 0, the number of gated entries with magnitude > e weeklyy decreases after one step (monotone sparsification). The combined effect is equivalent applying class-conditional. feature-wise proximal operator of the quadratic regularizer a (8jk + c)W jk to gated links before the ERM update, plus a mild global decay to stabilize scale.
Property 4 (Bias suppression with preservation of robust features) When gated coordinates align with spurious components on worst-group examples, the pairwise margin
mi m (x) = (wr - we)" ((x)
obeys
(x) 2 mz-)(()) + a 0spp ac (wx w.) (T(()))

---

Published as a conference paper at ICLR 2026
where Sy denotes the set of indices gated for class k, Ospu is the spurious component in the decompo- sition Φ = Prob + Ospu. and
Yspu
Σ JESk
(Ospu) 2 O.
This shows net margin gain on worst group inputs up to the small global decay term. Conversely, any feature whose smoothed alignment persistently remains above the classwise percentile, Ex Tk (defining the index Joob of robust, never-gated feature), is deterministically never gated and is therefore preserved.
Hence, AGS targets confident, class-aligned reliance in scale-invariant manner; stabilizes its budgeted decisions via EMA smoothing and percentile gates; * contracts and sparsifies persistently gated pathways through multiplicative decay; and > suppresses bias while preserving robust features that avoid the lower-alignment tail.
4 EXPERIMENTAL SETUP
In this section we describe the experimental framework used to evaluate our proposed method. We detail the datasets, evaluation metrics, and model selection strategy, followed by baseline comparisons and implementation specifics.
We fine-tune& ResNet50 He al. (2016) backbone with linear classifier, initialized from ImageNet weights. All models are trained end-to-ena with standard data augmentations (random resized crops, horizontal flips, mild color jitter). Our method (Sec. 3 is attached to the penultimate representation and the final classifier. Training overhead < 5% relative to vanilla fine-tuning.
Datasets, Metries, and Model Selection. We evaluate on three widely used spurious-correlation benchmarks and a COCO-based construction. Waterbirds (WILDS) Sagawa et al. (2020) induces a background shortcut: most waterbirds appear over water and most landbirds over land; minority (bias-cont (bias-conflicting) groups swap backgrounds. CelebA (WILDS) Liu et al. (2015) targets gender with hair color as a spurious attribute, where majority correlations are malee, dark hair) and (female, blond hair). BAR Nam et al. 2020) stresses action recognition when training contexts are prototypical (e.g., climbing outdoors) but test-time contexts are shifted (e.g.. indoor climbing). Finally, COCO Gender/Object Bias Zhao al. (2023) builds binary gender classification task with object-context correlations. Gender Is inferred from captions using curated keywords; bias categories are drawn from COCO instance annotations two disjoint groups: Sports/Outdoor (e.g., skateboard, skis, bicycle) and Kitchen/lndoor (e.g., oven, refrigerator, cup, dining table). We skew training so that male images are primarily paired with Sports/Outdoor objects and female images with Kitchen/Indoor objects; validation/test splits are stratified into Unbiased and Bias-Conflicting.
Our primary metric is worst-group accuracy (WGA) and also report average accuracy (Avg). For Waterbirds and CelebA, WGA is computed over the minority bias-conflicting groups. For BAR, since the test set consists solely of conflicting examples, we report average accuracy. For COCO, we report both Unbiased and Bias- ClaasCooffiitin accuracies within each bias category and their mean. Unless otherwise noted, model selection based on WGA on a held-out validation set.
Our method maintains only a D x C EMA buffer. Defaults that work robustly are (g. 3, a, Tw. c) (20, 0.75, 0.05, 5, 100 8). We keep [B| (Batch Size) at 32 for all datasets except the BAR dataset, which has value of 8.
4.1 RESULTS
We evaluate our method AGS three standard spurious-correlation benchmarks (Waterbirds, CelebA, BAR) and COCO (Gender/Object Bias), reporting average accuracy and worst-group accuracy (WGA). Unless noted, model selection follows the protocol in $4 and all runs average over 23 seeds (mean + std). Training overhead remains 5% relative to vanilla fine-tuming fne-tuning, as AGS adds only a light EMA buffer and percentile gate computed per class.
7

---

Published as a conference paper at ICLR 2026
Main results spurious-correlation benchmarks Across benchmarks, AGS consistently im- proves both robustness and accuracy. See Table foo a summary across benchmarks. On CelebA, attains the best results on both unbiased and bias-conflicting splits (95.63:1.228 and 93.95::1.06) outperforming the strongest prior (EvA-E](Ha et al. (2025) by over five points in each case and cutting the conflicting-syy accuracy at 97. 0.29, slightly surpassing EvA-E while lowering average error by 16.1 %2 and ob- tains a worst-group accuracy of 80.93+1.32, which statistically comparable EvA-E (81.31 though trailing JTT (84.98+0.55 by about four points, reflecting the classic average worst Pareto trade-off. On BAR, AGS delivers the strongest average accuracy at 0.309 gain of +2.39 points over EvA-E and +15588 over ERM. Taken together, these results show that AGS advances the Pareto frontier of average accuracy versus worst-case robustness: it is dominant on CelebA, state-of-the-art on BAR, and Pareloocompeliive on Waterbirds (top average, near-top WGA).
error from 11.26% 6.05%. On Waterbirds, AGS achieves the highest average
CelebA
Waterbirds
Method
BAR Average Acc.
Unbiased
Conflicting
Accuracy
Worst Acc.
Vanilla
60.51
70. 20.25_0.4
52.52 20.2
94. 04.10=4.5
63. 74.7443..
LIF
Nam et al.
2020
62.98
84.24_0.4
81.24
89.60
74.98
EIIl
Creager ct al. 2021
18.4
85.70=1.4
81.7021.5
95.882
77 77.20=1.0
JTT
Liu et al. 202
68.53
86.40-4.6
77.80+2.6
93. o0.
84.9840.5
LWBC
Kim al.
Z022
68, 68.45
83.90_1.6
87.22+1.1
Debian
Lt al. 2022
69.88
90.02=0.8
85.33
SiFER
Iwan
Shenoy
2023
72.08 10.4
90.00
88.0411.2
96. 11 10.6
77.22
EvA-E
He
2025
20.
90.51 110
88.7411.4
96.95
81.31 +1.5
Ours
76.09
95.63+0022
93.95
97.44
80.93 1.32
Table Test performance on CelebA, Waterbirds, and BAR (%): AGS improves average accuracy and strengthens worst-group robustness without group labels (ResNet-50 backhone).
COCO Gender/Object Bias On COCO with Sports/Outdoor and Kitchen//ndoor object biases, AGS attains the best average accuracy: 84.27, exceeding the strongest baseline (GMBM) by +0.73 points and ERM (Vanilla) by +14.77 points. Results are summarized in Table Crucially, AGS narrows bias-induced gaps: for Sports, the (unbiased-com conffictin))) gap shrinks from 6.20 to 0.67; ; for Kitchen, from 5.84 to 2.15. Per-split accuracies remain balanced (84.53/83.8 for Sports, 85.41/83.26 for Kitchen), indicating that AGS redistributes internal alignment toward context-invarias signals. It is also worth noting that GMBM Dwivedi et al. (2025) and Badd Sarridis et al. (2024) target multiple, co-occurring biases, and our method outperforms them well.
Sports Object
Kitchen Object
Method
Avg. Acc.
Unbiased
Conflicting
Unhiased
Conflicting
Vanilla
69.50
70.81
64.61
73.20
67.36
FairKL Burbano et al.
2023
73.67
76.32
67.11
74.35
76.90
EnD Tarlaghone al.
2021
76.95
77.11
70.97
82.38
77.34
FL.AC
Samrdis et al.
2025
79.88
80.02
77.31
80.22
79.95
BAdd
Sarniis ct al.
2024
81.76
81.28
77.81
82.91
83.05
GMBM Dwivedi
2025
83.54
83.78
83.85
83.19
83.35
Ours
84.27
84.53
83.86
85.41
83.26
Table 2: COCO gender/object bias (validation): AGS attains the best average accuracy and markedly reduces the unbiased and conflicting gap across both Sports and Kitchen splits.
Empirical sanity check. To verify that our gains are not merely due to the gating mechanism, we replace our parameter-space alignment signal with an EvA-style activation-space proxy while keeping the rest of the training-tima pipeline unchanged (EMA smoothing and within-class percentile gating). As shown in Table 3 this substitution substantially degrades worst-group accuracy, confirming that the parameter-space formulation and its coupling to confidence weighting, EMA, and percentile gating are essential for effective in-training suppression.
8

---

Published as a conference paper at ICLR 2026
Variant
Worst-group Acc. (%)
Avg. Acc. (%)
AGS (full, ours)
79.4
97.1
wlo confidence weighting (2)
73.9
91.8
w/o EMA (batch-only energies)
75.2
91.7
EvA-style activation-only proxy (in our loop)
70.1
90.9
Table -chek ablation on Waterbirds. Replacing our parameter-space alignment signal with an EvA-style activation-space proxy (while keeping EMA and percentile gating fixed) drops worst-group accuracy from 79.4% to 70.1%, showing that the full training- -ime design is not reducible to an activation-only proxy.
4.2 ANALYSIS AND ABLATION STUDIES
Decay a and EMA B. As shown in Fig. | predicted by the muliplicative-conraattio view (Property 3. increasing a strengthens sparsifying but can overshoot combined with small batches or highly entangled features. A moderate EMA (B [0.75]) reduces gate flips without blunting adaptation.
Batch size and class balance. Because alignment is computed on-batch and wiss (Eq. smaller batches increase sampligg variance and can push more fea- tures into the gated tail (Prop- erty 2 Class-balanced sampling, when available, notably stabilizes gate thresholds тę and improves WGA on datasets with rare mi- nority groups (e.g., Waterbirds). This also helps explain the resid- ual WGA gap to JTT on Water- birds: with rare bias-conflicting backgrounds, slightly larger or blass-balanced batches reduce vari- ance in (*) and improve the tail timate used for gating. (see Fig. 1
0.1
0.75
0.4
0,70
Average Accuracy
0.3
0.55
Alpha
0.26
8.2
2.19
0.55
2.00
8.4
16
256
Batch Size
AGS improves worst-group robust- ness and average accuracy by re- allocating confidence away from spurious lower-tail pathways. Its per-class, confidence-weighted gat- ing yields strong, stable gains on
Figure 1: Ablations on decay a and batch size: moderate a and sufficient batches stabilize gating and boost accuracy, while very small batches or large a risk over-suppression.
CelebA and BAR, competitive WGA on Waterbirds while achieving the best average accuracy, and large bias-gap reductions on COCO-all with negligible training overhead.
DISCUSSION
What is being regularized? AGS acts on internal, class-conditional flows of confidence-weighted alignment rather than data sampling loss surrogates. By tracking the EMA of per-neuron alignment energy Enft) jik (Eq. and shrinking only the more negative (i.c., strongest-confideneee class-aligned) connections per class via a percentile gate (Eq the method implements a targeted, budgeted contraction of high-confidence pathways that dominate the logit of the true class. This stands in contrast to uniform penalties (e.g., weight decay, dropout) or post-hoc hoc pruning and places the intervention precisely where shorteut reliance manifests during training. The resulting regularization (i) ceale-invariant under pegit-preservigg reparameterizations (Property [T), (ii) budgeted and stable by class via percentiles EMA (Property 2) and (iii) contracttieespprrifiingg on persistently gated

---

Published as a conference paper at ICLR 2026
coordinates (Property D.
Collectively, these properties explain why AGS improves sanorrtyy-group margins and calibration while minimally perturbing the remainder of the network (Property
Why do Pareto gains emerge? Across benchmarks, AGS advances the average-vs-worst-ee Pareto frontier by reducing the contribution of shortcut-dominate paths without uniformly shrinking capacity. On CelebA and BAR, AGS attains the best or oatte--f--h-hrrrrree accuracies, while on Waterbirds it secures the top average accuracy and near-top worst- worst-group accuracy a pattern consistent with selective contraction that curbs overconfident, spurious tails while preserving robust features that avoid the lower-alignment tail (qualitative alignment histograms and representation geometry support this view). The quantitative trends in Table show strong gains on BAR ( (+2.39 points over EvA-E; CelebA (e.g., confficting-sple error reduction of 46%), with competitive worst-group accuracy on Waterbirds. On our COCO construction, AGS both improves average accuracy and substantially shrinks bias-induced gaps (Table 2 These Pareto improvements are precisely what one would expect from a per-class, confidence seeddenne-weghhed contraction that narrows the heavy tail of shortcut alignment.
+15.8 over ERM) and large improvements
Mechanistic interpretation. A useful lens is the path-norm-like view: AGS decreases the product of weights and activations only along the caassconniitioall lower tail, which reduces the extreme logit contributions most responsible for miscalibration and minority- group errors. Because the gate is percentile-based based, the effective capacity budget is class-wise and scale-free, limiting the risk of global underfitting. The margin decomposition (Eq. T0 further clarifies how AGS can raise minority- group margins by attenuating spurious components in W Ospu while leaving W Órob comparatively more influential, matching the empirical observation of improved worst-case accuracy alongside strong average accuracy.
CONCLUSION
We propose Alignment-Gon Gated Suppression (AGS), a lightweight, group-agnostic regularizer that regulates where and how much class-conditionas confidence flows by contracting neuron-class links alignment. This percentile-budgeted, scale- invariant mechanism suppresses shortcut pathways while preserving robust features, improving average and worst-group accuracy and calibration with negligible overhead and without group labels across diverse benchmarks. AGS implicitly assumes that the most negative aligmment predominantly tracks thorteut-aligned routes; when robust cues dominate confidence or when robust and spurious features co-activate, such cues may transiently enter the lower tail and be contracted. While an EMA with a percentile budget reduces this risk, very small or imbalanced batches can destabilize the threshold т and over-suppress; early miscalibration can distort P& and thus the attribution signal exx(x). suggesting warm-up and conservative initial settings; and strong underfit in highly entangled regimes. Our current implementation chiefly gates the final classifier (or top-layer channels), yet spurious pathways emerge earlier in the network. These observations motivate adaptive, class-aware budgets and Mariance-reduced alignment estimation (especially for rare classes), layer- module-wise gating (e.g., channels attention heads), and combinations with group discovery or DRO to unify example-level and pathway-level robustness. Beyond vision, applying AGS structured prediction and multi-label or non-vision tasks could broaden the space of suribution-groundee regularizers for scalable robustness under spurious correlations.
that persist in the lower tail of confidence-weighee
large g may
ACKNOWLEDGMENTS
Rajeev is supported by the TCS Research Fellowship (TCS-RSP) Fellowship (Cycle 18: 2024-2028).
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Martin Arjovsky, Léon Bottou, Ishaan Gulrajani, and David Lopez-Paz. Invariant risk minimization. arXiv preprint arXiv: 1rXiiv/907.000 0907002893, 2019.
Hyojin Bahng. Sanghyuk Chun, Sangdoo Yun, Jaegul Choo, and Seong Joon Oh. Learning de-biased representations with biased representations. Hal Daumé and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning. volume 119 of Proceedings of Machine URL https: httep:://proceedinng
Learning Research, pp. 528-5288555 539. PMLR, 13-18 Jul 2020.
m1 mlr.presse w...psss//..... / v1 9/bahgg200 html
Carlo Alberto Barbano, Benoit Dufumier, Enzo Tartaglione, Marco Grangetto, and Pietro Gori. Unbiased supervised contrastive learning. In International Conference on Learning Representations (ICLR), 2023.
Sara Beery, Grant Van Horn, and Pietro Perona. Recognition in terra incognita. In Proceedings of the European Conference on Computer Vision (ECCV), September 2018.
Elliot Creager, Jörn-Henrik Jacobsen, and Richard Zemel. Environment inference for invariant
learning. In International Conference on Machine Learning, pp. 2189-2200. PMLR, 2021.
John C Duchi, Tatsunori Hashimoto, and Hongseok Namkoong. Distributionally robust losses against mixture covariate shifts. Under review, 2, 2019.
Rajeev Ranjan Dwivedi, Priyadarshini Kumari, and Vinod K. Kurmi. Cosfairnetaa parameter-space based approach for bias free learning. 35th British Machine Vision Conference 2024, RMVC2224 Glasgow, UK, November 25-28, 2024. BMVA, 2024. URL https::/papperr lomvc2024
bxg/0738.por
Rajeev Ranjan Dwivedi. Ankur Kumar, and Vinod K Kurmi. Multi attribute bias mitigation via representation learning. In Proceedings of the European Conference on Artificial Intelligence (ECAI), Bologna, Italy, October 2025.
Robert Geirhos, Jörn-Henrik Jacobsen, Claudio Michaelis, Richard Zemel, Wieland Brendel, Matthias Bethge, and Felix A. Wichmann. Shortcut learning in deep neural networks. Nature Machine Intelligence. 2(11):665-67) November 2020. ISSN 58399 doi: 10. 10.1038//42256- 0200000000000 -2t00/02222200000002------ URL / //oii www.rrrgo/..... 1 00251/022/6220022000000 020-00557---000550-000
Tatsunori Hashimoto, Megha Srivastava, Hongseok Namkoong, and Percy Liang. Fairness without demographics in repeated loss minimization. In International Conference Machine Learning,
pp. 1929-1938. PMLR, 2018
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and patter recognition, pp. 770-778, 2016.
Qiyuan He, Kai Xu, and Angela Yao. Eva: Erasing spurious correlations with activations. In The Thirteenth International Conference Learning Representations, 2025. URL https:
7/ //openreview..o ierenreriew.neer/ouuiiitttt www.iid0222ro0o1
Pavel Izmailov, Polina Kirichenko, Nate Gruver, and Andrew Gordon Wilson. On feature learning in the presence of spurious correlations. In Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates
Inc. ISBN 9781713871088.
Vinod K Kurmi, Rishabh Sharma, Yash Vardhan Sharma, and Vinay P. Namboodiri. Gradient based activations for accurate bias-free learning. In AAAI, Feb 2022.
Byungju Kim, Hyunwoo Kim, Kyungsu Kim, Sungjin Kim, and Junmo Kim. Learning learn: Training deep neural networks with biased data. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9012-9020, 2019.
Eungyeup Kim, Jihyeon Lee, and Jaegul Choo. Biaswap: Removing dataset bias with swapping augmentation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 14992-15001, 2021.
11

---

Published as a conference paper at ICLR 2026
Nayeong Kim, Sehyun Hwang, Sungsoo Ahn, Jaesik Park, and Suha Kwak. Learning debiased classifier with biased committee. Proceedings of the 36th International Conference on Neural Information Processing Systems, NIPS '22, Red Hook, NY, USA, 2022. Curran Associates Inc. ISBN 9781713871088
Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In Yoshua Bengio and Yann LeCun (eds.), 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings, 2015. URL http: //arxiv. org. abs 1aobo//1422 /1412. 6980
David Krueger, Ethan Caballero, Joern-Henrik Jacobsen, Amy Zhang, Jonathan Binas, Dinghuai Zhang, Remi Le Priol, and Aaron Courville. Out-of-distriuttin generalization via risk extrapola- tion (rex). In International Conference on Machine Learning, pp. 5815-5826. PMLR, 2021.
Jungsoo Lee, Jeonghoon Park, Daeyoung Kim, Juyoung Lee, Edward Choi, and Jaegul Choo. Revisiting the importance of amplifying bias for debiasing. AAAI-23, 5 2022. URL http?
Daniel Levy, Yair Carmon, John C Duchi, and Aaron Sidford. Large-scale methods for distributionally robust optimization. In NeurIPS, 2020.
Zhiheng Li, Anthony Hoogs, and Chenliang Xu. Discover and mitigate unknown biases with debiasing alternate networks. In ECCV, pp. 270-288. Springer, 2022.
Evan Z Liu, Behzad Haghgoo, Annie S Chen, Aditi Raghunathan, Pang Wei Koh, Shiori Sagawa, Percy Liang, and Chelsea Finn. Just train twice: Improving group robustness without training group information. In International Conference on Machine Learning, pp. 6781-6792 PMLR, 2021.
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In Proceedings of International Conference Computer Vision (ICCV), December 2015.
Ninarch Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1-33, 2021.
Junhyun Nam, Hyuntak Cha, Sungsoo Ahn, Jaeho Lee, and Jinwoo Shin. Learnnng from failure: De-biasing classifier from biased classifier. H. Larochelle, M. Ranzato, R. Hadsell, M.F Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems, volume 33, pp. 20673 20684. Curran Associates, Inc., 2020. URL https: htt.:ttss///occccdiinattt http::/wwceei.n.s.....eeeeeeeeee cc/
Maan Qraitem, Kate Saenko, and Bryan A Plummer. Bias mimicking: A simple sampling approach for bias mitigation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20311-20320, 2023.
Herbert Robbins and Sutton Monro. A Stochastic Approximation Method. The Annals of Math- ematical Statistics, 22(3):400- 407, 1951. doi: 10.1214/amm//117779588 URL https!
//doi.org/10.1222 121.:o////...111////..1 awt.//oo//////////////// /tt/////////////1/1222
Shiori Sagawa, Pang Wei Koh, Tatsunori B. Hashimoto, and Percy Liang. Distributionally robust
neural networks. In International Conference on Learning Representations, 2020. https
//openreview.net/ wttp://wwe..oommmmdddoxxxxxxear
Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos. and Christos Diou. Badd: Bias mitigation through bias addition. arXiv preprint arXiv:2408. /439, 2024.
Ioannis Sarridis, Christos Koutlis, Symeon Papadopoulos, and Christos Diou. FLAC: Fairness-Aware Representation Learning by Suppressing Attribute- Associations IEEE Transactions on Pattern Analysis & Machine Intelligence, (7(02):1148-111 1600 February 2025. ISSN 1939-3539. doi: 10.1 LL9/TTAAM..224.4444333833
URL httpa/pppee.ee.....e.aaatattttt. ozg/
10. 1 19/ AMT 22242 3487254
12

---

Published as a conference paper at ICLR 2026
Jure Sokolić, Raja Giryes, Guillermo Sapiro, and Miguel R. D. Rodrigues. Robust large margin deep neural networks. Trans. Sig. Proc., 65(16))4265 4280, August 2017. ISSN 1053-587X doi: 10.1 10.1109/TS22011227000399 /RL
2017 2708039
Enzo Tartaglione, Carlo Alberto Barbano, and Marco Grangetto. End: Entangling and disentangling deep representations for bias correction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 15508-13517 2021.
Antti Tarvainen and Harri Valpola. Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results. In Proceedings of the 31 International Conference on Neural Information Processing Systems, NIPS' 17, pp. 1195-1204, Red Hook, NY,
USA, 2017. Curran Associates Inc. ISBN 97815 97865098
Rachael Tatman. Gender and dialect bias in youtube's 's automatic captions. In Proceedings of the first ACL workshop on ethics in natural language processing, pp. 53-59, 2017.
Rishabh Tiwari and Pradeep Shenoy. Overcoming simplicity bias in deep networks using a feature sieve. Andreas Krause, Emma Brunskill, Kyunghyun Cho, Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learn- ing, volume 202 Proceedings of Machine Learning Research, pp. 34330-34343, PMLR, 23-29 .ww.mlrrm....mgmmgmmmmmmmmmmmm.m mlr www.m..r..rpp....mmmmmmmmmmm press / v202 http://wwaiiaaaaa 1tt.aaaaaaa............
Jul 2023. URL
Moritz Vandenhirtz, Laura Manduchi, Ričards Marcinkevičs, and Julia E Vogt. Signal is harder to learn than bias: Debiasing with focal loss. arXiv preprint 2rXiv:2305. 2023.
Yuzhe Yang, Haoran Zhang, Dina Katabi, and Marzych Ghassemi. Change is hard: closer look at subpopulation shift. In Proceedings of the 40th International Conference on Machine Learning. (C..'23. JogR.org, 2023.
Chenyu You, Haocheng Dai, Yifei Min, Jasjeet S. Sekhon, Sarang C. Joshi, and James S. Duncan. The silent majority: Demystifying memorization effect in the presence spurious correlations. CoRR, abs/2501.00911, January 2025. URL 00961
10 48550/ ar.iv.
Dora Zhao, Jerone Andrews, and Alice Xiang. Men also do laundry: Multi-attribute bias amplification. In Andreas Krause, Emma Brunskill, Kyunghyun Cho. Barbara Engelhardt, Sivan Sabato, and Jonathan Scarlett (eds.), Proceedings of the 40th International Conference on Machine Learming, volume 202 of Proceedings of Machine Learning Research, pp. 42000-42017 PMLR, 23-29 Jul 2023. URL https: http:///poce//igeeeeeeeee wig.e.iiig......m............... wtt.gss.....sss................e / w202 zha html
prw.s/
13

---

Published as a conference paper at ICLR 2026
A
EXPERIMENTAL DETAILS AND REPRODUCIBILITY
This section provides the necessary details reproduce experiments, including training protocols, dataset construction specifics, hyperparameter settings, and an analysis of computational overhead.
A.1 IMPLEMENTATIO DETAILS
All models were implemented in Py PyToch. We fine-tuned a ResNet-50 backbone pre-trained on ImageNet for all experiments. Standard data augmentations were applied, including random resized crops to 224 x 224, horizontal flips, and mild color jittering. The learning rate was managed with cosine decay schedule, preceded by a linear warm-up period of 5 epochs. Our method, AGS, was applied to the final linear classifier layer. Each experiment run with at least three different random seeds (77, 25 and 42), and we report the mean and standard deviation of the results.
Table 4 summarizes batch size, epochs, optimizer settings, and AGS hyperparameters (a,, Twr, 3,99) used the main results.
Table 4: Implementation and by perpaaraeeeeer details by dataset. All experiments fine-tune ResNet-50 with ImageNet initialization. AGS: a (decay), Tw (warm-up epochs), 3 (EMA momen- tum), q (percentile budget). Penultimate dimensionality is 2048.
Dataset
Size
Epochs
Optimizer Details
Hlpperparams (o. Ter
Waterbirds
32
100
Adam (Ir=1 x 10 wd=1 x 10
(0.075, 0.75, 20)
CelebA
32
30
Adam (Ir=1 x 10 4 wd=1 x 10
(0.075, 5, 0.75, 20)
COCO (our)
32
50
Adam (Ir=1 x 10-1, wd=1 x 107")
(0.035, 5, 0.75, 20) (0.075, 0.75, 20)
BAR
SGD (Ir=1 x 10 wd=1 x 10-*)
AGS maintains only a DxC EMA buffer and per-class percentile gates. Relative to vanilla fine- tuning, training-time overhead is < 5%, with no architectural changes and no gradients through gates/EMA. Our primary metric worst-group accuracy (WGA), complemented by average accuracy. Unless stated otherwise, model selection uses validation WGA; for BAR (confficting-only test), we report average accuracy.
B COMPLETE COCO GENDER BIAS DATASET CONSTRUCTION
We detail our COCO-based binary gender task to make clear which correlations AGS asked to overcome and how splits are formed. This section formalizes the caption- -aaeed labeling heuristic, object-category selection, and the split protocol used in $4 We infer gender labels from captions using exact-match lexicon (Table 5). Images with conflicting missing alignments are excluded. This heuristic is used only to form labels for the binary task; no group labels are used by AGS.
Attribute
Keyword list (exact strings)
Female
female, girl, woman, lady, girls, women, females, ladies, mother, girlfriend male, boy, man, leman boys, men, males, gentiemen, father, boyfriend
Male
Table 5: Caption keyword lexicon used for attribute inference (complete list).
To induce object-conteeeee context correlations, we select COCO categories reflecting common stereotypes (Table ( Training is skewed that male images co-occur more with Sports/Outdoor objects and female images with Kitchen/hddoor objects. Validation/test are stratified into Unbiased and Bias- Biax-Conflicting splits to probe robustness under shift.
14

---

Published as a conference paper at ICLR 2026
Context
COCO Objects
Stereotype Basis
Sports/Outdoor
sports ball, baseball bat, skateboard, suitcase, frisbee, skis, surfboard, tennis racket even, refrigerator, sink, cup, fork, knife, spoon, bowl
Traditional male-associated tivities and equipment
Kitchen/Indoor
Traditional Gemale-associacee domestic activities
Table Complete object categories with stereotype rationale.
C ALIGNMENT-GATED SUPPRESSION (AGS) ALGORITHM
instantiates the procedure described in 3 per-class alignment is computed on-batch, stabilized via EMA, and used to apply a percentile-uudgeted multiplicative decay before the ERM optimizer step. This gives the concrete training- loop placement required for reproducibility.
Algorithm I
Algorithm 1 Alignment-Gated Suppression (AGS)
Require: model (0, W), percentile q. EMA 3, decay warm-up Tu.
Initialize F:100 +- 0 for all (j, k)
2: for training step 1, 2, ... do
3: Sample mini-batch B
4: Forward pass: compute (1(x)), p(x) for x
5: For each k. form B2 {x E B: v()) = =}
6: For x E B& and all j: (x) +- ((()))))
b Eq
E(O) [Bll+e ExEB. (z)
7:
> Eq.
B())
jkk + -1 -1-11) B) + BE(t-1)
8:
b Eq.
9:
it't beyond warm-up Tw then
10:
For each k: Tk
+ Percentile, ((E (((()
b Eq.
11:
jih [
(t)
Tk (t)
> Eq.
12:
Wi + (1 as jb
(8)
0.05)
b Eq.
13:
end Backpropagate ERM loss w.r.t. decayed W; optimizer step
14:
15:
end for
Complexity and stability The additional memory O(DC) for the EMA buffer. Per-step overhead is dominated by vectorized percentile D features per class; in practice compute classwise thresholds on the host-side EMA buffer once per step. EMA and percentile gating make suppression budgeted, scale-free, and resistant noisy batches.
C.1 ALIGNMENT PROFILES OF SPURIOUS ROBUST FEATURES.
To better understand the mechanism of Alignment- Suppression (AGS), we follow the analysis of EVA and examine the distribution per-feature alignment energy. 2 shows that across all six action classes, features identified as more spurious (high gate frequency) consistently concentrate lower alignment energies compared their robust counterparts. This shift indicates that spurious features contribute less reliable alignment to the classifier's decision, and thus are preferentially targeted by AGS for suppression. The separation distributions provides quantitative support for claim that AGS leverages the alignment signal to distinguish and attenuate spurious predictors, leading to improved group robustness.
C.2 SPURIOCSNESS-ALIGNN
COUPLING.
To probe how AGS prioritizes features, we correlate each feature's spuriousness (measured by its gate frequency) with its per-class alignment energy E ERDDD, (- po(y=k: ar) Waj (x)))) Across all
15

---

Published as a conference paper at ICLR 2026
A Iynment denoities - Goos 0: Olinbing
Nignment dendities - Goos 1: Diving
Nignmeen-derdiies -Dass Rohing
103
02-
62-
00039 8.8 oitt
0.256 -0222
2.30
3009
0.000
Abgimaal OndEs regelive
atgament)
4igement
atgenanti
Alggment densilees Class Recing
giiggeentt dersilees
Class 4: Throee
Algrement danad
Class veuling
145
340
300
50
0..04
0.03
-3.00
5.00
6:31
60153001255 1006 DOTS 20.-0 OUD do0es c00s B.000 A Aigrmmen energy mmore nagultre scorpe algrmmnti
-00% 0,520 2015 20000 6.303 dod 0.006 3.00
Algererd energy
acrenger ge
Aigemntt energy STOrE regatre strarger algenentt
Figure 2: For each of the six action classes, we plot the density of per-feature alignment energies, comparing features deemed robust (blue) and spurious (orange) according to their gate frequency under AGS. Across classes, spurious features consistently exhibit more negative alignment energies, indicating weaker and less reliable contributions to the classifier's 's decision.
six classes, we observe a strong and systematic negative association: features that gate more often (more spurious) tend to exhibit lower (more negative) alignment energies. indicating they contribute less reliable class-consistent support and are thus suppressed by AGS. The reported Pearson r and Spearman in each panel, along with robust Theil-Sen fit, quantify this trend and show it holds beyond linear effects.
Climeing (nu 0 662)
Diving Ir--0 (2525
Rinhnn (tu 0.649)
3.000
0.004
1.005
0.600
Mayresk C10000
0.000
-2.00
0.000
0.00
0.600
000004
Spatourram Racing fr- 0.7151
Spurtarress
Sporisanaaa
Throwing (7504)
Polo vaullnng (r600)06)
3.0000
0.230
3.0025
-301
-3.000
-2.02
-1.005
-0.335
-3.04
-3 1.000
Spariouress
Sparkoccnos
Figure 3: Each subplot shows, for one action class, a scatter of per-feature spuriousness (gate frequency; z-axis) versus alignment energy (more negative stronger, y-axis). Points concentrate along a downward trend, and both Pearson r and Spearman p are consistently negative across classes, indicating that features most frequently gated by AGS are precisely those with low alignment energy.
16

---

Published as a conference paper at ICLR 2026
C.3 COMPONENT WISE ABLATION ANALYSIS
The monotonic improvements align with the intended mechanism (shown in Table 7 confidence weighting targets high-alignment pathways; EMA stabilizes gate membership; percentile gating delivers budgeted, scale-free contraction of the heavy tail. Together they explain the WGA lift without sacrificing average accuracy.
Table 7: Ablation on Waterbirds. Each row adds one AGS component to ERM. Gains track the mechanism in $ confidence weighting EMA smoothing percentile gating.
Component
Alignment
EMA
Percentile Gate
Avg Acc
Worst Acc
Baseline (ERM)
94.1 0.4
63.7 + 3.2
+ Confidence weighting
x
X
94.8 + 0.3
68.2 + 2.8
EMA smoothing
X
95.4 0.3
72.1 + 2.4
+ Percentile gating
97.4 0.3
80.9 1.3
C.4 DETAILED COMPARISON WITH CONCURRENT METHIODS
AGS is complementary to data- and loss-level approaches: it regularizes the internal signals that propagate shortcut alignment and can be combined with external interventions. (see Table
Table Comparison across where the intervention happens and whether group labels are required. AGS acts at the weight/conneettinn level and is plug-and and-play.
Method
Group Labels
Intervention Level
Overhead
Plug-and Play
GroupDRO
Required Required
Loss/Objective Loss/Objective
High
IRM
High
JTT
Not required
Data/Example
Medium
LIF
Not required
Data/Example
Medium
EvA-E
Not required
Activation
Low
AGS (Ours)
Not required
Weight/Connection
Low
C.5 RESULTS ON IMAGENET SCALE DATASETS
Results on ImageNet-Scale Datasets. On the ImageNet-9 Backgrounds Challenge (IN-9), ERM+AGS consistently improves performance across all evaluation variants compared to stan- dard ERM. While maintaining high accuracy on the Original split (98.69% vs. 98.05%) AGS yields notable gains under background-shifted settings, increasing accuracy on Mixed-Same Mixed-Rand, and Mixe--Nee Next. particular, performance on Mixed -andd improves from 80.17% 82.47%, indicating better robustness when sesground-backggooun rupted. Although the BG-Gap slightly increases (11.84 vs. 10.94), the overall trend suggests that AGS strengthens generalization under distribution shifts without sacrificing in-distribution accuracy, demonstrating its scalability and effectiveness at ImageNet seale.
correlations are dis-
Method
Original
Mixed-Same
Mixed-Rand
Mixed-Next
BG-Gap
ERM
98.05
91.11 94.31
80.17 82.47
77.46 79.41
10.94
ERM+AGS
98.69
11.84
Table 9: ImageNet-9 Backgrounds Challenge (IN-9) results. Models are adapted original /val (release-only val setting) and evaluated on IN-9 variants. BG-Gap defined the accuracy difference between mi xedx_eddat and mixed rand, following the IN-9 benchmark.
17

---

Published as a conference paper at ICLR 2026
C.6 CONNECTIONS TO BROADER ML PRINCIPLES
Relation to Attention Mechanisms. Alignment- Suppression (AGS) can be interpreted as an anli-allention operator that selectively attenuates overly confident, class-aligned pathways. Whereas dot-product attention amplifies contributions via a softmax over similarity scores,
(Qt K, V) = softmax
QK
V
AGS maintains a wenfidence-weighted alignment score at feature- class resolution,
(x) - pi (x) Wis ()(x),
aggregates it over time (e.g., EMA), and gates links in the lower-alignment tail:
5ik E₁
The
Wsk (1 - a sjx) 0.05a) Wsk-
Intuitively, attention boosts high-score routes; AGS contracts high-confidence routes that sit in cpass-specific heavy tail, reducing shortcut dominance while preserving the remaining signal. This yields natural complementarity:
Amplify robust, attenuate brittle. Attention layers can continue to surface salient inter- actions, while AGS curbs overconfident channels that repeatedly dominate a class logit, improving calibration and minority margins.
Scale-free control. Because AGS gates by within-class percentiles, it remains invariant to logit-preserving rescalings; thus coexists stably with attention temperature or normalization choices.
Compositional use. In attention stacks, AGS may be applied to value/feature channels or top-layer classifiers to down-weight spurious heads channels identified by persistent lower-tail alignment, without modifying the attention block itself.
Relation to Meta-Learning. AGS exhibits a lightweight, two-time-scale meta-learning behavior: fast ERM updates run in the inner loop. while slow, label-conditional alignment state drives outer-loop modulation.
(Inner loop)
(0, ()) + ERM step on (Ge W)
(Outer state)
Ezk
(1 - B) Esk + Eyk
(Policy)
8yk
I E5x < <T)]
W,k (1-aja)(11 -1000)))\10000)a,,a,,)
Here, the EMA buffer F, and percentile thresholds (Tx) constitute slow-moving class-conditional meta-parameters that adapt suppression budgets the observed alignment geometry. The effect mirrors bilevel prior that:
Adapts to dataset idiosyncrasies. Persistent shortcut patterns accrue strongly negative (high-confic cighh-oonfidence) alignment and are selectively contracted, reallocating capacity toward invariant features.
Stabilizes credit assignment. EMA smoothing reduces gate flips and noise sensitivity, acting like regularized outer objective that trades off plasticity (small 3) and inertia (large 3).
Implements structured shrinkage. The gated decay approximates a class-conditional prox- imal penalty, yielding path-norm-like capacity control focused shortcut-prone coordinates rather than uniform regularization.
In practice, this outer mechanism is differentiable only through its effects (no gradients through gates/EMA), making AGS plug-and-pla meta-regularizer that improves worst-case robustness without explicit group labels.
18

---

Published as a conference paper at ICLR 2026
Table 10: Notation used in the AGS properties.
Symbol
Meaning
Range / Notes
E R
Input; ((x) is its representation
VE [1....... K}
True class label
K: #classes
k, [1..... K}
Class indices (souree/target)
D
#fcatures (representation dimension)
= dim (6)
((x) EDD
Feature/repreeentation vector
01(2)
j-th coordinate of ((x)
[0)(1)) S Rs
W E B
Top-layer weight matrix
Columns are {wx}
W ERD
Class-k weight vector (column of W) Entry of W (feature j. class k) Predicted prob. for class k Alignment of feature for class k
||wx|l2 R.
Wik
(c(
Pa(x) [0,
(x)
ex((z) = (
BO)
On-batch alignment (per class, step t)
Batch statistic
BO
EMA alignment at step EMA smoothing parameter
=0 (11--))-V B + (-(11
1,
Tk
Class-F alignment threshold
Lower 9% percentile of (E.,)
9
Gate budget (percent)
most [4%] of D gated per class
Syk
Gate indicator for (j,A)
8jk (0,
&
Decay rate (gated shrinkage strength)
a a>0
Global decay coefficient
c 0.05
Per-step contraction on gated coords
P (1 a) ac) <
Sparsification threshold Mini-batch; [ its size Boundedness constants Pairwise margin & vs. t
Used in monotone shrink analysis
B
Larger [B] reduces variance
Rø Rw
[0)(1)) R4, ||wx||2 & R.
m)-s((a)
(wr wxy wxx) ((x)
me (x)
Margin after decay/gating step Set of gated indices for class Spurious component of Ø Spurious-alignment term
Used in Prop. 4
Depends T
Pspu
Decomposition & drob + Øspa
7spu
-EJEs. W (ospp))))
Jrab
Index robust (never-gated) feature
E jioh 2 Tk for all steps
D DERIVATION OF ALIGNMENT ENERGY
Recall from Eq. that the final classifier is linear the representation,
exp(2x(x))
(x) = Wi Φ0(x) Σ Wsk (z(x),
px(x)) = Σ ex)(2(()))
1=1
and that our per-neuron, per-class alignment score is defined
es((z) -x)x Wjk (z)
(A.1)
In the main text motivated eÿı (x) as confidence-weighte contribution of feature j to class k. Here we provide a derivation based on the free-energy view of the classifier, working in parameter space.
Free energy and sensitivity in weight space. model.
Consider the (temperature-1) free energy of the
E(x; W) log Σ exp (z<(x)) log Σ exp (w, (0(x)))
(A.2)
Diffcrentiating with respect to the logits yields
0E(x; W)
exp(zx(x))
- (x)
(A.3)
(2)(2) Σ, exp(21(x))
19

---

Published as a conference paper at ICLR 2026
Using the chain rule and the linearity of the logits, obtain the gradient of the free energy with respect to single weight Was:
(2 W)
30 0zz(x)
= (- (-Px(zz) (x)) (x)
(A.4)
aW, dzu @W
Hence infinitesimal changes A Wi yield the first-order perturbation
(x;;; W)
(E W)
AW jk
Σ 3.A
[p)(t)o5())AWH)
(A.5)
aW,k
J.k
Link-wise contribution and alignment energy. To isolate the contribution of a single link (j. k). consider straight-line interpolation weight space from reference value Wj to the current value Wi.- A first-order approximation attributes to this link the energy change
(x)((r) (E W)
(W,+ Wra).
(A.6)
SW,k
Setting Wi 0 for simplicity, and substituting Eq. (A.4), obtain
(x)x pa(x)) Φ5 (x)) Ws p4(x) Wsk 2z(2).
(A.7)
We therefore define the alignment score of link k) on input x as this first-order contribution:
(z)(z) = (
px(x) Ws Oz(),
(A.8)
which recovers Eq. This derivation provides a clear interpretation: (5t(x) is the first-order change in free energy induced by moving the weight Wj from 0 to its current value, holding the representation de (z) fixed. More negative alignment corresponds to stronger, confidence-weighte energy reduction along the link (j, k). Finally, averaging over inputs with label k yields the class- conditional alignment energy used in our method:
Ez = Ex>Da (z),
(A.9)
which is estimated while trining via exponential moving average and used for percentile-gated suppression during training.
E PROOFS OF THE FOUR PROPERTIES OF AGS
We consider multi-class classification with penultimate representation do(x) E R.D, final lin- ear layer W = [wi..... [wc] E DxC logits 2(x) W (1() and probabilities p& (I)
softmax, (Z)) For feature (neuron) j and class the (per-example) alignment score is
(x(())
(x(x) W, (x),
and class-k energy is the population average E = RZ [e)x(x)); in training we track an EMA
(i()) "jk of on-batch estimates (Eq. (5) The classwise percentile threshold (4) is the q-th percentile of
{E(1)
[(t) ik ..... E J. J. and the binary gate is o 16 (t), The weight update (applied before
backprop step f) is
Wjx + a1-
(2) (1 (1-ac) Wj with a > 0, 1=00000 0.05.
We will use the mild boundedness assumptions from 3.7
[0,(x) & Rs, ||wæel S Rw: Pk(x) [0,
Property (Confidence-weighted targeting and scale invariance). For y k, (st(22) becomes more negative as the alignment W Wiz0z(z) increases while (z(z) does not decrease thus higher- confidence, class-aligned contributions receive larger magnitude. Moreover, any logit-preserving rescaling ø' aQ, W W/a (with a 0) leaves WT ø W & and hence p' = pi consequently c',x()) e (x), the ordering of {E and the gated set are invariant. Lower- confidence samples (smaller pyy) produce alignment closer to 0 and thus exert less influence gating.
20

---

Published as a conference paper at ICLR 2026
Proof Fix example with = k. By definition (zk(z) p1( W ) If W W (z) increases and px(z) does not decrease, then -p((z) (x))(x) decreases (becomes more negative), i.e., lest (z) increases in magnitude with the same (negative) sign. Hence, larger confidence Pk pa( and stronger alignment W ))05(() jointly push (s((x) deeper into the negative tail.
For scale invariance, let ø' = a@ and W = -W/a with a > O. Then all classes W -W (x)) Pk((x), and
(zx(z) = (x(()) Wij 0f(2) = - (P(()) Ws
(a0, = ezx(z).
Therefore all (x) equal e5x(z) pointwise, their (EMA-Smoothed) classwise energies Esjt have identical order statistics within each class k, and the percentile-gated set {(j, k) Ezs < rK is unchanged. Finally, if Py is small, then (eju(x)] = Py|W,s9 رy (x) is closer to 0, so low-confidence examples contribute less to the tail. This proves the property:
Property 2 (Stable, budgeted gating via EMA and percentiles). The EMA update E(t) (1 F F(-1) (where 10, 1) is the EMA parameter) is convex combination with step-to-step drift bounded by 2(1 Ry. Per class k, at most [4%] features are gated (here q is the gate budget in percent), because decisions depend only on within-class order of {E.s). (t-1) + A (gap A > 0) and batch-inde aatch-indueed perturbations have size at most 8 (perturbation radius), then whenever
which is invariant under any strictly increasing transform. If E 2 Tk
2(1 3)5 < A,
feature j remains ungated at step (gate inertia). Larger mini-batches B (with size (B)) and bass-balanced sampling reduce the variance of E and hence the probability of flips.
Proof 2 Because (()) -1-3))EC 1BE(t-1 is a convex combination, for each (j, k).
F F((1-1) = (1 3) F - ((t-1) < (1 (3).
Hence (E(() jk
E(t-1) |1 < 2 R) establishing the stated drift bound.
Budgeted gating by percentiles. By definition, TE is the q-th percentile of the multiset {E(0 ..... ....,,,,),,,, With deterministic tie tie-breaking, most [4%] features are gated. Decisions are scale-free.
Gate inertia under bounded perturbations. Assume per-class sup-norm batch perturbation radius ò so that for i, (E(t) E b((t11) S 8. Then the entire vector E(()) deviates from b((t-1) by at most
(1 - (188)8 in sup norm, and any order statistic (thus the percentile) shifts by at most (1 3)5.
Consequently, if at step 1 we have a margin E(t-1) T
(t-1) A. then at step
((0)
2(1 3)8.
Jk
Larger mini-batches and class- balanced sampling reduce the variance of ( and hence the probability of flips. This concludes the proof.
Hence a sufficient condition for inertia 2(1 3)5 < A
21

---

Published as a conference paper at ICLR 2026
Property 3 (Multiplicative decay contracts and sparsifies), Under the update Wa (1 (1 ae) W jks with 55k E {0, 1] and 0.05, gated coordinates (8 1) contruct by = (1-a) a) ac) < 1 (the per-step contraction factor), yielding geometric attenuation across consecutive gated steps. For any threshold = > 0, the number of gated entries with magnitude weakly decreases after
step (monotone sparsification). To first order in a, the combined effect is equivalent to applying a class- conditiionn, feature-wist proximal shrinkage to gated links before the ERM update, plus a mild global decay stabilize scale.
Proof For any ki,
W = (1 (11 arc) [Waul with < (1 ai) ac) S ac) <1,
hence each coordinate contracts (strictly if either 8jk or c > 0). In particular, gated coordinate
(8ik- = 1J contracts by the factor (1 a1 ac) E (0, 1); iterating this over consecutive
gated steps yields geometric decay | S p² (1
jk
jk
For any fixed O. if 8yk and [W
then after step jel = p|W [W if [W S E remains Thus the count of gated coordinates with magnitude cannot increase, establishing monotone sparsification on the gated subset. (The global factor (1 - -xe) < 1 only helps this conclusion.)
Finally, expanding to first order in a gives
W = jas (c)) Wfa -(1 (1- c(ssa + +c) W 0(a2),
jk
which is exactly the proximal map of the quadratic penalty 2 (5zx +c) [W to first order (coordinate- wise shrinkage). This realizes a class-conditional, fealure-wise proximal shrinkage gated links (85k- = 1), plus a mild global shrinkage on all links, before the ERM gradient step, proving the property.
Property (Bias suppression with preservation of robust features), When gated coordinates align with spurious components on worst-group examples, the pairwise margin
my (x)) = (wr - we) (xx)
obeys
(x) 2 (x) + a "sspu ac (wk - (o)))((z))
where Sy denotes the set of indices gated for class k, Ospu is the spurious component in the decomposition o = &rab + Qapu, and "/spu
Σ Wjn (depa); 2 0. This shows nel margin
JESu
gain on worst-group inputs up to the small global decay term. Conversely, any feature whose
smoothed alignment persistently remains above the classwise percentile, Brack 2 Tk (defining the index jrob of a robust, never-gated feature), deterministically never gated and is therefore preserved.
Proof 4 Write @rob Ospu and recall the per-class gated set Sx (j Eygk Tk). Under one AGS step,
w = (1- - 8 O WK),
wt = (1 (cc a 8t C we),
where 8k (resp. 8t) is the 0/1 gate vector for class l (resp. 1) and "O" is the Hadamard product. Hence the updated margin satisfies
(x))(0) (wt TT To= -acc(we wx) T a(1 ac) [(s mx) (st wx) To].
22

---

Published as a conference paper at ICLR 2026
For any real u, (1 - ac)u 2 actul, u/. On worst- porst-group inputs the spurious part satisfies (sk O
wk) Qspu I 0. Therefore the term contributes a positive margin gain when we define
Tspu
Σ JESk
W (Sp(pss 2 O.
This yields (+) inputs up the small global decay term.
showing net margin gain on worst- worst-group group
Conversely, any feature whose smoothed alignment persistently remains above the classwise percentile, Bieak 2 Tk (defining the index Joob of a robust, rnver-gated feature), is deterministically never gated and is therefore preserved.
Notes assumptions and why they are used. We explicitly used the boundedness 10 RS. |un/22 [2 < Re. and Ps: E [0, 1] to: (i) upper-hound the magnitude of any alignment lejk(z) S Rg R (ii) control the EMA drift by 2(1 B) R.p... and (iii) ensure percentile thresholds well behaved. The gate inertia bound leverages a uniform per-batch perturbation radius to make a worst-case (yet clean) sup-norm argument; provided a simple sufficient condition 2(1 3)8 A and then noted it implies the (looser) condition spelled out in the paper for the default & range. Finally, Property 3 "proximal shrinkage" made precise via the first-order equivalence to the proximal map of quadratic penalty; this is the standard justification for multiplicative (coordinate- -wiee) shrinkage.
DISCLOSURE OF LLM USAGE
We have made limited use of a large language model to support tasks such as refining wording, improving readability, and suggesting alternative formulations. All conceptual contributions, experi- mental design, implementation, and interpretation of results were carried out by the authors, who take full responsibility for the final content.
23


Reviews:
[{"review_id": "zSBZSV9dBr", "reviewer": "Reviewer_2uU2", "summary": "This paper introduces Evidence-Gated Suppression (EGS), a lightweight, plug-in regularizer designed to combat shortcut learning in deep models without requiring group labels. EGS operates inside the network during training by tracking a class-conditional, confidence-weighted \"evidence energy\" for each neuron to identify which neurons contribute most strongly to the model's predictions. It then applies a percentile-based multiplicative decay to the weights of these extreme contributors, selectively suppressing overconfident shortcut pathways while leaving more robust features relatively influential. The authors demonstrate across several spurious correlation benchmarks (such as Waterbirds and CelebA) that EGS improves worst-group accuracy and calibration, achieving competitive performance with state-of-the-art methods while maintaining strong average accuracy and adding minimal training overhead.", "questions": "See weaknesses.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "While many methods address spurious correlations by reweighting data (e.g., JTT, LfF) or modifying the loss function (e.g., GroupDRO), EGS introduces a fundamentally different locus of intervention: the internal evidence pathways of the network itself. The concept of a dynamic, training-time, *neuron-level* regularizer that is both class-conditional and group-agnostic is original. The \"evidence energy\" metric, which combines model confidence (`pk(x)`) with feature-weight alignment (`Wjk * φj(x)`), provides a simple yet powerful signal for identifying and suppressing over-reliant pathways.", "weaknesses": "The central heuristic of EGS is that the most negative (i.e., highest-confidence, class-aligned) evidence corresponds to spurious features. While this holds true in many shortcut-learning scenarios, it is not guaranteed. A genuinely robust and highly discriminative feature could also consistently produce very strong evidence and be inadvertently suppressed by the percentile gate. The paper would be strengthened by an analysis that more directly validates this core assumption. For example, the authors could run EGS on a dataset known to have minimal spurious correlations (e.g., a balanced version of a dataset or even standard CIFAR-10) to demonstrate that the method does not harm performance when strong shortcuts are absent.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Za4eBMtLCl", "reviewer": "Reviewer_9w5c", "summary": "This paper proposes a method called Evidence-Gated Suppression (EGS).\n\nThe core idea is to track each neuron’s confidence-weighted evidence toward different classes and softly suppress those neurons that consistently provide strong but potentially spurious signals. \n\nIt uses a percentile-based gating mechanism and EMA smoothing to apply multiplicative decay to low-tail features, to weaken shortcut pathways while preserving robust ones.\n\nEGS achieves strong and consistent performance across CelebA, Waterbirds, BAR, and COCO.", "questions": "- I also how this kind of spurious correlation behaves in vision–language models (VLMs). Since VLMs often learn strong cross-modal associations, will it be different to the normal classification task like what are defined here?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- conceptually easy to understand and simple to implement. \n- works without any group or environment annotations, which makes it practical for real-world applications.\n- strong and consistent improvements on multiple spurious-correlation benchmarks (CelebA, Waterbirds, BAR, COCO).", "weaknesses": "- I noticed that the definition and formulation of evidence energy in Sections 3.1–3.2 look very similar to what was introduced in He et al. (2025). It would be helpful if the authors could clarify how their “evidence energy” differs conceptually or technically from that prior work.\n- it would be interesting to see how EGS works on Transformer architectures, since the paper mainly analyzes CNNs. A brief discussion or experiment on Transformers could make the work stronger.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "VrI4SANbMc", "reviewer": "Reviewer_mSKP", "summary": "The paper proposes Evidence-Gated Suppression (EGS), a regularizer that computes a per-neuron, class-conditional \"evidence\" score and suppress the feature neurons that are most likely spurious. The method tracks an EMA of the evidence energy and reduces those with over confidence. It achieves improvement across Waterbirds, CelebA, BAR and COCO with 5% computation overhead.", "questions": "1. The proposed evidence energy appears nearly identical to the formulation in EvA [1]. Could the authors clarify how their definition fundamentally differs from that of Eq. (7) in EvA? A more explicit comparison and proper attribution seem necessary.\n\n2. Have the authors evaluated EGS on large-scale benchmarks such as ImageNet or its spurious-feature variants [2, 3]?\n\n3. Have you compared with other classical penalties including weight decay, dropout or class weights adjusting?", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 1, "strengths": "1. The paper presents a clear conceptual framework: regulating internal, class-conditional confidence flows to mitigate shortcut reliance. Unlike many debiasing methods that act on the data level (e.g., reweighting, DRO), EGS directly intervenes on neuron–class weight inside the model.\n\n2. The paper supports its design with theoretical properties: (1) scale-invariance under logit reparameterization, (2) stability from EMA-based gating, (3) class-wise proximal shrinkage behavior, and (4) selective suppression of spurious features with preservation of robust ones.\n\n3. The paper shows strong empirical results on multiple spurious-correlation benchmark, demonstrating its effectiveness.", "weaknesses": "1. **The claimed contribution of defining evidence energy is highly concerning**. The formulation closely mirrors that of Eq. (7) in EvA [1], which already introduced a similar concept with exactly the same name evidence energy. Moreover, EvA-E also evaluates evidence energy on a class-wise basis to suppress spurious features. Both definitions essentially measure neuron confidence toward predictions. The primary distinction is that EGS integrates this process during training, whereas EvA applies it post-hoc. Although EvA-E is included in the experiments, the paper fails to explicitly discuss the differences and instead claims the definition of evidence energy as its own contribution.\n\n2. While the paper highlights low computational overhead, it omits critical details such as training FLOPs and total GPU hours. The claim that EGS is “plug-and-play” is weaker than that of previous post-hoc approaches, as it still requires full model retraining and tracking throughout the training process.\n\n3. The paper does not include experiments on large-scale or challenging datasets, such as spurious-feature variants on ImageNet-level [2, 3]. Such evaluations are essential to substantiate the claimed generalization and robustness benefits.\n\n[1] Erasing Spurious Correlations with Activations, ICLR 2025.\n\n[2] Salient ImageNet: How to discover spurious features in Deep Learning? ICLR 2022.\n\n[3] Large-Scale Detection of Harmful Spurious Features in ImageNet, ICCV 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "b37Ej4TUZP", "reviewer": "Reviewer_xEvJ", "summary": "This paper proposes tracking the contribution of each penultimate-layer feature to the classification prediction via an evidence score which is very negative for features which strongly support the prediction. The proposed Evidence Gated Scoring (EGS) algorithm estimates the evidence energy via an exponential moving average, then “gates” (zeros out with a stop-gradient) the features with the largest evidence score (in absolute magnitude). Experiments are provided which show EGS acts as a regularizer for overconfident “shortcut” neurons; empirical analyses, benchmarks, and a theoretical study are also provided.", "questions": "1. EGS is predicated on the assumption that any feature which disproportionately contributes to a certain class is likely to be a spurious feature (and so would be gated by EGS). What evidence is there that this assumption holds? I am concerned this assumption may not be justified in situations where the invariant and spurious features have similar “complexity”. For example, [1] show that the invariant and spurious features on CelebA (hair color and gender respectively) contribute roughly equally to the class prediction (Section 5.5).\n\n2. I’m curious how EGS relates to the line of work on feature diversification and ensembling, e.g., [2, 3, 4, 5], which similarly try to avoid overconfident shortcut-driven features.\n\n3. I wonder whether EGS can be formulated via a min-max objective which minimizes the maximum evidence energy contributed by any neuron, thus encouraging a roughly uniform contribution from each feature. If not, what is the difference with the EGS objective?\n\n4. Is EGS robust to model selection without group annotations, e.g., by selecting based on worst-class accuracy [6] or the bias-unsupervised validation score of [7]?\n\n5. Some clarity/grammatical improvements:\n    1. It would increase clarity to properly utilize \\citep to put parentheses around citations.\n\n    2. Missing semicolon or period in line 159\n\n    3. The order of Section 3.4 and Section 3.5 could potentially be switched since Section 3.4 references Section 3.5\n\n[1] Vasudeva et al. Mitigating Simplicity Bias in Deep Learning for Improved\nOOD Generalization and Robustness. TMLR 2024.\n\n[2] Addepalli et al. Feature Reconstruction From Outputs Can Mitigate Simplicity Bias in Neural Networks. ICLR 2023.\n\n[3] Rame et al. Model Ratatouille: Recycling Diverse Models for Out-of-Distribution Generalization. ICML 2023.\n\n[4] Zhang et al. Rich Feature Construction for the Optimization-Generalization Dilemma. ICML 2022.\n\n[5] Lin et al. Spurious Feature Diversification Improves Out-of-distribution Generalization. ICLR 2024.\n\n[6] Yang et al. Change is hard: A closer look at subpopulation shift. ICML 2023.\n\n[7] Tsirigotis et al. Group robust classification without any group information. NeurIPS 2023.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. The paper is very well-written with clear, intuitive explanations as well as rigorous experimental and theoretical justification. The “connections to broader ML principles” section in the Appendix is also a nice touch. Overall, beyond the proposed EGS method, this paper provides what I would term a “mechanistic lingua franca” for theorizing about how individual features affect group robustness.\n\n2. The analysis and ablation studies on the EGS method are well-done, especially the Section 5 discussion, and I appreciate the additional correlation studies in the Appendix. Overall, the EGS method is elegant and each of its components are strongly justified.\n\n3. The theoretical results complement the empirical analyses, with justified and clearly denoted assumptions.", "weaknesses": "The main weaknesses of the paper relate to the performance, evaluation, and comparison of the proposed EGS method.\n\n1. Worst-group accuracy performance is quite poor on Waterbirds; I disagree with the paper’s assessment that it has “near-top WGA” on this dataset (lines 370, 448). State-of-the-art methods should achieve 90% or higher WGA [1, 2]. More importantly, ERM with simple class-balancing can attain 82.9% WGA [3, Table 2], meaning that EGS is worse than a standard data balancing baseline on this dataset.\n\n2. More generally, the performance comparisons in Table 1 do not reflect the state-of-the-art; only a single reference from after 2023 is listed. Please consider including more recent competitive methods, e.g., [2, 4, 5, etc].\n\n3. I’m a bit confused on the difference between _conflicting_ accuracy and _worst-group_ accuracy, and why the former is reported for CelebA/COCO while the latter is reported for Waterbirds. It would be nice to also report WGA for CelebA/COCO to compare with results from the literature which also report WGA.\n\n4. If the main claim on the performance of EGS is that it “advances the Pareto frontier” of group robustness methods, then a method for interpolating between different average vs. worst-group trade-offs should be provided. In particular, a method which achieves Pareto gain at an _a priori_ undetermined point on the curve is not very useful in practice, as the user is unsure of what trade-off to expect before deployment, and it may lose out to techniques which more explicitly target WGA (e.g., as EGS does on Waterbirds).\n\n5. Only a single data domain (computer vision) and model architecture (ResNet50) are examined. While not strictly necessary, it would improve the claims to benchmark EGS using BERT on language datasets such as CivilComments or MultiNLI, as is common in the group robustness literature.\n\n[1] Kirichenko et al. Last Layer Re-Training is Sufficient for Robustness to Spurious Correlations. ICLR 2023.\n\n[2] Noohdani et al. Decompose-and-Compose: A Compositional Approach to Mitigating Spurious Correlation. CVPR 2024.\n\n[3] Idrissi et al. Simple data balancing achieves competitive worst-group-accuracy. CLeaR 2022.\n\n[4] Yang et al. Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias. AISTATS 2024.\n\n[5] Han et al. Improving Group Robustness on Spurious Correlation Requires Preciser Group Inference. ICML 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)