Scalable Energy-Based Models via Adversarial Training: Unifying Discrimination and Generation
Authors: Xuwang Yin
Claire Zhang
Julie Steele
Nir N Shavit
Tony T. Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Joint Modeling
Energy-Based Models (EBMs)
Adversarial Training
Robust Classification
Generative Modeling
PGD Attacks
explainability
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: Adversarial training replaces unstable SGLD in Joint Energy-Based Models, scaling EBM-based hybrids to ImageNet 256×256   with state-of-the-art robust classification and generation quality in a single model.

Abstract: Simultaneously achieving robust classification and high-fidelity generative modeling within a single framework presents a significant challenge. Hybrid approaches, such as Joint Energy-Based Models (JEM), interpret classifiers as EBMs but are often limited by the instability and poor sample quality inherent in training based on Stochastic Gradient Langevin Dynamics (SGLD). We address these limitations by proposing a novel training framework that integrates adversarial training (AT) principles for both discriminative robustness and stable generative learning. The proposed method introduces three key innovations: (1) the replacement of SGLD-based JEM learning with a stable, AT-based approach that optimizes the energy function through a Binary Cross-Entropy (BCE) loss that discriminates between real data and contrastive samples generated via Projected Gradient Descent (PGD); (2) adversarial training for the discriminative component that enhances classification robustness while implicitly providing the gradient regularization needed for stable EBM training; and (3) a two-stage training strategy that addresses normalization-related instabilities and enables leveraging pretrained robust classifiers, generalizing effectively across architectures.   Experiments on CIFAR-10/100 and ImageNet demonstrate that our approach: (1) is the first EBM-based hybrid to scale to high-resolution datasets with high training stability, simultaneously achieving state-of-the-art discriminative and generative performance on ImageNet 256$\times$256; (2) uniquely combines generative quality with adversarial robustness, enabling faithful counterfactual explanations; and (3) functions as a competitive standalone generative model, matching autoregressive models and surpassing diffusion models while offering additional versatility.

Published as a conference paper at ICLR 2026
SCALABLE ENERGY-BASED MODELS VIA ADVERSAR- IAL TRAINING: UNIFYING DISCRIMINATION AND GEN- ERATION
Xuwang Yin'
Claire Zhang MIT
Julie Steele
Independent
MIT
xuwangyinggmail.. com
clairefz@mit.ed
jssteele@mit.co edu
Nir Shavit MIT
Tony T. Wang MIT t twang5emit.edu
shanirecsail,
ABSTRACT
Simultaneously achieving robust classification and high-fidelity generative model- ing within single framework presents a significant challenge. Hybrid approaches, such as Joint Energy-Bautt -Baee Models (JEM), interpret classifiers as EBMs but are often limited by the instability and poor sample quality inherent in training based on Stochastic Gradient Langevin Dynamics (SGLD). We address these limita- tions by proposing novel training framework that integrates adversarial training (AT) principles for both discriminative robustness and stable generative learning. The proposed method introduces three key innovations: (1) the replacement of SGLD-based JEM learning with a stable, AT-based approach that optimizes the energy function through Binary Cross-Entrope (BCE) loss that discriminates be- tween real data and contrastive samples generated via Projected Gradient Descent (PGD); (2) adversarial training for the discriminative component that enhances classification robustness while implicitly providing the gradient regularization needed for stable EBM training; and (3) two-stage training strategy that ad- dresses normalization-relate
instabilities and enables leveraging pretrained robust classifiers, generalizing effectively across architectures. Experiments on CIFAR- 10/100 and ImageNet demonstrate that our approach: (1) is the first EBM-based hybrid to scale to high-resolution datasets with high training stability, simulta- neously achieving state-of-the-art discriminative and generative performance on ImageNet 256x (2) uniquely combines generative quality with adversarial robustness, enabling faithful counterfactual explanations; and (3) functions as a competitive standalone generative model, matching autoregressive models and surpassing diffusion models while offering additional versatility.
INTRODUCTION
Deep learning models have traditionally been developed with either discriminative generative objectives in mind, rarely excelling at both simultaneously (Ng and Jordan, 20001] Jebara 2004
Lasserre al.) 2006 Xie et al. 2016 Grathwohl et al. 2019). Discriminative models optimized for classification or regression tasks but lack the ability to model data distributions, while generative models can synthesize new data samples but may underperform
downstream classification tasks (Ng and Jordan] 2001 Jebara 2004). Recent research has explored unifying these approaches through joint discriminative- weweerrtgggeeeeeeeeeeeeeeeeee modeling frameworks that aim combine the predictive power of discriminative approaches with the rich data understanding of generative models (Xie et al.)
2016
Lazarow et al. 2017 Jin 2017; Du and Mordatch, 2019 Grathwohl al. 2019 Chen Guo et al. 2023 Deja et al. 2023). Such unification holds the promise of grounding
et al.
Guo
"Correspondence xuwangyl n@gma i com. Code: https://github: com http:/uuww..ooouuuuggggggggnnaaa www.wxwwwnggnnnoooo

---

Published as a conference paper at ICLR 2026
classification decisions in the model's understanding of the data distribution but realizing this potential requires more than simply combining discriminative and generative objectives.
Among these unification efforts, Energy- -aarey Models (EBMs) have emerged promising frame- work due to their flexibility and theoretical connections to both paradigms. In particular, Joint Energy-Basse Based Models (JEM) (Grathwohl et al. 2019) demonstrated that standard classifier archi- tectures could be reinterpreted to simultaneously function as EBMs, enabling both high-accuracy classification and reasonable sample generation. However, critical limitation of JEM and similar approaches is their reliance on Markov Chain Monte Carlo (MCMC) methods such as Stochastic Gradient Langevin Dynamics (SGLD; (SGLD:|Welling and Teh 201T) for training the generative component. CGLD-based EBM learning suffers from significant training instabilities, computational inefficiency. and often produces qua-qqualty samples (Grathwohl et al. 2019] Duvenaud et al., 2021: Du and Mordatch 2019: Nijkamp et al. 2019), limiting the practical adoption of these hybrid models.
We address these limitations by introducing Dual Adversarial Training (DAT), a novel framework that leverages adversarial training (AT) principles for both discriminative robustness and stable generative learning within unified JEM-based architecture. Our approach employs a dual application of adversarial training: (1) standard AT for the discriminative component to achieve robustness against adversarial perturbations, and (2) an AT-based energy function learning strategy for the generative component that replaces unstable SGLD- based JEM learning.
Our key technical contributions include:
1. A stable AT-based alternative to SGLD-based JEM learning. We replace the unstable SGLD- based JEM learning with an adversarial training approach that optimizes the energy function through a Binary Cross-Entropp loss that discriminates between real data and contrastive samples generated via PGD (Madry et al.) (017). This addresses the training instabilities of JEM, enabling reliable convergence and significantly improved sample quality.
2. Robustness and implicit regularization from adversarial training. We incorporate adversarial training for the discriminative component, which not only enhances classification robustness but also eliminates the need for the explicit Rú gradient penalty (Mescheder et al. (2018) required by previous AT-EBM frameworks (Yin et al. 2022),, simplifying the training procedure and avoiding constraints model expressiveness.
3. Two-stage training strategy. We introduce two-stage training strategy that leverages pre- trained robust classifiers and addresses normalization relltte instabilities, generalizing across architectures with batch normalization (ResNet) and layer normalization (ConvNeXt).
Experiments on CIFAR- and ImageNet demonstrate the effectiveness and scalability of our approach, establishing three advances in hybrid modeling:
1. First EBM-based hybrid that scales to high-resolution complex datasets. Prior EBM-based hybrid approaches could not scale beyond low resolution or achieve competitive generative performance ImageNet-level datasets due to SGLD instability. Our approach is the first to overcome these limitations, achieving competitive generative quality and strong classification performance ImageNet 256x with high training stability, demonstrating that EBM-based hybrid models can scale reliably to complex, high-resolution datasets.
Generative capability with adversarial robustness enables counterfactual explanations. Our approach uniquely combines state-of-the- generative quality with adversarial robustness, enabling the model to generate visual counterfactual explanations using the exact same energy function that determines its classification decisions. We demonstrate that our counterfactuals substantially more perceptually realistic and semantically faithful to target class than those from non-robust or robustness-only methods.
Competitive and flexible generative model. When evaluated on generation quality, our approach (with ConwNeXt--arge. matches the autoregressive model VAR-d16 and surpasses diffusion models on ImageNet 256x256. while achieving higher throughput than diffusion models. Beyond this competitive quality, our approach is versatile across diverse image synthesis tasks (Santurkar et al. 2019) and uniquely supports compositional generation (Du et al. 2020).
These results show that adversarial training provides an effective and scalable foundation for energy- based generative learning, enabling hybrid models that need not compromise on any dimension.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Joint discriminativv--eeeenai modeling The pursuit of joint discriminative-genenna modeling, or hybrid modeling, aims to combine the predictive power of discriminative approaches with the rich data understanding generative models within single framework. This line of research is motivated by the potential to improve classifier robustness, calibration, and -i--o---ittiiuttion detection (Grath wohl al. 2019 Du and Mordatch, 2019), while also enabling tasks like sample generation (e.g.. for counterfactual explanation (Deja et al. 2023)) and semi-superveen learning (Kingma et al.) 2014). A significant thrust this area involves Energy-Based Models (EBMs) (LeCun et al. 2006). Early work by Xie et al. 2016) showed how generative ConvNets could be derived from discriminative ones, framing them as EBMs. Du and Mordatch (2019) scaled EBM training to complex nigh-dimensional image datasets and showed that the same energy function can be used for discriminative tasks such as out-of-distrie of--iistribttoonn detection and robust classification, without introducing task fask-specific objectives. Grathwohl et al. (2019) introduced JEM, which explicitly reinterprets standard classifiers as EBMs over the joint distribution of data and labels p(2, y). allowing simultaneous classification and gen- eration. Yang et al. (2023) incorporated sharpness-aware minimization (SAM) to smooth energy landscapes and removed data augmentation from the EBM loss term to improve both classification accuracy and generation quality of JEM. Guo al. (202) proposed EGC, which employs Fisher divergence within diffusion framework to learn an unconditional score function V log p(z) and a conditional classifier p(y|æ) for unified classification and generation, thereby circumventing the training instability and scalability limitations of traditional EBMs.
Alternative architectural approaches have also been explored for joint modeling. Rather than energy- based formulations, joint diffusion models (Deja et al., (202) attach classifiers directly to diffusion model UNet encoders for joint end-to-end training. Another distinct approach is "introspective learning." where single model functions as both generator and discriminator through an iterative self-evaluation process, developed across works by Lazarow et al. (2017). Jin et al.) (2017). and Lee ct al. (2018). based models have also been explored for hybrid tasks; for instance, Residual Flows (Chen et al.] 2019) utilized invertible ResNet and showed competitive performance in joint generative and discriminative settings. These diverse approaches underscore the continued effort to create models that jointly leverage both discriminative and generative learning.
Joint Energy-Based Models (JEM) Grathwohl et al. (2019) showed that the logits of standard classifier can be reinterpreted as defining an energy function over the joint distribution p(r, v). enabling simultaneous classification and generation within a single architecture. Their hybrid training objective combines cross-entropy for ply (z) with an SGLD- b-bad EBM objective for p(æ). improving calibration, OOD detection, and adversarial robustness over standard training. Subsequent works improved JEM's s training stability: JEM++ (Yang and Ji. 2021) introduced proximal SGLD and informative initialization, while Robust-JEM (Korst and Asadulaev 2022) incorporated adversarial training into the discriminative component. Yang et al. (2023) applied sharpness-aware minimization to smooth energy landscapes and decoupled data augmentation from the EBM loss. However, all these methods fundamentally rely SGLD-based sampling for the generative component, inheriting its instability, and remain limited to CIFAR- scale (32 x 32) datasets.
Ad Adversarial training and energy- based models Several works have revealed deep connections between adversarial robustness and energy-based modeling. Zhu et al. (2021) showed that adversarial training implicitly flattens the energy landscape around real data, and proposed JEAT for joint classification and generation. WWan et al. (2022a) unified adversarial training and contrastive learning under EBM framework, showing that "GD-generated adversarial examples serve as implicit negative samples (see also MMiza et al. 2024). Separately, YYn et al. 2022) proposed AT-EBM, which replaces SGLD with PGD-based contrastive sampling and a BCE loss for learning the energy function, achieving more stable training and competitive generation. though limited to unconditional generation with an explicit Rj gradient penalty. /Augustin et al. (2020) proposed RATIO, combining in-distribution AT with OOD adversarial training for robust confidence calibration and OOD detection,
also enabling visual counterfactuals through l/2 robustness. /anntrrar et al. (2019) demonstrated that robust classifiers can serve as primitives for diverse image synthesis tasks via gradient-based optimization. Our work synthesizes these lines of research by incorporating AT-based EBM learning into the JEM framework for conditional generative modeling, with implicit R1 regularization from adversarial training eliminating the need for explicit gradient penaltics. See Appendix A.I for extended discussion.
3

---

Published as a conference paper at ICLR 2026
3 METHOD
3.1 JOINT ENERGY-BASED MODEL
Our approach builds upon the JEM framework (Grathwohl et al. 2019), which reinterprets the outputs of standard discriminative classifier as an energy-based model (EBM) over the joint distribution of data and labels y. Given classifier network that produces logits fo (z) E R for classes, JEM defines the joint energy function as:
Fo(x, v) = = (o)
where [0]
[y] is the logit corresponding to class y. This energy function can be normalized obtain joint probability density:
(= = (ep(-Fo(a,,)) v)) exp(fo(x)(v)
Z(A)
Z(0)
where =(0)=Ey, f exp (oroffo is the partition function (an intractable global normalizing constant). By marginalizing out the label marginal density over the input data can be obtained:
Σ (o 9) Ey exp(Jo(x))x))
Po
2(0)
Thus, a valid energy function for pe(x) is given by:
(x(x) = - log Σ exp((f(<x)())
(4)
A JEM is trained by maximizing the joint log-likelihood log po(,, y) over labeled training datapoints (2c, y) drawn from an empirical joint distribution (1, y). The joint log-likelihood is typically
factorized as log Pa (y]xx) + log (z)) The conditional term log (3(l1z) be maximized by minimizing the standard cross-entropy classification loss. The marginal term log (x is optimized using the EBM gradient (LeCun et al. 2006):
V Vg VoByr-mans)) [log pe (x:) Ex--pa Vo - - Eo
(5)
where (x) is the empirical marginal distribution obtained by marginalizing y from Pdata (z, v). This gradient decreases the energy of real data samples while increasing the energy of model- generated samples. At equilibrium when pz(z) Pdaa(:), these terms balance and the gradient becomes zero.
To approximate the expectation Ex-po(=) [.]. samples are drawn from (e(x) using SGLD (Welling and Teh 1)11), which starts from an initial distribution po(z) (e.g.. uniform noise) and iteratively applies the update rule:
F+41 Z4
(oEo( + Ee. where & a)),a)
(6)
3.2 LEARNING JEM WITH ADVERSARIAL TRAINING
The JEM framework successfully integrates generative modeling into classifiers, but its reliance
SGLD and EBM gradient (Eq. 5) causes significant training instabilities (Grathwohl et al. 2019
Duvenaud al. 2021) and results in poor sample quality. We address these limitations by replacing
The SGLD-based JEM with an adversarial training (AT) approach inspired by AT-EBM (Yin et al.
2022).
Concretely, we replace the standard EBM gradient (Eq. 5 with stabilized formulation:
Ex (w) V [x) E Ex)po(x) V Fx))
= E Ex)pan(e) - a ( () - -
(7)
where a(x) 1 - (x)) and ((x) = o-- (x))) are data-dependent scaling factors, and denotes the logistic sigmoid function. This formulation preserves the structural form of Eq.
4

---

Published as a conference paper at ICLR 2026
while introducing adaptive scaling factors that modulate gradient contributions according to the model's current energy values. These scaling factors stabilize training by providing automatic gradient regularization (Yin et al., 2022): when E (z) takes extreme values, the sigmoid saturation drives the corresponding scaling factor (a for Pdata samples, ,3 for contrastive samples) toward zero, attenuating gradient contributions and preventing numerical overflow and underflow. In contrast, the standard EBM gradient (Eq. 5) is unconstrained and permits - - to grow unbounded, resulting in numerical instability. The corresponding training objective, whose gradient with respect 0 recovers Eq. can be written as a Binary Cross- Cross-Entropy (BCE) loss:
(8) This gradient formulation stabilizes training the cost of limiting the EBM to modeling the support of Pdata rather than learning the full density. We provide a formal characterization distribution in Section [.2]] where we show that the optimal solution under the joint discriminative-
(BCE(O) -Er-paa(=) (oogelloo((((aaaaaaa
(s) [log(1 - o(--o(z)))]
the learned
generative objective learns f (x)[y) = log Panaa (yla( x)) on the support with constant marginal energy (x) = O.
In addition to the above gradient reformulation, we follow Yin et al. (2022) in replacing JEM's SGLD sampling with PGD, initializing from an auxiliary out-of-distribution dataset Pood (e.g.. the 80 Million Tiny Images dataset (Torralba et al., 2008) for CIFAR- 10), Concretely, contrastive samples are generated by performing T' steps of normalized gradient descent on (z)
Eg(x,)
It+1 = 24 - 7 Eo == = t. T -1
The PGD procedure transforms OOD images toward the data distribution during training. At test time, the same mechanism produces samples with competitive FID scores (Section (43 and enables counterfactual generation from existing images (Section 4.3.2). While OOD initialization yields the best generation quality, we show that DAT can also be trained from pure random noise, eliminating the dependence on auxiliary datasets (Appendix C.5
3.3 CLASSIFIER ROBUSTNESS AND IMPLICIT REGULARIZATION
Classifier robustness. While our AT-based approach improves the generative capabilities of JEM, the discriminative component still exhibits weaker adversarial robustness compared to standard AT classifiers. address this limitation, we complement the generative improvements by incorporating adversarial training for the discriminative term pe (y/zz)
For each input sample z with label y, we find an adversarial example Todv within an e-ball B(x, e)
around v that maximizes the classification loss:
Fade arg max LCE(0; z'EB(z.e)
(10)
where (0;(0;,',)) ) is the standard cross-entropy loss and Bxx, e) is an fp-norm ball. Similar to generative component, we approximate this optimization using the PGD attack, generating adversarial examples through iterative gradient steps within the constraint set. The classification term is then defined as:
LAUCE (0) ((x.))=vuxxz.)) |- log po [y]=ad)))]
(11)
Implicit regularization. Incorporating AT for the classifier not only ensures robust accuracy but also yields an additional benefit for the generative component. The original AT-EBM framework required explicit R2 gradient penalties (Mescheder et al. 2018) for training stability; we find that AT eliminates this need. Building on Roth al. (2020). we show that AT implicitly bounds the R1 penalty (Section[A.3) We empirically validate the effect of AT on R₁ gradients: AT maintains bounded R gradients throughout training, while standard training exhibits gradient explosion (Figure
2
3.4 DUAL AT FOR JOINT MODELING
Our complete approach applies adversarial training to both the generative and discriminative compo- nents, resulting in the combined objective:
L()) (ATCE(()) ))) Lucr (0)
(12)
5

---

Published as a conference paper at ICLR 2026
where CAT-CE (f) is the robust classification loss from Eq. 11 and (BC () is the AT-based generative loss from Eq. This combination corresponds to the factorization the joint log-likelihood the original JEM formulation: log po(z, y) log ( log po (Ir). The joint objective simultancously enhances the model's discriminative robustness and generative capabilities, addressing the key limitations of the original JEM framework; full algorithmic details are provided in Appendix AA.4
Our approach shares conceptual similarities with RATIO (Augustin et al. 2020,, which also combines adversarially robust classification with adversarial perturbations applied to out-of-distribution data:
LRATIO = CATCCE() + AB pood)))
max ('EB(Z,Co) LcE(0:z',1/1) 1/KK)
(13)
Despite this structural similarity, the approaches differ fundamentally in their objectives. RATIO's secondary term attacks OOD samples to maximize classifier confidence, then penalizes this confidence via cross-entropy against a uniform distribution, explicitly targeting robust OOD detection. In contrast, our LECE (0) leverages AT-based energy function learning Yin et al. 2022). using PGD to generate contrastive samples from OOD data and employing BCE loss to shape the energy landscape. While RATIO focuses primarily reducing confidence in OOD regions, approach prioritizes learning an energy function that enables high-quality sample generation alongside robust classification.
3.5 TWO-STAGE TRAINING
Neural network architectures typically incorporate normalization layers to stabilize and speed up training: ResNet (He et al.) 2016) uses batch normalization (BN) (lofte and Szegedy 2015), while modern architectures like ConvNeXt (Liu et al. 2022) and Vision Transformers (Dosovitskiy et al.) Vaswani Vaswani et al., 2017) use layer normalization (Ba al.) 2016). Training energy- joint models presents challenges with normalization layers. In particular, batch normalization has been identified as problematic for EBM training (Grathwohl al.] 2019 Yin et al.) [022]] Zhao al., 2020). Consistent with these findings, observe that enabling BN during joint training destabilizes the optimization of the generative modeling term LBCE. leading to oscillating losses and failure to converge.
To address these challenges while maintaining the benefits of normalization during discriminative training. we propose two-stage training strategy that generalizes effectively across architcctures:
Stage 1: Diseriminative training. We first train the network with its original normalization configuration, optimizing only the robust classification objective CAT-CE (Eq. [T]. This stage is equivalent to standard adversarial training and leverages normalization layers to achieve faster convergence and strong robust classification performance. Notably, this stage can be skipped when pretrained robust classifiers are available, making our approach immediately applicable existing robust models.
Stage Joint training. After robust discriminative training, modify the normalization behavior when necessary and continue training with the complete objective ((0) = LAFCH(()) (0) + LBCE(() (Eq. [2]. For architectures with batch normalization (ResNet, WRN), we disable BN by setting BN modules to eval mode, which freezes the BN statistics computed during Stage 1. For architectures with layer normalization (ConvNeXt), we maintain the normalization as-is.
This strategy not only addresses the incompatibility between batch normalization and EBM training, but also enables leveraging pretrained robust classifiers to reduce training costs (see Appendix B. for detailed computational analysis). As demonstrated Section 4.3) Stage 2 improves the generative modeling performance of pretrained robust classifiers with minimal impact on the robust accuracy established in Stage (see Appendix B.3 for training dynamics). The two-stage training strategy works effectively for both ResNet and ConvNeXt models, making it applicable to modern scalable architectures such as Vision Transformers (Dosovitskiy et al. 2021 [Singh et al., [203 Peebles and
Xie 2023a)

---

Published as a conference paper at ICLR 2026
4 EXPERIMENTS
4.1 TRAINING SETUP
Datasets and architectures. We evaluate our approach on CIFAR-10, CIFAR-I00 100 (Krizhessky et al. 2009). and ImageNet (Deng et al. 2009). For CIFAR-10/100 experiments, we use WRN-34- 10 Zagoruyko and Komodakis (016) fowig the official RATIO implementation (Augustin et al. ResNet-50 (He et al. 2016)) WRN-50-4 (Zagoruyko and Komodakis 2016), and LonyNeXt-Large with ConvStem (Singh et al. 2023).
2020). For ImageNet experiments,
Two-stage training. Since Stage 1 training is equivalent to standard adversarial training, pretrained standard AT checkpoints when available: a standard AT checkpoint from the RATIO codebase (Augustin al., 2020) for CIFAR-10, pretrained ImageNet ResNet-50 and WRN-50-4
models (Salman et al. 2020), and pretrained ConyNeXt-Large with ConvStem (Singh et al. 2023 (originally trained for &o = 4/255 robustness), while training our own CIFAR- 100 model following Augustin et al. 2222)) For Stage 2 training, we initialize from the Stage 1 model and continue joint training For ResNet and WRN architectures, we the BN modules to eval mode (which disables BN while preserving the BN statistics computed during Stage 1). Complete training hyperparameters can be found in Appendix B.I.T.
Data augmentation. Strong data augmentations are necessary for classifier robustness (Rebuffi ct al. 2021] Gowal al., (2200) but can distort the data distribution ways detrimental to generative modeling. Following Yang et al. (2023). we use separate augmentation strategies for Stage 2 training: strong augmentations for LAT-CE and basic transformations for LACF. Yang et al. (2023) found that augmentations such as random cropping with padding introduce artifacts (e.g.. black borders) in generated samples and therefore excluded them from generative training. We observe that this is not a limitation in framework-even with random cropping and padding applied, our generated samples do not exhibit such artifacts (Appendix B.1.2). Out-of-distribution data. Following RATIO (Augustin et al. images (Torralba et al.) (008) as the OOD dataset (Pood) for
the 80 million tiny CIFAR-10/100 experiments. For ImageNet, as there are no established OOD datasets, we follow OpenImage-O (Wang et al.) (0226) and construct an OOD dataset from the Open Images training set (Krasin et al. 2016). We randomly sample 350K images, restricting our selection those whose labels do not overlap with any ImageNet classes, yielding 300K samples for training and 50K for FID evaluation.
2020),
4.2 EVALUATION METRICS
We measure both classification and generative modeling performance. For classification, we report clean accuracy and robust accuracy against & attacks (e 0.5 for DEFAR-10/100 and 3.0 for ImageNet) computed using AutoAttack (Croce and Hein 2020). For generative modeling, we evaluate sample diversity and visual fidelity using Fréchet Inception Distance (FID) (Heusel et al. (017) and Inception Score (IS) (Salimans et al. (216)). We focus on conditional generation, which consistently outperforms unconditional generation across all datasets (Table 9): details of the generation setup are provided in Appendix B.4
To measure counterfactual quality, we apply targeted PGD attacks to training samples across a range of perturbation budgets and compute class-wise FID between the resulting counterfactuals and real samples of each target class. This differs from generative sample evaluation, where PGD is applied to OOD inputs rather than in-distribution data.
4.3 RESULTS
We evaluate DAT on CIFAR-10, CIFAR- 100, and ImageNet 256x256 (Tables and | [ below, our approach achieves the first successful scaling of EBM- based hybrids to high resolutions while simultaneously achieving robust classification and high-fidelity generation.
2). As detailed
First EBM-based hybrid to scale to high-resolution datasets with adversarial robustness. Prior EBM-based hybrid approaches (JEM, SADA-JEM) not explicitly optimized for adversarial robustness. On CIFAR-10, these methods achieve significantly lower robust accuracy than standard AT: JEM achieves 40.5% and SADA-JEM achieves 31. 11.9%, compared to 75.73% for standard AT
7

---

Published as a conference paper at ICLR 2026
(Table I. Our approach addresses this limitation, achieving 75. 75% robust accuracy comparable to standard AT while improving generative quality over prior EBM hybrids: FID 9.12 versus 38.4 (JEM) and 9.41 (SADA-JEM), Beyond robustness, prior EBM-baser hybrids could not scale beyond low resolution or achieve competitive generative performance on ImageNet-level datasets. Our approach is the first to overcome this limitation: on ImageNet 256x 256, our ConyNeXt-LLargy model achieves FID 3.29 (Table 2 with classification performance comparable to standard AT (Appendix C. demonstrating that EBM-based hybrid models can scale reliably complex, high- resolution datasets.
Unique combination of generation quality and adversarial robustness. While other scalable hybrids exist, they do not achieve both strong adversarial robustness and state-of-the tet-thh-artt art generative quality. The diffusion based EGC achieves 13.56% robust accuracy on ImageNet compared to our 56.40%, with worse FIID (6.05 vs. 3.29) (Table 2). Similarly, RATIO targets robustness but not generation quality (FID 21 .96 vs. our 9.12 on CIFAR-10) Qualitatively, Figures 6 7 and show that our method produces visually superior samples with fewer artifacts compared to RATIO and standard AT. This unique combination generative quality and robustness enables our model to produce substantially higher-qualin counterfactual explanations than both non-robust and robustness-only models (Section 4.3.2).
Competitive as standalone generative model. When evaluated purely generation quality, our approach achieves performance competitive with state-of-the-art specialized generative models. On ImageNet 256 DAT with ConwNeXt-L achieves FID 3.29, matching the state-of-the-an autoregressive model VAR-d16 (FID 3.30) while using fewer parameters (198M vs. 310M) and outperforming leading diffusion models including ADM-G (FID 4.59, 608M parameters) and LDM-4- G (FID 3.60, 400M parameters) (Table 2). The model also achieves relatively strong IS performance (310.2), likely due to PGD-based sampling explicitly optimizing for classifier confidence. Figure P shows representative samples demonstrating the visual quality achieved by our approach. Beyond quality, our approach achieves significantly higher throughput than diffusion models: ~99 x faster than ADM-G and ~5 faster than LDM-4-G (Table 7).
4.3.1 ANALYSIS AND ABLATIONS
Noise initialization. DAT can be trained without any OOD data by initializing PGD from pure random noise, eliminating the dependence auxiliary datasets. Noise-initialized DAT maintains comparable adversarial robustness to OOD initialization while achieving competitive generation quality (Appendix C.5).
trade-off. Our experiments reveal that the number of PGD iterations T (Eq. 9 controls the balance between discriminative and generative performance. On CIFAR- 10, increasing I" from 40 to 50 improves FID from 9.12 to 7.57 at the cost of standard and robust accuracy, similar trend observed across other datasets. In Appendix C.8.3] we investigate this tension, showing that the generative objective aligns representations with Pdata which can come at the cost of robustness. We further show that this trade-off can be explicitly tuned through loss weighting in addition to PGD iterations.
Generative-discriinnatie
Effect of model capacity and architecture. Our experiments on ImageNet demonstrate the benefits of increased model capacity and modern architectures. Scaling from ResNet-50 (26M parameters) to WRN-50-4 (223M parameters) yields consistent improvements across both discriminative and generative metrics. Beyond capacity, using slate-of-the-art architectures also provides clear benefits: ConvNeXI-L (198M parameters) substantially outperforms WRN-50-4 (223M parameters) both accuracy and generation quality despite having fewer parameters, demonstrating the importance of architectural design alongside model scale.
Component contributions. We isolate the contributions of our key components, showing that the generative loss and decoupled augmentation are essential for high-fidelity synthesis (oppndix(C.8..11
OOD data efficiency. We also demonstrate high data efficiency, achieving strong performance with limited auxiliary OOD data (Appendix (CS.2).
Robustness and stability. Beyond adversarial robustness, DAT maintains corruption robustness comparable to standard AT (Appendix(C.) and generalizes to Eo training (Appendix We also demonstrate high reproducibility with zero training divergence across all runs (Appendix C.9
8

---

Published as a conference paper at ICLR 2026
Computational efficiency. Finally, analyze computational cost, showing that two-stage training incurs only modest overhead (1.05-1.56x) relative to standard AT, and that our model achieves significantly higher inference throughput than diffusion models (Appendix B.2
Table I: Classification and generative modeling results on CIFAR-10 and CIFAR-100.
Method
Acc% f
Robust Acc% f
IS 1
FID !
CIFAR-1O hybrid models
Residual Flow (Chen
2019)
70.3
3.6
46,4
Glow Kingma and Dharrwal 2018
67.6
3.92
48,9
IGEBM (Du and Mordatch 2019
49.1
8.3
37.9
JEM Grathwohi et al. (019)
92.9
40.5
8.76
38.4
VERA Grathwohl
93.2
8.11
30.5
JEM++ Yang and Ji 2021)
94.1
8.11
38.0
JEAT (Zhu et al. 2021)
85.16
8,80
38.24
Robust-JEMT (Korst and Asadulacv 2022
8.71
41.17
SADA-JEM (Yang
(023))
95.5
31.93
8.77
9.41
WEAT (Mirza
[2024]
83.36
8.97
30.74
EGC (Guo all]]]10000 20223
95.9
9,43
3.30
loint-Dillusion Deja cl al 2023
96.4
6.4
RATIO Augusttn al. 2020)
92.23
76.25
9.61
21.96
Standard AT (Augustin
2020
92.43
75.73
9.58
28.41
DAT 40)
91.92
75.75
9,92
9.12
DAT (I' 50)
90.72
74.65
9.86
7.57
CIFAR-1O conditional generative models
SNGAN Miyato
2018
8.59
25.5
BigGAN Brack et al.
2018
9.22
14.73
StyleGANZ (Karras of al.
2020a
9.53
6.96
StyleGAN2 ADA (Karra
2020b
10.24
3.49
EDM (Karras
2022
1.79
CIFAR- 100 hybrid models
Joint-Diffusion (Deja al. 2023
77.6
16.8
SADA-JEM (Yang
75.0
11.63
14.4
EGC (Guo 2023
77.9
11.50
4.88
RATIO (Augustin et 2020
71.58
47.74
9.28
24.17
Standard AT (Augustin et al.
2020
72.16
47.78
9.54
23.59
DAT (T 15)
65.76
45.94
10.99
10.73
DAT (7 50)
60.12
42.55
11.12
9.53
Table Classification and generative modeling results on ImageNet 256x256.
Method
Acc%
Robust Acc% 1
FID L
IS
Params
Steps
Hybrid models
EGC (Guo at. [203)
78,90
13.56
6.05
231.3
543M (U-Net)
1000
Standard AT (Salman et al. 2020
64,91
39.96
15.12
286.2
26M (ResNet-50)
13
15)
61.31
39.96
322.65
(ResNei-50)
DAT 30)
55.96
37.14
5.28
319.3
26M (ResNet-50)
14
Standard AT (Salman 2020
71.25
45.86
37.33
260.2
223M (WRN-50-4)
12
DAT (f' 30)
64.45
45.84
6.23
341.0
223M (WRN-50-4)
17
DAT (T 65)
58.78
40.74
4.94
358.0
223M (WRN-50-4)
19
Standurd AT (Singh et al. 2023
78.25
33.38
44.46
27.32
198M (ComvNeXI-L-CvSi)
0
110)
75.78
56.40
310.2
(Com-NeXi-L-CSSh
Conditional generative models
BigGAN-deep (Brcck al. 2018
6.95
203.6
340M (ResNet)
ADM-G (Dhariwal and Nichol 2021
186.7
(U-Net)
LDM-4-G Rombach
3.60
247.7
400M (U-Net)
250
DIT-XI/2-0 Poeblls and Xie
2023h
2.27
278.2
675M
(Transformers)
250
VAR-d16 el al 20A
3.30
274.4
310M
(Trunsformers)
10
VAR-d30-1e
350.2
2.0B
(Transformrrs)
10

---

Published as a conference paper at ICLR 2026
4.3.2 COUNTERFACTUAL GENERATION, OOD DETECTION, AND CALIBRATION
Counterfactual generation. Visual counterfactual explanations (VCEs) reveal what minimal se- mantic changes would flip classifier's decision. Standard classifiers cannot generate meaningful VCEs because their input gradients lack semantic structure graddenttbbssee modifications produce adversarial noise rather than interpretable features. Prior VCE methods were therefore restricted to adversarially robust models (Augustin et al. 2020 [oreiko et al.) 2022). DVCE (Augustin et al. 2022) overcomes this limitation by guiding an unconditional diffusion model using gradients from an auxiliary adversarially robust classifier, projecting these onto a cone around the target classifier's gradient. This enables VCEs for arbitrary classifiers but requires multiple external dependencies: pretrained diffusion model, a robust classifier, and careful hyperparameter tuning.
In contrast, DAT is intrinsically explainable: it generates VCEs directly by gradient descent on the joint energy (z, y), requiring external models. Because our joint objective explicitly learns the energy landscape the joint distribution p(z, y), the resulting gradients V, Eg (z, y) naturally point toward semantically valid configurations. Therefore, our model's 's improved generative capability directly translates to higher-quality counterfactual explanations.
Figure 1 compares counterfactual quality across different models while accounting for classifier confidence. Our approach consistently generates counterfactuals with lower FIDs than baseline methods when achieving similar target class confidence. For instance, when RATIO reaches 0.89 confidence in the target class (at e = 8), its corresponding FID is 13.18. Our DAT model achieves a similar confidence level at 4 with significantly better FID of 25.53. This demonstrates that, for a comparable level of certainty that the counterfactual represents the target class, our generated faithful to the true visual characteristics of that class, indicating more
samples substantially plausible counterfactuals. We provide visualizations of counterfactuals in Appendix C.2
OOD detection. Our approach generally under- performs RATIO on OOD detection. Ablation studies show this gap persists even when us- ing identical aggressive augmentation for both the generative and discriminative components, indicating it stems from fundamental objective differences rather than the use of milder augmen- tation for the generative term: RATIO explicitly optimizes for OOD detection while our gener- ative loss prioritizes learning accurate energy functions for generation. The complete details can be found in Appendix
Aosrau Cles-wase Carf
1.0
Standard clussiter
RATI
DAT
0.2
0.9
7.5
Budget
Budget
Figure 1: Counterfactual FIDs and classifier confi- dences under different perturbations.
Calibration. Our model's calibration performance datasct-dependent, with detailed results provided in Appendix C.4 While the model is well-calibrated on CIFAR- outperforming the standard AT and RATIO baselines, it exhibits higher overconfidence on CIFAR-100 and ImageNet. The results suggest that prioritizing generative quality may come at the cost of calibration.
5 CONCLUSION
We presented Dual Adversarial Training (DAT), framework that replaces SGLD-based EBM learning with adversarial training, resolving the long-standing stability issues that have limited joint energy-based models. By integrating AT for both the discriminative and generative components within a JEM architecture, DAT achieves competitive results across classification, robustness, and generation on CIFAR- 10/100 and ImageNet 256x256, while producing faithful counterfactual explanations. These results suggest that adversarial training neyoon its well-known role in robustness provides an effective and scalable foundation for energy-based generative learning.
ACKNOWLEDGMENT
This work was supported in part by Advanced Micro Devices, Inc. under the AMD University Program's ' AI & HPC Cluster and part by a Lightspeed grant. TW was supported by a Vitalik Buterin PhD Fellowship.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Andrew Y Ng and Michael I Jordan. On discriminative vs. generative classifiers: A comparison of logistic regression and naive Bayes. In Advances in Neural Information Processing Systems, volume 14, 2001.
Tony Jebara. Machine Learning: Discriminative and Generative. The Springer International Series in Engineering and Computer Science. Springer, New York, NY, 2004.
Julia A Lasserre, Christopher M Bishop, and Thomas P Minka. Principled hybrids of generative and discriminative models. In 2006 IEEE Computer Society Conference Computer Vision and Pattern Recognition (CVPR 06), volume 1, pages 87-94. IEEE, 2006.
Jianwen Xie, Yang Lu, Song-Chun Zhu, and Yingnian Wu. A theory of generative convnet. In International conference on machine learning, pages 2635-2644. PMLR, 2016.
Will Grathwohl, Kuan-Chieh Wang. Bon-Henrik Jacobsen, David Duvenaud, Mohammad Norouzi, and Kevin Swersky. Your classifier is secretly an energy based model and you should treat it like one. arXiv preprint arXiv:1912.03200 2019.
Justin Lazarow, Long Jin, and Zhuowen Tu. Introspective neural networks for generative modeling. In Proceedings of the IEEE International Conference on Computer Vision, pages 2774-2783, 2017.
Long Jin, Justin Lazarow, and Zhuowen Tu. Introspective classification with convolutional nets. Advances in Neural Information Processing Systems, 30, 2017.
Yilun Du and Igor Mordatch. Implicit generation and modeling with energy based models. Advances in neural information processing systems, 32, 2019.
Ricky TQ Chen, Jens Behrmann, David K Duvenaud, and Jörn-Henrik Jacobsen. Residual flows for invertible generative modeling. Advances in Neural Information Processing Systems, 32, 2019.
Qiushan Guo, Chuofan Ma, Yi Jiang, Zehuan Yuan, Yizhou Yu, and Ping Luo. Ege: Image generation and classification via a diffusion energy -aaeed model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 22952-22962, 2023.
Kamil Deja, Tomasz. Trzciński, and Jakub M Tomezak. Learning data representations with joint diffusion models. Joint European Conference on Machine Learning and Knowledge Discovery in Databases, pages 543-559. Springer, 2023.
Max Welling and Yee W Tch. Bayesian learning via stochastic gradient langevin dynamics. In Proceedings the 28th international conference on machine learning (ICML-I1), pages 681-688. Citeseer, 2011.
David Duvenaud, Jacob Kelly, Kevin Swersky, Milad Hashemi, Mohammad Norouzi, and Will Grathwohl. No meme for me: Amortized samplers for fast and stable training of energy-based models. In International Conference on Learning Representations (ICLR), 2021.
Erik Nijkamp, Mitch Hill, Song-Chun Zhu, and Ying Nian Wu. Learning non-convergent non- persistent short-run meme toward energy-based model. Advances Neural Information Processing Systems, 32, 2019.
Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. Towards deep learning models resistant to adversarial attacks. arXiv preprint arXiv: (ee..00066..88 2017.
Lars Mescheder, Andreas Geiger, and Sebastian Nowozin. Which training methods for gans do actually converge? In International conference on machine learning, pages 3481-3490. PMLR, 2018.
Xuwang Yin. Shiying Li, and Gustavo K Rohde. Learning energy-based models with adversarial training. In European Conference Computer Vision, pages 209-226. Springer, 2022.
11

---

Published as a conference paper at ICLR 2026
Shibani Santurkar, Andrew Ilyas, Dimitris Tsipras, Logan Engstrom, Brandon Tran, and Aleksander Madry. Image synthesis with a single (robust) classifier. Systems, 32, 2019.
Classifier.Advancss Advances in Neural Information Processing
Yilun Du, Shuang Li, and Igor Mordatch. Compositional visual generation with energy based models. In Advances in Neural Information Processing Systems, volume 33, pages 6637-6647, 2020.
Diederik P Kingma, Danilo Jimenez Rezende, Shakir Mohamed, and Max Welling. Semi-supervised learning with deep generative models. In Advances in Neural Information Processing Systems, volume 27, 2014.
Yann LeCun, Sumit Chopra, Raia Hadsell, Marc Aurelio Ranzato, and Fu-Jie Huang. A tutorial on energy-based learning. In Predicting Structured Data, pages 1-59. MIT Press, 2006.
Xiulong Yang. Qing Su, and Shihao Ji. Towards bridging the performance gaps of joint energy- based models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15732-15744 2023.
Kwonjoon Lee, Weijian Xu, Fan Fan, and Zhuowen Tu. Wasserstein introspective neural networks, In Proceedings of the IEEE conference computer vision and pattern recognition, pages 3702-3711, 2018.
Xiulong Yang and Shihao Ji. Jem++: Improved techniques for training.jem In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 6494-6503, 2021.
Rostislav Korst and Arip Asadulaev. Adversarial training improves joint anergy-based generative modelling. arXiv preprint arXiv:2207.0888 2022.
Yao Zhu, Jiacheng Ma, Jiacheng Sun, Zewei Chen, Rongxin Jiang, Yaowu Chen, and Zhenguo Li. Towards understanding the generative capability of adversarially robust classifiers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 7737, 2021.
Yifei Wang, Yisen Wang, Jiansheng Yang, and Zhouchen Lin, A unified contrastive energy mergy-based based model for understanding the generative ability of adversarial training. In International Conference on Learning Representations, 2022a.
Mujtaba Hussain Mirza, Maria Rosaria Briglia, Senad Beadini, and Iacopo Masi. Shedding more light on robust classifiers under the lens of energy-based models. In European Conference on Computer Vision, pages 449 466. Springer, 2024.
Maximilian Augustin, Alexander Meinke, and Matthias Hein. Adversarial robustness on in-and out-distribution improves explainability. In European Conference on Computer Vision, pages 228-245 Springer, 2020.
Antonio Torralba, Rob Fergus, and William T Freeman. 80 million tiny images: A large data set for nonparametric object and scene recognition. IEEE transactions on pattern analysis and machine intelligence, 30(1 19:1998-1970 2008
Kevin Roth, Yannic Kilcher, and Thomas Hofmann. Adversarial training is form of data-dependent operator norm regularization. In Advances in Neural Information Processing Systems, volume 33, pages 149851 2020.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEF Conference Computer Vision and Puttern Recognition, pages 770-778, 2016.
Sergey Toffe and Christian Szegedy. Batch normalization: Accelerating deep network training by reducing internal covariate shift. In International conference on machine learning. pages 448 456. pmlr, 2015.
Zhuang Liu, Hanzi Mao, Chao- Yuan Wu, Christoph Feichtenhofer, Trevor Darrell, and Saining Xie. A convnet for the 2020s. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 1 11976-11988 2022.
12

---

Published as a conference paper at ICLR 2026
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations, 2021.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, pages 6000-6010 2017.
Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. Layer normalization. arXiv preprint arXiv:1607.06450. 2016.
Yang Zhao, Jianwen Xie, and Ping Li. Learning energy-based based generative models via coarse-to- to-fine expanding and sampling. In International Conference on Learning Representations, 2020.
Naman D Singh, Francesco Croce, and Matthias Hein. Revisiting adversarial training for imagenet: Architectures, training and generalization across threat models. In Advances in Neural Information Processing Systems, volume 36, pages 54424-54443, 2023.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the JEEE/CVF International Conference on Computer Vision, pages
2023a.
Alex Krizhevsky, Geoffrey Hinton, et al. Learning multiple layers of features from tiny images. 2009.
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hier- archical image database. In 2009 IEEE Conference Computer Vision and Pattern Recognition, pages 245-255. IEEE, 2009.
Sergey Zagoruyko and Nikos Komodakis. Wide residual networks. In Proceedings of the British Machine Vision Conference, 2016.
Hadi Salman, Andrew Ilyas, Logan Engstrom, Ashish Kapoor, and Aleksander Madry. Do adversari- ally robust imagenet models transfer better? Advances in Neural Information Processing Systems, 33:3533-3545, 2020.
Sylvestre-Alvise Rebuffi, Sven Gowal, Dan A Calian, Florian Stimberg, Olivia Wiles, and Tim- othy Mann. Fixing data augmentation to improve adversarial robustness. arXiv preprint arXiv:2103.01946, 2021.
Sven Gowal, Chongli Qin, Jonathan Uesato, Timothy Mann, and Pushmeet Kohli. Uncovering the limits of adversarial training against norm-bounded adversarial examples. arXiv preprint arXiv:2010.03593, 2020.
Haoqi Wang. Zhizhong Li, Litong Feng, and Wayne Zhang. Vim: Out-of-distribution with virtual- logit matching. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pages 4921-4930, 2022b.
Ivan Krasin, Tom Duerig, Neil Alldrin, Andreas Veit, Sami Abu-FilHaija, Serge Belongie, David Cai, Zheyun Feng, Vittorio Ferrari, Victor Gomes, Abhinav Gupta, Dhyanesh Narayanan, Chen Sun, and Gal Chechik. Openimages: A public dataset for large-scale multi-label and multi-class image classification. 2016. Dataset available from http:: /ttt//ttorreeeeeeeee googleapis com/ open web/ index. html
Francesco Croce and Matthias Hein. Reliable evaluation of adversarial robustness with an ensemble of diverse parameter-fre attacks. International conference on machine learning, pages 2206-2216. PMLR, 2020.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In Advances in Neural Information Processing Systems, 2017.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. In Advances in Neural Information Processing Systems, 2016.
13

---

Published as a conference paper at ICLR 2026
Durk P Kingma and Prafulla Dhariwal. Glow:
Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems, 31, 2018.
Will Grathwohl, Jacob Kelly, Milad Hashemi, Mohammad Norouzi, Kevin Swersky, and David Duvenaud. No meme for me: Amortized sampling for fast and stable training of energy-based models. In International Conference on Learning Representations, 2021.
Takeru Miyato, Toshiki Kataoka, Masanori Koyama, and Yuichi Yoshida. Spectral normalization for generative adversarial networks. arXiv preprint /rXiiv/802.055 2018.
Andrew Brock, Jeff Donahue, and Karen Simonyan, Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv: 1096, 2018.
Tero Karras, Samuli Laine, Miika Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8110-8119, 2020a.
Tero Karras, Miika Aittala, Janne Hellsten, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Training generative adversarial networks with limited data. Advances in neural information processing systems, 33: 10:12104-122 1244-12144 14, 2020b.
Tero Karras, Miik Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion- based generative models. In Advances in Neural Information Processing Systems, volume 35, pages 26565-26577 2022.
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 34:8780-877 8788448744 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pages 10684-10695, 2022.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 4195 4205, 2023b.
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arXiv preprint S.ti::200402200.. 2024.
Valentyn Boreiko, Maximilian Augustin, Francesco Croce, Philipp Berens, and Matthias Hein. Sparse visual counterfactual explanations in image space. In DAGM German Conference on Puttern Recognition, pages 138-148. Springer, 2022.
Maximilian Augustin, Valentyn Boreiko, Francesco Croce, and Matthias Hein. Diffusion visual counterfactual explanations. In Advances in Neural Information Processing Systems, volume 35, pages 364-377, 2022.
Yuichi Yoshida and Takeru Miyato. Spectral norm regularization for improving the generalizability of deep learning. arXiv preprint 109441, 2017.
AMD. Amd instinct mi300x accelerator. https: www.aamd.c www. com/en//ae com/en/producte...
acce Bcceleeratorrs/mnsiiit////oo.0. 300x h00 2024. Accessed: 2024.
Lambda Labs. Gpu benchmarks for deep 2024. Accessed: 2024.
rks
Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness common corruptions and perturbations. International Conference on Learning Representations, 2019. 2019,
14

---

Published as a conference paper at ICLR 2026
APPENDIX OVERVIEW
Contents
A Theory & Method
16
Extended related work
16
Optimal solution to the joint objective
18
A.3 Implicit R2 regularization
20
A.4 Training algorithm
22
Training & Evaluation
23
B.1
Implementation details
23
B.2
Computational efficiency
25
B.3 Training dynamics
26
B.4
Evaluation methodology
27
Results & Analyses
28
Qualitative samples
28
Visual counterfactuals
30
OOD detection
32
Calibration
34
Noise initialization
35
C.6
to generalization
36
Comuption robustness
37
8
Ablation studies
38
C.9
Reproducibility and stability
40
15

---

Published as a conference paper at ICLR 2026
A THEORY & METHOD
A.1 EXTENDED RELATED WORK
Connections between adversarial robustness and energy-based models Zhu et al. (2021) reinter- pret adversarially trained classifiers as joint energy-based models, showing that adversarial training implicitly flattens the energy landscape around real data by reducing the energy of nearby high-energy adversarial examples. They identify Ez y) as the key energy term for conditional generation and propose JEAT, which employs snergy-based adversarial perturbations and SGLD sampling for likelihood estimation and generation. Mirza et al. (2024) extend this analysis by decomposing the cross-entropy, loss as (rr 0) Eeg (x. y) Fi (a)) revealing that untargeted attacks increase the joint energy Eng Fa(x", ( (reducing classifier confidence of class y) while decreasing the marginal energy (xx). They show that robust overfitting corresponds to divergence between Egg (x) and Eg (a*) ), and that state-of-the-art robust models achieve better generalization by smoothing the marginal energy landscape around natural data. Wang et al.) (2022a) proposed unified Contrastive Energy- -based Model (CEM) framework that interprets adversarial training as biased maximum likelihood estimation of energy-based model po y) = exp ()). Unlike JEM (Grathwohl et al. 2019) which samples negative examples from random noise via Langevin dynamics, CEM shows that PGDD-generated adversarial perturbations from real data serve as implicit negative samples, providing more stable training without requiring random noise or OOD data. The framework unifies supervised (P-CEM) and unsupervised (NP-CEM) scenarios, revealing connections between adversarial training, contrastive learning, and energy- based modeling, and enables improved sampling algorithms that achieve improved generative performance over prior EBM-based methods.
Learning EBMs with adversarial training Yin et al.] (2022) explored an alternative approach to learning EBMs by leveraging the mechanism of Adversarial Training (AT). They established a connection between the objective of binary AT (discriminating real data from adversarially perturbed out-of-distribution data) and the SGLD-based maximum likelihood training commonly used for EBMs. Specifically, they showed that the binary classifier learned via AT implicitly defines energy function that models the support of the data distribution, assigning low energy to in-distribution regions and high energy out-of-distribution (OOD) regions. The PGD attack used in AT to generate adversarial samples from OOD data was interpreted as a non-convergent sampler that produces contrastive data, analogous MCMC sampling in EBM training. Although the resulting energy function can only capture the support rather than recover the exact density, their model achieves competitive image generation performance compared to explicit EBMs. Notably, this AT-based EBM learning approach is more stable than traditional MCMC-based based EBM training and demonstrated strong performance in worst-case nat-of-distribution detection, similar to methods like RATIO (Augustin) et al.) 2020). However, AT-EBM focuses on unconditional generative modeling and employs an explicit Rj gradient penalty to stabilize training, which can constrain model expressiveness. Our work incorporates AT-based EBM learning into the JEM framework to perform conditional generative modeling with implicit R, regularization from adversarial training, using ancestral sampling from the conditional distribution p(x y) rather than the marginal distribution p(r).
Improving joint energy-based models Building on the original JEM framework (Grathwohl et al. 2019), several works have explored techniques to improve training stability and performance. Yang and 202T) (JEM++) introduced multiple training improvements: (1) Proximal SGLD that con- strains samples within Ly-norm ball of previous samples via gradient clamping for improved stability; (2) YOPO-inspired acceleration (PYLD) that reduces redundant backpropagation by ex- ploiting the coupling between samples and first-layer weights; (3) Informative initialization from a class-conditional Gaussian mixture distribution estimated from training data, which accelerates SGLD convergence, improves stability, and enables batch normalization, Korst and Asadulaev (2022) (Robust-JEM) further enhanced JEM++ by incorporating adversarial traminn into the discriminative component, empirically observing improved training stability. At inference time, they propose a "combined inference" approach where initial samples from PGD adversarial attacks are refined using SGLD, improving generative performance. However, both JEM++ and Robust-JEM fundamentally still rely SGLD for sampling and MLE-based objectives for the generative component, inheriting SGLD's ' intrinsic instability issues. While these works introduced valuable techniques for improving SGLD stability, they did not fundamentally resolve the instability of SGLD-based training and both remain limited to CiFAR-scale (32x32) datasets. Our approach departs fundamentally from this line of work by: (1) providing mathematical analysis and empirical evidence showing that adversarial
16

---

Published as a conference paper at ICLR 2026
training offers implicit R1 regularization (Section A.3) ; (2) replacing the MLE-based objective with BCE-based gradients; (3) using deterministic PGD-based sampling instead of stochastic Langevin dynamics; and (4) introducing two-stage training strategy to address the incompatibility between batch normalization and EBM training. This enables scaling to high-resolution ImageNet synthesis (256x256) with state-of-the-an generative quality.
In- and out-distribution adversarial robustness Addressing the multifaceted challenge of creating models that are simultaneously accurate, robust, and reliable on out-of-distribution (OOD) data, Augustin et al. (020)) proposed RATIO (Robustness via Adversarial Training on In- and Out- distribution). Their approach combines standard adversarial training (AT) on the in-distribution data, aimed at improving robustness against adversarial examples, with form of AT on OOD data, which enforces low and uniform confidence predictions within neighborhood around OOD samples. The combined objective trains the model to maintain correct, robust classifications for in-distribution data while actively discouraging high-confidence predictions for OOD inputs, even under adversarial manipulation. Augustin et al. (2020) demonstrated that RATIO achieves state-of-the-art & robustness on datasets like CIFAR-IO, 10, often with less degradation clean accuracy compared to standard AT alone. Furthermore, they showed that RATIO yields reliable OOD detection performance, particularly in worst-case scenarios where OOD samples are adversarially perturbed to maximize confidence. Their work also highlighted that the f2 robustness fostered by RATTO enables the generation of meaningful visual counterfactual explanations directly in pixel space, where optimizing confidence towards a target class results in the emergence of corresponding class-specific visual features.
Robust classitiers for image synthesis and manipulation Santurkar et al. (019)) demonstrated that adversarially robust classifiers can serve as powerful primitives for diverse image synthesis tasks. The core insight of their work is that the process of adversarial training- loss over an le perturbation set rather than expected loss compels model to learn more perceptually aligned and human- interpretable feature representations by preventing reliance on imperceptible artifacts. Based on this insight, (Snnturar et al. (2019) showed that simple gradient ascent on class scores from such robust classifiers enables unified framework for image generation, inpainting, image-to-image translation, super-resolution, and interactive manipulation tasks typically requiring specialized GAN architectures or complex generative models.
optimizes the worst-case
17

---

Published as a conference paper at ICLR 2026
A.2 OPTIMAL SOLUTION THE JOINT OBJECTIVE
We formally characterize the distribution learned by our joint objective (Eqq. 8
Our analysis builds
on the theoretical framework from Yin al. (2022) (AT-EBM) and extends it the conditional modeling setting by deriving the optimal class logits under the joint objective.
Optimal solution for class logits fă [x) Following Yin al. (2022), the generative component of our objective can be expressed as maximin optimization problem. Let D(x:) = of Eg (x)) where Eg (z) - log My exp(fo(=)(x)
the marginal energy function. For the theoretical analysis
below, we assume (1) the PGD attack in Eq. converges to the global minimum of Ex(x), (2) the model has sufficient capacity, and (3) infinite training data from Pdata y). Under these assumptions, minimizing LBcu(f) implicitly solves:
max U (D, = En (z) [log D(x)] Bespr log(1 - D(x)) pr
(14)
where pr represents the distribution of samples after PGD attack initialized from the auxiliary outto--disttibution dataset Pood-
Under these assumptions, the optimal solution to Eq. [4 is characterized by Proposition of Yin et al. (2022). which shows that at optimum U(D",, pT) log(4) with:
1. (x) 1/2 for all E (upp(Paatu (z))
2. D (x) $ 1 for all x & Supp(Paam (z))
2
3. PT is supported on (x: (xx) )
where Supp(Puax denotes the support of the marginal data distribution.
The above result characterizes only the marginal energy Eg(x), leaving the individual class logits [y)] underdetermined. We now derive their optimal values by incorporating the discriminative objective LATCE.
Proposition A.1 (Optimal class logits), Under the assumptions stated above, at the optimal solution the joint objective C(0) (o)- + LBcE(0), the class logits satisfy on the support:
fō (z) [y] = log (1) for all æ E Supp(Paata(a))
(15)
Proof. From the AT-EBM result above, the support have D (xx) o(-Ea(1)) 2. which implies:
(x) = = - log Σ exp(ff = 00 - Σ exp(fá(x)(v)) =1
(16)
The conditional distribution is defined as:
exp( (z) (z)
po(u)z)) Ey exp(ffo fe (1/1))
(17)
Substituting the constraint from Eq. 16
(y(x)) = exp(S6 (x)
exp (x)
(18)
The adversarial cross-entropy objective LATCE minimizes log ())lx) over worst-case perturbations of (x, y) ~ (ar 2). Under our capacity assumption, at optimality on the support this reduces to standard cross-entropy behavior, yielding P& (t)a=) = Pama (z/lz)) Since the optimal predictive distribution naturally satisfies the normalization constraint Σ, Poata (y x = 1, the solution to the
discriminative objective is fully compatible with the marginal energy constraint E,, exp (x)[x)) =
1 derived from the generative objective. Therefore:
exp(f((x))x)) Pamaa (y(z)
(19)
Taking logarithms gives Eq. 15
18

---

Published as a conference paper at ICLR 2026
On-support behavior. Proposition A.I implies that on the support: (1) the joint energy equals the negative conditional probbabiily Eg (a. y) = f =
log Paua (v(2), (2) the marginal energy is constant (x) 0, and (3) the unnormalized marginal density is constant, P (z) o exp(- - Fig (a:) = 1. This confirms that the model assigns equal density to all points on the support, rather than learning the true data density (1). For datasets with deterministic labels where
Pdata (3) = 6 (x), this implies fg (<)]ll] = 0 for the true class and fø (z)[g) = -o otherwise.
Off-su behavior. For x. E Supp(Pasaa()) the optimal solution to Eq. 14 constrains (*)x) S and thus E,, exp(fā (z)(v) S L. Since CAT-CE only computed on the data support and its adversarial perturbations, the individual class logits underdetermined for points far from the support. From the constraint 2y exp(Se (x)(x) I 1, we have So (x)(vl S 0 for
2. which implies (x) 2
all classes and consequently E'g (z, y) (x) 2 O.
Comparison. For datasets with deterministic labels, this creates a hierarchical energy structure:
Valid pairs (x Supp, y = :Mme): The joint energy is exactly (2, y)
On-support, incorrect labels (z: E Supp, y + Ylve)) ) The joint energy diverges E (x.
20.
Off-support (OOD) (ar & Supp, any y): The joint energy is bounded below, Eg (z, y) 2 0. but the exact value is underdetermined.
Thus, the learned energy function Eg y) enables robust classification (via arg min, Eg (z, y)), generation (via minimizing (z, y) over x)), and OOD detection (via thresholding ming Eg (z. y)).
Finite-step vs. convergent PGD. The theoretical analysis above assumes the PGD attack converges to the global minimum of Ex)x), which leads to the maximin formulation in Eq. 14| In practice, however, we use finite-step PGD (Eq, (9) which does not explicitly solve the inner minimization problem, creating a crucial gap between theory and practice. Our experiments demonstrate that our approach achieves diverse, high-quality generation, suggesting that rather than converging to energy minima, finite-step PGD behaves like a sampler in practice: starting from different initializations in the OOD dataset, explores different regions of the energy landscape, leading to diverse generated samples.
Summary and implications. Our formal analysis reveals that our joint objective learns a funda- mentally different quantity than ML.E-based methods. While ML.E-based JEM theoretically learns the full density po(x) with valid partition function (though in practice short-run SGL.D fails to achieve this), approach explicitly learns the support of the data distribution with (z()) 0 on the support. The optimal class logits fō (x) [y] = log Pdata ( (r) (Proposition A.I reveal how the joint objective uniquely determines the solution the support.
This support-based characterization has important implications. The constant marginal energy (x) = 0 on the support means the model assigns equal unnormalized density to all points on the support, theoretically discarding frequency information about the data distribution. This represents a significant theoretical limitation: the model cannot distinguish between common and rare examples within the support, and thus cannot perform density estimation tasks that require modeling relative frequencies.
Despite this limitation, the support-based formulation provides clear advantages: superior training sta- bility compared to SGLD-based methods, robust classification, and effective OOD beeefiis that do not require density information. Additionally, our strong empirical generation results demon- strate that the finite-step PGD dynamics discussed above act as an effective sampler, capturing density variations despite the theoretical prediction of uniformity. Overall, the support-based approach provides practical and stable framework for joint modeling, trading full density estimation for superior training stability and strong empirical performance.
19

---

Published as a conference paper at ICLR 2026
A.3 IMPLICIT R1 REGULARIZATION
Building on the operator norm equivalence established by Roth et al. (020), we show that adver- sarial training implicitly bounds the R, gradient penalty, eliminating the need for explicit gradient regularization. Consider a classifier f : IRd + R producing logits for K classes.
Ri regularization directly penalizes large gradients of the true class logit:
(a..)) --aass
[1V=J1(<0]11
(20)
In practice, adversarial training uses cross-entropy loss to enforce consistent predictions:
CAT
E (z.y))
max L(f(x +6)
(21)
S
Adversarial training as operator norm regularization. constrained adversarial training with norrr loss on the logits equivalent data-dependent (p, opeorato norm regularization:
Roth et al. (2020) proved that nnrnr
CAT Cstandard + X(e) || (z)
(22)
where X(e) is an implicit regularization coefficient that scales linearly with the attack budget e. Here (x) E Kxd denotes the imput-Jacobian of the logits, and the (p, q) operratrr norm is defined as:
== max |
(23)
This measures the maximal signal amplification when propagating a norm-bounded input perturbation through the linearized network. For éy-constrained adversarial training, the (2, 2ooperatoo norm reduces to the spectral norm 0 maxx (z)( the largest singular value the Jacobian:
LAT Lstandard A(e) Tmax (Jf(Z))
(24)
The optimal perturbation direction aligns with the dominant right singular vector of (x))).
Connection to RS regularization. Since adversarial training regularizes T ( (x)( trace how this bound propagates to || (x) which R2 penalizes. By definition, the spectral norm satisfies ( (J)(x)) = auax (J)(Z)²) = max (a))")) ull2. Let Cy E TR be the standard basis vector for class y. The product (x) ey extracts the y-th column of Jy(x)", which corresponds
to the gradient (z)) Since ||,y|12 1, we have:
V v (x)112 (x)
max [1Jr(z) = omax (x)) ||
(25)
Thus, adversarial training's regularization of o maa (J)x)) upper-bounds the R₁ gradient penalty [x)]2)
Bound on all per-class gradients. This analysis reveals that adversarial training provides a stronger regularization than R; alone. While Ri penalizes only (x)((x)) adversarial training simultane- ously bounds all per-class gradient norms:
(z)x(=)ll < max (Jf(Z)), Vk E {l,..., K}
(26)
This includes the wroiahility-weightedd mean gradient Σn Pk fz)), since its norm is bounded by max; V (x)
Effective regularization strength scales with attack budget. An important consequence of this framework is that the effective regularization strength scales with the adversarial attack budget e.
From the equivalence established above, the adversarial training objective implicitly regularizes Fmax (J (x)) with strength X(e). For the network to maintain bounded total loss, the regularization term must remain bounded: X(e) Jmax (1)( S C for some constant C. Since X(e) scales linearly
with e, this implies:
C
m (nax(()(()) S
(27)
X(e)
20

---

Published as a conference paper at ICLR 2026
Combining with direct bound:
[Vz/y(r)(2 S mmxx( X(e)
(28)
Thus, the Ri gradient norm scales inversely with the regularization strength. Larger attack budgets induce stronger implicit regularization X(e), leading lower Rı gradients.
Connection to GAN regularization. GAN training commonly employs spectral normalization (Miyato et al., 2018) weight matrices and Ri gradient penalties (Mescheder al., 2018) for stability. The weight spectral norm provides global upper bound on the Jacobian spectral norm: (x)( S II. (W²) (Yoshida and Miyato 2017). However, Roth al. (2020) show this bound can be loose as it ignores data- data-dependent activation patterns. AT's equivalence to data-dependent spectral norm regularization more targeted, directly controlling sensitivity at actual data points. Combined with our analysis showing this bounds R1, AT provides a principled implicit alternative to both explicit GAN regularizers.
Empirical validation. We validate this theoretical analysis by tracking R₁ gradient norms during Stage 2 joint training on ImageNet 256x (ResNet-50, 7 15). Figure compares two settings: (1) adversarial training the discriminative loss (ExT-cE), and (2) standard training the discrim- inative loss. As predicted by the analysis above, the adversarial training curve remains bounded and stable throughout training, while standard training exhibits gradient explosion-with withh Rt values increasing by orders of magnitude in later stages. While the exact equivalence from Roth et al. (2020) holds under technical conditions (small relative ReLU cell size, infinite attack step size limit), our empirical observations are consistent with the theoretical predictions. This confirms that implicit regularization from adversarial training suffices to maintain bounded R₁ gradients, thereby enabling stable energy-based model training.
Adversarial training en the discriminative loss Standard training on the discriminative loss
Nmn
10a
100
10-1
0.0
800
1.6K
2.4K
3.2K
Iterations
Figure 2: Rj gradient norm during Stage 2 joint training on ImageNet 256 256 (ResNet-50, = 15). Y-axis is in log scale. Adversarial training maintains bounded gradients while standard training exhibits gradient explosion.
21

---

Published as a conference paper at ICLR 2026
A.4 TRAINING ALGORITHM
The complete training procedure for our combined objective (Eq. 12) rithm
is detailed in Algo-
We note that to train the generative component LECF, sample from po(x) estimate Exwepo(ie) V in Eq. In the context of JEM, there are broadly two strategies for drawing samples from po(
(Grathwohl et al. 2019):
1. Direct sampling from the marginal distribution using gradient-basee MCMC (e.g., SGLD PGD) on the marginal energy (z(z) = log My cx)(fo(=))v)) as implied by Eq
2. Ancestral sampling, which first draws label y ~ Paata then samples x ~ Pe (a( by running gradient-based MCMC on the joint energy Es(x, v) = [o(x))u).
Although both approaches yield unbiased estimates, we find ancestral sampling to be practically superior for training stability, possibly because it leverages the classifier's existing strong class representations to provide better mode coverage and mixing properties, while direct sampling from the marginal distribution often diverges.
Consequently, our implementation adopts ancestral sampling when generating contrastive samples (Algorithm D Specifically, first sample label y Pdaa (y). then generate contrastive sample Iy by performing T iterations of PGD on the negative joint energy function Eg( (z, y'), starting from an initial sample .ng Pood- This class-conditional contrastive sample æт is then used in the LaCe objective (Eq. whose gradient (Eq. 7 provides an approximation to the EBM gradient (Eq. 5)
Algorithm DAT training: Given network logits fa. in-distribution dataset Plata auxiliary out-of- distribution dataset Pood, classification AT bound e, PGD iterations T, PGD step size 1)
1: while not converged do
2: Sample (Ir, v) Pamaa (x, v). apply aggressive augmentation to
3: Sample - ~ Pdaa(()) 2g ~ (x), apply mild augmentation to and zo
4: Solve Tade arg max LCE (0; .v) via PGD attack
5: CXPCE (f)
log po ( (adu)
Robust classification loss
6:
Initialize I₁ & xo for = O, sample y' Pdata (v)
7:
for / {1, ..... T} do
Generate contrastive sample for EBM > Energy gradient
8:
V 0 Es (It-1, v/)
9:
24 - -t-1
9 |1g/22
Normalized gradient descent step
10:
end for
LBCE (0) log(o (+ log(1
D Generative modeling loss
12:
[(0) LAT-CE(O) - LACE (0)
13:
Compute parameter gradients VeL(O) and update
14:
end while
22

---

Published as a conference paper at ICLR 2026
B TRAINING & EVALUATION
B.1 IMPLEMENTATION DETAILS
B.1.1 HYPERPARAMETERS
We follow the two-stage training approach described in Section 3.5 hyperparameters for both stages across datasets.
Table 3 summarizes the
For Stage 1, we use a pretrained CIFAR-IO 10 model from RATIO (Augustin et al. 2020 and pretrained ImageNet ResNet-50 and WRN-50-4 models from (alman et al. (020), while training a CIFAR-100 model following RATIO with the hyperparameters in Table 3 For ConvNeXt-Land Largg experiments on ImageNet, use the pretrained checkpoint from (Singh et al. (2023). We select the EMA model with the best robust test accuracy as the final Stage I model.
For Stage 2, we initialize from the Stage model. For ResNet and WRN, we disable batch normal- ization by fixing all BN layers to evaluation mode. During this stage, we optimize the complete objective C(0) CAT-CE (0) + (BCE(() (0) using fixed learning rates specified in Table 3 For the discriminative component CAT-CE(O), CIFAR and ImageNet ResNet/WRN models continue to use the same adversarial training parameters as Stage 1, while ConvNeXt transitions from for to E2 perturbations with adjusted PGD parameters (see Table 4. The generative component LBCE (0) employs the parameters detailed in T
We select the Stage 2 checkpoint with the best FID score for the final evaluation reported in Sec- tion
Table 3:
Training hyperparameters for both stages. Epochs for Stage 2 are estimated based on the number of in-distribution images seen by the discriminative component during training.
CIFAR-10/100
ImageNet
ImageNet
Architecture
WRN-34-10
ResNat-50WRR-5004
ConvNcXt-L-CvSt
NarchNorn
Enabled (Stage 1), Disabled (Stage 2)
Enabled (Stage 1), Disabled (Stage 2)
NIA
LayerNorm
NIA
NIA Nesterov
Enabled
Optimizer
Nesterov
AdamW
Weight decay
10
1 10 (Stage 1).5 10 *(Stage 2)
0.05
size
$12
(Stage 1). (Stage 2)
EMA
Enabled 0.1 (cosine schedule)
Enabled
Enabled
LR (Stage
0.1 (step decay epochs 30, 90)
0.001 (cosine with warm-up)
1.R (Stage 2)
0.001 (CIFAR- 10), 0.009 (CIFAR-100) 100) 300
0.001
0.0003
Epochs (Stage
90
100
Epachs (Stage 2)
26 (CIFAR- 10), 30 (CIFAR-IO0)
0.78 (ResNet-50), 50), 0.39 (WRN-50-4)
1.09
Table 4: Adversarial training parameters for LAT.CE (identical across stages for CIFAR and ImageNet ResNet/WRN).
CIFAR-10/100
ImageNet (ResNet/WRN)
ImageNet (Con:NeXt-L-CvS))
PGD steps
10
PGD step size
0.1
2.0
2/255 (foc. Stage 1), 2.0 (f2, Stage 2) 4/255 (f... Stage e 3.0 (E2, Stage 2)
Perturbation bound
l2. 6 0.5
Eg = 3.0
Table 5: Adversarial training parameters for LACE (Stage 2 only).
CIFAR- 10/100
ImageNet (ResNet/WRN)
ImageNet (Com
Max PGD steps (7)
40/45/50
5/33/65
110
PGD step size
3,0
f perturbation bound
None (unconstrained)
None (unconstrained)
None (unconstrained)
00D data
80M Tiny Images Torralaa al. 2008
Open Images (Krasin al. 2016
Open Images (rrasin
2016
23

---

Published as a conference paper at ICLR 2026
B.1.2 DATA AUGMENTATION
As described in Section 4.1 use separate data augmentation pipelines for the discriminative summarizes the augmentation strategies for Stage 2 training.
and generative components. Table O For CIFAR-10/100 and ImageNet ResNet/WRN, the pretrained models were trained with the same augmentations as Stage 2 LXI-CE (from Augustin et al. (2020) for CIFAR Salman et al. (2020) for ImageNet). For ImageNet ConvNeXt-L-CvSt. trained with heavier augmentations (RandAugment MixUp + CutMix Random Hrasing) than the Examples of CIFAR-10 10 augmentations are shown in Figure
pretrained model from Singh al. (2023) was
Stage CAT-C strategy Table 6
Figure shows CIFAR-10 Stage 2 training curves under various augmentation strategies for LBCE (AutoAugment with Cutout is used throughout for LAT-CE). The choice of augmentation for the generative component also influences discriminative performance: robust test accuracy declines when using no augmentation. The best FID is achieved with no augmentation or random cropping with padding, both of which minimally distort Pdata- Overall, find random cropping with padding provides the optimal balance between discriminative and generative performance.
Table 6: Stage 2 data augmentation strategies for discriminative and generative components.
Dataset
Component
Augmentation Strategy
LADCC
AutoAugment Cutout RandomllorizzontalPipp) RandomCrop(32, padding=4) + Random/HorizontalPPipp
CIFAR-10/100
LECE
LAT-CE
RandomReesizcdCropp(55 + RandomHorizontalFFii Resize(256) + CenterCrop(256) + Random/HorzzoaalFPip
ImageNet
LOCE
sugmentation
crop eadding
AutoAugment cutout
Figure 3: CIFAR-10 samples under different augmentation strategies.
No augmentation
No augmentation
0.80
AutoAugment with cutout
30
AutoAugment cutout
0.78
Random crop with paddingg
Random crop with padding
Robust Test
0.76
10
4K
6K
2K
4K Iterations
8K
Iterations
Figure 4: CIFAR-10 training curves under different data augmentations during Stage 2 joint training.
24

---

Published as a conference paper at ICLR 2026
B.2 COMPUTATIONAL EFFICIENCY
Training overhead relative to standard AT. We estimate computational cost as follows: a forward pass costs I unit, input-gradient backward (for PGD) costs unit, and a full backward (for parameter updates) costs 2 units. This gives 2 units per PGD step and 3 units per training update. Under this model, Stage (standard AT) costs (2K + 3) units per iteration: 2K B for the steepp PGD attack plus 3B for the training update. Stage 2 adds T generative PGD steps (2T'B units) and processes three sample types per iteration (adversarial in-distribution, adversarial OOD, and clean in-distribution), tripling the training update cost to 9B units, for total of (2K + 2T + 9) B units per iteration. With Ej and Eg epochs for each stage, the total overhead relative to standard AT is:
E2 E
2K + 27 +
Overhead 1
(29)
2K +
This provides an upper bound, as our curriculum learning gradually increases T from a small initial value. Table repports overhead for all configurations: [411-.56x for CIFAR and 1.05-1.36 for ImageNet. Despite Stage 2's higher per-iteration cost, its short duration (less than epoch for most ImageNet configurations) results in minimal additional cost.
Absolute training time. Table 7 reports Stage effective training times on AMD M1210/M250/M13000 accelerators. For baseline models reported in V100-days (Rombach al.) 2022), we convert to MI300-hours using benchmark-base
ratios Our ImageNet 256 x56 ConvNeXt-L Stage 2 training takes 20 hours on 8 Mx33000, which ~5x faster than LDM-4- G (110 M1300-equivalent hours). Note that Stage 1 uses standard AT checkpoint, so total training time depends on whether pre-trained robust model available.
Inference efficiency. For classification, DAT requires only single forward pass, identical to standard classifiers. For generation, our models require 13-36 sampling steps (Table order of magnitude fewer than diffusion models (250 steps). Table repoorts the generation throughput, showing ~5x higher throughput than LDM-4-G while achieving better FID.
Table Computational cost and performance metrics DAT models. Training overhead computed relative to standard AT using Eq. 29 Stage training times effective durations excluding evaluation. Baseline training times converted from V100-days; throughput measured on a single MI300 accelerator.
Model
Params
FID 4
f
Training overhead
Training time (wall-clnck hours)
Throughput (img/s)
CIFAR-10
DAT (WRN-34-10 10, 40)
46M
9.12
9.96
1.4x
10 (4xM1210, Stage 2)
39
DAT (WRN-34-10, 50)
46M
7.57
9.86
1.49x
10 (4xM1210, Stage 2)
40
CIFAR-100
DAT (WRN-34-10, =
46M
10.70
10.83
1.52x
12 (4xM1210, Stage 2)
40
DAT (WRN-34-10, T
46M
9.53
11.12
1.56x
12 (4xM1210, Stage 2)
39
ImageNer 256x256
DAT (ResNet-50, T' = 15)
6.87
317.7
1.05
2.4 (4xM1210, Stage 2)
DAT (ResNet-50,, T 30)
26M
5.28
319.3
x.09
3.8 (4x Stage
DAT (WRN-50-2
223M
6.23
341.0
1.05
4.7 M4250 Stage 2)
14
DAT WRN-50-44 T 65)
223M
4.94
358.0
x.09
8.2 (4x MT250, Stage 2)
13
DAT Nex---eX-L
110)
198M
3.29
310.2
1.36x
20 (83M0000, Stage 2)
BigGAN-deepp (Brock et al. 2018
340M
6.95
203.6
52-104 (8 xM1300024)
ADM-G DDhrrial and Nichol 2021
608M
4.59
186.7
390 (8 35xM1300--0)
00.17
LDM-4-G (Rombach et 110-4-GGKoobaahh
2022
400M
3.60
247.7
110 (8 (Bx13000cq)
~0.9
'M1300/V100 7.4x based on AMD 2024
Lambda Labs 2024
25

---

Published as a conference paper at ICLR 2026
B.3 TRAINING DYNAMICS
Figure shows the Stage training curves across all datasets. FID improves rapidly and stabilizes, while robust accuracy is largely preserved. For ConvNeXt-L, which transitions from &oo to 82 perturbations in Stage 2, &g robust accuracy increases during training.
0.78
0.52
FID Robust Test Accuracy
Robust Test Accuracy
30
FID
Robust Test Accuracy
Robust Test Accuracy
0.77
0.51
20
0,76
0.50
15
15
10
0.75
0.49
10K
4K
Iterations
Iteraions
(a) CIFAR-10 (T' 40)
(b) CIFAR 100 (I' 45)
0.50
125
0.60
FID
Robust Test Accuracy
0.48
Robust Test Accuracy
100
FID
Robust Test Accuracy
0.58
Robust Test Accuracy
0.46
14
75
0.56
0.44
50
0.54
0.42
10
0.40
0.52
0.38
0.50
Iterations
Iterations
(c) ImageNet 256 (ResNet-50, T = 15)
(d) ImageNet 256x256 (ConvNeXi-L, T 110)
Figure 5: Training curves from Stage joint training demonstrating substantial FID improvements while maintaining robust test accuracy (evaluated via PGD attacks; FID measured using 10K generated samples).
26

---

Published as a conference paper at ICLR 2026
B.4 EVALUATION METHODOLOGY
We evaluate generative performance using FID and IS, FID is computed between 50K class-balanced generated samples and the full training set, while IS is computed the samples.
of 50K generated
Conditional generation. We generate an equal number of samples for each class. To generate samples for a given class we first sample an OOD data point x from the OOD data source, and then perform 7' steps of PGD according to:
V Ea(E,, v))
3++1
2t "1 V Ex y) y))
(30)
where 7' is the number of PGD steps and is the step size (see Table 8)
Unconditional generation. For unconditional generation, we sample from the marginal distribution using PGD (Eq. 1
compares conditional and unconditional FIDs across datasets; conditional generation consis- tently outperforms.
Table 8: Sample generation parameters for FID and IS evaluation. The number of PGD steps for each model and dataset combination determined through grid search.
Model
Dataset
PGD steps (T)
Step size
OOD data
(10 (T 10)
33
0.2
80M Tiny Images
CIFAR- (I' 50)
35
0.2
80M Tiny Images
CIFAR-I00 (I 45)
32
0.2
80M Tiny Images
CIFAR-100 (T 50)
33
80M Tiny Images
DAT
ImageNet (ResNet-50, T 15)
13
8.0
Open Images
ImageNet (ResNet-50, T [==30) =30)
14
8.0
Open Images
ImageNet (WRN-50-2 T= =300)
17
8.0
Open Images
ImageNet (WRN-50-4, =
19
8.0
Open Images
ImageNet (ConnNeXi-L-CvII
36
Open Images
CIFAR-10
31
0.2
80M Tiny Images
RATIO
CIFAR-100
0.2
80M Tiny Images
CIFAR-10
22
0.2
80M Tiny Images
CIFAR-100
15
0.2
80M Tiny Images
Standard AT
ImageNet (ResNet-50)
Open Images
ImageNet (WRN-50-4)
11
8.0
Open Images
ImageNet (ConNNcXt-L-CvSS)
0
8.0
Open Images
Table 9: Conditional vs. unconditional generation FIDs.
CIFAR-10
CIFAR-100
ImageNet 224x224 (ResNet50, T 15)
Conditional generation
9.07
10.70
6.64
Unconditional generation
20.57
13.56
18.67
27

---

Published as a conference paper at ICLR 2026
C RESULTS & ANALYSES
C.I QUALITATIVE SAMPLES
(a) Seed images generation.
(b) Uncurated conditional samples of DAT (T 50).
(c) Uncurated conditional samples
of RATIO.
Figure 6: CIFAR-10 class-conditional generation results.
(a) Seed images for generation.
Uncurated conditional samples of DAT (T = =50).
(c) Uncurated conditional samples of RATIO.
Figure 7: CIFAR- 100 conditional generation results.
(a) Seed images for generation.
(b) Uncurated conditional samples DAT (WRN-50-4, = 65).
(c) Uncurated conditional samples of standard AT (WRN-50-4).
Figure 8: ImageNet class-conditional generation results for the first 10 classes: tench, goldfish, great white shark, tiger shark, hammerhead, electric ray, stingray, cock, hen, ostrich (at 256 256 resolution).
28

---

Published as a conference paper at ICLR 2026
M
Figure 9: Selected ImtgeNet conditional generation results for class 88 (macaw), 107 (jellyfish), 130 (flamingo). 145 (king penguin), 248 (husky), 258 (Samoyed), 291 (lion), 511 (convertible), and 980 (volcano). Generated with DAT Con-NeXt-L-CVSt at 256
29

---

Published as a conference paper at ICLR 2026
C.2 VISUAL COUNTERFACTUALS
deer: 0.09 bird: 0.72
deer: 0.38 bird: 0.35
deer: 0.70 bird: 0.11
deer: 0.84
deer: 0.96 bird: 0.01
deer: n.98 bird: 0.00
dear: 0.98 bird: 0,00
bird: 0.04
deer:
deer:
deer:
deer:
plane: 0.15
plane: 0.29
plane: 0.45
plane: 0.
plane: 0.80
plane: 0.89
plane: 0.
horse: 0.28
horse: 0.16
horse: 0.09
horse: 0.06
horse: 0.03
horse: 0.02
horse: 0. 0.01
plane: 0.05 horse: 0.65
plane: horse: 0.85
0:01 horse: 0.95
plane: 00 horse: 0 96
horse: 0.43
horse: 0.92
dog:
dog:
dog:
dog:
dog:
dog:
cat: 0.10
cat: 0.05
cat: D. 03
cat: 0.02 dogg 0.90
cat: 0.01 dog: 0.93
cat: 0.01 dog: 0.95
Dog: 0.72
dag: 0.81
degi 0.87
Frog: 0.08
frog: 0.16 bird: 0.37
frog: 0 39
frog: 0.8
frog: 0.95 bird: 0.01
frog: 0.98 bird: 0.00
bird: 0.39
bind: 0. 26
bird: 0,08
bird: 0.03
frog: 0.05
frog: 0
frog: 0.02
frog:
frog: 0.00
frog:
Figure 10: CIFAR-I0 counterfactual examples 0.5, 1.0, 1.5, 2.0, 2.5, Each panel shows counterfactuals with classifier confidences for the correct class (top row) and target class (bottom row). As increases, counterfactuals progressively resemble the target class distribution with increasing target class confidence.
30

---

Published as a conference paper at ICLR 2026
Indign burning 0.20
indigo bunting: 0.02
indigo bunt ngr 6.01
indigo bunting: 0.00
Indign bunting 0.00
ndgio burting: 0.03
black
Nack 0.60
black
alack 5.00
black 0:00
indig bunting: 0.57
indigo Sunt nigi 0.90
indge b burting:
Inoigo Bunting 0.59
indigo Bunting: 0.99
bullfrog:
bu lftogg
oulffog::3
common
0.00
0.00
commen 0..00
common iguans: 0.29
common igians: 0.66
common iguans: 0:93
common iquana 0.97
common iguans: 0.59
chimsanzes: 0.02
chimpanzoe: 8.09
Chimgarzc:: 0.31
paaaaoo 0.55
chimpanzez: 6.65
chimpanaae: 0,76
chimpanzee:
chimpanzes: 0.05
chimpanase:
chimpanzee: 0.82
ady/ug: 0.01 deky
bodsbug 0:03 daby: 0.13
lodyeug 0:32 de sp. 3.04
ladybug; 0.67 delly: 9.01
Jodyy bug, 0:23 ky. 0.00
boybug: 0.89 daiyy: 0.85
ladebug: 0.00
ladyoup, 0.00 sy:
ladybug: 3,00
lady bug; 0.00 my.
ladsugg 0.00
Figure 11: ImageNet counterfactual examples at 10, 20, 30, 40, 50.
31

---

Published as a conference paper at ICLR 2026
C.3 OOD DETECTION
Using models trained with x24 generation, we evaluate both standard and worst-case (adver- sarial) OOD defection. For standard OOD detection, we measure AUROC between in-distribution test samples and clean OOD samples. For worst-case detection, we evaluate against adversarially perturbed OOD samples optimized to maximize the detection score. Results are computed using all in-distribution test samples and 1024 out-of-distribution samples. To generate adversarial OOD samples, we use an l perturbation budget of 1.0 for CIFAR-10/100 and 3.0 for ImageNet.
Energy-basee detection. We use an energy based function sy (a))) - Ex))), which is proportional log pe (z) up an additive constant. To find adversarial OOD inputs for this function, we use PGD to maximize the negative energy:
Rode = arg max -Ee(x') x' r'EB(z,eo)
(31)
where r is a clean OOD input and Bax, 6o) represents an lz-ball radius e, centered at x.
Maximum confidence detection. We use maximum confidence function sp(xr) maxy Pa (v)
that uses the confidence in the most likely class (also used by RATIO (Augustin et al. 2020 ). For this
detection function, following RATIO (Augustin et al. 20200), we generate adversarial O0D inputs by
maximizing the cross-entropy loss against a uniform distribution:
Tade
arg max LCE(O; 20; 1/K)
(32)
x'E,(a,).)
where 1/K represents uniform distribution over all K classes. Maximizing this loss encourages the model to produce a non non- uniform (confident) prediction, thereby maximizing the detection function.
Table I0 compares the two OOD detection functions. The results reveal complementary strengths: the energy-based function (- (-Ea (x)) achieves near-perfect AUROC on uniform noise detection, while the maximum confidence function (max, po (yl=)) performs better on natural image OOD datasets. Based on these findings, we adopt the maximum confidence score for subsequent comparisons with other methods.
Table 11 presents comparative results across different baselines. Notably, our DAT model achieves OOD detection performance comparable to standard AT on natural image datasets (CIFAR- 100, SVHN), despite incorporating an additional OOD dataset during training. This suggests that the generative component improves generation quality without significantly affecting OOD detection beyond standard AT.
Compared to RATIO, our model exhibits lower OOD detection performance across most datasets. To investigate whether this gap stems from our use of milder augmentations for the generative component, we trained an ablation model that applies RATIO's aggressive augmentation strategy to both loss terms. The results show that this variant performs similarly to our standard DAT model and still undcrperforms RATIO. This finding indicates that the performance gap is not primarily caused by the augmentation strategy but rather by the fundamental differences in the training objectives: RATIO's loss explicitly optimizes for OOD detection, while our generative loss prioritizes learning an accurate energy function for generation. A natural extension would be combining both objectives to preserve generation quality while improving OOD detection.
Table 10: Comparison of OOD detection functions on CIFAR-10.
CIFAR-100
SVHN
Uniform noise
Method
Clean
Adversarial
Clean
Adversarial
Clean
Adversarial
DAT (max, ()()z))
0.8709
0.6480 0.6647
0,9609
0.8334 0.6046
0.8922 0.9995
0.8257
DAT (-E((z))
0.8484
0.8011
0.9983
32

---

Published as a conference paper at ICLR 2026
Table 11: OOD detection performance (AUROC) with CIFAR-10 10 ID dataset (JEM results from
Augustin et al. 2020 ) All methods use the maximum confidence detection function sp(x)
maxy Po ())
CIFAR-100
SVHN
Uniform noise
Method
Clean
Adversarial
Clcan
Adversarial
Clean
Adversarial
JEM
0.8760
0.1920
0,8930
0.0730
0.1180
0.0250
Standard AT
0.8759
0,6364
0.9625
0.8306
0.8501
0.7902
DAT (T 40, uniform aug)
0.8751
0.6261
0.9642
0.8303
0,9546
0,9254
DAT (T = 40)
0,8709
0.6480
0,9609
0.8334
0,8922
0.8257
RATIO
0.9157
0.7516
0.9843
0.9130
0.9999
0.9999
Table 12: OOD detection performance (AUROC) with CIFAR-100 as ID dataset.
CIFAR-10
SVHN
Uniform noise
Method
Clean
Adversarial
Clean
Adversarial
Clean
Adversarial
Standard AT
0.7430
0.4093
0,8700
0.4863
0.7858
0.5048
RATIO
0.7320
0.3795
0.8439
0.4356
0.7769
0.5881
DAT (T = 45)
0.7027
0,5145
0,8271
0.5823
0.4024
0.2283
Table 13: OOD detection performance (AUROC) with ImageNet as ID dataset.
CIFAR-10
SVHN
Uniform noise
Method
Clean
Adversarial
Clean
Adversarial
Clean
Adversarial
Standard AT (ResNet-50) DAT (ResNet-50, T = 15)
0.7235
0.5304 0.4870
0.9239 0.8813
0.8089 0.7754
0.8678 0.6899
0.8377
0.6599
0.6268
33

---

Published as a conference paper at ICLR 2026
C.4
CALIBRATION
Figures 12 to 14 show reliability diagrams for CIFAR-10, CIFAR-I00, and ImageNet, using models trained with 224 224 generation.
1.0
1.0
1.0
3.39%
19.35%
1.73%
accuraty
2.6
2.8
2.6
9-
02
0.
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.9
0.2
0.4
0.6
0.8
1.0
confidence
confidence
confidence
(a) Standard AT
(b) RATIO
(c) DAT
Figure 12: Calibration diagrams on CIFAR-10 (without temperature scaling).
ECE: 4.23%
ECE: 4.48%
ECE: 18.35%
0.8
accuraty
accuracy
accuracy
0.2
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.0
0.2
0.4
0.6
0.8
1.0
confidence
confidence
confidence
(a) Standard AT
(b) RATIO
(c) DAT
Figure 13: Calibration diagrams on CIFAR-100 (without temperature scaling).
1.0
1.0
50%0
accuraty
0.6
3.6
0.6
02-
0.2
0.0
0.4
0.6
0.8
3.0
0.0
0.2
0.8
1.0
0.0
0.4
0.6
0.8
3.0
confidence
confidence
confidence
(a) Standard AT (ResNett-50)
(b) DAT (ResNet-50) 50, T' 15)
(c) DAT (ResNet-50,, T 30)
2.0
2.0
2.0
ECE: 2.66%
ECE: 20.20%
ECE: 74%
Rccurase
0.6
2.6
0.6
3.2
0.2
32-
0.5
0.2
0.4
0.6
0.8
3.0
0.0
0.2
0.4
0.6
0.8
1.0
0.0
0.2
0.4
0.6
0.8
1.5
confidence
confidance
confidence
(d) Standard AT (WRN-50-4)
(c) DAT (WRN-50-4, T 30)
(f) DAT (WRN-50-4, = 65)
Figure 14: Calibration diagrams on ImageNet (without temperature scaling).
34

---

Published as a conference paper at ICLR 2026
C.5 NOISE INITIALIZATION
We investigate whether DAT can be trained without any OOD data-initializing PGD from pure ran- dom noise instead of OOD images. We find that the DAT framework- loss, gradient-normalized PGD, and the implicit gradient regularization from adversarial -rrvving-roovides sufficient stability for training to succeed even from pure noise. Noise-initialized DAT achieves competitive Inception Score but worse FID compared to OOD initialization (Table T4]. Figure [5] shows class conditional samples generated from pure noise on CIFAR-10. demonstrating that the framework produces recog- nizable, class-consisten images without any auxiliary dataset. This experiment uses the same training methodology and hyperparameters (see Appendix [....]] as OOD initialization, except for a larger PGD step size 0.4 (vs. 0.1) to compensate for the greater distance from noise to the data manifold.
Table 14: Generation quality with noise OOD initialization CIFAR-10 10 (WRN-34- 10).
Method
Acc% f
Robust Acc% f
IS 1
FID |
DAT (OOD init, T 50) DAT (noise init, = 105)
90.72
74.65
9.86
7.57
90.60
74.03
9.00
13.72
Figure 15: Random class-conditional samples generated by noise-initialized DAT on CIFAR-10 (WRN-34-10). Each row corresponds to one class. PGD initialized from uniform noise 20 (0, 1) with no auxiliary OOD dataset.
35

---

Published as a conference paper at ICLR 2026
C.6 & GENERALIZATION
All our models in Tables | and 2 are trained with é2-based adversarial attacks for both the discrimi- native and generative components. To demonstrate that our approach generalizes l.c adversarial training, we train on ImageNet 256x256 with ConvNeXt-L. using foo attacks: for discriminative training, use 2-step PGD with 4/255 and step size 2/255; for generative training, we use step size 3/255 with T = 110 maximum steps. All other hyperparameters (optimizer, learning rate, weight decay, EMA, batch size) match our &₂ configuration exactly (see Table 3). FID evaluated on samples generated using & bbasd PGD with 36 steps and step size 0.03 (= 8/255).
2
Table 15 presents the results. Compared Standard AT, our trainedd DAT model trades modest reductions in clean accuracy (76.58% 78.25%) and robust accuracy (57.94% vs 59.40%) for significantly superior generation quality (FID 4.11 vs 44.46, IS 320.7 vs 27.32). This demonstrates that DAT successfully achieves joint discriminative -ewprinnnatiiieegatiiiie.a modeling under & training, maintaining competitive robustness while enabling high-quality generation.
Comparing our 6--- and t2-trained DAT models shows each model specializes in its training norm, achieving superior robustness under that norm while maintaining comparable clean accuracy and generation quality. However, observe notable difference in visual quality: Ey-trained models produce smooth generated images, while fx trained models exhibit high- high-frequency noise artifacts (grainy appearance with scattered bright pixels). This artifact stems from the different constraint geometries-f,, 6 perturbations allow independent bounded changes to each pixel, favoring per-pixel variations, whereas le perturbations enforce global constraint that naturally penalizes high-frequency noise. These results confirm that our approach successfully generalizes to the foo setting, though the choice of norm significantly influences the perceptual quality of generated samples despite similar FID scores.
Table 15: foo and lg results on ImageNet 256 256 with ConvNeXI-L. Standard AT uses the 6xo- trained (e 4/255) checkpoint from Singh et al. (2023). Both DAT models initialize from the same Stage 1 checkpoint and differ only in Stage perturbation norm. All models are evaluated under both too (c = 4/25 and lg (c 3.0) attacks using AutoAttack (Croce and Hein 2020).
Method
Training Norm
Clean Acc f
Robust Acc (Cxg 4/255) f Robust Acc (6, 3.0)
FID
18 f
Standard
78.25
33.38
44.46
27.32
DAT 110)
8
76.58
57.94
33.40
4.11
320.7
DAT (T 110)
2
75.73
51.90
56.40
3.29
310.2
36

---

Published as a conference paper at ICLR 2026
C.7 CORRUPTION ROBUSTNESS
To evaluate whether the joint training objective maintains robustness under distribution shift, we assess our CIFAR-10 10 models CIFAR-I0-C (Hendrycks and Dietterich 2019), benchmark testing
robustness to 15 common corruption types across severity levels.
As shown in Table 16, DAT achieves mCE of 19.84% (T' 40) and 21.84% (T' 50), compared 19.63% for standard AT. While corruption robustness slightly degrades as '' increases, DAT remains comparable standard AT across all corruption types.
Table 16:
Corruption robustness on CIFAR-10. Error rates (%) are averaged across 5 severity levels; mCE denotes mean corruption error across all 15 corruption types.
Metric
Standard AT
DAT (T = 10)
DAT (T = 50)
Noise corruptions (error %)
Gaussian noise
21.30 17.14
20.52 16.36
20.63 16.86
Shot noise
Impuise noise
23.80
23.54
24.72
Blur corruptions (error %)
Defocus blur
11.08
11.69
13.13
Glass blur
15.38
14.74
17.06
Motion blur
13.65
14.27
15.84
Zoom blur
11.72
12.39
13.89
Weather corruptions (error %)
Snow
12.19
12.19
13.62
Frost
12.44
12.13
13.60
Fog
24.75
26.67
29.45
Brightness
8.26
8.60
9.61
Digital corruptions (error %)
Contrast
33.31
32.89
36.24
Elastic transform
12.34
13.06
14.80
Pixelate
9,51
9.65
11.08
JPEG compression
9.56
9.73
Mean corruption error (mCF)
19.63
19.84
21.84
Clean Ace (%)*
92.43
91.86
90.72
Robust Ace (%)
75.73
75.66
74.65
FID |
28.41
9.07
7.57
37

---

Published as a conference paper at ICLR 2026
C.8
ABLATION STUDIES
We ablate individual model components, analyze OOD data efficiency, and investigate the trade-off between discriminative and generative performance.
C.8.1
COMPONENT ABLATION
To analyze the contribution of each component, we conduct an ablation study on CIFAR-10 with the following variants:
Standard AT: Baseline without generative component.
DAT with uniform augmentation: Same aggressive augmentation for both objectives.
DAT with decoupled augmentation: Aggressive augmentation for LAT-CE, mild augmentation for LBCB-
The results in Table 17 show the impact of the AT-based generative loss and decoupled augmenta- tion. Adding the generative loss reduces FID from 33.04 to 15.35, while robust accuracy remains comparable. Decoupled augmentation further reduces FID to 9.07.
Since both approach and RATIO extend standard AT baseline with objective that leverages outto--distriuttinn (OOD) data, it natural to compare their effect on generation quality. The RATIO objective, which is formulated for robust OOD detection, reduces the FID from 33.04 21.96. In contrast, our generative objective provides a much larger improvement, lowering the FID to 15.35. This confirms that dedicated generative loss is more effective for sample quality than OOD detection loss.
Table 17: Effect of generative loss and augmentation CIFAR-10.
Method
Acc%
Robust Acc%
FID 4
Standard AT
92.34
75.73
33.04
DAT (uniform aug)
92.68
75.93
15.35
DAT (decoupled aug)
91.86
75.66
9.07
RATIO (Augustin et al. 2020
92.23
76.25
21.96
C.8.2 OOD DATA EFFICIENCY
The dittt--diitribution (OOD) dataset is critical component of our training framework, as it provides initialization points for generating contrastive samples in the generative loss. Intuitively, a more diverse OOD dataset provides better coverage of the input space, allowing PGD discover broader range of spurious modes in the energy landscape. These modes are then suppressed as training progresses.
We ablate OOD dataset size on ImageNet with DAT ResNet-50 (T' = 15), varying from IK to 300K samples. As shown Table FID improves modestly from 6.96 with 1K samples to 6.64 with 300K samples. Classification accuracy remains stable across all dataset sizes, with similar robustness levels, indicating that the OOD dataset size primarily affects generation quality rather than discriminative performance.
These results demonstrate notable data efficiency, with only modest improvements when scaling from 1K to 300K OOD samples. One factor is data augmentation: we employ RandomResizedCrop with scale= 1.0) and aspect ratio= 75, 1.33), which can crop as little as 8% of the original image with varying aspect ratios, potentially amplifying the effective diversity of each sample. To investigate the contribution augmentation, we include a baseline using 1K OOD samples without data augmentation. While augmentation provides clear benefits benefits-improvine improving FID from 8.00 to 6.96- even without augmentation, our approach substantially outperforms standard AT in generation quality (FID 8.00 15.97).
38

---

Published as a conference paper at ICLR 2026
Table 18: Impact of OOD dataset size on ImageNet performance for DAT ResNet-50 (T' 15) with 224 x 224 generation.
Method
Acc%
Robust Acc%
FID J
IS 1
Standard AT
62.83
34.44
15.97
274.90
DAT IK w/o aug
57.50
33.80
8.00
320.64
DAT 1K
57.56
34.22
6.94
324.23
DAT 10K
57.82
34.70
6.84
320.78
DAT 100K
58.19
34.88
6.70
322.10
DAT 300K
57.88
34.84
6.64
339.55
C.8.3 DISCRIMINATIIVEGENERNNI TRADE-OFF
Our experiments--varyine PGD steps (Section 4.3), loss weights (see below), and augmentation strategies (Appendix B. 1.2 reveal trade-off between generative and discriminative performance. The augmentation ablation sheds light on this: varying only the generative augmentation, no augmen- tation achieves better FID but worse robust accuracy, while random cropping maintains similar FID but improves robustness. This sensitivity to the generative pipeline's 's data distribution suggests that model representations adapt to the data used for generative modeling. The large FID improvement of DAT over standard AT further supports this, as generation relies on gradients through the shared representation. This creates a tension: the generative objective encourages representations aligned with Pdara for high-fidelity generation, but this alignment can come at the cost of robustness. While this tension is inherent, the trade-off can be tuned through PGD iterations T and loss weighting.
Control via loss weighting, To demonstrate this controllability, we perform experiments on CIFAR 10 with three weighting configurations for the composite objective [(0) CAT-CE (0) CBCE
(1) standard loss with A A2 = 1.0; (2) emphasize generative with A = 0.6, )2 = 1.4; and
(3) emphasize classification with A 1.4, A2 0.6. The results in Table 19 confirm that the balance between generative and discriminative performance can be tuned by adjusting the loss term weights. Emphasizing the generative component improves FID at the cost slightly reduced classification performance, and vice versa. Notably, standard, unweighted loss corresponds to the natural factorization of the joint log- ligeliiel in the original JEM formulation: log pe (o y) = log Po (y()))) + log (2). This suggests that equal weighting is principled default that performs well without requiring additional hyperparameter tuning.
Table 19: Trading off generative and discriminative performance by weighting loss terms.
Method
Acc% t
Robust Acc% +
FID +
Standard loss
91.88
75.73
9.09
Emphasize generative modeling
91.16
8.77 75.11
Emphasize classification
92.52
75.97
10.02
39

---

Published as a conference paper at ICLR 2026
C.9 REPRODUCIBILITY AND STABILITY
Table 20 reports mean and standard deviation across five independent runs with different random
seeds. Performance consistent across all runs, with training divergences observed.
Table 20: Reproducibility of DAT across datasets (five runs with different random seeds).
Datasct
Acc% 1
Robust Ace%
IS f
FID +
CIFAR-10 (WRN-34-10. T 40)
91.92 = 0.09
75.75 0.07
9.92 + 0.05
9.12 0.05
CIFAR 100 (WRN-34-10, T = 45, LR=0.009)
65.76 + 0.75
45.94 + 0.48
10.99 +: 0.29
10.73 +: 0.25
ImageNet 256x256 (ResNet-50, 7 = 15)
61.31 1 0.16
39.96 1: 0.41
322.65 1: 2.28
6.87 ::: 0.05
40


Reviews:
[{"review_id": "mfxdikstCW", "reviewer": "Reviewer_WmTW", "summary": "The paper focusses on improving the Joint Energy Models (JEM) proposed by Grathwohl et al (2019) using techniques from Adversarial Training. They show that when such ideas are incorporated in the joint training of the generative + discriminative energy model, the training stability increases and surprisingly alleviates the need of gradient penalty regularizations, and improves the classification accuracy vs generative performance tradeoffs. Additionally, OOD detection and adversarial robustness and classifier calibration are auxiliary benefits with this approach.", "questions": "Please see weaknesses.\n\nAdditional Questions:\n1. - Line 319-320 - The authors say they use the RATIO pretrained models. However, they also show in Eq 13 how RATIO objective is different from their AT-CE objective. This totally changes their method fundamentally for CIFAR-10 and ImageNet models since the pretraining objective is RATIO objective which they elucidate how it's different from theirs in Eq 13. Since the datasets used in the paper are no where close to being considered large, I highly recommend the authors not use any pretrained models and train them from scratch using their proposed formulation without departure to other objectives like RATIO that makes it quite confusing to the reader on the exact proposal for training in this paper. \n2. - The weighted loss function modification in Eq (7) introduced by the authors is similar to the Focal Loss (https://arxiv.org/abs/1708.02002) albeit with gamma=1. Please compare this paper & suggest the readers why such a specific weight form is chosen (is it empirical? or is there a theoretical argument?)\n\nPlease also see weaknesses.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "- The objective of improving JEM training is made clear early on for the readers to get a grasp of the problem statement.\n- The overview of Grathwohl et al. 2019 in the Method section was essential in setting up the context and mathematical notation for the problem. I thank the authors for the great job done here.\n- The fundamental idea of incorporating Adversarial Training for more than just adversarial robustness is interesting and insightful. The authors successfully convince the reader how AT objectives improve the join distribution modeling in JEMs, which is quite different from the original focus of AT objectives in improving adversarial robustness.\n- Insightful findings on stabilizing training while still having BN layers and data augmentations.", "weaknesses": "Minor:\n- Numerous abbreviations are used before first expanding them. Example: PGD/SGLD in Abstract. PGD in the main paper before it's used in Line 63-65. \"DAT\" is used on line 249 without expanding it first.\n- Missing Citations:\n-- 1) Line 39 - \"...rarely excelling at both simultaneously.\" - Please cite?\n-- 2) Line 40 - \"...but may underperform on downstream classification tasks.\" - Please cite?\n- Misleading to say on Line 76: \"datasets of increasing complexity from CIFAR-10 to ImageNet...\" The paper only includes CIFAR-100 additionally and that is not a spectrum of datasets of increasing complexity as claimed.\n- Section 3.1 under Method: Please define Z(\\theta) before or after using it in the equations. Also, please make it clear and explicit that the Z(\\theta) used in P(x) on line 161 is not the same as the Z(\\theta) used in Eq. (2). Explicitly define them in each case with an integral etc to show the margnialized partition function.\n- Section 3.1, please also talk about how Z(\\theta) is handled in the loss function and how it is estimated.\n- Please clarify Equation 11: Are the authors only using the adversarial loss on samples x_adv for classification objective and No regular CE loss on the input samples \"x\"? Or are both losses turned on and the AT-CE loss is just an auxiliary loss to a regular CE classification loss on input samples \"x\"?\n- Unclear why OOD data is required in Line 328. Is it only for Eval? Or is it used during training too (which RATIO method requires)?\n\nMajor:\n- The *primary* focus of the paper is unclear. Is the focus on improving the discriminative-generative performance tradeoffs? Is the focus on improving the training stability of JEMs mainly? Is the focus on improving the adversarial robustness or OOD detection? It is quite confusing despite novel insights presented in the paper.\n- Continuing with the prev major weakness, the results in Table 1 are not clear to any extent where the method presented excels in relation to the other methods. For example, for CIFAR-100 hybrid models the DAT method is worse both in classification accuracy as well as FID than EGC method (sure, there is an extra benefit of Adversarial Robustness -- which comes back to the question of what aspect being the primary focus of the paper). Please BOLD the numbers of your method that you think are excelling at a certain aspect in relation to the rest of the models. \n- The datasets presented CIFAR-10/100 are not sufficient in convincing the readers of the merits of the work. These low res datasets are suited for a quick PoC run. The only substantial dataset used is ImageNet. Please consider including other datasets that compare to ImageNet in res like the LSUN dataset and/or CelebA faces etc. I think the results are insufficient to make a clear conclusion at the current form of the paper.\n- Line 195 - \"Preventing numerical overflow/ underflow\" -- Please either cite or show a toy example as evidence that this indeed happens while training. You can follow the plots used in this paper (https://openaccess.thecvf.com/content/WACV2022/papers/Bhaskara_GraN-GAN_Piecewise_Gradient_Normalization_for_Generative_Adversarial_Networks_WACV_2022_paper.pdf) where to convince that the gradient explodes, an explicit plot of the gradient norm is presented in Fig 3. \n- Line 199 \"The grad formulation stabilizes training at the cost of limiting the EBM to modeling the support of p_data....\" -- This is not clear to the reader how the cost here is limiting the EBM to modeling the support but not the full dentsity. Please include a citation or a toy experiment to prove this is true.\n- Line 215 - The paper mentions how the authors' method impoves the stability of training, however, no experiment is presented to back up this claim. For example, out of a random 10 experiments for each model, what fraction destabilize and diverge during training. Is the new method better in this quantitatively. See Fig 2 of (https://openaccess.thecvf.com/content/WACV2022/papers/Bhaskara_GraN-GAN_Piecewise_Gradient_Normalization_for_Generative_Adversarial_Networks_WACV_2022_paper.pdf) where a large FID/KID score implies training instability out of 5 random runs.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "afO8mfnpKF", "reviewer": "Reviewer_Y3JD", "summary": "This paper proposes a framework to simultaneously achieve robust classification and high-fidelity generative modeling within a single network. The key contribution is the use of a Dual Adversarial Training (DAT) strategy as an alternative to unstable sampling-based (SGLD) in Joint Energy-based Models. One AT component ensures discriminative robustness against adversarial attacks, while the other uses PGD-generated contrastive samples. A binary cross-entropy loss is used to perform a stable AI-based generative modeling approach. Experimental evaluations on standard datasets (CIFAR-10, CIFAR-100, and ImageNet) show improvement in adversarial robustness over existing hybrid models while maintaining good generative performance.", "questions": "* Have the authors explored whether the joint model maintains robustness when transferring to out-of-distribution (OOD) data or cross-domain settings (e.g., different image modalities or noise conditions)?\n* Would the model’s scalability or performance differ on higher-resolution or more complex datasets?\n* How do the dual objectives interact during training? For example, does improving the generative objective always enhance discriminative performance, or are there cases where they conflict? Some insight through visualization or empirical justification would be helpful.\n* How sensitive is the DAT's performance to the relative weighting and scheduling between the discriminative and the generative AT losses?\n* Can the authors provide insight into the computational cost? Specifically, what is the training time overhead (e.g., in GPU-hours) compared to training a non-robust JEM or a standard robust classifier on CIFAR-10/100?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The paper addresses the major stability and scaling issues of previous Joint Energy-based Models by replacing the unstable sampling-based SGLD with adversarial training-based optimization.\n* The proposed framework improves adversarial robustness in the discriminative tasks while simultaneously maintaining generative fidelity compared to existing hybrid models.\n* The dual adversarial training setup stabilizes training compared to standard GAN frameworks as demonstrated through detailed analysis and ablation studies.", "weaknesses": "* The dual optimization adds computational overhead, which could make the training slower and impractical for large-scale high-resolution images. Computational inefficiency is mentioned but not reported in terms of training/inference time and parameter size.\n* Experiments are performed mainly on standard image datasets (e.g., CIFAR-10/100, ImageNet), without extending to complex datasets or domains, which limits claims of broad applicability.\n* The sample selection for FID calculation needs to be better justified. \n* Performance comparison is weak as the paper shows comparisons against some old methods (mostly 2020 or earlier). The paper could have included comparisons with more recent hybrid models (e.g., diffusion-classifier hybrids or energy-based approaches) for a stronger empirical baseline.\n* Missing references: L-041 - \"Recent research .... understanding of generative models\" and L-094 - \"sample generation .... semi-supervised learning\"\n* The paper claims significant improvement without reporting any statistical significance analysis. \n* The training procedure can be dataset-specific, requiring tuning parameters and adjustments for new datasets. The current experiments are confined to image data. Extending it to other data modalities could introduce new stability challenges, making its generalizability uncertain.\n* L-365: \"Our best generative configuration ... achieves an FID of 5.39 ... requiring significantly less sampling steps.\" Not clear which dataset it's referring to.\n* The term PGD is never defined in the paper.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "I6urACSOuw", "reviewer": "Reviewer_feF3", "summary": "The authors propose a hybrid framework that aims to unify discriminative robustness and generative modeling within a single network. The method replaces the JEM-style SGLD negative sampling with adversarially generated negatives (PGD) and trains the energy with a binary classification loss, while the classifier head is trained with standard adversarial training (AT). Experiments on CIFAR-10/100 and ImageNet report improved adversarial robustness relative to hybrid baselines and competitive (sometimes strong) generative quality.", "questions": "1.\tPlease provide evidence that AT stabilizes SGLD-based JEM training: show training curves, failure/divergence rates, and input-gradient norm, not only final metrics.\n2.\tPlease sharpen the novelty narrative relative to AT-EBM, CEM, JEM++, and Robust-JEM: what is fundamentally new here (objective, theory, or capabilities) beyond an engineering consolidation?\n3.\tThe $l_\\infty$ is the more common norm in adversarial robustness. Why are experiments restricted to $l_2$? Please either justify this choice or add $l_\\infty$ evaluations with standard AutoAttack/RobustBench protocols. \n4.\tPlease report compute: FLOPs or wall-clock for training, plus sampling throughput (images/sec).\n5.\tWhat is the sampling procedure for generation—pure gradient-based synthesis or any MCMC steps? Please clarify the runtime and compute for each setting.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tReplacing SGLD with PGD negatives makes the training loop simple and scalable. The two-stage BN recipe is an effective engineering fix to a well-known incompatibility in EBM-style training.\n2.\tThe dual view (AT for the classifier and contrastive/AT-style learning for energies) offers a single model that can produce robust predictions, counterfactuals, and samples.\n3.\tResults are reported up to ImageNet, suggesting attention to large-scale feasibility.\n4.\tPublic code (if complete and reproducible) increases practical impact and adoption.", "weaknesses": "1.\tThe conceptual novelty is limited. The core idea (learn energies with adversarial negatives and train the classifier adversarially) has clear antecedents in AT-EBM/CEM/JEM++/Robust-JEM[1] lines of work that (i) use contrastive or adversarially produced negatives to shape an energy landscape, (ii) draw formal links between AT objectives and energy-based views, (iii) report stability benefits relative to SGLD, and (iv) An empirical study on AT improving JEM/JEM++.\n2.\tThe BCE-with-PGD objective likely corresponds to a contrastive density-ratio under a local worst-case neighborhood. It remains unclear what distribution is being learned and how/when this departs from MLE-style JEM; a formal treatment would strengthen the paper.\n3.\tThe paper does not provide convincing diagnostics that AT stabilizes SGLD-based JEM training (e.g., training curves, divergence/failure rates, gradient-norm statistics), beyond final performance metrics.\n4.\tThe proposed DAT is not compared head-to-head with prior AT-JEM methods (e.g., Robust-JEM) under matched backbones, budgets, and evaluation protocols, making it hard to attribute gains to the proposed ingredients rather than setup differences.\n\n[1] Korst, R., & Asadulaev, A. (2022). Adversarial training improves joint energy-based generative modelling. arXiv preprint arXiv:2207.08950.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "fL7OAtbjc3", "reviewer": "Reviewer_MpyS", "summary": "This paper presents DAT (Dual Adversarial Training), a framework that integrates adversarial training into Joint Energy-Based Models (JEMs) to achieve both robust classification and high-quality image generation. The method replaces unstable SGLD-based learning with an adversarial optimization using PGD-generated samples and introduces a two-stage procedure to handle batch normalization issues. Experiments on CIFAR-10, CIFAR-100, and ImageNet show competitive robustness (similar to standard AT) and improved generative quality (FID 5.39 on ImageNet). The motivation is to bridge discriminative and generative modeling, combining the accuracy of classifiers with the data awareness of generative models. While the formulation is clean and the results promising, the practical benefits of unifying these two objectives remain somewhat unclear.", "questions": "How does it compare in efficiency and scalability to modern diffusion and GAN models?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper addresses a well-known instability in JEM training through an elegant adversarial reformulation that improves both convergence and visual quality. The proposed method removes the need for gradient penalties and achieves stable training even on ImageNet, which is impressive for EBMs. The experiments are extensive and reproducible, comparing DAT against strong hybrid and adversarial baselines. Quantitatively, DAT outperforms RATIO and JEM in both robustness and FID, and qualitatively the generated samples show fewer artifacts. Overall, this is a careful and technically strong piece of work that meaningfully advances the robustness and scalability of energy-based hybrids.", "weaknesses": "The paper is technically competent and experimentally thorough, but the contribution feels incremental relative to prior hybrid frameworks. While the work pushes JEMs toward stability and competitive generative quality, it does not yet offer a compelling argument for hybrid modeling over modern specialized alternatives such as more recent, 2024 and beyond, diffusion models and GAN variants. Without stronger empirical evidence of unique practical benefits, the paper’s impact may be limited.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)