Toward Guidance-Free AR Visual Generation via Condition Contrastive Alignment
Authors: Huayu Chen
Hang Su
Peize Sun
Jun Zhu
Venue: ICLR 2025 Oral (2025)
Keywords: autoregressive
generative models
image generation
multimodal
alignment
RLHF
classifier-free guidance
Primary area: generative models

Abstract: Classifier-Free Guidance (CFG) is a critical technique for enhancing the sample quality of visual generative models. However, in autoregressive (AR) multi-modal generation, CFG introduces design inconsistencies between language and visual content, contradicting the design philosophy of unifying different modalities for visual AR. Motivated by language model alignment methods, we propose Condition Contrastive Alignment (CCA) to facilitate guidance-free AR visual generation. Unlike guidance methods that alter the sampling process to achieve the ideal sampling distribution, CCA directly fine-tunes pretrained models to fit the same distribution target. Experimental results show that CCA can significantly enhance the guidance-free performance of all tested models with just one epoch of fine-tuning (1% of pretraining epochs) on the pretraining dataset. This largely removes the need for guided sampling in AR visual generation and cuts the sampling cost by half. Moreover, by adjusting training parameters, CCA can achieve trade-offs between sample diversity and fidelity similar to CFG. This experimentally confirms the strong theoretical connection between language-targeted alignment and visual-targeted guidance methods, unifying two previously independent research fields.

Published as a conference paper at ICLR 2025
TOWARD GUIDANCE-FREE AR VISUAL GENERATION VIA CONDITION CONTRASTIVE ALIGNMENT
Huayu Chen', Hang Su', Peize Sun Jun Zhu Department of Computer Science & Technology, Institute for AI, BNRist Center, Tsinghua- -singhua-Bosch Joint ML Center, THBI Lab, Tsinghua University "The University of Hong Kong 3 SShnnghhu Technology, Beijing
ABSTRACT
Classifier-Free Guidance (CFG) is critical technique for enhancing the sample quality of visual generative models. However, autoregressive (AR) multi-modal generation, CFG introduces design inconsistencies between language and visual content, contradicting the design philosophy of unifying different modalities for vi- sual AR. Motivated by language model aligmment methods, propose Condition Contrastive Alignment (CCA) to facilitate guidance- free AR visual generation with high performance and analyzes its theoretical connection with guided sampling methods. Unlike guidance methods that alter the sampling process to achieve the ideal sampling distribution, CCA directly fine-tunes pretrained models to fit the same distribution target. Experimental results show that CCA can significantly enhance the guidance-free performance of all tested models with just one epoch of fine-tuning (~1% of pretraining epochs) on the pretraining dataset, on par with guided sampling methods. This largely removes the need for guided sampling AR visual generation and cuts the sampling cost by half. Moreover, by adjust- ing training parameters, CCA can achieve trade-offs between sample diversity and fidelity similar to CFG. This experimentally confirms the strong theoretical connection between language-targeted aligmment and visual-targeted guidance methods, unifying two previously independent research fields. Code and models: com cot.tcumm/ccccccc
CCA
wia gridanee
300
2423
20.4
25
pélrnce
wi o'gaddonno
240
202
6411
250
2--1
(CA(ou))
1131
200
150
wio axidanee gaidance
EDV
6.0
w/ gaduse
goiaano
100
150
5-40d
16.
21
11M
343M
3100
(a) LlamaGen
(b) VAR
Figure I: CCA significantly improves guidance-free sample quality for AR visual generative models with just one epoch of fine-tuning on the pretraining dataset.
INTRODUCTION
Witnessing the scalability and generalizability of autoregressive (AR) models language domains, recent works have been striving to replicate similar success for visual generation (Esser et al.) 2021 Lee et al. 2022). By quantizing images into discrete tokens, AR visual models can process images using the same next-token prediction approach Large Language Models (LLMs). This approach is
"oorrespnniing author

---

Published as a conference paper at ICLR 2025
attractive because it provides potentially unified framework for vision and language, promoting consistency in reasoning and generation across modalities (Team 2024 Xie et al. (024).
Despite the design philosophy of maximally aligning visual modeling with language modeling methods, AR visual generation still differs from language generation in a notable aspect. AR visual generation relies heavily Classifier-Free Guidance (CFG) (Ho & Salimans] 2022), sampling technique unnecessary for language generation, which has caused design inconsistencies between the two types of content. During sampling, while CFG helps improve sample quality by contrasting conditional and unconditional models, it requires two model inferences per visual token, which doubles the sampling cost. During training, CFG requires randomly masking text conditions to learn the unconditional distribution, preventing the simultaneous training of text tokens (Team 2024).
In contrast to visual generation, LLMs rarely rely on guided sampling. Instead, the surge of LLMs' corrucctioo-ollowinn abilities has largely benefited from tenntt--anseeee alignment methods (Schul) man et al., 2022). Motivated by this observation, we seek to study: "Can we avoid guided sampling in AR visual generation, but attain similar effects by directly fine-tuning pretrained models?"
In this paper, we derive Condition Contrastive Alignment (CCA) for enhancing visual AR performance without guided sampling. Unlike CFG which necessitates altering the sampling process to achieve more desirable sampling distribution, CCA directly fine-tunes pretrained AR models to fit the same distribution target, leaving the sampling scheme untouched. CCA is quite convenient to use since it does not rely on any additional datasets beyond the pretraining data. Our method functions by contrasting positive and negative conditions for given image, which can be easily created from the existing pretraining dataset as matched or mismatched image-condition pairs. CCA is also highly efficient given its fine-tuning nature. We observe that our method achieves ideal performance within just one training epoch, indicating negligible computational overhead (~19% % of pretraining).
In Sec. 4 we highlight theoretical connection between CCA and guided sampling techniques (Dhariwal & Nichol 2021; Ho & Salimans 2022). Essentially these methods all target at the same sampling distribution. The distributional gap between this target distribution and pretrained models is related to a physical quantity termed conditional residual (log p(ale)) ). Guidance methods typically p(m)
train an additional model (c.g.. unconditional model or classifier) to estimate this quantity and enhance pretrained models by altering their sampling process. Contrastively, CCA follows LLM alignment
techniques (Rafailov et al. 2023 Chen et al. 2024a) and parameterizes the conditional residual with the difference between our target model and the pretrained model, thereby directly training a sampling model. This analysis unifies language aunguage-targeedd alignment and visual-targeter guidance methods, bridging the gap between the two previously independent research fields.
We apply CCA to two state-of of---off-hhee -te autoregressive (AR) visual models, LLamaGen (Sun et al. 2024) and VAR (Tian et al.] 2024), which feature distinctly different visual tokenization designs. Both quantitative and qualitative results show that CCA significantly and consistently enhances the guidance-free sampling quality across all tested models, achieving performance levels comparable to CFG (Figure []. We further show that by varying training hyperparameters, CCA can realize controllable trade-off off between image diversity and fidelity similar to CFG. This further confirms their theoretical connections. We also compare our method with some existing LLM alignment methods (Welleck et al. 2019 Rafailov et al. 2023) to justify its algorithm design. Finally, we demonstrate that CCA can be combined with CFG to further improve performance.
Our contributions: 1. We take a big step toward guidance-free visual generation by significantly improving the visual quality of AR models. 2. We reveal a theoretical connection between alignment and guidance methods. This shows that language-targett alignment can be similarly applied to visual generation and effectively replace guided sampling, closing the gap between these two fields.
2

---

Published as a conference paper at ICLR 2025
2 BACKGROUND
2.1 AUTOREGRESSIVE (AR) VISUAL MODELS
Autoregressive models. Consider data æ represented by a sequence of discrete tokens i1,N := (21, J2,, NNJ. where each token In is integer. Data probability p(æ) can be decomposed as:
N
p(a)) = p(x1) II
n=2
AR models thus aim to learn (T x [cn)) = p( [<<n), where each token H, is conditioned only on its previous input &enn This is known next-token prediction (Radford al. 2018
Visual tokenization. Image pixels continuous values, making it necessary use vector- quantized tokenizers for applying discrete AR models to visual data (Van Den Oord et al. 2017] Esser et al. (021). These tokenizers are trained to encode images into discrete token sequences Ii.N and decode them back by minimizing reconstruction losses. In our work, we utilize pretrained and frozen visual tokenizers, allowing AR models to process images similarly text.
2.2 GUIDED SAMPLING FOR VISUAL GENERATION
Despite the motivation of developing unified model for language and vision, the AR sampling strategies for visual and text contents differ in one key aspect: AR visual generation necessitates a sampling technique named Classifier-Free Guidance (CFG) (Ho & Salimans 2022). During inference, CFG adjusts the sampling logits gample for each token as:
gample ll s(L ("))
where e and 8 are the conditional and unconditional logits provided by two separate AR models, Po (xlc) and po (x)). The condition e can be class labels or text captions, formalized as prompt tokens. The scalar is termed guidance scale. Since token logits represent the (unnormalized) log- likelihhoood in AR models, Ho & Salimans (2022) prove that the sampling distribution satisfies:
Po (x(e))
pample (x|c) x (a(())
px(x))
At s = 0, the sampling model becomes exactly the pretrained conditional model Pa- However, previous works (Ho Salimans 2022; Podell et al. 20033 Chang et al., 20023] Sun et al. (2024) have widely observed that an appropriate diversity, making training another unconditional model pa necessary. In practice, the unconditional model usually shares parameters with the conditional one, and can be trained concurrently by randomly dropping condition prompts e during training.
O is critical for an ideal trade-off between visual fidelity and
Other guidance methods, such Classifier Guidance (Ho & Salimans 2022) and Energy Guidance (Lu et al., (202)) have similar effects of CFG. The target sampling distribution of these methods can all be unified under Eq.
2.3 DIRECT PREFERENCE OPTIMIZATION FOR LANGUAGE MODEL ALIGNMENT
Reinforcement Learning from Human Feedback (RLHF) is crucial for enhancing the instruction- following ability of pretrained Language Models (LMs) (Schulman et al. 2022 OpenA] 2023). Performing RL typically requires reward model, which can be learned from human preference data. Formally, the Bradley-Terry preference model (Bradley & Terry 1952) assumes.
(c,www)
pl mylc) (cc.x.) + (r(c, (r((,, (,,(,,,,))))))) -((,, r(),
where Tn and o are respectively the winning and losing response for an instruction c, evaluated by human. r(-) represents an implicit reward for each response. The target LM Fe should satisfy (x(c)c) x riteee)) attain higher implicit reward compared with the pretrained LM ps-
3

---

Published as a conference paper at ICLR 2025
Direct Preference Optimization (Rafailov et al. 2023) allows to directly optimize pretrained LMs on preference data, by formalizing ro C, x) := Blog To - B log (z(c):
To (
3 log (
C (Ec....a log B
(5)
Hs (w)c)
(x(
DPO is more streamlined and thus often more favorable compared with traditional two-stage RLHF pipelines: first training reward models, then aligning LMs with reward models using RL.
3 CONDITION CONTRASTIVE ALIGNMENT
Autoregressive visual models are essentially learning a parameterized model pa (zzlc) to approximate the standard conditional image distribution p(> c). Guidance algorithms shift the sampling policy y pample (z)ee) e) away from p(æ(c) according to Sec. 2.2
p(z]cc)
sample
[ p(x/c)
(6)
p(x)
At guidance scale sampling from prample = p(x]ec) 2 (x(c)c is most straightforward. However, it is widely observed that an appropriate s > usually leads to significantly enhanced sample quality. The cost that we rely on an extra unconditional model pa(x)) z p(æ) for sampling. This doubles the sampling cost and causes an inconsistent training paradigm with language.
In this section, we derive a simple approach to directly model the same target distribution plamle using sample Specifically, our methods leverage a singular loss function for directly sample (æ/cc R paamle (xlc). Despite having similar effects as guided sampling, our approach does not require altering the sampling process. We theoretically derive our method in Sec. 3.1 and discuss its practical implementation in Sec. 3.2
a single AR model P/g
optimizing pretrained models (#lcc ~ p(æ[cc to become Pe
3.1
ALGORITHM DERIVATION
The core difficulty of directly learning Pa of prample However, we observe the distributional difference between p prample (axjer and p(ær(c) ()) is related simple quantity that can be potentially learned from existing datasets. Specifically, by taking the logarithm of both sides in Eq. and applying some algebra, we have
sample is that we cannot access datasets under the distribution
1 log p(alc)
pample (xlc) c)) p(x]cc) p(zz)
log
of which the right-hand side (i.e., log p(alc) p(z) corresponds to the log gap between the conditional probability and unconditional probability for an image æ, which we term as conditional residual.
Our key insight here is that the conditional residual can be directly learned through contrastive learning approaches (Gutmann & Hyvärinen 2012), as sated below:
Theorem 3.1 (Noise Contrastive Estimation, proof Appendix A). Let To be a parameterized model which takes in an image-condition pair (x, c) and oulputs scalar value ro(w c). Consider the loss function:
CV CNCE c) Ep( log (re (x: c)) pp(p)ppc) log logo((ro(z,e))
(8)
where a() is the standard logistic function: (w)) == 1/(1+e
Given unlimited model expressivity for rg, the optimal solution for minimizing CNCE satisfies
rō c) = log p(x(cc
(9)
p(x)
Now that we have a tractable way of learning ro c) R log p(alc) p(æ) the target distribution p sample can
be jointly defined by ro(æ, c) and the pretrained model Ps- However, we would still lack an explicitly
We ignore normalizing constant in Eq. 7 for brevity. A more detailed discussion is in Appendix B
4

---

Published as a conference paper at ICLR 2025
C1
C2 <Doop
CK um
Pusitive data (c. c)
p
<Bin>
Pg (Target)
Negative data p(x)p(c)
pa Pp
X1
log
log
X2
init
Us jo,Al
max
X3
tte
P (Pretrained)
pay p
log a
log
X
(t
(a) Training batch
(b) model likelihood
(c) Alignment loss
Figure An overview of the CCA method. Given a training batch of K <image, label> pairs, CCA treats these as positive samples, and generates K negative samples by randomly assigning negative label from remaining labels for each image. CCA then fine-tunes pretrained models by contrasting positive and negative data using an alignment loss. Pseudo code in Appendix [
parameterized model Po draw inspiration from the widely studied alignment techniques in language models (Rafailov et al. 2023) and parameterize ro ro c) with our target model Ps
sumple if ro (xe e) is another independent network. To address this problem,
sample (x/c) and (a((c) according to Eq.
sample (x/c)
Po
ro = loo
(10)
(p(a(c
Then, the loss function becomes
sample (aele)
sample (a)le)
Pe
log Pe
Lg -F pxp log
log
TF ( log a
(11)
Po (2(cc
s
(x(2
sample is learnable while pretrained pa is frozen. Po can be initialized from Pa- During training, Po
sample
This way we can sample with single AR model P4 sample eliminating the need for training separate sample
unconditional model for guided sampling. Sampling strategies for Po are consistent with standard language model decoding methods, which unifies decoding systems for multi-modal generation.
3.2 PRACTICAL ALGORITIM
Figure illustrates the CCA method. Specifically, implementing Eq. requires approximating two expectations: one under the joint distribution p(z, e) and the other under the product its two marginals p(=))((c) The key difference between these distributions is that in p(æ; c), images and conditions e are correctly paired. In contrast, xr and e are sampled independently from p(z)p((), meaning they are most likely mismatched.
In practice, we rely solely on the pretraining dataset to estimate CCCA Consider a batch of K data neg pairs (x, chi.k. We randomly shuffle the condition batch C1:K to become Ci:k where each c represents negative condition of image Fg. while the original e₁ is positive one. This results in our training batch (x, n J.k. The loss function
sample (It (ca) neg )= =- log a log Lp (mg, Ck. C+
sample (#E
neg
Po
-Alog 3 log
P6
C
(12)
Ps (x(((()))
Ps (xt C₁
relative likelihood for positive conditions
relative likelihoud for negative cunditions
where 3 and À are two hyperparameters that can be adjusted. B replaces the guidance scale parameter
while À is for controlling the loss weight assigned to negative conditions. The learnable Pe initialized from the pretrained conditional model Pg. making CCCA fine-tuning loss.
sample is
We give an intuitive understanding of Eq. 12 Note that log o-)) is monotonically increasing, The first term of Eq. aims to increase the likeldhood of image given a positive condition, with
5

---

Published as a conference paper at ICLR 2025
Method
Classifier Guidance
Classifier-Free Guidance
Condition Contrastive Alignment
Modeling log Rea)
pielet
log (o(lar) log p(c)
log (a)cc) log Ro(a))
Allog pauple (arle) (arle)
lo
Training loss
maxe Epta.e) log po(c)ar log pa(ale) llg po(laz)
mAXe By(e) log Pe(a)
ming C in Eq
Sampling policy
+ +) pa(xlcc pw(e))
log ample (z)ec
training
of learning Ps
~-10% learning Po
pretraining Ps
Sampling
x1.3
Applicable area
Diffusion
Diffusion & Autoregressive
Autoregressive
Table 1: Comparison of CCA (ours) and guidance methods in visual generative models.
similar effect to maximum likelihood training. For mismatched mmage-condition data, the second term explicitly minimizes its relative model likelihood compared with the pretrained Pa-
We name the above training technique Condition Contrastive Alignment (CCA) due its contrastive nature in comparing positive and negative conditions with respect to single image. This naming also reflects its theoretical connection with Noise Contrastive Estimation (Theorem
CONNECTION BETWEEN CCA AND GUIDANCE METHODS
As summarized in Table 1 the key distinction between CCA and guidance methods is how to model p(ale) which defines the distributional gap between the target yample (x|c) and p(æle) (Eq. 7
log p(m)
In particular, Classifier Guidance (Dhariwal & Nichol 2021 leverages Bayes' Rule and turn p(alc) into a conditional posterior: log p())
p c) log p(cl=) log p(c) pe log p(c),
log
p(x)
where p(c/æ) is explicitly modeled by classifier pa (c|æ), which is trained by standard classification loss. p(c) is regarded as a uniform distribution. CFG trains an extra unconditional model po (x) to estimate the unknown part of log p(alc).
p(x)
p
log
se log (=(x)c) log pa (x))
p(x)
Despite their effectiveness, guidance methods all require learning separate model and modified sampling process compared with standard autoregressive decoding. In comparison, CCA leverages Eq. and models log p(ale)
p(æ)
log p(x|c) p(x)
Bloogr sample (x)cc c)) - (ogps(alc))
which allows to directly learn Pa
sample instead of another guidance network.
Although CCA and conventional guidance techniques have distinct modeling methods, they all target at the same sampling distribution and thus have similar effects in visual generation. For instance, we show in Sec. 5.2 that CCA offers a similar trade-off between sample diversity and fidelity to CFG.
EXPERIMENTS
We seek to answer the following questions through our experiments:
1. How effective is CCA enhancing the guidance- free generation quality of pretrained AR visual models, quantitatively and qualitatively? (Sec. 5.1)
2. Does CCA allow controllable trade-offs between sample diversity (FID) and fidelity (IS) similar to CFG? (Sec. 5.2)
3. How does CCA perform comparison to alignment algorithms for LLMs? (Sec. 5.3
4. Can CCA be combined with CFG further improve performance? (Sec. 5.4

---

Published as a conference paper at ICLR 2025
w/o Guidance
w/ Guidance
Model
FID
1ST
Precision
Recallf
FID.
1St
ADM
Dhariwal & Nichol
2021
7.49
127.5
0.72
0.63
3,94
215.8
Diffusion
LDM-4
[Rombach et al. 2022)
10.56
103.5
0.71
0.62
3.60
247.7
U-ViT-H/Z
(Bao et al. 2023)
2.29
263.9
DiT-XL/2
Peeblee Xie
2023
9.62
121.5
0.67
0.67
2.27
278.2
MDTv2-XL/2 Gao al.
2023)
5.06
155.6
0.72
0.66
1.58
314.7
MaskGIT Chang al. 2022)
6.18
182.1
0.80
0.51
Mask
MAGVIT-v2 Yu et al.
2023)
3.65
200.5
1.78
319.4
MAGE (Li et al. 2023
6.93
195.8
VQGAN (Esser et al.
2021)
15.78
74.3
5,20
280.3
Autoregressive
ViT- Yuu et al.
2021
4.17
175.1
3.04
227.4
RQ-Trrasfformer (Lee et al. 2022
7.55
134.0
3.80
323.7
LlamaGen-3B (Sun et al. 2024)
9.38
112.9
0.69
0.67
2.18
263.3
+CCA (Ours)
2.69
276.8
0.80
0.59
VAR-d30 (Tian al. 2024
5.25
175.6
0.75
0.62
1.92
323.1
+CCA (Ours)
2.54
264.2
0.83
0.56
Table 2: Model comparisons on class-conditional ImageNet 256 x 256 benchmark.
LlamaGen (w/o Guidance)
LlamaGen CCA (w/o G.) IS=384.6
LlamaGen (w/ CFG) IS=404.0
1S=64.7
VAR (w/o Guidance) 15=154.3
VAR CCA (w/o G.) IS=350.4
VAR (w/ CFGv2) IS=390.8
Figure 3: CCA and CFG can similarly enhance the sample fidelity of AR visual models. The base models are LlamaGen-L (343M) and VAR-d24 (1.OB). We use s = 3.0 for CFG, and B = 0.02, A 10 for CCA. Figure 7 and Figure contain more examples.
5.1 TOWARD GUIDANCE-FREE AR VISUAL GENERATION
Base model. We experiment two families of publicly accessible AR visual models, LlamaGen (Sun et al., 2024) and VAR (Tian et al. 2024). Though both are class-conditioned models pretrained ImageNet, LlamaGen and VAR feature distinctively different tokenizer and architecture designs.
7

---

Published as a conference paper at ICLR 2025
LlamaGenn-L
VAR-d24
6-
+CCA
15
CPG
CRG
CFGv2
FID
A-104
10-
8-
2-
100
200
300
400
150
200
250
300
350
400
Figure 4: CCA can achieve similar FID-IS trade-offs to CFG by adjusting training parameter X.
LlamaGen focuses on reducing the inductive biases visual signals. It tokenizes images in the classic raster order and adopts almost the same LLM architecture as Llama (Touvron et al.) (023a). VAR, on the other hand, leverages the hierarchical structure images and tokenizes them in multi- scale, coarse-to-fine manner. VAR adopts GPT-2 architecture but tailors the attention mechanism specifically for visual content. For both works, CFG a default and critical technique.
Training setup. We leverage CCA to finetune multiple LlamaGen and VAR models of various sizes on the standard ImageNet dataset. The training scheme and hyperparameters are mostly consistent with the pretraining phase. We report performance numbers after only one training epoch and find this to be sufficient for ideal performance. We fix 3 = 0.02 in Eq. 12 and select suitable À for each model. Image resolutions 384 x 384 for LlamaGen and 256 256 for VAR. Following the original work, we resize LlamaGien samples to 256 x 256 whenever required for evaluation.
Experimental results. We find CCA significantly improves the guidance- free performance of all tested models (Figure[]) [T]. evaluated by metrics like FID (Heusel et al. (017)) and IS (Salimans et al.) 2016). For instance, after one epoch CCA fine-tuning, the FID score of LlamaGen-L (343M) improves from 19.07 to 3.41, and the IS score from 64.3 to 288.2, achieving performance levels comparable to CFG. This result compelling, considering that CCA has negligible fine-tunning overhead compared with model pretraining and only half of sampling costs compared with CFG.
Figure 3 presents qualitative comparison of image samples before and after CCA fine-tuning. The results clearly demonstrate that CCA can vastly improve image fidelity, as well as class-image alignment of guidance-fin free samples.
Table comparcs our best-performing models with several oftte-of-theaan visual generative models. With the help of CCA, we achieve a secord-brcaking FID score of 2.54 and an IS score of 276.8 for guidance-fiee free samples of AR visual models. Although these numbers still somewhat lag behind CFG-enhanced performance, they demonstrate the significant potential of alignment algorithms to enhance visual generation and indicate the future possibility of replacing guided sampling.
5.2 CONTROLLABLE TRADE-OFFS BETWEEN DIVERSITY AND FIDELITY
A distinctive feature of CFG is its ability to balance diversity and fidelity by adjusting the guidance scale. It is reasonable expect that CCA can achieve similar trade-off since essentially targets the same sampling distribution as CFG.
Figure 4 confirms this expectation: by adjusting the parameter for fine-tuning, CCA can achieve similar FID-IS trade-offs to CFG. The key difference is that CCA enhances guidance-fres models through training, while CFG mainly improves the sampling process.
It is worth noting that VAR employs a slightly different guidance technique from standard CFG, which we refer to as CFGv2. CFGv2 involves linearly increasing the guidance scale s during sampling, which was first proposed by [Chan et al. (2023) and found beneficial for certain models. The FID-IS curve of CCA more closely resembles that of standard CFG. Additionally, the hyperparameter also affects CCA performance. Although
algorithm derivation shows that 3 is directly related to the CFG scale s, we empirically find adjusting / is less effective and less predictable compared with adjusting X. In practice, we typically fix B and adjust A. We ablate 3 in Appendix C
8

---

Published as a conference paper at ICLR 2025
Model
FID]
1S
sFID]
Precision
Recall
Model
FID.
18
sFID]
Precision
Recall
DlmaaGen-L
19,00
64.7
8.78
0.61
0.67
VAR-024
6.20
154.3
8,30
0.74
0.62
+DPO
30.8
44.98
0.40
+DPO
7.53
232.6
19.10
0.34
+Unleam
12.22
111.6
7.99
0.66
0.64
+Unleam
5.55
165.9
8.41
0.75
0.61
+CCA
3.43
288.2
7.44
0.81
0.52
+CCA
2.63
298.8
7.63
0.84
0.55
Table 3: Comparision of CCA and LLM alignment algorithms in visual generation.
340
363
Opiket Croaaee (do (DD
sarpling gaklid sompling
gailed
puided sampling
SeCCAndyy
CCA
Se CEA C7G
200
97
100
1.0
0.0
25
300
CCA
Figure 5: The impact of training parameter À on the performance of CCA+CFG.
5.3 CAN LLM ALIGNMENT ALGORITHMS ALSO ENHANCE VISUAL AR?
Intuitively, existing preference-based LLM alignment algorithms such as DPO and Unlearning (Welleck et al., 2019) should also offer improvement for AR visual models given their similarity to CCA. However, Table shows that naive applications of these methods fail significantly.
DPO. As is described in Eq. 5 one can treat negative imagc-condition pairs dispreferred data and positive ones as preferred data to apply the DPO loss. We ablate Ba E (0.01, 0.1, 1.0, 10.0) and report the best performance in Table 3 Results indicate that DPO fails to enhance pretrained models, even causing performance collapse for LlamaGen-L. By inspecting training logs, we find that the likelihood of the positive data continuously decreases during fine-tuning, which may explain the collapse. This phenomenon is well-observed problem of DPO (Chen et al., 2024a Pal et al. 2024). stemming from its focus on optimizing only the relative likelihood between preferred and dispreferred data, rather than controlling likelihood for positive and negative image-condition pairs separately. We refer interested readers Chen al, (2024a) for detailed discussion.
Unlearning. Also known as unlikelihood training, this method maximizes log Pe(x)c) through standard maximum likelihood training on positive data, while minimizing log po( = unlearn negative data. A training parameter X, controls the weight of the unlearning loss. We find that with small 0.01 < X < 0.1, Unlearning provides some benefit, but it is far less effective than CCA. This suggests the necessity of including frozen reference model.
5.4 INTEGRATION OF CCA AND CFG
If extra sampling costs and design inconsistencies of CFG are not concerns, could CCA still be helpful? A takeaway conclusion is yes: CCA+CFG generally outperforms CFG (Figure 6 but requires distinct hyperparameter choices compared with CCA-only training.
Implementation. After pretraining the unconditional AR visual model by randomly dropping conditions, CFG requires us to also fine-tune the unconditional model during alignment. To achieve this, follow previous approaches by also randomly replacing data conditions with [MASK] tokens at a probability 10%. These unconditional samples are treated as positive data during CCA training. We provide pseudo-code in Appendix D
They require different hyperparameters. As shown in Figure larger À is typically needed for optimal FID scores in guidance- generation. For models optimized for guidance- free sampling, adding CFG guidance does not further reduce the FID score. However, with smaller X, CCA+CFG could outperform the CFG method.
Comparison of CCA-only and CCA+CFG.

---

Published as a conference paper at ICLR 2025
CCA oCAadly
100
CVG only
218
203
275-
250
256
2011
305
332
22.
CCA-CFG
CCA.ely oaly
CEG only
48225
113M
343M
775M
1.43
5.18
131M
343
775M
14B
1.18
Figure 6: Integration of CCA+CFG yields improved performance over CFG alone.
6 RELATED WORKS
Visual generative models. Generative adversarial networks (GANs) Goodfellow et al. 2014
Brock et al., 2018 Karras et al.) 2019 Kang et al. 2023) and diffusion models (Ho et al. 2020
Song Ermon 2019 Song al. 2020: Dhariwal & Nichol 2021 Kingma & Gao 2024 are
representative modeling methods for visual content generation, widely recognized for their ability to
produce realistic and artistic images (Sauer al. 2022 Ho al. 2022, Ramesh al. 2022 Podell
et al. 2023). However, because these methods are designed for continuous data like images, they struggle effectively model discrete data such as text, limiting the development of unified multimodal models for both vision and language. Recent approaches seek to address this by integrating diffusion
models with language models (Team 2024 Li et al.] [024]] Zhou et al. 2024). Another line works
(Chang et al. 2022; 2023 Yu et al. 2023 Xie al. [2024] Ramesh al. 2021: Yu et al. 2022)
explores discretizing images (Van Den Oord et al. [ Esser et al. 202 and directly applying
language models such as BERT-style (Devlin al. 2018) masked-prediction models and GPT--style
(Radford et al. 2018) autoregressive models for image generation.
Language model alignment. LLMs primarily employ training- alignment techniques to improve cntrucctionfollowinng abilities (Touvron af. 2203bb OpenAl 2023). Reinforcement Learning (RL) is well-suited for aligning LLMs with human feedback (Schulman et al.] [017]
Ouyang et al., 2022). However, this method requires learning a reward model before optimizing LLMs, leading to an indirect two-stage optimization process. Recent developments in alignment techniques (Rafailov et al.) 2023] Chen et al., 2024a Ji et al. 2024) have streamlined this process. They enable direct alignment of LMs through singular loss. Among all LLM aligmment algorithms, our method is perhaps most similar to NCA (Chen et al. 2024a). Both NCA and CCA are grounded in the NCE framework (Gutmann & Hyvärinen] 2012). Their differences mainly empirical regarding loss implementations, particularly how to estimate expectations under the product of two marginal distributions.
Visual alignment. Motivated by the success of alignment techniques in L.MSs, several studies have also investigated aligning visual generative models with human preferences using RI. (Black itt al. 2023a| Xu et al. 2024) or DPO (Black et al., 2023b Wallace et al. 2023). For diffusion models, the application is not straightforward and must rely some thcoretical approximations, as diffusion models do not allow direct likelihood calculation, which is required by most LLM alignment algorithms (Chen et al. 2024b).
7 CONCLUSION
In this paper, we propose Condition Contrastive Alignment (CCA) as a fine-tuning algorithm for AR visual generation models. CCA can significantly enhance the guidance-free sample quality of pretrained models without any modification of the sampling process. This paves the way for further development in multimodal generative models and cuts the cost of AR visual generation by half in comparison to CFG. Our research also highlights the strong theoretical connection between language- ttrgeted alignment and visual-targeted guidance methods, facilitating future research of unifying visual modeling and language modeling.
10

---

Published as a conference paper at ICLR 2025
ACKNOWLEDGEMENT
We thank Fan Bao, Kai Jiang, Xiang Li, Min Zhao, Keyu Tian and Kaiwen Zheng for their valuable suggestions and discussion. This work was supported by the National Natural Science Foundation of China (Nos. 62350080, 62106120, (BNR2022R00006)) Tsinghua Institute for Guo Qiang, and the High Performance Computing Center, Tsinghua University; J. Z was also supported by the XPlorer Prize.
136001, 92370124, 92248303, 92270001), BNRist
REPRODUCIBILIIY
We provide experimental details in Appendix E We submit our source code in the supplementary material. Code and model weights publicty accessible.
REFERENCES
Fan Bao, Shen Nie, Kaiwen Xue, Yue Cao, Chongxuan Li, Hang Su, and Jun Zhu. All worth words: A vit backbone for diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 11669-22679, 2023,
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2200. 13301, 2023a.
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning. arXiv preprint arXiv:2305. 13301, 2023b.
Ralph Allan Bradley and Milton E Terry. Rank analysis of incomplete block designs: I. the method of paired comparisons. Biometrika, 39(3/4):324-444 1952.
Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. arXiv preprint arXiv: I096, 2018
Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William Freeman. Maskgit: Masked generative image transformer. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 11315 11315-11325, 2022.
Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image generation via masked generative transformers. arXiv preprint vrXiv:2301.00700 2023.
Huayu Chen, Guande He, Hang Su, and Jun Zhu. Noise contrastive alignment of language models with explicit rewards. Advances in neural information processing systems, 2024a.
Huayu Chen, Kaiwen Zheng, Hang Su, and Jun Zhu. Aligning diffusion behaviors with fuuuction for efficient continuous control. Advances in neural information processing systems, 2024b.
Jacob Devlin, Ming- bidirectional transformers for language understanding. arXiv preprint 1rXiv:1810.04805, 2018.
Ming-Wei Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep
Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis. Advances in neural information processing systems, 0148870-8794 2021.
Patrick Esser, Robin Rombach, and Bjorn Ommer. Taming transformers for high-resolution image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 12873-12883, 2021.
Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is strong image synthesizer. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 23166-23173, 2023.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farles Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances neural information processing systems, 27, 2014.
11

---

Published as a conference paper at ICLR 2025
Michael U Gutmann and Aapo Hyvärinen. models, with applications to natural image statistics. Journal of machine learning research, 13(2), 2012.
estimation of unnormalized statistical
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by two time-scale update rule converge local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Jonathan Ho and Tim Salimans. Classifier-free diffusion guidance. arXiv preprint arXiv:2207.1259. 2022.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in neural information processing systems, 33:6840-6851, 2020.
Jonathan Ho, Chitwan Saharia, William Chan, David Fleet, Mohammad Norouzi, and Tim Salimans.
Cascaded diffusion models for high fidelity image generation. The Journal of Machine Learning Research, (5(1):22492228) 2022.
Haozhe Ji, Cheng Lu, Yilin Niu, Pei Ke, Hongning Wang, Jun Zhu, Jie Tang, and Minlie Huang. Towards efficient and exact optimization of language model alignment. arXiv preprint arXiv:2402.00886, 2024.
Minguk Kang, Jun-Yan Zhu, Richard Zhang, Jaesik Park. Eli Shechtman, Sylvain Paris, and Taesung Park. Scaling up gans for text-to-image synthesis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10124-10134, 2023.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 4401-4410, 2019.
Diederik Kingma and Ruiqi Gao. Understanding diffusion objectives as the elbo with simple data augmentation. Advances in Neural Information Processing Systems, 36, 2024.
Doyup Lee, Chiheon Kim, Saehoon Kim, Minsu Cho, and Wook-Shin Han. Autoregressive image generation using residual quantization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 11523- 11532, 2022.
Tianhong Li, Huiwen Chang, Shlok Mishra, Han Zhang, Dina Katabi, and Dilip Krishnan. Mage: Masked generative encoder to unify representation learning and image synthesis. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 2142-2152, 2023.
Tianhong Li, Yonglong Tian, He Li, Mingyang Deng, and Kaiming He Autoregressive image generation without vector quantization. arXiv preprint arXiv:2406. 1838, 2024.
Cheng Lu, Huayu Chen, Jianfei Chen, Hang Su, Chongxuan Li, and Jun Zhu. Contrastive energy prediction for exact grrddd diffusion sampling in offline reinforcement learning. In International Conference on Machine Learning, pp. 22825-22855. PMLR, 2023.
Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation guided diffusion models. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition (CVPR), pp. 14297-14306, June 2023.
OpenAl. Gpt-4 technical report. arXiv preprint arXiv:2303. 08774, 2023.
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 35:27730- 27744, 2022.
Arka Pal, Deep Karkhanis, Samuel Dooley, Manley Roberts, Siddartha Naidu, and Colin White. Smaug: Fixing failure modes of preference optimisation with dpo-positive. arXiv preprint arXiv:2402. 13228, 2024.
12

---

Published as a conference paper at ICLR 2025
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 4195 4205,,4205550 2023.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952. 2023.
Alec Radford, Karthik Narasimhan, Tim Salimans, llya Sutskever, et al. Improving language understanding by generative pre-training. article, 2018.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Thirty-seventh Conference on Neural Information Processing Systems, 2023.
Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and llya Sutskever. Zero-shot text-to-image generation. In International Conference on Machine Learning, pp. 8821-8831. PMLR, 2021.
Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text- conditional image generation with clip latents. arXiv preprint arXiv:2204.06125, 1(2):3, 2022.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Tim Salimans, Ian Goodfellow, Wojeiech Zaremba, Vicki Cheung, Alec Radford, and Xi Chen. Improved techniques for training gans. Advances in neural information processing systems, 29, 2016.
Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xL. Scaling stylegan to large diverse datasets. In ACM SIGGRAPH 2022 conference proceedings, pp. 1-10, 2022.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint
2017.
John Schulman, Barret Zoph, Christina Kim, Jacob Hilton, Jacob Menick, Jiayi Weng, Juan Fe- lipe Ceron Uribe, Liam Fedus, Luke Metz, Michael Pokorny, al. Chatgpt: Optimizing language models for dialogue. OpenAl blog, 2022.
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. arXiv preprint wwXiv:2010.025000 2020.
Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. Advances in neural information processing systems, 32, 2019.
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.065555 2024.
Chameleon Team. Chamelcon: Hixed-modas carly-fusion foundation models. arXiv preprint 0988084405009818. 2024.
Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng. and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction. arXiv preprint arXiv:2404.02905. 2024.
Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothéc Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302. 13971, 2023a.
Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaci, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, et al. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288. 2023b.
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
13

---

Published as a conference paper at ICLR 2025
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik. Diffusion model alignment using direct preference optimization. arXiv preprint arXiv:2311.12900 2023.
Sean Welleck, Ilia Kulikov, Stephen Roller, Emily Dinan, Kyunghyun Cho, and Jason Weston. Neural text generation with unlikelihood training. arXiv preprint urXiv:1908 04319, 2019.
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou. Show-o: One single transformer to unify multimodal understanding and generation. arXiv preprint arXiv:2408. 12528, 2024.
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. Advances in Neural Information Processing Systems, 36, 2024.
Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge. and Yonghui Wu. Vector-quantized image modeling with improved vogan. arXiv preprint arXiv:2/ 10.04627, 2021.
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content- rich -oxae generation. arXiv preprint arXiv:2206, 10789, 2(3):5, 2022.
Lijun Yu, José Lezama, Nitesh Gundavarapu, Luca Versari, Kihyuk Sohn, David Minnen, Yong Cheng, Agrim Gupta, Xiuye Gu, Alexander G Hauptmann, al. Language model beats diffusion- tokenizer is key to visual generation. arXiv preprint arXiv:2310.05737. 2023.
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy. Transfusion: Predict the next token and diffuse images with one multi-modal model. arXiv preprint arXiv:2408. 1039, 2024.
14

---

Published as a conference paper at ICLR 2025
w/o Guidance
+CCA (w/o Guidance)
w/ CFG Guidance
Figure 7: Comparison LlamaGen-L samples generated with CCA or CFG.
15

---

Published as a conference paper at ICLR 2025
2
w/o Guidance
+CCA (w/o Guidance)
w/ CFG Guidance
Figure Comparison of VAR-d24 samples generated with CCA or CFG.
16

---

Published as a conference paper at ICLR 2025
A THEORETICAL PROOFS
In this section, we provide the proof of Theorem 3.1
Theorem A.1 (Noise Contrastive Estimation ). Let re be a parameterized model which takes in an image-condition pair (x, c) and outputs scalar value re( c). Consider the loss function:
Co (z, c) Ep(m,e) log (r c)) pp(x)p(e) log ((-re(zx,)))
(13)
Given unlimited model expressivity for ro, the optimal solution for minimizing LICE satisfies
r0 c) = loog p(æ|c)
(14)
p(x)
Proof. First, we construct two binary (Bernoulli) distributions:
p(a,c) c)
p(x)p(()
( p(a)(c) c)
p(æ))
Qu ( p(a, c) + p(z)p(c) p(z, p(x)p(c) "p(zlc) (plzlc) p(x) p(z|c) p(xx)
pd etola,c)
(to(a,c) +1' cro(m,c)+1 (o(ro(z, (
Then we rewrite LNC (x, c) as
CNCE (x, c) -p(z,e) logo c)) Eplert log
/ ((z,,))logorr(t bogp://coo...oooooppoooo/ooooooo drde
/
+ +
p(a,c) c)
p(z)p(()
logo(ro(z,c)) p(x,c) p(x)p(()
log [1 (ro( c)]] dæde
pla, c) + p())p(c)
/ (p(x,c) + p()))(()) H(Qu.c Pe daode
/ (v( p(a))(()) (Kt.( H(Qm..) d
Here H(Qu.e Pd represents the cross-entropy between distributions Qu.e and P H H(Qu, is the entropy of Qx,c, which can be regarded as constant number with respect to parameter O. Due to the theoretical properties of Kl.-divergence, we have
NCF Cg
(xx c) = -/
(((()))(((((())))))))))))))))))) [DKL [paaQQucllP2.dd + (Qu..)) dade
> (p(a,c) c) ( + H(Qa,)ddade
constantly hold. The equality holds if and only Qu,e Pd m.o such that
p(æ, c)
arraw c)) (to(an.) + p(x, c) + p( m (())p()))))))))
ro(x, c) log p(a, c)
log p(xle)
p(x))p(c)
p(x)
17

---

Published as a conference paper at ICLR 2025
B THEORETICAL ANALYSES OF THE NORMALIZING CONSTANT
We omit normalizing constant Eq. for brevity when deriving CCA. Strietly speaking, the target sampling distribution should be:
1
p(a])c)
sample (xlc)- = Z(c)
p( (a)
p(x:)
such that
log p pasmplo (ar]c) log p(a: [c)cc
log Z(c).
p(a|c)
p(z)
The normalizing constant (c) ensures that p samplle
(a) c) is properly normalized, i.e., | = Bx(e)e ppale)
(z/c)da = We have Z(c) J [
p(a)
p(a)
To mitigate the additional effects introduced by Z(c). in our practical algorithm, introduce new training parameter À to bias the optimal solution for Noise Contrastive Estimation. Below, we present result that is stronger than Theorem 3.1
Theorem B.1. Let de > 0 be scalar function conditioned only on Consider the loss function:
CN c) = -pii,e) log (r c)) AoEp(e)pte) log (-re(xx c)).
(15)
Given unlimited model expressivity for rg. the optimal solution for minimizing Cy satisfies
p(a|c) log Ae.
(x, c)
log
(16)
p( a
Proof. We omit the full proof here, requires only redefinition of the distributions Qa,e from the proof of Theorem A.I.
p(x,c) Qu,c p(z, {ep(a)p(c) p(z, c) {cp(2)p(c) p(x|c) >ep(x) p(z]c) + )ep(a)
{ep(z)p(c)
p(acc)
)ep(x)
Then we can follow the steps in the proof of Theorem A.I arrive at the result.
let de == 7(c)+ [ppllel]] p(alec)
m()) [+] we could guarantee the convergence P9
sample to p sample
However, in practice estimating ((c) could be intricately difficult, formalize de as training parameter, resulting in our practical algorithm in Eq. 12
C ADDITIONAL EXPERIMENTAL RESULTS
We provide more image samples to compare CCA and CFG Figure Segureeane and Figure
We illustrate the effect of training parameter 3 on the FID-IS trade-off in Figure Overall, 3 affects the -ideeity-diversity trade-off similar and the CFG method.
Medel
15+
CCA (odiuting
(adjuating
50
150
200
220
300
350
Figure 9: Effect of varying 3 of CCA for the LlamaGen-L model. our CCA experiments, we either
fix À- = 1e3 and ablate 3 E [2, 5e - 3) (from left to right) or fix 3 = 0.02 and ablate À E [0, le4]. In
our CFG experiments, we ablate [0.3] 3).
18

---

Published as a conference paper at ICLR 2025
0.5
Relative Likellhood
0.1
Positive Like hood Ratio
CCA
Negative Likelihood Ratio
-400
Mmy
0.01
-600
0.005
3000
1000
300D
4000
5000
Gradient
Gradient Steps
Figure 10: Training curves of CCA for LlamaGen-L model (3 0.02, À 300). Left: CCA loss.
Right: Relative likelihood log (u(n)c) for positive and negative data during training. Pe (x/cc
D PSEUDO CODE
Algorithm CCA
Input: Pretraining dataset {ær, c), pretrained AR model ps, target model po- Initialize 0 = For each gradient step do Sample K data pairs {æ, c.K:K from the dataset as positive samples // p(x,
Randomly shuffle cı:k become auy and form negative samples {ar e 1:K 1/ p(x)p(e) If CCA+CFG then For each label o in Ci.K and nee do Replace with Ø with probability of 10% // Random masking
La 0
For For each data f Ck in training batch {æ, c)1. and {æz c 1:Kk do
La Le log 3 log Pe (axle))
if (æx, ex is positive sample es 2e (mxlee)
Le Lo Alog
(8 log Pa(#xln))
if (az,, Ck } negative sample and ct +
- Voto (Eq.
12
We provide an example of training curves for CCA in Figure 10
E TRAINING HYPERPARAMETERS
Table reports hyperparameters for chosen models in Figure [ and Figure Other unmen- tioned design choices and hyperparameters are consistent with the default setting for LlamaGen www.///ooaaaiooooooooooooooooooe wwt.iiisiinn.... sioonnl amaGen| and VAR hwam/Poundaatinvissiiooo VAR repo. All models are fine-tuned for 1 epoch on the ImageNet dataset. We use a mix NVIDIA NVIDIA A100, and NVIDIA A40 GPU cards for training.
/
Type
LlamaGen
VAR
Model
B
L
XL
XXL
3B
d16
d20
d24
d30
Size
111M
343M
775M
1.4B
3.1B
310M
600M
1.0B
2.0B
CCA
0.02
0.02
0.02
0.02
0.02
0.02
0.02
0.02
0.02
CCA À
1000
300
1000
1000
500
50
50
100
1000
CCA+CFG
0.1
0.02
0.1
0.1
0.1
CCA+CFG
1
1
1
Learning rate
le-5
le-5
le-5
le-5
le-5
2e-5
2e-5
2e-5
2e-5
Dropout?
Yes
Yes
Yes
Yes
Yes
None
Yes
Yes
Yes
Batch size
256
256
256
256
256
256
256
256
256
Table 4: Hyperparameter table.
All our reported models are trained individually for each hyperparameter. However, we note that hyperparameters like À and can input for target AR visual model using existing
19

---

Published as a conference paper at ICLR 2025
distillation techniques (Meng al. 2023) that way CCA can allow test-time flexibility just like CFG. We present an initial result conditioning the LlamaGen-L model parameter À in Table In order to additionally condition input A, we use the same embedding method as the one used by DiT (Peebles & Xie 2023) and directly add the À embedding during training. The model is trained for 3 epochs.
tune them only during inference. This
extra scalar
the class token embeddings. We randomly sample À
(e 10000)
Inference-ting À
10
100
300 (Chosen)
1000
3000
10000
FID IS
7.23
4.18
3.59
3.73
4.12
5.33
153.1
218.8
256.2
277.5
307.7
341.2
Table 5: Performance for different inference-time À values. For reference, the pretrained LlamaGen model has an IS of 64.3 and an FID of 19.07. After CCA finetuning with fixed 300, the finetuned model has an IS 288.2 of and FID of 3.43.
20


Reviews:
[{"review_id": "PHTuAzTuU7", "reviewer": "Reviewer_bJkX", "summary": "To bridge the gap between autoregressive language and visual generation, this work targets at removing the need of CFG in AR image generation, therefore unifying the sampling scheme and reducing the sampling cost. The authors choose to fine-tuning the pretrained AR models with the contrastive alignment loss. The intuition behind the contrastive alignment is to maximize the likelihood of positive conditions and minimize the likelihood of the negative conditions. More discussion with classifier guidance and classifier-free guidance is also illustrated, providing more insights for difference sampling schemes. Experiments are conducted on autoregressive visual generation methods like LlamaGen and VAR.", "questions": "Could the authors clarify the necessity of removing CFG, except sampling cost?\n\nShould the model be fine-tuned individually for each lambda and beta?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "* The problem of sampling schemes for autoregressive models is fundamental and should be explored.\n\n* Detailed illustrations are provided. For example, the authors review preliminaries, give formal derivation, and also the practical implementation.\n\n* It is interesting to see that CCA provides extra gain based on CFG.", "weaknesses": "* For the motivation, one confusion to me is why removing the CFG is necessary for unifying visual and language generation. I mean, there may no influence for sampling them with slightly different schemes? I recognize that introducing CCA can reduce the half cost. But it is achieved at the cost of sampling quality as demonstrated in Table 2.\n\n* To my understanding, the model should be fine-tuned for each lambda and beta individually. Is that correct? If so, this will be a big limitation since other sampling schemes can adjust trade-offs during inference, which is more flexible.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MxjH8pqRvq", "reviewer": "Reviewer_WGPM", "summary": "This paper proposes CCA to replace CFG in auto-regressive models, which saves sampling costs while keeps the sampling performance on part with CFG. And this paper gives the theoretical connection", "questions": "1. Your pipeline requires only 1% of the pre-trained epochs. However, could you provide insights into the corresponding GPU memory usage during training in comparison to CFG?\n2. Regarding the proposed loss function in Equation 11, what is its convergence speed?\n3. A more detailed inquiry: in line 372, the experiments do not utilize a common default value of s=7.5, for Classifier-Free Guidance (CFG). Could you elaborate on the reason behind this choice?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "1. The theoretical derivation is clear and logically structured, allowing for a comprehensive understanding of the underlying concepts.\n2. The authors have conducted a thorough series of experiments that effectively validate the theoretical claims made in the paper.", "weaknesses": "1. concerns mentioned in questions.\n2. lack of explanation for notations when first mentioned, such as $p_{\\theta}$  in line 168, $\\sigma$ in line 189", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2BKPYjtfvn", "reviewer": "Reviewer_8sAn", "summary": "This paper introduces CCA which a new alternative to CFG to finetune autoregressive image generation models so that they work like the sampling model used by CFG during inference. The key idea lies in modeling the conditional residual by learning to maximize the likelihood of an image given positive conditions, while minimizing the likelihood given negative conditions. In my opinion, it can be intuitively seen as a way to absorbing the benefits of the unconditional model (which complements the conditional model as demonstrated by CFG) right into single conditional model inference. The resulting approach is more elegant than CFG and is more efficient than CFG because it only requires 1x sampling cost. Furthermore, it can be paired with CFG for additional inference quality boost.", "questions": "Please refer to weaknesses", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "- The paper tackles an important but often overlooked problem in autoregressive image generation.\n\n- The methodology section of the paper is mostly well written except that it lacks key information in certain areas. Overall, it provides strong motivations and reasonable derivations based on prior work that lead to the proposed CCA method. \n\n- The experimental results using strong AR visual generation models (LLamaGen, VAT) as baselines on ImageNet are very promising. It is impressive that without any additional inference costs (and a bit more training/finetuning costs), CCA can bring significant improvements to the FID and IS metrics. When paired with CFG, improved FIDs can be achieved.", "weaknesses": "- Some important details are missing in the paper, such as how positive and negative data samples are chosen for CCA training. Sec 3.2 only briefly mentions that and Fig. 2 shows something about it without any verbal explanation. It is unclear if we just use the ImageNet labels as the sole criterion for constructing positives and negatives or there’s some other strategy being used. In Sec 5.4, it’s mentioned that certain unconditional data samples are used as positives but it is not clear how exactly that works. Without this key information, it may be hard for readers to understand how and why CCA works.\n\n- Fig 2‘s caption is totally uninformative. There should be a caption describing the method in detail.\n\n- CCA uses models pretrained with CFG training (i.e., condition dropout). For CCA to work, is there a strong requirement on this? What is the relationship of CFG pretraining and CCA finetuning?\n\n- When CCA is combined with CFG, IS becomes worse while FID becomes better. Can the authors explain this phenomenon? \n\n- The proposed method resembles CLIP-like contrastive loss. I wonder if it can only work if the training batch size is large enough.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1VEUyH6LpH", "reviewer": "Reviewer_CPMH", "summary": "The authors propose condition contrastive alignment (CCA), replacing classifier-free guidance (CFG). Based on CCA, they implement a guidance-free AR visual generation and unifies the alignment mechanism between language and visual models. Compared with models using CFG, generation models with CCA can reduce the sampling cost by half. To implement CCA, the authors transform CFG into a contrastive learning formulation and train the sampling model only with a fine-tuning loss. In addition, they conduct extensive experiments to validate the effectiveness of the proposed method.", "questions": "To make the proposed method better, the authors need to provide practical solutions to some issues like training cost with large-scale datasets.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "+ This paper proposes condition contrastive alignment (CCA) for a guidance-free AR visual generation. Concretely, the authors transform CFG into a contrastive learning formulation and train the sampling model only with a fine-tuning loss. Thus, CCA is simple and easy to implement.\n+ The proposed method can unifies the alignment mechanism between language and visual models.  Besides, unlike CFG, generation models with CCA can reduce the sampling cost by half, which is a practical technique.\n+ The authors conduct extensive experiments to validate the proposed method. Compared with CFG, CCA achieves comparable performance in FID score under different visual models.", "weaknesses": "- Training cost. CCA needs one training epoch to get ideal performance. The training cost is negligible for ImageNet datasets. But for existing large visual generation models (like SD3), the training data is huge. The cost of training one epoch with large-scale datasets is expensive. Is there a practical solution to reduce the cost?\n- Optimal $\\beta$ and $\\lambda$. To obtain the optimal values for $\\beta$ and $\\lambda$, we need to train CCA many times. The cost of this process is large. Is there a potential guideline to mitigate the substantial cost of finding optimal $\\beta$ and $\\lambda$?\n- Negative samples in CCA. To my knowledge, negative samples have a large effect on the performance of contrastive learning. For CCA, it is based on contrastive learning framework. The selection of negative samples affects the performance of CCA largely. Is there a selection mechanisms of  negative samples?\n- Implementation about integration of CCA and CFG. I do not understand the implementation completely due to lacks of details. It is necessary to clarify the implementation with more details.\n- IS score. As shown in Figure 1 and Figure 6, IS score starts to degrade when model size is beyond a certain size, such as LlamaGen>775M. To my knowledge, this is an unusual phenomenon. Larger model means better IS score.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "9WgjVWlsr5", "reviewer": "Reviewer_Zf6F", "summary": "The authors propose a novel method for fine-tuning auto-regressive models to enable guidance-free sampling from the same distribution that would be sampled if classifier-free guidance were used. The training objective is derived by noise contrastive estimation in which genuine pairs of condition and data are used as positive while random pairs are negative ones. The experimental results show that just one epoch fine-tuning with the proposed method greatly improves the performance of the guidance-free sampling.", "questions": "- It is not clearly descirbed why the performance with varying lambda leads to a similar behaviour to that with varying beta. For example, why has IS score been improved by increasing the value of lambda(namely, overweighing the loss on negative pairs)?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- The motivation of this study is clear and should be important to make auto-regressive models efficiently generate high-quality visual contents. As of 2024, CFG is commonly used for auto-regressive models to achieve high-quality image generation, while it doubles the computational cost of the generation process. How to get rid of the necessity of CFG would be an important problem especially for practical applications.\n\n- The derivation of the proposed method, particularly Eq. (10) and (11), is elegant. The theoretical connection with guidance-based sampling is clear, and the practical implementation is reasonable.\n\n- The experimental results validate the advantage of the proposed method; it greatly boosts the performance of guidance-free sampling. Although its performance is slightly worse than that of sampling with CFG, it would be acceptable in some practical applications considering that the computational cost for the generation is halved.\n\n- The manuscript is well-written and easy to follow.", "weaknesses": "- I do not find any crucial weakness or major concerns on this paper. The followings are just minor points.\n  - As the proposed method requires to fix a target distribution represented by beta, it cannot control the fidelity-diversity trade-off in the inference phase, which might reduce some flexibility compared with CFG.\n    - It might be worth exploring whether the model could be extended to accept additional condition beta to work with various beta in the inference phase, similar to the approach in the distillation of guided diffusion models [R1]. However, it should be non-trivial how to achieve such extention in AR models.\n\n      [R1] \"On Distillation of Guided Diffusion Models,\" CVPR 2023.\n  - It would be beneficial if the author could discuss how stable the proposed method is. As the negative pairs in Eq. (12) are random pairs of x and c, p(x|c) appeared in the loss computation could have very small values, which might cause numerical instability during the fine-tuning.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MAXMKJHe33", "reviewer": "Reviewer_qqAa", "summary": "This submission proposes an alternative way to achieve classifier-free guidance (CFG) for visual autoregressive model, by directly finetuning the model, which avoids an extra forward pass when deploying CFG. The experiment results show the proposed finetuning method introduces huge performance gain.", "questions": "Please check the weakness.", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The problem the submission addresses has very high practical value to deploying the visual autoregressive model.\n- The algorithm derivation gives a clear explanation about the methods. And the derived training loss function is very concise, which is easy to train.\n- The performance gain is huge, on both generation quality and diversity.\n- Experimental section is thorough. Additionally, sec 5.3 is informative and appreciated, especially unlearning.", "weaknesses": "In short, this is a good work. I only have one question about the training loss function, as $\\beta$ is expected to be responsible for CFG degree. But the experimental results show that $\\lambda$ actually dominates the CFG controllability, any clue about it?\n\nOne open question is, will the proposed method also improve the directional autoregressive model, a.k.a., Masked Generative Models such as MaskGIT and MAGE. Or, to think even more boldly, will the method still applicable to (continuous) diffusion models, since MaskGIT could be regarded as a discrete diffusion model.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Oral)