Toward Guidance-Free AR Visual Generation via Condition Contrastive Alignment

## Abstract

Abstract Classifier-Free Guidance (CFG) is a critical technique for enhancing the sample quality of visual generative models. However, in autoregressive (AR) multi-modal generation, CFG introduces design inconsistencies between language and visual content, contradicting the design philosophy of unifying different modalities for visual AR. Motivated by language model alignment methods, we propose Condition Contrastive Alignment (CCA) to facilitate guidance-free AR visual generation with high performance and analyze its theoretical connection with guided sampling methods. Unlike guidance methods that alter the sampling process to achieve the ideal sampling distribution, CCA directly fine-tunes pretrained models to fit the same distribution target. Experimental results show that CCA can significantly enhance the guidance-free performance of all tested models with just one epoch of fine-tuning ( ∼ similar-to \sim ∼ 1% of pretraining epochs) on the pretraining dataset, on par with guided sampling methods. This largely removes the need for guided sampling in AR visual generation and cuts the sampling cost by half. Moreover, by adjusting training parameters, CCA can achieve trade-offs between sample diversity and fidelity similar to CFG. This experimentally confirms the strong theoretical connection between language-targeted alignment and visual-targeted guidance methods, unifying two previously independent research fields. Code and model weights: https://github.com/thu-ml/CCA .

## 1 Introduction

Witnessing the scalability and generalizability of autoregressive (AR) models in language domains, recent works have been striving to replicate similar success for visual generation (Esser et al., ; Lee et al., ). By quantizing images into discrete tokens, AR visual models can process images using the same next-token prediction approach as Large Language Models (LLMs). This approach is attractive because it provides a potentially unified framework for vision and language, promoting consistency in reasoning and generation across modalities (Team, ; Xie et al., ).

Despite the design philosophy of maximally aligning visual modeling with language modeling methods, AR visual generation still differs from language generation in a notable aspect.
AR visual generation relies heavily on Classifier-Free Guidance (CFG) (Ho & Salimans, ), a sampling technique unnecessary for language generation, which has caused design inconsistencies between the two types of content. During sampling, while CFG helps improve sample quality by contrasting conditional and unconditional models, it requires two model inferences per visual token, which doubles the sampling cost. During training, CFG requires randomly masking text conditions to learn the unconditional distribution, preventing the simultaneous training of text tokens (Team, ).

In contrast to visual generation, LLMs rarely rely on guided sampling. Instead, the surge of LLMs’ instruction-following abilities has largely benefited from fine-tuning-based alignment methods (Schulman et al., ). Motivated by this observation, we seek to study: “Can we avoid guided sampling in AR visual generation, but attain similar effects by directly fine-tuning pretrained models?”

In this paper, we derive Condition Contrastive Alignment (CCA) for enhancing visual AR performance without guided sampling. Unlike CFG which necessitates altering the sampling process to achieve a more desirable sampling distribution, CCA directly fine-tunes pretrained AR models to fit the same distribution target, leaving the sampling scheme untouched. CCA is quite convenient to use since it does not rely on any additional datasets beyond the pretraining data. Our method functions by contrasting positive and negative conditions for a given image, which can be easily created from the existing pretraining dataset as matched or mismatched image-condition pairs. CCA is also highly efficient given its fine-tuning nature. We observe that our method achieves ideal performance within just one training epoch, indicating negligible computational overhead ($\sim$1% of pretraining).

In Sec. 4, we highlight a theoretical connection between CCA and guided sampling techniques (Dhariwal & Nichol, ; Ho & Salimans, ). Essentially these methods all target at the same sampling distribution. The distributional gap between this target distribution and pretrained models is related to a physical quantity termed conditional residual ($\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}$). Guidance methods typically train an additional model (e.g., unconditional model or classifier) to estimate this quantity and enhance pretrained models by altering their sampling process. Contrastively, CCA follows LLM alignment techniques (Rafailov et al., ; Chen et al., ) and parameterizes the conditional residual with the difference between our target model and the pretrained model, thereby directly training a sampling model. This analysis unifies language-targeted alignment and visual-targeted guidance methods, bridging the gap between the two previously independent research fields.

We apply CCA to two state-of-the-art autoregressive (AR) visual models, LLamaGen (Sun et al., ) and VAR (Tian et al., ), which feature distinctly different visual tokenization designs. Both quantitative and qualitative results show that CCA significantly and consistently enhances the guidance-free sampling quality across all tested models, achieving performance levels comparable to CFG (Figure 1). We further show that by varying training hyperparameters, CCA can realize a controllable trade-off between image diversity and fidelity similar to CFG. This further confirms their theoretical connections. We also compare our method with some existing LLM alignment methods (Welleck et al., ; Rafailov et al., ) to justify its algorithm design. Finally, we demonstrate that CCA can be combined with CFG to further improve performance.

Our contributions: 1. We take a big step toward guidance-free visual generation by significantly improving the visual quality of AR models. 2. We reveal a theoretical connection between alignment and guidance methods. This shows that language-targeted alignment can be similarly applied to visual generation and effectively replace guided sampling, closing the gap between these two fields.

## 2 Background

### 2.1 Autoregressive (AR) Visual Models

##### Autoregressive models.

Consider data ${\bm{x}}$ represented by a sequence of discrete tokens ${\bm{x}}_{1:N}:=\{x_{1},x_{2},...,x_{N}\}$, where each token ${\bm{x}}_{n}$ is an integer. Data probability $p({\bm{x}})$ can be decomposed as:

$$ $p({\bm{x}})=p({\bm{x}}_{1})\prod_{n=2}^{N}p({\bm{x}}_{n}|{\bm{x}}_{<n}).$ (1) $$

AR models thus aim to learn $p_{\phi}({\bm{x}}_{n}|{\bm{x}}_{<n})\approx p({\bm{x}}_{n}|{\bm{x}}_{<n})$, where each token ${\bm{x}}_{n}$ is conditioned only on its previous input ${\bm{x}}_{<n}$. This is known as next-token prediction (Radford et al., ).

##### Visual tokenization.

Image pixels are continuous values, making it necessary to use vector-quantized tokenizers for applying discrete AR models to visual data (Van Den Oord et al., ; Esser et al., ). These tokenizers are trained to encode images ${\bm{x}}$ into discrete token sequences ${\bm{x}}_{1:N}$ and decode them back by minimizing reconstruction losses. In our work, we utilize pretrained and frozen visual tokenizers, allowing AR models to process images similarly to text.

### 2.2 Guided Sampling for Visual Generation

Despite the core motivation of developing a unified model for language and vision, the AR sampling strategies for visual and text contents differ in one key aspect: AR visual generation necessitates a sampling technique named Classifier-Free Guidance (CFG) (Ho & Salimans, ). During inference, CFG adjusts the sampling logits $\ell^{\text{sample}}$ for each token as:

$$ $\ell^{\text{sample}}=\ell^{c}+s(\ell^{c}-\ell^{u}),$ (2) $$

where $\ell^{c}$ and $\ell^{u}$ are the conditional and unconditional logits provided by two separate AR models, $p_{\phi}({\bm{x}}|{\bm{c}})$ and $p_{\phi}({\bm{x}})$. The condition ${\bm{c}}$ can be class labels or text captions, formalized as prompt tokens. The scalar $s$ is termed guidance scale. Since token logits represent the (unnormalized) log-likelihood in AR models, Ho & Salimans () prove that the sampling distribution satisfies:

$$ $p^{\text{sample}}({\bm{x}}|{\bm{c}})\propto p_{\phi}({\bm{x}}|{\bm{c}})\left[ \frac{p_{\phi}({\bm{x}}|{\bm{c}})}{p_{\phi}({\bm{x}})}\right]^{s}.$ (3) $$

At $s=0$, the sampling model becomes exactly the pretrained conditional model $p_{\phi}$. However, previous works (Ho & Salimans, ; Podell et al., ; Chang et al., ; Sun et al., ) have widely observed that an appropriate $s>0$ is critical for an ideal trade-off between visual fidelity and diversity, making training another unconditional model $p_{\phi}$ necessary. In practice, the unconditional model usually shares parameters with the conditional one, and can be trained concurrently by randomly dropping condition prompts ${\bm{c}}$ during training.

Other guidance methods, such as Classifier Guidance (Ho & Salimans, ) and Energy Guidance (Lu et al., ) have similar effects of CFG. The target sampling distribution of these methods can all be unified under Eq. 3.

### 2.3 Direct Preference Optimization for Language Model Alignment

Reinforcement Learning from Human Feedback (RLHF) is crucial for enhancing the instruction-following ability of pretrained Language Models (LMs) (Schulman et al., ; OpenAI, ). Performing RL typically requires a reward model, which can be learned from human preference data. Formally, the Bradley-Terry preference model (Bradley & Terry, ) assumes.

$$ $p({\bm{x}}_{w}\succ{\bm{x}}_{l}|{\bm{c}}):=\frac{e^{r({\bm{c}},{\bm{x}}_{w})}} {e^{r({\bm{c}},{\bm{x}}_{l})}+e^{r({\bm{c}},{\bm{x}}_{w})}}=\sigma(r({\bm{c}}, {\bm{x}}_{w})-r({\bm{c}},{\bm{x}}_{l})),$ (4) $$

where ${\bm{x}}_{w}$ and ${\bm{x}}_{l}$ are respectively the winning and losing response for an instruction ${\bm{c}}$, evaluated by human. $r(\cdot)$ represents an implicit reward for each response. The target LM $\pi_{\theta}$ should satisfy $\pi_{\theta}({\bm{x}}|{\bm{c}})\propto\mu_{\phi}({\bm{x}}|{\bm{c}})e^{r({\bm{c
}},{\bm{x}})/\beta}$ to attain higher implicit reward compared with the pretrained LM $\mu_{\phi}$.

Direct Preference Optimization (Rafailov et al., ) allows us to directly optimize pretrained LMs on preference data, by formalizing $r_{\theta}({\bm{c}},{\bm{x}}):=\beta\log\pi_{\theta}({\bm{x}}|{\bm{c}})-\beta
\log\mu_{\phi}({\bm{x}}|{\bm{c}})$:

$$ $\mathcal{L}^{\text{DPO}}_{\theta}=-\mathbb{E}_{\{{\bm{c}},{\bm{x}}_{w}\succ{ \bm{x}}_{l}\}}\log\sigma\left(\beta\log\frac{\pi_{\theta}({\bm{x}}_{w}|{\bm{c} })}{\mu_{\phi}({\bm{x}}_{w}|{\bm{c}})}-\beta\log\frac{\pi_{\theta}({\bm{x}}_{l }|{\bm{c}})}{\mu_{\phi}({\bm{x}}_{l}|{\bm{c}})}\right).$ (5) $$

DPO is more streamlined and thus often more favorable compared with traditional two-stage RLHF pipelines: first training reward models, then aligning LMs with reward models using RL.

## 3 Condition Contrastive Alignment

Autoregressive visual models are essentially learning a parameterized model $p_{\phi}({\bm{x}}|{\bm{c}})$ to approximate the standard conditional image distribution $p({\bm{x}}|{\bm{c}})$. Guidance algorithms shift the sampling policy $p^{\text{sample}}({\bm{x}}|{\bm{c}})$ away from $p({\bm{x}}|{\bm{c}})$ according to Sec. 2.2:

$$ $p^{\text{sample}}({\bm{x}}|{\bm{c}})\propto p({\bm{x}}|{\bm{c}})\left[\frac{p( {\bm{x}}|{\bm{c}})}{p({\bm{x}})}\right]^{s}.$ (6) $$

At guidance scale $s=0$, sampling from $p^{\text{sample}}({\bm{x}}|{\bm{c}})=p({\bm{x}}|{\bm{c}})\approx p_{\phi}({\bm
{x}}|{\bm{c}})$ is most straightforward. However, it is widely observed that an appropriate $s>0$ usually leads to significantly enhanced sample quality. The cost is that we rely on an extra unconditional model $p_{\phi}({\bm{x}})\approx p({\bm{x}})$ for sampling. This doubles the sampling cost and causes an inconsistent training paradigm with language.

In this section, we derive a simple approach to directly model the same target distribution $p^{\text{sample}}$ by fine-tuning pretrained models. Specifically, our methods leverage a singular loss function for optimizing $p_{\phi}({\bm{x}}|{\bm{c}})\approx p({\bm{x}}|{\bm{c}})$ to become $p_{\theta}^{\text{sample}}({\bm{x}}|{\bm{c}})\approx p^{\text{sample}}({\bm{x}
}|{\bm{c}})$. Despite having similar effects as guided sampling, our approach does not require altering the sampling process. We theoretically derive our method in Sec. 3.1 and discuss its practical implementation in Sec. 3.2.

### 3.1 Algorithm Derivation

The core difficulty of directly learning $p^{\text{sample}}_{\theta}$ is that we cannot access datasets under the distribution of $p^{\text{sample}}$. However, we observe the distributional difference between $p^{\text{sample}}({\bm{x}}|{\bm{c}})$ and $p({\bm{x}}|{\bm{c}})$ is related to a simple quantity that can be potentially learned from existing datasets. Specifically, by taking the logarithm of both sides in Eq. 6 and applying some algebra, we have(^1^11We ignore a normalizing constant in Eq. 7 for brevity. A more detailed discussion is in Appendix B.):

$$ $\frac{1}{s}\log\frac{p^{\text{sample}}({\bm{x}}|{\bm{c}})}{p({\bm{x}}|{\bm{c}} )}=\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})},$ (7) $$

of which the right-hand side (i.e., $\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}$) corresponds to the log gap between the conditional probability and unconditional probability for an image ${\bm{x}}$, which we term as conditional residual.

Our key insight here is that the conditional residual can be directly learned through contrastive learning approaches (Gutmann & Hyvärinen, ), as sated below:

###### Theorem 3.1 (Noise Contrastive Estimation, proof in Appendix A ) .

Let $r_{\theta}$ be a parameterized model which takes in an image-condition pair $({\bm{x}},{\bm{c}})$ and outputs a scalar value $r_{\theta}({\bm{x}},{\bm{c}})$. Consider the loss function:

$$ $\mathcal{L}^{\text{NCE}}_{\theta}({\bm{x}},{\bm{c}})=-\mathbb{E}_{p({\bm{x}},{ \bm{c}})}\log\sigma(r_{\theta}({\bm{x}},{\bm{c}}))-\mathbb{E}_{p({\bm{x}})p({ \bm{c}})}\log\sigma(-r_{\theta}({\bm{x}},{\bm{c}})).$ (8) $$

Given unlimited model expressivity for $r_{\theta}$, the optimal solution for minimizing $\mathcal{L}^{\text{NCE}}_{\theta}$ satisfies

$$ $r_{\theta}^{*}({\bm{x}},{\bm{c}})=\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}.$ (9) $$

Now that we have a tractable way of learning $r_{\theta}({\bm{x}},{\bm{c}})\approx\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}$, the target distribution $p^{\text{sample}}$ can be jointly defined by $r_{\theta}({\bm{x}},{\bm{c}})$ and the pretrained model $p_{\phi}$. However, we would still lack an explicitly parameterized model $p^{\text{sample}}_{\theta}$ if $r_{\theta}({\bm{x}},{\bm{c}})$ is another independent network. To address this problem, we draw inspiration from the widely studied alignment techniques in language models (Rafailov et al., ) and parameterize $r_{\theta}({\bm{x}},{\bm{c}})$ with our target model $p^{\text{sample}}_{\theta}({\bm{x}}|{\bm{c}})$ and $p_{\phi}({\bm{x}}|{\bm{c}})$ according to Eq. 7:

$$ $r_{\theta}({\bm{x}},{\bm{c}}):=\frac{1}{s}\log\frac{p_{\theta}^{\text{sample}} ({\bm{x}}|{\bm{c}})}{p_{\phi}({\bm{x}}|{\bm{c}})}.$ (10) $$

Then, the loss function becomes

$$ $\mathcal{L}^{\text{CCA}}_{\theta}=-\mathbb{E}_{{\color[rgb]{0,0,1}p({\bm{x}},{ \bm{c}})}}\log\sigma\Big{[}\frac{1}{s}\log\frac{p_{\theta}^{\text{sample}}({ \bm{x}}|{\bm{c}})}{p_{\phi}({\bm{x}}|{\bm{c}})}\Big{]}-\mathbb{E}_{{\color[rgb ]{0,0,1}p({\bm{x}})p({\bm{c}})}}\log\sigma\Big{[}-\frac{1}{s}\log\frac{p_{ \theta}^{\text{sample}}({\bm{x}}|{\bm{c}})}{p_{\phi}({\bm{x}}|{\bm{c}})}\Big{]}.$ (11) $$

During training, $p_{\theta}^{\text{sample}}$ is learnable while pretrained $p_{\phi}$ is frozen. $p_{\theta}^{\text{sample}}$ can be initialized from $p_{\phi}$.

Figure: Figure 2: An overview of the CCA method.
Refer to caption: x3.png

This way we can fit $p^{\text{sample}}$ with a single AR model $p_{\theta}^{\text{sample}}$, eliminating the need for training a separate unconditional model for guided sampling. Sampling strategies for $p_{\theta}^{\text{sample}}$ are consistent with standard language model decoding methods, which unifies decoding systems for multi-modal generation.

###### Theorem 3.1 (Noise Contrastive Estimation, proof in Appendix A ) .

### 3.2 Practical Algorithm

Figure 2 illustrates the CCA method. Specifically, implementing Eq. 11 requires approximating two expectations: one under the joint distribution ${\color[rgb]{0,0,1}p({\bm{x}},{\bm{c}})}$ and the other under the product of its two marginals ${\color[rgb]{0,0,1}p({\bm{x}})p({\bm{c}})}$. The key difference between these distributions is that in $p({\bm{x}},{\bm{c}})$, images ${\bm{x}}$ and conditions ${\bm{c}}$ are correctly paired. In contrast, ${\bm{x}}$ and ${\bm{c}}$ are sampled independently from $p({\bm{x}})p({\bm{c}})$, meaning they are most likely mismatched.

In practice, we rely solely on the pretraining dataset to estimate $\mathcal{L}^{\text{CCA}}_{\theta}$. Consider a batch of $K$ data pairs $\{{\bm{x}},{\bm{c}}\}_{1:K}$. We randomly shuffle the condition batch ${\bm{c}}_{1:K}$ to become ${\bm{c}}^{\text{neg}}_{1:K}$, where each ${\bm{c}}^{\text{neg}}_{k}$ represents a negative condition of image $x_{k}$, while the original ${\bm{c}}_{k}$ is a positive one. This results in our training batch $\{{\bm{x}},{\bm{c}},{\bm{c}}^{\text{neg}}\}_{1:K}$. The loss function is

$$ $\mathcal{L}^{\text{CCA}}_{\theta}({\bm{x}}_{k},{\bm{c}}_{k},{\bm{c}}^{\text{ neg}}_{k})=-\underbrace{\log\sigma\Big{[}\beta\log\frac{p_{\theta}^{\text{ sample}}({\bm{x}}_{k}|{\bm{c}}_{k})}{p_{\phi}({\bm{x}}_{k}|{\bm{c}}_{k})}\Big{ ]}}_{\text{relative likelihood for positive conditions}\;\uparrow}-\lambda \underbrace{\log\sigma\Big{[}-\beta\log\frac{p_{\theta}^{\text{sample}}({\bm{x }}_{k}|{\bm{c}}^{\text{neg}}_{k})}{p_{\phi}({\bm{x}}_{k}|{\bm{c}}^{\text{neg}} _{k})}\Big{]}}_{\text{relative likelihood for negative conditions}\;\downarrow},$ (12) $$

where $\beta$ and $\lambda$ are two hyperparameters that can be adjusted. $\beta$ replaces the guidance scale parameter $s$, while $\lambda$ is for controlling the loss weight assigned to negative conditions. The learnable $p_{\theta}^{\text{sample}}$ is initialized from the pretrained conditional model $p_{\phi}$, making $\mathcal{L}^{\text{CCA}}_{\theta}$ a fine-tuning loss.

We give an intuitive understanding of Eq. 12. Note that $\log\sigma(\cdot)$ is monotonically increasing. The first term of Eq. 12 aims to increase the likelihood of an image ${\bm{x}}$ given a positive condition, with a similar effect to maximum likelihood training. For mismatched image-condition data, the second term explicitly minimizes its relative model likelihood compared with the pretrained $p_{\phi}$.

We name the above training technique Condition Contrastive Alignment (CCA) due to its contrastive nature in comparing positive and negative conditions with respect to a single image. This naming also reflects its theoretical connection with Noise Contrastive Estimation (Theorem 3.1).

## 4 Connection between CCA and guidance methods

As summarized in Table 1, the key distinction between CCA and guidance methods is how to model $\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}$, which defines the distributional gap between the target $p^{\text{sample}}({\bm{x}}|{\bm{c}})$ and $p({\bm{x}}|{\bm{c}})$ (Eq. 7).

In particular, Classifier Guidance (Dhariwal & Nichol, ) leverages Bayes’ Rule and turn $\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}$ into a conditional posterior:

$$ $\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}=\log p({\bm{c}}|{\bm{x}})-\log p( {\bm{c}})\approx\log p_{\theta}({\bm{c}}|{\bm{x}})-\log p({\bm{c}}),$ $$

where $p({\bm{c}}|{\bm{x}})$ is explicitly modeled by a classifier $p_{\theta}({\bm{c}}|{\bm{x}})$, which is trained by a standard classification loss. $p({\bm{c}})$ is regarded as a uniform distribution.
CFG trains an extra unconditional model $p_{\theta}({\bm{x}})$ to estimate the unknown part of $\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}$:

$$ $\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}\approx\log p_{\phi}({\bm{x}}|{\bm {c}})-\log p_{\theta}({\bm{x}}).$ $$

Despite their effectiveness, guidance methods all require learning a separate model and a modified sampling process compared with standard autoregressive decoding.
In comparison, CCA leverages Eq. 7 and models $\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}$ as

$$ $\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}\approx\beta[\log p_{\theta}^{ \text{sample}}({\bm{x}}|{\bm{c}})-\log p_{\phi}({\bm{x}}|c)],$ $$

which allows us to directly learn $p_{\theta}^{\text{sample}}$ instead of another guidance network.

Although CCA and conventional guidance techniques have distinct modeling methods, they all target at the same sampling distribution and thus have similar effects in visual generation. For instance, we show in Sec. 5.2 that CCA offers a similar trade-off between sample diversity and fidelity to CFG.

Table: Table 1: Comparison of CCA (ours) and guidance methods in visual generative models. Computational costs are estimated according to Dhariwal & Nichol () and Ho & Salimans ().

## 5 Experiments

We seek to answer the following questions through our experiments:

- 1.
How effective is CCA in enhancing the guidance-free generation quality of pretrained AR visual models, quantitatively and qualitatively? (Sec. 5.1)
- 2.
Does CCA allow controllable trade-offs between sample diversity (FID) and fidelity (IS) similar to CFG? (Sec. 5.2)
- 3.
How does CCA perform in comparison to alignment algorithms for LLMs? (Sec. 5.3)
- 4.
Can CCA be combined with CFG to further improve performance? (Sec. 5.4)

### 5.1 Toward Guidance-Free AR Visual Generation

**Table 2: Model comparisons on class-conditional ImageNet $256\times 256$ benchmark.**
|  | Model | w/o Guidance | w/ Guidance |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| FID$\downarrow$ | IS$\uparrow$ | Precision$\uparrow$ | Recall$\uparrow$ | FID$\downarrow$ | IS$\uparrow$ |  |  |
| Diffusion | ADM  (Dhariwal & Nichol, ) | 7.49 | 127.5 | 0.72 | 0.63 | 3.94 | 215.8 |
| LDM-4  (Rombach et al., ) | 10.56 | 103.5 | 0.71 | 0.62 | 3.60 | 247.7 |  |
| U-ViT-H/2  (Bao et al., ) | – | – | – | – | 2.29 | 263.9 |  |
| DiT-XL/2  (Peebles & Xie, ) | 9.62 | 121.5 | 0.67 | 0.67 | 2.27 | 278.2 |  |
| MDTv2-XL/2  (Gao et al., ) | 5.06 | 155.6 | 0.72 | 0.66 | 1.58 | 314.7 |  |
| Mask | MaskGIT  (Chang et al., ) | 6.18 | 182.1 | 0.80 | 0.51 | – | – |
| MAGVIT-v2  (Yu et al., ) | 3.65 | 200.5 | – | – | 1.78 | 319.4 |  |
| MAGE  (Li et al., ) | 6.93 | 195.8 | – | – | – | – |  |
| Autoregressive | VQGAN  (Esser et al., ) | 15.78 | 74.3 | – | – | 5.20 | 280.3 |
| ViT-VQGAN  (Yu et al., ) | 4.17 | 175.1 | – | – | 3.04 | 227.4 |  |
| RQ-Transformer  (Lee et al., ) | 7.55 | 134.0 | – | – | 3.80 | 323.7 |  |
| LlamaGen-3B  (Sun et al., ) | 9.38 | 112.9 | 0.69 | 0.67 | 2.18 | 263.3 |  |
| $\quad+$CCA (Ours) | 2.69 | 276.8 | 0.80 | 0.59 | – | – |  |
| VAR-d30  (Tian et al., ) | 5.25 | 175.6 | 0.75 | 0.62 | 1.92 | 323.1 |  |
| $\quad+$CCA (Ours) | 2.54 | 264.2 | 0.83 | 0.56 | – | – |  |

Figure: Figure 3: CCA and CFG can similarly enhance the sample fidelity of AR visual models. The base models are LlamaGen-L (343M) and VAR-d24 (1.0B). We use $s=3.0$ for CFG, and $\beta=0.02,\lambda=10^{4}$ for CCA. Figure 7 and Figure 8 contain more examples.
Refer to caption: extracted/5919300/pics-resize/c2i_L_384cfg1.0sample_207.png

##### Base model.

We experiment on two families of publicly accessible AR visual models, LlamaGen (Sun et al., ) and VAR (Tian et al., ). Though both are class-conditioned models pretrained on ImageNet, LlamaGen and VAR feature distinctively different tokenizer and architecture designs. LlamaGen focuses on reducing the inductive biases on visual signals. It tokenizes images in the classic raster order and adopts almost the same LLM architecture as Llama (Touvron et al., ). VAR, on the other hand, leverages the hierarchical structure of images and tokenizes them in a multi-scale, coarse-to-fine manner. VAR adopts a GPT-2 architecture but tailors the attention mechanism specifically for visual content. For both works, CFG is a default and critical technique.

##### Training setup.

We leverage CCA to finetune multiple LlamaGen and VAR models of various sizes on the standard ImageNet dataset. The training scheme and hyperparameters are mostly consistent with the pretraining phase. We report performance numbers after only one training epoch and find this to be sufficient for ideal performance. We fix $\beta=0.02$ in Eq. 12 and select suitable $\lambda$ for each model. Image resolutions are $384\times 384$ for LlamaGen and $256\times 256$ for VAR. Following the original work, we resize LlamaGen samples to $256\times 256$ whenever required for evaluation.

##### Experimental results.

We find CCA significantly improves the guidance-free performance of all tested models (Figure 1), evaluated by metrics like FID (Heusel et al., ) and IS (Salimans et al., ). For instance, after one epoch of CCA fine-tuning, the FID score of LlamaGen-L (343M) improves from 19.07 to 3.41, and the IS score from 64.3 to 288.2, achieving performance levels comparable to CFG. This result is compelling, considering that CCA has negligible fine-tunning overhead compared with model pretraining and only half of sampling costs compared with CFG.

Figure 3 presents a qualitative comparison of image samples before and after CCA fine-tuning. The results clearly demonstrate that CCA can vastly improve image fidelity, as well as class-image alignment of guidance-free samples.

Table 2 compares our best-performing models with several state-of-the-art visual generative models. With the help of CCA, we achieve a record-breaking FID score of 2.54 and an IS score of 276.8 for guidance-free samples of AR visual models. Although these numbers still somewhat lag behind CFG-enhanced performance, they demonstrate the significant potential of alignment algorithms to enhance visual generation and indicate the future possibility of replacing guided sampling.

### 5.2 Controllable Trade-Offs between Diversity and Fidelity

A distinctive feature of CFG is its ability to balance diversity and fidelity by adjusting the guidance scale. It is reasonable to expect that CCA can achieve a similar trade-off since it essentially targets the same sampling distribution as CFG.

Figure 4 confirms this expectation: by adjusting the $\lambda$ parameter for fine-tuning, CCA can achieve similar FID-IS trade-offs to CFG. The key difference is that CCA enhances guidance-free models through training, while CFG mainly improves the sampling process.

Figure: Figure 4: CCA can achieve similar FID-IS trade-offs to CFG by adjusting training parameter $\lambda$.
Refer to caption: x4.png

It is worth noting that VAR employs a slightly different guidance technique from standard CFG, which we refer to as CFGv2. CFGv2 involves linearly increasing the guidance scale $s$ during sampling, which was first proposed by Chang et al. () and found beneficial for certain models. The FID-IS curve of CCA more closely resembles that of standard CFG. Additionally, the hyperparameter $\beta$ also affects CCA performance. Although our algorithm derivation shows that $\beta$ is directly related to the CFG scale $s$, we empirically find adjusting $\beta$ is less effective and less predictable compared with adjusting $\lambda$. In practice, we typically fix $\beta$ and adjust $\lambda$. We ablate $\beta$ in Appendix C.

### 5.3 Can LLM Alignment Algorithms also Enhance Visual AR?

**Table 3: Comparision of CCA and LLM alignment algorithms in visual generation.**
| Model | FID$\downarrow$ | IS | sFID$\downarrow$ | Precision | Recall |
| --- | --- | --- | --- | --- | --- |
| LlamaGen-L | 19.00 | 64.7 | 8.78 | 0.61 | 0.67 |
| $\quad+$DPO | 61.69 | 30.8 | 44.98 | 0.36 | 0.40 |
| $\quad+$Unlearn | 12.22 | 111.6 | 7.99 | 0.66 | 0.64 |
| $\quad+$CCA | 3.43 | 288.2 | 7.44 | 0.81 | 0.52 |

Intuitively, existing preference-based LLM alignment algorithms such as DPO and Unlearning (Welleck et al., ) should also offer improvement for AR visual models given their similarity to CCA. However, Table 3 shows that naive applications of these methods fail significantly.

##### DPO.

As is described in Eq. 5, one can treat negative image-condition pairs as dispreferred data and positive ones as preferred data to apply the DPO loss. We ablate $\beta_{d}\in\{0.01,0.1,1.0,10.0\}$ and report the best performance in Table 3. Results indicate that DPO fails to enhance pretrained models, even causing performance collapse for LlamaGen-L. By inspecting training logs, we find that the likelihood of the positive data continuously decreases during fine-tuning, which may explain the collapse. This phenomenon is a well-observed problem of DPO (Chen et al., ; Pal et al., ), stemming from its focus on optimizing only the relative likelihood between preferred and dispreferred data, rather than controlling likelihood for positive and negative image-condition pairs separately. We refer interested readers to Chen et al. () for a detailed discussion.

##### Unlearning.

Also known as unlikelihood training, this method maximizes $\log p_{\theta}({\bm{x}}|{\bm{c}})$ through standard maximum likelihood training on positive data, while minimizing $\log p_{\theta}({\bm{x}}|{\bm{c}}^{\text{neg}})$ to unlearn negative data. A training parameter $\lambda_{u}$ controls the weight of the unlearning loss. We find that with small $0.01\leq\lambda_{u}\leq 0.1$, Unlearning provides some benefit, but it is far less effective than CCA. This suggests the necessity of including a frozen reference model.

### 5.4 Integration of CCA and CFG

Figure: Figure 5: The impact of training parameter $\lambda$ on the performance of CCA+CFG.
Refer to caption: x6.png

Figure: Figure 6: Integration of CCA+CFG yields improved performance over CFG alone.
Refer to caption: x8.png

If extra sampling costs and design inconsistencies of CFG are not concerns, could CCA still be helpful? A takeaway conclusion is yes: CCA+CFG generally outperforms CFG (Figure 6), but it requires distinct hyperparameter choices compared with CCA-only training.

##### Implementation.

After pretraining the unconditional AR visual model by randomly dropping conditions, CFG requires us to also fine-tune the unconditional model during alignment. To achieve this, we follow previous approaches by also randomly replacing data conditions with [MASK] tokens at a probability of 10%. These unconditional samples are treated as positive data during CCA training.

##### Comparison of CCA-only and CCA+CFG.

They require different hyperparameters. As shown in Figure 5, a larger $\lambda$ is typically needed for optimal FID scores in guidance-free generation. For models optimized for guidance-free sampling, adding CFG guidance does not further reduce the FID score. However, with a smaller $\lambda$, CCA+CFG could outperform the CFG method.

## 6 Related Works

##### Visual generative models.

Generative adversarial networks (GANs) (Goodfellow et al., ; Brock et al., ; Karras et al., ; Kang et al., ) and diffusion models (Ho et al., ; Song & Ermon, ; Song et al., ; Dhariwal & Nichol, ; Kingma & Gao, ) are representative modeling methods for visual content generation, widely recognized for their ability to produce realistic and artistic images (Sauer et al., ; Ho et al., ; Ramesh et al., ; Podell et al., ). However, because these methods are designed for continuous data like images, they struggle to effectively model discrete data such as text, limiting the development of unified multimodal models for both vision and language. Recent approaches seek to address this by integrating diffusion models with language models (Team, ; Li et al., ; Zhou et al., ). Another line of works  (Chang et al., ; ; Yu et al., ; Xie et al., ; Ramesh et al., ; Yu et al., ) explores discretizing images  (Van Den Oord et al., ; Esser et al., ) and directly applying language models such as BERT-style (Devlin et al., ) masked-prediction models and GPT-style (Radford et al., ) autoregressive models for image generation.

##### Language model alignment.

Different from visual generative models which generally enhance sample quality through sampling-based methods (Dhariwal & Nichol, ; Ho & Salimans, ; Zhao et al., ; Lu et al., ), LLMs primarily employ training-based alignment techniques to improve instruction-following abilities (Touvron et al., ; OpenAI, ). Reinforcement Learning (RL) is well-suited for aligning LLMs with human feedback (Schulman et al., ; Ouyang et al., ). However, this method requires learning a reward model before optimizing LLMs, leading to an indirect two-stage optimization process. Recent developments in alignment techniques (Rafailov et al., ; Cai et al., ; Azar et al., ; Ethayarajh et al., ; Chen et al., ; Ji et al., ) have streamlined this process. They enable direct alignment of LMs through a singular loss. Among all LLM alignment algorithms, our method is perhaps most similar to NCA (Chen et al., ). Both NCA and CCA are theoretically grounded in the NCE framework (Gutmann & Hyvärinen, ). Their differences are mainly empirical regarding loss implementations, particularly in how to estimate expectations under the product of two marginal distributions.

##### Visual alignment.

Motivated by the success of alignment techniques in LLMs, several studies have also investigated aligning visual generative models with human preferences using RL (Black et al., ; Xu et al., ) or DPO (Black et al., ; Wallace et al., ). For diffusion models, the application is not straightforward and must rely on some theoretical approximations, as diffusion models do not allow direct likelihood calculation, which is required by most LLM alignment algorithms (Chen et al., ). Moreover, previous attempts at visual alignment have primarily focused on enhancing the aesthetic quality of generated images and necessitate a different dataset from the pretrained one. Our work distinguishes itself from prior research by having a fundamentally different optimization objective (replacing CFG) and does not rely on any additional data input.

## 7 Conclusion

In this paper, we propose Condition Contrastive Alignment (CCA) as a fine-tuning algorithm for AR visual generation models. CCA can significantly enhance the guidance-free sample quality of pretrained models without any modification of the sampling process. This paves the way for further development in multimodal generative models and cuts the cost of AR visual generation by half in comparison to CFG. Our research also highlights the strong theoretical connection between language-targeted alignment and visual-targeted guidance methods, facilitating future research of unifying visual modeling and language modeling.

#### Acknowledgments

We thank Fan Bao, Kai Jiang, Xiang Li, and Min Zhao for providing valuable suggestions. We thank Keyu Tian and Kaiwen Zheng for the discussion.

## Appendix A Theoretical Proofs

In this section, we provide the proof of Theorem 3.1.

###### Theorem A.1 (Noise Contrastive Estimation ) .

Let $r_{\theta}$ be a parameterized model which takes in an image-condition pair $({\bm{x}},{\bm{c}})$ and outputs a scalar value $r_{\theta}({\bm{x}},{\bm{c}})$. Consider the loss function:

$$ $\mathcal{L}^{\text{NCE}}_{\theta}({\bm{x}},{\bm{c}})=-\mathbb{E}_{p({\bm{x}},{ \bm{c}})}\log\sigma(r_{\theta}({\bm{x}},{\bm{c}}))-\mathbb{E}_{p({\bm{x}})p({ \bm{c}})}\log\sigma(-r_{\theta}({\bm{x}},{\bm{c}})).$ (13) $$

Given unlimited model expressivity for $r_{\theta}$, the optimal solution for minimizing $\mathcal{L}^{\text{NCE}}_{\theta}$ satisfies

$$ $r_{\theta}^{*}({\bm{x}},{\bm{c}})=\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}.$ (14) $$

###### Proof.

First, we construct two binary (Bernoulli) distributions:

$$ $Q_{{\bm{x}},{\bm{c}}}:=\{\frac{p({\bm{x}},{\bm{c}})}{p({\bm{x}},{\bm{c}})+p({ \bm{x}})p({\bm{c}})},\frac{p({\bm{x}})p({\bm{c}})}{p({\bm{x}},{\bm{c}})+p({\bm {x}})p({\bm{c}})}\}=\{\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}}|{\bm{c}})+p({\bm{ x}})},\frac{p({\bm{x}})}{p({\bm{x}}|{\bm{c}})+p({\bm{x}})}\}$ $$

$$ $P_{{\bm{x}},{\bm{c}}}^{\theta}:=\{\frac{e^{r_{\theta}({\bm{x}},{\bm{c}})}}{e^{ r_{\theta}({\bm{x}},{\bm{c}})}+1},\frac{1}{e^{r_{\theta}({\bm{x}},{\bm{c}})}+1 }\}=\{\sigma(r_{\theta}({\bm{x}},{\bm{c}})),1-\sigma(r_{\theta}({\bm{x}},{\bm{ c}}))\}$ $$

Then we rewrite $\mathcal{L}^{\text{NCE}}_{\theta}({\bm{x}},{\bm{c}})$ as

$$ $\displaystyle\mathcal{L}^{\text{NCE}}_{\theta}({\bm{x}},{\bm{c}})$ $\displaystyle=-\mathbb{E}_{p({\bm{x}},{\bm{c}})}\log\sigma(r_{\theta}({\bm{x}} ,{\bm{c}}))-\mathbb{E}_{p({\bm{x}})p({\bm{c}})}\log\sigma(-r_{\theta}({\bm{x}} ,{\bm{c}}))$ $\displaystyle=-\int\Big{[}p({\bm{x}},{\bm{c}})\log\sigma(r_{\theta}({\bm{x}},{ \bm{c}}))+p({\bm{x}})p({\bm{c}})\log\sigma(-r_{\theta}({\bm{x}},{\bm{c}}))\Big {]}\mathrm{d}{\bm{x}}\mathrm{d}{\bm{c}}$ $\displaystyle=-\int\Big{[}(p({\bm{x}},{\bm{c}})+p({\bm{x}})p({\bm{c}}))\Big{]}$ $\displaystyle\Big{[}\frac{p({\bm{x}},{\bm{c}})}{p({\bm{x}},{\bm{c}})+p({\bm{x} })p({\bm{c}})}\log\sigma(r_{\theta}({\bm{x}},{\bm{c}}))+\frac{p({\bm{x}})p({ \bm{c}})}{p({\bm{x}},{\bm{c}})+p({\bm{x}})p({\bm{c}})}\log\big{[}1-\sigma(r_{ \theta}({\bm{x}},{\bm{c}}))\big{]}\Big{]}\mathrm{d}{\bm{x}}\mathrm{d}{\bm{c}}$ $\displaystyle=\int\Big{[}(p({\bm{x}},{\bm{c}})+p({\bm{x}})p({\bm{c}}))\Big{]}H (Q_{{\bm{x}},{\bm{c}}},P_{{\bm{x}},{\bm{c}}}^{\theta})\mathrm{d}{\bm{x}} \mathrm{d}{\bm{c}}$ $\displaystyle=\int\Big{[}(p({\bm{x}},{\bm{c}})+p({\bm{x}})p({\bm{c}}))\Big{]} \Big{[}D_{\mathrm{KL}}(Q_{{\bm{x}},{\bm{c}}}\|P_{{\bm{x}},{\bm{c}}}^{\theta})+ H(Q_{{\bm{x}},{\bm{c}}})\Big{]}\mathrm{d}{\bm{x}}\mathrm{d}{\bm{c}}$ $$

Here $H(Q_{{\bm{x}},{\bm{c}}},P_{{\bm{x}},{\bm{c}}}^{\theta})$ represents the cross-entropy between distributions $Q_{{\bm{x}},{\bm{c}}}$ and $P_{{\bm{x}},{\bm{c}}}^{\theta}$. $H(Q_{{\bm{x}},{\bm{c}}})$ is the entropy of $Q_{{\bm{x}},{\bm{c}}}$, which can be regarded as a constant number with respect to parameter $\theta$. Due to the theoretical properties of KL-divergence, we have

$$ $\displaystyle\mathcal{L}^{\text{NCE}}_{\theta}({\bm{x}},{\bm{c}})$ $\displaystyle=\int\Big{[}(p({\bm{x}},{\bm{c}})+p({\bm{x}})p({\bm{c}}))\Big{]} \Big{[}D_{\mathrm{KL}}(Q_{{\bm{x}},{\bm{c}}}\|P_{{\bm{x}},{\bm{c}}}^{\theta})+ H(Q_{{\bm{x}},{\bm{c}}})\Big{]}\mathrm{d}{\bm{x}}\mathrm{d}{\bm{c}}$ $\displaystyle\geq\int\Big{[}(p({\bm{x}},{\bm{c}})+p({\bm{x}})p({\bm{c}}))\Big{ ]}H(Q_{{\bm{x}},{\bm{c}}})\mathrm{d}{\bm{x}}\mathrm{d}{\bm{c}}$ $$

constantly hold. The equality holds if and only if $Q_{{\bm{x}},{\bm{c}}}=P_{{\bm{x}},{\bm{c}}}^{\theta}$, such that

$$ $\sigma(r_{\theta}({\bm{x}},{\bm{c}}))=\frac{e^{r_{\theta}({\bm{x}},{\bm{c}})}} {e^{r_{\theta}({\bm{x}},{\bm{c}})}+1}=\frac{p({\bm{x}},{\bm{c}})}{p({\bm{x}},{ \bm{c}})+p({\bm{x}})p({\bm{c}})}$ $$

$$ $r_{\theta}({\bm{x}},{\bm{c}})=\log\frac{p({\bm{x}},{\bm{c}})}{p({\bm{x}})p({ \bm{c}})}=\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}$ $$

∎

###### Theorem A.1 (Noise Contrastive Estimation ) .

###### Proof.

## Appendix B Theoretical Analyses of the Normalizing Constant

We omit a normalizing constant in Eq. 7 for brevity when deriving CCA. Strictly speaking, the target sampling distribution should be:

$$ $p^{\text{sample}}({\bm{x}}|{\bm{c}})={\color[rgb]{0,0,1}\frac{1}{Z({\bm{c}})}} p({\bm{x}}|{\bm{c}})[\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}]^{s},$ $$

such that

$$ $\frac{1}{s}\log\frac{p^{\text{sample}}({\bm{x}}|{\bm{c}})}{p({\bm{x}}|{\bm{c}} )}=\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}{\color[rgb]{0,0,1}-\frac{1}{s} \log Z({\bm{c}})}.$ $$

The normalizing constant $Z({\bm{c}})$ ensures that $p^{\text{sample}}({\bm{x}}|{\bm{c}})$ is properly normalized, i.e., $\int p^{\text{sample}}({\bm{x}}|{\bm{c}})\mathrm{d}{\bm{x}}=1$. We have $Z({\bm{c}})=\int p({\bm{x}}|{\bm{c}})[\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}
]^{s}\mathrm{d}{\bm{x}}=\mathbb{E}_{p({\bm{x}}|{\bm{c}})}[\frac{p({\bm{x}}|{
\bm{c}})}{p({\bm{x}})}]^{s}$.

To mitigate the additional effects introduced by $Z({\bm{c}})$, in our practical algorithm, we introduce a new training parameter $\lambda$ to bias the optimal solution for Noise Contrastive Estimation. Below, we present a result that is stronger than Theorem 3.1.

###### Theorem B.1 .

Let $\lambda_{\bm{c}}>0$ be a scalar function conditioned only on ${\bm{c}}$. Consider the loss function:

$$ $\mathcal{L}^{\text{NCE}}_{\theta}({\bm{x}},{\bm{c}})=-\mathbb{E}_{p({\bm{x}},{ \bm{c}})}\log\sigma(r_{\theta}({\bm{x}},{\bm{c}}))-{\color[rgb]{0,0,1}\lambda_ {\bm{c}}}\mathbb{E}_{p({\bm{x}})p({\bm{c}})}\log\sigma(-r_{\theta}({\bm{x}},{ \bm{c}})).$ (15) $$

Given unlimited model expressivity for $r_{\theta}$, the optimal solution for minimizing $\mathcal{L}^{\text{NCE}}_{\theta}$ satisfies

$$ $r_{\theta}^{*}({\bm{x}},{\bm{c}})=\log\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})} {\color[rgb]{0,0,1}-\log\lambda_{\bm{c}}}.$ (16) $$

###### Proof.

We omit the full proof here, as it requires only a redefinition of the distributions $Q_{{\bm{x}},{\bm{c}}}$ from the proof of Theorem A.1:

$$ $Q_{{\bm{x}},{\bm{c}}}:=\{\frac{p({\bm{x}},{\bm{c}})}{p({\bm{x}},{\bm{c}})+ \lambda_{\bm{c}}p({\bm{x}})p({\bm{c}})},\frac{\lambda_{\bm{c}}p({\bm{x}})p({ \bm{c}})}{p({\bm{x}},{\bm{c}})+\lambda_{\bm{c}}p({\bm{x}})p({\bm{c}})}\}=\{ \frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}}|{\bm{c}})+\lambda_{\bm{c}}p({\bm{x}})}, \frac{\lambda_{\bm{c}}p({\bm{x}})}{p({\bm{x}}|{\bm{c}})+\lambda_{\bm{c}}p({\bm {x}})}\}$ $$

Then we can follow the steps in the proof of Theorem A.1 to arrive at the result. ∎

If let $\lambda_{\bm{c}}:=Z({\bm{c}})^{\frac{1}{s}}=\big{[}\mathbb{E}_{p({\bm{x}}|{\bm
{c}})}[\frac{p({\bm{x}}|{\bm{c}})}{p({\bm{x}})}]^{s}\big{]}^{\frac{1}{s}}$, we could guarantee the convergence of $p^{\text{sample}}_{\theta}$ to $p^{\text{sample}}$. However, in practice estimating $Z({\bm{c}})$ could be intricately difficult, so we formalize $\lambda_{\bm{c}}$ as a training parameter, resulting in our practical algorithm in Eq. 12.

###### Theorem B.1 .

###### Proof.

## Appendix C Additional Experimental Results

We provide more image samples to compare CCA and CFG in Figure 7 and Figure 8.

We illustrate the effect of training parameter $\beta$ on the FID-IS trade-off in Figure 9. Overall, $\beta$ affects the fidelity-diversity trade-off similar to CCA $\lambda$ and the CFG method.

Figure: Figure 9: Effect of varying $\beta$ of CCA for the LlamaGen-L model. In our CCA experiments, we either fix $\lambda=1e3$ and ablate $\beta\in[2,5e-3]$ (from left to right) or fix $\beta=0.02$ and ablate $\lambda\in[0,1e4]$. In our CFG experiments, we ablate $s\in[0,3]$.
Refer to caption: x9.png

## Appendix D Training Hyperparameters

Table 4 reports hyperparameters for chosen models in Figure 1 and Figure 6. Other unmentioned design choices and hyperparameters are consistent with the default setting for LlamaGen [https://github.com/FoundationVision/LlamaGen](https://github.com/FoundationVision/LlamaGen) and VAR [https://github.com/FoundationVision/VAR](https://github.com/FoundationVision/VAR) repo. All models are fine-tuned for 1 epoch on the ImageNet dataset. We use a mix of NVIDIA-H100, NVIDIA A100, and NVIDIA A40 GPU cards for training.

**Table 4: Hyperparameter table.**
| Type | LlamaGen | VAR |  |  |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
| Model | B | L | XL | XXL | 3B | d16 | d20 | d24 | d30 |
| Size | 111M | 343M | 775M | 1.4B | 3.1B | 310M | 600M | 1.0B | 2.0B |
| CCA $\beta$ | 0.02 | 0.02 | 0.02 | 0.02 | 0.02 | 0.02 | 0.02 | 0.02 | 0.02 |
| CCA $\lambda$ | 1000 | 300 | 1000 | 1000 | 500 | 50 | 50 | 100 | 1000 |
| CCA+CFG $\beta$ | 0.1 | 0.02 | 0.1 | 0.1 | 0.1 | - | - | - | - |
| CCA+CFG $\lambda$ | 1 | 1 | 1 | 1 | 1 | - | - | - | - |
| Learning rate | 1e-5 | 1e-5 | 1e-5 | 1e-5 | 1e-5 | 2e-5 | 2e-5 | 2e-5 | 2e-5 |
| Dropout? | Yes | Yes | Yes | Yes | Yes | None | Yes | Yes | Yes |
| Batch size | 256 | 256 | 256 | 256 | 256 | 256 | 256 | 256 | 256 |

## Peer Reviews

confidence: 3
contribution: 3
initial_rating: 6
presentation: 3
questions: Could the authors clarify the necessity of removing CFG, except sampling cost?

Should the model be fine-tuned individually for each lambda and beta?
soundness: 3
strengths: * The problem of sampling schemes for autoregressive models is fundamental and should be explored.

* Detailed illustrations are provided. For example, the authors review preliminaries, give formal derivation, and also the practical implementation.

* It is interesting to see that CCA provides extra gain based on CFG.
summary: To bridge the gap between autoregressive language and visual generation, this work targets at removing the need of CFG in AR image generation, therefore unifying the sampling scheme and reducing the sampling cost. The authors choose to fine-tuning the pretrained AR models with the contrastive alignment loss. The intuition behind the contrastive alignment is to maximize the likelihood of positive conditions and minimize the likelihood of the negative conditions. More discussion with classifier guidance and classifier-free guidance is also illustrated, providing more insights for difference sampling schemes. Experiments are conducted on autoregressive visual generation methods like LlamaGen and VAR.
weaknesses: * For the motivation, one confusion to me is why removing the CFG is necessary for unifying visual and language generation. I mean, there may no influence for sampling them with slightly different schemes? I recognize that introducing CCA can reduce the half cost. But it is achieved at the cost of sampling quality as demonstrated in Table 2.

* To my understanding, the model should be fine-tuned for each lambda and beta individually. Is that correct? If so, this will be a big limitation since other sampling schemes can adjust trade-offs during inference, which is more flexible.

---

confidence: 3
contribution: 3
initial_rating: 6
presentation: 2
questions: 1. Your pipeline requires only 1% of the pre-trained epochs. However, could you provide insights into the corresponding GPU memory usage during training in comparison to CFG?
2. Regarding the proposed loss function in Equation 11, what is its convergence speed?
3. A more detailed inquiry: in line 372, the experiments do not utilize a common default value of s=7.5, for Classifier-Free Guidance (CFG). Could you elaborate on the reason behind this choice?
soundness: 3
strengths: 1. The theoretical derivation is clear and logically structured, allowing for a comprehensive understanding of the underlying concepts.
2. The authors have conducted a thorough series of experiments that effectively validate the theoretical claims made in the paper.
summary: This paper proposes CCA to replace CFG in auto-regressive models, which saves sampling costs while keeps the sampling performance on part with CFG. And this paper gives the theoretical connection
weaknesses: 1. concerns mentioned in questions.
2. lack of explanation for notations when first mentioned, such as $p_{\theta}$  in line 168, $\sigma$ in line 189

---

confidence: 4
contribution: 3
initial_rating: 5
presentation: 3
questions: Please refer to weaknesses
soundness: 4
strengths: - The paper tackles an important but often overlooked problem in autoregressive image generation.

- The methodology section of the paper is mostly well written except that it lacks key information in certain areas. Overall, it provides strong motivations and reasonable derivations based on prior work that lead to the proposed CCA method. 

- The experimental results using strong AR visual generation models (LLamaGen, VAT) as baselines on ImageNet are very promising. It is impressive that without any additional inference costs (and a bit more training/finetuning costs), CCA can bring significant improvements to the FID and IS metrics. When paired with CFG, improved FIDs can be achieved.
summary: This paper introduces CCA which a new alternative to CFG to finetune autoregressive image generation models so that they work like the sampling model used by CFG during inference. The key idea lies in modeling the conditional residual by learning to maximize the likelihood of an image given positive conditions, while minimizing the likelihood given negative conditions. In my opinion, it can be intuitively seen as a way to absorbing the benefits of the unconditional model (which complements the conditional model as demonstrated by CFG) right into single conditional model inference. The resulting approach is more elegant than CFG and is more efficient than CFG because it only requires 1x sampling cost. Furthermore, it can be paired with CFG for additional inference quality boost.
weaknesses: - Some important details are missing in the paper, such as how positive and negative data samples are chosen for CCA training. Sec 3.2 only briefly mentions that and Fig. 2 shows something about it without any verbal explanation. It is unclear if we just use the ImageNet labels as the sole criterion for constructing positives and negatives or there’s some other strategy being used. In Sec 5.4, it’s mentioned that certain unconditional data samples are used as positives but it is not clear how exactly that works. Without this key information, it may be hard for readers to understand how and why CCA works.

- Fig 2‘s caption is totally uninformative. There should be a caption describing the method in detail.

- CCA uses models pretrained with CFG training (i.e., condition dropout). For CCA to work, is there a strong requirement on this? What is the relationship of CFG pretraining and CCA finetuning?

- When CCA is combined with CFG, IS becomes worse while FID becomes better. Can the authors explain this phenomenon? 

- The proposed method resembles CLIP-like contrastive loss. I wonder if it can only work if the training batch size is large enough.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: To make the proposed method better, the authors need to provide practical solutions to some issues like training cost with large-scale datasets.
soundness: 3
strengths: + This paper proposes condition contrastive alignment (CCA) for a guidance-free AR visual generation. Concretely, the authors transform CFG into a contrastive learning formulation and train the sampling model only with a fine-tuning loss. Thus, CCA is simple and easy to implement.
+ The proposed method can unifies the alignment mechanism between language and visual models.  Besides, unlike CFG, generation models with CCA can reduce the sampling cost by half, which is a practical technique.
+ The authors conduct extensive experiments to validate the proposed method. Compared with CFG, CCA achieves comparable performance in FID score under different visual models.
summary: The authors propose condition contrastive alignment (CCA), replacing classifier-free guidance (CFG). Based on CCA, they implement a guidance-free AR visual generation and unifies the alignment mechanism between language and visual models. Compared with models using CFG, generation models with CCA can reduce the sampling cost by half. To implement CCA, the authors transform CFG into a contrastive learning formulation and train the sampling model only with a fine-tuning loss. In addition, they conduct extensive experiments to validate the effectiveness of the proposed method.
weaknesses: - Training cost. CCA needs one training epoch to get ideal performance. The training cost is negligible for ImageNet datasets. But for existing large visual generation models (like SD3), the training data is huge. The cost of training one epoch with large-scale datasets is expensive. Is there a practical solution to reduce the cost?
- Optimal $\beta$ and $\lambda$. To obtain the optimal values for $\beta$ and $\lambda$, we need to train CCA many times. The cost of this process is large. Is there a potential guideline to mitigate the substantial cost of finding optimal $\beta$ and $\lambda$?
- Negative samples in CCA. To my knowledge, negative samples have a large effect on the performance of contrastive learning. For CCA, it is based on contrastive learning framework. The selection of negative samples affects the performance of CCA largely. Is there a selection mechanisms of  negative samples?
- Implementation about integration of CCA and CFG. I do not understand the implementation completely due to lacks of details. It is necessary to clarify the implementation with more details.
- IS score. As shown in Figure 1 and Figure 6, IS score starts to degrade when model size is beyond a certain size, such as LlamaGen>775M. To my knowledge, this is an unusual phenomenon. Larger model means better IS score.

---

confidence: 4
contribution: 4
initial_rating: 8
presentation: 4
questions: - It is not clearly descirbed why the performance with varying lambda leads to a similar behaviour to that with varying beta. For example, why has IS score been improved by increasing the value of lambda(namely, overweighing the loss on negative pairs)?
soundness: 3
strengths: - The motivation of this study is clear and should be important to make auto-regressive models efficiently generate high-quality visual contents. As of 2024, CFG is commonly used for auto-regressive models to achieve high-quality image generation, while it doubles the computational cost of the generation process. How to get rid of the necessity of CFG would be an important problem especially for practical applications.

- The derivation of the proposed method, particularly Eq. (10) and (11), is elegant. The theoretical connection with guidance-based sampling is clear, and the practical implementation is reasonable.

- The experimental results validate the advantage of the proposed method; it greatly boosts the performance of guidance-free sampling. Although its performance is slightly worse than that of sampling with CFG, it would be acceptable in some practical applications considering that the computational cost for the generation is halved.

- The manuscript is well-written and easy to follow.
summary: The authors propose a novel method for fine-tuning auto-regressive models to enable guidance-free sampling from the same distribution that would be sampled if classifier-free guidance were used. The training objective is derived by noise contrastive estimation in which genuine pairs of condition and data are used as positive while random pairs are negative ones. The experimental results show that just one epoch fine-tuning with the proposed method greatly improves the performance of the guidance-free sampling.
weaknesses: - I do not find any crucial weakness or major concerns on this paper. The followings are just minor points.
  - As the proposed method requires to fix a target distribution represented by beta, it cannot control the fidelity-diversity trade-off in the inference phase, which might reduce some flexibility compared with CFG.
    - It might be worth exploring whether the model could be extended to accept additional condition beta to work with various beta in the inference phase, similar to the approach in the distillation of guided diffusion models [R1]. However, it should be non-trivial how to achieve such extention in AR models.

      [R1] "On Distillation of Guided Diffusion Models," CVPR 2023.
  - It would be beneficial if the author could discuss how stable the proposed method is. As the negative pairs in Eq. (12) are random pairs of x and c, p(x|c) appeared in the loss computation could have very small values, which might cause numerical instability during the fine-tuning.

---

confidence: 3
contribution: 3
initial_rating: 8
presentation: 3
questions: Please check the weakness.
soundness: 3
strengths: - The problem the submission addresses has very high practical value to deploying the visual autoregressive model.
- The algorithm derivation gives a clear explanation about the methods. And the derived training loss function is very concise, which is easy to train.
- The performance gain is huge, on both generation quality and diversity.
- Experimental section is thorough. Additionally, sec 5.3 is informative and appreciated, especially unlearning.
summary: This submission proposes an alternative way to achieve classifier-free guidance (CFG) for visual autoregressive model, by directly finetuning the model, which avoids an extra forward pass when deploying CFG. The experiment results show the proposed finetuning method introduces huge performance gain.
weaknesses: In short, this is a good work. I only have one question about the training loss function, as $\beta$ is expected to be responsible for CFG degree. But the experimental results show that $\lambda$ actually dominates the CFG controllability, any clue about it?

One open question is, will the proposed method also improve the directional autoregressive model, a.k.a., Masked Generative Models such as MaskGIT and MAGE. Or, to think even more boldly, will the method still applicable to (continuous) diffusion models, since MaskGIT could be regarded as a discrete diffusion model.

Meta-review: CFG improves visual generative models but creates inconsistencies in autoregressive (AR) multi-modal generation. To address this, this paper proposes Condition Contrastive Alignment (CCA), which directly fine-tunes pre-trained models to achieve guidance-free AR visual generation. CCA significantly enhances model performance with minimal fine-tuning, reducing sampling costs by half while maintaining a balance between sample diversity and fidelity. 

All reviewers unanimously agree to accept it.

Though limitations such as models need to be fine-tuned with each parameter exist, this paper could still be widely interesting to the whole visual generation community. Given the current promising results presented in this paper, I recommend to accept it.

Decision: Accept (Oral)