StyleAR: Customizing Multimodal Autoregressive Model for Style-Aligned Text-to-Image Generation

## Abstract

Abstract In the current research landscape, multimodal autoregressive (AR) models have shown exceptional capabilities across various domains, including visual understanding and generation. However, complex tasks such as style-aligned text-to-image generation present significant challenges, particularly in data acquisition.
In analogy to instruction-following tuning for image editing of AR models, style-aligned generation requires a reference style image and prompt, resulting in a text-image-to-image triplet where the output shares the style and semantics of the input. However, acquiring large volumes of such triplet data with specific styles is considerably more challenging than obtaining conventional text-to-image data used for training generative models.
To address this issue, we propose StyleAR, an innovative approach that combines a specially designed data curation method with our proposed AR models to effectively utilize text-to-image binary data for style-aligned text-to-image generation. Our method synthesizes target stylized data using a reference style image and prompt, but only incorporates the target stylized image as the image modality to create high-quality binary data. To facilitate binary data training, we introduce a CLIP image encoder with a perceiver resampler that translates the image input into style tokens aligned with multimodal tokens in AR models and implement a style-enhanced token technique to prevent content leakage which is a common issue in previous work.
Furthermore, we mix raw images drawn from large-scale text-image datasets with stylized images to enhance StyleAR’s ability to extract richer stylistic features and ensure style consistency. Extensive qualitative and quantitative experiments demonstrate our superior performance.

## 1 Introduction

Multimodal autoregressive (AR) models [34; 17; 4; 40; 3; 32] have revolutionized cross-modal content synthesis by using the next-token prediction mechanism, demonstrating superior performance in tasks such as image understanding and text-to-image generation. In particular, in text-to-image generation, AR models outperform traditional diffusion models [26; 23; 5; 16; 45; 46; 54; 43] in terms of prompt adherence and image quality. However, when tackling style-aligned text-to-image generation, a task requiring the style and semantic of generated images precise alignment with reference style images and input prompts, respectively, AR models exhibit significant performance gaps compared to diffusion-based approaches [10; 38; 51; 14; 30; 12; 18; 41; 27; 9]. This limitation stems from that the fact that AR models require text-image-to-image triplet data (that is, high-quality annotated data containing reference style images, prompts, and corresponding stylized generated images) in an instruction-following tuning manner to perform style-aligned text-to-image generation, while existing public datasets [28] predominantly provide text-image binary data.

Existing approaches for style-aligned text-to-image generation which are predominantly diffusion-based and can be broadly categorized into two paradigms. 1) Optimization-free direct inference methods [51; 38; 12], exemplified by IP-Adapter [51] which trains decoupled cross-attention layers to integrate style features with text conditions, allowing zero-shot style transfer. 2) Methods requiring parameter tuning or latent inversion [14; 10; 27], such as Dreambooth [27] which performs low-rank [11] adaptation (LoRA) for each new reference style, and StyleAligned [10] which extracts the image feature of reference images via DDIM inversion [20], then enforces style alignment through shared attention layers. In contrast, AR models remain underexplored for style-aligned text-to-image generation. Compared to diffusion models [23; 26] that flexibly blend multimodal features in latent space, AR models [17; 34; 32] face dual challenges: their heavy dependence on text-image-to-image triplet data and the inefficiency of extraction of style feature via image tokenizer, which collectively hinder their competitiveness in this domain.
To overcome these limitations, we present StyleAR, the first study to enable AR models to perform style-aligned text-to-image generation. We primary focus on two innovative aspects, i.e., data curation and designs of AR models.

On the data aspect, while we can use diffusion-based methods for creating triplet data for instruction-following tuning, they suffer from quality issues due to two main reasons. First, the generated stylized images exhibit low style consistency with the input reference style image, which limiting dataset quality. Second, the generated triplet data serve as ground truth for instruction-following tuning, making the capability frontier of diffusion models an upper bound for AR models, thus challenging the purpose of tailoring AR models for style-aligned text-to-image generation. To address these issues, we propose to drop the reference style image in the data generation process and use only the prompt and the generated stylized image for binary data construction, illustrated as Fig. [2](https://arxiv.org/html/2505.19874v1#S1.F2)(a). Moreover, we discover that mixing raw images drawn from large-scale text-image dataset [28] with stylized images at an optimal 3:1 ratio during training stage can significantly improve our StyleAR’s ability to extract richer stylistic features, illustrated as Fig. [2](https://arxiv.org/html/2505.19874v1#S1.F2)(b)(c). Using these curated stylized image data along with raw image data, we can create a high-quality dataset for AR model tuning.

The remaining part focuses on customizing AR models for style-aligned image generation using binary data. We introduce two key innovations to enhance the AR model to learn from binary data and improve inference performance. 1) We integrate the image encoder [24] with a perceiver [13; 1] resampler module to convert the input image to a unified token space that integrates effectively with multimodal tokens. 2) We introduce a style-enhancing mechanism for tokens using SAM [15] and Gaussian noise injection to remove irrelevant semantic features from the reference style image.

The key contributions of this work can be summarized as follows:

- •
We investigate a novel data curation method for crafting binary stylized text-image data from a diffusion counterpart model for training AR models, achieving high quality and avoiding upper bound limit of the base data generation model.
- •
We propose a training framework for AR models to use binary text-to-image data to perform stylized-aligned text-to-image generation, preventing the difficulty of scaling up text-image-to-image triplet data for instruction-following tuning.
- •
We also propose the style-enhanced tokens technique, which can effectively solve the problem of content leakage in the stylized-aligned text-to-image generation task and significantly improve prompt adherence and style consistency.
- •
Extensive quantitative evaluation, qualitative experiments, and user study demonstrate that our StyleAR achieves state-of-the-art performance in both prompt adherence and style consistency, surpassing existing diffusion-based approaches. Moreover, StyleAR integrates additional conditions effectively, such as depth map and other structural control.

Figure: Figure 2: The pipeline of our method. a) We first investigate a novel stylized image data curation to form binary data with high prompt adherence and prevent low style consistency. b) We use a mixed dataset to enhance rich stylistic features learning. c) With the designed data curation and model framework, our method achieve high prompt adherence and style consistency.
Refer to caption: x2.png

## 2 Related Work

### 2.1 Multimodal Understanding and Generation Unified Models

In recent times, unified models [34; 17; 4; 40; 3; 54; 33; 7; 46; 45; 44; 31] for multimodal generation and understanding have achieved significant development. These works fall primarily into two categories: autoregressive-based models represented by Chameleon [34], and diffusion-based models exemplified by Transfusion [54]. In the first category, Chameleon [34] pioneered an early fusion approach in which all multimodal inputs are projected onto a shared representational token space from the start, training a transformer to process sequences of multimodal tokens. Building on this foundation, Lumina-mGPT [17] enhances training datasets and achieves flexible image generation of varying aspect ratios, and Emu3 [40] adopts a two-stage training framework and introduces the DPO [25] (Direct Preference Optimization) algorithm to improve visual quality and prompt alignment. Furthermore, Janus-Pro [3] decouples visual encoding for image understanding and image generation tasks, employing different image encoders for each objective. HART [33] and FLUID [7] explore integrating continuous tokens into AR models to improve the quality of image generation. In the second category, Transfusion [54] innovatively combines autoregressive and diffusion training paradigms within a single transformer framework: images are trained via diffusion process, while text follows next-token prediction pattern. Furthermore, Show-o [46] and OmniGen [45] expand this approach by scaling the training datasets. In particular, OmniGen [45] incorporates diverse computer vision tasks (e.g., inpainting, deblurring) and subject-driven image generation task into its training data, enabling strong performance across multiple common benchmarks.

### 2.2 Style-Aligned Text-to-Image Generation

Existing approaches for style-aligned text-to-image generation are predominantly diffusion-based and can be broadly categorized into two paradigms. 1) Optimization-free methods [51; 38; 12], exemplified by IP-Adapter [51] which trains decoupled cross-attention layers to dynamically integrate style features from pretrained image encoders [24] with text conditions, allowing zero-shot style transfer. Similarly, InstantStyle [38] identifies content-specific and style-specific layers in Stable Diffusion XL [23] model, proposing a layer-wise decoupling mechanism to suppress content leakage from the reference style images. 2) Methods [14; 10; 27] require parameter tuning or latent inversion [20], such as StyleAligned [10] which extracts image features via DDIM inversion [20] and enforces stylization through attention layers. In contrast, AR models remain underexplored for style-aligned text-to-image generation. Compared to diffusion models [23; 26] that flexibly blend multimodal features in latent space, AR models [17; 34; 32] face dual challenges: heavy dependence on instruction-following data and inefficiency of style feature extraction in next-token prediction frameworks via image tokenizer, which hinder their competitiveness.

## 3 Method

Figure: Figure 3: The framework of our StyleAR. During training, we utilize a frozen CLIP [24] image encoder along with a trainable perceiver [13; 1] resampler module to efficiently extracted features. Subsequently, style tokens are combined with the injected Gaussian noise and concatenated with multimodal tokens by replacing the placeholder tokens. During inference, we incorporate SAM [15] to remove irrelevant semantic contents in the reference style image.
Refer to caption: x3.png

### 3.1 Preliminaries

During the image generation training process of the AR model, for an input image $x\in\mathbb{R}^{H\times W\times 3}$, it is first quantized into $q\in\mathbb{Q}^{h\times w}$ discrete tokens by an image tokenizer [35; 6; 52], where $h=H/p_{h}$, $w=W/p_{w}$, $p_{h}$ and $p_{w}$ are the downsample ratios of the image tokenizer in the vertical and horizontal directions respectively, and $q^{(i,j)}$ is the indices of the image codebook. Then the image tokens are flattened into a sequence of length $h\times w$ and is concatenated with the text tokens passed into a Transformer-based [36] autoregressive model for training. During the inference stage, given the text tokens $t$, the autoregressive model can generate image tokens through next-token prediction:

$$ $\Pi^{h\cdot w}_{t=1}p(q_{t}|q_{<t},t).$ (1) $$

Finally, the generated image tokens are converted to pixel space via a image decoder.

### 3.2 Data Curation

A primary contribution of our method focuses on the data curation part. If we aim at creating triplet data for instruction-following tuning, while we can use InstantStyle [38] to create such data, they suffer from low style consistency and make the capability frontier of diffusion models an upper bound for AR models. In contrast, we drop the reference style image in the data generation process and use only the prompt and the generated stylized image for binary data construction. In this way, we acquire high-quality stylized binary data and prevent low style consistency. Moreover, through practical experiments, we have found that if we rely solely on this stylized dataset for model training, the model’s ability to capture image features during the inference stage is unsatisfactory (as shown in Section [4.3](https://arxiv.org/html/2505.19874v1#S4.SS3)), which leads to the style consistency staying low between the generated images and the reference style images. Moreover, considering model nature during pre-training phase of the text-to-image generation task, the training sets it uses mostly consist of raw images that are not stylized. If we only use the stylized dataset for training, the domain gap brings difficulties to the model training process. In view of this, when training our StyleAR, we simultaneously use the raw image dataset and the stylized image dataset in a certain proportion to serve as training dataset.

### 3.3 Framework of StyleAR

Training with Binary Data.
The framework of our StyleAR are shown in Figure [3](https://arxiv.org/html/2505.19874v1#S3.F3).
To enable binary data training, we design the model to use the input image in a self-supervised manner, extracting style features and learning to predict the image tokens of the same image. In particular, the input image $I$ is first processed through CLIP [24] image encoder $E_{I}$ to extract image features. The image features are converted to style tokens $s\in\mathbb{R}^{M\times C}$, which fit the unified token space of the AR model by a perceiver [13; 1] resampler module $R$, where $M=16$ is the number of style tokens and $C=4,096$ is the dimension of the unified token space of the AR model. Additionally, to alleviate the content leakage issue [14], we inject Gaussian noise $n$ to style tokens to weaken irrelevant semantic features and enforce the AR model to pay attention to the semantic information from prompts during image generation. The process of image tokens generation is formulated as $\Pi^{h\cdot w}_{t=1}p(q_{t}|q_{<t},t,\hat{s})$, where $\hat{s}=s+\gamma\cdot n$, and $\gamma$ is the strength of Gaussian noise injection.

Style-Enhanced Inference.
We incorporate the SAM (Segment Anything Model) [15], represented as $S_{I}$, combining with the Gaussian noise injection mechanism to form the style-enhanced tokens technique to further reduce the risk of content leakage and facilitate accurate and reliable inference. Concretely, the input image $I$ and its segmented image $I_{S}$ are passed through the CLIP [24] image encoder to obtain the corresponding image features $F$ and $F_{S}$. Via feature subtraction $F-F_{S}$ to exclude semantic information, the result is mapped to the unified token space through the perceiver resampler module. Moreover, to preserve more fine-grained style features, we also introduced a residual path in the unified token space. The style-enhanced tokens $\hat{s}_{e}$ can be formulated as follows:

$$ $\hat{s}_{e}=\alpha\cdot R(F)+(1-\alpha)\cdot R(F-F_{S})+\gamma\cdot n,$ (2) $$

where $\alpha$ is the residual ratio of the residual path. This inference mechanism significantly improves the stylized image quality, achieving high prompt adherence and high style consistency.

Post-Training.
Recent numerous research [22; 25; 29] have demonstrated the potential of post-training, mainly via reinforcement learning, to enhance the reasoning capabilities of large language models (LLMs) and human preference alignment. In the field of image generation, whether in diffusion models [2; 8; 19] or AR models [40; 39], post-training is frequently used to improve prompt alignment and visual quality of generated images [47; 48; 37; 8]. In this work, we utilize Direct Preference Optimization [25] (DPO) algorithm to boost the prompt alignment in the style-aligned text-to-image generation. Specifically, we implement a standard DPO strategy via ranking data creation. For each prompt, we use our StyleAR to generate two images and use the VLM [49; 50] to select the image that better aligns with the semantics of the corresponding prompt from these two images. Based on the scoring results, we construct a triplet $(p_{i},x_{i}^{chosen},x_{i}^{rejected})$ for DPO training.

## 4 Experiments

### 4.1 Experimental Details

Details of Model and Dataset. Our StyleAR is implemented based on the FP-SFT@768 version of Lumina-mGPT [17]. The raw image training dataset is sourced from the open source dataset [21]. For the stylized image training dataset, we collect 80 distinct artistic style images from the open source dataset [45; 42] and generate 200 semantically diverse images per style using InstantStyle [38], resulting in a total training dataset of 16,000 stylized image data. During each epoch, we randomly sample 10% (49,368 images) data from the raw image dataset and mix with the full stylized image dataset to construct the training dataset. The training configuration employs a batch size of 64 with a learning rate of 2e-5 and the rank of LoRA [11] parameters employed in the AR model is set to 4.

Details about Evaluation Metrics. Following previous work [10; 18; 30; 12], we use the CLIP-T [24] metric to evaluate prompt adherence, which is the cosine similarity between the CLIP text embeddings of the input prompts and the CLIP image embeddings of the corresponding generated images. CLIP-I and DINO [53] metric are used to evaluate style consistency, which is the cosine similarity between the image embeddings of the reference style images and the image embeddings of the corresponding generated images. To robustly measure performance and generalization capabilities of the methods, we collected 10 diverse reference style images and 20 various prompts including human activities, animals, buildings, vehicle, musical instruments, and furniture. For the evaluation suite, we generate four images per style and per prompt, totaling 800 images.

Figure: Figure 4: Qualitative comparison. We conducted a comprehensive qualitative evaluation by comparing our StyleAR with various existing methods which are all diffusion-based, including InstantStyle [38], IP-Adapter [51], StyleAligned [10], StyleCrafter [18], StyleShot [9].
Refer to caption: x4.png

### 4.2 Comparisons

We perform a comprehensive comparison between our proposed StyleAR (AR-based) with existing methods which are diffusion-based, including InstantStyle [38], IP-Adapter [51], StyleAligned [10], StyleCrafter [18], and StyleShot [9]. With the exception of StyleShot [9], which uses the implementation of the Stable Diffusion 1.5 [26] implementation, all other comparative methods are based on the Stable Diffusion XL architecture [23]. All comparative experiments were conducted using the official open source implementations of the baseline methods, with hyperparameter settings strictly adhering to the configurations prescribed in their respective technical documentations.

Qualitative Comparison. The quantitative comparison is demonstrated in Figure [4](https://arxiv.org/html/2505.19874v1#S4.F4). According to the results, InstantStyle [38] shows a superior prompt adherence, achieving a notable semantic alignment of the input prompt and the generated image. However, it shows an inferior style consistency between the generated images and the reference style images. The IP-Adapter [51] often exhibits a failure in prompt adherence, where the generated images deviate from the input prompts. As demonstrated in the first row of Figure [4](https://arxiv.org/html/2505.19874v1#S4.F4), given the prompt “a train”, the model incorrectly generates an aircraft image. This artifact originates from content leakage, a phenomenon in which the semantic content of the reference style image (e.g., aircraft) inappropriately propagates into the image generation process via decoupled cross-attention layers in the IP-Adapter [51], overriding the semantic content of the input prompts. StyleAligned [10] demonstrates unstable generation outcomes and semantic chaos due to the inherent limitations of its shared attention layers in achieving stable decoupling between content and style features. StyleCrafter [18] and StyleShot [9] demonstrate satisfactory prompt adherence, but exhibit notable deficiencies in style consistency. In contrast, our StyleAR demonstrates excellent prompt adherence and accurately captures the overall and detailed features of the reference style.

Quantitative Comparison. The quantitative comparison results are shown in Table [1](https://arxiv.org/html/2505.19874v1#S4.T1). Our StyleAR achieves a superior balance between prompt adherence and style consistency. On the one hand, StyleAR attains the second highest performance in prompt adherence, slightly below InstantStyle [38], while InstantStyle shows poor style consistency. On the other hand, StyleAR ranks second in CLIP-I and DINO metrics, marginally behind IP-Adapter [51]. However, IP-Adapter [51] suffers from poor prompt adherence and severe content leakage (as shown in Figure [4](https://arxiv.org/html/2505.19874v1#S4.F4) of qualitative results), which lead to abnormal increases in CLIP-I and DINO metrics. In contrast, our method effectively extracts the style features of the reference style image and generate the target images without any content leakage.

**Table 1: Quantitative comparison. We conduct a comprehensive qualitative evaluation by comparing our StyleAR with various existing methods, including InstantStyle [38], IP-Adapter [51], StyleAligned [10], StyleCrafter [18], StyleShot [9]. The CLIP-T metric reflects the prompt adherence. The CLIP-I and DINO metrics indicate the style consistency. Notably, the content leakage issue of IP-Adapter leads to a abnormal increase in CLIP-I and DINO metrics.**
|  | StyleAR (Ours) | InstantStyle | StyleAligned | StyleCrafter | IP-Adapter | StyleShot |
| --- | --- | --- | --- | --- | --- | --- |
| CLIP-T ($\uparrow$) | 0.2893 | 0.2944 | 0.2778 | 0.2815 | 0.2655 | 0.2807 |
| CLIP-I ($\uparrow$) | 0.7456 | 0.6899 | 0.7003 | 0.7098 | 0.7971 | 0.6888 |
| DINO ($\uparrow$) | 0.6136 | 0.4902 | 0.6080 | 0.6094 | 0.6617 | 0.5668 |

Figure: Figure 5: User study. We conducted a user study by comparing StyleAR with existing methods, including InstantStyle [38], IP-Adapter [51], StyleAligned [10], StyleCrafter [18], StyleShot [9].
Refer to caption: x5.png

User Study. The results of the user study are shown in Figure [5](https://arxiv.org/html/2505.19874v1#S4.F5). In terms of prompt adherence and image quality, our method performs on par with InstantStyle [38], both significantly outperforming other methods. Moreover, in terms of style consistency, our method far surpasses all others. In contrast, the InstantStyle [38] method exhibits poor style consistency. It can be seen that our method not only strictly adheres to the input prompts to generate high-quality images but also ensures a high degree of style consistency between the generated images and the reference style images.

Additional Results. Freezing the original parameters of the AR model, StyleAR can retain the original generation capabilities of the AR model, such as structural control. Compared with diffusion-based methods like IP-Adapter [51] and InstantStyle [38], our StyleAR method performs better on image quality, condition fidelity, and style consistency, shown in Figure [6](https://arxiv.org/html/2505.19874v1#S4.F6).

Figure: Figure 6: Qualitative comparison of integration with additional conditions. We show the comparison results of control generation with Ours and diffusion models.
Refer to caption: x6.png

Figure: Figure 7: Ablation study of composition of training datasets. We investigate the impact of the composition of training datasets in our StyleAR. The compared training datasets include pure stylized image data, the ratios of stylized image data to raw image data are 1:3, 1:6 and 1:30 respectively.
Refer to caption: x7.png

### 4.3 Ablation Study

In this section, we conduct ablation experiments to examine how the training dataset’s elements and the design modules impact StyleAR’s results.

Composition of training dataset. We have meticulously designed different compositions of the training datasets to conduct ablation experiments, in order to explore the impact of the training datasets on StyleAR. Specifically, the compared training datasets include pure stylized image data, the ratios of stylized image data to raw image data are 1:3, 1:6 and 1:30 respectively. The qualitative results are shown in Figure [7](https://arxiv.org/html/2505.19874v1#S4.F7) and the quantitative results are shown in Table [2](https://arxiv.org/html/2505.19874v1#S4.T2). According to the results, when the training dataset only contains stylized image data, prompt adherence is relatively good, but style consistency is relatively poor. In contrast, when raw images are added, specifically when the ratio of stylized image data to raw image data is 1:3, the style consistency is significantly improved and the prompt adherence also remains at a good level. However, when the ratios are further increased to 1:6 and 1:30, the content leakage occurred and the generated images show “overfitting” to the reference style images. The irrelevant semantic content also appears in the generated images, causing the semantics of the generated images to not follow the semantics of the input prompt. Consequently, we conclude that in stylization tasks, in addition to stylized image data, appropriately adding some raw image data can improve style consistency and will not degrade the prompt adherence.

Figure: Figure 8: Ablation study of style-enhanced (SE) tokens. We investigate the style-enhanced tokens technique in our StyleAR. The ablation study demonstrates the effectiveness.
Refer to caption: x8.png

**Table 2: Quantitative ablation study of composition of training dataset, style-enhanced (SE) tokens and DPO post-training. The compared variants covers the training dataset study and module designs. The ranking pattern is similar as that in Table [1](https://arxiv.org/html/2505.19874v1#S4.T1), indicating the content leakage phenomenon.**
|  | Stylized data | Ratio 1:3 (Ours) | Ratio 1:6 | Ratio 1:30 | Ours w/o SE tokens | Ours w/o DPO |
| --- | --- | --- | --- | --- | --- | --- |
| CLIP-T ($\uparrow$) | 0.2970 | 0.2893 | 0.2688 | 0.2170 | 0.2683 | 0.2874 |
| CLIP-I ($\uparrow$) | 0.7022 | 0.7456 | 0.7475 | 0.7798 | 0.7552 | 0.7151 |
| DINO ($\uparrow$) | 0.5241 | 0.6136 | 0.6657 | 0.7677 | 0.6765 | 0.6193 |

Impact of Style-Enhanced Tokens. To evaluate the effectiveness of our proposed style-enhanced tokens technique, we conduct quantitative and qualitative ablation study to compare our StyleAR and our StyleAR without style-enhanced tokens. The results are shown in Figure [8](https://arxiv.org/html/2505.19874v1#S4.F8) and in the six columns of Table [2](https://arxiv.org/html/2505.19874v1#S4.T2). According to the results, when without style-enhanced tokens, the irrelevant semantic features from reference style images show in the generated images, which leads to the generated images not conforming to the semantic control of the input prompt and the situation of chaotic generation occurs. In contrast, prompt adherence and image quality are both improved when style-enhancing mechanism is employed, which enables the style-enhanced tokens to assist the model in filtering out the irrelevant semantic information of reference style images, ensuring that generated images are highly consistent with the input prompt and significantly improving image quality.

Impact of DPO. To evaluate the effectiveness of DPO post-training in our StyleAR, we conduct quantitative ablation study to compare our StyleAR and our StyleAR without DPO pose-training. The quantitative results are shown in the seventh column of Table [2](https://arxiv.org/html/2505.19874v1#S4.T2). DPO post-training can improve the prompt adherence and slightly enhance the style consistency of StyleAR.

## 5 Conclusion and Limitations

In this work, we present StyleAR, the first work to use image-text binary data to enable the multimodal autoregressive model to perform style-aligned text-to-image generation which is dominated by diffusion-based methods. Compared with using triplet data in instruction-following tuning of previous AR models, our use of image-text binary data training can easily scale up the size of the training dataset, thereby improving the model’s performance. Furthermore, ablation experiments verify the effectiveness of our module designs, including stylized-raw mixed training strategy and style-enhanced tokens technique to enhance style consistency and prompt adherence. However, the current implementation requires depth map extraction for content control rather than direct input of content image to realize style transfer. Future research will focus on leveraging the multimodal input capabilities of autoregressive models to enable simultaneous integration of style reference images and content-specific visual image inputs for further conditional image generation.

## 6 Acknowledgments

This research is supported by the Anhui Provincial Natural Science Foundation (Grant No.2408085QF214), the Fundamental Research Funds for the Central Universities (Grant No.WK2100000045), the Opening Project of the State Key Laboratory of General Artificial Intelligence (Grant No.SKLAGI2024OP10, Grant No.SKLAGI2024OP11), the Research Grants Council of Hong Kong (C5055-24G and T45-401/22-N) and in part by the National Natural Science Foundation of China (No. 62201483).

## References

- [1]
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al.
Flamingo: a visual language model for few-shot learning.
Advances in neural information processing systems, 35:23716–23736, 2022.
- [2]
Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine.
Training diffusion models with reinforcement learning.
arXiv preprint arXiv:2305.13301, 2023.
- [3]
Xiaokang Chen, Zhiyu Wu, Xingchao Liu, Zizheng Pan, Wen Liu, Zhenda Xie, Xingkai Yu, and Chong Ruan.
Janus-pro: Unified multimodal understanding and generation with data and model scaling.
arXiv preprint arXiv:2501.17811, 2025.
- [4]
Ethan Chern, Jiadi Su, Yan Ma, and Pengfei Liu.
Anole: An open, autoregressive, native large multimodal models for interleaved image-text generation.
arXiv preprint arXiv:2407.06135, 2024.
- [5]
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al.
Scaling rectified flow transformers for high-resolution image synthesis.
In Forty-first international conference on machine learning, 2024.
- [6]
Patrick Esser, Robin Rombach, and Bjorn Ommer.
Taming transformers for high-resolution image synthesis.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 12873–12883, 2021.
- [7]
Lijie Fan, Tianhong Li, Siyang Qin, Yuanzhen Li, Chen Sun, Michael Rubinstein, Deqing Sun, Kaiming He, and Yonglong Tian.
Fluid: Scaling autoregressive text-to-image generative models with continuous tokens.
arXiv preprint arXiv:2410.13863, 2024.
- [8]
Ying Fan, Olivia Watkins, Yuqing Du, Hao Liu, Moonkyung Ryu, Craig Boutilier, Pieter Abbeel, Mohammad Ghavamzadeh, Kangwook Lee, and Kimin Lee.
Dpok: Reinforcement learning for fine-tuning text-to-image diffusion models.
Advances in Neural Information Processing Systems, 36:79858–79885, 2023.
- [9]
Junyao Gao, Yanchen Liu, Yanan Sun, Yinhao Tang, Yanhong Zeng, Kai Chen, and Cairong Zhao.
Styleshot: A snapshot on any style.
arXiv preprint arXiv:2407.01414, 2024.
- [10]
Amir Hertz, Andrey Voynov, Shlomi Fruchter, and Daniel Cohen-Or.
Style aligned image generation via shared attention.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 4775–4785, 2024.
- [11]
Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al.
Lora: Low-rank adaptation of large language models.
ICLR, 1(2):3, 2022.
- [12]
Nisha Huang, Kaer Huang, Yifan Pu, Jiangshan Wang, Jie Guo, Yiqiang Yan, and Xiu Li.
Artcrafter: Text-image aligning style transfer via embedding reframing.
arXiv preprint arXiv:2501.02064, 2025.
- [13]
Andrew Jaegle, Felix Gimeno, Andy Brock, Oriol Vinyals, Andrew Zisserman, and Joao Carreira.
Perceiver: General perception with iterative attention.
In International conference on machine learning, pages 4651–4664. PMLR, 2021.
- [14]
Jaeseok Jeong, Junho Kim, Yunjey Choi, Gayoung Lee, and Youngjung Uh.
Visual style prompting with swapping self-attention.
arXiv preprint arXiv:2402.12974, 2024.
- [15]
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al.
Segment anything.
In Proceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023.
- [16]
Black Forest Labs.
Flux.
[https://github.com/black-forest-labs/flux](https://github.com/black-forest-labs/flux), 2024.
- [17]
Dongyang Liu, Shitian Zhao, Le Zhuo, Weifeng Lin, Yu Qiao, Hongsheng Li, and Peng Gao.
Lumina-mgpt: Illuminate flexible photorealistic text-to-image generation with multimodal generative pretraining.
arXiv preprint arXiv:2408.02657, 2024.
- [18]
Gongye Liu, Menghan Xia, Yong Zhang, Haoxin Chen, Jinbo Xing, Yibo Wang, Xintao Wang, Yujiu Yang, and Ying Shan.
Stylecrafter: Enhancing stylized text-to-video generation with style adapter.
arXiv preprint arXiv:2312.00330, 2023.
- [19]
Yanting Miao, William Loh, Suraj Kothawade, Pascal Poupart, Abdullah Rashwan, and Yeqing Li.
Subject-driven text-to-image generation via preference-based reinforcement learning.
Advances in Neural Information Processing Systems, 37:123563–123591, 2024.
- [20]
Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or.
Null-text inversion for editing real images using guided diffusion models.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 6038–6047, 2023.
- [21]
opendiffusionai.
laion2b-en-aesthetic-square-cleaned.
[https://huggingface.co/datasets/opendiffusionai/laion2b-en-aesthetic-square-cleaned](https://huggingface.co/datasets/opendiffusionai/laion2b-en-aesthetic-square-cleaned), 2025.
- [22]
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al.
Training language models to follow instructions with human feedback.
Advances in neural information processing systems, 35:27730–27744, 2022.
- [23]
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach.
Sdxl: Improving latent diffusion models for high-resolution image synthesis.
arXiv preprint arXiv:2307.01952, 2023.
- [24]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.
Learning transferable visual models from natural language supervision.
In International conference on machine learning, pages 8748–8763. PmLR, 2021.
- [25]
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn.
Direct preference optimization: Your language model is secretly a reward model.
Advances in Neural Information Processing Systems, 36:53728–53741, 2023.
- [26]
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.
High-resolution image synthesis with latent diffusion models.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022.
- [27]
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman.
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.
In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 22500–22510, 2023.
- [28]
Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al.
Laion-5b: An open large-scale dataset for training next generation image-text models.
Advances in neural information processing systems, 35:25278–25294, 2022.
- [29]
Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Y Wu, et al.
Deepseekmath: Pushing the limits of mathematical reasoning in open language models.
arXiv preprint arXiv:2402.03300, 2024.
- [30]
Kihyuk Sohn, Nataniel Ruiz, Kimin Lee, Daniel Castro Chin, Irina Blok, Huiwen Chang, Jarred Barber, Lu Jiang, Glenn Entis, Yuanzhen Li, et al.
Styledrop: Text-to-image generation in any style.
arXiv preprint arXiv:2306.00983, 2023.
- [31]
Kaiyue Sun, Xian Liu, Yao Teng, and Xihui Liu.
Personalized text-to-image generation with auto-regressive models.
arXiv preprint arXiv:2504.13162, 2025.
- [32]
Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan.
Autoregressive model beats diffusion: Llama for scalable image generation.
arXiv preprint arXiv:2406.06525, 2024.
- [33]
Haotian Tang, Yecheng Wu, Shang Yang, Enze Xie, Junsong Chen, Junyu Chen, Zhuoyang Zhang, Han Cai, Yao Lu, and Song Han.
Hart: Efficient visual generation with hybrid autoregressive transformer.
arXiv preprint arXiv:2410.10812, 2024.
- [34]
Chameleon Team.
Chameleon: Mixed-modal early-fusion foundation models.
arXiv preprint arXiv:2405.09818, 2024.
- [35]
Aaron Van Den Oord, Oriol Vinyals, et al.
Neural discrete representation learning.
Advances in neural information processing systems, 30, 2017.
- [36]
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.
Attention is all you need.
Advances in neural information processing systems, 30, 2017.
- [37]
Bram Wallace, Meihua Dang, Rafael Rafailov, Linqi Zhou, Aaron Lou, Senthil Purushwalkam, Stefano Ermon, Caiming Xiong, Shafiq Joty, and Nikhil Naik.
Diffusion model alignment using direct preference optimization.
In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 8228–8238, 2024.
- [38]
Haofan Wang, Matteo Spinelli, Qixun Wang, Xu Bai, Zekui Qin, and Anthony Chen.
Instantstyle: Free lunch towards style-preserving in text-to-image generation.
arXiv preprint arXiv:2404.02733, 2024.
- [39]
Junke Wang, Zhi Tian, Xun Wang, Xinyu Zhang, Weilin Huang, Zuxuan Wu, and Yu-Gang Jiang.
Simplear: Pushing the frontier of autoregressive visual generation through pretraining, sft, and rl.
arXiv preprint arXiv:2504.11455, 2025.
- [40]
Xinlong Wang, Xiaosong Zhang, Zhengxiong Luo, Quan Sun, Yufeng Cui, Jinsheng Wang, Fan Zhang, Yueze Wang, Zhen Li, Qiying Yu, et al.
Emu3: Next-token prediction is all you need.
arXiv preprint arXiv:2409.18869, 2024.
- [41]
Zhouxia Wang, Xintao Wang, Liangbin Xie, Zhongang Qi, Ying Shan, Wenping Wang, and Ping Luo.
Styleadapter: A unified stylized image generation model.
arXiv preprint arXiv:2309.01770, 2023.
- [42]
wikiart.
wikiart.
[https://huggingface.co/datasets/huggan/wikiart](https://huggingface.co/datasets/huggan/wikiart), 2022.
- [43]
Yi Wu, Ziqiang Li, Heliang Zheng, Chaoyue Wang, and Bin Li.
Infinite-id: Identity-preserved personalization via id-semantics decoupling paradigm.
In European Conference on Computer Vision, pages 279–296. Springer, 2024.
- [44]
Yi Wu, Lingting Zhu, Lei Liu, Wandi Qiao, Ziqiang Li, Lequan Yu, and Bin Li.
Proxy-tuning: Tailoring multimodal autoregressive models for subject-driven image generation.
arXiv preprint arXiv:2503.10125, 2025.
- [45]
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu.
Omnigen: Unified image generation.
arXiv preprint arXiv:2409.11340, 2024.
- [46]
Jinheng Xie, Weijia Mao, Zechen Bai, David Junhao Zhang, Weihao Wang, Kevin Qinghong Lin, Yuchao Gu, Zhijie Chen, Zhenheng Yang, and Mike Zheng Shou.
Show-o: One single transformer to unify multimodal understanding and generation.
arXiv preprint arXiv:2408.12528, 2024.
- [47]
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong.
Imagereward: Learning and evaluating human preferences for text-to-image generation.
Advances in Neural Information Processing Systems, 36:15903–15935, 2023.
- [48]
Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, et al.
Dancegrpo: Unleashing grpo on visual generation.
arXiv preprint arXiv:2505.07818, 2025.
- [49]
An Yang, Baosong Yang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Zhou, Chengpeng Li, Chengyuan Li, Dayiheng Liu, Fei Huang, Guanting Dong, Haoran Wei, Huan Lin, Jialong Tang, Jialin Wang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Ma, Jin Xu, Jingren Zhou, Jinze Bai, Jinzheng He, Junyang Lin, Kai Dang, Keming Lu, Keqin Chen, Kexin Yang, Mei Li, Mingfeng Xue, Na Ni, Pei Zhang, Peng Wang, Ru Peng, Rui Men, Ruize Gao, Runji Lin, Shijie Wang, Shuai Bai, Sinan Tan, Tianhang Zhu, Tianhao Li, Tianyu Liu, Wenbin Ge, Xiaodong Deng, Xiaohuan Zhou, Xingzhang Ren, Xinyu Zhang, Xipin Wei, Xuancheng Ren, Yang Fan, Yang Yao, Yichang Zhang, Yu Wan, Yunfei Chu, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zhihao Fan.
Qwen2 technical report.
arXiv preprint arXiv:2407.10671, 2024.
- [50]
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu.
Qwen2.5 technical report.
arXiv preprint arXiv:2412.15115, 2024.
- [51]
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang.
Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.
arXiv preprint arXiv:2308.06721, 2023.
- [52]
Jiahui Yu, Xin Li, Jing Yu Koh, Han Zhang, Ruoming Pang, James Qin, Alexander Ku, Yuanzhong Xu, Jason Baldridge, and Yonghui Wu.
Vector-quantized image modeling with improved vqgan.
arXiv preprint arXiv:2110.04627, 2021.
- [53]
Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum.
Dino: Detr with improved denoising anchor boxes for end-to-end object detection.
arXiv preprint arXiv:2203.03605, 2022.
- [54]
Chunting Zhou, Lili Yu, Arun Babu, Kushal Tirumala, Michihiro Yasunaga, Leonid Shamis, Jacob Kahn, Xuezhe Ma, Luke Zettlemoyer, and Omer Levy.
Transfusion: Predict the next token and diffuse images with one multi-modal model.
arXiv preprint arXiv:2408.11039, 2024.

## Peer Reviews

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: - In Table 1, it is unclear why NanoBanana and Flux-Kontext are excluded from quantitative metrics. 
- The paper would be stronger if it demonstrated that the approach generalizes to multiple AR backbones, not just Lumina-mGPT.
- The method may struggle when style information is localized (e.g., reference image has a stylized object but plain background). Explicit discussion would help clarify limitations and future improvement directions.
soundness: 3
strengths: - The qualitative results are strong, showing that the method often produces visually pleasing stylized outputs compared to diffusion-based baselines.
- The proposed style-enhanced token mechanism addresses the content leakage problem that is common in existing style transfer methods.
- The work explores style alignment in autoregressive models, which is a less-studied direction compared to diffusion-based approaches.
summary: This paper proposes StyleAR, a framework enabling multimodal AR models to perform stylized image generation. A CLIP-based image encoder and perceiver resampler are used to extract style tokens, and a style-enhanced token mechanism is introduced to mitigate content leakage. Experiments show qualitative improvements in style consistency, supported by a user study and ablation analysis.
weaknesses: - The methodological novelty appears limited. The overall pipeline mainly involves constructing (text, stylized image) pairs and fine-tuning the AR model, while the style token extraction and integration mechanism resembles an adaptation of existing approaches.
- The proposed model requires training, which makes it computationally more expensive than training-free stylization approaches.
- Quantitative performance lags behind some baselines.

---

confidence: 4
contribution: 2
initial_rating: 4
presentation: 3
questions: Please refer to the weakness part.
soundness: 3
strengths: 1. The paper eliminates the need for difficult-to-acquire triplet data (as illustrated in Fig. 3), which can lower the data barrier for style-aligned generation tasks.

2. From the user study and the qualitative results, StyleAR achieves better performance compared to existing approaches.
summary: This paper introduces StyleAR for style-aligned text-to-image generation using multimodal autoregressive (AR) models. To solve the challenge of acquiring the "text-image-to-image" triplet data required for style transfer task, the author adopts InstantStyle to construct the binary data and combines the raw datasets to joint the model. Besides, the author uses a CLIP encoder with a perceiver resampler to create "style tokens," and a "style-enhanced token" technique designed to prevent content leakage.
weaknesses: 1. The idea in this method is a little bit confusing: 

a). The paper's core premise is difficult to follow. A central claim is that the method does not require triplet data, yet the authors use InstantStyle to synthesize stylized data for training. This seems contradictory. 

b). Furthermore, it is not explained what loss or constraint is used to enforce style consistency during training. 

c). The claim that Gaussian noise (n) "weakens irrelevant semantic features" is also questionable, as this noise would likely degrade the essential style information from the reference image simultaneously, undermining the stated goal.

2. The style injection mechanism appears to be adopted from prior work, lacking novelty. More critically, the proposed "Style-Enhanced Inference" method is overly simplistic and suffers from a significant practical limitation: it relies on obtaining a segmented image (mask) of the style reference. This is a non-trivial step and often infeasible for arbitrary style images. The failure of this approach is evident in the qualitative results, particularly in Figure 6 (rows 2 and 4). The 4th row, for instance, exhibits vibrant colors, suggesting the strategy is not robust.

3. Insufficient Experimental Analysis and Unclear Results: The experimental section lacks depth and fails to explain several critical findings:

a). Missing Hyperparameters: The paper does not specify the values for alpha and r (for Style-Enhanced tokens) or describe the methodology for tuning them.

b). Counter-intuitive Ablations: The analysis of training with different ratios of raw vs. stylized images is baffling. Why does increasing the proportion of raw images lead to the model overfitting on style? This is highly counter-intuitive and requires a much clearer explanation. This result raises concerns about potential data leakage: are the styles and prompts used for testing also part of the training set?

c). Persistent Content Leakage: Despite the claims, content leakage is still evident in the generated images (e.g., the balloon on line 343 and the image on line 393). The evaluation of style transfer would be more convincing if the authors tested on more specific or abstract styles (e.g., "fire," "ice") to truly assess the model's ability to separate style from content.

---

confidence: 5
contribution: 1
initial_rating: 2
presentation: 2
questions: see weaknesses
soundness: 1
strengths: The chapter organization of the paper is clear, and the selected images are feasible.
summary: This paper primarily addresses the issue of style transfer in AR models. The authors claim that high-quality AR style transfer models can be trained simply by using binary groups.Freezing CLIP and fine-tuning the resampler are common techniques in diffusion models.
weaknesses: The introduction in Section 3.2 on training data is not sufficiently addressed. How can binary groups ensure that the prompts cover a wide range of styles? The explanation is very unclear, and it does not convey the efforts made in terms of data.

The training framework seems similar to the approach of diffusion models for style control, lacking novelty.

Many recent state-of-the-art methods were not included in the comparison.

Why is CLIP still used as the style encoder? In fact, there are already some commonly used style encoders, such as CSD, OneIG-StyleEncoder, etc.

Carefully selecting some cases is not very difficult, but I feel that this article does not address the actual challenges.The ablation study is relatively incomplete, as it fails to demonstrate the advantages of the data organization approach in other models.

---

confidence: 3
contribution: 2
initial_rating: 2
presentation: 2
questions: The stylized image generation is conditioned on textual representations of prompts augmented by image representations of the style reference. How to balance the generation guidance when there is semantic conflict in these two conditions, for example text prompt ‘a cat’ with a stylized image of a dog?
soundness: 2
strengths: - The presented qualitative samples demonstrate high visual quality, suggesting the efficiency of the proposed method as it accomplishes this text-image-to-image task with only text-to-image data.
- The overall model design is reasonable. To avoid content leakage from the encoded image representation, the authors introduce some Gaussian noise to perturb the generation condition. Such a design helps to improve the robustness of their self-supervised architecture.
summary: This paper proposes a new method for enabling stylized text-to-image generation in multimodal auto-regressive (AR) models. Despite the recent progress in unified multimodal generative models, the pre-trained foundation models fail in stylized text-to-image generation, where the model is challenged with controllable generation based on both text prompts and a reference style image. This limitation is due to the lack of high-quality text-image-to-image triplet data. To address this limitation, the authors propose to utilize the text-image binary data, which could be generated from existing diffusion-based stylized generation models. For crafting the style conditions, the authors introduce a Resampler module combined with a CLIP image encoder to extract style features from the training image to augment the text prompts, effectively transforming the task into text-to-image generation.
weaknesses: - There is no evidence for the claim ‘the performance of AR models on stylized text-to-image generation is upper bounded by the base data generation model like diffusion models’ in line 129. Specifically, as discussed in lines 228-229 Section 3.2, the low-quality text-image-to-image triplets produced by InstantStyle is the bottleneck for training AR models on this task. To better support this claim, the author shall consider an ablation study on model performance trained on such triplet data compared to that trained on binary data. On the other hand, if the low similarity between style reference image and the stylized output of InstantStyle presents the difficulty, wouldn’t it be simpler to filter the generated data as post-processing, or employ a stronger base model like FLUX? Such a motivation is not sound, especially given the existence of such triplet stylized dataset, or data generation pipeline [1].
- The quantitative metrics for showing the effectiveness of the proposed method is sparse. Currently there are only 3 quantitative metrics adopted. More established quantitative metrics like CSD score [2] would help.
- The novelty of the proposed method is somehow limited. As claimed as the first innovation in line 107, the integration of image encoders and a perceiver resampler is exactly the design choice of IP-Adapter, which is one of the references of this paper.
- The manuscript is a little bit hard to follow. There is substantial space for improvements in the readability. For instance, consider splitting the sentence ‘To facilitate …’ in lines 49-52.

[1] Xing, P., Wang, H., Sun, Y., Wang, Q., Bai, X., Ai, H., Huang, R. and Li, Z., 2024. Csgo: Content-style composition in text-to-image generation. arXiv preprint arXiv:2408.16766.

[2] Somepalli, G., Gupta, A., Gupta, K., Palta, S., Goldblum, M., Geiping, J., Shrivastava, A. and Goldstein, T., 2024. Measuring style similarity in diffusion models. arXiv preprint arXiv:2404.01292.


Decision: Reject