Mod-Adapter: Tuning-Free and Versatile Multi-concept Personalization via Modulation Adapter
Authors: Weizhi Zhong
Huan Yang
Zheng Liu
Huiguo He
Zijian He
Xuesong Niu
Di ZHANG
Guanbin Li
Venue: ICLR 2026 Poster (2026)
Keywords: Multi-concept Customization
DiT
text-to-image generation
Primary area: generative models
TL;DR: A novel tuning-free method for versatile multi-concept personalization, capable of customizing both object and abstract concepts without test-time fine-tuning.

Abstract: Personalized text-to-image generation aims to synthesize images of user-provided concepts in diverse contexts. Despite recent progress in multi-concept personalization, most are limited to object concepts and struggle to customize abstract concepts (e.g., pose, lighting). 
Some methods have begun exploring multi-concept personalization supporting abstract concepts, but they require test-time fine-tuning for each new concept, which is time-consuming and prone to overfitting on limited training images.
In this work, we propose a novel tuning-free method for multi-concept personalization that can effectively customize both object and abstract concepts without test-time fine-tuning. 
Our method builds upon the modulation mechanism in pre-trained Diffusion Transformers (DiTs) model, leveraging the localized and semantically meaningful properties of the modulation space. Specifically, we propose a novel module, Mod-Adapter, to predict concept-specific modulation direction for the modulation process of concept-related text tokens.
It introduces vision-language cross-attention for extracting concept visual features, and Mixture-of-Experts (MoE) layers that adaptively map the concept features into the modulation space.
Furthermore, to mitigate the training difficulty caused by the large gap between the concept image space and the modulation space, we introduce a VLM-guided pre-training strategy that leverages the strong image understanding capabilities of vision-language models to provide semantic supervision signals.
For a comprehensive comparison, we extend a standard benchmark by incorporating abstract concepts. Our method achieves state-of-the-art performance in multi-concept personalization, supported by quantitative, qualitative, and human evaluations.

Published as a conference paper at ICLR 2026
MOD-ADAPTER: TUNING-FREE AND VERSATILE MULTI-CONCEPT PERSONALIZATION VIA MODULA- TION ADAPTER
Weizhi Zhong'; Huan Yang", Zheng Liu', Huiguo Hc Zijian He*,
Xuesong Niu², Di Zhang", Guanbin Li
Schooo of Computer Science and Engineering, Sun Yat-sen University, Guangzhou, China
2 Kolors Team, Kuaishou Technology
3 Shenzhen Loop Area Institute, Shenzhen, China
GGunngdong Key Laboratory of Big Data Analysis and Processing, Guangzhou, China
'oouhh China University of Technology, Guangzhou, China
Iupui Concegts
Ispei Cencpit
surface
Fight
vaboi with poie under the light.
rilbx wiR In imago style.
Right
the fires, cal and Rower vise made with surfiice
A Rower vise with surface under light cave
Concupts
light
degg
color
surface
surface
A dog and lundba snade wall starlaat sandy beadh
dig and wallet mnde with sut face under the light cere
Figure Results our multi-concept personalized image generation method. Our method enables customizing both object and abstract concepts (e.g.. pose, light, surface) without test-time fine-tuning. The colored words the prompt below image indicate concepts to be personalized.
ABSTRACT
Personalized text-to-image generation aims to synthesize images of user-provided concepts in diverse contexts. Despite recent progress in multi-concept person- alization, most are limited to object concepts and struggle to customize abstract concepts (e.g., pose, lighting). Some methods have begun exploring multi-concept personalization supporting abstract concepts, but they require test-time fine-tuning for each new concept, which is time-consuming and prone to overfitting on limited training images. In this work, we propose novel tuning-free method for multi- concept personalization that can effectively customize both object and abstract concepts without test-time fine-tuning. Our method builds upon the modulation mechanism in pre-trained Diffusion Transformers (DiTss) model, leveraging the localized and semantically meaningful properties of the modulation space. Specif- ically, we propose a novel module, Mod-Adapter, to predict concept-specific modulation direction for the modulation process of concept-related text tokens. It introduces lasiuaelanguage cross-attention for extracting concept visual features, and Mixture-of oixtere-of-Experts (MoE) layers that adaptively map the concept features into the modulation space. Furthermore, to mitigate the training difficulty caused by the large gap between the concept image space and the modulation space, we introduce VLM-guided pre-training strategy that leverages the strong image understanding
Work done while interning at Kuaishou Technology. Corresponding author.

---

Published as a conference paper at ICLR 2026
capabillities of dision-language models provide semantic supervision signals. For a comprehensive comparison, we extend a standard benchmark by incorporating ab- stract concepts. Our method achieves sfate-of-the the- performance in multi-concept personalization, supported by quantitative, qualitative, and human evaluations. Project Page: https:/ /wei zhi zhong . gith... wtw.chu..oooo..... io/Mod- Adapter
INTRODUCTION
Personalized text-to-image generation aims to synthesize images of the concepts specified by user- provided images in diverse contexts. Recently, this technique has attracted increasing research attention due to its broad applications, such as poster design and storytelling. However, existing personalized generation methods primarily focus on object concepts (c.g., common objects and animals) and struggle to personalize non-object/abstrrat concepts (e.g.. pose and lighting), limiting their wider applicability. Recently. Token Verse (Garibi et al.) 2025) proposes a multi-concept personalization framework supporting both object and abstract concepts. However, its concept- specific test-time fine-tuning paradigm requires fine-tuning for each new concept image at test time, which is time-consuming and tends to overfit on the single training image, leading to suboptimal results. In this work, we take the first step toward tuning-fiee free framework enabling versatile multi-concept personalization for both objects and abstract concepts as shown in Fig
Existing tuning-free personalized generation methods (Sun et al. 2024 Wang al. 2025 Huang et al. 2025 Ma et al., 2024) often face two challenges when customizing abstract concepts. First, these methods failed to decouple the object concept and abstract concept from the input image due to the lack of an effective mechanism for extracting abstract features. As a result, they tend to directly replicate the object into the generated image. For example, when personalizing persons" pose concept, the generated person often closely resembles the one in the input concept image, rather than merely reflecting the pose features. This compromises the alignment between the generated image and the input prompt. Second, the features of abstract concepts are easily influenced by textual features or other concept features during generation, hindering the accurate preservation of the customized concept. This issue arises because these methods either concatenate concept image features with text features, or fuse them through additive cross-attention layers et al. 2023), resulting in limited localized control over the generated content.
To address these challenges, we propose a novel tuning-free framework for personalizing multiple concepts (both objects and abstract concepts) by leveraging the localized and semantically meaningful properties of the modulation space in DiTs. Specifically, we design a novel module, Mod-Adapter, to predict modulation directions for customized concepts. The directions are further integrated into the modulation process of concept-related textual tokens (e.g.. "surface"), facilitating disentangled and localized control over the generated content. To effectively extract the target concept features from the input image, the Mod-Adapter introduces vision-language cross-attention layers that utilize the CLIP model's alignment capability between image and textual features. Besides, for accurately mapping the extracted concept visual features into the direction of DiT modulation space, we introduce a Mfxture-of-Eaan of-Experts (MoE) mechanism within Mod-Adapter, where each expert is responsible for handling concepts with similar mapping patterns. Furthermore, to mitigate the difficulty of training Mod-A aod-Adapter from scratch due to the large gap between the concept image space and the DiT modulation space, we propose a VLM- leverages the strong image understanding capabilities of vision-language models to provide semantic supervision signals. To summarize, our key contributions are as follows:
pre-training strategy for better initialization, which
We propose novel tuning-free and versatile multi-concept personalization generation method that can effectively customize both object and abstract concepts, such as pose, lighting, and surface, without requiring test-time fine-tuning for new concepts.
We propose an innovative module, Mod-Adapter, to predict concept-specific personalized directions within the modulation space in novel tuning-free paradigm. Within Mod-Adapter, the designed Dision-languse gross-attention extract concept visual features by leveraging the image-text align- ment capability of CLIP, and the oftture-of-Expoor layers adaptively project these features into the modulation space. In addition, we propose a novel pre- training strategy guided by a languullan model to facilitate training Mod-Adapter.
2

---

Published as a conference paper at ICLR 2026
We extend the commonly used benchmark by incorporating abstract concepts, resulting in new benchmark named DreamBench-Abe Experimental results demonstrate that our method achieves state-of-the- art performance in multi-concept personalization generation on this benchmark, as validated by quantitative and qualitative evaluations as well as user studies.
RELATED WORKS
In contrast to single-concept personalization works (Tan et al. 2025 Zhang al. 2025) that can only customize a single concept, our discussion centers on multi-concept personalization, which can be further categorized into tuning- luning-based and tuning- funing-free methods.
Suning-based multi-concept personalization ap- proach (Choi et al. [2025] Jin et al. 2225 Kong et al. 2024 Kwon & Ye 2025 Gu et al. 2023 Dong et al.
Tuning-Base Multi-concept Personalization.
et al.) 2025 Kumari et al. 2023) requires one or more reference images of each concept to fine-tune the model before performing multi-concept personalization test time. For example, Textual Inversion (Gal ct al. 2023) and P+ (Voynov et al. 2023) introduce personalized text-to-image generation by learning pseudo-word text embedding represent each input concept. MuDI (Jang et al. 2024) and ConceptGuard (Guo & Jin 2025) propose solutions to mitigate the concept confusion problem in multi-concept customization, using form of data augmentation and bonncpttbinddig prompts techniques, respectively. Recently, Token Verse (Garibi et al.) 2025) proposed a disentangled multi-concept personalization method, which supports decoupling abstract concepts beyond objects, such as lighting conditions and material surfaces, from images. It optimizes a small MLP for each image to predict the modulation vector offsets for words in the image caption, learning to disentangle concepts of the image. Despite these advancements, tuning- -baed methods require test-time fine-tuning for unseen concepts, which time-consuming and prone to overfitting on limited training images, often leading to suboptimal results. Contrasting with Token Verse, propose a novel tuning-free free paradigm along with an innovative module, Mod-Adapter, and a novel pretraining strategy guided by VLM.
Tuning-Fris Free Multi-concept Personalization. Many studies (Sun et al. 2024 Ding et al. 2024 Wu
al. 2024 HHungg et al., 2025 Shi et al., (025 Chen et al. 2025) have attempted explore tuning- free multi-concept personalized generation, which can generalize to unseen concepts without the need for test-time fine-tuning, Among these, Subject-Diffusion (Ma et al. 2024) and FastComposer (Xiao
et al.) 2024) integrate subject image features into the text embeddings of subject words. The combined features are then incorporated into the diffusion model through cross-attention layers to enable personalized generation for multiple subjects. Similarly, X-ECLIPSE (Patel et al. 2024) projects image-text interleaved features into the latent space of the image generation model using a contrastive pre-training strategy. BI.P-DDifusson (I. et al. (023a) follows a VLM BLIP-2 (Li et al. 2023b) to pre-train an encoder that produces text-aligned object-type concept representations. In contrast, our pre-training leverages the strong image understanding capabilities of frozen VLM to facilitate the alignment of concept image features with the DiT modulation space. Benefit from the localized and semantically meaningful properties of the modulation space, method can effectively customize both object and abstract concepts. Emu2 (Sun et al. (2024) proposes a generative autoregressive multimodal model for various multimodal tasks including multi-concept customization. Instructlmagen (Hu et al. 2024a) introduces multimodal instructions and employs multimodal instruction tuning to adapt text-to-image models for customized image generation. MS- Diffusion (Wang et al., 2025) proposes a layout-guided multi-subject personalization framework equipped with grounding resampler module and multi-subject cross-attention mechanism. MIP Adapter (Huang et al. 2025) introduces
merge mechanism to alleviate the concept confusion problem in multi-concept personalized generation. UniReal (Chen et al. 2025) proposes unified framework for various image generation tasks, including multi-subject personalization. by learning real-world dynamics from large-scale video data. However, all existing tuning-free free multi-conces personalization methods primarily focus on personalizing object concepts, and struggle to handle customization of abstract concepts. In this study, we propose a novel tuning-froe free framework for multi-concept personalization that can effectively customize both object and abstract concepts.
3

---

Published as a conference paper at ICLR 2026
MSE
CLP
GLIP broake
GUP
CUI Tex: Eneoder
(a) Training framework
(b) Pre-training
Trandde
G: Sudrncionn @ Asdton
Acapler
Trespprent cyan-grean glase.
NX
Food-Forvwnd Forward
VLM
Joint Altentin
Micturooof-xpartt
surface
...
tmeetep
VL Attention
(c) Inference
sbock
cel
Mod- Acaaperr
Vid
DiT
CLIP
Mod- Adaptter
Vig
surface
A table with surface
foreel, cat and floer visre
Figure 2: Overview of the proposed method. (a) During training, the proposed Mod-Adapter module takes as input concept image and its corresponding concept word, and predicts a concept-specific modulation direction for each DiT block. The predicted directions (Mod) process of the concept-related text tokens DiT. (b) Pre-training of the Mod-Adapter module. The concept image is fed into vision-language model (VLM) to obtain detailed descriptive caption of the target concept in the image, which is further encoded by CLIP text encoder and mapped by an MLP layer (M) into the DiT modulation space. The resulting feature provides the semantic supervision signals for Mod-Adapter. (c) At inference, Aod-AAdater predicts a modulation direction for each customized concept. These directions integrated into the modulation process of their corresponding text tokens to enable multi-concept customization.
integrated into the modulation
3 METHOD
The overview our proposed method is shown in Fig. We propose modulation space adapter module, named Mod-Adapter, for versatile multi-concept customization generation, along with a VLM-suupervised adapter pretraining mechanism. In the following sections, we first introduce preliminaries token modulation in DřT's (Sec. [. and then detail the design of the Mod-Adapter module (Sec. 3.2p and the proposed pretraining strategy (Sec. 3.3
3.1 PRELIMINARIES
Diffusion Transformers(DiTs Peebles Xie 2023) have recently emerged as a promising architec- ture for diffusion models (Ho et al. 2020), owing to the strong scalability of transformer (Vaswani al. 2017). In text-to-image DiT's (Esser al. 2024 Labs 2024), text tokens and noisy image tokens jointly processed through N DiT blocks, each consisting of joint attention and feed-forward layers, to predict the noise added to the image VAE (Kingma & Welling 2013) latent. In addition, the diffusion timestep condition and global representation of the text prompt integrated into the generation process via a token modulation mechanism before joint attention. In this work, we build on FLUX (Labs 2024), state-of-the tht-hhe-rt art DiT-based text-to-image to-image model, for multi-concept personalization. Specifically, in each DiT block, all image and text tokens are modulated by a shared conditioning vector y through Adaptive Layer Normalization (AdaLN) (Peebles & Xie 2023). The modulation vector y is computed by summing the diffusion timestep embedding temb and projection of the CLIP (Radford et al. 202T pooled prompt embedding, as follows:
Me(tems) + (CLIP(p)))
where p the text prompt, M, and M are two distinct MLP mapping layers. Token Verse (Garibi et al. 2025) demonstrates that modulating individual tokens differently enables localized manipulations over the concept of interest during generation process. Specifically, instead of using the same modulation vector for all tokens, the text tokens associated with the target concept are modulated using adjusted modulation vectors:
y + $A attriibute:
4

---

Published as a conference paper at ICLR 2026
where is scale factor, the direction Aattribute captures the personalized attributes of the concept in the modulation space. The updated vectors 3/ will induce localized effects on concept-related image regions through the joint attention layers. Thanks to the semantically additive properties of CLIP
textual embedding, previous works (Baumann et al. 2025; Hu et al. 2024b] Garibi et al. (0255 have shown that the semantic direction of attribute be estimated using contrastive prompts with and without the specific attribute. Specifically, Aottribate can be approximated as:
Auttribute R M(CLIP(p*)) - M(CLIP(P))).
where p+ is positive prompt with some attribute added (e.g., "transparent cyan-green glass surface"), p" is a neutral prompt without attribute (e.g., "surface"), ) i.e., the concept word itself. Token- Verse (Garibi al. 2025) proposes training separate MLP for each image to predict personalized directions Aattribute for the concept in the image. However, it requires fine-tuning a model at test time for each new concept image. In contrast, we propose a tuning-free method for multi-concept customization that can generalize to unseen concepts without tisttimm fine-tuning, as detailed in the following sections.
3.2 MODULATION SPACE ADAPTER
As illustrated in Fig. a). our proposed Mod-Adapter takes concept image and its corresponding concept word as input, and predicts a personalized direction Aattribute in the modulation space. Since FLUX (Labs 2024) contains N DiT blocks, Mod-Adapter predicts distinet modulation direction A; for each block to enhance the model's 's expressiveness, forming the set (A: ii= ......N) As illustrated in Fig. [(a), in the i-th DiT block, only A; will be added the original modulation vector. Inspired by the formulation in Eq. 3 fod--Adpterr first predicts the attribute feature of the customized concept in the modulation space, denoted as Ftt. Then, the personalized modulation directions { {A; I.....N] N} are computed as follows:
A = F M(CLIP(()))
To obtain attribute feature F+t. the Mod-Adapter is designed with vision- languaee cross- cots-attenttin mechanism and a Mfxture-of-ENaee Expertss (MoE) component, as detailed below.
Vision Language Cross-Attention. The proposed alignment capability of the CLIP model (Radford et al. 2021) between image and text features. Specifically, to extract the desired concept features from the input concept image, the corresponding concept word p" (e.g., "surface") is first passed through the CLIP text encoder followed by the MLP mapping layer M obtain neutral feature(i.e. M(CLIP()")) To generate personalized modulation direction for each of the N DiT blocks, the neutral feature is further projected by linear layer into N queries, denoted as Q1..... QN. Sinusoidal positional embeddings are added to these queries for distinguishing the direction of different DiT blocks. Meanwhile, we encode the input concept image using the CL.IP image encoder and project the fine- fine-grained features from the penultimate layer into key and value, denoted as K and V, respectively. Then, cross-attention between the text and image features is computed using the following formula:
fully exploits the cross-modal
Attention (Q,, K, V) Softmax( Q,K
V
(5)
vd
where d is the dimension of key and i= I,...,N. N.
Mixture of Experts. After extracting the concept visual feature via vision-language cross-attention, the feature must be mapped into the modulation space of the pre-trained DiT model for effective integration. A straightforward approach is to use an MLP layer for this mapping. However, we find that this leads to suboptimal performance, possibly due to the fact that different types of concepts exhibit distinet mapping patterns. This suggests that concepts with similar mapping patterns should be handled by the same mapping function, while those with significantly different patterns should be processed separately. Motivated by this intuition, we introduce a Mixture-of--xpertt (MoE) mechanism to adaptively map various concept visual features into the modulation space, where each expert corresponds to a distinct MLP mapping network. The core of the MoE lies in the routing network, which is responsible for assigning different inputs to different experts. A common practice is to use a learnable linear gating network to perform the routing. However, we find that this approach tends to suffer from the well-known imbalanced expert utilization problem, where many experts
5

---

Published as a conference paper at ICLR 2026
remain underused during training, even when using load balancing loss (Shazeer et al. 2017 To address this issue, we design a simple parameter-free routing mechanism based on the k-means clustering algorithm. Specifically, we perform k-means clustering over the neutral features of all concept words (i.e,,((CL (((LIIP())) in the training dataset, with the number of clusters equal to the number of experts. Each resulting cluster corresponds to concepts of certain categories, which are assigned to specific expert for processing.
Training and Inference. The training framework of Mod-Adapter illustrated in Fig. a). The proposed Mod-Adapter is the only component that requires training, while the pre-trained DiT-based text-to-image generation model is kept frozen. During training. only a single customized concept condition is added to the modulation space. while at inference time, multiple concept conditions can be added to the modulation of their corresponding concept tokens to enable multi-concept personalization, as illustrated in Fig. [[c). We train Mod-Adapter using the same diffusion objective as the original DiT model (Labs) 2024). However, we find that training Mod- pod-Adapter from scratch with this objective alone is challenging, possibly due to the large gap between the modulation space in DiT and the concept image space. To address this issue, we propose a VLM-supervised pretraining mechanism for Mod-Adapte to obtain better initialization, described below.
3.3 MOD-ADAPTER PRE-TRAINING SUPERVISED BY VLM
Our proposed pre-training approach is illustrated in Fig. bb), and is inspired by the formulations in land Eq. [ from the previous analysis. The attribute feature Ftt predicted by Mod-Adapter can be coarsely supervised during pre-training by the modulation-spall space representation (i.e., M(CLIP(pp* ))) of a positíve prompt pz describing the attribute of the concept in image. To obtain an accurate positive prompt pt of the input concept image, we leverage a pretrained vision-language model (VLM) that already possesses strong image understanding capabilities. Specifically, the concept image and pre-defined system prompt are fed into the VLM, where the system prompt guides the model to describe the detailed attributes of the target concept in the image, resulting in the output p+. The positive prompt pt is then encoded by the CLIP text encoder and mapped by the MLP layer M into the modulation space. The resulting feature is used to supervise the Adapter' 's output Fi during pre-training, with the MSE loss defined as:
N ||, M(CLIP(p*))
1
Cpretrain
(6)
During pre-training, only the objective Lpretrain is used, and the output of Mod-Adapter is not integrated into the memory-intensin DiT model, which enables efficient and lightweight pretraining. After pretraining, the Mod-Adapter is incorporated into the DiT model and trained further using only the diffusson objective mentioned above.
EXPERIMENTS
4.1 EXPERIMENTAL SETUPS
Datasets. We train our model using the open-source MVImgNet object dataset YYu et al. 2023). the Animal Faces-HQ (AFHQ) dataset (Choi et al., 2020), and synthetic data generated by the FLUX (Labs) 2024) model. MVImgNet contains multi-view images of real-world objects. We select objects of 40 commonly seen categories from it and use a single-view image per object for training. AFHQ is high-quality animal face dataset containing common categories such as cats, dogs, and wildlife. For abstract concepts, we synthesize training data using the FLUX model. which has been demonstrated by recent works (Tan et al. 2024 Cai et al. 2025a) to be an effective diffusion self-distillation strategy. The resulting abstract concept dataset covers range of common categories, including those used in Token (Garibi et al. 2025) (environment light, human pose, scene, and surface), as well as our additional extensions of image style and color tone. In total, our final training dataset contains 106,100 104 images paired with corresponding captions.
Implementation Details. We adopt the pre-trained FLUX.I-dev model as our DiT backbone containing N = 57 blocks. The Mod-Adapter contains N' = 4 blocks and the number of experts is set to 12 according to the ablation study. It is the only module that requires optimization during training, with total parameter count of just 1,667.77N, For more details, please see Appendix C

---

Published as a conference paper at ICLR 2026
Table Quantitative comparison. CP and PF score evaluates concept preservation and image-text alignment respectively. Their product CP-PF is a comprehensive evaluation score. CLIP-T evaluates the image-text alignment. All scores range from 0 to 1. t: higher is better.
Multi- Concept
Single-Concap
Methods
CPt
PF
CP-PF
CLIP-TI
CP
PFt
CP-PFt
CLIP-TT
Emu2 (Sun et al. 2024
0.53
0.48
0.25
0.299
0.73
0.57
0,42
0.288
MIP-Adapter (Huang ct al. 2025
0.68
0.55
0.37
0.328
0.70
0.39
0.27
0.277
MS-Ditfusion ( (Wang el al 2025
0.62
0.51
0.32
0.326
0.57
0.40
0.23
0.282
Veren (Garibi ct al.
2025
0.56
0.56
0.31
0.316
0.58
0.66
0.38
0,312
Mod-A
0.70
0.89
0.62
0.330
0.61
0.89
0.54
0.315
Comparison Methods. We compare our method with ofte--of-the-art multi-subject personalization
approaches, including tuning-free methods Emu2 (Sun et af.) 2024). MIP-Adapter (Huang et al. 2025), and MS-Diffusion (Wang et al.] 2025). as well as the tuning-based method Token Verse (Garibi et al. 2025). Since the training datasets used by MIP-Adapter and MS-Diffusion do not include abstract concept data, we fine-tune their released model weights using our abstract concept data to ensure a fair comparison. Emu2 has not released its training code, but its training data includes common abstract concepts such style and lighting.
Evaluation Benchmarks. Following prior work (Wang et al. 2025 Huang et al. 2025; Sun et al.
2024), we evaluate the performance of both single-concept and multi concept personalization on the DreamBench benchmark (Ruiz et al.] (023)), which contains 30 object or animal concepts and 25 template prompts. In addition, extend the DreamBench by incorporating 20 abstract concepts for a more comprehensive evaluation, resulting in a new benchmark named DreamBench- Bea.bebbchh-bb.... For multi-concept evaluation, we follow Token Verse (Garibi al. 2025) to construct 30 combinations from all 50 concepts. None of the test images or prompts appear in our training set. Among all methods, only TokenVerse based on per-sample training, requires using the test images for training.
Metries. We follow prior methods (Wang et al. 2025 Huang et al. 2025 Sun et al., 2024 Garibi al., 2025) to evaluate both single-concept and multi-concept personalization from two perspectives: the fidelity of the generated concepts (Concept Preservation) and the alignment between the generated images and the textual prompt (Prompt Fidelity). As our task involves both object and abstract concepts, we follow Voren (Garibi et al. 2025) and adopt multimodal LLM-based (OpenAl 2024, Peng et al. 2225) scoring approach. For each generated image, the multimodal LLM outputs a Concept Preservation (CP) and Prompt Fidelity (PF) score, which evaluate concept preservation and image-text alignment, respectively. Since there is often a tradc-off between the two metrics, their product (CP-PF) is also reported comprehensive score (Peng et al.) 2025). Besides, we evaluate image-text alignment by measuring the similarity between their CLIP embeddings (CLIP-T).
4.2 COMPARISONS
Quantitative Comparison. We report the quantitative comparison results in Tab. D In multi-concept personalization, our method outperforms all previous approaches across all metrics. Specifically, it achieves the highest CP-PF score of 0.62, demonstrating a substantial +67.6% improvement over the second-best method, MIP-Adapter (0.37). While AIP--dapter and MS-Diffusion achieve competitive CP scores (0.68 and 0.62, respectively), their PF scores (0.55 and 0.51) are significantly lower than ours (0.89). Consistent with observations from prior work, the CLIP-T metric is insensitive to variations in image-text alignment, with all methods scoring around 0.3. In single-concept personalization, our method still achieves significantly higher performance on the combined metric CP-PF compared to all other methods. Although Emu2 and MIP-Adapter outperform us on the CP metric. they perform significantly worse on both PF and CLIP-T, indicating that they sacrifice prompt fidelity to achieve higher concept preservation. In addition, while Emu2 performs well in single-concept personalization, its performance drops notably in the multi-concept setting.
Qualitative Comparison. Fig. 3 presents representative qualitative comparisons between our method and other approaches. In the first row of single-concept personalization, our method successfully generates a wallet with brown leather surface consistent with the input concept image. In contrast, MS-Diffusion, MIP-Adapter, and Emu2 fail to effectively disentangle the abstract concept 'brown leather' from the object (handbag). As result, they simply replicate the original object in the generated image, producing an undesired brown leather handbag instead of the desired wallet. This observation aligns with their high CP scores but low PF scores in Tab. T In the multi-subject per-
7

---

Published as a conference paper at ICLR 2026
Input Concepts
Ours
TokenVerse
MS-Diffusion
MIP-Adapter
Emu2
surface
Prompt: a wallet made with surface
dog
surface
Prompt: in the road, a dog and a flower vase made with surface
light
Prompt: purple T-shirt girl sitting with pose on stage under light
pose
dog
Image style
Prompt: dog and with pose in image style
Figure 3: Qualitative comparison. The left dashed box shows input concept images. Colored words in the prompt indicate concepts to personalized, while underlined text highlights elements that reflect differences in prompt alignment performance between methods.
Table User study results, CP and PF respectively record the average scores given by volunteers for concept preservation and image-text alignment. Scores range from to 5. t: higher is better.
Multi-Concept
Single-Coneept
CP1
PP1
CP1
PP1
Emau (Sun et 2024
2.10
2,02
2.66
3,04
MIP Aupper
MS Diffision Wane el
3.16
3.14
2.42
2.60
Token Verse Ganhi
3.35
3.48
3.43
2.87
Mod-Adapper(Ours)
sonalization setting, our method continues to demonstrate superior concept preservation and prompt alignment performance, whereas Emu2 shows reduced effectiveness, often generating unnatural concept combinations. MS-Diffusion and MIP-Adapter are still prone to "copy-paste" artifacts (e.g., the glass table in the second row and the man in the third row), which negatively affect prompt alignment. Meanwhile, their concept preservation performance also declines. In both single- and multi-concept personalization settings, tuning-based methods Token Verse tend overfit due the need for fine-tuning on each input concept image, which compromises both concept preservation and prompt alignment. Its per-sample training paradigm only encourages the model to better reconstruct the training images with the input of corresponding image captions, but it does not ensure good performance on unseen test prompts real-world scenarios. All these results demonstrate the superior performance of our method in multi-concept personalized generation.
User Study. Following the evaluation setting of Token (Garibi et al. 2025), we conducted a user study with 32 participants, asking them to rate each generated image in terms of concept preservation (CP) and prompt fidelity (PF). Each participant evaluated five results per method on both single- and multi-come multi-concept personalization settings, resulting in 4000 votes. The results are reported in Tab. 2 Our method receives consistently higher user ratings than all compared methods in both CP and PF For more details about the user study, please Appendix D
8

---

Published as a conference paper at ICLR 2026
4.3
ABLATION STUDY
Input concepts
w/o pre-training
w/o
k-means
surface
Prompt: backpack
dog
surface
Prompt: dog handbag serface
dog
Prompt: dog green sofa
Figure 4: Qualitative ablation results. The left dashed box shows input concept images. Eliminating
any proposed component degrades qualitative performance.
Table 3: Quantitative ablation results.
Multi-Concepy
Single-Concee
Methods
CPt
PPt
CP-PFt
CLIP-T1
CP
PPt
CP-PF
CLIP-T7
w/o k-means routing
0.59
0.83
0.49
0.324
0.54
0.82
0.44
0.311
w/o MoE
0.52
0.68
0.35
0.313
0.51
0.82
0.42
0.311
w/o VL-attn
0,52
0.75
0.39
0.329
0.57
0.86
0.49
0.317
w/o pre-training
0.58
0.306
0.33
0.72
0.24
0.308
Mod-Adapter(Ours)
0.70
0.89
0.62
0.330
0.61
0.89
0.54
0.315
Mod-Adapter Pre-training. We design an ablation variant excluding the Mod-Adapter pre-training strategy. Specifically, we train the Mod-Adapter from scratch using only the diffusion objective, under the same training settings and for the same total number of steps (176K) as the full model. The quantitative results of this variant are shown in the "w/o Pre-training" row of Tab. Both CP and PF scores drop significantly for both single-concepp and multi multi-concept personalization. Furthermore, as illustrated in the "w/o pre-trainn pre-training" column of Fig. 4 the generation quality degrades terms of both concept preservation and prompt fidelity compared our full model. This degradation is due to the large gap between the modulation space in DiT and the concept image space. contrast, our proposed VLM ILM-supervised pre- pre-training mechanism effectively mitigates this training difficulty by cawadiiiiiiia leveraging the strong understanding capabilities of the VLM to provide an initialization.
Vision Language Cross-Attention. We design ablation variant removing VL cross-attention from the Mod-Adapter module. In this variant, the concept word is not used as input, and the VL-attention replaced with standard cross-attention layer, where the queries are N learnable query tokens following MS-Diffusion (Wang et al. 2025). As shown in the "w/o VL-atten" results in Tab. and Fig. [ this variant shows slightly degraded performance in both concept preservation and prompt fidelity in single-concept personalization. However, in the multi-concept setting, the performance drop more significant, possibly due the learnable query mechanism's inability effectively extract target concept features. As mentioned earlier, the CLIP-T metric is insensitive to variations prompt alignment; thus, the CLIP-T score of this variant remains comparable to, or even slightly higher than that of our full model.
Mixture of Experts. We design an ablation variant by replacing the MoE layer with single MLP, while keeping the overall parameter count of the Mod-Adapter unchanged. As shown in the "w/o MoE" results on Tab. 3 and Fig. this variant performs worse than our full model both concept preservation and prompt fidelity. This is because simple MLP is insufficient to accurately project diverse concept features into the modulation space the pre-trained DiT network. For more ablation analysis on the number of experts, please refer to the Appendix F
K-means MoE Routing, We design a variant following the common practice of using a learnable linear gating network for routing. As shown in Tab. and Fig. 4 compared to full model, this variant shows degraded performance in both concept preservation and prompt fidelity. but still performs better than the "w/o MoE" variant. This is because the learnable linear gating network tends

---

Published as a conference paper at ICLR 2026
to result in the under-utilization of certain experts, which is equivalent to using fewer experts than our full model. For more detailed analysis and the ablation on the clustering method, please refer to the Appendix
CONCLUSION
We propose novel tuning-free framework for versatile multi-concept personalization, capable of customizing both object and abstract concepts without test-time fine-tuning. Our method contains novel module, Mod-Adapter, consisting of vision-language cross-attention layers for concept visual feature extraction and Mixture-of--Exeee layers for projecting features into the modulation space. Additionally, we introduce an innovative VLM-guided pretraining strategy to facilitate Mod-Adapter training. We conduct extensive experiments and demonstrate the superiority and effectiveness of our proposed method. For the limitation discussion of our method, please refer to the Appendix
6 ACKNOWLEDGMENTS
This paper is sponsored by CCF-Kuaishou Large Model Explorer Fund (No. CF-KuaiShou 2024007), and is also supported by the National Natural Science Foundation of China (No. 62322608), and supported by Kuaishou Technology.
REFERENCES
Stefan Andreas Baumann, Felix Krause, Michael Neumayr, Nick Stracke, Melvin Sevi, Vincent Tao Hu, and Björn Ommer. Continuous, Subject-Specific Attribute Control in T2I Models by Identify- ing Semantic Directions. In CVPR, 2025.
Shengqu Cai, Eric Chan, Yunzhi Zhang, Leonidas Guibas, Jiajun Wu, and Gordon. Wetzstein. Diffusion oif-distillation for zero-shot customized image generation. In CVPR, 2025a.
Yinqi Cai, Jichang Li, Zhaolun Li, Weikai Chen, Rushi Lan, Xi Xie, Xiaonan Luo, and Guanbin Li. Deepshield: Fortifying deepfake video detection with local and global forgery analysis. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 12524- 2025b.
Xi Chen, Zhifei Zhang, He Zhang, Yuqian Zhou, Soo Kim, Qing Liu, Yijun Li, Jianming Zhang, Nanxuan Zhao, Yilin Wang, et al. UniReal: Universal image generation and editing via learning real-world dynamics. CVPR, 2025.
Yongjin Choi, Chanhun Park, and Seung Jun Baek. DynASyn: Multi-subject personalization enabling dynamic action synthesis. AAAI, volume 39, pp. 1564-2574 2025.
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-We Woo Ha. StarGAN v2: Diverse image synthesis for multiple domains, In CVPR, pp. 8188 8897 2020.
Ganggui Ding, Canyu Zhao, Wen Wang, Zhen Yang, Zide Liu, Hao Chen, and Chunhua Shen. FreeCustom: Tuning-free customized image generation for multi-concept composition. In CVPR. pp. 1189-9098, 2024.
Jiahua Dong, Wenqi Liang, Hongliu Li, Duzhen Zhang, Meng Cao, Henghui Ding, Salman H Khan, and Fahad Shahbaz Khan. How to continually adapt text-to-image diffusion models for flexible customization? Advances in Neural Information Processing Systems. 37:130057-13003 2024.
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. In ICML, 2024.
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashnik, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In ICLR, 2023.
10

---

Published as a conference paper at ICLR 2026
Daniel Garibi, Shahar Yadin, Roni Paiss, Omer Tov, Shiran Zada, Ariel Ephrat, Tomer Michaeli, Inbar Mosseri, and Tali Dekel. Token Verse: Versatile multi-concept personalization in token modulation space. ACM SIGGRAPH 2025, 2025.
Yuchao Gu, Xintao Wang, Jay Zhangjie Wu, Yujun Shi, Yunpeng Chen, Zihan Fan, Wuyou Xiao, Rui Zhao, Shuning Chang, Weijia Wu, et al. Mix-of-show: Decentralized low-rank adaptation for multi-concept customization of diffusion models. Advances in Neural Information Processing Systems, 36: 16900-91115902 2023.
Zinan Guo, Pengze Zhang, Yanze Wu, Chong Mou, Songtao Zhao, and Qian He. Musar: Exploring multi-subject customization from single-subject dataset via attention routing. arXiv preprint arXiv:2505.02883, 2025.
Zirun Guo and Tao Jin. ConceptGuard: Continual personalized text-to-image generation with forgetting and confusion mitigation. In CVPR, 2025.
Will Hawkins, Brent Mittelstadt, and Chris Russell. Deepfakes on demand: The rise of accessible non-consensual deepfake image generators. In Proceedings of the 2025 ACM Conference on Fairness, Accountabilityy and Transparency, pp. 1602-1614, 2025.
Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. NeurIPS, 33: 6840-6851, 2020.
Hexiang Hu, Kelvin C.K. Chan, Yu-Chuan Su, Wenhu Chen, Yandong Li, Kihyuk Sohn, Yang Zhao, Xue Ben, Boqing Gong, William Cohen, Ming-Wai Chang, and Xuhui Jia. Instruct-Imagen Image June 2024a.
generation with multi-modal instruction. In CVPR, pp.
Taihang Hu, Linxuan Li, Joost van de Weijer, Hongcheng Gao, Fahad Shahbaz Khan, Jian Yang. Ming-Ming Cheng, Kai Wang, and Yaxing Wang. Token merging for training-fre free semantic binding in text-to-image synthesis. NeurlPS, 37:137646-137672, 2024b.
Qihan Huang, Siming Fu, Jinlong Liu, Hao Jiang, Yipeng Yu, and Jie Song. Resolving multi-condition confusion for finetuning-free personalized image generation. In AAAI, volume 39, pp. 3707-3714, 2025.
Sangwon Jang, Jaehyeong Jo, Kimin Lee, and Sung Ju Hwang, Identity decoupling for multi-subject personalization of text-to-image models. In NeurlPS, 2024.
Jiaxiu Jiang, Yabo Zhang, Kailai Feng, Xiaohe Wu, Wenbo Li, Renjing Pei. Fan Li, and Wangmeng Zuo. Mc" 2: Multi- concept guidance for eustomized multi-concept generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 2802-2812 2025.
Jian Jin, Yu Zhenbo, Shen Yang, Zhenyong Fu, and Jian Yang. LatexBlend: Scaling multi-concept customized generation with latent textual blending. In CVPR, 2025.
Diederik P Kingma and Jimmy Ba. Adam: method for stochastic optimization. arXiv preprint arXiv:1412.6980 2014.
Diederik Kingma and Max Welling, Auto-encoding variational bayes. arXiv e-prints, pp. arXiv- 1312, 2013.
Zhe Kong, Yong Zhang, Tianyu Yang, Tao Wang, Kaihao Zhang, Bizhu Wu, Guanying Chen, Wei Liu, and Wenhan Luo. OMG: Occlusion-friees friendly personalized multi-concept generation in diffusion models. In ECCV, pp. 253-270. Springer, 2024.
Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun- Yan Zhu. Multi-concept customization of text-to-image diffusion. In CVPR, pp. 1931-1941, 2023.
Gihyun Kwon and Jong Chul Ye. TweedieMix: Improwing multi-concept fusion for diffusion-based image/video generation. In http:///eriioorrddas222240..,..0 2025.
Black Forest Labs. Flux. https::/githhb...
latreett-abbs/flaee 2024.

---

Published as a conference paper at ICLR 2026
Dongxu Li, Junnan Li, and Steven Hoi. Blip- Blip-diffusion: Pre-trained subject representation for controllable sikt-to-image generation and editing. Advances in Neural Information Processing Systems, 36:30146-333 6016066, 2023a.
Junnan Li, Dongxu Li. Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-iman -image pre-training with frozen image encoders and large language models. In International conference on machine learning, pp. 19730-19742. PMLR, 2023b.
Ming Li, Taojiannan Yang. Huafeng Kuang, Jie Wu, Zhaoning Wang, Xuefeng Xiao, and Chen Chen. ControlNet++: Improving conditional controls with efficient consistency feedback: Project page: liming-ai. github. io/controlnet plus_plus. In ECCV, pp. 129-147. Springer, 2024.
Wenxue Li, Tian Ye, Xinyu Xiong, Jinbin Bai, Feilong Tang, Wenxuan Song, Zhaohu Xing, Lie Ju,
Guanbin Li, and Lei Zhu, Glasswizard: harvesting diffusion priors for glass surface detection. In Proceedings of the IEEE/CVF international conference computer vision, pp. 17848-17858. 2025.
Jian Ma, Junhao Liang, Chen Chen, and Haonan Lu. Jubject-Diffusiin Open domain personalized text-to-image generation without test-time fine-tuning. In ACM SIGGRAPH 2024 Conference Papers, pp. 1-12, 2024.
Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi, and Ying Shan. T21-Adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In AAAI, volume 38, pp. 4296-4304, 2024.
OpenAl, Introducing gpt-4o and more tools to chatgpt free users. 2024. URL
Maitreya Patel, Sangmin Jung, Chitta Baral, and Yezhou Yang. X-ECLIPSE: Multi-concept personal- ized text-to-image diffusion models by leveraging clip latent space. TMLR, 2024.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In ICCV, pp. 4195-4205 2023.
Yuang Peng, Yuxin Cui, Haomiao Tang, Zekun Qi, Runpei Dong, Jing Bai, Chunrui Han, Zheng Ge, Xiangyu Zhang, and Shu-Tao Xia. DreamBench++: A human-aligned benchmark for person- alized image generation. In ICLR, 2025, URL httpp:::/pperrreeee....t net. www.:nennf/ofiiio.mmmmo [GSCESJTkk
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In /CML, pp. 8748 8763. PMLR, PMLR, 2021.
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. DroahBooth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, pp. 22500-20022 22510, 2023.
Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outrageously large neural networks: The sparsely-gad mixture-offexxpere layer. arXiv preprint arXiv: 00656888,..,.. 2017.
Qingyu Shi, Lu Qi, Jianzong Wu, Jinbin Bai, Jingbo Wang, Yunhai Tong, and Xiangtai Li. DreamRe- lation: Bridging customization and relation generaion. In CVPR, 2025.
Quan Sun, Yufeng Cui, Xiaosong Zhang, Fan Zhang, Qiying Yu. Yueze Wang. Yongming Rao, Jingjing Liu, Tiejun Huang, and Xinlong Wang. Generative multimodal models are in-context learners. In CVPR, pp. 13398-14499, 2024.
Zhenxiong Tan, Songhua Liu, Xingyi Yang. Qiaochu Xue, and Xinchao Wang. OminiControl: Minimal and universal control for diffusion transformer. arXiv preprint arX1v:2411.15098, 2024.
Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer. 2025.
12

---

Published as a conference paper at ICLR 2026
Qwen Team. Qwen2.5-VL, January 2025. URL http:://ween.n.gittbbb...o. qwen2.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention all you need. NeurIPS, 30, 2017.
Andrey Voynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman, p+: Extended textual condition- ing in text-to-image generation. arXiv preprint arXiv:2303.09522, 2023.
Xierui Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang. MS-Diffusion: Multi-subjeet
zero-shot image personalization with layout guidance. In JCLR, 2025.
Shaojin Wu, Mengqi Huang, Wenxu Wu, Yufeng Cheng, Fei Ding, and Qian He. Less-to-more generalization: Unlocking controllability by in-context generation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 18682-18692, 2025.
Zhi-Fan Wu, Lianghua Huang, Wei Wang. Yanheng Wei, and Yu Liu. MultiGen: Zero-shot image generation from multi-modal prompts. ECCV, pp. 297-313. Springer, 2024.
Guangxuan Xiao, Tianwei Yin, William T Freeman, Frédo Durand, and Song Han. FastComposer: Tuning-free multi-subject image generation with localized attention. IJCV, pp. 1-20, 2024.
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. IP-Adapter: Text compatible image prompt adapter for tex--to-image diffusion models. arXiv preprint arXiv:2308.06721 2023.
Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Tianyou Liang, Guanying Chen, Shuguang Cui, and Xiaoguang Han. MVImgNet: large-scale dataset of multi-view images. In CVPR, 2023.
Lvmin Zhang, Anyi Rao, and Mancesh Agrawala. Adding conditional control to text-to-image diffusion models. ICCV, pp. 3836-3847, 2023.
Yuxuan Zhang, Yirui Yuan, Yiren Song, Haofan Wang, and Jiaming Liu. Easycontrol: Adding efficient and flexible control for diffusion transformer. arXiv preprint arXiv:2503.07022, 2025.
Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan- Yee K Wong. Uni-ControlNett All-in-one control to text-to-image diffusion models. NeurlPS, 36: 11127-11150, 2023.
13

---

Published as a conference paper at ICLR 2026
APPENDIX
A LIMITATIONS DISCUSSION
First, similar to Token Verse (Garibi et al. 2025), our method may fail when two customized concepts have the same name in the prompt- for example, generating image based on prompt where the word "dog" appears twice. Second, when personalizing more than three concepts simultaneously. method tends to deviate from the concept conditions. At the time of submission, trained only on single-concept data, which may be the primary cause for the performance drop in multi-concept customization with an extreme number of concepts. Additionally, our framework relied solely on the CLIP image encoder for image feature extraction, which captures high-level features but tends lose low-level object details. We present a potential solution: we synthesized training data involving two concepts (abstract and object) using the FLUX model, and combined it with the open-source MuSAR- -eR (Guo et al., 2025) multi-concept dataset, resulting in total of 52,000 multi-concept samples for training. Furthermore, to mitigate the loss of low-level object details, we concatenated the VAE latents concept images with the noise latents as additional conditions. We also introduced UnoPE (Wu et al., 2025) positional encoding to distinguish VAE latents from multiple reference images and applied lightweight LoRA fine-tuning to the joint attention layers in DIT. We present qualitative results for customizing 4 to 5 concepts in Figure 10 We leave addressing these limitations as future work.
model was
B MORE RELATED WORKS
Adapter: Adapters offer an efficient paradigm for customized generation. Several adapter- -bsed methods (Zhang et al. 2023] Mou et al. 2024 Ye et al. 2023 Zhao al., 2023 Li et al. 2024
2023) employ lightweight modules for task speccfcc adaptation while keeping the foundation model weights frozen. For example, ControlNet (Zhang et al. 2023) proposed trainable copy module of its U-Net-encoder to incorporate spatial control conditions (e.g., edge maps). IP-adapter (Ye et al. 2023) proposed a decoupled cross-attention module to enable the image prompt control. Based on it, MiP-Adapter (Huang et al. 2025) further introduces weighted-merg mechanism to alleviate the subject confusion problem in multi-subject personalized generation. In this paper, we propose a modulation space adapter module, Mod- Add-Ar,pter, to predict concept-specific modulation directions while keeping the parameters of the pre-trained DiT backbone frozen.
C MORE IMPLEMENTATION DETAILS
TTam 2025 as the vision- model (VLM) for attribute caption and image caption. We train the model using the Adam W (Kingma & Bal 2014) optimizer with learning rate of x 10 on 8x 80GB GPUs. Mod-Adapter is first pre-trained for 50K steps with a batch size of 32 without being integrated into the DiT model, followed by an additional 126K steps of further training with batch size of 1. Our method has the advantage of supporting multi-concept personalization at inference time while training only on single-concept data, especially given the difficulty of obtaining high-quality multi- concept training data. This ability stems from the
We use Qwen2.5-VL
localized and composable nature of the DiT modulation space. The scale factor s in Eq. 2 is set to 1 during training and testing.
D USER STUDY DETAILS
Fig. shoow the user study interface for evaluating single- (a) and multi-concept (b) personalization. To assess concept preservation, participants were asked to rate the similarity between each generated concept and the corresponding concept in the reference image. For multi-concept cases, each concept was evaluated separately, and the final score was calculated by averaging the scores across all concepts. Participants selected from five options, each corresponding to score from to 5: "Very inconsistent" (I), "Somewhat inconsistent" (2), "Fair" (3), "Quite consistent" (4), and "Very consistent" (5). To assess prompt fidelity, participants evaluated how well the content of the generated image matched the given text prompt, using the same five-point scale.
14

---

Published as a conference paper at ICLR 2026
Reference Image: toy Iay ducki
20.
Reffrrnne Image Icpow
Taheroro Image 2iightig
Generated image: Trextual tebric)
Generpdd image. Thexal pianic a bay pressed red sancing with poss street A byhe?
Vay
COPLAMT
Very
mocraseen
Scmestul
Far
ta conmiitent
Vary conmialent
peciue
Please evtrurte the cimiliaity between the duck gerersted Image ant the ons in infreence tragge
Please ovaluano how wall the content the generited inaee mandhis prondde tostal promett.
sukies the
Single Concagtt
Conosst
Figure 5: Screenshot of our user study rating interface. (a) Single-concept personalization evaluation. (b) Multi-concept personalization evaluation.
Table 4: More quantitative ablation results.
Multi concept
single-concepi
Methods
CP
PF
CP-PF
CLIP-T
CP
PF
CP.PF
CLIP-T
Ours (N Lexert==)
0,40
0.54
0.22
0.272
0.36
0.48
0.17
0.278
Ours (N _expert=4)
0.43
0.59
0.25
0.305
0.40
0.78
0.31
0.310
Ours (N exxxeet=88)
0.57
0.65
0.37
0.310
0.50
0.79
0.40
0.308
Ours (N (2_exper==12)
0.70
0.89
0.62
0,330
0.61
0.89
0.54
0.315
Hierarchical Clustering [Neppert=112 12)
0.72
0.63
0.331
0.62
0.54
E BROADER IMPACTS
The development of the proposed versatile multi-concept personalization generation technique holds potential for broad societal benefits, such as facilitating personalized image creation, storytelling, poster design, and other creative applications. However, this technology may also raise concerns about potential negative societal impacts, such as its misuse to generate misleading or deceptive content, enabling style-specific personalization that may infringe upon artists' rights. These risks be mitigated through techniques such as watermarking generated images and developing robust deepfake detection algorithms (e.g. (Cai al. 2025b Hawkins al. 2025) for content authentication.
F
MORE ABLATION ANALYSIS
The number of experts and the clustering method. In our MoE routing design, we suspect that concepts within the same cluster may share similar mapping patterns and should therefore be handled by the same expert. Based this assumption, we the number of clusters equal the number of experts, such that each expert is responsible for processing concepts of multiple categories within one cluster. This design choice empirically validated by our ablation study. Besides, we conducted experiments to investigate the impact of the number of clusters/experts (N_ (N_experts) and the choice of clustering method (k-means vs. hierarchical clustering). The results are shown Tab As observed, increasing the number of experts (i.c., the number of clusters) leads to improved model performance. We used 12 experts our setup, the maximum that could be accommodated simultancously within the 80GB GPU memory constraint. To accommodate more experts, the CPU offloading technique be adopted at the cost additional offloading overhead. Additionally, the choice between hierarchical and k- k-means clustering has negligible impact performance.
K-means MoE Routing, In the ablation study, we design a variant that employs a learnable linear gating network for MoE routing. This variant exhibits suboptimal performance, possibly due the
15

---

Published as a conference paper at ICLR 2026
Table 5: Licenses for released assets
Asset
License
FLUX.1 I [dev]
Apache-2.0 license
Qwen2.5-VL Apache-2.0 license
undlizttilization of certain experts. We further analyze the expert utilization pattern in this section and observe that several experts are underused, which is equivalent using fewer experts than our full model. We visualize the expert assignment patterns of the MoE layers in the 1st and 2nd blocks of Mod-Adaptes during training in 6 and (b), respectively. The training set contains 106, 104 samples, and for each sample. Mod-A Mod-Adapter predicts N 57 modulation directions, resulting in 6,047,99 total inputs (106, 104 57) to each MoE layer per training epoch. Each bar in Fig 6 indicates the percentage of inputs routed to each expert. As shown, Expert 7 (3.8%) and Expert (4.1%) in the first Mod-Adapter block, as well as Expert (1.1%) in the second block, receive significantly fewer imputs during training. This may lead to slower training of these experts, which in turn makes them less likely to be selected in subsequent iterations, resulting in their insufficient utilization. In contrast, our parameter-free k-means-baset routing mechanism pre-defines expert assignments before training. based on k- k-means clustering over the neutral features of all concept words in the training dataset, helping ensure sufficient utilization of all experts throughout the training process as shown in Fig (c).
Assigement
/9:02-
60000
J000/0
Figure 6: Distribution of Expert Assignment. Each bar indicates the percentage of inputs routed to each expert.
G ADDITIONAL QUALITATIVE RESULTS
In this section, we present additional qualitative results of our method (see Fig.
and
Fig. 12
H LICENSES FOR RELEASED ASSETS
This work uses the listed projects in Tab. released under their licenses. We strictly adhered to their license requirements; the original projects' copyright notices and license texts can found their official repositories.
16

---

Published as a conference paper at ICLR 2026
Input Concept
surface
made with surface on wood table
house made with surface forest
bridge made with surface over a muddy river
made surface parked display stand
Input Concept
style
The Eiffel Tower in image style
wedding image style
sleepy mountain vilage
A clessroom style
Input Concept
color tone
bridge over image with color tone
picture on mountain with a tone
room, image with color tone
image with color tone
Input Concept
light
The Eiffel Tower under light.
A boll of lightning with
with light
A bed in the under
light
light
Figure 7: More single-concept personalization results of our method. Colored words in the prompt indicate concepts to be personalized; underlined text highlights elements reflecting prompt alignment.
Concepts
A made of light
Ihs coos under light
surisce on graze
surfice on the fnor
pose
style
Fight
bay with poss Imoge ttyle
boy poss under
light
dog
Adoo woating sunglascoo
Addn in Boary with light
Figure More multi-concept personalization results of method. Colored words in the prompt indicate concepts to be personalized; underlined text highlights elements reflecting prompt alignment.
17

---

Published as a conference paper at ICLR 2026
style Inpui Concept
Jspur Concept
A dog made with surface road.
Jupur Concept
backpack made with surface bunch.
A living room style.
light
color
pose
living image in color tume.
a boy with pose on blanket.
under light.
pose
girl with on
Ixpur Concent
Jnpur Concent
Jnpur Conreor S..k
lon the seene.
SCE90
Scene
There as toy the
Car Impost Concepps
VAM
vase cup with surracee
Inpai Concepts
Inpust Concepols
surface
light
box made with surface.
under light.
toy
By the ses, there is boy pose.
Input Concepts
Input Concepty
Inpurt Concepis
surface
bagg
Inpus Concepts
Input Conzeptr
hat made with surface and teapot the beach, dog.
On D seome, there a hag near
Figure 9: More qualitative examples of our method on abstract concept personalization. Colored words in the prompt indicate concepts to be personalized, including abstract concepts and their combinations with object concepts.
18

---

Published as a conference paper at ICLR 2026
Lado that tmselt pladie toy.
plantic oocppet wade
De bàke sade ligit placlie
TELL
10f, putaaran aurfice
plalioadd
Figure 10: The qualitative results of our method for customizing an extreme number of concepts (4 to 5 concepts). Colored words in the prompt indicate concepts to be personalized.
cal
surface
hat
Inpurt Concents
Input Concegtts
cat with surface.
made surface.
Figure 11: Qualitative results of combining abstract concepts and concrete objects. Our method effectively decouples abstract concepts and concrete objects and successfully recombines them. Colored words in the prompt indicate concepts to be personalized.
Input Concepts
weather
teapot
weather
plushie
weather, teapot and plushic by lake.
In weather, teapot and plushie the road.
Input
Fog
toy
Concapts
Surronnded fog, there plushic and toy the road.
In frent of fog, plushie and toy wooden table.
fog
plushie
Figure 12: Qualitative results demonstrating the generalization of our method to concept categories completely unseen during training. Colored words the prompt indicate concepts to be personalized.
19


Reviews:
[{"review_id": "GAlQWxMXGA", "reviewer": "Reviewer_Bc9H", "summary": "This paper proposes a tuning-free multi-concept personalized image generation method called Mod-Adapter, which can simultaneously customize specific objects and abstract visual concepts. Based on the modulation space of the pre-trained Diffusion Transformer, the method predicts concept-specific modulation directions via lightweight adapters and incorporates vision-language cross-attention and MoE mechanisms for efficient feature mapping. The authors also design a VLM-guided pre-training strategy to mitigate training difficulties and extend the evaluation benchmark DreamBench-Abs.", "questions": "See above.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- Achieving tuning-free multi-concept personalization. Unlike previous methods, such as TokenVerse that require fine-tuning small MLPs for each new concept, Mod-Adapter is trained once to be universally applicable to all concepts. During inference, only reference images and concept words need to be input without any optimization steps.\n- Extensive experiments are conducted on several datasets. The proposed method achieves good performance in multi-concept personalization.", "weaknesses": "- The proposed Mod-adaptor is depedent on the training data. Mod-Adapter needs to be trained on datasets containing abstract concepts, synthetic data + MVImgNet + AFHQ. If the user concepts fall outside the training distribution, its generalization ability is questionable.\n- The performance degrades when customizing more than three concepts simultaneously, which limits its application in extremely complex scenarios.\n- Why in Table 1, the performance of Mod-adaptor is 0.61, which is lower than previous Emu2 method?", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MO6txQpRNF", "reviewer": "Reviewer_5SAg", "summary": "This paper presents Mod-Adapter, a tuning-free framework for versatile multi-concept personalization in text-to-image generation that can customize both object and abstract concepts without test-time fine-tuning. The method leverages the localized and semantically meaningful properties of the modulation space in pre-trained Diffusion Transformers (DiTs). The key contributions include: (1) a novel Mod-Adapter module that predicts concept-specific modulation directions using vision-language cross-attention for feature extraction and Mixture-of-Experts (MoE) layers for mapping to modulation space; (2) a VLM-guided pre-training strategy to facilitate training; and (3) an extended benchmark DreamBench-Abs incorporating abstract concepts. Experimental results show state-of-the-art performance in multi-concept personalization.", "questions": "1. What is the computational overhead of the method during inference compared to baseline approaches? Have the authors measured inference time and memory usage?\n2. The paper mentions performance degradation with more than 3 concepts. Could the authors analyze this limitation more thoroughly? Are there specific types of concepts that are more prone to causing interference?\n3. How does the method handle concept drift or domain shift? For example, would it maintain performance when applied to concepts from domains not represented in the training data?\n4. Could the authors provide more insights into the MoE routing mechanism? Why does k-means clustering perform better than learnable gating networks? How is balanced expert utilization ensured?\n5. What is the effect of varying the number of DiT blocks (N) on performance? The paper uses all 57 blocks from FLUX, but would a subset be sufficient?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "- The paper addresses an important and underexplored challenge in multi-concept personalization - the simultaneous customization of both object and abstract concepts without test-time fine-tuning. This represents a significant practical limitation of existing methods that the authors successfully tackle.\n- The insight to leverage the localized and semantically meaningful properties of the DiT modulation space is particularly clever. This approach enables the additive combination of multiple concept modulations without interference, which is crucial for multi-concept personalization.\n- The creation of DreamBench-Abs by incorporating abstract concepts into the standard benchmark is a valuable contribution to the research community. This enables more rigorous evaluation of methods claiming to handle diverse concept types.\n- The authors provide extensive quantitative, qualitative, and human evaluation results demonstrating the superiority of their method over state-of-the-art approaches. The ablation studies effectively validate the contribution of each proposed component.\n- The tuning-free nature of the approach makes it highly practical for real-world applications where test-time fine-tuning is infeasible or impractical. The ability to handle both object and abstract concepts significantly expands the range of possible applications.", "weaknesses": "- The Mod-Adapter contains 1.67B parameters, which is large for an adapter module. This raises questions about the practical deployment of the method, especially in resource-constrained environments. The paper does not adequately address this concern.\n- The paper does not sufficiently explore the generalization capabilities of the method to unseen concept types or combinations. The ablation studies focus on component removal but lack analysis of performance under varying conditions.\n- The paper mentions that performance degrades with more than 3 concepts, but does not provide a detailed analysis of this limitation or potential solutions. This represents a significant practical constraint.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Iftx7xYQzx", "reviewer": "Reviewer_4ZPx", "summary": "This paper addresses the limitations of existing multi-concept personalized text-to-image generation methods. The authors propose MOD-ADAPTER, a tuning-free framework that can effectively customize both object and abstract concepts, with the use of vision-language cross-attention, mixture-of-experts, and a VLM-guided pre-training strategy. The authors also extend the DreamBench benchmark to include abstract concepts and validate the method through quantitative and qualitative comparisons and user studies, showing state-of-the-art performance.", "questions": "1.\tWhat’s the ratio of concepts in evaluation that are not used in pre-training and training? For these unseen concepts, please provide comparisons.\n\n2.\tPlease refer to the weaknesses.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1.\tThe paper addresses the limitations of existing methods by proposing a tuning-free framework that can handle both object and abstract concepts. The extended benchmark is valuable for further studies.\n\n2.\tThe integration of Vision-Language Cross-Attention and MoE layers in the MOD-Adapter module is well-justified and effective, proven in ablation studies.\n\n3.\tExperimental results underscore the effectiveness of the proposed method.", "weaknesses": "1.\tWhile MOD-ADAPTER introduces a tuning-free improvement, the core idea of leveraging the DiT modulation space for multi-concept personalization builds on TokenVerse. \n\n2.\tThe paper seems to claim its ability to deal with unseen concepts (neither used in training nor pre-training), however, the experiments are not split based on whether the concept is used in training. Besides, I doubt that some designs of the method, like the number of experts in MOE, may be highly dependent on the pre-training and training data.\n\n3.\tThe main difference between cross-attention and the proposed Vision Language Cross-Attention is the usage of CLIP image encoder and text encoder, but in Figure 2, these two encoders are outside of the VL attention block, which is confusing.\n\n4.\tA common advantage of tuning-free frameworks is fast inference speed, which is also claimed in your paper. However, I don’t see any time complexity analysis in the experiments to prove it.\n\n5.\tThere are misunderstanding typos in the paper, such as “w/o k-means” in Figure 4.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oXLiU0EsGl", "reviewer": "Reviewer_Vhnb", "summary": "The author propose a tunning-free personalization method for abstract concepts such as pose, lighting and surface by an innovative module Mod-Adapter. Also, they extend the DreamBench to form a new benchmark DreamBench-Abs for evaluating baselines and Mod-Adapter", "questions": "- Is it possible to combine abstract concepts and concrete objects? The authors claim that this approach effectively decouples abstract concepts and concrete objects. So, for example, is it possible to achieve an effect like \\<img1\\> surface \\<img2\\> cat?\n\n- What is the approximate proportion of abstract concept data in the training dataset? How many types of abstract concepts are inclued in the dataset?\n\nMinor questions:\n\n- Which VLM do authors use for evaluating CP and PF?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "- This work is the first to introduce the 'mod' concept from TokenVerse into tuning-free abstract concept personalization.\n- This work uses a reasonable design for the Mod-Adapter and VLM-guided pre-training.\n- The Mod-Adapter shows good results on sample examples.", "weaknesses": "- Limitation in qualitative results: This paper only shows a very limited number of abstract concept customization. Many abstract concepts are repeatedly displayed in different figures, which greatly reduces the reliability of the method's effectiveness. I am not sure the success rate of this method is beyond these examples. Can authors show more examples, even report the fail rate? If not, is it beacuse the distribution of training dataset?\n- Insufficient motivation for using MoE: Authors discuss the reason for using MoE instead of simply MLP, do authors have any experiements to show how the mapping different between objects? Do the difference happen between objects (cat, dog, $\\cdots$) and abstract concepts, or there are more complex principle for the mapping pattern gap? More discussion and experiments here might enhance the motivation for using MoE.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)