CoCoIns: Consistent Subject Generation via Contrastive Instantiated Concepts
Authors: Lee Hsin-Ying
Kelvin C.K. Chan
Ming-Hsuan Yang
Venue: Accepted by TMLR ()

Abstract: While text-to-image generative models can synthesize diverse and faithful content, subject variation across multiple generations limits their application to long-form content generation. Existing approaches require time-consuming fine-tuning, reference images for all subjects, or access to previously generated content. We introduce Contrastive Concept Instantiation (CoCoIns), a framework that effectively synthesizes consistent subjects across multiple independent generations. The framework consists of a generative model and a mapping network that transforms input latent codes into pseudo-words associated with specific concept instances. Users can generate consistent subjects by reusing the same latent codes. To construct such associations, we propose a contrastive learning approach that trains the network to distinguish between different combinations of prompts and latent codes. Extensive evaluations on human faces with a single subject show that CoCoIns performs comparably to existing methods while maintaining greater flexibility. We also demonstrate the potential for extending CoCoIns to multiple subjects and other object categories.

Published in Transactions on Machine Learning Research (11/2025)
CoColns: Consistent Subject Generation via Contrastive Instantiated Concepts
Lee Hsin-Ying University of California, Menced
hlee3077@numerree.cee edu
Kelvin C.K. Chan Google DeepMind
keloknckchan&Sggggge..ccom
Ming-Hsuan Yang University of California, Merced
mbyang@uumerred..de
Reviewed OpenReview: https: // opwnreviee.e.eee.e. net/ forum? id= fPZYDNLOS
A [o] woman
A [x] man
Figure 1. Contrastive Concept Instantiation (CoCoIns) is generation framework that achieves subject con- sistency across multiple independent generations without fine-tuning or reference images. Unlike prior work that requires customization fine-tuning, adopts additional encoder for references, generates images in batches, Co- Colns creates instances of concepts with unique associations connecting latent codes to subject instances. Given latent code (o/x), CoColns converts it into pseudo-word ([o]/[x] that determines the appearance of subject concept. By reusing the same code, users can consistently generate the same subject instances across generations.
Abstract
While text-to-image generative models can synthesize diverse and faithful content, subject variation across multiple generations limits their application long-form content generation. Existing approaches require time-consuning fine-tuning, reference images for all subjects, access to previously generated content. We introduce Contrastive Concept Instantia- tion (CoCoIns), framework that effectively synthesizes consistent subjects across multiple independent generations. The framework consists of generative model and a mapping network that transforms input latent codes into pseudo-words associated with specific con- cept instances. Users can generate consistent subjects by reusing the same latent codes. To construct such associations, we propose contrastive learning approach that trains the net- work to distinguish between different combinations of prompts and latent codes. Extensive cvaluations human faces with single subject show that CoCoIns performs comparably to existing methods while maintaining greater flexibility. We also demonstrate the potential for extending CoColns to multiple subjects and other object categories. The source code and model weights are available at htt////otrraiiiieooeeeereeeeeepe
github. io.

---

Published in Transactions on Machine Learning Research (11/2025)
1 Introduction
Text-to-image generation has made remarkable advances (Rombach et al., 2022; Saharia et al., 2022; Podell et al., 2023), opening up numerous downstream applications, including editing and style transfer. Among these applications, maintaining subject consistency has been long-standing challenge in long-form content creation, including storytelling (Li et al., 2019), comics (Wu el al., 2024a), and movie generalion (Tulyakov et al., 2018; Polyak et al., 2025). These applications consist of sequences of images and clips, where consistent characters and objects facilitate recognizing subjects across scones and following narratives.
While numerous approaches have been explored to ensure subject consistency, they are often labor-intensive or time-consuming. One straightforward approach is to gather all existing generations and manually swap generated subjects with reference subjects using face swapping (Nirkin et al., 2019; Bitouk et al., 2008). Another direction is to customize generators by optimizing virtual word tokens fine-tuning model weights to represent reference subjects and produce new generations (Gal et al., 2023a; Ruiz et al., 2022). To reduce the overhead of tuning generators, recent methods (Wei et al., 2023; Ye et al., 2023) incorporate additional encoders that convert references into representations. However, these methods still require users to prepare references for all subjects.
In contrast to addressing each generation individually, one can generate all images in a batch, allowing samples within the batch to interact and achieve consistency (Tewel et al., 2024; Zhou et al., 2024; Liu al., 2025). Specifically, the target prompts are merged into the same batch, and the latents of all samples are processed together, allowing subjects within the batch to converge toward a similar appearance. Although these approaches achieve promising results, they require storing generated outputs to recreate the same subjects in the future.
We propose a generation framework that maintains subject consistency across individual generations with- out manual swapping, fine-tuning, or preparing references. Building such framework presents numerous challenges. While we aim to generate a consistent subject appearance from concept, preserving diversity among all instances of the concept remains important. Since the generator is already trained and exhibits high diversity and generalizability, we need to strike a balance between minimizing variation among the same subject instances across individual generations while maintaining diversily between different instances. Addi- tionally, collecting large-scale, high-quality data organized by subjects is challenging. Training the generator directly on annotated subjects in low-quality datasets could hamper both output quality and diversity.
To minimize variation among instances of the same subject while preserving diversity among different in- stances, we introduce a latent space to model the distribution of instances for each concept. The proposed method is motivated by common practices in natural and programming languages. If I user provides sulficient descriptions that encompass every intricate aspect of a concept, the generator may be able to consistently output the same appearance. Although covering comprehensive details is impractical with the limited vocab- ulary of human language, prior work on customizing generative models has shown the efficacy of pseudo-words (Gal et al., 2023a; Ruiz et al., 2022), which can convey essential information to represent particular subject instances. Our framework is built upon instantiating concepts (Anderson al., 1976; Dershowitz, 1985) via pseudo-words. words. We associate codes in the latent space with specific concept instances in the output space, if creating instances identified by latent codes. These latent codes are embeddings sampled from the space, taken as input by the generation framework, and transformed into heudo--ords that guide the generator to synthesize specific instances.
To establish the association between input latent codes and output subject instances, we develop a light mapping network that converts latent code into a pseudo-word, which then combined with a concept token to represent a specific instance of the concept. We then develop a contrastive learning strategy to train the mapping network. Instead of relying on subject annotations, the model learns to differentiate latent codes by comparing its own outputs generated from various combinations of prompts and latent codes. This self-supervised paradigm enables scalability and generalizability while avoiding the need to learn directly from limited data.
We refer to our generation framework as Contrastive Concept Instantiation (CoCoIns). As illustrated in Figure 1, given a concept in a prompt indicating subject (e.g. woman), the framework creates an instance of that concept by transforming a sampled latent code into a porddo-word (e.g. [o] in the example prompt)
2

---

Published in Transactions on Machine Learning Research (11/2025)
that describes the concept. Each latent code and its transformed pscudo-word is uniquely tied to a specific instance and can be utilized for future generations. Different latent codes yield different instances, showcasing the preserved output diversity.
We conduct experiments on human images and perform systematic evaluations, including generating portrait photographs and free-form images. We achieve favorable subject consistency and prompt fidelity compared to aatch-generation approaches. We also demonstrate early success in extending our approach to multi-subject and general concepts. The main contributions of this work are:
To the best of our knowledge, we propose the first subject-consistent generation framework for multiple independent generations without fine-tuning or encoding references.
We develop a contrastive learning method that avoids reliance on limited subject annotations while preserving output quality and diversity.
We perform extensive evaluations and demonstrate favorable performance compared to approaches that require time-consuming fine-tuning or batch generation.
2 Related Work
Subject-Driven Generation. One approach to subject consistency is based on subject-driven generation, which aims to generate customized content according to user-provided input. By learning new tokens (Gal et al., 2023a; Voynov et al., 2023; Tewel et al., 2023) or model weights (Ruiz et al., 2022; Kumari et al., 2023; Han et al., 2023; Ruiz et al., 2024), pretrained generative models can be customized to produce outputs based on specific references. Textual Inversion (Gal et al., 2023a) learns virtual tokens that capture subject information inverted from reference images. DrcamBooth (Ruiz et al., 2022) finc-tunes the parameters of pretrained models and learns unique identifiers that represent references. While subject consistency can be achieved by customizing each target concept with user-provided references, these methods are time- consuming as they require fine-tuning for every subject.
To reduce the time and computational cost of luning-based methods, another line of research incorporates additional encoders to obtain representations from reference images, which generative models take as con- ditions via augmented prompt embeddings (Wei et al., 2023; Shi et al., 2024; Xiao et al., 2024; Li et al., 2023; Wang et al., 2024a; He et al., 2024; Chen et al., 2023; Gal et al., 2023b; Avrahami et al., 2023), self- attention (Ding al., 2024; Wang al., 2024b), or cross-attention (Wei et al., 2023; Shi et al., 2024; Jia et al., 2023; Ye et al., 2023; Wang et al., 2024b; 2025). In addition, some approaches (Valevski et al,, 2023; Wang et al., 2024d; Li et al., 2024; Peng et al., 2024; Papantoniou et al., 2024; Wu et al., 2024b; Yue et al., 2024) focus solely on specific domains, e.g. human faces, in exchange for adopting more powerful encoders dedicated to those domains, e.g. face recognition models (Deng et al., 2022). While encoders case the tuning process, users still need to prepare reference images for all target concepts. Designing specific mechanisms to insert reference features into generative models also necessary. In contrast. our approach operates in the text embedding space, offering the potential be applied to a wide range of text-based generative models.
Subject- Conisstttnniitettt Generation. While subject-driven generation produces new generations featuring the same subjects as references, another line of work focuses on generating a set of images with consistent subjects from a set of prompts. The Chosen One (Avrahami et al., 2024) iteratively selects a cluster of similar images and fine-tunes the model using that cluster. Consistory (Tewel et al., 2024), JeDi (Zeng et al., 2024), and Story Diffussin (Zhou et al., 2024) uuilize the self-attention features of all samples in a batch. However, these approaches are less flcxible as they require access to other samples or fcatures when performing new generations. IPrompt (Liu et al., 2025) consolidates all prompts into a single lengthy prompt in a specific format, where multiple context settings for subject follow single description of that subject. This format limits the expressiveness of the prompts. In contrast, we achieve subject consistency with greater flexibility by treating each generation independently while retaining complete prompts.
Storytelling. Generating coherent stories (Li et al., 2019) requires maintaining character consistency over time. Some approaches (Rahman et al., 2023; Pan et al., 2024; Liu et al., 2024; Shen et al., 2025; Maharana et al., 2022) utilize cross-attention to access information from previous frames and prompts. Others introduce memory bank (Maharana et al., 2021), perform auxiliary foreground segmentation (Song et al., 2020), or
3

---

Published in Transactions on Machine Learning Research (11/2025)
(a) Anchor
Insert
*
Wi
man on
DDIM
Wi
soccer held
nolsy image
code
pseudo word
modulated prompt
predicted noise
predicted image
pull
(b) Positive
"An image captures man playing.... ..."
Wi
(c) Negative
man field
E1
42
Wz
Mapping
Denoiser
Taaned Froenn
Figure Overview of Contrastive Concept Instantiation (CoColus). We develop contrastive learning approach to build associations between input latent codes and concept instances. For each training image, we generate two descriptions and randomly sample two latent codes zı and zg. The mapping network transforms the latent codes into pseudo-words wi and We then construct triplet of combinations of descriptions and latent codes. build (a) an anchor sample with description embedding e" modulated by inserting wi before the target concept token, (b) positive sample ex with similar description modulated with wi, and (c) negative sample e with the same prompt as anchor but modulated with different pseudo-word w2. The network trained with a triplet loss to differentiate approximated images 4², it and à from denoiser predictions é", ēt, and ē
generate particular reference characters (Gong et al., 2023; Shen & Elhoseiny, 2023; Wang et al., 2024e). These methods often involve training on storytelling datasets and focus on generating complete stories or continuing existing ones. Our approach emphasizes equipping existing generative models with the ability to maintain subject consistency.
StyleGAN. Our generation framework shares similar insights with StyleGAN (Karras et al., 2019; 2020). Both methods use mapping network to transform input latents into intermediate and more disentangled latent space. The space in StyleGAN enables better control over generated image attributes by modulating the generator through adaptive instance normalization (Huang & Belongie, 2017). In our framework, the intermodiate latents operate in the same space as text embeddings, enabling better manipulation of subject appearances via lext conditions. CharacterFactory (Wang el al., 2024c) also learns a mapping network that maps random noise virtual tokens associated with human names in celebrity dataset. These virtual tokens produce consistent identities but lack control over semantics and attributes.
3 Methodology
Our goal is maintain subject consistency across individual generations without time-consuming fine- tuning labor-intensive reference collection. We introduce Contrastive Concept Instantiation (CoCoIns). generation framework that models concept instances in a latent space and uniquely associates latent codes with output concept instances via contrastive learning. We introduce the base text-to-image model in Section 3.1, then describe the framework in Section 3.2 and the contrastive learning strategy in Section 3.3.
3.1 Text-to-Image Diffusion Models
We explore subject consistency in the context of text-to-image generation and base our approach on a latent diffusion model (Rombach et al., 2022; Podell al., 2023). We utilize a pretrained text-to-image model comprising an autoencoder (Kingma & Welling, 2014), a text encoder (Radford et al., 2021), and a denoiser. Given image 1 and prompt P, obtain the latent image representation and prompt embedding e
4

---

Published in Transactions on Machine Learning Research (11/2025)
via the autoencoder and text encoder, respectively. The denoiser co reverses the diffusion process:
3t Va,e x + vI a,e, e e~ E~
(1)
where our E 17 is a decreasing sequence, by predicting é from the noisy image Tt, prompt e, and timestep l:
= ea (2t, e,t)
(2)
3.2
Instantiating Concepts
To generate consistent instances of a concept, we model the distribution of instances in a latent space and associate latent codes in the space with concept instances in output images. As illustrated in Figure 2, mapping network takes a latent code as input and produces pseudo-word, which conveys the necessary descriptive details to create a particular concept instance. The framework thus achieves subject consistency by generating the same subject with a fixed latent code across multiple generations.
The proposed mapping network transforms a latent code into a virtual word token, which is then inserted into the prompt embedding and guides generation alongside other words. Let e E Raxd denote the embedding of prompl P obtained via dictionary lookup, where is the sequence length. The concept token that user wants to generate consistently (e.g. man in Figure 2) is at location i, Given latent code E R the mapping network f R + Rd produces pseudo-word embedding w B that represents an instance:
w f()), ~~N(0,1
Then we insert the output w into the prompt embedding e at location before the concept token and obtain the modulated prompt embedding è:
è = insert(e, w.
(4)
where insert denotes the insertion operation. The modulated prompt embedding é is further encoded by the text encoder and serves as the text condition during generation.
3.3 Contrastive Association
We aim to establish unique associations between input latent codes and pscudo-words that represent visual instances in the output images, such that a latent code can be reused to generate the same concept instance.
A naive way is to train the network f to synthesize subjects from a dataset with identity annotations, such as face recognition datasots (Huang et al., 2008; Liu et al., 2015). However, we empirically find that training with common noise prediction (Ho et al., 2020) often compromises the generalizability and output quality of the generator, as these datasets are typically collected from data domains that are much narrower than the pretraining data of the generator. The network learns synthesize subjects from these datasets, but it may overfit to the limited distributions. Thus, we develop contrastive lcarning approach that does not require ityntity annotations, allowing us to train the mapping network in a self-supervised manner.
Constructing Triplets. We prepare multiple combinations of prompts and latent codes. As illustrated in Figure 2, the same prompts (e.g. "a man on a soccer field ...") are paired with different latent codes, and a similar prompt (e.g. "an image captures a man playing soccer ...") paired with the same latent code. The network is trained to generate pscudo- words that, when inscrted into prompts, guide the synthesis of specific instances.
Specifically, we prepare image and triplet of prompts for each training sample. The triplet consists of (a) an anchor prompt, (b) a positive prompt, and (c) negative prompt. The anchor prompt is caption describing the image, modulated by a latent code. The positive prompt is another description of the image, modulated with the same latent code. The negative prompt has the same caption as the anchor prompt but is modulated with a different latent code. Formally, let e, and e2 denote the embeddings of two descriptions P1 and P2 of the image I. Here, 21 and 2₂ denote two different latent codes, and i and j are the locations of target concept tokens in er and e2, respectively. We create an anchor prompt e", a positive prompt et.
5

---

Published in Transactions on Machine Learning Research (11/2025)
and negative prompt e via
e' insert(e1. w1,t), et insert((e, insert(e1. W2,1),
w1 f(11); W2 f22)).
(5)
Then, we construct three noisy image latents to be paired with the three prompt embeddings. Since, in practice, a user may perform multiple generations with different initial noises, we obtain the noisy image latents by adding two different noises, 61 and E2, sampled from a normal distribution, to the image latent x to maintain subject consistency across multiple generations. We add the same noise the anchor and negative samples to create challenging scenario:
Vare açé1, x Voye + vI 0,E2, = Va-a + vI 0gg61
(6)
where of at, and o7 denote the anchor, positive, and negative noisy image latents, which are then paired with the modulated prompt embeddings e", et, and et, respectively, to form the inputs to the denoiser.
Building Association. To encourage the generative model to synthesize subjects associated with pscudo- words, we apply a triplet loss (Schroff et al., 2015) to the denoiser outputs é" ēt, and é of the anchor, positive, and negative samples, pulling the anchor and positive samples closer while pushing the negative sample away. Since consistency is meaningful only in image latents rather than noise, we first obtain the predicted image latents "", x+, and & with DDIM (Song al., 2021) approximation. Then, the distances between the three approximated latents are measured via
Loon = Ldis( x*))==+=+ + Xueg Lais , x-))
where Liis denotes distance function. Note that we empirically find that the common form of triplet loss with subtraction leads to less distinction between different input latent codes; thus, triplct loss is based the reciprocal of the distance between the anchor and negalive samples.
Subject Masks. Furthermore, since we only pursue subject consistency across images rather than similarity of entire images, we calculate the loss only in subject regions by applying masks to the output images. Subject masks can be obtained through an off-the-shelf referring segmentation model (Kirillov el al., 2023; Liu al., 2023b; Ren et al., 2024), which annotates pixels corresponding to input words. Let m denote the mask with boolean values that covers the pixels of target concepts. We replace Lais(s.)) with C in Eq. (7) to denote the masked distance function with mask m, where Liss (z, Lais (m z, m y).
Background Presorvation. With the aforementioned subject mask m, we negate the subject mask to obtain the background mask m = - m. The background preservation loss is defined to minimize the distance between the backgrounds of the images generated with and without the pseudo-words:
bback = L (+ .1) + Cii ( (22) + L
(8)
Here i and 22 are DDIM approximations of the denoiser outputs co e1, t) and co(x,, e2, e2, respectively. The final loss function consists of the contrastive and background preservation losses:
L= Acnn Leon + Aback Liack:
(9)
where Aeon and Aback are weights balancing the two losses.
4 Experimental Results
4.1 Implementation Details
Architecture. We implement the mapping network f as an n-layer MLP and leverage Stable Diffusion XL (Podell et al., 2023) as the text-to-image diffusion model. We train only the mapping network f, with all other model weights frozen.

---

Published in Transactions on Machine Learning Research (11/2025)
(b)
(d)
(e)
Figure 3. Qualitative comparisons Portraits from (a) StoryDiffusion (Zhou et al., 2024), (b) Consistory (Tewel et al., 2024), (c) 1Prompt1Story (Liu et al., 2025), (d) DreamBooth (Ruiz et al., 2022), and (e) CoColns. The left four columns generated with man the subject, and the right four with woman. We achieve subject consistency without generating images in batches or performing reference fine-tuning.
Negative Distance Weight Schedule. We empirically find that the distance between auchor and negative samples is often too large the beginning of training. The model tends to ignore the randomly initialized input latent codes and produces identical outputs. Therefore, we implement the weight of negative distance Aneg (k/K)8 as increasing function over training steps, where k and K are the current and total training steps, and y and 3 are hyperparameters.
4.2 Experiment Setups
We conduct comprehensive experiments on single-subject human faces with our approach, followed by mul- tiple subjects and other object categories. More details on data collection can be found in Appendix D.
Training. We train the mapping network using the CelebA dataset (Liu et al., 2015), which comprises 20K images and 10K identities. We generate prompts with the captioning model LLa VA-Next (Liu et al., 2023a) and masks with the zero-shot referring segmentation model Grounded SAM 2 (Kirillov et al., 2023; Liu et al., 2023b; Ren et al., 2024).
Evaluation. We design two prompt sets for comprehensive ovaluation:
This set evaluates face similarity with clear frontal faces. It contains 1K sentences composed of the template "A [subject] is looking at the camera.", where [subject] is one of {man, woman, boy, girl, person). Each subject contains 200 sentences, resulting in total of 1K samples.
Portraits:
Scenes: This set represents real-world performance with free-form prompls, where face poses and angles vary. It comprises 1K sentences generated by a Large Language Model (LLM) (Ouyang et al., 2022). We prompt the LLM to generate sentences of the same five subjects doing something in diverse situations, including four settings: daily lives, professional environments, cultural or recreational occasions, and outdoor activities, each with 50 samples.

---

Published in Transactions on Machine Learning Research (11/2025)
Quantitative performance on Por- traits. We achieve comparable consistency and bet- ter diversily compared to approaches that generate images in batches.
Table 1.
Table 2. Quantitative performance on Scenes. We achieve the best face similarity while maintaining similar subject diversity and prompt fidelity compared other methods. In addition, generale images independently, enabling high flexibility for future generations.
Simt
Divt
CLIPT
Simt
Divt
CLIPt
DSt
CelebA
0.590
0.992
0.299
Consistory 0.098
0.883
0.297
0.383
Cousistory
0.356
0.774
0.218
StoryDiffusion
0.159
0.814
0.290
0.407
StoryDiffusion
0.637
0.577
0.217
1Prompt
0.307
0.611
0.228
Ours 0.256
0.847
0.290
0.388
Ours 0.600
0.799
0.193
We measure subject similarity and diversity over Portraits and Scenes. For faces, we estimate face similarity (Sim) and diversity (Div) of cropped and aligned images. Face similarity is the pairwise cosine similarity of ArcFace (Deng et al., 2022) embeddings between images with the same identity. To estimate diversity, we first average the face embeddings for each identity. We then calculate the pairwise cosine similarity bet the averaged embeddings of all identities.
Since Scenes considers images in real-world applications where faces are not always clear and large, we calculate DreamSim (Fu et al., 2023) (DS), a learned perceptual distance aligned with human preference, for subject similarity, We also measure prompt fidelity for both sets using CLIP, which computes the cosine similarity between the projected embeddings of the CLIP text and image encoders.
Evaluated Methods. We evaluate our method against tuning-free subject-consistent generation and buning-based customization. Tuming-free schemes include Consistory (Tewel et al., 2024), StoryDiffu- sion (Zhou et al., 2024), and 1Prompt1Story (Liu et al., 2025). Since tuning- based methods require fine- tuning for all subjects, which incurs heavy computational costs, we use DreamBooth (Ruiz et al., 2022) as an example and present only quantitative results.
4.3 Empirical Results
Table shows the quantitative performance on Portraits, and Figure 3 displays two subjects, a man and woman, each with four images, generated by all approaches. We measure the similarity (Sim) and di- versity (Div) of the training dataset CelebA for reference. Although StoryDiffusion exhibits the highest similarity, even surpassing CelebA, its diversity remains low. Subjects generated from different batches with different initial noise converge toward similar appearance. Our approach achieves comparable similarity to Siffuuiff and CelebA while generating diverse subjects.
We demonstrate examples from Scenes in Figure 4 and present the quantitative performance in Table 2. We compare our approach against Consistory and StoryDiffusion because IPrompt1Story does not support free-form prompts. It operates with a specific prompt structure where a subject description is followed by multiple context descriptions. Our model performs favorably on face similarity while performing comparably diversity and fidelity.
4.4 Ablation Study
Consistency Loss. We analyze the elficacy of triplet. loss in maintaining consistency. We calculate face similarity and diversity on Portraits and evaluate prompt fidelity Scenes. Table 3 shows the performance of applying the loss of only positive distance (Pos), positive and negative distance (Pos + Neg), using the common form of triplet loss with negative distance subtraction (subtract), and setting the weight of negative distance as an increasing schedule over training iterations. The four settings yield similar prompt fidelity, while implementing negative distance, the reciprocal form, and the weighting schedule enhances performance.
Background Loss. We also evaluate the effect of removing background preservation loss under the same test sets. As shown in Table 4, using only consistency loss results in unsatisfactory face similarity and fidelity.
8

---

Published in Transactions on Machine Learning Research (11/2025)
(b)
(c)
(d)
Figure 4. Qualitative comparisons Scenes from (a) StoryDiffusion Zhou al. (2024), (b) Consistory Tewel et al. (2024), (c) DreamBooth Ruiz et al. (2022), and (d) CoCoIns. Prompt1Storry Liu et al. (2025) is absent because it requires specific prompt format with unified subject descriptions. The left and right four columns show two different subjects in diverse contexts. The prompts can be found in Appendix D.
Table 3. Performance of ablating consistency loss. We train the network with distance from the positive sample (Pos) and the reciprocal of the negative sample (Neg). Instead of common triplet loss with subtrac- tion of negative distances (subtract), minimize its ciprocal and apply weighting schedule that increases training iterations (Schedule). The final setting achieves the best face similarity and diversily with sim- ilar prompt fidelity.
Table 4. Performance of ablating background loss. addition to the triplet loss for consistency (Consistency Loss), we adopt segmentation mask (Mask) to control the variation region and a back- ground preservation loss (Background) to keep back- grounds close to the original predictions, thereby sig- nificantly improving face diversity and prompt fidelity while maint ainnng similar face similarity.
Simt
Divt
CLIP
Simt
Divt
CLIP1
Consistency Loss
0.444
0.395
0.138
Pos
0.394
0.500
0.293
Mask
0.619
0.352
0.128
Pos Neg
0.492
0.750
0.290
Mask + Background
0.600
0.799
0.290
Pos Neg (subtract)
0.380
0.444
0.294
Pos Neg + Schedule
0.600
0.799
0.290
While training with masked distances (+ Mask) without preserving backgrounds yields slightly higher face similarity, it results in low diversity and fidelity. In this setting, although the loss calculated only within masked regions, the model modifies non-masked regions, possibly due to the self-attention mechanism, which allows information to interact globally. Applying background preservation (+ Background) significantly improves diversity and fidelity.
Prompt and Noise Combinations. We compare strategies for constructing training triplets. We evaluate the performance of using the same two prompts creating noisy latent images with the same noise for anchor and positive samples. Table 5 shows that using different prompts and noise achieves the best similarity and nearly the same prompt fidelity.
Training with Subject Annotations. We show results from directly training the mapping network as a noise prediction problem on CelebA. Figure shows that the model learns maintain subject consistency, but the output quality is also affected by the limited, low-quality data.
9

---

Published in Transactions on Machine Learning Research (11/2025)
Table 5. Performance of prompt and noise com- binations for constructing training triplets. Com- pared to using the same prompts (=) or noise for the anchor and positive samples, using two different (*) prompts and noise yields the best face similarity and diversity with similar prompt fidelity.
BN
oan
Figure 5. Results of training the mapping network as a noise prediction problem. The nelwork overfits to the dataset and generales low-quality images.
Prompts
Noise
Simt
Divt
CLIP
*
0.548
0.686
0.290
0.306
0.772
0.292
0.600
0.799
0.290
Figure Results of general concept consistency. Our approach makes no assumptions about object cat- egories. It can potentially be applied to other concepts such as cats, dogs, and cars.
Figure Results of multi-subject consistency. Given two different latent codes, the model trained on single-subject images can maintain consistency for mul- tiple subjects.
4.5 Extensions
General Concepts. Since our approach imposes no constraints on subject classes, we also demonstrate that it can be applied to general concepts. We train the model with animal (Choi al., 2020) and car (Yu et al., 2015) images. The examples in Figure 6 show that the model can potentially be applied to more concept categories beyond humans.
Multi-Subject Consistency. In addition to analyzing single subjects, we demonstrato our ability to maintain consistency across images with multiple subjects. Figure contains two sets of examples of a man and a woman different settings. We use two different input codes for the two subjects and generate the images with the model trained on single-subject data. While the model has never seen two faces in an image, it can identify face regions and maintain consistency for multiple subjects. Despite some entanglement and influence between subjects, the results demonstrate the potential to extend the model to multi-subject scenarios.
5 Conclusion
In this work, we propose Contrastive Concept Instantiation (CoCoIns), the first approach to achieve subject consistency without the need for time-consuming fine-tuning and labor-intensive reference collection. Our key idea is to model concept instances a latent space and train a mapping network using contrastive learning to associate latent codes with output concopt instances. We demonstrate efficacy on single- subject human faces and extend it to multi-subject scenarios and general concepts. We believe this work establishes foundation for controllable content creation.
10

---

Published in Transactions on Machine Learning Research (11/2025)
References
Richard C. Anderson, James W. Pichert, Ernest T. Goetz, Diane L. Schallert, Kathleen V. Stevens, and Stanley R. Trollip. Instantiation of general terms. Journal of Verbal Learning and Verbal Behavior, 1976.
2
Omri Avrahami, Kfir Aberman, Ohad Fried, Daniel Cohen-Or, and Dani Lischinski. Break-a-scene: Extract- ing multiple concepts from single image. In SIGGRAPH Asia, 2023. 3
Omri Avrahami, Amir Hertz, Yael Vinker, Moab Arar, Shlomi Fruchter, Ohad Fried, Daniel Cohen-Or, and Dani Lischinski. The chosen one: Consistent characters in text-to-image diffusion models. In SIGGRAPH, 2024. 3
Dmitri Bitouk, Neeraj Kumar, Samreen Dhillon, Peter Belhumeur, and Shree K. Nayar. Face swapping: automatically replacing faces in photographs. In SIGGRAPH, 2008. 2
Wenhu Chen, Hexiang Hu, Yandong Li, Nataniel Ruiz, Xuhui Jia, Ming-Wee Wei Chang, and William W Cohen. Subject-driven text-to-image generation via apprenticeship learning. In NeurIPS, 2023. 3
Yunjey Choi, Youngjung Uh, Jaejun Yoo, and Jung-Woo Woo Ha. Stargan v2: Diverse image synthesis for multiple domains. In CVPR, 2020. 10
Jiankang Deng, Jia Guo, Jing Yang, Niannan Xue, Irene Kotsia, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. IEEE TPAMI, 2022. 3, 8
Nachum Dershowitz. Program abstraction and instantiation. ACM Trans. Program. Lang. Syst., 1985. 2
Ganggui Ding, Canyu Zhao, Wen Wang, Zhen Yang, Zide Liu, Hao Chen, and Chunhua Shen. Freecustom: Tuning-free free customized image generation for multi-concept composition. In CVPR, 2024. 3
Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. In NeurIPS, 2023. 8, 17
Rinon Gal, Yuval Alaluf, Yuval Atzmon, Or Patashník, Amit Haim Bermano, Gal Chechik, and Daniel Cohen-or. An image is worth one word: Personalizing text-to-image generation using textual inversion. In ICLR, 2023a. 2, 3
Rinon Gal, Moab Arar, Yuval Atzmon, Amit H. Bermano, Gal Chechik, and Daniel Cohen-Oa Encoder- based domain tuning for fast personalization text-to-image models. ACM TOG, 2023b. 3
Yuan Gong, Youxin Pang, Xiaodong Cun, Menghan Xia, Yingqing He, Haoxin Chen, Longyue Wang, Yong Zhang, Xintao Wang, Ying Shan, and Yujiu Yang. Interactive story visualization with multiple characters. In SIGGRAPH Asia, 2023.
Ligong Han, Yinxiao Li, Han Zhang, Peyman Milanfar, Dimitris Metaxas, and Feng Yang. Svdiff: Compact parameter space for diffusion fine-tuning. In ICCV, 2023. 3
Zecheng He, Bo Sun, Felix Juelei-Xu, Haoyu Ma, Ankit Ramchandani, Vincent Cheung, Siddharth Shah, Anmol Kalia, Harihar Subramanyam, Alireza Zareian, Li Chen, Ankit Jain, Ning Zhang, Peizhao Zhang, Roshan Sumbaly, Peter Vajda, and Animesh Sinha. Imagine yourself: Tuning-free personalized image generation. arXiv:2409.188 18346, 2024. 3
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yacl Pritch, and Daniel Cohen-Or. Prompt-to- prompt image editing with cross attention control. arXiv preprint arXiv:2208 01626, 2022. 22
Jonathan Ho, Ajay Jain, and Picter Abbecl. Denoising diffusion probabilistic models. In NearIPS, 2020. 5
Gary Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. Labeled faces in the wild: A database forstudying face recognition in unconstrained environments. In Workshop on Faces in Real-Life Images: Delection, Alignment, and Recognition, 2008. 5
11

---

Published in Transactions on Machine Learning Research (11/2025)
Xun Huang and Serge Belongie. Arbitrary style transfer in real-time with adaptive instance normalization. In ICCV, 2017. 4
Xuhui Jia, Yang Zhao, Kelvin C. K. Chan, Yandong Li, Han Zhang, Boqing Gong, Tingbo Hou, Huisheng Wang, and Yu-Chuan Su. Taming encoder for zero fine-tuning image customization with text-to-image diffusion models. arXix:20440020 02642, 2023. 3
Tero Karras, Samuli Laine, and Timo Aila. A stylo-based generator architccture for generative adversarial networks. In CVPR, 2019.
Tero Karras, Samuli Laine, Miik Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In CVPR, 2020. 4
Diederik P Kingma and Max Welling. Auto-encoding variational bayes. ICLR, 2014.
Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan- Yen Lo, et al. Segment anything. In CVPR, 2023. 6, 7
Nupur Kumari, Bingliang Zhang, Richard Zhang, Eli Shechtman, and Jun- Yan Zhu. Multi-concept cus-
tomization of text-to-image diffusion. In CVPR, 2023. 3
Dongxu Li, Junnan Li, and Steven C. H. Hoi. Blip-diffusion: Pre-trained subject representation for control- lable text-to-image generation and editing. (XXi::305.14720, 2023. 3
Yitong Li, Zhe Gan, Yelong Shen, Jingjing Liu, Yu Cheng, Yuexin Wu, Lawrence Carin, David Carlson, and Jianfeng Gao. Storygan: A sequential conditional gan for story visualization. In CVPR, 2019. 2, 3
Zhen Li, Mingdeng Cao, Xintao Wang, Zhongang Qi, Ming-Ming Cheng, and Ying Shan. Photomaker: Customizing realistic human photos via stacked id embedding. In CVPR, 2024.
Chang Liu, Haoning Wu, Yujie Zhong, Xiaoyun Zhang, Yanfeng Wang, and Weidi Xie. Intelligent grimm open-ended visual storytelling via latent diffusion models. In CVPR, June 2024. 3
Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. In NewrIPS, 2023a. 19
Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Chunyuan Li, Jianwei Yang, Hang Su, Jun Zhu, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. orXiv:2303.05.000 2023b. 6,
Tao Liu, Kai Wang, Senmao Li, Joost van de Weijer, Fahad Shahbaz Khan, Shiqi Yang, Yaxing Wang, Jian Yang, and Ming-Ming Cheng. One-prompt-one--stryy:: Free-lunch consistent. text-to-image generation using a single prompt. In ICLR, 2025. 2, 3, 7, 8,
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang. Deep learning face attributes in the wild. In ICCV, 2015. 5, 7, 22
Adyasha Maharana, Darryl Hanman, and Mohit Bansal. Improving generation and evaluation of visual stories via semantic consistency. In NAACL, 2021.
Adyasha Maharana, Darryl Hannan, and Mohit Bansal. Storydall-e: Adapting pretrained text-to-image transformers for story continuation. In ECCV, 2022. 3
Yuval Nirkin, Yosi Keller, and Tal Hassner. Fsgan: Subject agnostic face swapping and reenactment. In ICCV, 2019. 2
Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishknnn Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. Training language models to follow instructions with human feedback. In NewrIPS, 2022. 7
12

---

Published in Transactions on Machine Learning Research (11/2025)
Xichen Pan, Pengda Qin, Yuhong Li, Hui Xue, and Wenhu Chen. Synthesizing coherent story with auto- regressive latent diffusion models. In WACV, 2024. 3
Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, Jiankang Deng, Bernhard Kainz, and Stefanos Zafeiriou. Arc2face: A foundation model for id-consistent human faces. In ECCV, 2024.
Xu Peng, Junwei Zhu, Boyuan Jiang, Ying Tai, Donghao Luo, Jiangning Zhang, Wei Lin, Taisong Jin, Chengjie Wang, and Rongrong Ji. Portraitbooth: A versatile portrait model for fast identity-preserver personalization. In CVPR, 2024. 3
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv:2307.01922 2023. 2, 4,
Adam Polyak, Amit Zohar, Andrew Brown, Andros Tjandra, Animesh Sinha, Ann Lee, Apoorv Vyas, Bowen Shi, Chih-Yaa Ma, Ching- Yao Chuang, David Yan, Dhruv Choudhary, Dingkang Wang, Geet Sothi, Guan Pang, Haoyu Ma, Ishan Misra, Ji Hou, Jialiang Wang, Kiran Jagadeesh, Kunpeng Li, Luxin Zhang, Mannat Singh, Mary Williamson, Matt Le, Matthew Yu, Mitesh Kumar Singh, Peizhao Zhang, Peter Vajda, Quentin Duval, Rohit Girdhar, Roshan Sumbaly, Sai Saketh Rambhatla, Sam Tsai, Samanch Azadi, Samyak Datla, Sanyuan Chen, Sean Bell, Sharadh Ramaswamy, Shelly Sheynin, Siddharth Bhat- tacharya, Simran Motwani, Tao Xu, Tianhe Li, Tingho Hou, Wei-Ning Hsu, Xi Yin, Xiaoliang Dai, Yaniy Taigman, Yaqiao Luo, Yen-Cheng Liu, Yi-Chiao Wu, Yue Zhao, Yuval Kirstain, Zecheng He, Zijian He, Albert Pumarola, Ali Thabet, Artsiom Sanakoyeu, Arun Mallya, Baishan Guo, Boris Araya, Breena Kerr, Carleigh Wood, Ce Liu, Cen Peng, Dimitry Vengertsev, Edgar Schonfeld, Elliot Blanchard, Felix Juefei- Xu, Fraylie Nord, Jeff Liang, John Hoffman, Jonas Kohler, Kaolin Fire, Karthik Sivakumar, Lawrence Chen, Licheng Yu, Luya Gao, Markos Georgopoulos, Rashel Moritz, Sara K. Sampson, Shikai Li, Simone Parmeggiani, Steve Fine, Tara Fowler, Vladan Petrovic, and Yuming Du. Movie gen: A cast of media foundation models. Xrv: 10. 18720, 2025.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya. Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Πly Sutskever. Learning transferable visual models from natural language supervision. 3Xxi:2103.00022) 2021. 4
Tanzila Rahmaan, Hsin-Ying Lee, Jian Ren, Sergey Tulyakov, Shwela Mahajan, and Leonid Sigal. Make-a- story: Visual memory conditioned consistent story generation. In CVPR, 2023. 3
Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, Zhaoyang Zeng Hao Zhang, Feng Li, Jie Yang, Hongyang Li, Qing Jiang, and Lei Zhang, Grounded sam: Assembling open-world models for diverse visual tasks. arx 240.11400..415. 14159, 2024. 6, 7, 22
Robin Rombach, Andreas Blattmann, Dominík Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. In CVPR, 2022. 2, 4
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman. Dream- booth: Fine tuning text-to-image diffusion models for subject-driven generation. In CVPR, 2022. 2, 3, 7, 8,
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Wei Wei, Tingbo Hou, Yael Pritch, Neal Wadhwa, Michael Rubinstein, and Kfir Aberman. Hyperdreambooth: Hypernetworks for fast personalization of text-to- image models. trXtt:2307.069/0 2024.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Bureu Karagol Ayan, S. Sara Mahdavi, Rapha Goutijo Lopes, Tim Salimans, Jonathan Ho, David J Flect, and Mohammad Norouzi. Photorcalistic text-to-image diffusion models with deep language understanding. arXiv:2205.114. 11487, 2022. 2
Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. Tn CVPR, 2015. 6
13

---

Published in Transactions on Machine Learning Research (11/2025)
Fei Shen, Hu Ye, Sibo Liu, Jun Zhang, Cong Wang, Xiao Han, and Wei Yang. Boosting consistency in story visualization with rich-contextual conditional diffusion models. Tn AAAT, 2025. 3
Xiaoqian Shen and Mohamed Elhoseiny. Storygpt-v: Large language models as consistent story visualizers. arXiv:2312.0220 02255, 2023.
Jing Shi, Wei Xiong, Zhe Lin, and Hyun Joon Jung. Instantbooth: Personalized text-to-image generation without test-time finetuning, In CVPR, 2024. 3
Jiaming Song, Chenlin Meng, and Stefano Ermon. Denoising diffusion implicit models. In ICLR, 2021.
Yun-Zhu Song, Zhi Rui Tam, Hung-Jen Chen, Huiao-Han Lu, and Hong-Han Shuai. Character-prservingg coherent story visualization. In ECCV, 2020. 3
Yoad Tewel, Rinon Gal, Gal Chochik, and Yuval Atzmon. Key-locked rank one editing for text-to-image personalization. In SIGGRAPH, 2023.
Yoad Tewel, Omri Kaduri, Rinon Gal, Yoni Kasten, Lior Wolf, Gal Chechik, and Yuval Atzmon. Training-fiee free consistent text-to-image generation. In SIGGRAPH, 2024. 2, 3, 7, 8, 9
Sergey Tulyakov, Ming- Yu Liu, Xiaodong Yang, and Jan Kautz, Mocogan: Decomposing motion and content for video gencration. In CVPR, 2018. 2
Dani Valevski, Danny Lumen, Yossi Matias, and Yaniv Leviathan. FaceO: Instantaneously conditioning a text-to-image model on a face. In SIGGRAPH Asia, 2023.
Andrey Voynov, Qinghao Chu, Daniel Cohen-Or, and Kfir Aberman. P+: Extended textual conditioning in text-to-image generation. MXXi:23333.09222 2023. 3
Jiahao Wang, Caixia Yan, Haonan Lin, Weizhan Zhang, Mengmeng Wang, Tieliang Gong, Guang Dai, and
Hao Sun. Oneactor: Consistent subject generation via custer-condiitonee guidance. In NewrIPS, 2024a.
3
Kuan-Chich Wang, Daniil Ostashev, Yuwci Fang, Sergey Tulyakov, and Kfir Aberman. Moa: Mixture- of-attention for subject-context disentanglement in personalized image generation. In SIGGRAPH Asia,
2024b. 3
Qinghe Wang, Baolu Li, Xiaomin Li, Bing Cao, Liqian Ma, Huchuan Lu, and Xu Jia. Characterfactory: Sampling consistent characters with gans for diffusion models. arXiv:2404.1567, 2024c. 4
Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and Anthony Chen. Instantid: Zero-shot identity-preserving generation in seconds. arXiv:2/01.074 07519, 2024d.
Wen Wang, Canyu Zhao, Hao Chen, Zhekai Chen, Kecheng Zheng, and Chunhua Shen. Autostory: Gener- aling diverse storytelling images with minimal human ellort. IJCV, 2024e. 4
Xierui Wang, Siming Fu, Qihan Huang, Wanggui He, and Hao Jiang. MS-diffusion: Multi-subject zero-shot image personalization with layout guidance. In ICLR, 2025.
Yuxiang Wei, Yabo Zhang, Zhilong Ji, Jinfeng Bai, Lei Zhang, and Wangmeng Zuo. Elite: Encoding visual concepts into textual embeddings for customized text-to-image generation. In ICCV, 2023. 2,
Jianzong Wu, Chao Tang, Jingbo Wang, Yanhong Zeng, Xiangtai Li, and Yunhai Tong. Diffsensei: Bridging multi-modal Iims and diffusion models for customized manga generation. arXiv:2412.0758, 2024s. 2
Yi Wu, Ziqiang Li, Heliang Zheng, Chaoyuc Wang, and Bin Li. Infinite-id: Identity-preserved personalization via id-semantics decoupling paradigm. In ECCV, 2024b. 3
Guangxuan Xiao, Tianwei Yin, William T. Freeman, Frédo Durand, and Song Han. Fastcomposer: Tuning- free multi-subject image generation with localized attention. IJCV, 2024.
14

---

Published in Transactions on Machine Learning Research (11/2025)
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXim:2308.0077 06721, 2023. 2, 3
Fisher Yu, Yinda Zhang, Shuran Song, Ari Seff, and Jianxiong Xiao. Lsun: Construction of large-scale
image dataset using deep learning with humans in the loop. arXiv preprint (XXx::500000380 2015. 10
Dongxu Yue, Maomao Li, Yunfei Liu, Qin Guo, Ailing Zeng, Tianyu Yang, and Yu Li. Addme: Zero-shot group-photo synthesis by inserting people into scenes. In ECCV, 2024. 3
Yu Zeng, Vishal M. Patel, Haochen Wang, Xun Huang, Ting-Chun Wang, Ming-Yo Yu Liu, and Yogesh Balaji. Jedi: Joint-image diffusion models for finetuning-free personalized text-to-image generation. In CVPR, 2024.
Yupeng Zhou, Daquan Zhou, Ming-Ming Cheng, Jiashi Feng, and Qibin Hou. Storydiffusion: Consistent self-attention for long-range image and video generation. In NewrIPS, 2024. 2, 3, 8,
15

---

Published in Transactions on Machine Learning Research (11/2025)
A Comparison of Settings and Implementation
We provide comparison of settings and implementation between our approach and prior work in Table 6. As discussed in Section 2, prior approaches can be categorized into subject-driven generation and subject- consistent generation.
Subject-driven approaches personalize a generative model by fine-tuning parameters on reference images or incorporating additional pretrained encoders. These approaches are either con-cconsuming due to subject- specific fine-tuning or require integrating general encoders, such as DINO, or domain-specific encoders, such face recognition models.
Subject-consistent approaches modify the prompt attention mechanisms of the base generator, thereby avoiding the need for extra modules, reference images, or additional training. Their main limitations are the requirement to generate images in batches or the reliance on stored features.
Our method lighter and more flexible. It requires only a single MLP training and supports independent inference.
Table 6. Comparison of seltings and implementation between our approach and prior work.
Approach
Extra Modules
Reference Training
Inference Constraints
Tuming-based lersonalization
None
Yes
Subject Tuning
None
Encoder- basede Personalization
Pretrained Encoders
Yes
Once
None
Sobjecct-Consstennt Generation
None
No
None
Batch
CoCoIns (Ours)
Lightweight. ML.P
No
Once
None
B
Additional Implementation Details
Computational Resources. The experiments conducted AMD EPYC 9354 CPU and four NVIDIA A6000 GPUs. Each training round takes approximately eight hours.
Hyperparameters. We list the hyperparameters in Table 7. They are determined by grid search. The distance function Lais is the Mean Squared Error.
Latent Code Sampling. Latent codes are randomly sampled from the Gaussian distribution N/ I), as stated in Eq. (3). In each training triplet, the anchor and positive samples share the same latent code, and the negative sample is paired with another randomly sampled code, detailed in Eq. (5).
Table 7. Hyperparameters.
Hyperparameters
Value
256
Aoon
Aback
30
0.00001
8
2
8
K
5000
Batch Size
128
Learning Rate
0.0001
Learning Rate Decay
Cosine
Learning Rate Warmup
500
Optimizer
Adam
Weight Decay
0.2
16

---

Published in Transactions on Machine Learning Research (11/2025)
Table 8, FID score between each approach and its base models.
Table 9. Comparison of performance using MSE and DreamSim loss as the distance function.
FID
Distance
Sim f
Div t
CLIP t
Consistory 13.3
MSE
0.600
0.799
0.193
StoryDiffusion
15.2
DreamSim
0.314
0.724
0.291
Ours
9.2
Table 10.
Ablation study on
Table
Ablation study on weighting negative distances.
Table Ablation study on sched-
weighting background loss.
ules of negative distance weighting.
Aback
Sim f
Div f
CLIP t
Sim
Div t
CLIP 1
8
Sim 1
Div t
CLIP t
10
0.637
0.603
0.265
10-4
0.666
0.766
0.288
1
0.528
0.735
0.291
30
0.600
0.799
0.290
10-5
0.600
0.799
0.290
2
0.600
0.799
0.290
50
0.516
0.707
0.292
10-6
0.445
0.635
0.293
3
0.520
0.710
0.291
C Additional Experimental Results
C.1 Additional Comparison
Image Quality. We evaluate the quality of the generated images using the Fréchet Inception Distance (FID) score. Since various approaches are based different models, we compute FID scores by comparing the images produced by each approach with those generated by its respective base model. Consistory and our method use SDXL; StoryDiffusion utilizes RealVisXL. Given that the FID score is sensitive to sample size, we duplicate the Scenes dataset ten times to create a total of 10K prompts. We then generate 10K images using both the compared approaches and their base models. As shown in Table 8, our generated images are more closely aligned with the distribution of the base model.
Distance Function. We investigate the effect of different distance functions for measuring subject similarity during training. In our training strategy, the model is trained to generate similar appearances from the same image corrupted by two different noises, using two similar prompts and the same latent code. When reconstruction is nearly perfect, a simple pixel-wise metric such as Mean Squared Error (MSE) is sufficient. However, in more realistic scenarios where outputs are imperfect, we may need a subject-aware similarity measure. One option is to use a subject encoder; however, existing encoders are typically designed for narrow domains (e.g. face recognition) and often rely on non-differentiable operations, such as cropping and landmark alignment, making them unsuitable for end-to-end training. An alternative loss, such as DreamSim (Fu et al., 2023), which is trained to align with human perception of similarity. However, its notion of similarity may reflect factors like layout and color rather than subject identity.
learned perceptual
Therefore, we primarily use MSE as our distance function but also include a comparison with DreamSim. To apply DreamSim as a perceptual similarity metric, we decode the DDIM-approximated image latents using the autoencoder and compute the cosine similarity between their DreamSim embeddings. We use the DINOv2 chockpoint for DreamSim. As shown in Table 9, compared to MSE, DreamSim loss struggles generate consistent subjects.
C.2 Additional Ablation Study
We examine the impact of hyperparameter choices on performance, specifically (1) the balance between the consistency and background loss and (2) the weighting schedule of negative distances.
Table 10 shows the comparison between different background loss weights, denoted by Aback in Eq. (9). Increasing the importance of background loss improves prompt fidelity the cost of lower face similarity and diversity. We choose the Aback that balances these factors.
17

---

Published in Transactions on Machine Learning Research (11/2025)
Interpolation
Figure 8. Results generated with interpolations of two latent codes. The leftmosst and rightmost images are generated from two randomly sampled codes. The intermediate images the results of interpolations between the two codes, demonstrating the gradual transition between the two faces.
Selected
Close
Far
Figure Results generated with neighbors of word We generate 100 images with randomly sampled latent codes and fixed initial noise. Given a randomly selected code and its corresponding image, we find the neighbors of the selected code by sorting the cosine similarity between the pseudo-word transformed from the code and the others. The result shows that the pseudo-words words of closer neighbors (L.e. high cosine similarity) produce more similar faces.
Additionally, as discussed in Section 4.1, the weighting negative distances is implemented as an increasing schedule paraneterized by ( where ls the current training step, and K is the total number of train- ing steps. y and B are hyperparameters. We also examine the effect of varying these hyperparameters.
Table 11 presents the results for different values of 7, and Table 12 provides comparisons between different values of B. Similar to our previous study on consistency and background loss, some hyperparameter sets lead to higher face similarity but lower prompt fidelity or diversity. We choose the hyperparameter set that balances all of those important metrics.
C.3 Analysis of the Latent Space
We analyze the latent space that models the instance distribution of concepts. To better understand the rela- tionships between latent codes in the space and the information captured by pseudo-words, we demonstrate results generated with interpolations of latent codes and neighbors of pseudo-words.
Interpolation of Latent Codes. We reveal the relationships between latent codes by visualizing their interpolations. Given two randomly sampled latent codes, we generate images by gradually interpolating between them, starting from a fixed initial noise. As illustrated in the two examples Figure 8, the leftmost and rightmost images correspond to the two original latent codes, while the intermediate images depict the
18

---

Published in Transactions on Machine Learning Research (11/2025)
Figure 10. Eight sets of examples from Portraits. The subjects for each row are woman, man, girl, and boy.
transition between them. Using a fixed initial noise results similar background appearances across the interpolated images.
Neighbors of Pscudo-Words. Words. We also illustrate the information captured by a pscudo-word by retrieving its neighbors. Specifically, we randomly sample 100 latent codes and generate images from them. We then select one code and its corresponding image, and compute the cosine similarity between the pseudo-word dcrived from this code and those from the remaining codes. The other images are sorted based on this similarity. shown in Figure 9, the leftmost image generated from the selected latent code, while the images to right are arranged in descending order of pseudo-word similarity. The results indicate that closer neighbors, t.e. pseudo-words with higher similarity, tend to generate more visually similar faces.
C.4 Additional Generated Samples
We provide additional samples from Portraits Figure 10 and Scenes in Figure 11. Each row contains two subjects with four examples. The subjects for each row are woman, man, girl, and boy.
More multi-subject examples are in Figure 12. We also include comparisons with two previous approaches, Consistory and StoryDiffusion. Our examples demonstrate high image quality and consistency. The prompts in Figure 7 and Figure 12 are "a man and a woman sitting on the sofa", "a man and woman taking the bus", "a man and a woman walking on the street", and "a man and a woman dancing on the stage".
More examples in Figure demonstrate consistency for general concepts. The prompts in Figure and Figure 13 are "a photo of cat", "dog", "car".
Data Collection and Generation
Identifying Subjects in Descriptions. We generate descriptions for training images with a multi-modal captioner (Liu et al., 2023a). Then, we prompt an LLM to identify the words that are most likely to be the subjects in the sentences. The prompt is follows:
These are two captions of an image. Tag the words related to the subjects of the captions. Return the captions in JSON with keys "caption1" " and "caption2".
19

---

Published in Transactions on Machine Learning Research (11/2025)
Figure Eight sets of examples from Scenes. The subjects for each row are woman, man, girl, and boy.
Consistory
StoryDiffusion
CoCoins
Figure 12. More examples and comparison with previous work of multi-subject consistency.
1. Identify words related to a person: Look for specific nouns or noun phrases that refer to a person. Exclude pronouns (e.g., "he", "she", "they", "it") and collective nouns (e.g., "people") from tagging.
2. Tag these words: Surround each identified word or noun phrase with <subj> and </subj> tags. Use <subj1>, <subj2>, etc., for different items. Apply the same index number to all references to the same item.
3. If no relevant. words are found: Return the original captions without any changes.
20

---

Published in Transactions on Machine Learning Research (11/2025)
Figure 13. Two sets of examples demonstrating consistency for general concepts.
4. Handling ambiguity: If a word has ambignous references or unclear roles, do not tag it.
5. Pronouns: Do not tag pronouns.
Then, we implement a tag parser to provide subject locations during training. inference, users can provide locations indicate target concepts that must be consistent.
Generating Diverse Scenes for Evaluation. One of the test datasets is collected to represent diverse, real-world contexts. Taking "man" as example, we prompt LLM to generate these sentences using the following instruction:
Generate fifty scene descriptions man in daily life.
1. Focus on the subject
The subject should always be "man".
Provide descriptions of diverse scenes that feature specific subject performing action in a certain place.
Make actions and locations diverse
Always use a verb and a location that has not appeared in previous sentences.
The scenes should related to everyday life, such as cooking, driving, walking, reading, etc.
3. Portrait Details:
The descriptions should feature close-up views of the subject's face.
Sensory details should enhance the scene (lighting, surrounings, sounds, smells, etc.), but keep the focus on the subject's face. The environment should be vivid but relevant to the subject's 's action or setting.
4. Tag the subjects:
Tag the subject with <subj1> </subj1>. For example,"a "a <eubji>man<<subji>> stands in the room"
Do not lag pronouns (such "he", "his", "him", etc.).
If there are multiple subjects, different indexes for each individual (e.g., <subj1>, <subj2>, etc.).
Use the same index for all references to the same subject.
5. Length: No more than three sentences.
Now generate 50 more samples with scenes related technical or professional settings. For example, locations can be offices, schools, hospitals, labs, farms, factories, studios, kitchens, etc.
Now generate 50 more samples with scenes related to casual, cultural, or recreational occa- sions, such as dances, music, dramas, movics, sports, arts, etc.
Now generate 50 more samples with scenes related to outdoor activities or in nature, such as gardens, parks, mountains, forests, beaches, rivers, lakes, etc.
Prompts of Qualitative Comparison. The images in Figure 4 generated with the following prompts:
21

---

Published in Transactions on Machine Learning Research (11/2025)
A man fixes his bicycle chain in the workshop, grease streaking his concentrated face. The smell of rubber and oil surrounds him as he works by lamplight.
A man kneads bread dough in his sun-drenched kitchen, flour dusting his smile lines as morning light streams through gauzy curtains. His focused gaze follows the rhythmic movements of his hands, while the yeasty aroma fills the warm air.
At the television studio, man directs live broadcast, speaking calmly into his headset. His focused gaze darts between multiple monitors as he calls camera changes.
A man tracks wildlife in the misty forest, his experienced eyes reading subtle signs in the undergrowth.
The early morning light filters through the canopy, illuminating his weathered face as he studies fresh prints.
A woman tends to her rooftop beehives, moving with calm confidence among the buzzing insects. Her peaceful expression reflects years experience as she checks each frame.
Under fluorescent lights at the corner store, woman browses magazine covers, her reflection ghosted in the glossy pages. Her fingers trace headlines as she squints slightly, the artificial brightness highlighting the fine lines around her eyes.
In the acoustically treated recording studio, woman masters audio tracks, her trained ear catching subtle nuances. Her eyes close briefly as she adjusts levels with precise movements. A woman fills her bird feeder in the backyard, morning dew soaking the hem of her robe. She squints against the rising sun, watching finches dart around her head as she pours the seeds.
Tackling Subject Ambiguity. Our experiments focus on single-subject consistency. To create a clean training dataset and minimize ambiguity, we use CelebA (Liu et al., 2015), which primarily contains human portraits. We generate masks using powerful off-the-shelf model, Grounded SAM 2 (Ren et al., 2024). We then manually filter out images with more than one face according to the segmentation results. This step ensures that the training images contain single subjects and reduces ambignous masks.
Extension to Multi-Subject Scenes. Although the model is trained only on single-subjoct portrait images, it can handle subject consistency in more challenging conditions, such as free-form prompts in Scenes and images with two subjects. For more challenging scenarios with multi-subject images, the proposed loss function framework can be further extended. Since prior work (Hertz et al., 2022) has shown the relationships between word embeddings and feature maps, and pseudo-word operates in the text embedding space to describe the subject that follows, we hypothesize that the model can learn to associate a pseudo-word with features of its corresponding subject, even in a multi-subject scene.
More thorough experimentation and evaluation are needed for multi-subject scenarios, which will be part of our future work.
Segmentation Accuracy. In single-subject experiments, inaccurate segmentation is less pronounced. Seg- menting person in portrait, is common in the CelebA dataset, is relatively straightforward for a powerful model like Grounded SAM 2. In addition, we manually validate 100 randomly sampled images and find that the predictions are consistently reliable for this use case.
Licenses. The main experiments are conducted on CelebA (Liu et al., 2015). It is made available for noncommercial research purposes and requires users to comply with the terms outlined in the official usage agrocment.
E Further Discussions
Preventing Prompt Corruption. To ensure that a pseudo-word alfects only the subject and does not corrupt the rest of the prompt, use bakkgrouud-presevvatie in the background regions between an image generated with the pseudo-word and one generated without it. This localizes the effect of a pseudo-word to the masked subject region, thereby preserving the overall scene context dictated by the original prompt. The ablation study in Table 4 shows that removing this loss significantly harms prompt fidelity.
loss. This loss minimizes the difference
Place of Pseudo-Worts CoCoIns is developed based on the assumption that the pseudo-word word lies in the text embedding space. Our mapping network is explicitly designed to take latent code and output
22

---

Published in Transactions on Machine Learning Research (11/2025)
pscudo-word w vector that has the same dimension the text embeddings of the pretrained text encoder. This pseudo-word vector is then inserted directly into the sequence of prompt embeddings before the subject token.
Aligning Pseudo- Pseudo-Words. The consistency loss is not defined directly on the pseudo-words. Instead, a pseudo-word made meaningful through an indirect alignment process via a contrastive loss applied to the generated images.
We use triplet loss applied to the predicted image latents from the denoiser. The model is trained to minimize the visual difference between images generated with the same pseudo-word (even with different prompts and noise) while maximizing the visual difference between images generated with different pseudo- words. This process forces the model to treat each pseudo-word as a unique identifier for specific visual appearance. The meaning of a pseudo-word becomes the consistent subject identity it produces.
We also analyze the space of pscudo-words Appendix C.3, which provides strong evidence that this indirect alignment is successful. The experimental results show that:
Smoothly interpolating between two latent codes results in smooth visual transition between the two corresponding faces.
Interpolation:
Similarity: Pseudo-words that are closer to each other in the embedding space (Le have higher cosine similarity) produce subjects with more similar appearances.
These results demonstrate that the learned space of pseudo-words is well-structured and meaningful.
F Broader Impacts
Our approach enables image generative models maintain subject consistency, extending their applicability across various tasks and modalities. Although this versatility could potentially be misused, such risks can be effectively mitigated through responsible deployment strategies, including strict usage policios, gated releases, and watermarking techniques.
23


Reviews:
[{"review_id": "NhLVQwZvXr", "reviewer": "Reviewer_HQRe", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "## Summary\nThis work proposes Contrastive Concept Instantiation (CoCoIns), a lightweight framework for subject preservation in text-to-image generative models. The authors propose a simple and intuitive contrastive learning method by constructing positive and negative prompt pairs for a given subject and learning a mapping function from a sampled latent code to a pseudo-word as a learnable prefix to the subject token. Empirical results show the effectiveness of the proposed method in terms of subject preservation and diversity in generation.\n\n## Strengths\n- Overall, the paper is written very well with a clear motivation for developing the proposed CoCoIns method.\n- The method is intuitive and proposes a strategy similar to \"prefix tuning\" or soft prompting that has been popularized in LLM/VLMs for a unique application in the subject preservation task.\n- The experimental results are very convincing and show that CoCoIns produces high-quality generated images preserving subjects while maintaining diversity.\n- The masked contrastive loss for the subject region in the image coupled with a background preservation loss is intuitive for the subject preservation task.\n\n## Weaknesses\n- Although the method is interesting, the applicability to long and arbitrary prompts is not particularly addressed. The experiments include a free-form prompt setting, but it is not clear how this method would work for complex subject relationships in prompts.\n- The masking component may not always be accurate, especially when the subject can be ambiguous in images.\n- It is not clear how the original latent code $\\mathbf{z}$ is sampled.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims made in this paper are accurate, convincing, and clear. The motivation behind developing this method comes from the computational cost of previous subject preservation methods. The proposed CoCoIns framework is intuitive in formulation and is reminiscent of \"prefix tuning\" or soft prompting that has been popularized in LLMs/VLMs. Using this idea for improving the deterministic components of text-to-image models is interesting and the empirical evaluations clearly show an improvement in generation quality compared to other methods.", "audience_yn": "Yes", "audience_explanation": "This work is very clearly in line with the interests of TMLR's audience. With the popularity of text-to-image generative models, this paper makes a contribution to interpretability and model steering. Furthermore, the findings of this paper can be utilized for a variety of different applications in the generative modeling and computer vision domains.", "requested_changes": "Explaining the following in more details in the main text would be critical for clarity:\n- How do you make sure that after the pseudo-word is prepended to the subject that the text encoder doesn't corrupt the embeddings? Is the assumption that the pseudo-word lies in the text embedding space? If so, how do you align this to make sure it is meaningful? The loss seems to be with respect to the generated images and a loss is not explicitly defined on the pseudo-words. Making this clear is important.\n- How are the original latent codes sampled? Clarifying this is important since the method is largely dependent on transforming these codes to derive pseudo-words. \n- How do you deal with subject ambiguity and more complex prompts? \n- The out-of-the-box segmentation models are not always accurate for more complex images. How is the masked subject loss defined in such scenarios?", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "bcbwP7Vuba", "reviewer": "Reviewer_pT3i", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "Though existing text-to-image generative models follow the prompts and produce diverse images, they fall short on generating consistent appearances for the same subjects across multiple generations. This paper introduces Contrastive Concept Instantiation (CoCoIns) to improve the consistency of the same subjects across multiple individual generations. CoCoIns only learns a mapping function in a self-supervision setting to produce virtual word tokens which correspond to different subjects. The experimental results show the effectiveness of CoCoIns in terms of human subjects (quantitatively and qualitatively), and also demonstrate its potential on other subjects (e.g., dogs, cats) and on multiple-subject scenarios.", "claims_and_evidence_yn": "Yes", "claims_explanation": "**Strengths**\n1. The learning overhead is low. CoCoIns only learns a mapping function, instead of tuning the text-to-image backbones. Plus, it does not require reference images/subjects due to its self-supervision nature.\n2. The virtual word tokens correspond to language embeddings, which allows easy manipulation of subject appearances through text conditions.\n3. The experimental results show that CoCoIns can generate images with consistent subjects over multiple generations in terms of humans, cats, dogs, and even multiple-subject scenarios.\n\n\n**Weaknesses**\n1. The negative prompts (same caption as the anchor prompt but inserted with a different latent code) can be considered as easy negative samples (or at least too different from anchor samples which may contribute to the large distance between anchor samples and negative samples at the beginning of training as mentioned in Section 4.1). Given the contrastive learning nature of CoCoIns, have the authors considered (1) increasing batch size (i.e., the number of negatives) and (2) making the negative samples more challenging (e.g., applying different amount of random noise to the anchor latent code). \n2. There are a lack of ablation studies on (1) \\lambda_con and \\lambda_back and (2) the negative distance weight schedule hyperparameters of \\gamma and \\beta.\n3. The demonstration of multi-subject consistency is interesting. However, there are no comparisons with other existing methods to demonstrate the performance of CoCoIns.", "audience_yn": "Yes", "audience_explanation": "See above.", "requested_changes": "1. Can the authors add a table to show which prior works require generating images in batches or through reference tuning or need to tune the text-to-image backbones? This can better highlight the limitations of prior works and promote CoCoIns.", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "dnLKpDfRk5", "reviewer": "Reviewer_3wKr", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "No", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "This paper mainly targets subject-driven generations. The authors point out that previous methods mainly rely on time-consuming tuning or referencs for all subjects, leading to limited usage. To this end they propose a new method which maps latent codes to pseudo word embeddings, which are merged into text prompt and used in the imgae generation process. The authors leverage contrastive learning to train this framework. They conduct several experiments to validate the effectiveness of the proposed method.", "claims_and_evidence_yn": "No", "claims_explanation": "1. For quantitative experiment, the proposed mehod fails to perform the best in terms of almost all of the metrics.\n2. For qualitative experiment, it is obvious that the generated images by the proposed method as shown in Fig.3(e) and Fig.7 has severe deformed human faces. This leads to the concern that the proposed method can damage the generation ability of the pretrained model.\n3. Given that more advanced diffusion-based models are commonly used in recent papers, it is insufficient to only leverage SDXL in the experiments.\n4. There is no specific metric or visualization to show the effectiveness of contrastive learning.\n5. One can find that the dogs and cars in Fig.6 are not consistent among the images.", "audience_yn": "No", "audience_explanation": "While this paper is indeed handling a widely studied problem, i.e. the subject-consistent generation, the specific setting is essentially different from the previous works. The reason that this task is important is that one can generate consistent images based on some reference images. However, the proposed method can only control the ID information by latent code, but not some extra source knowledge. This makes the method less useful compared to previous ones.", "requested_changes": "Please refer to the above comments.", "broader_impact_concerns": "NA", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)