WithAnyone: Toward Controllable and ID Consistent Image Generation

## Abstract

Abstract Identity-consistent generation has become an important focus in text-to-image research, with recent models achieving notable success in producing images aligned with a reference identity. Yet, the scarcity of large-scale paired datasets containing multiple images of the same individual forces most approaches to adopt reconstruction-based training. This reliance often leads to a failure mode we term copy-paste , where the model directly replicates the reference face rather than preserving identity across natural variations in pose, expression, or lighting. Such over-similarity undermines controllability and limits the expressive power of generation.
To address these limitations, we (1) construct a large-scale paired dataset MultiID-2M tailored for multi-person scenarios, providing diverse references for each identity; (2) introduce a benchmark that quantifies both copy-paste artifacts and the trade-off between identity fidelity and variation; and (3) propose a novel training paradigm with a contrastive identity loss that leverages paired data to balance fidelity with diversity. These contributions culminate in WithAnyone , a diffusion-based model that effectively mitigates copy-paste while preserving high identity similarity.
Extensive qualitative and quantitative experiments demonstrate that WithAnyone significantly reduces copy-paste artifacts, improves controllability over pose and expression, and maintains strong perceptual quality. User studies further validate that our method achieves high identity fidelity while enabling expressive controllable generation.

## 1 Introduction

With the rapid progress of generative artificial intelligence, controllable image generation via reference images or image prompting [44, 16, 66, 59, 19, 57] and identity-consistent (ID-consistent) generation [64, 14, 50, 21, 8, 68, 15] have achieved remarkable advances: modern models can synthesize portraits that closely match the provided individual. Recent efforts [8, 4] push resemblance toward near-perfect reproduction. While pursuing higher similarity seems natural, beyond a certain point, excessive fidelity becomes counterproductive.

In real photographs of the same person, identity similarity varies substantially due to natural changes in pose, expression, makeup, and illumination (Fig. [2](https://arxiv.org/html/2510.14975v1#S1.F2)). By contrast, many generative models adhere to the reference image far more rigidly than this natural range of variation. Although such over-optimization may seem beneficial, it suppresses legitimate variation, reducing controllability and limiting practical usability. We term this failure mode the copy-paste artifact: rather than synthesizing an identity in a flexible, controllable manner, the model effectively copies the reference image into the output (see Fig. [2](https://arxiv.org/html/2510.14975v1#S1.F2)). In this work, we formalize this artifact, develop metrics to quantify it, and propose a novel training strategy to mitigate it.

Mitigating copy-paste artifacts is fundamentally constrained by the lack of suitable training data. While numerous large-scale face datasets exist [29, 47, 9, 67, 22, 70, 51], they remain ill-suited for controllable multi-identity generation. Critically, few datasets provide paired references for each identity-multiple images of the same person across diverse expressions, poses, hairstyles, and viewpoints.
As a result, most prior work resorts to single-person, reconstruction-based training [14, 50], where the reference and target coincide. This setup inherently promotes copying and exacerbates copy-paste artifacts. Constructing datasets with multiple references per identity, particularly in group photos, and developing methods to effectively exploit such data remain open challenges.

Figure: Figure 2: Our Observation. Natural variations, such as head pose, expression, and makeup, may cause more face similarity decrease than expected. Copying reference image limits models’ ability to respond to expression and makeup adjustment prompts.
Refer to caption: x7.png

In this work, we introduce a large-scale open-source Multi-ID dataset, MultiID-2M, together with a comprehensive benchmark, MultiID-Bench, designed for intrinsic evaluation of multi-identity image generation. MultiID-2M contains 500k group photos featuring 1–5 recognizable celebrities. For each celebrity, hundreds of individual images are provided as paired references, covering diverse expressions, hairstyles, and viewing angles. In addition, 1.5M unpaired group photos without references are included.
MultiID-Bench establishes a standardized evaluation protocol for multi-identity generation. Beyond widely adopted metrics such as ID similarity [45, 11], it quantifies copy-paste artifacts by measuring distances between generated images, references, and ground truth. Evaluation on 12 state-of-the-art customization models highlights a clear trade-off between ID similarity and copy-paste artifacts (see Fig. [5](https://arxiv.org/html/2510.14975v1#S5.F5)).

Furthermore, we present WithAnyone, a novel identity customization model built on the FLUX [27] architecture, as a step toward mitigating copy-paste artifacts. WithAnyone maintains state-of-the-art identity similarity (with regard to target image) while substantially reducing copy-paste, thereby breaking the long-observed trade-off between fidelity and artifacts. This advance is enabled by a paired-training strategy combined with an ID contrastive loss enhanced with a large negative pool, both made possible by our paired dataset. The labeled identities and their reference images enable the construction of an extended negative pool (images of different identities), which provides stronger discrimination signals during optimization.

In summary, our main contributions are:

- •
MultiID-2M: A large-scale dataset of 500k group photos containing multiple identifiable celebrities, each with hundreds of reference images capturing diverse variations, along with 1.5M additional unpaired group photos. This resource supports pre-training and evaluation of multi-identity generation models.
- •
MultiID-Bench: A comprehensive benchmark with standardized evaluation protocols for identity customization, enabling systematic and intrinsic assessment of multi-identity image generation methods.
- •
WithAnyone: A novel ID customization model built on FLUX that achieves state-of-the-art performance, generating high-fidelity multi-identity images while mitigating copy-paste artifacts and enhancing visual quality.

## 2 Related Work

Figure: Figure 3: Overview of WithAnyone. It builds on a large-scale dataset, MultiID-2M, constructed through a four-step pipeline: (1) collect and cluster single-ID data based on identity similarity; (2) gather multi-ID data via targeted searches using desired identity names with negative keywords for filtering; (3) form image pairs by matching faces between single-ID and multi-ID data; and (4) apply post-processing for quality control and stylization. Training proceeds in four stages: (1) pre-train on single-ID, multi-ID, and open-domain images with fixed prompts; (2) train with image-caption supervision; (3) fine-tune with ID-paired data; and (4) perform quality tuning using a curated high-quality subset.
Refer to caption: x10.png

#### Single-ID Preservation.

The generation of Identity-preserving images is a core topic in customized synthesis [48, 49, 63, 60, 58, 52, 5, 20, 35]. Many methods in the UNet/Stable Diffusion era inject learned embeddings (e.g., CLIP or ArcFace) via cross‑attention or adapters [17, 43, 40, 64, 41, 42]. With the rise of DiT‑style backbones [38, 13, 27] (e.g., SD3, FLUX), progress in ID preservation like PuLID [14], also attracts great attention.

#### Multi-ID Preservation.

Multi‑ID preservation remains relatively underexplored. Some works target spatial control of multiple identities [25, 15, 68], while others focus on identity fidelity. Methods such as XVerse [4] and UMO [8] use VAE‑derived face embeddings concatenated with model inputs, which can produce pixel‑level copy‑paste artifacts and reduce controllability. DynamicID [18](^1^11Excluded from our experiments due to unavailability of code and pretrained models.) achieves improved controllability but is constrained by limited task‑specific data and evaluation standards. Other general-purpose customization and editing models [36, 30, 37, 56, 61, 54, 55, 2, 53] can also synthesize images containing multiple identities, but their ID similarity is often compromised for generality.

#### ID-Centric Datasets and Benchmarks.

Although there are numerous single‑ID datasets [23, 51] and multi‑ID collections [9, 22], paired reference images are scarce, so reconstruction remains the dominant training objective for multi‑ID datasets. Representative datasets are listed in Table [4](https://arxiv.org/html/2510.14975v1#A2.T4). Evaluation protocols are underdeveloped: several works (e.g., PuLID [14], UniPortrait [15], and others [60, 68]) construct test sets by sampling identities from CelebA [29], which undermines reproducibility. Recent efforts benchmark multiple reference generation [71, 54] while focusing on general customization. To address this, we release a curated multi‑ID benchmark with standardized splits and comprehensive metrics to facilitate future research.

## 3 MultiID-2M: Paired Multi-Person Dataset Construction

MultiID-2M is a large-scale multi-person dataset constructed via a four-stage pipeline: (1) collect single-ID images from the web and construct a clean reference bank by clustering ArcFace [11] embeddings, yielding $\sim$1M reference images across $\sim$3k identities (averaging 400 per identity); (2) retrieve candidate group photos via multi-name and scene-aware queries and detect faces; (3) assign identities by matching ArcFace embeddings to single-ID cluster centers using cosine similarity (threshold 0.4); and (4) perform automated filtering and annotation, including Recognize Anything [69], aesthetic scoring [12], OCR-based watermark/logo removal, and LLM-based caption generation [1]. The final corpus comprises $\sim$500k identified multi-ID images with matched references from the reference bank, as well as $\sim$1.5M additional unidentified multi-ID images for reconstruction training, covering $\sim$25k unique identities, with diverse nationalities and ethnicities.
Further details of the construction pipeline and dataset statistics are provided in Appendix [B](https://arxiv.org/html/2510.14975v1#A2).

Figure: (a) Model Architecture
Refer to caption: x11.png

## 4 MultiID-Bench: Comprehensive ID Customization Evaluation

MultiID-Bench is a unified benchmark for group-photo (multi-ID) generation. It samples rare, long-tail identities with no overlap to training data, yielding 435 test cases. Each case consists of one ground-truth (GT) image containing 1–4 people, the corresponding 1–4 reference images as inputs, and a prompt describing the GT. Detailed statistics are provided in Appendix [B](https://arxiv.org/html/2510.14975v1#A2).

Evaluation considers both identity fidelity and generation quality. Let $\mathbf{r},\mathbf{t},\mathbf{g}$ denote the face embeddings of the reference identity, the target (ground-truth), and the generated image, respectively. We define similarity between two embeddings as $\mathrm{Sim}(\mathbf{a},\mathbf{b})$, specifically we term the generated image’s face similarity with regard to GT as $\mathrm{Sim_{GT}}$, and to reference as $\mathrm{Sim_{Ref}}$,

$$ $\mathrm{Sim}(\mathbf{a},\mathbf{b})=\frac{\mathbf{a}^{\top}\mathbf{b}}{\|\mathbf{a}\|\,\|\mathbf{b}\|},$ (1) $$

Specially, we denote $\mathrm{Sim_{Ref}}=\mathrm{Sim}(\mathbf{r},\mathbf{g})$ and $\mathrm{Sim_{GT}}=\mathrm{Sim}(\mathbf{t},\mathbf{g})$.
Prior works [68, 15, 14, 8] has largely reported only $\mathrm{Sim_{Ref}}$, which inadvertently favors trivial copy-paste: directly replicating the reference appearance maximizes the score, even when the prompt specifies changes in pose, expression, or viewpoint. In contrast, MultiID-Bench uses $\mathrm{Sim_{GT}}$ the similarity to the ground-truth identity described by the prompt as the primary metric. This design penalizes excessive copying when natural variations (e.g., pose, expression, occlusion) are expected, while rewarding faithful realization of the prompted scene.

We define the angular distance as $\theta_{ab}=\arccos(\mathrm{Sim}(a,b))$ (geodesic distance on the unit sphere). The Copy-Paste metric is given by

$$ $\mathrm{M_{CP}}(\mathbf{g}\mid\mathbf{t},\mathbf{r})=\frac{\theta_{gt}-\theta_{gr}}{\max(\theta_{tr},\,\varepsilon)}\in[-1,1],$ (2) $$

where $\varepsilon$ is a small constant for numerical stability. The metric thus captures the relative bias of $\mathbf{g}$ toward the reference $\mathbf{r}$ versus the ground truth $\mathbf{t}$, normalized by angular distance of $\mathbf{r}$ and $\mathbf{t}$. A score of $1$ means $\mathbf{g}$ fully coincides with the reference (perfect copy-paste), while $-1$ means full agreement with the ground truth.

We additionally report identity blending, prompt fidelity (CLIP I/T), and aesthetics; formal definitions and further details are provided in Appendix [C](https://arxiv.org/html/2510.14975v1#A3).

## 5 WithAnyone: Controllable and ID-Consistent Generation

Building on the scale and paired‑reference supervision of the MultiID‑2M, we devise training strategies and tailored objectives that transcend reconstruction to enable robust, identity‑conditioned synthesis. This rich, identity‑labeled supervision not only substantially improves identity fidelity but also suppresses trivial copy–paste artifacts and affords finer control over multi‑identity composition. Motivated by these advantages, we introduce WithAnyone - a unified architecture and training recipe designed for controllable, high‑fidelity multi‑ID generation. Architectural schematics and implementation details are provided in Fig. [4](https://arxiv.org/html/2510.14975v1#S3.F4) and Appendix [E](https://arxiv.org/html/2510.14975v1#A5).

### 5.1 Training Objectives

Diffusion Loss. We adopt the mini-batch empirical flow-matching loss. For each batch, we sample a data latent $x_{1}\sim p_{\text{data}}$, Gaussian noise $x_{0}\sim\mathcal{N}(0,I)$, and a timestep $t\sim\mathcal{U}(0,1)$. We then form the interpolated latent $x_{t}=(1-t)x_{0}+tx_{1}$ and regress the target velocity $(x_{1}-x_{0})$:

$$ $\mathcal{L}_{\mathrm{diff}}=\big\|v_{\theta}(x_{t}^{(i)},t^{(i)},c^{(i)})-(x_{1}^{(i)}-x_{0}^{(i)})\big\|_{2}^{2},$ (3) $$

where $c^{(i)}$ denotes the conditioning signal.

**Table 1: Quantitative comparison on the single-person subset of MultiID-Bench and OmniContext. , , and indicate the first-, second-, and third-best performance, respectively. For Copy-Paste ranking, only cases with $\mathrm{Sim(GT)}>0.40$ are considered.**
| Method | Identity Metrics | Generation Quality |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- |
| Sim(GT) $\uparrow$ | Sim(Ref) $\uparrow$ | CP $\downarrow$ | CLIP-I $\uparrow$ | CLIP-T $\uparrow$ | Aes $\uparrow$ |  |
| DreamO | 0.454 | 0.694 | 0.303 | 0.793 | 0.322 | 4.877 |
| OmniGen | 0.398 | 0.602 | 0.248 | 0.780 | 0.317 | 5.069 |
| OmniGen2 | 0.365 | 0.475 | 0.142 | 0.787 | 0.331 | 4.991 |
| FLUX.1 Kontext | 0.324 | 0.408 | 0.099 | 0.755 | 0.327 | 5.319 |
| Qwen-Image-Edit | 0.324 | 0.409 | 0.093 | 0.776 | 0.316 | 5.056 |
| GPT-4o Native | 0.425 | 0.579 | 0.178 | 0.794 | 0.311 | 5.344 |
| UNO | 0.304 | 0.428 | 0.141 | 0.765 | 0.314 | 4.923 |
| USO | 0.401 | 0.635 | 0.286 | 0.790 | 0.329 | 5.077 |
| UMO | 0.458 | 0.732 | 0.359 | 0.783 | 0.305 | 4.850 |
| UniPortrait | 0.447 | 0.677 | 0.265 | 0.793 | 0.319 | 5.018 |
| ID-Patch | 0.426 | 0.633 | 0.231 | 0.792 | 0.312 | 4.900 |
| InfU | 0.439 | 0.630 | 0.233 | 0.772 | 0.328 | 5.359 |
| PuLID | 0.452 | 0.705 | 0.315 | 0.779 | 0.305 | 4.839 |
| InstantID | 0.464 | 0.734 | 0.337 | 0.764 | 0.295 | 5.255 |
| Ours | 0.460 | 0.578 | 0.144 | 0.798 | 0.313 | 4.783 |
| GT | 1.000 | 0.521 | -0.999 | N/A | N/A | N/A |
| Ref | 0.521 | 1.000 | 0.999 | N/A | N/A | N/A |

Ground-truth-Aligned ID Loss. Since ArcFace embedding requires landmark detection and alignment, directly extracting landmarks from $I_{\text{gen}}$ is unreliable because generated images are obtained through noisy diffusion or one-step denoising. Prior methods compromise: PortraitBooth [39] applies the loss only at low noise levels ($t<0.25$), discarding supervision at higher noise, while PuLID [14] fully denoises generated results at significant computational cost. In contrast, we align the generated image using GT landmarks, thereby avoiding noisy landmark extraction. We minimize the cosine distance between GT-aligned ArcFace embeddings of the generated and ground-truth (GT) faces:

$$ $\mathcal{L}_{\text{ID}}=1-\cos(\mathbf{g},\mathbf{t})$ (4) $$

where $\mathbf{g}$ and $\mathbf{t}$ are ArcFace embeddings of the generated and GT images. This design (1) enables applying the ID loss across all noise levels, (2) incurs negligible overhead throughout training, and (3) implicitly supervises generated landmarks. Ablation studies (Sec. [6.3](https://arxiv.org/html/2510.14975v1#S6.SS3)) demonstrate more accurate identity measurement and substantially improved identity preservation.

Denoting the face recognition model as $f(\cdot,\cdot)$ (Arcface [11], in our case), and the coupled detection model as $g(\cdot)$ (RetinaFace [10]), the generated image as $\mathbf{G}$, and the ground-truth image as $\mathbf{T}$, a embedding extraction should be performed as follows:

$$ $\mathbf{t}=f(g(\mathbf{T}),\mathbf{T}),$ (5) $$

where $g(\mathbf{T})$ are the detected landmarks, and $f(\cdot,\cdot)$ extracts the aligned face embedding. Instead of using $g(\mathbf{G})$ as landmarks for $\mathbf{G}$, our GT-aligned ID loss is computed as:

$$ $\mathcal{L}_{\text{id}}=1-\cos(f(g(\mathbf{T}),\mathbf{G}),f(g(\mathbf{T}),\mathbf{T})).$ (6) $$

ID Contrastive Loss With Extended Negatives. To further strengthen identity preservation, we introduce an ID contrastive loss that explicitly pulls the generated image closer to its reference images in the face embedding space while pushing it away from other identities. The loss follows the InfoNCE [31] formulation:

$$ $\mathcal{L}_{\text{CL}}=-\log\frac{\exp(\cos(\mathbf{g},\mathbf{r})/\tau)}{\sum_{j=1}^{M}\exp(\cos(\mathbf{g},\mathbf{n}_{j}))/\tau)},$ (7) $$

where $\mathbf{r}$ is the embedding of a reference image of the same identity as the generated image, $\mathbf{n}_{j}$ are embeddings of $M$ negatives from different identities, and $\tau$ is a temperature hyperparameter. This formulation relies on ID-labeled datasets, which make it possible to draw thousands of negatives per sample from the reference bank, thereby greatly enriching the diversity of negative examples.

The overall training objective is a weighted sum of the above losses:

$$ $\mathcal{L}=\mathcal{L}_{\mathrm{diff}}+\lambda_{\text{ID}}\mathcal{L}_{\text{ID}}+\lambda_{\text{CL}}\mathcal{L}_{\text{CL}},$ (8) $$

where $\lambda_{\text{ID}}$ and $\lambda_{\text{CL}}$ are hyper-parameters controlling the contributions of the ID loss and contrastive loss, respectively. Both are set to $0.1$ across all training phases described below.

Figure: (a) Single-ID subset
Refer to caption: x13.png

**Table 2: Quantitative comparison on the multi-person subset of MultiID-Bench. , , and indicate the first-, second-, and third-best performance, respectively. For Copy-Paste ranking, only cases with $\mathrm{Sim(GT)}>0.35$ are considered. GPT exhibits prior knowledge of identities from TV series in subsets with more than two IDs, leading to abnormally high similarity scores.**
| Method | Identity Metrics | Generation Quality |  |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Sim(GT) $\uparrow$ | Sim(Ref) $\uparrow$ | CP $\downarrow$ | Bld $\downarrow$ | CLIP-I $\uparrow$ | CLIP-T $\uparrow$ | Aes $\uparrow$ |  |
| DreamO | 0.359 | 0.514 | 0.179 | 0.105 | 0.763 | 0.319 | 4.764 |
| OmniGen | 0.345 | 0.529 | 0.209 | 0.110 | 0.750 | 0.326 | 5.152 |
| OmniGen2 | 0.283 | 0.353 | 0.081 | 0.112 | 0.763 | 0.334 | 4.547 |
| GPT | 0.332 | 0.400 | 0.061 | 0.092 | 0.774 | 0.328 | 5.676 |
| UNO | 0.223 | 0.274 | 0.043 | 0.082 | 0.735 | 0.325 | 4.805 |
| UMO | 0.328 | 0.491 | 0.176 | 0.111 | 0.743 | 0.316 | 4.772 |
| UniPortrait | 0.367 | 0.601 | 0.254 | 0.075 | 0.750 | 0.323 | 5.187 |
| ID-Patch | 0.350 | 0.517 | 0.183 | 0.085 | 0.767 | 0.326 | 4.671 |
| Ours | 0.405 | 0.551 | 0.161 | 0.079 | 0.770 | 0.321 | 4.883 |

### 5.2 Training pipeline

Copy–paste artifacts largely arise from reconstruction-only training, which encourages models to replicate the reference image rather than learn robust identity-conditioned generation. Leveraging our paired dataset, we employ a four-phase training pipeline that gradually transitions the objective from reconstruction toward controllable, identity-preserving synthesis.

Phase 1: Reconstruction pre-training with fixed prompt. We begin with reconstruction pre-training to initialize the backbone, as this task is simpler than full identity-conditioned generation and can exploit large-scale unlabeled data. For the first few thousand steps, the caption is fixed to a constant dummy prompt (e.g., “two people”), ensuring the model prioritizes learning the identity-conditioning pathway rather than drifting toward text-conditioned styling. The full MultiID-2M is used in this phase, which typically lasts for $20\text{k}$ steps, at which point the model achieves satisfactory identity similarity. To further enhance data diversity, CelebA-HQ [23], FFHQ [24], and a subset of FaceID-6M [51] are also incorporated.

Phase 2: Reconstruction pre-training with full captions. This phase aligns identity learning with text-conditioned generation and lasts for an additional $40\text{k}$ steps, during which the model reaches peak identity similarity.

Phase 3: Paired tuning. To suppress trivial copy–paste behavior, we replace $50\%$ of the training samples with paired instances drawn from the $500\text{k}$ labeled images in MultiID-2M. For each paired sample, instead of using the same image as both input and target, we randomly select one reference image from the identity’s reference set and another distinct image of the same identity as the target. This perturbation breaks the shortcut of direct duplication and compels the model to rely on high-level identity embeddings rather than low-level copying.

Phase 4: Quality tuning. Finally, we fine-tune on a curated high-quality subset augmented with generated stylized variants to (i) enhance perceptual fidelity and (ii) improve style robustness and transferability. This phase refines texture, lighting, and stylistic adaptability while preserving the strong identity consistency established in earlier phases.

## 6 Experiments

Figure: Figure 6: Qualitative Results of Different Generation Methods. The text prompt is extracted from the ground-truth image shown on the leftmost side.
Refer to caption: x15.png

In this section, we present a comprehensive evaluation of baselines and our WithAnyone model on the proposed MultiID-Bench.

Baselines.
We evaluate two categories of baseline methods: general customization models and face customization methods. The general customization models include OmniGen [61], OmniGen2 [54], Qwen-Image-Edit [53], FLUX.1 Kontext [2], UNO [56], USO [55], UMO [8], and native GPT-4o-Image [32]. The face customization methods include UniPortrait [15], ID-Patch [68], PuLID [14] (referring to its FLUX [27] implementation throughout this paper), and InstantID [50]. All models were evaluated on the single-person subset of the benchmark, while only those supporting multi-ID generation were additionally tested on the multi-person subset.
Further implementation details are provided in Appendix [F.1](https://arxiv.org/html/2510.14975v1#A6.SS1).

### 6.1 Quantitative Evaluation

The quantitative results are reported in Tables [1(b)](https://arxiv.org/html/2510.14975v1#S5.T1.st2) and [2(b)](https://arxiv.org/html/2510.14975v1#S5.T2.st2). We observe a clear trade-off between face similarity and copy-paste artifacts. As shown in Fig. [5](https://arxiv.org/html/2510.14975v1#S5.F5), most methods align closely with a regression curve, where higher face similarity generally coincides with stronger copy-paste. This indicates that many existing models boost measured similarity by directly replicating reference facial features rather than synthesizing the identity. In contrast, WithAnyone deviates substantially from this curve, achieving the highest face similarity with regard to GT while maintaining a markedly lower copy-paste score.

WithAnyone also achieves the highest score among ID-specific reference models on the OmniContext [54] benchmark. However, VLMs [1, 32] exhibit limited ability to distinguish individual identities and instead emphasize non-identity attributes such as pose, expression, or background. Despite that general customization and editing models often outperform face customization models on OmniContext, WithAnyone still has best performance among face customization models.

### 6.2 Qualitative Comparison

To complement the quantitative results, Fig. [6](https://arxiv.org/html/2510.14975v1#S6.F6) presents qualitative comparisons between our method, state-of-the-art general customization/editing models, and face customization generation models.

It shows that identity consistency remains a significant weakness of general customization or editing models, consistent with our quantitative findings. Many VAE-based approaches where references are encoded through a VAE, such as FLUX.1 Kontext and DreamO tend to produce faces that either exhibit copy-paste artifacts or deviate markedly from the target identity. A likely reason is that VAE embeddings emphasize low-level features, leaving high-level semantic understanding to the diffusion backbone, which may not have been pre-trained for this task.
ID-specific reference models also struggle with copy-paste artifacts. For example, they fail to make the subject smile when the reference image is neutral and often cannot adjust head pose or even eye gaze. In contrast, WithAnyone generates flexible, controllable faces while faithfully preserving identity.

### 6.3 Ablation and User Studies

**Table 3: Ablation Study. indicate the first, second, third performance respectively. We ablate paired data training (without stage 2, w/o s2), GT-Aligned landmark ID loss (Self-aligned, S.A.), extended negative samples in InfoNCE (w/o neg). And model trained on FFHQ is also compared.**
|  | Ablation | Identity Metrics | Generation Quality |  |  |  |  |
| --- | --- | --- | --- | --- | --- | --- | --- |
| Sim(G) $\uparrow$ | Sim(R) $\uparrow$ | CP $\downarrow$ | CLIP-I $\uparrow$ | CLIP-T $\uparrow$ | Aes $\uparrow$ |  |  |
| Phases | w/o Phase 3 | 0.406 | 0.625 | 0.239 | 0.755 | 0.307 | 4.955 |
| Loss | w/o GT-Align | 0.385 | 0.549 | 0.175 | 0.763 | 0.317 | 4.754 |
| w/o Ext. Neg. | 0.368 | 0.455 | 0.074 | 0.740 | 0.304 | 4.984 |  |
| Data | FFHQ only | 0.224 | 0.246 | 0.027 | 0.658 | 0.330 | 5.039 |
| Ours | Full Setting | 0.405 | 0.551 | 0.161 | 0.770 | 0.321 | 4.883 |

Figure: Figure 7: Comparison of GT-aligned and Prediction-aligned landmarks.
Refer to caption: x16.png

To better understand the contribution of each component in WithAnyone, we conduct ablation studies on the training strategy, the GT-aligned ID loss, the InfoNCE-based ID loss, and our dataset. Due to space constraints, we report the key results here, with additional analyses provided in Appendix [G](https://arxiv.org/html/2510.14975v1#A7).

As shown in Table [3](https://arxiv.org/html/2510.14975v1#S6.T3), the paired-data fine-tuning phase reduces copy-paste artifacts without diminishing similarity to the ground truth, while training on FFHQ performs significantly worse than on our curated dataset. Fig. [7](https://arxiv.org/html/2510.14975v1#S6.F7) further demonstrates that the GT-aligned ID loss lowers denoising error at low noise levels and yields higher-variance, more informative gradients at high noise, thereby strengthening identity learning. By ablating extended negatives, leaving only $63$ negative samples from the batch (originally extended to $4096$), the effectiveness of ID contrastive loss is greatly reduced. More ablation results can be found in Appendix [G](https://arxiv.org/html/2510.14975v1#A7).

We conduct a user study to evaluate perceptual quality and identity preservation. Ten participants were recruited and asked to rank 230 groups of generated images according to four criteria: identity similarity, presence of copy-paste artifacts, prompt adherence, and aesthetics. The results, shown in Fig. [8](https://arxiv.org/html/2510.14975v1#S6.F8), indicate that our method consistently achieves the highest average ranking across all dimensions, demonstrating both stronger identity preservation and superior visual quality. Moreover, the copy-paste metric exhibits a moderate positive correlation with human judgments, suggesting that it captures perceptually meaningful artifacts. Further details of the study design, ranking protocol, and statistical analysis are provided in Appendix [H](https://arxiv.org/html/2510.14975v1#A8).

Figure: Figure 8: User study. Bigger bubbles indicate higher ranking.
Refer to caption: x18.png

## 7 Conclusion

Copy-paste artifacts are a common limitation of identity customization methods, and face-similarity metrics often exacerbate the issue by implicitly rewarding direct copying. In this work, we identify and formally quantify this failure mode through MultiID-Bench, and propose targeted solutions. We curate MultiID-2M and develop training strategies and loss functions that explicitly discourage trivial replication. Empirical evaluations demonstrate that WithAnyone significantly reduces copy-paste artifacts while maintaining and in many cases improving identity similarity, thereby breaking the long-standing trade-off between fidelity and copying. These results highlight a practical path toward more faithful, controllable, and robust identity customization.

## References

- [1]
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin.
Qwen2.5-vl technical report.
arXiv preprint arXiv:2502.13923, 2025.
- [2]
Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, Sumith Kulal, et al.
Flux. 1 kontext: Flow matching for in-context image generation and editing in latent space.
arXiv e-prints, 2025.
- [3]
Anthony Chen, Jianjin Xu, Wenzhao Zheng, Gaole Dai, Yida Wang, Renrui Zhang, Haofan Wang, and Shanghang Zhang.
Training-free regional prompting for diffusion transformers.
arXiv preprint arXiv:2411.02395, 2024.
- [4]
Bowen Chen, Mengyi Zhao, Haomiao Sun, Li Chen, Xu Wang, Kang Du, and Xinglong Wu.
Xverse: Consistent multi-subject control of identity and semantic attributes via dit modulation.
arXiv preprint arXiv:2506.21416, 2025.
- [5]
Weifeng Chen, Jiacheng Zhang, Jie Wu, Hefeng Wu, Xuefeng Xiao, and Liang Lin.
Id-aligner: Enhancing identity-preserving text-to-image generation with reward feedback learning.
arXiv preprint arXiv:2404.15449, 2024.
- [6]
Wei Cheng, Ruixiang Chen, Siming Fan, Wanqi Yin, Keyu Chen, Zhongang Cai, Jingbo Wang, Yang Gao, Zhengming Yu, Zhengyu Lin, et al.
Dna-rendering: A diverse neural actor repository for high-fidelity human-centric rendering.
In ICCV, 2023.
- [7]
Wei Cheng, Su Xu, Jingtan Piao, Chen Qian, Wayne Wu, Kwan-Yee Lin, and Hongsheng Li.
Generalizable neural performer: Learning robust radiance fields for human novel view synthesis.
arXiv preprint arXiv:2204.11798, 2022.
- [8]
Yufeng Cheng, Wenxu Wu, Shaojin Wu, Mengqi Huang, Fei Ding, and Qian He.
Umo: Scaling multi-identity consistency for image customization via matching reward.
arXiv preprint arXiv:2509.06818, 2025.
- [9]
Jiaming Chu, Lei Jin, Yinglei Teng, Jianshu Li, Yunchao Wei, Zheng Wang, Junliang Xing, Shuicheng Yan, and Jian Zhao.
Uniparser: Multi-human parsing with unified correlation representation learning.
TIP, 2024.
- [10]
Jiankang Deng, Jia Guo, Evangelos Ververas, Irene Kotsia, and Stefanos Zafeiriou.
Retinaface: Single-shot multi-level face localisation in the wild.
In CVPR, 2020.
- [11]
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou.
Arcface: Additive angular margin loss for deep face recognition.
In CVPR, 2019.
- [12]
discus0434.
aesthetic-predictor-v2-5.
[https://github.com/discus0434/aesthetic-predictor-v2-5](https://github.com/discus0434/aesthetic-predictor-v2-5), 2023.
Accessed: 2025-05-12.
- [13]
Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al.
Scaling rectified flow transformers for high-resolution image synthesis.
In ICML, 2024.
- [14]
Zinan Guo, Yanze Wu, Zhuowei Chen, Lang Chen, Peng Zhang, and Qian He.
Pulid: Pure and lightning id customization via contrastive alignment.
In NeurIPS, 2024.
- [15]
Junjie He, Yifeng Geng, and Liefeng Bo.
Uniportrait: A unified framework for identity-preserving single-and multi-human image personalization.
ICCV, 2025.
- [16]
Amir Hertz, Ron Mokady, Jay Tenenbaum, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or.
Prompt-to-prompt image editing with cross attention control.
ICLR, 2023.
- [17]
Jonathan Ho, Ajay Jain, and Pieter Abbeel.
Denoising diffusion probabilistic models.
NeurIPS, 2020.
- [18]
Xirui Hu, Jiahao Wang, Hao Chen, Weizhan Zhang, Benqi Wang, Yikun Li, and Haishun Nan.
Dynamicid: Zero-shot multi-id image personalization with flexible facial editability.
ICCV, 2025.
- [19]
Yuqi Hu, Longguang Wang, Xian Liu, Ling-Hao Chen, Yuwei Guo, Yukai Shi, Ce Liu, Anyi Rao, Zeyu Wang, and Hui Xiong.
Simulating the real world: A unified survey of multimodal generative models.
arXiv preprint arXiv:2503.04641, 2025.
- [20]
Junha Hyung, Jaeyo Shin, and Jaegul Choo.
Magicapture: High-resolution multi-concept portrait customization.
In AAAI, 2024.
- [21]
Liming Jiang, Qing Yan, Yumin Jia, Zichuan Liu, Hao Kang, and Xin Lu.
Infiniteyou: Flexible photo recrafting while preserving your identity.
ICCV, 2025.
- [22]
Qing Jiang, Lin Wu, Zhaoyang Zeng, Tianhe Ren, Yuda Xiong, Yihao Chen, Qin Liu, and Lei Zhang.
Referring to any person.
2025.
- [23]
Tero Karras, Timo Aila, Samuli Laine, and Jaakko Lehtinen.
Progressive growing of gans for improved quality, stability, and variation.
ICLR, 2018.
- [24]
Tero Karras, Samuli Laine, and Timo Aila.
A style-based generator architecture for generative adversarial networks.
In CVPR, 2019.
- [25]
Chanran Kim, Jeongin Lee, Shichang Joung, Bongmo Kim, and Yeul-Min Baek.
Instantfamily: Masked attention for zero-shot multi-id image generation.
arXiv preprint arXiv:2404.19427, 2024.
- [26]
Minchul Kim, Anil K Jain, and Xiaoming Liu.
Adaface: Quality adaptive margin for face recognition.
In CVPR, 2022.
- [27]
Black Forest Labs.
Flux.
[https://github.com/black-forest-labs/flux](https://github.com/black-forest-labs/flux), 2024.
- [28]
Black Forest Labs.
Flux.1 krea.
[https://huggingface.co/black-forest-labs/FLUX.1-Krea-dev](https://huggingface.co/black-forest-labs/FLUX.1-Krea-dev), 2025.
- [29]
Ziwei Liu, Ping Luo, Xiaogang Wang, and Xiaoou Tang.
Deep learning face attributes in the wild.
In ICCV, 2015.
- [30]
Chong Mou, Yanze Wu, Wenxu Wu, Zinan Guo, Pengze Zhang, Yufeng Cheng, Yiming Luo, Fei Ding, Shiwen Zhang, Xinghui Li, et al.
Dreamo: A unified framework for image customization.
SIGGRAPH Asia, 2025.
- [31]
Aaron van den Oord, Yazhe Li, and Oriol Vinyals.
Representation learning with contrastive predictive coding.
arXiv preprint arXiv:1807.03748, 2018.
- [32]
OpenAI.
Addendum to gpt-4o system card: Native image generation, 2025.
- [33]
Maxime Oquab, Timothée Darcet, Theo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Russell Howes, Po-Yao Huang, Hu Xu, Vasu Sharma, Shang-Wen Li, Wojciech Galuba, Mike Rabbat, Mido Assran, Nicolas Ballas, Gabriel Synnaeve, Ishan Misra, Herve Jegou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski.
Dinov2: Learning robust visual features without supervision.
arXiv:2304.07193, 2023.
- [34]
Dongwei Pan, Long Zhuo, Jingtan Piao, Huiwen Luo, Wei Cheng, Yuxin Wang, Siming Fan, Shengqi Liu, Lei Yang, Bo Dai, et al.
Renderme-360: A large digital asset library and benchmarks towards high-fidelity head avatars.
NeurIPS, 2023.
- [35]
Foivos Paraperas Papantoniou, Alexandros Lattas, Stylianos Moschoglou, Jiankang Deng, Bernhard Kainz, and Stefanos Zafeiriou.
Arc2face: A foundation model for id-consistent human faces.
In ECCV, 2024.
- [36]
Gaurav Parmar, Or Patashnik, Kuan-Chieh Wang, Daniil Ostashev, Srinivasa Narasimhan, Jun-Yan Zhu, Daniel Cohen-Or, and Kfir Aberman.
Object-level visual prompts for compositional image generation.
arXiv preprint arXiv:2501.01424, 2025.
- [37]
Or Patashnik, Rinon Gal, Daniil Ostashev, Sergey Tulyakov, Kfir Aberman, and Daniel Cohen-Or.
Nested attention: Semantic-aware attention values for concept personalization.
In SIGGRAPH, 2025.
- [38]
William Peebles and Saining Xie.
Scalable diffusion models with transformers.
In ICCV, 2023.
- [39]
Xu Peng, Junwei Zhu, Boyuan Jiang, Ying Tai, Donghao Luo, Jiangning Zhang, Wei Lin, Taisong Jin, Chengjie Wang, and Rongrong Ji.
Portraitbooth: A versatile portrait model for fast identity-preserved personalization.
In CVPR, 2024.
- [40]
Guocheng Qian, Kuan-Chieh Wang, Or Patashnik, Negin Heravi, Daniil Ostashev, Sergey Tulyakov, Daniel Cohen-Or, and Kfir Aberman.
Omni-id: Holistic identity representation designed for generative tasks.
CVPR, 2025.
- [41]
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.
Learning transferable visual models from natural language supervision.
In ICML, 2021.
- [42]
Xingyu Ren, Alexandros Lattas, Baris Gecer, Jiankang Deng, Chao Ma, and Xiaokang Yang.
Facial geometric detail recovery via implicit representation.
In FG, 2023.
- [43]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox.
U-net: Convolutional networks for biomedical image segmentation.
In MICCAI, 2015.
- [44]
Nataniel Ruiz, Yuanzhen Li, Varun Jampani, Yael Pritch, Michael Rubinstein, and Kfir Aberman.
Dreambooth: Fine tuning text-to-image diffusion models for subject-driven generation.
In CVPR, 2023.
- [45]
Florian Schroff, Dmitry Kalenichenko, and James Philbin.
Facenet: A unified embedding for face recognition and clustering.
In CVPR, 2015.
- [46]
Erich Schubert, Jörg Sander, Martin Ester, Hans Peter Kriegel, and Xiaowei Xu.
Dbscan revisited, revisited: why and how you should (still) use dbscan.
TODS, 2017.
- [47]
Lorenzo Stacchio, Alessia Angeli, Giuseppe Lisanti, Daniela Calanca, and Gustavo Marfia.
Imago: A family photo album dataset for a socio-historical analysis of the twentieth century.
arXiv preprint arXiv:2012.01955, 2020.
- [48]
Dani Valevski, Danny Lumen, Yossi Matias, and Yaniv Leviathan.
Face0: Instantaneously conditioning a text-to-image model on a face.
In SIGGRAPH Asia, 2023.
- [49]
Qinghe Wang, Xu Jia, Xiaomin Li, Taiqing Li, Liqian Ma, Yunzhi Zhuge, and Huchuan Lu.
Stableidentity: Inserting anybody into anywhere at first sight.
TMM, 2025.
- [50]
Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, and Anthony Chen.
Instantid: Zero-shot identity-preserving generation in seconds.
arXiv preprint arXiv:2401.07519, 2024.
- [51]
Shuhe Wang, Xiaoya Li, Jiwei Li, Guoyin Wang, Xiaofei Sun, Bob Zhu, Han Qiu, Mo Yu, Shengjie Shen, Tianwei Zhang, et al.
Faceid-6m: A large-scale, open-source faceid customization dataset.
arXiv preprint arXiv:2503.07091, 2025.
- [52]
Yibin Wang, Weizhong Zhang, Jianwei Zheng, and Cheng Jin.
High-fidelity person-centric subject-to-image synthesis.
In CVPR, 2024.
- [53]
Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu.
Qwen-image technical report.
arXiv preprint arXiv:2508.02324, 2025.
- [54]
Chenyuan Wu, Pengfei Zheng, Ruiran Yan, Shitao Xiao, Xin Luo, Yueze Wang, Wanli Li, Xiyan Jiang, Yexin Liu, Junjie Zhou, Ze Liu, Ziyi Xia, Chaofan Li, Haoge Deng, Jiahao Wang, Kun Luo, Bo Zhang, Defu Lian, Xinlong Wang, Zhongyuan Wang, Tiejun Huang, and Zheng Liu.
Omnigen2: Exploration to advanced multimodal generation.
arXiv preprint arXiv:2506.18871, 2025.
- [55]
Shaojin Wu, Mengqi Huang, Yufeng Cheng, Wenxu Wu, Jiahe Tian, Yiming Luo, Fei Ding, and Qian He.
Uso: Unified style and subject-driven generation via disentangled and reward learning.
arXiv preprint arXiv:2508.18966, 2025.
- [56]
Shaojin Wu, Mengqi Huang, Wenxu Wu, Yufeng Cheng, Fei Ding, and Qian He.
Less-to-more generalization: Unlocking more controllability by in-context generation.
ICCV, 2025.
- [57]
Tong Wu, Yinghao Xu, Ryan Po, Mengchen Zhang, Guandao Yang, Jiaqi Wang, Ziwei Liu, Dahua Lin, and Gordon Wetzstein.
Fiva: Fine-grained visual attribute dataset for text-to-image diffusion models.
NeurIPS, 2024.
- [58]
Yi Wu, Ziqiang Li, Heliang Zheng, Chaoyue Wang, and Bin Li.
Infinite-id: Identity-preserved personalization via id-semantics decoupling paradigm.
In ECCV, 2024.
- [59]
Chufeng Xiao and Hongbo Fu.
Customsketching: Sketch concept extraction for sketch-based image synthesis and editing.
In Computer Graphics Forum. Wiley Online Library, 2024.
- [60]
Guangxuan Xiao, Tianwei Yin, William T Freeman, Frédo Durand, and Song Han.
Fastcomposer: Tuning-free multi-subject image generation with localized attention.
IJCV, 2025.
- [61]
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing, Ruiran Yan, Chaofan Li, Shuting Wang, Tiejun Huang, and Zheng Liu.
Omnigen: Unified image generation.
arXiv preprint arXiv:2409.11340, 2024.
- [62]
Hengyuan Xu, Liyao Xiang, Hangyu Ye, Dixi Yao, Pengzhi Chu, and Baochun Li.
Permutation equivariance of transformers and its applications.
In CVPR, 2024.
- [63]
Yuxuan Yan, Chi Zhang, Rui Wang, Yichao Zhou, Gege Zhang, Pei Cheng, Gang Yu, and Bin Fu.
Facestudio: Put your face everywhere in seconds.
arXiv preprint arXiv:2312.02663, 2023.
- [64]
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang.
Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models.
arXiv preprint arxiv:2308.06721, 2023.
- [65]
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer.
Sigmoid loss for language image pre-training.
In ICCV, 2023.
- [66]
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.
Adding conditional control to text-to-image diffusion models.
In ICCV, 2023.
- [67]
Ning Zhang, Manohar Paluri, Yaniv Taigman, Rob Fergus, and Lubomir Bourdev.
Beyond frontal faces: Improving person recognition using multiple cues.
In CVPR, 2015.
- [68]
Yimeng Zhang, Tiancheng Zhi, Jing Liu, Shen Sang, Liming Jiang, Qing Yan, Sijia Liu, and Linjie Luo.
Id-patch: Robust id association for group photo personalization.
In CVPR, 2025.
- [69]
Youcai Zhang, Xinyu Huang, Jinyu Ma, Zhaoyang Li, Zhaochuan Luo, Yanchun Xie, Yuzhuo Qin, Tong Luo, Yaqian Li, Shilong Liu, et al.
Recognize anything: A strong image tagging model.
arXiv preprint arXiv:2306.03514, 2023.
- [70]
Yujie Zhong, Relja Arandjelovic, and Andrew Zisserman.
Compact deep aggregation for set retrieval.
In ECCV, 2018.
- [71]
Cailin Zhuang, Ailin Huang, Wei Cheng, Jingwei Wu, Yaoqi Hu, Jiaqi Liao, Hongyuan Wang, Xinyao Liao, Weiwei Cai, Hengyuan Xu, et al.
Vistorybench: Comprehensive benchmark suite for story visualization.
arXiv preprint arXiv:2505.24862, 2025.

## Appendix A Family of WithAnyone

FLUX.1 comprises a family of models, including FLUX.1 [27], FLUX.1 Kontext [2] and FLUX.1 Krea [28]. Krea is a text-to-image model with improved real-person face generation, whereas Kontext is an image-editing model that excels at making targeted adjustments while preserving the rest of the image. However, as reported in Table [1(b)](https://arxiv.org/html/2510.14975v1#S5.T1.st2), Kontext shows limited consistency with the reference face identity.

Our method, WithAnyone, can be seamlessly integrated into Kontext for the face customization downstream tasks like face editing. As illustrated in Fig. [9](https://arxiv.org/html/2510.14975v1#A1.F9), WithAnyone effectively injects identity information from the reference images into the target image.

The overall training pipeline follows the procedure described in Sec. [5](https://arxiv.org/html/2510.14975v1#S5), with a single modification: the input image provided to Kontext (whose tokens are concatenated with the noisy latent at each denoising step) is set to the target image with the face region blurred.

Figure: Figure 9: Application of WithAnyone-Kontext. Marrying editing models, WithAnyone is capable of face editing given customization references.
Refer to caption: x19.png

## Appendix B MultiID-2M Construction Details

To fill in the void left by the lack of publicly available multi-ID datasets, a data constraction pipeline is proposed to create a large-scale dataset of multi-person images with paired identity references for identities on the data record. Based on this pipeline, $500$k group photo images are collected, featuring $3$k identities, each with hundreds of single-ID reference images. Another $1$M images that cannot be identified are also included in the dataset for image reconstruction training purpose for image reconstruction training purpose.

### B.1 Dataset Construction Pipeline

The pipeline contains four steps, as shown in Fig. [3](https://arxiv.org/html/2510.14975v1#S2.F3). The detailed pipeline are as follows.

Figure: (a) ID Appearance.
Refer to caption: x20.png

Single-ID images. To construct a ID reference set, single-ID images were collected from the web using celebrity names as search queries on Google Images. For each image, facial features were extracted with ArcFace [42], ensuring that only images containing exactly one face were retained. To remove outliers, DBSCAN [46] clustering was applied to the embeddings for each celebrity, resulting in a set of cluster centers and hundreds of reference images per identity. This process established a reliable reference set for each unique identity. Human review confirms the accuracy of the ID bank built in this step.

Multi-ID images. To achieve best searching efficiency, group photos were obtained using more complex queries that combined multiple celebrity names, keywords indicating the number of people (e.g., “two celebrities”), scene descriptors (e.g., “award ceremony”), and negative keywords to filter out irrelevant results. ArcFace embeddings were extracted for these images, yielding a large pool of candidate multi-ID images. At this stage, the dataset comprised more than 20 million images.

Retrieval. To provide ID reference for the multi-ID images, it is necessary to retrieve the IDs on it. All single-ID cluster centers were aggregated into an embedding matrix. For each detected face in every multi-ID image, its ArcFace embedding was compared to all single-ID cluster centers to determine identity. The similarity between two embeddings was calculated as:

$$ $\mathrm{sim}(id_{1},id_{2})=\cos(f(id_{1}),f(id_{2}))$ (9) $$

where $id_{1}$ and $id_{2}$ denote two faces, and $f$ is the ArcFace embedding network.

Each face in a multi-ID image was assigned the identity of the single-ID cluster center with the highest similarity, provided the similarity exceeded a predefined threshold (0.5). This approach enabled accurate and automated identity assignment in group images and facilitated retrieval of corresponding reference images.

Filtering and labelling. To further improve dataset quality, a series of annotation and filtering steps were applied. The Recognize Anything model [69], an aesthetic score predictor [12], and other auxiliary tools were used for annotation. Images with low aesthetic scores or those identified as collages rather than genuine group photos were excluded. Optical Character Recognition (OCR) tools detected watermarks and logos, which were cropped out when possible; otherwise, the images were discarded. Finally, descriptive captions were generated for the images using a large language model, enriching the dataset with textual information.

So far, a dataset with three parts is obtained: (1) $1$M single-ID images as reference bank, or single-ID cross-paired training; (2) $500$k paired multi-ID images with identified persons; (3) $1$M unpaired multi-ID images, which can be used for training scenario without the need of references, such as reconstruction.

Figure: (a) Clothes & Accessories Distribution.
Refer to caption: x23.png

### B.2 Dataset Statistics

Following prior arts [29, 7, 6, 34], comprehensive statistics of the dataset are provided in Fig. [11](https://arxiv.org/html/2510.14975v1#A2.F11), including the distribution of nationalities, the count of appearances per identity, and a word cloud illustrating the most frequent terms in the generated image captions, offering insights into the diversity and richness of the dataset.
A long-tail distribution is observed in the count of appearances per identity in Fig. [11(a)](https://arxiv.org/html/2510.14975v1#A2.F11.sf1), with a few identities appearing frequently while many others are less common. This provide a diverse set of identities, as well as a perfect test dataset without identity interaction with the training set. Fig. [11(b)](https://arxiv.org/html/2510.14975v1#A2.F11.sf2) and Fig. [10(c)](https://arxiv.org/html/2510.14975v1#A2.F10.sf3) illustrate MultiID-2M’s nationality distribution and action diversity respectively.
The comparison between the proposed dataset and existing multi-ID datasets are listed in Table [4](https://arxiv.org/html/2510.14975v1#A2.T4), highlighting MultiID-2M’s outstanding volume and paired references.

**Table 4: Statistic comparison for multi-identity group photo datasets. #Img refers to total scale of the dataset; #Paired refers to paired group image number; #Img / ID indicates number of reference image for each single ID; #ID / Img means number of IDs appears on group photos.**
| Dataset | #Img | #Paired | #Img / ID | #ID / Img |
| --- | --- | --- | --- | --- |
| IMAGO [47] | $80$k | 0 | 0 | - |
| MHP [9] | $5$k | 0 | 0 | $2-10$ |
| PIPA [67] | $40$k | $40$k | cross | $1-10$ |
| HumanRef [22] | $36$k | $36$ | $1+$ | $1-14+$ |
| Celebrity Together [70] | $194$k | 0 | 0 | $1-5$ |
| MultiID-2M | $1.5$M | $500$k | $100+$ | $1-5$ |

## Appendix C Benchmark and Metrics Details

Most existing methods are evaluated on privately curated test sets that are seldom released, and even when datasets are shared, the accompanying evaluation protocols vary widely. For example, ID-Patch [68] and UniPortrait [15] measure identity similarity using ArcFace embeddings, whereas UNO [56] relies on DINO [33] and CLIP similarity scores. This heterogeneity together with the common practice of reporting only the cosine similarity between matched ArcFace embeddings fails to capture more nuanced insights and can even encourage degenerate behavior in which models produce images that are effectively “copy‑pastes” of the reference photos.

In this work, MultiID-Bench  is introduced as a unified and extensible evaluation framework for group photo (multi-ID) generation. It standardizes assessment along two complementary axes: (i) identity fidelity (preserving each target identity without unintended copying and blending), and (ii) generation quality (semantic faithfulness to the prompt/ground truth and overall aesthetic quality).

The data used in MultiID-Bench  are drawn from the long-tail portion of MultiID-2M. We first select the least frequent identities and gather all images containing them. To prevent information leakage, the training split is filtered to ensure zero identity overlap with the benchmark set. The final benchmark contains 435 samples; each sample provides 1–4 reference identities (with their images), a corresponding ground-truth image, and a text prompt describing that ground-truth scene.

Identity Blending. In the similarity matrix, the off-diagonal elements correspond to the similarity between different identities. The average of the diagonal elements is used as the metric for identity fidelity, and the average of the off-diagonal elements serves as the metric for identity blending, as in Eq. [10](https://arxiv.org/html/2510.14975v1#A3.E10).

$$ $\mathrm{M_{Bld}}(x^{g},x^{t})=\frac{1}{N^{2}-N}\sum_{i=1}^{N}\sum_{j=1,j\neq i}^{N}\cos(g_{i},t_{j})$ (10) $$

where $g_{i}$ is the embedding of the $i$-th face in the generated image $x^{g}$, and $t_{j}$ is the embedding of the $j$-th face in the ground-truth image $x^{t}$. A lower value indicates less unintended blending between different identities, which is desirable.

Generation quality. The overall generation quality is evaluated based on CLIP-I and CLIP-T, which are the de facto standards for evaluating the prompt-following capability [41], are employed to measure the cosine similarity in the CLIP embedding space between the generated image and the ground truth image or caption. Additionally, an aesthetic score model [12] is used to assess the aesthetic quality of the generated images.

## Appendix D Galleries of WithAnyone

Figure: Figure 12: Galleries of Single-ID Generation.
Refer to caption: x25.png

Figure: Figure 13: Galleries of 2-person Generation.
Refer to caption: x26.png

Figure: Figure 14: Galleries of 3-to-4-person Generation.
Refer to caption: x27.png

We show more results of WithAnyone in Fig. [12](https://arxiv.org/html/2510.14975v1#A4.F12), Fig. [13](https://arxiv.org/html/2510.14975v1#A4.F13), and Fig. [14](https://arxiv.org/html/2510.14975v1#A4.F14).

## Appendix E Model Framework Details

We follow prior work [64, 14] and integrate a lightweight identity adapter into the diffusion backbone. Identity embeddings are injected by cross-attention so that the base generative prior is preserved while controllable identity signals are added.

Face embedding. Each reference face is first encoded by ArcFace, producing a $1\times 512$ identity embedding. To match the tokenized latent space of the DiT backbone, this vector is projected with a multi-layer perceptron (MLP) into $8$ tokens of dimension $3072$ (i.e., an $8\times 3072$ tensor). This tokenization provides sufficient capacity for the cross-attention layers to integrate identity cues without overwhelming the generative context.

Controllable attribute retention. Completely suppressing copy-like behavior is not always desirable: users sometimes expect certain mid-level appearance attributes (e.g., hairstyle, accessories) to be preserved. ArcFace focuses on high-level, identity-discriminative geometry and texture cues but omits many mid-level semantic factors. To expose controllable retention of such attributes when needed, we optionally incorporate SigLIP [65] as a secondary encoder. SigLIP provides more semantically entangled representations, enabling selective transfer of style-relevant traits while ArcFace anchors identity fidelity.

Attention mask and location control. To further improve identity disentanglement and precise localization in the generated images, an attention mask and location control mechanism are incorporated [3, 62]. Specifically, ground-truth facial bounding boxes are extracted from the training data and used to generate binary attention masks. These masks are applied to the attention layers of the backbone model, ensuring that each reference token only attends to its corresponding face region in the image, providing location control at the same time.

Feature injection. After each transformer block of the DiT backbone, we inject face features through a cross-attention modulation:

$$ $H^{\prime}=H+\lambda_{\text{id}}\,\text{softmax}\!\left(\frac{(HW_{Q})(EW_{K})^{\top}}{\sqrt{d}}+M\right)(EW_{V}),$ (11) $$

where $H$ denotes the current hidden tokens, $E$ the stacked face-embedding tokens, and $W_{Q},W_{K},W_{V}$ the projection matrices; $d$ is the query/key dimension, and $\lambda_{\text{id}}=1.0$ during training. When SigLIP is enabled, its tokens are processed by a parallel cross-attention with an independent scaling coefficient.

## Appendix F Experimental Details

### F.1 Implementation Details

WithAnyone is trained on 8 NVIDIA H100 GPUs, with a batch size of $4$ on each GPU. The learning rate is set to $1e^{-4}$, and the AdamW optimizer is employed with a weight decay of $0.01$.
The pre-training phase runs for $60k$ steps, with a fixed prompt used during the first $20k$ steps. The subsequent paired-tuning phase lasts $30k$ steps: $50\%$ of the samples use paired (reference, ground-truth) data, while the remaining $50\%$ continue reconstruction training. Finally, a quality/style tuning stage of $10k$ steps is performed with a reduced learning rate of $1\times 10^{-5}$.

For the extended ID contrastive loss, the target is used as the positve sample, while other IDs from samples in the same batch serve as negative samples. With the global batch size of $32$, this yields less than a hundred negative samples. Extended negative samples are drawn from reference bank. If this ID is identified as one of the $3$k ID in the reference bank, we simply omit its own ID and draw the from other IDs. If this ID is not identified, then it makes things easier – all the IDs in the reference bank can be used as negative samples.

For other baseline methods, official implementations and checkpoints (or API) are used with default settings. Methods are tested on MultiID-Bench and real-human subset of OmniContext [54]. OmniContext uses Vision-Language Models (VLMs) to evaluate the prompt-following (PF) and subject-consistency (SC) of generated images. For reproducibility, the VLM is fixed to Qwen2.5-VL [1]. ID-Patch [68] requires pose condition, and we use the ground-truth pose for it.

Single face embedding model may induce biased evaluation on ID similarity, thus we average three de-facto face recognition models’ consine similarity to compute the overall ID similarity metric, namely ArcFace [11], FaceNet [45], and AdaFace [26].

### F.2 More Discussion on the Quantitative Results

The performance of GPT on our 3‑and‑4‑people subset offers a useful validation of our copy‑paste metric, as shown in Table [2(b)](https://arxiv.org/html/2510.14975v1#S5.T2.st2). This subset largely comprises group photographs from TV series that GPT may have encountered during pretraining, so GPT attains unusually high identity‑similarity scores both to the ground truth (GT) and to the reference images. Actually, in one case GPT even generates an ID from the TV series that is not present in the reference images. This behaviour approximates an idealized scenario in which a model fully understands and faithfully reproduces the target identity: similarity to GT and to references are both high, and the copy‑paste measure the difference between distances to GT and to references approaches zero. These observations are consistent with our metric design and support its ability to distinguish true identity understanding from trivial copy‑and‑paste replication.

We report the experimental limit in Table [1(b)](https://arxiv.org/html/2510.14975v1#S5.T1.st2). If one model completely copy the reference image, $\mathrm{Sim_{GT}}=0.521$, $\mathrm{Sim_{Ref}}=1.0$, and copy-paste is $0.999$, which aligns with the theoretical limit $1.0$ of copy-paste.

The prompt-following ability is measured by CLIP-I and CLIP-T in our benchmark, and is judged by VLM in OmniContext. WithAnyonegains state-of-the-art performance in both metrics, and is ranked the highest in our user study. However, the credibility of CLIP scores and the aesthetic scores may be debated, as they are not always consistent with human perception.

## Appendix G Ablation Study Details

Figure: Figure 15: ID Loss Curves with $\lambda\times$ InfoNCE Loss. $0.1$ is $0.1\times$ InfoNCE Loss without extended negative samples, and $0.1+$ is $0.1\times$ InfoNCE Loss with extended negative samples.
Refer to caption: x28.png

In this section, we systematically evaluate the impact of training strategy, GT-aligned ID-Loss, InfoNCE ID Loss, and our dataset construction. User study is also conducted to validate the consistency of the proposed metrics with human perception, as well as evaluate the human preference on different methods.

SigLIP signal. SigLIP [65] signal is introduced to retain copy-paste effect when user tend to retain the features from reference images like hairstyle, accessories, etc. As shown in Fig. [16](https://arxiv.org/html/2510.14975v1#A7.F16), increasing the SigLIP signal weight effectively amplifies the copy-paste effect while simultaneously boosting ID similarity to the reference images exactly as expected, since stronger SigLIP guidance enforces tighter semantic alignment and transfers more fine-grained appearance cues (e.g., hairstyle, accessories, local textures).

Training strategy. We evaluate the effect of a paired-data fine-tuning stage. After an initial reconstruction training phase, we either continue training with paired (reference, ground-truth) data or keep training under the reconstruction objective for 10k steps. As shown in Table [3](https://arxiv.org/html/2510.14975v1#S6.T3), continuing with paired data effectively reduces the copy-paste effect without compromising similarity to the ground truth.

Dataset construction. To validate the effectiveness of our dataset, we trained a model on FFHQ [24] using reconstruction training for the same number of steps. As shown in Table [3](https://arxiv.org/html/2510.14975v1#S6.T3), the FFHQ-trained model performs poorly across all metrics. This likely stems from FFHQ’s limited diversity and size, as it contains only  70k face-only portrait images.

GT-aligned ID-Loss. We validate the GT-aligned ID-Loss with a simple experiment that visualizes predicted faces at different denoising time steps during training. As shown in Fig. [7](https://arxiv.org/html/2510.14975v1#S6.F7), at low noise levels the GT-aligned ID-Loss is substantially lower than the loss computed using prediction-aligned landmarks, indicating that aligning faces to ground-truth landmarks reduces denoising error and yields a more accurate identity assessment. At high noise levels the GT-aligned ID-Loss shows greater variance, producing stronger and more informative gradients that help the model learn identity features.

InfoNCE Loss. The InfoNCE loss with extended negative samples is crucial for the convergence in the early training stage. We conduct a toy experiment with 1000 training samples, and record ID Loss curves with no InfoNCE loss, $0.1\times$ InfoNCE loss without extended negatives, and $0.1\times$ InfoNCE loss with extended negatives. As shown in Fig. [16](https://arxiv.org/html/2510.14975v1#A7.F16), ID loss fits a lot faster with InfoNCE loss with extended negatives, demonstrating its effectiveness in accelerating training convergence. It also largely increases the ID similarity score, as shown in Table [3](https://arxiv.org/html/2510.14975v1#S6.T3).

**Table 5: Correlation Statistics Between Machine Ranking and Human Ranking. Reported values include Pearson’s $r$, Spearman’s $\rho$, and Kendall’s $\tau$ with corresponding $p$-values.**
| Dimension (N) | Pearson $r$ (p) | Spearman $\rho$ (p) | Kendall $\tau$ (p) |
| --- | --- | --- | --- |
| Copy-Paste | $0.4417$ ($7.98e{-48}$) | $0.4535$ ($1.26e{-50}$) | $0.3405$ ($1.10e{-46}$) |
| ID Sim | $0.3254$ ($1.54e{-26}$) | $0.3237$ ($2.91e{-26}$) | $0.2423$ ($1.11e{-25}$) |

## Appendix H User Study Details

Figure: Figure 17: User Study Interface.
Refer to caption: files/user_study_interface.jpg

Our user study is conducted with the same data samples and generated results in our quantitative experiments. Due to a tight financial budget, we randomly select $100$ samples from single-person subset, $100$ samples from 2-people subset, and all samples from 3-and-4 people subset. 10 participants are recruited for the study, all of whom are trained with a brief tutorial to understand the task and evaluation criteria.

We illustrate the interface used in our user study in Fig. [17](https://arxiv.org/html/2510.14975v1#A8.F17).

### H.1 Correlation Analysis

We analyze the correlation between our proposed metrics and user study results. As shown in Table [5](https://arxiv.org/html/2510.14975v1#A7.T5), our copy-paste metric shows a moderate positve correlation with user ratings on copy-paste effect.

### H.2 Participant Instructions

We provide the instructions for training the participants in the following table.

#### Identity similarity

#### Copy-and-paste effect (excessive mimicry of the reference)

#### Prompt following

#### Aesthetics

## Appendix I Prompts for Language Models

Large language models (LLMs) and vision-language models (VLMs) are used in various stages of our work, including dataset captioning and OmniContext evaluation.

### I.1 Dataset Captioning

Besides the system prompt, we design 6 different prompts to generate diverse captions for each image. 1 prompt is randomly selected for each image during captioning.

## Peer Reviews

confidence: 4
contribution: 3
initial_rating: 8
presentation: 3
questions: Generalization to Non-Celebrity or Low-Quality Data: Since MultiID-2M primarily contains celebrity faces with high-quality, well-lit imagery, how does WithAnyone perform on non-celebrity or in-the-wild datasets with lower resolution, occlusions, or varied lighting conditions? Would the model’s performance degrade significantly outside of this curated domain?

Dataset Ethics and Bias: Although the paper addresses ethical data sourcing and the exclusion of minors, could the authors elaborate on how potential biases (e.g., nationality, ethnicity, or gender imbalance) in MultiID-2M might impact the model’s fairness and generalization? Have any bias analyses or mitigation strategies been conducted?

Reproducibility and Accessibility: The paper states that only a small subset of MultiID-2M is included in the supplementary material. Are there concrete plans or timelines for releasing the full dataset and benchmark, or alternative means (e.g., data access agreements) to allow other researchers to replicate results?


Validity of the Copy-Paste Metric: The proposed Copy-Paste (MCP) score depends on cosine distances in face embedding space. How sensitive is this metric to the choice of embedding model (e.g., ArcFace vs. ElasticFace or CurricularFace etc)? What about face verfic?ation metrics, e.g., FMR10000? Have the authors verified that the metric aligns with perceptual similarity judgments beyond the limited user study?
soundness: 3
strengths: The paper’s main strengths lie in its clear problem formulation, methodological thoroughness, and comprehensive evaluation. It identifies and formalizes the “copy-paste” artifact in identity-consistent image generation, offering a concrete metric to quantify this often-overlooked issue. The introduction of the MultiID-2M dataset and MultiID-Bench benchmark represents a valuable infrastructural contribution, enabling more systematic and reproducible research on multi-identity generation. The proposed WithAnyone framework integrates a ground-truth–aligned identity loss and a contrastive loss with extended negatives in a diffusion-based pipeline, leading to measurable improvements in controllability and identity preservation. Extensive experiments, strong ablation studies, and user evaluations convincingly support the claimed benefits, while the paper’s clear presentation, detailed appendix, and ethical considerations reflect a high level of polish and maturity in execution.
summary: This paper makes a strong and timely contribution to controllable, identity-consistent image generation by introducing WithAnyone, a diffusion-based framework trained with a novel contrastive identity loss and a ground-truth–aligned ID loss to mitigate “copy-paste” artifacts—where models overfit to reference faces instead of generalizing identity
weaknesses: The proposed “copy-paste” failure mode, while intuitively appealing, overlaps conceptually with previously discussed overfitting and reconstruction issues in personalization models such as DreamBooth or PuLID, making the novelty somewhat incremental rather than fundamental.

The dataset MultiID-2M, though large, is primarily composed of celebrity data scraped from the web, raising concerns about bias, consent, and representativeness; furthermore, the lack of public access to the full dataset severely limits reproducibility and undermines the paper’s claim to openness. Evaluation metrics like the Copy-Paste score depend heavily on specific embedding spaces (e.g., ArcFace), which could bias results toward models aligned with that embedding and may not fully capture perceptual similarity.

The use of a similarity metric is clearly insufficient to justify identity consistency. Additional face verification metrics are needed, EER, FRM1000 etc.

---

confidence: 4
contribution: 3
initial_rating: 6
presentation: 3
questions: Please see Weaknesses for full details. In summary, weakness 1 and 2 need clarifications and updating manuscript to more precisely summarize the motivation and contribution of this paper, and weakness 3 needs additional experiments and explanations to help audience better understand the proposed method, e.g. which components/properties contribute the most. I am happy to raise my rating if my concerns are well addressed, especially for weakness 3, because I think such ablations provide new understanding and shed light on future work, adding strength to this paper.
soundness: 3
strengths: 1. There are previous benchmark papers which propose datasets of similar functionalities, but MultiID-2M is larger in scale (2M images) and uniquely focuses on group photos with labeled identities, beyond objects, anime characters, and human faces. The rich annotation also enables fine-grained control aspects. Thus, I think the high quality of MultiID-2M is the main strength/contribution of this paper and cannot be substituted by simply assembling a few previous datasets.

2. Using contrastive loss directly on reference images sounds clever to me. There are previous work which adds contrastive loss to diffusion model, e.g. Customcontrast [1] pulls representations of the same subject closer. WithAnyone links target, generated, and reference images together via reconstruction loss + contrastive loss, which repurposes contrastive learning in another way, but I think this paper is among the first papers to leverage contrastive learning here, which sounds interesting.

[1] Chen, Nan, et al. "Customcontrast: A multilevel contrastive perspective for subject-driven text-to-image customization." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 39. No. 2. 2025.
summary: This paper terms the failure mode of the reconstruction-based diffusion training where it fails to produce diversity as **copy-paste**, such that it focuses too much on fidelity and cannot adapt to human preference and diversity requirement. This paper proposes a well annotated MultiID-2M benchmark to provide diverse references for each identity under multi-person scenarios, addressing balance between identity preservation and generation diversity with a contrastive identity loss (and corresponding metrics to measure performance).
weaknesses: 1. The intuition/motivation from the ''copy-paste'' effect has been proposed and studied [2,3], where they also leverage multiple images per identity to resolve overfitting and target leakage that leads to copy-paste effect. Then, it will be better for the authors to compare their datasets and proposed methods to demonstrate the improvements from this work -- I agree MultiID-2M is more comprehensive and unique as I summarized in strength 1, but it will be good to discuss it in the manuscript to better illustrate the motivation and improvement. 

2. Following weakness 1, since it is a grounded problem with previous studies, this paper shares some similarities in their approach and evaluation. For example, [2,3] use multiple different images of the same identity as the target image, and [3] also proposes metrics to measure the alignment/balance between fidelity and diversity. This weakens the novelty of this work for the motivation part.

3. I hope there can be more ablation studies to demonstrate what makes this method works better, besides the good MultiID-2M data. A. I see ablation on number of negative examples for InfoNCE, but I think it is more of parameter tuning, can the authors show results w/o contrastive loss at all? B. Compared to previous work, is the capability of learning from MultiID reference images brought by larger models than initially SD 1.4/1.5? I am curious about what contributes the most to make WithAnyone works well on MultiID-2M, whether it is due to contrastive loss or larger model's expressivity. I suggest try baseline minus CL loss, baseline with reduced model size, and baseline minus CL loss and with reduced model size.

[2] Li, Zhen, et al. "Photomaker: Customizing realistic human photos via stacked id embedding." Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2024.

[3] Tang, Haoran, et al. "Retrieving conditions from reference images for diffusion models." arXiv preprint arXiv:2312.02521 (2023).

---

confidence: 4
contribution: 2
initial_rating: 6
presentation: 3
questions: How Ground-truth-Aligned ID Loss is computed is still unclear to me. How do authors obtain the generated face image to compute this loss at high noise level? Is it to estimate an approximate $z_0$ or to run the whole denoising process to obtain a clean $z_0$? I would expect a more detailed explanation as the current description in the paper doesn't distinguish WithAnyone with existing methods in obtaining generated images.
soundness: 3
strengths: 1. The paper makes a conceptual step forward by explicitly distinguishing between the reference face and the intended target face, moving toward more controllable identity-consistent person generation.

2. It contributes a large-scale paired dataset with diverse identities and multi-person compositions, which is likely to be valuable for future work on identity-conditioned generation.

3. It proposes new evaluation metrics designed to capture the copy-paste failure mode in identity-consistent generation, offering a more diagnostic way to assess whether a model is truly preserving identity rather than naively pasting faces.

4. The proposed model achieves strong results and is competitive with SOTA methods in both single-person and multi-person generation settings.
summary: The paper studies identity-consistent image generation conditioned on a reference face image and a text prompt. The authors argue that current identity-preserving generators often fall into a "copy-paste" failure mode: instead of synthesizing a coherent new face, they directly paste the reference face into the generated scene, leading to visible artifacts in pose, expression, and lighting mismatch.

To address this, the authors introduce (1) a large-scale paired dataset containing multi-person images together with target face identities, aimed at training models that handle multi-person composition; and (2) a new evaluation benchmark with metrics designed to distinguish between naive face copy-paste and proper identity transfer, by explicitly comparing the generated face to both the provided reference and the intended target identity.

Building on these resources, the paper proposes a four-phase finetuning pipeline based on the Flux image generation model, following the PuLID architecture, to improve multi-identity conditioning and reduce copy-paste behavior. Empirically, the finetuned model WithAnyone is reported to perform competitively with SOTA approaches in both single-person and multi-person generation settings.
weaknesses: 1. *Single-person quality not clearly superior*. In the single-subject setting, the qualitative comparisons do not convincingly demonstrate an advantage of WithAnyone. In particular, the claim that the method avoids copy–paste style artifacts would be more credible if shown under challenging conditions (extreme lighting, large pose changes, strong facial expressions). The revision should include more targeted visual examples in these stress cases.

2. *Underpowered user study*. The user study is based on only 10 raters, which limits statistical reliability and generalizability. A larger participant pool is needed, ideally with demographic diversity and significance reporting.

3. *Limited ablations*. The ablation analysis is incomplete. It would be valuable to (i) report performance after each training phase/stage to show incremental gains, and (ii) include a baseline trained with only the diffusion/denoising loss to isolate the benefit of the proposed objectives.

4. *Unclear ID contrastive loss formulation*. Equation 5 defines the ID Contrastive Loss using features from the reference image rather than the generated (target) image, but the paper does not justify this design choice or discuss alternatives. The rationale and expected effect on identity preservation need to be explained.

5. *Inconsistent table highlighting*. The visual highlighting in Tables 1 and 2 appears misleading. For example, in Table 1(a), Qwen-Image-Edit seems to have the best CP score among non-ground-truth methods, but WithAnyone is marked as “best.” The authors should fix or clarify the ranking/formatting so that highlighted cells actually correspond to the strongest scores.

---

confidence: 5
contribution: 3
initial_rating: 6
presentation: 4
questions: 1. All cases in this paper are based on celebrity images. What are the results when applying the model to ordinary people or synthetic portraits?
soundness: 4
strengths: 1. Solid experiments: A large number of data experiments and qualitative results support the observations presented in this paper.
2. Promising research direction: Most existing works on ID consistency focus on ID consistency itself, while systematic research on editability-related topics is lacking. This paper conducts in-depth exploration of this idea, proposing corresponding data-driven solutions and relevant benchmarks. I believe the research community should invest more efforts in this area to achieve a better trade-off between similarity and editability, rather than overemphasizing similarity.
summary: To address the conflict between identity consistency and controllability in text-to-image generation—where existing models tend to suffer from the "copy-paste" issue —the authors construct the large-scale MultiID-2M dataset, which contains 500k group photos with paired references and 1.5 million unpaired group photos. They also design MultiID-Bench, a benchmark for quantifying copy-paste artifacts and the trade-off between ID fidelity and variation. Additionally, the authors propose the WithAnyone model based on the FLUX architecture. By leveraging an ID contrastive loss (with an extended negative sample pool) and a ground-truth-aligned ID loss, combined with a four-stage training pipeline (reconstruction pre-training with fixed prompts, reconstruction pre-training with full captions, paired fine-tuning, and quality tuning), WithAnyone reduces copy-paste artifacts while maintaining high ID similarity. Experiments validate its advantages in pose/expression controllability and perceptual quality.
weaknesses: 1. Lack of innovation in the experimental design: Most of the methods adopted have been explored by previous studies (e.g., contrastive loss), or only involve minor modifications based on prior work (e.g., GT-Align).
2. The experimental results are suboptimal: The reduction in similarity is quite significant. For instance, the second ID from the right in the bottom image of Figure 12 does not closely resemble the reference identity.


Decision: Conditional Accept (Poster)