Durian: Dual Reference Image-Guided Portrait Animation with Attribute Transfer
Authors: Hyunsoo Cha
Byungjun Kim
Hanbyul Joo
Venue: ICLR 2026 Poster (2026)
Keywords: Attribute Transfer
Portrait Animation
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: Given a portrait and one or more reference images specifying target attributes, Durian generates a portrait animation with attribute transfer in a single-stage pipeline.

Abstract: We present Durian, the first method for generating portrait animation videos with cross-identity attribute transfer from one or more reference images to a target portrait. Training such models typically requires attribute pairs of the same individual, which are rarely available at scale. To address this challenge, we propose a self-reconstruction formulation that leverages ordinary portrait videos to learn attribute transfer without explicit paired data. Two frames from the same video act as a pseudo pair: one serves as an attribute reference and the other as an identity reference. To enable this self-reconstruction training, we introduce a Dual ReferenceNet that processes the two references separately and then fuses their features via spatial attention within a diffusion model. To make sure each reference functions as a specialized stream for either identity or attribute information, we apply complementary masking to the reference images.
Together, these two components guide the model to reconstruct the original video, naturally learning cross-identity attribute transfer.
To bridge the gap between self-reconstruction training and cross-identity inference, we introduce a mask expansion strategy and augmentation schemes, enabling robust transfer of attributes with varying spatial extent and misalignment. Durian achieves state-of-the-art performance on portrait animation with attribute transfer. Moreover, its dual reference design uniquely supports multi-attribute composition and smooth attribute interpolation within a single generation pass, enabling highly flexible and controllable synthesis.

Published as a conference paper at ICLR 2026
DURIAN: DUAL REFERENCE IMAGE-GUIDER PORTRAIT ANIMATION WITH ATTRIBUTE TRANSFER
Hyunsoo Cha, Byungjun Kim, Hanbyul Joo
Scoul National University
22 3443st stehenn, byung jun. k kim, hb joo) @snu. kr
Portrait
Glasses
Portrait Animation with Glasses Transfer
Portrait
Hair
Portrait Animation with Hair Trassfer
Glasses
Reard
Portrait
Hair
HHt
Portzait Animation with Multi-Altribute Transfer
Figure 1: Portrait Animation with Attribute Transfer. Given portrait image and single or multiple reference images specifying target attributes (e.g., hairstyle, eyeglasses), method generates portrait animation with facial attribute transfer conditioned on keypoint sequence.
ABSTRACT
We present Durian, the first method for generating portrait animation videos with cross-identity attribute transfer from one or more reference images to target por- trait. Training such models typically requires attribute pairs of the same individual, which are rarely available at scale. To address this challenge, we propose a self- reconstruction formulation that leverages ordinary portrait videos to learn attribute transfer without explicit paired data. Two frames from the same video act as a pseudo pair: one serves as an attribute reference and the other as an identity refer- ence. To enable this self-recone oeffrrcconttr training, we introduce Dual ReferenceNet that processes the two references separately and then fuses their features via spatial attention within diffusion model. To make sure each reference functions as specialized stream for either identity or attribute information, we apply complemen- tary masking to the reference images. Together, these two components guide the model to reconstruct the original video, naturally learning cross-identity attribute transfer. To bridge the gap between self-reconstrruction training and cross-identity inference, we introduce a mask expansion strategy and augmentation schemes, enabling robust transfer attributes with varying spatial extent and misalignment. Durian achieves state-of-the-art performance on portrait animation with attribute transfer. Moreover, its dual reference design uniquely supports autti- composition and smooth attribute interpolation within single generation pass, enabling highly flexible and controllable synthesis. Project page and code are available at: hotps: httpss//hhhnnoohhhaee github. io/durian

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Personalized appearance editing, such as virtually trying on glasses or experimenting with new hairstyles, is becoming a key feature of virtual styling applications. However, most existing solutions are highly specialized and limited in scope. Hairstyle preview apps typically rely on fixed templates, which may look realistic from a single view but fail to adapt to head pose expression changes. Glasses try-on systems often depend on pre-scanned 3D product models, restricting users to a predefined catalog. Furthermore, these systems focus multiple elements, such as hair, glasses, or hats, within a unified experience.
single attribute and cannot combine
A key challenge in building such system is obtaining suitable training data. Disentangling identity from attributes ideally requires paired images of the same person with different attributes, which are rarely available and expensive collect at scale. This difficulty grows exponentially for multiple attributes, as capturing all combinations quickly becomes infeasible. For example, Li et al. (2023) collect multi-view images of subjects wearing different eyeglasses to model realistic glasses try-on, but the dataset remains too limited to generalize broadly. ZZhng et al. (2025) propose a synthetic pipeline that predicts a bald version of portrait and generates reference hair images using pretrained diffusion model. However, this approach is not easily scalable beyond hair.
This naturally raises the question: can we train a model for portrait animation with attribute transfer without any explicit altribute-paired data? Motivated by this question, we propose self- reconstruction framework that learns this task directly from widely available in-the-wild portrait videos. During training, we randomly sample two frames from single video: one the attribute reference and the other as the identity reference. The remaining frames are treated as targets to be generated, conditioned on keypoint sequence representing the motion of the video. To prevent identity leakage we apply complementary masking to the two reference frames so that the network must disentangle and combine the attribute and identity information to reconstruct the original video.
To enable this framework, we design a Dual ReferenceNet architecture that explicitly encodes the attribute and portrait references through two separate branches and fuses their disentangled features for generation via spatial attention. This design enables the network to move beyond simple pose driving, generating keypoint-driven portrait animations that seamlessly combine the attribute from one image with the identity from the other. Surprisingly, although the model is trained with only a single attribute reference at time, the spatial attention mechanism allows more advanced operations at inference time. Since different attributes (e.g.. hair, glasses, beard, hats) occupy distinct spatial regions, their features can be jointly injected without conflict, enabling seamless multi-attribuu transfer. Furthermore, by interpolating the features of two attribute references, our model can achieve attribute interpolation, generating smooth transitions between the attributes. These emergent capabilities make our framework especially valuable for real- world styling scenarios, where users may want to explore diverse combinations and gradual transformations of facial attributes.
While self reconsrccntttittoon training is effective for learning to separate identity and attributes, it operates within a single video, leading to a domain gap when the model is applied to cross- cross-identtyy inference, where the attribute and portrait come from different individuals. To mitigate this gap, we introduce a mask expansion strategy and lightweight augmentation schemes. These techniques expose the model to a broader range of attribute configurations during training, enabling robust transfer across spatial and structural variations of the attribute region. These designs form a unified framework capable of robust cross-identity attribute transfer. As a result, our method achieves versatile system that generates portrait animations with diverse appearance edits in a zero-shot manner.
We summarize the key contributions of our work, as follows:
We propose the first method to generate keypoint-driven portrait animations with transferred attributes directly from two images, generalized across diverse facial attributes beyond hair.
We design Dual ReferenceNet architecture that disentangles attribute and identity through two branches fused via spatial attention, enabling selftreconstruuctonn training directly on uncurated in-the-wild videos without paired data.
We propose mask expansion strategy and lightweight augmentations to bridge the domain gap for cross-identity transfer, improving robustness to diverse spatial configurations.
Our framework exhibits an emergent ability to support multi-attribute composition and interpolation in single generation pass, without requiring any additional training.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
Face Editing. Generative models have advanced facial editing from unconditional synthesis to fine-grained manipulation of existing images (GoodIellow et al. 2014 Rezende & Mohamed] 2015
Ho et al. 2020). Latent-space editing with StyleGAN (Karras et al. (020) and GAN inversion (Zhu
et al. 2016 Abdal et al. 2019 Richardson et al., [022]) has been extended to video via latent
trajectory modeling (Yao et al. 2021 TTaaaan et al. (022) and 3D-aware editing (Bilecen et al. 2024]
Xu et al. 2024). However, such approaches often rely on attribute classifiers or fixed editing controls. Diffusion-based models have introduced more flexible editing through prompt-driven (Brooks et al., 2023) or edentity-preservin techniques YYe ct al.
2023; Wang et al. 2024). with extensions to video improving temporal consistency (Ku et al. 2022 Kim et al. 2023 Still, these methods are limited to modifying existing content and cannot generate new motions or expressions.
Diffusion-based Attribute Transfer. Diffusion- attribute transfer methods typically formulate editing as masked inpainting, where reference content is inserted into target image using explicit
masks (Yang et al.) 2023] Chen et al.] 2024, Mou et al., 2025 Chen et al. 2025 Song et al.
2025). These approaches have been adapted to domain- specific tasks such as hairstyle (Zhang et al.
2025 Chung et al., 2025), clothing (Kim et al. 2024a Li et al. 2024| Chong et al. (024). and
makeup (Zhang al. 2024b). While effective for static images, they rely on category labels mask annotations. Video extensions (Fang et al.] 2024 Tu et al.] 2025) apply per-frame inpainting with post-hoc smoothing. but predefined masks are hard specify for deformable facial attributes that vary over time. Recent works have also explored attribute transfer in 3D avatars (Kim et al. 2024b Nam al. 2025] Cha et al., 2024 2025] Wang et al. 2025 Kim et al. 2025), but such approaches often require specialized capture setups or are not easily generalizable to in in-the-wild scenarios, In contrast, our model performs attribute transfer and animation jointly in a single forward pass, conditioned only on a pair of reference images and facial keypoint sequence. This eliminates the need for per-fram per-frame masks, text prompts, or category labels, enabling zero-shot transfer of diverse facial attributes.
Portrait Animation from Single Image. Portrait animation aims to generate motion from
static image, typically guided by facial keypoints, audio, or motion trajectories. Early methods rely on GANs with implicit keypoint modeling (Guo al. 2024 Wang et al. 202I), while recent approaches use diffusion models (Hu) 2024 Zhu et al. 2024 Yang et al. 2025 for improved realism and temporal stability. These methods primarily focus on reenactment and identity preservation. Others incorporate paired motion (Xie et al. (2024) or audio (Yang et al. 2025). but require multi- stage inference or fine-tuning. Our model jointly performs facial attribute transfer and motion generation, producing photorealistic, ddentity-preservig videos from diverse attribute references and keypoint-driven motion in a single pass.
3 METHOD
3.1 OVERVIEW: LEARNING ATTRIBUTE TRANSFER FROM SELF-RECCOSTRUCCION
We propose diffusion-basse generative framework for portrait animation with cross-identity attribute transfer. At high level, our model generates an F-frame animation sequence V {I,) as:
V = Durrian(aatt:: Mattr; Iport, Mport. K),
(I)
conditioned on an attribute image Lattrs portrait image Iport. and a sequence of driving facial keypoint images K = ] Each reference image has a binary mask: Mattr localizes the attribute region (e.g., hair or glasses) in the reference image, while Mport specifies the candidate region in the portrait where the attribute will be transferred. Using these masks, we construct two masked inputs: the attribute-nlly image Iatu Lattr C Mattr, where only the attribute region is preserved, and the attrbbutemmsked portrait image Iport Iport @ (1 Mport), where the corresponding region is removed. These masked inputs are fed into the Dual ReferenceNet, consisting of the Attribute ReferenceNet (ARNet) and Portrait ReferenceNet (PRNet), which extract multi-scale spatial features. These features are then injected into a diffusion- based generator, the Denoising UNer (DNer), to synthesize the remaining frames of the video with keypoint guidance K (Section 3.2).
3

---

Published as a conference paper at ICLR 2026
PRNNet
Expansion (Eg,
Aftenton
ARNO
Fipet
Antheranny Terage
Vadao Pounse
Speiial
Sdff
Cooe
Pertua Acteuitoon
Spacial (Eay (50)
Overview of Training Pipeline. Given an attribue-masken portrait image I port and an attribute-only image Lattr- Durian synthesizes portrait animation with the transferred attribute. These inputs are constructed by randomly sampling two frames from a training video and applying the estimated masks. A sequence of facial keypoints {k, ) is extracted from the video to guide the motion. During generation, spatial features from PRNet and ARNet are fused via spatial attention into the DNet, ensuring identity preservation and attribute consistency in the synthesized video.
Figure 2:
To enable training without requiring explicitly annotated triplets (i.e., combinations of a target attribute image, an original portrait image, and an edited portrait image), we adopt self-reconstruction strategy based on portrait videos (Yu et al. 2023 Xie 2022). Specifically, we simulate attribute transfer by sampling two frames Lttr and port from the same video, treating one as the attribute
reference and the other as the target portrait. We then construct the masked inputs Iattr and Iort using the same masking formulation as inference, based on a segmentation mask of randomly selected attribute. Although the two frames come from the same identity, the complementary masking enforces a clear separation between identity and attribute inputs, encouraging the model to learn meaningful mappings from these features output frames without requiring cross-identity supervision. To enhance the model's ability to generalize beyond the self attribut transfer setup. we introduce an augmentation scheme that improves robustness to spatial and appearance variations(Section) 3.3).
At inference time, we estimate refined attribute masks by aligning the attribute image to the portrait through lightweight alignment process, mitigating spatial misalignment between them. Conditioned on the two masked reference images and the driving keypoint sequence, our model then synthesizes portrait animations with attribute transfer. Notably, our design also supports multi-attribute composi- tion and smooth interpolation within a single generation pass, without requiring additional training or post-processing (Section 3.4.. Fig. shows our generated portrait animations with attribute transfer.
3.2 MODEL ARCHITECTURE: DUAL REFERENCENET
Inspired by recent approaches (Guo et al. 2023 Hu 2024 Zhu et al. 2024) that leverage Refer- enceNet to inject spatial features into diffusion models, we propose a Dual ReferenceNet architecture tailored for portrait animation with attribute transfer. Unlike previous work, our model includes two separate encoders: Attribute ReferenceNet (ARNet) and Portrait ReferenceNet (PRNet), each sharing the same architecture as the Denoising U-Net (DNet) in the diffusion model, excluding the temporal layers. The networks follow the U-Net (Long et al., 2015) architecture used in latent diffusion models (Rombach al. 2022), with each block containing convolutional layers followed by self- and cross-attention modules. The overall architecture is shown in Fig 2
Reference inputs. Given an attribute image Lattr E R and a portrait image Ioort E x3 along with their binary masks Mattr E x and Mport E x which localize the attribute region and the candidate transfer region respectively, we construct two masked inputs: the attribute-only image Ī Iattr @ Matrr where only the attribute region is preserved, and the atrrbbut-masked portrait image Ioort Ioott ② Mport), where the corresponding candidate
4

---

Published as a conference paper at ICLR 2026
region is removed. We then encode these masked images into latent representations using the pretrained VAE from the latent diffusion model (Rombach et al. 2022), yielding Eattr: 2port E RexAxw The corresponding masks Mattr. Mport are downsampled to match the latent resolution, producing Mattr port E 1xxxx These downsampled masks are concatenated with the latents along the channel dimension to form (c + 1) )>-chnneel inputs Eattr, Eport. E (e(+1)xxxx as follows:
attr concatc(Zattr: matte),
purt
concate (xport Mport).
(2)
Spatial attention. The augmented latents are passed to ARNet Eater and PRNet Eport to extract multi-scale feature maps after convolutional layers of each block:
Fattr (Faurt (ãater Oattr); Foort == {FL port
(port(2port: port))
where θ (attr.port) are the parameters of Dual ReferenceNet. Let F IR x denote the feature map of the frame т at the 1-th block of the denoising U-Net. While the original denoising U-Net includes a self-attention layer at each resolution, we replace it with our spatial attention to integrate identity and attribute features in spatially-awar manner. We denote width-wise concatenation as concat (-), and define our spatial attention SA(:...)
FTJ ref. = concalw ((F)*) ppon: F Fatur)) E Boxxh x3mm FTd SF F port> Fl. Flurr) (ottention(WoF"" W Frd
(4)
ref WvF ret,)):
(5)
where F. E Rexxkxxx is the feature map after the spatial attention, Attention(Q, K. softmax(QN (ORixxQQT///III
is the standard scaled dot-product attention (Vaswani al. 2017). Wo. WK. Wv are linear projection layers. This width-wise concatenation preserves spatial resolu- tion and allows the model to attend across all positions in the combined reference and target features. As result, the model can leverage both attribute and portrait guidance at every step.
Cross-attention with semantic embeddings. After applying spatial attention, we further inject semantic guidance into both the Dual ReferenceNet and the denoising U-Net via cross-attention. For ARNet, we use the CLIP (Radford et al. 2021) embedding of the attribute-only image Latte as the attribute embedding @atur which is injected via cross-attention into each block of ARNet. For PRNet and DNet, we construct portrait embedding &port by combining ArcFace (Deng et al. 2019) and
CLIP embeddings of the attribute-masker portrait image Iorrt following Animatt (Tu et al. 2024). This embedding is injected into both PRNet and DNet to enhance identity preservation. We define the cross-attention operation CA(::))
CA(F, Φ) Attention W/F, W WV
(6)
where F is the input feature map, & is the conditioning embedding, and W Wik Wir are learned linear projections. Let Fattr and F Foort be the self-attended features of the i-th block in ARNet and PRNet, and F. the spatially attended feature of DNet. Then, the cross-altention updates are given by:
F (nattr,port)
CA(F (nstr,port): & ((attr,portt),, Fr CA(F7 @port),
(7)
where Fi F' attre ports and F. are the feature maps after cross-attention in ARNet, PRNet, and DNet.
Temporal extension and keypoint guidance. Our model incorporates temporal awareness to generate coherent portrait animations by inserting temporal self-attention into each U-Net block,
following Hu (024)) Zhu et al. (2024). To control pose and expression, we use sequence of facial [ 1--1 extracted by Sapiens (Kirookkar et al. 2024). Each keypoint image k is encoded into spatial feature map Fi.. via pose encoder and combined with the noisy latent z(()
keypoints K
following Zhu et al. 2024).. For each frame т, DNet co predicts the added noise e!T7) from the noisy latent x! at timestep f, using the reference features, semantic embeddings, and keypoint features:
2 co (2(")) ,t, Fattr, Fport, Pattr: Φpart: kpp
(8)
The predicted noise is used to recover the denoised latent x0 then decoded by the VAE decoder D to produce the final video frame I, D(=(1)) for 7 1...., F.
5

---

Published as a conference paper at ICLR 2026
3.3 TRAINING STRATEGY
Training loss. To effectively train our model, we adopt a two-stage training scheme following the previous approaches (Hu 2024 Zhu al.) 2024). In the first stage, we optimize the entire model except the temporal attention layers, treating each video frame as independent training sample. We define the per-frame conditioning bundle as C' == (Fattr- Fport Pattr. portt) where Fport Fatty are the multi-scale spatial features from PRNet and ARNet and Pprrt Pattr are the semantic embeddings. Then, the training objective the standard denoising diffusion loss:
dir Ezo,c,t c,6 [Ic- co (Zz, t, Flpe))l
where 2; is the noised latent diffusion timestep t. is the sampled noise, and kpt the feature map of the corresponding facial keypoint image. In the second stage, we freeze all modules except the temporal attention layers and train them using multi- frame inputs. The temporal objective considers sequence of noised latents and corresponding keypoints:
((2) E ( Ff e |||tt-- (f2 C, ( (pppt)== ) diff
(10)
where e1.F = {e(r) denotes the per-frame noise sequence. This staged training improves convergence and allows the temporal attention module to focus on modeling motion dynamics without disrupting the spatial fidelity learned in the first stage.
Attribute-aware mask expansion. To ex- pose the model to diverse spatial extents of fa- cial attributes during training. we introduce attribute-awar mask expansion strategy, illus- trated in the top right of Fig. 2 Given a training frame I, we first select a target attribute (e.g., hair, eyeglasses, beard) and obtain its binary mask Mattr using Sapiens (Khirodkar et al. 2024). To simulate variation in the shape and coverage of this attribute, we generate modi- fied image Igen with SDXL (Podell et al. (203) and ControlNet (hang al. 2023), conditioned on the facial keypoints of and a text prompt de-
Mins
Pentrill Apon
Mpai
Alignet
1
Altibute Insae
Man.
Lau
Figure 3: Aligned Attribute Mask Estimation.
To improve porribtttt-oaaittt alignment, esti- mate an aligned attribute mask via Face Aligner.
scribing an altered appearance (e.g.. "long wavy hair"). To enable fully automated prompt generation without any human intervention, we construct dictionary of descriptive attribute modifiers (e.g., long, short, wavy, curly) and randomly sample their combinations to generate prompts for image generation. A new mask Mgen is then extracted from Igen using Sapiens. The final training mask is computed as the union of the original and generated masks, and the two masked inputs are constructed as:
M Mirain = Mattr U gens I = IO Mattr: iport I O
M )
(11)
where denotes element- wise multiplication. Here, Matty localizes the original attribute region, while Mtriie defines the expanded region into which the attribute will be inserted during generation. it preserves the intended attribute category while diversifying its spatial extent. Unlike HairFusion (Chung et al.) 2025). which expands masks using fixed heuristics specific to hair, our approach generalizes across multiple facial attributes and enables the model to learn spatially flexible yet semantically grounded transfer patterns.
This expansion process is atrrbbuteaware
Reference image augmentation. To address the limited diversity of reonstrecottt setups, we introduce an augmentation pipeline that improves robustness to pose, alignment, and appearance variations in attribute-portrait pairs. We perturb both the attribute-only and masked portrait images to simulate realistic spatial and photometric variations. We apply random affine transformations (trans- lation, scaling, rotation) to induce spatial misalignment, and use the FLUX outpainting model (Labs) 2024) to inpaint newly exposed regions. Additionally, color jittering tone, contrast, saturation, and hue accounts for appearance variations. This strategy exposes the model to diverse configurations, enabling more robust attribute transfer and animation under real-world variations.

---

Published as a conference paper at ICLR 2026
3.4 INFERENCE FRAMEWORK AND EXTENSIONS
Inference pipeline. At inference time, our system takes as input a portrait image, an attribute image, and a keypoint sequence. We first construct two masked reference images: the attribute-only image Lattr and the attribute-maske portrait image port. by applying segmentation masks predicted by Sapiens (Khirodkar et al.) (024) to the attribute image lattr and the portrait image port. To improve spatial alignment between the attribute and portrait inputs, we introduce Face Aligner module, which repurposes a lightweight ma-ge-to-3D avatar model (Chu & Harada (024) solely for alignment. This module reconstructs coarse 3D avatar from the attribute image and aligns its shape and pose to the portrait using FLAME (Li et al. 2017) parameters (3, 0, v/) estimated by EMOCA (Dančček al. 2022). From the resulting pose-aligned image attr we extract refined attribute mask Malig" using Sapiens. This mask is then merged with the initial portrait mask Mpor
to define the final transferable region Miderr M port U attr The updated mask is applied to construct the final attribute-maan maskedd portrait image, [prrt = port (1 MMpor); as illustrated in Fig. 3] Finally, spatial features F and semantic embeddings Pattr Pport extracted from the two masked reference images. Conditioned on these features and the keypoint sequence,
DNet synthesizes video of the target identity with the desired attribute through iterative denoising (Eq. (8)
Multi-attribute transfer. Our model supports zero-shot composition of multiple attributes without additional training, by generalizing the spatial attention formulation in Eq. (5) Instead of using single attribute feature, we concatenate multiple attribute feature maps along the width dimension:
F = SA FF F
port: concat, (F'. 1 astee> attrs
F1.2
F...Nut))) )),
(12)
where each F.. denotes the feature map extracted from the k-th attribute-only image using the ARNet. To construct the final attribute- attribute-masked masked portrait in this setting, we also generalize the mask fusion process by taking the union of all aligned attribute masks:
Nater
Lileer
M port
U U A=1
Malign.k
(13)
actr
where each Malign& the aligned mask extracted from the k-th attribute image. This composite mask is then used to remove all attribute regions from the portrait image before generation. The rest of the attention computation remains unchanged, allowing the model to jointly attend all attributes and synthesize coherent multi-attribute compositions without retraining.
Attribute interpolation. Our model enables zero-shot interpolation between two attributes of the same category (e.g., hairstyle A and B) without fine-tuning (Zhang et al. 2024a; Cha et al. 2025). Given two attribute-only images, extract spatially attended features F. and F7..2 using spatial attention, and interpolate them as follows:
F = (1 a) FuLll a FT.42
(14)
where E [0, 1] controls the interpolation ratio. The interpolated feature F is then passed DNet for generation. This enables smooth and semantically consistent transitions between attributes.
EXPERIMENTS
Experimental setup. To address the lack of ground-truth data for cross-identity attribute transfer, we design two evaluation settings: self-attribute transfer and cross-atribute transfer. In self-attribute transfer, a single video is split into a portrait and an attribute image from different frames of the same identity, and the model reconstructs the original video. While useful for controlled evaluation, this provides only a pseudo ground-trun truth and mainly reflects reconstruction ability rather than the full complexity of cross-identity transfer. In cross-attribute transfer, the portrait and attribute
7

---

Published as a conference paper at ICLR 2026
Table 1: Quantitative Comparison. We compare our method with recent approaches that (1)
synthesize portraits with transferred hairstyles, and (2) animate the synthesized portrait image.
Self-Attribute Transfer
Cros-Aitribute Transfer
Ien.
Animation
La
PSNRT
SSIM
LPIPS
FID
mCLIP-I"
IDINO!
ID-Sim.7
VPID,o
ViIDaasex
LivePortrait
0.1059
3,3868
PbB
X-Portrait
0.1180
15.33
0,5270
0.2977
59.20
0.8393
0.5916
0.5458
36,7030
2,9008
MegAcctor>>
0.1268
14.82
0.4840
0.3157
62.77
0,8535
0.6266
0.4863
38.2746
6.2743
LivePortrait
0.143
13.76
0:4801
0.3792
46.34
0.6512
30.5632
2.6719
HairFusion
X-Portrait
0.1511
13.30
0.4334
0.3733
59.02
0.8509
0.6914
0.6520
30,2570
4,9184
MogActor-3 Σ
0.1650
12.75
0,4138
0.4015
65.59
0.8736
0.6708
0.6044
30,9702
5,3057
LivePortrait
0.6564
29.5014
X-Portrait
0.1229
0.6447
MegActor- Σ
0.1301
14.62
0,4706
0.3347
63.47
0,8548
0.7271
0.6130
30,4087
1.4672
LivePortrait
0.1023
16.52
0.5511
0.2924
57.86
0.8540
0,6163
0.2776
32,5600
8.9103
TriplancFdit
X-Portrait
0.1051
16.05
0.5401
0.2760
0,8366
0.6216
2.3355
MegActor-) Σ
0.1248
15.10
0.4828
0.3295
7044
0.8210
015674
0.2770
32,5679
2.8542
Ours
0.0744
1883
0.6527
0,1568
38,00
0.9043
0,7801
0.7098
27.1547
2,4052
SH++X
Ours
Portrait
Hair
Portrait Animation with Attribute Transfer
Figure 4: Qualitative Comparison for Cross-Attribute Transfer. We compare our method and the baselines that combine X-Portrait (Xie et al., 2024) with StableHair (Zhang al., 2025) in cross-identity transfer setup. We provide more results our Supp. Mat.
images come from different individuals. Without exact ground-truth, this setting instead evaluates semantic consistency, identity preservation, and temporal realism. Together, the two settings offer comprehensive evaluation of both low-level fidelity and high-level transfer quality.
Dataset. We train model on CexebV-Text YYu et al. 2023). VFHQ Xie al. 2022). and Nersemble (Kirschstein et al. 2023), totaling 2.747 videos. For evaluation, we sample 200 videos for self-attribute transfer and 50 videos for cross-atrribute transfer from CelehV-Text and VFHQ, ensuring diverse and unseen identities, head poses, and expressions. The masks for the portrait and attribute frames are generated following the procedure used in each compared method.
Metries. For self-attribute transfer, we evaluate reconstruction fidelity using L,, PSNR, SSIM, and LPIPS, and perceptual quality with FID (Parmar et al.) (022).. For cross cots-attribbte transfer, measure
attribute transfer quality with CLIP-I (Radford et al. 2021 Hessel et al. 2021) and DINO (Caron
al., 2021), identity preservation with ArcFace (Deng al. 2019), and temporal realism with VFID (Fang et al. 2024) using I3D (Carreira & Zisserman 2017) and ResNeXt (Hara et al. 2018).
4.1 COMPARISON
Baselines. As no prior work directly tackles portrait animation with attribute transfer from in-the- wild reference images, we construct two-stage baselines by combining image-level attribute transfer with video animation methods, resulting in 12 model combinations. For attribute transfer (stage 1),
we consider: Paint-by-Example (PbE) (Yang et al.) (023)) mask-conditioned diffusion method for reference image insertion; HairFusion (Chung et al., 2025) and StableHair (Zhang et al. (022).. diffusion-basee based models for hairstyle transfer with and without masks; and TriplaneEdit (Bilecen et al. 2024). a 3D-aware GAN-based face editor. For portrait animation (stage 2). we use: LivePortrait (Guo al. 2024 X-Portrait (Xie et al. 2024), and MegActor->. (Yang al. 2025).
8

---

Published as a conference paper at ICLR 2026
Portrait
Hair
Single ReferenceNet
w/o mask expansion
wio rof. image sug.
ref. image input
Clurs
mask input
Figure 5: Ablation Study. Omitting components or altering training scheme degrades visual quality.
Results. As shown T our method consistently outperforms baseline combinations across both fidelity and perceptual quality metries in self-attribute transfer. Fig. presents a qualitative comparison against baselines using LivePortrait (Guo et al. 2024) as the animation module (stage 2). Our method generates coherent and realistic hairstyle animations that preserve the identity and maintain consistency in spatial extent, shape, and fine details across frames. Please refer to our Supp. Mat. for additional qualitative comparisons with other baseline combinations.
4.2 ABLATION STUDY
We evaluate the contributions of key compo- nents in our model and training strategy. Ta- ble 2 presents quantitative results, and Fig. 5 shows corresponding qualitative comparisons. "single ReferenceNet" replaces the braacb architecture with shared encoder that receives the portrait and attribute images concatenated along the channel dimension, following CAT- VTON (Chong et al. 2024). This setup fails to separate the roles of the two inputs, resulting in undesired blending of attribute and identity cues.
Table 2: Ablation Study. Bold indicates the best, underline the second.
Variant
L 4
PSNRT
SSIM
LPIPS]
single ReferenceNet
0.0813
17.95
0.6314
0.1973
w/o expansion
0.0881
0.5915
0.2073
wlo ref. image
0.0900
16.97
0.5973
0.2248
wlo mask input
0.0747
fall ref. image inpus
0.0670
19.47
0.6698
0.1310
Ours
0.0744
18.83
0.6527
0.1565
"w/o mask expansion" omits the attribute-awar -ware augmentation that simulates variations in spatial extent. Without this strategy, the model tends to rely on the default shape the portrait's original attribute mask, making it less capable of handling diverse attribute shapes during inference. "wwo ref. image aug." disables spatial and photometric augmenta- tions applied to the reference images during training. As a result, the model fails to accurately transfer the desired attribute with misaligned reference images. "w/o ref. mask input" removes the binary mask concatenation from the inputs to the ReferenceNets. This weakens spatial localization and often leads to artifacts residual traces of the original attribute in the output. "full ref. image input" uses unmasked portrait and attribute images during training. Interestingly, this variant achieves the best quantitative scores in Table which evaluates the self-atrribuee transfer setting, since full images simplify the task by allowing the model to copy content more easily. However, as shown in Fig. 5] this model fails disentangle identity and attribute roles, leading visible identity leakage during cross-identiyy transfer. Ours achieves spatially consistent, identity-preservinn results, and quantitatively outperforms all other ablated variants except the full reference image variant.
4.3 APPLICATION
Multi-attribute transfer. Our model supports the composition multiple attributes (e.g., glasses, hat, hairstyle) in a single generation pass by extending the spatial attention mechanism as described in Eq. 12 Fig. 6 shows qualitative results where multiple attributes are simultaneously transferred from different reference images. Remarkably, seamlessly but also handles interactions between overlapping regions, such as between hair and a hat. Despite the reference images exhibiting diverse lighting conditions and spatial alignments, the model
model not only combines multiple attributes

---

Published as a conference paper at ICLR 2026
5
S
Portruit
Hair
Glasses
Beard
Hat
Portrait Animation with Attribute Transfer
Figure 6: Multi-Attribute Transfer. Our model supports composition of multiple attributes (e.g.. hair, eyeglasses, beard, hat) in single forward pass without additional training.
Portrait
Hair A
Hair B
0.0
0.2
0.4
a
0.6
Q =
0.8
=
1.0
Portrait
Beard A
Beard B
0.0
0.2
0.4
0.6
0.8
1.0
Portrait
Eyeglasses A
Eyeglasses B
a 0.0
a 0.2
a 0.4
a
11.6
a 11.8
a
1.0
Portrait
Hat A
Hat B
a 0.0
= 0.2
0.4
0.6
0.8
1.0
Figure 7: Attribute Interpolation. Our method generates zero-shot, single-stage portrait animations with interpolated attributes, even for rigid objects such as hats and eyeglasses. The animations interpolate naturally according to the values.
successfully integrates all attributes into the portrait image while maintaining coherent and natural appearance.
Attribute interpolation. Our model enables attribute interpolation by linearly blending the ref- erence features of two attributes, as in Eq. (I4). Fig. 7 shows the results with smooth transitions in shape and appearance. The interpolations exhibit smooth changes in visual attributes. demonstrating that our model effectively captures semantically meaningful directions in the attribute feature space.
5 CONCLUSION
We present Durian, a zero-shot framework for portrait animation with cross-identity attribute transfer, given portrait image and one more reference images specifying the target attributes. Our diffusion model, equipped with a Dual ReferenceNet, leamns attribute transfer directly from uncurated portrait videos through reconstccnttttii training strategy, eliminating the need for triplet supervision. This is further enhanced by our attribute-awe mask expansion and augmentation scheme. Moreover, Durian naturally extends to multi-attributt composition and attribute interpolation within a single generation pass, without requiring any additional training.
Ackmowledgments. This work was supported by NRF grant funded by the Korean government (MSIT) (No. 20--202--NR070490 and IITP grant funded by the Korea government (MSIT) [No. RS 2544224440449844, No. -000160, No. RS-2025-2544411 and No. RS-2021-I12 13333).31 H.
Joo is the corresponding author.
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Rameen Abdal, Yipeng Qin, and Peter Wonka. Image2stylegan: How to embed images into the stylegan latent space? In Proceedings of the IEEE International Conference on Computer Vision, 2019.
Bahri Batuhan Bilecen, Yigit Yalin, Ning Yu, and Aysegul Dundar. Reference-based 3d-aware image editing with triplanes. arXiv preprint arXiv:2404.03632. 2024.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties self-supervised vision transformers. Proceedings of the IEEE International Conference on Computer Vision, 2021.
Joao Carreira and Andrew Zisserman. Quo vadis, action recognition? a new model and the kinetics dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patlern Recognition, 2017.
Hyunsoo Cha, Byungjun Kim, and Hanbyul Joo. Pegasus: Personalized generative 3d avatars with composable attributes. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Hyunsoo Cha, Inhee Lee, and Hanbyul Joo. Perse: Personalized 3d generative avatars from a single portrait. In Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, 2025.
Lan Chen, Qi Mao, Yuchao Gu, and Mike Zheng Shou, Edit transfer: Learning image editing via vision in-context relations. arXiv preprint ar 2rXhv:2503.13227 2025.
Xi Chen, Lianghua Huang, Yu Liu, Yujun Shen, Deli Zhao, and Hengshuang Zhao. Anydoor: Zero- shot object-level image customization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Zheng Chong, Xiao Dong, Haoxiang Li. Shiyue Zhang, Wenqing Zhang, Xujic Zhang, Hanqing Zhao, Dongmei Jiang, and Xiaodan Liang. Catvton: Concatenation is all you need for virtual try-on with diffusion models. arXiv preprint 2.Xiv:2407.118866, 2024.
Xuangeng Chu and Tatsuya Harada. Generalizable and animatable gaussian head avatar. Advances in Neural Information Processing Systems, 2024.
Chaeyeon Chung, Sunghyun Park, Jeongho Kim, and Jaegul Choo. What to preserve and what to transfer: Faithful, AAAI Conference on Artificial Intelligence, 2025.
diffusion-based hairstyle transfer. In Proceedings of the
Radek Daněček, Michael Black, and Timo Bolkart. Emoca: Emotion driven monocular face capture and animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVE Conference on Computer Vision and Pattern Recognition, 2019.
Zixun Fang, Wei Zhai, Aimin Su, Hongliang Song, Kai Zhu, Mao Wang, Yu Chen, Zhiheng Liu, Yang Cao, and Zheng-Jun Jun Zha. Vivid: Video virtual try-on using diffusion models. arXiv preprint arXiv:240S.11794 2024.
Ian J Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farle Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. Advances in Neural Information Processing Systems, 2014.
11

---

Published as a conference paper at ICLR 2026
Jianzhu Guo, Dingyun Zhang, Xiaoqiang Liu, Zhizhou Zhong, Yuan Zhang, Pengfei Wan, and Di Zhang. Liveportrait: Efficient portrait animation with stitching and retargeting control. arXiv preprint arXiv:2407.03168, 2024.
Yuwei Guo, Ceyuan Yang. Anyi Rao, Zhengyang Liang, Yaohui Wang, Yu Qiao, Maneesh Agrawala, Dahua Lin, and Bo Dai. Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. arXiv preprint Sttt:2307.04725. 2023.
Kensho Hara, Hirokatsu Kataoka, and Yutaka Satoh. Can spatiotemporal 3d enns retrace the history of 2d cnns and imagenet? In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2018.
Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference- free evaluation metric for image captioning. arXiv preprint arXiv:2104.087/6 18, 2021.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. Advances in Neural Information Processing Systems, 2017.
Jonathan Ho. Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models. Advances in
Neural Information Processing Systems, 2020.
Li Hu. Animate anyone: Consistent and controllable image-to-video synthesis for character animation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2024.
Tero Karras, Samuli Laine, Miik Aittala, Janne Hellsten, Jaakko Lehtinen, and Timo Aila. Analyzing and improving the image quality of stylegan. In Proceedings of the IEEE/CVF Conference Computer Vision and Patlern Recognition, 2020.
Rawal Khirodkar, Timur Bagautdinov, Julieta Martinez, Su Zhaoen, Austin James, Peter Selednik, Stuart Anderson, and Shunsuke Saito. Sapiens: Foundation for human vision models. In European Conference on Computer Vision, 2024.
Byungjun Kim, Shunsuke Saito, Giljoo Nam, Tomas Simon, Jason Saragih, Hanbyul Joo, and Junxuan Li. Haircup: Hair compositional universal prior for 3d gaussian avatars. arXiv preprint 250,. 2025.
Gycongman Kim, Hajin Shim, Hyunsu Kim, Yunjey Choi, Junho Kim, and Eunho Yang. Diffusion video autoencoders: Toward temporally consistent face video editing via disentangled video encod- ing. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023.
Jeongho Kim, Guojung Gu, Minho Park, Sunghyun Park, and Jaegul Choo. Stableviton: Learning semantic correspondence with latent diffusion model for virtual try-on. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024a.
Taeksoo Kim, Byungjun Kim, Shunsuke Saito, and Hanbyul Joo. Gala: Generating animatable layered assets from a single scan. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024b.
Tobias Kirschstein, Shenhan Qian, Simon Giebenhain, Tim Walter, and Matthias Nießner. Nersemble: Multi-view radiance field reconstruction of human heads. ACM Transactions on Graphics, 2023.
Max Ku, Cong Wei, Weiming Ren, Harry Yang, and Wenhu Chen. Anyv2v: A tuning-free framework for any video-to-video editing tasks. arXiv preprint arxiv:22222:::40004446 2024.
Black Forest Labs. Flux. httppss//ggtaab...commbbblaeesss
2024.
Junxuan Li, Shunsuke Saito, Tomas Simon, Stephen Lombardi, Hongdong Li, and Jason Saragih. Megane: Morphable eyeglass and avatar network. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2023.
Tianye Li, Timo Bolkart, Michael J Black, Hao Li, and Javier Romero. Learning a model of facial shape and expression from 4d scans. ACM Transactions Graphics, 2017.
12

---

Published as a conference paper at ICLR 2026
Yuhan Li, Hao Zhou, Wenxiang Shang, Ran Lin, Xuanhong Chen, and Bingbing Ni. Anyfit: Controllable virtual try-on for any combination of attire across any scenario. arXiv preprint arXiv:2405.18172, 2024.
Jonathan Long. Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2015.
Davide Morelli, Matteo Fincato, Marcella Comia,, Federico Landi, Fabio Cesari, and Rita Cucchiara. Dress code: High-resolution multi-category virtual try-on. In Proceedings of the IEEE/CVF Conference on Computer Vision and Puttern Recognition, 2022.
Chong Mou, Yanze Wu, Wenxu Wu, Zinan Guo, Pengze Zhang, Yufeng Cheng, Yiming Luo, Fei Ding, Shiwen Zhang, Xinghui Li, et al. Dreamo: A unified framework for image customization. arXiv preprint arXiv:2504. 25911 2025.
Hyeongjin Nam, Donghwan Kim, Jeongtaek Oh, and Kyoung Mu Lee. Decloth: Decomposable 3d cloth and human body reconstruction from single image. In Proceedings of the Computer Vision and Pattem Recognition Conference, pp. 1636-56433 2025.
OpenAl. Chatgpt: Large language model. 2025-09-25.
com/ 2025. Accessed:
Gaurav Parmar, Richard Zhang, and Yan Zhu. On aliased resizing and surprising subtleties in gan evaluation. Proceedings of the IEEF/CVF Conference on Computer Vision and Puttern Recognition, 2022.
Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis. arXiv preprint arXiv:2307.01952. 2023.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In Proceedings of the International Conference on Machine Learning, 2021.
Danilo Rezende and Shakir Mohamed. Variational inference with normalizing flows. In Proceedings of the International Conference Machine Learning, 2015.
Elad Richardson, Yuval Alaluf, Or Patashnik, Yotam Nitzan, Yaniv Azar, Stav Shapiro, and Daniel Cohen-Or. Encoding in style: stylegan encoder for image-to-image translation. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2021.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2022.
Wensong Song, Hong Jiang, Zongxing Yang, Ruijie Quan, and Yang. Insert anything: Image insertion via in-context editing in dit. arXiv preprint arXiv:2504. 15009, 2025.
Shuyuan Tu, Zhen Xing, Xintong Han, Zhi-Qi Cheng, Qi Dai, Chong Luo, and Zuxuan Wu. Stableani- mator: High-quality identity- preserving human image animation. arXiv preprint ar arXiv22411.17697, 2024.
Yuanpeng Tu, Hao Luo, Xi Chen, Sihui Ji, Xiang Bai, and Hengshuang Zhao. Videoanydoor: High-fidelity video object insertion with precise motion control. arXiv preprint arX14 2025.
Rotem Tzaban, Ron Mokady, Rinon Gal, Amit Bermano, and Daniel Cohen-Or. Stitch it in time: Gan-based facial editing of real videos. ACM Transactions on Graphics (Proc. SIGGRAPH Asia), 2022.
13

---

Published as a conference paper at ICLR 2026
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in Neural Information Processing Systems, 2017.
Cong Wang, Di Kang. Heyi Sun, Shenhan Qian, Zixuan Wang, Linchao Bao, and Song- Song-Hai Zhang. Mega: Hybrid mesh-gaussian head avatar for high-fidelity rendering and head editing. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025.
Qixun Wang, Xu Bai, Haofan Wang, Zekui Qin, Anthony Chen, Huaxia Li, Xu Tang, and Yao Hu. Instantid: Zero-shot identity-preservngg generation in seconds. arXiv preprint arXiv:2401.07511
2024.
Ting-Chun Wang. Ming-Yo Yu Liu, Yan Zhu, Guilin Liu, Andrew Tao, Jan Kautz, and Bryan Catanzaro. Video-to- video synthesis. arXiv preprint arXiv: wa...100....00 2018.
Ting-Chun Wang, Arun Mallya, and Ming-Yo Yu Liu. One-shot free-view neural talking-head synthesis for video conferencing. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, 2021.
Linngin Xie, Xintao Wang, Honglun Zhang, Chao Dong, and Ying Shan. Vfhq: A high-quality dataset and benchmark for video face susor-nesolution. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, 2022.
You Xie, Hongyi Xu, Guoxian Song, Chao Wang. Yichun Shi, and Linjie Luo. X-portrait: Expressive portrait animation with hierarchical motion attention. In ACM Transactions on Graphics (Proc. SIGGRAPHH. 2024.
Yiran Xu, Zhixin Shu, Cameron Smith, Seoung Wug Oh, and Jia-Bin Huang. In-n-out: Faithful 3d gan inversion with volumetric decomposition for face editing, In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024.
Binxin Yang. Shuyang Gu, Bo Zhang, Ting Zhang, Xuejin Chen, Xiaoyan Sun, Dong Chen, and Fang Wen. Paint by example: Exemplar- based image editing with diffusion models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Shurong Yang. Huadong Li, Juhao Wu, Minhao Jing, Linze Li, Renhe Ji, Jiajun Liang, Haoqiang Fan, and Jin Wang. Megactor-sigma: Unlocking flexible mixed-modal control in portrait animation with diffusion transformer. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025.
Xu Yao, Alasdair Newson, Yann Gousseau, and Pierre Hellier. A latent transformer for disentangled face editing in images and videos. In Proceedings of the IEEE International Conference Computer Vision, 2021.
Hu Ye, Jun Zhang, Sibo Liu, Xiao Han, and Wei Yang. Ip-adapter: Text compatible image prompt adapter for text-to-image diffusion models. arXiv preprint arXiv:2308.0662/, 2023.
Jianhui Yu, Hao Zhu, Liming Jiang, Chen Change Loy, Weidong Cai, and Wayne Wu. Celebv-text: A large-scale facial text-video dataset. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023.
Kaiwen Zhang, Yifan Zhou, Xudong Xu, Bo Dai, and Xingang Pan. Diffmorpher: Unleashing the capability of diffusion models for image morphing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024a.
Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE International Conference Computer Vision, 2023,
Yuxuan Zhang, Lifu Wei, Qing Zhang, Yiren Song, Jiaming Liu, Huaxia Li, Xu Tang, Yao Hu, and Haibo Zhao, stable-makeupp When real-world makeup transfer meets diffusion model. arXiv preprint Xww.:xi::44007766660 2024b.
Yuxuan Zhang, Qing Zhang, Yiren Song, Jichao Zhang, Hao Tang, and Jiaming Liu. Stable-hair Real-world hair transfer via diffusion model. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025.
14

---

Published as a conference paper at ICLR 2026
Jun- Yan Zhu, Philipp Krähenbühl, Eli Shechtman, and Alexei A Efros. Generative visual manipulation on the natural image manifold. In European Conference on Computer Vision, 2016.
Shenhao Zhu, Junming Leo Chen, Zuozhuo Dai, Zilong Dong, Yinghui Xu, Xun Cao, Yao Yao, Hao Zhu, and Siyu Zhu. Champ: Controllable and consistent human image animation with 3d parametric guidance. In European Conference on Computer Vision 2024.
15

---

Published as a conference paper at ICLR 2026
A IMPLEMENTATION DETAILS
A.1 TRAINING DETAILS
We adopt the two-stage training strategy following Zhu et al. (2024). In the first stage, we resize all videos to uniform resolution of 512 x 512 pixels and train with global batch size of 8 for 60,000 steps. During this phase, all layers except the temporal attention layers are set to be trainable, as the latter are not yet incorporated into the UNet. In the second stage, we insert temporal attention layers into the Denoising UNet (DNet) and train only these newly added layers. This stage uses 24-frame inputs. global batch size of 8. and also runs for 60,000 steps. For both stages, we fix the learning rate at 1e-5, with each stage requiring approximately three days of training. We train our model using NVIDIA RTX A6000 GPUs. As initialization, we use the UNet checkpoint from Yang al. 2023), while the temporal attention layers are initialized from Guo et al. (2023). Our training dataset consists of 2,747 samples drawn from Celeb (Yu et al., 2023). VFHQ (Xie et al. 2022 and Nersemble (Kirschstein et al. 2023). Our method focuses on four attribute categories, with the following distribution: Hair 886 samples from CexebV-Text, 935 from Nersemble, and 265
from VFHQ (total 2,086); Beard 253 samples from CelebV-Texx Eyeglasses 279 samples from VeTeb;-Text; Hat - 129 samples from CelebV-Text. On average, each video contains 292 frames.
A.2 EVALUATION DETAILS
For self-attribute transfer, we randomly sample 200 videos from CelebV-Text (Yu et al. (023) and VFHQ (Xie al., 2022), ensuring that these videos contain unseen identities, facial poses, and expressions relative to the training dataset. For coos-attt transfer, we additionally sample 50 videos. Masks required for image editing baselines are constructed following the procedures provided by the respective authors. To construct cross-attributt transfer pairs, we use the 50 sampled identities and randomly select corresponding face images from VFHQ and CelebV-Text that do not overlap with the training dataset.
We evaluate the results using several metrics. mCLIP-I (masked CLIP-I (Radford et al., 2021 Hessel eet al. 2021p) and mDINO (Caron al. 22221) (masked DINO) assess whether the target attribute is accurately transferred into the generated portrait animation video. To this end, we fill the background of attribute-only images with white and segment the target attribute region from the generated portrait animation video using Sapiens (Khirodkar et al. 2024). We then fill the segmented background with white and compute frame-wise cosine similarity embeddings with CLIP-I and DINO. ID-Sim evaluates identity preservation. Specifically, we mask attribute regions in portrait images by filling them with black, segment the target attribute regions in the generated videos with Sapiens, and replace them with black before computing frame-wise cosine similarity embeddings with ArcFace. Finally, VFID (Video Fréchet Inception Distance) (Heusel et al. 2017 Wang et al.) (018)) extends FID the video domain. Following Fang et al. (2024), we adopt VFID to measure temporal consistency and overall video quality.
A.3 KEYPOINT GUIDANCE GENERATION
Our model generates portrait animations using a guidance video composed of facial keypoints, as shown in Fig. 2 of our main paper. These keypoints encode entangled facial shape information, such as interocular distance and the relative positions of eyes, nose, and ears. While this rich representation supports accurate animation in self-attribute transfer scenarios, we observe that, in cross-attribute settings, the generated animation tends to follow the facial shape of the guidance video rather than the portrait image. Also, in these real-world scenarios, significant shape and scale discrepancies between the source and the driver can degrade the model's 's performance. To address this, we propose a method that preserves the portrait's facial shape while transferring only the motion from a different identity. Specifically, we employ LivePortrait (Guo et al.) (024)) to generate an animation of the portrait image that maintains its original shape while being driven by the motion in the guidance video. We then extract facial keypoint guidance video from this animation using Sapiens (Khirodkar et al.) 2024), effectively creating a reenatteentttttt like scenario that allows our model to operate more reliably. Note that for all quantitative results reported in paper and tables, follow the standard relffreenactment setting (Kim et al., 2024a Morelli et al. 2022). The facial keypoint guidance is extracted directly from the ground-truth videos, not generated by LivePortrait.
16

---

Published as a conference paper at ICLR 2026
FROU
Portrait
Attribute
Face Aligner Disabled
Face Aligner Enabled (Ours)
Figure 8: Ablation Study for Face Aligner. Omitting Face Aligner at inference time degrades the visual quality of the generated animation.
Portrait
Attribute
Erode 50
Erode 20
Sapiens (Ours)
Dilate 20
Dilate 50
Figure 9: Sensitivity Analysis of the Attribute Mask. We present analysis showing how output quality changes with mask quality by applying crosion and dilation to the attribute mask derived from the Sapiens Mask.
B ADDITIONAL RESULTS
B.1 ADDITIONAL ABLATION STUDY FOR FACE ALIGNER
We perform ablation study on our Face Aligner, as described in Section and illustrated in Fig. of the main paper. As shown in Fig. [ removing Face Aligner still allows the long blonde hair from the attribute image be transferred the portrait's ss target attribute region. However, the generation becomes unstable, with the left hair strand intermittently appearing and disappearing, In contrast, ours, which applies the face aligner at inference time, enables stable transfer, ensuring that the long blond hair remains consistently preserved throughout the animation.
B.2 SENSITIVITY ANALYSIS OF THE ATTRIBUTE MASK
We show a mask sensitivity analysis by systematically eroding and dilating the hair masks Matu in Fig. 9 When the mask is eroded, it longer fully covers the target hair region, resulting spatially shorter transferred hairstyle. Nonetheless, the model still produces a visually plausible hair transfer video. Conversely, moderate mask dilation has little impact on the overall visual quality, indicating robustness to typical boundary uncertainties in real-world segmentation.
B.3 ADDITIONAL QUALITATIVE COMPARISON
Qualitative comparison of self-atte attribute transfer. We additionally provide qualitative results with other baseline combinations in self-attribute transfer setup. Note that generate portraits with transferred hair attributes using recent image insertion and face editing methods (Chung et al., 2025 Yang al. 2023 Zhang al. 2025 Bilecen et al. 2024), and compare the resulting animation
videos produced by applying recent animation techniques (Guo et al. 2024 Xie et al. 2024 Yang
al. 2025) with those generated by our method, as shown Fig. 10
Qualitative comparison of cross-attribute transfer. We extend the comparison in Fig. 4 of the main paper and present results in Fig. [T against baselines for gross-attribute transfer setup. Our method best preserves the identity of the portrait image while most accurately transferring the hairstyle from the attribute image. Furthermore, our results are perceived as the most natural and visually coherent.
B.4 ADDITIONAL QUANTITATIVE COMPARISON
As shown in Fig. 12 and Table 3, we compare TriplaneEdit (Bilecen et al. +iiveoLLioorrtrai (Guo al. 2024) with method, since Triplane also supports transfer for eyeglasses. Our method
17

---

Published as a conference paper at ICLR 2026
Frimess
ha
on D An o
Bs
LivePartrii SoParaait MegAstue LivePertaal X.Pectrit MogAstur LivePocual
MegAssor
Livehaatrat
Magheson
GT
HairFusion
PbE
StableHair
TriplanEdit
Ours
Figure 10: Qualitative Comparison of Self-Attribuu Transfer in the Hair Category. We compare our method and the baselines that combine portrait animation method with image or hairstyle editing methods. Our results show the highest quality closest to the ground truth, while other methods produce artifacts or unnatural appearances.
Portrait Altribute
Liveooman:
XoPoraor MegActonn Livelonmir
X-Porma MegAttoe Livehorar
X-Pomaa Meghane Livefomnnn X-Pomar MepArinn Ours
HairFusion
PbE
Stabeellai
TrplaneEdi
Figure 11: Qualitative Comparison of Artss-AAtributt Transfer in the Hair Category. We compare our method with the baselines that combine image editing and portrait animation. Our results best preserve the identity of the portrait image while most effectively transferring the hairstyle.
consistently outperforms the baseline across all self-attribute transfer metrics. Morcover, it produces results that are closer to the ground truth and more natural than the baseline.
B.5
USER STUDY
We conduct a user study to evaluate portrait animations generated using portrait and attribute inputs from different identities, as shown in Table 4 Each of the 100 participants viewed 9 randomly selected videos from a pool of 44 and rated how well each output preserved the hairstyle of the attribute image and the identity of the portrait image. Our method achieves the highest user preference, demonstrating superior performance in cross- idennty transfer. Participants were asked: "Which video most naturally combines the face from the "face" image with the hairstyle from the 'hair" image?"
18

---

Published as a conference paper at ICLR 2026
GI
TE+LP
Ours
Video Frames
Video Frames
Figure 12: Qualitative Comparison of Self-Attribute Transfer in the Eyeglasses Category. TE represents TriplaneEdit and LP denotes LivePortrait. In the self-attribute transfer setting on the eyeglasses category, compare our results with baseline. Our method produces portrait animations most similar the ground truth while remaining the most natural.
Quantitative Comparison on Eyeglasses Category. Our method outperforms this baseline Table 3:
on every evaluation metric.
Img.Gen.
Animation
L 4
PSNR1
SSIM
LPIPS
FID
TriplancEdit
LivePortrait
0.151
13.53
0,433
0.435
106.28
Ours
0.078
18.19
0.627
0.181
75.59
B.6 ADDITIONAL RESULTS
Single-attribute transfer. We extend the results of Fig. 1 in the main paper and present in Fig. 17 animations generated by transferring a single attribute to the portrait. Our method preserves the identity of the portrait image while faithfully transferring the attribute from the attribute image, resulting in natural portrait animations with attribute transfer.
we present portrait animations generated by simultaneously transferring two and three attributes in single stage under the zero-shot setting. Through various combinations of the four supported categories (beard, eyeglasses, hair, hat), our method produces portrait animations where attributes are transferred naturally and with high quality, without any additional optimization.
Multi-attribute transfer. In Fig. 18 and Fig. 19
B.7 TEXT-TO-IMAGE GENERATED ATTRIBUTE TRANSFER FOR PORTRAIT ANIMATION
Our method generates portrait animation video with attribute transfer given image containing the desired attribute. extend this capability by synthesizing the attribute image directly from a text prompt, enabling text-driven control over the target attribute, as illustrated in Fig. [3 Specifically, we leverage the FLUX (Labs) 2024) text-to-image model to generate realistic attribute images, which are then transferred to the portrait image to produce the final attribute- transfrrrrrdde portrait animation.
B.8 GENERALIZATION ON CARTOON DOMAIN
We present cartoon-style result in Fig. 14 Despite being trained exclusively real human video data, our Durian shows strong generalization to the cartoon domain without additional fine-tunigg benefiting from the pretrained diffusion prior.
19

---

Published as a conference paper at ICLR 2026
Table 4: User Study. We conduct user study on baseline methods that achieve strong performance in both self-attribute transfer and cross-attribute transfer. Our approach receives the highest preference among participants.
Ien.
Animation
User Study(%6)f
TriplaneEdit
LivePortrait
4.45
PbE
LivePortrait
19.04
Ours
76.50
Portrait
Hair
Portrait
Hur
Portrait
Portzast
C
Portrait
Reference
Portrail Animation with Attribute Transter
Figure 13: Text-to-Image Generated Attribute Transfer for Portrait Animation. We generate portrait animation with attribute transfer from a textual description by using FLUX (Labs 2024) to synthesize high-quality portrait image with the desired hair attribute.
B.9 FAILURE CASES
Conflicting Lighting We present animation with hair transfer results using a portrait image captured under extremely dark, blue-tinted lighting, while the target hairstyle taken from a subject photographed outdoors under bright daylight with white-colored hair appearance. In the resulting animation as shown in Fig. [5| the white hairstyle transferred accurately; however, the hair appearance does not fully adapt to the portrait's low-light illumination. Nonetheless, we observe that back lighting partially reflected the synthesized hair, indicating that the model captures some lighting cues even under severe illumination mismatch.
Occlusion We conduct qualitative experiment occluded face input as shown in Fig. 16 We
demonstrate hair transfer with animation using a portrait image in which part of the face is occluded by hand with complex manicure patterns. In the resulting animation, minor artifacts appear around the nose region, likely due to the challenging occlusion. Nevertheless, the hair transfer and the generation of the occluded mouth region successful, and the mouth motion aligns well with the
20

---

Published as a conference paper at ICLR 2026
Portrait
Attribute
Portrait Animation with Hair Transfer
Figure 14:
Generalization on Cartoon Domain. We present our Durian's portrait animation with hat transfer results generated from a cartoon portrait image and cartoon hat image.
Portrait
Attribute
Portrait Animation with Hair Transfer
Figure 15: Failure Case under Conflicting Lighting Conditions. We present failure case under large lighting discrepancies. Although the white hairstyle correctly transferred, its color does not adapt the portrait lighting.
keypoint guidance video, indicating that the model can robustly synthesize motion-consistn facial regions under partial occlusion.
C DISCUSSION
C.I EVALUATION IN THE SELF-ATTRIBUTE TRANSFER SCENARIO
In ideal evaluation, would use ground-truth videos that contain before and after versions of the attribute transfer. Since such paired data does not exist, the commonly used alternative attribute transfer and VTON literature is the self-attribute(or garment) transfer setting (Kim al. 2024a Chung et al. 2025 Zhang et al. (025). In this setup, we take ground-truth video and select two arbitrary frames as the portrait image and the attribute image. The generated video then compared against the original ground truth video using reconstruction metrics such as L1, PSNR, SSIM, and LPIPS. as reported Table I and Table 2 Although this setting cannot directly evaluate genuine cross-identity transfer, it still provides useful information because the attribute region in the portrait image and the identity region in the attribute image are masked out (see the inputs in Fig. 2 This forces the model to combine complementary cues in order to reconstruct the video.
C.2 REGARDING FULL REF. IMAGE INPUT VARIANT ABLATION STUDY
We provide additional clarification regarding the full reference image input variant presented in the ablation study. The ablation study shown Table 2 is via self-attribute transfer setting. However, in the "full ref. image input" ablation where masks are not used, the model can simply copy either the portrait or the attribute input because both already contain the required face and attribute cues. As result, the model can obtain strong reconstruction metries in Table without learning true disentanglement. Importantly, this shortcut is specific to this variant: for all other ablations, the identity region in the attribute image and the attribute region in the portrait image masked out, preventing such leakage and ensuring a fair and meaningful comparison. The limitation of the unmasked shortcut becomes evident cross-attribule transfer. where the model fails separate identity and attribute cues when the two inputs come from different sources, as shown in Fig. We therefore suggest interpreting Fig. together with Table understand this contrast.
USE OF LARGE LANGUAGE MODELS (LLMS)
In accordance with the ICLR policy on the use of Large Language Models (LLMs), we disclose that ChatGPT (OpenAl 2025) (an LLM developed by OpenAl) used during the preparation of this
21

---

Published as a conference paper at ICLR 2026
Portrait
Attribute
Portrait Animation with Hair Transfer
Figure 16: Failure Case under Portrait Occlusion. We present a failure case when the portrait image contains occlusions. Artifacts appear around the nose region.
manuscript. The model was employed exclusively for sentence-level grammar checking and minor style corrections.
No parts of the research ideas, methodology, experimental design, conclusions were generated by the LLM. All scientific contributions are solely attributable to the authors.
22

---

Published as a conference paper at ICLR 2026
Portrait
Hair
Portait
Hair
Porteait
Har
Postraic
S
S
S
S
Hat
Portrait
Beard
Portrait
Beard
Portrait
Poctrait Eyoglasses
Eyoglassos
Reference
Portruit Animation with Attribute Transfer
Figure 17: Qualitative Results for Siriie-Attribute Transfer. We present additional results on hair, hat, eyeglasses, and beard attribute transfer for portrait animation. Our method preserves the fine details of the original portrait while achieving natural and seamless attribute transfer.
23

---

Published as a conference paper at ICLR 2026
Fyrgboses Beamd
Flir
Eyeglosees
S
S
S
S
Tat Beard
Eyoolosses
Reference
Portrait Animation with Attribute Transfer
Figure 18: Qualitative Results for Dual-Attribute Transfer. We demonstrate the results of
simultaneously transferring two attributes for portrait animation.
Hair Baid-Ba Band-fyyyllues explsee
Har Eyelhoos
$
Reference
Portrait Animation with Attribute Transfer
Figure 19: Qualitative Results for Triple-Attribte Transfer. We present the results of simulta-
neously transferring three attributes. each example, the image in the top-left corner indicates the target portrait.
24


Reviews:
[{"review_id": "07JMWwptT4", "reviewer": "Reviewer_5oxU", "summary": "This paper presents Durian, which generates portrait animation videos with cross-identity attribute transfer from one or more reference images to a target portrait", "questions": "Pls see weakness.", "limitations": "", "rating": 2, "confidence": 5, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "The task may be new and the method does not need paired data.", "weaknesses": "- Figure 2 shows the proposed method use the guidance video frames only. But in A.3, it is stated that the methods use LivePortrait to generate a \"self-reenactment\" video. This video preserves the shape of the target portrait while matching the movements of the driving video.\nThen, extract key points from this intermediate generated video, which will serve as the actual input for the Durian model. The proposed Durian seems to rely on the ability of LivePortrait on the self-reenactment video. The comparison with LivePortrait may not be fair.\n\n- “full ref. image input” uses unmasked portrait and attribute images during training. This variant achieves the best quantitative scores in Table 2. The paper provide explanation in Fig 5. This may show the used metrics are not reliable and the performance in Table 1 is in question. In Figure 10 shows liveportrait with StableHair performs better.\n\n- The paper mention that \"no previous work has directly addressed this task\". Therefore, it constructs a \"two-stage\" baseline: Using an image editing model (such as StableHair) to transfer attributes, and then using a video animation model (such as LivePortrait) to create animations. Such comparison is unfair. The two-stage method inherently has disadvantages (error accumulation, domain mismatch). It is expected that the authors' end-to-end model outperforms these \"patched-together\" baselines in terms of metrics. This is not a solid contribution.\n\n- The core critique of the novelty of the Durian architecture lies in the fact that its \"Dual ReferenceNet\" is a combination of Adapters. The core mechanism of IP-Adapter (Ye et al., 2023) is as follows: it uses a (typically frozen) image encoder (CLIP) to extract features from a reference image, and injects these features into the cross-attention layers of a pre-trained diffusion model. This enables additional control over the styel.Durian’s ARNet performs exactly the same function. It takes an attribute reference image (hairstyle), encodes it also with CLIP , and injects its spatial featureinto the diffusion U-Net to control the style again. Besides, InstantID (Wang et al., 2024) designs IdentityNet that integrates facial and key-point images into the generation process by imposing strong semantic and weak spatial conditions, thereby achieving high-fidelity id preserve. Durian also performs an identical function. It takes a portrait reference image, encodes it also with ArcFace and injects its feature Diffusion to ensure identity preservation. It seems such two branches are, in fact, a combination of existing adapters.\n\n\n## Comprehensive Analysis  \nThe so-called \"novel architecture\" of Durian is essentially a **parallel combination** of an IP-Adapter (for attributes) and an InstantID-like module (for identity). This is a **compositional and incremental engineering step**, not a fundamental architectural invention. In their paper, the authors avoid using the standard term \"Adapter\" and instead coin the new term \"Dual ReferenceNet\"—to some extent, this obscures the limited architectural novelty of their method.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "BWgL5ZHZMP", "reviewer": "Reviewer_2dwE", "summary": "This paper introduces portrait animation system while transferring facial attributes based on reference images. To avoid the attribute-paired dataset issues, it propose self-reconstruction strategies by designing dual reference branches upon diffusion models, significantly outperforming existing SoTAs.", "questions": "In multi-attributes transfer, the main component is to concatenate the feature $F^{l, N_{attr}}_{attr}$ with attention module. However, in general, projecting features into key and value vectors relies on learnable projection layer. Since multiple-attributes transfer increase the width of the features thus requiring variable size of linear projections, how the model handles this?\n\nDo the system use separate $N_{attr}$-specific models than single attribute transfer or use some inference-level adaptive manner in single generation pass?\n\nAlso, it is recommended to discuss is there any degradation according to the $N_{attr}$ and the maximal number for this.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "- High fidelity on attribute transfer: The proposed system exhibits significantly faithful outcomes on attribute transfer with high quality and relatively no visual artifacts.\n- Single generation pass for multiple attributes: For multiple attributes transfer, the concat-based module produces pretty plausible results.", "weaknesses": "- Discussion about masking step: It is well known that extracting particular pixels alongside given attributes often fails within in-the-wild condition. It is better to discuss how such scenarios can be further alleviated for practicality.\n- Width-wise concatenations: The system propose width-wise concatenation to mix the reference information from both paths in main denoising process. However, in reference-based editing manner, there are some common approaches such as cross attention, per-layer attention map swapping and so forth. It is recommended to discuss how the proposed width-concatenation works compared to existing approaches.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sluoRHVGTj", "reviewer": "Reviewer_VRMj", "summary": "This paper proposes Durian, a diffusion-based framework for generating portrait animation videos with cross-identity facial attribute transfer (e.g., hairstyles, eyeglasses, beards, hats) from one or more reference images. A core challenge in this field is the scarcity of large-scale paired data (same individual with/without target attributes), which limits existing methods to specialized tasks (e.g., hairstyle-only transfer) or multi-stage pipelines. Durian addresses this by introducing a self-reconstruction training strategy that leverages uncurated in-the-wild portrait videos (CelebV-Text, VFHQ, Nersemble) instead of paired data.", "questions": "- Can Durian be extended to deformable attributes (e.g., clothing, scarves) or subtle attributes (e.g., makeup)? What modifications to masking (e.g., dynamic masks for moving clothing) or feature extraction (e.g., finer-grained ARNet layers) would be required?\n- If external tools (e.g., Sapiens, LivePortrait) are replaced with a built-in masker/aligner, how would performance change? Is end-to-end integration feasible without sacrificing quality?\n- Cross-identity transfers with large lighting differences (e.g., a dark-haired reference and a bright-portrait identity) produce subtle artifacts. Would adding an illumination alignment module (e.g., as in PERSE, Cha et al. 2025) improve attribute-identity coherence?\n- Training data is imbalanced (2,086 hair samples vs. 129 hat samples). Does this imbalance hurt transfer quality for rare attributes (e.g., hats)? Could few-shot learning or data augmentation for rare attributes improve generalization?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "- Existing attribute transfer methods (e.g., HairFusion, Chung et al. 2025; StableHair, Zhang et al. 2025) rely on attribute-paired data or synthetic pipelines (e.g., generating bald portraits to pair with hairstyles). In contrast, Durian’s self-reconstruction strategy uses uncurated videos (2,747 total) to simulate attribute transfer, eliminating the need for expensive paired data. This makes it scalable to diverse attributes without dataset curation.\n- Single-reference models (e.g., CAT-VTON, Chong et al. 2024; TriplaneEdit, Bilecen et al. 2024) concatenate attribute and identity inputs into a shared encoder, leading to attribute-identity blending. Durian’s ARNet (attribute) and PRNet (identity) separate feature extraction, enforced by complementary masking. Ablation studies confirm this design outperforms single ReferenceNet variants (L₁: 0.0744 vs. 0.0813; SSIM: 0.6527 vs. 0.6314).\n- Most SOTA methods are task-siloed: static attribute transfer (e.g., Stable-Makeup, Zhang et al. 2024b) cannot animate, while animation models (e.g., LivePortrait, Guo et al. 2024; X-Portrait, Xie et al. 2024) cannot transfer attributes. Durian uniquely enables multi-attribute composition (e.g., combining hair + glasses + hat) and attribute interpolation (smooth hairstyle transitions) in one generation pass—critical for real-world styling applications (e.g., virtual try-ons).\n- The self-reconstruction paradigm inherently suffers from a domain gap (training on single identities vs. inferring across identities). Durian’s mask expansion (union of original/generated attribute masks) and augmentations (affine misalignment, FLUX outpainting) outperform fixed-heuristic methods (e.g., HairFusion’s hair-specific mask expansion). This explains its strong cross-identity results (ID-Sim: 0.7098 vs. baseline 0.2776).", "weaknesses": "- Durian only supports four attributes (hair, eyeglasses, beard, hat), narrowing its applicability compared to methods that handle makeup (Stable-Makeup, Zhang et al. 2024b), clothing (AnyFit, Li et al. 2024), or accessories beyond hats (e.g., necklaces). The paper provides no rationale for excluding these attributes, nor discusses modifications needed for deformable attributes (e.g., scarves) or subtle attributes (e.g., lipstick).\n- Durian relies heavily on third-party tools for critical components:\nMask generation: Sapiens (Khirodkar et al. 2024) for attribute segmentation. Outpainting: FLUX (Labs, 2024) to fill regions after affine transformations. Keypoint alignment: LivePortrait (Guo et al. 2024) to fix shape mismatch in cross-identity inference. This lack of end-to-end integration limits deployment flexibility and introduces potential errors (e.g., Sapiens failing to segment small attributes like thin eyeglasses).\n- The two-stage training (60,000 steps per stage, ~6 days on 8 NVIDIA RTX A6000 GPUs) is computationally expensive. For comparison, single-stage animation models (e.g., Animate Anyone, Hu 2024) train faster while maintaining temporal consistency. The paper does not explore optimization (e.g., adaptive step scheduling) to reduce training time, which is a barrier for real-world adoption.\n- Insufficient Theoretical and Ablation Depth\n  - The “full ref. image input” ablation (unmasked training) achieves better self-transfer metrics (L₁: 0.0670 vs. Durian’s 0.0744) but fails in cross-identity transfer. However, the paper does not analyze why unmasked inputs break disentanglement (e.g., do they allow the model to copy identity cues instead of learning transfer?).\n  - Ablations for temporal attention (critical for animation) are missing. The paper only confirms temporal layers improve consistency but does not quantify their impact (e.g., VFID without temporal layers).\n- Durian is only evaluated on 3-second clips. For longer videos (e.g., 10+ seconds), its temporal attention may struggle with drifting attributes (e.g., hairstyle shape changing over time). SOTA video diffusion models (e.g., VideoCrafter2, Chen et al. 2024a) use factorized spatiotemporal attention to maintain long-horizon consistency, which Durian does not adopt.\n- For blurred images or unrecognizable faces (e.g., low resolution, occlusions), Durian fails to extract valid attribute masks or align references. This is a critical gap compared to robust methods like HunyuanPortrait (Xu et al. 2025), which uses an enhanced appearance encoder to handle low-quality inputs.\n- When combining attributes from references with large pose differences (e.g., a hat from a left-angled face and glasses from a front-facing face), Durian suffers from misalignment. Unlike MimicMotion (Zhang et al. 2024b), which uses pose-aware guidance to align multi-reference features, Durian lacks explicit pose correction for compositional tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Rh6b24MlCB", "reviewer": "Reviewer_3NqY", "summary": "This paper proposes DURIAN, a diffusion-based framework for dual-reference image-guided portrait animation, which enables attribute transfer across identities (e.g., hairstyle, glasses, beard, hat) while maintaining the target person’s identity consistency and generating high-fidelity dynamic videos.The paper introduces four core innovations:\n1. Dual ReferenceNet architecture – separately processes the attribute reference image and the identity reference image, and fuses their representations through a Spatial Attention mechanism within the diffusion process.\n2. Self-Reconstruction Training Strategy – trains solely on natural portrait videos by sampling two frames from the same video as pseudo attribute–identity pairs, enabling unpaired attribute transfer learning.\n3. Attribute-Aware Mask Expansion and Augmentation – bridges the gap between self-reconstruction training and real cross-identity inference, improving robustness to spatial misalignment and diverse attribute layouts.\n4. Emergent Abilities – DURIAN exhibits zero-shot multi-attribute composition and interpolation in a single forward pass without any additional training.\nExtensive experiments on CelebV-Text, VFHQ, and Nersemble datasets show that DURIAN significantly outperforms multiple attribute transfer baselines both quantitatively and visually.", "questions": "1. Could the proposed method be extended to audio-driven scenarios, where an attribute reference is applied to a portrait and animated according to an input audio signal?\n2. How does the model handle overlapping attributes during multi-attribute transfer (e.g., hat and hair regions)?\n3. Can the model generalize to non-human domains such as stylized or 3D avatars?\n4. How sensitive is DURIAN to the accuracy of Sapiens masks? Does noisy segmentation significantly degrade results?\n5. Could the self-reconstruction setup be extended to multi-frame attribute co-training (e.g., three or more frames to jointly learn multiple attributes)?\n6. Could the authors provide examples of failure cases under extreme conditions (occlusion, conflicting lighting, overlapping attributes, etc.)?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "1. Clever self-reconstruction strategy: The proposed self-supervised training method effectively eliminates the dependence on paired attribute data, which is scarce in real-world scenarios, making it an important advancement in attribute transfer.\n2. High technical originality: The dual-reference architecture independently encodes identity and attribute information, and fuses them via cross-attention, representing a novel design in diffusion-based frameworks.\n3. Robust identity preservation: The introduction of mask expansion and augmentation significantly improves robustness under cross-identity attribute generation, ensuring consistent identity preservation.\n4. Excellent writing and structure: The paper clearly motivates the research problem, presents the method with logical flow, and provides solid reasoning for each design choice. The methodology section follows a top-down structure, clearly explaining design rationale and detailed implementations.\n5. Detailed equations and reproducibility: The mathematical formulations are clear and complete. The appendix provides comprehensive implementation details, enhancing reproducibility.\n6. Comprehensive experiments: Quantitative metrics include both reconstruction fidelity and perceptual quality; Ablation studies verify the effect of each module (mask expansion, augmentation, dual-branch design, etc.); Comparison with strong baselines shows clear superiority.\n7. Emergent properties: The model demonstrates emergent multi-attribute composition and smooth interpolation in a single forward pass, indicating strong representational power and generalization.", "weaknesses": "1. Handling of overlapping attributes: The paper does not explicitly describe how the model handles the interaction between overlapping attributes such as hair and hat regions. Providing a detailed explanation or visualization of how the spatial attention mechanism resolves these conflicts would make the method more transparent and convincing.\n2. Heavy dependency on external tools: The approach heavily relies on multiple external pretrained modules, including Sapiens, EMOCA, FLUX, and SDXL. While this integration improves practicality, it may reduce the model’s reproducibility and independence. Since the mask quality from Sapiens directly affects transfer results (for example, incomplete glasses detection can cause attribute failure), it would be beneficial to conduct a sensitivity analysis or robustness evaluation with alternative segmentation tools.\n3. Limited evaluation domain: The current evaluation focuses only on human facial portraits, without testing on stylized or non-human avatars. Including results on cartoon or 3D-rendered portraits would help demonstrate the model’s generalization ability across different visual domains.\n4. Empirical design of mask expansion: The mask expansion strategy, which leverages SDXL and ControlNet with prompt engineering to simulate variations such as long hair or reshaped hairstyles, appears somewhat heuristic. It would strengthen the technical rigor to include quantitative results or introduce a more automatic and controllable expansion mechanism.\n5. Lack of semantic consistency validation in interpolation: The interpolation experiments (Fig. 7) only show visual examples, without verifying whether the interpolation path corresponds to a semantically smooth transformation. Incorporating semantic distance or latent-space curvature analysis would help confirm that the interpolation indeed reflects continuous attribute transitions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)