LaTo:  Landmark-tokenized Diffusion Transformer for Fine-grained Human Face Editing
Authors: Zhenghao Zhang
Ziying Zhang
Junchao Liao
Xiangyu Meng
Qiang Hu
Siyu Zhu
Xiaoyun Zhang
Long Qin
Weizhi Wang
Venue: ICLR 2026 Poster (2026)
Keywords: Image Editing; Face Editing; Identity Preservation; Landmark-tokenized
Primary area: generative models

Abstract: Recent multimodal models for instruction-based face editing enable semantic manipulation but still struggle with precise attribute control and identity preservation. Structural facial representations such as landmarks are effective for intermediate supervision, yet most existing methods treat them as rigid geometric constraints, which can degrade identity when conditional landmarks deviate significantly from the source (e.g., large expression or pose changes, inaccurate landmark estimates). To address these limitations, we propose LaTo, a landmark-tokenized diffusion transformer for fine-grained, identity-preserving face editing. Our key innovations include: (1) a landmark tokenizer that directly quantizes raw landmark coordinates into discrete facial tokens, obviating the need for dense pixel-wise correspondence; (2) a location-mapped positional encoding and a landmark-aware classifier-free guidance that jointly facilitate flexible yet decoupled interactions among instruction, geometry, and appearance, enabling strong identity preservation; and (3) a landmark predictor that leverages vision–language models to infer target landmarks from instructions and source images, whose structured chain-of-thought improves estimation accuracy and interactive control. To mitigate data scarcity, we curate HFL-150K, to our knowledge the largest benchmark for this task, containing over 150K real face pairs with fine-grained instructions. Extensive experiments show that LaTo outperforms state-of-the-art methods by 7.8% in identity preservation and 4.6% in semantic consistency. Code is available at https://github.com/alibaba/landmark-tokenized-dit.

Published as a conference paper at ICLR 2026
LATO: LANDMARK-TOKENTER DIFFUSION TRANSFORMER FOR HUMAN FACE EDITING
Zhenghao Zhang Siyu Zhu
Ziying Zhang
Junchao Liao
Xiangyu Meng'
Qiang Hu?
Xiaoyun Zhang
Long Qin
Weizhi Wang
Alibaba Cloud Computing Shanghai Jiao Tong University"
Fudan University
ABSTRACT
Recent multimodal models for instruction-oes based face editing enable semantic manipulation but still strug- gle with precise attribute control and identity preservation. Structural facial representations such as land- marks are effective for intermediate supervision, yet most existing methods treat them rigid geometric constraints, which can degrade identity when conditional landmarks deviate significantly from the source (e.g., large expression or pose changes, inaccurate landmark estimates). To address these limitations, we propose LaTo, a landmark-tokennze, diffusion transformer for fine-grained, editing. Our key innovations include: (1) a landmark tokenizer that directly quantizes raw landmark coor- dinates into discrete facial tokens, obviating the need for dense pixel-wise correspondence: (2) location- mapped positional encoding and a landmark- aware classifier-free guidance that jointly facilitate flexible yet decoupled interactions among instruction, geometry. and appearance, enabling strong identity preser- vation; and (3) a landmark predictor that leverages vision- language models to infer target landmarks from instructions and source images, whose structured mhain-of-thougll improves estimation accuracy and interactive control. To mitigate data scarcity, we curate LIFL-150K, to our knowledge the largest benchmark for this task, containing over 150K real face pairs with fine- mne-grained instructions. Extensive experiments show that LaTo outperforms state-of-the-art methods by 7.8% in identity preservation and 4.6% in semantic consistency. Code is available at https://uthub.commllbbbbuu hatp///bbooommbbbbabaannaaaaaar. landmark landmark-tokeeizzedddl tokenized-ditt
face
INTRODUCTION
Generating photorealistic facial images (Lin et al. 2025) with controllable expressions, head pose, and other attributes while preserving subject identity remains a core challenge in face editing (Preechakul et al. 2022, ZZhng et al. 2025). These capabilities are critical for applications such virtual avatar creation, digital human synthesis, and identity-preserving facial modifications. Recent proprictary multimodal models (c.g.. SeedEdit3 (Wang et al. 2025). Step1X-Edii (Liu et al., 2025). FLUX.1 1 KKontext (Labs al., 2025)) have markedly advanced instruction- based image editing. Leveraging large- scale vision-language modeling (Li et al. 20244 Deng et al. 2025). they deliver higher-fidelity edits across diverse scenarios than prior face editing methods (Liu et al. 2022, Pernuu et al. 2023 Cheng et al. 2024). Predictably, to enable fine-grained control, users typically must provide detailed, standardized textual descriptions (e.g.. "turn the subject's head 450 to the left and make the facial expression slightly happy"). However, existing models (Liu et al. 2025) Xiao et al. 2025 Labs et al. 2225)) exhibit limitations in accurate instruction following and identity preservation during in-context generation. We attribute these inconsistencies to their exclusive reliance on high-level semantic encoders, which struggle to capture the structural facial cues required for precise control.
A common strategy for improving edit fidelity is to employ facial landmarks as an intermediate structural prior (Yang & Guo 2020 Wei et al., 2025 Liang et al. 2024). Unlike text prompts, landmarks (Li et al. 2022 Sun et al. 2024] impose
explicit geometric constraints via precise 2D coordinates of key facial features (eyes, nose, mouth), thereby localizing edits to the appropriate regions. However, most existing approaches
built on GANs (Goodfellow et al.] 2020) UNet-
based (Ronneberger et al. 2015) diffusion models and transfer poorly to modern Diffusion Transformer (DiT) (Peebles & Xie 2023) due to fundamental architectural differences. Recent DiT-based editors like OminiControl (Tan 2024)
and OmniGen (Xiao et al. 2025) adopt a general- purpose control strategy for face editing: they rasterize landmarks into 2D images, encode them via Variational Autoencoder (VAE) (Kingma & Welling (022) to obtain dense visual tokens, and use these as in-context guidance. Despite improving identity preservation, this strategy introduces two core limitations: (1) conditioning on rendered landmark images encourages pixel-wise copying of fixed facial shapes rather than geometric reasoning, leading to identity drift and artifacts when the conditional landmarks substantially deviate from the source in shape or position, as shown in Figure [] and (2) because self-attention scales quadratically with sequence length (Huang et al. Avrahami et al. 2025), appending long dense visual tokens to diffusion tokens incurs prohibitive memory and compute costs, limiting practical applicability in complex scenarios.
general-purpose
"These authors contributed equally to this work. Corresponding authors.

---

Published as conference paper at ICLR 2026
Instruction
scured, rihhi
ungry, 15 riight
huppy, 30"
surprised
neutral, 30 30'right
Inpur image
Landmarks
Rendered-
image-based
Coordinates- based (LaTo)
Figure/ Landmark tokenization in LaTo preserves identity and produces natural results, whereas pixelwise alignment baselines rigidly follow the rendered landmark image and often lose identity under cross poss-identity landmark conditions (first four columns) or when self-identity landmarks differ substantially from the source.
In this paper, we present LaTo, a landmark-tooenizzed Diffusion Transformer for complex facial editing. Instead of relying dense pixelwise landmark renderings, introduce a landmark tokenizer that directly quantizes landmark coordinates into discrete facial tokens. The tokenizer adopts a VQVAE-style codebook (Van Den Oord et al. 2017). mapping coordinate inputs embeddings with the same dimensionality image tokens, faithfully preserving facial structure. To route sparse, spatially discontinuous landmark tokens to their target facial regions, we design a location location-mapping mapping positional encoding that anchors each token its physical location in the latent grid, ensuring precise regional guidance in the generated image. Following Step1 X-Edit, we integrate these sparse landmark tokens as contextual inputs for unified token processing within DiT blocks, enabling flexible yet decoupled interactions among geometry, appearance, and instruction while maintaining high efficiency, strong identity preservation, and semantic consistency. We further introduce landmark- classifier-free guidance balance visual quality and geometric fidelity.
To address training data limitations, we develop an automated synthesis-and-curatio pipeline to construct HFL-150K, large-scale dataset of more than 150,000 face editing pairs with diverse attributes and strict identity consistency. Each pair is annotated with fine-grained editing instructions and high-precision facial landmarks, providing the rich supervision necessary to fully realize LaTo's capabilities. users. To alleviate this requirement, we introduce a landmark predictor that employs vision-language model (VLM) to infer target landmarks from the source image and textual instruction using a structured chain-of- -fuhouhht. We collect set of high-quality instuctioonlandmme substantially improving landmark estimation accuracy and usability. Equipped with HFL-150P 500 and the landmark predictor, LaTo achieves sate-of-the-ar face editing performance, particularly in semantie consistency and identity preservation. Our contributions can be summarized as follows:
inference, supplying precise landmark inputs can be impractical for end
annotations with explicit change magnitudes and fine-tune a lightweight VLM,
We introduce HFL-150K. face-editing dataset comprising 150K face pairs annotated with fine- fine-graine grained editing instructions. To the best of our knowledge, it the largest resource this area.
We present LaTo, the first landmark-tokenizzz landmark tokenization of raw coordinates, (ii) location- location-mapping positional encoding, and (ii) landmark- classifier-free guidance. This design affords flexible geometric control, strong identity preservation, and reduced computational cost compared with rendered-image conditioning.
diffusion transformer for precise face editing that integrates (i)
We develop a landmark predictor lightweight VLM that infers target landmarks from source image and an editing instruction, bridging semantics and precise facial geometry for intuitive user control.

---

Published as conference paper at ICLR 2026
Head Pose
SP
X
Input image
30"
30"Ictt 30"up
30'down
30'right
60'right
Expression
Input image
happy
angry
disgusted
surprised
scared
Expression Intensity
OO
Input image
slightly angry
amgry
strongly angry
slightly surprised
surprised
strongly surprised
Combination
Input image
happy 60f
angry 30"
disgusted 30 down
sad 30'up
surprised 305right
scared 60"right
Figure 2: LaTo enables fine-grained facial expression editing, parametric head-pose editing, or their combination. The small images visualize generated landmarks via landmark predictor, enabling intuitive control signal acquisition.
leveraging HFL-150K and the Landmark Predictor, LaTo delivers precise facial attribute control under complex, fine-grained instructions and achieves state-of-the-ee performance.
As shown in Figure 2
2 RELATED WORK
2.1 INSTRUCTION BASED IMAGE EDITING MODELS
Diffusion models have become the defacto paradigm for high fidelity text to image synthesis and underpin many instruction driven editing systems. Existing approaches fall into two groups. Training-free methods manipulate the reverse process through latent inversion (Tumanyan et al. 2023; Rombach al.] 2022: Mokady et al., 2023) or attention control (Cao 2023 Wang et al.) (024).. These methods are efficient but often fail on complex spatially constrained edits. Training-based methods fine tune on large scale paired image data and achieve stronger results, InstructPix2Pix (Brooks
ct al.
ct al.
2023) pioneered synthetic supervision, while MGIE (Fu et al., 2023) and Emu Edit (Sheynin et al. 2024) incorporate to improve instruction grounding. To further narrow the gap between instructions and edits, recent work couples VLM with diffusion models, including SmartEdit (Huang et al. 2024) AnyEdit (Yu et al.) 2025). UltraEdit (Zhao et al.
VLM
2024) and unified frameworks such as OmniGen (Xiao et al. 2025), BAGEL (Deng et al. 2025), and ACE (Han et al.
2024 Another line fuses VLMMs latents into diffusion decoders (DreamEngine (Chen et al. (2025), MetaQueries (Pan al.
2025a). StepIX-Edi (Liu et al. 2025)). Generalist systems, for example GPT-4o (Hurst et al. 20024) and Gemini (Comanici ct al.] 2025). also show strong vision- language coherence. Despite these advances, precise spatial alignment and identity preservation for human face editing remain challenging because most systems rely on high level semantic signals rather than explicit geometric constraints.
2.2 FACE EDITING MODELS
Face editing seeks to modify facial attributes while preserving identity (Preechakul et al. 2022). Recent driven ap- proaches, including StyleCLIP (Patashnik al. (2221) and ChatFace (Yue al. 2023), have demonstrated strong qual- itative performance. Nevertheless, they often produce entangled edits and unintended changes to identity or appearance, particularly under large instruction variations. To improve fine-grained control and anatomical consistency, subsequent work introduces structured geometric conditions such as face masks (Zhang et al. 2025), semantic layouts (Mofayezi et al. 2024), landmark images (Li et al.) 2022 Sun et al. 2024). In advanced ATT-based general editing systems (Tan et al.

---

Published as a conference paper at ICLR 2026
Table I:
Key attributes of human face editing benchmarks. HFL-150K surpasses existing face benchmarks in both scale and diversity, with unique strengths in fine-grained instruction alignment.
Benchmarks
Size
Real Image
Training
Fine-grained Instruction
Expression
Head pose
ICE-Bench (Pan et al. 2025b)
206
X
X
SeqDeepFake (Shao et al 2022)
49,920
SEED (Zhu et al. 2025
91,526
X
/
X
HFL-150K
302,014
2024 Pan et al. 2025a), landmarks are typically rasterized into 2D images and encoded by visual VAE to condition the diffusion process, which strengthens geometric alignment. However, pixel-wise conditioning can encourage template copying and leads to identity drift when the target geometry differs substantially from the source. Moreover, full res- olution conditionals expand the token sequence and impose high memory and computation. These limitations motivate LaTo, which directly models the relationship between landmark coordinates and target facial regions, decouples geometric structure from pixel-level appearance control.
3
METHODOLOGY
3.1 HFL-150K DATASET CONSTRUCTION
Large-scale editing datasets have been proven critical for developing advanced editing models. In face editing, existing datasets such as SeqDeepFake (Shao al. 2022) and SEED (Zhu et al., (025) suffer from two fundamental limitations: (1) they rely on coarse-grained facial attribute instructions and outdated synthesis models (Karras et al.] 2019: Tsaban & Passos| )023), resulting unrealistic editing artifacts and limited result diversity; (2) their scale constrained by poor-quality samples that require extensive filtering to remove invalid examples. To address the limitations of existing benchmarks, we introduce HFL-150K, a large-scale human face editing dataset comprising 150k image-edit instruction triplets (source, instruction, edited). As summarized in Table T HFL-150K is constructed through hybrid approach combining real-world image curation and synthetic generation using advanced editing models.
Fine-grained attribute definition. We focus two facial editing tasks: expression editing (e.g., "make him smile gently") and parametric head pose editing (e.g.. "rotate her head 30 left".. For expression categorization, we adopt standard emotion recognition protocols (Huang et al. 2023) define seven canonical expressions and assign intensity levels (slightly, normally, strongly) based on visual saliency. For head pose parameters, we formulate spatial transformations using yaw and pitch angles with 30 as the base unit of motion amplitude, as shown in Figure (c-d).
Synthetic data collection. As shown in Figure 3 (a), we employ advanced editing models (Step (SteplX-Edit, GPT-4o (Hurst et al.) 2024), BAGEL (Deng et al. 2025), and sion or head pose edits, aligning with their single-turn training objectives. To ensure generation quality, we implement instruction specific filtering: (1) For expression edits, an expression validator computes semantic similarity between gener- ated outputs and input instructions using Qwen2.5-V1. (Bai et al. (025). (2) For pose edits, a pose discriminator estimates head orientation via Fuler angle regression (Yang et al. 2019) from facial landmarks and verifies alignment with target rotations Or, which can be described as:
to generate synthetic dataset focusing on either expres-
AO 0 0.11= = V(,, 02,p) + (0,, -00-000 0r.g)²
(0,-0r.))
(1)
where 0 = (0p.0.) Q,y) represents the estimated pitch, yaw angles. Only validated samples are retained, resulting in 34K- sample dataset. This synthetic dataset provides a simplified prior that enhances model interpretability.
we further construct real-world face subset from human-centric video datasets (Li et al. 2025), leveraging natural dynamics to capture intra-identity variation in expression and head pose. We apply a multi-stage filtering process comprising: (1) a quality filter using face detector (Deng et al. 2019) to drop occlusions and enforce centering, a Laplacian of Gaussian (LoG) to remove motion blur, and Dover (Wu et al., 2023) for aesthetic and technical assessment; (2) diversity filter combining geometric analysis (2D facial landmarks (Zhu et al., 2016)) and semantic analysis ( (Qwen2.5-VI. for high level facial changes). Pairs with scores beyond thresholds are removed, including abnormally high values indicating copy-paste artifacts and critically low values caused by variability between different people. Finally, an image matcher with CLIP (Radford et al. (021) performs cross frame identity verification, removing residual pairs from different people. Through this pipeline, a total 116K high-quality, semantically diverse image pairs are generated, reflecting natural facial variations.
Real-world data collection. As illustrated in F
For deriving fine- grained editing instructions, we leverage a facial captioner to recognize expressions and estimate intensity. However, even advanced Bwen2.5-VL-72B shows limitations in fine-gra grained expression recognition. To address this, we
4

---

Published as conference paper at ICLR 2026
(a) Synthetic data editing models
Expression distribution
"Mcke her surprised"
Labe
editing models
Sartety BAGEL
Expression
Valičator
8%
27%
T StepFun
15%
S-pp
GPT--40
Discrimintoor
9%
"Tunn her head 45" left"
Happy Surprised
Angry Scared
Disgusted Neutni
Sad
(b) Real-world data from videos
(d) Pose distrinution
"Walee his /acial expression etigyy digflly and fare his dogrees
25%
20%
15%
Video
Valid image list
Ensage pairs
Triplets
10%
Frame extraction
Quality
Divensity filter
Image
Facial Captioner
[-00] (60)--0000000000
[-30,
[0.
[30, 60)
[60,
90)
Real processing
-Yaw
Pitch
Figure 3: Data collection pipeline and statisties of HFL-150K. (a) Synthetic data generation via advanced editing models. (b) Real image pair extraction from video datasets. (c) Expression distribution across categories. (d) Head pose angles aligned with 30 motion budgets.
curate high-quality dataset and fine-tune the model to improve sensitivity more subtle magnitudes. We manually annotate 18k samples with seven predefined expression categories and intensity levels (see Appendix for guidelines). For head pose estimation, we use both horizontal and vertical optical flow angles (Karaev et al., 2024) quantify motion amplitude relative to our base unit. A unified template is designed for instruction generation:
Make his/her facial expression (expression-type) (intensity-level) and turn hissher head {angle-degree).
3.2
LATO
Building upon the Step1X- Seep1X-Edee we propose LaTo, fine-grained human face editing framework that effectively leverages compact facial tokens. As illustrated Figure 4 LaTo achieves precise and seer-frienlly face editing through three core mechanisms: landmark tokenizer, multi-modal token fuser and landmark predictor.
3.2.1 LANDMARK TOKENIZER
Building on the widely adopted VQVAE architecture for discrete tokenization, the landmark tokenizer combines an encoder-decoder framework with a lightweight quantizer. Given raw landmark sequence F {(X,, YD) (with n 2D locations), the encoder maps into continuous latent space E E Rexd via residual blocks with convolutions. A quantizer then discretizes these latents through nearest-neighhor lookup in a learnable codebook C E Ro of size m, generating compact yet expressive facial tokens in unified geometric space. The decoder, structured to mirror the encoder, reconstructs the input sequence, ensuring spatial coherence. The complete training objective combines reconstruction loss and commitment loss:
L = F Fl12 h + F sg [C]13
where sg[:] denotes the stop-gred gradient operation, and 3 is the weight of the commitment loss. This loss encourages faithful reconstruction while promoting effective codebook utilization, enabling model to learn both geometric accuracy and semantic expressiveness in facial token representations.
3.2.2 MULTI-MODA, TOKEN FUSER
We develop token fuser to flexibly integrate landmark, image, and semantic tokens through three components: location- mapped landmark positional encoding, unified representation and landmark- classifier-free guidance.
Location- Location-mapping landmark positional encoding. Step 1xpp1XEB LX-Edit employs 3D Rotary Positional Encoding (RoPE) (Su et al. 2024) to encode spatial information for both image and text tokens. For each position in image tokens, the 3D RoPE
5

---

Published as conference paper at ICLR 2026
Andlyne the face to understand.
Tum her head 30" left
VLM
Connector
Anelys
Break down the instrccior...
Landwark Predicter
Incodes
Positional
token
escoding
Decomposition>
Location
Desonbe how these parts nuse...
F112800,
Landmark Tokenizer
mapping
Uhcondtional heddmnkk taken
DiT Block
68-paint target landruuk
Eneratic Choin Ressoning
Decoder
Codeibook
Transiate Estinatian>.
X Y
Ensode
Structured chain -houggt
quaniizttionn
Target image
Figure 4: Overview of LaTo. The landmark predictor infers target landmarks from source image and instruction via structured chain of thought. landmark tokenizer and visual VAE encode predicted landmarks and source image into tokens. The location- mapppin positional encoding anchors each landmark token to its physical location, ensuring unified yet flexible alignment with instruction and visual tokens. The learned unconditional landmark token further guides the denoising process, keeping the edited image aligned with both the specified landmarks and instructions.
is computed as:
P: Concat RT((), RH (1 Row R (i%)h)
where represents the height of the latent grid. Here, each R(-) implements ID rotary embeddings applied to the text, height, and width dimensions. These embeddings are independently repeated across spatial axes model positional rela- tionships between tokens. To maintain spatial consistency, we design location mapping mechanism that links landmark tokens to their physical location in the latent grid:
P: = Concat (0)( RH (v:) Rw(xx))
where (Is; yi) denote downsampled coordinates of landmark points from the original image. This ensures each compressed representation accurately guides its corresponding area, preserving conditioning fidelity. Our experiments demonstrate that, in the absence of this correction, the model struggles to learn spatial relationships between landmark inputs and generated tokens, leading blurry misaligned facial features.
Unified representation. Following StepIX-Edit, we treat all modal tokens as a unified representation. A trainable facial landmark adapter first projects facial tokens into the same latent space noisy tokens, denoted as 2f Rl The input is formulated as:
Z Concat(a, Zf: 2n) R tllettettt
(5)
where E Rloxd 2, E R..xd and zn E Rlaxd xd denote semantic text tokens, source visual tokens, and noisy image tokens, respectively. The multi-modal attention mechanism generates query and key via:
P = Concat Pt, Ps, Py, Pn)
(6)
Q RoPE (Z)
K (Z))
with P, Ps. Pa derived from Equation and P, from Equation This formulation enables flexible token interactions
via DiT's multi-modal attention mechanism, allowing direct relationships between any token pair without rigid spatial constraints. Given the facial landmark token length I = 68 is significantly smaller than noisy image tokens (h, 1024), this approach maintains computational efficiency comparable to the baseline model.
Landmark-awan -aware Classifier-frr guidance. To balance image quality and landmark fidelity, we introduce landmark- aware classifier-freg guidance (CFG). conventional image-conditioned pipelines, the unconditional branch is obtained by feeding zero image. By analogy, zeroing landmark coordinates for the unconditional path, especially at high CFG weights, often makes the model copy the reference face and suppress appearance variations that should covary with landmarks. We argue that zeroed landmark embeddings do not encode an unconstrained geometric state and they conflict with the physical dynamics of facial motion. This yields an undesired coupling between landmark conditions and the generated content even in the unconditional branch. Inspired by CTVCrafter (Ding et al.) 2025), we train learnable unconditional tokens that replace the position-awee awaee landmark tokens during unconditional training passes. This produces a semantically meaningful unconditional distribution and improves robustness.
6

---

Published as a conference paper at ICLR 2026
Table 2: Quantitative evaluation of state-of-the-art editing methods on HFL-150K test set and face attribute editing subsets from Bodii-Bench/IC--Benchh Indicates models fine-tuned on HFL-150K training set.
H5L-150K
GEdit&ICE (encCC-beee
Method
1
vQ
NA f
IP
SC
vQ
NA
IP
Dustruct-PixPii BBookk et al. 2023
0.518
0.582
0,675
0.381
0.573
0.567
0,643
0.405
AnyEdit et al. 2025
0.612
0.641
0.702
0.446
0.669
0,654
0.676
0,479
OmniGen (Xiao el al 2025)
0.737
0.688
0.731
0.503
0.755
0.707
0.697
0.536
Bagel (Deng ct al. 2025
0.786
0.709
0.759
0.539
0.797
0.718
0.733
0.579
StepIX-Edit (Liu cf al.
2025
0.751
0.706
0.732
0,518
0.767
0.694
0.705
0.541
StepIX-Editt (Lu
20255
0.804
0.725
0,801
0.571
0.803
0.732
0.788
0.594
FLUX.I-Kontext (aabs el
2025
0.712
0.720
0.779
0.556
0.749
0.693
0.751
0.576
FLUX. I-ontexttt
(Labs et al.
2025
0.786
0.737
0.816
0.593
0.801
0.713
0.771
0.609
LaTo (ours)
0.832
0.749
0.805
0.634
0.829
0.724
0.793
0.651
3.3 LANDMARK PREDICTOR
To enable intuitive interaction and improve landmark estimation, we fine-tune Qwen2.5-VL-3BB into a landmark predictor (LP). Given a source image and an instruction, LP generates target landmark coordinates via structured CoT. The CoT supervision comprises four stages: (I) initial state analysis, characterizing the starting pose, expression, and landmark alignment; (2) instruction decomposition, breaking the instruction into primary anatomical motions; (3) kinematic-chain reasoning, separating rigid motions (head rotation, translation) from non-rigid deformations (muscle-driven expression changes); and (4) coordinate estimation, mapping these components to numerical displacements on normalized 512 512 canvas and producing a canonical, machine-parsabi list of (X, ())Y) pairs. We generated CoT traces for 23,145 triplets sampled from HFL.-150K using a rule-guided pipeline that ingests the source image, instruction and target landmarks, and after manual verification retained 19,398 high-quality examples for fine-tuning, During training, the visual and textual inputs are encoded and fused in the multimodal transformer and the model is optimized with next-token supervision to produce the structured CoT token sequence. Coordinates normalized and encoded with compact tokenization scheme and a fixed output grammar to improve numeric fidelity. At inference, smoothing and geometric sanity checks are applied to preserve identity-consistemi rigid distances. This design delivers interpretable, numerically precise landmark predictions that connect robust instruction understanding to explicit geometric control for downstream face editing. Detailed CoT procedures are provided in the Appendix.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETUP
Implement details. The landmark tokenizer is trained from scratch on HFL.-150K with an 8, 8.192-entry codebook and 3.072-dimmnsional codes matched to the X-Edi hidden size. Training uses NVIDIA A100 GPUs with batch size 128 per GPU for 100k iterations, and unused codes reset every 50 steps to prevent saturation. For LaTo, we fine-tune the base model with LoRA (rank 64) and add an unfrozen linear landmark adapter. The model is trained on 16 NVIDIA A100 GPUs with total batch size 32, learning rate 1 x 10-4, We replace conditional landmark tokens with unconditional ones with probability 0.1 to encourage diversity, and train for a total of 40k iterations.
Dataset. We split HFL-150K into 150,000 training samples and 1,007 test samples. For the test set, stratified sampling en- sures diverse coverage of expression types, head poses, and their combinations. We additionally perform manual validation to ensure all samples include high-quality instructions and image-landmark pairs. We also curate an auxiliary test set by selecting face-attribute editing samples from GEdit-Bench (Liu et al. 2025) and ICE-Bench (Pan et al. (025b)) yielding 103 samples for further evaluation. Detailed curation protocols are provided in the Appendix.
Metrics. We propose a four-criteria framework to evaluate both identity preservation and accuracy of requested modifica- tions, comprising Semantic Consistency (SC), Visual Quality (VQ), Natural Appearance (NA), and Identity Preservation (IP), SC, VQ. and NA are scored by Qwen2.5-VE VL- using normalized [0. 1] scale via visual reasoning. For IP, we first use ArcFace similarity Sare (Deng al. 2019) but some methods inflate it by copying or barely altering the source. We therefore define rectified score that penalizes such cases: Qwen2.5-VI, parses the source face and predicts expected edit amplitude iin from the instruction, while the actual amplitude Preat is derived from SSIM between source and edit. The rectified IP score calculated as:
PPins Preal )2),
Srip max(0, Sare
(7)
Pins +
7

---

Published as conference paper at ICLR 2026
Predicted
Inpul image
Landmarks
Ours
OmniGen
Step1-EEdit
BAGEL
FLUX. FLUX.IKOnEE Kontext
Instruction: strongly scared
Instruction:
303 right
30"down
Instruction:
angry right
Instruction: disgusted 30f
Figure 5: Qualitative comparison with sate-offthhearr image editing methods.
Table 3: Ablation study on landmark condition types and landmark positional encoding. We calculate the LI distance between the edited image and provided landmark as the Landmark Error to evaluate landmark alignment precision.
Landmark type
Encoder
Additional Setting SC VQ f NA f IP Latency(s) Landmark Error
0.804 0.725 0.801 0.571
49.6
0.821 0.712 0.744 0.584
83.6
1.76
rendered image
visual VAE
compression
0.816 0.704 0.709 0.569
61.3
3.07
(1) w/o PE
0.654 0.611 0.630 0.512
50.7
58.9
landmark tokenizer
(2) RoPE
0.778 0.751 0.786 0.621
52.1
25.4
coordinates
(3) learnalle RoPE
0.803 0.737 0.791 0.617
52.9
9.63
(4) ours
0.832 0.749 0.805 0.634
52.1
2.34
4.2 QUALITATIVE AND QUANTITATIVE ANALYSTS
We benchmark LaTo against state-of-the-an methods on two datasets: HFL--10K (fine-grained edit instructions) and the antribatti splits of Biadit-Bench/ICE. Bench (global descriptions). For fairness, we fine-tune Step1X X-Edii and Kontext on HFL-150K using their official implementations. Table 2 reports statistically significant gains across all metrics. We ob- serve 5.3% and 7.4% relative improvements in SC on HFL- 150K for the two approaches, suggesting that our dataset better reflects real-world diversity for this task. On HFL--150K LaTo surpasses the second- best method, Bagel, by 4.6% SC, and LaTo-IP exceeds FLUX.1 Kontext by 7.8%. On Betttt-eeenneeeeeeeeeeeeeeee
LaTo outperforms Bagel by 7.2%, demonstrat- ing stronger identity preservation. Despite sharing the same training data and base models 2.9% average absolute improvement across all metrics, validating the effectiveness of our landmark tokenization design. Qualitative results (Figure 5) show superior pose accuracy and identity consistency, while maintaining photorealism under large expression changes where most baselines introduce cartoon-like or synthetic artifacts.
ReepX-Eche LaTo achieves
4.3 ABLATION STUDY
Facial condition formulations. We compare against two standard landmark conditioning schemes: (1) rendering land- marks 2D images and extracting facial tokens with shared VAE, and (2) downsampling landmark images with position shifting (inspired by OmininControl2 (Tan et al. 2025)) to improve efficiency. As shown Table 3 relative to the fine- tuned baseline, these image-bases based variants are limited: NA decreases by 5.7%, IP increases only by 1. 1.3%, and compression

---

Published as conference paper at ICLR 2026
Cross-Identity
Left: Imput image Right: Edit result
Incomplete Instruction:
Happy 30"
Input image
Mask 10%
Input image
Mask 30%
Input image
Mask 50%
Stylistic
Input image
happy
Input image
30
Input image
scared
Qualitative results on challenging inputs, including cross-identity landmarks, incomplete landmarks, and stylistic inputs. For cross-identity cases, the corresponding driving images are shown in the bottom-right corner.
Figure 6:
Comparison of landmark predictor performance using unified prompting conditions.
Table 4:
Table 5: Performance analysis across different CFG scales for landmark condition.
Methods
Accuracy
CFG-scale SC vQ NA
IP
Landmark Error 1
Qwen2.5-VIs VVLL3B
0.477
0.803 0.733 0.816 0.619
9.63
Owen2.5-VL-722
0.597
4
0,832
0.749 0.805 0.634
2.34
Gemini 2.5 Pro
0.613
0.821
0.698 0.751 0.616
1.94
Ours
0.730
0.807
0.656 0.679 0.588
1.82
further widens the gap despite a 26% speedup. In contrast, our landmark tokenization models spatial relations between co- ordinates and facial attributes and decouples control strength, achieving 6.1% % higher NA, 1.1% higher SC, and 5.0% higher IP than rendered-image conditioning. It also attains 37% speedup. matching the baseline's 's computational efficiency.
Landmark positional encoding effectiveness. We investigate various positional encoding (PE) strategies for landmark tokens, including original relative encoding (RoPE), learnable RoPE, no PE, and our location-mapping RoPE. The results in demonstratee that: (1) No PE leads to unstable training and unnatural results due to the absence of spatial aware- ness; (2) Original RoPE fails to effectively capture spatial relationships between landmarks and target images, achieving suboptimal landmark error of 25.4; (3) Learnable RoPE improves both instruction adherence and landmark conditioning but remains inferior to our approach; (4) our method provides the model with a strong geometric prior, enabling rapid geometry information extraction and achieving superior performance both landmark conditioning and identity fidelity.
Landmark predictor accuracy evaluation. To evaluate the effectiveness of landmark predictor, conducted a manual accuracy assessment against Gemini 2.5 Pro. Qwen2.5-VLL722B, and Qwen2.5-VL-33 Specifically, we selected 50 human faces from the HFL-150K test set and randomly generated instructions for each image by altering expression, head pose angle, or their combinations (6 variations per image), resulting in 300 samples. The predicted landmarks were rendered on source images, and participants were asked to evaluate whether the predicted landmarks aligned with the given instructions,

---

Published as conference paper at ICLR 2026
Happy Surprised
Angry Disgusted
Sad Angry
30" left 60" right
Figure 7: Visualization of the performance of landmark predictor under ambiguous instructions. Predicted landmarks are overlaid on the source images.
input
predicted landmark
cfg
cfg
cfg w=7
cfg w=10
Figure 8: Visualization of the landmark-aware CFG scale w.
assigning score of (incorrect) or (correct). We recruited 10 human evaluators and collected their results, which are presented in Table Our fine-tuned model achieves the highest average accuracy among the compared models, outper- forming the baseline by 25.3%, demonstrating its superiority in landmark analysis. To further evaluate LP under realistic noisy instructions, we test on composite expressions (e.g.. happy + surprised) and conflicting pose instructions (e.g., turn left and turn right), as shown in Figure 7 I. remains robust to instruction- ambiguity. For composite expressions, it produces plausible intermediate states the landmark space. For conficcing poses, the CoT module infers a coherent head orientation and often defaults near-frontal view when cues conffict.
Landmark-awe -wware aware CFG scaling analysis. Figure g and Table 5 illustrate the impact of our CFG scale. Raising the CFG scale improves landmark alignment, yet may generate more artifacts and compromise video quality. We adopt CFG scale of 4 as the optimal baseline configuration.
The sensitivity to challenging inputs. Although LaTo is mainly celf-landmark-drivenn via the proposed LP, also test in three challenging settings: (I) missing landmarks, (2) cross-identity landmarks, and (3) stylized inputs. As shown in Figure 6, LaTo mostly preserves identity under cross-identity conditioning and moderate landmark dropout, and keeps stable editing quality on non- -ootpoooorraliitic inputs. We attribute this to three components. First, the landmark tokenizer separates shape structure from pixel appearance, stabilizing geometric reasoning and helping capture expression dynamics and identity-agnostic pose changes. Second, the token fuser makes the model jointly interpret driving geometry, editing instructions, and identity features, enabling implicit, feature-level geometry control instead of explicit geometric copying. Third, learned unconditional landmark tokens let each landmark position flexibly balance geometry and identity, so even with severe landmark loss (about 50%), LaTo still preserves basic visual quality and key facial traits.
CONCLUSION
We presented LaToo, landmark-tokenizeed diffusion transformer for fine-grained, identity-preserving face editing. LaTo quantizes landmark coordinates into discrete facial tokens and aligns them with image tokens via a location mapping posi- tional encoding, which decouples geometry from appearance and enables precise control with strong identity preservation and high efficiency. A lasion-l landmark predictor with structured reasoning infers target landmarks from instruc- tions and source images, improving robustness and interactive controllability. This design removes the need for dense pixel-wise correspondence and mitigates identity drift under large pose expression changes. To support research at scale, we curate HFL-150K, a large-scale benchmark face pairs with fine-grained instructions, spanning real-world imagery and outputs from advanced models. Extensive experiments demonstrate that LaTo delivers state-of-the-art photorealism. semantic consistency, and computational efficiency, establishing a strong foundation for controllable, human noman-centric edit- ing.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This work supported by National Natural Science Foundation of China (62571322, 62431015, 62271308), STCSM (24ZR1453 1432000, 24511 24511106902, 106902, 24511106909 106900, 22DZ2229005), 111 plan 18007,9010) and State Key Laboratory of UHD Video and Audio Production and Presentation.
REFERENCES
Omri Avrahami, Or Patashnik, Ohad Fried, Egor Nemchinov, Kfir Aberman, Dani Lischinski, and Daniel Cohen-Or. Stable flow: Vital layers for training-free image editing. In Proceedings of the Computer Vision and Patlern Recognition Conference, pp. 7877-7888, 2025.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang. Shijie Wang. Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
Tim Brooks, Aleksander Holynski, and Alexei A Efros. Instructpix2pix: Learning to follow image editing instructions. In Proceedings of the IEEE/CVF conference on computer vision and paltern recognition, pp. 18392-18402, 2023.
Mingdeng Cao, Xintao Wang, Zhongang Qi, Ying Shan, Xiaohu Qie, and Yinqiang Zheng. Masactrl: Tuning-free mu- tual self-attention control for consistent image synthesis and editing. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 113600-2570,,0 2023.
Liang Chen, Shuai Bai, Wenhao Chai, Weichu Xie, Haozhe Zhao, Leon Vinci, Junyang Lin, and Baobao Chang. Mul- timodal representation alignment for image generation: Text-image interleaved control is easier than you think. arXiv preprint
2025.
Yuhao Cheng, Zhuo Chen, Xingyu Ren, Wenhan Zhu, Zhengqin Xu, Di Xu, Changpeng Yang, and Yichao Yan. 3d-aware face editing via warping-guided latent direction learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,, pp. 916-926, 2024.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261 2025.
Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, et al. Emerging properties in unified multimodal pretraining. arXiv preprint 1rXi::2505.1468 2025.
Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Areface: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVE conference on computer vision and pattern recognition, pp. 4690-4699 2019.
Yanbo Ding, Xirui Hu, Zhizhi Guo, Chi Zhang, and Yali Wang. Mtverafter: 4d motion tokenization for open-world human image animation, 2025, URL http::///a/: http:///aaaxii...a/aa/////ee
org/ abs////s.////////////
10238
Tsu-Jui Fu, Wenze Hu, Xianzhi Du, William Yang Wang, Yinfci Yang, and Zhe Gan. Guiding instruction-basse image editing via multimodal large language motebbroks<2222nstteppii//
arXiv preprint arXiv:2309.17100 2023.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengi Generative adversarial networks. Communications of the ACM, 63(1 ((11):1399144 2020.
Zhen Han, Zeyinzi Jiang, Yulin Pan, Jingfeng Zhang, Chaojie Mao, Chenwei Xie, Yu Liu, and Jingren Zhou Ace: All- round creator and editor following instructions via diffusion transformer. arXiv preprint arXiv:2410.00086, 2024.
Yuzhou Huang, Liangbin Xie, Xintao Wang, Ziyang Yuan, Xiaodong Cun, Yixiao Gc. Jiantao Zhou, Chao Dong, Rui Huang, Ruimao Zhang, et al. Smartedit: Exploring complex instruction-based image editing with multimodal large language models. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 8362- 8371, 2024.
Yu Huang, Chia-Chin Chiang, Jian-Hao Chen, Yi-Chian Chen, Hsin-Lung Chung, Yu-Ping Cai, and Hsiu-Chuan Hsu. A study on computer vision for facial emotion recognition. Scientific reports, 13(1):8425, 2023.
Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint 2rXiv:2410.21276. 2024.

---

Published as a conference paper at ICLR 2026
Nikita Karaev, lurii Makarov, Jianyuan Wang, Natalia Neverova, Andrea Vedaldi, and Christian Rupprecht. Cotracker3: Simpler and better point tracking by pseudo-labelling real videos, 2024.
Tero Karras, Samuli Laine, and Timo Aila. A style-based generator architecture for generative adversarial networks. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp.
Diederik P Kingma and Max Welling. Auto-encoding variational bayes, 2022. URL
1512.6113
Black Forest Labs, Stephen Batifol, Andreas Blattmann, Frederic Boesel, Saksham Consul, Cyril Diagne, Tim Dockhorn, Jack English, Zion English, Patrick Esser, et al. Flux. 1 kontext: Flow matching for in-context image generation and editing in latent space. arXiv preprint arXiv:2506.15742, 2025.
Hui Li, Zidong Guo, Seon-Min Rhee, Seungju Han, and Jae-Joon Han. Towards accurate facial landmark detection via cascaded transformers. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 4176 4176-4185, 2022.
Hui Li, Mingwang Xu, Yun Zhan, Shan Mu, Jiaye Li, Kaihui Cheng, Yuxuan Chen, Tan Chen, Mao Ye, Jingdong Wang, et al. Openhumanvid: A large-scale high-quality dataset for enhancing human-centric video generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 7752-7762, 2025.
Zhimin Li, Jianwei Zhang, Qin Lin, Jiangfeng Xiong, Yanxin Long, Xinchi Deng, Yingfang Zhang, Xingchao Liu, Minbin Huang, Zedong Xiao, Dayou Chen, Jiajun He, Jiahao Li, Wenyue Li, Chen Zhang, Rongwei Quan, Jianxiang Lu, Jiabin Huang, Xiaoyan Yuan, Xiaoxiao Zheng, Yixuan Li, Jihong Zhang, Chao Zhang. Meng Chen, Jie Liu, Zheng Fang, Weiyan Wang, Jinbao Xue, Yangyu Tao, Jianchen Zhu, Kai Liu, Sihuan Lin, Yifu Sun, Yun Li, Dongdong Wang, Mingtao Chen, Zhichao Hu, Xiao Xiao, Yan Chen, Yuhong Liu, Wei Liu, Di Wang, Yong Yang, Jie Jiang, and Qinglin Lu. Hunyuan-dit: A powerful melti-resolltion diffusion transformer with fine-grained chinese understanding, 2024. htttstts:://art
org/ /tt.b////2000005055...
Jayy Liang, Haotian Liu, Hongteng Xu, and Dixin Luo. Generalizable face landmarking guided by conditional face warping. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 2425-2435, 2024.
Li Lin, Santosh Santosh, Mingyang Wu, Xin Wang, and Shu Hu. Ai-face: A million-scale demographically annotated ai-generated face dataset and fairness benchmark. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 3503-3515, 2025.
Kanglin Liu, Gaofeng Cao, Fei Zhou, Bozhi Liu, Jiang Duan, and Guoping Qiu. Towards disentangling latent space for unsupervised semantic face editing. IEEE Transactions on Image Processing, 31:1475-1489, 2022.
Shiyu Liu, Yucheng Han, Peng Xing, Fukun Yin, Rui Wang, Wei Cheng, Jiaqi Liao, Yingming Wang, Honghao Fu, Chunrui Han, et al. Steplx-edit: A practical framework for general image editing. arXiv preprint arXiv:2504. 17761, 2025.
Mohammadreza Mofayezi, Reza Alipour, Mohammad Ali Kakavand, and Ehsaneddin Asgari. MD face: A unified multi- modal multilingual framework for human face generation and editing. arXiv preprint 20X.v2240..02369, 2024.
Ron Mokady, Amir Hertz, Kfir Aberman, Yael Pritch, and Daniel Cohen-Or. Null-text inversion for editing real images using guided diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 6038 6048-604722200 2023.
Xichen Pan, Satya Narayan Shukla, Aashu Singh, Zhuokai Zhao, Shlok Kumar Mishra, Jialiang Wang. Zhiyang Xu, Jiuhai Chen, Kunpeng Li, Felix Juefei-Xu, et al. Transfer between modalities with metaqueries. arXiv preprint 2rXiv:2504.06256. 2025a.
Yulin Pan, Xiangteng He, Chaojie Mao, Zhen Han, Zeyinzi Jiang, Jingfeng Zhang, and Yu Liu. Ice-bench: A unified and comprehensive benchmark for image creating and editing. 2025b.
14482]
Or Patashnik, Zongze Wu, Eli Shechtman, Daniel Cohen-Or, and Dani Lischinski. Styleclip: Text-driven manipulation of stylegan imagery. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 2085-2094, 2021.
William Peebles and Saining Xie. Scalable diffusion models with transformers. In Proceedings of the IEEF/CVF interna- lional conference on computer vision, pp. 4195-4205, 2023.
Martin Pernuš, Vitomir Štruc. and Simon Dobrišek. Maskfacegan: High- High-resolution face editing with masked gan latent code optimization. IEEE Transactions on Image Processing, 02:5893-5908 2023.
12

---

Published as a conference paper at ICLR 2026
Konpat Preechakul, Nattanat Chatthee, Suttisak Wizadwongsa, and Supasorn Suwajanakorn. Diffusion autoencoders: Toward meaningful and decodable representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10619-10629, 2022.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Míshkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In PmLR, 2021.
International conference on machine learning, pp.
Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bjöm Ommer. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention, pp. 234-241. Springer, 2015.
Rui Shao, Tianxing Wu, and Ziwei Liu. Detecting and recovering sequential deepfake manipulation. In European Confer- ence on Computer Vision, pp. 712-728. Springer, 2022.
Shelly Sheynin, Adam Polyak, Uriel Singer, Yuval Kirstain, Amit Zohar, Oron Ashual, Devi Parikh, and Yaniv Taigman. Emu edit: Precise image editing via recognition and generation tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8871-8879 2024.
Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan. Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. Neurocomputing, 568:127063, 2024.
Zhonglin Sun, Chen Feng, loannis Patras, and Georgios Tzimiropoulos. Lafs: Landmark-based facial self-supervised
learning for face recognition. In Proceedings of the IEEE/CVF Conference Computer Vision and Pattern Recognition, pp. 1639-1649, 2024.
Zhenxiong Tan, Songhua Liu, Xingyi Yang, Qiaochu Xue, and Xinchao Wang. Ominicontrol: Minimal and universal control for diffusion transformer. arXiv preprint arXiv:2411.15096 2024.
Zhenxiong Tan, Qiaochu Xue, Xingyi Yang, Songhua Liu, and Xinchao Wang. Ominicontrol2: Efficient conditioning for 08280
diffusion transformers, 2025. URL
Linoy Tsaban and Apolinário Passos. Ledits: Real image editing with ddpm inversion and semantic guidance. arXiv preprint vrXii:2307.00522,, 2023.
Narek Tumanyan, Michal Geyer, Shai Bagon, and Tali Dekel. Plug-and-play diffusion features for text-driven image- to-image translation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1921-1930, 2023.
Aaron Van Den Oord, Oriol Vinyals, et al. Neural discrete representation learning. Advances in neural information processing systems, 30, 2017.
Jiangshan Wang, Junfu Pu, Zhongang Qi, Jiayi Guo, Yue Ma, Nisha Huang, Yuxin Chen, Xiu Li, and Ying Shan. Taming rectified flow for inversion and editing, arXiv preprint arXiv:2411.0474 2024.
Peng Wang, Yichun Shi, Xiaochen Lian, Zhonghua Zhai, Xin Xia, Xuefeng Xiao, Weilin Huang, and Jianchao Yang. Seededit 3.0: Fast and high-quality generative image editing, 2025. URL htttps://aaxxi..orrrr argp/b////bb.. 05083
Mengting Wei, Tuomas Varanka, Yante Li, Xingxun Jiang, Huai- Khor, and Guoying Zhao. Towards consistent and controllable image synthesis for face editing. arXiv preprint 2502:2500..2265, 2025.
Haoning Wu, Erli Zhang, Liang Liao, Chaofeng Chen, Jingwen Hou, Annan Wang, Wenxiu Sun, Qiong Yan, and Weisi Lin. Exploring video quality assessment on user generated contents from aesthetic and technical perspectives. In Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 20154-20154, 2023.
Shitao Xiao, Yueze Wang, Junjie Zhou, Huaying Yuan, Xingrun Xing. Ruiran Yan, Chaofan Li. Shuting Wang, Tiejun Huang, and Zheng Liu. Omnigen: Unified image generation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 13294-13304 2025.
Tsun-Yi Yang, Yi-Ting Chen, Yen-Ye Yu Lin, and Yung- Yu Chuang. Fsa-net: Learning fine-grained structure aggregation for head pose estimation from a single image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1087-1096, 2019.
13

---

Published as conference paper at ICLR 2026
Yang Yang and Xiaojje Guo. Generative landmark guided face inpainting. In Chinese Conference on Pattern Recognition and Computer Vision (PRCV), pp. 14-26. Springer, 2020.
Qifan Yu, Wei Chow, Zhongqi Yue, Kaihang Pan, Yang Wu, Xiaoyang Wan, Juncheng Li, Siliang Tang, Hanwang Zhang, and Yueting Zhuang. Anyedit: Mastering unified high-quality image editing for any idea. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 26125-26133 2025.
Dongxu Yue, Qin Guo, Munan Ning. Jiaxi Cui, Yuesheng Zhu, and Li Yuan. Chatface: Chat-guides real face editing via diffusion latent space manipulation. arXiv preprint arX1v:2305.14742, 2023.
Xin Zhang, Siting Huang, Xiangyang Luo, Yifan Xie, Weijiang Yu, Heng Chang, Fei Ma, and Fei Yu. Museface: Text- driven face editing via diffusion-based mask generation approach. arXiv preprint arXiv:2503.23888 2025.
Haozhe Zhao, Xiaojian Shawn Ma, Liang Chen, Shuzheng Si, Rujie Wu, Kaikai An, Peiyu Yu, Minjia Zhang, Qing Li, and Baobao Chang. Ultraedit: Instruction-basse fine-grained image editing at scale. Advances Neural Information Processing Systems, 37:3058-3093, 2024.
Shizhan Zhu, Cheng Li, Chen- Chen-Change Loy, and Xiaoou Tang. Unconstrained face alignment via cascaded compositional learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3409-3417, 2016. 2016
Yule Zhu, Ping Liu, Zhedong Zheng, and Wei Liu. Seed: A benchmark dataset for sequential facial attribute editing with diffusion models. arXiv preprint arXiv:2506. 00562, 2025.
14


Reviews:
[{"review_id": "smnswDCsQl", "reviewer": "Reviewer_Cy7B", "summary": "This paper introduces LaTo, a landmark-tokenized diffusion transformer for fine-grained, identity-preserving human face editing. \nLaTo quantizes landmark coordinates into discrete facial tokens, integrating them into diffusion transformers using location-mapped positional encoding. \nIt also introduces a landmark predictor based on a fine-tuned vision–language model with structured chain-of-thought reasoning, allowing intuitive user control without explicit landmark input. \nThe authors further curate HFL-150K, a large-scale dataset containing 150k real and synthetic face editing pairs with precise annotations. Extensive experiments demonstrate that LaTo achieves superior identity preservation and semantic consistency compared to prior state-of-the-art models.", "questions": "1. How are identity-dependent facial variations (e.g., different muscle or bone structures) handled when generating landmarks for the same expression?\n2. How are identity preservation metrics validated against human perceptual ratings? Is there a correlation analysis?\n3. What is the sensitivity of the model to noisy or inaccurate landmark predictions from the VLM?\n4. Can the proposed landmark predictor generate accurate and stable landmark coordinates for non-realistic, stylized cartoon faces? Does the full LaTo pipeline (landmark tokenization + DiT fusion) maintain editing quality when the input lies outside the natural-face manifold?\n5. Can the landmark tokenizer or location-mapping positional encoding be applied to other structured domains, such as full-body or hand motion editing?\n6. How does the landmark predictor perform under noisy or ambiguous instructions?\n7. Are there any plans to release HFL-150K in subsets (e.g., synthetic only)?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. Direct landmark tokenization provides an efficient geometric prior for diffusion transformers.\n2. LaTo achieves consistent improvement across benchmarks (HFL-150K, ICE-Bench, GEdit-Bench).\n3. The effect of positional encoding and classifier-free guidance is well quantified.\n4. HFL-150K is a valuable, large-scale dataset that provides both real and synthetic samples with fine-grained instructions — a useful resource for the community.", "weaknesses": "1. The landmark predictor tends to produce nearly identical landmark configurations for the same expression instruction (e.g., “make him happy strongly”) across subjects, for example, the first four columns of Figure 1.\nThis over-templating reduces individual expressiveness and may lead to uniform, less personalized facial deformations.\nThe paper lacks analysis on inter-identity landmark variance.\n2. Figure 1 (second to last column) includes a cartoon-style input image as an example. In such cases, landmark prediction becomes substantially more difficult because the facial geometry deviates from human anatomy — e.g., mouths drawn as single lines, exaggerated eyes, missing noses, or highly stylized jawlines. \nThese features make standard 68-point facial landmark detection ill-defined.\nHowever, the paper does not clarify whether the proposed landmark predictor is trained or evaluated on such stylized data. \nGiven that HFL-150K is described as primarily real-human-face-based (Section 3.1), it is doubtful that the model has seen sufficient cartoon or synthetic faces during training.\n3. Although the paper promises public release “upon acceptance,” both LaTo’s code and the HFL-150K dataset are currently unavailable.\nFor a paper emphasizing both model and dataset contributions, this limits reproducibility.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "Cni0Xd9aKk", "reviewer": "Reviewer_oX4b", "summary": "This paper propose LaTo, a landmark-tokenized diffusion transformer for fine-grained and identity-preserving face editing, and curate a face editing dataset HFL-150K. The core designs of LaTo are (i) landmark tokenization of raw coordinates, (ii) location-mapping positional encoding, and (iii) landmark-aware classifier-free guidance. Experiments demonstrate that LaTo achieves state-of-the-art performance.", "questions": "1. Why did you choose to use a VLM for landmark prediction instead of traditional, well-established landmark detectors? Were any experiments conducted to demonstrate the effectiveness of your chosen approach?\n2. Was the accuracy of the head rotation estimation evaluated qualitatively? (Alternatively, if emphasizing the lack of such evaluation: The experimental evaluation appears incomplete. Specifically, is there any qualitative assessment of the accuracy for the estimated head rotation angles?\n3. Was a user study conducted?\n4. Were there any ablation studies on the proposed token fusing mechanism? How does it compare to other fusion strategies?\n5. How was the size of the codebook determined? Were ablation studies performed to validate the chosen size?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The edited images presented in this work demonstrate effective fine-grained control, with experiments showing improved performance over previous state-of-the-art methods. It enables more precise and adaptable face editing, holding significant practical value for relevant applications.\n- The overall pipeline is logical and intuitive, and the writing is well-structured.\n- Dataset contribution: This work develops an automated synthesis-and-curation pipeline and constructs a face-editing dataset comprising over 150K face pairs annotated with fine-grained instructions.", "weaknesses": "1. Weaknesses in Ablation Studies: Although the paper includes some ablation studies, they are relatively limited. Please provide the rationale for not using traditional landmark extractors as the landmark predictor, along with supporting ablation studies on its utility. Furthermore, it would be beneficial to include ablation analyses on the codebook size and the design of the multi-modal token fuser.\n2. Lack of user study. For the fine-grained face editing task, the quantitative objective metrics are insufficient to fully validate the effectiveness of the editing results.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "zVPid4hiE8", "reviewer": "Reviewer_ApJ8", "summary": "This paper achieves better facial control while maintaining high identity fidelity by tokenizing key points, as opposed to other methods that condition on rendered landmark images. Furthermore, the integration of a large model supporting text input further enriches the application scenarios. Additionally, the authors propose a new face editing dataset. I believe this is a complete and highly practical paper.", "questions": "1. In cross-identity scenarios, could there be an issue with inadequate decoupling between geometry and identity?\n2. What is the inference speed?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 4, "strengths": "1. By tokenizing key points, this method achieves better facial control while maintaining high identity fidelity, in contrast to other approaches that condition on rendered landmark images.\n2. The support for text input further expands the application scenarios.\n3. The data and experiments are sufficient, and the writing is clear and easy to understand.", "weaknesses": "1. The DIT structure results in significant resource consumption, and it is necessary to evaluate the inference speed.\n2. Keypoints have limited capability in modeling geometric information and are not fully decoupled from identity, which may pose challenges in cross-identity tasks.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "MJzHVBiyA8", "reviewer": "Reviewer_bTdN", "summary": "This paper introduces LaTo, a novel face-editing framework that leverages landmark-tokenized diffusion transformers (DiTs) for identity-preserving and fine-grained facial attribute control. The model quantizes facial landmark coordinates into discrete tokens and uses a landmark tokenizer along with a location-mapping positional encoding. LaTo aims to overcome challenges faced by existing methods by improving computational efficiency, identity preservation, and semantic consistency. The paper further introduces a new dataset, HFL-150K, containing over 150,000 face pairs with fine-grained editing instructions, which is used to train and benchmark LaTo.", "questions": "- Can the model be optimized for real-time use in applications like live avatar creation or video conferencing? What specific optimizations are needed to enable efficient on-the-fly processing?\n- How well does LaTo perform across various demographic groups such as age, ethnicity, or gender? Could you provide an in-depth analysis of its results on more diverse faces to ensure fair representation?\n- While the paper notes limitations of pixel-wise methods when landmarks deviate from the source image, how does LaTo handle extreme cases where landmark positions are highly inaccurate due to large pose changes, occlusions, or exaggerated expressions? Is there any noticeable degradation in output quality under these conditions?", "limitations": "", "rating": 6, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- Landmark tokenization is an innovative approach that moves beyond traditional pixel-based conditioning in face editing tasks. This method helps prevent identity drift when landmarks differ significantly from the source image.\n- The HFL-150K dataset is a valuable contribution, offering a large collection of high-quality real-world face pairs with fine-grained instructions essential for training advanced face editing models.\n- Comprehensive ablation studies and both qualitative and quantitative evaluations demonstrate LaTo’s effectiveness across multiple metrics, supporting its proposed innovations.", "weaknesses": "- The paper mainly focuses on facial expression and head pose editing but provides limited discussion on how well LaTo generalizes to other image domains or tasks (e.g., full-body or complex scene editing). Assessing its broader applicability would strengthen the claims.\n- Although HFL-150K is extensive, there is little information about its diversity regarding age, ethnicity, or other demographics—factors crucial for ensuring robust performance across different populations. More details on how these aspects were considered during dataset construction would enhance transparency.\n- There is an inconsistency: cfg is mentioned in the Methods section but not in the Abstract.\n- The main contribution is not included in the framework, making it confusing and difficult to understand.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)