SesaHand: Enhancing 3D Hand Reconstruction via Controllable Generation with Semantic and Structural Alignment
Authors: Zhuoran Zhao
Xianghao Kong
Linlin Yang
Zheng WEI
Pan Hui
Anyi Rao
Venue: ICLR 2026 Poster (2026)
Keywords: Controllable Hand Image Generation
3D Hand Reconstruction
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Recent studies on 3D hand reconstruction have demonstrated the effectiveness of synthetic training data to improve estimation performance. However, most methods rely on game engines to synthesize hand images, which often lack diversity in textures and environments, and fail to include crucial components like arms or interacting objects. Generative models are promising alternatives to generate diverse hand images, but still suffer from misalignment issues. In this paper, we present SesaHand, which enhances controllable hand image generation from both semantic and structural alignment perspectives for 3D hand reconstruction. Specifically, for semantic alignment, we propose a pipeline with Chain-of-Thought inference to extract human behavior semantics from image captions generated by the Vision-Language Model. This semantics suppresses human-irrelevant environmental details and ensures sufficient human-centric contexts for hand image generation. For structural alignment, we introduce hierarchical structural fusion to integrate structural information with different granularity for feature refinement to better align the hand and the overall human body in generated images. We further propose a hand structure attention enhancement method to efficiently enhance the model's attention on hand regions. Experiments demonstrate that our method not only outperforms prior work in generation performance but also improves 3D hand reconstruction with the generated hand images.

Published as a conference paper at ICLR 2026
SESAHAND: ENHANCING 3D HAND RECONSTRUC- TION VIA CONTROLLABLE GENERATION WITH SEMANTIC AND STRUCTURAL ALIGNMENT
Zhuoran Zhao' Xianghao Kong", Linlin Yang', Zheng Wei', Pan Hui! Anyi Rao²
I "The Hong Kong University of Science and Technology (Guangzhou)
The Hong Kong University of Science and Technology
*Communication University of China 2ehao0748conneeee hkust-gz edu. en anyiraolust.
ABSTRACT
Recent studies on 3D hand reconstruction have demonstrated the effectiveness of synthetic training data to improve estimation performance. However, most meth- ods rely on game engines to synthesize hand images, which often lack diversity in textures and environments, and fail to include crucial components like arms interacting objects. Generative models ate diverse hand images, but still suffer from misalignment issues. In this pa- per, we present SesaHand, which enhances controllable hand image generation from both semantic and structural alignment perspectives for 3D hand reconstruc- tion. Specifically, for semantic alignment, we propose pipeline with Chain- of-Thought inference to extract human behavior semantics from image captions generated by the Vision- Lsagun-e Model. This semantics suppresses human- irreevant environmental details and ensures sufficient human-centric contexts for hand image generation. For structural alignment, we introduce hicrarchical struc- tural fusion to integrate structural information with different granularity for fea- ture refinement to better align the hand and the overall human body in generated images. We further propose a hand structure attention enhancement method to ef- ficiently enhance the model's attention on hand regions. Experiments demonstrate that our method not only outperforms prior work in generation performance but also improves 3D hand reconstruction with the generated hand images.
promising alternatives to gener-
INTRODUCTION
3D hand reconstruction from single image plays a crucial role computer vision, human-computer interaction, and AR/VR applications (Han et al., 2022; Hosain et al., 2021; Arimatsu & Mori, 2020).
also has great potential in dexterous robotic manipulation for embodied intelligence (Gavryushin al., 2025; Luo al., 2025). Current state-of-the-art methods (Xu al., 2023; Moon & Lee, 2020; Huang et al., 2023) rely on large amounts of accurate ground-truth labels for training, but obtaining these labels is expensive and time-consuming, Therefore, synthetic data has gained growing interest in the computer vision community (Tesch et al., 2025; Black et al., 2023; Wood et al., 2021; Hasson al., 2019). Synthetic hand datasets typically feature various hand poses and are rendered by game engines to enhance realism (Gao et al., 2022; Li et al., 2023). Several recent studies have explored training 3D hand reconstruction models with real and synthetic data, achieving better results than only training with real data (L.i et al., 2023; Zhao et al., 2025). However, hand texture maps and backgrounds used to create synthetic hand datasets are limited, which affects the diversity of syn- thetic data. Hands are often placed in environments incompatible with the contextual semanties. Moreover, most synthetic hand datasets only show floating hands without arms and hardly feature hand-object interaction (see Fig. (a)), conflicting with the human body structure and failing to reflect real human behavior.
Diffusion models have shown impressive capability in generating realistic images from textual de- scriptions (Rombach et al., 2022; Gu et al., 2022). Texts are used as semantic controls to provide images with diverse settings. Several studies further explore controllable image generation with

---

Published as a conference paper at ICLR 2026
Image Synthesis/Gnnrratin Generation
[b) In-the-wildd
wM Cnop
Game Engine- based
Generative Model-based
Ours
Ours
The person about to take a bite of the donut...
Figure 1: (a) We present controllable hand image generation method that generates diverse hand images with semantic and structural alignment. (b) 3D hand reconstruction performance in the wild can be improved with better semantic- and structural- aligned generated images.
additional control modalities (Zhang al., 2023a; Mou al.. 2024), which unlocks the potential for paired data generation. Pre-trained diffusion models gain rich human prior information (Chang al., 2025; Kim et al., 2025) from massive training data, and controllable generative models ensure image-label alignment. These make diffusion models good alternative to mitigate the limitations of existing hand image synthesis approaches.
Several works employ controllable diffusion models to generate hand data. However, they are stricted to lab environments (Xu et al., 2024), suffer from inefficient training for hand-related fea- ture refinement (Park al., 2024), et al., 2025a). Moreover, prior work (Park et al., 2024) directly uses the large lision-language model (VLM) for image description generation, which poses challenges for plausible hand image gener- ation due to the overthinking issues (Xiao & Gan, 2025) of VLMs. The overthinking issue can introduce irrelevant information into image descriptions, creating semantic gap from insufficient human behavior context and leading to implausible hand image generation. Moreover, since the hand is critical part of the human body, neglecting the structural information of the human body associated with the hand can cause misalignment issues in generated images, such as floating hands and implausible human poses (see Fig. (a)).
ikeleton-based. ignoring the influence of hand shape (Chen
With these motivations in mind, we propose SesaHand, enhancing text-conditioned controllable hand image generation from both semantic and structural alignment perspectives. For semantic alignment, we find that the human-centric context within the image caption is crucial for gener- ating plausible hand images and can be decomposed into four components: human pose, action, hand action, and the environment through prompt influence and attention analyses. We define this human-centric context as human behavior semantics. To extract this semantics effectively, we pro- pose pipeline with Chain-of-Thhoure (CoT) inference (Wei et al., 2022; Zhang et al., 2023b), which employs a step-by-step thinking approach to identify and extract human behavior semantics while eliminating irrelevant details. Specifically, the CoT inference consists of three stages. First, a Captioner generates an initial image caption for an input image. Second, an Extractor identi- fies and decomposes essential components of the image caption with few-shot learning. Third, Composer composes the extracted components together into the final text prompt. The proposed CoT inference can generate image descriptions with sufficient semantic information about human behavior, facilitating the construction of improved text-image pairs to train the text-to-image model.
In diffusion- based models, self-attention maps preserve geometric and structural information of im- ages, which has been proven useful for image editing (Liu et al., 2024a) and human image ani- mation (Chang et al., 2025). This motivates us to strengthen the alignment of the hand and human body in controllable image generation by leveraging the structural information within self-attention maps. Specifically, we integrate structural information with different granularity with both global and local representations by extracting multi-resolution self-attention maps in the encoding and middle blocks of the control module (Zhang et al., 2023a) and hierarchically fuse them to refine the features fed into the Stable Diffusion generative backbone (Rombach et al.. 2022). This helps pro- vide enhanced structural information useful for hand image generation. Moreover, cross-attents attenntnnon maps reveal the semantic correspondence between image features and text embeddings. To high-
2

---

Published as a conference paper at ICLR 2026
light the local hand structure in image features, propose efficient approach of adding bias terms to the hand-related cross-attention maps, rather than relying on a slow embedding refinement process (Park et al., 2024).
Extensive experiments demonstrate that our method outperforms existing approaches in hand im- age generation. Notably, we further show that the generated hand images improve the 3D hand reconstruction performance on in-the-wild datasets (see Fig. 1(b)), paving the path for exploring the use of synthetic data from generative models in 3D hand reconstruction task. We summarize our contributions as follows:
1. We propose CoT inference- based pipeline that extracts human behavior semantics to con- struct text-image pairs for training. which mitigates overthinking issues in VLM -LM-ggnerated captions and improves semantic alignment in hand image generation.
2. We improve the structural alignment in hand image generation through a hierarchical struc- tural fusion method that integrates multi-leve features for hand-bod alignment, along with a hand structure attention mechanism that efficiently highlights hand-related regions.
3. We demonstrate through extensive experiments that our method not only outperforms exist- ing hand image generation methods, but also enhances in-the-wild 3D hand reconstruction performance using our generated images.
RELATED WORKS
Hand Image Synthesis. Creating real-world 3D hand datasets is both time-consuming and labor- intensive. Data collection typically involves using multi-camera or depth sensor setups (Chao al., 2021; Zimmermann et al., 2019; Hampali et al., 2020; Zhan et al., 2024; Chen et al., 2025b). Manual labeling is required to obtain ground-truth annotations. Synthetic hand data becomes an effective alternative. Early methods synthesize realistic hand images with game engines, which can render large-scale synthetic data efficiently (Gao et al., 2022; Moon et al., 2023; Li et al., 2023; Hasson et al., 2019). However, these images often show floating hands without arms, which is unnatural from the human behavior perspective, and incorporating human bodies requires an extra fitting pro- cess. Also, synthesizing realistic images requires additional efforts to collect hand texture maps and environment backgrounds, which are often limited in quantity and are not customizable (Li et al., 2023: Moon al., 2023). Our method generates hand images with natural human bodies and uses text prompts to provide diverse textures and enviromments. Recent works explore hand image gen- eration with diffusion models. HanDiffuser (Narasinhaswamy et al., 2024) and Hand1000 (Zhang et al., 2025) generate hand images from text prompts directly, but the generated images cannot be used to train 3D hand reconstruction models since they have no reliable corresponding labels. HoundHand (Chen et al., 2025a) achieves controllable hand image generation from 2D keypoints, which ignores the hand shape factor. HandBooster (Xu et al., 2024) supports hand image generation with various conditions, but it is limited to lab environments. Similar to method, Attention- Hand (Park et al., 2024) generates in-the-wild hand images with hand mesh conditions and text prompts. However, it suffers from huge training time cost in optimizing hand-related features and two-stage learning for global and local features. Our work achieves better generation perfor- mance without sacrificing time cost, and the generated hand images are useful for improving 3D hand reconstruction performance in in-the-wild scenarios.
Text-to-Image Generation. Text-to- Image (T21) generation has seen significant progress in recent years, enabling diverse and realistic image generation based on text descriptions. Early approaches utilize the generative adversarial network (GAN) (Reed et al., 2016; Xu al., 2018), and trans- formers also show potential for conditional generation (Ding et al., 2021; Yu et al., 2022). The emergence of diffusion models has marked a new era for T2I generation (Rombach et al., 2022; Saharia et al., 2022; Chen et al., 2024). Since text prompts cannot provide enough structural guid- ance, recent research introduces additional control modalities to enable customized and controllable T2I generation. ControlNet (Zhang et al., 2023a) integrates user-specific conditional information into the image generation process by creating a trainable copy of the Stable Diffusion model. T2I- Adapter (Mou et al., 2024) also adapts Stable Diffusion external control modalities with trainable adapters. Despite these advances, challenges still remain in hand image generation due to the small size of the hand and ambiguous occlusion problems. In this work, we enhance hand region learning
3

---

Published as a conference paper at ICLR 2026
person doing hunds in image?
apuert
Captioner
Caption
males suyme
Input Image
Promrut Copttonn sotruct the bey entittes and their personss nctton, hand action, envirommnn
Image Caprion
lesd weurr
Fow-shot Esample
T21
Human Behawior Semantics
Response
Compaser
Ituman Belasiir Image Caption Human Behavior Semantics
CoT Inference
Figure 2: (a) Comparison of hand image generation with VLM-generated caption (top) and human behavior semantics (bottom). Overthinking in VLM captions leads to attention shifts toward irrele- vant objects in later denoising steps, while human behavior semantics guide the model to focus human-related regions, generating more plausible hand images. (b) CoT inference in human behav- ior semanties extraction pipeline.
through hand structure attention enhancement and reduce ambiguity by extracting human behavior semantics from image descriptions for generation.
3 METHOD
Our method aims to enhance text-conditioned hand mesh image-controlled image generation with both semantic and structural alignment. the preliminary, we provide an overview of ControlNet (Sec. 3.1), a control module for Stable Diffusion, which facilitates controllable hand image genera- tion based on the hand mesh image. Firstly, we introduce our human behavior semanties extraction pipeline with Chain-of- Thought (CoT) inference that incorporates crucial components to achieve se- mantic alignment for hand image generation (Sec. 3.2). Secondly, we demonstrate our hierarchical structural fusion method to improve the hand-body alignment (Sec. 3.3. 1) and our hand structure attention enhancement method to emphasize local hand regions efficiently (Sec. 3.3.2).
3.1 PRELIMINARY
Latent diffusion model (Rombach et al., 2022) is an image generation method that performs the de- noising process in the image latent space. It utilizes pretrained image encoder to encode the input image into its latent embedding: = Exx). During training, the model optimizes a UNet (Ron- neberger et al., 2015) cœ predict the noise added noisy latent sample 2 conditioned on the text prompt embedding çt. Here, is noisy sample of 2n at random step t [0, TJ. At inference time, the model generates desired samples from the image latent space, starting from Gaussian noise 2T ~ (0, 1) and denoising through T' timesteps.
ControlNet (Zhang et al., 2023a) adds conditional control to a pretrained diffusion model, such as Stable Diffusion. It preserves the original model's 's capabilities by locking its parameters and creating a trainable copy of its encoding blocks, which is connected to the locked model via zero convolution layers Z(:.)) to stabilize training in initial steps. The training objective of ControlNet is given by:
L(0)= = E.,, [[(e-= [ c0(2z, (,,,),,,
where cy is task-specific latent condition encoded by small neural network from input condition e; in image space: Cj = (c)). In our case, c; denotes the hand mesh image and cy denotes its latent representation.
4

---

Published as a conference paper at ICLR 2026
3.2 SEMANTIC ALIGNMENT: HUMAN BEHAVIOR SEMANTICS EXTRACTION
To achieve plausible hand image generation, it is essential to include sufficient semantic information about human behavior in text prompts while eliminating irrelevant details of other elements. VLMs are effective for image captioning (Lin et al., 2025; Ge et al., 2024) and can be used to efficiently collect paired text-image data for training T2I models. However, VLMs exhibit overthinking is- sues (Xiao & Gan, 2025), which tend to describe every element in images in detail regardless of the specific scenario. Such irrelevant information about non-human elements can introduce bias to the generative model and degrade the generated hand image quality (Park al., 2024). As shown in Fig. 2(a), the image description generated by VLM (Bai et al., 2025) contains redundant descriptions such utensils, which causes unnecessary occlusions of the hand in the generated image.
Although overthinking issues exist, we find that the human-centric context in image caption is cru- cial for generating plausible hand images. A hand image can be semantically decomposed into human and environment components. To better capture human-centric information, we further de- compose the human component into three finer-grained parts: human pose, action, and hand action (see Fig. 2(a)). We define this human-centric context as human behavior semantics, which represent essential elements within hand image while eliminating irrelevant details that bias the T2I gener- ation process. To verify the superiority of the proposed human behavior semanties, we perform a prompt influence analysis by first generating captions and extracting human behavior semantics on 100 image samples. We then use either captions or human behavior semantics as text prompts along with hand mesh images to generate hand images and compute average hand confidence scores with a hand detector (Zhang et al., 2020). Hand images generated with human behavior semantics achieve a higher confidence score (97%) compared to VLM captions (86%). Moreover, we analyze attention maps in the T2I model to uncover the generation process (see Appendix A). We observe that over- thinking in VLM caption causes model's attention to deviate towards irrelevant object generation, especially in later denoising timesteps. In contrast, the T2I model with human behavior semantics generates a more plausible hand image by preserving focused attention on human-related regions.
Based on this observation, propose pipeline with COT inference to extract human behavior semantics effectively (see Fig. 2(b)). CoT has been validated as an effective means of guiding the reasoning process of LLMs (Wei et al., 2022). Therefore, we introduce CoT to our pipeline gen- erate human behavior semantics utilizing image caption and critical decomposed components with few-shot learning. Specifically, for an image X, its associated image description X, is first generated by Captioner: X, = Captioner(X). Then we use Extractor to extract the human behavior seman- ties within the image given X,:
Extractor(X,, Pa). where P. is a few-shot example of input and output that guides Extractor to extract key entities and attributes more accurately. We restrict the ouput to the JSON format since it has better parsability, enabling casier interpretations (Mitra et al., 2024). Then we employ Composer to extract the components from the output JSON and compose them together to obtain the final text Pf:
Pi Composer (P pose, P Paction: P Phand action: P Peny).
(2)
The final composed texts P1 are subsequently used to construct text-image pairs for training our controllable hand image generation model.
3.3
STRUCTURAL ALIGNMENT
3.3.1
HIERARCHICAL STRUCTURAL. FUSION
To strengthen the hand-body alignment in hand image generation, we propose multi-level structural information fusion approach. We adopt Stable Diffusion as the generative backbone and utilize Con- trolNet to incorporate 2D hand mesh control. ControlNet preserves the capability of the pre- pra-ted Stable Diffusion model by locking its parameters while creating trainable copy of its encoding and middle blocks. The encoding and middle blocks consist of ResNet layers and transformer blocks, which generate features of different resolutions (64, 32, 16, 8). Each transformer block consists of self- and cross-attention layers. We extract the self-attention maps generated from 2 and cy, denoted eyy, where represents the resolution. Self-attention maps with higher resolution capture fine-grained local human structural information, while lower resolution maps focus on the global human structural information (Chu et al., 2017).
5

---

Published as a conference paper at ICLR 2026
The person fowarce the camere, One hand is holding pink unbrella and wrope around the handle The other hand not visible within the frome."
smiling while looking
zero
UNet
Control- Not
Encoder
refine
Malti-Blsskatinn maps
UNet
Decoder
6gl2, k...
smillng
looking
hand
holding
hand
Figure 3: Hierarchical Structural Fusion. Multi- level self-attention maps are extracted from the ControlNet encoder and middle blocks, which capture the structural information of the input image. These maps are aggregated and applied to obtain the refined feature fed to the Decoder.
Figure 4: Hand Structure Attention Enhance- ment. Applying the enhancement (bottom) ef- fectively highlights the local structural human- and hand-related features compared to the orig- inal cross-attention maps (top).
As shown in Fig. 3, the noisy image latent oN first combined with the conditioning hand mesh fea- ture ey and then input to the control module. Self-attention maps in encoding and middle blocks are extracted refine the original feature fe output by the control module. Specifically, self-attention maps in different resolutions are first resized to the same resolution by applying the max pooling operation A and then aggregated by summation:
Σ M(
(3)
8,1632,664
Then the aggregated self-attention map & applied to the fa to produce the refined feature fè:
fz fo 2 v/,
where @ denotes matrix multiplication.
The refined feature is integrated into the original output f of the Stable Diffusion model through zero convolution layers Z:
Z()) +
(5)
This refined feature preserves more human structural information, which helps the UNet Decoder to generate hand images with improved structural alignment.
3.3.2 HAND STRUCTURE ATTENTION ENHANCEMENT
To emphasize the local structural features of the hand region in images efficiently, we introduce our hand structure attention enhancement approach. The cross-attention layers in the Stable Diffusion model inject textual information into the image generation process, enabling T2I models to gener- images consistent with the text prompts. To enhance the generation of hand-related regions, we first perform hand-related tagging on text prompts using part-of-speech tagging (Chiche & Yitagesu, 2022) from NLTK library (Loper Bird, 2002) following (Park al., 2024). We extract the index list I of input tokens belonging to the "Verb" category and containing the word "hand". We then derive the hand-related features through the cross-attention layers in the control module with the extracted index list T. Instead of refining noisy embedding through time-consuming optimiza- tion process (Park et al., 2024), we enhance hand-related features by applying a bias term to specific cross-attention maps, which enables hand-related attention maps contribute more to the final fea- ture in attention attention-weighter combination, improving hand image generation. In the cross-attention layer of the control module, the query Q; is derived from dr(2t,C)) where 2, is the noisy image embedding, cy is the latent hand mesh conditioning image, and (2 cy) is the input of the i-th cross-attention layer. The keys K; and values Vú are derived from the text embeddings a, produced by the CLIP text encoder (Radford et al., 2021). The cross-attention maps Marass are computed as:
Q.KT Vd
Moross = softmax
+ B),
(6)

---

Published as a conference paper at ICLR 2026
Nechirngg
State Difluston
121 Adightr
Contro
Atontanttas)
Meskieage
UniCorto Not.
121 Aagater
Certro
Adtentoniaadd
Figure 5: Qualitative comparisons with state-of-the-art text-to-image generation models.
Table I: Quantitative comparisons MSCOCO.
Method
FID
KID
FID-H
KID-H
Hand Coaf.
MSE-2D
MSE-3D
User Pref.
Stable Diffusion (Rombach et al., 2022)
40.52
0.806844
50.78
2,932
4,591
1,50
Coi-ContzllNet (Zlaoo al., 2023)
30.34
0.00744
37.77
0.02004
0.855
2.105
3,039
2.56
T21 P21-Adpteer (Mou 2024)
22.00
0.00761
32,08
0,01568
0.914
1.546
2.451
2.91
ControlNet (Zhang al,, 2023s)
21.67
0.00K58
40.52
0.02098
0,810)
2,182
3,21
AttentionHand (Park et al., 2024)
20.71
0.00301
27.09
0.01287
0.965
1.026
1,986
4.05
Ours
18.63
0.00553
17.77
0,00718
0.966
1.155
1,537
4,55
where d, is the dimension of the key, B is bias matrix introduced to enhance the hand-related feature to improve the generation of hand regions. The bias matrix B is defined as:
Sa, a, KEI 10, otherwise"
By =
where a is positive hyperparameter, is the query index and k is the key index.
The output of the cross-attention layer is computed by applying the enhanced cross-attention maps Mcross to the value V/:
c1) = Mcross Vi-
(8)
As shown in Fig. 4, this approach enhances the model's ability to capture and emphasize hand- related features, thus improving the generation of hand regions.
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTING
Datasets. For text-to-image generation, we follow prior work (Park et al., 2024) to use MSCOCO (Lin et al., 2014; Jin et al., 2020), which is preprocessed with RGB hand and hand mesh images for training and evaluation. We generate new image descriptions with our human behavior semantics extraction pipeline. For 3D hand mesh reconstruction, test datasets Hands-In-Actin
in-the-wild (HIC) (Tzionas et al., 2016) and Re:InterHand (RelH) (Moon et al., 2023). HIC consists of single and interacting hand sequences recorded with an RGBD camera with 732 samples in the test set. ReIH is synthetic interacting hand dataset, which is relighted with HDR backgrounds, RelH test has 126,640 samples. We show our qualitative results on MSCOCO.
evaluate
Evaluation Metries. To evaluate image generation performance, we adopt Frechet Inception Dis- tance (FID) (Heusel et al., 2017) and Kernel Inception Distance (KID) (Binkkowski al., 2018). We evaluate the quality of generated images in the hand regions with FID-H and KID-H by com- puting FID and KID on hand crops. We also evaluate the hand confidence score (Hand Conf.) to measure the hand quality in generated images with an off-the-shelf hand detector Mediapipe (Zhang et al., 2020) to compute the average confidence scores for detection following (Park et al., 2024; Narasimhaswamy et al., 2024). We follow prior work (Park et al., 2024) to report mean squared error of 2D keypoints (MSE-2D) with Mediapipe and mean squared error of 3D keypoints (MSE- 3D) using InterWild (Moon, 2023). For preference (User Pref.), rate 10 groups of generate hand images in the following aspects with a 5 5-iont Likert scale: visual quality, hand mesh alignment, and text alignment. To evaluate 3D hand reconstruction performance,
asked 12 volunteers to
7

---

Published as a conference paper at ICLR 2026
Table 2: Quantitative comparisons of 3D hand reconstruction methods with and without our gener-
ated images. indicates our re-implemented results.
Datasets
HIC (Tzionas al., 2016)
ReIH (Moon et al., 2023)
Method
1PVPE
RRVE
MRRPE 4
MPVPE L
RRVE
MRRPE
DIR (Ren et al., 2023)
21.89
26.11
43.11
21.82
29.66
37,01
AttentionHand
20.66 (-5.0%)
25.87 (-0.9%)
40.54 (-0.0%)
19.91 (-8.8%)
26.67 (-10.1%)
35.05 (-5.3%)
DIR" (Ren et al., 2023)
21.94
29.09
42.60
22.14
29.41
40.48
Ours
20. 18 (-6.7%)
27.78 (-4.5%)
40.01 (-6.3%)
19. (-13/3%)
27.42 (-5.85)
38. 10 (-8.9%)
InterWild (Moon, 2023)
15.30
21:35
31.26
13,99
20.07
22.38
AttentionHand
14.74 (-3.2%)
21.10 (-1.2%)
29.26 (-0.4%)
13:95 (-0.3%)
19.94 (-0.85))
22 05 (-1.8%6)
InterWild" (Moon, 2023)
15.30
24,20
31,26
13.99
20.07
22.38
Ours
14.70 (39.9%) 23.36 (-8.%% 28.57 (- 8.6%)
13.01 (-7.0%)
19.32 (-9.79)
22.17 (-9.95)
Input Image
w/o Ours
w Ours
Figure 6: Qualitative comparison of InterWild with and without our generated images on MSCOCO validation set.
we report three metrics: mean per-vertex position error (MPVPE), right hand-relative vertex error (RRVE), and mean relative-root position error (MRRPE) in mm. See Appendix C for more details.
4.2
MAIN RESULTS
Comparison on Image Generation. We compare our method with state-of-the-art controllable im- age generation methods. As shown in Tab. I, our method outperforms other methods on FID, FID-H, KID-H, and hand confidence score. Specifically, our method achieves improvements of 34% of FID- H and 44% of KID-H over AttentionHand, demonstrating the effectiveness of introducing semantic and structural information to improve hand image generation. Also, AttentionHand directly resizes images for training, which can result in a hand distortion problem that affects image generation performance. We further perform qualitative comparison to visually evaluate the effectiveness of our method (see Fig. 5). Our method achieves high-fidelity generation results compared with other methods. The hands in the generated images are aligned with the hand mesh image conditions, and the associated human bodies are in plausible poses. The results also show that the model can generate human bodies compatible with varying hand poses and shapes.
Comparison on 3D Hand Reconstruction. We evaluate the effectiveness of our method by fine- tuning two 3D hand reconstruction methods InterWild (Moon, 2023) and DIR (Ren et al., 2023) with our generated images on HIC and RelH. As shown in Tab. 2, our method outperforms Atten- tionHand, especially in MPVPE, achieving an improvement of 3.9% v.s 3.7% on HIC and 7.0% v.s 0.3% on ReIH with InterWild, and 6. 7% v.s 5.6% on HIC and 13.2% v.s 8.8% on RelH with DIR. We also show qualitative results in Fig. 6. Although InterWild is trained with large-scale real and synthetic datasets, it still underperforms on challenging in-the-wild samples, such as those involy- ing occlusions and truncations. Our generated images can enhance the model's robustness in hand detection well as hand mesh predictions.
8

---

Published as a conference paper at ICLR 2026
Ablation study of different components. SE, SF, and AE denote semantics extraction, struc- tural fusion, and attention enhancement. HC de- notes the hand confidence score.
Table 3:
Table 4: Speed comparison.
Method
ControlNet
AttentionHand
Ours
Duration (s/iter)
0.41
27.25
0.44
Table 5: Comparison different bias
SE
SF
AE
FID
KID
FID-H
KID-H
HC
X
21.04
0.00744
21.90
0.00982
0.942
X
X
19.83
0,00664
18.90
0.00807
0.944
1.0
1.5
2.0
2.5
X
19.05
0.00542
18.17
0.00722
0.960
FID
20.41
19.69
18.63
19.97
18.63
0.00553
17.77
0.00718
0.966
FID-H
19.75
18.98
17.77
19.01
4.3 ABLATION STUDIES
Component Ablations. As shown in Tab. 3, we conduct ablation study of our method compo- nents on generation performance. By incorporating human behavior semantics extraction pipeline, the performance can be improved from FID 21.04 19.83, demonstrating the effectiveness of human behavior semantics integration for improving image generation performance. We then in- troduce the hierarchical structural fusion and hand-related attention enhancement incrementally to improve structural alignment and highlight hand-related local features. Each addition fur- ther improves model performance. These experiments validate the contribution of each compo- nent to the overall performance. We further show visualizations of ablation studies (see Fig. 7). We show two challenging samples: one real image with motion blur and occlusion issues (both self- and object-occlusion), and another with varying hand shape scales. The results show that adding human behavior semantics ex- traction improves the hand image quality since highlights human- and hand-related context while eliminating irrelevant information. Hier- archical structural fusion injects the structural information of the human body into the gener- ation process, facilitating the hand-body align- ment. Furthermore, hand-related attention en- hancement enables better generation of hand regions.
Image
w/
Figure 7: Ablation examples. SE, SF, and AE de- note semantics extraction, structural fusion, and attention enhancement.
Training Efficiency Analysis. We perform training efficiency analysis compared with ControlNet and Attentionl Hand. As shown in Tab. 4, although our training speed is slightly slower than Control- Net, we are much faster than AttentionHand which requires a complex optimization process during training for hand-related feature refinements.
Ablations on the Bias Term. We conduct a hyperparameter search on the a of the bias matrix to determine its value. As shown in Tab. 5, bias value of 2.0 yields the best performance while other values have decreased results. This indicates that small bias enhancement may fail to highlight the useful hand-related regions while high bias introduces noise.
4.4 HUMAN PERCEPTUAL STUDY
We further evaluate SesaHand with human perceptual studies. The first is evaluate our human behavior semantics extraction pipeline compared to different VLMs, including Qwen- VL (Bai et al., 2023), Qwen2.5-VL-7B (Bai et al., 2025), and LLaVA-v1.6-7B (Liu al., 2024b). We randomly select 12 images and their de- scriptions generated by the VL.Ms and our pipeline, and conduct our study with 20 users. Users are asked to select the best description of human behavior in images among four descrip- tions. We collect 240 answers and report the corresponding preference rate. Fig. 8(a) shows that our pipeline outperforms other VLMs, achieving 41% % preference rate. The second
10 10 B
Perennage (56
Pereentage
33
(6) Models
(b) Metheds
Figure 8: Human perceptual studies.

---

Published as a conference paper at ICLR 2026
Hand Mash image
GPT-4o
Gemini 2.0 Flash
Nano Banana
Nano Banana Pro
Ours
The person is engaged in phone call. One hand is holding mobile, phone to their ear..
Figure 9:
Comparison with commercial models. Given a conditioning hand mesh image and text prompt, GPT-4o, Gemini 2.0 Flash, Nano Banana, and Nano Banana Pro fail to generate well- aligned and realistic hand images, despite their capability for multiple image generation tasks.
is to compare images generated by the model without our improvements to those generated with all improvements applied. We present 12 groups of generated images with text prompts and hand mesh conditions. Users are asked to select the best image regarding text alignment, structural control, and image quality. Our preference rate is 67% (see Fig. 8(b)), indicating the effectiveness of our approach. See Appendix B for more details.
4.5 COMPARISON WITH COMMERCIAL MODELS.
We further explore the ability of existing commercial models generate well-aligned hand im- ages. For qualitative comparison, we choose four widely-used commercial image generation mod- els: GPT-40 (Hurst al., 2024), Gemini 2.0 Flash (Comanici et al., 2025), Nano Banana (Gemini 2.5 Flash), and Nano Banana Pro (Gemini Pro Image). As shown in Fig. 9, despite providing these four models with additional explanations about the task, these models fail follow the instructions to generate aligned hand images. GPT-4o generates a realistic image, but it does not align with the conditioning hand mesh image. Gemini 2.0 Flash and Nano Banana are affected by the conditioning image and generate an undesired 3D rendering without texture. Nano Banana Pro performs better in generating realistic images, but still suffers from a misalignment problem with the conditioning mesh image. In contrast, our method can generate realistic and well-aligned hand image.
CONCLUSION
We propose SesaHand, controllable hand image generation method that achieves semantic and structural alignment to enhance 3D hand reconstruction. Previous hand image synthesis methods lack diversity and struggle integrate critical components, while diffusion-based hand image gen- eration methods suffer from misalignment and training inefficiency issues. For semantic aligmment, we propose human behavior semantics extraction pipeline with CoT inference identify and gen- erate crucial semantics, which helps extract essential human-centric context and reduces irrelevant information for better hand image generation. For structural alignment, we perform hierarchical structural fusion to refine the image features to enhance the hand-body alignment in the generated images. We also propose hand structure attention enhancement method that emphasizes the local hand features to improve the generation of hand regions. With these improvements, our method outperforms state-of -aree-of-th-httt the-art methods in both quantitative and qualitative evaluations.
6 LIMITATION AND FUTURE WORK
Our method may generate blurry hand images when the conditioning hand is small. When hand- object interaction exists, the hand and object sometimes fuse with each other at contact points. In future work, we plan to enhance the learning of the hand-object interaction region to achieve more realistic and natural hand-object interactions in generated images. Also, adopting a prioritized train- ing strategy that emphasizes higher noise levels training with higher sampling probability could further improve hand structure generation quality, since higher noise levels focus on generating images' structural information while lower noise levels focus refining details. Finally, incorpo- rating additional paired training data from ego-centric datasets can enhance our method's ability to generate better ego-centric images, benefiting downstream tasks such robotic manipulation.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGEMENT
This research was supported in part by the NSFC/RGC Collaborative Research Scheme Project No.CRS_I HKUST605/25, the National Natural Science Foundation of China 15))))561660115, the National Key Research and Development Program of China under Grant 2024YFC3307602, and the Guangdong Provincial Talent Program, Grant No. 20233000000000 Additionally, this research was supported by National Natural Science Foundation of China (No.62406298)
ETHICS STATEMENT
This work adheres to the ICLR Code of Ethics. All user data involved in the human perceptual study was rigorously anonymized. The datasets used in our research were accessed from published literature and open-source platforms such as GitHlub and Hugging Face, Our work aims to improve the hand image generation from a technical perspective and is not intended for malicious use.
REFERENCES
Kazuyuki Arimatsu and Hideki Mori. Evaluation of machine learning techniques for hand pose estimation handheld device with proximity sensor. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, pp. 2-1.,2000
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, local- ization, text reading, and beyond. arXiv preprint arXiv:2308. 12966, 2023.
Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang. Jun Tang. Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding. Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. arXiv preprint arXiv:2502. 13923, 2025.
Mikolaj Binkowski, Danica J Sutherland, Michael Arbel, and Arthur Gretton. Demystifying mndd gans. In International Conference on Learning Representations, 2018
Michael Black, Priyanka Patel, Joachim Tesch, and Jinlong Yang. Bedlam: A synthetic dataset of bodies exhibiting detailed lifelike animated motion. In innPro Proceedings of the IEEÉ/CVF Conference on Computer Vision and Pattern Recognition, pp. 8726-8737, 2023.
Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Openpose: Realtime multi- person 2d pose estimation using part affinity fields. IEEE transactions on pattern analysis and machine intelligence, 4321): 2019.
Zhe Cao, Ilija Radosavovic, Angjoo Kanazawa, and Jitendra Malik. Reconstructing hand-object interactions in the wild. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 12417-12426, 2021.
Di Chang, Hongyi Xu, You Xie, Yipeng Gao, Zhengfei Kuang, Shengqu Cai, Chenxu Zhang, Guox- ian Song, Chao Wang, Yichun Shi, al. X-dyna: Expressive dynamic human image animation. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 5499-5509, 2025.
Yu-Wei Chao, Wei Yang, Yu Xiang, Pavlo Molchanov, Ankur Handa, Jonathan Tremblay, Yashraj Narang, Karl Van Wyk, Umar Igbal, Stan Birchfield, et al. Dexycb: A benchmark for capturing hand grasping of objects. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9044-9053, 2021.
Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang. James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-a: Fast training of diffusion transformer for photore- alistic text-to-image synthesis. In The Tweifth International Conference on Learning Representa- fions, 2024.
11

---

Published as a conference paper at ICLR 2026
Kefan Chen, Chaerin Min, Linguang Zhang, Shreyas Hampali, Cem Keskin, and Srinath Sridhar. Foundhand: Large-scale domain-speciff learning for controllable hand image generation. In Pro- ceedings of the Computer Vision and Pattern Recognition Conference, pp. 77448-174605 2025a.
Shuhang Chen, Xianliang Huang, Zhizhou Zhong, Juhong Guan, and Shuigeng Zhou. A focused human body model for accurate anthropometric measurements extraction. In Proceedings of the Computer Vision and Puttern Recognition Conference, pp. 23658-22665 2025b.
Alebachew Chiche and Betselot Yitagesu. Part of speech tagging: a systematic review of deep learning and machine learning approaches. Journal of Big Data, 9(1):10, 2022.
Xiao Chu, Wei Yang, Wanli Ouyang, Cheng Ma, Alan L Yuille, and Xiaogang Wang. Multi-context attention for human pose estimation. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1831-1840, 2017.
Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blistein, Ori Ram, Dan Zhang, Evan Rosen, al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capa- bilities. arXiv preprini arXiv:2507.06261 2025.
Ming Ding, Zhuoyi Yang, Wenyi Hong, Wendi Zheng, Chang Zhou, Da Yin, Junyang Lin, Xu Zou, Zhou Shao, Hongxia Yang, et al. Cogview: Mastering text-to-image generation via transformers. Advances in neural information processing systems, 34: 19822-19855, 2021.
Daiheng Gao, Yuliang Xiu, Kailin Li, Lixin Yang, Feng Wang. Peng Zhang, Bang Zhang, Cewu Lu, and Ping Tan. Dart: Articulated hand model with diverse accessories and rich textures. Advances in Neural Information Processing Systems, 35:37055-37067, 2022.
Alexey Gavryushin, Xi Wang, Robert J. S. Malate, Chenyu Yang, Xiangyi Jia, Shubh Goel, Da- vide Liconti, René Zurbrügg, Robert K. Katzschmann, and Mare Pollefeys. Maple: Encoding dexterous robotic manipulation priors learned from egocentric videos, 2025. URL https: / //aaxxiv.rr//aaaaaaaaaaa 2200.. 06084..
Yunhao Ge, Xiaohui Zeng. Jacob Samuel Huffmann, Tsung- Yi Lin, Ming- Yu Liu, and Yin Cui. Visual fact checker: Enabling high-fidelity detailed caption generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14033-14044 2024.
Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The Ilama 3 herd of models. arXiv preprint 2407 2024.
Shuyang Gu, Dong Chen, Jianmin Bao, Fang Wen, Bo Zhang, Dongdong Chen, Lu Yuan, and Baining Guo. Vector quantized diffusion model for text-to-image synthesis. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 10696-10706, 2022.
Shreyas Hampali, Mahdi Rad, Markus Oberweger, and Vincent Lepetit. Honnotate: A method for 3d annotation of hand and object poses. In CVPR, 2020.
Shangchen Han, Po-chen Wu, Yubo Zhang, Beibei Liu, Linguang Zhang, Zheng Wang, Weiguang Si, Peizhao Zhang, Yujun Cai, Tomas Hodan, et al. Umetrack: Unified multi-view end-to-end hand tracking for vr. SIGGRAPH Asia 2022 conference papers, pp. 1-9, 2022.
Yana Hasson, Gul Varol, Dimitrios Tzionas, Igor Kalevatykh, Michael J Black, Ivan Laptev, and Cordelia Schmid. Learning joint reconstruction of hands and manipulated objects. In Proceedings of the JEFF/CVF conference on computer vision and pattern recognition, pp. 11807-11816, 2019.
Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge local nash equilibrium. Advances in neural information processing systems, 30, 2017.
Al Amin Hosain, Panneer Selvam Santhalingam, Parth Pathak, Huzefa Rangwala, and Jana Kosecka. Hand pose guided 3d pooling for word-level sign language recognition. Proceedings of the IEFF/CVF winter conference on applications of computer vision, pp. 3439. 2021.
12

---

Published as a conference paper at ICLR 2026
Lin Huang, Chung Ching Lin, Kevin Lin, Lin Liang, Lijuan Wang, Junsong Yuan, and Zicheng Liu. Neural voting field for camera-space space 3d hand pose estimation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9969-8978, 2023.
Aaron Hurst, Adam Lerer, Adam Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Os- trow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21271 2024.
Sheng Jin, Lumin Xu, Jin Xu, Can Wang, Wentao Liu, Chen Qian, Wanli Ouyang, and Ping Luo. Whole-body human pose estimation in the wild. In European Conference on Computer Vision, pp. 196-214. Springer, 2020.
Hyeonwoo Kim, Sangwon Baik, and Hanbyul Joo. David: Modeling dynamic affordance of 3d objects using pre-trained video diffusion models. Proceedings of the IEEE/CVF International Conference Computer Vision, pp. 10330-10341, 2025.
Lijun Li, Linrui Tian, Xindi Zhang, Qi Wang, Bang Zhang, Liefeng Bo, Mengyuan Liu, and Chen Chen. Renderih: large-scale synthetic dataset for 3d interacting hand pose estimation. In Proceedings of the IEEL/CVF international conference on computer vision, pp. 20395-20405, 2023.
Qiuxia Lin, Rongyu Chen, Kerui Gu, and Angela Yao. Semantics-awar test-time adaptation for 3d human pose estimation. Forty-second International Conference on Machine Learning. 2025.
Tsung-V Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pp. 740-755. Springer, 2014.
Bingyan Liu, Chengyu Wang, Tingfeng Cao, Kui Jia, and Jun Huang. Towards understanding cross and self-attention in stable diffusion for text-guided image editing. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 7817-7826, 2024a.
Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Improved reasoning, ocr, and world knowledge, January 2024b. URL https:// 01 30 1lava -lava-next/.
llava-ve senva-vl.ge vl.gitiiii trrttuub... wwtboo///////eee 2024
Yunze Liu, Yun Liu, Che Jiang, Kangbo Lyu, Weikang Wan, Hao Shen, Boqiang Liang, Zhoujie Fu, He Wang, and Li Yi. Hoi4d: A 4d egocentric dataset for category-level human-object interac- tion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 21013-21022 June 2022.
Edward Loper and Steven Bird. Nltk: The natural language toolkit. arXiv preprint cs/0205028, 2002.
Hao Luo, Yicheng Feng, Wanpeng Zhang, Sipeng Zheng, Ye Wang, Haoqi Yuan, Jiazheng Liu, Chaoyi Xu, Qin Jin, and Zongqing Lu. Being-hO: hO: sssionlanguaac-actiio pretraining from large- scale human videos. arXiv preprint (.0iv::507./559999 2025.
Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain-of- thought prompting for large multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 14420-14431, 2024.
Gyeongsik Moon. Bringing inputs to shared domains for 3d interacting hands recovery in the wild. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp- 17028-17037 2023.
Gyeongsik Moon and Kyoung Mu Lee. 121-meshnet: Image-to-lixet prediction network for accu- rate 3d human pose and mesh estimation from a single rgb image. In European Conference on Computer Vision, pp. 752-768. Springer, 2020.
Gyeongsik Moon, -hoou-I Yu, He Wen, Takaaki Shiratori, and Kyoung Mu Lee. Interhand2. 6m: A dataset and baseline for 3d interacting hand pose estimation from a single rgb image. In European Conference Computer Vision, pp. 548-564. Springer, 2020.
13

---

Published as a conference paper at ICLR 2026
Gyeongsik Moon, Shunsuke Saito, Weipeng Xu, Rohan Joshi, Julia Buffalini, Harley Bellan, Nicholas Rosen, Jesse Richardson, Mallorie Mize, Philippe De Bree, et al. A dataset of relighted 3d interacting hands. Advances in Neural Information Processing Systems, 36:17689-17701, 2023.
Chong Mou, Xintao Wang, Liangbin Xie, Yanze Wu, Jian Zhang, Zhongang Qi. and Ying Shan. T2i-adapter: Learning adapters to dig out more controllable ability for text-to-image diffusion models. In Proceedings of the AAAI conference on artificial intelligence, pp. 4296-43044 2024.
Supreeth Narasimhaswamy, Uttaran Bhattacharya, Xiang Chen, Ishita Dasgupta, Saayan Mitra, and Minh Hoai. Handiffuser: Text-to-image generation with realistic hand appearances. In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1468-2479 2024.
Junho Park, Kycongbo Kong, and Suk-Ju Kang. Attentionhand: Text-driven controllable hand image generation for 3d hand reconstruction in the wild. In European Conference on Computer Vision, pp. 329-345, Springer, 2024.
Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Reconstructing hands in 3D with transformers. In CVPR, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- 9748-8763. PmLR, 2021.
Scott Reed, Zeynep Akata, Xinchen Yan, Lajanugen Logeswaran, Bernt Schiele, and Honglak Lee. Generative adversarial text to image synthesis. In International conference on machine learning, pp. 1060-1069. Pmir, 2016.
Pengfei Ren, Chao Wen, Xiaozheng Zheng, Zhou Xue, Haifeng Sun, Qi Qi, Jingyu Wang, and Jianxin Liao. Decoupled iterative refinement framework for interacting hands reconstruction from single rgb image. In Proceedings of the IEEE/CVF international conference on computer vision,
pp. 8014 8025, 2023.
Robin Rombach, Andreas Blattmann, Dominík Lorenz, Patrick Esser, and Bjöm Ommer. High- resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF confer- ence on computer vision and pattern recognition, pp. 10684-10695, 2022.
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-net: Convolutional networks for biomedi- cal image segmentation. In International Conference on Medical image computing and computer- assisted intervention, pp. 234-241. Springer. 2015.
Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily L Denton, Kamyar Ghasemipour, Raphael Gontijo Lopes, Burcu Karagol Ayan, Tim Salimans, et al. Photorealistic text-to-image diffusion models with deep language understanding. Advances in neural informa- tion processing systems, 35:36479-36498 2022.
Joachim Tesch, Giorgio Becherini, Prerana Achar, Anastasios Yiannakidis, Muhammed Kocabas, Priyanka Patel, and Michael J Black. Bedlam2, O: Synthetic humans and cameras in motion. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2025.
Dimitrios Tzionas, Luca Ballan, Abhilash Srikantha, Pablo Aponte, Mare Pollefeys, and Juergen Gall. Capturing hands in action using discriminative salient points and physics simulation. Inter- national Journal of Computer Vision, 118(2):172-993, 2016.
Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824-24837, 2022.
Erroll Wood, Tadas Baltrušaitis, Charlie Hewitt, Sebastian Dziadzio, Thomas J Cashman, and Jamie Shotton. Fake it till you make it: face analysis in the wild using synthetic data alone. In Proceed- ings of the IEEF/CVF international conference on computer vision, pp. 3681-3691, 2021.
14

---

Published as a conference paper at ICLR 2026
Wenyi Xiao and Leilei Gan. Fast-slow thinking grpo for large vision- model reasoning. In The Thirty-nimi Anmual Conference on Neural Information Processing Systems, 2025.
Hao Xu, Tianyu Wang, Xiao Tang, and Chi-Wing Fu. H2onet: Hland-oclusion--ndd-rrrrntaioi aware network for real-time 3d hand mesh reconstruction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 17048-17058, 2023.
Hao Xu, Haipeng Li, Yinqiao Wang. Shuaicheng Liu, and Chi-Wing Fu. Handbooster: Boosting 3d hand-mesh reconstruction by conditional synthesis and sampling of hand-object interactions. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
10159-10169, 2024.
Tao Xu, Pengchuan Zhang, Qiuyuan Huang, Han Zhang, Zhe Gan, Xiaolei Huang, and Xiaodong He. Attngan: Fine-grained text to image generation with attentional generative adversarial net- works. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 1316-1324, 2018.
Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content- rich text-to-image generation. Transactions on Machine Learning Research, 2022.
Xinyu Zhan, Lixin Yang, Yifei Zhao, Kangrui Mao, Hanlin Xu, Zenan Lin, Kailin Li, and Cewu Lu. Oakink2: A dataset of bimanual hands-object manipulation in complex task completion. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp-
445-456, 2024.
Fan Zhang, Valentin Bazarevsky, Andrey Vakunov, Andrei Tkachenka, George Sung, Chuo-Ling Chang, and Matthias Grundmann. Mediapipe hands: On-device real-time hand tracking. arXiv preprint Xrxiv:2006. 102/4, 2020.
Haozhuo Zhang. Bin Zhu, Yu Cao, and Yanbin Hao. Hond1000: Generating realistic hands from text with only 1,000 images. In Proceedings of the AAAI Conference Artificial Intelligence, 2025.
pp.
Lymin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 3836-3847, 2023a.
Zhuosheng Zhang, Aston Zhang, Mu Muli, and Alex Smola. Automatic chain of thought prompting in large language models. The Eleventh International Conference on Learning Representations, 2023b.
Shihao Zhao, Dongdong Chen, Yen-Chun Chen, Jianmin Bao, Shaozhe Hao, Lu Yuan, and Kwan- Yee Wong. Uni-controlnet: All-in-one control to text-to-image diffusion models. Advances in Neural Information Processing Systems, 36:1 177-111500 2023.
Zhuoran Zhao, Linlin Yang, Pengzhan Sun, Pan Hui, and Angela Yao. Analyzing the synthetic-to- real domain gap in 3d hand pose estimation. In Proceedings of the Computer Vision and Pattern Recognition Conference (CVPR), pp. 12255-12265, June 2025.
Christian Zimmermann, Duygu Ceylan, Jimei Yang, Bryan Russell, Max Argus, and Thomas Brox. Freihand: A dataset for markerless capture of hand pose and shape from single rgb images. In Proceedings of the IEEE/CVF international conference on computer vision, pp. 813-822 2019.
15

---

Published as a conference paper at ICLR 2026
APPENDIX OVERVIEW
The appendix includes human behavior semantics extraction pipeline details, human perceptual study details, implementation details, additional quantitative and qualitative comparisons. The ap- pendix is organized as follows:
Sec. provides our attention analysis and the few-shot example used in our human behav- ior semantics extraction pipeline.
Sec. B shows more details of our human perceptual study.
Sec. C provides the implementation details.
Sec, D presents the module-wise runtime analysis.
Sec. E. shows more qualitative results and comparisons with FoundHand (Chen et al., 2025a), HandBooster (Xu al., 2024), ControlNet (Zhang et al., 2023a) and Attention- Hand (Park et al., 2024).
A HUMAN BEHAVIOR SEMANTICS EXTRACTION DETAILS
A.1 ATTENTION ANALYSIS
In this section, we show our attention analysis of the Stable Diffusion model with VLM -LM-generated image caption and human behavior semantics. As shown Fig. 10, we visualize the attention maps in the UNet Decoder during different timesteps. We choose early block and later block of the UNet Decoder as examples. In initial timesteps, both attention maps from the VLM--gnerated im- age caption and human behavior semantics focus the generation of human structure and the hand region. As the timestep increases, the model starts to add more irrelevant environmental details depicted in the prompt with VLM caption, which severely occlude the hand. In contrast, human behavior semantics enables the model to refine details while still preserving the hand region. This attention deviation towards gen- observation indicates that overthinking in image caption generation. Human behavior semantics erating those irrelevant objects that degrade the hand image with essential components mitigate this deviation and improve hand image generation performance.
Early Block
Later block
Generated Hand Image
VLM Image Caption
t-1
Larfy Block
Leier Block
Generated Hand image
t=T
Human Behavior Semantics
Figure/ 10: Visualization of attention maps in UNet Decoder early and later blocks with VLM-- generated image caption (top) and human behavior semantics (bottom). VLM-generated caption causes attention deviation towards irrelevant objects later denoising steps, while human behavior semantics enables more focused attention human- and
regions.
16

---

Published as a conference paper at ICLR 2026
A.2 FEW-SHOT EXAMPLE
In human behavior semantics extraction pipeline, we provide a few-shot example to guide Extractor generate more accurate responses that conform the JSON format for better composition. Here is the few-shot example:
"In the image, the person is holding pink umbrella with hand and appears be smiling while looking towards camera..."
Example Input:
Example Output:
"key entities": "person, umbrella".
"pose": "standing casually".
"action": "appears to be smiling while looking towards the camera",
"hand actionn:: "One hand is holding pink umbrella and wraps around the handlc..." "env": "possibly sunny beach"
B HUMAN PERCEPTUAL STUDY
We recruited 20 users via social media, including gender (12 female and 8 male) and age (M 26.33, SD = 1.24). 7 users are Al researchers, 3 are HCI researchers, 3 have technical backgrounds, and 7 are from other fields. We design questionnaire and share it with each user. There are 12 groups of questions, and we collected 240 answers in total. The preference rate for each model is calculated its number of selections divided by the total number of answers.
In the first human perceptual study, we ask each user to select the description that best reflects human behavior for each image, based on the following question and background information:
Question: "Which image description most accurately and clearly describes the human behavior without hallucination?"
Human behavior refers to the range of actions, reactions, and interac- tions exhibited by people in response to their environment.
Hallucination refers to descriptions that are irrelevant redundant for understanding human behavior.
As shown Fig. of the main paper, our visual prompt generation pipeline outperforms state-of- the-art vision-language models in describing human behavior within the image. Our visual prompt generation pipeline achieves 41 % preference rate, surpassing the second-best model Qwen2.5- VL-7B (Bai et al., 2025) with a 31% preference rate. LLaVA-v1.6-7B and Qwen-VL have 23% and 5% preference rates, respectively. We observe that Qwen-VL. (Bai et al., 2023) tends to generate brief image descriptions and cannot provide detailed information about human behavior, likely due to its limited visual reasoning ability. Swwen2.5-VL-7B is stronger than Qwen-VL, but exhibits overthinking issues and produces descriptions unrelated human behavior. LLa LLaVA-v1.6-7BB (Liu et al., 2024b) performs well in some cases, but still suffers from hallucination and struggles to describe human behavior accurately.
The second human perceptual study is about image generation. We sample 12 images generated by the model without our improvements (baseline) and the model with our improvements applied. Users are asked to select the better image in each group based on three criteria: text alignment, hand alignment, and generation quality:
Question: "Which image better matches the following text prompt, aligns with the hand mesh image condition, and has better image quality?"
17

---

Published as a conference paper at ICLR 2026
A total 240 answers collected and the preference rate is caleulated using the same method as in the first human perceptual study. As shown in Fig. 8 of the main paper, our method achieves superior performance in human preferences with a 67% preference rate, further validating its effectiveness. During the interview, users reported that images generated by the baseline model exhibit mismatched hand joints, unnatural hand-object interactions (with the hand and object fusing contact points), and incorrect hand shapes. Moreover, the baseline model fails to handle perspective well, which results in improper size of the arm relative to the hand in the generated images.
C IMPLEMENTATION DETAILS
For CoT inference-based human behavior semanties extraction, use Qwen2.5-VL (Bai et al., 2025) as Captioner and LLaMA 3 70B (Grattafiori et al., 2024) as Extractor for semantics ex- traction. For Composer, we integrate different output components together using a concatenation operation.
We Stable Diffusion 1.5 (Rombach et al., 2022) as our generative backbone and freeze its weight during training. An Adam W optimizer is utilized with a learning rate of le-5 to train the control module for 50 epochs with a batch size of 2. The hyperparameter a is set to 2.0. Experiments are conducted on one NVIDIA RTX 5000 Ada. The training takes around 2 days (50 GPU hours). We crop the input RGB image and the hand mesh image the training set centered on the two-hands region and resize it to 512 x 512 for training prevent distortion of the hand and human body caused by direct resizing.
For the evaluation of 3D hand reconstruction, reimplement InterWild (Moon, 2023) and DIR (Ren et al., 2023) using their official code. For InterWild, we select the checkpoint trained with InterHand2.66 (H+M) (Moon al., 2020), MSCOCO (Lin al., 2014), and Intentadannd (Ego_cat ) (Moon et al., 2023), since it produces results most similar to those reported the At- tentionHand paper, which does checkpoint released in its GitHuub repository. For DIR evaluation, we preprocess the Re:InterHand test set by first cropping the hand region in each image based on 2D joints, following the data pre- processing procedure used in the original implementation of DIR.
provide evaluation implementation details. For DIR,
the
For evaluation metries, MPVPE calculates the Euclidean distance between predicted and ground- truth 3D hand meshes after aligning the translation of the right and left hand separately. RRVE computes the relative position between two hands, aligning the translation of the right hand's root joint. MRRPE measures the 3D relative distance between two hands, which is the 3D distance between the predicted and ground-truth right hand root-relative left hand root position.
D MODULE-WISH WISE RUNTIME ANALYSIS.
We report module-wise runtime analysis in Tab. Both Captioner and Extractor are deployed locally. The GPU memory usage is 26 GB during training and 17 GB during inference. Training and inference are both conducted on a single NVIDIA RIX 5000 Ada.
Table 6: Module-wise Runtime Analysis.
Module
Captioner
Extractor
Attention Fusion
Diffusion Sampling
Inference
Runtime (s)
1.7
0.8
2.9x
0.44
4
E ADDITIONAL COMPARISONS
E. COMPARISON WITH FOUNDHAND.
Quantitative Comparison. We conduct a quantitative comparison with FoundHand (Chen et al., 2025a) by finetuning HaMeR (Pavlakos al., 2024) 10k images generated by FoundHand and our method in Tab. 7. Since FoundHand does not provide detailed experiment setups about the domain transfer experiment from Int (RelH) EpicKitchen in its paper, follow the
18

---

Published as a conference paper at ICLR 2026
Table Quantitative comparison with FoundHand by finetuning HaMeR (Pavlakos et al., 2024) on 10k images generated by FoundHand and SesaHand (ours). The results of FoundHand are from its paper.
FoundHand
PA- PA-MPJPE +
PA-MPVPE
SesaHand (ours)
PA-MPJPE &
PA-MPVPE J
Before finetune
14.41
7.52
Before finetune
9.71
9.89
After finetune
13.68 (-0.73)
6.29 (-1.23)
After finetune
8.56 (-1.15)
8.72 (-1.17)
RelH Image
2D Keypoints
FoundHand
Hand Image
SesaHand (ours)
Figure 11: Comparison with FoundHand (Chen et al., 2025a) on hand image generation based on Re:InterHand (ReIH) (Moon et al., 2023) images. FoundHand shows floating hands in generated images whereas our method preserves proper hand-body alignment.
setting of HaMeR to train and report the evaluation results. We randomly sample 10k images from ReIH and render the corresponding hand mesh images. The text prompts are generated leveraging CoT pipeline with semantic alignment. These hand mesh images and text prompts used for hand image generation. We finetune HaMeR by these 10k images generated with our method. As shown in Tab. 7, our method can generate hand images to further improve HaMeR's 3D hand reconstruction performance, which both improve 3D joint and mesh prediction. On the contrary, FoundHand shows limited performance but only 13.68 mm PA-MPJPE after finetuning. Moreover, we show a comparison of the generated images of FoundHand and our method in Fig. 11. We use FoundHand to generate images from RelH to EpicKitchen by using 2D keypoints from RelH and target domain conditions from EpicKitchen. As observed, FoundHand shows floating hands the generated hand images while our method produces hand images with hand-body alignment.
3D joint prediction, achieving 6.29 mm PA-MPVPE
Qualitative Comparison. compare our method with HoundHand, which supports gesture trans- fer, generating hand image given reference hand image and target pose. We select third-view image from MSCOCO and an ego-centric image from MOW (Cao et al., 2021). We obtain their corresponding 2D keypoints with Mediapipe, following the original implementation in FoundHand, and render the hand mesh image with the 3D ground-truth hand mesh provided by MOW. As shown in Fig. 12, the hand image generated by FoundHand has hand-bod body misalignment issue, where the human body in the generated image is not coherent with the hand. Furthermore, the generation results restricted by the reference images and cannot be easily controlled by text prompts, which limits its diversity. Moreover, FoundHand sticks to generating images in 256 x 256 resolution. Our method ensures the hand-bodh alignment the generated images and can control the image setting with text prompts. We support resolution of 512 x 512 with higher image quality. Notably, we can also generate a realistic hand image from an ego-centric view, while FoundHand fails to generate plausible hand image.
E.2 COMPARISON WITH HANDBOOSTER.
HandBooster (Xu et al., 2024) generates hand images conditioned on mesh images and additional normal images, which demonstrates its effectiveness for improving 3D hand reconstruction perfor-
19

---

Published as a conference paper at ICLR 2026
Third-person Image
Egocentric Image
2D Keypoints
Hand Mesh Image
2D Keypoints
Hand Mesh Image
is
The smoothing paper pressing poper. environment room woodinn possibly environment.
comf oetblrybbly
anjeying food,
pizz.
pizzo
smoothing
hand placed
indear, possibly at
dining setting.
Reference Image
Text Prompt
Reference Image
Text Prompt
256 256
512 x 512
256 256
522 x 512
FoundHand
Ours
FoundHand
Ours
Comparison with FoundHand (Chen et al., 2025a). FoundHand generates hand image in the target pose based on a reference image, which suffers from misalignment of the hand and body, well as limited diversity. Our method achieves better hand-body alignment and provide diverse settings through text prompts.
Figure 12:
mance on lab environments (e.g., Dex- YCB (Chao et al., 2021) and HO3D (Hampali et al., 2020)) To evaluate its performance, render the hand mesh and normal images from the Dex-YCB test set and conduct hand image generation with these conditions using HandBooster with its DCB s0 checkpoint. As shown in Fig. 13, the generated results show limited diversity in environment and texture, and the floating- hand issue remains noticeable. In contrast, our method generates well- aligned hand image with only a mesh image and text prompt, even without training on the Dex- YCB dataset.
We further evaluate HandBooster by generating hand images with hand mesh and normal images from MSCOCO. As shown in Fig. 14 rows and 2, HandBooster exhibits limited generalization ability for generating hand images conditioned on out-of-distribution mesh and normal conditions. Moreover, it also shows limited performance for generating small or interacting hands (see Fig. 14 rows and 4), likely due to insufficient human priors and the restricted expressiveness its training data. Our method can generate diverse in-the-wild hand images with plausible hand poses and hand- body alignment.
Moreover, try to train HandBooster on the MSCOCO dataset to perform comparison for im- proving 3D hand reconstruction as in Tab. 2. However, HandBooster hard to converge when is trained images with complex and diverse environments, likely because it is trained from scratch without internet-scale human prior information. Additionally, HandBooster requires normal maps essential input condition, whereas method does require normal map condition.
20

---

Published as a conference paper at ICLR 2026
Mesh Image
Normal Image
HandBooster
Mesh Image
Ours
Figure 13: Qualitative comparison between HandBooster and our method with mesh and normal conditions from Dex- YCB. Normal image is required only by HandBooster.
Image
Normal Image
HandBooster
Image
Ours
Figure 14: Qualitative comparison between HandBooster and our method with mesh and normal conditions from MSCOCO. Normal image is required only by HandBooster.
21

---

Published as a conference paper at ICLR 2026
E.3 QUALITATIVE RESULTS OF HAND-OBJECT INTERACTION (HOI).
As shown in Fig. 15, our method can generate diverse hand images with hand-object interactions with text prompts and hand mesh renderings. Moreover, a challenging HOI of fingering a guitar is shown in Fig. 16. We randomly choose an image fingering guitar from Pexels. The text prompt is generated with our CoT pipeline and the hand mesh rendering is rendered by HaMeR for hand image generation.
The person is sitting with arms folded to hold the dog on a boat in the sea...
The person is leaning forward slightly and playing game on nintendo wii console...
The person is standing and preparing for skiing...
The man is standing while both hands holding sandwich with fingers wrapped around it...
The person is seated casually and holding a snack...
Figure 15: More qualitative results of the hand image generation with hand-object interactions.
22

---

Published as a conference paper at ICLR 2026
Real Image
Hand Image
Ours
The person seated with arms playing position and playing on the ukulele...
Figure 16: Qualitative result on a challenging hand-object interaction (fingering guitar). The real image is randomly chosen from Pexels. The text prompt is generated using our CoT pipeline and the hand mesh image is rendered by HaMeR.
16nd Vaage Orouned
Daallaage
TUAN Mannina Cenersdd Image
Figure 17: Qualitative results of hand image gen- eration from different viewpoints.
Figure 18: Qualitative results of egocentric hand image generation using HOI4D (Liu et al., 2022) and MOW (Cao et al., 2021).
E.4
QUALITATIVE RESULTS OF VIEWPOINT PERSPECTIVES AND EGOCENTRIC VIEWPOINTS.
We show more qualitative results of the hand image generation given hand mesh renderings from different viewpoint perspectives Fig. and egocentric viewpoints in Fig. 18.
E.5 COMPARISON WITH CONTROL.ET OPENPOSE FULL AND ATTENTIONHAND.
We provide more qualitative comparisons in Fig. 19, Fig. 20, and Fig. 21. Since AttentionHand has not released its checkpoint yet, we train its model using its official code for two months. However, due the slow refinement process, the model still does tive results presented in the main paper are taken from its paper. We further compare our method with ControlNet v Openpose Full, which enables image generation controlled by Openpose (Cao et al., 2019) body, hand, and face keypoints. As shown in Fig. 19, Openpose sometimes fails to detect the hand, leading undesired generation results. Moreover, missing fingers occur in the generated images by ControlNet Openpose Full, even when hand keypoints are accurately detected. Additionally, the human body pose the generated image constrained by the detected pose from the input image, which limits the diversity of hand image generation. In contrast, our method can generate realistic and well-aligned hand images in various hand poses and shapes.
perform well. Note that the qualita-
F THE USE OF LARGE LANGUAGE MODELS
In this work, large language models were used only for language polishing and minor editing. All research ideas, methods, and experiments were carried out by human authors.
23

---

Published as a conference paper at ICLR 2026
+ Opennn Coennoo Full
ControlNet
AttentionHand
Ours
Condition
The person about take bite of the donut...
+ Openpose Full
ControlNet
AttentionHand
Ours
Condition
...The person appears to be about to take a sip...
Qualitative comparison with ControlNet + Openpose Full and AttentionHand on con- ditional hand image generation. ControlNet + Openpose Full uses full-body keypoints detected by Openpose as input, while AttentionHand and our method take the hand mesh image as input. Our method demonstrates better generation performance with both semantic and structural alignment.
Figure 19:
24

---

Published as a conference paper at ICLR 2026
ControlNet Openpose Full
AttentionHand
Ours
Condition
The person is preparing something related to crafting...
ControlNet + Openpose Full
AttentionHand
Ours
Condition
... The person holds a slice pizza...
Figure 20: More qualitative comparison with ControlNet + Openpose Full and AttentionHand on conditional hand image generation.
25

---

Published as a conference paper at ICLR 2026
+ Openpose Full
ControlNet
AttentionHand
Ours
Condition
The person engaged in phone call...
ControlNet + Openpose Full
AttentionHand
Ours
Condition
...One hand is holding phone his ear and the other hand is gesturing expressively...
Figure 21: More qualitative comparison with ControlNet + Openpose Full and AttentionHand on conditional hand image generation.
26


Reviews:
[{"review_id": "ytdWQ337P8", "reviewer": "Reviewer_bUzN", "summary": "This paper presents a pipeline to generate synthetic hand images to improve 3D hand reconstruction. It addresses:\n(1) Semantic alignment using COT to extract \"human behavior semantics\" for text prompts.\n(2) Structural alignment using hierarchical fusion for hand-body coherence and an attention bias to focus on hand regions. \nExperiments show that this generated data improves 3D hand reconstruction on in-the-wild datasets.", "questions": "1. In Fig. 7 w/ SE+SF+AE, the coffee-holding hand in the first example has a \"broken\" finger, and the small left hand in the second example has only 4 fingers. Despite all the improvement modules, the model still seems to struggle with generating plausible hand structures. What is the cause of this, and how do authors plan to address this next?\n2. Why not consider an explicit object condition, like object mask, for HOI alignment? How does the proposed method perform for more challenging HOIs, like fingering a guitar?\n3. FoundHand shows in their Domain Transfer experiment that finetuning HaMeR with 10k generated images from ReInterHand to EpicKitchen improves 3D reconstruction accuracy. How does SesaHand compare with this baseline?\n4. Most examples are shown in 3rd third-person view. The authors claim that their method can also generate an egocentric view, but Fig.10 only shows one sample of an egocentric view. Why not include a comprehensive experimental comparison on some egocentric hand data, e.g. HOI4D, ARCTIC, EpicKitchen, etc? Also, FoundHand shows many examples of egocentric generation in their results.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The problem statement of improving hand image generation quality to improve hand estimation task is sound. Data scarcity and fidelity is a key problems and current SOTA image generation models still fail to generate good hands occasionally.\n2. This work shows using COT to improve the caption alignment of VLM and attention bias improves hand image generation compared with some prior works. Thereby, improving 3D hand reconstruction from synthetic images.\n3. Writing and presentation clarity are good, and the paper is well-organized.", "weaknesses": "1. Though the paper focuses on improving semantic and hand structural alignment, objects in HOI alignment seem to be neglected; most examples only show simple cases of hands holding stuff. This work would be stronger if a more challenging and comprehensive study of HOI generation were demonstrated.\n2. The examples lack diversity in style and viewpoint perspectives. It's difficult to judge how robust the proposed method is.\n3. Comparison with the latest works, like FoundHand, is under investigation. See questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "iAu9TXpWHq", "reviewer": "Reviewer_46de", "summary": "The paper proposes SesaHand, a controllable hand-image generation framework that aims to improve downstream 3D hand reconstruction by enforcing both semantic and structural alignment. On the semantic side, the method uses a CoT pipeline to distill human-behavior semantics from VLM captions. On the structural side, it aggregates multi-resolution self-attention maps via hierarchical fusion and adds a lightweight bias in cross-attention to emphasize hand regions. Built on ControlNet-conditioned diffusion with hand-mesh images, the model generates diverse, human-centric images and mitigates common synthetic-data issues such as “floating hands” and hand–scene misalignment.", "questions": "The following questions are based on the weaknesses discussed above; please refer to that section.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 2, "contribution": 2, "strengths": "1. The CoT-driven semantic extraction and the hierarchical structural fusion are well motivated and technically coherent; together they target complementary failure modes (semantic drift vs. structural misalignment).\n2. The supplemental comparisons against commercial models indicate task-specific controllability and suggest that the proposed design adapts well to hand-centric scenarios where generic generators struggle.", "weaknesses": "1. The perceptual quality of generated images appears limited: several qualitative examples show unnatural finger articulation and color artifacts. Relative to SOTA hand rendering methods [i-iii] (acknowledging different goals and toolchains), the realism gap remains noticeable.\n2. Section 4.2 does not provide sufficient detail for the reconstruction setup. The paper does not specify the number of synthetic images used, training schedule, the proportion of generated vs. original data, or whether models are trained from scratch or fine-tuned. These omissions hinder reproducibility and attribution of gains.\n3. Efficiency is not quantified. The paper does not report module-wise runtime (CoT captioning, attention fusion, diffusion sampling), end-to-end throughput, training/inference cost, or memory usage. Given that efficiency is a practical criterion for synthetic data generation, the absence of such measurements weakens the empirical case.\n4. The discussion of hand–object interaction synthesis is limited. The paper does not address how to generate more natural interaction images with plausible contact, occlusion handling, and object pose consistency—factors that matter for reconstruction in interactive scenes.\n\n[i] Chen, Zhaoxi, et al. \"URhand: Universal relightable hands.\" CVPR. 2024.\n\n[ii] Potamias, Rolandos Alexandros, et al. \"Handy: Towards a high fidelity 3d hand shape and appearance model.\" CVPR. 2023.\n\n[iii] Iwase, Shun, et al. \"Relightablehands: Efficient neural relighting of articulated hand models.\" CVPR. 2023.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "sxfQc0lblF", "reviewer": "Reviewer_j2h5", "summary": "This paper proposes SesaHand, a framework for generating hand images conditioned on 3D hand parameters, achieving enhanced semantic and structural alignment in controllable hand image generation compared to existing methods. The synthesized images can be utilized for training 3D hand reconstruction models. The technical contributions are twofold. First, for semantic alignment, the framework employs chain-of-thought inference to generate effective human-centric image captions (used as inputs to the image diffusion model), thereby improving the semantic conditioning of the generated hand images. Second, for structural alignment, it introduces a hierarchical structural fusion mechanism to integrate hand structure information, along with a hand structure attention enhancement module. Experimental results demonstrate that the proposed method produces higher-quality hand images and yields better performance when used to train hand estimation models than the existing baselines.", "questions": "Please refer to the weaknesses (1) and (2) above.", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "**(1) Good presentation quality**\n\nThe paper is well written overall, and the text is easy to follow.\n\n**(2) Good experimental results**\n\nThe proposed method achieves strong results in both (1) controlled hand image generation and (2) hand pose estimation (when training an estimation model on the generated hand images). However, I have a few questions regarding the comparison settings (see the weaknesses section below).\n\n**(3) Good analysis to justify model design choices**\n\nAlthough the proposed method is relatively simple and straightforward, it provides good analysis to justify the design choices. For example, I appreciate the prompt influence analysis used to validate the proposed text prompt extraction strategy.", "weaknesses": "**(1) Questionable omission of FoundHand in quantitative comparisons**\n\nI believe FoundHand is one of the most relevant baseline works to this paper. Although it is discussed in the related work section and qualitative comparisons are provided in the supplementary material, I wonder why this baseline is omitted from the main quantitative experiments for both hand image generation and hand pose estimation. Since the code is publicly available, was there any specific reason why this comparison could not be performed?\n\n**(2) Questionable experimental setting**\n\nIn the experiments, when explaining the suboptimal KID performance of this method compared to AttentionHand, it is mentioned that “Although our KID score slightly underperforms AttentionHand, this may be due to the random sampling strategy in the KID calculation.” However, this statement raises concerns about the whole experimental rigor. In related fields (e.g., diffusion-based motion generation [1]), it is standard practice to run N = 10 experiments and report the mean metric values to mitigate the influence of such randomness. Have you also considered this?\n\n[1] Tevet et al., Human Motion Diffusion Model, ICLR 2023.\n\n**(3) Novelty**\n\nThe proposed modules are not highly novel, as they largely combine existing techniques for enhancing the controllability of diffusion-based generative models. However, I consider this limitation relatively minor.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ahM02YT7S0", "reviewer": "Reviewer_CFW1", "summary": "This paper presents a mesh-controlled hand-image generation model, trained by a well-prepared dataset extracted through Chain-of-Thought (COT), and enhanced by multi-scale structural information and text-guided hand structure attention. Experiments demonstrate its superior performance in generation quality and improvement to the 3D hand reconstruction task, compared to SOTA methods.", "questions": "Although this task uses hand mesh to control image generation, I'm still wondering, can the model generate hand images without any hand mesh, just guided by the text? Will the SesaHand still perform better than previous methods in this case?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "1. This paper is in a good writing style.\n2. The algorithm design, including image semantics extraction and two structural alignment modules, is reasonable. They are suited to the hand-image generation task. \n3. The results show promising performance in both generation quality and benefiting hand reconstruction.", "weaknesses": "### Major Weakness\n- The metrics reported in Table 1 are not complete compared to AttentionHand [1], including MSE-2D, MSE-3D, and user preference.\n- In Table 2, HandBooster [2] can be compared, as it also uses mesh-controlled generated images to enhance 3D hand reconstruction.\n- In evaluation, what's the text prompt input to the ControlNet/Stable Diffusion in SesaHand? Do they align with AttentionHand? This is important to ensure a fair comparison.\n- In Table 2, for the re-implemented results of InterWild, why does only **RRVE on HIC** differ so much from the official one (24.20 vs. 21.35)? Yet the other metrics are all the same. Not sure if this is correct or not.\n\n### Minor Weakness\n- In the section of related work, ``Hand Image Synthesis`` and ``Diffusion-Based Hand Image Generation`` can be merged into one paragraph, as they are closely related.\n\n\n[1] AttentionHand: Text-driven Controllable Hand Image Generation for 3D Hand Reconstruction in the Wild, ECCV 2024.\n[2] HandBooster: Boosting 3d hand-mesh reconstruction by conditional synthesis and sampling of hand-object interactions, CVPR 2024.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)