EgoHandICL: Egocentric 3D Hand Reconstruction with In-Context Learning
Authors: Binzhu Xie
Shi Qiu
Sicheng Zhang
Yinqiao Wang
Hao Xu
Muzammal Naseer
Chi-Wing Fu
Pheng-Ann Heng
Venue: ICLR 2026 Poster (2026)
Keywords: 3D Hand Reconstructin
Egocentric Vision
In-Context Learning
Primary area: applications to computer vision, audio, language, and other modalities
TL;DR: EgoHandICL is the first in-context learning framework for egocentric 3D hand reconstruction, achieving strong generalization and state-of-the-art results through vision-language model guidance and multimodal context integration.

Abstract: Robust 3D hand reconstruction is challenging in egocentric vision due to depth ambiguity, self-occlusion, and complex hand-object interactions. Prior works attempt to mitigate the challenges by scaling up training data or incorporating auxiliary cues, often falling short of effectively handling unseen contexts. In this paper, we introduce EgoHandICL, the first in-context learning (ICL) framework for 3D hand reconstruction that achieves strong semantic alignment, visual consistency, and robustness under challenging egocentric conditions. Specifically, we develop (i) complementary exemplar retrieval strategies guided by vision–language models (VLMs), (ii) an ICL-tailored tokenizer that integrates multimodal context, and (iii) a Masked Autoencoders (MAE)-based architecture trained with 3D hand–guided geometric and perceptual objectives. By conducting comprehensive experiments on the ARCTIC and EgoExo4D benchmarks, our EgoHandICL consistently demonstrates significant improvements over state-of-the-art 3D hand reconstruction methods. We further show EgoHandICL’s applicability by testing it on real-world egocentric cases and integrating it with EgoVLMs to enhance their hand–object interaction reasoning. Our code and data will be publicly available.

Published as a conference paper at ICLR 2026
EGOHANDICL: EGOCENTRIC 3D HAND RECON- STRUCTION WITH IN-CONTEXT LEARNING
Binzhu Xie Shi Qiu 1 Sicheng Zhang", Yinqiao Wang 12 Hao Xu
Muzammal Naseer", Chi-Wing Fu ? Pheng-Ann Heng!.
Department of Computer Science and Engineering, The Chinese University of Hong Kong
2 Institute of Medical Intelligence and XR, The Chinese University of Hong Kong
3 Department of Computer Science, Khalifa University [bzxie, cbiqiu)ecse cuhk. edu.
Hand MoMoooa
Background Event Noise Key Joints
Hand Mask
Input
(i) Previous Methods for Egocentric
GT
Step Template Retrieval
Step B ICL Demonstration
EBotanddo
?
VLM
Retrieved Template Image
Template Input
Template Target
Query Input
Query Target
Query Image
(ii) Our Proposed EgoHandICL for Egocentric Hand Reconstruction
Figure 1: Comparing EgoHandICL with previous methods. (i) Prior works improve egocentric
hand reconstruction by exploiting auxiliary cues (Prakash et al. 2024 Hara et al. 2025). thereby having limited capabilities in handling challenging scenarios with severe occlusions. (ii) Ego-
HandICL improves the precision through two key steps. Step A: We prompt vision- vision-language models with egocentric cues retrieve template images for the input query image. Step B: We construct ICL demonstrations by aligning the input-target pairs of both template and query images.
ABSTRACT
Robust 3D hand reconstruction is challenging in egocentric vision due to depth ambiguity, self-occlusion, and complex hand-ohject interactions. Prior works at- tempt to mitigate the challenges by scaling up training data or incorporating auxil- iary cues, often falling short of effectively handling unseen contexts. this paper, we introduce EgoHandICL,, the first in-context learning (ICL) framework for 3D hand reconstruction that achieves strong semantic alignment, visual consistency, and robustness under challenging egocentric conditions. Specifically, de- velop (i) complementary exemplar retrieval strategies guided by lanion-language models (VLMs), (ii) an ICI-tailored tokenizer that integrates multimodal con- text, and (iii) Masked Autoencoders (MAE))-based architecture trained with 3D hand-guided geometric and perceptual objectives. By conducting comprehen- sive experiments on the ARCTIC and EgoExo4D benchmarks, our EgoHandICL consistently demonstrates significant improvements over state-of-the-art 3D hand reconstruction methods. We further show EgoHandICL's applicability by test- ing on real-worke egocentric cases and integrating it with VLo to en- hance their hand-object interaction reasoning. Our code and data are available /ittps://github.. www.. com http:tu....ccce.ocooooo//aaaaaaa
: /
"Equal contribution.
Corresponding author.

---

Published as a conference paper at ICLR 2026
INTRODUCTION
Reconstructing 3D hands from monocular RGB images has long been a core computer vision task, with a wide range of applications in extended reality (XR), human computer interaction (HCI, robotics, etc. By leveraging transformer-bas based backbones trained on large-scale datasets to extract rich visual representations (Cui et al. 2023p. recent methods, such as HaMeR (Pavlakos et al. 2024) and WiL.OR (Potamias et 2025), achieve strong 3D hand reconstruction performance across var- ious benchmarks. While effective, these models can still encounter practical challenges, such as depth ambiguity, self-occlusion, and domain shift, which often affect their robustness and gener- alization in real-world applications (Hong et al., 2022 Zhu et al. 2024] Cui et al. 2024). These challenges are further aggravated in egocentric settings, where severe occlusions, perspective distor- tions, and complex hand objectt interactions are commonly present.
Previous works (Prakash et al. 2024 Hara et al. 2025) have also explored specialized solutions tai- lored for egocentric 3D hand reconstruction: as illustrated in Fig. (i), current methods utilize aux- iliary supervision cues that require additional annotations, but still fail resolve severe occlusions or ambiguous hand-object interactions in egocentric views. These limitations highlight the urgent need for flexible and generalizable framework that can adapt to diverse egocentric contexts with ro- bustness. Although each egocentric scenario presents unique visual ambiguities, humans naturally resolve them by drawing on prior experience, multimodal context, and task task-specific knowledge. This remarkable ability inherently aligns with the core concept of in-context learning (ICL) (Brown) ett al. 2020), which adapts to solve new problems by conditioning on a few relevant contextual ex- amples. While ICL has achieved remarkable progress in language modeling (Dong et al., [022]
Ferber et al. 2024), recent studies have begun to apply this paradigm for vision tasks, where rapid problem adaptation and example-gue -ugiddd reasoning are equally critica (Zong et al. (024). Given that ICL mirrors how humans exploit relevant cues to overcome ambiguity. it provides a natural paradigm for egocentric vision. Motivated by this inherent connection, we formulate a new frame- work, namely EgoHandICL, to advance egocentric 3D hand reconstruction by exploiting the strong reasoning capabilities of the in-context learning paradigm. As the first ICL-based approach for 3D hand reconstruction, we address the following two critical issues.
First, ICL's effectiveness highly depends on selecting relevant examples (Liu al.. 2022 Zhang eet al. 2023 Rubin et al.) 2002). yet egocentric scenarios make this selection particularly difficult. As shown in Fig. (i)) we propose complementary retrieval strategies for exemplar template selee- tion: (i) using four hand-involvemenn modes (left-hand, right-hand, two-hand, and non-hand) that broadly cover typical egocentric hand-related activities, we retrieve examples with similar hand in- volvement to ensure visual consistency between the query image and the matched templates; and (ii) we design interaction-specife prompts with a vision-language model (VLM) to retrieve adap- tive templates conditioned on semantic context. Together, these strategies ensure that exemplars with strong semantic alignment and visual fidelity are exploited during in-context learning.
Second, unlike conventional ICL methods addressing single-modal mappings, 3D hand reconstruc- tion requires bridging 2D visual inputs and 3D parametric outputs. To ensure structural consistency between queries and templates,
represent both inputs and outputs using unified MANO param- eterization (Romero et al.] (017). which yield semantically aligned input- output pairs that facilitate effective ICL. Moreover, we incorporate egocentric hand priors and multimodal cues encode vi- sual, textual, and structural context into a learnable token space, and adopt a Mask Autoencoders (MAE)-based architecture He et al. (2022) trained with 3D geometric and perceptual objectives. By conducting extensive experiments, we demonstrate that these designs enable our EgoHandICL framework to achieve more robust and generalizable egocentric hand reconstruction performance. In summary, our main contributions are threefold:
We present the first in-context learning approach to 3D hand reconstruction, showcasing a promis- ing way tackle severe occlusions and diverse interactions in egocentric vision.
We develop the EgoHandICL framework, which retrieves effective exemplars, tokenizes multi- modal context, and learns through MAE-style architecture for robust and generalizable 3D) hand reconstruction.
We present extensive experiments, demonstrating that our EgoHandICL achieves state-of-the-art performance on egocentric benchmarks and exhibits strong practicality in real-world scenarios, including self captured data and enhancing hand -ojject interaction reasoning in VLMs.
2

---

Published as a conference paper at ICLR 2026
2 RELATED WORK
3D Hand Reconstruction. Hand reconstruction has been extensively studied for long time. where early works used to employ depth cameras to recover 3D hand joints and meshes (Ge et al.
| Oikonomidis et al., [ Tagliasacchi et al.] [015] Rogez et al., 2014] [Simon et al. 2017 Sridhar et al.. 2016; /Snn et al.
2013; Tompson et al. 2014). A milestone is the introduction of -lndiimenioonal parametric hand model, which enables single- image 3D hand reconstruction by regressing associated posed shapes. [Boukhayma et al. (2019) then demonstrates the applicability of MANO by using a CNN model and an articulated hand mesh. This learning-based approach has inspired following methods that either directly regress MANO parame- ters (Baek al., 2019| Potamias al. 2023 Baek al. 2020) or predict hand vertices for improved
MANO (Romero et al. 2017).
P
image alignment (Kulon et al. 2019 Choi et al. 20200 Ge et al. [019] Kulon et al.) (020). Beyond parametric regression, alternative strategies have been explored, including voxel-based (Iqbal et al.)
2018 Moon & Lee 2020) and vertex regression methods (Chen et al. 2021). with a few work fur- ther emphasiziig efficiency and robustness of reconstruction (Chen et al. [022] Park et al. [022 Oh ct al. 2023 Jiang et al. 2023). More recently, large-scale vision transformers pretrained
millions of images demonstrate that scaling both model capacity and data volume significantly enhances the generalization of hand reconstruction models in-the-wild scenarios (Dong et al. 2024 Kim al.) 2023 Lin et al.) 2021). In particular, HaMeR (Pavlakos et al., (024) presents a powerful vision transtormer-based (Vaswani et al., 2017) pipeline that utilizes patched image tokens to reconstruct the MANO parametric model, achieving state-of-the-art accuracy in both egocentric and in-the-wild settings. However, existing methods mainly focus on general reconstruction and lack robustness under challenging interactions and occlusions: e.g., when hands cross and one is heavily obscured (Fig.B) 3. or when a hand blends into the background due a black glove (Fig. D. State-of-the-aag ap- proaches tend to miss hands, confuse left and right identities, or distort occluded regions. In contrast, our method leverages multimodal cues and acquires in-context knowledge from retrieved exemplars, achieving consistent and accurate hand reconstruction in these difficult egocentric scenarios.
In-Context Learning Vision. Popularized by GPT-3 (Brown et al., 2020), in-context learning (ICL) allows models to adapt to new tasks by conditioning without updating model parameters (Radford et al.) 2021 Rubin et al.] 2022, Xie et al. (02T). In- spired by the success of ICL in LLM exploration, researchers have also extended this paradigm to computer vision research, with representative efforts on segmentation (Li et al. 2024b). recogni- tion (Zhang et al. 2025a), and multimodal understanding (Zong et al. 2024). However, applying ICL to 3D vision is more challenging because 3D data involves complex spatial and temporal struc- tures that are not easily captured or represented by simple input-output pairs. Consequently, only a few studies have explored this area, such as PIC (Fang et al., 2023) for point cloud recognition, HiC (Liu al. 2025) for human motion, and TrajICL (Fujii al., 2025) for pedestrian trajectory forecasting, etc. Yet, none of them learns the contextual information needed to handle the large modality gap between 2D images and 3D meshes that exists the reconstruction problem. To this end, we unify both modalities in the MANO parameter space and develop an ICI--seecific tokenizer to effectively incorporate multimodal context. Moreover, within our EgoHandICL framework, the proposed VLM- exemplar retrieval strategies and MAE-driven reconstruction pipeline to- gether facilitate semantic alignment and geometric consistency, enabling robust egocentric hand reconstruction even under severe occlusion and visual ambiguities.
few imput-oupun demonstrations
3 METHOD
In this section, we first formulate in-context learning for egocentric 3D hand reconstruction (Sec. (II.. Then, we present our EgoHandICL framework (Scc.[3.2 including template retrieval strategies that select exemplar samples, an ICL tokenizer that integrates visual, textual, and struc- tural context, as well as the training and inference in practice. Finally, we introduce the specific losses that used to train the EgoHandICL framework (Sec. 3.3)
3.1 MODELING IN- -N-COTTEXT LEARNING IN EGOCENTRIC 3D HAND RECONSTRUCTION
ICL Preliminaries. In-context learning (ICL) typically models a task as few-shot inference, where model F conditions on small set of input-target exemplars and a query within shared context.
3

---

Published as a conference paper at ICLR 2026
Formally, given query 24h and context set C { b.) of N task-specific exemplar pairs (i.e., "templates"",, the model predicts yov by conditioning on both the query and the templates:
y = F(x"9(())
R Hx we aim reconstruct the 3D mesh of each hand i represented by the MANO model (Romero et al. 2017). We denote the set of MANO hand parameters as M {O,, B,, 4 P= where &₁ E R are the pose parameters, 9r E Io are the shape parameters, and +; E R is the global orientation. Our objective is to learn a mapping:
3D Hand Reconstruction. Given an image
M M M=G(T),
(2)
where 9 predicts the MANO parameters for all visible hands in the given image 1. The estimated parameters M are then passed to the MANO decoder obtain the corresponding 3D hand meshes.
ICL for Egocentric 3D Hand Reconstruction. Although we can realize 3D hand reconstruction by directly regressing MANO parameters from single image, it becomes far more difficult in egocen- tric views, which often suffer from occlusions, complex hand-object interactions, and ambiguous viewpoints. To mitigate this, leverage ICL with dedicated egocentric contextual exemplars, en- abling example-base reasoning for accurate and robust egocentric 3D hand reconstruction. To fulfill the objective of Eq, within the ICL formulation of Eq. [
we proceed as follows.
Given a query image /gry. we first retrieve template images that are contextually related to the query. For each retrieved template image Tppl- we construct an input-target exemplar pair (Mas. Mapi), aligning with both the objective of the 3D hand reconstruction task as well as the requirement of the ICL paradigm. Particularly, Mipi denotes coarse MANO parameters estimated by applying a trained reconstruction model (e.g., HaMeR (Pavlakos et al., 2024) WiLoR (Potamias al. 2025 to Iipl. as in Eq. 2
while Mip denotes the retrivvdd template' ground-truth MANO parameters from the database. Then, we obtain context set of egocentric exemplars Cм (MMM))MMp)))) Finally, we estimate the coarse MANO parameters Mar of the query image Igry with the same reconstruction model, and then refine it by conditioning on the exemplar set CM via the ICI. paradigm (as Eq. [
Mary F(Mary (CM).
This formulation establishes the foundation of applying ICL for egocentric 3D hand reconstruction.
3.2 EGOHANDICL FRAMEWORK
Fig. illustrates the overall EgoHandICL framework. It is composed of three key components: template retrieval strategies, multimodal context tokenizer, and ICL leaning and inference pipelines, where a VLM is prompted to retrieve the template mainly based on semantic descriptions derived from user-specified prompts regarding interactions, occlusions, and other egocentric cues.
S.c. a key step in our framework is retrieving a contextually relevant template image Tpp for the input query image Tary- As shown in Fig. (Part A), propose two complementary retrieval strategies that exploit both visual and textual information for contextual adaptation within the ICL paradigm.
Template Retrieval. As detailed in Sec. 3.1
Specifically, we introduce Pre-defined Visual Templates, where the VLM classifies each image into one of four pre-defined egocentric handdinvollvements: left hand-, right hand-, two hands-, and non hand-imo invvemmenteeent types. This categorization covers common egocentric hand configurations, al- lowing us to retrieve visually consistent exemplars from the dataset. In addition, hand involvement alone is not always sufficient, as egocentric scenarios often include object interactions, occlusions, missing hands, etc. To address these issues, we further introduce Adaptive Textual Templates, where the VLM retrieves the template mainly based semantic descriptions derived from user-specified prompts regarding interactions, occlusions, and other egocentric cues. This strategy enables more flexible and dynamic retrieval of semantically consistent exemplars, providing finer-grained contex- tual cues in addition to the basic pre-defined visual templates. In this work, we retrieve one template image per query image. Further implementation details are provided in Appendix B
ICL Tokenizer. Once the template image is retrieved, we construct the ICL tokens by incorporating multimodal context from the template image TTp and the query image Tyy- As shown in Fig. (Part
4

---

Published as a conference paper at ICLR 2026
Part A- Template Retrieval
Part B ICL Tokenizer
Part C- TTaainnng and Inference
Pre-defned Visual template
Encoder
00000000
Random Mask
Mi
00000000
Query Imaggn
Templne Image
& Adapiive Tociual Template
100000
Decoderr
tarn anfferveggtt sendreondrreiinn
P0 MarcMer
OutBuC
Temprene
ILL Tokera
0000 00000000
1888000
The hand Keppiig
Template Input
Tranuformar
cdason ovatt secuaios
Pour frgit brartdd les
Template Target
Equery input
Query Torget
Text of Query imae
Text of Templtte Image
Music Token
Figure 2: Overview of our EgoHandICL framework. Part A: Given a query image, we retrieve templates via two complementary strategies. Pre-defined Visual Templates: a VLM classifies the hand-involvement type and retrieves a template image of the same type, Adaptive Textual Templates: we prompt the VLM to generate semantic descriptions, and retrieve template image given textual similarity. Part B: We encode image tokens F,, structural tokens F. and text tokens F,, respectively; and then apply cross-attention to tokenize four structured sets of ICL tokens. Part C: We follow a MAE-style design, where the template and query target tokens are partially masked to train the Transformer. In inference, the query target tokens are fully masked for the Transformer's s prediction.
B), for both Ipp and /ap.. we estimate their coarse MANO parameters, collect their ground-truth MANO parameters, and feed these four sets of MANO parameters (i.e., Miph, Mary, Masi Mary) into a MANO encoder, generating the structural tokens Fm that preserve 3D hand articulation and shape priors. In addition, we encode Jus and Iuy. respectively, with pretrained ViT en- coder (Potamias et al.] 2025), capturing appearance and spatial details and generating the image tokens F,, Moreover, the VLMMggeerated semantic descriptions used for retrieval are embedded as the text tokens F, through text encoder. Finally, we apply cross-attention to fuse the multi- modal structural, image, and text tokens, producing unified ICL. tokens for context-aware reasoning. Specifically, after processing Iul and Lgry, our ICL Tokenizer produces four sets of ICL tokens: Tip (representing "input" of "template"), ) Tipl (representing "target" of "template"), Thp (representing "input" of "query"), and ary (representing "target" of "query"). Together, these ICL tokens are utilized in the bassd reconstruction model contextual exemplars.
Masked Reconstruction for ICL. Masked Autoencoders (MAE) (He et al. 2022) have been widely used in vision tasks, where part of the input tokens is randomly masked and model is trained to re- construct the missing ones. This architecture has proven effective for learning robust representations and handling incomplete or ambiguous visual signals. However, there is a key challenge when ap- plying it to the ICL paradigm: during training, the model has access to the ground truths of both the template and the query samples; while in inference, the query target unavailable, since it is exactly what we aim to infer. To tackle this, prior ICL methods for vision tasks (Bar et al. (0022 Wang et al.) 2023 Fang et al. (023) introduce masking into the target data during training: by partially masking the targets of both template and query samples, the model is trained to predict under incomplete supervision. As illustrated in Fig. (Part C), our EgoHandICL employs a transformer trained for masked reconstruction over ICL tokens. During training. we randomly and partially mask the target tokens, i.e., the ICL tokens of T and T to simulate such incomplete supervision. In inference, T is fully masked (unavailable), yet the trained model can decode (reconstruct) the target of query MANO parameters Mary from the remaining ICL tokens of contextual exemplars. In summary, this MAE-driven design offers an effective reasoning, enabling in-context hand reconstruction under challenging egocentric settings.
architecture for exemplar-
3.3 LOss FUNCTION
We follow standard practices in parametric hand reconstruction and employ parameter-level (Lmono) and vevel (Lv) supervisions to train our EgL to reconstruct 3D hand meshes:
Lmana
||0
0.| 2 + ||B
.e5 1 + ||
Lv
1|VV
v&b
(4)
5

---

Published as a conference paper at ICLR 2026
While parameter and vertex supervisions provide essential geometric constraints, they remain in- sufficient in egocentric settings with lacking enough information only from the input image itself. Inspired by perceptual losses (Johnson al. 2016) used for image restoration, hand-specific 3D) perceptual loss L3D. which aims to align high-level embeddings of predicted and ground-truth hand meshes. This loss enforces semantic consistency under occlusion and ambiguity, leading to reconstructions that are both geometrically accurate and perceptually realistic:
introduce
LND lp)(P) - ((P
(5)
where is the point cloud formed by the vertices (or joints) of the MANO-generated hand mesh, and (6-) is a pretrained 3D feature encoder. The overall loss L can be defined as:
L Xmaoo Xn + >anLan:
(6)
where Am. Au. and Aap are empirical weights. For datasets without MANO ground truth, e.g., EgoExo4D (Chen et al. 2024), the loss applies 3D key-joint J3D constraints, weighted by Az:
LJ = |J Ja L = ==A + AaDLaD-
3.4 IMPLEMENTATION
For the retrieval process, we employ Qwen2.5- Siven2.5-VI--72B, For the ICI. tokenizer, we use the same pretrained ViT backbone (Dosovitskiy et al. 2020) as in WiLoR, while the text encoder is Qwen-7B (Bai et al. 2023). The MANO encoder and decoder are implemented as MLPs, training together within the EgoHandICL framework. We implement the reconstruction model as a lightweight Transformer encoder. We also apply Uni3D-ti (Zhou et al. 2024) the 3D feature encoder & used in the loss LAD. All models with a learning rate of 1e-4. 4. The loss weights are set as Am = 0.05, Ae = 5.0, X, = 5.0, and Xan = 0.01. The training is conducted on single RTX 4090 GPU with batch size of 64 and AdamW optimizer (lyy 2018). We use 4 A100 GPUs for data preprocessing and retrieval.
( YYnng et al. 2024) as our VLM.
trained for 100 epochs
EXPERIMENTS
4.1 DATASETS AND EVALUATION METRICS
Datasets. We evaluate the EgoHandICL framework on two benchmarks: the ARCTIC dataset (Fan all.|12000 2023) provides high-quality MANO parameter annotations in controlled laboratory environ- ments; and the EgoExo4D dataset (Grauman et al.) 2024) consists of diverse egocentric videos with challenging hand-ohject interactions captured in realistic, unconstrained scenarios. These two datasets allow us to comprehensively evaluate the 3D hand reconstruction performance of Ego- HandICL from the perspectives of hand mesh vertices (ARCTIC, 118.2K training 16.9K testing samples) and hand skeleton joints (EgoExo4D, 17.3K training / / testing samples).
Baselines. We compare the performance of EgoHandILL with state-of-the-art 3D hand reconstruc- tion baselines. For the ARCTIC dataset, we mainly evaluate against 3D hand mesh reconstruc- tion methods, including HaMeR (Pavlakos et al.] (024)). WiLoR (Potamias et al. 2025). Wild- Hand (Prakash et al., 2024) and HaWok ęZhang al. 2025b). For the EgoFxo44 dataset, primarily assess against 3D hand joint estimation approaches, including POTTER (Zheng et al. 2023). PCIE-EgooaandPose (Chen al. 2024), and hand mesh reconstruction methods.
Evaluation Metrics. We evaluate egocentric hand reconstruction using both vertex-level and joint- level metrics, under two evaluation settings. In the general setting, metrics are computed for each detected hand, where cases with failed hand detection are excluded from evaluation. For ARCTIC, we report Procrustes-Alinne Mean Per Vertex Position Error (P-MPVPE), along with the fraction of vertices within 5mm and 15mm error thresholds (F@5, F@ 15). For EgoExo4D, we report the Mean Per Joint Position Error (MPJPE) and its Procrustes-aligned variant (P-MPJPE), together with F@ 10 and F@ 15 to measure the proportion of accurately reconstructed joints. To further capture the challenges of egocentric views while ensuring fair comparison under consistent detection conditions, we also evaluate under the bimanual setting, where only samples with both hands correctly detected are counted. In addi- tion to P-MPVPE, we compute the Mean Relative Root Position Error (MRRPE) in this setting, to quantify the spatial consistency between the left and right hands (Fan et al. 2021 2023 Moon et al.) 2020). All metrics for both vertex-level and lovet-level evaluations reported in millimeters.
Mean Per Joint Position Error (P-MPJPE) and Procrustes-Alignee

---

Published as a conference paper at ICLR 2026
Table Quantitative results on the ARCTIC dataset. We follow the standard evaluation protocol and report both the joint- and vertex-level metrics.
General Setting
Bimanual Setting
Method
P-MPJPE]
P-MPYPEL
F@51
F@
P-MPVPEL
MRRPE]
HaMeR (Pavlakos et al. 2024
9.9
9.6
0.046
0.911
9.9
10.1
WiLoR Potamaas ct al. 2025
5.5
5.5
0.524
0.994
5.7
9.8
Wand (Prakash el al 2024
5.8
5.6
0.746
0.928
4.9
7.1
HaWoR Zhhnng et al. 20255
6.2
0.474
0.896
EgoHandICL (ours)
4.0
3.8
0.801
0.996
3.7
6.2
Table Quantitative results the EgoExo4D dataset. We follow the standard evaluation proto- col and report the joint-level metrics.
General Setting
Bimanual Setring
Method
MPJPR
P-MPJPEJ.
F@ 101
F@R
P-MPJPE_
MRRPEJ.
PCIn-oeeoundPose (Chen et al. 2024
8.5
0.544
0.910
130.9
Potter (Zheng et al. 2023
28.9
11.1
0.491
0.910
10.3
148.9
HaMek Paviakos et at. 2024
30.1
11.2
0.453
0.883
10.6
361.2
Wil.oR (Polams et al.
31.1
12.5
0.528
0.905
11.0
378.0
Ego (ours)
21.1
7.7
0.789
0.935
110.9
4.2 EXPERIMENT ANALYSIS
Experimental Results of 3D Hand Mesh Reconstruction. We the ARCTIC dataset for both joint- and vertex-level evaluations. As shown Tab. [ while WiLoR performs well when evalu- ating only detected hands in the general setting, its accuracy drops in bimanual cases, often failing to recover heavily occluded hands or confusing left/right identities (see the bottom case in Fig. (3) Compared to the second best, EgoHandICL consistently improves PA-MPVPE by 31.1% and 24.5% in the general and bimanual settings, respectively, and further reduces MRRPE by 12%. This in- dicates finding that, with in-context learning, EgoHandICL is particularly effective resolving egocentric occlusions and estimating spatial relations between the two hands. For the EgoExo4D dataset (Tab. D. where MRRPE substantially higher than in the ARCTIC dataset due dynamic egocentric viewpoints, EgoHandICL. again achieves notable gains with reduced errors, demonstrat- ing strong spatial consistency between the two hands. Qualitative comparisons in Figs. 3 and 4 highlight that egocentric reconstruction involves not only single-band accuracy but also maintain- ing reasonable relative geometry between the two hands in rapidly changing egocentric views. In general, our EgoHandICL consistently improves both visual accuracy and geometric fidelity across scenarios with egocentric occlusions and bimanual interaction, providing a more robust solution for real-world cases. Further discussions are provided in Appendix C
Table 3: In-context reasoning analysis across different hand-involvement types. L,, R, T, and N denote training on the left-hand, right-hand, two-hand, and non- involvement type sub-dataset, respectively. Results are tested the ARCTIC under these four sub-dataset divisions.
Type
Hand
Right Hand
Hands
Non Hand
Method
P-MPJPEL
P-MPPPE
P-MPIPE
P-MPYPE
P-MPIPE,
P-MPYPEL
P-MPJPEJ
P-MPYPE_
Proposed L
4.6
4.4
4.6
4.5
5.1
4.8
5.2
5.1
Proposed
5.0
5.2
5.2
Proposed T
4.7
4.6
4.9
4.4
4.3
4.2
5.0
5.1
Proposed
4.9
4.7
5.1
5.5
5.0
Proposed Full
3.9
4.7
In-context Reasoning Analysis. To verify the in-context reasoning abilities of the EgoHandICL framework, as well as the effectiveness of our categorization of pre-defined visual templates, we train variants of EgoHandICL on sub-datasets, each containing only single hand-involvement type. As shown in Tab, each variant performs the best when the evaluation condition matches the hand involvement type of its training templates. For example, Proposed-L achieves the lowest errors when testing the sub-dataset of the left-hand involvement type. However, this configuration also

---

Published as a conference paper at ICLR 2026
ARCTIC
GT
EgoHandICL
WiLoR
WildHand
Figure 3: Qualitative results on the ARCTIC dataset. Note: In the bottom case, where the two hands cross and the left hand is severely occluded, WiL.R (Potamias et al. 2025) reconstructs only the right hand but mistakenly identifies as the left.
introduces side effects: models trained only on a specific sub-dataset (e.g., Proposed-N) generalize poorly to other types of samples (e.g., the Two Hands sub-dataset). To this end, our Proposed-Fuu model, trained on all sub-datasets, achieves the best accuracy across all hand-involvement types, demonstrating the synergistic benefits of in-context learning for stronger generalization and effective adaptation to diverse scenarios.
EgoExo4D
EgoHandICL
HaMeR
WiLoR
Self-captured
EgoHandICL
HaMeR
WiLoR
Figure 4:
Qualitative results the EgoExo4D dataset (left) and self-captured cases (right). Note: In the bottom-left case with a single heavily occluded hand, HaMeR (Pavlakos et al., 2024) mistakenly reconstructs two hands, whereas WiLoR (Potamias et al. 2025) fails reconstruct any.
Different Prompts for Retrieval. To assess the role of adaptive textual templates, com- pare two prompting strategies for generating retrieval texts via the VLM (Tab. 4.. When using only templates retrieved from the same hand-involvementt type without any adaptive textual templates (W/o. Prompts), the model achieves relatively stable performance on F@5 and F@ 15. Introducing adaptive textual tem- plates with descriptive prompts (Des. Prompts:
Table 4: Comparison of different prompts for adaptive textual templates retrieval. Results are tested the ARCTIC dataset.
Method
P-MPJPEL
P-MPVPE
F@51
F@
W%o. Prompts
4.3
3.9
0.838
0.996
Prompts
3.7
0.978
Reas. Prompts
3.9
3.7
0.766
0.975
e.g., "I am performing 3D hand reconstruction, please describe the interaction occlusion details in the egocentric image.") ") reduces reconstruction errors by providing explicit cues about occlusions. In particular, reasoning-style prompts (Reas. Prompts: e.g., "..., please provide guidance for han-
8

---

Published as a conference paper at ICLR 2026
dling occlusions and complex interactions.") further improve accuracy, they encourage VLM to exploit richer contextual cues with semantic grounding. These results indicate that leveraging VLM's reasoning capabilitiess enables more effective and robust in-context learning for hand recon- struction. Our default setting balances this trade-off by applying the reasoning prompts under heavy occlusion and the description prompts in clearer scenarios.
4.3 ABLATION ANALYSIS
Table 5: Comparison of coarse MANO pre- diction backbones. Improvement denotes rela- tive gains over the corresponding baseline per- formance on the ARCTIC dataset in Tab,
Table 6: Comparison of different mask ratios for ICL tokens. Results are tested on the ARC- TIC dataset.
Mask Ratio
P.MPJPE
P.MPYPE
F@51
F@51
P-MPJPEL
P.MPPPE
F@51
Ours W. HaMeR
8.3
8.1
0.052
4.3
0.761
0.995
Improvement
+16.1%
-0.4%
+13.4%
0.5
4.1
0.788
0.996
Ours W. WildHamd
5.1
4,9
0.823
0.6
4.1
0.791
0.997
Improvement
12.1%
12.5%
( 110.3%
4.0
3.8
0.801
0.998
Ours
0.8
4.2
4.0
0.782
0.996
Improvement +27.3%
30.9%
+7.3%
Backbone for Coarse MANO Prediction. Our EgoHandICL framework is designed be indepen- dent of the choice of coarse MANO prediction backbone. To verify this, we use different MANO prediction backbones our EgoHandICL framework. As shown in Tab. []] EgoHandICL consis- tently and significantly improves over the corresponding baselines. These results indicate that the gains are attributed to the in-context learning paradigm itself, demonstrating the generalization and robustness of EgoHandICL. regardless of the coarse MANO prediction backbone.
Impact of Mask Ratio for ICI. Tokens. We evaluate the mask ratio over a broad range (40%-80%) As shown in Tab. 6 lower mask ratios moderately degrade performance, while a 70% mask ratio yields the best results. This suggests that in egocentric hand reconstruction, where occlusions and ambiguous interactions to exploit stronger contextual cues and employ deeper reasoning to infer hidden structures. The observation consistent with the key insight of MAE (He et al. (002), where higher masking ratio enables the model to learn more informative latent features.
common, masking larger portion of ICL tokens encourages the model
3D Perceptual Loss. We analyze the role of our introduced hand-specific 3D perceptual loss, which designed improve implicit hand rep- resentation alignment, complementing geometric parameter supervision. As Tab. 7 shows, incor- porating our 3D perceptual supervision LAD pro- vides additional benefits over standard geometric regressions Lv and Lmano- This indicates that hand-specific 3D perceptual loss LAD adds com- plementary 3D cues that are not fully captured by Lv or Liano alone.
Table Comparison of different loss items for the EgoHandICL training. Results are tested on the ARCTIC dataset.
Loss function
P-MPVPE
F@5t
F@ 151
Lv
4.7
0.6
0.982
Ly + Lmaan
4.3
0.994
Lv + Lan
3.9
0.998
Lv + CaD + Lmano
3.8
0.998
EXPLORING EGOVLMS WITH EGOHANDICL
Recent work has investigated egocentric video- language models (EgoVLMss) for hand-object in- teraction understanding. To explore whether Ego- HandICL can facilitate this task, we evaluate on the EgoHOIBench (Xu et al., (025)) dataset with different ELo As shown in Tab. 8 EgoGPT (Yang et al.] (025). which is built on LLa LLaVA-One Vision (Li et al. 2024a) and fine- tuned on a wide range of egocentric data, un- derperforms the VA--OOOne Vision base model.
Table Comparison VLo on hand- object interaction reasoning.
Model
avg."
verb.?
none.t
action"
L.aVA-OneVVssoo
0.75
0.68
0.82
0.58
+ Proposed
0.78
0.71
0.84
0.61
EgoGPT
0.71
Proposed
0.76
0.70
0.90
0.61
Qwen2.5-VL-7B--mme
0.82
0.74
0.90
0.64
Proposed
0.85
0.74
0.91
0.69

---

Published as a conference paper at ICLR 2026
Q: Please understand the characters' hands and objects in the egocentric video.
right hand performs primary task of mixing the rice. The right hand uses utensil to interact with the rice, moving it in circular motions to ensure thorough mixing.
with visual prompts
by EgoHandICL.
Both hands appear be working in tandem, ... The right hand is the primary actor, using fingers to press and shape the rice within the bowl. the right hand would likely have been moving rhythmically, pressing and shaping the rice.
Figure 5: EgoVLM's hand-object interaction reasoning with and without EgoHandICL. By incorporating our hand reconstructions visual prompts, hand-related actions in egocentric videos can be recognized reliably with finer details.
This suggests that excessive finetuning egocentric data may bias EgoVLMs toward event-level understanding at the cost of fine-grained hand-object interaction reasoning. mitigate this issue, feed EgoHandICL's reconstruction outputs to the Ego as additional visual prompts, consis- tently improving their hand-object interactions reasoning capabilities (an example shown in Fig. 5 This exploration demonstrates the practical value of EgoHandICL for downstream applications.
CONCLUSION AND FUTURE WORK
We present EgoHandICL,, a novel framework for egocentric 3D hand reconstruction with in-context learning. Our method features three main components: (1) VLM-guided retrieval of exemplar tem- plates; (2) an ICL tokenizer synergizing multimodal context; and (3) an MAE-based architecture trained with specialized 3D geometric and perceptual constraints. Extensive experiments on bench- marks and real-world applications validate the effectiveness of our approach. While effective, the computational cost of the VLM-based retrieval presents a limitation for real-time deployment. the future, plan to refine the training pipeline and generalize EgoHandICL framework broader egocentric settings, such as 3D hand pose tracking and hand-object reconstruction.
ACKNOWLEDGMENTS
The work described in this paper was supported by the Research Grants Council of the Hong Kong Special Administrative Region, China, under Project T45-401/22-N; and in part by The Chinese University of Hong Kong, under Projects 4055212 and 6907743.
ETHICS STATEMENT
We take ethical considerations priority in this work. All datasets used strictly follow their of- ficial licenses and community standards, and we adhere to the ICLR Code of Ethics throughout our research. Our study does not involve any new human subject data collection, sensitive per- sonal information, practices that raise ethical concerns. Further discussion of social impact and compliance details can be found in Appendix A.3
REPRODUCIBILITT STATEMENT
We place high importance the reproducibility of our work. We provide an anonymous im- plementation of EgoHandICL, including training and evaluation scripts, at the following link:
https: pttpss://itae / com..ccom/i// http://wwccco///cc////////ee//e/ http://ccoous/////aaaann...
10

---

Published as a conference paper at ICLR 2026
REFERENCES
Seungryul Back, Kwang In Kim, and Tae-Kyun Kim. Pushing the envelope for rgb-based dense 3d hand pose estimation via neural rendering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 1067-1076, 2019.
Seungryul Back, Kwang In Kim, and Tae-Kyun Kim. Weakly-supervised domain adaptation via gan and mesh model for estimating 3d hand poses interacting objects. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, Pp. 6121-6131, 2020.
Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, Binyuan Hui, Luo Ji, Mei Li, Junyang Lin, Runji Lin, Dayiheng Liu, Gao Liu, Chengqiang Lu, Keming Lu, Jianxin Ma, Rui Men, Xingzhang Ren, Xuancheng Ren, Chuanqi Tan, Sinan Tan, Jianhong Tu, Peng Wang, Shijie Wang, Wei Wang. Shengguang Wu, Benfeng Xu, Jin Xu, An Yang, Hao Yang, Jian Yang, Shusheng Yang, Yang Yao, Bowen Yu, Hongyi Yuan, Zheng Yuan, Jianwei Zhang, Xingxuan Zhang. Yichang Zhang, Zhenru Zhang, Chang Zhou, Jingren Zhou, Xiaohuan Zhou, and Tianhang Zhu. Qwen technical report, 2023. URL https: /taps://aaxivv org abs
http://////339/// 2309. 11009.16666 16609
Amir Bar, Yossi Gandelsman, Trevor Darrell, Amir Globerson, and Alexei Efros. Visual prompting via image inpainting. Advances in Neural Information Processing Systems, 35:25005-25017, 2022.
Adnane Boukhayma, Rodrigo de Bem, and Philip HS Torr. 3d hand shape and pose from images in the wild. In Proceedings the IEEE/CVF conference on computer vision and pattern recognition, pp. 10843-10852 2019.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. Language models are few-shot learners. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information P Processing Systems, volume 33, pp. 1877-1901. Curran Associates, Inc., 2020.
Feng Chen, Ling Ding, Kanokphan Lertniphonphan, Jian Li, Kaer Huang, and Zhepeng Wang. Pcie egohandpose solution for egoexo4d hand pose challenge, 2024. /tRpstt/:///axii..o
abs/2406. 12219
Xingyu Chen, Yufeng Liu, Chongyang Ma, Jianlong Chang. Huayan Wang, Tian Chen, Xiaoyan Guo, Pengfei Wan, and Wen Zheng. Camera-space hand mesh recovery via semantic aggregation and adaptive 2d-ld registration. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 13274-13283, 2021.
Xingyu Chen, Yufeng Liu, Yajiao Dong, Xiong Zhang, Chongyang Ma, Yanmin Xiong, Yuan Zhang, and Xiaoyan Guo. Mobrecon: Mobile-friendly hand mesh reconstruction from monoeular image. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 20544 20554, 2022.
Hongsuk Choi, Gyeongsik Moon, and Kyoung Mu Lee. Pose2mesh: Graph convolutional network for 3d human pose and mesh recovery from 2d human pose. In European Conference on Com- puter Vision, pp. 769-787. Springer, 2020.
Ruikai Cui, Siyuan He, and Shi Qiu. Adaptive low rank adaptation of segment anything to salient object detection. arXiv preprint arXiv:2308.05422. 2023.
Ruikai Cui, Binzhu Xie, Shi Qiu, Jiawei Liu, Saeed Anwar, and Nick Barns.. Numgrad- pull: Numerical gradient guided tri-plane representation for surface reconstruction from point clouds. arXiv preprint arXiv:2411.17322 2024.
Haoye Dong, Aviral Chharia, Wenbo Gou, Francisco Vicente Carrasco, and Fernando D De la Torre. Hamba: Single-view 3d hand reconstruction with graph-guided bi-scanning mamba. Advances in Neural Information Processing Systems, 37:2127-2160 2024.

---

Published as a conference paper at ICLR 2026
Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Jingyuan Ma, Rui Li, Heming Xia, Jingjing Xu, Zhiyong Wu, Tianyu Liu, et al. A survey on in-context learning, arXiv preprint arXiv:2301.00234 2022.
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold. Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010./1929, 2020.
Zicong Fan, Adrian Spur, Muhammed Kocabas, Siyu Tang, Michael J Black, and Otmar Hilliges. Learning to disambiguate strongly interacting hands via probabilistic per-pixel part segmentation. In 202/ International Conference on 3D Vision (3DV), pp. 1-10. IEEE, 2021.
Zicong Fan, Omid Taheri, Dimitrios Tzionas, Muhammed Kocabas, Manuel Kaufmann, Michael J Black, and Otmar Hilliges. Arctie: dataset for dexterous bimanual hand-object manipulation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp- 2023.
Zhongbin Fang, Xiangtai Li, Xia Li, Joachim M Buhmann, Chen Change Loy, and Mengyuan Liu. Explore in-context learning for 3d point cloud understanding. Advances in Neural Information Processing Systems, 36:42382-422 -2392,,42252---- 2023.
Dyke Ferber, Georg Wölflein, Isabella C Wiest, Marta Ligero, Srividhya Sainath, Narmin Ghaf- fari Laleh, Omar SM EI Nahhas, Gustav Müiler-Franzes, Dirk Jäger, Daniel Truhn, et al. In- context learning enables multimodal large language models to classify cancer pathology images. Nature Communications, 15114 2024.
Ryo Fujii, Hideo Saito, and Ryo Hachiuma. Towards predicting any human trajectory in context. arXiv preprint arXiv:2506.008: 00871, 2025.
Liuhao Ge, Hui Liang, Junsong Yuan, and Daniel Thalmann. Robust 3d hand pose estimation in single depth images: from single-view cnn to multi-view cnns. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3593-3601, 2016.
Liuhao Ge, Zhou Ren, Yuncheng Li, Zehao Xue, Yingying Wang, Jianfei Cai, and Junsong Yuan. 3d hand shape and pose estimation from a single rgb image. Proceedings of the IEÉE/CVF conference on computer vision and pallern recognition, pp. 10833-10842, 2019.
Kristen Grauman, Andrew Westbury, Lorenzo Torresani, Kris Kitani, Jitendra Malik, Triantafyllos Afouras, Kumar Ashutosh, Vijay Baiyya, Siddhant Bansal, Bikram Boote, et al. Ego-exo4d: Understanding skilled human activity from first-and third-person perspectives. In Proceedings of the IEEF/CVF Conference on Computer Vision and Pattern Recognition, pp. 19383-19408 2024.
Ryosei Hara, Wataru Ikeda, Masashi Hatano, and Mariko Isogawa. Eventegobands: Event-based egocentric 3d hand mesh reconstruction. In IEEE International Conference on Image Processing (ICIP), pp. 1 1199-1204 2025. doi: 10. 109//CIP555113.22255 1084451.
Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick. Masked au- toencoders are scalable vision learners. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 16000-16009, 2022.
Jie Hong, Shi Qiu, Weihao Li, Saeed Anwar, Mehrtash Harandi, Nick Barnes, and Lars Petersson. Pointcam: Cut-and-mix for open-set point cloud learning. arXiv preprint arXiv:2212.02011, 2022.
Frank Hutter Ilya. Fixing weight decay regularization in adam, 2018. openreview net/ forumm 2
URL
https://
Umar Iqbal, Pavlo Molchanov, Thomas Breuel Juergen Gall, and Jan Kautz. Hand pose estimation via latent 2.5 d heatmap regression. In Proceedings of the European conference on computer vision (ECCV), pp. 118-134, 2018.
12

---

Published as a conference paper at ICLR 2026
Zheheng Jiang, Hossein Rahmani, Sue Black, and Bryan M Williams. A probabilistic attention model with occlusion-awart texture regression for hand reconstruction from single rgb im- age. In Proceedings the the IEEE/CVF conference computer vision and pattern recognition, pp- 758-767, 2023.
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. In European conference on computer vision, pp. 694-711. Springer, 2016.
Jeonghwan Kim, Mi-Gyeong Gwon, Hyunwoo Park, Hyukmin Kwon, Gi-Mun Um, and Wonjun Kim. Sampling matter: Soint-guided 3d human mesh reconstruction. Proceedings of the IEEE/CVF Conference computer vision and pattern recognition, pp. 12880-128888 2023.
Dominik Kulon, Haoyang Wang, Riza Alp Güler, Michael Bronstein, and Stefanos Zafeiriou. Single image 3d hand reconstruction with mesh convolutions. arXiv preprint I 1905.013250 2019.
Dominik Kulon, Riza Alp Guler, lasonas Kokkinos, Michael M Bronstein, and Stefanos Zafeiriou. Weakly-supervised mesh-convolee convoluutoll hand reconstruction in the wild. In Proceedings of the IEEF/CVF conference on computer vision and pattern recognition, pp. 4990-5000, 2020.
Taein Kwon, Bugra Tekin, Jan Stühmer, Federica Bogo, and Mare Pollefeys. H2o:
Two hands manipulating objects for first person interaction recognition. In Proceedings of the IEEE/CVF International Conference Computer Vision (ICCV), pp. 10138-10148, October 2021.
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li. Ziwei Liu, and Chunyuan Li. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408. 033226 2024a.
Feng Li, Qing Jiang, Hao Zhang, Tianhe Ren, Shilong Liu, Xueyan Zou, Huaizhe Xu, Hongyang Li, Jianwei Yang, Chunyuan Li, et Visual in-context prompting. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12861-12871, 2024b.
Kevin Lin, Lijuan Wang, and Zicheng Liu. End-to-end human pose and mesh reconstruction with transformers. In Proceedings of the IEEF/CVF conference on computer vision and pattern recog- nition, pp. 1954-1963, 2021.
Jiachang Liu, Dinghan Shen, Yizhe Zhang, Bill Dolan, Lawrence Carin, and Weizhu Chen. What makes good in-context examples for GPT-3? In Eneko Agirre, Marianna Apidianaki, and Ivan Vulié (eds.). Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pp. 100-114, Dublin, Ireland and Online, May 2022. Association for Computational Linguistics. doi: 10. 18653/v1/ 1022.deelio-11 1.10. URL
2022 deelio-1.
Mengyuan Liu, Xinshun Wang, Zhongbin Fang, Deheng Ye, Xia Li, Tao Tang, Songtao Wu, Xi- angtai Li, and -iunn Yang. Human-in-contextt: Unified cross-domain 3d human motion modeling via co-tentext learning, arXiv preprint arXiv:2508. 100::29908.0097, 2025.
Gyeongsik Moon and Kyoung Mu Lee. 121-meshnet: Image-to-lixel prediction network for accu- rate 3d human pose and mesh estimation from a single rgb image. In European Conference on Computer Vision pp. 752-768. Springer. 2020.
Gyeongsik Moon, Shoou-I Yu, He Wen, Takaaki Shiratori, and Kyoung Mu Lee, Interhand2, 6m: A dataset and baseline for 3d interacting hand pose estimation from a single rgb image. In European Conference on Computer Vision, pp. 548-564. Springer, 2020.
Yeonguk Oh, JoonKyu Park, Jaeha Kim, Gyeongsik Moon, and Kyoung Mu Lee. Recovering 3d hand mesh sequence from a single blurry image: A new dataset and temporal unfolding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 554- 563, 2023.
lason Oikonomidis, Nikolaos Kyriazis, and Antonis Argyros. Efficient model-based 3d tracking of hand articulations using kinect. In Procedings of the British Machine Vision Conference 2011, 2011.
13

---

Published as a conference paper at ICLR 2026
JoonKyu Park. Yeonguk Oh, Gycongsik Moon, Hongsuk Choi, and Kyoung Mu Lee. Handoccnet: Occlusion-robust 3d hand mesh estimation network. In Proceedings of the IEEE/CVF conference computer vision and pattern recognition, pp. 1496-1505, 2022.
Georgios Pavlakos, Dandan Shan, Ilija Radosavovic, Angjoo Kanazawa, David Fouhey, and Jitendra Malik. Reconstructing hands in 3d with transformers. In Proceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pp. 1826-9836, 2024.
Rolandos Alexandros Potamias, Stylianos Ploumpis. Stylianos Moschoglou, Vasileios Triantafyllou and Stefanos Zafeiriou. Handy; Towards a high fidelity 3d hand shape and appearance model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 4670-4680, 2023.
Rolandos Alexandros Potamias, Jinglei Zhang, Jiankang Deng, and Stefanos Zafeiriou. Wilor: End- to-end 3d hand localization and reconstruction in-the-wild. In Proceedings of the Computer Vision and Pattern Recognition Conference, pp. 12242-12254, 2025.
Aditya Prakash, Ruisen Tu, Matthew Chang, and Saurabh Gupta. 3d hand pose estimation in eryday egocentric images. In European Conference on Computer Vision, pp. 183-202. Springer, 2024.
Alec Radford, Jong Wook Kim, Chris Hallacy. Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Machine Learning, volume 139 of Proceedings of Machine Learming Research, pp. 8748-8763, PMLR, 24 Jul 2021. html
URL https: petts:://rrocedd.nng mlr wttpeaereeee//1soffrrorrooe
Grégory Rogez, Maryam Khademi, JS Supančič III, Jose Maria Martinez Montiel, and Deva Ra- manan. 3d hand pose detection in egocentric rgb-d images. In European Conference on Computer Vision, pp. 356-371. Springer, 2014.
Javier Romero, Dimitrios Tzionas, and Michael Black. Embodied hands: modeling and capturing hands and bodies together. ACM Transactions Graphics (TOG), 36(6):1-17, 2017.
Ohad Rubin, Jonathan Herzig, and Jonathan Berant. Learning to retrieve prompts for in-context learning. In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Proceedings of the 2022 Conference of the North American Chapter of the Association for Com- putational Linguisties: Human Language Technologies, pp. July 2022. Association for Computational Linguistics. doi: 12.18653/v122222.nmm mai.222...a--mmi11911111 URL https:://acl antho ogy. 202g22222..0m naac
Seattle, United States,
n. 1
Tomas Simon, Hanbyul Joo, lain Matthews, and Yaser Sheikh. Hand keypoint detection single images using multiview bootstrapping. In Proceedings of the IEEE conference Computer Vision and Pattern Recognition, pp. 1145-1153, 2017.
Srinath Sridhar, Franziska Mueller, Michael Zollhöfer, Dan Casas, Antti Oulasvirta, and Christian Theobalt. Real-time joint tracking of a hand manipulating an object from rgb-d input. In European conference on computer vision, pp. 294-310. Springer, 2016.
Xiao Sun, Yichen Wei, Shuang Liang, Xiaoou Tang, and Jian Sun. Cascaded hand pose regression. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 824-832, 2015.
Andrea Tagliasacchi, Matthias Schröder, Anastasia Tkach, Sofien Bouaziz, Mario Botsch, and Mark Pauly. Robust articulated-icc for real-time hand tracking. In Computer graphics forum, volume 34-5, pp. 101-114, 14. Wiley Online Library, 2015.
Jonathan Tompson, Murphy Stein, Yann Lecun, and Ken Perlin. Real-time continuous pose recovery of human hands using convolutional networks. ACM Transactions on Graphics (ToG), 33(5):1-
10, 2014.
14

---

Published as a conference paper at ICLR 2026
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural informa- tion processing systems, 30, 2017.
Xinlong Wang, Wen Wang, Yue Cao, Chunhua Shen, and Tiejun Huang. Images speak in images: A generalist painter for in-context visual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6830-6839, 2023.
Sang Michael Xie, Aditi Raghunathan, Percy Liang. and Tengyu Ma. An explanation of in-context learning implicit bayesian inference. arXiv preprint arXiv:2111.02080 2021.
Boshen Xu, Ziheng Wang, Yang Du, Zhinan Song, Sipeng Zheng, and Qin Jin. Do egocentric video- language models truly understand hand-object interactions? In The Thirteenth International Conference Learning Representations, 2025. httphhtppss//pppeeeeeeeee net/ forumm?
idd-8ggSFGkkn Gkn2
An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang. Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang. Le Yu, Mei Li, Mingfeng Xue, Pei Zhang, Qin Zhu, Rui Men, Runji Lin, Tianhao Li, Tingyu Xia, Xingzhang Ren, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yu Wan, Yuqiong Liu, Zeyu Cui, Zhenru Zhang, and Zihan Qiu. Qwen2.5 technical report. arXiv preprint 2IXi::242.115115. 2024.
Jingkang Yang. Shuai Liu, Hongming Guo, Yuhao Dong, Xiamengwei Zhang, Sicheng Zhang. Pengyun Wang, Zitang Zhou, Binzhu Xie, Ziyue Wang, Bei Ouyang, Zhengyu Lin. Marco Cominelli, Zhongang Cai, Bo Li, Yuanhan Zhang, Peiyuan Zhang, Fangzhou Hong, Joerg Wid- mer, Francesco Gringoli, Lei Yang, and Ziwei Liu. Egolife: Towards egocentric life assistant. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 088855289900 June 2025.
Fei Zhang, Pei Zhang. Baosong Yang, Fei Huang, Yanfeng Wang, and Ya Zhang. Context: Driving in-context learning for text removal and segmentation. http:://arri.oorgaa////550221122 2025a.
Jinglei Zhang, Jiankang Deng, Chao Ma, and Rolandos Alexandros Potamias. Hawor: World-space hand motion reconstruction from egocentric videos. In Proceedings of the Computer Vision and Patlern Recognition Conference, pp. 1805-1815, 2025b.
Yuanhan Zhang, Kaiyang Zhou, and Ziwei Liu. What makes good examples for visual in-context learning? In A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (eds.), Advances in Neural Information Processing Systems, volume 36, pp. 17773-17794, Curran As- sociates, Inc., 2023.
Ce Zheng, Xianpeng Liu, Guo-Jun Qi, and Chen Chen. Potter: Pooling attention transformer for efficient human mesh recovery. In Proceedings of the IEEE/CVF Conference Computer Vision and Puttern Recognition, 2023.
Junsheng Zhou, Jinsheng Wang, Baorui Ma. Yu-Shen Liu, Tiejun Huang, and Xinlong Wang. Uni3d: Exploring unified 3d representation scale. In International Conference on Learning Represen- tations (ICLR), 2024.
Runsong Zhu, Di Kang, Ka-Hei Hui, Yue Qian, Shi Qiu, Zhen Dong, Linchao Bao, Pheng- Ann Heng, and Chi-Wing Fu. Ssp: Semi-signed prioritized neural fitting for surface reconstruction from unoriented point clouds. In Proceedings of the IEEE/CVF Winter Conference Applica- tions of Computer Vision, pp. 3769-3778, 2024.
Yongshuo Zong, Ondrej Bohdal, and Timothy Hospedales. VI-icl bench: The devil in the details of multimodal in-context learning. arXiv preprint arXiv:2403. 13164, 2024.
15

---

Published as a conference paper at ICLR 2026
APPENDIX
A DISCUSSIONS
A.1 LIMITATIONS
While EgoHandICL achieves strong performance on egocentric 3D hand reconstruction, it has sev- eral limitations. First, the reliance on VLM-based retrieval introduces non-trivial computational overhead, which limits real-time deployment on resource-constrained devices. Second, the frame- work depends high-quality template databases, and retrieval performance may degrade if tem- plates lack diversity. Third, egocentric datasets with complete annotations remain limited. For example, the widely used EgoExo4D dataset provides only keypoint-level ground truth for hand pose estimation, lacking full MANO parameter supervision as available in ARCTIC. Relying solely on keypoint annotations restricts model generalization to complex real-world scenarios. Moreover, most current evaluations are performed with ground-truth bounding boxes, while robust hand detec- tion itself is a major challenge in egocentric views. The absence of fair end-to-end benchmark that jointly evaluates detection and reconstruction limits progress toward more realistic deployments.
A.2 FUTURE WORK
Building on the identified limitations, several promising directions emerge for future work. First, to mitigate the computational overhead of VL.M-based retrieval, we plan explore more effi cient retrieval mechanisms, such as lightweight VIMSs, approximate nearest-neighbor search, or retrieval-free strategies that embed contextual exemplars directly into the learning pipeline. Sec- ond, addressing the limited availability of fully annotated egocentric datasets, emphasize the importance of constructing comprehensive 3D egocentric benchmarks with consistent annotations of hand pose, object pose, and body pose. Such datasets would enable more holistic evaluation of hand-object-bodf interactions and facilitate robust generalization in real-world scenarios. Third, to move beyond evaluations that rely on ground-truth bounding boxes, we advocate for the construction of fair end-to-end benchmarks that jointly assess detection and reconstruction in egocentric settings, better reflecting real deployment challenges. Finally, we will explore temporal extensions of Ego- HandICL for continuous 3D hand tracking and broaden its scope to encompass richer egocentric tasks, such as joint aaje-o reconstruction and gaze-conditioned interaction modeling, with particular emphasis on lightweight designs suitable for AR/VR applications.
A.3 SOCIAL IMPACT
Egocentric 3D hand reconstruction has promising applications in extended reality, assistive tech- nologies, and robotics. However, also introduces potential risks. First, continuous capture of ego- centrie data may raise privacy concerns, particularly regarding bystanders sensitive environments. Second, the reconstructed hand data could be misused in surveillance or unauthorized tracking ap- plications. To mitigate these risks, we emphasize that all datasets used in this work comply with their official licenses and community standards, and we strictly adhere to ethical guidelines throughout data usage and research practices.
A.4 LLM USAGE
Large language models (LLMs) are used moderately polish the paper writing. Specifically, LLMs are employed for improving grammar and formatting consistency of LaTeeX content.
B MORE DETAILS OF EGOHANDICL
B. PRE-DEFINED VISUAL TEMPLATES
We first introduce the construction of Pre-defined Visual Templates, which categorize egocentric images into four canonical hand- involvement types: left-hand, right-hand, two-hand, and non-hand involvement.

---

Published as a conference paper at ICLR 2026
Prompt Design. To enable VLM to automatically categorize each image, design explicit textual prompts describing the four indolenvovvemntt types. Our prompt is:
SYSTEM_PROMPT
You are an image understanding agent. Your task is to analyze a first-person perspective image and classify the interaction status of the left and right hands.
Classi fication rules:
0: Only the left hand is interacting with an object
Only the right hand is interacting with object
Both hands are interacting with an object
3: Neither hand is interacting with any object
Important notes:
Occlusion caused by objects must be considered determining whether a hand is interacting.
Use your best reasoning based on the visual content to make this decision.
Your reaponse must be single number: one of 10, 1, 2, -1). Do not include any explanation or additiona text.
USER PROMPT .*.
Please analyze the first-person perspective image and determine the interaction status of the hands.
Return only the correct label number based on the following:
Only the left hand is interacting
1: Only the right hand is interacting
2: Both hands are interacting
3:
Neither hand is interacting
This prompt is fixed across both training and inference to ensure consistency.
Data Distribution. Based on the above categorization, we partition the dataset into the four hand- involvement subsets. The overall distribution of samples across categories is summarized in Tab. This categorization allows us balance template retrieval across common egocentric scenarios.
B.1
Table B.1: Data distribution of pre-defined visual templates. Each hand-involvement type is split into training, validation, and testing sets.
Hand Involvement Type
Train
Val
Test
Total
Left Hand
7,991
2,283
1,143
11,417
Right Hand
14,334
4,095
2,049
20,478
Both Hand
92,917
26,547
13,275
132,739
None Hand
2,996
856
428
4,280
Total
118,238
33,781
16,895
168,914
Training Strategy. During training-time data preprocessing, we perform the template-retrieval procedure three times for every candidate training sample. This repetition is used to verify the correctness of hund-involvement classification and ensure that each exemplar is assigned to the proper category. Only samples with consistent predictions across all three runs are kept in the exemplar pool, After this offline verification, the training process itself uses random sampling within
2

---

Published as a conference paper at ICLR 2026
each validated subset to construct the exemplar set for each query image at every iteration. This stochastic sampling maintains exemplar diversity and mitigates overfitting.
Inference Protocol. At inference time, we adopt the same VLM prompt to categorize the query image and retrieve templates from the training set accordingly.
B.2 ADAPTIVE TEXTUAL TEMPLATES
Prompt Design. We design two families of prompts with different semantic emphases:
Description-style prompts (Des. Prompts): These provide explicit cues about the egocentric scene, encouraging the VLM to describe visible details. Our Prompts are:
Please analyze the egocentric image and return concise sentence describing the visible hands and their interactions with surrounding objects for 30 hand reconstruction The description should specify:
Which aand are present (Left hand, right hand, or both).
2. The type of interaction (e.g., grasping, holding, touching, pointing, resting).
3. The object involved, if any (e.g., cup, phone, keyboard), If hand is clearly visible, explicitly state "no hand involvement."
Examples:
"Left hand grasping cup."
"Right hand pointing at phone."
"Both hands holding a book."
"No hand
The prompts is especially effective when the hand is clearly visible, helping the VLM retrieve templates with matching appearance and interaction patterns.
Reasoning-style prompts (Reas. Prompts): These encourage the VLM to infer strategies for handling challenging cases. Our Prompts are:
SYSTEM PROMPT SYSTEM_PROMPT
You are a reasoning agent specialized in egocentric hand-object interaction understanding. Your task is to analyze images captured from a first-person perspective and generate caption that describes all hand and object interaction dctails relevant to 3D hand reconstruction.
Given single input image, output one concise sentence that speci fies which hand (s) (Left/right/both) are visible, their pose, and their interaction with any objects. Be precise and descriptive about the hand-object interaction without adding explanatiens or speculations.
USER_PROMPH m""
Please analyze the image and zeturn one senterce describing the hands and their interactions with objects for hand recenstruction (e.g., left hand grasping cup, right hand resting the table).
unm
The prompt is mostly useful for heavily occluded or ambiguous views, where direct visual cues are insufficient and deeper contextual reasoning required.

---

Published as a conference paper at ICLR 2026
Query Image with its tempaite
Hand-object Involvement Type:
Left
Right
Description Prompts:
Left hand holding a toy slide, right hand open and hovering nearby.
Right hands holding scissors and cutting paper on a wooden box.
Reasoning Prompts:
Left hand grasping curved wooden toy with pastel-ce colored accents, right hand open and slightly raised above the table.
Right hand gripping black scissors with blades positioned over wooden box, thumb and fingers engaged in cutting motion
Hand-object Involvement Type:
Left
Right
Right hand holding black scissors above a white surface with a wooden box nearby.
Description Prompts:
Left hand touching lid circular container.
Left hand grasping light blue and white toy structure with wooden base, right hand open and positioned near the object if preparing assist stabilize.
Right hand gripping black scissors with thumb through upper loop and fingers through lower loog, blades open angled downward; left hand partially visible at bottom left.
Reasoning Prompts:
Figure B.1: Query-Template Example
B.3 EXAMPLES.
In Fig. B.I and Fig. B.2 we illustrate examples of the two template retrieval strategies. During both training and inference, the retrieved images and the query image can serve as interchangeable query-ter query-template pairs, enabling flexible construction of in-context demonstrations.
C MORE DETAILS OF EXPERIMENTS
Evaluation Settings. All baselines are fine-tuned on the same training splits, without incorpo- rating any external data, ensuring fully fair comparison. In the General setting, evaluation is performed on all hands detected within each image. However, egocentric scenarios often involve challenging cases such partially invisible hands, severe self-occlusions, hands from different persons. Under such conditions, different baseline models occasionally fail to detect all hands, leading to small proportion of missing detections. Across our benchmarks, we observe that the proportion of such missing cases remains below 5%, and therefore the reported results in the general setting are still representative of overall performance.
multiple interacting
Bimanual setting. To provide stricter and fairer comparison, further adopt the Bimanual setting, where only samples with both hands correctly detected by each evaluated method are con- sidered. This ensures that reconstruction accuracy is compared under consistent detection condi-

---

Published as a conference paper at ICLR 2026
Query Image with its tempalte
Hand-object Involvement Type:
Both
None
Description Prompts:
Both hands lifting and tilting a wooden box.
Both hands resting near circular device white surface.
Both hands gripping light wooden box hand supporting the bottom edge, right hand holding the corner with fingers curled along the side.
Both hands hovering above the table with fingers slightly curled and marked for tracking, not direct contact with the circular white object.
Reasoning Prompts:
Hand-object Involvement Type:
Both
None
Description Prompts:
Both hands holding a wooden board.
Two hands suspended the table.
Reasoning Prompts:
Both hands holding light wooden plank left hand supporting the bottom edge with fingers extended, right hand gripping the top corner with thumb front and fingers curled underneath.
Both hands are poised just above tabletop, lightly elevated with relaxed, curved fingers, while a foldable phone atop small wooden box the scene.
Figure B.2: Query-Template Example 2.

---

Published as a conference paper at ICLR 2026
tions across models, thereby eliminating potential biases caused by unequal detection performance. The resulting bimanual test set contains 12,041 and 3,802 samples on ARCTIC and EgoExo4D respectively, providing a robust basis for evaluating hand-to-ham spatial consistency and overall reconstruction reliability in dual-hand interactions.
Performance across hand-involvement categories. To better understand the behavior of differ- ent models under varying egocentric conditions, we further evaluate them separately on the four pre-defined sand-involvement types: left-hand, right-hand, two-hand, and no-hand. This break- down allows us to quantify how well each model generalizes to different interaction settings, and to analyze whether certain models are more prone performance degradation in specific categories.
Table C.I: Performance of Model Zoos across different hand-involvement types.
Type
Left Hand
Hand
Hands
Non Hand
Method
P-MPIPE]
P-MPVPE]
P-MPIPE
P-MPV
P-MPJPE]
P-MPYPHL
P-MPIPK:
P-MPPPE
HaMeR Pavllkos al. 2024
10.5
10.2
10.1
9.7
10.0
9.6
7,6
WiLaR Pryzmmiss of 2025
5.6
5.5
5.2
5.1
5.5
5.4
5.1
5.0
WildHard (1Ta
HaWoR Zhing 01 20238
6.7
6.1
5.9
53
5.0
Average
7.2
6.9
6.8
6.5
6.5
6.3
5,7
5,5
As shown in Table C.I the reconstruction errors vary across the four hand-involvement types. We observe that left-hand cases exhibit slightly higher errors (7.2/6.9) compared right-hand cases (6.8/6.5). This asymmetry is consistent with prior observations that egocentric datasets are often right-hand dominant (e.g., ARCTIC (Fan al., 2023). EgoExo4D (Chen et al. 2024), ), leading to stronger representations for the right hand. Interestingly. two-hand scenarios achieve lower errors (6.5/6.3) than single-hand cases, suggesting that the presence both hands provides additional ge- ometric constraints that help the model resolve occlusions and perspective ambiguities. This trend contrasts with conventional baselines such as HaMeR (Pavlakos et al.) (022)) and WiLoR (Potamias) al. 2025), which typically degrade in bimanual settings due to severe inter-hand occlusions. Fi- nally, the lowest errors are obtained in the no-hand category (5.7/5.5), demonstrating the model's robustness in avoiding false positives when hand is present.
D RETRIEVAL QUALITY AND OOD ANALYSIS.
Performance improvement Clp Smilarity
Improvement vs Similarity
Dute
3.55
0.70
650
3:55
0dt
0.55
0.70
p Semilenity
Cip Simirrtyy
(a) Image Similarity Arctic Test Dataset
(b) Image Similarity on H2O Test Dataset for OOD Analysis
Figure D.1: Retrieval Quality and OOD Analysis.
D. RETRIEVAL QUALITY ANALYSIS
On the ARCTIC dataset, we assess the quality of the retrieved templates by computing the cosine similarity between the CLIP (Radford et al., 202I) image features of the query and its exemplar. We further correlate this similarity with the performance gain over the strongest baseline WiLoR (in P-MPVPE). As shown in Figure ()), over 90% of retrieved exemplars achieve similarity above 0.7, indicating that our retrieval stage provides nigh-quality contextual matches. Moreover, higher similarity consistently leads to larger improvements over WiLoR, confirming that EgoHandICL ef- fectively leverages high-quality exemplars for contextual refinement of 3D hand reconstruction.

---

Published as a conference paper at ICLR 2026
D.2 OUT-OF-DISTRIBUTTI (OOD) ANALYSIS
We further analyze EgoHandICL under OOD conditions on the H2O dataset (Kwon et al.) 2021) where retrieved templates less aligned with the query image due distribution shift. As shown in Figure D.I (b), the cosine similarity between the query and its retrieved exemplar is predomi- nantly concentrated in the range [0.6, 0.7), reflecting the difficulty of retrieval in OOD scenarios. Correspondingly, the absolute improvement over WiLoR is smaller than in the in-distribution case Figure D.I (a), EgoHandICL still consistently outperforms WiLoR across the observed similar- ity range, demonstrating robust generalization of our exemplar-driven ICL framework even when template quality is degraded.
E
FAILURE CASE
Template (Retrieval Quality
Template (Retrieval Quality
Template Image Quality 0.67)
Template Image (Retrieval Quality 0.60)
Figure E.1: Failure Cases. Failure cases under low-quality retrieval.
As shown in Figure E. EgoHandICL may fail when the retrieved exemplar of low quality. Such low-quality retrieval typically arises under challenging visual conditions (e.g., severe occlusion, mo- tion blur, non-typical hand-object configurations), where the visual evidence becomes insufficient for accurate matching. In these scenarios, the retrieved exemplar offers weak contextual cues, lead- ing to degraded reconstruction quality. Also, these are very challenging cases that existing prior methods can hardly handle.


Reviews:
[{"review_id": "slWzC6c5M7", "reviewer": "Reviewer_bCSC", "summary": "This paper proposes EgoHandICL, an in-context learning (ICL)-based framework for 3D hand reconstruction in egocentric visual settings. It focuses on handling unseen contexts. Especially, it reformulates 3D hand reconstruction as exemplar retrieval for contextual adaptation and multimodal (image, MANO parameter, text ) token fusion based on MAE-style VLM for robust representation learning. Experiments on ARCTIC and EgoExo4D benchmarks show significant improvements over previous works (HaMeR, WiLoR, WildHand, and HaWoR). The paper further explores downstream integration with EgoVLMs, showing that reconstructed hand cues enhance hand–object interaction reasoning.", "questions": "See Weaknesses.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "- The proposed method is an effective method applying in‑context learning (ICL) in 3D hand reconstruction. The retrieval and then multimodal learning strategy is sound and provides an interesting direction to contextual adaptation when interpreting complex egocentric scenes. \n\n- The paper is well structured and easy to follow. \n\n- Quantitative results on ARCTIC and EgoExo4D clearly demonstrate the advantage of EgoHandICL, including bimanual and occlusion-heavy cases. Ablations cover mask ratios, loss combinations, backbone variations, and prompting strategies, substantiating design choices. Integration with EgoVLMs (Tab.8, Fig.5) shows meaningful downstream benefits beyond reconstruction itself.", "weaknesses": "1. The retrieval is confusing and not clear, especially the definition of template/visual images. Based on \"few shot\" (L.154) and Fig.2, it seems that the template images are from the same dataset or even the frames of the same video clip. In this case, it seems the retrieval is not necessary and using template images is easier. Moreover, it would be better to discuss the influence if the retrieval/prompt is not good enough.\n\n2. The proposed method takes many pretrained/foundational models. Eq.2 requires coarse MANO parameters from HaMeR or WiLoR, Eq.4 requires the pretrained Vit (WiLoR, L.212 and L.278), Eq.5 requires the pretrained 3D feature encoder Uni3D-ti (L.269 and L.281). Moreover, the backbone is Qwen2.5-VL-72B-Instruct and Qwen-7B (L.278). This raises concerns about efficiency. It would be better to report the latency, parameters, inference time (ms), FLOPs and GPU memory compared to other sotas. Moreover, it would be better to report the efficiency of each step in the proposed method.\n\n3. The training details are not clear. It would be better to explicitly state the dataset used for training (Eq.4). In addition, I am not sure if the comparison is fair. For Tabs.1 and 2, are ARCTIC and EgoExo4D used during finetuning for all sotas? It would be better to show the training dataset of all methods.\n\n4. More ablations are needed. The paper mentions a cross-attention fusion among visual, text, and MANO tokens, but does not ablate their individual contributions. It would be better to conduct an ablation study to evaluate performance without each modality (Image-only, Image+Text, Image+Structure) and quantify the benefit of multimodal fusion. Moreover, it would be better to show the performance after removing/reducing quality such as template, text, $M_{qry}$, $M_{tpl}$. \n\n5. The details of data preprocessing and hand-crafted prompts are not clear. This makes it difficult to reproduce the results faithfully.\n\n6. While the experiments show that ICL improves occlusion reasoning, the paper lacks analysis explaining why ICL achieves this. A qualitative visualization or attention-weight inspection between exemplar templates and query tokens could help substantiate the claimed “context reasoning” mechanism.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "tMkml4osme", "reviewer": "Reviewer_CWKC", "summary": "Authors proposed a solution to 3D hand reconstruction from egocentric RGB images, which is challenging due to depth ambiguity, severe self-occlusion, and complicated hand-object interactions. The core idea is to bring in-context learning into the problem. The method first retrieves a template egocentric image that is semantically and visually similar, using VLMs. Then, for that template, authors construct an exemplar pair: coarse MANO estimate from a pretrained reconstructor and ground-truth MANO parameters for that template. They do the same for the query frame. The model is then conditioned on the paired examples to refine the query’s coarse MANO estimate into a final high-quality 3D hand mesh prediction, leveraging retrieved context without explicit fine-tuning at test time. Authors evaluate the method on ARCTIC and EgoExo4D datasets, and show that their reconstructions improve fine-grained hand-object reasoning.", "questions": "It might be useful to report how large the exemplar pool is. \nPlease provide failure cases, especially for the retrieval stage.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "Clear motivation: egocentric hand reconstruction is challenging. The paper successfully pointed out and tackled the challenges  in the problem and provided sound solutions.\n\nNovel and sound idea: The retrieval and in-context learning are novel and sound approaches. Instead of simply training a bigger network or adding more auxiliary cues, authors explicitly retrieve semantically similar examples and feed them as context to guide inference. Especially, instead of simply retrieving raw RGB crops, authors proposed to align coarse prediction and GT pairs and let the model learn how to correct errors in the query. This seems an interesting and effective idea. Multimodal tokenizer also looks effective.\n\nStrong empirical performance: Authors showed that injecting the reconstructed hands into the egocentric VLMs improves its ability to reason about fine-grained HOIs. This looks like a good downstream example.", "weaknesses": "The method may depend on the quality of retrieval: Even though the overall performance might depend on the quality of template, the paper does not quantify the success/failure of the retrieval stage. It might be better to include such results.\n\nMore careful analysis required to validate the effectiveness: Deep learning models are frequently suffering from out-of-distribution (OOD) samples, which denote testing samples far from training samples which are exploited during training. Using the retrieval scheme, it might be even hard to secure similar samples for all the testing samples in the limited database, which may worsen the overall accuracy. It might be necessary to analyze the relationship between average image similarity during testing and overall accuracy.\n\nEfficiency issue: Since the method is based on VLMs, it might be hard to deploy the algorithm in the real-time manner.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "oTXVw2rfom", "reviewer": "Reviewer_Pnsi", "summary": "This paper tackles 3D hand reconstruction from egocentric views by applying in-context learning (ICL) paradigm, named EgoHandICL. Given a query image, a Vision-Language Model (VLM) is prompted to retrieve a single, contextually relevant template image from a database. The core EgoHandICL model, a Masked Autoencoder (MAE)-based Transformer, receives multimodal tokens (representing image features, textual descriptions, and structural MANO parameters) for four components: the template input (a coarse 3D estimate), the template target (its ground truth), and the query input (its coarse 3D estimate) . The model is trained to predict the query target. Experiments on the ARCTIC and EgoExo4D benchmarks show that EgoHandICL significantly outperforms state-of-the-art methods like HaMeR and WiLoR. Furthermore, the ablation studies, regarding hand types, prompt designs, masking, and losses, confirm the effectiveness of the proposed module of the method.", "questions": "In addition to the Weaknesses section, please clarify points below.\n- Is the MANO encoder and decoder architecture a simple, continuous autoencoder? Or does it implement a discrete latent variable, similar to a VQ-VAE?\n- Need the details for visual prompts in EgoVLMs: How are the hand reconstructions used as visual prompts for EgoVLMs? Does this involve simply overlaying the predicted 3D hand mesh sequence onto the input video, or were other prompt variants tested?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 4, "strengths": "- Novelty: The application of in-context learning to 3D hand reconstruction is highly novel. On top of the state-of-the-art pose estimators, it provides dynamic, example-based reasoning at inference time. Using a VLM for semantic retrieval (e.g., finding similar interactions or occlusion types ) rather than just visual similarity is a powerful and unique idea for this problem.\n- Technical soundness: The method is well-formulated. Refining a corse prediction with exemplar mano pair is a logical and effective approach. Plus, the multimodal ICL tokenizer and MAE-style learning are clearly aligned to this problem and seem to be a strong prior for ICL in visual and 3D hand parameter domains. \n- Strong experimental support: The results are state-of-the-art across the board. The improvements are particularly large in highly occluded scenarios. Thorough ablation studies also strengthen the paper's argument.", "weaknesses": "Further clarifications on the following points would be appreciated. \n- Inference cost trade-off: The proposed method introduces significant computational overhead (VLM text generation, template retrieval, multimodal tokenization, and MAE transformer inference) beyond the base regressor. An estimation of this additional burden, using metrics like FPS or FLOPs, is needed to provide a comprehensive view of the test-time bottleneck for potential real-time applications.\n- Impact of template size (N): The framework is evaluated using only a single template (N=1). Drawing an analogy from LLMs, an ablation study on using additional context (N>1) would be an insightful exploration to understand how context quantity benefits this task.\n- Missing failure analysis: A discussion on the effect of noisy or poorly-matched exemplars is needed. For instance, out-of-distribution (non-typical) test samples may retrieve poor exemplars due to a lack of similar data. It would be insightful to know if this situation worsens the reconstruction performance compared to the baseline, which would clarify the framework's robustness.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "UXgfVpzW3D", "reviewer": "Reviewer_EXnq", "summary": "This paper proposes the first in-context learning framework (EgoHandICL) for 3D hand reconstruction under challenging egocentric conditions. EgoHandICL consists of a VLM-based exemplar retrieval strategy, a multi-modal feature tokenizer, and a Masked Autoencoders (MAE)-based architecture for hand mesh reconstruction. The proposed framework achieves significant performance improvement over state-of-the-art 3D hand reconstruction models on two public benchmarks of ARCTIC and EgoExo4D.", "questions": "The questions are listed above in the Weakness part. Overall, the paper proposes a novel method and the experiments are extensive and convincing. My positive rating to this paper would be solidified if the above concerns are addressed.", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1) The in-context learning framework for 3D hand construction is novel. It augments existing hand reconstruction models from an interesting perspective. The motivation is clear, and the method design based on VLM and masked autoencoder is also insightful.  \n2) Comprehensive experiments are conducted to analyze the performance of the proposed framework which significantly improve the performance over SOTA methods. \n2) The paper is well presented and it is easy to follow the story.", "weaknesses": "1) Overall, the paper is clearly written. However, several technical details are still missing.   \na) For the ICL tokenizer part, it is not clear how each set of the ICL tokens are composed of (e.g., number of tokens, dimension of token embedding).   \nb) The detailed architecture of MANO encoder/decoder and Transformer-based reconstruction model are not given.  \nc) In Section 4.2, it is claimed that \"Our default setting balances this trade-off by applying the reasoning prompts under heavy occlusion and the description prompts in clearer scenarios\". However, it is unclear about the implementation details, such as how to quantify occlusion or scene clearness.  \n2) Several experiments are missing, which are important to demonstrate the overall performance of the proposed framework:  \na) The ablation study for the ICL tokenizer (especially the cross-attention to fuse multi-modal tokens) is not given.  \nb) In Section 3.3, the loss fuctions require ground-truth MANO parameters as well as 3D mesh/vertex. However, such information are not provided in the EgoExo4D dataset. Then, how the proposed method is trained on EgoExo4D?  \nc) It is also important to show the runtime evaluation. In particular, as explained in Appendix B.1, the retrieval is repeated 3 times for visual template. Considering the large size of training data, the template retrieval would be very time-consuming.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)