GenCape: Structure-Inductive Generative Modeling for Category-Agnostic Pose Estimation
Authors: Jiyong Rao
Yu Wang
Shengjie Zhao
Venue: ICLR 2026 Poster (2026)
Keywords: Category-Agnostic Pose Estimation
Graph Learning
Variational Autoencoder
Primary area: applications to computer vision, audio, language, and other modalities

Abstract: Category-agnostic pose estimation (CAPE) aims to localize keypoints on query images from arbitrary categories, using only a few annotated support examples for guidance. Recent approaches either treat keypoints as isolated entities or rely on manually defined skeleton priors, which are costly to annotate and inherently inflexible across diverse categories. Such oversimplification limits the model’s capacity to capture instance-wise structural cues critical for accurate pixel-level localization. To overcome these limitations, we propose \textbf{GenCape}, a \textbf{Gen}erative-based framework for \textbf{CAPE} that infers keypoint relationships solely from image-based support inputs, without additional textual descriptions or predefined skeletons. Our framework consists of two principal components: an iterative Structure-aware Variational Autoencoder (i-SVAE) and a Compositional Graph Transfer (CGT) module. The former infers soft, instance-specific adjacency matrices from support features through variational inference, embedded layer-wise into the Graph Transformer Decoder for progressive structural priors refinement. The latter adaptively aggregates multiple latent graphs into a query-aware structure via Bayesian fusion and attention-based reweighting, enhancing resilience to visual uncertainty and support-induced bias. This structure-aware design facilitates effective message propagation among keypoints and promotes semantic alignment across object categories with diverse keypoint topologies. Experimental results on the MP-100 dataset show that our method achieves substantial gains over graph-support baselines under both 1- and 5-shot settings, while maintaining competitive performance against text-support counterparts.

Published as a conference paper at ICLR 2026
GENCAPE: STRUCTURE-INDUCTIVV GENERATIVE MODELING FOR CATEGORY-AGNOSTIC POSE ESTIMA-
TION
Jiyong Rao', Yu Wang" Shengjie Zhao
School of Computer Science and Technology, Tongji University
ABSTRACT
Category-agnostic pose estimation (CAPE) aims to localize keypoints on query images from arbitrary categories, using only few annotated support examples for guidance. Recent approaches either treat keypoints as isolated entities or rely on manually defined skeleton priors, which are costly to annotate and in- herently inflexible across diverse categories. Such oversimplification limits the model's capacity to capture instance-wise structural cues critical for accurate pixel-Ievel localization. To overcome these limitations, we propose GenCape, Generative-based framework for CAPE that infers keypoint relationships solely from image- based support inputs, without additional textual descriptions or prede- fined skeletons. Our framework consists of two principal components: an iterative Structure-aware Variational Autoencoder (i-SVAE) and Compositional Graph Transfer (CGT) module. The former infers soft, instance-specific adjacency ma- trices from support features through variational inference, embedded layer-wise into the Graph Transformer Decoder for progressive structural priors refinement. The latter adaptively aggregates multiple latent graphs into query-aware struc- ture via Bayesian fusion and attention- based reweighting, enhancing resilience visual uncertainty and support-inducee bias. This structure-ane awatre design fa- cilitates effective message propagation among keypoints and promotes semantic alignment across object categories with diverse keypoint topologies. Experimen- tal results on the MP- 100 dataset show that our method achieves substantial gains over graph-support baselines under both 1- and 5-shot settings, while maintaining competitive performance against text-support counterparts.
INTRODUCTION
Category-Agnostee Pose Estimation (CAPE) Xuet al. (022a); : Shi et al. (2023): Hirschorn & Avidan (024); Rusanovsky et (2025): Ren et al. (2024); Chen et al. (2025a) has emerged as funda-
mental yet challenging task in computer vision, aiming to localize semantic keypoints on arbitrary object categories using only handful of annotated support samples. Unlike conventional 2D pose estimation task (Sun et al. (2019); Xu et al. (2022b); Yuan et al. (2021): Rao et al. (2025), which depends heavily on predefined templates or class-specific priors, CAPE requires robust general- ization across semantically diverse and structurally heterogeneous object classes. This capability extends the applicability of pose estimation from closed-worke scenarios to open-world scenarios, enabling scalable deployment in domains such human motion analysis Zheng al. (2023); Yang ctt al. (2023), cross-species behavior understanding Ye al. (2024); (Stofft al. (2024). and robotic manipulation Zheng al. (2025); ); Ma et al. (2024) dynamic environments.
In the CAPE paradigm, the objective is to estimate keypoints for objects from novel categories, conditioned on few annotated support images. Despite recent advancements, existing CAPE ap- proaches are hindered by two critical limitations. On the one hand, many existing approaches either treat keypoints isolated semantic entities, neglecting the latent spatial dependencies essential for accurate pixel-level localization, or rely heavily on external priors such as manually pre-defined skeleton connections or auxiliary textual descriptions. These external priors not only incur high
"Corresponding author; yuwangt j@yeah net and shengji ezhan@tonggii .ddu.cn

---

Published as a conference paper at ICLR 2026
annotation overhead but also restrict the model's adaptability to novel instances with large pose variations, non-rigid deformation, or diverse structural characteristics.
On the other hand, the stochastic nature of support sets selection in few-shot setting makes CAPE methods particularly vulnerable to the low quality support examples. real-world scenarios, sup- port images may contain severe occlusions, incomplete annotations, structural discrepancies relative to the query, which can misguide structural inference and significantly impair prediction accuracy and generalization. Together, these limitations underscore the need for more flexible, data-driven approach to structure modeling and robust support-query adaptation.
To address the limitations of fixed priors and structural rigidity in CAPE, propose GenCape, generative latent structure learning framework that automatically infers keypoint relationships, rep- resented as latent adjacency matrices, exclusively from image-based support inputs, without any external priors, as illustrated in Figure [ At its core of, GenCape integrates two complementary components: an iterative Structure-aware Variational Autoencoder (i-SVAE) and a Compositional Graph Transfer (CGT) module. Specifically, the i-SVAE leverages variational inference to learn a distribution over snstance-specific graph structures, iteratively generating and refining latent ad- jacency matrices that serve as flexible and data-driven structural priors. Compared to the recently related deterministic approach SDPNet Ren et al. (2024). this generative formulation captures the epistemic uncertainty in sparse and ambiguous support signals, allowing for more expressive and robust message passing within the Graph Transformer Decoder. The progressive refinement enables the model propagate contextual cues across spatially correlated keypoints and adapt to complex object configurations. In parallel, the CGT module dynamically aggregates multiple latent graph hypotheses into a mnery-conditione representation through a principled Bayesian fusion and an attention-based re-weighting strategy. This dynamic compositional mechanism explicitly accounts for support-query query inconsistencies and mitigates the adverse impact of noisy or misleading support examples caused by occlusion, deformation, or pose variation. Together, these modules enhance structural generalization and resilience to support noise, setting a new direction for few-shot key- point reasoning under the CAPE paradigm. Remarkably, our approach surpasses the performance of cxisting CAPE methods, showcasing a new state-of-the-ar performance.
In summary, contributions are as follows:
We introduce GenCape, a novel generative framework for category-agnostic pose estima- tion, which incorporates iterative Structure-aware Variational Autoencoder (i-SVAE) to infer latent, instance-specific skeletons solely from image-based support sets, eliminating the need for predefined anatomical priors or textual descriptions.
We propose Compositional Graph Transfer (CGT) mechanism that dynamically ag- gregates multiple structural hypotheses into a unified, query-conding query-conditioned graph through attention-guided fusion, significantly enhancing robustness under ambiguous, noisy, or structurally mismatched support scenarios.
Our framework achieves new state-of-the-art results on the representative and challenging MP-100 benchmark under both 1-shot and 5-shot settings, surpassing existing methods by a substantial margin +1.59% mPCK averaged across evaluation thresholds, without relying any external structural or textual annotations.
RELATED WORK
2.1 CATRGORY-AGNON POSE ESTIMATION
Category-agnostee pose estimation (CAPE) Xu et al. (2022a) has emerged as a compelling general-
ization of conventional pose estimation Sun et al. (2019); Yu et al. (202T): ; Xu et al. (2022b 2025);
Rao et al. (2025) by localizing keypoints for arbitrary category objects with only few annotated support images. The pioneering POMNet Xu et al. (2022a) employed a metric-learning paradigm to match support and query features in latent space, while CapeFormer Shi et al. (2023). two-stage refinement framework that first produces initial keypoint proposals and then refines their positions in second stage. Recent efforts Hirschom & Avidan (0024); Liang al. (2024) further extended this paradigm by integrating fixed skeleton priors into graph reasoning modules to capture latent keypoint relations. However, it remains limited by the rigidity of manually defined structures, which
2

---

Published as a conference paper at ICLR 2026
Skeleton Graph Support
Support Sets
Bel baaney
Keypoint Identifier Support
Text Graph Support
marida
CAPE Model
Support Sets
Query
Prediction
CAPE Model
Query
Prediction
Figure 1: Left: Existing CAPE frameworks rely additional structured priors within the support set, such as @ keypoint identifiers, 2 fixed skeleton graphs, or textual descriptions with skeleton graphs, to enhance structural reasoning. Right: In contrast, our framework directly infers latent keypoint relationships solely from support images, learning instance-specific adjacency matrices without relying on handcrafted priors.
constrains adaptability to novel categories with topological variation. WeakShot Chen al. (2025b) learns aagegory-agnostic keypoints via diffusion-basee keyness prediction and correspondence trans- fer. Another line of works Yang et al. (2024); Kim et al. (2024); Lu al. (2024); Rusanovsky et al. 2025) also leverage textual descriptions for guidance, enhancing category ereegor--annotic generalization but still depending on auxiliary language priors. Most relevant to our work, SDPNet Ren et al. (2024) adopts discriminative approach by predicting fixed adjacency matrix from support features. How- ever, it lacks mechanisms to model structural uncertainty, limiting robustness under support-query mismatch. In contrast, we propose generative framework that infers flexible, instance-specific graphs from support images, enabling more adaptive and resilient structure modeling.
2.2 LATENT STRUCTURE LEARNING FOR POSE ESTIMATION
Latent structure learning has been widely adopted to reason inter-keypoint dependencies in pose estimation. Early approaches Wang et al. (2020); Hassan & Hamza (2023) leverage graph convolu- tional networks to refine keypoint predictions by modeling predefined skeletal connections, which restrict applicability to human- hand-specific topologies. Generative models, particularly vari- ational frameworks like the Variational Graph Autoencoder (VGAE) Kipf & Welling (2016) and CVAM-Pose Zhao al. (2024) have demonstrated promise in capturing structural variability and uncertainty, but typically remain tied to specific classes or predefined topologies. More recently, I-VIPELLvy & Shrivastava (2024) leverages variational autoencoder framework learn view- invariant latent pose representation. ProPose Han et al. (0225) reformulates 3D human pose es- timation probabilistic generative task by modeling instance-level pose distributions, enabling uncertainty- and sample-efficien inference. Following these advancements, we explore a gen- erative formulation learn instance-level latent structures, aiming enhance generalization and move beyond the reliance predefined priors. While learning latent structures improves flexibil- ity, remains insufficient under the CAPE setting, where support sets are sampled stochastically. This insight suggests mechanism that aggregates multiple latent graphs into guery-conditioned structure, dynamically emphasizing support information most aligned with the query.
METHOD
To effectively learn optimal keypoint structural dependencies and eliminate the adverse effects of framework tailored for Eatteory-Agnosiiee
appropriate support sets, we propose generative-baue GraphCape/Hinschom Pose Estimation (CAPE). This method is grounded in GraphCape Hirschorn & Avidan (2024) frame- work without reliance on skeleton priors. We begin by presenting concise overview of the pipeline before introducing our generative graph learning module and query-aware fusion technique.
3.1 OVERALL PIPELINE
The goal of CAPE is to estimate the locations of semantic keypoints K2 R for query image In. given a small set of annotated exemplars from an unseen category, where M, denotes the maximum possible number of keypoints. In the N-shot setting, provided with set of

---

Published as a conference paper at ICLR 2026
2 &.F.
Self Atteationn
Imszo aai
Image
Proposal Generator
Gymey faraties
Graph Transformet Decoder
rraus Trausformed former Encoder
(R
06300
Iuitd
MI P
Euceder
Dexaler
Cross Attention
Image Beckbons
repport lpe fort
[
boxd
Predictoen R.
Graph FFN
Norm +
Herutiive StructureVAE
Graph Transformer Deceder
Figure 2: Architecture Overview. Our approach utilizes pre-trained backbone extract image features, which are refined by transformer encoder through self-attention. A proposal generator employed along- side graph transformer decoder. Subsequently, we employ itcrative StructureVAB to generate probabilistic adjacency matrices, and integrates them to a query-aware graph in the graph transformer decoder, improving localization accuracy by graph-oriente decoding.
N support pairs {[T, K;) where each support image I is annotated with a set of keypoints K E M2 for category с (which may vary in keypoint count Mc). Initially, a shared backbone <(-) extracts visual features F, <L,,) and F" = o(I"). The support features are then aggregated with their corresponding keypoint targets produce keypoint-aware embeddings F. RD where M represents the maximum number of potential keypoints, and is the embedding size. Then, similarity aware proposal generator computes correlations between F, and Fg. yielding po- sition proposals / E I 4xx2 As shown in Figure to model keter-keypoint dependencies and learn flexible skeleton knowledge, we introduce an iterative Structure-aware Variational Autoencoder (i- SVAE) that infers latent adjacency matrix A IR M conditioned on F,. This probabilistic graph captures iettance-specffee keypoint relations and is fused with visual cues in the graph trans- former decoder. We further mitigate visual uncertainty and reduce the adverse effects of improper support images through a Compositional Graph Transfer (CGT) strategy, which aggregates multiple latent hypotheses into a query- aware graph. This composition is injected into the graph transformer decoder to guide self- and cross-attention, progressively refining keypoint predictions.
3.2 ITERATIVE STRUCTURE LEARNING
In CAPE, the support and query mismatch in visibility, poses and topologies, making structural alignment essential. Our framework addresses this discrepancy jointly with i-SVAE that learns layer- wise, instance-specific graphs from support features., and CGT that adapts them to the query. Most CAPE methods make oversimplified assumptions: either modeling keypoints as independent en- tities relying manually defined priors. Such assumptions hinder the capture of topological consistency and pose variability across instances. To this end, we reformulate structure inference as graph learning problem: keypoints are nodes, and their relationships encoded in latent ad- jacency matrix. Instead of using static, category-specife graphs, we propose an i-SVAE that learns and refines instafce-specifie keypoint graphs across decoding stages.
Graph Formulation. Let the graph G EV,E) consist of M keypoint nodes E V with initial node feature matrix X E M and an initial noisy adjacency matrix ()) E RM MM encoding edge set E. Our goal is learn function F= that maps support keypoint embeddings F, to soft adjacency matrix A, capturing latent inter-keypoin keypoint dependencies. We formulate this within the principled Variational Graph Autoencoder (VGAE) Kipf & Welling (2016), wherein the graph structure is treated posterior and z the latent code. The code is decoded into a soft adjacency matrix that models probabilistic keypoint connectivity.
latent variable: gs (z) z E RR: with Us denoting the approximate
Iterative StructureVAF. The i-SVAE consists two components: a probabilistic encoder that parameterizes a latent graph distribution, and decoder that constructs the adjacency matrix from this latent space. A detailed figure is shown in the supplementary material. At each layer I, given support node embeddings F!!) M from the previous graph transformer decoder layer,
4

---

Published as a conference paper at ICLR 2026
first perform variational inference to estimate the latent structural embeddings:
( log( (eg((o1)) [)] Enc( u log ( RP: go (2 F = (2(2)) ;1,)) diag((!"))
(1)
where Enc denotes the graph encoder that produces the approximate posterior distribution over the latent graph codes z. We then employ the reparameterization trick sample z R=:
c 6, <~~ <0,(0,
Next, the latent code is passed through fully connected decoder construct the adjacency matrix Á = Dzc TRM where Dec represents the graph decoder. To ensure the undirectionality and interpretability of the adjacency matrix, we symmetrize and normalize row-wise:
A sym (A AlO) T
A (A4)) sym),
(3)
Then, take the CGT strategy to fuse multiple sampled adjacency matrices into unified, query- aware graph, which is introduced the next subsection. This fusion strategy is performed via Bayesian averaging, reweighted by graph-level uncertainty and relevance the query features:
- = CGT((AL), (n!)), (,))),F Fq).
(4)
To ensure meaningful latent representations and regulate structural uncertainty, we minimize the Kullback-Leible divergence between the approximate posterior g X) and a Gaussian prior p(z) (O0 I). Besides, we impose an l sparsity constraint on the learned adjacency to encourage minimal and interpretable connectivity. The total i-SVAE loss at the I-th decoder layer is defined as:
VAE
(.)) +B- C(U) ssarrs
DKL 20l (2)(2()) p(2())
IIP.
(5)
M
Tinal
Prior Regularization
Sparse Matrix
where the syper-parameter / 0.1. To leverage the learned structural priors, we follow the Graph- Cape and incorporate graph convolutional layer conditioned on the final matrix A
final
F(-1)= @Wan) (() A W WeerF!!)).
where Wadj Wserr E D are learnable weights, and o-) denotes ReLU activation. The first term aggregates features from semantically spatially connected neighbors, while the second term retains individual node semantics via self transformatiion.
final skeleton final serves as the structural guidance for message passing:
F(++) (C A00) p((+1)
(P()) MLP(F((+1)
where (! E Kx2 is the predicted keypoint locations 1-th layer used for intermediate supervi- sion, with the output from the final layer as the final keypoint prediction. And a and o are the sigmoid and its inverse function.
By embedding i-SVAE within each decoder layer, our method performs iterative structural refine- ment, progressively updating latent pose graphs in response evolving visual semantics and local- ization cues. This layer-wise iterative design enables the model to capture diverse structural patterns and encode high-order keypoint dependencies, thereby strengthening relational reasoning and im- proving generalization to novel categories. See Appendix A.I for further details.
3.3 COMPOSITIONAL GRAPH TRANSFER
While i-SVAE enables laser-wise modeling of mustance-specingg pose graphs, its stochastic sampling process introduces uncertainty across multiple latent graphs. To this end, we propose Compositional Graph Transfer (CGT), a query-aware aware graph fusion mechanism that aggregates multiple sampled adjacency matrices into robust and expressive structural representation. Specifically, given set
5

---

Published as a conference paper at ICLR 2026
of N₂ latent graphs sampled from i-SVAE the 1-th decoder layer, denoted as {Ax ( each associated with a latent distribution parameterized by (n(), o ) our goal to construct matrix Anial E XMM that best reflects the robust structure relationships among keypoints conditioned on the query context, while simultaneously alleviating over-reliance on the support-driven guidance.
(0
To achieve this, we adopt a Bayesian confidence-weighted aggregation strategy. Firstly, we define the confidence of each sampled graph as the inverse of the total variance:
Wn
FL
ΣP D.
(8)
2==1
Wm
where e -e a small constant to ensure numerical stability. These normalized weights w. re- flect the epistemic uncertainty of each latent sample and serve to guide the fusion process. The fused adjaceney is then computed via weighted average: A Ew i. Al To further align the fuse structure with query-specific evidence, we incorporate query-guided gating. Let Fg R denote the query feature, where [h, w] denotes the patch size in image backbone. We compute attention-base gating scores a by comparing global query descriptors with each means μ!);
sim(Pool(F)), (1))
Li E1 sim(Pool(Fg) (0))
where sim(.,-) denotes cosine similarity and Pool is global average pooling operator. The final fused graph becomes A = Et= a11)) A Auseed where I. E [1, L.]] is the current decoder layer. The fusion process enhances robustness against sampling stochasticity and grounds structural reasoning in the visual context of the query. The resulting graph A is propagated into the GCN layer, enabling structure-aware refinement of keypoint predictions. See Appendix A. for CGT details.
3.4 TRAINING AND INFERENCE
For the category-agnostic pose estimation task, we employ the commonly used loss loss|Shi et al. (2023);
Hirschorn & Avidan 20244; Rusanovsky et al. (2025) Eprea:
Lered Aheatmap
Lheatmop + foffsets
(10)
Cheatmup
Σ H H.
Lotieet
K K,
Me. H W
where H. denotes the output similarity heatmap and H. is the ground-truth heatmap. KJ is the output keypoint location from the Graph Transformer layer and K, is the ground-truth location. By our framework and internal modules 3.2 our overall training objective is as follow:
Cpred + + LVVE:
(11)
where y = 1e-3 is the hyper-parameter. During inference, our model uses the final layer output K as the predicted location. Within the i-SVAE variational inference process, the latent code & is equal to the mean j of the approximate posterior, effectively collapsing the stochastic sampling. This deterministic substitution ensures stable and consistent structural priors, aligning with the learned feature distribution while eliminating inference-time uncertainty.
4 EXPERIMENTS
4.1 IMPLEMENTATION DETAILS
We train and evaluate our method on machine with an NVIDIA A GPU with 40 GB of memory. The architecture is implemented within the MMPose framework [Contributors (020). To ensure fair comparison, the configuration settings remain consistent with GraphCape Hirschorn & Avidan (0224) and CapeFormer Xu et al.) (2022a). During training, we use 256 x 256 input images and apply data augmentation including random scaling in the range ([-0.15, 0.15]) and random rotation within ([-15 15"l) for fair comparisons. All models are trained for 200 epochs with step- wise learning rate scheduler that decreases by a factor of 10 at the 160th and 180th epochs. We use Adam optimizer to train the model for 200 epochs with batch size of 16. See Section A.2 for details.

---

Published as a conference paper at ICLR 2026
Table 1: Comparisons on MPP-100: PCK@0.2 performance under the 1-shot setting. GenCape achieves the best average performance on the average of all splits, outperforming state-of-the-an methods under all three support sets types.
Type
Method
Support
Split
Split 2
Split 3
Split 4
Split 5
Avg.
POMNeI Xu et al.
2022a
Image
84,23
78.25
78.17
78.68
79.17
79.70
CapeFormer Slu et a
2025
Image
89.45
84.88
83.59
83.53
85.09
85.31
ESCAPE Nguyen el sl. 2024
Image
86.89
82.55
81.25
81.72
81.32
82.74
Image support
MetaPoint+ [ bon et
2024
Image
90.43
85.59
84.52
84.34
85.96
86.17
CapeFormer-11 Shi
Image
89.48
86.69
84.79
84.97
86.25
SDPNet (HINNet-3)) Ren
2024
Image
91.54
86,72
85.49
85.77
87.26
87.36
SCAPE Liang al.
2024
Image
91.67
86.87
87.29
85.01
86.92
87.55
CLAMP ChaMP al. 2023
Text
72.37
Texpos
X-Pose Yang al 2024
Imagel,Text Text
89.07
85.05
85.26
85.52
79.79
86. 14
PPM+CPT Fong et al 2024
Image Text
91.03
88. 06
84.48
86.73
87.40
87.54
CapeX-S Rusanovaky cl.
2025
Image Text + Graph
95.17
88.88
87.72
88.24
91.81
90.37
GraphCape-T
HapphCaappT(Hischhmm Hiirchomm & Avidan
2024
Image Graph
91.19
87.81
85 68
85.87
85.61
87.23
Gmph-support
GenCape-1 (Ours)
Image (Graph)
92.05
88.69
86.89
85.88
87.02
88.09
GruphCape-S5 SaaiiCCppeSSHisschoort |Hirschom Avidan
2024
Image Graph
94.73
89.79
GenCape-S
Image (Graph)
95.23
90.60
89.46
89.32
90.43
91.01
Table 2: Performance comparisons under 5-shot
Table 3:
Performance comparison of
setting with SwinV2-small as image backbone.
CAPE methods under stricter thresholds.
Method
Split
Split 2 Split 3 Split
Split
Avg.
Method
Th0.05
Th0.1
Th0.15
Th0.2
mPCK
Fine-tune
71.67
57.84
66.76
66.53
60.24
64.61
POMNet
44.39
68.87
79.39
84.23
69.22
POMNet
84.72
79.61
78.00
80.38
80.71
CapeFormer
51.03
75.17
84.87
89.45
75.13
CapeFormer
91.94
88.92
89.40
88.01
88.25
89.30
ESCAPE
48.24
72.25
82.30
86.89
72.42
SDPNet
93.68
90.23
89.67
89.08
89.46
90.42
MctaPoint
55.08
77.12
85.81
90.43
77.11
PPM+CPT
93.64
92.71
91.76
92.85
91.94
92.58
GraphCape
48.55
73.43
83.71
88.19
73.47
SCAPE
95.18
91.25
91.78
90.74
91.10
92.01
SCAPE
54.09
77.34
87.02
91.67
77.53
GraphCupe
96.67
91.48
92.62
90.95
92.41
92.83
FMMP
57.30
78.48
87.28
91.82
78.72
GenCape (Ours)
97.19
92.94
92.26
91.93
93.34
93.53
GenCape-R50
58.91
80.63
88.97
92.74
80.31
4.2 DATASET AND METRIC
We train and evaluate our method on the MP-100 Xu et al. (2022a) dataset, which is currently the only public dataset for CAPE tasks. MP-100 contains 100 sub-categories and supercategories, with total of 18K images and 20K annotations. The number of annotated keypoints covers wide range, from to 68. To ensure and identify performance stability on unseen categories, following previous methods Xu et al. (2022a), the dataset is divided into 5 splits. In each split, these categories are split into train, validation, and test sets 70/10/20 ratio without any category overlap. We use the Probability of Correct Keypoint (PCK) as the evaluation metric. We follow the standard metric PCK @ 0.2 as the default metric for performance reporting. And we further evaluate model perfor- mance under stricter threshold conditions ((0.05, 0.1, 0.15, 0.2)) for comprehensive comparisons.
4.3 BENCHMARK RESULTS
We conduct a comparative analysis of our approach with SwinV2 Liu et al. (0022) as backbone, against the graph-support method: GraphCape[Hirsehhee Hirschorn & Avidan (2024) as our baseline: image- support methods: POMNet Xu et al. (2022a). CapeFormer Shi et al. (2023), ESCAPE Nguyen
al. 2024). MetaPoint Chen et (2024). SDPNet Ren et (2024). FMMP Chen al. (2025));
text-support methods: CLAMP Zhang et al. (2023), XPose Yang et al. (2024), PPM+CPT Peng al. (2024), CapeX Rusanovsky et al. (2025). Our evaluation conducted on the MP- 100 dataset, considering the 1- and 5-shot settings. We denote models as GenCape-T and GenCape-S for ab- breviation, corresponding to employing Swin V2-tiny and small as the image backbone, respectively.
1-shot results. As reported in Table GenCape-T achieves average PCK of 88.09%, surpassing the strong graph-based GraphCape- baseline (87.23% by +0.86%. Remarkably, without relying on class-level text, method still outperforms multimodal CAPE models such as XPose (6.14%) and PPM+CPT (87.54%). indicating that the learned structure-awe aware representation serves as an effective surrogate for external semantic cues. Moreover, GenCape-S attains 91. 91.01% % PCK, exceed-

---

Published as a conference paper at ICLR 2026
ing CapeX-S (90.37%) which leverages both textual and skeleton support. 5-shot results. Under the 5-shot setting (Table 2). GenCape-S further improves, achieving an average PCK of 93.53% and outperforming all representative CAPE methods, including PPM+CPT (92.58%), SCAPE (91.01 %), and GraphCape (92.83%). The model delivers consistent gains across all splits except Split 3, with maximum of 97.19% on Split and minimum of 91. 91.93% on Split 4.
More detailed comparisons. [ableepppeseeee results under stricter thresholds (0.2. 0.15, 0.1, 0.05) on Split-1 with ResNet-50 as backbone. Threshold choice strongly affects relative gaps: GenCape- R50 outperforms FMMP by 0.92% at PCK @0.2, and the margin increases to 1.61 % at PCK @0.05, showing that coarse thresholds may mask fine-grained discriminative ability. GenCape achieves the highest accuracy at all thresholds and improves mPCK by +1.59% over FMMP.
4.4 ABLATION STUDY
In this section, we conduct all of the ablation studies of our proposed method on the MP-100 dataset Split-1 using the S2inV2-Tiny backbone, unless otherwise specified. We now present key ablation experiments. Additional ablations can be found in Appendix B
Effects of Different Components. To rigorously quan- tify the contribution of each module in our framework, we conduct a comprehensive ablation study under 1-shot setting. We isolate and examine the effects of variational regularization (LKL). sparsity penalization (Cspurse, and the CGT module. Table 4 reports the results. Starting from the baseline that excludes all components, we ob- serve a base PCK of 99.1%. Introducing the KL diver- gence term LK yields modest improvement (+0.24%),
Table 4: Ablation studies different components (Split-1).
CKI. Cyparee CGT
PCK
A
91.19
0
91.43
+0.24
91.75
+0.56
92.05
+0.86
suggesting its stabilizing role by constraining posterior distributions with the Gaussian prior. Com-
bining both constraints, the performance improves substantially to 91.75%, confirming their syn- ergistic effect in enforcing informative and interpretable structural cues. Further incorporating the CGT mechanism yields 0.86% gains, underscoring the importance of compositional graph fusion in consolidating uncertainty-awart structural hypotheses and enhancing query-specific robustness.
Effects of Hyper-parameter. We in- vestigate the inffuence of four key hy- perparameters: latent dimensionality D,, sample count N.. and loss weights and 7 (5q [... As shown in Table 5 the model achieves opti- mal performance at D2 = 32, with larger dimensions leading to perfor- mance degradation (e.g., 92.05% at 32 PCK
Table 5: Ablation studies on hyper-parameters, includ- ing latent dimension D2, number of posterior samples N., and weighting factors 3 and 7 for the training objectives.
of i-SVAE
Parameter D, (N, = 3)
32
64
128
PCK
92.05
91.89
91.54
91.40
Parameter N. (D. 32)
2
3
5 91.63
91.60
92.05
91.47
lower at 64/128), indicating that excessive latent capacity introduces re- dundancy and weakens structural com- pactness. Fixing D2 = 32, we vary N, and observe the best results at N. 3, while both smaller (N, 2) and larger (N, 5) values slightly reduce accu- racy. This suggests trade-off between uncertainty modeling and variance over-smoo, For loss weights, we separately tune y (KL loss) and / (sparsity loss). The model peaks at y = 10 when is fixed, balancing reconstruction and regularization. Likewise, B = 0.1 yields the best PCK, high- lighting its role in regulating structure sparsity without over werrpennliizing connectivity. These results confirm that our framework remains robust across hyperparameter variations, with optimal settings jointly promoting expressiveness and structural regularity.
Objective weighting factor
Parameter 3 (7 1c-3) 3)
1
1e
1c 1e-
le -e- 91.74
PCK
91.43
92.05
91.65
Parameter 7 (8 0.1)
1
1e
1e-2
1e-3
PCK
90.99
91.14
91.71
92.05
Effects of Cross-Care Category Generalization. To assess structural generalization beyond cat- egory boundaries, we conduct three pross-supercategory pairs experiments: Person<=Felidae, Felidae Ursidae,, and ArisoalaAniallFaaee These settings cover diverse appearance and topology gaps, including GenCape consistently outperforms GraphCape across all pairs, with margins up +1 1.8 points (Fe-
transfers and full-body to face shifts. As shown in Table 6
8

---

Published as a conference paper at ICLR 2026
GT
Tradic
Prudett
Predetieed
Lager
Layee
Lyper
Support Iz?
Quary GT
GopaCCape
GanClep
Figure 3: Comparisons on adjacency matrices inferred by i-SVAE and predefined graph. The Predict" is the predicted locations with the prior connections, while the Predict is with learned connections.
Figure 4: Comparisons of qualitative visualization. The red boxes high- light the differences.
Pirson) demonstrating the robustmess of compositional latent graphs under structural shifts. We further evaluate the challenging cross-species setting, where models trained on human bodies and tested on morphologically distinct animal bodies. GraphCape suffers substantial performance drops, e.g., 31.09 on Rabbit Body, due to the rigidity of static priors. In contrast, GenCape main- tains strong transferability, achieving +21.05 improvement and scaling effectively across species with varying skeletons on Squirrel Body. These results validate that generative, instance-specifee graphs better capture structural uncertainty and enable robust pose reasoning across categories.
Qualitative Analysis. To intuitively un- derstand how our model mines effective skeletons, we visualize the pre-normalized adjacency matrices across decoder layers with the inferred skeletons in Figure B Compared with fixed skeletons, whose manually chosen edges are not guaranteed to be semantically correct and can mislead message passing, the graph inferred by i- SVAE are query-conditioned and instance- specitic. So pose, viewpoint and other changes are directly reflected in the ad- jacency matrix. The learned structures are clearly task- task-driven, emphasizing high- influence keypoints that most contribute to accurate localization, e.g., nose and eyes as anchors for klipspringer face. four corners for bed, and central torso for long sleeved outwear. They receive denser and stronger connections and thus provide more effective geometric constraints. Figure 4 highlights GenCape's 's superior adaptability when exposed to varying support-query pairs in 1-shot setting. Compared to GraphCape, which fails under pose misalignment and occlusion (e.g., bison and swivelchair), our model consistently localizes keypoints accurately by leveraging its uncertainty-awarc graph. Interestingly, we further visualize the impact of varying support images for the same query instance, as shown in the fourth and fifth rows. When the fox undergoes different forms of occlusion, GraphCape suffers from inconsistent errors. while GenCape maintains stable predictions. These results suggest that our method is significantly less susceptible to the adverse effects introduced by suboptimal support sets, Additional analyzes are provided in Appendix
Table 6: Cross category evaluation comparing Graph- Cape and GenCape based on V2-nV22-maal backbone.
Train
Test
GraphCape Ours
Cross super-category
Person
Felidae
58.46
58,82
Felidae
Person
56.26
68.08
Felidae
Ursidae
73.10
74.10
Ursidac
Felidae
73.83
77.76
AnimalFace
Person
65.49
77.93
Person
AnimalFace
65.08
68.41
Cross species
Human Body
Fox Body 46.65
57.47
Human Body
Rabbit Body 31.09
52.14
Human Body
Squirrel Body 54.30
73.37
CONCLUSION
In this paper, we introduce GenCape, a generative framework for CAPE that infers keypoint rela- tionships solely from visual inputs. GenCape integrates an iterative Structure-aware Variational Au- toencoder to progressively infer cestancc-specifii keypoint relationships, alongside a Compositional Graph Transfer module that aggregates multiple latent graph hypotheses into query-aware structural cues. Extensive experiments on MP-100 demonstrate that GenCape achieves the state-of the-art performance. These results demonstrate the effectiveness of framework.

---

Published as a conference paper at ICLR 2026
6 ACKNOWLEDGMENT
This work was supported in part by New Generation Artificial Intelligence-Nationn Science and Technology Major Project under Grant 237001 in part by the National Key Research and Development Project under Grant 2023YFC3806000, in part by the National Natural Science Foun- dation of China under Grant 62406226, in part by the Shanghai Municipal Science and Technology Major Project under Grant 001SHZZZX0000 in part sponsored by Shanghai Sailing Program un- der Grant 24YF2748700, in part by -ww-Geneaation Information Technology under the Shanghai Key Technology R&D Program under Grant 25511 103500.
REFERENCES
Junjie Chen, Jiebin Yan, Yuming Fang, and Li Niu. Meta-point learning and refining for category- agnostic pose estimation. In Conference on Computer Vision and Pattern Recognition, pp. 23534- 23543, 2024.
Junjie Chen, Weilong Chen, Yifan Zuo, and Yuming Fang. Recurrent feature mining and keypoint mixup padding for category-agnostic pose estimation. In Conference on Computer Vision and Pattern Recognition, pp. 22035-22044, 2025a.
Junjie Chen, Zeyu Luo, Zezheng Liu, Wenhui Jiang, Li Niu, and Yuming Fang. Weak-shot keypoint estimation via keyness and correspondence transfer. The Thirty-ninth Anmual Conference on Neural Information Processing Systems, 2025b.
MMPose Contributors. Openmmlab pose estimation toolbox and benchmark. httttt::/tgtthh/ee httpl//oon..o..t/.t/////////g// com/ open mm ab/ ab/mmpose 2020.
Jumin Han, Jun-Hee Kim, and Seong-Woy Whan Lec. Propose:
Probabilistic 3d human pose estimation with instance-level distribution and normalizing flow. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pp. 2338-3346. 2025.
Md Tanvir Hassan and A Ben Hamza. Regular splitting graph network for 3d human pose estimation. IEEE Transactions on Image Processing, 32:4212 42222,2-222, 2023.
Or Hirschorn and Shai Avidan. graph-based approach for category-agnostic pose estimation. In European Conference on Computer Vision, pp. 469-485. Springer, 2024.
Junho Kim, Hyungjin Chung, and Byung-Hoon Kim. Capellm: Support-free cagegory-agnostic pose estimation with multimodal large language models. arXiv preprint arXiv:2411.06869, 2024.
Diederick P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International
Conference Learning Representations, 2015.
Thomas N Kipf and Max Welling. Variational graph auto-encoders. NIPS Workshop on Bayesian
Deep Learning, 2016.
Mara Levy and Abhinav Shrivastava. V-vipe: Variational view invariant pose embedding. In Con- ference on Computer Vision and Patlern Recognition, pp. 1633-1642, 2024.
Yujia Liang, Zixuan Ye, Wenze Liu, and Hao Lu. Scape: A simple and strong category-agnostic pose estimator. In European Conference on Computer Vision, pp. 478-494. Springer, 2024.
7e Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, al. Swin transformer v2: Scaling up capacity and resolution. Conference on Computer Vision and Pattern Recognition, pp. 12009-12019, 2022.
Changsheng Lu, Zheyuan Liu, and Piotr Koniusz. Openkd: Opening prompt diversity for zero- and few-shot keypoint detection. European Conference on Computer Vision, pp. 148-165. Springer, 2024.
Xiao Ma, Sumit Patidar, lain Haughton, and Stephen James. Hierarchical diffusion policy for kinematics- multi-task robotic manipulation. In Conference on Computer Vision and Pattern Recognition, pp. 1808 18090,, June 2024.
10

---

Published as a conference paper at ICLR 2026
Khoi Due Nguyen, Chen Li, and Gim Hee Lee. Escape: Encoding super-keypoints for category- agnostic pose estimation. In Conference on Computer Vision and Pattern Recognition, pp. 23491- 23500, 2024.
Duo Peng, Zhengho Zhang, Ping Hu, Qiuhong Ke, David KY Yau, and Jun Liu. Haresssing text- to-image diffusion models for category-agnon pose estimation. In European Conference on Computer Vision, pp. 342-360, 2024.
Jiyong Rao, Brian Nlong Zhao, and Yu Wang. Probabilistic prompt distribution learning for animal pose estimation. In Conference on Computer Vision and Pattern Recognition, pp. 29438-29447, 2025.
Pengfei Ren, Yuanyuan Gao, Haifeng Sun, Qi Qi, Jingyu Wang, and Jianxin Liao. Dynamic support information mining for category-agnoste pose estimation. In Conference on Computer Vision and Pattern Recognition, pp. 1921-1930, 2024.
Matan Rusanovsky, Or Hirschorn, and Shai Avidan. Capex: Category-agnostic pose estimation from textual point explanation. In International Conference on Learning Representations, 2025.
Min Shi, Zihao Huang, Xianzheng Ma, Xiaowei Hu, and Zhiguo Cao. Matching is not enough: A pose estimation. Conference on Computer Vision
two-stage framework for and Pattern Recognition, pp. 7308-7317, 2023.
Lucas Stoffi, Andy Bonnetto, Stéphane d' Ascoli, and Alexander Mathis. Elucidating the hierarchical nature of behavior with masked autoencoders. In European conference on computer vision, pp- 106-125. Springer, 2024.
Ke Sun, Bin Xiao, Dong Liu, and Jingdong Wang. Deep high-resolution representation learning for human pose estimation. In Conference on Computer Vision and Puttern Recognition, pp. 5693-5703, 2019.
Jian Wang, Xiang Long, Yuan Gao, Errui Ding, and Shilei Wen. Graph-pemn: Two stage human pose estimation with graph pose refinement. In European Conference on Computer Vision, pp- 492-508. Springer, 2020.
Lumin Xu, Sheng Jin, Wang Zeng, Wentao Liu, Chen Qian, Wanli Ouyang, Ping Luo, and Xiaogang Wang. Pose for everything: Towards category-agnostic pose estimation. In European conference on computer vision, pp. 398-416. Springer, 2022a.
Tianyang Xu, Jiyong Rao, Xiaoning Song. Zhenhua Feng, and Xiao-Jun Wu. Learning structure- supporting dependencies via keypoint interactive transformer for general mammal pose estima- tion. International Journal of Computer Vision, pp. 1-19, 2025.
Yufei Xu, Jing Zhang, Qiming Zhang, and Dacheng Tao. Vitpose: Simple vision transformer base- lines for human pose estimation. In Advances in neural information processing systems, vol- ume 35, pp. 38571-38584, 2022b.
Jie Yang, Ailing Zeng, Ruimao Zhang, and Lei Zhang. X-pose: Detecting any keypoints. In Euro- pean Conference on Computer Vision. pp. 249-268, 2024.
Sen Yang, Wen Heng, Gang Liu, GUOZHONG LUO, Wankou Yang, and Gang YU. Capturing the motion of every joint: 3d human pose and shape estimation with independent tokens. In International Conference on Learning Representations, 2023.
Shaokai Ye, Anastasiia Filippova, Jessy Lauer, Steffen Schneider, Maxime Vidal, Tian Qiu, Alexan- der Mathis, and Mackenzie Weygandt Mathis. Superanimal pretrained pose estimation models for behavioral analysis. Nature Communications, 15(1):1165, 2024.
Hang Yu, Yufei Xu, Jing Zhang, Wei Zhao, Ziyu Guan, and Dacheng Tao. AP-10k: A benchmark for animal pose estimation in the wild. In Advance Neural Information Processing System, 2021.
Yuhui Yuan, Rao Fu, Lang Huang, Weihong Lin, Chao Zhang, Xilin Chen, and Jingdong Wang. Hrformer: High-resolution vision transformer for dense predict. In Advances in neural informa- fion processing systems, volume 34, pp. 7281-7293, 2021.

---

Published as a conference paper at ICLR 2026
Xu Zhang. Wen Wang, Zhe Chen, Yufei Xu, Jing Zhang, and Dacheng Tao. Clamp: Prompt-based based contrastive learning for connecting language and animal pose. In Conference on Computer Vision and Pattern Recognition, pp. 13272-23282 2023.
Jianyu Zhao, Wei Quan, and Bogdan J Matuszewski. Cvam-pose: Conditional variational autoen- coder for multi-object monocular pose estimation. In British Machine Vision Conference. BMVA, 2024.
Ce Zheng, Wenhan Wu, Chen Chen, Taojiannan Yang. Sijie Zhu, Ju Shen, Nasser Kehtarnavaz, and Mubarak Shah. Deep learning-based human pose estimation: A survey, ACM computing surveys, 56(1):1-37, 2023.
Ying Zheng, Lei Yao, Yuejiao Su, Yi Zhang, Yi Wang, Sicheng Zhao, Yiyi Zhang, and Lap-Pui Chau. A survey of embodied learning for object-centric robotic manipulation. Machine Intelligence Research, pp. 1-39, 2025.
12

---

Published as a conference paper at ICLR 2026
APPENDIX
A ADDITIONAL IMPLEMENTATION DETAILS
A.1 DETAILS OF NETWORK STRUCTURE
Algorithm Compositional Graph Transfer (CGT)
Require: Support keypoint embedding Es, query feature Pg
I: for I..... Lg
2:
Structure VAE encoding:
( log(( = Enc (F
3: Random sampling (N, turns)
z No 1 + o Oe
Structure VAE decoding:
1A Jn=l
10) 1
10)
5:
Computing confidence:
Wn.
Wn
Si=l [00) +
Wr
Σ
No
m=
6:
Bayesian aggregation: A > Em Wn -All)
7:
Update support keypoint embedding:
p GCN(F!D) A
Compute Gating Scores: (1) amm(Poo((Fg)deo (0)
E [1, La]
Σt simiPooll Uyl.
9: Fusion across Layers: A + Et= a1) A
10:
end for
Table 7 provide detailed architecture con- figurations for i-SVAE module. M denotes the maximum number of keypoints, D is the node feature emhedding size, and hidden_dim refers to the hidden dimen- sionality of the VAE en- coder. The latent.dii used for graph sampling is denoted by Dz, consistent with the main manuscript. Figure S) illustrates the pipeline Compositional Graph Transfer. In the I-th decoder layer, the i-SVAE produces multiple sampled adjacency matrices ( - together with their posterior statisties y(()) and (i)) Bayesian aggregation fuses these samples into a single adjacency estimate, and query-guie suery- gating further reweights the fused graphs using the query embedding F.. The resulting matrix final serves the final structure used for message passing in that layer. The details can be found in Section 3.3
Compositional Graph Transfer
Bayesian Aggregation
Query-guided Gating
matrices (2
Σ
Eqith
256
Al Tinal
On
Taserd
.2/1
Faaned
Figure 5: The pipeline Compositional Graph Transfer. The i-SVAE the I-th decoder layer outputs sampled adjacencies (AA! Jn=i with posterior (!)) ((1), Bayesian aggregation fuses samples into a single adjacency esti- mate A(t) and query-guided gating reweights layers using the query embed- ding Fg. The result is the final graph A(!)) in that layer.
The complete pipelines of two proposed modules are outlined in Algorithm I Due the limited structural cues present in raw keypoint features, the iterative learning paradigm faces challenges in capturing meaningful latent graph structures. The resulting graphs F: (1+1) serve refined structural
13

---

Published as a conference paper at ICLR 2026
Table Architecture details of the iterative BrructurVAE, including graph encoder, latent sampling. and graph decoder in the Figure 2 (main manuscript).
Norm & Activation
Output Shape
Graph Encoder (for posterior distribution)
Input node feature æ
B, M, D
Reshape -> x'
B, M D]
Linear
ReLU
B3 hidden.dim)
Linear
[B, latent_dim)
Split + logo")
B, latent_dim] 2
Latent Sampling (Reparamcterizationn)
If training: t
B, latent_dim)
Else:
B, latent.dim)
Graph Decoder (to soft adjacency matrix)
Linear
Sigmoid
B, M M)
Reshape
B, M, M]
Symmetrization
A = (A+
B,M, M, M]
priors, facilitating the downstream reasoning module learn more expressive and structure aware keypoint representations.
A.2 DETAILED TRAINING CONFIGURATIONS
Table 8 summarizes the training and evaluation settings for the GenCape-T/S models. We adopt the Adam Kingma & Ba (2015) optimizer with base learning rate of 1.0 x 10-5, scheduled linearly and warmed up over 1,000 iterations with a warmup ratio of 0.001. Training performed for 175,000 epochs with batch size of 16 in float32 precision, ensuring stable convergence. For few- shot evaluation, the model is assessed under both 1-shot and 5-shot settings, with each episode comprising 15 query samples and total of 200 episodes ensure statistical reliability.
B ADDITIONAL EXPERIMENTAL RESULTS
B.1 ABLATIONS ON DIFFERENT STRATEGIES
To further assess the effectiveness of our proposed framework, we investigate the impact of different graph construction methods and different composi- tional graph transfer strategies. Table 9 compares different methods for constructing adjacency matrix. Random initialization results significantly per- formance drop, confirming the importance of struc- tural priors Learnable graphs consistently outper- form static counterparts, and removing the symme- try constraint leads to only a slight decrease, sug- gesting that bidirectionality is more critical than di- rectional specificity. We replace our layer-wise i- SVAE updates with the first-layer adjacency matrix predicted only once from the encoder output. we ob- serve: iter (92.05) vs. non-iter (91.48). This +0.57 improvement demonstrates that iterative refinement indeed beneficial. A fixed adjacency estimated once from static support features cannot adapt to the evolving decoder representations. Furthermore, we conducted an experiment that directly used the self- attention weights from Figure as the adjacency matrix. This variant achieves only 89.33 PCK P.2 (2.72 drop vs. 92.05), indicating that attention-induced "connectivity" fails to provide meaningful structure. This is expected: self-atee attentttnn mainly captures appearance-driven correlations, lacks un- certainty modeling, especially when support features ambiguous.
Table 8: Training configuration used for GenCape-T/S.
Training recipe: optimizer
Adam
Learning hyper- pper-parameters:
base learning rate
1.0E-05
learning rate schedule
lincar
batch size training steps Ir warmup iters warmup ratio warmup schedule data type norm opsilon
16
175,000
1.000
0.001
lincar
float32
1.0E-06
Few-shot testing hyper-parameters:
shots num_query num_episodes
1/5
15
200
composi-
14

---

Published as a conference paper at ICLR 2026
tional graph transfer strategies. The combination of query weighting the layer level and Bayesian fusion at the sampling level achieves the highest 92.05% PCK. In contrast, using the same strategy at both levels (e.g., query weighting for both) yields suboptimal results. We attribute this improvement to the complementary strengths of the two mechanisms: query weighting enables dynamic alignment of structural importance across layers based the semantic context of the query, while Bayesian fusion effectively mitigates uncertainty introduced by latent graph sampling. Conversely, a mis- match between Bayesian fusson across layers and query weighting across samples performs worse (91.55%) These findings highlight the importance of hybrid fusion strategies that jointly consider semantic relevance and structural reliability. Layer-wise representations encode semantically dis- tinct structural abstractions that benefit from query-adaptive weighting rather than confidence-bel averaging. Applying Bayesian fusion at this level can obscure semantically salient but uncertain layers, effectively flattening meaningful hierarchical distinctions.
Table 9: Comparisons of different adjacency matrix construction strategies under 1-shot setting with S2inV2-Thin -iny backbone.
Table 10:
Comparisons on compositional graph transfer strategies under 1-shot setting with Swin backbone.
Type
Symmetric
PCK
Static Graph
91.19
Layer-wise
Sampling-wis wise
PCK
Learnable Graph
92.05
Bayesian Fusion
Bayesian Fusion
91.36
Learnable Graph
X
91.71
Query Weighting
Query Weighting
91.74
Random Initialized Graph
84.39
Bayesian Fusion
Query Weighting
91.55
Non-iter Self-attention
91.48
Query Weighting
Bayesian Fusion
92.05
V
89.33
B.2 ADDITIONAL CROSS-SUPERCATEGORY RESULTS
Table 11: Cross-upperaategory results. PCK @0.2 performance under the 1-shot setting Split- Following the standard super-category partitioning protocol, our method achieves the best perfor- mance across all splits, demonstrating its strong generalization.
Method
HumanBody
HumanFace
Vehicle
Furniture
ProtoNet
37.61
57.80
28.35
42.64
MAML
51.93
25.72
17.68
20.09
Fine-tune
52.11
25.53
17.46
20.76
POMNet
73.82
79.63
34.92
47.27
CapeFormer
83.44
80.96
45.40
52.49
GraphCape
88.38
83.28
44.06
45,56
GenCape
89.69
93.76
47.74
66.63
We follow prior works Liang et al. (2024): Hirschorn & Avidan (2024); Rusanovsky et al. (2025) and perform a soorrsuuprra evaluation rigorously assess the generalization ability of our model across semantically diverse object classes. Concretely, in Table II we treat one of the four MP-100 pnperraategoress-ummaallt
HumanFace, Vehicle, Furniture-20 the test domain and train on the remaining three, creating four disjoint train-test splits. GenCape consistently achieves the highest accuracy across all cross cross- susercceprrt splits. These improvements highlight the strong generalization of our generative structural modeling, which captures keypoint dependencies that remain robust under large variations.
We further evaluate a more challenging setting: training one category and testing on differ- ent, structurally mismatched category. Across Train-+Test pairs, Table [2] shows that Gen- Cape consistently surpasses GraphCape, indicating stronger resilience to cross-category discrep- ancies. Notably, GenCape achieves substantial gains HumanBody- cle cle-+HuannBody (+8.40), and Chair-HumanBood (+4.23), showing its ability adapt entirely different topologies. Overall, these results highlight that GenCape learns transferable, generative keypoint relations that generalize reliably across heterogeneous object categories.
(+11.36), Vehi-
15

---

Published as a conference paper at ICLR 2026
Table 12: Cross domain transfer evaluation. PCK@0.2 performance under the I-shot setting on Split-1. Training on one super-category and testing the other.
Train
Test
GraphCape
GenCape
HumanBody
FamenFace
33.87
45.23
HumanFace
HumanBody
55.90
56,43
Furniture
HumanBody
73.79
73.09
HumanBody
Furniture
31.11
50.47
Vchicle
HumanBody
50.13
58.53
HumanBody
Vchicle
28.52
32.64
Chair
HumanBody
49.10
53.33
B.3 ADDITIONAL METRICS RESULTS
Table 13: Additional Metrics. AUC, EPE and NME performance under 1-shot setting.
Method
AUC % (f)
EPE (1)
NME (U)
PCK % (t)
GraphCape-T
89.10
41.04
0.08
91.19
GenCape-TT
89.50
39.65
0.08
92.05
GraphCape-S
91.16
30.05
0.06
94.73
GenCape-S 91.37
29.62
0.06
95.23
We further evaluate our model using three standard keypoint localization metries, as summarized in Table 13 AUC measures the area under the PCK curve and reflects overall accuracy across range of distance thresholds. EPE computes the Euclidean distance between predicted and ground- truth keypoints. NME reports the mean localization error normalized by object scale. These metrics provide a comprehensive assessment of both absolute and scale-invariant localization performance.
B.4 COMPARISONS ON COMPUTATIONAL COMPLEXITY
Table 14: Comparison computational complexity and accuracy across methods.
Method
GFLOPs
Params
FPS
PCK
POMNet
38.01
48.21M
6.80
46.05
One-Stage
22.65
26.86M
36.90
CapeFormer
23.68
31.14M
26.09
89.45
GraphCape-T
15.48
43.68M
15.36
91.19
GenCape-T 15.66
44.47M
14.89
92.05
GraphCape-S 27.75
65.06M
10.45
94.73
GenCape-S 27.93
65.85M
10.44
95.23
To further clarify computational efficiency, provide a comparison of GFLOPs, parameter counts, and inference speed. GraphCape and GenCape are tested on A100, while the results of POMNet, One-Stage, and CapeFormer were obtained on a RTX 3090. As shown in Table [4] despite this hardware discrepancy, the com- parison still reveals clear trend: GenCape introduces negligible computational overhead relative GraphCape (e.g., +0.18 GFLOPs and +0.8M params for the Tiny variant), while consistently deliv- ering higher accuracy. This confirms that our generative structural modeling improves performance without sacrificing efficiency.
taken from CapeFormer Shi al. (2023) where all measurements
C ADDITIONAL VISUALIZATION RESULTS
In this section, we present more qualitative results. As shown the first row of Figure 6 the support image depicts inverted top-view of dog, where GraphCape Hirschorn Hirschon & Avidan (024 exhibits
16

---

Published as a conference paper at ICLR 2026
(a) Support GT
(b) Query GT
(c) GraphCape
(d) GenCape
Figure 6: Comparative qualitative results. We compare more keypoint predictions with GraphCape SaaphhapeeHrrsseeeee Hirschom Avidan 2024) under the 1-shot setting. The red boxes highlight the regions with significant differences.
noticeable prediction drift on the left hind leg and right foreleg, while our method achieves more ac- curate localization. In the second and third rows, GraphCape overly relies on support instances with dissimilar poses, resulting in incorrect keypoint predictions. The fourth row presents a challenging case involving swivel chair, where structural reasoning becomes critical for precise keypoint infer- ence. Despite preserving the overall skeleton shape, GraphCape relics solely manually defined connections and fails to localize the seat correctly, producing an upward shift as highlighted by the red box. A similar issue is observed in the fifth row, where the predicted seat location is misaligned to the edge of the cat. Figure 7 provides additional antegory-agnostie pose estimation examples to further illustrate the effectiveness of our approach.
Figure X shows more visualizations between predefined skeletons and the latent adjacency matrices inferred across different decoder layers. In the first example (giraffe), the learned adjacency matrices
17

---

Published as a conference paper at ICLR 2026
SHI
Figure 7: Qualitative visualization. We visualize more keypoint predictions across different splits.
Predict
Predefined
Layer
Layer #2
Layer
Figure 8: Adjacency matrix visualization. We visualize more latent graph structures across different splits.
progressively capture long-range dependencies critical for reasoning along the elongated neck and legs, outperforming the sparse predefined skeleton. The second case (monkey) presents occlusion challenges from tree branches, where the model adaptively strengthens cross-limb connections in deeper layers to improve structural consistency, though minor prediction drift remains. For the bird, although predefined structures already offer reasonable symmetry, latent graphs further refine bilat- eral dependencies, enhancing accuracy. The trouser case represents particularly challenging case due its dense, repetitive keypoints and weak structural cues. These visualizations also enhance
18

---

Published as a conference paper at ICLR 2026
the interpretability of our GenCape. However, from Figure and observe that localization rors are primarily caused by visual feature ambiguity. Predictions on the swivelchair category show large structural deviations, suggesting that the weak and homogeneous textures of this class hinder the transfer of support-driven structural priors. We further evaluate robustness by randomly masking the query image. When 25% of the image is masked, GenCape-S shows only mild drop (93.05), but performance collapses at 50% masking (76.81), showing that heavy occlusion severely disrupts the visual evidence required for localization. This sharp degradation reinforces the need for gen- erative, uncertainty-awer structural modeling to cope with missing keypoints and support-que query mismatch.
Predict
Predefined
Layer
Layer Layer
Figure 9: Additional adjacency matrix and skeleton visualization. We visualized samples from gories (bed and klipspringer face) Split-1 more comprehensive illustration. Predict* column predicted locations with the prior connections, while the Predict column is with learned connections.
More qualitative analysis of the adjacency matrix of the same category. Figure 9 presents ad- ditional adjacency-ni matrix and skeleton visualizations for two same categories Figure 3 Each row corresponds to the layer-wise adjacency matrices progression produced and finally the skeleton. Brighter colors the matrices indicate stronger relational dependencies between corresponding key- points. We observe that adjacency patterns remain similar across samples of the same category. For instance, in first rows of bed category, the adjacency matrices consistently shows a core keypoint showing strong influence the others. From left to right: initially the core keypoint fires broadly, then adjacent points start to dominate local neighborhoods, and the final layer produces localized high-response clusters and suppressed irrelevant edges, indicating confident and diseriminative dependency graph. For the klipspringer face category in the last three rows, the progression follows different pattern: early layers emphasize local smoothness among neighboring keypoints, then the model increasingly attends to the central nose region as an anchor, and the final layer converges to distinct pattern. These differences across categories reflect that the layer- weyer-wise-a graphs represent coarse-to-fine refinement of functional dependencies. The evidences that the learned graphs encode structural dependencies useful for CAPE, rather than merely aiding optimization convergence.
19


Reviews:
[{"review_id": "9sfCv2XUiP", "reviewer": "Reviewer_Dn8o", "summary": "The submission focused on the task of category-agnostic pose estimation with few annotated example images. Specifically, the authors propose a novel generative-based framework named GenCape to estimate keypoints without additional textual descriptions or predefined skeletons. The authors propose an Structure-aware Variational Autoencoder to infer instance-specific adjacency matrices from support features, and also propose a Graph Transformer Decoder to progressively refine the estimated results. The experiments are conducted on a large-scale benchmark datasets, indicating the effectiveness of the proposed novel framework.", "questions": "See Weakness.", "limitations": "", "rating": 8, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 3, "strengths": "1. The task of category-agnostic pose estimation is interesting and fundmental for extending the category number of pose estimation.\n\n2. The idea of using generative framework is reasonable and makes sense.\n\n3. The proposed Structure-aware Variational Autoencoder and Compositional Graph Transfer are novel and effective to model the pose structure information.\n\n4. The performances of proposed framework are shown on large-scale benchmark dataset, and outperform SOTA dramatically.\n\n5. The experimental analyses are comprehensive and clear.", "weaknesses": "1. As discussed in Line 69-74, the support images may contain severe occlusions or incomplete annotations, and how does the proposed method address this issue? E.g., the query image has 2 occluded keypoints, while the support image has another 3 occluded keypoints. Can the proposed method estimate all the visible keypoints?\n\n2. What's the complexity of proposed framework? It seems to be about O(M^2). Is the proposed method cost-effective?\n\n3. Can the proposed method produce diverse results based on VAE sampling? How to understand the contradiction of diversity v.s. consistency in the proposed VAE-based method?\n\n4. Closely related works are missing in the related works.&#x20;\n\n   > 1. @inproceedings{chen2025weakshot, title={Weak-shot Keypoint Estimation via Keyness and Correspondence Transfer}, author={Chen, Junjie and Luo, Zeyu and Liu, Zezheng and Jiang, Wenhui and Li, Niu and Fang, Yuming}, booktitle={The Thirty-ninth Annual Conference on Neural Information Processing Systems}, year={2025} }\n   >\n   > 2. @inproceedings{lu2024openkd,   title={Openkd: Opening prompt diversity for zero-and few-shot keypoint detection},   author={Lu, Changsheng and Liu, Zheyuan and Koniusz, Piotr},   booktitle={European Conference on Computer Vision},  year={2024} }", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "ksPHZHo7N2", "reviewer": "Reviewer_E4Zn", "summary": "The paper suggests a novel CAPE method, which utilizes predicted graph structure for enhanced keypoint localization accuracy.\nThe method uses a graph VAE formulation to predict the graph, and further implements it iteratively within each decoder layer.\nUsing CGT, several sampled graphs are combined into a query-aware graph structure that aids in localization.\nThe authors show competitive results on the MP100 dataset.", "questions": "- CGT - the adjacency matrices are sampled using the predicted mean and variance. Thus, it's not clear to me why each sample has its own mean and variance values (line 263), given that they are sampled from the same distribution.\nThis is further shown in equation 9, where alpha_n is not dependent on n at all.\n\n- See weaknesses for other questions.\n\n I'm willing to raise my score if the authors address my concerns.", "limitations": "", "rating": 4, "confidence": 5, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "- The paper suggests a novel method that deals with a limitation of recent graph-based methods.\n- The paper is well written, and the proposed solution looks solid and practical.\n- SOTA results compared to other CAPE methods on the MP100 dataset.", "weaknesses": "- Using only Fs to predict the adjacency matrix suggests that the structure information is embedded in Fs in the first place.\nAs self-attention can be seen as an all-to-all information sharing mechanism, an explanation of why self-attention can't learn the relevant connections between keypoints should be added or even proven.\nSpecifically, the authors should explain how the current i-SVAE design adds to the self-attention already in the decoder.\n\n- Iterative Graph Prediction - The suggested method works iteratively, predicting a different adjacency matrix for each decoder layer. \nAn ablation study, showing why using a different adjacency matrix in each decoder layer, compared to using only one predicted adjacency matrix (using the output features of the encoder, for example), should be presented to support the iterative superiority claim.\n\n- Qualitative skeleton visualization - Figure 3 is hard to understand. \nIt would be helpful to add the skeleton visualizations on top of the images, and not only show the adjacency matrix.\nMaybe the width or opacity can correspond to the weight. It is crucial to make it easier to understand what structure is actually learned.\n\nSmall Note:\n- Figures 4 and 5 label your method as AutoCape instead of GenCape.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "DDd0ZYjg7G", "reviewer": "Reviewer_5bdU", "summary": "This works suggests solving CAPE by progressivley inferring instance-specific keypoint relationships from the support, instead of using predefined annotated adjacency matrices. The authors also introduce the Compositional Graph Transfer module that aids with incoroporating the query features, thus allowing for less reliance on inferred keypoint relationships from the support. This makes the model more robust to occlusions and discepencies between the support and query. The new GenCape approach is tested on the known MP-100 benchamrk, achieving SOTA results.", "questions": "You mentioned Text Graph Support as an approach for CAPE. Will fusing text with your approach might increase performance? Could you hint on how whould you incorporate text as a future work with your approach (maybe also infer it from the support?).", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 3, "contribution": 3, "strengths": "1. The paper is written in a clear language that was easy to follow.\n\n2. While predicting the keypoints relations from the data is not new, the novel i-SVAE and CGT components suggest some interesting insights that might interest the CAPE community.\n\n3. The suggested approach achieves SOTA while dropping the need for predefined annotated data (keypoint connectivity) that was used by previous methods.\n\n4. Other than the main experiment in Table 1, the design choices are justified in the ablations conudcted (Table 4, Table 5 and Table 6).", "weaknesses": "My main issues are with the presentation, not with the method. After resolving these issues, I would positively consider increasing my rating.\n\n1. The technical text (mostly) in the Methods section:\n\nLine 157: M_C is not defined in the right place. Move it to this sentence.\n\nLine 179: remove 1 between.\n\nLines 190-195: i-SVAE also infers graphs from the support. And as you mention, there is sometimes a discrepency between the support and the query. So I'm not sure that i-SVAE alone will solve the issue mentioned in these lines. However, i-SVAE combined with CFG will.\n\nLine 208: F_s^(l-1) is not defined properly - what is its value where l=1?\n\nLine 212: in the second row of Equation 1, should this be F_s^(l-1) or F_s^(l)?\n\nEquations 3 and 4: A^~(l) is defined twice?\n\nEquation 6: F_s^(l) is in the input and output\n\nLine 244: A^~(l) - different notation compared to Equation 3. Should be/not be in bold?\n\nLine 248: This is not clear. Do keypoint locations are predicted in each layer? Each layer of what? the Graph Transformer Decoder? Clarify what is the output of each layer in the Graph Transformer Decoder.\n\nLine 275: missing ')' in mu^(l \n\nLine 377: \"More detailed comparisons.\": should be a sperate paragraph?\n\n2. Figures:\n\nFigure 2: Consider adding CGT to Figure 2 (A^l_fused is not enough to easily follow).\n\nFigure 3: It is challenging to interpret the adjacency matrices. Consider showing the “best” links from the adjacency matrix as colored edges in your prediction.\n\nFigure 4: last column is AutoCape.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "bPP8Dw76Ke", "reviewer": "Reviewer_cNF7", "summary": "This paper introduces GenCape, a novel generative framework for Category-Agnostic Pose Estimation (CAPE) that learns structural relationships directly from support images. The key innovation lies in automatically inferring keypoint connectivity patterns (soft adjacency matrices) without requiring predefined skeleton graphs, keypoint identifiers, or text descriptions. The framework comprises two main components: The Iterative Structure-aware Variational Autoencoder (i-SVAE) learns instance-specific graph structures from support features using variational inference, with iterative refinement across decoder layers. The Compositional Graph Transfer (CGT) module then dynamically combines multiple graph hypotheses through Bayesian fusion and query-guided attention mechanisms.", "questions": "Figures 4 and 5 reveal some localization errors. What is the primary cause of these errors, structural inference failures or visual feature ambiguity? \n\nCould the authors provide quantitative analysis of these failure modes? \n\nWhat is the method's robustness to scale variations, cropping, and other common transformations? \n\nHave additional evaluation metrics beyond PCK been considered, such as AUC or other standard pose estimation metrics?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "This is the first CAPE method to achieve fully automatic learning of structural relationships from image support sets, removing the need for predefined skeletons, keypoint IDs, or text descriptions, which enhances both generality and practical deployment. The i-SVAE approach models structural uncertainty through variational inference, demonstrating superior robustness compared to discriminative methods like SDPNet, particularly when handling support-query mismatches or occlusion scenarios. The method consistently outperforms various baselines on MP-100, with particularly notable advantages under strict thresholds (e.g., PCK@0.05).", "weaknesses": "1. While the paper claims to evaluate cross-supercategory generalization on MP-100, the definition of supercategories appears inconsistent with the original MP-100 benchmark and prior CAPE literature (e.g., CapeFormer).\nThe original MP-100 dataset is widely understood to group categories into four high-level semantic domains: human body, human/animal face, vehicle, and furniture. However, this work instead uses a finer-grained 8-supercategory split (e.g., separating Felidae, Canidae, Ursidae as distinct supercategories), which blurs the line between \"cross-category\" and \"cross-subcategory\" generalization. For instance, transferring from Felidae to Ursidae involves structurally similar quadruped animals with comparable keypoint layouts—this is arguably intra-domain transfer, not the more challenging cross-domain shift (e.g., chair → human) that truly tests category-agnostic capability. Worse, the paper does not include any cross-domain transfer between the canonical four domains (e.g., furniture → human body). This omission is critical, as if the method cannot generalize from chair to person, its claim of \"structure-inductive\" modeling is significantly weakened.\n\n2. The paper does not provide any comparison of computational efficiency, such as inference time, FLOPs, model size, or throughput, against baseline methods like GraphCape or CapeFormer. While it introduces additional modules (i-SVAE and CGT) that likely increase computational cost, no quantitative analysis or efficiency trade-offs are reported.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)