Reference Grounded Skill Discovery
Authors: Seungeun Rho
Aaron Trinh
Danfei Xu
Sehoon Ha
Venue: ICLR 2026 Poster (2026)
Keywords: Skill Discovery
Imitation Learning
Unsupervised RL
Reinforcement Learning
Motion Imitation
Primary area: reinforcement learning
TL;DR: We present a novel skill discovery algorithm that scales to high-DoF agents using reference motion guidance.

Abstract: Scaling unsupervised skill discovery algorithms to high-DoF agents remains challenging. As dimensionality increases, the exploration space grows exponentially, while the manifold of meaningful skills remains limited. Therefore, semantic meaningfulness becomes essential to effectively guide exploration in high-dimensional spaces. In this work, we present **Reference-Grounded Skill Discovery (RGSD)**, a novel algorithm that grounds skill discovery in a semantically meaningful latent space using reference data. RGSD first performs contrastive pretraining to embed motions on a unit hypersphere, clustering each reference trajectory into a distinct direction. This grounding enables skill discovery to simultaneously involve both imitation of reference behaviors and the discovery of semantically related diverse behaviors.
On a simulated SMPL humanoid with $359$-D observations and $69$-D actions, RGSD successfully imitates skills such as walking, running, punching, and sidestepping, while also discover variations of these behaviors. In downstream locomotion tasks, RGSD leverages the discovered skills to faithfully satisfy user-specified style commands and outperforms imitation-learning baselines, which often fail to maintain the commanded style.

Published as a conference paper at ICLR 2026
REFERENCE GROUNDED SKILL DISCOVERY
Seungeun Rho, Aaron Trinh, Danfei Xu, Sehoon Ha
School of Interactive Computing Georgia Institute of Technology
Atlanta, GA 30332, USA
{srho31, at atrinh31, danfei, sehoonhal@gateeh...
ABSTRACT
Scaling unsupervised skill discovery algorithms to high-DoF agents remains chal- lenging. As dimensionality increases, the exploration space grows exponentially, while the manifold of meaningful skills remains limited. Therefore, seman- tic meaningfulness becomes essential to effectively guide exploration high- dimensional spaces. In this work, present Geference-Grounded Skill Dis- covery [ novel algorithm that grounds skill discovery in a semantically meaningful latent space using reference data. RGSD first performs contrastive pretraining to embed motions on a unit hypersphere, clustering each reference trajectory into distinct direction. This grounding enables skill discovery si- multaneously involve both imitation reference behaviors and the discovery semantically related diverse behaviors. On simulated SMPL humanoid with 359-D observations and 69-D actions, RGSD successfully imitates skills such as walking, running, punching, and sidestepping, while also discover variations of these behaviors. In downstream locomotion tasks, RGSD leverages the discov- ered skills to faithfully satisfy user-specified style commands and outperforms imitation-learning baselines, which often fail to maintain the commanded style.
INTRODUCTION
METRA
RGSD
(Ours)
Figure 1: Comparison of skills learned by METRA and RGSD, To our knowledge, this is the first work to discover structured skills in high-DoF systems by grounding them in reference motions.
The ultimate goal of unsupervised skill discovery acquire set of reusable skills that can be ap- plied to arbitrary downstream tasks. Achieving this goal requires the learned skills to satisfy two key desiderata: diversity and semantic meaningfulness.. First, the skill set should be sufficiently diverse to cover the wide distribution of possible downstream tasks. For example, humanoid agent should be equipped with both turn-lefi and curn-right skills; if only learns wa k rd, it will fail on general locomotion tasks, such goal reaching heading. Second, the discovered skills must be semantically meaningful, as downstream tasks are typically defined in semantic terms. For example, consider set of unstructured humanoid behaviors such as independent vibrating or
Videos seungeunrho. thub..
thub. htpprojjctt s/ /RGSI

---

Published as a conference paper at ICLR 2026
jittering of individual body parts, Although these behaviors may be diverse, it is difficult to envision realistic tasks for which such skills would be useful. Therefore, is essential that discovered skills not only diverse, but also structured and semantically interpretable.
Recent progress in skill discovery has demonstrated the ability to learn skills that satisfy both desiderata, particularly in relatively low
(DoF) systems such benchmark en- vironments (Eyenbacch et al.] 2018 Gregor et al., 2016; Laskin et al. (022). quadrupeds (Cheng
[025; Cathomen al. 2025), or simple robotic ma- 2023aa] Rho et al. (024). However, scaling unsupervised skill discovery to high-DoF agents remains an open challenge. As the DoF increases, the exploration space grows exponentially, while the portion of the semantically meaningful manifold remains rel- atively small. This mismatch renders skill discovery in high-DoF systems particularly difficult. As shown in Fig. [1] skills learned by the state-of-the-arr unsupervised skill discovery algorithm, ME- TRA (Park et al.] (023b). fail to yield meaningful behaviors. Joints move randomly, producing highly unstructured motions in which arms, legs, torso, and head move independently and arbitrar- ily. This highlights the necessity of a mechanism that grounds skill discovery within a meaningful manifold.
al. 2024 Atanassov et al.) 2024 Rho
nipulators with grippers (Park et al.
Our key insight is that to tame the curse of dimensionality in high-DoF skill discovery, we need to construct semantically meaningful skill latent space a priori and constrain exploration within this space. In this paper, we propose Reference-Groundee Skill Discovery (RGSD), a simple yet effec- tive algorithm that achieves this by leveraging reference data to ground the latent skill space before exploration begins. Building upon DIAYN (Eysenbach et al. 2018). RGSD first uses contrastive learning on reference trajectories to embed motions on a unit hypersphere, where each reference behavior corresponds to a distinct direction. This grounding step establishes a semantically mean- ingful manifold that constrains subsequent exploration. RGSD then utilizes this pre-structured latent space to simultaneously imitate reference skills and discover novel behaviors that are semantically related to the references. This two-stage approach is analogous to recent large language model train- ing regimes (Guo et al.) 2025 Yang et al. 2025). where pretraining with self-supervised objectives establishes a meaningful exploration space before reinforcement learning fine-tuning. As a result, RGSD not only reproduces motions present in the reference data but also discovers new skills that emerge as coherent variations of existing ones, rather than arbitrary high-dimensional movements.
We empirically demonstrate that RGSD discovers structured skills in humanoid SMPL agent with a diseeimensii observation space and a -i-dimensional action space. RGSD successfully imi- tates motions such walking, running, side-stepping, punching, and moving backward, while also discovering related novel skills. Furthermore, it achieves superior performance on downstream tasks compared to both pure skill-discovery and imitation-based skill acquisition baselines.
The main contributions of this paper are as follows: 1) We propose a novel skill discovery algorithm
that scales high-DoF agents by grounding the latent space with reference data. 2) We empiri- cally demonstrate that RGSD discovers both diverse and structured motions on a 69-DoF SMPL humanoid and achieves superior downstream task performance. 3) We provide a theoretical proof establishing the validity of the proposed reward as a legitimate imitation signal. 4) We offer insights into why mutual information- dependency- based methods fail to extend in the same way.
methods integrate well with our approach, whereas Wasserstein
2 RELATED WORKS
Our work can be framed "imitation for discovery," because it performs imitation based on refer- ence data to discover semantically similar yet novel behaviors. Accordingly, we first review prior work on pure unsupervised skill discovery, and then discuss imitation-based skill acquisition approaches that also learn skills by relying on reference data.
2.1 UNSUPERVISED SKILL DISCOVERY
The most widely adopted approach for acquiring diverse skills is based on maximizing the mutual information (MI) I(S: Z2) between latent variable and the agent's states (Gregor et al. 2016
Eysenbach et al. 2018 Hansen et al., 2019 Liu & Abbeel 2021a Laskin et al. 2022). However, because MI is typically estimated via KI. divergence, it sufters from key limitation: even minor
2

---

Published as a conference paper at ICLR 2026
differences between skills can fully maximize the objective, long the discriminator dis- tinguish them. To address this, methods based on the Wasserstein Dependency Measure (WDM, Ozair al. (2019)) have been proposed (He et al., 2022; Park et al. 2021; 2023b), which explicitly seek maximal differences between skills. Nevertheless less, our method builds on MI-based approaches. and in Section 6 we explain why extending WDM-based algorithms with idea faces inherent challenges. Another relevant line of work is CIC (Laskin et al.) 2022), which, like our approach, employs contrastive learning. The key difference is that CIC optimizes contrastive objective on online data gathered by the agent, leaving it with the same scaling difficulties in high-Dol -ooF systems.
More recently, approaches such as LGSD (Rho et al. 2024) and DoDont (Kim al., 2024) have explored injecting guidance from LLMs or videos to semantically ground skill discovery. While motivated by similar goal, these methods still struggle guidance is provided only at high level. In contrast, our method leverages reference data directly, enabling a more effective grounding mechanism that scales to complex, high-DoF agents.
scale to high-DoF systems, as their
2.2
MIITATIONBASSER SKILL ACQUISITION
Imitation learning offers another major paradigm for acquiring repertoire skills. A common ap- proach is based on GAIL (Ho & Ermon (016)) which uses adversarial training to align the marginal state distribution of the learned policy with that of expert demonstrations. Building on this idea, ASE (Peng et al.) 2022) learns controllable skill embedding from large-scale human motion data using a GAlL-style objective. CALM (Tessler et al. (023)) mitigates the mode collapse observed in ASE by incorporating conditional discriminator and motion encoder, enabling skill generation conditioned on specific reference motions. More recently, Meta- Motivo (Tirinzoni et al. (025) com-
bines GAIL with forward-bras backwardd representation learning (Blier al. 2021 Touati & Ollivier 2021 to extract a skill set tailored for zero-shot RI..
In contrast to these imitation-based approaches, highlight key conceptual difference: imitation objectives aim to match the state visitation measure of the demonstrations, whereas RGSD is discovery algorithm that explicitly encourages visiting states not present from the dataset. This fundamental difference allows RGSD to discover a wider range of skills, as demonstrated in our experiments.
PRELIMINARIES
Unsupervised skill discovery can be formulated as reward-free Markov Decision Process (MDP) M == AS.A, "T.,A,,P,,10 7), which consists of the state space S, the action space A, the transition dynamics p Pr(s' s, a) the initial state distribution Po and the discount factor 7. The objective is to associate latent skill vector with the states visited by the skill-conditioned policy Tt 8, 2). such that different & induce distinct behaviors. Typically, 2 ~ p(2) is sampled from fixed prior at the beginning of each episode and held constant throughout.
A common formulation is to maximize the mutual information (MI) I(S; Z) between skills and visited states. Prior work (Grregor et al.) 2016) |Eysenbach al.) 2018:
Laskin et al. 2022
Liu &
Abbeel 20216) leverages the decomposition 2(8;2) H(s) H(s 2) = H(2) 7 8).
Because H( s) is intractable, [Eysenbach et al. (2018) introduced neural encoder (2(s)) to
obtain a variational lower bound G(O, Φ) of the skill discovery objective F(O):
F(0)= = I(s;)) + H(aa 8,
> Ex=p(() (()) logp(>) + log (o (=) + H[To s,, 2 G(0,
where H[-] denotes Shannon entropy.
Optimization proceeds as:
ua maximize via RL with reward rz log (ogp(=) log qa(z (s,
go : maximize log (s(2 s), 8 ~ T(- s),
(2)
along with the entropy bonus for the policy. Intuitively, de learns to infer which skill & led to a given state, while my is reinforced visit states that make z easily identifiable.
3

---

Published as a conference paper at ICLR 2026
Pretrsining
Imitation
Pelicy
Enviioament
Freale
(s st*
Nr
Innoder
Data
Ryef
minimize
Discovery
Encoddr
minimiize
210
Policy
Environment
Ti.
3
Figure 2: We present the overall training pipeline RGSD. It starts with contrastive pretraining of an encoder using reference motions, followed by parallel training of imitation and discovery.
4 REFERENCE GROUNDED SKILL DISCOVERY
At high level, RGSD approaches skill discovery in the reverse order of conventional methods. Standard skill discovery begins with policy exploration, and only afterward induces latent space to enforce different regions represent different behaviors. In contrast, we start with a dataset of target behaviors wish to capture and first embed them into unit hyperspherical latent space without any environmental interaction. The ideal latent space would have each motion represented by single directional vector 2, with clear separation between different motions. This will turn exploration of the latent space into the discovery of structured skills in the state space. To achieve this, we employ contrastive learning.
Once the latent space is grounded with motion representations, we proceed with skill discovery in this reference-equipped space. Here, we exploit the geometry of the hypersphere: sampling along directions aligned with reference vectors triggers imitation, while sampling in-between reference directions drives the discovery novel skills. This design enables semantically meaningful expan- sion of known behaviors and structured exploration of new ones.
In the following subsections, we detail how motion priors are injected into the latent space and how imitation and discovery are jointly achieved within this framework.
4.1 PRETRAINING: GROUNDING LATENT SPACE ON REFERENCE MOTIONS
The goal of pretraining is associate each motion with latent vector 2, enabling the skill- conditioned policy (-|ss, 2) to reproduce the corresponding trajectory. To achieve this, we place each motion at a point the hypersphere while ensuring separation between different motions. We employ contrastive learning: positive pairs are sampled from the same trajectory, and negative pairs from different trajectories. This encourages motions cluster tightly around unique latent directions, forming a well-structured space for downstream imitation and discovery.
More coneretely, we train encoder (z s) that maps state probability distribution over latent vector z Z, where {u k || 1}. We model (a 8) with von Mises-Fisher (yMF) distribution:
go(z (s) x exp (xp(kuof (s)T
(3)
where (s) is the mean direction parameterized by neural network ó, and k is the concentration parameter. We fix n. constant during training.
Training is performed with contrastive learning on a reference dataset M = {m, - where each m; is reference trajectory consisting of sequence of states. During pretraining, motion m Uniform(MM (M), then select a pair of states (s",s" s+) from m, with s" as the anchor and + as the positive sample. A negative sample s- is drawn from M \ (m). Given the anchor sa the
first sample

---

Published as a conference paper at ICLR 2026
positive stt and negatives {s, }. we embed them
uo(s"), zt = =o((s)) 2j = Ho(s)),
and optimize the InfoNCE loss (Ord et al. 2018):
exp(sim(2",, (xp(sim(22,2*//TT
LInfONCE llg
2aimmz",=+)/7) + Σ, exp (2 2
Here, sim( 23) 2 2j denotes cosine similarity since all vectors lie on the unit hypersphere, and T = 1/s is the temperature. A detailed derivation is provided in Appendix
As depicted in pretrainn.,. the mapping of motions in latent space is spread out before training. However when pretraining is completed, we end up with a within-motion alignment, meaning that the mapping of every state E m in each motion m points in exactly the same direction. This characteristic is important for the following section; therefore, for completeness, we provide a proof of this claim in Appendix C
4.2 IMITATION REFERENCE SKILLS
After pretraining, we freeze qs and proceed to the second phase, where imitation and discovery are trained in parallel. Interestingly, both processes rely on the same reward term defined in Eq. [ We first show that this reward can be repurposed as an imitation reward when conditioned on the mo- tion's 's embedding of the pretrained encoder q6. Specifically, we sample a motion m Uniform(M) and compute its embedding vector 2m as the average its state embeddings:
Σ Mo(s),
(5)
8Em
where is the length of motion m. At theoretical optimum, 2m should be aligned with the embedding of any individual state m.
Conditioning the policy on 2m. i.e. executing rollouts with (-|ss, zm). does not by itself repro- duce m. However, reinforcement learning with the DIAYN reward (Eq. drives (T-|s, 2m) toward imitating m. Substituting in the vMF formulation (Bq. 3 gives
r(s, Zm) log p(=) log qo (am 1))
= C + K 1 (8) Zm:
where the prior p(2) and concentration K fixed, and is frozen. C' is a constant term indepen- dent of and žm- Intuitively, this reward depends on the angle between (s(ss and 2m, effectively measuring the similarity between the state visited by the agent and the states in the reference mo- tion. Moreover, initializing 8p from m provides an aligned starting point; as trajectories deviate, the cosine alignment decreases and the reward diminishes.
This formulation can be viewed as feature-based imitation with structured representations, as pro- posed by Li et al. (2025). Unlike DeepMimic (Peng et al. 2018) or MaskedMimic (Tessler al., 2024). which reward joint-level similarity, our method evaluates similarity in a learned latent space.
Guarantee as imitation reward For the proposed reward serve as a valid imitation objec- tive, it must satisfy two key conditions: (1) it should attain its optimum when the agent visits the exact states of the reference motion, and (2) the reward landscape around the reference states should be locally quasi-concave, ensuring that deviations from the reference lead to a monotonic decrease the reward within a neighborhood of the optimum. We claim that, under the assumption of "per- fect alignment within motion" introduced in the previous section, our reward formulation in Eq. satisfies both conditions, and we provide the proof in Appendix D.I To exploit this local concavity in practice, apply early termination: whenever the agent deviates from the reference motion be- yond specified threshold measured by cartesian error, the episode is terminated. This mechanism guarantees that the reward functions a legitimate imitation objective.
5

---

Published as a conference paper at ICLR 2026
Run
ats, Arun
Z
1
Rback
Backward
Zynuh
[Ts, (TS, #ban))
Zpunen
(d) Punch xtS, (punco))
at 22n))
(c) Sidestep
Figure 3: Example skills. RGSD generates diverse behaviors when conditioned different la- tent vectors. The figure shows motions from single policy conditioned on distinct latent vectors sampled near the embedding of the (a) running, (b) backward, (c) sidestepping, and (d) punching motion.
4.3 DISCOVERY OF NEW SKILLS
The discovery process follows DIAYN, but with several key differences. First, to protect the learned latent space during discovery, we initialize separate encoder 4/g from the frozen i6 and continue propagating gradients from minimizing KI-divergence between 1' and qp- Second, train discov- ery in parallel with imitation, so that the shared policy and value function transfers knowledge about high fidelity behavior from imitations into discovery. Note that these two processes share same form of reward function and latent space, these shared components can be optimized in stable man- ner. Third, we adopt reference state initialization (RSI), which samples initial states directly from the reference motions. RSI prevents the emergent of disjoint skill sets by ensuring that imitation and discovery operate over overlapping state distributions.
In detail, imitation and discovery trained in parallel, with ratio parameter pt
M (m),
with probability p, with probability 1
k/|lal|, k~ N(0,
where is the motion sampled by RSI. The full algorithmic details are provided in Appendix
5 EXPERIMENTAL RESULTS AND DISCUSSION
In this section, we investigate four questions: (1) Can RGSD imitate reference motions with high fidelity? (2) Can RGSD discover novel skills that are still semantically related the references? (3) Can we control degree of diversity at test time? (4) Can the learned skills be effectively leveraged for downstream tasks?
Baselines. We compare RGSD against both unsupervised skill discovery (USD) methods and im- itation learning (II.) based skill acquisition methods. To assess the impact of reference guidance, we include pure USD baselines. Specifically, we compare against DIAYN (Eysenbach et al. 2018) and METRA PPar et al.) 2023b). DIAYN, which forms the backbone of RGSD, also serves as ablation without reference grounding, while METRA represents a disstaccemaxximizatinn approach to USD that explicitly seeks maximal differences between skills.
For the IL baselines, we consider ASE, CALM, and Meta-Motivo. all of which utilize a GAlL-style objective (Ho & Ermon 2016) for imitation. ASE builds InfoGAIL (Li et al. 2017). combining informaaiooonnnoonnn maximization with an adversarial imitation reward, making it the closest base- line that jointly addresses skill discovery and imitation. CALM, variant of ASE, further augments

---

Published as a conference paper at ICLR 2026
Table 1: Cartesian ERR(cm) and FID scores for imitation across tasks. Lower values are better.
Walk
Run
Sidestep
Backward
Punch
ERR
FID
ERR
FID
ERR
FID
ERR
FID
ERR
FID
DIAYN
46.7
70.1
52.8
132.0
27.4
23.7
36.7
72.8
50.7
53.2
METRA
42.0
67.6
51.8
140.3
44.7
32.8
47.4
75.7
51.5
40.9
ASE
8.2
2.9
16.4
11.3
10.3
22.2
11.6
62.6
9.0
4.9
CALM
7.2
1.4
15.0
13.9
11.8
3.2
10.1
45.8
9.2
4.2
Meta-Motivo
10.9
1.7
15.4
4.9
11.8
1.4
8.6
2.1
8.1
2.8
RGSD (Ours)
7.4
4.7
7.7
9.4
6.7
8.0
6.7
4.0
7.7
8.8
the imitation objective using a motion-condiiioned discriminator. Meta-Motivo integrates imitation rewards with forward-backward representation learning.
We emphasize that while these methods rely on GAlL-based imitation rewards, our approach em- ploys a novel imitation reward derived from the DIAYN objective. To ensure a clean algorithmic comparison, we remove additional engineering techniques used in the original ASE and CALM implementations. The exact modifications are described Appendix G.2
Experimental setup. We selected 20 reference motions from the ACCAD dataset. These mo- tions are categorized into five tasks: walk, run, sidestep. backward, and punch, with each task containing 2 to 6 relevant motions. A complete list of motions is provided in Appendix G.3 Training was conducted in the GPU-based simulator Isaac Gym (Makoviychuk ing PPO (Schulman et al., 2017) as the RL algorithm. The full set of hyperparameters is provided in Appendix G.4] To ensure stable training, all methods employed an early termination condition: whenever the robot fell, the episode was terminated.
al.) (021). us-
5.1 EVALUATION OF IMITATION
We first assess how well RGSD reproduces reference motions. For each motion, the agent is ini- tialized with the first state su of the motion. Then condition the policy (so: 2m), where 2m computed with Eq. [ We generate 500 trajectories per each motion and compared against the reference compute two metries:
Cartesian error: the average & distance between corresponding body-part positions per frame, averaged over the trajectory.
Motion FID: the Fréchet distance between Gaussian feature distributions fitted to reference motions and generated motions. Lower FID indicates better naturalness.
For CALM and Meta-Motivo, selecting the right latent that represents each motion is straightforward since it also includes motion encoders. For methods without encoders, uniformly sample 500 latent vectors, select the one that minimizes Cartesian error, and re-evaluate using this vector to ensure fairness. We found that 500 samples were sufficient, increasing the number further did yield noticeable improvements.
RGSD achieves high-fidelity imitation. As shown in Table RGSD achieves low Cartesian error while Meta-Motivo achieves low FID scores across most motions. In contrast, pure USD baselines fail discover skills that closely resemble the reference dataset. While DIAYN has demonstrated the ability to discover basic locomotion behaviors in low DoF ranging from 3 to 6 environments such as HalfCheetah, Ant, and Hopper (Brockman et al. (0166), they struggle with the 69-DoF SMPL agent, producing behaviors that are not semantically meaningful. This emphasizes the impact of reference guidance.
When compared to Metivo, the results highlight a trade-off between fidelity and naturalness. Meta-Motivo achieves lower FID scores 4 out of tasks, indicating that its motions generally appear more natural. However, RGSD outperforms Meta-Motivo in Cartesian enror 4 out of tasks, demonstrating higher trajectory fidelity. This contrast reflects a key difference in objective design: Meta-Motivo employs accupancy-matehing objective at the motion level, leading to smoother but less precise reproductions, whereas RGSD relies on frame-level similarity rewards, yielding high fidelity motions.

---

Published as a conference paper at ICLR 2026
RGSD(OUrr)
Meta-Motive
CALM
ASE
DIAYN
METRA
Walk (turn right by 909)
FID
FID: L.
FID
FID:
57.5
FID: $1.3
Backward (while running))
FIB: 13.3
FID: 15.0
FID: 32.4
FID: A2.
FE1:: 61.
FID: 66.9
Run (tr left by 135")
FID:
FIL::
FIL:: 26.7
FID: 118
FIl: 91.9
Sidestep to the right)
(m))
Reference traj.
Figure 4: view trajectories of the robot's base when conditioned on latent vectors sampled from the neighborhood of each motion embedding. For each method, we visualize 150 trajectories.
5.2 EVALUATION OF DISCOVERED SKILLS
Now we evaluate whether RGSD can discover skills that are semantically similar to existing ones while also producing novel variations. Semantic similarity is enforced by sampling latent vectors from a local neighborhood around each motion embedding using a vMF distribution with the fixed concentration parameter at 50.
RGSD can discover semantically similar yet novel behaviors. Fig. illlstrates behaviors gen- erated by the policy when conditioned on sampled latent vectors. The method produces diverse behaviors that remain semantically consistent with the reference motions. For example, while the reference dataset contains only a single sidestepping motion to the right, RGSD discovers skills that preserve the sidestepping style but introduce diverse degree of turns, enabling sidestepping in multiple directions. Similarly, although the dataset includes a punching motion aimed at a fixed target, our method extends this behavior to generate punches toward variety of target positions.
To compare against baselines, we visualize the 150 top-view trajectories of the robot's 's base for each method in Fig. alongside the trajectories from the reference motions. Trajectories produced by RGSD remain tightly clustered around the references, indicating that the policy captures the intended motion structure. In contrast, the baseline methods often produce degenerate or drifting trajectories, revealing their inability to preserve the characteristics of the original movements.
These observations are further supported by the FID scores. For three out of four motions, RGSD achieves the lowest FID. While Meta-Motivo typically achieves the second-lowest FID, its generated trajectories still fail to remain centered around the references. CALM shows particularly degraded performance: its FID increases from 1.4 to 15.5 for walking and from 13.9 to 26.7 for running, indicating difficulty in producing diverse yet high-fidelity behaviors. In contrast, the FID scores of RGSD remain stable, demonstrating that the method successfully preserves motion style while enabling variation.
We attribute this difference to the training setup. In both Meta-Motivo and CALM, the policy is always conditioned directly the motion embedding and therefore never encounters nearby latent vectors during training. By contrast, RGSD explicitly separates imitation and discovery phases: the imitation phase exposes the policy motion- reriionnemeddding latents, while the discovery phase condi- tions it on diverse samples from the latent neighborhood. Consequently, the policy learns to han- dle both exact motion embeddings and their local variations, enabling consistent and semantically meaningful skill discovery.
RGSD enables test-time control over behavioral diversity. A key advantage of RGSD is that allows users modulate the diversity of generated behaviors test time by adjusting the sampling distribution of the latent vector 2. Recall that we sample ~ vMF(z,,. k). a von Mises-Fisher dis- tribution centered at the motion embedding 2m with concentration parameter A larger s. produces
8

---

Published as a conference paper at ICLR 2026
Froestyll
Sidestepping
Backward
1.0
Ri. 20.3
FID- 100.2
Success Rate
FID 6.8
0.5
FID 17.9
Molivo
ASE
F10-41.7
0.2
FIO 10.4
FI0-9188
1k
3k
5k
2k
6k 10c
2k
Number of Updates
Figure 5: Training curves for the downstream task, along with corresponding FID scores.
samples tightly clustered around 2m- yielding behaviors that closely imitate the reference motion but exhibit lower diversity. Conversely, a smaller и. yields broader distribution, producing more diverse behaviors that may deviate further from the original motion m.
We evaluate this effect three reference motions by sampling with E {20, 100, 1000). Results are shown Fig. 6 As illustrated, increasing k results in trajectories that stay stylistically similar to the reference motion with limited variation, while decreasing produces increasingly diverse trajectories that still preserve the core semantic style.
5.3 DOWNSTREAM TASK EVALUATION
Setup. We consider the GoalReaching task with three style conditions: freestyle, sidestepping, and backward. The agent must reach a randomly spawned goal within 20 x 20m arena while adhering to the specified style. Following the setup the previous sec- tions, we first train a low-level policy Tlo (- 8,2) using the 20 reference motions for each method. We then train high-level policy Thigh 8, g), which takes the goal g as input and outputs a high- level latent action z. The corresponding low-level policy Tlo (- s, z) then produces the joint action a. During this phase, now frozen, and we optimize only Thigh.
To encourage both goal reaching and adherence to the commanded style, we adopt reward similar to Tessler et al. (2023)
exp(p-0000 -0.25 || + exp( -8 * || 2m1l))
where dgow is the distance to the goal, and e 2m denotes the cosine distance between the high-level action z and the embedding 2m of the directed motion. For the freestyle task, only the first reward term is applied. All experiments are conducted with three random seeds.
Only RGSD consistently respects the commanded style while reaching the goal. Figure re- ports the success rates and FID scores of RGSD and baseline methods over the course of training- In the freestyle setting, since no specific style is required, we evaluate only the success rate. Although all methods eventually achieve near-perfect success, RGSD converges significantly faster. In contrast, ASE frequently fails to reach the goal; we observed that it often remains stationary, exploiting the fact that its reward remains positive even without producing the desired motion.
For the s sidestepping and backward tasks, we evaluate both the success rate and the FID score, Only RGSD reliably follows the commanded style. Although CALM achieves higher success rates early in training, RGSD eventually matches it. Furtermore, CALM ultimately ignores the style command and simply runs forward. On the other hand, Meta-Motivo respects the style only under certain conditions. For example, in the backward task, it walks backward when the goal is spawned behind the agent, but when the goal appears in front, the agent decides to walk forward. In contrast, RGSD consistently adheres to the commanded style: even when the goal is spawned in front during backward task, it takes a large detour to maintain backward motion. This difference is reflected in the FID scores: Meta- Meta-oooivoo CALM, and ASE all exhibit higher FID values, indicating substantial deviations from the intended styles. We refer the reader to the accompanying video for qualitative examples.
Note that these tasks are inherently challenging because the reference motions for sidestepping and wackward running do not include any turning behaviors. To reach diverse goal locations while main-

---

Published as a conference paper at ICLR 2026
taining style fidelity, agent must discover additional skills that are semantically consistent with the style, such as backward turns at varying angles. Imitation- based baselines lack this flexibility, whereas RGSD acquires rich set of turning behaviors, enabling it to reach goals while preserving the commanded style.
UNDERSTANDING THE CHALLENGES OF EXPANDING RGSD WITH METRA
Our backbone skill discovery module is DIAYN, which relies on maximizing mutual information (MI). However, well-known limitation of MI-based objectives is that they be fully optimized even when the underlying differences between skills are minimal, as long as the discriminator can distinguish them. This raises natural question: can extend RGSD top of distance- maximization based approaches such as METRA?
To explore this, we conduct experiments with METRA-based variant of RGSD. Similar DIAYN, we first ground the latent space with reference motions while respecting the two conditions that METRA enforces. First, each motion should align with unique directional vector 2, which can be satisfied using the original contrastive loss. Second, the latent space should capture the notion of temporal distance: Vz, y E Sudj. [|x(z) - <(y)]l S 1. To enforce this, we add loss term that drives [[ő(z) <(v) toward its maximum value of 1 for adjacent state pairs. The resulting latent space, shown in Fig. places each motion along a distinct line.
However, this formulation exposes a critical issue when dealing with repetitive motions. Consider walking: the agent begins at some pose, takes a step, and eventually returns to pose nearly identical to the starting one. Because all observations are computed in the local frame, the initial state and final state ST become identical. In this case, the METRA reward (ø(TT) ((s))) collapses to zero. As result, repetitive behaviors cannot be framed as reward maximization under METRA, and more broadly, WDM- -based approaches defined in a local coordinate system are not straightforward to capture repetitive dynamics. This observation is consistent with the concurrent findings (Park let al.).
One might suspect that this problem could be bypassed by augmenting the state with additional variables, such as a time variable or global coordinates. However, this does not resolve the issue. In practice, it causes the latent space to be dominated by these added dimensions, leading the policy to focus on them rather than discovering meaningful behaviors. For brevity, we defer a more detailed discussion to Appendix H To properly represent a substantial portion repetitive motions dataset, we choose DIAYN as our backbone.
CONCLUSION
We introduced Reference-Grounded Skill Discovery (RGSD), simple yet effective framework that scales unsupervised skill discovery to high-DoF agents by grounding exploration in a semanti- cally meaningful latent space constructed from reference motions. On 69-DoF SMPL humanoid with 359-D observations, RGSD reproduces complex motions, such as walking, running, sidestep- ping, backward walking, punching, with high fidelity, and also discovers coherent variants, out- performing state-of-the-arr unsupervised discovery and imitation-based baselines on both motion metries and downstream control.
Despite these advancements, several avenues remain open for further exploration. One promising direction is to move beyond variants of individual skills toward genuinely compositional behaviors and principled interpolations that blend primitives (e.g.. "walking while punching"). Another in- teresting direction would be scaling across embodiments and datasets, with the long-term vision of building a skill foundation model for control, analogous to large language models in natural lan- guage processing. Overall, we believe our work represents the beginning of a practical recipe for scaling skill discovery in high-DoF agents through reference grounding.
10

---

Published as a conference paper at ICLR 2026
REPRODUCIBILITY STATEMENT
We have made extensive efforts to ensure the reproducibility of our work across multiple dimensions:
A complete pseudo-code description of our algorithm is provided in Appendix F
The full list of reference motions used in our experiments is detailed in Appendix G.3
Comprehensive specifications of states, actions, and hyperparameters are given in Ap- pendix G.4
Rigorous derivations and proofs of our theoretical claims are included in Appendices B D.I
ACKNOWLEDGMENTS
This research has been funded by the Industrial Technology Innovation Program (P0028404, devel- opment of a product level humanoid mobile robot for medical assistance equipped with bidirectional customizable human-robot interaction, autonomous semantic navigation, and dual-arm complex ma- nipulation capabilities using large-scale artificial intelligence models) of the Ministry of Industry, Trade and Energy of Korea. In addition, we thank Jeonghwan Kim for the discussion.
REFERENCES
OSU ACCAD. Accad motion lab system data. http:::/acaad..csueee/eeeeeeee motion laaa-labbaaa htpp/yytem data Accessed: 2025-09-20.
Vassil Atanassov, Wanming Yu, Alexander Luis Mitchell, Mark Nicholas Finean, and Ioannis Havoutis. Constrained skill discovery: Quadruped locomotion with unsupervised reinforcement learning. arXiv preprint arX1v:2410.07877, 2024.
Léonard Blier, Corentin Tallec, and Yann Ollivier. Learning successor states and goal-dependent values: A mathematical viewpoint. arXiv preprint arXiv:2101.07122. 2021.
Greg Brockman, Vicki Cheung, Ludwig Pettersson, Jonas Schneider, John Schulman, Jie Tang, and Wojciech Zaremba. Openai gym. arXiv preprint arXiv:1606.01544 2016.
Rafacl Cathomen, Mayank Mittal, Marin Vlastelica, and Marco Hutter. Divide, discover, deploy: Factorized skill learning with symmetry and style priors. arXiv preprint arXiv:2508. SCI::2508.19953, 2025.
Jin Cheng. Marin Vlastelica, Pavel Kolev, Chenhao Li, and Georg Martius. Learning diverse skills for local navigation under multi-constraint optimality. In 2024 IEEE International Conference on Robotics and Automation (ICRA), pp. 5083-5089. IEFE, 2024.
Benjamin Eysenbach, Abhishek Gupta, Julian Ibarz, and Sergey Levine. Diversity is all you need: Learning skills without a reward function. In International Conference on Learning Representa- tions, 2018.
Karol Gregor, Danilo Jimenez Rezende, and Daan Wierstra. Variational intrinsic control. arXiv preprint arXiv:1611.07530 2016.
Daya Guo, Dejian Yang. Haowei Zhang, Junxiao Song. Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-rl: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501. 12948, 2025.
Steven Hansen, Will Dabney, Andre Barreto, David Warde-Farley, Tom Van de Wiele, and Volodymyr Mnih. Fast task inference with variational intrinsic successor features. In Interna- tional Conference on Learning Representations, 2019.
Shuncheng He, Yuhang Jiang, Hongchang Zhang, Jianzhun Shao, and Xiangyang Ji. Wasserstein unsupervised reinforcement learning. Proceedings of the AAAI Conference on Artificial Intel- ligence, volume 36, pp. 6884-6892, 2022.
11

---

Published as a conference paper at ICLR 2026
Jonathan Ho and Stefano Ermon. Generative adversarial imitation learning. Advances in neural information processing systems, 29, 2016.
Hyunseung Kim, BYUNG KUN LEE, Hojoon Lee, Dongyoon Hwang, Donghu Kim, and Jaegul Choo. Do's and don'ts: Learning desirable skills with instruction videos. Advances in Neural Information Processing Systems, 37:47741-47766 2024.
Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint 1411v:1412.69880 2014.
Michael Laskin, Hao Liu, Xue Bin Peng, Denis Yarats, Aravind Rajeswaran, and Pieter Abbeel. Unsupervised reinforcement learning with contrastive intrinsic control. In Advances in Neural Information Processing Systems, 2022.
Chenhao Li, Marco Hutter, and Andreas Krause. Feature-bused vs. gan-based learning from demon- strations: When and why. arXiv preprint arXiv:2507.05906 2025.
Yunzhu Li, Jiaming Song, and Stefano Ermon. Infogail: Interpretable imitation learning from visual demonstrations. Advances in neural information processing systems, 30, 2017.
Hao Liu and Pieter Abbeel. Aps: Active pretraining with successor features. Conference on Machine Learning, pp. 673666764447 6747. PMLR, 2021a.
In International
Hao Liu and Pieter Abbeel. Behavior from the void: Unsupervised active pre-training. Advances in Neural Information Processing Systems, 34:1844415 18118999-1477,, 2021b.
Zhengyi Luo, Jinkun Cao, Alexander W. Winkler, Kris Kitani, and Weipeng Xu. Perpetual humanoid control for real-time simulated avatars. In International Conference on Computer Vision (ICCV), 2023.
Viktor Makoviychuk, Lukasz Wawrzyniak, Yunrong Guo, Michelle Lu, Kier Storey, Miles Macklin, David Hoeller, Nikita Rudin, Arthur Allshire, Ankur Handa, et al. Isaac gym: High performance gpu-based physics simulation for robot learning. arXiv preprint urXiv:2108. 10470, 2021.
Aaron van den Oord, Yazhe Li, and Oriol Vinyals. Representation learning with contrastive predic- tive coding. arXiv preprint arXiv:1807.0331 2018
Sherjil Ozair, Corey Lynch, Yoshua Bengio, Aaron Van den Oord, Sergey Levine, and Pierre Ser- manet. Wasserstein dependency measure for representation learning. Advances in Neural Infor- mation Processing Systems, 32, 2019.
Jonghae Park, Daesol Cho, Jusuk Lee, Dongseok Shim, Inkyu Jang, and Jin Kim. Periodic skill discovery. In The Thirty-ninth Annual Conference Neural Information Processing Systems.
Scohong Park, Jongwook Choi, Jackycom Kim, Honglak Lee, and Gunhee Kim. Lipschitz- constrained unsupervised skill discovery. In International Conference on Learning Represen- tations, 2021.
Scohong Park, Kimin Lee, Youngwoon Lee, and Pieter Abbeel. Controllability unsupervised skill discovery. arXiv preprint arXiv:2302.05503, 2023a.
Seohong Park, Oleh Rybkin, and Sergey Levine. Metra: Scalable unsupervised with metric-aware
abstraction. In The Twelfth International Conference on Learning Representations, 2023b.
Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel Van de Panne. Deepmimic: Example- guided deep reinforcement learning physics-based character skills. ACM Transactions On Graphics (TOG), 37(4):1-14, 2018.
Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, and Sanja Fidler. Ase: Large-scale reusable adversarial skill embeddings for physically simulated characters. ACM Transactions On Graphics (TOG), -14):11-17, 2022.
Seungeun Rho, Laura Smith, Tianyu Li, Sergey Levine, Xue Bin Peng, and Sehoon Ha. Language guided skill discovery. In The Thirteenth International Conference on Learning Representations, 2024.
12

---

Published as a conference paper at ICLR 2026
Seungeun Rho, Kartik Garg, Morgan Byrd, and Schoon Ha. Unsupervised skill discovery explo- ration for learning agile locomotion. Conference on Robot Learning, Pp. 2698.2694. PMLR, 2025.
John Schulman, Philipp Moritz, Sergey Levine, Michael Jordan, and Pieter Abbeel. High- dimensional continuous control using generalized advantage estimation. arXiv preprint arXiv:1506.024333 02438. 2015.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017.
Chen Tessler, Yoni Kasten, Yunrong Guo, Shie Mannor, Gal Chechik, and Xue Bin Peng. Calm: Conditional adversarial latent models for directable virtual characters. In ACM SIGGRAPH 2023 Conference Proceedings, pp. 1-9, 2023.
Chen Tessler, Yunrong Guo. Ofir Nabati, Gal Chechik, and Xue Bin Peng. Maskedmimic: Uni- fied physics-based character control through masked motion inpainting, ACM Transactions on Graphics (TOG), 43(6):1-21, 2024.
Andrea Tirinzoni, Ahmed Touati, Jesse Farebrother, Mateusz Guzek, Anssi Kanervisto, Yingchen Xu, Alessandro Lazaric, and Matteo Pirotta. Zero-shot whole-body humanoid control via behav- ioral foundation models. arXiv preprint arXiv 2025.
Ahmed Touati and Yann Ollivier. Learning one representation to optimize all rewards. Advances in Neural Information Processing Systems, 34:13-23, 2021.
An Yang, Anfeng Li, Baosong Yang. Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao. Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09.0 093888 2025.
13

---

Published as a conference paper at ICLR 2026
A
QUALITATIVE RESULTS FOR CONTROLLING DIVERSITY EXPERIMENT
Less diverse (x=1,000)
Medium diversity (k-1100)
diverse (k=20)
jsi hackward
FID: 25.4
Rum backward
F10: 27.2
Run hackward
FID: 45,9
Sidestep
4,2
Steestp
FID: 4.4 Sidesieg
8.D
tum 1386
135
Figure 6: Red dots indicate the agent's trajectory. From left to right, sample the latent variable 2 from increasingly wider distributions centered around the reference motion, the resulting behaviors become more diverse while still preserving the core characteristics of the original motion.
B FROM VMF ENCODER TO INFONCE LOSS
Recall the encoder defines von Mises-Fisher (vMF) density on the unit hypersphere: gp(= s) Ca(re) exp(1e (s))s z). ||ж||2 = (s) 1, where Ca(R) is the normalizing constant and x > 0 is fixed.
Given an anchor 8 and candidate set C = (==t,......... _ containing one positive and K negatives, consider the conditional probability that the positive at was generated from go(- 88) among the candidates:
(2 (5))
= Ca(x) (F(
(pos = 2+ |5* = Σ
(0 84)
exp (x)()))000)0 sEC
Σ
ZEC
Because & is fixed, (x(x)) cancels:
exp (50)
Po (pos 2 C)
(xp()10(80)TT8 + ΣØ exp (o(sa)T x25)
Maximizing the log-likelihood of the correct positive is therefore equivalent to minimizing the cross- entropy loss
CNCE log exp(1: (s)(s)T2*)))) ++) Σ exp(1 (o()))
Identifying the similarity as the dot product on the unit sphere, sim(u, v) = To (cosine similarity), and setting the temperature T 1/k, obtain the standard InfoNCE form:
exp (s"))
LinfONCE llog exp (s))sim((((s)), + Σ exp (s)m(
(8)
If instantiate the candidates by their mean directions, i.c., za us(s"), z ps(s)), ). and 27 (,77)), then the loss reduces exactly to the temperature-scales dot-product InfoNCE used in the main text (with T 1/5).
14

---

Published as a conference paper at ICLR 2026
C PROOF OF WITHIN-MOTION ALIGNMENT
We prove that under the InfoNCE objective with "same-motion" positives, the optimal encoder perfectly aligns all frames of a given motion single unit vector the sphere.
Proof. Let there be N motions. Each motion
{m1....,, mx) is sequence of states E IRd. An encoder Hs maps each state to a unit vector u = (s(s) Z.
For an anchor we draw positive E (independent) and negatives R1,..,, Rl from other motions. The InfoNCE loss for this triplet is
exp(u b/T)
f(u, (ngH) = - log
Exp(uTu/I1 + Σj=ı exp(u' no n//T)
Fix and negatives {ny}. Write u T u [-1, and C' E3 exp(u (eflunnn/2// nş/T), Then
{(s) T + log (xp(x/T)) CC..
Differentiating LHS with
ds 7 (-espls/T)+c)
dt
exp(s/T) +c) <0,
so {(s) is monotonically decreasing in Hence, the loss is minimized by maximizing u u. which at most and attained only when e = u. This implies that the angle between embeddings of frames from the same motion should be minimized, resulting in perfect alignment.
Reduction of the loss. After the alignment, for motion m we have reduces
Mm,, and equation
e//T
im
log 2/TT
Ejtm em
The total loss Σn lm now depends only the pairwise inner products Mnn Minimizing this objective pushes the vectors {Am) as far apart as geometry allows, yielding a regular simplex when M < 1.
D PROOF OF GUARANTEE AS AN IMITATION REWARD
Our goal to prove properties the reward function in Eq. 7 (i) the reward achieves its optimum when the agent visits the exact states of motion m. and (ii) the reward function is locally quasi-coneave around the states m.
D.1 PROOF OF REWARD OPTIMALITY ON MOTION STATES
Assumption. We assume that the pretraining of μış in Section mum. More concretely, for all m E M and for all {81, 82,.... St,,. C m, we have
has reached its theoretical opti-
( = (x2) = (
where mm the number of states motion m. Therefore, from Eq. 5
2m
Σ Mo(s)
Im Im = Vi.
(10)
PEm
Proof. From Eq. the reward can be written
r(s, 2m) = p(z) + log (zm
(11)
C (s) 2m:
(12)
15

---

Published as a conference paper at ICLR 2026
where C' and g are constants that do not affect the optimal point. Discarding the constants, write
r(s, 2m) (1p(()), 2m))
Substituting us(s) with Eq. 10 we obtain
vs E m, r(8, 2m) = (Mp(s), 2m) (Zm= 2m) 1.
Since r(s, 2m) is bounded within [-1, 1], this value is the global optimum. Hence, every state m achieves the global optimum.
D.2 PROOF OF LOCAL QUASI-CONCAVITY THE REWARD
Assumption. We assume from Eq. that for every 8" m have pa(s") 2m the neural network üi to be piecewise linear in a neighborhood of s*, and normalization is applied the output so that pa(s)) (s)ss/ (
We designed
lies in unit hypersphere for all E S.
Lemma. s* E S be a state a motion that satisfies ua = (/1i)/)) (s))222 2m- Then r(s) (1)( 2m) is quasi-concave in neighborhood of s*
Proof. To show that r(s) is quasi-concave in a neighborhood of s", we want to show that inside the neighborhood (s)),, the superlevel set (r) {s r((s) 2 a) is convex.
To show this, we first check the convexity of the superlevel set in the latent space. Then, by leverag- ing the fact that the preimage of an affine transform preserves convexity, we conclude the convexity of the superlevel set in the state space.
Convexity of superlevel set in latent space. We define function in latent space f(z) -T ==///|=/22 This satisfies (()(())) = r(s). We now show the convexity the superlevel set of f:
S.(f) = =2: f(=) = 2m 2
= { ZT 2 a al|z||2)
To show its convexity, for V2 22 E Sf) and E [0, 1). we plug in + to, + (1 - t)z2:
=T2( + (1 t)2²) = t + (1 t)
2 ta||>ı||2 + (1 - t)a
alltzzlle ||(1 +)
tzi + (1 - 1)22
Therefore, t2, + (1 t)22 E S.(f), so S.) is convex.
Convexity of superlevel set in state space. Here, we use the fact that the preimage of an affine transform also preserves convexity: when (ss) = As + b is affine, then T T-1(S.(f) is convex.
The superlevel set of r(s) = f(Po(s)) is defined as:
Sa(r) = (s r(s) 2a} = {s= f(P 2 a.
Since the affine property only applies near the neighborhood of s* can define an open ball around 8 that satisfies such a condition:
(s)
As +
for all E B.(s") {s: ||s 8'112 <e).
16

---

Published as a conference paper at ICLR 2026
Restricting the domain to the ball, we have:
{s E Be(s") : r(s) 2 a = (.(())B.(s')
(s:f(i(())2a))BBe
(o.f(T(s)2aa]BBBBB
=())))((())))))(( B(())
(S-)
This set also convex because the intersection of convex sets also convex. This proves that the superlevel set of r(-) is convex local to s*.
Conclusion By definition of quasi-concavity, we conclude that r(-) is quasi-concave the neigh- borhood of 8" (s))).
17

---

Published as a conference paper at ICLR 2026
E WITHIN-MOTION ALIGNMENT: LATENT SPACE VISUALIZATION AFTER PRETRAINING
(a) Epoch 0
(b) Epoch I
(c) Epoch
-22
-14
(d) Epoch 10
(e) Epoch 100
(f) Epoch 1000
Figure 7: Latent embeddings of all 20 reference motions using the pretrained encoder. Fach point corresponds to a single frame, and different colors represent different motions.
To evaluate the degree of aithin-motion alignment, we visualize the latent embeddings each mo- tion in 3D space using the pretrained encoder μgg. Because the latent vectors are 16-dimensional, we display only their first three dimensions. As shown in Fig. 7 after 1000 epochs of training with the loss in Eq. 8 the state-wise embeddings within each motion form tight clusters and nearly perfectly aligned, which is consistent with our theoretical results in Section
We also quantify this alignment using cosine similarity. For each motion m, .,, we first compute its motion-level embedding žm, using Eq. Then, for every state E mi we measure the cosine similarity between its latent embedding and 2m;, and average these values to obtain the within- motion alignment score for that motion. Across all 20 motions in the dataset, the average within- motion ilignment 0.99998-8.9111 - empirically confirming the effectiveness of the alignment objective.
In practice, overlapping states across different motions can make the motion embeddings harder to distinguish. We mitigate this issue by introducing a phase variable t and concatenating it to each frame or, as well as defining each state as a stack of the most recent n frames, 8; = [ot-n+1: 0t-n+2: Or). This reduces the likelihood that two motions share identical frame se- quences at the same phase t. In our dataset, n = works well, though the optimal value may vary depending the data. Additionally, our method is robust to small number of overlapping states, as long as they do not significantly pull the motion-level embeddings of different motions closer together.
18

---

Published as a conference paper at ICLR 2026
F FULL ALGORITHM OF RGSD
Algorithm RGSD Pretraining
Initialize encoder function (s). reference dataset M
2: for + to # of epochs do
3: Sample m Uniform(M)
4: Sample x" x=,x" xx
from m, x from [M - m} 5: Update 4s with LInfONCE on Eq.
6: end for
After pretraining is completed, imitatino and discovery happen in parallel.
Algorithm 2 RGSD: Imitation & Discovery
1: Initialize: skill-conditioned policy wo, frozen pretrained encoder dg trainable encoder qø. im- itation ratio p. KL loss coefficient reference dataset M, set of reference motion embeddings Zret, feature mask Omusk, and replay buffer D
2: Initialize qá 6 96
3: for <- to number of epochs do
4: for pj to episodes per epoch do
5: Sample task indicator ~ Bernoullipp)
// 1 being imitation, being discovery
6: Sample reference m Uniform(M)
7: Initialize Uniform(n
8:
Compute 2m using Eq.
5 with 9
9:
2m c k/ |+|||
c), where & ~ (0)
10:
11:
while episode not done do Execute TT(-] 8, z), observe s'
12:
13:
Compute reward r +- Const. + log g (2 s²) c log loggo(z (1-d) -
14:
Add (s, a,,,,r, c) to D s+s
16:
end while
end for
18:
for each (8,
E E do
19:
Compute L. + (1 c)
c) (-
(- log 0 s')) + (L((a(-(s) (+1s)
20:
Update á to minimize L.
Update using PPO with rewards
22:
end for
23:
end for
G EXPERIMENTS DETAILS
G.1 STATE AND ACTION SPACE
We use a 3D SMPL humanoid character, commonly adopted in prior work (Luo et al. 2023
Tessler
al. 2024
Tirinzoni al. 2025). The state representation is defined follows:
phase variable: 1-D
root (pelvis) height 1-D
root rotation relative to the local coordinate frame, 69-D
local rotation of each joint, 144-D
local velocity of each joint, and, 72-D
positions of the hands and feet in the local coordinate frame. 72D
19

---

Published as a conference paper at ICLR 2026
While policy and value function takes single frame observation input, encoder takes 5 frames concatenated the latest 5 steps of observations as imput. & = (0t-1, 0,-3, 2,-2, Ot-1, 0r). The agent controls the character by outputting target rotations for PD controllers at each joint. In total the character has 23 spherical joints, resulting in 69I) action space.
G.2 IMPLEMENTATION DETAILS OF BASELINE ALGORITHMS
To isolate the algorithmic contributions, removed all additional engineering techniques applied in the original implementations to improve motion robustness.
For ASE, the original training introduces recovery strategies by initializing 10% of the environments in random fallen states. While this clearly improves robustness, we removed this component consis- tently across method and all baselines. ASE also incorporates auxiliary diversity objective:
(Dkula() ( (Paulot Dz(21,
21), T(- 22))
27
Jaiv Ex Beg.
(13)
where Dy is distance function. Together with the mutual information objective, Jaiv encourages policies exhibit diverse behaviors when conditioned on different latent variables. Although this objective is modular and could be applied to any baseline, we excluded it to focus strictly on the main algorithmic differences.
For CALM, we encoded entire motions rather than segmenting them into 2-second clips, since motion lengths we used (between 2 and seconds) are relatively short. Shorter motions were zero-padded to standardize sequence lengths.
Finally, following conventions in skill discovery frameworks (Gregor et al. 2016 Eysenbach et al. 2018 Laskin et al. 2022 Park et al.) 2023b: Rho al., (024)) sampled a skill latent at the beginning of each episode and kept fixed throughout the rollout. While transitioning between skills orthogonal and can be incorporated into both method and the baselines further improve robustness, we excluded here to focus on a clean understanding of algorithmic contributions.
G.3 LIST OF REFERENCE MOTIONS
We list the file names (relative paths) of all motions used in our experiments:
Wa Walkk_turn_left__
2. ACCaDDssmgg//aaallellaaalnncnac Walk_turn_left_99
3. ACCAD ACCCDDDmmmm//aall222aaal11132333
Kalkk_turnleef__135
4. ACCDDD=mm|///aalee2222222/222111 2
5. ACCaD-smmm//aalle2e222222/3/3333 o
Welkg_ournrright_900
6. ACCAD- oCCa:e/rre/eeeeeeeaaaaaaaaaaaaaa stt_s________dd_________________
7. Accalpammm///eennaaeaaaaaaaaaaaa stt_____________eeeh_h_a
8. AcCh00ommmm//lall///uunnnn/eeae2 5
9. ACClD0-mam///llll2222unnnaacaaa_ run_turn_left_90
10. ACCaDD-mmg//Mllle///uunnna/eaaa S
11. ACCa0-uuap///aale/////nnnneaaa23 ||tpa///e__///////aaaaaaaaaaaaa0
12. ACCD00aa|h///llll//000mmnccaa2aa http:/o444___rnuuuuuughhhhhh_110
13.
14. octp::mmm//aaaaannnnnnoaaacaa111
15.
16. acco::/mmm//aaelllaaaa.......d.. left
17. ACCAD http:///o...aeeaaaaaaaaaaaaaaaaa cross_left
18. accppp/m../aaaaaaaaaaaaaaaaeeaaa uppercut_left
cross_right
19. hoc0:o/wwm.aaalca....raaaaaleaee __tpa/aeea_ccca___ppppeeeeee___e
20. accaremmm./aalllarrrraaaaaeeeeea uppercut_right
20

---

Published as a conference paper at ICLR 2026
G.4 HYPERPARAMETERS
Table 2: Hyperparameters of RGSD
Name
Value
Network Architecture Dim. of latent
16
Policy network π Activaion of
MLP with [1024, 1024, 1024, 512], tanh
Encoder network do Activaion of d
MLP with [1024, 1024, 1024, 512]
ReLU
4s input frame stacks
5
Contrastive pretraining
Minibatch size
256
Total number of epochs Optimizer
3,000
Adam(Kingma Ba
2014)
Learning rate for encoder
1c-4
Imitation and Discovery
Ratio of imitation environments
0.7
Learning
2e-5(actor), le-4(critic), le-4(encoder)
Optimizer
Adam
Minibatch size
32768
Horizon length
32
PPO clip threshold
0.2
PPO number of epochs
5
GAE A (Schulman al. 2015)
0.95
Discount factor
0.99
Entropy coefficient
0.1
KL loss coefficient
0.5
H SUPPLEMENTARY DISCUSSION: CHALLENGES OF EXPANDING RGSD WITH METRA
In Section we have discussed why the METRA objective conflicts with learning repetitive mo- tions when only local observations are available. A potential workaround is to augment the state with additional variables, such as a global coordinate or time variable. These variables could help distin- guish between locally identical motions, e.g., before and after taking step from walking motion. However, each approach ultimately faces a challenge for different reasons.
Global coordinate information. Given that all state information computed in the agent's local coordinate system, the first and last states of a walking behavior appear identical. Concatenating global coordinate to the state would allow the state to capture this progress. However, this approach still cast issues. First, not all motions produce meaningful displacement in global coordinates (c.g., shaking hands). More importantly, incorporating global position creates an imbalance in the state representation. All other state dimensions are bounded, which implies that the output of Φ is bounded as well. contrast, global position is unbounded, and METRA can exploit this property. Recall that the METRA reward defined (ø(s') Φ(+)) 2, where reward increases when the agent visits new states along the z direction in latent space. Because global position is unbounded, changes in position always yield new states, regardless of behavior. Consequently, the agent ends up learning skills that simply move in different directions when conditioned on different since this suffices to maximize the reward.
Time variable. A time variable could serve meaningful alternative the global coordinate. as it is less likely to be exploited by the METRA reward. Although the time variable also unbounded, it advances uniformly across all behaviors regardless of the underlying dynamics, increments by
21

---

Published as a conference paper at ICLR 2026
Figure Conceptual figure of latent space learned by METRA with time variable.
tag :48.2
tmp0-14633
2
DRE 245.8
D2J1 145.5
Dag 103.5
TAp
153.5
2304
230.4
TA81
160
150
22 2-5-3
10 90 15
Lemme
110 12 30.35
lag LIMIEE from
Den
x
Labue
Lemeet
50
(a)
Figure 9: Latent space learned with METRA. It shows seven reference trajectories and the agent's trajectory (blue line with markers). We introduce time variable to differentiate locally identical state pairs in repetitive motions, which causes the agent trajectory to exhibit discontinuous transi- tions in the latent space.
constant amount at every step. This ensures that no behavior can accelerate decelerate its progression. Moreover, it effectively differentiates identical states over time. Therefore the agent retains the opportunity learn meaningful behaviors, rather than exploiting state changes of specific dimensions.
However, this design still introduces issues. Figure illustrates an ideal latent space of reference motions formed by METRA with a time variable. Each motion is aligned with distinct directional vector, and consecutive states are placed a distance 1. this setup, the time variable naturally induces a contour: distances in the latent space correspond to "temporal distances." i.c., the mini- mum number of transition steps required to reach a given state, For example, if the time variable is 3, then reaching that state requires exactly three transitions, and the optimum of the representation function Ø, all behaviors at timestep should lie on the same contour.
The core problem arises during METRA exploration. When the agent transitions at large t val- ues-for example, from the 1=100 contour to the 1=101 contour-it may undergo an abrupt tran- sition of considerable magnitude in the latent space, especially if the subsequent pose lies the opposite side of the contour. This phenomenon is illustrated Fig. [9] Since METRA enforces that the latent distance between successive states must be less than 1, such discontinuities break the contour structure. As result, temporal coherence in the latent space is lost, and the representation collapses. In short, constructing an idealized latent space from reference data while respecting ME- TRA constraints with phase variable introduces fundamental conflict, leading to highly unstable training.
22


Reviews:
[{"review_id": "f3GRkj5a8G", "reviewer": "Reviewer_a8F6", "summary": "This paper introduces Reference-Guided Skill Discovery (RGSD), a method that enables unsupervised skill discovery to scale to high degree-of-freedom robots by grounding the exploration process in a semantically meaningful latent space derived from reference motion data. The key insight is to first use contrastive learning to embed reference motions onto a unit hypersphere, then leverage this pre-structured latent space to simultaneously imitate reference behaviors (by sampling latent vectors along reference directions) and discover novel related skills (by sampling novel directions). Applied to the SMPL humanoid, RGSD successfully learns both to reproduce complex motions like walking, running, and punching with high fidelity, and to discover semantically coherent variations of these behaviors-addressing the fundamental challenge that pure exploration in high-dimensional spaces tends to produce meaningless, unstructured behaviors while pure imitation fails to generalize beyond the reference dataset.", "questions": "- **Validation of within-motion alignment.** While Appendix B proves that contrastive pretraining achieves within-motion alignment at optimality, can you provide empirical evidence that this property is achieved in practice? Specifically: (a) quantitative metrics showing the variance of embeddings $\\mu_\\phi(s)$ for states $s$ within the same trajectory, (b) visualization of how embeddings cluster on the unit hypersphere for different motions, and (c) analysis of whether this alignment quality correlates with downstream imitation performance. Additionally, does this alignment property degrade when sampling latent vectors far from reference embeddings during discovery?\n\n- **Sensitivity to reference dataset composition.** The experiments use a fixed set of 20 reference motions (first version, lines 336-342). How does the method's performance scale with: (a) the number of reference motions per category (e.g., 1 vs. 5 walking motions), (b) the total number of skill categories, and (c) the diversity within each category (e.g., straight walking vs. walking with various turns)?\n\n- **Qualitative comparison on downstream task behaviors.** While Sec.5.3 reports FID scores showing RGSD (34.3) outperforms CALM (46.7) on the goal-reaching task, could you provide visual comparisons showing the actual behavioral differences? Given that both methods achieve similar success rates, understanding how they differ qualitatively would better support the claim of learning \"semantically meaningful\" skills.", "limitations": "", "rating": 2, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper provides rigorous theoretical justification for its core technical claims (Sec.A, Sec.B, Sec.C.1, etc.).", "weaknesses": "- **Unclear scope of \"discovery\" and potential overselling.** The paper's motivation (first version, lines 44-53) suggests learning skills beyond the reference data that are still \"semantically meaningful,\" using the example of a manipulator learning diverse skills like pushing and grasping. However, the experimental results demonstrate a more limited form of discovery: variations of existing reference behaviors (e.g., punching in different directions, sidestepping with varying turns) rather than genuinely novel skill categories. While this is still valuable, it conflicts with the initial framing. The paper would benefit from: (a) explicitly clarifying that discovered skills remain within the semantic manifold spanned by references, or (b) experiments sampling latent vectors farther from reference embeddings to empirically characterize what \"novel\" behaviors emerge and whether they remain meaningful or degenerate into unstructured motion.\n\n- **Strong assumptions on data segmentation limit practical applicability.** The method assumes each reference motion corresponds to a single, consistent skill throughout its duration. While this holds for the curated dataset used in experiments, most real-world video or robot demonstration data contains multiple skills within single trajectories and would require extensive manual segmentation and annotation. The paper provides no discussion of: (a) how to handle multi-skill demonstrations, (b) robustness to segmentation errors, or (c) potential extensions to learn from unsegmented data. This significantly limits the method's applicability beyond carefully curated motion capture datasets.\n\n- **Insufficient downstream task evaluation.** Sec.5.3 presents only a single downstream task (goal-reaching with sidestepping), which is insufficient to validate the learned skills' general utility. Additionally, critical experimental details are missing, e.g., how to obtain the frozen $\\pi_\\text{low}$\n\n- No supplementary material (e.g., videos of rollout examples) is provided.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "W3hdKF66dZ", "reviewer": "Reviewer_8xWQ", "summary": "* The main contribution of this paper is a framework that improves the initialization of skill discovery by using a seed set of reference motions that the discovery algorithm first imitates, followed by an unsupervised phase that introduces novelty and diversity without straying too far from the reference behaviors. The approach relies on constructing and exploiting a semantically grounded latent space, obtained through a contrastively pretrained encoder that maps states onto a unit-sphere latent representation via a von Mises–Fisher (vMF) distribution.\nIn practice, the imitation and discovery processes are performed in parallel: the pretrained encoder is frozen for reward computation, while a second encoder is trained through reinforcement learning to track it under a KL-divergence penalty.\n\n* Experimentally, on a 69-DoF SMPL humanoid with a 359-dimensional observation space, the proposed method successfully imitates the reference motions (walk, run, sidestep, backward, and punch) with low Cartesian error and achieves competitive FID metrics compared to DIAYN, METRA, ASE, and CALM. For the downstream “GoalReaching-Sidestepping” task, a high-level policy based on RGSD attains a success rate comparable to CALM, while achieving a better FID score.", "questions": "See suggestions of improvement in the \"Weaknesses\"  Section.", "limitations": "", "rating": 4, "confidence": 3, "soundness": 2, "presentation": 3, "contribution": 3, "strengths": "* The paper is clearly written and well structured. The proposed method is well motivated and exhibits an elegant design, both in its overall pipeline and in the chosen geometry of the latent space. \n* The experimental section provides clear and convincing qualitative evidence supporting the effectiveness of the approach.", "weaknesses": "* Several claims in the paper appear overstated. In particular, the theoretical analysis relies on the assumption of perfect within-motion alignment, which is unlikely to hold in practice. The encoder has finite capacity, and although the InfoNCE objective promotes alignment, it does not guarantee it. Moreover, the authors introduce several heuristic mechanisms—such as artificially terminating episodes when deviations from alignment become too large—to make the method work in practice. Unfortunately, all theoretical developments are built upon this idealized assumption, and no convergence or stability bounds are provided to quantify the impact of suboptimal alignment.\n\n* The comparisons with baselines are also somewhat unfair. For example, ASE and CALM were originally proposed with explicit diversity components, yet these elements were omitted in the reported baseline implementations. This undermines the strength of the comparative results.\n\n* The paper further relies heavily on the Fréchet Inception Distance (FID) as a key evaluation metric, but the justification for its use in this context is insufficient. Moreover, visual inspection of the figures suggests that small FID differences sometimes contradict qualitative impressions, casting doubt on its reliability as the primary measure of performance.\n\n* Finally, the choice of downstream task appears somewhat ad hoc. The corresponding reward function is explicitly designed to favor a latent skill close to the “sidestep” motion encoding, introducing a bias that aligns the task with the model’s strengths. In more general settings, downstream tasks will not necessarily be directional in this sense. The paper implicitly assumes that “motions” correspond directly to “directions” in latent space, which limits the generality of the approach.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "eL4iUF7QeL", "reviewer": "Reviewer_Yhxb", "summary": "The paper introduces Reference-Guided Skill Discovery (RGSD), a framework for reproducing and controlling diverse motions using reference data, outperforming prior imitation and unsupervised skill discovery methods. RGSD first learns a latent skill space by contrastively embedding reference trajectories, clustering each motion around a unique latent direction. It then jointly trains a skill-conditioned policy to imitate reference motions and discover novel skills by conditioning on both reference and interpolated latents. In experiments, RGSD consistently outperforms prior approaches by achieving lower Cartesian errors and maintaining high trajectory fidelity across reference trajectories, demonstrating strong improvements over both imitation-based and unsupervised skill discovery baselines in high-DoF SMPL humanoid.", "questions": "- How does RGSD perform when reference trajectories are noisy or highly variable, as in real-world human reference trajectories due to inaccuracies in sensing or mapping?\n\n- Can RGSD effectively scale and generalize as the number and diversity of reference trajectories increase?\n\n- Is the set of discovered skills strictly limited to local variants of those present in the reference dataset? If yes, I would suggest clarifying this in the paper\n\n- How does RGSD compare to frameworks like Meta-Motivo, which also address a similar problem of acquiring skills from a reference dataset, despite not focusing exclusively on skill discovery? \n\n- How does RGSD offer advantages over CALM? Both methods achieve similar downstream task performance. Are there settings where RGSD’s skill interpolation provides unique benefits?", "limitations": "", "rating": 8, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "**Strengths:**\n\n- Clear, easy to read\n- Demonstrates strong, consistent improvements over established baselines\n- Provides a principled advancement over both imitation-based and unsupervised skill discovery approaches", "weaknesses": "**Weaknesses:**\n\n- Downstream task evaluation is limited to a single, relatively simple task\n- Assumes coverage of skill in the pretraining reference dataset\n- Limited exploration of performance with diverse or noisy reference data", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AvGuY12zBt", "reviewer": "Reviewer_1tD4", "summary": "This paper proposes a way to do unsupervised skill discovery based on reference motion data. The authors first learn a latent mapping function (from $\\mathcal{S}$ to $\\mathcal{Z}$) with contrastive learning. In the next phase, they train a DIAYN-like objective with a KL constraint to train a skill policy. As a result, they get a set of skills that are diverse while looking natural. They compare their method with two unsupervised skill discovery baselines (DIAYN and METRA) and two imitation learning baselines (ASE and CALM) in terms of $\\ell^2$ distance and FID, showing that their method produces better behaviors.", "questions": "I don't have particular questions about this paper as of now. My main issue is the lack of discussion and empirical comparison with related work (see the weaknesses section above), which I hope the authors can address during the rebuttal period.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "The paper is well-motivated and generally well-written. The problem setting in this paper (i.e., how to do skill discovery based on some prior knowledge -- in this case, a reference motion dataset) is important and timely. While the individual components of the proposed method are not necessarily novel, the combination of these components has (to my knowledge) not been previously done in the literature. I also like that the proposed method is straightforward and relatively easy to implement. The authors convincingly demonstrate that RGSD is more effective than ASE and CALM.", "weaknesses": "The main weakness of this work is in the empirical comparison. While the authors compare their method against ASE and CALM, there are many more relevant previous works (essentially the whole literature on \"data-driven/offline skill discovery\") that are closely related to the proposed method but are not properly discussed/compared against. For example, OPAL and SPiRL also do the conceptually similar thing as this method -- they first learn a latent space (via a VAE) and then do policy learning based on the learned latent space. I know these methods are not exactly the same as the proposed one (and the problem settings are also not identical), but I think they are still close enough, to the degree that I would have expected to see comparisons with prior works in data-driven skill discovery. I also would have expected more extensive discussions of this line of research.\n\nThe paper distinguishes its method from Motivo by saying that \"it does not explicitly aim to discover new skills, which distinguishes our contribution,\" which I find unconvincing. I agree that the latent space is not densely covered by $z$'s from the dataset trajectories, but given that the distribution of latent vectors tends to be uniform with contrastive learning (https://proceedings.mlr.press/v119/wang20k/wang20k.pdf), interpolated latents are still more or less in-distribution. Moreover, even if one treats these interpolated vectors as \"new skills,\" the same argument could apply to other works including Motivo, FB, etc. as well, where one samples $z$'s from the prior distribution (typically a (unit) Gaussian) to train a policy. Hence, I think the authors should either provide empirical quantitative evidence that shows RGSD indeed learns more novel skills, or tone down the claim about the ability to discover \"new\" skills. I also think the paper could have been much stronger with comparisons with Motivo, as it is one of the most closely related works both algorithmically and empirically.\n\nI'd be happy to adjust my score if the authors address these points.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)