NeurIPS 2025

Periodic Skill Discovery
Jonghae Park1 Daesol Cho2 Jusuk Lee1 Dongseok Shim1 Inkyu Jang1 H. Jin Kim1
1Seoul National University
2Georgia Institute of Technology
bdfire1234@snu.ac.kr
Abstract
Unsupervised skill discovery in reinforcement learning (RL) aims to learn diverse
behaviors without relying on external rewards. However, current methods often
overlook the periodic nature of learned skills, focusing instead on increasing the
mutual dependence between states and skills or maximizing the distance traveled
in latent space. Considering that many robotic tasks—particularly those involving
locomotion—require periodic behaviors across varying timescales, the ability
to discover diverse periodic skills is essential. Motivated by this, we propose
Periodic Skill Discovery (PSD), a framework that discovers periodic behaviors
in an unsupervised manner. The key idea of PSD is to train an encoder that maps
states to a circular latent space, thereby naturally encoding periodicity in the latent
representation. By capturing temporal distance, PSD can effectively learn skills
with diverse periods in complex robotic tasks, even with pixel-based observations.
We further show that these learned skills achieve high performance on downstream
tasks such as hurdling. Moreover, integrating PSD with an existing skill discovery
method offers more diverse behaviors, thus broadening the agent’s repertoire.
Our code and demos are available at https://jonghaepark.github.io/psd
1
Introduction
A fundamental observation in nature is that nearly all forms of locomotion are inherently periodic.
Rhythmic gaits of quadrupeds, the oscillatory motions of fish, and even human walking patterns
share a distinct periodic structure, which can be flexibly modulated across multiple timescales
[39, 28, 31]. This inherent periodicity not only enables energy-efficient movement [39, 82] but also
provides adaptability under varying conditions [28, 31, 82]. Motivated by this understanding, robotics
research has le...