ICML 2025

Online Laplacian-Based Representation Learning in Reinforcement Learning
Maheed H. Ahmed 1 Jayanth Bhargav 1 Mahsa Ghasemi 1
Abstract
Representation learning plays a crucial role in re-
inforcement learning, especially in complex envi-
ronments with high-dimensional and unstructured
states. Effective representations can enhance the
efficiency of learning algorithms by improving
sample efficiency and generalization across tasks.
This paper considers the Laplacian-based frame-
work for representation learning, where the eigen-
vectors of the Laplacian matrix of the underlying
transition graph are leveraged to encode mean-
ingful features from raw sensory observations of
the states. Despite the promising algorithmic ad-
vances in this framework, it remains an open ques-
tion whether the Laplacian-based representations
can be learned online and with theoretical guaran-
tees along with policy learning. We address this
by formulating an online optimization approach
using the Asymmetric Graph Drawing Objective
(AGDO) and analyzing its convergence via online
projected gradient descent under mild assump-
tions. Our extensive simulation studies empiri-
cally validate the convergence guarantees to the
true Laplacian representation. Furthermore, we
provide insights into the compatibility of differ-
ent reinforcement learning algorithms with online
representation learning.
1. Introduction
Representation learning is an important part of machine
learning that involves learning compact and useful represen-
tations of data. The quality of these representations signifi-
cantly impacts the performance and efficiency of machine
learning algorithms (Bengio et al., 2013). In reinforcement
learning (RL), agents often deal with complex environments
characterized by high-dimensional and unstructured states.
1Electrical and Computer Engineering, Purdue University, West
Lafayette, IN 47907, USA. Correspondence to: Maheed H. Ahmed
<ahmed237@purdue.edu>.
Proceedings of the 42 nd International C...