ICML 2025

Robot-Gated Interactive Imitation Learning with Adaptive Intervention
Mechanism
Haoyuan Cai 1 Zhenghao Peng 1 Bolei Zhou 1
Abstract
Interactive Imitation Learning (IIL) allows agents
to acquire desired behaviors through human in-
terventions, but current methods impose high
cognitive demands on human supervisors. We
propose the Adaptive Intervention Mechanism
(AIM), a novel robot-gated IIL algorithm that
learns an adaptive criterion for requesting hu-
man demonstrations. AIM utilizes a proxy Q-
function to mimic the human intervention rule
and adjusts intervention requests based on the
alignment between agent and human actions. By
assigning high Q-values when the agent deviates
from the expert and decreasing these values as the
agent becomes proficient, the proxy Q-function
enables the agent to assess the real-time align-
ment with the expert and request assistance when
needed. Our expert-in-the-loop experiments re-
veal that AIM significantly reduces expert moni-
toring efforts in both continuous and discrete con-
trol tasks. Compared to the uncertainty-based
baseline Thrifty-DAgger, our method achieves a
40% improvement in terms of human take-over
cost and learning efficiency. Furthermore, AIM
effectively identifies safety-critical states for ex-
pert assistance, thereby collecting higher-quality
expert demonstrations and reducing overall ex-
pert data and environment interactions needed.
Code and demo video are available at https:
//github.com/metadriverse/AIM.
1. Introduction
Reinforcement learning (RL) has been successful in a wide
variety of tasks, from playing Atari games (Mnih et al.,
2013) to autonomous driving (Li et al., 2022a) and robotic
1Computer Science Department, University of Califor-
nia, Los Angeles (UCLA). Correspondence to:
Bolei Zhou
<bolei@cs.ucla.edu>.
Proceedings of the 42 nd International Conference on Machine
Learning, Vancouver, Canada. PMLR 267, 2025. Copyright 2025
by the author(s).
I can have a rest.
I can handle it.
Can you help me?
...