Agency Is Frame-Dependent
Authors: name: David Abel, name: André Barreto, name: Michael Bowling, name: Will Dabney, name: Shi Dong, name: Steven Hansen, name: Anna Harutyunyan, name: Khimya Khetarpal, name: Clare Lyle, name: Razvan Pascanu, name: Georgios Piliouras, name: Doina Precup, name: Jonathan Richens, name: Mark Rowland, name: Tom Schaul, name: Satinder Singh

Agency is a system's capacity to steer outcomes toward a goal, and is a
central topic of study across biology, philosophy, cognitive science, and
artificial intelligence. Determining if a system exhibits agency is a
notoriously difficult question: Dennett (1989), for instance, highlights the
puzzle of determining which principles can decide whether a rock, a thermostat,
or a robot each possess agency. We here address this puzzle from the viewpoint
of reinforcement learning by arguing that agency is fundamentally
frame-dependent: Any measurement of a system's agency must be made relative to
a reference frame. We support this claim by presenting a philosophical argument
that each of the essential properties of agency proposed by Barandiaran et al.
(2009) and Moreno (2018) are themselves frame-dependent. We conclude that any
basic science of agency requires frame-dependence, and discuss the implications
of this claim for reinforcement learning.