Title: Optimal Control of the Future via Prospective Foraging
Primary area: learning on time series and dynamical systems
Keywords: Machine Learning, Statistical Learning, Reinforcement Learning, Dynamic Systems

Abstract: Optimal control of the future is the next frontier for AI.  Current approaches to this problem are typically rooted in either reinforcement learning  or online learning.  While powerful, these frameworks for learning are mathematically distinct from Probably Approximately Correct (PAC) learning, which has been the workhorse for the recent technological achievements in AI. We therefore build on prior work on prospective learning, an extension of PAC learning  (without control) in non-stationary environments.  Here, we further extend the PAC learning framework to address learning and control in non-stationary environments.   Using this framework, called ''Prospective Control'', we  prove that under certain fairly general assumptions, empirical risk minimization (ERM) asymptotically achieves the Bayes optimal policy.  We then take a specific instance of prospective control, foraging---which is a canonical task for any mobile agent---be it natural or artificial.  We illustrate that existing reinforcement learning algorithms fail to learn in these non-stationary environments, and even with modifications they are orders of magnitude less efficient than our prospective foraging agents.