Abstract Statistical methods are used widely in machine learning and artificial intelligence to account for uncertain environments. In this talk, I focus specifically on statistical inference and planning in dynamic systems.
Dynamic systems are essential to many machine learning tasks.
For instance, they are used in robotics, computer vision, computational finance, language processing, and bioinformatics. Using the visual tracking of human motion as an example application, I first describe a formal framework for the modeling of dynamic inference and decision problems. Then I suggest a new learning algorithm for reinforcement learning in this domain that determines optimal decision rules based on previous observations. This algorithm uses local averaging to estimate the value function of the dynamic system, and it is the first algorithm of its kind that is provably convergent to a unique fixed point. I also develop a statistical learning theory for reinforcement learning which is useful to assess the uncertainty of the value function estimate.