An Interpretive Model of Hand-Eye Coordination

Overview Normal cognitive function requires coordination bet w en perception and action. However, both action and perception pose enormous challenges to o ur scientific understanding, and, as a result they ar often studied separately, with experimental paradig ms carefully crafted to tease them apart (e.g., gaz e fixation). We present an interpretive model that a ddresses both faculties within a single framework. A simplified task of hand-eye coordination is modeled as a stochastic optimal control problem, and the solution is a coordinated motion of eye and hands, derived from first principles. Our numerical resul t reveal a rich behavioral repertoire, with both smoo th pursuit and saccades emerging as components of t he optimal solution to the eye-hand control task. The Model We present an abstract model of a reaching task – t wo free-moving points (“hands”) move on a twodimensional plane (the “scene”), which includes fou r bstacles and a target (depicted in figure 1). T he hands start at both sides of the bottom of the scen e, and at a final time they are penalized proportio nal to their distance from the target, located at the topmiddle of the scene. Each hand moves through a pai r of obstacles on its path to the goal. The linear dyna mics of the hands are subject to constant process n oi e, and the agent is provided with noisy observations o f the state of the scene. State estimation and fee dback control of the hands are therefore necessary to com plete the task. The positions of both obstacles an d target are fixed, but the learning agent starts wit h an uncertain estimate of these positions, and so they have to be observed and estimated as well. The eye is involved in the observation process – it provides information that allows for better state estimation, thereby improving feedback control and so contributing to task performance. Motivated by this abstraction, we model the eye as a “point of g aze”, a free-moving point in the scene. The effect of this gaze is a local modulation of the observation n ise: if an object (hand, obstacle or target) is c lo e to the point of gaze, the noise perturbing the observa tion of the object’s position will be reduced (this reduction being a Gaussian function of distance fro m the gaze point, as illustrated in figure 2B). Therefore, directing the gaze at an object allows t he learning agent to generate better estimation of its position, which yields more accurate feedback contr ol of the hands and hence better overall performanc e. Solution Since the observation noise is state-dependent (and he ce not Gaussian), estimation requires a nonlinear filter (we employ the Extended Kalman Filter ); furthermore, since the cost is not quadratic (du e to the obstacles), the Stochastic Optimal Control (SOC ) problem cannot be solved using standard LinearQuadratic-Gaussian techniques. Instead, we solve the SOC using Minimax Differential Dynamic Programming (the complete mathematical description was published in [1]; see references therein). The optimal solution is a feedback policy for controlli ng the positions of both hands and eye through the scene that exhibits both saccades (fig. 3) and smooth pur suit (fig. 4). In order to make this high-dimensional optimization problem tractable, we shape the solution by gradually reducing the width of the eye’s “fovea” ( the width of the Gaussian inside which observation noise is reduced) – at first, wider fovea allows th e agent to detect and learn what areas of the scene ar most relevant at every stage; eventually, a narrowe r fovea leads to more distinct eye motions.