Calculation of object position in various reference frames with a robotic simulator

Calculation of object position in various reference frames with a robotic simulator Marcel Svec (svec.marcel@gmail.com) and Igor Farkaˇs (farkas@fmph.uniba.sk) Department of Applied Informatics, Comenius University 84248 Mlynsk´a dolina, Bratislava, Slovakia Abstract The brain encodes the space in various reference frames. The key role in spatial transformations is played by the posterior parietal cortex where neurons combine retinal location of vi- sual stimulus with gaze direction to encode spatial informa- tion. This nonlinear dependence of neuronal responses, gain modulation, is considered a fundamental computational prin- ciple used in the brain. The important insight can be obtained through computational models, typically artificial neural net- works. In this paper, we test the Zipser–Andersen model but use more realistic and variable stimuli, employing the sim- ulated iCub robot. The multi-layer perceptron was able to successfully perform coordinate transformation from eye- to body-centered reference frame, using gaze information. Model achieves high accuracy of 2 to 4 degrees on testing data, de- pending on the dataset variability. We provide visualisation techniques for analysing the network, and the effects of gain modulation and shifting receptive fields. Our results confirm previous findings that hidden neurons use various intermediate codings that mediate transformations. Keywords: reference frames; coordinate transformation; neural network; robotic simulator; gain field puts for training the model. In this paper, we take advan- tage of the simulated iCub robotic environment (Tikhanoff et al., 2008) that naturally provides embodied data of higher complexity for learning the task. The classical view, com- ing from geometry, applied commonly in robotics, assumes that coordinate transformations are computed explicitly and applied sequentially. On the contrary, in a novel view, being more consistent with neuroscientific data, coordinate trans- formations are computed implicitly and in parallel (Blohm & Crawford, 2009). Hence, we analyze the phenomenon of coordinate transformations in the context of the progressive cognitive robotics that offers a promising pathway to build- ing autonomous systems. We are not aware of this type of work with iCub. For coordinate transformation, we use the original Zipser–Andersen model described below, and show that the implicit transformation can be learned equally well despite using more complex data. We also introduce visual- ization techniques that reveal model behavior. Background Introduction Determining the object position in space is always related to some point at known location. This relationship is captured by the concept of reference frame in which we can define a concrete coordinate system (Batista, 2002). Humans are able to use both egocentric and allocentric reference frames, which can be combined to support behavior according to the task (Burgess, 2006). In general, reference frame may be an- chored to practically anything, to our head, hand, or any other object. Neuroscientists naturally adopted the concept of refer- ence frames to better understand how the space is represented in the brain. Cells in the visual system respond to the stimuli located only in particular location called the receptive field in cases where the response is considered to be strictly sensory. The receptive field (also called response field) can also refer to the set of patterns that evoke neuron’s activation. Research on reference frames that has been progressing for the past 25 years suggests that reference frames do not always exist in an explicit form, but rather as some intermediate rep- resentations of space that are further processed for specific purposes, for instance to generate reaching commands. The process of converting sensory stimuli into the motor plans is referred to as sensorimotor transformation. These are of- ten formalised in terms of spatial transformations from eye- centred (retinotopic) or head-centered coordinates into hand- centered coordinates. Coordinate transformations are also a key component for learning the body schema (Hoffmann et al., 2011). In most cases, the authors use artificially generated inputs and out- Andersen and Mountcastle (1983) discovered that neurons in area 7a of posterior parietal cortex (PPC) of monkeys com- bine retinal location of visual stimulus with gaze direction to encode spatial information. The role of PPC as a sensorimo- tor interface for visually guided eye and arm movements has been also supported by later findings (Buneo & Andersen, 2006; Khan, Pisella, & Blohm, 2012). Cells in PPC appear to nonlinearly combine information from different modal- ities, while their sensitivity is modulated by one modality (e.g. gaze direction) without changing their selectivity to the other modality (visual stimuli). This phenomenon was coined as gain modulation and the changes in neuron’s sensitivity as gain fields. The subsequent studies of gain modulation have revealed that it is an extremely widespread mechanism that appears to be a fundamental computational principle behind coordinate transformations (Salinas & Thier, 2000; Salinas & Sejnowski, 2001; Blohm & Crawford, 2009). The types of signals that could produce gain fields in- clude gaze direction, head position, eye vergence, target dis- tance, chromatic contrast or attention, all together leading to the suggestion that gain modulation is a general mechanism for multimodal integrations that underlie important cogni- tive functions like sensorimotor transformation, object recog- nition, motion processing or focusing attention (Salinas & Thier, 2000). The essential feature of gain fields is nonlin- earity. However, the biophysical basis that allows neurons to combine information from two sources such that their output is close to the product of two functions is still unclear. In the first computational model, Zipser and Andersen