Decoding the perception and memory trace: Neuronal representations of perceptual and memory content in time and space

Visual processing is a complex task which is best investigated using sensitive multivariate analysis methods that can capture representation-specific brain activity over both time and space. In this study, we applied a multivariate decoding algorithm to MEG data of subjects engaged in passive viewing of images of faces, scenes, bodies and tools. We used reconstructed source-space time courses as input to the algorithm in order to localize brain regions involved in optimal image discrimination. Applying this method to the interval of 115 to 315 ms after stimulus onset, we show a focal localization of regression coefficients in the inferior occipital, middle occipital, and lingual gyrus that drive decoding of the different perceived image categories. Classification accuracy was highest (over 90% correctly classified trials, compared to a chance level accuracy of 50%) when dissociating the perception of faces from perception of other object categories. Furthermore, we applied this method to each single time point to extract the temporal evolution of visual perception. This allowed for the detection of differences in visual category perception as early as 85 ms after stimulus onset. Furthermore, localizing the corresponding regression coefficients of each time point allowed us to capture the spatiotemporal dynamics of visual category perception. This revealed initial involvement of sources in the inferior occipital, inferior temporal and superior occipital gyrus. During sustained stimulation additional sources in the anterior inferior temporal gyrus and superior parietal gyrus became involved. We conclude that decoding of source-space MEG data provides a suitable method to investigate the spatiotemporal dynamics of ongoing cognitive processing. Introduction Visual perception is an intricate yet fast process. This is not only the case for isolated low-level image properties such as object location and luminance. We are also able to discern different image categories such as faces and tools very rapidly. Indeed, saccade studies show that as little as 120 ms is required before a saccade differentiating between animals and scenes is initiated (Kirchner & Thorpe, 2006), and only 100-110 ms before saccading to human faces (Crouzet et al., 2010). As these latencies include the process of saccade planning, differential brain activity for faces as compared to other semantic categories should already be present as early as 80 ms after stimulus onset (Crouzet et al., 2010). However, neuroimaging research has been inconclusive about this exact timing, demonstrating variable latencies ranging from 40-75 ms (Bacon-Macé et al., 2005; Kirchner et al., 2009; Ramkumar et al., 2013; Seeck et al., 1997; VanRullen & Thorpe, 2001) up to 100-150 ms (Amano et al., 2006; Bode et al., 2012; Carlson et al., 2011; Fuentemilla et al., 2010; Jafarpour et al., 2013; Liu et al., 2009; Simanova et al., 2010; Thorpe et al., 1996; Wokke et al., 2012). In addition to the fast onset of vision-related activity, the temporal evolution of this activity is also complex, consisting of multiple visual event-related potential components, including the P1, N1, and the face-related N170 (Bentin et al., 1996). Next to this rapid and complex temporal evolution, a multitude of brain regions is employed in the process of visual perception. Areas along both the ventral and dorsal stream of visual processing are recruited, ranging from the striate cortex in the occipital lobe to the anterior temporal and parietal lobe (Mishkin et al., 1983). Furthermore, this activity is thought to proceed in rapid succession (Kirchner & Thorpe, 2006; Riesenhuber & Poggio, 1999; Serre et al., 2007; Thorpe & Fabre-Thorpe, 2001). Therefore, a method is required that can capture both the temporal and spatial aspect of visual processing. Although electrophysiological methods do provide exquisite temporal resolution, they generally lack the spatial resolution to localize the aforementioned temporal evolution. We therefore aim to improve spatial localization by applying a multivariate classification algorithm to reconstructed source-space activity, calculated from magnetoencephalography (MEG) recordings. Multivariate methods can be more sensitive than univariate methods when discriminative information is distributed across multiple sources (Kriegeskorte et al., 2006; Kriegeskorte, 2011; Lange et al., 1999). Previous work with functional magnetic Spatiotemporal dynamics of visual category perception | 19 18 | Chapter 2