Neural Basis of 3 D Surface Representation in Primate Visual Cortex R 01 : Research Plan

R01: Research Plan A. Research Aims The general goal of this project is to characterize the neural mechanisms in the early visual system of the macaque for representing and computing the three-dimensional (3D) surface structure of objects in visual scenes. 3D surface inference and representation is a fundamental problem in vision. Recent neurophysiological studies have shown that neurons in the higher order extrastriate cortices (IT, CIP, MT) are sensitive to large 3D structures and the shape of objects (Bradley et al. 1998; DeAngelis et al. 1998; DeAngelis and Newsome, 1999; Janssen et al. 1999, 2000, 2001; Nguyenkim and DeAngelis, 2003; Tsutsui et al. 2002). There are also several new studies, including ours, that suggest V2 might be involved in the first stage of computing and representing surfaces (Bakin et al. 2000; Lee et al. 2002; von der Heydt et al. 2000, Thomas et al. 2002). However, these studies primarily demonstrate indirect ‘sensitivity’ to surfaces, or tuning to absolute or relative disparity. While quantitative and systematic techniques have been used to study the encoding of binocular disparity by V1 neurons (Ohzawa et al. 1990, 1996, 1997; Anzai et al. 1999a, 1999b, 1999c), similarly rigorous techniques have not been applied to the study of encoding local 3D surface structures in V1 or V2 neurons. Many basic questions remain unanswered: How are arbitrary local 3D surface structures (surface shapes, orientation and discontinuity) represented in the brain? What are the basic neural representational elements? How are they derived from various visual cues and influenced by a contextual surround? To answer these questions, we propose to investigate the neural activity of early visual cortex (V1, V2) in response to 3D surface shape structures, both locally and in the context of larger scenes. The receptive fields (RFs) of early visual neurons are retinotopically organized and small in size, and thus are ideal candidates for representing local 3D surface structures. Neurons in higher cortical areas (V4, MT, IT) have larger receptive fields and might be useful for representing larger surface patches or more global surface structures. We propose to carry out experiments to systematically characterize the sensitivity of V1 and V2 neurons to surface orientation and other basic surface structures, and to determine their possible 3D receptive field structures which would provide insight to the neural strategies for encoding 3D scenes. We will investigate how these representations are derived and disambiguated in the brain under the influence of various bottom-up visual cues as well as the global image context. Aim 1: Representation of local 3D surface structures. Experiments are proposed to characterize the sensitivity of neurons in V1 and V2 to 3D surface structures rendered with dynamic random dot stereograms (DRDS). The basic hypothesis is that neurons in these early visual areas are sensitive or tuned to 3D surface structures, and potentially each has a 3D receptive field for encoding a particular basic shape primitive. We will study neural responses to basic absolute disparity, surface orientations (slant and tilt), and oriented convex and concave structures. Furthermore, we will characterize the 3D receptive fields of these neurons, predicted from efficient coding theory and the statistical analysis of surface variation in natural scenes. Aim 2: Representation as a function of visual cues. Here, we propose experiments to determine whether neuronal sensitivity to 3D surface shape is a function of visual cues. The sensitivity of neurons to 3D shapes rendered in shading, texture and stereo disparity cues will be tested both in isolation and in combination. The basic objective is to evaluate whether surface sensitivities based on different cues are carried by the same or different neural channels. We also plan to study the time course of surface shape sensitivity as a function of visual cues as a means to understand the inference processes. We will test the hypothesis that abstract 3D surface structure is inferred and represented early on in the visual areas. Aim 3: Contextual influence of the surrounding surfaces. In this aim, we have designed experiments to investigate the influence of global context on the interpretation of local surface cues, and to understand how surfaces are organized in a global context. Surface shape tuning