Mounting evidence suggests that 'core object recognition,' the ability to rapidly recognize objects despite substantial appearance variation, is solved in the brain via a cascade of reflexive, largely feedforward computations that culminate in a powerful neuronal representation in the inferior temporal cortex. However, the algorithm that produces this solution remains poorly understood. Here we review evidence ranging from individual neurons and neuronal populations to behavior and computational models. We propose that understanding this algorithm will require using neuronal and psychophysical data to sift through many computational models, each based on building blocks of small, canonical subnetworks with a common functional goal. Recognizing the words on this page, a coffee cup on your desk, or the person who just entered the room all seem so easy. The apparent ease of our visual recognition abilities belies the computational magnitude of this feat: we effortlessly detect and classify objects from among tens of thousands of possibilities (Biederman, 1987) and we do so within a fraction of a second (Potter, 1976; Thorpe et al., 1996) , despite the tremendous variation in appearance that each object produces on our eyes (reviewed by Logothetis and Sheinberg, 1996) . From an evolutionary perspective, our recognition abilities are not surprisingour daily activities (e.g., finding food, social interaction, selecting tools, reading, etc.), and thus our survival, depend on our accurate and rapid extraction of object identity from the patterns of photons on our retinae. The fact that half of the nonhuman primate neocortex is devoted to visual processing (Felleman and Van Essen, 1991) speaks to the computational complexity of object recognition. From this perspective, we have a remarkable opportunity-we have access to a machine that produces a robust solution, and we can investigate that machine to uncover its algorithms of operation. These to-be-discovered algorithms will probably extend beyond the domain of vision-not only to other biological senses (e.g., touch, audition, olfaction), but also to the discovery of meaning in high-dimensional artificial sensor data (e.g., cameras, biometric sensors, etc.). Uncovering these algorithms requires expertise from psychophysics, cognitive neuroscience, neuroanatomy, neurophysiology, computational neuroscience, computer vision, and machine learning, and the traditional boundaries between these fields are dissolving. What Does It Mean to Say ''We Want to Understand Object Recognition''? Conceptually, we want to know how the visual system can take each retinal image and report the identities or categories of one or more objects that are present in that scene. Not everyone agrees on what a sufficient answer to object recognition might look like. One operational definition of ''understanding'' object recognition is the ability to construct an artificial system that performs as well as our own visual system (similar in spirit to computer-science tests of intelligence advocated by Turing (1950) . In practice, such an operational definition requires agreed-upon sets of images, tasks, and measures, and these ''benchmark'' decisions cannot be taken lightly (Pinto et al., 2008a; see below). The computer vision and machine learning communities might be content with a Turing definition of operational success, even if it looked nothing like the real brain, as it would capture useful computational algorithms independent of the hardware (or wetware) implementation. However, experimental neuroscientists tend to be more interested in mapping the spatial layout and connectivity of the relevant brain areas, uncovering conceptual definitions that can guide experiments, and reaching cellular and molecular targets that can be used to predictably modify object perception. For example, by uncovering the neuronal circuitry underlying object recognition, we might ultimately repair that circuitry in brain disorders that impact our perceptual systems (e.g., blindness, agnosias, etc.). Nowadays, these motivations are synergistic-experimental neuroscientists are providing new clues and constraints about the algorithmic solution at work in the brain, and computational neuroscientists seek to integrate these clues to produce hypotheses (a.k.a. algorithms) that can be experimentally distinguished. This synergy is leading to high-performing artificial vision systems (Pinto et al., 2008a (Pinto et al., , 2009b Serre et al., 2007b) . We expect this pace to accelerate, to fully explain human abilities, to reveal ways for extending and generalizing beyond those abilities, and to expose ways to repair broken neuronal circuits and augment normal circuits. Progress toward understanding object recognition is driven by linking phenomena at different levels of abstraction. Neuron 73, February 9, 2012 ª2012 Elsevier Inc. 415 ''Phenomena'' at one level of abstraction (e.g., behavioral success on well-designed benchmark tests) are best explained by ''mechanisms'' at one level of abstraction below (e.g., a neuronal spiking population code in inferior temporal cortex, IT). Notably, these ''mechanisms'' are themselves ''phenomena'' that also require mechanistic explanations at an even lower level of abstraction (e.g., neuronal connectivity, intracellular events). Progress is facilitated by good intuitions about the most useful levels of abstraction as well as measurements of well-chosen phenomena at nearby levels. It then becomes crucial to define alternative hypotheses that link those sets of phenomena and to determine those that explain the most data and generalize outside the specific conditions on which they were tested. In practice, we do not require all levels of abstraction and their links to be fully understood, but rather that both the phenomena and the linking hypotheses be understood sufficiently well as to achieve the broader policy missions of the research (e.g., building artificial vision systems, visual prosthetics, repairing disrupted brain circuits, etc.). To that end, we review three sets of phenomena at three levels of abstraction (core recognition behavior, the IT population representation, and IT single-unit responses), and we describe the links between these phenomena (sections 1 and 2 below). We then consider how the architecture and plasticity of the ventral visual stream might produce a solution for object recognition in IT (section 3), and we conclude by discussing key open directions (section 4). The Behavioral Phenomenon of Interest: Core Object Recognition Vision accomplishes many tasks besides object recognition, including object tracking, segmentation, obstacle avoidance, object grasping, etc., and these tasks are beyond the scope of this review. For example, studies point to the importance of the dorsal visual stream for supporting the ability to guide the eyes or covert processing resources (spatial ''attention'') toward objects (e.g., Ikkai et al., 2011; Noudoost et al., 2010; Valyear et al., 2006) and to shape the hand to manipulate an object (e.g., Goodale et al., 1994; Murata et al., 2000) , and we do not review that work here (see Cardoso-Leite and Gorea, 2010; Jeannerod et al., 1995; Konen and Kastner, 2008; Sakata et al., 1997) . Instead, we and others define object recognition as the ability to assign labels (e.g., nouns) to particular objects, ranging from precise labels (''identification'') to course labels (''categorization''). More specifically, we focus on the ability to complete such tasks over a range of identity preserving transformations (e.g., changes in object position, size, pose, and background context), without any object-specific or location-specific pre-cuing (e.g., see Figure 1 ). Indeed, primates can accurately report the identity or category of an object in the central visual field remarkably quickly: behavioral reaction times for singleimage presentations are as short as 250 ms in monkeys (Fabre-Thorpe et al., 1998) and 350 ms in humans (Rousselet et al., 2002; Thorpe et al., 1996) , and images can be presented sequentially at rates less than 100 ms per image (e.g., Keysers et al., 2001; Potter, 1976) . Accounting for the time needed to make a behavioral response, this suggests that the central visual image is processed to support recognition in less than 200 ms, even without attentional pre-cuing (Fabre-Thorpe et al., 1998; Intraub, 1980; Keysers et al., 2001; Potter, 1976; Rousselet et al., 2002; Rubin and Turano, 1992) . Consistent with this, surface recordings in humans of evoked-potentials find neural signatures reflecting object categorization within 150 ms (Thorpe et al., 1996) . This ''blink of an eye'' time scale is not surprising in that primates typically explore their visual world with rapid eye movements, which result in short fixations (200-500 ms), during which the identity of one or more objects in the central visual field (10 deg) must be rapidly determined. We refer to this extremely rapid and highly accurate object recognition behavior as ''core recognition' ' (DiCarlo and Cox, 2007) . This definition effectively strips the object recognition problem to its essence and provides a potentially tractable gateway to understanding. As describe below, it also places important constraints on the underlying neuronal codes (section 2) and algorithms at work (section 3). Core object recognition is the ability to rapidly (<200 ms viewing duration) discriminate a given visual object (e.g., a car, top row) from all other possible visual objects (e.g., bottom row) without any object-specific or location-specific pre-cuing (e.g., DiCarlo and Cox, 2007) . Primates perform this task remarkably well, even in the face of identity-preserving transformations (e.g., changes in object position, size, viewpoint, and visual context). 416 Neuron 73, February 9, 2012 ª2012 Elsevier Inc. Neuron The Crux Computational Problem: Core Recognition Requires Invariance To gain tractability, we have stripped the general problem of object recognition to the more specific problem of core recognition, but we have preserved its computational hallmark-the ability to identify objects over a large range of viewing conditions. This so-called ''invariance problem'' is the computational crux of recognition-it is the major stumbling block for computer vision recognition systems (Pinto et al., 2008a; Ullman, 1996) , particularly when many possible object labels must be entertained. The central importance of the invariance problem is easy to see when one imagines an engineer's task of building a recognition system for a visual world in which invariance was not needed. In such a world, repeated encounters of each object would evoke the same response pattern across the retina as previous encounters. In this world, object identity could easily be determined from the combined responses of the retinal population, and this procedure would easily scale to a nearly infinite number of possible ''objects.'' This is not object recognition, and machine systems that work in these types of worlds already far outperform our own visual system. In the real world, each encounter with an object is almost entirely unique, because of identity-preserving image transformations. Specifically, the vast array of images caused by objects that should receive the same label (e.g., ''car,'' Figure 1 ) results from the variability of the world and the observer: each object can be encountered at any location on the retina (position variability), at a range of distances (scale variability), at many angles relative to the observer (pose variability), at a range lighting conditions (illumination variability), and in new visual contexts (clutter variability). Moreover, some objects are deformable in shape (e.g., bodies and faces), and often we need to group varying three-dimensional shapes into a common category such as ''cars,'' ''faces, '' or ''dogs'' (intraclass variability). In sum, each encounter of the same object activates an entirely different retinal response pattern and the task of the visual system is to somehow establish the equivalence of all of these response patterns while, at the same time, not confuse any of them with images of all other possible objects (see Figure 1 ). Both behavioral (Potter, 1976; Thorpe et al., 1996) and neuronal (Hung et al., 2005) evidence suggests that the visual stream solves this invariance problem rapidly (discussed in section 2). While the limits of such abilities have only been partly characterized (Afraz and Cavanagh, 2008; Bü lthoff et al., 1995; Kingdom et al., 2007; Kravitz et al., 2010 Kravitz et al., , 2008 Lawson, 1999; Logothetis et al., 1994) , from the point of view of an engineer, the brain achieves an impressive amount of invariance to identity-preserving image transformations (Pinto et al., 2010) . Such invariance not only is a hallmark of primate vision, but also is found in evolutionarily less advanced species (e.g., rodents; Tafazoli et al., 2012; Zoccolan et al., 2009) . In sum, the invariance of core object recognition is the right place to drive a wedge into the object recognition problem: it is operationally definable, it is a domain where biological visual systems excel, it is experimentally tractable, and it engages the crux computational difficulty of object recognition. The Invariance of Core Object Recognition: A Graphical Intuition into the Problem A geometrical description of the invariance problem from a neuronal population coding perspective has been effective for motivating hypothetical solutions, including the notion that the ventral visual pathway gradually ''untangles'' information about object identity (DiCarlo and Cox, 2007) . As a summary of those ideas, consider the response of a population of neurons to a particular view of one object as a response vector in a space whose dimensionality is defined by the number of neurons in the population (Figure 2A ). When an object undergoes an identity-preserving transformation, such as a shift in position or a change in pose, it produces a different pattern of population activity, which corresponds to a different response vector (Figure 2A) . Together, the response vectors corresponding to all possible identitypreserving transformations (e.g., changes in position, scale, pose, etc.) define a low-dimensional surface in this high-dimensional space-an object identity manifold (shown, for the sake of clarity, as a line in Figure 2B ). For neurons with small receptive fields that are activated by simple light patterns, such as retinal ganglion cells, each object manifold will be highly curved. Moreover, the manifolds corresponding to different objects will be ''tangled'' together, like pieces of paper crumpled into a ball (see Figure 2B , left panel). At higher stages of visual processing, neurons tend to maintain their selectivity for objects across changes in view; this translates to manifolds that are more flat and separated (more ''untangled'') ( Figure 2B , right panel). Thus, object manifolds are thought to be gradually untangled through nonlinear selectivity and invariance computations applied at each stage of the ventral pathway (DiCarlo and Cox, 2007) . Object recognition is the ability to separate images that contain one particular object from images that do not (images of other possible objects; Figure 1 ). In this geometrical perspective, this amounts to positioning a decision boundary, such as a hyperplane, to separate the manifold corresponding to one object from all other object manifolds. Mechanistically, one can think of the decision boundary as approximating a higher-order neuron that ''looks down'' on the population and computes object identity via a simple weighted sum of each neuron's responses, followed by a threshold. And thus it becomes clear why the representation at early stages of visual processing is problematic for object recognition: a hyperplane is completely insufficient for separating one manifold from the others because it is highly tangled with the other manifolds. However, at later stages, manifolds are flatter and not fused with each other, Figure 2B ), so that a simple hyperplane is all that is needed to separate them. This conceptual framework makes clear that information is not created as signals propagate through this visual system (which is impossible); rather, information is reformatted in a manner that makes information about object identity more explicit-i.e., available to simple weighted summation decoding schemes. Later, we extend insights from object identity manifolds to how the ventral stream might accomplish this nonlinear transformation. Considering how the ventral stream might solve core recognition from this geometrical, population-based, perspective shifts emphasis away from traditional single-neuron response properties, which display considerable heterogeneity in high-level visual areas and are difficult to understand (see section 2). We argue Neuron 73, February 9, 2012 ª2012 Elsevier Inc. 417 Neuron Perspective that this perspective is a crucial intermediate level of understanding for the core recognition problem, akin to studying aerodynamics, rather than feathers, to understand flight. Importantly, this perspective suggests the immediate goal of determining how well each visual area has untangled the neuronal representation, which can be quantified via a simple summation decoding scheme (described above). It redirects emphasis toward determining the mechanisms that might contribute to untanglingand dictates what must be ''explained'' at the single-neuron level, rather than creating ''just so'' stories based on the phenomenologies of heterogenous single neurons. (B) All possible identity-preserving transformations of an object will form a low-dimensional manifold of points in the population vector space, i.e., a continuous surface (represented here, for simplicity, as a one-dimensional trajectory; see red and blue lines). Neuronal populations in early visual areas (retinal ganglion cells, LGN, V1) contain object identity manifolds that are highly curved and tangled together (see red and blue manifolds in left panel). The solution to the recognition problem is conceptualized as a series of successive re-representations along the ventral stream (black arrow) to a new population representation (IT) that allows easy separation of one namable object's manifold (e.g., a car; see red manifold) from all other object identity manifolds (of which the blue manifold is just one example). Geometrically, this amounts to remapping the visual images so that the resulting object manifolds can be separated by a simple weighted summation rule (i.e., a hyperplane, see black dashed line; see DiCarlo and Cox, 2007) . (C) The vast majority of naturally experienced images are not accompanied with labels (e.g., ''car,'' ''plane''), and are thus shown as black points. However, images arising from the same source (e.g., edge, object) tend to be nearby in time (gray arrows). Recent evidence shows that the ventral stream uses that implicit temporal contiguity instruction to build IT neuronal tolerance, and we speculate that this is due to an unsupervised learning strategy termed cortical local subspace untangling (see text). Note that, under this hypothetical strategy, ''shape coding'' is not the explicit goal-instead, ''shape'' information emerges as the residual natural image variation that is not specified by naturally occurring temporal contiguity cues. The Ventral Visual Stream Houses Critical Circuitry for Core Object Recognition Decades of evidence argue that the primate ventral visual processing stream-a set of cortical areas arranged along the occipital and temporal lobes ( Figure 3A )-houses key circuits that underlie object recognition behavior (for reviews, see Gross, 1994; Miyashita, 1993; Orban, 2008; Rolls, 2000) . Object recognition is not the only ventral stream function, and we refer the reader to others (Kravitz et al., 2010; Logothetis and Sheinberg, 1996; Maunsell and Treue, 2006; Tsao and Livingstone, 2008 ) for a broader discussion. Whereas lesions in the posterior ventral stream produce complete blindness in part of the visual field (reviewed by Stoerig and Cowey, 1997), lesions or inactivation of anterior regions, especially the inferior temporal cortex (IT), can produce selective deficits in the ability to distinguish among complex objects 418 Neuron 73, February 9, 2012 ª2012 Elsevier Inc. Perspective (e.g., Holmes and Gross, 1984; Horel, 1996; Schiller, 1995; Weiskrantz and Saunders, 1984; Yaginuma et al., 1982) . While these deficits are not always severe, and sometimes not found at all (Huxlin et al., 2000) , this variability probably depends on the type of object recognition task (and thus the alternative visual strategies available). For example, some (Schiller, 1995; Weiskrantz and Saunders, 1984) , but not all, primate ventral stream lesion studies have explicitly required invariance. While the human homology to monkey IT cortex is not well established, a likely homology is the cortex in and around the human lateral occipital cortex (LOC) (see Orban et al., 2004 for review). For example, a comparison of monkey IT and human ''IT'' (LOC) shows strong commonality in the population representation of object categories (Kriegeskorte et al., 2008). Assuming these homologies, the importance of primate IT is suggested by neuropsychological studies of human patients with temporal lobe damage, which can sometimes produce remarkably specific object recognition deficits (Farah, 1990) . Temporary functional disruption of parts of the human ventral stream (using transcranial magnetic stimulation, TMS) can specifically disrupt certain types of object discrimination tasks, such as face discrimination (Pitcher et al., 2009) . Similarly, artificial activation of monkey IT neurons predictably biases the subject's reported percept of complex objects (Afraz et al., 2006) . In sum, long-term lesion studies, temporary activation/inactivation studies, and neurophysiological studies (described below) all point to the central role of the ventral visual stream in invariant object recognition. Ventral Visual Stream: Multiple, Hierarchically Organized Visual Areas The ventral visual stream has been parsed into distinct visual ''areas'' based on anatomical connectivity patterns, distinctive anatomical structure, and retinotopic mapping (Felleman and Van Essen, 1991) . Complete retinotopic maps have been revealed for most of the visual field (at least 40 degrees eccentricity from the fovea) for areas V1, V2, and V4 (Felleman and Van Essen, 1991) and thus each area can be thought of as conveying a population-based re-representation of each visually presented image. Within the IT complex, crude retinotopy exists over the more posterior portion (pIT; Boussaoud et al., 1991; Yasuda et al., 2010) , but retinotopy is not reported in the central and anterior regions (Felleman and Van Essen, 1991) . Thus, while IT is commonly parsed into subareas such as TEO and TE (Janssen et al., 2000; Saleem et al., 2000 Saleem et al., , 1993 Suzuki et al., 2000; Von Bonin and Bailey, 1947) or posterior IT (pIT), central IT (cIT), and anterior IT (aIT) (Felleman and Van Essen, 1991) , it is unclear if IT cortex is more than one area, or how the term ''area'' should be applied. One striking illustration of this is recent monkey fMRI work, which shows that there are three (Tsao et al., 2003) to six (Tsao et al., 2008a) or more (Ku et al., 2011) smaller regions within IT that may be involved in face ''processing' ' (Tsao et al., 2008b) (also see Op de Beeck et al., 2008; Pinsk et al., 2005) . This suggests that, at the level of IT, behavioral goals (e.g., object categorization) (Kriegeskorte et al., 2008; Naselaris et al., 2009 ) many be a better spatial organizing principle than retinotopic maps. All visual cortical areas share a six-layered structure and the inputs and outputs to each visual area share characteristic patterns of connectivity: ascending ''feedforward'' input is received in layer 4 and ascending ''feedforward'' output originates in the upper layers; descending ''feedback'' originates in the lower layers and is received in the upper and lower layers of the ''lower'' cortical area (Felleman and Van Essen, 1991). Neuron 73, February 9, 2012 ª2012 Elsevier Inc. 419 Neuron Perspective These repeating connectivity patterns argue for a hierarchical organization (as opposed to a parallel or fully interconnected organization) of the areas with visual information traveling first from the retina to the lateral geniculate nucleus of the thalamus (LGN), and then through cortical area V1 to V2 to V4 to IT (Felleman and Van Essen, 1991) . Consistent with this, the (mean) first visually evoked responses of each successive cortical area are successively lagged by 10 ms (Nowak and Bullier, 1997; Schmolesky et al., 1998; see Figure 3B ). Thus, just 100 ms after image photons impinge on the retina, a first wave of imageselective neuronal activity is present throughout much of IT (e.g., Desimone et al., 1984; DiCarlo and Maunsell, 2000; Hung et al., 2005; Kobatake and Tanaka, 1994a; Logothetis and Sheinberg, 1996; Tanaka, 1996) . We believe this first wave of activity is consistent with a combination of intra-area processing and feedforward inter-area processing of the visual image. The only known means of rapidly conveying information through the ventral pathway is via the spiking activity that travels along axons. Thus, we consider the neuronal representation in a given cortical area (e.g., the ''IT representation'') to be the spatiotemporal pattern of spikes produced by the set of pyramidal neurons that project out of that area (e.g., the spiking patterns traveling along the population of axons that project out of IT; see Figure 3B ). How is the spiking activity of individual neurons thought to encode visual information? Most studies have investigated the response properties of neurons in the ventral pathway by assuming a firing rate (or, equivalently, a spike count) code, i.e., by counting how many spikes each neuron fires over several tens or hundreds of milliseconds following the presentation of a visual image, adjusted for latency (e.g., see Figures 4A and 4B) . Historically, this temporal window (here called the ''decoding'' window) was justified by the observation that its resulting spike rate is typically well modulated by relevant parameters of the presented visual images (such as object identity, position, or size; Desimone et al., 1984; Kobatake and Tanaka, 1994b; Logothetis and Sheinberg, 1996; Tanaka, 1996) (see examples of IT neuronal responses in Figures 4A-4C ), analogous to the well-understood firing rate modulation in area V1 by ''low level'' stimulus properties such as bar orientation (reviewed by Lennie and Movshon, 2005) . Like all cortical neurons, neuronal spiking throughout the ventral pathway is variable in the ms-scale timing of spikes, resulting in rate variability for repeated presentations of a nominally identical visual stimulus. This spike timing variability is consistent with a Poisson-like stochastic spike generation process with an underlying rate determined by each particular image (e.g., Kara et al., 2000; McAdams and Maunsell, 1999) . Despite this variability, one can reliably infer what object, among a set of tested visual objects, was presented from the rates elicited across the IT population (e.g., Abbott et al., 1996; Aggelopoulos and Rolls, 2005; De Baene et al., 2007; Heller et al., 1995; Hung et al., 2005; Li et al., 2009; Op de Beeck et al., 2001; Rust and DiCarlo, 2010) . It remains unknown whether the ms-scale spike variability found in the ventral pathway is ''noise'' (in that it does not directly help stimulus encoding/decoding) or if it is somehow synchronized over populations of neurons to convey useful, perhaps ''multiplexed'' information (reviewed by Ermentrout et al., 2008) . Empirically, taking into account the fine temporal structure of IT neuronal spiking patterns (e.g., concatenated decoding windows, each less than 50 ms) does not convey significantly more information about object identity than larger time windows (e.g., a single, 200 ms decoding window), suggesting that the results of ventral stream processing are well described by a firing rate code where the relevant underlying time scale is 50 ms (Abbott et al., 1996; Aggelopoulos and Rolls, 2005; Heller et al., 1995; Hung et al., 2005) . While different time epochs relative to stimulus onset may encode different types of visual information (Brincat and Connor, 2006; Richmond and Optican, 1987; Sugase et al., 1999) , very reliable object information is usually found in IT in the first 50 ms of neuronal response (i.e., 100-150 ms after image onset, see Figure 4A ). More specifically, (1) the population representation is already different for different objects in that window (DiCarlo and Maunsell, 2000) , and (2) responses in that time window are more reliable because peak spike rates are typically higher than later windows (e.g., Hung et al., 2005) . Deeper tests of ms-scale synchrony hypotheses require large-scale simultaneous recording. Another challenge to testing ms-scale spike coding is that alternative putative decoding schemes are typically unspecified and open ended; a more complex scheme outside the range of each technical advance can always be postulated. In sum, while all spike-timing codes cannot easily (if ever) be ruled out, rate codes over 50 ms intervals are not only easy to decode by downstream neurons, but appear to be sufficient to support recognition behavior (see below). The IT Population Appears Sufficient to Support Core Object Recognition Although visual information processing in the first stage of the ventral stream (V1) is reasonably well understood (see Lennie and Movshon, 2005 for review), processing in higher stages (e.g., V4, IT) remains poorly understood. Nevertheless, we know that the ventral stream produces an IT pattern of activity that can directly support robust, real-time visual object categorization and identification, even in the face of changes in object position and scale, limited clutter, and changes in background context (Hung et al., 2005; Li et al., 2009; Rust and DiCarlo, 2010) . Specifically, simple weighted summations of IT spike counts over short time intervals (see section 2) lead to high rates of cross-validated performance for randomly selected populations of only a few hundred neurons (Hung et al., 2005; Rust and DiCarlo, 2010) (Figure 4E) , and a simple IT weighted summation scheme is sufficient to explain a wide range of human invariant object recognition behavior (Majaj et al., 2012) . Similarly, studies of fMRI-targeted clusters of IT neurons suggest that IT subpopulations can support other object recognition tasks such as face detection and face discrimination over some identity-preserving transformations (Freiwald and Tsao, 2010) . Importantly, IT neuronal populations are demonstrably better at object identification and categorization than populations at earlier stages of the ventral pathway (Freiwald and Tsao, 2010; Hung et al., 2005; Li et al., 2009; Rust and DiCarlo, 2010) . Similarly, while neuronal activity that provides some discriminative information about object shape has also been found in dorsal stream visual areas at similar hierarchical levels 420 Neuron 73, February 9, 2012 ª2012 Elsevier Inc. Perspective (Sereno and Maunsell, 1998 ), a direct comparison shows that it is not nearly as powerful as IT for object discrimination (Lehky and Sereno, 2007) . Taken together, the neurophysiological evidence can be summarized as follows. First, spike counts in 50 ms IT decoding windows convey information about visual object identity. Second, this information is available in the IT population beginning 100 ms after image presentation (see Figure 4A) . Third, the IT neuronal representation of a given object across changes in position, scale, and presence of limited clutter is untangled A) Poststimulus spike histogram from an example IT neuron to one object image (a chair) that was the most effective among 213 tested object images (Zoccolan et al., 2007) . (B) Left: the mean responses of the same IT neuron to each of 213 object images (based on spike rate in the gray time window in A). Object images are ranked according to their effectiveness in driving the neuron. As is typical, the neuron responded strongly to 10% of objects images (four example images of nearly equal effectiveness are shown) and was suppressed below background rate by other objects (two example images shown), with no obvious indication of what critical features triggered or suppressed its firing. Colors indicate highly effective (red), medium-effective (blue), and poorly effective (green) images. Right: data from a second study (new IT neuron) using natural images patches to illustrate the same point (Rust and DiCarlo, unpublished). (C) Response profiles from an example IT neuron obtained by varying the position (elevation) of three objects with high (red), medium (blue), and (low) effectiveness. While response magnitude is not preserved, the rank-order object identity preference is maintained along the entire tested range of tested positions. (D) To explain data in (C), each IT neuron (right panel) is conceptualized as having joint, separable tuning for shape (identity) variables and for identity-preserving variables (e.g., position). If a population of such IT neurons tiles that space of variables (left panel), the resulting population representation conveys untangled object identity manifolds ( Figure 2B
DiCarlo et al. REF review the evidences of neurons and computational models, and discover the computation process which can mimic powerful neuronal representations in the inferior temporal cortex.
How Does the Brain Solve Visual Object Recognition?
Neuron
Neuron
Psychology Medicine Computer Science