Untangling invariant object

Despitetremendousvariationintheappearanceofvisualobjects, primates can recognize a multitude of objects,each in a fraction of a second, with no apparent effort.However, the brain mechanisms that enable this funda-mentalabilityarenotunderstood.Drawingonideasfromneurophysiology and computation, we present a grap-hical perspective on the key computational challenges ofobjectrecognition,andarguethattheformatofneuronalpopulation representation and a property that we term‘object tangling’ are central. We use this perspective toshow that the primate ventral visual processing streamachieves a particularly effective solution in which single-neuroninvarianceisnotthegoal.Finally,wespeculateonthe key neuronal mechanisms that could enable thissolution, which, if understood, would have far-reachingimplications for cognitive neuroscience.IntroductionOur daily activities rely heavily on the accurate and rapididentiﬁcation of objects in our visual environment. Theapparenteaseofwithwhichwerecognizeobjectsbeliesthemagnitude of this feat: we effortlessly recognize objectsfrom among tens of thousands of possibilities and we do sowithin a fraction of a second, in spite of tremendousvariation in the appearance of each one. Understandingthebrain mechanisms thatunderlie thisabilitywould bealandmark achievement in neuroscience.Object recognition is computationally difﬁcult for manyreasons, but the most fundamental is that any individualobject can produce an inﬁnite set of different images on theretina, due to variation in object position, scale, pose andillumination, and the presence of visual clutter (e.g. [1–5]). Indeed, althoughwe typically seean objectmany times,weeffectively never see the same exact image on our retinatwice.Althoughseveralcomputationaleffortshaveattackedthisso-called‘invarianceproblem’(e.g.[1,3,6–12]),arobust,real-world machine solution still evades us and we lack asatisfyingunderstandingofhowtheproblemissolvedbythebrain. We believe that these two achievements will beaccomplished nearly simultaneously by an approach thattakes into account both the computational issues and thebiological clues and constraints.Because it is easy to get lost in the sea of previousstudies and ideas, the goal of this manuscript is to clearthetable,bringforthkeyideasinthecontextoftheprimatebrain, and pull those threads together into a coherentframework. Below, we use a graphical perspective to pro-vide intuition about the object recognition problem, showthat the primate ventral visual processing stream pro-duces a particularly effective solution in the inferotem-poral (IT) cortex, and speculate on how the ventral visualstream approaches the problem. Along the way, we arguethat some approaches are only tangential to, or evendistract from, understanding object recognition.What is object recognition? We deﬁne object recognition as the ability to accuratelydiscriminate each named object (‘identiﬁcation’) or set ofobjects (‘categorization’) from all other possible objects,materials, textures other visual stimuli, and to do thisover a range of identity-preserving transformations oftheretinalimageofthatobject(e.g.imagetransformationsresulting from changes in object position, distance, andpose). Of course, vision encompasses many disparate chal-lenges that may interact with object recognition, such asmaterial and texture recognition, object similarity esti-mation,objectsegmentation,objecttrackingandtrajectoryprediction. Exploring such possible interactions is not ourgoal. Instead, we aim to see how far a clear focus on theproblem of object recognition will take us. We concentrateon what we believe to be the core of the brain’s recognitionsystem – the ability to rapidly report object identity orcategory after just a single brief glimpse of visual input(<300 ms; see Box 1) [13,14]. What computational processes must underlie objectrecognition? Tosolvearecognitiontask,asubjectmustusesomeinternalneuronal representation of the visual scene (populationpatternofactivity)tomakeadecision(e.g.[15,16]):isobjectA present or not? Computationally, the brain must apply adecision function [16] to divide an underlying neuronalrepresentational space into regions where object A is pre-sentandregionswhereitisnot(Figure1b;onefunctionforeach object to be potentially reported). Because brainscompute with neurons, the subject must have neuronssomewhere in its nervous system – ‘read-out’ neurons –that can successfully report if object A was present [17]. Ofcourse, there are many relevant mechanistic issues, forexample, how many such neurons are involved in comput-ing the decision, where are they in the brain, is theiroperation ﬁxed or dynamically created with the task atOpinion