Discovering the visual features and representations used by the brain to recognize objects is a central problem in the study of vision. Recently, neural network models of visual object recognition, including biological and deep network models, have shown remarkable progress, which starts to rival human performance in some challenging tasks. These models are trained on image examples, learn to extract features and representations, and use them for categorization. It remains unclear, however, whether the representations and learning processes discovered by current models are similar to the ones used by the human visual system. Here we show, by introducing and using minimal recognizable images, that the human visual system uses features and processes not used by current models, and which are critical for recognition. We found by psychophysical studies that at the level of minimal recognizable images, a minute change of the image can have a drastic effect on recognition, identifying features that are critical for the task. Simulations then showed that current models cannot explain this sensitivity to precise feature configurations, and more generally, do not learn to recognize minimal images at a human level. The role of the features shown here is revealed uniquely at the minimal level, where the contribution of each feature is essential. A full understanding of the learning and use of such features will extend our understanding of visual recognition and its cortical mechanisms, and enhance the capacity of computational models to learn from visual experience and deal with recognition and detailed image interpretation. Discovering the visual features and representations used by the brain to recognize objects is a central problem in the study of vision. The successes of recent computational models of visual recognition naturally raise the question: do computer systems and the human brain use similar or different computations? We show by combining a novel method ('minimal images') and simulations that the human recognition system uses features and learning processes not used by current models, which are critical for recognition. The study uses a 'phase transition' phenomenon in minimal images, where minor changes to the image make a drastic effect on its recognition. The results show fundamental limitations of current approaches and suggest directions to produce more realistic and better performing models. \body
Ullman et al. REF showed that minor changes to the image, which are barely perceptible to humans, can have drastic effects on computational recognition accuracy.
Atoms of recognition in human and computer vision
Proceedings of the National Academy of Sciences of the United States of America
Proceedings of the National Academy of Sciences of the United States of America
Computer Science Medicine