Learning to predict visual brain activity by predicting future sensory states
Deep predictive coding networks are neuroscience-inspired unsupervised learning models that learn to predict future sensory states.We build upon the PredNet implementation by Lotter, Kreiman, and Cox to investigate if predictive coding representations are useful to predict brain activity in the visual cortex.We use representational similarity analysis to compare PredNet representations to functional magnetic resonance imaging and magnetoencephalography data from the Algonauts Project.In contrast to previous findings in the literature, we report empirical data suggesting that unsupervised models trained to predict frames of videos without further fine-tuning may outperform supervised image classification baselines in terms of correlation to spatial and temporal data.
We show empirical evidence that predictive coding models yield representations more correlated to brain data than supervised image recognition models.