Modulation of early visual processing alleviates capacity limits in solving multiple tasks Sushrut Thorat1, Giacomo Aldegheri1, Marcel A. J. Van Gerven, Marius V. Peelen 1 Equal contribution Donders Institute, Radboud University, Nijmegen, The Netherlands Correspondence: s.thorat & g.aldegheri @donders.ru.nl E X P E R I M E N T A L M A N I P U L A T I O N R E S U LT S C O N C L U S I O N S R E F E R E N C E S N E T W O R K D E S C R I P T I O N P R O B L E M D E S C R I P T I O N Cheung, B., Terekhov, A., Chen, Y., Agrawal, P., & Olshausen, B. (2019). Superposition of many models into one. arXiv preprint arXiv:1902.05522. Gilbert, C. D., & Li, W. (2013). Top-down influences on visual processing. Nature Reviews Neuroscience, 14 (5), 350–363. Thorat, S., van Gerven, M., & Peelen, M. (2018). The functional role of cue-driven feature-based feedback in object recognition. In Conference on Cognitive Computational Neuroscience, CCN 2018 (pp. 1–4) Identity? Emotion? modulation Object detection as multiple one vs. all classification tasks (is this object present/absent?) • Objects: MNIST digits and their permutations When could early modulation be beneficial, in addition to late modulation, in a neural network? (i.e. overcome its wiring cost) When there are capacity limits, caused by e.g.: Small number of neurons in the network Large number of tasks the network has to perform How does the detection performance boost provided by the addition of early modulation vary with the number of tasks and units in the network? Main contrast: modulating early & late neural processing (L1 → L2 & L2 → L3) vs. modulating late only (L2 → L3) • Measure of interest: detection performance • Manipulation: number of tasks (100, 250, 500), and number of units in the hidden layer (32, 64, 128) The performance boost provided by additionally modulating early processing is • directly proportional to the number of tasks • inversely proportional to number of units in hidden layer M E T H O D S In a multi-task scenario, extracting features common to all tasks might not always be possible. Modulating early visual processing can allow early selection of task-relevant information (Gilbert & Li, 2013) 784 N CUE L1 L2 L3 x3 = [W3(g2 ∘x2)+ b3]+ L1 → L2 L2 → L3 Decision (Present / Absent) • MLP with one hidden layer • Variable number of units (N) in the hidden layer • One-hot cue indicates the task (which digit to detect) • Cue modulates transformations via bias and gain • Main network and modulation are trained jointly • Cue either modulates only the late or both early & late transformations Modulation of early neural processing increases multi-tasking performance in conditions of limited network capacity. Are the advantages of early modulation dependent on a neural network’s capacity limits? Capacity limits can be a result of there being limited number of neurons in the network and/or too many (difficult) tasks. Is this the role of early sensory modulation in biological brains? Forgoing wiring costs in favour of optimality? Further work should look at more natural datasets and tasks: Permuted-MNIST limits the sharing of early visual features In a naturalistic dataset (CIFAR, ImageNet) the effect of early modulation might be different because low-level features are shared across categories Bias modulation does not aid performance on top of gain modulation. e.g. Network (32 units, 25 tasks, early+late) trained with: • gain only modulation: 94.8% • bias only modulation: 90.9% • bias & gain modulation: 94.7% I N T R O D U C T I O N Increasing task difficulty (adding noise to digits) also increases performance boost. Modulation Permutation 1 Permutation 2