In the past five years we have observed the rise of incredibly well performing feed-forward neural networks trained supervisedly for vision related tasks. These models have achieved super-human performance on object recognition, localisation, and detection in still images. However, there is a need to identify the best strategy to employ these networks with temporal visual inputs and obtain a robust and stable representation of video data. Inspired by the human visual system, we propose a deep neural network family, CortexNet, which features not only bottom-up feedforward connections, but also it models the abundant top-down feedback and lateral connections, which are present in our visual cortex. We introduce two training schemes -the unsupervised MatchNet and weakly supervised TempoNet modes -where a network learns how to correctly anticipate a subsequent frame in a video clip or the identity of its predominant subject, by learning egomotion clues and how to automatically track several objects in the current scene. Find the project website at tinyurl.com/CortexNet.
Canziani and Culurciello proposed the CortexNet neural architecture REF in order to obtain a robust and stable representation of temporal visual inputs.
CortexNet: a Generic Network Family for Robust Visual Temporal Representations
ArXiv
ArXiv
Computer Science