We propose an alternative design of memory augmented neural networks (MANNs) called Labeled Memory Networks (LMNs) suited for tasks requiring fast adaptation of batch-trained classification models. LMNs have two key differences with existing MANNs. First, LMNs organize the memory with the discrete class label as the primary key unlike the existing practice of key being a real vector derived from the input. Second, LMNs treat memory as a second boosted stage following a regular neural network thereby allowing the memory and the primary network to play complementary roles. Unlike existing MANNs that write to memory for every instance and use LRU based memory replacement, LMNs write only for instances with non-zero loss and use label-based memory replacement. These properties make them particularly suited for classification tasks requiring fast adaptation and memorizing rare events. We demonstrate significant accuracy gains on three such tasks: online sequence prediction, life-long learning of rare events, and few-shot learning. Recently, a new architecture in the form of Memory augmented neural networks (MANNs) has emerged as a partial solution to the above challenges. Memory networks were originally developed with goals far loftier than plain supervised learning. For example, in [7] , an end-to-end differentiable memory network is used to learn basic tasks and in [28] memory is used to memorize facts relevant 31st
Shankar et al. REF organized the memory with the discrete class label as the primary key unlike the previous key being a real vector derived from the input.
Label Organized Memory Augmented Neural Network
ArXiv
ArXiv
Mathematics Computer Science