Hybrid computing using a neural network with dynamic external memory
Authors: Demis Hassabis
Helen King
Tim Harley
Ivo Danihelka
Agnieszka Grabska-Barwińska
John Agapiou
Yori Zwols
Malcolm Reynolds
Edward Grefenstette
Karl Moritz Hermann
Tiago Ramalho
Adrià Puigdomènech Badia
Christopher Summerfield
Greg Wayne
Sergio Gómez Colmenarejo
Phil Blunsom
Alex Graves
Adam Cain
Georg Ostrovski
Koray Kavukcuoglu
Journal: Nature (2016)
Discipline: Multidisciplinary
Keywords: Artificial neural networks; sensory processing; sequence learning; reinforcement learning; external memory
Affiliations: Google DeepMind, 5 New Street Square, London EC4A 3TW, UK.

Abstract:  Artificial neural networks are remarkably adept at sensory processing, sequence learning and reinforcement learning, but are limited in their ability to represent variables and data structures and to store data over long timescales, owing to the lack of an external memory. Here we introduce a machine learning model called a differentiable neural computer (DNC), which consists of a neural network that can read from and write to an external memory matrix, analogous to the random-access memory in a conventional computer. Like a conventional computer, it can use its memory to represent and manipulate complex data structures, but, like a neural network, it can learn to do so from data. When trained with supervised learning, we demonstrate that a DNC can successfully answer synthetic questions designed to emulate reasoning and inference problems in natural language. We show that it can learn tasks such as finding the shortest path between specified points and inferring the missing links in randomly generated graphs, and then generalize these tasks to specific graphs such as transport networks and family trees. When trained with reinforcement learning, a DNC can complete a moving blocks puzzle in which changing goals are specified by sequences of symbols. Taken together, our results demonstrate that DNCs have the capacity to solve complex, structured tasks that are inaccessible to neural networks without external read-write memory. 

Retracted: No

PubPeer comments:
 I sent a letter to Editor-in-Chief of Nature titled: A Report to Nature about Technical Flaws of Post-Selections Using Test Sets (PSUTS) Charge C1: Post-Selection. Their experimental protocols should have included a stage, called Post-Selections Using Test Sets (PSUTS). In the simplest form, all the data were divided into two disjoint sets, T and Tu2032. In a training stage, n systems where trained to fit the training set T, each starting from a different set of weights determined by a different random seed. Next, in the Post-Selection stage, all the n systems were tested on the test set Tu2032 and finally only the performance of the luckiest system that has the best performance on the test set Tu2032 was reported in the corresponding paper, but not the performances of the remaining less lucky n u2212 1 systems. This is a version of u201ctesting on the training setu201d as textbook Duda, Hart Stork, Pattern Classification (Wiley, New York, 2001) warned against, because the Post-Selection stage utilized the test set Tu2032. Charge C2: Lack of transparency. There is a lack of transparency in reporting the Post-Selection stage. The paper wrote:  20 replicated training runs with different random-number seeds for a DNC and LSTM ... A single DNC was ... some failures to satisfy all constraints (incomplete).u201d However, the paper still lacks due transparency about the Post-Selection stage. For example, do the performance data, including those in Fig. 4(b), correspond to the luckiest system among these n = 20 trained systems? If the answer is positive, what is the distribution of performances of other less lucky 20 u2212 1 = 19 systems? For more detail, see http: www.cse.msu.edu ~weng research 2021-06-28-Report-to-Nature-specific-PSUTS.pdf ,