TASK DEFINITION: In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
PROBLEM: We present a novel approach for training neural abstract architectures which in- corporates (partial) supervision over the machine’s interpretable components. To cleanly capture the set of neural architectures to which our method applies, we introduce the concept of a differential neural computational machine (∂NCM) and show that several existing architectures (e.g., NTMs, NRAMs) can be instantiated as a ∂NCM and can thus benefit from any amount of additional supervision over their interpretable components. Based on our method, we performed a detailed experimental evaluation with both, the NTM and NRAM architectures, and showed that the approach leads to significantly better convergence and generalization capabilities of the learning phase than when training using only input-output examples.


SOLUTION: We increase the amount of trace supervision possible to utilize when training fully differentiable neural machine architectures.

PROBLEM: Asynchronous distributed gradient descent algorithms for training of deep neural
 networks are usually considered as inefficient, mainly because of the Gradient delay
 problem. In this paper, we propose a novel asynchronous distributed algorithm
 that tackles this limitation by well-thought-out averaging of model updates, computed
 by workers. The algorithm allows computing gradients along the process
 of gradient merge, thus, reducing or even completely eliminating worker idle time
 due to communication overhead, which is a pitfall of existing asynchronous methods.
 We provide theoretical analysis of the proposed asynchronous algorithm,
 and show its regret bounds. According to our analysis, the crucial parameter for
 keeping high convergence rate is the maximal discrepancy between local parameter
 vectors of any pair of workers. As long as it is kept relatively small, the
 convergence rate of the algorithm is shown to be the same as the one of a sequential
 online learning. Furthermore, in our algorithm, this discrepancy is bounded
 by an expression that involves the staleness parameter of the algorithm, and is
 independent on the number of workers. This is the main differentiator between
 our approach and other solutions, such as Elastic Asynchronous SGD or Downpour
 SGD, in which that maximal discrepancy is bounded by an expression that
 depends on the number of workers, due to gradient delay problem. To demonstrate
 effectiveness of our approach, we conduct a series of experiments on image
 classification task on a cluster with 4 machines, equipped with a commodity communication
 switch and with a single GPU card per machine. Our experiments
 show a linear scaling on 4-machine cluster without sacrificing the test accuracy,
 while eliminating almost completely worker idle time. Since our method allows
 using commodity communication switch, it paves a way for large scale distributed
 training performed on commodity clusters.

SOLUTION: A method for an efficient asynchronous distributed training of deep learning models along with theoretical regret bounds.

PROBLEM: Model-free reinforcement learning (RL) has been proven to be a powerful, general tool for learning complex behaviors. However, its sample efficiency is often impractically large for solving challenging real-world problems, even for off-policy algorithms such as Q-learning. A limiting factor in classic model-free RL is that the learning signal consists only of scalar rewards, ignoring much of the rich information contained in state transition tuples. Model-based RL uses this information, by training a predictive model, but often does not achieve the same asymptotic performance as model-free RL due to model bias. We introduce temporal difference models (TDMs), a family of goal-conditioned value functions that can be trained with model-free learning and used for model-based control. TDMs combine the benefits of model-free and model-based RL: they leverage the rich information in state transitions to learn very efficiently, while still attaining asymptotic performance that exceeds that of direct model-based RL methods. Our experimental results show that, on a range of continuous control tasks, TDMs provide a substantial improvement in efficiency compared to state-of-the-art model-based and model-free methods.

SOLUTION:
We show that a special goal-condition value function trained with model free methods can be used within model-based control, resulting in substantially better sample efficiency and performance.