In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
--------
Question: Reinforcement learning algorithms rely on carefully engineered rewards from the environment that are extrinsic to the agent. However, annotating each environment with hand-designed, dense rewards is difficult and not scalable, motivating the need for developing reward functions that are intrinsic to the agent. 
 Curiosity is such intrinsic reward function which uses prediction error as a reward signal. In this paper: (a) We perform the first large-scale study of purely curiosity-driven learning, i.e. {\em without any extrinsic rewards}, across $54$ standard benchmark environments, including the Atari game suite. Our results show surprisingly good performance as well as a high degree of alignment between the intrinsic curiosity objective and the hand-designed extrinsic rewards of many games. (b) We investigate the effect of using different feature spaces for computing prediction error and show that random features are sufficient for many popular RL game benchmarks, but learned features appear to generalize better (e.g. to novel game levels in Super Mario Bros.). (c) We demonstrate limitations of the prediction-based rewards in stochastic setups. Game-play videos and code are at https://doubleblindsupplementary.github.io/large-curiosity/.

Answer: An agent trained only with curiosity, and no extrinsic reward, does surprisingly well on 54 popular environments, including the suite of Atari games, Mario etc.


Question: We provide a theoretical algorithm for checking local optimality and escaping saddles at nondifferentiable points of empirical risks of two-layer ReLU networks. Our algorithm receives any parameter value and returns: local minimum, second-order stationary point, or a strict descent direction. The presence of M data points on the nondifferentiability of the ReLU divides the parameter space into at most 2^M regions, which makes analysis difficult. By exploiting polyhedral geometry, we reduce the total computation down to one convex quadratic program (QP) for each hidden node, O(M) (in)equality tests, and one (or a few) nonconvex QP. For the last QP, we show that our specific problem can be solved efficiently, in spite of nonconvexity. In the benign case, we solve one equality constrained QP, and we prove that projected gradient descent solves it exponentially fast. In the bad case, we have to solve a few more inequality constrained QPs, but we prove that the time complexity is exponential only in the number of inequality constraints. Our experiments show that either benign case or bad case with very few inequality constraints occurs, implying that our algorithm is efficient in most cases.

Answer: A theoretical algorithm for testing local optimality and extracting descent directions at nondifferentiable points of empirical risks of one-hidden-layer ReLU networks.


Question: Transferring knowledge across tasks to improve data-efficiency is one of
 the open key challenges in the area of global optimization algorithms. Readily
 available algorithms are typically designed to be universal optimizers and, thus,
 often suboptimal for specific tasks. We propose a novel transfer learning method to
 obtain customized optimizers within the well-established framework of Bayesian
 optimization, allowing our algorithm to utilize the proven generalization
 capabilities of Gaussian processes. Using reinforcement learning to meta-train an
 acquisition function (AF) on a set of related tasks, the proposed method learns to
 extract implicit structural information and to exploit it for improved data-efficiency.
 We present experiments on a sim-to-real transfer task as well as on several simulated
 functions and two hyperparameter search problems. The results show that our
 algorithm (1) automatically identifies structural properties of objective functions
 from available source tasks or simulations, (2) performs favourably in settings with
 both scarse and abundant source data, and (3) falls back to the performance level
 of general AFs if no structure is present.

Answer:
We perform efficient and flexible transfer learning in the framework of Bayesian optimization through meta-learned neural acquisition functions.