In this task, you are given the abstract of a research paper. Your task is to generate a summary of this abstract. Your summary should not be very short, but it's better if it's not more than 30 words.
--------
Question: The weight initialization and the activation function of deep neural networks have a crucial impact on the performance of the training procedure. An inappropriate selection can lead to the loss of information of the input during forward propagation and the exponential vanishing/exploding of gradients during back-propagation. Understanding the theoretical properties of untrained random networks is key to identifying which deep networks may be trained successfully as recently demonstrated by Schoenholz et al. (2017) who showed that for deep feedforward neural networks only a specific choice of hyperparameters known as the `edge of chaos' can lead to good performance.
 We complete this analysis by providing quantitative results showing that, for a class of ReLU-like activation functions, the information propagates indeed deeper for an initialization at the edge of chaos. By further extending this analysis, we identify a class of activation functions that improve the information propagation over ReLU-like functions. This class includes the Swish activation, $\phi_{swish}(x) = x \cdot \text{sigmoid}(x)$, used in Hendrycks & Gimpel (2016),
 Elfwing et al. (2017) and Ramachandran et al. (2017). This provides a theoretical grounding for the excellent empirical performance of $\phi_{swish}$ observed in these contributions. We complement those previous results by illustrating the benefit of using a random initialization on the edge of chaos in this context.

Answer: How to effectively choose Initialization and Activation function for deep neural networks


Question: Recent work has shown that deep reinforcement-learning agents can learn to follow language-like instructions from infrequent environment rewards. However, this places on environment designers the onus of designing language-conditional reward functions which may not be easily or tractably implemented as the complexity of the environment and the language scales. To overcome this limitation, we present a framework within which instruction-conditional RL agents are trained using rewards obtained not from the environment, but from reward models which are jointly trained from expert examples.   As reward models improve, they learn to accurately reward agents for completing tasks for environment configurations---and for instructions---not present amongst the expert data. This framework effectively separates the representation of what instructions require from how they can be executed.
 In a simple grid world, it enables an agent to learn a range of commands requiring interaction with blocks and understanding of spatial relations and underspecified abstract arrangements. We further show the method allows our agent to adapt to changes in the environment without requiring new expert examples.

Answer: We propose AGILE, a framework for training agents to perform instructions from examples of respective goal-states.


Question: Compressed forms of deep neural networks are essential in deploying large-scale
 computational models on resource-constrained devices. Contrary to analogous
 domains where large-scale systems are build as a hierarchical repetition of small-
 scale units, the current practice in Machine Learning largely relies on models with
 non-repetitive components. In the spirit of molecular composition with repeating
 atoms, we advance the state-of-the-art in model compression by proposing Atomic
 Compression Networks (ACNs), a novel architecture that is constructed by recursive
 repetition of a small set of neurons. In other words, the same neurons with the
 same weights are stochastically re-positioned in subsequent layers of the network.
 Empirical evidence suggests that ACNs achieve compression rates of up to three
 orders of magnitudes compared to fine-tuned fully-connected neural networks (88×
 to 1116× reduction) with only a fractional deterioration of classification accuracy
 (0.15% to 5.33%). Moreover our method can yield sub-linear model complexities
 and permits learning deep ACNs with less parameters than a logistic regression
 with no decline in classification accuracy.

Answer:
We advance the state-of-the-art in model compression by proposing Atomic Compression Networks (ACNs), a novel architecture that is constructed by recursive repetition of a small set of neurons.