Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: The paper used the lottery ticket hypothesis to study the over-parameterization of deep neural networks (DNNs). The main idea is that overparametrization increases the probability of a “lucky” sub-network initialization being present rather than by helping the optimization process. 

The paper conducted experiments to evaluate whether “winning ticket” initializations exist in two different domains: natural language processing (NLP) and reinforcement learning (RL). The authors confirm that winning ticket initializations
generally outperform parameter-matched random initializations, even at extreme pruning rates for both NLP and RL. The results suggest that the lottery ticket
hypothesis is not restricted to supervised learning

The similarity between supervised learning and RL and NLP problem is obvious from a function approximation and optimization point of view. The paper is empirical in nature, and do not offer any additional insight. 
The experiments are not very conclusive. 
review_title: Official Blind Review #1

final_score: rating: 3: Weak Reject
review_content: This paper studies the existence of “winning ticket” initialization in natural language processing (NLP) and reinforcement learning (RL). The lottery ticket hypothesis has been found effective in over-parameterized deep neural networks, which provides better sub-network initialization if not outperforming the original full network. Experiments show that winning ticket initializations generally outperform parameter-matched random initializations on recurrent LSTM models, large-scale Transformer, and discrete-action space RL tasks, including both classic control and pixel control.  A substantial number of parameters can be saved.

This paper is clearly motivated and easy to follow.  The results are interesting.  However, my major concern is its intellectual merit.  The paper does not seem to propose any new algorithm, and the application of lottery ticket (late rewinding) to LSTM, Transformer, and RL looks quite straightforward. There are a few insights drawn from the experiment, such as when only the Transformer weights are pruned.  However they does not appear substantial. Overall, I do not find the paper innovative enough for publication at top conferences like ICLR.  I understand this is subjective, so I leave it to the AC for further evaluation.
review_title: Official Blind Review #2

final_score: rating: 6: Weak Accept
review_content: This paper describes an application of lottery ticket hypothesis to NLP and RL problems. An extensive set of experiments on very strong baseline models on language modeling, machine translation and atari games demonstrates that lottery ticket hypothesis is not only present in feed-forward and convolutional nets on image classification tasks as demonstrated originally in papers by Frankle & Carbin, 2019 and Frankle et al 2019. I don't have any major complaints regarding this work and I believe it is well executed.

For NLP problems it would be nice to have additional ablation studies comparing lottery ticket hypothesis pruning with methods like distillation (Hinton et al 2014) and attention pruning (Michel et al 2019 Are Sixteen Heads Really Better than One?). 

For RL it is quite interesting that pruning weights sometimes improves the final scores on Atari games like Berzerk, Kangaroo, Krull and Centipede perhaps due to exploration. As a future work it would be interesting to see a way to using lottery ticket hypothesis to guide exploration in RL.

Overall it is well executed paper, although it is mainly an application of existing lottery ticket hypothesis techniques to NLP and RL.
review_title: Official Blind Review #3

Meta-review: This paper explores the application of the lottery ticket hypothesis to NLP and RL problems for better initialisations of deep networks and reduced model sizes. This is evaluated in a variety of settings, including continuous control and ATARI games for RL, and LSTMs and Transformers for NLP, showing very positive results.

The main issue raised by the reviewers was the lack of algorithmic novelty in the paper. Despite this, I believe the paper to present an important contribution that could stimulate much additional research. The paper is well written and the results are rigorous and interesting. For these reasons I recommend acceptance.
Decision: Accept (Poster)