Venue: ICLR 2018 conference

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper introduces sparse persistent RNNs, a mechanism to add pruning to the existing work of stashing RNN weights on a chip. The paper describes the use additional mechanisms for synchronization and memory loading. 

The evaluation in the main paper is largely on synthetic workloads (i.e. large layers with artificial sparsity).  With evaluation largely over layers instead of applications, I was left wondering whether there is an actual benefit on real workloads. Furthermore, the benefit over dense persistent RNNs for OpenNMT application (of absolute 0.3-0.5s over dense persistent rnns?) did not appear significant unless you can convince me otherwise. 

Storing weights persistent on chip should give a sharp benefit when all weights fit on the chip. One suggestion I have to strengthen the paper is to claim that due to pruning, now you can support a larger number of methods or method configurations and to provide examples of those.

To summarize, the paper adds the ability to support pruning over persistent RNNs. However, Narang et. al., 2017 already explore this idea, although briefly. Furthermore, the gains from the sparsity appear rather limited over real applications. I would encourage the authors to put the NMT evaluation in the main paper (and perhaps add other workloads). Furthermore, a host of techniques are discussed (Lamport timestamps, memory layouts) and implementing them on GPUs is not trivial. However, these are well known and the novelty or even the experience of implementing these on GPUs should be emphasized.
review_title: Sparse Persistent RNNs Review: Limited novelty over persistent RNNs

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposes improving performance of large RNNs by combing techniques of model pruning and persistent kernels. The authors further propose model-pruning optimizations which are aware of the persistent implementation.

It's not clear if the paper is relevant to the ICLR audience due to its emphasize on low-level optimization which has little insight in learning representations. The exposition in the paper is also not well-suited for people without a systems background, although I'll admit I'm mostly using myself as a proxy for the average machine learning researcher here. For instance, the authors could do more to explain Lamport Timestamps than a 1974 citation.

Modulo problems of relevance and expected audience, the paper is well-written and presents useful improvements in performance of large RNNs, and the work has potential for impact in industrial applications of RNNs. The work is clearly novel, and the contributions are clear and well-justified using experiments and ablations.
review_title: Novel and impactful contributions, but unclear relevance and expected audience

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper devises a sparse kernel for RNNs which is urgently needed because current GPU deep learning libraries (e.g., CuDNN) cannot exploit sparsity when it is presented and because a number of works have proposed to sparsify/prune RNNs so as to be able to run on devices with limited compute power (e.g., smartphones). Unfortunately, due to the low-level and GPU specific nature of the work, I would think that this work will be better critiqued in a more GPU-centric conference. Another concern is that while experiments are provided to demonstrate the speedups achieved by exploiting sparsity, these are not contrasted by presenting the loss in accuracy caused by introducing sparsity (in the main portion of the paper). It may be the case by reducing density to 1% we can speedup by N fold but this observation may not have any value if the accuracy becomes  abysmal.

Pros:
- Addresses an urgent and timely issue of devising sparse kernels for RNNs on GPUs
- Experiments show that the kernel can effectively exploit sparsity while utilizing GPU resources well

Cons:
- This work may be better reviewed at a more GPU-centric conference
- Experiments (in main paper) only show speedups and do not show loss of accuracy due to sparsity
review_title: The paper devises sparse GPU kernels for RNNs

Meta-review: The reviewers find the work interesting and well made, but are concerned that ICLR is not the right venue for the work.  I will recommend that the paper be accepted, but ask the authors to add the NMT results to the main paper (any other non-synthetic applications they could add would be helpful).
Decision: Accept (Poster)