Venue: ICLR 2017 workshop

Reviews:

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content:  Introducing the advantages of weight pruning and sparsity, in decreasing number of multiplications, into the Winograd domain results in a larger decrease in number  of multiplications required per forward pass. 
The authors propose training the kernel in transform domain and also pruning the weights in the same thereby enabling Winograd transform to be used with sparse weights. 
The Relu is now applied after Winograd transform for not loosing the advantage of operating on sparse activations.

Experimental validation supports the authors claim of decrease in number of multiplications by using proposed approach.
review_title: simple yet quite useful approach for reducing the number of multiplications required in a CNN.

final_score: rating: 7: Good paper, accept
review_content: The paper presents two tricks to reduce the number of multiplications needed to compute convolutions with small kernels. First, the trainable filters are stored directly in the Winograd domain, rather than stored in feature domain and then transformed. This directly impacts the number of multiplications that need to be performed. Second, the  Winograd transformation and ReLU are swapped for network activations bringing even more sparsity into the Winograd domain. Oddly enough this change doesn't affect the performance of the networks, even though they no longer perform convolutions followed by ReLUs. The two tricks reduce the number of multiplications 2.2 times over a pruned network, while losing little accuracy.

The paper is a nice starting point for investigation of possivle speedups and efficient implementation of pruned networks. The empirical evaluation is slightly underwhelming, but sufficient for a workshop paper. I would like to see more evidence that the ReLU trick works (it drastically changes the semantics of the network). I would also want to see baselines such as smaller pruned networks. 

Nit: I found Figure 1 hard to read and would prefer to have it accompanied by the equations for the computations performed.
review_title: Nice trick to reduce the number of multiplications required to evaluate a CNN

Decision: Accept