Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: The plots suggest that the proposed FacNet is almost always outperformed by LISTA and L-FISTA. This is surprising since the paper seem to suggest that LISTA is in some sense a specialization of FacNet. Any comment about this?
review_title: Empirical performance of FacNet

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper performs theoretical analysis to understand how sparse coding could be accelerated by neural networks. The neural networks are generated by unfolding the ISTA/FISTA iterations. Based on the results, the authors proposed a reparametrization approach for the neural network architecture to enforce the factorization property and recovered the original gain of LISTA, which justified the theoretical analysis. My comments are listed below.

It is not clear about the purpose of Section 2.3.2. Adapting the factorization to the input distribution based on (15) would be time consuming because the overhead of solving (15) may not save the total time. In fact, the approach does not use (15) but back propagation to learn the factorization parameters. 

Minor comments:

- E(z_k) in (3) and (4) are not defined.

- E_x in (19) is not defined.

- Forward referencing (“Equation (20) defines…”) in the paragraph above Theorem 2.2. needs to be corrected.

review_title: null

final_score: rating: 5: Marginally below acceptance threshold
review_content: This paper proposes a method for neural sparse coding inspired by LISTA (Gregor and LeCun 2010). A theoretical analysis is presented that attempts to explain the non-asymptotic acceleration property of LISTA (via Theorem 2.2. and Corollary 2.3).

FacNet is a specialization of LISTA, sharing the same network architecture but with additional constraints on the parameters. In numerical experiments, LISTA outperforms FacNet, up to some optimization errors. It is not clear what is the advantage of using FacNet instead of LISTA.

Overall, the paper lacks clarity in several parts. It would be good to state beforehand what the main contribution is. As stated in the clarification question/answer below, this paper would benefit from a more clear explanation about the connection of FacNet with LISTA. 

Minor comments/typos:
- p. 6: "memory taps" -> tapes?
- sec 3.2: "a gap appears has the number of iterations increases" -> as?
- sec. 4: "numerical experiments of 3" -> of sec 3
review_title: Connection to LISTA should be made clearer

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: This work presents an analysis of LISTA, which originally proposes to accelerate sparse coding algorithms with some prior on the structure of the problem. The authors here propose a solid analysis of the acceleration performance of LISTA, using a specific matrix factorisation of the dictionary. 

The analysis is well structured, and provides interesting insights. It would have been good to tie more closely these insights to specific properties of data or input distributions.

The learned dictionary results in Section 3.3 are not very clear: is the dictionary learned with a sort of alternating minimisation strategy that would include LISTA as sparse coding step? Or is it only the sparse coding that is studied, with a dictionary that has been learned a priori?

Overall, the paper does not propose a new algorithm and representation, but provides key insights on a well-known and interesting acceleration method on sparse coding. This is quite a nice work. The title seems however a bit confusing as 'neural sparse coding' is actually rather 'LISTA', or 'neural network acceleration of sparse coding' - basically, it is not immediate to understand what 'neural sparse coding' means...
review_title: Very good work, title potentially confusing

Meta-review: The work is fairly unique in that it provides a theoretical explanation for an empirical phenomenon in the world of sparse coding. The reviewers were overall favourable, although some reviewers thought parts of the paper were unclear or had confusion about the relationship to LISTA. I suspect the analysis here could also shed light on other problems.
Decision: Accept (Poster)