Venue: ICLR 2018 Workshop

Reviews:

final_score: rating: 6: Marginally above acceptance threshold
review_content: The paper proposed a novel approach to explore what networks learn. The idea revolves around asking questions about a trained network for relearning the dataset. 

The influence of depth and labeled data shows how to define a simple task for nn. Relating learnability to generalization on a theoretical footing for a simple 1nn would improve the merits of the approach.

Based on the author rebuttal from conference review this work needs more attention. Worthy to explore as a part of the workshop. 
review_title: Interesting idea. Worthy to explore. 

final_score: rating: 5: Marginally below acceptance threshold
review_content: The authors of this paper reported their intensive experimental results on learnability when varying network complexity and using random data. This paper does not provide any theoretical justification of learnability, but their empirical results may interest some people. 

Minors:
1. I could not understand this: "structured prediction,on,"
2. The format of references should be consistent.

review_title: Empirical study on learnability

final_score: rating: 4: Ok but not good enough - rejection
review_content: This paper proposes the measure of learnability and empirically evaluate it using various neural network architectures.
However, I am not fully convinced with the notion of learnability.
Since it checks whether or not the results of two trained models matches, it is more suitable to call it the "stability" of models.
Moreover, the concept would be strongly related with bias and variance which are well studied in the context of statistical learning theory.
Hence the notion of learnability is not appropriate and I recommend to carefully discuss with the proposed measure and such existing measures.


review_title: The notion of learnability in this paper is questionable

Meta-review: Based on the reviews, this paper has not been accepted for presentation at the ICLR workshop. However, the conversation and updates can continue to appear here on OpenReview.
Decision: Reject