Venue: ICLR 2017 conference

Reviews:

final_score: rating: null
review_content: Would any rule recognized in the self-learning loop be consistent with the music theory? If not, how did you interpret and address the exception?
review_title: Completeness

final_score: rating: null
review_content: Just to clarify: would the described system be able to handle voice leading such as “resolving a suspension downwards”, and “avoid parallel fifths” (even though a *fifth* is fine), which depend on multiple voices as they move through time? The only place I saw that might address this is the end of Section 6, but it was not clear to me how well such characteristics are actually handled, if at all. 

If this *can* be handled in principle, then the next question is whether the resulting rules be in a form that would make any intuitive sense to a beginning student? (i.e. "resolve the 4th down to the 3rd" is a fairly intuitive single phrase that is easy to for a student to understand, but i could imagine this same concept being expressed as a conjunction of various rules involving interval counting, etc, that would not be nearly as clear.) Given that the system is intended for students, and given that a big part of generating Bach chorales is understanding voice leading, this is an important consideration.
review_title: rules about joint motion of multiple voices over time

final_score: rating: null
review_content: In the paper, I didn't see detailed introductions about the student optimization (right part of Fig. 1). What is S_q? Could the authors elaborate on it? I assume this is part of the author's previous system. It would be great to mention it briefly in the text.

Also, what is the meaning of mod_12, diff, order, etc (in Table. 1). The introduction in Appendix A.1 is a bit abstract and might not be helpful for people who are not familiar to music. 
review_title: Some notations need explanation. 

final_score: rating: 8: Top 50% of accepted papers, clear accept
review_content: Summary: 
The paper presents an advanced self-learning model that extracts compositional rules from Bach's chorales, which extends their previous work in: 1) the rule hierarchy in both conceptual and informational dimensions; 2) adaptive 2-D memory selection which assumes the features follow Dirichlet Distribution. Sonority (column of 4 MIDI numbers) acts as word in language model: unigram statistics have been used to learn the fundamental rules in music theory, while n-grams with higher order help characterize part writing. Sonorities have been clustered together based on feature functions through iterations. The partition induced by the features is recognized as a rule if it is sufficiently significant. As a result, two sample syllabi with different difficulty strides and "satisfactory gaps" have been generated in terms of sets of learned rules. 

1. Quality:
 a) Strengths: In the paper, the exploration of hierarchies in two dimensions makes the learning process more cognitive and interpretable. The authors also demonstrate an effective memory selection to speed up the learning.

 b) Flaws: The paper only discussed N<=5, which might limit the learning and interpretation capacities of the proposed model, failing to capture long-distance dependence of music. (In the replies to questions, the authors mentioned they had experimented with max N=10, but I'm not sure why related results were not included in the paper). Besides the elaborated interpretation of results, a survey seeking the opinions of students in a music department might make the evaluation of system performance more persuasive.

2. Clarity:
 a) Pros: The paper clearly delivers an improved automatic theorist system which learns and represents music concepts as well as thoroughly interprets and compares the learned rules with music theory. Proper analogies and examples help the reader perceive the ideas more easily.

 b) Cons: Although detailed definitions can be found in the authors' previous MUS-ROVER I papers, it would be great if they had described the optimization more clearly (in Figure 1. and related parts).  The "(Conceptual-Hierarchy Filter)" row in equations (3): the prime symbol should appear in the subscript.

3. Originality:
The representation of music concepts and rules is still an open area, the paper investigate the topic in a novel way. It illustrates an alternative besides other interpretable feature learning methods such as autoencoders, GAN, etc. 

4. Significance:
It is good to see some corresponding interpretations for the learned rules from music theory. The authors mentioned students in music could and should be involved in the self-learning loop to interact, which is very interesting. I hope their advantages can be combined in the practice of music theory teaching and learning.

review_title: null

final_score: rating: null
review_content: Do you have any intuition of what is happening in the bottom row (“5-gram”) of the top part of Figure 4? 

review_title: figure clarification

final_score: rating: 6: Marginally above acceptance threshold
review_content: This paper proposes an interesting framework (as a follow-up work of the author's previous paper) to learn compositional rules used to compose better music. The system consists of two components, a generative component (student) and a discriminative component (teacher). The generative component is a Probabilistic Graphical Models, generating the music following learned rules. The teacher compares the generated music with the empirical distribution of exemplar music (e.g, Bach’s chorales) and propose new rules for the student to learn so that it could improve.

The framework is different from GANs that the both the generative and discriminative components are interpretable. From the paper, it seems that the system can indeed learn sensible rules from the composed music and apply them in the next iteration, if trained in a curriculum manner. However, there is no comparison between the proposed system and its previous version, nor comparison between the proposed system and other simple baselines, e.g., an LSTM generative model. This might pose a concern here. 

I found this paper a bit hard to read, partly due to (1) lots of music terms (e.g, Tbl. 1 does not make sense to me) that hinders understanding of how the system performs, and (2) over-complicated math symbols and concept. For example, In Page 4, the concept of raw/high-level feature, Feature-Induced Partition and Conceptual Hierarchy, all means a non-overlapping hierarchical clustering on the 4-dimensional feature space. Also, there seems to be no hierarchy in Informational Hierarchy, but a list of rules. It would be much clearer if the authors write the paper in a plain way. 

Overall, the paper proposes a working system that seems to be interesting. But I am not confident enough to give strong conclusions.
review_title: null

final_score: rating: null
review_content: I am curious how well the described approach might scale. Certain aspects (seem to me to) depend on exhaustive searches/enumerations, but this was not entirely clear. Suppose, for example, that rather than Bach chorales, the system was optimized to help students learn orchestration, where there can easily be 10-20 instruments. In this case, the family of windows in Eq(1) would therefore correspond to a significantly larger power set. Furthermore, the pieces are much longer, so counting occurrences of arbitrary compositions of atomic operators could be combinatorially non-trivial, etc. Yet, conceptually organizing the rules of orchestration hierarchically might actually be quite a helpful application. Any comment on this would be appreciated.
review_title: how does the approach scale

final_score: rating: 6: Marginally above acceptance threshold
review_content: After the discussion below, I looked at previous work by the authors (MUS-ROVER) on which this paper was based. On one hand, this was very helpful for me to better understand the current paper. On the other hand, this was very needed for me to better understand the current paper.

Overall, while I think that I like this work, and while I am familiar with the JSB chorales, with probabilistic approaches, with n-grams, etc, I did find the paper quite hard to follow at various parts. The extensive use of notation did not help the clarity.

I think the ideas and approaches are good, and certainly worth publishing and worth pursuing. I am not sure that, in the paper's current form, ICLR is an appropriate venue. (Incidentally, the issue is not the application as I think that music applications can be very appropriate, nor is the problem necessarily with the approach... see my next suggestion..). I get the sense that a long-form journal publication would actually give the authors the space necessary to fully explain these ideas, provide clearer running examples where needed, provide the necessary background for the appropriate readership, provide the necessary background on the previous system, perhaps demonstrating results on a second dataset to show generality of the approach, etc. A short conference paper just seems to me to be too dense a format for giving this project the description it merits. If it were possible to focus on just one aspect of this system, then that might work, but I do not have good suggestions for exactly how to do that. 

If the paper were revised substantially (though I cannot suggest details for how to do this within the appropriate page count), I would consider raising my score. I do think that the effort would be better invested in turning this into a long (and clearer) journal submission.

[Addendum: based on discussions here & revisions, I have revised my score]

review_title: Final Review

Meta-review: Given that all reviewers were positive aobut this paper and given the unusual application domain, we recommend to accept this paper for poster presentation at the main conference.
Decision: Accept (Poster)