Venue: NeurIPS 2022 Track Datasets and Benchmarks

Reviews:

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: The paper proposes combinational music generation, a task for creating different background music based on given conditions. The task creates music samples with musical metadata and combines them to produce complete music. The paper also introduces 'ComMU', a symbolic music dataset consisting of 12,025 short music samples and their metadata. The dataset is manually created by professional music composers and features 12 musical metadata that embrace the composers' intentions.
strengths: - The paper is a good contribution to datasets for machine learning research in music, especially with music generation. This is a worthwhile contribution given that music datasets have not advanced as rapidly as other areas.
- The proposed dataset contains several metadata that provide a decent amount of interpretiveness to the music and control over generated music using combinatorial music generation.
weaknesses: - One of the claims of the proposed dataset is the diversity of the music samples, but that is difficult to justify given that the samples were manually generated by a constrained set of composers. Plus, no information is given about the composers, so the compositions may be limited by the composers' skillset or orientation (e.g., genre preference, chord diversity, etc.)
- There is a lot more to music than just chord progressions and the other metadata contained in the proposed approach. As such, the generality might be quite limited, e.g., in the absence of other musical qualities like scales, arpeggios, etc.
- Like the prior point, chord progressions are extremely complex and diverse. It would have been nice to get a sense of the kinds of progressions that were included in the dataset and what the patterns represent or how the patterns were created.
- An apparent major contribution of the paper is combinatorial music generation, but very little time is spent on describing it in the paper. It seems to me like the main contribution is the dataset and combinatorial music generation would be one of the potential uses of the dataset. Unless I'm mistaken, the paper may need to be restructured to better represent the contributions (some minor restructuring of sentences in the abstract/intro may solve this issue).
correctness: The claims made in the paper seem correct. Some clarification might be helpful for any rules that were followed in the composition of the music samples. Or, was it arbitrary?
clarity: The paper is generally well-written and easy to understand. There are some typos, like an undefined reference on line 86. There is a disparity in the number of MIDI samples between line 46 and 97.
relation to prior work: The paper clearly distinguishes the proposed work from prior work. The main difference is that prior symbolic music datasets do not have enough metadata for sophisticated control.
additional feedback: - How do you ascertain the quality of the music samples, given the apparent anonymity of the composers?
- It seems like the quality of the samples would be very subjective. For example, genres are very subject to interpretation by the composers (e.g., one composer may describe new age music as cinematic). Or, a riff would vary substantially based on the skill of the composer. This could have implications for training, wherein a desire for cinematic music may be misinterpreted due to a different interpretation in the training data. How would this be accounted for in the dataset?
review_title: Dataset for combinatorially creating music samples based on given conditions

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This paper provides a dataset for combinatorial music generation. The dataset consists of 12,025 MIDI files with 12 musical metadata (bpm, genre, key, instrument, track-role, time signature, 48 pitch range, number of measures, chord progression, min/max velocity, and rhythm). The paper takes combinatorial music generation as a two-phase problem: note sequence generation and multiple note sequences combination. The dataset is aimed for the first phase, note sequence generation with metadata. The authors formulate it as a conditional generation task which generate note sequence autoregressively conditioned on the metadata provided. Some data analyses and baseline eperiments have been conducted to validate the usefulness of the provided dataset. 
strengths: This paper provides a valuable MIDI dataset for combinatorial music generation. Music generation has recently attracted much attention in the community since the success of generative models. However, the publicly avaible music data for research on this topic are lacking due to the copyright and license issues. While previous research has open some datasets for generative music continuation, little has been done for  controllable music generation. To bridge this gap, this paper provides a valuable MIDI dataset with abundant meta features. Meanwhile, the paper demonstrates the use of the dataset with combinatorial music generation tasks. The dataset and generated samples are availble. I checked the generated samples on the website, the quality is high which verfies the usefulness of the dataset. The dataset should benefit the community to perform research on controllable music generation.




weaknesses: The paper is well written and easy to follow. My concern is that in Section 4.1 the authors define the task as a autoregressive generation problem. But it does not clarify its relationship to existing work and why not implement an exsiting approach as the baseline.
correctness: This paper provides a dataset. The dataset contains 12,025 MIDI files. The only question is how the number of samples is determined and whether they are sufficient for commercial-level music generatoin study?
clarity: The paper is overall well-written. But it would be better to provide more background information for readers who have no prior knowledge in the field.
relation to prior work: The paper present a new set of MIDI files with detailed meta data. The dataset is new to the research on controllable music generatoin task. Also, the author compare the dataset with previous ones in Table 1. For the methodology part, the authors define the task as a autoregressive generation problem. Yet, its relationship to existing work is lacking. 
additional feedback: I suggest to specify a clear license for the code on github.
review_title: Review for ComMU: Dataset for Combinatorial Music Generation

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: The paper discussed a new data set of 11.144 short music samples created by professional human experts and annotated with 12-dimensional metadata. The metadata is then used to investigate a combinatorial analysis and synthesis of longer music sequences. This is a very interesting and new line of work.
strengths: 1. The paper provides O(10^4) samples of short musical compositions with 12-dimensional annotations to the community.
2. There is an interesting baseline solution presented for the problem of combinatorial music generation using this data set.
3. All the code, data set and website are of high quality.
weaknesses: 1. I am not an expert in music. So, I cannot comment on the academic significance of the problem in music. However, as an end user, I am quite happy with the quality of music being generated here. 
correctness: There are no issues with correctness. 
clarity: The paper is very clearly written. 
relation to prior work: The paper cites related work in the area. 
additional feedback: This is wonderful work put together in a complete and clear manner. I greatly enjoyed reading the paper and browsing the related website.
review_title: Professionally created 11,144 MIDI samples annotated with 12-dimensional meta data

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This paper introduces MIDI-based symbolic music dataset, ComMU, with rich and unique metadata for combinatorial music generation which consists of two stages; 1) note sequence generation given metadata, and 2) combination of the sequences for one completed music. The authors quantitatively analyze the dataset and evaluate Transformer baseline in diversity, controllability, and fidelity. The dataset seems useful for music generation field and combinatorial music generation task seems promising approach for high-quality music generation.
strengths: 1. The music dataset, collected by professional composers, is of such high-quality that it can be used commercially.
2. It contains rich and unique metadata which is helpful controllable and diverse music generation.
3. Documentation and materials including website and GitHub repository are well-organized.

weaknesses: 1. The dataset lacks a big of genre diversity since there are only two genres (cinematic and new-age). More diversity in genre will strengthen the usefulness.
2. The paper mainly focuses on stage 1 so it lacks description on stage 2. I think more contents of stage 2 would strengthen the completeness of this work, e.g., Are there any patterns or tendencies of professional composers? How about almost randomly generated combination compared to manually generated one?
correctness: The paper seems correct and claims well-supported.
clarity: The paper is well-written and easy to follow.
relation to prior work: The relation to prior work and contribution are clearly discussed.
additional feedback: 1. (line 86) The reference for ADL Piano MIDI is missing, while one is correct in Table 1.
2. (Table 1) LakhMIDI -> Lakh MIDI
3. (line 105) Where do "reference music" come from? Do they utilize external music sources such as online free music repository?
4. (Eq 1) I think $\sum^T_{t=12}\log p_\theta(x^S_t|x_{\lt t})$ is more accurate, because metadata consists of $x_{1:11}$.
review_title: High-quality music dataset with rich metadata

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: This paper introduces an interesting new task called combinatorial music generation, and provides a symbolic music dataset ComMu for illustration. The ComMU dataset is manually constructed by professional composers and includes 12 musical metadata, two of which, track and extended chord quality, are unique metadata in this dataset. 
strengths: 
(1)  The dataset is manually made by professional composers;  

(2)  The dataset includes two unique metadata, track and extended chord quality, that can improve the capacity of the automatic composition. 

(3) The dataset includes abundant metadata (12 metadata), while the previous works generate music with much fewer metadata. 
weaknesses: In the experiment, the authors only use one baseline (Transformer-XL). Therefore, all conclusions are drawn from this baseline. It's important add some baselines to enhance the conclusions. 
correctness: Yes
clarity: This paper is well written. 
relation to prior work: Yes
additional feedback: I'm not familiar with the field of automatic music composition. Here are some questions I'm curious about. 

(1) What are the difficulties and challenges of compositional music generation tasks? 

(2) How this article can help researchers in the field?

(3) What is the relationship between combinatorial music generation and automatic music composition?

Here are some further questions about the experiment. 

(1) In Figure 7, there is no error bar or confidence intervals. 

(2) In Table 3, the authors keep the parameters ($K$ and $\tau$) fixed. This makes me a little confused. If the parameters are fixed, what is the role of the validation set? Are there other parameters that need to be selected through the validation set?

(3) Why make all the metadata in the validation set exclusive to the training set?

(4) Why are the evaluation metrics in Figures 6 and 7 different from those in Table 3?
review_title: Valuable dataset for Interesting combinatorial music generation tasks

final_score: rating: 7: Good paper, accept
review_content: summary and contributions: The paper offers an excellent groundwork for furthering research in the space of human-computer co-creativity and more specifically in the context of combinatorial music generation. The proposed method values the intention of the composers, a factor that is crucial for the development of responsible art.  Furthermore, the dataset includes unique metadata that can augment the quality of generated music. 
strengths: 1. The dataset is constructed by expert opinions, thus making it more reliable and robust. 

2. Offers rich meta data including extended chord quality and track role, taking into account the intentions of the composers. 


weaknesses: 1. Music genres and instruments explored are largely dominated by Western cultures. 

2. Paper can benefit from providing a detailed data sheet - please see comments below. 
correctness: seems ok
clarity: Paper is very well organized and presented. 
relation to prior work: Although the work concerns music generation, it might be still useful to contextualize some related work in the space combinatorial music selection such as https://arxiv.org/abs/2104.12432. This paper may be a relevant prior work- https://arxiv.org/abs/2104.12432 
additional feedback: Please see responses above.
review_title: A unique combinatorial music generation dataset informed by expert opinions

Meta-review: metareview: All six reviewers recommend acceptance. Reviewers found the large dataset of professional MIDI music and metadata to be a useful resource. Authors should attend to main points in the reviews when preparing a final version. These issues include those raised by Reviewer 2G3b around more clearly presenting the contributions, the description of stage 2 and equation typos (7zFc), and clarifying which baselines were run (VTd3). The AC sees no basis to overturn reviews, and thus recommends acceptance.
confidence: 3: The area chair is somewhat confident
Decision: Accept