Kati8/ICLR_2023_2025 huggingface dataset:
authors: Giorgio Mariani
authors: Irene Tallini
authors: Emilian Postolache
authors: Michele Mancusi
authors: Luca Cosmo
authors: Emanuele Rodolà
reviews: 
Comments: This paper proposes a unified approach (called Multi-Source Diffusion Model) to solve audio generation, source inputation and source separation. In training time, the diffusion model learns the joint distribution of the audio mixture and solves the three tasks using different inference methods. Audio generation is done via directly sampling the prior. Source inputation is done using the inpainting technique in diffusion models. The source separation is a tailored method: reconstructing audio signals under the constraint that audio mixture is the summation of each audio tracks. Experiment results show the model is successful in all three tasks.
Strengths: 1. Originality. The idea of using a generative way to view music source separation is so natural and experiments show diffusion model can perform pretty well. The diffusion sampling process under summation constraint is borrowed from works in CV but it is a natural fit for source separation tasks.
2. Quality: theory is presented clearly and experiments are effective.
3. Clarity: paper is well-written. High-level idea and detail are both clear.
4. Significance: the result is convincing according to the demo in the supplementary material.
Weaknesses: The paper is well-written. I have some minor comments. If it can be clarified, the significance of the work will be increased.

1. The idea of using a generative way to model audio and solve multiple audio tasks in one model is not a new idea. For example, we have a unified VAE approach to do source separation and transcription [1]. Of course the current work is novel but it would be better to include those studies and compare the approaches in related work.
2. What is the novelty compared to NCSN-BASIS? Any intuition for MSDM Dirac? Is it a marginal improvement from existing methods, or not?
3. The introduction of ISDM method is not clear. Is it a baseline method where the assumptions are obviously wrong? Is it another valid approach? Compared to MSDM, is there a trade-off in terms of generation quality and separation performance?
4. The introduction of correction step is okay. Since it is evaluated in the experiment section, could you explain more in the paper? What is the statement to be expected prior to the experiment regarding correction step?

[1] Liwei Lin, Gus Xia, Qiuqiang Kong, Junyan Jiang: A unified model for zero-shot music source separation, transcription and synthesis. ISMIR 2021: 381-388
Recommendation: 8: accept, good paper
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 4 excellent
reviews: 
Comments: The authors propose to tackle the source separation problem by seeing it as a separate-tracks music generation problem. 
They model this distribution using a diffusion model.
This approach allows to tackle more use cases such as "source inputation" (accompaniment generation) using the same model.
Strengths: The paper is well written, well-organized, self-contained and the background and reference section are comprehensive and detailed. The proposed method is sound and the authors obtain positive results. Even if not state of the start (source separation is a highly studied task), this original method shows promising results and has the advantage of being conceptually simpler and more general.

The new proposed modeling method IDSM Dirac seems relevant, clearly improves on previous methods and might be of interest in other application domains.

This method is of course more costly in terms of the amount of data required or in terms of computational resources needed. But this is addressed in the limitations section.

The appendix showcases interesting hyperparameter searches about the MSDM Dirac approach, like the impact on the constrained source.
Weaknesses: Adding details on the correction step could make the paper even more self-contained.
Recommendation: 8: accept, good paper
Reviewer confidence: 4: You are confident in your assessment, but not absolutely certain. It is unlikely, but not impossible, that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work.
Soundness/correctness: 4 excellent
Clarity: 4 excellent
reviews: 
Comments: My review did not seem to have the right visibility, so I am re-submitting.

The authors present an approach for mixture-based diffusion modeling, complete with a novel inference approach that leverages a posterior based on Dirac delta functions and a Monte Carlo approximation of the Dirac likelihood in place of the mixture density. The authors also demonstrate that this approach can be used for generation. The authors evaluate for stem separation using Slakh2100 against established baselines, and also show quantitative and qualitative human-evaluated results for a generation task, establishing a new task and baseline in the process.
Strengths: - The approach the authors proposed is novel and in fact quite general. Orthogonally, I am interested in understanding the potential of the approach in settings with other mixture-based diffusion.
- The manuscript is compelling to read, well organized, and very clear. The discussion of related work is quite comprehensive.
- Using a waveform as the representation of data generalizes the approach further, opening the door to applicability in other domains.
- The authors provide a strong baseline for accompaniment generation with a data-rich setup (Slakh2100) which will serve to be a solid foundation. If not already planned, I would encourage the authors to release as much as they can with respect to evaluation methodology and reproducible artifacts that others can use to evaluate in a similar manner.
Weaknesses: The results of the paper would be made stronger with more discussion of the computational footprint and details for training and inference. How does the footprint compare to Demucs or other methods? How does does computation scale with the amount of data?

Further, some brief qualitative analysis of the results might be warranted. Do the authors have an explanation for the relative performance on certain stems relative to Demucs? What are the high-level takeaways from the qualitative results in Table 3 beyond those that the reader could intuit or speculate about?

Some feedback on the manuscript:
- Section 2.1, in the last paragraph: should "the minute" read "a minute" denoting duration of the context length?
Recommendation: 8: accept, good paper
Reviewer confidence: 3: You are fairly confident in your assessment. It is possible that you did not understand some parts of the submission or that you are unfamiliar with some pieces of related work. Math/other details were not carefully checked.
Soundness/correctness: 3 good
Clarity: 4 excellent