Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: This paper introduces deep audio prior (DAP), which uses CNN'S prior to perform classical tasks in audio processing: source separation, denoising, texture synthesis, co-separation. This paper gets inspiration from deep image prior and adapts it to audio, by introducing a lot of insights in the audio domain, which I believe is a good amount of contribution.
I have the following questions on the paper:
(1) The source generation part is a bit confusing. When testing on real-world examples, do you need to generate sources? I want more explanations on the ablations studies in Section 4, temporal/dynamic sources. What are the input/output, how is the noise generated, how is the model trained.
(2) In general, I expect more details in the paper, like the model architecture (does that affect performance), how training is performed (iteration, convergence, etc).
(3) Some notations are missing, e.g. equation (4).
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: The authors detail a set of priors for unsupervised decomposition of individual spectrograms into their component parts. The introduce reasonable constraints on temporal coherence (consistency and dynamic shifts) and mask activations (at least one component always activated). They also regularize sources to not overlap spectrotemporally. Decomposition is performed by training the weights of a U-Net on a single spectrogram as in deep image priors. The authors demonstrate quantitative improvements on blind source separation over other data-agnostic techniques, and qualitative use of the model for interactive editing, audio texture synthesis, and audio watermark removal. The work also performs an ablation study to qualitatively demonstrate the importance of each element for the prior. The experiments are performed well and explained clearly. They also introduce a dataset of diverse mixtures for future comparisons.

Pros:
* Important motivation for why audio has different properties than images (even if it can be represented as a "image" spectrogram). The priors are well-motivated by the dynamics of audio.
* Good ablations and quantitative comparisons to baselines.

Cons:
* Some details could be better demonstrated / explained (even if only in the appendix). For example the paper cites the network architecture, but a local description would be helpful. Similarly, the latent dynamics are carefully regularized, so visualizing them would be helpful to understand the dynamics.
* The scaling of the technique is not supported by the current experiments. The authors claim they have extended to 4 sources, but all experiments in the paper seem to only involve two sources. 
* More motivation could help in terms of the value of non-amortized methods like deep priors, vs. other approaches such as pretraining or self-supervised methods. While it is difficult to get lots of labeled data for a specific task, the argument was not convincingly made that methods like deep priors should outperform methods that use pretrained priors on adjacent tasks (where collecting data is easy).
review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: This paper presents a method for blind source separation relying on randomly initialized networks to decompose an input audio spectrogram into two components.
The networks are designed to promote temporal contiguity of spectral energy in the estimated signals, which are modulated (in time) by estimated masks.
The proposed method is evaluated on a collection of 150 random mixes of sounds, and performs favorably relative to some standard baseline methods (RPCA, NMF, KAM).

This seems like a promising line of work, but at this point I think the weaknesses of the paper outweigh its strengths (as detailed below).  Some of these points may be addressed during discussion, but I currently lean toward reject.


Strengths of the paper:

- The ideas are interesting, and appear to perform well on a simulated and real(ish) data.

- The authors investigate several variations and applications of source separation, including interactive editing, co-separation, and texture synthesis.

- A small (qualitative) ablation study is included to clarify the importance of different components of the loss function.


Weaknesses of the paper:

- Much of the presentation is vague or opaque.  There is little detail provided about the specific architectural parameters of the model, and the diagram (figure 1) does not appear to match the equations.  Specifically, it's unclear whether S_1^* is a function of S_mixture or not.

- The quantitative evaluation focuses entirely on one (new) dataset with unclear characteristics.  No details are provided about the evaluation protocol, and in particular, the tuning of hyperparameters for the various methods under comparison.  Aggregate statistics are included (mean? SDR, etc), but no notion of variance or error bars are included.  It's not ultimately clear how fair this evaluation is.  There should at minimum be a comparison on a standard source separation or speech enhancement dataset, in addition to the new set presented here.

- Most of the spectrogram figures appear to be upside-down, which is confusing.  The details of the audio processing are omitted: STFT parameters are stated, but not the sampling rate.

- There are numerous typos ("grounth", "spectrogram stokes", etc), indicative of the authors not running a spell-checker on their submission.


Questions for the authors:

- The model itself consists of several competing loss functions, but it seems like they may have a trivial, optimal solution at S1 = S_mix (M_1 = 1) and S_2 = 0 (M_2 = 0 or 1).  As far as I can tell, this solution would trivially minimize each term of equation 9.  (If S_1* does not depend on S_mixture, this may be less of an issue, but the trivial solution may still exist when the driving noise is of sufficiently high dimension and the optimization is run long enough.)  Am I misunderstanding the algorithm, or is there some deeper reason why this solution would not be preferred?

review_title: Official Blind Review #2

Meta-review: This paper proposes to use CNN'S prior to deal with the tasks in audio processing. The motivation is weak and the presentation is not clear.  The technical contribution is trivial.
Decision: Reject