Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: In this work, the authors consider the task of supervised music source separation, i.e., separating out the components (bass, drums, voice, other) out of a mixed music track. In particular, the work considers the task of supervised source separation where the individual target tracks are available during training time. The main contribution of this work is the improvement of an end-to-end waveform-to-waveform separation models through a number of architectural changes that allow such waveform-to-waveform models to perform comparably with other current state-of-the-art methods that instead operate in the spectrogram domain.

Overall, the paper is generally well written and the method is easy to follow. However, I have a couple of concerns about this work, based on which I would rate the work as “weak reject”:

1. The specific architecture proposed in this task does appear to improve performance for this particular task, but it is not clear to me that the conclusions drawn based on the study in this work will be generally applicable to other related tasks. Also, the final model is still only comparable to the spectrogram-based methods overall, and appears to do a slightly worse job in separating the ‘vocal’ and ‘other’ tracks than these baseline methods. As such, I’m unsure about the impact of this work.
2. The section describing the evaluation metric SDR wasn’t very clear to me. Perhaps it would be better to just refer back to (Vincent et al., 06) and just describe what SDR captures instead? Or alternatively, more details could be added to explain the computation more clearly.
3. Section 4.2: The authors mention that they multiply each source by +/- 1. I didn’t follow why this is being done. Could the authors please clarify.

Minor comments:
1. The notation in Equation 1 is slightly confusing because ‘s’ is used to index both the mixture and the sources. I think x \in \mathcal{D} would be clearer. Similarly I wonder if x_s \in \mathbb{R}^C \times \mathbb{R}^T would be clearer than x_s \in \mathbb{R}^{C,T}
2. The reference (Oord et al., 2017) should be rendered as (van den Oord et al., 2017) 
3. The Figure uses K, and S to denote Kernel Width and stride, but this isn’t explicitly mentioned in the text. Perhaps it would be useful, in Section 3.1 to write: “... convolution with kernel width, K=8, and stride, S=4, ...” or something similar, and updating the caption to explain the notation.
4. Section 3.2: “The decoder is almost the symmetric of the encoder” --> “The decoder is almost the inverse of the encoder” would perhaps be better?
5. Section 3.2: “The final layer … S.C_0 ...” --> “The final layer … S * C_0 ...”
6. The notation in Equation 1 and Equation 2 is slightly inconsistent (lower-case vs. upper-case L for the loss function)
7. Section 4.3: “additionnaly” --> “additionally”; “32 GB or RAM” --> “32 GB of RAM”

review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This paper suggests using a Unet type architecture to perform end-to-end source separation. They are reporting performance improvement over another architecture which uses Unets architecture in conjunction with Wavenet decoders. They also report a very marginal performance improvement over an STFT based model (open unmix) The improment is 0.02 dB (table 1), and I am not sure if it is statistically significant.


Also, there is no mention of algorithms which adaptively learn the basis and then do masking similar to what we do in the STFT domain. A very popular example for this is tasnet, which performs well on speech source separation tasks. I would like to see comparisons with this model. If you think there is a specific reason why not to use adaptive basis approaches such as TASNET, please do let me know. 
review_title: Official Blind Review #1

final_score: rating: 8: Accept
review_content: The authors present modifications to the state of the art (waveform based) source separation model (Wave-U-Net) and improve the state of the art to be comparable to spectral masking based methods. They clearly outline all the architectural changes they make (GLU nonlinearities, strided upsampling, bidirectional RNN), perform thorough evaluations against strong baselines, and a complete ablation study to demonstrate the value of each component. The paper reads well and quickly informs newcomers to the field with proper motivation and context. For all these reasons I think it should be accepted.  One weakness of the paper is that the relative incremental nature of the study, but given the thoroughness and clarity of the experiments, and the non-triviality of the architecture search, I think this is a valuable contribution for the ICLR community.

One small suggestion, the language difference between "upsampled" convolution and "transposed" convolution is a bit confusing. I might suggest focusing on the striding of the convolution vs. bilinear upsampling, as a non-strided convolution can still be "transposed" from a standard API point of view in pytorch or tensorflow.
review_title: Official Blind Review #3

Meta-review: The paper proposed a waveform-to-waveform music source separation system. Experimental justification shows the proposed model achieved the best SDR among all the existing waveform-to-waveform models, and obtained similar performance to spectrogram based ones. The paper is clearly written and the experimental evaluation and ablation study are thorough. But the main concern is the limited novelty, it is an improvement over the existing Wave-U-Net, it added some changes to the existing model architecture for better modeling the waveform data and compared masking vs. synthesis for music source separation.  
Decision: Reject