Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 6: Weak Accept
review_content: The authors propose complex valued neural networks to perform audio source separation in the Fourier domain. The adapt a well known U-Net architecture to the task by introducing a complex-valued FiLM layer and a new complex similarity loss that explicitly takes magnitude and phase into account. They motivate the use of complex values well and demonstrate performance and parameter efficiency improvements over real-valued baselines. Importantly, they do not need to perform spetrogram inversion because their network works natively in the complex domain. Despite the quantitative improvements over spectral models, they still slightly underperform the ConvTasNet baseline that operates directly in the waveform domain (which was slightly misleading to not include in the table). The authors perform an extensive hyperparameter search to tune the model and provide sufficient detail to reproduce their experiments. While the results did not improve upon the best baseline, they do provide further evidence to the value of using complex-valued neural networks to handle complex-valued data (where phase and synchronicity matter), which I believe will be of value to the ICLR community, and thus I lean slightly in favor of acceptance. 
review_title: Official Blind Review #2

final_score: rating: 3: Weak Reject
review_content: This work researches the deep complex-valued neural networks. Specifically, it proposes a new signal extraction mechanism that operates in frequency domain and applies to address the speech separation issue. Also, a function is proposed to explicitly consider both the magnitude and phase information of a signal. Related work on learning representation in frequency domain and speech separation is well introduced. Theoretical analysis is conducted to show the motivation and connection to signal processing. The architecture of the deep neural networks is presented in details, with the elaboration of the complex mask generation. Experimental study is conducted on a benchmark dataset to compare the proposed complex networks with those using real-part values only to demonstrate the improvement. 

The rating is 3: Weak Reject considering that the novelty is limited and the experimental study is weak. 

1. The significance of the theoretical analysis in Eq.(1) to Eq.(4) needs to be better explained. Currently, they seem to be some straightforward results in the field of signal processing;
2. The proposed CSimLoss is interesting. However, its effectiveness seems to be limited as demonstrated in Table 1. It can be found that the CSimLoss in some cases is only comparable (or even inferior) to the L2freq loss;
3. The mask generation proposed in Section 6 conceptually is largely an attention mechanism that has been widely applied in deep networks; 
4. The experimental comparison in Table 1 is limited, although some improvements have been demonstrated. This work shall also make a comparison with some of the existing methods on speech separation as described in Section 2.2.
5. It was mentioned in the last paragraph of Section 7 that (Shi et al. 2019) uses different data preparation than this paper. Can this paper use the same data preparation as (Shi et al. 2019) and perform some comparisons?
6. Why did (Shi et al. 2019) achieve better SDR (12.1 vs. 11.3) than the proposed method using standard setup?
7. What if the mechanism of mask generation is also applied to Real U-Net? How much improvement can this bring? 

review_title: Official Blind Review #3

final_score: rating: 6: Weak Accept
review_content: This paper proposes a new method for source separation, by using deep learning UNets, complex-valued representations and the Fourier domain. Concretely, their contribution is : i) a complex-valued convolutional version of the Feature-Wise Linear Modulation, able to optimise the parameters needed to create multiple separated candidates for each of signal sources that are then combined using signal averaging; ii) the design of a loss that takes into account magnitude and phase while being scale and time invariant. It was then tested and compared with real-valued versions, and also some state-of-the-art methods.

Overall, I think this paper is of good quality and proposes an interesting method for this crucial task of source separation. However, I found the paper too dense and difficult to read (even if well written), and it looks like a re-submission from a journal paper of more than 8 pages. I would suggest the authors to shrink the paper so it *really* fits into the 8-pages (without important figures or important implementation details in the appendices), maybe at the cost of leaving some parts (such as old related works) out of the paper. The experiments are important here, and it is too bad that the comparison with state-of-the-art is just in the last paragraph, while the results do not seem to show any improvements compared to the other methods. The computational time might be very important here, as the claim is that FFT reduces time computation, but I did not had time to go through all the appendices.

Positive aspects:
- The work is well documented and motivated, and I found that the reflexion leading to the method is of good quality. 
- Concretely, I found interesting the use of the FiLM, originally designed for another application, for minimizing the SNR of the signal sources. The motivation/proof is quite clear too.
- Equally, the motivation for the design of the new loss is clear and interesting.
- I also found important the experiments, that shows in the same table the difference between the method without the complex-valued part and with different parameter values.

Questions and remarks:
- I have to recall that I am not an expert on source separation and complex-valued deep learning. Yet, I have had difficulties in understanding the structure of the method, even if the different parts were clearly explained. The figure 1 is very useful, but I found it not clear enough and too small. I went to find some informations in the appendices, but there are too much crucial information there and I did not have time to go through all of it.
- The use of the U-net architecture is not explained (just some citations are given). What is supposed to be the output of it? 
- When you say 'to be more rigorous, we can assume in the cse of speech separation that, for each speaker, there exists an impulse response such that when it is convolved with the clean speech of the speaker, it allows to reconstruct the mix' : why can we be sure that it is always possible, and why is it more rigourous?
- Why are the additive noises epsilon_i supposed to have the same E(|epsilon_i|^2|) ? Even if they are uncorrelated, what is the hypothesis behind that?
- In the CSimLoss, why (i) is the real part negative and the imaginary part positive; (ii) is the imaginary part squared?
- have you tested with a higher lambda_imag (as the larger is now the best)?
- It looks from the end of the paper that the method is still not achieving better results than the state-of-the-art. I agree with the authors as it might not be the scope of the paper, but then what is it? If it's time computation, it is not shown in the paper. If it is just a methodology, what would be required in the future to beat the best method?
- In the results, table 1, in the last 4 lines: it looks from 1st and 2nd line that the new loss CSimLoss is not very different from the L2 (9.88 compared to 9.87). The best result, in the 4th line, cannot be compared to the 3rd line as both the loss and the number of transforms are different. I then found those values in the appendices, but it would be best to show fewer parameters varying in the main paper, but show some results that can be easily compared. 
- What is the importance of the first paragraph in 2.1? I was not aware of the holographic reduced representations, but I don't understand it more now, and I don't see why explaining that for 15 lines.

Small remarks:
- 'deep complex valued models have *just* started to gain momentum'... with citations beginning in 2014, I would not say 'just'.
- 'in the frequncy domain is then, ...' --> frequency + no coma
- Figure 2 is in the appendix, while in the text it is not said so. I was lost. This figure should not be in the appendix as the appendix should not have key elements, but just details that are not important for the understanding of the paper.

review_title: Official Blind Review #1

Meta-review: The paper discusses audio source separation with complex NNs.  The approach is good and may increase an area of research.  But the experimental section is very weak and needs to be improved to merit publication.
Decision: Reject