Venue: ICLR 2022 Conference

Reviews:

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: I am sorry, by mistake I posted the incomplete review version! 

This manuscript proposes a sound source localization method utilizing "synperiodic filter" bank representation as to the input of a deep convolutional neural network. 
Authors claim that the convolution of the proposed filterbanks with the raw waveform helps to achieve multi-scale perception in the time domain which results in performance improvement. 
The authors demonstrated that their method could outperform some of the state-of-the-art similar work.  

main_review: Accurately estimating sound sources' temporal location can be used in a wide range of machine cognition tasks, such as inference on the type of environment, self-localization, navigation without visual input, smart-home applications, scene visualization systems, and etc. 
This manuscript covers a very interesting topic of sound sources' temporal localization. 
The authors suggest using Synperiodic Filterbank with the exiting methods to outperform the traditional methods. NOTE, they have proposed a new combination of feature sets and new model structures. 
However, I have the following few concerns. 
1- The writing of the manuscript requires improvements, there are many ambiguous sections/ paragraphs that require more elaboration. There are many typos and grammatical errors in the manuscript. 
2- It is not clear to me whether the improvement in the performance is mainly due to the Synperiodic filterbank or the Synperiodic filterbank+ new structure, which brings the total param size to 60M! 
3- the experimental section requires more supporting materials. It is not clear how the training/testing is conducted. It would be more helpful if the authors could elaborate more on some implementation details. 
4- The authors in section 3.1 claim their mathematical formulation covers the "moving spatial motion sp" as well. However, they have not discussed/supported this claim in the experimental section. 

summary_of_the_review: This manuscript addresses an ongoing and open problem of sound source localization. I like the main idea in the paper, however, I think this work requires some minor improvements to be in proper shape for this proceedings.
review_title: null

final_score: rating: 5: marginally below the acceptance threshold
review_content: summary_of_the_paper: This paper addresses the problem of sound event detection and localization from multi-channel raw audio waveforms. Essentially, a different audio front end feature extraction scheme - "Synperiodic Filterbank" if proposed which can be parameterized and jointly learnt along with a backbone classifier in an end-to end manner. The feature extraction scheme proposed is grounded in principle of uncertainty governing time-frequency resolution. Through ablation studies the authors show that this feature extraction scheme performs better (albeit marginally) compared to some of the traditionally used feature extraction schemes like MFCC, and LFBEs. Subsequently, the authors show that combining the proposed "Synperiodic Filterbank" with a significantly large model (~3X times the closest baseline model in terms of No. of trainable parameters) based on the Transformer Architecture offers a dramatic improvement in the results compared to the baseline. 
main_review: Strengths:
1. The premise of the paper that using a more appropriate Filterbank which learns to choose different time-frequency resolution tradeoffs based on data is indeed interesting. The authors have made a nice effort in figuring out the details and their experimental evaluation. 
2. The ablation study of using the proposed "Synperiodic Filterbank" features with existing methods was definitely needed and insightful to show that features do carry more information than the traditional features. 

Weakness:
1. Personally I think using a significantly larger backbone of transformers on the proposed features to show improvement was not a fair comparison. I would recommend the authors to also experiment with the same or similar Transformer architecture using traditional MFCC or LFBE features to clearly understand what brings more value to the table - the model architecture or the feature extractor. From the existing ablation study it feels like the feature extractor offers a modest improvement at best. 
2. It appears that the authors have reported single / best model accuracy numbers and it is not clear how reliable these numbers are - perhaps reporting average metrics  + standard deviation across multiple model runs (with random initialization) would be better ?
3. Several related works concerning raw waveform based sound detection and separately raw waveform based source localization are missing and hence the related works section does not offer a balanced review of the problem space:
[1] T. Kim, J. Lee, and J. Nam, “Comparison and analysis of sample CNN architectures for audio classification,” IEEE Journal of Selected Topics in Signal Processing, vol. 13, no. 2, pp.285–297, May 2019. - Review of Raw-audio based architectures using CNN for event classification.
[2] J. M. Vera-Diaz, D.Pizarro, and J. M. Guarasa, “Towards end-to-end acoustic localization using deep learning: from audio signal to source position coordinates,” CoRR, vol. abs/1807.11094, 2018. Raw audio based single source localization
[3] H. Sundar, W. Wang, M. Sun and C. Wang, "Raw Waveform Based End-to-end Deep Convolutional Network for Spatial Localization of Multiple Acoustic Sources," IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 4642 - 4646. 
4. The paper can benefit greatly from re-writing the motivations and editing. A few examples of unnecessary claims not justified in the experiments and sentences which can benefit from revision are :
a) In the Abstract: "...Existing methods fail to do so because they either extract hand-engineered features (i.e. STFT, LogMel) that require a great deal of parameter tuning work (i.e. filter length, window size), or propose to learn a single filter bank to process sound waveforms in a single-scale that often leads to a limited time-frequency resolution capability" - I don't think the existing methods are "failing". They may not be performing as well as the model the authors of this paper have. 
b) In multiple places the phrase - "filter’s length and frequency response are inversely related" is used. This is not making sense to me. Should this be "filter’s length and frequency resolution are inversely related" ?
c) Sentence needs revision:
    i) "Traditional sound features are human-auditory system biased."
   ii) "Sound source detection has been previously treated as sound structure (Thrun, 2006), sound object(He et al., 2021) and sound event detection and localization (SELD task)"
  iii) "Our motivation is based on the fact that high frequency sound signal can be adequately detected with a shorter filter, enlarging the filter length doesn’t necessarily increase the frequency resolution but inevitably reduces the filter’s time resolution." Amenable to wrong interpretation - Are the authors referring to bio-inspired processing where humans are less sensitive resolving between 
  iv) "The design of this window function is motivated by the mel-scale frequency initialization strategy, in which two neighboring frequencies are more closely spaced in lower frequency part and higher frequency part" - not clear what this means. 
  v) "The parameter number is just linear to filter number." - needs revision. 
d) In Eq. (5) what is l (ell) ?

summary_of_the_review: Overall I think the authors address a curious idea. The presentation is not entirely widely accessible and can benefit greatly from revision. The experiments are interesting but not conclusive enough to understand the root cause of improvements.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This submission addresses the sound source detection problem. Compared to previous method, the authors propose a novel synperiodic filerbank compared to syndistance bank, which learns a data-dependent time-frequency resolution map. Experiments show that the proposed method achieves state-of-the-art performance on several datasets. 
main_review: **Note**: I am not an experienced researcher in the sound source detection. My reviews may miss some published results or technology. 

## Strengths

+ The paper is clearly written, with sufficient technical details.
+ The novelty and motivation is valid and the proposed method achieves state-of-the-art performance.

## Weakness

- Does the synperiodic filterbank consider the physical characteristics of the sound receiver system ($e.g.$, spatially-deployed microphone array)? If the receiver system structure is changed, does the synperiodic filterbank still generalize well?
summary_of_the_review: **Note**: I am not an experienced researcher in the sound source detection. My reviews may miss some published results or technology. 

Overall, I think this submission has solid novelty and state-of-the-art performance on the sound source detection task. Thus, I am inclined to accept the submission. However, my comments may not be professional and I would wait for authors response before making the final decision.
review_title: null

final_score: rating: 6: marginally above the acceptance threshold
review_content: summary_of_the_paper: This paper proposes a method for improving the detection and the localization of several sound sources using a 4-channel signal (The addressed configuration, determined or under-determined case is not detailed in this paper).
The proposed approach is based on a so-called "synperiodic" filter bank representation that is used as the input of a deep convolutional neural network. Finally method proposed by the authors is comparatively evaluated on the DCASE2020 dataset and pretends to provide the best results.


main_review: Despite interesting presented results with the proposed approach, this paper contains several major flaws.
For all the reasons reported below, I cannot recommend to accept this paper in the current form, however I encourage the authors to submit again an improved version of their work.

I) the overall organization of the paper should be improved. First,  it contains a lot of references to previous work for which the link with the present paper is not very clear (cf. Section 2). I recommend to formulate the problem before discussing the existing approaches to explain in what the new proposed method is new and significant. I also suggest to add a formal and clear definition of "synperiodicity" that is missing since it is not a standard concept in signal processing theory. This point is important  since it is a possible new contribution claimed by the authors. For the moment, I understand that "synperiodic" filter-bank is a reformulation of well-known concepts already rigorously presented in time-frequency analysis books such as "P. FLANDRIN, Time-frequency/time-scale analysis. Academic press, 1998". or "S. MALLAT, A wavelet tour of signal processing. Elsevier, 1999."  The authors should explain rigorously what is "synperiodicity" and "syndistance" with consideration with previously established theoretical results. The handled concepts such as time-frequency resolution are currently too superficially presented in the actual form of the paper.

II)The chosen mathematical notations are not always adequate. For example from Eq. (1), the goal of sound source detection is to estimate the number of sources K with their parameters of the sources S. Here, the link between the representation $\mathcal P$ and $S$ is not clear. Moreover, source detection and tracking is a problem not directly related to deep neural network that is only a possible approach to address this problem. I suggest to the author the use a standard and general introduction to the problem as presented in previous works to make the link.  Looking in detail, the formal definition of what is $sp_k$, $sl_k$ is not very clear, this is also the case for several parameters implicitly introduced in the paper such as $\sigma_i$, $\rho$, etc. The other equations and the following notations are not very rigorous too (eg. Eqs. (2)- (5) ) and don't help to clarify the understanding of the method and to make the link with standards tools such as the Fourier transform. For example, one should guess to what refers the frequency response $\phi(\omega_i)$ multiplied by the window function  $w$ that should be a function of time and should be parametrized by a width $\sigma_i$... In Eq.(5) the window now becomes a function of $\omega_i$ and $\rho$ that is no more consistent.
To fix that, I suggest to the authors to  only describe in detail their own contribution mainly summarized in the neural network architecture presented in Table 4 and to discuss the changes in comparison to similar deep neural-based approaches with dedicated simulations or theoretical results to motivate their choices. For example, I would be interested in a accurate comparison of the resolution of the proposed method with several existing TF representation such as STFT or CWT using different analysis windows.

III) Despite good claimed results, the experiments are not sufficiently detailed to be reproduced. What is exactly the evaluation protocol here ? What are the set of training/validation and test ? The computation time ?  Do the authors have reimplemented the compared state-of-the-art methods or did they only reported results from the literature ? Can please provide more details about the implementation of your method, used language and framework, providing implementation considerations and/or with source code ? Can you please explain how Fig. 4 and 5 are computed ? Can you please detail what are the axis with the proper units in all the figures ? Moreover, the authors should explain the evaluation metrics with a fair discussion with the other methods. What is the motivation of the ablation study? This  problem of interpreting the results is related to my Section II) where the authors should explain why their method perform better, discuss several relevant points with a consideration of the time-frequency resolution of the proposed "synperiodic" filterbank claimed as the main new contribution.

IV) The paper could also benefit of a major rewriting and English spell-check to fix typos and grammatical errors.
summary_of_the_review: The authors address a very interesting topic considering the computation of the suitable signal representation as the input of a deep neural network to address the problem of estimating the signal parameters of interest.
Despite the presented "good" results, the experiments are not reproducible since relevant information related to the protocol and the implementation are missing. Moreover, the theoretical concepts introduced in this study to describe the main contributions from the authors  are too superficially presented. 
Hence, I recommend to reject this paper in present state but I strongly encourage the authors to resubmit a more clear and more transparent study (for example to DCASE 22) to introduce their possibly novel approach.
review_title: null

Meta-review: This work studies the task of sound source localization from multi-channel audio. An approach to design a wavelet-like filter bank for audio feature extraction is proposed.

After discussion, all reviewers have given this work borderline ratings. Concerns were raised about the quality of the writing, missing related work, experimental methodology, and especially with regards to confounding factors in the experimental results, which make it difficult to assess the individual merit of the different components in the proposed system (i.e. features vs. transformer model). The authors have addressed this to some extent with additional experiments in the updated version of the manuscript, but this revealed that the gains obtained by the proposed feature extraction method in isolation are actually quite modest. This is in contrast with how the paper is written, with much more emphasis on this particular contribution than seems to be warranted by the empirical results.

Additionally, I believe the manuscript would benefit from a more careful and thorough revision to improve clarity and accessibility, beyond what is possible within a single review cycle. Therefore I am recommending rejection.
Decision: Reject