Venue: ICLR 2020 Conference

Reviews:

final_score: rating: 3: Weak Reject
review_content: ====================================== Updated Review =====================================
I would like to thank the authors for providing more experiments and details regarding their work.
However, after reading the authors rebuttal, I still think that there is more work to do in terms of comparison to prior work. That way it would be much clearer what is contribution of this work, and how it can be used for future research in that field. 

Hence, I would like to keep my score as is. 
==========================================================================================

This paper describes a method for fricative phonemes boundary detection with zero delays. The authors suggest optimizing a convolutional based neural network with a binary cross loss function to detect such events. 
The authors provide results on the TIMIT dataset and compare the proposed model to several baselines. 

The task of phoneme boundary detection was well studies under different setups and is very important for various applications, including the one proposed in this paper.

However, I have some major concerns regarding this paper, which I would like the authors to clarify. Without these, it is hard to understand the contribution in this paper.

1) the authors chose to model the problem from the raw wave. Although it is getting popularity in several speech processing tasks, it is not clear why not using magnitude/MFCC, for example. In case the authors claim that learning from the waveform is better, I suggest providing a comparison to other features.  
Additionally, did the authors experience with simpler architectures Maybe more shallow models? Regarding supervision, did the authors tried comparing to the method proposed by [2] but with a unidirectional RNN? Similar to [3].

2)  If I understand it correctly, the motivation for this task was: accurate detection of fricatives boundary can be used to shift into lower frequency bands in hearing aids. It seems like the boundaries are more important than other phoneme parts such as a mid phoneme, for example. 
In that case, a better metric might be Presicion + Recall + F1 + R-val next to the boundaries (for instance, with a tolerance level of 10-20ms). Those metrics were suggested on several studies of phoneme segmentation, [1], [2]. 

3) The comparison in Table 3 is very strange. Results are reported on different datasets. Although the authors mentioned it in the caption, it is still misleading. I suggest the authors to compare either obtain results on the same benchmark or compare to other baselines.

Minor comments: 
"If for the majority of the samples in a phoneme our network’s output is greater than the threshold we set" -> not a clear sentence. 

[1] Franke, Joerg, et al. "Phoneme boundary detection using deep bidirectional lstms." Speech Communication; 12. ITG Symposium. VDE, 2016.
[2] Michel, Paul, et al. "Blind phoneme segmentation with temporal prediction errors." arXiv preprint arXiv:1608.00508 (2016).
[3] Adi, Yossi, et al. "Automatic Measurement of Voice Onset Time and Prevoicing Using Recurrent Neural Networks." INTERSPEECH. 2016.
review_title: Official Blind Review #1

final_score: rating: 6: Weak Accept
review_content: This paper apples supervised deep learning methods to detect exact duration of a fricative phoneme in order to improve practical frequency lowering algorithm. A major challenge compared to existing work is to have an algorithm with nearly zero delay while preserving detection accuracy. A deep convolutional neural network is trained for this purpose and it is validated on TIMIT dataset.

After a careful preprocessing of the data, long segments of raw audio are given as input to the convolutional net. It is trained as a binary classification problem. Therefore, for each different phenome, a different network is needed. To improve the accuracy, Majority voting is also adopted. This however increases the computational cost. To address this issue, an extrapolation detection problem is formulated to predict the fricative phoneme a few ms in advance. Extensive numerical results show that the approach still outperforms the method of Ruinskiy & Lavner (2014) in Unweighted Average Recall. 

I find the accuracy attained by the neural nets quite impressive, although more insights would be favored to understand what is going on. This is yet an interesting application of deep learning useful for real-life problems. If the method could be tested on another dataset, the result would be more convincing. 

review_title: Official Blind Review #3

final_score: rating: 3: Weak Reject
review_content: 
## Updated review

I have read the rebuttal. I have some concerns about the new version of the paper: The addition of section 2.3 about the MFCCs is welcome but feels a bit out of place. The first part about the MFCC is interesting and relevant, but it could be in the introduction as a motivation. The second part about the "proprietary high-quality time-frequency filterbank" is not clear at all. Firstly, results are discussed so this part should be in the Evaluation section. Secondly, why using proprietary filterbanks and not the standard Mel filterbanks ?

Given that the rebuttal and the new version of the paper didn't address my major concerns, I am keeping my original rating.

## Original review

This paper presents an approach to detect fricative phoneme in speech with as little delay as possible, in the context of hearing aids improvement. The model is based on CNN and is trained to detect fricative given the past context. The model is evaluated in terms of recall and compared with recent published works. The results show that the proposed approach outperforms the baselines and yields state-of-the-art performance with no delay. The paper concludes with some analysis on the computational cost and draw possible future work.

This paper should be rejected for the following reasons:
- The novelty is very limited: this work applied a well-known architecture (CNN) to a common problem, phoneme recognition. This only novelty is the zero-delay constraint, which is probably not sufficient for ICLR.
- The significance is also limited given the very specialized application.  
- Some references are missing (see below).
- The presented results are not very clear.
- The computational considerations section is interesting but is missing some important elements.

Detailed comments:
- The authors selected raw speech signal as input to the CNN, which is not trivial and should be motivated and discussed in the paper. For instance, using the standard features like Mel filterbanks or MFCC will introduce a delay as they are computed on an overlapping window of 25ms. Phoneme recognition using raw speech as input to a CNN has been presented before, the authors should cite [1] and [2] for instance.
- Table 4 is confusing as the first four lines are not actually evaluated on TIMIT, so I don't see the point of adding these numbers to the table, as they cannot be compared anyway. I would remove these four lines from the Table.
- In terms of previous works, phoneme recognition on TIMIT is a very popular task, and many others could be cited, such as [3-5].
- On the computation consideration, the analysis is interesting, but a discussion on the size (i.e. number of parameter) of the network is missing: one way to decrease computation time is to have a smaller network, which is in line with the application: hearing aids probably do not have gigabytes of ram available. 
- Question about the network: the input segment seems to be of size 3072 samples, why ? any motivation for this particular input size ?

My review can seem to be a bit harsh, I actually enjoyed the paper, but I don't think ICLR is the right conference for it, and I would advise the authors to improve it and submit it to a speech conference.

References:
[1] Palaz, D., Magimai Doss, M. and Collobert, R.. "Estimating Phoneme Class Conditional Probabilities from Raw Speech Signal using Convolutional Neural Networks." Proceedings of Interspeech 2013.
[2] Zeghidour, N., Usunier, N., Kokkinos, I., Schaiz, T., Synnaeve, G., & Dupoux, E. "Learning filterbanks from raw speech for phone recognition". Proceedings of ICASSP 2018.
[3] Zhang, Ying, Mohammad Pezeshki, Philémon Brakel, Saizheng Zhang, Cesar Laurent, Yoshua Bengio, and Aaron Courville. "Towards end-to-end speech recognition with deep convolutional neural networks." arXiv preprint arXiv:1701.02720 (2017).
[4] Chorowski, Jan K., et al. "Attention-based models for speech recognition." Advances in neural information processing systems. 2015.
[5] Tóth, László. "Phone recognition with hierarchical convolutional deep maxout networks." EURASIP Journal on Audio, Speech, and Music Processing 2015.1 (2015): 25.
review_title: Official Blind Review #2

Meta-review: The reviewers appreciate the importance of the problem, and one reviewer particularly appreciated the gains in performance. However, two reviewers raised concerns about limited novelty and missing comparisons to prior work. While the rebuttal helped address these concerns, the novelty is still limited. The authors are encouraged to revise the presentation to clarify the novelty.
Decision: Reject