Paper:
People with high-frequency hearing loss rely on hearing aids that employ frequency lowering algorithms. These algorithms shift some of the sounds from the high frequency band to the lower frequency band where the sounds become more perceptible for the people with the condition. Fricative phonemes have an important part of their content concentrated in high frequency bands. It is important that the frequency lowering algorithm is activated exactly for the duration of a fricative phoneme, and kept off at all other times. Therefore, timely (with zero delay) and accurate fricative phoneme detection is a key problem for high quality hearing aids. In this paper we present a deep learning based fricative phoneme detection algorithm that has zero detection delay and achieves state-of-the-art fricative phoneme detection accuracy on the TIMIT Speech Corpus. All reported results are reproducible and come with easy to use code that could serve as a baseline for future research.

Review:
## Updated review
I have read the rebuttal. I have some concerns about the new version of the paper: The addition of section 2.3 about the MFCCs is welcome but feels a bit out of place. The first part about the MFCC is interesting and relevant, but it could be in the introduction as a motivation. The second part about the "proprietary high-quality time-frequency filterbank" is not clear at all. Firstly, results are discussed so this part should be in the Evaluation section. Secondly, why using proprietary filterbanks and not the standard Mel filterbanks ?
Given that the rebuttal and the new version of the paper didn't address my major concerns, I am keeping my original rating.
## Original review
This paper presents an approach to detect fricative phoneme in speech with as little delay as possible, in the context of hearing aids improvement. The model is based on CNN and is trained to detect fricative given the past context. The model is evaluated in terms of recall and compared with recent published works. The results show that the proposed approach outperforms the baselines and yields state-of-the-art performance with no delay. The paper concludes with some analysis on the computational cost and draw possible future work.
This paper should be rejected for the following reasons:
- The novelty is very limited: this work applied a well-known architecture (CNN) to a common problem, phoneme recognition. This only novelty is the zero-delay constraint, which is probably not sufficient for ICLR.
- The significance is also limited given the very specialized application.  
- Some references are missing (see below).
- The presented results are not very clear.
- The computational considerations section is interesting but is missing some important elements.
Detailed comments:
- The authors selected raw speech signal as input to the CNN, which is not trivial and should be motivated and discussed in the paper. For instance, using the standard features like Mel filterbanks or MFCC will introduce a delay as they are computed on an overlapping window of 25ms. Phoneme recognition using raw speech as input to a CNN has been presented before, the authors should cite [1] and [2] for instance.
- Table 4 is confusing as the first four lines are not actually evaluated on TIMIT, so I don't see the point of adding these numbers to the table, as they cannot be compared anyway. I would remove these four lines from the Table.
- In terms of previous works, phoneme recognition on TIMIT is a very popular task, and many others could be cited, such as [3-5].
- On the computation consideration, the analysis is interesting, but a discussion on the size (i.e. number of parameter) of the network is missing: one way to decrease computation time is to have a smaller network, which is in line with the application: hearing aids probably do not have gigabytes of ram available. 
- Question about the network: the input segment seems to be of size 3072 samples, why ? any motivation for this particular input size ?
My review can seem to be a bit harsh, I actually enjoyed the paper, but I don't think ICLR is the right conference for it, and I would advise the authors to improve it and submit it to a speech conference.
References:
[1] Palaz, D., Magimai Doss, M. and Collobert, R.. "Estimating Phoneme Class Conditional Probabilities from Raw Speech Signal using Convolutional Neural Networks." Proceedings of Interspeech 2013.
[2] Zeghidour, N., Usunier, N., Kokkinos, I., Schaiz, T., Synnaeve, G., & Dupoux, E. "Learning filterbanks from raw speech for phone recognition". Proceedings of ICASSP 2018.
[3] Zhang, Ying, Mohammad Pezeshki, Philémon Brakel, Saizheng Zhang, Cesar Laurent, Yoshua Bengio, and Aaron Courville. "Towards end-to-end speech recognition with deep convolutional neural networks." arXiv preprint arXiv:1701.02720 (2017).
[4] Chorowski, Jan K., et al. "Attention-based models for speech recognition." Advances in neural information processing systems. 2015.
[5] Tóth, László. "Phone recognition with hierarchical convolutional deep maxout networks." EURASIP Journal on Audio, Speech, and Music Processing 2015.1 (2015): 25.