From: NIPS.cc/2018/Workshop/IRASL
title: Acceptance Decision
decision: Accept

From: NIPS.cc/2018/Workshop/IRASL/Paper13/AnonReviewer2
title: a relevant paper with sufficient novelty
review: In recent years, approaches have emerged where speech signal is directly modeled by artificial neural networks (typically, convolutional neural networks) for speech processing. This paper is a contribution in that direction where the filters in the first convolution layer parameters are constrained by assuming a functional form (sinc function).  In doing so, it reduces the number of first convolution layer parameters and also leads to faster convergence. The approach has been validated on speaker identification and speech recognition tasks. Although a big chunk of the paper appears in SLT workshop paper, the contribution is sufficiently novel and is relevant for the audience of the conference.  

Having said that, the paper has a few weaknesses.

1. The paper focuses on intepretability of convolutional filters in raw speech modeling. It is stated that  
"The downside of raw speech processing lies in the possible lack of interpretability of the filter bank
learned in the first convolutional layer"
This is not entirely true. As the pertinent question is ascertaining the information modeled or learned by the CNNs. Previous works on raw speech signal modeling studies have shown that the filters learned in the first convolution layer can be interpreted as a "spectral dictionary" and the information learned by the CNNs can be ascertained.  See,
(a) D. Palaz, M. Magimai-Doss and R. Collobert,"End-to-End Acoustic Modeling using Convolutional Neural Networks for Automatic Speech Recognition", Idiap Research Report Idiap-RR-18-2016, 2016.
(b) Citation [35] in the paper
(c) H. Muckenhirn, M. Magimai.-Doss and S. Marcel,"On Learning Vocal Tract System Related Speaker Discriminative Information from Raw Signal Using CNNs ", In Proc. of Interspeech 2018.
(d) S. H. Kabil, H. Muckenhirn and M. Magimai.-Doss,"On Learning to Identify Genders from Raw Speech Signal Using CNNs", in Proc. of Interspeech 2018.
The paper does not touches any of these points.

The spectral dictionary interpretation is also supported by signal theoretic analysis of CNNs. For instance, see
(a) Vardan Papyan, Yaniv Romano, and Michael Elad,  “Convolutional neural networks analyzed via convolutional sparse coding,” Journal of Machine Learning Research, vol. 18, no. 83, pp. 1–52, 2017.
(b) Stéphane  Mallat, “Understanding  deep  convolutional  networks,” Philosophical Transactions of the Royal Society A: Mathematical, Physical and Engineering Sciences,  vol.  374, no. 2065, 2016.

Furthermore, raw speech signal based CNNs can be analyzed used gradient-based visualization techniques, which have originated from the computer vision community.  See
H. Muckenhirn, V. Abrol, M. Magimai.-Doss and S. Marcel,"Gradient-based spectral visualization of CNNs using raw waveforms ", Idiap Research Report Idiap-RR-11-2018, 2018.

Yet another work that deserves to be discussed is the following
Hardik B. Sailor, Hemant A. Patil, "Novel Unsupervised Auditory Filterbank Learning Using Convolutional RBM for Speech Recognition". IEEE/ACM Trans. Audio, Speech & Language Processing 24(12): 2341-2353 (2016)

In summary, although it may not be easy to interpret the learned filters from just the parameters, the information learned by the CNNs can be found. Also the filters need not be band pass in nature. Some spectral information may need more support than others depending upon the task at hand. Cumulative frequency response provides that information.  

It is good to consider the point of "interpretability of convolutional filters" taking these existing works into account.

2.  The paper does not provides any information about the architecture of the systems studied. Without that information it is difficult to see beyond the numbers presented as results.
The reviewer is specifically interested in the block length "L" and shift considered for the study. In works pointed above the block length is a hyper parameter that is found via cross validation. For instance, in speech recognition studies it is about 2ms and in speaker recognition studies the block length is about 2ms or about 20 ms depending upon the type of information to be learned. What were the considerations to decide the block length and shift in the present work? Furthermore the cumulative frequency response of the first convolution layer is also related to the whole architecture.
rating: 4: Top 50% of accepted papers, clear accept
confidence: 3: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature

From: NIPS.cc/2018/Workshop/IRASL/Paper13/AnonReviewer1
title: Good paper; just enough novelty for a workshop paper
review: This paper explores a new approach to parameterizing the first-layer filters in convolutional nets applied to audio tasks such as speech recognition and speaker identification. In the standard approach, the filters (which are simply finite-impulse response filters applied to the input audio) are fully parameterized, with no constraints imposed on the parameters or the structure of the learned filters. In contrast, the proposed approach parameterizes the filters such that they are constrained to be rectangular bandpass filters (up to the limits of the window function used in the network), with only the low- and high-frequency cutoffs for each filter being learned from data. This proposed approach has a number of potential advantages, including a reduced number of parameters, potentially faster convergence during training, and enhanced interpretability. The proposed model is called "SincNet," after the sinc function, which is the Fourier transform of an ideal low-pass filter. An experiment in which standard and SincNet models are trained to perform speech recognition on TIMIT data corrupted with band-limited additive noise shows that while both models learn to ignore the corrupted frequency range, SincNet learns to do this faster. Plots of the cumulative frequency response of filterbanks learned by conventional and SincNet models learning a speaker identification task suggest that the SincNet model is more able to focus on frequency ranges that are important for the speaker-ID task (namely, the ranges corresponding to pitch and the first and second formants). Finally, experiments show that SincNet models can outperform other models (both ones operating at the signal level and ones operating on higher-level features like MFCCs or log mel spectra) on both speaker ID and speech recognition tasks.

This is a clear, well-written paper that is nicely matched to the theme of the workshop. The proposed parameterization makes a lot of sense, and I find the experiments to be convincing.

About the only flaw I see in the paper is that there is not a lot of novel content here compared to the SLT paper on the same subject, but there is enough for a workshop paper, especially given that the IEEE SLT workshop and the NIPS workshops cater to different communities.

It would be interesting to compare the cumulative frequency responses of SincNet models trained to do speech recognition on two different languages: one in which tone is a distinctive feature (such as Chinese or Vietnamese) and one in which it is not (such as French or English). Would the network trained on the tonal language devote more resources to representing pitch?

Does Fig. 3 show frame error rate on the training data, held-out data, or test data? Are the trends similar on all portions of the TIMIT dataset?

Footnote 3 should have been anonymized for review.

Proofreading

but also raised crucial concerns => but also raises crucial concerns

results not reported here reveals no significant => results not reported here reveal no significant

rather far from what achieved with DNN systems => rather far from what is achieved with DNN systems

and it more interpretable than => and is more interpretable than


rating: 4: Top 50% of accepted papers, clear accept
confidence: 3: The reviewer is absolutely certain that the evaluation is correct and very familiar with the relevant literature

From: NIPS.cc/2018/Workshop/IRASL/Paper13/AnonReviewer3
title: The paper deals with learning the parameters of a speech processing filter bank  by integrating it into a CNN as its lowest layer. There have been several such attempts earlier. The novelty of the paper is that it constraints the shape of the filters based on a priori assumptions. The paper is technically correct, references are plenty, the experiments are convincing. However, the paper has a 70-80% overlap with an earlier paper.
review: Critics:
The authors failed to stay anonymous, as the github link given in the paper contains their name. Moreover, they cite their own earlier work ([38]), which is not even released yet (although available on arxiv).  Comparing this work with that paper, there is about 70-80% overlap. The theoretical description and half of the experiments is the same (including the text, the formulas and the figures). The new contribution of this paper is the speech recognition experiments summarized in  Figure 5 and Table 3. I am not familiar with the official policy of NIPS about this, but I think I should mention.

Technical comments:
all elements of each filter - "elements of a filter" sounds a bit sloppy me, perhaps "each coefficient of the impulse response of each filter" would be better

Figure 3. Frame error rate - on the train or the test set?

Figure 5. shows the result after an early training stage (after processing one hour of speech) - This is unclear. To me,  "early training stage" means after a  few training epochs, where each epoch goes through all the training data. So it seems to contradict "after processing one hour of speech". Or is the figure is made after not even finishing the first epoch?? Please, add a sentence that clarifies this.

Please take care to force all abbreviations to capital in the references (there is at least one "cnn").

Philosophical comments:
Earlier similar papers try to learn the filters without any restriction. Here, the authors force them to be bandpass filters with a fixed shape. Although the authors do not mention, i guesss this is motivated by what we know about the shape of auditory filters of human hearing. Introducing such a priori constraints leads us back to the classic debate whether to use manually designed features or let deep learning solve everything. While I would like to avoid taking a stand on this, I think that the statements that the filters learned without restriction are "less meaningful", "do not appeal to "human intuition" and "lack interpretability" are too strong. They just do not fulfill are expectations, thats all. They might still work, 
so I prefer seeing the experimental results.
In line with this, saying that filters that take weird shapes are less "interpretable", or worse, they do no have a "well-defined frequency response"(!) is very controversial to me. The frequency response is alwas well-defined and displayable. Can we call it "less interpretable" simply because it does not fulfill are expectation?

For the speaker recognition task, the authors say that the network "tend to correctly focus on the lower part of the spectrum", in particular to the 1st and 2nd formants. Pehaps a citation would be good here, to underpin why it is correct.
At least, if I recall my lessons from phonetic, I learned that the lower formants are more informative about the linguistic content, while "the fourth and higher formants are indicative of a speakers voice quality" (see, eg. Ladefoged, Johnson: A course in phonetics). 
rating: 4: Top 50% of accepted papers, clear accept
confidence: 2: The reviewer is fairly confident that the evaluation is correct