From: NIPS.cc/2018/Workshop/IRASL
title: Acceptance Decision
decision: Accept

From: NIPS.cc/2018/Workshop/IRASL/Paper6/AnonReviewer3
title: a study on quaternion neural networks based phone recognition
review: This paper investigates quaternion convolutional neural networks (qCNN) and quaternion recurrent neural networks (qRNN) for phone recognition. Through investigation on TIMIT corpus, it is demonstrated that qCNN and qRNN approach yield systems that have less parameters and yield better performance  than the conventional approach of modeling static and dynamic features as a concatenated feature.

While the comparative study is somewhat interesting, the insight gained is little. For instance, conventional approaches limit the dynamic feature up to second order (static+delta+delta-delta). In qCNN and qRNN, the modeling goes up to third order derivative. Did the baseline systems model third order derivative? Did the authors try to figure out the impact of this difference?

The quaternion approach separates modeling of the different features through fundamental quaternion units. In the literature, there have been works where modeling of static and dynamic features can be treated as a multi stream combination problem. For instance, see
A. Hagan and A. Morris, Recent advances in the multi-stream HMM/ANN hybrid approach to noise robust ASR, Computer Speech & Language, 19(1), 2005, pp 3--30.
Can a similarity be drawn to such an approach although a single stream posterior probability is estimated with quaternion approach (i.e. there is no probability combination involved)?

rating: 3: Marginally above acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct

From: NIPS.cc/2018/Workshop/IRASL/Paper6/AnonReviewer1
title: Feedback on Quaternion Neural Networks 
review: The paper reads very well.

My main concern with the paper is novelty. As the authors cited QCNN and QRNN are not new.
The comparison is interesting. I like the fact, they are releasing the code. They could have released it for review as well under anonyms URL. It would have been interesting to report real time in their comparison as they report #params. RT or size may be indicative as well for what does. Do the authors plan to study the quaternion setup for LVCSR as well?

rating: 3: Marginally above acceptance threshold
confidence: 1: The reviewer's evaluation is an educated guess

From: NIPS.cc/2018/Workshop/IRASL/Paper6/Authors
title: Thanks you + answers 
comment: We would like to first thank reviewer 2 for the useful feedback. In the following, we address the points raised:

1. We agree with reviewer 2 that the first occurence of the “realistic” term is not appropriate, indeed TIMIT is not a realistic task, and we changed this word. However, the second occurence of “realistic” denotes the best PER observed with respect to the validation data-set, highlighting realistics testing conditions from an unknown test set. 

2. Reviewer 2 stated that both models (QRNN and QCNN) presented in this paper have  already been published, and this is partially true. While the QCNN has indeed been presented in an oral session at Interspeech 2018, the QRNN remains an unofficial ArXiv preprint, and therefore not peer reviewed yet. We also would like to recall the fact that the IRASL workshop is willing to accepts already submitted works. 

3. We regret the fact that reviewer 2 finds the contribution not clear enough, and we clarify it in both the paper and the following. One of the official question of the IRASL workshop is: “How can we design more powerful networks with simpler architectures ?”. We believe that QNNs answer this question by introducing better multidimensional input representations, and smaller neural networks with better results compared to real-valued NN. In the case of speech recognition, QCNN and QRNN remain mostly unexplored with only one published work (QCNN) and one published but non-peer reviewed one (QRNN). Therefore, we propose a small overview of QNNs to highlight the ability of such models to answer an important question raised by the NIPS IRASL workshop. 

4.a) In the case of QCNN, results differ due to 1) Different input features. Indeed original QCNN paper employed two time derivatives, while the current implementation is based on three derivatives. 2) Different architectures as mentioned by reviewer 2.

4.b) The ArXiv version of QRNN is an old version of the QRNN which is about to be updated. The differences on the results are explained by: Different learning rate optimizer (Adam / RMSPROP) and different activation functions (ReLU / tanh). Also, the quaternion acoustic feature Section of the ArXiv version unfortunately contains an error (About to be corrected in a near update of the preprint). Indeed, inputs are fMLLR features that are adapted to speakers (i.e. better results). 

We would like to thanks again reviewer 2 for the remarks, and we would like to mention that all the requested changes are done. Therefore, a new version of the paper will be uploaded for camera-ready.

From: NIPS.cc/2018/Workshop/IRASL/Paper6/AnonReviewer2
title: Quaternion neural networks revisited
review: This paper provides an analysis of quaternion neural networks on the TIMIT database. Specifically, a quaternion convolutional neural network (QCNN) is compared to a real-valued CNN in a end-to-end framework, and a quaternion recurrent neural network (QRNN) is compared to an RNN within a more traditional HMM-based system. However, I beg to differ: this setup is *NOT* one of the "realistic conditions of speech recognition".

The paper uses essentially the same model as [27] (CNN and QCNN using a CTC loss function), and adds a comparison with a traditional HMM-based ASR system using RNN and QRNN models (previously published in [26]). Other than putting these two papers together into one, it is not clear what the contribution of this paper is going to be. Confusingly, the results reported are not always the same. Sometimes, the difference can be explained by a different model structure (3 or 10 layers vs 4 layers), but in the case of [26]'s Table 1 vs Table 2 here, I am not sure what changed. Please clarify.

rating: 2: Marginally below acceptance threshold
confidence: 2: The reviewer is fairly confident that the evaluation is correct