Paper:
Published as a conference paper at ICLR 2018 LEARNING FROM BETWEEN-CLASS EXAMPLES FOR DEEP SOUND RECOGNITION
Deep learning methods have achieved high performance in sound recognition tasks. Deciding how to feed the training data is important for further performance improvement. We propose a novel learning method for deep sound recognition: Between-Class learning (BC learning). Our strategy is to learn a discriminative feature space by recognizing the between-class sounds as between-class sounds. We generate between-class sounds by mixing two sounds belonging to different classes with a random ratio. We then input the mixed sound to the model and train the model to output the mixing ratio. The advantages of BC learning are not limited only to the increase in variation of the training data; BC learning leads to an enlargement of Fisher's criterion in the feature space and a regularization of the positional relationship among the feature distributions of the classes. The experimental results show that BC learning improves the performance on various sound recognition networks, datasets, and data augmentation schemes, in which BC learning proves to be always beneficial. Furthermore, we construct a new deep sound recognition network (EnvNet-v2) and train it with BC learning. As a result, we achieved a performance surpasses the human level 1 .

Review:
This manuscript proposes a method to improve the performance of a generic learning method by generating "in between class" (BC) training samples. The manuscript motivates the necessity of such technique and presents the basic intuition. The authors show how the so-called BC learning helps training different deep architectures for the sound recognition task.
My first remark regards the presentation of the technique. The authors argue that it is not a data augmentation technique, but rather a learning method. I strongly disagree with this statement, not only because the technique deals exactly with augmenting data, but also because it can be used in combination to any learning method (including non-deep learning methodologies). Naturally, the literature review deals with data augmentation technique, which supports my point of view.
In this regard, I would have expected comparison with other state-of-the-art data augmentation techniques. The usefulness of the BC technique is proven to a certain extent (see paragraph below) but there is not comparison with state-of-the-art. In other words, the authors do not compare the proposed method with other methods doing data augmentation. This is crucial to understand the advantages of the BC technique.
There is a more fundamental question for which I was not able to find an explicit answer in the manuscript. Intuitively, the diagram shown in Figure 4 works well for 3 classes in dimension 2. If we add another class, no matter how do we define the borders, there will be one pair of classes for which the transition from one to another will pass through the region of a third class. The situation worsens with more classes. However, this can be solved by adding one dimension, 4 classes and 3 dimensions seems something feasible. One can easily understand that if there is one more class than the number of dimensions, the assumption should be feasible, but beyond it starts to get problematic. This discussion does not appear at all in the manuscript and it would be an important limitation of the method, specially when dealing with large-scale data sets.
Overall I believe the paper is not mature enough for publication.
Some minor comments:
- 2.1: We introduce --> We discussion
- Pieczak 2015a did not propose the extraction of MFCC.
- the x_i and t_i of section 3.2.2 should not be denoted with the same letters as in 3.2.1.
- The correspondence with a semantic feature space is too pretentious, specially since no experiment in this direction is shown.
- I understand that there is no mixing in the test phase, perhaps it would be useful to recall it.