## 3 Inter-Floor Noise Classification
### Convolutional Neural Network-Based Classifier  
The CNN-based classifiers designed for image recognition already demonstrated source type classification and localization in the previous studies [9; 10]. These prior studies extended the application of CNN to source type classification and localization of inter-floor noise. Several state-of-the-art CNNs [49; 50; 51], which have been widely employed in many other applications, were tested against inter-floor noise classification tasks using a dataset obtained from the campus building. The inter-floor noise classification tasks to be stated and solved in this study proceeded upon the assumption that CNNs designed for image recognition can be adopted for these tasks. In this study, VGG16 [51] is employed as a CNN-based feature extractor from an inter-floor noise in the form of log-scaled Mel-spectrogram \(\mathbf{P}\). This architecture showed the best performance among several state-of-the-art CNNs for inter-floor noise classification in the previous study [10]. An alternative to this approach is a one-dimensional CNN, which does not require conversion of the inter-floor noises to image-like features. The kernels of one-dimensional CNN eventually learn a set of Mel-filters, which can be considered as a set of basis functions in different frequencies [52; 53; 54]. An audio signal filtered by the bottom convolutional layers of a one-dimensional CNN can be compared with \(\mathbf{P}\). At this point, \(\mathbf{P}\) is almost equivalent to the output of the bottom convolutional layers of a one-dimensional CNN. Figure 4 illustrates the flows of the inter-floor noise classification using the two-dimensional CNN (Figure 4a) and one-dimensional CNN (Figure 4b). Two-dimensional CNN-based classification starts with (1) conversion of a signal to a log-scaled Mel-spectrogram \(\mathbf{P}\), (2) convolutional layers and fully connected layers composing VGG16 as a feature extractor, (3) an adaptation layer (fc), and finishes with (4) classification using a softmax function. These were implemented with TensorFlow [55]. An inter-floor noise signal is converted to a log-scaled Mel-spectrogram \(\mathbf{P}\in\mathbb{R}^{H\times W}\) through the following steps, where the height \(H\) and width \(W\) are both defined as 224 by the input size of VGG16. This conversion was implemented with librosa [56]. A signal \(\mathbf{s}\in\mathbb{R}^{k_{s}}\) containing an inter-floor noise event is extracted from an audio clip in the dataset, where \(l_{s}\) represents \(f_{s}\) times the signal length \(t\). The event start position in \(\mathbf{s}\) is detected using the onset detection described in Section 3.1. Several \(t=\{0.152,\,0.501,\,1.00,\,1.50,\,2.00,\,3.00\}\) s are tested to identify the effect of \(t\), where it can be considered a hyperparameter. The amplitudes of \(\mathbf{s}\) are rescaled to values between \([-1,1]\). Short time samples with size of \(l=1024\) from \(\mathbf{s}\) are windowed, where the window is an \(N=2^{13}\)-point Hanning window. \(l\) sets the spectral line resolution as approximately 40 Hz for the given \(f_{s}\). The windowed short time sample \(\mathbf{x}_{w}\), \((w=0,\cdots,W-1)\) is converted to a spectral power using a discrete Fourier transform  
\[ \begin{split}\mathbf{X}_{w}(k)=|\text{DFT}[\mathbf{x}_{w}(n)]|^{ 2}=\left|\sum_{n=0}^{N-1}\mathbf{x}_{w}(n)e^{-\jmath\pi nk/N}\right|^{2},\\ k=0,1,\cdots,N.\end{split}  \tag{6}\]  
The start position of the next short time sample is determined by the hop size, \(h=\{30,\,99,\,197,\,296,\,394,\,591\}\) sample, to achieve \(W\) of \(\mathbf{P}\) for the given \(t\). A block of the windowed short time samples \(\mathbf{x}=[\mathbf{x}_{0}\,\mid\,\cdots\,\mid\,\mathbf{x}_{W-1}]\in\mathbb{ R}^{N\times W}\) is converted to a power spectrogram \(\mathbf{X}\) by Equation (6). Then, it is converted to a Mel-spectrogram  
\[\mathbf{M}=\mathbf{F}\mathbf{X}, \tag{7}\]  
Figure 4: A flow of the inter-floor noise classification.