# IoTBystander: A Non-Intrusive Dual-Channel-Based Smart Home Security Monitoring Framework
## 5 Event Recognition
### Audio-Based Event Recognition (AER)  
We developed an AER model consisting of two modules, _Audio2 Image_ and _Event Recognizer_, to identify physical events in smart home environments. The detailed framework is illustrated in Figure 4.  
Figure 3: System overview of IoTBystander. The ellipsis (“...”) indicates that the event sequence continues with additional events following the same pattern.  
The _Audio2 Image_ module converts audio signals into spectrogram representations. We begin by segmenting each WAV-format audio signal into 960 ms frames to capture local temporal features while minimizing the effects of dynamic variations. Each frame is sampled at 48 kHz with a bit depth of 32 bits, followed by the application of a 25 ms Hann window with a 10 ms stride. The Short-Time Fourier Transform (STFT) is then applied, resulting in a linear spectrogram of size \(96\times 257\). While this transformation captures frequency components across both high and low frequencies, the uniform resolution does not effectively represent low-frequency details, which are critical for audio recognition tasks. To address this, we use a 64-band Mel filter bank to divide the linear spectrogram into 64 frequency bands, which improves resolution for low frequencies and filters out high-frequency noise. This reduces dimensionality, producing a Mel spectrogram of size \(96\times 64\). Finally, we apply logarithmic smoothing to further compress energy fluctuations, enhancing feature extraction and mitigating audio noise interference, ultimately producing a log-Mel spectrogram with improved robustness for recognition.  
The _Event Recognizer_ module is designed to identify events from audio data, represented by log-Mel spectrograms. It consists of two general components: _Feature Extractor_ and _classifier_, which can be implemented using various neural network models. In this work, we utilize the pre-trained _Vggish_ model for feature extraction and a support vector machine with a Radial Basis Function (_SVM-RBF_) for classification. This structure was chosen because of its ability to achieve high recognition performance with a relatively small amount of labeled audio data, which is particularly beneficial in situations where time and expertise for extensive labeling are limited. The _Vggish_ architecture includes convolution and pooling operations with kernels of size \(3\times 3\) and \(2\times 2\), respectively, and a stride of 1, as detailed in Table 4. After extracting features from the spectrogram, we apply principal component analysis (PCA) and whitening using pre-trained parameters (_pca_matrix_ and _pca_means_), yielding a more efficient and sparse feature representation. This results in a 128-dimensional vector for each short-term frame. While both _SVM-RBF_ and Multi-Layer Perceptron (MLP) models have demonstrated favorable results in previous studies [12], we selected _SVM-RBF_ for the classification task in this work. Notably, the framework is highly flexible, and alternative models can be employed for both the feature extraction and classification components in different IoT applications or settings.