# IoTBystander: A Non-Intrusive Dual-Channel-Based Smart Home Security Monitoring Framework
## 4 Design Overview of IoTBystander  
In this section, we provide an overview of the proposed framework, IoTBystander. The notations used throughout this paper are summarized in Table 3. As shown in Figure 3, IoTBystander follows a modular design, comprising three key modules: audio-based event recognition (AER), traffic-based event recognition (TER), and dual-channel aggregation (DCA).  
\begin{table}
\begin{tabular}{c l} \hline \hline
**Symbol** & **Description** \\ \hline AER & Audio-based Event Recognition \\ TER & Traffic-based Event Recognition \\ DCA & Dual-Channel Aggregation \\ \(ES_{AER}\) & Event sequence obtained based on AER \\ \(ES_{TER}\) & Event sequence obtained based on TER \\ \(ES_{res}\) & Event sequence obtained by aggregating \(ES_{AER}\) and \(ES_{TER}\) \\ \(ks_{AER}\) & prior-knowledge set for audio channel \\ \(ks_{TER}\) & prior-knowledge set for traffic channel \\ \hline \hline \end{tabular}
\end{table}
Table 3: Notations.  
Figure 2: Interactions among entities in smart homes.  
The module AER identifies physical events related to both smart and dumb devices through the audio channel, which contains sounds made by these events. Audio signals are captured by microphones and transformed into spectrograms (i.e., image-style representations) via an _Audio2 Image_ component. The spectrograms are then taken as input by a _Recognizer_ model to identify physical events. The recognition result is encoded as an event sequence \(ES_{AER}\).  
Similarly, the TER module recognizes activities of the smart device (generating event notifications) from the traffic channel, where the network traffic generated during activities resides. Due to the generality of Wi-Fi in home area networks, this paper focuses on Wi-Fi traffic. We hook the access point in the home to obtain all the encrypted packets going between IoT devices/hubs and platforms. The captured packets are converted into _Feature tuples_ for further processing. As network packets carrying the same type of IoT event notification have highly fixed patterns denoted as signatures, events are recognized from ongoing packets through a _signature matching_ process and are denoted as an event sequence \(ES_{TER}\).  
The DCA module takes \(ES_{AER}\) and \(ES_{TER}\) as input and outputs a final event sequence \(ES_{aggr}\) after a line of three components: _event alignment_, _event verification_, and _event fusion_. Due to the distinct underlying mechanism of AER and TER, \(ES_{AER}\) and \(ES_{TER}\) denote the recognized physical events in different modalities (e.g., in terms of event types, time boundaries, etc.) and cannot be aggregated directly. _Event alignment_ handles this issue and converts \(ES_{AER}\) and \(ES_{TER}\) into a unified format. After that, _event verification_ handles the erroneous recognition results in \(ES_{AER}\) and \(ES_{TER}\) through cross verification. Finally, _event fusion_ fuses the monitoring results from both channels and outputs \(ES_{aggr}\), which can be used by decision models (e.g., anomaly detection) to further detect security/security issues, e.g., anomalies, attacks, failure, etc.  
In the following sections, we present the design details of the three key modules of IoTBystander: AER, TER, and DCA in Sections 5.1, 5.2 and 6, respectively.