# Random Segmentation: New Traffic Obfuscation against Packet-Size-Based Side-Channel Attacks
## 5 Evaluation and Results
### Effectiveness  
To evaluate the randomness in the packet length introduced in our technique, we developed a program to simulate our defense on a WiFi trace of four IoT devices: doorbell,camera, light bulb, and smart plug. We captured the encrypted traffic for one hour in different operating modes (e.g., ON-OFF).  
Our program reads the pcap file of each device and produces the obfuscated traffic to test our defense against DF attacks. Table 1 outlines our configuration for the adjustable system parameters introduced in Sections 4.2 and 4.3. The probability was manually chosen with the goal of minimizing the overhead (i.e., reducing the number of segmented packets) while maintaining a lower accuracy. The initial value of 0.6 resulted in a high accuracy (>80). Consequently, we increased the probability to 0.7, and the accuracy remained consistently high. Subsequently, when we further elevated the probability to 0.8, a reduction in accuracy was achieved. We used the same defense parameters for each group of devices categorized based on the traffic intensity because different parameters will likely create new patterns to distinguish the devices.  
Furthermore, we run another simulation to obfuscate our captured trace using the random padding introduced in [15] to compare the effectiveness of a traditional noise-based solution with our defense.  
The attacker's profiling classifier is trained using the training data from the original trace. To assess the attack's performance without defense, we test the classifier using the testing data derived from the original trace.  
Likewise, we evaluate the defense approach by initially creating a modified trace using our defense program, relying on the original trace as a foundation. Subsequently, we proceed to train the attack classifier using the training data from the modified trace and then test the classifier using the corresponding test data in the modified trace.  
We use Random Forest for our classification due to its outperformance on similar IoT device-identification attacks compared with several other ML algorithms [2; 4]. We randomly divided our dataset into 70% for training and 30% for testing and quantified the performance of our classifier using the following metrics: accuracy, precision, recall, and F1 score. The specific computation of these metrics can be found in the Appendix A.  
Note that we show the indistinguishability of devices by applying our technique to training and testing data; the closer the accuracy to random guessing (50%), the more effective our defense is in confusing the classifier. Random guessing attains an accuracy rate of \(1/k\), where \(k\) is the number of labels/devices. As we confuse the attacker between two devices of similar traffic intensity, then \(k=2\).  
We evaluate against an attacker who exploits the packet size and direction only. Thus, we construct our dataset using packet sizes in a binary format to represent directions; a positive size represents incoming packets, and a negative size, outgoing ones. Similar to [2; 16], we break the trace into sequences observed within a 30 s time window for classification.