# Dual-IoTID: A Session-Based Dual IoT Device Identification Model
## 5 Performance Evaluation
### Experimental Results  
We conducted two experiments to evaluate the performance of Dual-IoTIID. The first experiment included only the communication traffic of 23 IoT devices, excluding non-IoT device traffic. Table 5 presents the accuracy, precision, recall, and F1 score for all devices in our dataset. It can be observed that our approach achieved high accuracy for most devices, with identification precision and recall exceeding 99%. Some IoT devices like Nest Dropcam and Dropcam exhibited relatively lower accuracy due to insufficient data volume, making it challenging to extract an adequate amount of features for identification. However, overall, we achieved excellent performance, with a global accuracy of 99.48% for all devices. \begin{table}
\begin{tabular}{c c c} \hline \hline
**Initial Classifier** & **Second Classifier** & **Accuracy** \\ \hline \multirow{3}{*}{RF} & RF & 0.9935 \\  & DT & 0.9934 \\  & XGB & 0.9936 \\ \hline \multirow{3}{*}{DT} & RF & 0.9930 \\  & DT & 0.9940 \\  & XGB & 0.9935 \\ \hline \multirow{3}{*}{XGB} & RF & 0.9940 \\  & DT & 0.9942 \\ \cline{1-1}  & XGB & 0.9948 \\ \hline \end{tabular}
\end{table} 
Table 4: Score of second classification. To better demonstrate the effectiveness of our proposed method, Figure 3 illustrates the confusion matrix for each device. It can be observed that only a small number of devices are misclassified. Additionally, our method exhibits a low misclassification rate for devices with different purposes from the same vendor, indicating high identification accuracy even for similar devices from the same manufacturer. In this regard, our method outperforms previous works. Figure 3: Confusion Matrix of the experiment without non-IoT device. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Device Name** & **Precision** & **Recall** & **F1 Score** \\ \hline Smart Things & 0.9573 & 0.9739 & 0.9655 \\ Amazon Echo & 0.9993 & 0.9933 & 0.9963 \\ Netemto Welcome & 0.9917 & 0.9987 & 0.9952 \\ TP-link Cloud Cam & 0.9896 & 0.9917 & 0.9907 \\ Samsung SmartCam & 0.9987 & 0.9927 & 0.9957 \\ Dropcam & 0.7500 & 0.8400 & 0.7925 \\ Insteon Camera & 0.9977 & 0.9977 & 0.9977 \\ Withings Baby Mon & 0.9994 & 0.9988 & 0.9991 \\ Belkin Werno switch & 0.9963 & 0.9980 & 0.9972 \\ TP-Link Smart plug & 0.9787 & 0.9892 & 0.9840 \\ iHome & 0.9516 & 0.9219 & 0.9365 \\ Belkin werno sensor & 0.9977 & 0.9967 & 0.9972 \\ NEST-P smoke alarm & 0.9565 & 1.0000 & 0.9778 \\ Netatmo weather station & 0.9972 & 0.9986 & 0.9979 \\ Withings Smart scale & 0.9565 & 1.0000 & 0.9778 \\ Biipcare BPMeter & 1.0000 & 1.0000 & 1.0000 \\ Withings slp sensor & 0.9982 & 0.9963 & 0.9973 \\ Light LiFX Smart Bulb & 0.9900 & 1.0000 & 0.9950 \\ Triby Speaker & 0.9861 & 0.9467 & 0.9660 \\ PIX-STAR Photo-frame & 0.9976 & 0.9928 & 0.9952 \\ HP Printer & 0.9761 & 0.9784 & 0.9772 \\ Samsung Galaxy Tab & 0.9868 & 0.9943 & 0.9905 \\ Nest Dropcam & 0.6667 & 0.3636 & 0.4706 \\ Accuracy & & 0.9948 \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Performance evaluation results for the experiment without non-IoT device. To verify whether our proposed method can accurately identify IoT devices in an Internet environment, we conducted Experiment 2, which included traffic generated by non-IoT devices. In this experiment, 23 labels were assigned to 23 IoT devices, while all non-IoT devices were assigned a different label. Table 6 presents the overall device identification results of Experiment 2. From the table, it can be seen that with the inclusion of data generated by non-IoT devices, the overall identification precision, recall, F1 score, and accuracy decreased slightly compared to when there were no non-IoT devices. However, the identification accuracy still reached 98.5%, indicating that our proposed method can accurately identify IoT devices even in an Internet environment and distinguish them from non-IoT devices. Figure 4 shows the confusion matrix for this experiment.