# TwSense: Highly Robust Through-the-Wall Human Detection Method Based on COTS Wi-Fi Device
## 5 Results and Evaluation
### System Performance Evaluation  
In order to prove that the combined algorithm of Kmeans+HOG+G-SVM (KHG) used in this paper was better than the single algorithm for classification and to verify the reliability and high performance of the method proposed in this experiment, we used the ROC curve as an evaluation criterion, thus analyzing the effectiveness of the KHG classification method. The results are shown in Figure 14. With the simultaneous use of  
Figure 12: The experimental effect of different equipment spacing. Figure 13: The experimental results of the different locations of the personnel: (**a**) Error ratios at near positions from the device; (**b**) Error ratios at middle positions from the device; (**c**) Error ratios at far positions from the device. Doppler shift, we compared the two classification methods without using the K-means clustering method and only using the G-SVM algorithm. From the comparison results, it can be seen that, when the actual positive rate (TPR) reached 0.8, the KHG method proposed in this paper had a false positive rate (FPR) of 0.1, which was the best performance, and the FPR of the method without using the K-means clustering method reached 0.4, which was slightly lower than that of KHG. The FPR of only using the G-SVM method reached 0.64, which was the worst performance. From the comparison results, we see that the KHG method in this paper had the best performance, and the method using only the G-SVM method without HOG feature extraction had the worst performance, indicating that the combination algorithm was better than a single algorithm for classification. To evaluate the classification performance of the TwSense system, this paper was compared with the typical detection systems DeMan [23], R-TTWD [24], and TWMD [25], three methods related to human detection under the use of four wall materials conditions. The classification method we used was the combined algorithm (HKG) of HOG+K-means+G-SVM, and then the ROC curve was used as the evaluation standard to analyze the effect of the KHG classification method. The results are shown in Table 3.  
Table 3 shows the experimental results of the average recognition accuracy of human detection using the four methods. It can be seen from the table that in the glass wall experiment scene, our system used glass material and had a simple structure. Thus, the attenuation of the Wi-Fi signal after passing through the glass was small, and the classification accuracy was the highest. Compared with the glass wall, the thickness of the door had been improved, the structure was slightly complicated, and the classification effect was slightly worse. The structure of the gypsum wall and concrete wall was more complex, resulting in excellent attenuation of the signal, and the classification accuracy was the lowest but still more than  
Figure 14: Comparison of ROC curves of different classification algorithms. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline \multirow{2}{*}{**Experimental Scene**} & \multirow{2}{*}{**System**} & \multirow{2}{*}{**Method**} & \multicolumn{4}{c}{**Different Material Recognition Accuracy (\%)**} \\ \cline{3-6}  & & & **Concrete** & **Wall** & **Gypsum** & **Wooden** & **Glass** \\ \hline \multirow{4}{*}{
\begin{tabular}{c} bedroom, \\ meeting room, \\ hall \\ \end{tabular} } & TwSense & OR-PCA+HOG+Doppler+G-SVM & 93.6 & 94.8 & 96.5 & 97.3 \\ \cline{2-6}  & TWMD & BP Network & 90 & 91 & 93 & 95.5 \\ \cline{1-1} \cline{2-6}  & R-TTWD & PCA+SVM & 87 & 89 & 91.5 & 93 \\ \cline{1-1} \cline{2-6}  & [PERSON] & [PERSON] model+Parameter estimation & 50 & 71 & 79 & 85 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Comparison of different classification systems for different materials. 90%. However, DeMan's detection effect in several wall-penetrating scenes was not ideal, and its accuracy was about 50%.