# PEiD: Precise and Real-Time LOS/NLOS Path Identification Based on Peak Energy Index Distribution
## 5 Experimental Evaluation
### Comparison with Previous Research  
We compare PEiD with LiFi [30], the first LOS/NLOS identification based on commodity WiFi we have ever known. Similar to our approach, LiFi explores CIR obtained from CSI (PhaseU [31] exploit phase difference of CSI, which clearly differs from PEiD and LiFi. As a result, we only consider the performance of LiFi), but it focuses on envelope distribution and neglects the time index of CIR. Two statistical features, the Rician-K factor and skewness, are employed to characterize the difference between LOS and NLOS paths. It is observed that the increased NLOS component of channel properties can cause the factors to grow as the shape of the envelope become asymmetric. Therefore, a threshold-based classifier is built to achieve LOS/NLOS identification.  
Figure 8: The identification accuracy with different number of packets.  
The experiments are conducted on the dataset described above. To demonstrate the effectiveness of our approach, we test the performance of two methods with different packet numbers of 250, 500, 1000, and 2000. Their corresponding sampling times are 500 ms, 1000 ms, 2000 ms, and 4000 ms, respectively. The experimental results are plotted in Figure 9.  
It shows that PEiD outperforms LiFi in each configuration. When only 250 packets are used (500 ms of sampling), LiFi achieves an accuracy of 75.1% in LOS cases and 80.6% in NLOS cases. In contrast, PEiD can improve that to 93.8% and 92.6%, respectively. Such a performance gap shrinks as the packet increases. The best performance of LiFi reaches 90% and 93.1% when 2000 packets are used (4000 ms of sampling). Nevertheless, PEiD still outperforms it by 7.4% and 4.3%.  
The above observation illustrates the improvements on both accuracy and packets number. We attribute this to two reasons. First, we go deeper into the characteristics of CIR. Both time-related and power-related features are adopted in our approach. The techniques of machine learning are also adopted to improve the accuracy, while LiFi only adopts a threshold-based method. Second, PEiD exploits a novel feature based on peak energy index distribution of CIR, which distinguishes LOS and NLOS paths more significantly than envelope features.  
To compare the best performance of LOS/NLOS identification, we also consider some more recent works based on WiFi. They include PhaseU [31], AmpN [32], RNN-based [33] methods, and CNN-based methods [37]. The approaches that utilize FTM-based ranging are not taken into account due to the differences in feature selection. The experimental results are shown in Figure 10.  
It can be seen that PEiD achieves the highest accuracy of identifying the LOS paths. PhaseU and AmpN have the following accuracy rates, which are 94.4% and 94.2% respec  
Figure 10: Comparison of best performance between PEiD and other previous works based on WiFi: (a) LOS case; (b) NLOS case.  
Figure 9: Comparison between PEiD and LiFi under different packet numbers and collection time: (a) LOS case; (b) NLOS case.  
tively. The performance of RNN- and CNN-based methods are relatively poor--below 92%. We attribute this to the limited time resolution of WiFi and ambient measurement noise introduced by commodity devices. Although deep learning models possess greater feature extraction capabilities than machine learning algorithms, they can still be disturbed by irrelevant information in the raw images of CSI and CIR. In terms of identifying NLOS paths, the performance of the five schemes is comparable, with the exception of AmpN, which achieves an accuracy of 97.6%. We attribute this to the implementation of a neural network and combination of several designed features such as kurtosis and skewness. However, their default packet number is 500, which is slightly higher than PEiD.