# Abnormal Activity Detection Using Pyroelectric Infrared Sensors
## 7 Experimental Evaluation
### Experimental Results  
In our experiments, because the output of the sensor node was a seven-dimensional data stream with continuous values, as shown in Figure 6, we employed HMMs with Gaussian observation density. The number of hidden states and the number of Gaussian models are determined by the log-likelihood of the training samples [3]. Specifically, we use HMMs with two Gaussians and eight states to profile the general human normal activity and each cluster of normal activities. For OSVM, the parameter \(\
u\) was set to 0.01. Experiments were conducted to compare the performance of all three of the algorithms. We randomly selected 240 normal samples for training. The other 80 normal samples and all the 80 abnormal samples were randomly mixed together for testing. Figure 7 shows the ROC curve with respect to the detection rate and the false alarm rate. We can see from the figure that SC+OSVM gives the best detection result. This is mainly because the self-turning spectral clustering can estimate the number of different types of activities automatically and then cluster the similar activities accordingly. The discriminative feature vectors are helpful to improve the accuracy of OSVM. \begin{table}
\begin{tabular}{l c c c} \hline \hline  & & \multicolumn{2}{c}{**Actual Activity**} \\ \cline{3-4}  & & **Abnormal** & **Normal** \\ \hline Predicted & Abnormal & True Positive (TP) & False Postivitive (FP) \\ Label & Normal & False Negative (FN) & True Negative (TN) \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Confusion matrix. We also conducted experiments to investigate the effect of varying the number of training samples on the performance of the three algorithms. In these experiments, we kept the amount of testing data unchanged and reduced the number of training samples. Figures 8 and 9 show the experimental results using 160 and 80 normal traces for training, respectively. We can see from the figure that, when the number of normal traces for training decreases, the performance of the three algorithms decreases  
Figure 6: The original output of the sensor node with seven PIR sensors for different activities: (**a**) Falling; (**b**) Sitting down; (**c**) Standing up; (**d**) Walking; (**e**) Jogging. The horizontal axis represents the sampling points (25 Hz) of the activity. Figure 7: Comparison of the detection rate and the false alarm rate _versus_ different numbers of training samples: training on 240 normal samples. correspondingly. The reason lies in two aspects. First, for HMMs, which are generative models [3; 9], sparse training samples affect the accuracy of parameter estimation of the models. Second, for OSVM, when the training samples are sparse, the calculated decision boundary may not exactly capture the characteristics of normal activities. Therefore, the abilities of three algorithms to distinguish normal and abnormal activities degrade. As shown in Figure 9, when there are only 80 normal samples for training, the performance of OneHMM and SC+iForest are comparable to that of SC+OSVM because all of them could not model the normal activities well with less training samples. When the training samples are less than 80, the performance of three algorithms will be worse and unsatisfactory. Again, as shown in Figure 8, when we have 160 normal samples for training, SC+OSVM still performs the best among the three algorithms. Figure 8: Comparison of the detection rate and the false alarm rate _versus_ different numbers of training samples: training on 160 normal samples. Figure 9: Comparison of the detection rate and the false alarm rate _versus_ different numbers of training samples: training on 80 normal samples. To explicitly compare the performance of the three algorithms, we computed the AUC values by calculating the area under the ROC curves depicted in Figures 7-9. The results are summarized in Table 2. We can see from the first column of the table that, when 240 normal samples are used for training, the AUC value for SC+OSVM, SC+iForest and OneHMM are 0.863, 0.375 and 0.354, respectively.