# Using Common Spatial Patterns to Select Relevant Pixels for Video Activity Recognition
## 5 Experimental Results
### Comparison  
Deciding whether our approach is better than the Histogram of Optical Flow is not trivial and can not be assumed by the obtained results due to the lack of clear differences. For that reason, a statistical test needs to be performed to determine if there is a difference between the approaches presented and if the test indicates that a difference exists, we could determine which model is better by the individual results. The statistical test that has been used is the _Friedman Test_. The Friedman Test is used when the data is dependant and it can be considered as an extension of Wilcoxon signed-rank test for more than two groups. The Friedman Test is computed this way: \begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & \multicolumn{2}{c}{**Variance**} & \multicolumn{2}{c}{**More Info**} & \multirow{2}{*}{**HOF**} \\ \cline{2-2} \cline{4-5}  & **Best** & **Mean Best** & **Best** & **Mean Best** & \\ \hline KNN & 0.7572 & 0.7355 & 0.7752 & 0.7503 & 0.754 \\ LDA & 0.7571 & 0.7273 & **0.85** & 0.8298 & 0.7305 \\ RF & **0.7902** & **0.7646** & **0.8567** & **0.8365** & **0.8435** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Summary results for each classifier. 1. Being \(\{x_{ij}\}_{m\times n}\) a data table with \(m\) rows and \(n\) columns, \(\{r_{ij}\}_{m\times n}\) is calculated where \(r_{ij}\) is the order of \(x_{ij}\) in every block \(i\). 2. Then, the statistic is calculated: \[Q=\frac{12m}{n(n+1)}\sum_{j=1}^{n}\left(\overline{r}_{j}-\frac{k+1}{2}\right) ^{2}\] (1) \[\overline{r}_{j}=\frac{1}{m}\sum_{i=1}^{m}r_{ij}.\] (2)
3. Finally, the p-value is defined this way, approximating the probability distribution of \(Q\) by a \(\chi^{2}\) distribution: \[P(\chi^{2}_{n-1}\geq Q).\] (3)  
When applying this test, the null hypothesis is that there are no differences between the tested groups. As in Equation (3), if the calculated probability is lower than the significance level, the null hypothesis is rejected, which indicates that at least 2 of the tested groups are significantly different from each other. In our approach, the hypotheses are defined this way:  
* H0: there is no difference between the tested models \(\Leftrightarrow\)\(p\)-value \(\geq\) 0.05
* H1: at least 2 of the tested models are different from each other \(\Leftrightarrow\)\(p\)-value \(<\) 0.05  
The dependant variable is formed by the accuracy values obtained with each model, the grouping variable is the definition of the models and the blocking variable the ranking of the models, from 1 to 15 in our case. After computing the Friedman Test, the obtained results indicate that there is evidence that at least two of the tested models are different (\(\chi^{2}(20)=123.68\), \(p\)-value \(<\) 2.2 \(\times\) 10\({}^{-16}\)), therefore the null hypothesis is rejected. Although it has been proven that a difference exists between the tested models, a post-hoc analysis is required to determine which groups are significantly different from each other. To do so, we have used the Nemenyi test [31]. In Table 6 the obtained \(p\)-values can be seen. As our objective is to compare our results to the ones achieved by the Histogram of Optical Flow method, the values that are presented in Table 6 are specifically these comparisons. The names of our approaches are defined by the regular expression of Equation (4). \[CSP(3|5|10)(\_var)?-((KNN)|(LDA)|(RF)) \tag{4}\]  
where 3, 5, 10 indicate the \(q\) value, _var indicates if just the variance is taken as feature and KNN, LDA or RF define the algorithm that is used to create the model. In the obtained results most of the \(p\)-value are not significant. However, there are some of them that indicate an evident difference between the models. The values in green are significant (<0.05) and the values in red are very significant (<0.01). There are a total of eleven pairs where a significant differences have been detected. Referring to the original values, we observe that HOF-RF beats the CSP approach five times, while CSP-RF beats HOF-LDA three times.