# Two-Step Classification with SVD Preprocessing of Distributed Massive Datasets in Apache Spark
## 5 Evaluation
### PAMAP Dataset  
The second dataset that we examined was PAMAP in terms of multinomial classification. We performed a series of classification analyses, as with the Higgs dataset, with the exception of using one-vs-rest classifier instead of gradient boosted trees classifier, as the latter does not support multinomial classification.  
In Table 11, the results of the decision tree classifier are presented. It can be immediately seen that the overall scores were higher than the ones examined in the Higgs dataset. In the same context as before, as the dataset got larger, the better the classifier results got and the more complex the resulting tree. Worth noting is the fact that in this case, the height of the tree had the highest possible value in all cases, but its complexity increased since nodes kept increasing in number. In addition, the computational time needed, as expected, kept increasing along with the size of the dataset, but not in a linear manner; when the size of the dataset increased by 40 times (from 5K to 200K rows), the time needed nearly doubled. Comparing the proposed two-step classifier with the one-step classifier, we observe that the former performed almost the same as the formal classification method in terms of the \(F1\) metric, with the resulting tree being less complex. In addition, it required about 10-15% less time to produce the classification model.  
In following, in Table 12, the results of the random forest classifier are presented. Again, as the dataset grew in size the results got better and the resulting model got more complex in terms of the total number of nodes, although we should mention that the values of the metrics were high in almost all the cases. Both one- and two-step classification methods performed almost the same in terms of the \(F1\) metric (about 0.5% less accurate results), with the model of the two-step classifier being a bit simpler. In addition, in all but the 200K rows case, two-step model required about 5% less time to be computed (in the case of 200K rows required 0.5% extra time). As the dataset size grew bigger, so did the three metrics, except the case of the 10K rows. In overall, this classifier produced slightly lower metrics compared with the decision tree classifier. Furthermore, the training time of this classifier was higher enough than the decision tree one. When the size of the dataset increased by 40 times, it needed almost 3 times more for the classifier to be trained.  
The next classifier is the multinomial logistic regression, which is depicted in Table 13. Again, as in the Higgs dataset, there was no parameter optimization performed. With few exceptions, we can argue that as the dataset size increased, so did the metrics, although some fluctuation existed. The metrics' scores was lower that the decision trees as well as random forest classifier. As for the scaling of time regarding the size of the dataset, for a 20 times bigger dataset, from 10K to 200K rows, the classifier's training time increased by 7 times. When comparing the two classification methods, the results were almost the same, with a 4% drop in \(F1\) metric in the case of two-step classifier, which was also about 10% faster.  
Table 14 depicts the results of the multilayer perceptron classifier. Concretely, the results got better as the dataset grew larger. Both one- and two-step classification methods provided almost the same results, both in \(F1\) metric and in time needed to produce the model.  
Finally, the last classifier examined is one-vs-all in Table 15. As the base classifier, we have chosen logistic regression. This classifier performance was rather poor, both in time needed to train the classifier and in the metrics' scores. Examining again the \(F1\) metric, it achieved 77% while the logistic regression achieved about 80%. Considering time scaling, with regard to the dataset size, for 20 times bigger dataset, the classifier's training time was increased by 3 times.