# Two-Step Classification with SVD Preprocessing of Distributed Massive Datasets in Apache Spark
## 3 Proposed Method  
In this section, we will discuss the classification algorithms utilized in our experiments as well as the proposed methodology. The focus lied in the relationships between the dataset size and the computation time needed to perform classification as well as between the dataset size and the metrics evolved. The results of our work are presented in Tables 1-14. Recall, precision, and the \(F1\) metric were used as the evaluation metrics of the different algorithms. In addition, the time needed to train each classifier is presented, as well as information about the analysis cases. However, before presenting the various classifiers used in the experiments, the important characteristics of 10V data will be discussed. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dataset Size** & **Precision** & **Recall** & **\(F1\) Metric** & **Time** & **Depth** & **Number of Nodes** \\ \hline \multicolumn{8}{c}{one-step classifier} \\ \hline
5.000 & 0.633 & 0.631 & 0.632 & 0:12:19 & 4 & 31 \\ \hline
10.000 & 0.665 & 0.665 & 0.665 & 0:14:09 & 6 & 119 \\ \hline
25.000 & 0.662 & 0.662 & 0.662 & 0:18:53 & 7 & 231 \\ \hline
50.000 & 0.682 & 0.683 & 0.682 & 0:39:13 & 8 & 469 \\ \hline
75.000 & 0.688 & 0.687 & 0.687 & 0:50:17 & 7 & 251 \\ \hline
100.000 & 0.688 & 0.688 & 0.688 & 1:03:39 & 9 & 961 \\ \hline
200.000 & 0.692 & 0.692 & 0.692 & 1:42:47 & 9 & 1001 \\ \hline \multicolumn{8}{c}{two-step classifier} \\ \hline
5.000 & 0.618 & 0.618 & 0.618 & 0:11:43 & 2 & 7 \\ \hline
10.000 & 0.657 & 0.657 & 0.657 & 0:14:26 & 7 & 205 \\ \hline
25.000 & 0.638 & 0.639 & 0.638 & 0:18:13 & 7 & 237 \\ \hline
50.000 & 0.657 & 0.657 & 0.657 & 0:37:21 & 6 & 119 \\ \hline
75.000 & 0.661 & 0.662 & 0.661 & 0:48:17 & 7 & 255 \\ \hline
100.000 & 0.659 & 0.659 & 0.659 & 1:04:29 & 10 & 1667 \\ \hline
200.000 & 0.663 & 0.663 & 0.663 & 1:39:18 & 9 & 963 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Decision tree. \begin{table}
\begin{tabular}{c c c c c c c} \hline \hline
**Dataset Size** & **Precision** & **Recall** & \(F1\) **Metric** & **Time** & \
\begin{tabular}{c} **Number of** \\ **Trees** \\ \end{tabular} &
\begin{tabular}{c} **Number of** \\ **Nodes** \\ \end{tabular} \\ \hline \multicolumn{6}{c}{one-step classifier} \\ \hline
5.000 & 0.683 & 0.682 & 0.682 & 0:18:04 & 58 & 32,662 \\ \hline
10.000 & 0.698 & 0.697 & 0.697 & 0:24:52 & 52 & 40,712 \\ \hline
25.000 & 0.699 & 0.700 & 0.699 & 0:38:09 & 46 & 50,044 \\ \hline
50.000 & 0.704 & 0.704 & 0.704 & 1:32:04 & 38 & 48,974 \\ \hline
75.000 & 0.713 & 0.713 & 0.713 & 2:02:18 & 54 & 75,846 \\ \hline
100.000 & 0.713 & 0.713 & 0.713 & 2:28:50 & 44 & 65,826 \\ \hline
200.000 & 0.710 & 0.711 & 0.710 & 4:10:07 & 54 & 90,658 \\ \hline \multicolumn{6}{c}{two-step classifier} \\ \hline
5.000 & 0.661 & 0.661 & 0.661 & 0:17:04 & 44 & 22,474 \\ \hline
10.000 & 0.686 & 0.684 & 0.685 & 0:22:46 & 42 & 29,762 \\ \hline
25.000 & 0.671 & 0.671 & 0.671 & 0:39:18 & 60 & 61,290 \\ \hline
50.000 & 0.679 & 0.678 & 0.679 & 1:35:13 & 52 & 61,358 \\ \hline
75.000 & 0.681 & 0.681 & 0.681 & 2:20:12 & 54 & 72,610 \\ \hline
100.000 & 0.683 & 0.684 & 0.684 & 2:47:25 & 54 & 78,582 \\ \hline
200.000 & 0.684 & 0.684 & 0.684 & 4:44:11 & 52 & 82,542 \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Random forest.