# Ensemble and Deep Learning for Language-Independent Automatic Selection of Parallel Data
## 4 Results  
In this section, we will show more details about our experiments and their results. Most of our features were numerical and were normalized by using the Feature scaling method. Our experiment was run using the Weka machine learning workbench [44] for training and testing. For the classifier's evaluation, we used the Positive Predictive Value (Precision) and the Sensitivity (Recall) metrics. The former is the number of positive predictions divided by the total number of positive class values predicted, while the latter is the number of positive predictions divided by the total number of actual positive class values. To improve the experiments' accuracy, we applied extra filters. We noticed unequal values between the classes, class A being the minority class. We applied the SMOTE unsupervised filter to the minority [45]. In that way, the number of our instances increased from 2787 to 3150. Finally, we wanted to compare the performance between the 82 and the 96 feature dimensions, with and without using the SMOTE filter (Table 3). It is noteworthy that when we apply Random Forest with the new additional features, there is an increase between 1% and 4%, i.e., we have better accuracy results for all classes, which is quite promising. Additionally, after applying the SMOTE filter, the minority class A has a much better prediction accuracy than before. We do not observe significant differences in the other two classes. Furthermore, we wanted to see if this feature set would give good results in the English-Italian corpus. Therefore, we applied the same methodology to the English-Italian corpus, extracting 82 features and comparing the results with the 96 features (Table 4). We observed that the minority \begin{table}
\begin{tabular}{c c c c c} \hline \hline  & \multicolumn{2}{c}{**82 Features**} & \multicolumn{2}{c}{**96 Features**} \\ \hline
**Class** & **Precision** & **Recall** & **Precision** & **Recall** \\ \hline \multicolumn{5}{c}{**Classifier: RandomForest-2687 instances**} \\ \hline A & 49\% & 22\% & 53\% & 20\% \\ B & 46\% & 36\% & 47\% & 38\% \\ C & 50\% & 70\% & 52\% & 72\% \\ \hline \multicolumn{5}{c}{**Classifier: RandomForest\_SMOTE-3150 instances**} \\ \hline A & 77\% & 63\% & 77\% & 69\% \\ B & 44\% & 32\% & 48\% & 35\% \\ C & 50\% & 68\% & 53\% & 69\% \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Precision and Recall of English-Greek corpus. class also in this corpus is class A, so we applied again the SMOTE unsupervised filter to the minority class A [46] and our instances increased from 2745 to 3340. The results using the 96 feature set and the SMOTE filter are generally better. In particular, we observed that the extra features improve the accuracy of class A (precision increase between 25% and 29%, recall increase between 6% and 7%) and we have similar results for the other two classes. The SMOTE filter also helps the minority class A (precision increase from 29% to 71%, and recall increase from 7% to 63%). We decided to use the Deeplearning4j (DL4J) framework [43] using the Weka tool. DL4J is designed to handle large corpora. We used the previous 96 features and the SMOTE filter. Firstly, we used k fold CV, which is a reliable method for testing our models, and a value of k = 10 is very common in the field of machine learning [47]. Then, we hold out a certain percentage (70%) of the data for training and the remaining part is used for testing (Table 5). As we observe, using Deeplearning4j with 10 fold CV we have satisfactory results, compared to Random Forest (Table 3), especially for class B. When we use the 70% Percentage split we notice a small performance improvement for class A, but we also have good results for class B (Precision increase from 45% to 50% and Recall increase from 40% to 48%), in contrast to Radon Forest results. As can be seen in Table 6, we have tried the same architecture for the English-Italian corpus.