# A Deep Belief Network Classification Approach for Automatic Diacritization of Arabic Text
## 6 Experiments and Results
### Comparisons with Literature  
In this subsection, we present our results on ATB3 and Tashkeela datasets by comparing them to the best published systems. To the best of our knowledge, [PERSON] and Abdel-Karim [38], is the system that has the best reported accuracy on the same datasets used in our work. Table 5 provides a summary of our results along with the best published systems in terms of DER, WER, DER-1 and WER-1. To the best of our knowledge, [PERSON] et al. bidirectional LSTM based systems in [33; 38] has the best reported accuracy's on the ATB3 and Taskeela datasets. In terms of the ATB3 dataset our DBN model provides 19% DER improvement and 26% WER improvement compared to [PERSON]'s et al. system in [33]. And a 10% DER improvement and 17% WER compared to the system proposed in [38]. Looking at Table 5, it can be observed that the DER is also decreasing in this work compared to the best published systems presented in the table. For the Tashkeela dataset, our system provides 14% DER improvement and 20% WER improvement compared to [PERSON]'s system in [33]. From Table 5, it is recognized that, over the past several years, diacritization performance has been improved overall and our system has greatly improved the performance of its predecessor while continuing its trend. In 2015, [PERSON] et al. (2015) [33; 38] reported the best accuracy on ATB3 and Tashkeela datasets. For these two datasets, we report a higher improvement on error rates as mentioned earlier. Recently, over the past 2 years, [PERSON] et al. (2019) [36], [PERSON] et al. (2019), and [PERSON] and [PERSON] (2020) [19] have reported interesting results on the ATB3 and Tashkeela datasets. Still, our results outperform their reported results in terms of DER, WER, DER-1 and WER-1. Our system provides 19% DER improvement compared to [PERSON] el al. (2015), 18% WER improvement compared to [PERSON] et al. (2019), and 76% DER improvement compared to [PERSON] and [PERSON] (2020) regarding the ATB3 dataset. Figures 9 and 10 present the results of using our DBN model and the Bi-LSTM proposed in [33] on the Tashkeela corpus. Apparently, the DER and the WER of the DBN model outperform the competing model of Bi-LSTM. The reason DBN works so well is related to the fact that the pre-training phase in an unsupervised fashion of DBN improves \begin{table}
\begin{tabular}{l c c c c c} \hline \hline \multirow{2}{*}{**System**} & \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c}{**All Diacritics**} & \multicolumn{2}{c}{**Ignore Last**} \\ \cline{3-6}  & & **DER** & **WER** & **DER-1** & **WER-1** \\ \hline [PERSON] \& [PERSON] (2005) [28] & ATB3 & 12.8 & 23.6 & 6.5 & 7.3 \\ [PERSON] et al. (2006) [14] & ATB3 & 5.5 & 18.0 & 2.5 & 7.9 \\ [PERSON] \& [PERSON] (2007) [84] & ATB3 & 4.8 & 14.9 & 2.2 & 5.5 \\ [PERSON] et al. (2008) [85] & ATB3 & 4.3 & 19.9 & 1.7 & 6.8 \\ [PERSON] et al.(2010) [86] & ATB3 & 13.8 & 46.8 & 9.3 & 26.0 \\ [PERSON] et al. (2011) [21] & ATB3 & 3.8 & 12.5 & 1.2 & 3.1 \\ [PERSON] et al. (2013) [4] & ATB3 & 3.6 & 11.4 & 1.6 & 4.4 \\ [PERSON] et al. (2015) [33] & ATB3 & 2.72 & 9.07 & 1.38 & 4.34 \\ [PERSON] et al. (2019) [39] & ATB3 & 2.8 & 8.2 & - & - \\ [PERSON] \& [PERSON] (2019) [38] & ATB3 & 2.46 & 8.12 & 1.24 & 3.81 \\ [PERSON] \& [PERSON] (2020) [19] & ATB3 & 9.32 & 28.51 & 6.37 & 12.85 \\
**This work** & ATB3 & **2.21** & **6.73** & **1.2** & **2.89** \\ \hline [PERSON] (2015) [33] & [PERSON] & 2.09 & 5.82 & 1.28 & 3.54 \\ [PERSON] (2017) [87] & [PERSON] & 3.73 & 11.19 & 2.88 & 6.53 \\ [PERSON] \& [PERSON] (2019) [38] & [PERSON] & 1.97 & 5.13 & 1.22 & 3.13 \\ [PERSON] et al. (2019) [36] & [PERSON] & 2.60 & 7.69 & 2.11 & 4.57 \\ [PERSON] \& [PERSON] (2020) [19] & [PERSON] & 3.39 & 9.94 & 2.61 & 5.83 \\
**This work** & [PERSON] & **1.79** & **4.63** & **1.15** & **2.13** \\ \hline \hline \end{tabular}
\end{table} 
Table 5: Diacritization results of previous systems and our system. model performance by avoiding over-fitting and enhancing the model generalization. Furthermore, a DBN exploits hidden features and their context layer by layer.