# A Deep Belief Network Classification Approach for Automatic Diacritization of Arabic Text
## 7 Conclusions and Future Work  
In this study, we present a novel diacritization approach based on a DBN. DBN uses a network structure composed of multiple RBMs, which is more effective for data modeling and feature extraction. The key idea of DBN is to use a greedy layer-wise training to extract deep hierarchical representation of input data, followed by fine-tuning to achieve competitive classification performance. This approach adds diacritic marks to undiacritized Arabic text using DBN. We have used two well-known benchmarks, which are Tashkeela and LDC ATB3. From the Tashkeela benchmark, we used ten books that are used in previous research by other researchers so that we can compare our results to theirs. Our results were evaluated in terms of DER and WER. Experimentation on the ATB3 yielded a DER of 2.21%, which made an improvement of 19% over state-of-the-art systems. In addition, the WER scored by our approach was 6.73% outperforming competitive systems with an improvement of 26%. On the Tashkeela benchmark, our system continues to achieve high accuracy. Our DBN-based approach scored a DER of 1.79% and 14% improvement over the best published systems. On the other hand, the WER recorded for our system was 4.63% and achieved an improvement of 20% over the best published system. With respect to the DER-1 and WER-1 metrics which imply ignoring the last letter diacritization mark, our system makes an improvement of DER and WER of 36% and 54%, respectively. This approach outperforms state-of-the-art approaches. It does not require the use of rule-based techniques nor morphological, syntactic or diacritic rules. More importantly, it does not require any post-processing. When compared to state-of-the-art approaches, the DBN model significantly improves error rates for both the ATB3 and Tashkeela datasets. The limitation of freely available resources including the availability of diacritized Arabic corpus creates a challenge and sometimes a barrier for researchers that work in this problem domain. Therefore, we presented in this work a new corpus of diacritized Arabic text collected from children's stories that consists of about 26K MSA words. For future work, we plan to incorporate a hybrid approach of RBMs, DBNs and LSTMs as a preprocessing approach to examine to what extent it may make any considerable improvement to the performance of the DBN. It is expected that when using a DBN for preprocessing and then employing it with an LSTM for training a deep learning model the accuracy will improve considerably. We will further use the principal component analysis technique for feature extraction as it may be useful for a better understanding of the performance of the implemented approach. Moreover, to gain a better understanding of the effectiveness of the implemented approach we will use the K-fold cross validation approach with evolutionary techniques to optimize the training time and performance. In addition to that, a comparative work with different types of neural networks could be researched. Furthermore, we seek to extend the range of the children's stories corpus to a couple more thousands or millions of words. \begin{table}
\begin{tabular}{c c c c c} \hline \hline \multirow{2}{*}{**Dataset**} & \multicolumn{2}{c}{**All Diacritics**} & \multicolumn{2}{c}{**Ignore Last**} \\ \cline{2-5}  & **DER** & **WER** & **DER-1** & **WER-1** \\ \hline Children stories & 2.4 & 6.57 & 1.33 & 2.83 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Diacritization results of our DBN model on the children’s stories dataset. **Author Contributions:** Conceptualization, W.A., M.A. and O.A. ; Methodology, W.A. ; Validation, W.A. ; Data curation, W.A. ; Writing--original draft preparation, W.A. ; Writing--review and editing, W.A., M.A and O.A. ; Supervision M.A. and O.A. All authors have read and agreed to the published version of the manuscript. **Funding:** This research received no external funding. **Institutional Review Board Statement:** Not applicable. **Informed Consent Statement:** Not applicable.