# A Deep Belief Network Classification Approach for Automatic Diacritization of Arabic Text
## 2 Literature Review
### Hybrid Systems  
On the other hand, hybrid techniques have gained a lot of interest by NLP researchers seeking to improve the diacritization problem for Arabic language. Hybrid techniques combine the rule-based techniques with statistical techniques [12]. For instance, [PERSON] et al. [13], proposed a hybrid approach to build an Arabic diacriitizer. The diacritizer uses a lexicon and a Support Vector Machine (SVM). Experimental results showed that statistically based methods are promising in addressing the ambiguity resolution problem in Arabic language diacritization. [PERSON] et al. [29], used a Viterbi decoder and a support vector machine to guess the diacritics of words and their case endings. Their experimentation results in a 3.29% and 12.77% WER without and with case endings. [PERSON] et al. [30] proposed a hybrid system that uses a deep LSTM network and Alkhalil Morpho Sys 2 analyzer [31]. The analyzer associates each word with its morphological-syntactic features. The purpose of using the analyzer is to take the word out of its context and provide all possible diacritization forms for it along with its POS. At the same time, the LSTM is trained to diacritize input text. Next, undiacritized words by the LSTM are selected with their context and further associated with their corresponding diacritized forms and POS outputted by Alkhalil Morpho Sys 2. [PERSON] et al. [32] used a deep learning framework with the Confused Sub-set Resolution (CSR) method to improve the classification accuracy of diacritics and Arabic PoS tagging using deep neural networks. The authors report a syntactic WER of 9.9% on the LDC ATB3 corpus. [PERSON] et al. [33] proposed a sequence transcription approach for the automatic diacritization of Arabic text using RNN. The authors used a bidirectional LSTM network that builds high-level linguistic abstractions of text and exploits long range context. This approach does not require any lexical, morphological, or syntactical analysis to be performed on the data before being processed by the RNN. For experimentation, the authors used LDC ATB3, the simple version of the holy Quran and ten books drawn from the Tashkeela dataset. Results achieved were as follows: for Tashkeela books the DER and WER were 2.09% and 5.82%, respectively. For LDC ATB3 DER and WER were 2.72% and 9.07%, respectively. [PERSON] and [PERSON] [34] studied the effect of combining acoustic information with morphological and contextual constraints to automatically diacritize letters with missing diacritics in a text for use in acoustic model training for ASR systems. Experimental results achieved a DER and WER of 11.5% and 27.3%, respectively. [PERSON] et al. [14] proposed a maximum entropy based approach that exploits the lexical, segment-based and part-of-speech features to build a diacritization model. The authors formulate the problem of diacritics restoration as a sequence classification problem such that every character in a sequence is assigned a diacritic mark. The model was trained and tested using LDC ATB3. The approach achieves DER and WER of 5.5% and 18%, respectively. [PERSON] and [PERSON] [35] extend the use of their morphological analyzer to generate all the possible analysis of a word. An SVM was used to help narrow down the list of generated words and a language model was constructed to select the best solution from the list. This approach was trained and tested using LDC ATB3 dataset. The experiments achieve DER and WER of 4.8% and 14.9%, respectively. [PERSON] et al. [21] used a hybrid system of language factorizing and un-factorizing textual features. In order to infer the most likely diacritization and phonetic transcription of the input text, different diacritization system rely on factorizing text using morphological analysis and case diacritics and then statistically recapitulate the most likely sequence using lattice search. In this work, the most likely sequence is selected using the n-gram probability estimation and A* lattice search.