# Error Detection for Arabic Text Using Neural Sequence Labeling
## 6 Discussion and Analysis  
For error detection in Arabic, [PERSON] et al. [26] approached the task as a sequence labeling problem by training an SVM classifier using morphological and lexical features, such as lemmas and POS tags. The model labels the incorrect words in the text with a tag before passing them on for correction. Recall of the error detection model was 80%, whereas precision was 34.5%. Both scores are considered low in comparison to the results of the neural network models in Table 5.  
On the other hand, the first notable application of neural sequence labeling to the task of error detection was done by [PERSON] and [PERSON] [1]. They performed neural sequence labeling on English corpora and reported the results using precision, recall, and F\({}_{0.5}\). They achieved the best result on BiLSTM with F\({}_{0.5}\) = 41.1%, which was improved to F\({}_{0.5}\) = 41.88% when integrating the character-level component in [24]. Similarly, [PERSON] et al. [47] treated error finding as a neural sequence labeling task with the character-level component, while using a small set of features that were easily computed and needed no linguistic input. Their detection model achieved an F\({}_{0.5}\) score of 51.14%. On the other hand, [PERSON] et al. [48] utilized Conditional Random Field for sequence labeling to automate Chinese grammar error detection without the use of any external features. [PERSON] et al. [49], however, did incorporate some external features to their sequence labeling model for English language error detection and achieved 82.52% in F\({}_{0.5}\). Additionally, [PERSON] et al. [50] used BiLSTM sequence labeling for detecting errors but the model used a forward Language Model to predict the following token and a backward Language Model to predict the preceding token achieving F\({}_{0.5}\) = 63.57%. All of these works utilized multiple large corpora and more complex models for training and testing. However, we believe that this is the first application of neural sequence labeling on Arabic text. Considering the small size of training corpus, we had expected that precision would be easier to achieve than recall. On the contrary, the results revealed a noticeable trend throughout all experiments: recall was mostly easier to achieve than precision. This is most probably due to the shortness of the sentences in the training set. In addition, we found that smaller batches performed better than larger batches on our small dataset, which agrees with the work in [52]. Additionally, configurations using smaller batches were able to generalize better on new data than larger batches. This was apparent in our results for SimpleRNN and LSTM. Table 9 presents four examples and their labels in five cases: gold standard and the baseline commercial Arabic grammar checker, as well as the testing results of SimpleRNN, LSTM, and BiLSTM models. Additionally, the specific type of error for each example is shown, as was mentioned in [32]. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Feature** & **Precision** & **Recall** & **F\({}_{0.5}\)** \\ \hline Baseline & 75.6 & 91.96 & 78.39 \\ SimpleRNN & 84.25 & 67.72 & 80.33 \\ LSTM & 84.81 & 68.04 & 80.83 \\ BiLSTM & 82.97 & 41.61 & 69.21 \\ \hline \hline \end{tabular}
\end{table} 
Table 8: QALB Test Results. Overall, when observing the predictions on test sets, BiLSTM had the best performance; it was better at learning sentence structure in general since LSTMs are capable of handling long-term dependencies by applying a linear update to the internal cell representation, unlike SimpleRNN. More importantly, it is because BiLSTMs are able to incorporate context on both sides of every token. While SimpleRNN and LSTM also performed well, they were able to correctly learn and identify that correct words were correct. This is due to the fact that there were far more correct words in the training corpus than incorrect words. This might be an indication that a more balanced corpus would have enabled models to learn both correct and incorrect words equally.