## 4 Experiments and Results  
All the experiments were run using an Apple Macintosh computer with a 2 GHz Quad-Core Intel Core i5 with 16 GB memory, and the implementation was carried out using the high-level technical computing language Python 3 version 3.7.6. For the classical machine learning classification algorithm, the Natural Language ToolKit (NLTK) was used, ref. [33] as well as the scikit-learn Python library [34]. For the CNN approach, the Keras library with TensorFlow as the back-end was used [35]. Two different experiments were run: the first as a multiway ADI problem, including four-way and three-way, and the second as a two-way ADI problem. For both experiments, the considered CNN and classical machine learning algorithms were trained on 80% of our dataset and tested on the rest of the dataset. For classical machine learning in both experiments, the models were trained by using the popular approach of a five-fold validation, in which the data were trained in four folds, and the remaining fold was used as the validation set. This method was repeated five times, and then the average result was recorded. However, for both experiments a pipeline was developed to determine the best parameters using a grid-search approach, which includes the following tested parameters:  
* Different values for the document frequency threshold (max_df), which required when calculating TF-IDF, including 0.5, 0.75, and 1.0. * Different values for the minimum document frequency threshold (min_df), including 1, 5 and 10. * Different n-grams combination for character model as shown in Table 2.  
For the CNN, the approach was tested with the data normalised in terms of removing tashkeel from the text in Models 2 and 4, and without the data normalised in Models 1 and 3. For the training, it was also tested by adding two drop layers in Models 3 and 4, and no drop layer in Models 1 and 2. Table 3 shows the combination of these tested parameters. Tables 4-6 represent the results of the considered algorithms for the four-way, three-way, and two-way ADI problems, respectively. The following subsections discuss the results along with their experiments in more detail. \begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Classifier** & **Precision** & **Recall** & **F-Measure** & **Accuracy** \\ \hline Model 1 & 41.7\% & 33.9\% & 28.2\% & 35.4\% \\ \hline Model 2 & 25.0\% & 32.2\% & 28.0\% & 33.2\% \\ \hline Model 3 & 26.1\% & 33.5\% & 28.7\% & 33.7\% \\ \hline Model 4 & 41.5\% & 33.3\% & 27.5\% & 34.5\% \\ \hline MultinomialNB & & & & \\ max\_df = 0.75, min\_df = 1, & 34.83\% & 34.42\% & 34.25\% & 35.8\% \\ ngram\_range = (1, 2) & & & & \\ \hline BernoulliN & & & & \\ max\_df = 0.75, min\_df = 1, & 38.68\% & 36.0\% & 36.2\% & 36.3\% \\ ngram\_range = (1, 2) & & & & \\ \hline SGDC & & & & \\ max\_df = 0.5, min\_df = 1, & 38.7\% & 38.8\% & 38.7\% & 38.9\% \\ ngram\_range = (1, 4) & & & & \\ \hline SVC & & & & \\ max\_df = 0.75, min\_df = 1, & 40.5\% & 39.6\% & 39.5\% & 40.6\% \\ ngram\_range = (1, 4) & & & & \\ \hline Logistic Regression & & & & \\ max\_df = 0.75, min\_df = 1, & 40.6\% & 40.1\% & 40.2\% & 40.9\% \\ ngram\_range = (1, 4) & & & & \\ \hline LinearSvc & & & & \\ max\_df = 0.75, min\_df = 1, & 39.9\% & 39.7\% & 39.8\% & 40.2\% \\ ngram\_range = (1, 4) & & & & \\ \hline NuSVC & & & & \\ max\_df = 0.75, min\_df = 1, & 40.6\% & 40.4\% & 40.4\% & 40.9\% \\ ngram\_range = (1, 4) & & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Results of four-way ADI problem. \begin{table}
\begin{tabular}{c c} \hline \hline
**Character n-grams Model** & **Type of Character n-grams** \\ \hline (1, 1) & Unigrams \\ (1, 2) & Unigrams + bigrams \\ (1, 3) & Unigrams + bigrams + trigrams \\ (1, 4) & Unigrams + bigrams + four-grams \\ (2, 2) & Bigrams \\ (2, 3) & Bigrams + trigrams \\ (2, 4) & Bigrams + trigrams + four-grams \\ (3, 3) & Trigrams \\ (3, 4) & Trigrams + four-grams \\ (4, 4) & Four-grams \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The different n-gram combinations that were tested in our experiments.