# Arabic Document Classification: Performance Investigation of Preprocessing and Representation Techniques

Authors: Abdullah Y. Muaad; Hanumanthappa Jayappa Davanagere; D.S. Guru; J.V. Bibal Benifa; Channabasava Chola; Hussain AlSalman; Abdu H. Gumaei; Mugahed A. Al-antari
Journal: Mathematical Problems in Engineering
 (2022)

---

## Abstract

With the increasing number of online social posts, review comments, and digital documentations, the Arabic text classification (ATC) task has been hugely required for many spontaneous natural language processing (NLP) applications, especially within the coronavirus pandemics. The variations in the meaning of the same Arabic words could directly affect the performance of any AI-based framework. This work aims to identify the effectiveness of machine learning (ML) algorithms through preprocessing and representation techniques. This effectiveness is measured via different AI-based classification techniques. Basically, the ATC process is influenced by several factors such as stemming in preprocessing, method of feature extraction and selection, nature of datasets, and classification algorithm. To improve the overall classification performance, preprocessing techniques are mainly used to convert each Arabic word into its root and decrease the representation dimension among the datasets. Feature extraction and selection always play crucial roles to represent the Arabic text in a meaningful way and improve the classification accuracy rate. The selected classifiers in this study are performed based on various feature selection algorithms. The overall classification evaluation results are compared using different classifiers such as multinomial Naive Bayes (MNB), Bernoulli Naive Bayes (BNB), Stochastic Gradient Descent (SGD), Support Vector Classifier (SVC), Logistic Regression (LR), and Linear SVC. All of these AI classifiers are evaluated using five balanced and unbalanced benchmark datasets: BBC Arabic corpus, CNN Arabic corpus, Open-Source Arabic corpus (OSAc), ArCovidVac, and AlKhaleej. The evaluation results show that the classification performance strongly depends on the preprocessing technique, representation methods and classification technique, and the nature of datasets used. For the considered benchmark datasets, the linear SVC has outperformed other classifiers overall when prominent features are selected.


---

## Body

## 1. Introduction

The concept of text mining (knowledge discovery) has gained much attention in recent years to extract meaningful information from the text [1]. It is a more recent discipline that emerged from the fields of linear algebra, statistics, ML, natural language processing (NLP), and linear algebra [2]. In addition, text mining has been categorized as a big data problem because of the volume, variety, veracity, and velocity of data [3]. Text mining techniques can mine/discover information from huge datasets, and on the other hand, they allow flexibility and adaptability. This analysis helps in many domains such as classification of text data in many real scenarios such as classification of articles, e-mail spam classification, text translation, and sentiment analysis. Almost 447 million people speak Arabic as a native language, and it is declared as the official language for 22 countries across the globe. Moreover, it is the mother script for other languages such as Persian and Urdu. The Arabic language comprises 29 letters/characters, and typically, the writing for Arabic is opposite to English from the right side to the left side. One of the main characteristics of the Arabic text is that the letters are position-dependent with different forms and shapes [4]. In comparison to other languages, Arabic is a root-based language, and the majority of Arabic words consist of vertical stacking of letters. Besides that, Arabic has three different forms known as classical, standard, and dialectal. Therefore, working on Arabic text mining is a difficult task as compared to other languages. In addition, the Arabic language is more challenging to learn because of the dialects, phonology, orthography, and morphology characteristics [5]. Further, only limited research works have been done in ATC in comparison with the English language [6].Data preprocessing is the first practice of preparing and cleaning the text for further processing. It is the first phase in the text classification pipeline before representation because the text is often unstructured. The unstructured text does not have a predefined data model, and it is not suitable for further processing [7]. Therefore, specific algorithms for preprocessing are required to reduce the text size, eliminate noise from the text, and extract useful patterns. Mainly, preprocessing also includes text cleaning, space, and stop words removal, stemming, and handling of negation words [8]. Preprocessing text plays the main role in enhancing the accuracy of Arabic documents in many applications [9]. Finally, most of the accuracy results are affected by preprocessing difficulties [10].The Arabic text representation and feature extraction are a seriously important step in the text mining and classification process to extract the most prominent features and select the optimal subset. It is a process of representing text or documents as vectors and transforming from unstructured to structured equivalent such that the ML algorithms can understand it precisely [11]. Several feature extraction techniques were applied such as BoW, TF-IDF [12], Term-Class Relevance TCR, Term Class Weight-Inverse Class Frequency (TCW-ICF) [3], phrases representation [13], and a symbolic used for feature representation [14]. Most of the researches use TF-IDF or BoW, which have some major feature dimensionality reduction (FDR) drawbacks like missing the order of the words, and they also ignore semantics of the words so that different sentences can have the same representation, as long as the same words are used. TF-IDF and BoW techniques are usually used as representation models but still, they have limitations such as sparse matrices. Therefore, the way of factorization and FDR is inevitable to finalize the relatively small number of features for every document [3]. FDR technique can be done in the preprocessing step such as stop word removal. Feature Selection (FS) is done by any of the three methods known as embedded, wrapper, and filtering techniques. Feature transformation is one of the FDR techniques that belong to the category of multivariate analysis. Principal Component Analysis (PCA), Nonnegative Matrix factorization (NMF), Random Projection (RP), and Linear Discriminant Analysis (LDA) are used for unsupervised feature extraction process [15]. Based on the observations, these methods can be applied effectively for decreasing the size of the representation matrix in Arabic text documents. Once the representation of text is created through optimal feature extraction and selection methods, then the classifier algorithm has to be trained. Subsequently, the trained model should classify the documents into target classes with improved accuracy, based on the knowledge gained at the training phase [3]. Text categorization is a significant task that detects whether a piece of writing text belongs to any of the predefined sets of classes [16]. In addition, text categorization remains one of the most difficult computational tasks in the ML field [17]. There are several applications of text mining, such as spam and junk e-mail filtering, Web page classification, and sentiment analysis. Finally, compared to English, a few researches have been done for ATC by NB [18], SVM [19], ANN [20], and DT [21].The goal of this study is to investigate the impact of preprocessing and representation of various techniques on the Arabic text classification. In addition to these feature extraction and selection methods, six AI-based classifiers are adopted and employed for the Arabic text classification task. Consequently, this research enables the developers in the domain to select a robust AI-based technique for the robust ATC applications. The main contributions of this study are summarized as follows:(i)
A comprehensive study is performed to investigate the effectiveness of Arabic text classification workflow procedure: preprocessing, representation, and classification.(ii)
Several AI-based techniques are adopted and tested along with different preprocessing algorithms to identify their performance with respect to ATC.(iii)
To show the effectiveness of preprocessing and representation techniques, all AI-based classifiers are evaluated via five Arabic text benchmark datasets, namely, CNN Arabic (CNN), BBC Arabic (BBC), Open-Source Arabic Corpus (OSAc), ArCovidVac, and AlKhaleej.(iv)
We show the effectiveness of balanced and unbalanced Arabic datasets in multiclass classification scenarios.The rest of the article is structured as follows. In Section2, the related previous works are summarized regarding English and Arabic language text classification. In Section 3, the proposed methodology and the mathematical model are discussed. Then, the detailed experimental analysis and the outcomes are described in Section 4. The key findings and the inferences are detailed in Section 5 with appropriate concluding remarks.


## 2. Related Work

In English and other languages, many different approaches are available to solve the problem of text classification. However, only a sparse amount of research works has reported the key issues involved in the Arabic text classification (ATC). In recent days, the revolution created around the globe by social media and search engines has stressed the need for ATC in terms of Arabic text mining, sentimental analysis, and document classification. Here, a survey of related works is addressed to characterize the challenges involved in the pipeline of text classification as stated in the introduction part of the article. The ATC process consists of four different phases, namely, prepossessing, representation, FDR, and classification.


### 2.1. Text Preprocessing

There is a collection of preprocessing methods such as stop words, which are used to eliminate unwanted words with high weights. For example, a, an, and the do not have significant meaning; therefore, deleting these words will not affect the model but will decrease the dimension like the authors in [1] have done for English. In contrast, the same idea has been used for the removal of Arabic text stop words such as [11] [2] [22] لاجل ، مع، لى. Stemming is the method of converting a word into its corresponding root or stem. Thus, it is seen as an important preprocessing step before handling any NLP tasks. Stemming is a very important process for Arabic text mining. Primarily, there are three types of stemmers known as root-based (Khoja), light-based (Light 10), and statistical-based (N-Grams) stemmers, and occasionally they will be further subdivided into many other types. It is important to apply the steaming process to the text data after tokenization. In Arabic text, Yousif et al. (2015) evaluated the effect of stemming algorithm along with NB classifier and concluded that the root extractor exhibits the best performance [23]. Previously, Rehab Duwairi et al. (2013) investigated the effects of stemming strategies on Arabic document classification [24]. They used a stemming algorithm called light stemming, and word clusters were investigated, and they conclude that the light stemming method improves the result in terms of accuracy. Mamoun and Ahmed (2016) proposed a new light stemmer for Arabic called P-stemmer. They modified the version of Larkey’s light stemmers. The validation of the stemmer has shown significantly increasing accuracy during the classification process when NB, SVM, and RF classifiers are applied. They conclude that the SVM classifier performs better than other classifiers [4]. Shargabi et al. (2011) used many classification algorithms such as NB, KNN, and J48 to build the classification models, and they conclude that the Light stemmer was better compared to Khoja stemmer [25]. Salam et al. (2016) presented the ATC system by studying the effect of normalization and stemming techniques such as ISRI and Tashaphyne stemmer. Finally, they concluded that the normalization was best for the outcome, but the stemmers could not produce the required accuracy [26]. Oraby et al. (2013) presented Arabic sentiment analysis, and they addressed the effect of the stemming algorithm. They showed the results of accuracy equal to 93.2%, 92.6%, and 92.2% for Tashaphyne, ISRI, and Khoja stemmers, respectively [27]. Qusay Walid et al. (2005) processed a document clustering process by studying the influence of ISRI stemmer [28]. They have concluded that the method of ISRI outperforms nonstemming methods. Al haj et al. (2018) introduced responsibilities of the stemming technique for Arabic text mining, which have been explored, and it has been found that stemming minimizes the computational difficulties for the classification process [11].


### 2.2. Text Representation

The text cannot be processed in the native form, because it is unstructured. The term unstructured is a very subjective property of text data, so there are many representation schemes available to address this problem with its own set of characteristics. The representation schemes are broadly classified into two types: the first is vector space-based representation model (VSM), and the second is a graph-based representation (GBR) [3].The VSM is a representation method that has been used because it provides a matrix representation for the text data. According to Sebastiani et al. (2002), ML techniques can be utilized for the process of text classification. There are different types of VSMs present depending on the usage of BoW, in which the words are present in a linearized way [18]. It can be used as features/terms to represent the text data in the form of a vector [12, 14]. Hence, a document-term matrix is created using the vectors corresponding to each document. This representation has limitations like lack of correlation among sentences, and it misses the contextual meaning of keywords in the text document [29].In contrast, text-based phrases carry high semantic qualities; however, they lead to the loss of statistical quality as Hammouda and Kamel (2004) mentioned in [13]. Kyriakopoulou et al. (2006) applied clustering to the documents followed by a simple BoW model [30]. Term weighting schemes represent the text in the form of a document-term matrix such as TF-IDF. The different variants have unsupervised weighting schemes, which are similar to the IR fields. Elmasry et al. (2018) applied the TCW-ICF for ATC and obtained comparatively better results than TF-IDF [31]. In ATC, the ontology of semantics model has been presented across the words of a text document [32]. As compared to the VSM, the documents are represented in the form of a graph in GBR to capture such correlating words with an appropriate weight. Here, the connected ontology representation preserves the semantic meaning among the words in a text. Guru et al. (2010) proposed a novel symbolic text clustering method along with a symbolic feature selection method [33]. Syntactic Word Representation such as N-gram and other techniques mentioned by Grigori Sidorov et al. (2019) in [34] consist of sparse levels. In contrast, due to the deep learning revolution, many other techniques are also proposed like word embedding with variants by Earlier, Rong (2014). They proposed the Word2Vec method using Skipping N-gram and Continuous BoW (CBoW) technique, but it has failed to capture the contextual meaning of the word from the document, and it could not capture the word that is not present in the training phase, namely, out-of-vocabulary [35]. Bojanowski (2017) proposed the fast text approach that cannot capture the word that has a different meaning (polysemy). It needs huge memory for storing and pretraining purposes in comparison with GloVec [36] and Word2Vec [37]. Matthew E. Peters et al. (2018) propounded a new idea for representing text with handling contextualized meaning, but the required memory for storage and the performance of downstream tasks are also found computationally more expensive. It could not handle OOV words from the corpus and did not work at a character and word level [38]. Earlier, Mesleh et al. (2007) addressed the enhancement of two metrics precision and recall values by the usage of feature selection technique for Arabic text [19, 39]. In contrast, once deep learning techniques are proposed for Arabic text mining, high accuracy is achieved as compared to ML techniques using public datasets [40]. M. Alhawarat et al. applied a deep learning model, which got an accuracy range of 97.58% to 99.90%; however, the CNN model takes a longer time to complete the training process as compared to the traditional ML algorithms. Hence, it is evident that the ML techniques have the potential to reduce the training time by adding FDR methods. Elnagar et al. (2020) introduced novel unbiased single and multilabeled datasets for Arabic text categorization known as SANAD and NADiA. These datasets are investigated to identify the impact of word2vec embedded models on the performance improvement of the classification tasks [41]. Fatima et al. (2018) presented an Arabic text classification system depending on BoW, utilizing Arabic word sense [42], deep neural network [43], and deep Autoencoder representations [44]. The proposed method has achieved good results using precision and F-measure of 94% and 93%, respectively. Here, they failed to handle the issue of Arabic language ambiguity that enhances the performance of ATC by sense embedding techniques. A. Y. Muaad et al. in [45] proposed a new model for Arabic text called computer-aided recognition (ArCAR), which represented text in character-level. The same idea was presented in [46].


### 2.3. Feature Dimensional Reduction (FDR)

In the text mining process, thousands of terms are created through the process known as BoW. The major issues in text mining are known as noise, redundancy, and huge numbers of features, and they can be handled effectively by feature extraction techniques. FDR is done by feature selection or feature transformation. It is good to mention that less scalability and high time consumption are the key issues in dimensionality reduction. Hence, the majority of methods use feature selection rather than feature transformation [3]. There are plenty of feature selection techniques available for handling text data such as Chi-square, information gain, mutual information, and document frequency [31]. Nehar et al. (2016) introduced ATC by discussing a method for word root extraction without relying on any dictionary [47]. Here, they removed any non-Arabic letters and stop words, etc. Subsequently, LibSVM is used to build the classification model, and the result mentions that the performance of the classification was increased by root extraction. Bahassine et al. (2018) proposed the improved feature selection (FS), known as ImpCHI, and it is compared with other feature selection algorithms, namely, MI, IG, and CHI. The results indicate that ImpCHI with SVM was the best [48]. Alhaj et al. (2019) studied document classification using FS techniques such as CHI and IG [49]. They also studied different kinds of stemmers known as ISRI, Tashaphyne, and ARLStem along with TF-IDF and Chi-square to select the highest ranked Features. The results show that ARLStem accompanied by SVM offers a good classification performance. Elmasry et al. (2018) proposed a new FS technique by transforming the attributes of the TF-IDF term weighting technique [31]. Mohamed (2020) compared three-dimension reduction algorithms for identifying the pros and cons among each other. They concluded that principal component analysis (PCA) has produced effective results in ATC [50]. Chantar et al. (2019) proposed binary Grey Wolf Optimizer (GWO) as a wrapper-based FS technique. From all analyses of results for the proposed model, the SVM enhances the performance for ATC problems as compared to the other [51].From this related work, we observed that many researchers have applied classical representation like TFIDF and BoW in ATC to represent the text. Besides these representation methods, we try to reduce the size and enhance the performance using the FDR technique [52]. So, an efficient FDR technique is required to reduce the representation cost.


### 2.4. Text Classification

Once the representation matrix for a given collection of documents or corpus is created through an optimal set of features, a classifier has to train and classify the documents of different classes. The selection of a classifier is an extremely crucial task in the text classification process [3, 17]. Over the last few decades, text categorization problems have been intensively investigated and handled in a variety of real-world applications. Many researchers are increasingly interested in designing applications that use text categorization algorithms, especially in light of recent advances in NLP and text mining. In the literature of ML, there have been plenty of classifiers proposed including both parametric [17] and nonparametric classifiers [7, 12]. Mesleh et al. (2007) implemented a text classification system for Arabic language articles using SVM for classification and Chi to select features [19]. Mohamed El Kourdi (2013) used KNN and FS techniques with stemming and light stemming in preprocessing to achieve classification purposes. The word is reduced to their root, but using KNN needs to find an optimal k-value, and it is computationally expensive [53]. Al-Harbi et al. (2008) proposed Arabic text documents categorization on seven different Arabic corpora using a statistical technique. It is applied using traditional FS with SVM and C5.0 and concluded that a novel FS and weighting strategy is required for achieving an optimal accuracy [22]. Harrag and El-Qawasmah (2009) utilized the Term Weighting Scheme (TWS), Singular Value Decomposition (SVD), and ANN to classify documents in the Arabic language. From the literature survey of classification algorithms for Arabic text documents, it is inferred that several research gaps exist as stated in the sections above. In addition, a novel stemmer and FS strategy is mandatory to achieve good ATC performance [20].


## 2.1. Text Preprocessing

There is a collection of preprocessing methods such as stop words, which are used to eliminate unwanted words with high weights. For example, a, an, and the do not have significant meaning; therefore, deleting these words will not affect the model but will decrease the dimension like the authors in [1] have done for English. In contrast, the same idea has been used for the removal of Arabic text stop words such as [11] [2] [22] لاجل ، مع، لى. Stemming is the method of converting a word into its corresponding root or stem. Thus, it is seen as an important preprocessing step before handling any NLP tasks. Stemming is a very important process for Arabic text mining. Primarily, there are three types of stemmers known as root-based (Khoja), light-based (Light 10), and statistical-based (N-Grams) stemmers, and occasionally they will be further subdivided into many other types. It is important to apply the steaming process to the text data after tokenization. In Arabic text, Yousif et al. (2015) evaluated the effect of stemming algorithm along with NB classifier and concluded that the root extractor exhibits the best performance [23]. Previously, Rehab Duwairi et al. (2013) investigated the effects of stemming strategies on Arabic document classification [24]. They used a stemming algorithm called light stemming, and word clusters were investigated, and they conclude that the light stemming method improves the result in terms of accuracy. Mamoun and Ahmed (2016) proposed a new light stemmer for Arabic called P-stemmer. They modified the version of Larkey’s light stemmers. The validation of the stemmer has shown significantly increasing accuracy during the classification process when NB, SVM, and RF classifiers are applied. They conclude that the SVM classifier performs better than other classifiers [4]. Shargabi et al. (2011) used many classification algorithms such as NB, KNN, and J48 to build the classification models, and they conclude that the Light stemmer was better compared to Khoja stemmer [25]. Salam et al. (2016) presented the ATC system by studying the effect of normalization and stemming techniques such as ISRI and Tashaphyne stemmer. Finally, they concluded that the normalization was best for the outcome, but the stemmers could not produce the required accuracy [26]. Oraby et al. (2013) presented Arabic sentiment analysis, and they addressed the effect of the stemming algorithm. They showed the results of accuracy equal to 93.2%, 92.6%, and 92.2% for Tashaphyne, ISRI, and Khoja stemmers, respectively [27]. Qusay Walid et al. (2005) processed a document clustering process by studying the influence of ISRI stemmer [28]. They have concluded that the method of ISRI outperforms nonstemming methods. Al haj et al. (2018) introduced responsibilities of the stemming technique for Arabic text mining, which have been explored, and it has been found that stemming minimizes the computational difficulties for the classification process [11].


## 2.2. Text Representation

The text cannot be processed in the native form, because it is unstructured. The term unstructured is a very subjective property of text data, so there are many representation schemes available to address this problem with its own set of characteristics. The representation schemes are broadly classified into two types: the first is vector space-based representation model (VSM), and the second is a graph-based representation (GBR) [3].The VSM is a representation method that has been used because it provides a matrix representation for the text data. According to Sebastiani et al. (2002), ML techniques can be utilized for the process of text classification. There are different types of VSMs present depending on the usage of BoW, in which the words are present in a linearized way [18]. It can be used as features/terms to represent the text data in the form of a vector [12, 14]. Hence, a document-term matrix is created using the vectors corresponding to each document. This representation has limitations like lack of correlation among sentences, and it misses the contextual meaning of keywords in the text document [29].In contrast, text-based phrases carry high semantic qualities; however, they lead to the loss of statistical quality as Hammouda and Kamel (2004) mentioned in [13]. Kyriakopoulou et al. (2006) applied clustering to the documents followed by a simple BoW model [30]. Term weighting schemes represent the text in the form of a document-term matrix such as TF-IDF. The different variants have unsupervised weighting schemes, which are similar to the IR fields. Elmasry et al. (2018) applied the TCW-ICF for ATC and obtained comparatively better results than TF-IDF [31]. In ATC, the ontology of semantics model has been presented across the words of a text document [32]. As compared to the VSM, the documents are represented in the form of a graph in GBR to capture such correlating words with an appropriate weight. Here, the connected ontology representation preserves the semantic meaning among the words in a text. Guru et al. (2010) proposed a novel symbolic text clustering method along with a symbolic feature selection method [33]. Syntactic Word Representation such as N-gram and other techniques mentioned by Grigori Sidorov et al. (2019) in [34] consist of sparse levels. In contrast, due to the deep learning revolution, many other techniques are also proposed like word embedding with variants by Earlier, Rong (2014). They proposed the Word2Vec method using Skipping N-gram and Continuous BoW (CBoW) technique, but it has failed to capture the contextual meaning of the word from the document, and it could not capture the word that is not present in the training phase, namely, out-of-vocabulary [35]. Bojanowski (2017) proposed the fast text approach that cannot capture the word that has a different meaning (polysemy). It needs huge memory for storing and pretraining purposes in comparison with GloVec [36] and Word2Vec [37]. Matthew E. Peters et al. (2018) propounded a new idea for representing text with handling contextualized meaning, but the required memory for storage and the performance of downstream tasks are also found computationally more expensive. It could not handle OOV words from the corpus and did not work at a character and word level [38]. Earlier, Mesleh et al. (2007) addressed the enhancement of two metrics precision and recall values by the usage of feature selection technique for Arabic text [19, 39]. In contrast, once deep learning techniques are proposed for Arabic text mining, high accuracy is achieved as compared to ML techniques using public datasets [40]. M. Alhawarat et al. applied a deep learning model, which got an accuracy range of 97.58% to 99.90%; however, the CNN model takes a longer time to complete the training process as compared to the traditional ML algorithms. Hence, it is evident that the ML techniques have the potential to reduce the training time by adding FDR methods. Elnagar et al. (2020) introduced novel unbiased single and multilabeled datasets for Arabic text categorization known as SANAD and NADiA. These datasets are investigated to identify the impact of word2vec embedded models on the performance improvement of the classification tasks [41]. Fatima et al. (2018) presented an Arabic text classification system depending on BoW, utilizing Arabic word sense [42], deep neural network [43], and deep Autoencoder representations [44]. The proposed method has achieved good results using precision and F-measure of 94% and 93%, respectively. Here, they failed to handle the issue of Arabic language ambiguity that enhances the performance of ATC by sense embedding techniques. A. Y. Muaad et al. in [45] proposed a new model for Arabic text called computer-aided recognition (ArCAR), which represented text in character-level. The same idea was presented in [46].


## 2.3. Feature Dimensional Reduction (FDR)

In the text mining process, thousands of terms are created through the process known as BoW. The major issues in text mining are known as noise, redundancy, and huge numbers of features, and they can be handled effectively by feature extraction techniques. FDR is done by feature selection or feature transformation. It is good to mention that less scalability and high time consumption are the key issues in dimensionality reduction. Hence, the majority of methods use feature selection rather than feature transformation [3]. There are plenty of feature selection techniques available for handling text data such as Chi-square, information gain, mutual information, and document frequency [31]. Nehar et al. (2016) introduced ATC by discussing a method for word root extraction without relying on any dictionary [47]. Here, they removed any non-Arabic letters and stop words, etc. Subsequently, LibSVM is used to build the classification model, and the result mentions that the performance of the classification was increased by root extraction. Bahassine et al. (2018) proposed the improved feature selection (FS), known as ImpCHI, and it is compared with other feature selection algorithms, namely, MI, IG, and CHI. The results indicate that ImpCHI with SVM was the best [48]. Alhaj et al. (2019) studied document classification using FS techniques such as CHI and IG [49]. They also studied different kinds of stemmers known as ISRI, Tashaphyne, and ARLStem along with TF-IDF and Chi-square to select the highest ranked Features. The results show that ARLStem accompanied by SVM offers a good classification performance. Elmasry et al. (2018) proposed a new FS technique by transforming the attributes of the TF-IDF term weighting technique [31]. Mohamed (2020) compared three-dimension reduction algorithms for identifying the pros and cons among each other. They concluded that principal component analysis (PCA) has produced effective results in ATC [50]. Chantar et al. (2019) proposed binary Grey Wolf Optimizer (GWO) as a wrapper-based FS technique. From all analyses of results for the proposed model, the SVM enhances the performance for ATC problems as compared to the other [51].From this related work, we observed that many researchers have applied classical representation like TFIDF and BoW in ATC to represent the text. Besides these representation methods, we try to reduce the size and enhance the performance using the FDR technique [52]. So, an efficient FDR technique is required to reduce the representation cost.


## 2.4. Text Classification

Once the representation matrix for a given collection of documents or corpus is created through an optimal set of features, a classifier has to train and classify the documents of different classes. The selection of a classifier is an extremely crucial task in the text classification process [3, 17]. Over the last few decades, text categorization problems have been intensively investigated and handled in a variety of real-world applications. Many researchers are increasingly interested in designing applications that use text categorization algorithms, especially in light of recent advances in NLP and text mining. In the literature of ML, there have been plenty of classifiers proposed including both parametric [17] and nonparametric classifiers [7, 12]. Mesleh et al. (2007) implemented a text classification system for Arabic language articles using SVM for classification and Chi to select features [19]. Mohamed El Kourdi (2013) used KNN and FS techniques with stemming and light stemming in preprocessing to achieve classification purposes. The word is reduced to their root, but using KNN needs to find an optimal k-value, and it is computationally expensive [53]. Al-Harbi et al. (2008) proposed Arabic text documents categorization on seven different Arabic corpora using a statistical technique. It is applied using traditional FS with SVM and C5.0 and concluded that a novel FS and weighting strategy is required for achieving an optimal accuracy [22]. Harrag and El-Qawasmah (2009) utilized the Term Weighting Scheme (TWS), Singular Value Decomposition (SVD), and ANN to classify documents in the Arabic language. From the literature survey of classification algorithms for Arabic text documents, it is inferred that several research gaps exist as stated in the sections above. In addition, a novel stemmer and FS strategy is mandatory to achieve good ATC performance [20].


## 3. Material and Method

The proposed framework presented the Arabic text classification model using different preprocessing and representation techniques such as bag of word (BoW) and term frequency-inverse document frequency (TF-IDF). Meanwhile, several classifiers are adopted and performed, such as Multinomial NB, Bernoulli NB, LR, SGD Classifier, SVC, and Linear SVC. The findings of the present study are utilized to understand the influence of different methods in the performance improvement of the ATC system. The high-level workflow of the proposed system includes preprocessing, representation, FS, and classification algorithm as highlighted in Figure1. The preprocesses of the input text start by removing the stop words. Subsequently, the normalization and stemming process have been done to get the root for that text, so that the dimensions can be reduced.Figure 1
Schematic diagram of the proposed Arabic text classification architecture.The dimension-reduced text is passed through TF-IDF and BoW that generates a matrix as input. Then, the machine learning started passing this matrix into the ML algorithm after splitting it into two parts: 80% for training, and 20% for testing.


### 3.1. Preprocessing

The preprocessing technique is most commonly used for preparing the raw data into a specific input data format that ML models can work with. In document classification techniques, preprocessing method refers to the process of converting documents into a suitable form and making text ready for representation. This method is helpful in reducing the computational complexity [11, 22]. In Arabic text, the preprocessing phase eliminates all characters that do not have significant meaning, stop words, and punctuations [54, 55]. The various steps of preprocessing include (a) tokenization, (b) normalization, (c) stop word elimination, and (d) stemming as follows:


#### 3.1.1. Tokenization

The process of splitting text into tokens and replacing each word by number is known as tokenization (features). In Arabic, sentences are frequently divided by distinct signals such as commas, quotes, semicolons, spaces, and periods. These tokens could be single words (noun, verb, pronoun, etc.) that have been altered without regard for their meanings or relationships [56, 57].


#### 3.1.2. Normalization

Normalization is referred to as the transformation of a letter in the text into a canonical form or to remove the diacritics; for instance, transforms هــ into ة. Reference [46].


#### 3.1.3. Stop Word Elimination

It is the process of removing words in the sentences that do not hold any important meaning, for example, for لاجل, which means “so.” The researcher in Arabic mentions a list of words in [58, 59].


#### 3.1.4. Stemming

The process of removing most frequently prefixes and suffixes and definite articles from the word is named the steaming process. This process is to make a word in a root/base form. There are many types of stemming techniques available such as root, light [11], and hybrid stemming [59].


### 3.2. Feature Representation (Extraction)

Typically, the text processing cannot be done in its native form, because it is unstructured in nature. Many text feature representation schemes have been discussed already with their own characteristics. Initially, the dataset contains a group of documents with many classes as expressed in (1). Then, the technique for representation in terms of Term Frequency (TF) is presented in (2) displayed in the average number of times that occurs in a specific topic, which is divided by the rate of occurrences. (1)AD=d1,d2……dN,(2)tft,d=fdtmaxw∈dfdw.Thefdt is the rate of term ‘t’ in the document ‘d’ and ‘w’ is a set of words in the document ‘d’ and D is the corpus of documents.The measure of information provided by TF is IDF, to see whether a particular term is frequently or rarely used in all documents (common) that is equal to the logarithm of the quotient divided by the total number of documents as expressed in equation (3) and tf−idf is presented in equation (4).(3)idft,D=lnDd∈D;t∈d,(4)tf−idft,d,D=tft,d⋅idft,D.Generally, in text classification, thousands of terms are obtained through the process of representation such as BoW creation vector. It is a major issue with course dimension due to the characteristics such as noise, redundancy, and a large number of features. Hence, in the feature extraction process, determination of an efficient technique for representation is a critical phase, and there are many methods and different levels for representation [29, 45].


### 3.3. Arabic Document Classification

Classification of text documents based on their content is known as document classification. Several works have been reported herein based on text classification-oriented examples such as sentimental analysis, rating classification, and document classification with the help of many classification algorithms, with few well-known algorithms as follows: multinomial NB, Bernoulli NB, stochastic gradient descent (SGD), and logistic regression [21, 60, 61]. Classifiers have been introduced in recent works. In the proposed work, five open-source datasets, namely, CNN, BBC, OSAC, ArCovidVac, and ALKHALEEJ, are used with 6, 7, 10, and 10 classes, respectively, for document classification in Arabic text. Here, three datasets are unbalanced, and one dataset is balanced. Classification is the labeling of data documents or text to their classes based on their content. In the present work, the classification performance of the classifiers, namely, Multinomial NB [61–63], Bernoulli NB [64], SGD Classifier [65], SVC [66], Linear SVC [67, 68], and logistic regression, is explored, and the appropriate discussions are included in the subsequent sections.


#### 3.3.1. Multinomial Naive Bayes (MNB)

Naive Bayes in general is a probabilistic model. It has different forms such as MNB. Based on the explanation of MNB distribution and Bayes’ rule in [513], MNB classifiers use the following formulas in equation (5). The document ‘d’ belongs to class ‘c’ that is estimated using equation to get its probability:(5)Pc|dαPcΠ1≤k≤ndPtk|c,where Ptk|c is the conditional probability, Pc is the prior probability, and t1,t2,…tnd are the tokens in ‘d’. nd is the number of tokens in ‘d’.The aim is to find the best class for the text or document we want to classify. The maximum a posteriori (MAP)cmap is the best classification in NB classification(6)cmap=argmaxc∈CP^c|d=argmaxc∈CP^cΠ1≤k≤ndP^tk|c,(7)cmap=argmaxc∈ClogP^c+∑1≤k≤ndlogP^tk|c,(8)P^c=NcN;P^t|c=Tct∑t′∈VTct′,where Nc is the number of documents in class c and N is the total number of documents. The conditional probability P^t|c is the relative frequency of term ‘t’ in documents belonging to class c as in (8). Here, Tct is the number of occurrences of ‘t’ in training documents from class c.


#### 3.3.2. Bernoulli Naive Bayes (BNB)

These models are independent binary variables that describe the inputX=1;0;1::1, which means that the binary term occurrence is used instead of the frequency of the term in the BoW model as in(9)Pd|c=Pe1⋯eM|c=∏1≤i≤MPUi=ei|c.Ut=1iff ‘t’ occurs in the document, d=e1,e2,…,eM, where ei=e|c, P^Ui=e|c and P^c∏ti∈VP^Ui=e|ci.d=e1,e2…,eM is a binary vector of dimensionality M that indicates, for each term, whether it occurs in d or not, and ‘c’ is the class.


#### 3.3.3. Logistic Regression

LR is an expanded form of linear regression, which creates discrete probability scores using logistic sigmoid functions. This aids in classification or prediction by generating a regression function. The logistic regression model used binary classification problems, but it can also be expanded to multiclassification situations. In logistic regression, any input value is mapped to the [0, 1] range, and a predicted value is obtained. Predictions and probabilities can be mapped using the sigmoid function. The sigmoid function in equation (8) helps shrink the continuous input into a range of [0, 1]. Equation (11) fx consists of features ‘xj‘ and their corresponding weights/coefficients ‘βj‘ in a linear form shown in equation (12).(10)sigt=11+e−t,(11)PY|X=11+e−fx,(12)fx=x0+x1β1+⋯+xkβk+ε,where x,β,fx∈Rk and ε is the random error.


#### 3.3.4. Stochastic Gradient Descent (SGD)

This algorithm is good for huge training sets because this is a simplification of gradient descent algorithm. To reduce the computation cost, a stochastic version of the algorithm is being used in deep learning module training. Here, computing gradient is a more complex function, so the stochastic gradient will be descent trained on a random sample. The method is repeated over the training instances, updating the model parameters based on the update rule for each example as given in the following equation:(13)w←w−ηα∂Rw∂w+∂LwTxi+b,yi∂w,where η is the learning rate, which controls the step size in the parameter space. The intercept ‘b’ is updated similarly but without regularization. The learning rate η can be either constant or gradually decaying. For classification, the default learning rate schedule is given in (14)ηt=1αt0+t,where ‘t’ is the time step, and t0 is determined based on a heuristic.


#### 3.3.5. Support Vector Classifier

SVM is a statistical learning theory algorithm for classification tasks that has different forms such as SVC. The SVC is used in pattern recognition problems and most commonly in document classification based on the statistical learning theory [15, 69]. SVM is the most widely used algorithm for the classification of text documents. SVCs learn n-dimensional hyperplanes to classify the linear and nonlinear data into appropriate classes. Let us consider a training set of labeled instances that are known as paired linear functions, and it can be expressed in the following equation:(15)y=argmaxyiωT⟶Φx⟶,y′,(16)∀i∀y=yiωT⟶Φxi⟶,y′−ωT⟶Φxi⟶,y′≥1−ξi.


#### 3.3.6. Linear Support Vector Classifier

One application of SVM classifiers is SVC, Linear SVC. They are working based on Library for SVMs (LIBSVM). Linear SVC works with more options, so it is flexible. SVC with parameter kernels equal to ‘linear’ has more flexibility in the choice of penalties and loss functions and should be scaled better to large numbers of samples.


### 3.4. Evaluation Metrics

The performance of different classification algorithms was analyzed with various FS and feature representation methods with different sizes of public benchmark Arabic text datasets. The proposed method performance is evaluated in terms of precision, recall, accuracy, and F-Measure as used subsequently [70–75].


#### 3.4.1. Accuracy

In the measurement of a set, accuracy is defined as the closeness of the measured value to a specific actual value [76].(17)Accuracy=TP+TNTP+FN+TN+FP.


#### 3.4.2. Precision

Precision is the closeness among the measured quantities to each other. It is the fraction of retrieved items that are relevant to the classification results:(18)Precision=TPTP+FP.


#### 3.4.3. Recall/Sensitivity

Recall in information retrieval is the fraction of the items successfully retrieved that are relevant to the posted query [46, 77]:(19)Recall=TPTP+FN.


#### 3.4.4.F-Measure

TheF-measure is a measure of a test’s accuracy. It considers both precision and the recall of the test samples to compute the score. The traditional F-measure or balanced F-score (F1 score) is the harmonic mean of precision and recall:(20)F1−score=2⋅TP2⋅TP+FP+FN.


### 3.5. Arabic Text Classification

One of the supervised machine learning techniques is a classification algorithm that is used to classify new observations on the testing based on learning the pattern from training data.Algorithm 1: Classification with TFIDF.
(i)
Inputs:          D = {D1, D2, …, Dn}(ii)
wheren of Arabic documents(iii)
Di selected document(iv)
Where ∀ (Di) ∃ Cj (C is name of class) and (j) number of classes(v)
Output:          Assign Di (unknown Document) to correct class Cj(vi)
Begin               Read All collection of document in (corpus)(vii)
ForD = 1 to n(viii)
Do Preprocessing for Document(ix)
D[I]1 ← Tokenization (D[i])(x)
D[I]2 ← Stop word removal (D[i]1)(xi)
D[I]3 ← Stemming(D[i]2(xii)
D[I] ← TFIDF(D[i]3) OR D[I] ← BoW(D[i]3)(xiii)
D[I]Train = 80%(xiv)
D[I]Train = 20%(xv)
“Training phase”   W = Input matrix with weights Train (TFIDF; Document)(xvi)
Weight(W) for document and add Label(L) for each document(xvii)
MNB ←W+L where W is referred text and L referred to Label(xviii)
BNB ←W+L(xix)
SGD ←W+L(xx)
LR ←W+L(xxi)
SVC ←W+L(xxii)
Linear SVC ←W+L(xxiii)
“Testing phase”    W = Input matrix with weights Train (BoW; Document)(xxiv)
MNB ←W where W is referred text and L referred to Label(xxv)
BNB ←W(xxvi)
SGD ←W(xxvii)
LR ←W(xxviii)
SVC ←W(xxix)
MNB ←W(xxx)
End for(xxxi)
Push vector value without correspond label to classification algorithm then let the algorithm predictL(xxxii)
ML ← (text)(xxxiii)
Class(L) ← ML (Predict)(xxxiv)
End


## 3.1. Preprocessing

The preprocessing technique is most commonly used for preparing the raw data into a specific input data format that ML models can work with. In document classification techniques, preprocessing method refers to the process of converting documents into a suitable form and making text ready for representation. This method is helpful in reducing the computational complexity [11, 22]. In Arabic text, the preprocessing phase eliminates all characters that do not have significant meaning, stop words, and punctuations [54, 55]. The various steps of preprocessing include (a) tokenization, (b) normalization, (c) stop word elimination, and (d) stemming as follows:


### 3.1.1. Tokenization

The process of splitting text into tokens and replacing each word by number is known as tokenization (features). In Arabic, sentences are frequently divided by distinct signals such as commas, quotes, semicolons, spaces, and periods. These tokens could be single words (noun, verb, pronoun, etc.) that have been altered without regard for their meanings or relationships [56, 57].


### 3.1.2. Normalization

Normalization is referred to as the transformation of a letter in the text into a canonical form or to remove the diacritics; for instance, transforms هــ into ة. Reference [46].


### 3.1.3. Stop Word Elimination

It is the process of removing words in the sentences that do not hold any important meaning, for example, for لاجل, which means “so.” The researcher in Arabic mentions a list of words in [58, 59].


### 3.1.4. Stemming

The process of removing most frequently prefixes and suffixes and definite articles from the word is named the steaming process. This process is to make a word in a root/base form. There are many types of stemming techniques available such as root, light [11], and hybrid stemming [59].


## 3.1.1. Tokenization

The process of splitting text into tokens and replacing each word by number is known as tokenization (features). In Arabic, sentences are frequently divided by distinct signals such as commas, quotes, semicolons, spaces, and periods. These tokens could be single words (noun, verb, pronoun, etc.) that have been altered without regard for their meanings or relationships [56, 57].


## 3.1.2. Normalization

Normalization is referred to as the transformation of a letter in the text into a canonical form or to remove the diacritics; for instance, transforms هــ into ة. Reference [46].


## 3.1.3. Stop Word Elimination

It is the process of removing words in the sentences that do not hold any important meaning, for example, for لاجل, which means “so.” The researcher in Arabic mentions a list of words in [58, 59].


## 3.1.4. Stemming

The process of removing most frequently prefixes and suffixes and definite articles from the word is named the steaming process. This process is to make a word in a root/base form. There are many types of stemming techniques available such as root, light [11], and hybrid stemming [59].


## 3.2. Feature Representation (Extraction)

Typically, the text processing cannot be done in its native form, because it is unstructured in nature. Many text feature representation schemes have been discussed already with their own characteristics. Initially, the dataset contains a group of documents with many classes as expressed in (1). Then, the technique for representation in terms of Term Frequency (TF) is presented in (2) displayed in the average number of times that occurs in a specific topic, which is divided by the rate of occurrences. (1)AD=d1,d2……dN,(2)tft,d=fdtmaxw∈dfdw.Thefdt is the rate of term ‘t’ in the document ‘d’ and ‘w’ is a set of words in the document ‘d’ and D is the corpus of documents.The measure of information provided by TF is IDF, to see whether a particular term is frequently or rarely used in all documents (common) that is equal to the logarithm of the quotient divided by the total number of documents as expressed in equation (3) and tf−idf is presented in equation (4).(3)idft,D=lnDd∈D;t∈d,(4)tf−idft,d,D=tft,d⋅idft,D.Generally, in text classification, thousands of terms are obtained through the process of representation such as BoW creation vector. It is a major issue with course dimension due to the characteristics such as noise, redundancy, and a large number of features. Hence, in the feature extraction process, determination of an efficient technique for representation is a critical phase, and there are many methods and different levels for representation [29, 45].


## 3.3. Arabic Document Classification

Classification of text documents based on their content is known as document classification. Several works have been reported herein based on text classification-oriented examples such as sentimental analysis, rating classification, and document classification with the help of many classification algorithms, with few well-known algorithms as follows: multinomial NB, Bernoulli NB, stochastic gradient descent (SGD), and logistic regression [21, 60, 61]. Classifiers have been introduced in recent works. In the proposed work, five open-source datasets, namely, CNN, BBC, OSAC, ArCovidVac, and ALKHALEEJ, are used with 6, 7, 10, and 10 classes, respectively, for document classification in Arabic text. Here, three datasets are unbalanced, and one dataset is balanced. Classification is the labeling of data documents or text to their classes based on their content. In the present work, the classification performance of the classifiers, namely, Multinomial NB [61–63], Bernoulli NB [64], SGD Classifier [65], SVC [66], Linear SVC [67, 68], and logistic regression, is explored, and the appropriate discussions are included in the subsequent sections.


### 3.3.1. Multinomial Naive Bayes (MNB)

Naive Bayes in general is a probabilistic model. It has different forms such as MNB. Based on the explanation of MNB distribution and Bayes’ rule in [513], MNB classifiers use the following formulas in equation (5). The document ‘d’ belongs to class ‘c’ that is estimated using equation to get its probability:(5)Pc|dαPcΠ1≤k≤ndPtk|c,where Ptk|c is the conditional probability, Pc is the prior probability, and t1,t2,…tnd are the tokens in ‘d’. nd is the number of tokens in ‘d’.The aim is to find the best class for the text or document we want to classify. The maximum a posteriori (MAP)cmap is the best classification in NB classification(6)cmap=argmaxc∈CP^c|d=argmaxc∈CP^cΠ1≤k≤ndP^tk|c,(7)cmap=argmaxc∈ClogP^c+∑1≤k≤ndlogP^tk|c,(8)P^c=NcN;P^t|c=Tct∑t′∈VTct′,where Nc is the number of documents in class c and N is the total number of documents. The conditional probability P^t|c is the relative frequency of term ‘t’ in documents belonging to class c as in (8). Here, Tct is the number of occurrences of ‘t’ in training documents from class c.


### 3.3.2. Bernoulli Naive Bayes (BNB)

These models are independent binary variables that describe the inputX=1;0;1::1, which means that the binary term occurrence is used instead of the frequency of the term in the BoW model as in(9)Pd|c=Pe1⋯eM|c=∏1≤i≤MPUi=ei|c.Ut=1iff ‘t’ occurs in the document, d=e1,e2,…,eM, where ei=e|c, P^Ui=e|c and P^c∏ti∈VP^Ui=e|ci.d=e1,e2…,eM is a binary vector of dimensionality M that indicates, for each term, whether it occurs in d or not, and ‘c’ is the class.


### 3.3.3. Logistic Regression

LR is an expanded form of linear regression, which creates discrete probability scores using logistic sigmoid functions. This aids in classification or prediction by generating a regression function. The logistic regression model used binary classification problems, but it can also be expanded to multiclassification situations. In logistic regression, any input value is mapped to the [0, 1] range, and a predicted value is obtained. Predictions and probabilities can be mapped using the sigmoid function. The sigmoid function in equation (8) helps shrink the continuous input into a range of [0, 1]. Equation (11) fx consists of features ‘xj‘ and their corresponding weights/coefficients ‘βj‘ in a linear form shown in equation (12).(10)sigt=11+e−t,(11)PY|X=11+e−fx,(12)fx=x0+x1β1+⋯+xkβk+ε,where x,β,fx∈Rk and ε is the random error.


### 3.3.4. Stochastic Gradient Descent (SGD)

This algorithm is good for huge training sets because this is a simplification of gradient descent algorithm. To reduce the computation cost, a stochastic version of the algorithm is being used in deep learning module training. Here, computing gradient is a more complex function, so the stochastic gradient will be descent trained on a random sample. The method is repeated over the training instances, updating the model parameters based on the update rule for each example as given in the following equation:(13)w←w−ηα∂Rw∂w+∂LwTxi+b,yi∂w,where η is the learning rate, which controls the step size in the parameter space. The intercept ‘b’ is updated similarly but without regularization. The learning rate η can be either constant or gradually decaying. For classification, the default learning rate schedule is given in (14)ηt=1αt0+t,where ‘t’ is the time step, and t0 is determined based on a heuristic.


### 3.3.5. Support Vector Classifier

SVM is a statistical learning theory algorithm for classification tasks that has different forms such as SVC. The SVC is used in pattern recognition problems and most commonly in document classification based on the statistical learning theory [15, 69]. SVM is the most widely used algorithm for the classification of text documents. SVCs learn n-dimensional hyperplanes to classify the linear and nonlinear data into appropriate classes. Let us consider a training set of labeled instances that are known as paired linear functions, and it can be expressed in the following equation:(15)y=argmaxyiωT⟶Φx⟶,y′,(16)∀i∀y=yiωT⟶Φxi⟶,y′−ωT⟶Φxi⟶,y′≥1−ξi.


### 3.3.6. Linear Support Vector Classifier

One application of SVM classifiers is SVC, Linear SVC. They are working based on Library for SVMs (LIBSVM). Linear SVC works with more options, so it is flexible. SVC with parameter kernels equal to ‘linear’ has more flexibility in the choice of penalties and loss functions and should be scaled better to large numbers of samples.


## 3.3.1. Multinomial Naive Bayes (MNB)

Naive Bayes in general is a probabilistic model. It has different forms such as MNB. Based on the explanation of MNB distribution and Bayes’ rule in [513], MNB classifiers use the following formulas in equation (5). The document ‘d’ belongs to class ‘c’ that is estimated using equation to get its probability:(5)Pc|dαPcΠ1≤k≤ndPtk|c,where Ptk|c is the conditional probability, Pc is the prior probability, and t1,t2,…tnd are the tokens in ‘d’. nd is the number of tokens in ‘d’.The aim is to find the best class for the text or document we want to classify. The maximum a posteriori (MAP)cmap is the best classification in NB classification(6)cmap=argmaxc∈CP^c|d=argmaxc∈CP^cΠ1≤k≤ndP^tk|c,(7)cmap=argmaxc∈ClogP^c+∑1≤k≤ndlogP^tk|c,(8)P^c=NcN;P^t|c=Tct∑t′∈VTct′,where Nc is the number of documents in class c and N is the total number of documents. The conditional probability P^t|c is the relative frequency of term ‘t’ in documents belonging to class c as in (8). Here, Tct is the number of occurrences of ‘t’ in training documents from class c.


## 3.3.2. Bernoulli Naive Bayes (BNB)

These models are independent binary variables that describe the inputX=1;0;1::1, which means that the binary term occurrence is used instead of the frequency of the term in the BoW model as in(9)Pd|c=Pe1⋯eM|c=∏1≤i≤MPUi=ei|c.Ut=1iff ‘t’ occurs in the document, d=e1,e2,…,eM, where ei=e|c, P^Ui=e|c and P^c∏ti∈VP^Ui=e|ci.d=e1,e2…,eM is a binary vector of dimensionality M that indicates, for each term, whether it occurs in d or not, and ‘c’ is the class.


## 3.3.3. Logistic Regression

LR is an expanded form of linear regression, which creates discrete probability scores using logistic sigmoid functions. This aids in classification or prediction by generating a regression function. The logistic regression model used binary classification problems, but it can also be expanded to multiclassification situations. In logistic regression, any input value is mapped to the [0, 1] range, and a predicted value is obtained. Predictions and probabilities can be mapped using the sigmoid function. The sigmoid function in equation (8) helps shrink the continuous input into a range of [0, 1]. Equation (11) fx consists of features ‘xj‘ and their corresponding weights/coefficients ‘βj‘ in a linear form shown in equation (12).(10)sigt=11+e−t,(11)PY|X=11+e−fx,(12)fx=x0+x1β1+⋯+xkβk+ε,where x,β,fx∈Rk and ε is the random error.


## 3.3.4. Stochastic Gradient Descent (SGD)

This algorithm is good for huge training sets because this is a simplification of gradient descent algorithm. To reduce the computation cost, a stochastic version of the algorithm is being used in deep learning module training. Here, computing gradient is a more complex function, so the stochastic gradient will be descent trained on a random sample. The method is repeated over the training instances, updating the model parameters based on the update rule for each example as given in the following equation:(13)w←w−ηα∂Rw∂w+∂LwTxi+b,yi∂w,where η is the learning rate, which controls the step size in the parameter space. The intercept ‘b’ is updated similarly but without regularization. The learning rate η can be either constant or gradually decaying. For classification, the default learning rate schedule is given in (14)ηt=1αt0+t,where ‘t’ is the time step, and t0 is determined based on a heuristic.


## 3.3.5. Support Vector Classifier

SVM is a statistical learning theory algorithm for classification tasks that has different forms such as SVC. The SVC is used in pattern recognition problems and most commonly in document classification based on the statistical learning theory [15, 69]. SVM is the most widely used algorithm for the classification of text documents. SVCs learn n-dimensional hyperplanes to classify the linear and nonlinear data into appropriate classes. Let us consider a training set of labeled instances that are known as paired linear functions, and it can be expressed in the following equation:(15)y=argmaxyiωT⟶Φx⟶,y′,(16)∀i∀y=yiωT⟶Φxi⟶,y′−ωT⟶Φxi⟶,y′≥1−ξi.


## 3.3.6. Linear Support Vector Classifier

One application of SVM classifiers is SVC, Linear SVC. They are working based on Library for SVMs (LIBSVM). Linear SVC works with more options, so it is flexible. SVC with parameter kernels equal to ‘linear’ has more flexibility in the choice of penalties and loss functions and should be scaled better to large numbers of samples.


## 3.4. Evaluation Metrics

The performance of different classification algorithms was analyzed with various FS and feature representation methods with different sizes of public benchmark Arabic text datasets. The proposed method performance is evaluated in terms of precision, recall, accuracy, and F-Measure as used subsequently [70–75].


### 3.4.1. Accuracy

In the measurement of a set, accuracy is defined as the closeness of the measured value to a specific actual value [76].(17)Accuracy=TP+TNTP+FN+TN+FP.


### 3.4.2. Precision

Precision is the closeness among the measured quantities to each other. It is the fraction of retrieved items that are relevant to the classification results:(18)Precision=TPTP+FP.


### 3.4.3. Recall/Sensitivity

Recall in information retrieval is the fraction of the items successfully retrieved that are relevant to the posted query [46, 77]:(19)Recall=TPTP+FN.


### 3.4.4.F-Measure

TheF-measure is a measure of a test’s accuracy. It considers both precision and the recall of the test samples to compute the score. The traditional F-measure or balanced F-score (F1 score) is the harmonic mean of precision and recall:(20)F1−score=2⋅TP2⋅TP+FP+FN.


## 3.4.1. Accuracy

In the measurement of a set, accuracy is defined as the closeness of the measured value to a specific actual value [76].(17)Accuracy=TP+TNTP+FN+TN+FP.


## 3.4.2. Precision

Precision is the closeness among the measured quantities to each other. It is the fraction of retrieved items that are relevant to the classification results:(18)Precision=TPTP+FP.


## 3.4.3. Recall/Sensitivity

Recall in information retrieval is the fraction of the items successfully retrieved that are relevant to the posted query [46, 77]:(19)Recall=TPTP+FN.


## 3.4.4.F-Measure

TheF-measure is a measure of a test’s accuracy. It considers both precision and the recall of the test samples to compute the score. The traditional F-measure or balanced F-score (F1 score) is the harmonic mean of precision and recall:(20)F1−score=2⋅TP2⋅TP+FP+FN.


## 3.5. Arabic Text Classification

One of the supervised machine learning techniques is a classification algorithm that is used to classify new observations on the testing based on learning the pattern from training data.Algorithm 1: Classification with TFIDF.
(i)
Inputs:          D = {D1, D2, …, Dn}(ii)
wheren of Arabic documents(iii)
Di selected document(iv)
Where ∀ (Di) ∃ Cj (C is name of class) and (j) number of classes(v)
Output:          Assign Di (unknown Document) to correct class Cj(vi)
Begin               Read All collection of document in (corpus)(vii)
ForD = 1 to n(viii)
Do Preprocessing for Document(ix)
D[I]1 ← Tokenization (D[i])(x)
D[I]2 ← Stop word removal (D[i]1)(xi)
D[I]3 ← Stemming(D[i]2(xii)
D[I] ← TFIDF(D[i]3) OR D[I] ← BoW(D[i]3)(xiii)
D[I]Train = 80%(xiv)
D[I]Train = 20%(xv)
“Training phase”   W = Input matrix with weights Train (TFIDF; Document)(xvi)
Weight(W) for document and add Label(L) for each document(xvii)
MNB ←W+L where W is referred text and L referred to Label(xviii)
BNB ←W+L(xix)
SGD ←W+L(xx)
LR ←W+L(xxi)
SVC ←W+L(xxii)
Linear SVC ←W+L(xxiii)
“Testing phase”    W = Input matrix with weights Train (BoW; Document)(xxiv)
MNB ←W where W is referred text and L referred to Label(xxv)
BNB ←W(xxvi)
SGD ←W(xxvii)
LR ←W(xxviii)
SVC ←W(xxix)
MNB ←W(xxx)
End for(xxxi)
Push vector value without correspond label to classification algorithm then let the algorithm predictL(xxxii)
ML ← (text)(xxxiii)
Class(L) ← ML (Predict)(xxxiv)
End


## 4. Datasets

Most of the researchers in Arabic text classification studies have collected their datasets individually. In this article, six well-known classifiers are operated on five publicly available Arabic datasets, namely, CNN, BBC, OSAC, ArCovidVac, and AlKhaleej Arabic corpora, and the performance metrics are studied [41, 78].


### 4.1. BBC Arabic Corpus

The Arabic BBC corpus dataset contains 4,763 documents in Arabic, and all documents belong to 1 class from 7. The number of documents for each class is as follows: 2,356 documents for Middle East News, 1,489 documents for world news, 296 documents for business and economics, 219 Sports documents, 49 documents for international press, 232 science and technology documents, and 122 art and culture documents [78]. In total, this corpus contains about 1,860,786 (1.8 M) words and 1,06,733 distinct keywords as summarized in Table 1.Table 1
BBC corpus dataset distribution.
#Class typeNo. of documentsTraining/Validation set (80%)Test set (20%)1Middle east news2,3561,8854712Science and Technology232186463International press4939104Art and culture12298245Sports219175446Business and economy296237597World news1,4891,191298


### 4.2. CNN Arabic Corpus

The number of documents in the CNN Arabic corpus is 5,070. Every single document belongs to 1 class. The number of documents for each class is as follows: 836 business documents, 474 entertainment documents, 1,462 documents for middle east news, 526 documents for science and technology, 762 sports documents, and 1010 documents for world news [78]. In total, this corpus contains about 2,241,348 (2.2 M) words and 1, 44, 460 distinct keywords, and the number of documents in each category is summarized in Table 2.Table 2
CNN corpus dataset distribution.
#Class typeNo. of documentsTrainTest1Entertainments474379952Science and technology5264211053Sports7626101524International press---5Business and economy8366691676World news1,0108082027Middle east news1,4621,170292


### 4.3. Open-Source Arabic Corpus (OSAc)

This dataset included 22,429 Arabic text documents collected from multiple sources [78]. Every single text document belongs to 1 class from 10 classes. The number of documents for each class is different as mentioned in Table 3. This corpus contains about 1,81,83,511 (18M) words and 4,49,600 distinct keywords.Table 3
OSAC corpus dataset distribution.
#Class typeNo. of documentsTrainTest1Health2,2961,8374592Sports2,4191,9354843Cooking recipes2,3731,8984754Religion3,1712,5376345Education and family3,6082,8867226History3,2332,5866477Economy3,1022,4826208Stories7265811459Low94475518910Astronomy557446111


### 4.4. AlKhaleej

This dataset is scraped from all articles published in the news portal from 2008 to 2018 [41]. The collected text dataset exceeds the volume of 4 GB, and most of the articles are published on the websites. AlKhaleej has seven classes, which are Finance, Sports, Culture, Technology, Politics, Medical, and Religion. The dataset is generated in a balanced way where each class contains 6,500 Arabic articles. The training and testing sets are randomly split to be 5,200 and 1,300 per class, respectively.


### 4.5. Covid-19 Dataset

One more dataset about Covid-19 has been used to classify Arabic comments. Those comments belong to short text [79]. The data distribution of the Covid-19 dataset is shown in Table 4.Table 4
Covid-19 corpus dataset distribution.
#Class typeNo. of documentsTrainTest1Positive7,9626,3691,5922Negative6355081363Natural1,3911,113286


## 4.1. BBC Arabic Corpus

The Arabic BBC corpus dataset contains 4,763 documents in Arabic, and all documents belong to 1 class from 7. The number of documents for each class is as follows: 2,356 documents for Middle East News, 1,489 documents for world news, 296 documents for business and economics, 219 Sports documents, 49 documents for international press, 232 science and technology documents, and 122 art and culture documents [78]. In total, this corpus contains about 1,860,786 (1.8 M) words and 1,06,733 distinct keywords as summarized in Table 1.Table 1
BBC corpus dataset distribution.
#Class typeNo. of documentsTraining/Validation set (80%)Test set (20%)1Middle east news2,3561,8854712Science and Technology232186463International press4939104Art and culture12298245Sports219175446Business and economy296237597World news1,4891,191298


## 4.2. CNN Arabic Corpus

The number of documents in the CNN Arabic corpus is 5,070. Every single document belongs to 1 class. The number of documents for each class is as follows: 836 business documents, 474 entertainment documents, 1,462 documents for middle east news, 526 documents for science and technology, 762 sports documents, and 1010 documents for world news [78]. In total, this corpus contains about 2,241,348 (2.2 M) words and 1, 44, 460 distinct keywords, and the number of documents in each category is summarized in Table 2.Table 2
CNN corpus dataset distribution.
#Class typeNo. of documentsTrainTest1Entertainments474379952Science and technology5264211053Sports7626101524International press---5Business and economy8366691676World news1,0108082027Middle east news1,4621,170292


## 4.3. Open-Source Arabic Corpus (OSAc)

This dataset included 22,429 Arabic text documents collected from multiple sources [78]. Every single text document belongs to 1 class from 10 classes. The number of documents for each class is different as mentioned in Table 3. This corpus contains about 1,81,83,511 (18M) words and 4,49,600 distinct keywords.Table 3
OSAC corpus dataset distribution.
#Class typeNo. of documentsTrainTest1Health2,2961,8374592Sports2,4191,9354843Cooking recipes2,3731,8984754Religion3,1712,5376345Education and family3,6082,8867226History3,2332,5866477Economy3,1022,4826208Stories7265811459Low94475518910Astronomy557446111


## 4.4. AlKhaleej

This dataset is scraped from all articles published in the news portal from 2008 to 2018 [41]. The collected text dataset exceeds the volume of 4 GB, and most of the articles are published on the websites. AlKhaleej has seven classes, which are Finance, Sports, Culture, Technology, Politics, Medical, and Religion. The dataset is generated in a balanced way where each class contains 6,500 Arabic articles. The training and testing sets are randomly split to be 5,200 and 1,300 per class, respectively.


## 4.5. Covid-19 Dataset

One more dataset about Covid-19 has been used to classify Arabic comments. Those comments belong to short text [79]. The data distribution of the Covid-19 dataset is shown in Table 4.Table 4
Covid-19 corpus dataset distribution.
#Class typeNo. of documentsTrainTest1Positive7,9626,3691,5922Negative6355081363Natural1,3911,113286


## 5. Experimental Analysis and Discussion

The algorithms such as MNB, BNB, LR, SGD Classifier, SVC, and linear SVC are implemented herein using Python 3.8.0 programming with Anaconda [Jupyter notebook]. The Python-based ML libraries such as NLTK, pandas, and scikit-learn are utilized to investigate the performance metrics by the proposed methods. The results and discussions concerning various techniques incorporated are highlighted in the subsequent sections.


### 5.1. Evaluation Result Based on Representation and Preprocessing

The six classification algorithms are operated on an unbalanced CNN benchmark dataset for ATC along with two types of representation such TF-IDF and BoW with and without preprocessing. Here, the SGD classifier and linear SVC with TF-IDF yield an accuracy of about 93%. In contrast, with BoW representation, LR followed by linear SVC offers good accuracy. For a balanced dataset (AlKhaleej), the linear SVC is found to be the best choice with preprocessing and in contrast, the accuracy decreases without preprocessing. Altogether, every classification algorithm except BNB works well and the best was when the dataset was preprocessed. The BNB algorithm works well only with binary classification, and its performance decreases with multiclass classification problems as listed in Tables5 and 6. Another dataset has been studied, which is called Covid-19 as short text. We implemented the same idea to study the effectiveness of representation and preprocessing. We conclude that logistic regression and support vector classifiers are the best results and the effectiveness of preprocessing in some cases positive such as BoW and in others is negative such as TFIDF in Tables 7.Table 5
Data description of representation for CNN with and without pre-processing.
ClassifiersBag of words (BoW)Term frequency inverse document frequency (TFIDF)With preprocessingWithout preprocessingWith preprocessingWithout preprocessingMultinomial Naive Bayes95939391Bernoulli Naive Bayes90858587Logistic regression97969694Stochastic gradient descent97959594Support vector classifier97969695Linear support vector classifier97969694Table 6
Data description of representation for Alkhaleej with and without pre-processing.
ClassifiersBag of words (BoW)Term frequency inverse document frequency (TFIDF)With preprocessingWithout preprocessingWith preprocessingWithout preprocessingMultinomial Naive Bayes88886458Bernoulli Naive Bayes61736173Logistic regression93929091Stochastic gradient descent91919392Support vector classifier90919092Linear support vector classifier92919392Table 7
Data description of Covid-19 with BoW and cross-validation.
ClassifiersBag of words (BoW)Term frequency inverse document frequency (TFIDF)With preprocessingWithout preprocessingWith preprocessingWithout preprocessingMultinomial Naive bayes80818080Bernoulli Naive Bayes80807573Logistic regression80818180Stochastic gradient descent79808080Support vector classifier80808180Linear support vector classifier79798080


### 5.2. Evaluation Result Based on the Selected Features

From the experiments carried out for different numbers of features that have been chosen, we observed that as the number of features is increased, there is FDR drastic rise in the execution time. Meanwhile, no changes have been observed in the performance of the classifiers. It should be noted that the accuracy did not change with all the classifiers except SVC after increasing the features from 7,000 to 10,000. Another interesting observation is that the results are similar with the reduced and actual number of features, while SVC is operated on CNN. Conversely, while SVC is operated on the AlKhaleej dataset with 8,000 features, the accuracy was increased to 95% as listed in Figures2 and 3.Figure 2
Data description of feature selection for CNN with TFIDF.Figure 3
Data description of feature selection for Alkhaleej with TF-IDF.


### 5.3. Evaluation Result Based on Cross-Validation

The relationship between reduced or increased features with cross-validation is studied simultaneously with two representation methods, namely, TF-IDF and BoW. It is observed that an increased number of features would intensify or reduce the accuracy of ATC in a few cases. From the results summarized in Tables8–15, it is inferred that the CNN with the increased number of features leads to performance improvement in Linear SVC. In contrast, there are no changes in the performance of Linear SVC that were observed with respect to increased features in the balanced dataset. The results prove that not only representation and feature selection affect the text classification performance, but also the nature of data (balanced/unbalanced) have a significant impact. With OSAC dataset, the accuracy remains the same among all the classifiers compared herein except BNB. The various interesting observations concerning the accuracy of ATC, while the features increased or decreased are highlighted in Tables 8–15.Table 8
Data description of CNN with TIDF and cross-validation.
ClassifiersHigh accuracy with 500 featuresAverage accuracy for 10 cross-validation with 500 featuresMultinomial Naive Bayes9088Bernoulli Naive Bayes8178Logistic regression9190Stochastic gradient descent9290Support vector classifier9391Linear support vector classifier9491Table 9
Data description of CNN with BoW and cross-validation.
ClassifiersHigh accuracy with 500 featuresAverage accuracy for 10 cross-validation with 500 featuresMultinomial Naive Bayes8481Bernoulli Naive Bayes8884Logistic regression9491Stochastic gradient descent9592Support vector classifier9492Linear support vector classifier9592Table 10
Data description of BBC with TFIDF and cross-validation.
ClassifiersHigh accuracy with 500 featuresAvg. Accuracy for 10 cross validation with 500 featuresMultinomial Naive Bayes8480Bernoulli Naive Bayes8886Logistic regression8785Stochastic gradient descent9391Support vector classifier8785Linear support vector classifier9390Table 11
Data description of BBC with BoW and cross-validation.
ClassifiersHigh accuracy with 5000 featuresAverage accuracy for 10 cross-validation with 5000 featuresMultinomial Naive Bayes7674Bernoulli Naive Bayes9391Logistic regression8885Stochastic gradient descent9190Support vector classifier8785Linear support vector classifier9189Table 12
Data description of OSAC with TFIDF and cross-validation.
ClassifiersHigh accuracy with 500 featuresAverage accuracy for 10 cross-validation with 500 featuresMultinomial Naive Bayes9594Bernoulli Naive Bayes8281Logistic regression9797Stochastic gradient descent9898Support vector classifier9898Linear support vector classifier9898Table 13
Data description of OSAC with BOW and cross-validation.
ClassifiersHigh accuracy with 5000 featuresAvg. Accuracy for 10 cross-validation with 5000 featuresMultinomial Naive Bayes9595Bernoulli Naive Bayes8786Logistic regression9898Stochastic gradient descent9999Support vector classifier9898Linear support vector classifier9999Table 14
Data description of Alkhaleej 500 feature with TFIDF and cross-validation.
ClassifiersHigh accuracy with 500 featuresAvg. Accuracy for 10 cross-validation with 500 featuresMultinomial Naive Bayes9493Bernoulli Naive Bayes8585Logistic regression9696Stochastic gradient descent9595Support vector classifier9696Linear support vector classifier9696Table 15
Data description Alkhaleej 10000 feature with TFIDF and cross-validation.
ClassifiersHigh accuracy with 10000 featuresAvg. Accuracy for 10 cross-validation with 10000 featuresMultinomial Naive Bayes9595Bernoulli Naive Bayes9090Logistic regression9797Stochastic gradient descent9797Support vector classifier9797Linear support vector classifier9797


## 5.1. Evaluation Result Based on Representation and Preprocessing

The six classification algorithms are operated on an unbalanced CNN benchmark dataset for ATC along with two types of representation such TF-IDF and BoW with and without preprocessing. Here, the SGD classifier and linear SVC with TF-IDF yield an accuracy of about 93%. In contrast, with BoW representation, LR followed by linear SVC offers good accuracy. For a balanced dataset (AlKhaleej), the linear SVC is found to be the best choice with preprocessing and in contrast, the accuracy decreases without preprocessing. Altogether, every classification algorithm except BNB works well and the best was when the dataset was preprocessed. The BNB algorithm works well only with binary classification, and its performance decreases with multiclass classification problems as listed in Tables5 and 6. Another dataset has been studied, which is called Covid-19 as short text. We implemented the same idea to study the effectiveness of representation and preprocessing. We conclude that logistic regression and support vector classifiers are the best results and the effectiveness of preprocessing in some cases positive such as BoW and in others is negative such as TFIDF in Tables 7.Table 5
Data description of representation for CNN with and without pre-processing.
ClassifiersBag of words (BoW)Term frequency inverse document frequency (TFIDF)With preprocessingWithout preprocessingWith preprocessingWithout preprocessingMultinomial Naive Bayes95939391Bernoulli Naive Bayes90858587Logistic regression97969694Stochastic gradient descent97959594Support vector classifier97969695Linear support vector classifier97969694Table 6
Data description of representation for Alkhaleej with and without pre-processing.
ClassifiersBag of words (BoW)Term frequency inverse document frequency (TFIDF)With preprocessingWithout preprocessingWith preprocessingWithout preprocessingMultinomial Naive Bayes88886458Bernoulli Naive Bayes61736173Logistic regression93929091Stochastic gradient descent91919392Support vector classifier90919092Linear support vector classifier92919392Table 7
Data description of Covid-19 with BoW and cross-validation.
ClassifiersBag of words (BoW)Term frequency inverse document frequency (TFIDF)With preprocessingWithout preprocessingWith preprocessingWithout preprocessingMultinomial Naive bayes80818080Bernoulli Naive Bayes80807573Logistic regression80818180Stochastic gradient descent79808080Support vector classifier80808180Linear support vector classifier79798080


## 5.2. Evaluation Result Based on the Selected Features

From the experiments carried out for different numbers of features that have been chosen, we observed that as the number of features is increased, there is FDR drastic rise in the execution time. Meanwhile, no changes have been observed in the performance of the classifiers. It should be noted that the accuracy did not change with all the classifiers except SVC after increasing the features from 7,000 to 10,000. Another interesting observation is that the results are similar with the reduced and actual number of features, while SVC is operated on CNN. Conversely, while SVC is operated on the AlKhaleej dataset with 8,000 features, the accuracy was increased to 95% as listed in Figures2 and 3.Figure 2
Data description of feature selection for CNN with TFIDF.Figure 3
Data description of feature selection for Alkhaleej with TF-IDF.


## 5.3. Evaluation Result Based on Cross-Validation

The relationship between reduced or increased features with cross-validation is studied simultaneously with two representation methods, namely, TF-IDF and BoW. It is observed that an increased number of features would intensify or reduce the accuracy of ATC in a few cases. From the results summarized in Tables8–15, it is inferred that the CNN with the increased number of features leads to performance improvement in Linear SVC. In contrast, there are no changes in the performance of Linear SVC that were observed with respect to increased features in the balanced dataset. The results prove that not only representation and feature selection affect the text classification performance, but also the nature of data (balanced/unbalanced) have a significant impact. With OSAC dataset, the accuracy remains the same among all the classifiers compared herein except BNB. The various interesting observations concerning the accuracy of ATC, while the features increased or decreased are highlighted in Tables 8–15.Table 8
Data description of CNN with TIDF and cross-validation.
ClassifiersHigh accuracy with 500 featuresAverage accuracy for 10 cross-validation with 500 featuresMultinomial Naive Bayes9088Bernoulli Naive Bayes8178Logistic regression9190Stochastic gradient descent9290Support vector classifier9391Linear support vector classifier9491Table 9
Data description of CNN with BoW and cross-validation.
ClassifiersHigh accuracy with 500 featuresAverage accuracy for 10 cross-validation with 500 featuresMultinomial Naive Bayes8481Bernoulli Naive Bayes8884Logistic regression9491Stochastic gradient descent9592Support vector classifier9492Linear support vector classifier9592Table 10
Data description of BBC with TFIDF and cross-validation.
ClassifiersHigh accuracy with 500 featuresAvg. Accuracy for 10 cross validation with 500 featuresMultinomial Naive Bayes8480Bernoulli Naive Bayes8886Logistic regression8785Stochastic gradient descent9391Support vector classifier8785Linear support vector classifier9390Table 11
Data description of BBC with BoW and cross-validation.
ClassifiersHigh accuracy with 5000 featuresAverage accuracy for 10 cross-validation with 5000 featuresMultinomial Naive Bayes7674Bernoulli Naive Bayes9391Logistic regression8885Stochastic gradient descent9190Support vector classifier8785Linear support vector classifier9189Table 12
Data description of OSAC with TFIDF and cross-validation.
ClassifiersHigh accuracy with 500 featuresAverage accuracy for 10 cross-validation with 500 featuresMultinomial Naive Bayes9594Bernoulli Naive Bayes8281Logistic regression9797Stochastic gradient descent9898Support vector classifier9898Linear support vector classifier9898Table 13
Data description of OSAC with BOW and cross-validation.
ClassifiersHigh accuracy with 5000 featuresAvg. Accuracy for 10 cross-validation with 5000 featuresMultinomial Naive Bayes9595Bernoulli Naive Bayes8786Logistic regression9898Stochastic gradient descent9999Support vector classifier9898Linear support vector classifier9999Table 14
Data description of Alkhaleej 500 feature with TFIDF and cross-validation.
ClassifiersHigh accuracy with 500 featuresAvg. Accuracy for 10 cross-validation with 500 featuresMultinomial Naive Bayes9493Bernoulli Naive Bayes8585Logistic regression9696Stochastic gradient descent9595Support vector classifier9696Linear support vector classifier9696Table 15
Data description Alkhaleej 10000 feature with TFIDF and cross-validation.
ClassifiersHigh accuracy with 10000 featuresAvg. Accuracy for 10 cross-validation with 10000 featuresMultinomial Naive Bayes9595Bernoulli Naive Bayes9090Logistic regression9797Stochastic gradient descent9797Support vector classifier9797Linear support vector classifier9797


## 6. Conclusion

For classifying Arabic text, a comprehensive investigation study is performed to show the effectiveness of preprocessing, feature extraction, feature selection, and the nature of the dataset. Several AI-based techniques have been presented to highlight the effectiveness of various methods on classifying Arabic text. The findings of the study show that there are many methods that affect the accuracy of the system performance on ATC. Our observation for this study proves that representation (feature extraction and feature selection) is highly important in ATC. At the same time, preprocessing, classification, and the nature of the dataset all affect the performance of classification. The results demonstrate the advantages of the feature representation approach that affect the text classification performance. Based on our understanding in this article, there are still many open issues for future work such as lack of benchmark dataset, lexicons, and simultaneously, finding techniques that handle the context meaning of ATC. Many other tools for ATC can be improved such as the augmentation of data. At last, an improving preprocessing technique for ATC specially is stemming.



---

*Source: 3720358-2022-04-30.xml*