A Novel Fuzzy-Logic-Based Multi-Criteria Metric for Performance Evaluation of Spam Email Detection Algorithms  
[PERSON]  
1 College of Computing and Information Sciences, Karachi Institute of Economics and Technology, Karachi 75190, Pakistan; [EMAIL_ADDRESS]  
[PERSON]  
1 College of Computing and Information Sciences, Karachi Institute of Economics and Technology, Karachi 75190, Pakistan; [EMAIL_ADDRESS]  
[PERSON]  
2 Cyber Security Center, Prince Mohammad Bin Fahd University, Dhahran 34754, Saudi Arabia; [EMAIL_ADDRESS]  
[PERSON]  
3 Computer and Information Sciences Department, Universiti Teknologi PETERONAS, Seri Iskandar 32610, Malaysia; [EMAIL_ADDRESS]  
[PERSON]  
4 Center for Intelligent Signal and Imaging Research (CISIR), Department of Electrical and Electronics Engineering, Universiti Teknologi PETERONAS, Seri Iskandar 32610, Malaysia; [EMAIL_ADDRESS]  
Emgar A. A. 2022. This number is estimated to reach 376.4 billion in 2025, with 4.6 billion users [3]. Consequently, the increase in spam emails is proportional to the increase in overall email traffic. The continuous increase in spam emails at a global scale has resulted in a high cost in terms of time, storage requirements, network bandwidth usage, and indirect costs of protecting privacy and security [4]. In addition, the nuisance caused by spam emails on human mental health and moods may have its own role, as indicated by studies [5; 6]. The problem of spam email detection has several dimensions. The first dimension is concerned with the algorithms used for the spam detection. The prime objective of a spam detection algorithm is to segregate legitimate and spam emails as correctly as possible. For over two decades, researchers have proposed various solutions and methods, such as the use of Blacklist [7], Real-Time Blackhole List [8], and Content-Based Filters [9] to identify and filter spam from hams. Active research is still being conducted to develop methods that are faster and more accurate. In particular, artificial intelligence (AI)-based methods have received notable attention from researchers in recent years. Special emphasis has been placed on the methods based on machine learning [1; 10; 11; 12; 13; 14; 15; 16; 17; 18; 19; 20; 21; 22; 23; 24; 25]. In addition, deep learning methods have recently been successfully applied to spam email detection [10; 11; 12; 13; 14; 26; 27; 28]. The outcomes of these studies indicate that machine learning and deep learning algorithms provide an effective platform for efficiently solving the spam detection problem; yet there is still a room for further improvement and development. The second dimension of the spam detection problem is the datasets used to assess the performance of the spam detection algorithm. The dataset contained in a corpus plays a critical role in evaluating the performance of any spam detector [2]. A number of datasets have been reported in the literature, such as Spambase [29], Lingspam [29], PU1 [30], PU2 [31], PU3 [31], PU3 [31], PUA [31], Enron [32; 33], Genspam [34], Trec 2005 [35], Trec 2006 [36], and Trec 2007 [37]. A comprehensive list of email spam datasets available in the public domain is reported by [PERSON] et al. [2]. The datasets have distinctive features and their suitability and effectiveness is determined by the type of experiments performed to evaluate the performance of the spam detection algorithm. In addition, several researchers use datasets that are not well known, or developed by the researchers themselves [1; 12; 17; 19; 20; 21; 22; 24; 28]. The third dimension is the performance metrics. In order to evaluate the performance of a spam detection algorithm using a dataset, an appropriate performance measure is necessary. One of the common metrics used for the evaluation is the _accuracy_, which measures the proportion of the total number of messages correctly classified (whether ham or spam) [15]. However, using accuracy as the only performance measure is not sufficient [2]. Therefore, other performance metrics such as _recall_ and _precision_ are also used [15].