Effective spam filter based on a hybrid method of header checking and content parsing

IET NetworksVolume 9, Issue 6 p. 338-347 Research ArticleFree Access Effective spam filter based on a hybrid method of header checking and content parsing Ko-Tsung Chu, Ko-Tsung Chu Department of Finance, Minghsin University of Science and Technology, Hsinchu, 30401 TaiwanSearch for more papers by this authorHua-Ting Hsu, Hua-Ting Hsu Department of Information Management, National Dong Hwa University, Hualien, 97401 TaiwanSearch for more papers by this authorJyh-Jian Sheu, Corresponding Author Jyh-Jian Sheu jjsheu@nccu.edu.tw College of Communication, National Chengchi University, Taipei, 11605 TaiwanSearch for more papers by this authorWei-Pang Yang, Wei-Pang Yang Department of Information Management, National Dong Hwa University, Hualien, 97401 TaiwanSearch for more papers by this authorCheng-Chi Lee, Cheng-Chi Lee Department of Library and Information Science, Research and Development Center for Physical Education, Health, and Information Technology, Fu Jen Catholic University, New Taipei City, 24205 Taiwan Department of Photonics and Communication Engineering, Asia University, Taichung, 413 TaiwanSearch for more papers by this author Ko-Tsung Chu, Ko-Tsung Chu Department of Finance, Minghsin University of Science and Technology, Hsinchu, 30401 TaiwanSearch for more papers by this authorHua-Ting Hsu, Hua-Ting Hsu Department of Information Management, National Dong Hwa University, Hualien, 97401 TaiwanSearch for more papers by this authorJyh-Jian Sheu, Corresponding Author Jyh-Jian Sheu jjsheu@nccu.edu.tw College of Communication, National Chengchi University, Taipei, 11605 TaiwanSearch for more papers by this authorWei-Pang Yang, Wei-Pang Yang Department of Information Management, National Dong Hwa University, Hualien, 97401 TaiwanSearch for more papers by this authorCheng-Chi Lee, Cheng-Chi Lee Department of Library and Information Science, Research and Development Center for Physical Education, Health, and Information Technology, Fu Jen Catholic University, New Taipei City, 24205 Taiwan Department of Photonics and Communication Engineering, Asia University, Taichung, 413 TaiwanSearch for more papers by this author First published: 03 November 2020 https://doi.org/10.1049/iet-net.2019.0191AboutSectionsPDF ToolsRequest permissionExport citationAdd to favoritesTrack citation ShareShare Give accessShare full text accessShare full-text accessPlease review our Terms and Conditions of Use and check box below to share full-text version of article.I have read and accept the Wiley Online Library Terms and Conditions of UseShareable LinkUse the link below to share a full-text version of this article with your friends and colleagues. Learn more.Copy URL Share a linkShare onFacebookTwitterLinkedInRedditWechat Abstract In recent years, hazardous e-mails arose, such as the e-mails infected with 'viruses' or 'worms' spreading destructive programs and the 'Phishing Mails' defrauding e-mail accounts of the users. The number of spams continue to grow. With the related problems of spam coming to be more severe, the spam topics have become significant in various research domains. The common filtering methods include black/white list, rule learning, and those based on text classification, such as Naïve Bayes, support vector machine, and boosting trees, multi-agent and genetic algorithm. Among these, the methods based on text classification are most widely applied. Moreover, some efficient methods were proposed to consider only the e-mail's header section, based on which both operating efficiency and classification efficiency could be improved. By applying machine learning technique and decision tree data mining algorithm C4.5, this study aims to propose an efficient spam filtering method with the following features: (i) proposing a two-phase filtering mechanism to scan mainly e-mail's header and auxiliary content. (ii) Reducing the problem of false positive. The experimental results show that the authors' method has a considerably high accuracy rate of 98.76%. Compared with some other methods of using the same spam data sets or of deep learning-based, their method obviously has an excellent performance. 1 Introduction As the sending of e-mail requires low cost, e-mail becomes one of the tools heavily exploited by advertising agents. A large number of business e-mails are sent without the consent of the recipients, which are known as 'spams'. To date, e-mail boxes are always full of spams. In recent years, hazardous e-mails arose, such as the e-mails infected with 'viruses' or 'worms' spreading destructive programs and the 'Phishing Mails' defrauding e-mail accounts of the users. The number of spams continue to grow. According to Symantec spam report dated in May 2014 [1 ], the global spam rate is 60.6% for May. In other words, almost 61 in 100 e-mails are spams, which is an astonishing figure. From the economic point of view, it is conservatively estimated that American firms and consumers experience costs of almost $20 billion annually due to spam [2 ]. Moreover, two major industries, spam industry (spammers and spam-advertised merchants) and anti-spam industry (blocks spams), have been created; on the one hand, the spam industry collects gross worldwide revenues on the order of US$200 million per year and on the other hand, the anti-spam industry generates US$500 million annually [2 –4 ]. With the related problems of spam coming to be more severe, the spam topics have become significant in various research domains. To address the problem of spam, many research studies have proposed various filtering techniques. The common methods include black/white list [5 ], rule learning [6, 7 ], and those based on text classification, such as Naïve Bayes [8 –10 ], support vector machine (SVM) [11 ] and boosting trees [12, 13 ], multi-agent [14 –16 ] and genetic algorithm [17, 18 ]. Among those, the methods based on text classification are most widely applied. Moreover, some efficient methods were proposed to consider only e-mail's header section, based on which both operating efficiency and classification efficiency could be improved [19 –22 ]. Khan et al. suggested some intelligent methods such as evolutionary algorithms to fight against spam botnets [23 –25 ]. Moreover, the ensemble-based classifiers [26, 27 ] integrate multiple models to increase their prediction performance and have been effectively used in filtering spams [28, 29 ]. However, most of the existing methods are facing some problems as follows. First, although spam filtering methods based on text classification are superior to traditional methods, they usually make misjudgements [30 ], whose costs are rather high. Among these misjudgements, FP (false positive, which classifies legitimate e-mails as spams) produces more severe results than FN (false negative, which classifies spams as legitimate e-mails) does. Secondly, most methods scan e-mail's content, but the process of scanning content may increase complexity and reduce operating efficiency. The copious information for identifying whether an e-mail is legitimate or not could be obtained by applying the content-based filtering methods to parse the whole content of e-mail; however, the process of parsing is time consuming. On the other hand, the header-based methods, which examine only e-mail's header, may unavoidably suffer from the deficiency of information. Thirdly, current methods have the trade-off between strengthening the spam detection capability and reducing the problem of FP. In other words, if high spam detection capability intends to be maintained, FP should be sacrificed and raised; low FP may decline spam detection capability. Over the years, people are increasingly using mobile services as a way of communication. Hence, the problems and detection of short message service (SMS) spams have become hot research issues. Recently, some researchers achieved a high accuracy to detect the e-mail/SMS spams by applying various methods. Olatunji [31 ] proposed an SVM-based model for spam detection while paying attention to appropriately search for the optimal parameters to achieve better performance. By using deep learning technique, Jain et al. [32 ] implemented a long short-term memory (LSTM) model of the recursive neural network (RNN) to classify spams. Yang et al. [33 ] used the LSTM model and the convolutional neural network (CNN) model to filter spams. Gupta et al. [34 ] proposed an ensemble learning method for spam detection. Mohammad [35 ] created a lifelong spam e-mail classification model to filter spams. Roy et al. [36 ] used deep learning technique to employ CNN and LSTM models for SMS spam detection. Even though people are increasingly using mobile text messages as a convenient way of communication, traditional e-mails have not been eliminated. At present, traditional e-mail service is still the most formal and important communication tool for individuals or businesses. Thus, even though many spam detection methods of SMS, Botnet, and social network have been proposed and achieved high accuracy, finding more efficient and simple spam filtering methods for traditional e-mail system is still an important research topic in the current time. With high operating efficiency and low handling complexity, decision tree data mining algorithm may effectively raise operating speed, and its generated rules are human-readable. By applying machine learning technique and decision tree data mining algorithm C4.5, this research aims to propose an efficient spam filtering method with the following features for improving the above-mentioned problems: (i) Proposing a two-phase filtering mechanism to scan mainly e-mail's header and auxiliary content: This research presents a two-phase filtering mechanism, mainly handling e-mail's header section, in order to mostly avoid the low operating efficiency in scanning e-mail content. When e-mails cannot be judged only based on the header, content scanning can be utilised to facilitate judgment. Therefore, the problem of misjudgement due to one-time judgment can be effectively improved. (ii) Reducing the problem of FP: This research will propose a training mechanism to establish, respectively, spammy keyword table and legitimate keyword table. Coupled with the two-phase filtering mechanism, it can effectively solve the problem of the trade-off between strengthening the spam detection capability and reducing the problem of the filter misjudging legitimate e-mails as spams. The remaining part of this paper is organised as follows. Section 2 is the literature review; Section 3 presents the two-phase spam filtering mechanism that can effectively integrate header handling and content scanning; Section 4 discusses the experimental results; Section 5 gives the conclusions and suggestions to future studies. 2 Decision tree data mining algorithm The decision tree is the data mining method upon the tree data structure. The general statistical method usually can only calculate the distribution of the surface of data whereas the decision tree can find the potential association rules between the data attributes. Let 'Target Attribute ' be the data attribute which is concerned objective of our research. For example, the attribute 'E-mail type ' whose value is either 'spammy' or 'legitimate' is chosen as the target attribute in this paper. Also, let 'Critical Attributes ' be the other important data attributes which interest us in this research. The decision tree algorithm can analyse the potential association rules between the target attribute and the critical attributes. Moreover, the association rules can be applied to predict or classify the unknown data. Among all decision tree methods, ID3 (iterative dichotomiser 3) and C4.5 are the two most well-known decision tree data mining algorithms [37, 38 ]. The former was proposed by Quinlan in 1986, and the latter was developed by Quinlan by improving ID3 in 1993. C4.5 algorithm is a new algorithm presented by Quinlan by improving the ID3 algorithm. ID3 only supports categorical attributes. However, C4.5 can support both categorical attributes and numerical attributes at the same time. As the values of data attributes adopted in this research are composed of numerical type and categorical type, C4.5 is selected as the tool of data mining in this paper. This research only proposes an ideal filtering methodology by using original C4.5 in view of its conciseness. However, there exist some optimised approaches of C4.5 [39 ]. Note that it is likely that the improved versions of C4.5 will acquire better performance in implementing practical filtering software. A decision tree is constructed from the top (the root) to bottom (the leaves). Suppose that all data instances are stored in the root node, in the beginning, the C4.5 algorithm will choose a critical attribute with the maximum Gain Ratio but not yet selected, and allocate data instances into child nodes according to their values of the chosen critical attribute. Each child node will repeat the procedures of choosing critical attribute mentioned above, until any of the stop condition is satisfied, to complete the building procedures of the decision tree. The C4.5 algorithm has three stop conditions as follows: (i) all data instances in this child node have the same value of target attribute; (ii) all critical attributes have been selected; and (iii) the number of data instances in child node is less than a specific number M which is defined in advance. We describe the detailed computation process of C4.5 algorithm [38 ] in the Appendix at the end of this paper. 3 Effective two-phase hybrid spam filtering method integrating header handling and content scanning Fig. 1 shows the proposed system framework. The process of the whole method is divided into two stages: (i) Training stage : The purpose of this stage is to learn spam keywords, which will be used to establish the keyword tables, and acquire the association rules about spams by utilising C4.5 algorithm through training e-mails. This stage contains two major modules: (I) Keyword table establishing module: The task of this module is to learn spam keywords form training e-mails and establish the keyword tables. (II) Two-phase hybrid training module: This module contains two training tasks. One is to find association rules based on the attributes captured from the e-mail's header; and the other is to find association rules based on the attributes captured from the e-mail's content. Fig 1Open in figure viewerPowerPoint Architecture of the two-phase hybrid spam filtering method (ii) Classification stage : The framework of this stage is a hybrid of the header-based method and the content-based method, which will dynamically apply the two kinds of association rules found in training stage to classify the test e-mails. We present a two-phase filtering mechanism, mainly handling the header section of test e-mail in the first phase, in order to improve the low operating efficiency in scanning the whole content of the test e-mail. When the test e-mail cannot be judged only based on the header section, the second phase based on content scanning will be utilised to facilitate judgment. Therefore, the problem of misjudgement due to one-time judgment can be effectively improved. Detailed contents of the two stages will be introduced in the remaining part of this section. 3.1 Module (I) of training stage: keyword table establishing module In order to detect the suspicious words in e-mail's header and content, the explicit spam keyword tables are indispensable to a spam filter. Note that both the two major modules in training stage contain two kinds of filters simultaneously: 'header-based filter' and 'content-based filter'. As header-based filter will parse e-mail's header section, while content-based filter will parse e-mail's body section, the considerable difference occurs in the characteristics of keywords. Therefore, keyword tables should be established, respectively, for these two filters. Our spam filtering system proposed in this research is equipped with four keyword tables: (i) H L -table is a table of keywords about legitimate e-mails for header-based filter; (ii) H s -table is a table of keywords about spams for header-based filter; (iii) C L -table is a table of keywords about legitimate e-mails for content-based filter; and (iv) C S -table is a table of keywords about spams for content-based filter. The process of this module is illustrated in Fig. 2, which is introduced as follows: Step 1. Extraction of header and body: Extract the header section and the body section for each training e-mail, and handle them separately. Step 2. Processing of tokenisation and stop-word: The purpose of this step is to remove spaces, punctuations, and stop-words of the words in header and body. Step 3. Word stemming: Without word stemming performed, 'apple' and 'apples' will be considered as two independent words; however, they are the same, but with the difference in type. After stemming performed, 'apple' and 'apples' will be reduced to 'appl'. In this way, repetitively arising words with different types can be reduced. This research adopts the Porter stemming algorithm [40 ] for the work of stemming in this step. Step 4. Keyword extraction: Step 3 may generate many thousands of words. Obviously, we only need to retain those representative words as keywords. Thus, we set a threshold tf, and choose each word whose frequency is more than tf to be a formal keyword. Given a formal keyword chosen from the header section, it will be stored into H s -table if this e-mail is spammy; and H L -table otherwise. Moreover, given a formal keyword chosen form the body section, it will be stored into C S -table if this e-mail is spammy; and C L -table otherwise. Step 5. Scoring: Calculate the scores of keywords for each keyword table. The calculation details will be introduced as follows. Note that the usage of word stemming (Step 3) would cause a reduction in the performance of classification techniques owing to the generation of over-steaming and under-steaming errors [41 ]. The adoption of word ontologies such as WordNet or other alternatives can substitute for the usage of word stemming [42, 43 ]. Obviously, the frequent appearance of a keyword usually implies that this keyword may play a key role in recognising spams. Given a keyword table, we suppose that there exist p keywords, which are denoted by Ki for 1 ≤ i ≤ p. Let the score of keyword Ki be expressed as Score(Ki ). Then the following formula will compute the score for each keyword in this keyword table: where Count(Ki ) is the frequency of the keyword Ki, CountMIN is the minimum frequency, and CountMAX is the maximum frequency in this keyword table. The purpose of adding 0.1 is to prevent the score of the keyword with minimum frequency from being 0. Therefore, all of the scores will be converted into 0.1–1.1. A calculation example of H L -table is shown in Table 1. Assume that we set the threshold tf as 155. Then we will choose only three keywords ('hpl', 'nom', and 'meter') and store them into the H L -table. Suppose that'hpl', 'nom', and 'meter' are denoted as K 1, K 2, and K 3, respectively. Then their scores can be calculated as follows: Table 1. Ten candidates for keywords of HL-table No. Word Frequency Score 1 hpl 581 1.1 2 nom 485 0.874647887 3 meter 357 0.574178404 4 actual 279 0.391079812 5 enron 271 0.372300469 6 nomin 262 0.351173709 7 ga 236 0.290140845 8 deal 196 0.196244131 9 fw 177 0.151643192 10 revis 155 0.1 Fig 2Open in figure viewerPowerPoint Process of the keyword table establishing module 3.2 Module (II) of training stage: two-phase hybrid training module The architecture of this module is shown in Fig. 3. There are two kinds of training tasks in this module: (i) header-based filter will analyse header sections of training e-mails to compute the association rules, which will be store into the rule database 'Rule_Database (H )'; and (ii) content-based filter will scan the body sections of training e-mails to compute the association rules, which will be stored into the rule database 'Rule_Database (C )'. Fig 3Open in figure viewerPowerPoint Architecture of the two-phase hybrid classification module In this module, the same copies of training e-mails will be concurrently sent to both header-based filter and content-based filter to compute association rules, respectively. The detailed process is described as follows. 3.2.1 Training task of header-based filter Header-based filter takes the header section of training e-mail as calculation basis, then utilises algorithm C4.5 through the related attributes in the header section, and finally extracts the association rules and stores them into Rule_ Database(H ). The detailed training flow is shown in Fig. 4, which is introduced as follows: Step 1 : Extract the header sections from training e-mails. Step 2 : The extracted header of each training e-mail will be checked, respectively, to capture the values of all necessary attributes: the target attribute 'E-mail type' is a categorical attribute, whose value is either 'spammy' or 'legitimate'. As shown in Table 2, this research defines two critical attributes of binary values and six critical attributes of numerical values for header-based filter. The values of H 1, H 2, …, H 8 can be easily decided by checking the corresponding fields of header section. Also, the value of attribute 'Score of keywords in header section' (H 8) will be calculated by looking up the keyword tables (i.e. H L -table and H S -table). The value of H 8 is the summed scores of keywords in H S -table minus the summed scores of keywords in H L -table. Now we describe the value calculation of H 8 by using an example as follows. Assume that both H L -table and H S -table contain the same three keywords 'how', 'you' and 'as'. Moreover, the related scores of keywords are shown in Table 3. Given a training e-mail, we assume that its subject contains three keywords 'how,' 'you' and 'do'. The keyword 'do' will be ignored since it is not contained in H L -table and H S -table. Since the summed scores of 'how' and 'you' in H S -table and H L -table are 0.2 + 0.9=1.1 and 1.1 + 0.6=1.7, respectively, we can conclude that the value of H 8 is 1.1 − 1.7=−0.6. Obviously, the smaller value of H 8 indicates that the subject of this e-mail contains more keywords of legitimate tendency. Furthermore, the attribute H 5 is to check whether the difference between 'Receiving Time on e-mail's envelope' and 'Sending Date on e-mail's header' is more than 1 h. Receiving Time on e-mail's envelope is the time of simple mail transfer protocol (SMTP) host server receiving the e-mail, which is not generally easy to be modified; Sending Date on e-mail's header is the time of the sender sending the e-mail, which is easy to be altered. The spammer may alter Sending Date by forward adjusting the time (resulting in sending time later than today) or backward adjusting the time (resulting in sending time earlier than today). When the receiving software of the recipient arranges sequences in terms of Sending Date, the junk e-mail may arise in the first or last sequence. Therefore, when the difference between the 'Receiving Time ' and the 'Sending Date ' is more than 1 h, it indicates abnormality. Step 3 : Apply the decision tree algorithm C4.5 to construct decision tree: The captured attributes of training e-mails mentioned above will be input into the algorithm C4.5 to build a decision tree, which will bring out the potential association rules of 'if–then' pattern between the eight critical attributes and the target attribute ('spammy' or 'legitimate'). Step 4 : Score each association rule extracted from the resulted decision tree, and then store the rules into the rule database 'Rule_Database(H )': Assume that there are m association rules, which are denoted as Ri for 1 ≤ i ≤ m. Let the associated leaf node of rule Ri be denoted as Leaf(Ri ), and the total number of training e-mails contained in Leaf(Ri ) be |Leaf(Ri )|. Moreover, let the number of legitimate training e-mails and the number of spammy training e-mails contained in Leaf(Ri ) be L# (Leaf(Ri )) and S# (Leaf(Ri )), respectively. The scoring formulas in this research are modified from [20, 21 ], which are introduced as follows: (i) Spam_Intention(Ri ): Calculate the 'intensity' of the rule Ri judging e-mails as spams. Obviously, larger Spam_Intention(Ri ) indicates higher possibility of Ri judging e-mails as spams. The formula is as follows: (ii) Rule_ Support(Ri ): to calculate the degree of support of rule Ri, which is used to indicate the importance of Ri among all rules. The higher degree of rule support indicates that most e-mails can be classified by this rule. The formula is as follows: where N is total quantity of training e-mails. (iii) Weight (Ri ): Calculate the weighted value of the rule Ri. The formula below converts the resulted value into 0–100%. The formula is as follows: where SupportMIN is the minimum value and SupportMAX is the maximum value of Rule_ Support(Rj ) for 1 ≤ j ≤ m, respectively. (iv) Confident(Ri ): Calculate the level of confidence of the rule Ri. The level of confidence indicates the possibility of the rule Ri making wrong judgments. :The formula is as follows: where MAX(L# (Leaf(Ri )), S# (Leaf(Ri ))) is the bigger one of L# (Leaf(Ri )) and S# (Leaf(Ri )). (v) Rule_ Score(Ri ): Calculate the score of the rule Ri. The formula consists of three formulas Confident(Ri ), Spam_ Intention(Ri ), Weight (Ri ). Furthermore, Spam_ Intention(Ri ), and Weight (Ri ) are multiplied by α and (1−α ) in their front, representing the proportion of them, and finally multiplied by Confident(Ri ), the formula is as follows: where α is a coefficient between 0 and 1. Fig 4Open in figure viewerPowerPoint Training process of the header-based filter Table 2. Attributes of header-based filter ID Critical attribute Type Value H 1 length of sender name numerical — H 2 domain length of sender address numerical — H 3 number of recipients numerical — H 4 absence or inexistence of subject categorical binary: true or false H 5 abnormality of receiving date and sending date categorical binary: true or false H 6 mail size numerical — H 7 mail text size numerical — H 8 score of keywords in header section numerical — target attribute: e-mail type categorical 'S': spammy; 'L': legitimate Table 3. Example of scores of keywords in header-based filter Keyword Score in H L -table Score in H S -table how 1.1 0.2 you 0.6 0.9 as 0.5 0.3 Table 4. Attributes of content-based filter ID Critical attribute Type Value C 1 Length of sender name numerical — C 2 domain length of sender address numerical — C 3 number of recipients numerical — C 4 absence or inexistence of subject categorical binary: true or false C 5 abnormality of receiving date and sending date categorical binary: true or false C 6 mail size numerical — C 7 mail text size numerical — C 8 score of keywords in body section numerical — target attribute: e-mail type categorical 'S': spammy;'L': legitimate Fig 5Open in figure viewerPowerPoint Training process of the content-based filter Table 5. Example of scores of keywords in content-based filter Keyword Score in C L -table Score in C S -table current 1.1 0.2 development 0.9 0 test 0.6 0.8 as 0.5 0.3 name 0.1 0.1 3.2.2 Training task of content-based filter Content-based filter captures the attributes in e-mail's body section and takes them as calculation basis to find the association rules by utilising algorithm C4.5. Then the extracted association rules will be stored into Rule_Database (C ). The training flow is as shown in Fig. 5, which is introduced as follows: Step 1: Extract the body section from each training e-mail. Step 2 : Respectively calculate the values of all classification attributes of extracted body sections, according to the designed attribute table as shown in Table 4. Note that the target attribute 'E-mail type' is identical to that in Table 2. Moreover, the critical attributes C 1, C 2, …, C 7 in Table 4 are exactly the same as H 1, H 2, …, H 7 in Table 2. Therefore, the values of these attributes have been generated by header-based filter, requiring no re-calculation, and only the critical attribute 'Score of keywords in body section' (C 8) needs to be calculated. The value of H 8 is the summed scores of keywords in C S -table minus the summed scores of keywords in C L -table. A calculation example is as follows: Assume that both C L -table and C S -table contain five keywords, which are shown in Table 5 with the related scores. Given a training e-mail, we assume that its body section contains the keywords of 'current', 'test', 'development', 'snapshot' and 'name'. Note that the keyword 'snapshot' will be ignored since it is not included in the keyword tables. Obviously, the summed scores of keywords in C S -table is 0.2 + 0 + 0.8 + 0.1 = 1.1, and the summed scores of keywords in C L -table is 1.1 + 0.9 + 0.6 + 0.1=2.7. Thus, the value of H 8 is 1.1 − 2.7=−1.6. The smaller score indicates that the body section of the training e-mail conta