S pam C lus: An Agglomerative Clustering Algorithm for Spam Email Campaigns Detection (2024)

Abstract: Spam emails constitute a significant proportion of emails received by users, and can result in financial losses or in the download of malware on the victim’s device. Cyberattackers create spam campaigns to deliver spam messages on a large scale and benefit from the low economic investment and anonymity required to create the attacks. In addition to spam filters, raising awareness about active email scams is a relevant measure that helps mitigate the consequences of spam. Therefore, detecting campaigns becomes a relevant task in identifying and alerting the targets of spam. In this paper, we propose an unsupervised learning algorithm, SpamClus_1, an iterative algorithm that groups spam email campaigns using agglomerative clustering. The measures employed to determine the clusters are the minimum number of samples and minimum percentage of similarity within a cluster. Evaluating SpamClus_1 on a set of emails provided by the Spanish National Cybersecurity Institute (INCIBE), we found that the optimal values are 50 minimum samples and a minimum cosine similarity of 0.8. The clustering results show 19 spam datasets with 3048 spam samples out of 6702 emails from a range of three consecutive days and eight spam clusters with 870 spam samples out of 1469 emails from one day.

## introduction in 2023, more than 45% of the emails received by individuals were spam @xcite and this figure is projected to reach 4.48 billion emails per day by 2024 @xcite . due to the characteristics provided by emails, such as low economic investment and anonymity, spam campaign emails have become a useful tool employed by cyberattackers to perform criminal activities. some examples are the advertising of fake products, scams that cause financial losses, the mass mailing of malware, or illegal activities that end, in many cases, in economic losses for companies or even individuals @xcite . to reduce the impact or potential damage to users, companies try to run awareness campaigns indicating actions to be avoided, such as downloading files from unknown email addresses. in addition, platforms like gmail in turn implement filters that label emails as spam so that the user handles them with the greatest possible care, and thus avoids becoming a victim of malicious spam emails. those filters use black and white lists of email addresses to identify spam, however, spammers are constantly developing techniques to bypass the spam filters of e-mail clients @xcite . these solutions are most of the time lagging due to spammers' ability to innovate and troubleshoot to bypass filters. some of the techniques they use include hiding text, images, or even html tags in the email body. these elements may add noise when we try to cluster messages by their content @xcite . because of these reasons, it is important to understand and develop systems that can remove these spammers' tricks and classify effectively spam emails from legitimate ham emails. clues for identifying spammers are usually hidden in multiple aspects such as content, behavior, relationships, and interaction with the review @xcite . authors usually try to group emails by tagging them by subject. however, spam campaigns can contain a wide variety of topics and limiting the number of subjects can be an unrealistic scenario. in this paper, we propose the first version of an iterative algorithm that uses the agglomerative cluster to detect spam campaigns based on a minimum number of examples and a similarity percentage between emails from the same campaign, in this case measured as cosine similarity. the rest of the paper is organized as follows: the literature review is presented in section 2. in section 3, we explain the methodology. then, we introduce the experiments and results in section 4. finally, the discussion, conclusions and future work are described in section 5 and section 6. ## literature review authors have applied different approaches to address spam campaign detection and spam clustering. most of the works in this field use the content of the emails to group them and identify the campaigns. @xcite followed this approach and applied a topic modeling technique based on latent dirichlet allocation to detect spam reviews, but did not consider the identification of campaigns. however, authors such as @xcite used the url as a basis to cluster spam campaigns using the amount of money mentioned in the email as an additional feature. several email features have also been used for clustering phishing campaigns, as proposed by @xcite , who employed mean shift algorithm to group emails based on the email sender, subject, body, and url. dinh et al. (2015) also used several email features in their work, including the email content type, character set, subject, layout, url, and attachment. their proposal consisted of a software framework that identifies campaigns in real-time and labels and scores the campaigns detected. they employed a database to handle a large number of spam emails, a scoring mechanism to highlight severe spam campaigns and a visualization tool. typically, spam campaign detection is addressed as a binary classification problem. for example, @xcite proposed an unsupervised algorithm that clusters emails into ham and spam, based on the domain and header information. they used a dataset with 22,000 emails from several sources, such as @xcite , trec @xcite and @xcite datasets. however, some authors manage this problem as a multi-classification problem, where they create clusters based on the topic of the spam campaign @xcite . @xcite proposed a model based on auto-encoders and clustering algorithms for spam review detection, although they do not identify campaigns. our approach addresses spam campaign detection as a clustering problem where we group our spam campaigns in clusters with the same topic without labeling the dataset. the samples of the clusters in most of the cases should be similar except for small differences in specific data such as personalized information. therefore, the objective of our research is not to detect spam or identify spam topics, but rather to identify campaigns, which consist of sets of emails with the same goal and similar characteristics, usually sent within a certain period of time. these campaigns often target users who have something in common, such as being clients of the same organization. it would be helpful for computer emergency response teams to detect if a campaign is taking place, allowing an early response to the attacks and would enable them to alert users. while the most recent dataset for email clustering dates from 2019-2020 @xcite , we use a set of emails provided by the spanish national cybersecurity institute (incibe) consisting of spam emails from 2021. additionally, in contrast with existing proposals by other authors who used techniques such as dbscan @xcite or topic modeling @xcite , we propose a technique based on agglomerative clustering. moreover, the goal of spamclus is to group emails together if they are likely to belong to the same spam campaign, as opposed to other authors who aim to cluster emails by topic, without considering if they belong to a spam campaign. in previous work, only @xcite have used an approach based on autoencoders. transformer-based models have obtained promising performance in clustering tasks @xcite and they are able to provide information about context. thus, regarding the input for the spamclus algorithm, we use bert embeddings instead of features. finally, regarding spammer tricks present in spam emails, only @xcite considered the presence of such techniques by adding the recognition words with separate letters. in contrast, we add a pre-processing step that removes hidden text using ocr. ## experiments and results we aim at optimizing the number of clusters detected as spam campaigns while ensuring that these clusters do not mix emails of different topics. to this end, we fixed the input parameter, min_samples to 50 and calculate the min_similarty to 0.8. the min_samples was set due to the recommendation of a cybersecurity technician from incibe as the optimal number to detect a campaign. we computed the min_similarty as follows. we ran spamclus for several ranges of days taking from 14069 to 6702 samples and manually checked the content of the email clustered as spam campaigns. the objective was to identify the instances where the algorithm clusters unrelated campaigns, with the aim of maximizing the number of samples clustered as spam campaigns but avoiding clusters of mixed topics. spamclus 1 spam detection algorithmic based on agglomerative clustering min_samples min_similarity emails_df spam_df ← dataf rame[∅] initial_threshold ← compute_threshold(df [content]) emails_content ← preprocess_content(df [content]) embeddings ← bert _encode(emails_content) threshold ← initial_threshold do emails_df [cluster] ← agglomerative_clustering(embeddings, threshold) number_samples ← count_samples_per_cluster(emails_df [cluster]) cosine_similarities ← cosine_similarity_per_cluster(emails_df [cluster]) spam_df ← spam_df.append(emails_df where(@xmath0, emails_df ▷ return spam campaigns clustered and emails no considered campaigns ## discussion in addition to the aforementioned techniques, other methodologies were also evaluated. initially, a fixed-threshold approach was employed, yet encountered difficulties with each threshold utilized. when we used a very high threshold, the small finally, to avoid the problem described above, we propose the algorithm spamclus 1. with regard to the input parameters minimum number of samples and minimum similarity, it can be anticipated that we encounter similar issues min # non-spam # spam # spam mixed similarity samples samples clusters clusters? 0.9 261 1208 4 no 0.8 423 986 8 no 0.7 599 870 8 no 0.6 599 870 8 no 0.5 823 646 9 yes ## conclusions and future work in this work, we presented a baseline algorithm that addressed the problem of spam campaign detection using agglomerative clustering named spamclus 1. this algorithm avoids having a fixed threshold, which creates small clusters with high similarity or big clusters with low similarity. the output of this algorithm is two variables that contain the clusters considered spam and non-spam. this output depends on two criteria to consider a campaign: the minimum number of samples and the similarity of the emails in the cluster. we fixed the first parameter to 50 based on experts' recommendations and calculated the second to 0.8 depending on the most frequent optimal value tested on several ranges of dates emails. with those fixed values, we obtained 19 spam clusters with a total of 3048 samples for a range of three days (see in future work, we will explore new approaches to automatically compute the minimum cosine similarity depending on the emails being clustered. in addition, we might explore new options to remove hidden text from the email content because the ocr approach takes too long to extract only visible text. furthermore, the current proposal to remove hidden text from emails using ocr requires a significant computational cost. therefore, in future work, we propose to explore alternative methods for removing hidden text and preserving only the relevant email content.