title: Collaborative human-AI approach to mitigate phishing attacks
title: A collaborative human-AI approach to mitigate phishing attacks
titleType: AlternativeTitle
Creators: name: Saka, Tarini
Publisher: name: The University of Edinburgh
Year: 2025
Subjects: subject: phishing
subject: artificial intelligence
subject: AI
subject: phishing detection
subject: context-based features
subject: Phishing Codebook
subject: algorithm training
subject: online security
subject: Louvain community detection algorithm

description: Phishing involves sending fraudulent emails with links or attachments that can contain malware or trick users into revealing sensitive information or transferring money. Given the potential risks posed by phishing to both individuals and organizations, it is important to have the most advanced detection and mitigation algorithms. In this thesis, I examine the potential of combining human expertise with artificial intelligence to create hybrid defences against phishing, with a special focus on supporting two user groups (IT staff and end-users) in their email security practices. In order to gain a thorough understanding of the research area and to review existing work, I begin with a structured literature review and systemization of knowledge of prior research on grouping spam and phishing emails. The study revealed that traditional text representation techniques are primarily used, with little exploration of advanced methods such as language models, and that researchers often overlook contextual features. Therefore, I conducted a study to investigate the possibility of using unsupervised clustering techniques to group emails by utilizing a combination of contextual and technical features extracted from the emails. I found that features based on context perform well and should be more widely used in this topic. Based on the findings from these studies, I learned that phishing emails present challenges to machine learning (ML) algorithms. First, emails are unstructured documents with significant variations in length and structure, making it difficult to capture context accurately. Second, attackers often use obfuscation techniques, such as white-on-white text, to deceive both computers and humans. Third, very few studies have utilized descriptive, contextual features in emails. To address these gaps, I conducted a qualitative study to analyse the structure of phishing emails and understand the factors influencing human decision-making when assessing suspicious emails. I identified a novel set of descriptive features and created the Phishing Codebook, a framework for systematically extracting key information from such emails. Additionally, I developed a dataset of 503 annotated emails with high inter-rater reliability. Subsequently, I developed PhishCoder to automate the coding process and extract contextual information from phishing emails. I defined seven information extraction tasks based on the codebook, four of which were text classification tasks with predefined labels, and three as text extraction tasks. These tasks were performed using fine-tuned language models, employing text classification for the former and question answering for the latter. As the results showed an impressive performance, I performed experiments to study the utility of PhishCoder outputs for two tasks: identifying phishing email campaigns and providing user guidance. I used the Louvain community detection algorithm and text similarity measure on the PhishCoder outputs to detect email campaigns. Additionally, I conducted a user survey on Prolific to evaluate the impact of real-time user guidance on user phishing detection accuracy and confidence. The results demonstrated that personalized real-time guidance significantly enhanced user performance. The findings of this thesis highlight the novel ways in which artificial intelligence can be leveraged to assist users and cultivate a safer and more effective organizational environment. Furthermore, this thesis demonstrates how qualitative methodologies can offer fresh perspectives on research problems and contribute to the development of better tools to support users. Given that much of the existing phishing research focuses on preventive measures before an attack (such as phishing detection), rather than post-attack mitigation measures, this thesis introduces new directions for future researchers and security professionals.