Pathways to Radicalisation: On Research for Online Radicalisation in Natural Language Processing and Machine Learning (2025)

Abstract: Online communities play an integral part in communication for communication across the globe. Online communities that are known for extremist content. As a field of surveillance technologies, NLP and other ML fields hold particular promise for monitoring extremist communities that may turn violent.Such communities make use of a wide variety of modalities of communication, including textual posts on specialised fora, memes, videos, and podcasts. Furthermore, such communities undergo rapid linguistic evolution, thus presenting a challenge to machine learning technologies that quickly diverge from the data that are used. In this position, we argue that radicalisation is a nascent area for which machine learning is particularly apt. However, in addressing radicalisation research it is important that avoids falling into the temptation of focusing on prediction. We argue that such communities present a particular avenue for addressing key concerns with machine learning technologies: (1) temporal misalignment of models and (2) aligning and linking content across modalities.

## introduction internet-facilitated radicalisation is an urgent modern challenge, with links to both acts of physical violence and intangible social harms. the proliferation of online content that espouses extremist views presents a challenge for scalable content moderation and prevention of radicalization. for nlp methods to be applied for such purposes, they must take into account the nature of radicalisation and communication in fora where radicalisation occurs. first, language use in radicalised communities is highly dissimilar from standard language use in more sanitised areas of the internet due to an over-emphasis on negative rhetoric and discussions around target groups.second, communication in radicalised communities is characterized by large temporal shifts. fast-moving norms present a challenge to traditional nlp methods which remain static once they have been trained, yet hold potential for modern nlp methods under few-shot settings. moreover, these communities employ direct democracies in the governance of their policies, i.e., their members can vote for policy changes. one such example is incel.is, which frequently updates their terms and conditions to address the changing norms of their community foot_0 . while the terms and conditions often take into account community wishes, they are also accountable to laws in effect where they are legally registered. this has led to subtle distinctions between, e.g., celebrating news of someone having "gone er"-referring to having committed a mass shooting against perceived or actual group targets-and stating that you will "go er" or encouraging others to do so, where only the latter is sanctioned. third, research points to that radicalisation is a longitudinal process where data across different modalities-such as memes, podcasts, videos, and written documents-collectively act to shift opinions, beliefs, and actions towards exclusionary and violent ideologies. in this position paper, we discuss theories of radicalisation and the potential and limitations of nlp for radicalisation research. we argue that taking the different factors of radicalisation together holds several implications for nlp, urging the need to develop (1) datasets and processes that better represent conversations in such communities; (2) methods that better address the rapidly changing norms and vocabularies of radicalised communities; (3) models that take into account the multi-modal nature of radicalisation; and (4) methods for mapping and tracking shifts of opinion in a large body of multi-modal data. to our knowledge, this is the first paper to provide (i) a holistic discussion of the challenges of radicalisation research within nlp and (ii) a roadmap for how future nlp researchers can frame their research questions. ## related work defining radicalisation a challenge for studying radicalisation and extremism lies in the lack of agreed-upon definitions @xcite . different disciplines have conceptualised it according to the particular interests of the field, leading to difficulties in cross-disciplinary research. for instance, in the social sciences radicalisation research often focuses on identity formation, group dynamics, grievances, or ideological pull factors (e.g., @xcite . in contrast, political science often views radicalisation through the lens of political violence, state responses, or the dynamics of extremist movements (e.g., @xcite . psychological research, in turn, has focused on individual pathways, cognitive vulnerabilities, and the role of social influence (e.g., @xcite . the computational study of radicalisation has traditionally been situated within the information retrieval and web science communities. in these communities, the operational definition of radicalisation is often implicit as it seeks to identify patterns of behaviour. for example, @xcite use sharing of incitement material and using language from an extremism lexicon to signal radicalisation, and find that users are more likely to adopt new terminology and to interact with new users in the period before they exhibit these signals. @xcite construct a dataset of content from users who have been sanctioned on twitter for involvement with extremist movements. they use social and timing features (e.g., follower count and time between tweets) to develop methods for predicting whether nonsanctioned users will retweet extremist content or respond to engagement from extremist users. in nlp, research has similarly sought to distinguish posts from extremist web-fora and mainstream fora @xcite . while this body of work relies on language as a signal, it tends to treat language as static and do not consider context, and therefore do not provide evidence for how or why individuals adopt extremist views. nlp for radicalisation more recent work in nlp has sought to examine extremism and radicalisation in more detail. for example, @xcite and @xcite seek to the relation between extremist content and users and regular content and users. @xcite annotate radicalisation using an ordinal approach to capture different levels of extremism, @xcite seek to predict a user's eventual network centrality, their usage of lexicon terms, and the duration of their interaction with extremist communities using early engagement features. kock (2025) further develops a method for identifying extremist in-group language using social and temporal cues. importantly for nlp, recent studies have identified high propensities for linguistic innovation in extremist communities @xcite identify more than 1500 variants of the word '-cel' from the incel.is platform and mendelsohn et al. (2023 ## towards machine learning and nlp for radicalisation studies given the abundance of data that can constitute as relevant to processes of radicalisation, advanced pattern recognition methods hold potential for easing research into radicalisation, particularly in academic settings where large resource constraints exist. however, contemporary pattern recognition systems may need further development to realise their potential. in this section, we discuss why machine learning and nlp systems may be of service to research, and outline the challenges that have yet to be resolved by the research community. ## recommendations we now turn to presenting recommendations for the challenges for using nlp for radicalisation. treat radicalisation as a process radicalisation is a process unfolding over time. yet, prior research-which focuses on classifying posts as "radicalising" or "extremism-promoting"obscures this. we argue that research should seek to identify and analyse how the radicalism of users shifts over time, instead of identifying individual "radicalising posts". this could include identifying radicalising events for a particular user's journey, and identifying indicators that a particular user has "drifted" into radicalism. nlp techniques from parallel tasks, such as mental health monitoring, can be repurposed-e.g., temporal change point detection @xcite , timeline extraction @xcite , or longitudinal personalised language modelling for social media users @xcite . account for temporal drift if the aim is to study change over time, models must be able to incorporate information from different points in time. as we discussed in section 3.2, the language used in radical communities varies greatly over time. this includes the introduction of new lingo, changes in behavioural norms, and events and topics the community discuss. however, traditional nlp models are trained on static snapshots of discussion in communities, and may not adapt well to rapid linguistic shifts @xcite . we argue that new models should be built which are able to quickly adapt to new language and norms, using e.g. specialised architectures @xcite or metalearning @xcite . model network structure like posts, users cannot be modelled properly in isolation. users interact with other members of the community, and modify their behaviour based on these interactions. radicalisation journeys intersect, and users mutually drive radicalisation. further, content across sites is reposted, repurposed, referenced, and used as the basis for new content. we argue therefore that models should account for network structure when attempting to predict user journeys, for example through the use of graph-based models @xcite . account for non-textual content external events-and the discussion and reference to them in videos, podcasts, and memes-are watched and shared in extremist communities and often act as key points in radicalisation journeys @xcite @xcite . for this reason, it will be vital that the research community develops robust automatic speech recognition, vision, and vision-language models to account for the nontextual content that is shared on extremist platforms @xcite . ## towards a framework for radicalisation research here, we turn to presenting a proposal for how research on radicalisation may be actualised. ## conclusion in this paper, we have introduced and discussed challenges in the nascent field of nlp for radicalisation research. we argue that while nlp technologies present an opportunity for radicalisation research, the nature of radicalisation-i.e., a longitudinal process where influence is manifested through multiple modalities-presents challenges for existing nlp methods which require new approaches to model processes of radicalisation across data from different modalities. to this end, we provide recommendations for future work in nlp for radicalisation and propose a framework for radicalisation research in nlp. we hope that our consideration can further encourage work in the field of nlp researching online radicalisation.