Integrating linguistic knowledge into DNNs: Application to online grooming detection

1 INTRODUCTION . Online grooming ( OG ) is a communicative process of entrapment in which an adult lures a minor into taking part in sexual activities online and , at times , offline ( Lorenzo-Dus et al. , 2016 ; Chiang & Grant , 2019 ) . Our aim is to detect instances of OG . This is achieved through binary classification of whole conversations into OG ( positive class ) or neutral ( negative class ) . This classification requires the ability to capture subtleties in the language used by groomers . Corpus Linguistic ( CL ) analysis provides a detailed characterisation of language in large textual datasets ( McEnery & Wilson , 2003 ; Sinclair , 1991 ) . We argue that , when integrated into ML models , the products of CL analysis may allow a better capture of language subtleties , while simplifying and guiding the learning task . We consider two types of CL products and explore strategies for their integration into several stages of DNNs . Moreover , we show that CL knowledge may help law enforcement in interpreting the ML decision process , towards the production of evidences for potential prosecution . Our text heavily uses slang and sms-style writing , as many real-world Natural Language Processing ( NLP ) tasks for chat logs . Text normalisation methods were proposed to reduce variance in word choice and/or spelling and simplify learning , e.g . ( Mansfield et al. , 2019 ) for sms-style writing . However , they do not account for the final analysis goal and may discard some informative variance , e.g . the use of certain forms of slang possibly indicative of a user category . CL analysis provides with the preferred usage of spelling variants or synonyms . We propose to use this domain knowledge to selectively normalise chat logs while preserving the informative variance for the classification task . As demonstrated by the CL analysis in ( Lorenzo-Dus et al. , 2016 ) , the theme and immediate purpose of groomer messages may vary throughout the conversation , in order to achieve the overarching goal of entrapping the victims . Groomers use a series of inter-connected ” sub-goals ” , referred to as OG processes here , namely gaining the child ’ s trust , planning activities , building a relationship , isolating them emotionally and physically from his/her support network , checking their level of compliance , introducing sexual content and trying to secure a meeting off-line . The language used within these processes is not always sexually explicit , which makes their detection more challenging . However , CL analysis additionally flags some contexts associated to the OG processes , in the form of word collocations ( i.e . words that occur within a same window of 7 words ) that tend to occur more frequently in , and therefore can be associated with , OG processes . We propose to exploit the relations between the OG processes and their overarching goal of OG to improve the final OG classification . We use the CL identified context windows to guide the learning of our DNN . Our main contributions are : 1 ) We explore different strategies for integrating CL knowledge into DNNs . They are applied to two architecture types and demonstrated on OG detection , but may generalise to other NLP applications that involve digital language and/or complex conversational strategies . 2 ) The principle and several implementations of selectively normalising text through modifying a word embedding in support to classification . 3 ) The decomposition of conversation analysis into identifying sub-goals . Our DNN implicitly models the relations between these sub-goals and the conversation ’ s overarching final goal . 4 ) A new attention mechanism for LSTM based on the direct stimulation of its input gates , with two proposed implementations . 5 ) A state-of-the-art ( SoTA ) and interpretable OG detector . 6 ) A new corpus for OG detection , to be publicly released on demand , and that extends PAN2012 with more conversations and with products of CL analysis . 2 RELATED WORK . Villatoro-Tello et al . ( 2012 ) detected OG chat logs using a DNN to classify binary bag-of-words . This simple approach highlights the importance of commonly used words amongst groomers which we exploit for selective text normalisation . This is emphasised in ( Vartapetiance & Gillam , 2014 ; Hidalgo & Dı́az , 2012 ) where a set of phrases are derived from the important features of a Naı̈ve Bayes classifier to describe common behaviours among groomers . Liu et al . ( 2017 ) obtained the current OG detection SoTA using a word embedding for semantic of important words and an LSTM . Integrating domain knowledge into DNNs is often done with additional losses that assist with sparse and low quality data . ( Muralidhar et al. , 2018 ) penalise a DNN ’ s output violating logical rules w.r.t . the input features . ( Hu et al. , 2018 ) use the posterior regularisation framework of ( Ganchev et al. , 2010 ) to encode domain constraints for generative models . A teacher-student architecture in ( Hu et al. , 2016 ) incorporates first-order logic rules to create an additional loss for the student network . Other works integrated prior knowledge in the design of the DNN architecture . In BrainNetCNN ( Kawahara et al. , 2017 ) , the convolutions of a convolutional neural network ( CNN ) are defined based on the graph data ’ s locality to account for the brain ’ s connectivity . The training procedure may also integrate priors without modifying the DNN ’ s architecture . Derakhshani et al . ( 2019 ) use assisted excitation of CNN neurons in the images ’ areas of interest , thus providing both localisation and semantic information to the DNN . An attention mechanism was used in a supervised way to focus a DNN on important words in ( Nguyen & Nguyen , 2018 ) . We experiment with these various approaches and adapt them to our domain knowledge and DNN architectures . Linguistic knowledge was integrated to learnt word embeddings in the past . Knowledge in the form of lexicons , that carry a manual categorisation and/or ranking of words , is combined with a learnt word embedding in ( Margatina et al. , 2019 ) . Three strategies are proposed , namely concatenating the lexicon and embedding features , and using the lexicon features to conditionally select or transform the word embeddings . In our study , we are concerned with a different type of linguistic knowledge . However , our modification of word embedding ( Section 4.1 ) may also exploit this lexicon knowledge . 3 AUGMENTED PAN2012 DATASET . PAN2012 ( Inches & Crestani , 2012 ) is a standard corpus for OG detection . It was gathered from Omegle ( one-to-one conversations , IRC ( technical discussions in groups ) , and the Perverted Justice ( PJ ) website1 ( chat logs from convicted groomers interacting with trained adult decoys ) , with 396 groomers and 5700 / 216,121 OG / non-OG conversations . Some non-OG chat logs contain sexual wording , making the OG classification more challenging . Conversations are truncated to 150 messages each , which limits both CL and ML analyses . To resolve this limitation , we augment the corpus with full OG conversations and the addition of new groomers from PJ , totalling 623 groomers in 6204 OG conversations ( same negatives which could not be augmented to fuller conversations due to no access to the original data ) . Final OG / non-OG conversations total an average ( std ) of 215 ( 689 ) / 13 ( 23 ) messages and 1010 ( 3231 ) / 94 ( 489 ) words , respectively . Statistics on the dataset content are in the sup . materials . PJ data is freely available online and was largely used in previous social science and NLP studies , thus its use does not raise any peculiar ethical concern . For a debate on its usability see ( Chiang & Grant , 2019 ; Schneevogt et al. , 2018 ) . Our dataset also includes the results of a CL analysis of the new corpus using the method described in ( Lorenzo-Dus et al. , 2016 ) , which involves a heavy use of manual analysis by CL experts . As part of data preparation for CL analysis , word variants are identified , which are either spelling variations ( mistakes or intentional e.g . ‘ loool ’ → ‘ lol ’ ) , or the same semantic meaning behind two terms ( e.g . ‘ not comfy ’ → ‘ uncomfortable ’ ) . These variants are not specific to OG , but rather reflect digital language , 1http : //perverted-justice.com and are therefore valid for other real-world chat logs . The CL analysis also identified the variants that are most used among groomers . The CL products in our dataset include : 1 ) the set of variants both general and groomer-preferred , 2 ) a set of frequent 3-word collocates ( not necessarily direct neighbours , but located within a window of 7 words ) that are used among many different users , and 3 ) a manual annotation of 2100 samples of OG processes ( there are 7 types of OG processes , as identified in ( Lorenzo-Dus et al. , 2016 ) and listed in the introduction and detailed in the sup . materials ) that could be associated to 3-word collocates and the context windows that these latter define . These CL products are sensitive data that might be used to help groomers refine their strategies , therefore they will only be shared on request . They are used in Sections 4-5 to train a DNN model , but this model does not require CL analysis to be performed at testing phase , as it takes raw text only as input . 4 METHODOLOGY . Overarching vision and general applicability – We integrate two CL priors into DNNs : the word variants and the identification of OG processes . Word variants provide knowledge of same semantic meaning , which allows reducing variance in the text . The knowledge of groomer ’ s preferred variants brings an implicit and selective text normalisation that supports the classification task . It is achieved through a reduction of distances between non-discriminative variants in a word embedding . This selective normalisation is applicable to other classification tasks from real-world chat logs , provided an updated selection of the preferred and discriminative variants . As highlighted in Section 3 , the variants reflect digital language and are relevant to different analyses of chat conversations . The selection of discriminative variants is done easily and automatically following a procedure described in Section 4.1 using empirical occurrences in positive and negative conversations . This knowledge integration is also applicable to all DNNs that use a word embedding to capture word semantic . The use of OG processes aides in differentiating between causal conversations involving sexual language , and OG conversations with complex strategies and sub-goals ( i.e . OG processes ) . The language associated to OG processes , reflected by the 3-word collocates and context windows that they define , may be more informative than traditionally used simple sexual wording in making this distinction . We propose 3 strategies to integrate this knowledge , namely the definition of sub-tasks and two stimulations of DNN attention . They all guide the learning by providing focus on contexts of interest ( a valuable complement to attention mechanisms , as demonstrated in our experiments ) , and by implicitly modelling the relation between sub- and final goals . This CL knowledge integration principle is generally applicable to the analysis of complex conversations , provided an appropriate CL identification of the conversation ’ s sub-goals and of their associated language through context windows . This identification of sub-goals has been the focus of many social science studies . For example , a large corpus of works have identified strategies for persuasion and manipulation in extreme ideology groups , e.g . ( Brindle , 2016 ; Nouri & Lorenzo-Dus , 2019 ; Lorenzo-Dus & Nouri , 2020 ; Saridakis & Mouka , 2020 ) for radical right hate speech and ( Baker et al. , 2021 ) for jihadi radicalisation . This established baseline of knowledge may be integrated into DNNs in multidisciplinary works . The identification of frequent 3-word collocates is automated , as described in ( Lorenzo-Dus et al. , 2016 ) . The association of their occurrences to the identified sub-goals is the only task that may require additional manual work . Our stimulation of DNN attention may also be more generally used to focus a DNN ’ s attention on a priori known important elements of a training set . Base models – We demonstrate the general applicability of our CL integration strategies by applying them to two DNN architecture types representative of the two NLP standards of recurrent and transformer models . The recurrent DNN of Liu et al . ( 2017 ) is the current SoTA for OG classification . It comprises a language model that builds two word and sentence embeddings , and an OG classifier with two LSTMs and a linear projection . Our base model # 1 is a modified version ( Fig . 1 left ) with the word embedding provided as input to the OG classifier in place of the sentence embedding . This word embedding will be more directly impacted by our CL integration , and it increases explainability as will be seen next . It may be replaced by similar embeddings , and we also present results using the pre-trained GloVe ( Pennington et al. , 2014 ) . Further , to compensate for the loss of sentence structure modelling previously provided by the sentence embedding , and to account for the longer sequences of inputs into the classifier , we add an unsupervised attention mechanism ( Luong et al. , 2015 ) into the classifier . Following the method in ( Luong et al. , 2015 ) , the hidden states of the last LSTM for all words of the conversation are provided to the attention mechanism that outputs a conversation embedding of the same size as the LSTMs hidden state , namely 256 . XLNet ( Yang et al. , 2019 ) is a popular transformer model , a SoTA for many NLP tasks , and therefore a strong baseline for this study . It iteratively refines word embeddings , starting from an initial embedding that captures word semantic similarly to that of Liu et al . ( 2017 ) , and attaining richer word representations that account for word relationships within a sentence using a positional embedding and self attention layers . The refined contextualised word embeddings are classified by linear projection . In our application , this projection fails to handle our class imbalance and always outputs the same class with F-score at 0.392 . Providing the contextualised word embeddings to a two-layer LSTM , whose last hidden state is used as a conversation embedding to be classified by the linear projection , solves this issue2 and forms base model # 2 ( Fig . 1 right ) . The combination of a transformer model with LSTM is not new , see for example ( Ma , 2019 ) , and has the advantage of allowing the use of our LSTM-based knowledge integration strategies ( see ‘ Stimulating LSTM input gates ’ in Section 4.2 ) . Input to the models – The analysis is performed on whole conversations , and the final OG / non-OG classification is obtained for the whole conversation , rather than per-message . Messages are separated by the [ SEP ] token , so that inter-text representations can be modelled . Messages from both users are included with no distinction . For base model # 2 , the [ CLS ] token is added at the beginning of conversations following the XLNet standard . Conversations longer than 2,000 words are truncated to retain their end part ( 12 % / 8e-5 % of OG / non-OG conversations ) . All base and CL-augmented DNNs take raw text as input only . The only text preparation prior to the DNN is tokenisation of named entities . We do not apply explicit text normalisation such as ( Mansfield et al. , 2019 ) as part of text preparation , since the methodological premise of the paper is the design of a hybrid approach where an ML model incorporates its own text normalisation informed by CL knowledge .

Summary: This work proposes the approach of integrating priors into a DNN in the form of Linguistic sub-models that capture characteristics of OG. The authors use the example of the PAN-12 dataset for sexual predators to use information about linguistics behaviour for the grooming phases. The work then goes to highlight the augmentations that are done on baseline DNN models to include these CL characteristics. The authors then go on to show the impact of these augmenations on performance of classification on the PAN-12 dataset.