Distilling Meta Knowledge on Heterogeneous Graph for Illicit Drug Trafficker Detection on Social Media

1 Introduction . As the market of illicit drugs ( e.g. , heroin , synthetic opioids such as Fentanyl ) is considerably lucrative , the crime of drug trafficking ( a.k.a . illicit drug trading ) has never stopped but co-evolved with the advance of modern technologies , e.g. , the practice of illicit drug trade has transformed from the physical world to online platforms . It has been shown that major social media platforms , including Instagram , Twitter , Snapchat , and Facebook , have become direct-to-consumer marketing mediums for illicit drug traffickers [ 25 ] . As illustrated in Figure 1 , due to the convenience and popularity , drug traffickers can easily create accounts on these platforms to advertise and sell drugs by posting codewords ( e.g. , street name ) and images of drugs that are in stock ( shown in Figure 1 . ( a ) ) ; they can also post their drug sale websites and utilize social media platforms for promotion ( shown in Figure 1. ( b ) ) . Buyers can access illicit drugs and trade with vendors easily through these social media platforms . ∗Corresponding authors . 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . Illegal drug trading has turn into a global concern due to its catastrophic consequences on society , from violent crimes to public health ( e.g. , over 87,000 people died from drug overdose in 2020 in the U.S. and it broke the record with a 20 % increase in a year [ 41 ] ) . Despite the persistent effort by law enforcement agencies , due to the considerable profits , activities of online drug trafficking are nimble and resilient : as shown in Figure 1 . ( c ) , to avoid being banned , skilled drug traffickers ( e.g. , with 1,002 followers , and 118 followings ) rarely post drug description or images on social media , but instead , they advertise drugs implicitly using the slang ( e.g. , using “ KET ” to refer ketamine ) and leave encrypted chat tool contact information ( e.g. , Kik Messenger ) through their comments to other users ’ posts . To combat online drug trafficking , there is an imminent need to develop novel techniques for effective detection of drug traffickers on social media and thus enable law enforcement for proactive interventions to disrupt and dismantle illicit drug trades . To tackle the above problem , there have been many research efforts on the investigation of online drug trafficking . Some existing works focus on analyzing the drug trafficking activities on darknet markets [ 46 , 47 , 9 ] , while some methods have been proposed for detecting illicit drug trading activities on social media [ 20 , 21 , 22 ] . Most of these works usually rely on a single type of content feature ( e.g. , text or image ) while ignoring the structured relationship information among entities on social media . For instance , Instagram monitors drug traffickers simply by filtering drug-related keyword hashtags ( e.g. , weed4sale ) [ 14 ] . Unfortunately , as illustrated in Figure 1 . ( c ) , drug traffickers may always invent new tactics to evade detection . Those methods that fail to consider multi-modal features ( e.g. , both text and image ) and the structural relationships among entities ( e.g. , users and posts ) could limit the effectiveness of detecting drug trafficking activities . Additionally , most recent works [ 22 , 23 , 20 ] require sufficient labeled samples to train the model for the detection of drug trafficking . For example , Mackey et al . [ 23 ] collected almost 612,000 posts on Twitter and only found 1,778 drug trafficking-related posts . Obtaining labeled samples on social media is always expensive , and existing models may suffer from the constraint of few labeled data to detect drug traffickers . To address the above challenges , in this paper , we propose a holistic framework named MetaHG ( as shown in Figure 3 ) to automatically detect illicit drug traffickers on social media ( i.e. , using Instagram as a showcase ) . First , to overcome the shortcomings of merely analyzing a single type of content information ( e.g. , text or image ) , we propose to consider the multi-modal content features ( i.e. , both text and image ) and introduce a heterogeneous graph ( HG ) to model the relationships among three types of entities ( i.e. , user , post , and keyword ) . The constructed HG is able to comprehensively characterize the complex ecosystem of drug trafficking on social media . And then we exploit relationbased graph convolutional neural networks ( R-GCNs ) [ 32 ] to fuse both relational information among entities and content features to obtain initial node embeddings over the built HG . Due to spare relational connections among some entities , we further introduce graph structure refinement ( GSR ) [ 24 , 16 ] to learn more robust node representations on the HG . Furthermore , to address the few labeled data issue , we propose a meta-learning framework to transfer knowledge from training tasks and effectively adapt them to testing tasks ( e.g. , new types of drug traffickers with few labeled samples ) . More specifically , to exploit unlabeled information for better performance , our proposed model consists of two additional modules : an embedding similarity-based self-supervised learning module to augment R-GCNs for node representation refinement , and a knowledge distillation module to facilitate better model optimization . In summary , the major contributions of our work include : • We study the illicit drug trafficker detection on social media and create a new dataset on Instagram ( including text , image and relation information ) for the problem , which is novel and urgent . • To solve the problem , we develop a holistic yet novel framework ( i.e. , MetaHG ) to jointly model both structured relations and unstructured content information for illicit drug trafficker detection , accounting for the constraints of graph sparsity and limited labeled data for model training . • Comprehensive experiments on the real-world dataset demonstrate the outstanding performance of MetaHG by comparison with the state-of-arts methods . To the best of our knowledge , MetaHG is the first work that automatically detects drug traffickers on social media using few labeled data . • As drug overdose deaths have continued to increase over the past decade across the country , our proposed technique will have a significant societal impact to help address this critical issue . 2 Related Work . In this section , we briefly introduce relevant studies in three aspects : drug trafficker detection , graph neural networks , and meta-learning . Drug Trafficker Detection . Most existing works analyze drug traffickers on darknet markets [ 2 , 3 , 46 , 47 , 48 ] . For instance , Zhang et al . [ 46 ] developed a system by leveraging user ’ s styles to detect drug traffickers on darknet markets . As drug trafficking appears on the surface web dramatically recently , some recent works [ 20 , 22 , 23 ] detect illicit drug traffickers on social media platforms by analyzing the single type of text information or image information of posts . For example , Li et al . [ 20 ] used a RNN model to learn the text pattern of posts to detect drug dealers on Instagram . However , these methods seldom consider multi-modal features and the relational information among entities . Different from existing works , we detect these illicit drug traffickers on social media by incorporating structural relation and unstructured multi-modal content information . Graph Neural Networks . Most existing graph neural networks ( GNNs ) learn the node embedding by aggregating the features of neighborhood nodes [ 12 , 17 , 39 , 44 ] . GCN [ 17 ] implements layer-wise propagation rule to learn the node embedding , and GAT [ 39 ] proposes to learn different attention scores for neighbors when aggregating neighborhood information . However , most of these existing models work with homogeneous graphs . Some recent models [ 32 , 34 , 43 , 45 ] are proposed to deal with heterogeneous graphs which are more practical in reality . Inspired by GCN , Weighted-GCN [ 34 ] and R-GCNs [ 32 ] propose to learn node embedding based on multiple relational neighborhoods . Additionally , inspired by GAT , HAN [ 43 ] leverages the attention mechanism to learn the importance between nodes and meta-path simultaneously in heterogeneous graphs . Motivated by these studies , we employ R-GCNs as our base model to learn the node embedding in HG . Meta-Learning . Recent meta-learning models generally can be divided into two groups : metricbased meta-learning [ 4 , 18 , 28 , 36 , 40 ] and gradient-based meta-learning [ 1 , 8 , 10 , 11 , 19 , 27 ] . For metric-based models , they implement a generalized metric and matching functions from training tasks to train the model . For instance , Matching Networks [ 40 ] learns a network that maps the labeled support set and an unlabelled example to the label in support set . Prototypical networks [ 36 ] learns a metric space in which classification can be performed by computing distances to prototype representations of each class . For gradient-based models , they employ existing tasks data to learn well initialized model parameters that can be fast updated to new tasks with few data [ 11 ] or directly implement a meta-optimizer to learn the optimization process [ 38 ] . For example , Finn et al . [ 10 ] proposed MAML , in which the parameters of the model are explicitly trained such that several gradient steps with few training data from a new task will produce good generalization performance on that task . In this paper , motivated by MAML , we introduce a meta-learning framework to address small labeled data challenge of illicit drug traffickers detection . 3 Preliminary . 3.1 Problem Definition . Given a social media dataset , we build a heterogeneous graph ( HG ) to comprehensively depict the rich information among the dataset for learning the user embedding . Let G = ( V , E , X ) denote a HG , where V is the set of different types of nodes , E ⊆ V × V is the set of edges , and X is the node attributes ( features ) set . Specifically , there are three types of nodes ( user , post , and keyword ) and eight types of relations ( e.g. , user-reply-post ) in HG ( see Figure 2 ) . A node can be regarded as any type of entities and an edge can be regarded as any type of relations between two nodes . The node attribute can be considered as the feature applied to the node . The goal is to learn the user embedding which can be fed to a classifier for drug trafficker detection . Given the features of all user nodes Xr = ( x1 , . . . , xN ) ( N : number of users ) and their binary labels Y = ( y1 , . . . , yN ) ( yi = 1 denotes drug trafficker and yi = 0 represents regular user ) , we target learning a mapping function ( detection function ) Uϕ : Xr → Y ( with parameter ϕ ) . Unlike existing works that use sufficient samples for model training , as the number of drug trafficker samples collected from social media is limited , we consider a more practical scenario that only few labeled data are available . Following the definition of few-shot learning [ 10 ] , given the labeled data of existing different types of drug traffickers ( e.g. , stimulants trafficker ( refer to the Supplementary Material 1.1 ) ) , we aim to build a classifier that can effectively detect drug traffickers of new types with few labeled data ( e.g. , opioid trafficker ) . Formally , the problem is defined as follows . Problem 1 . Drug Trafficker Detection on Social Media . Given a social media data denoted as HG G and a set of different types of drug traffickers with their features Xr = ( x1 , . . . , xN ) and corresponding labels Y = ( y1 , . . . , yN ) ( training data ) , the problem is to build a machine learning model to detect illicit drug traffickers of new types that only have few labeled samples ( testing data ) .

Summary: Summary:  The authors propose a model to detect drug-trafficker profiles on Instagram. The model consists of many steps and phases, but in short: they create a heterogeneous graph from users, keywords, and posts, and in addition to the regular cross-entropy loss they propose to add multiple regularizer terms extracted from several stages of their algorithm (i.e., a term to build the graph, a term to learn the node representations, and a term to distill the knowledge from an out-of-domain model).  To evaluate their model, the authors created a dataset and compared their model with many baselines and their combinations. They also reported several experiments, including user visualization and a case study.