A   RECURRENT NEURAL CASCADE-BASED MODEL FOR CONTINUOUS-TIME DIFFUSION PROCESS

1 INTRODUCTION . The recent development of online social networks enabled researchers to suggest methods to explain and predict observations of diffusion across networks . Classical cascade models , which are at the heart of the research literature on information diffusion , regard the phenomenon of diffusion as an iterative process in which information transits from users to others in the network ( Saito et al. , 2008 ; Gomez-Rodriguez et al. , 2011 ) , by a so-called word-of-mouth phenomenon . In this setting , diffusion modeling corresponds to learning probability distributions of content transmission . Various cascade models have been proposed in the literature , each inducing its own learning process to explain some observed diffusion episodes and attempting to extract the main dynamics of the network . However , most of these models rely on a strong markovian assumption for which the probabilities of next infections1 only depend on who is already infected at the current step , not on the past trajectories of the diffused content . We claim that the history of spread contains much valuable information that should be taken into account by the models . Past trajectories of the diffusion can give insights about the different natures of the contents . Also , the content may be changed during the diffusion , with different transformations depending on which nodes re-transmit the information . On the other hand some recent approaches rely on representation learning and recurrent neural networks ( RNN ) to predict the future spread of diffusion given the past . A naive possibility would be to consider diffusion episodes as sequences of infections and propose temporal point process approaches to model the dynamics . Using the Recurrent Marked Temporal Point Process model ( Du et al. , 2016 ) , the current hidden state of the RNN would embed the history of the whole diffusion sequence , which would be used to output the next infected node and its time of infection . However , since diffusion episodes are not sequences but trees , naive recurrent methods usually fail in capturing the true dynamics of the networks . Embedding the whole past in the state of a given node rather than restraining it to its specific ancestor branch leads to consider many independent and noisy events for future predictions . A model that would consider the true diffusion paths would be more effective , by focusing on the useful past . If the true diffusion paths were known , it would be possible to adapt works on recurrent neural models for tree structures such as successfully proposed in ( Tai et al. , 2015 ) for NLP tasks . Unfortunately , in most of applications the topology of diffusion is unknown while learning . In the task considered in this paper , the only observations available are the timestamps of the infected nodes . 1Throughout this paper , we refer to infection for denoting the participation of a node of the network in the diffusion . To cope with it , ( Wang et al. , 2017a ) proposed Topo-LSTM , a Long-Short Time Memory network that considers a known graph of relationships between nodes to compute hidden states and cells of infected nodes . The hidden state and cell of a given node at t depend on those from each of its predecessors that have been infected before t. Since nodes may have multiple predecessors that are infected at time t , the classical LSTM can not be applied directly . Instead , ( Wang et al. , 2017a ) proposed a cell function that aggregates infector candidate states via mean pooling . This allows to take the topology of the possible diffusion into account , but not the past trajectory of the content ( it averages all possible paths ) . To overcome this , ( Wang et al. , 2017b ) proposed a cascade attentionbased RNN , which defines a neural attention mechanism to assign weights to infector candidates before summing their contribution in the new hidden state at t. The attention network is supposed to learn to identify from whom comes the next infection based on past states . However , such an approach is likely to converge to most of attention weight vectors in the center of the simplex , since diffusion is a stochastic process with mostly very weak influence probabilities . The deterministic inference process of the approach limits its ability to produce relevant states by mixing from multiple candidates rather than sampling the past trajectories from their posterior probabilities . Note the similar approach in ( Wang et al. , 2018 ) , which does not use a RNN but defines a composition module of the past via attention networks to embed the episode in a representation space from which the probability of the next infected node can be deduced . Beyond the limits discussed above w.r.t . deterministic mixing of diffusion trajectories , no delay of infection is considered in this work , which makes it impossible to use for diffusion modeling purposes . Recently , many works in representation learning used random walks on graphs to sample trajectories that can be used to learn a network embedding which respects some topological constraints . While DeepWalk ( Perozzi et al. , 2014 ) only uses structural information , models proposed in ( Nguyen et al. , 2018 ) or ( Shi et al. , 2018 ) include temporal constraints in random walks to sample feasible trajectories w.r.t . observed diffusion episodes . The approach DeepCas from ( Li et al. , 2016 ) applies this idea for the prediction of diffusion cascades . However , such approaches require a graph of diffusion relations as input , which is not always available ( and not always representative of the true diffusion channels of the considered network as reported in ( Ver Steeg & Galstyan , 2013 ) ) . In our work , we consider that no graph is available beforehand . Moreover , no inference process is introduced in DeepCas to sample trajectories from their posterior probabilities given the observed diffusion sequences . The sampling of trajectories is performed in an initialization step , before learning the parameters of the diffusion model . In this paper , we propose the first bayesian topological recurrent neural network for sequences with tree dependencies , which we apply for diffusion cascades modelling . Rather than building on a preliminary random walk process , the idea is to consider trajectory inference during learning , in order to converge to better representations of the infected nodes . Following the stochastic nature of diffusion , the model infers trajectories distributions from observations of infections , which are in turn used for the inference of infection probabilities in an iterative learning process . Our probabilistic model , based on the famous continuous-time independent cascade model ( CTIC ) ( Saito et al. , 2009 ) is able to extract full paths of diffusion from sequential observations of infections via black-box inference , which has multiple applications in the field . Our experiments validate the potential of the approach for modeling and prediction purposes . The remaining of the paper is structured as follows . Section 2 presents some background and notations of the approach . Section 3 presents the proposed model . Section 4 reports experimental results of the approach compared to various baselines . 2 BACKGROUND . 2.1 INFORMATION DIFFUSION . Information diffusion is observed as a set of diffusion episodes D. Classically , episodes considered in this paper only contain the first infection event of each node ( the earlier time a content reached the node ) . Let U = { u0 , u1 , .... , uN−1 } be a set of N nodes , u0 standing for the world node , used to model influences from external factors or spontaneous infections ( as done in ( Gruhl et al. , 2004 ) for instance ) . A diffusion episodeD = ( UD , TD ) reports the diffusion of a given content in the network as a sequence of infected nodes UD = ( UD0 , ... , U D |D|−1 ) and a set TD = { tDu ∈ N+∞|u ∈ U } of infection time-stamps for all u ∈ U . UD is ordered chronologically w.r.t . the infection time-stamps TD . Thus , UDi ∈ U corresponds to the i-th infected node in D for all i ∈ { 0 , ... , |D| − 1 } , with |D| the number of infected nodes in the diffusion . Every episode in D starts by the world node u0 ( i.e. , UD0 = u0 for all episodes D ) . We note t D u the infection time-stamp in D for any node in U , ∞ for nodes not infected in D. Time-stamps are relative w.r.t . to tD1 , arbitrarily set to t D 1 = 1 in the data . Note that we also set tD0 = 0 for every episode D. In the following , Di = ( U D i , t D UDi ) denotes the i− th infected node in U with its associated time-stamp . Cascades are richer structures than diffusion episodes , since they describe how a given diffusion happened . The cascade structure stores the first transmission event ( u→ v ) that succeeded from any node u to each infected node v. Thus , a cascadeCD = ( UD , TD , ID ) corresponds to a transmission tree rooted in u0 and reaching nodes in UD during the diffusion , according to a sequence ID of infector indices in UD : for any j ∈ { 1 , ... , |D|−1 } and any i ∈ { 0 , ... , |D|−2 } , IDj equals i iff UDj was infected by UDi in the diffusion D ( i.e. , U D i is the infector of U D j ) . We arbitrarily set I D 0 = 0 ( no infector for the world node ) . Note that ID1 is always equal to 0 , since there is no other candidate for being the infector of UD1 than the world u0 . For convenience , we note I D ( i ) ∈ UD the infector of UDi for i ∈ { 0 , ... , |D| − 1 } . Note that the cascade structures respect that tDID ( j ) < t D UDj for every D and all j ∈ { 1 , ... , |D| − 1 } ( the infector of a node v is mandatory a node that was infected before v ) . Several different cascade structures are possible for a given diffusion episode . Cascade models usually perform assumptions on these latent diffusion structures to learn the diffusion parameters . 2.2 CASCADE MODELS . The Independent Cascade model ( IC ) ( Goldenberg et al. , 2001 ) considers the spread of diffusion as cascades of infections over the network . We focus in this paper on cascade models such as IC , which tend to reproduce realistic temporal diffusion dynamics on social networks ( Guille et al. , 2013 ) . The classical version of IC is an iterative process in which , at each iteration , every newly infected node u gets a unique chance to infect any other node v of the network with some probability θu , v . The process iterates while new infections occur . Since the expectation-maximization algorithm proposed in ( Saito et al. , 2008 ) to learn its parameters , IC has been at the heart of diffusion works . However , in real life , diffusion occurs in continuous time , not discrete as assumed in IC . ( Lamprier et al. , 2016 ) proposed DAIC , a delay-agnostic version of IC , where diffusion between nodes is assumed to follow uniform delay distributions rather than occurring in successive discrete timesteps . A representation learning version of DAIC has been proposed in ( Bourigault et al. , 2016 ) , where nodes are projected in a continuous space in a way that the distance between node representations render the probability that diffusion occurs between them . This allowed the authors to obtain a more compact and robust model than the former version of DAIC . Beyond uniform time delay distributions , two main works deal with continuous-time diffusion . NetRate ( Gomez-Rodriguez et al. , 2011 ) learns parametric time-dependent distributions to best fit with observed infection timestamps . As NetRate , CTIC ( Saito et al. , 2009 ) uses exponential distributions to model delays of diffusion between nodes , but rather than a single parameter for each possible relationship , delays and influence factors are considered as separated parameters , which leads to more freedom w.r.t . to observed diffusion tendencies . Delays and influence parameters are learned conjointly by an EMlike algorithm . Note the continuous-time cascade model extension in ( Zhang et al. , 2018 ) , which embeds users in a representation space so that their relative positions both respect some structural community properties and can be used to explain infection time-stamps of users . In our model we consider that diffusion probabilities from any infected node v depend on a latent state associated to v , which embeds the past trajectory of the diffused content . This state depends on the state of the node u who first transmitted the content to v. Therefore , we need to rely on a continuous-time model such as CTIC ( Saito et al. , 2009 ) , which serves as a basis for our work . In CTIC , two parameters are defined for each pair ( u , v ) of nodes in the network : ku , v ∈ ] 0 ; 1 [ , which corresponds to the probability that node u succeeds in infecting v , and ru , v > 0 , which corresponds to a time-delay parameter used in an exponential distribution when u infects v. If u succeeds in infecting v in an episode D , v is infected at time tDv = t D u + δ , where δ ∼ ru , v exp ( −ru , vδ ) . These parameters are learned via maximizing the following likelihood on a set of episodes D : p ( D ) = ∏ D∈D p ( D ) = ∏ D∈D ∏ v∈UD hDv ∏ v 6∈UD gDv ( 1 ) where hDv stands for the probability that v is infected at t D v by previous infected nodes in D and g D v is the probability that v is not infected by any infected node in D. We build on this in the following , but rather than considering a pair of parameters ku , v and ru , v for each pair of nodes ( u , v ) ( which implies 2 × |U| × ( |U| − 1 ) parameters to store ) , we propose to consider neural functions which output the corresponding parameters according to the hidden state of the emitter u , depending on its ancestor branch in the cascade , and a continuous embedding of the receiver v .

Summary: The problem that the paper tackles is very important and the approach to tackle it id appealing. The idea of regarding the history as a tree looks very promising. However, it’s noteworthy that embedding to a vector could be useful too if the embedding espace is representative of the entire history and the timing of the events.