Clinical Risk: wavelet reconstruction networks for marked point processes

1 INTRODUCTION . Clinical risk scores are commonly used analytic devices in health care . There are risk scores for predicting strep throat from sore throats ( Centor et al. , 1981 ) , mortality from vital signs ( GardnerThorpe et al. , 2006 ) , heart attacks from routine clinic visits ( D ’ Agostino et al. , 2008 ) , and many more . Policy is implemented around these risk scores , from rates of reimbursement to physician compensation ( Asch et al. , 2015 ) . When used for early warning , risk scores have been associated with reduced mortality ( Seymour et al. , 2017 ) . Underlying these approaches is the formulation of risk over time given some set of features . For example , in Cox models , a study time t0 = 0 is defined and the risk model is λ ( t ; t′ , x ) , where x are defined by features timestamped before time t′ = t0 , i.e. , a time invariant model . In Hawkes processes , the risk model is also λ ( t ; t′ , x ) but where x contains all history up to time t′ = t , i.e. , a nowcasting model . Cox and Hawkes models can be limited by their assumptions , which often are inappropriate for the health care setting , and include , for example , the assumptions of proportional hazards and summation over kernel activations . We formulate a point process model to address these limitations and develop multi-forecasting , the forecasting task across the two dimensions of time : t′ and t. To motivate our specific formulation , consider the limitations of the Hawkes process in health care . First , the Hawkes process encodes an additive relationship of change in rate from recurring precursors , i.e. , burstiness , whereas in health care , the repeated measurement of an event beyond the first , say of glucose , might be irrelevant . Second , clinical event timing may be routine , scheduled , or emergent , which suggests that kernel learning will improve model performance because changes in the rate may be time-dependent and not immediate . Third , clinical event processes are marked , with marks that could be categorical , real , or null values : e.g. , bacterial culture : staphylococcus aureus , glucose : 200 , and ketoacidosis : NA Our model addresses each of these limitations . To address the first limitation , summation , we adopt a reduction layer where we allow for reductions other than “ sum ” of kernel contributions from recurring events . To address the second and third limitations , non-specific timing and lack of marks , we propose a kernel learning method over one dimension ( time ) and two dimensions ( time-value ) using wavelet reconstructions . The motivation for wavelets is illustrated in Figure 1 , where a discrete wavelet reconstruction encodes the relationship of time-delayed events identified through cross-correlation . While cross-correlations capture all relative timings , many may be spurious or coincidental . The wavelet representation of relative timing instead is learned through likelihood optimization . To capture the effect of the value distribution of marks on events , two-dimensional wavelet reconstructions are value indexed , producing a one-dimensional reconstruction that is passed forward to the reduction layer . We show that we can encode the wavelets , the relative-to-absolute mapping , and the reduction step on a computation graph to conduct learning . We apply our model first in simulations of heart attacks and scheduled hemoglobin A1c checks . We then evaluate the model in a real cohort to forecast complications and adherence to clinical participation in patients with diabetes . Empirically , we show that our model has improved performance over baselines in single forecasts and explore its performance in multi-forecasts in terms of prediction and representation . Our results provide benefit for diabetes risk assessment . Whereas the central focus of the diabetes adherence literature is on adherence to medication and therapeutic regimen ( García-Pérez et al. , 2013 ; Edelman and Polonsky , 2017 ) , our method provides a lens to investigate adherence to continued diabetes care participation , which is important because , for one , participation is associated with better diabetes control ( Schectman et al. , 2008 ) . Additionally , forecasts of complications and adherence to participation , at different times in the care process and at different forecast distances , are central concerns in clinical decision-making . For example , many diabetes medications require regular clinical monitoring , and the risk of complications and noncompliance to clinical participation both affect regimen choice . By focusing on these outcomes , our forecasting tools provide relevant information for the clinical decision-making process . Related work . Both neural networks and Hawkes process variants are used for rate modeling in health care : a few recent examples include Choi et al . ( 2015 ) , Du et al . ( 2016 ) , Alaa et al . ( 2017 ) , and Bao et al . ( 2017 ) . To effectively model the hazard two key properties are used : ( 1 ) the ability of Hawkes processes to capture relative timing of interdependent events , and ( 2 ) the flexible functional forms of neural networks that are able to capture relative timings , albeit somewhat opaquely . The closest work is likely that of Bao et al . ( 2017 ) , where the authors adopt dyadic influence functions . However in that work marks are not used and the dyads selected are a subset of the Haar wavelet basis . Outside of health applications , related literature includes Hawkes kernel learning , e.g . Zhou et al . ( 2013 ) ; Linderman and Adams ( 2014 ) ; Lee et al . ( 2016 ) . Our method follows these approaches , instead using a wavelet representation across value and time to capture long-range dependencies . Compared to Mei and Eisner ( 2017 ) that generalizes the Hawkes process with neural networks , our method ( 1 ) allows events to have marks , ( 2 ) enables forecasting and multi-forecasting ( not just nowcasting ) , and ( 3 ) can be seen to have a generalized linear form . Our method of mapping relative timing hazard components onto absolute time possesses the advantages of the Hawkes approach and adopts a simple but well-performing neural network architecture . To achieve the mapping , a stepwise hazard approximation is made , as is done in Jing and Smola ( 2017 ) and Weiss ( 2017 ) , however , instead of LSTMs and forests that have challenging interpretations , our method remains interpretable for small data sets , where exploration with visualization similar to that of Caruana et al . ( 2015 ) can be performed . Our method uses wavelets to represent event contributions , and several survival analysis approaches have also adopted them in univariate models , e.g. , in Antoniadis et al . ( 1994 ) and Brillinger ( 1997 ) . Outside of survival analysis and point processes , wavelet-inspired neural networks have seen success , with Wave-net using wavelets to classify time series ( Bakshi and Stephanopoulos , 1993 ) , and Wavenet adopting a multi-layer hidden neural architecture to connect distant time steps ( Van Den Oord et al. , 2016 ) . Contributions . The contributions of this work are as follows : our work generalizes multivariate Hawkes processes to allow for non-additive event rate relationships . Like other works , e.g . Linderman and Adams ( 2014 ) , our work learns the kernel function that relates multivariate event histories to the rate . However , in our work we use wavelets as the kernel , akin to a multivariate development from Brillinger ( 1997 ) . We leverage the scaling property of wavelets to formulate a regularization that balances spatiotemporal generalizability with deterministic or near-deterministic event timing . Unlike many sequence models , e.g . Hochreiter and Schmidhuber ( 1997 ) , which are affected strongly by choice of time step , our work adopts an absolute and relative time frame , and therefore the granularity of the absolute time domain need not be determined a priori . Additionally , unlike some point process formulations , our work models marks that are 1-d ( event times ) and 2-d ( event times and their category or real value ) . Our work is explicit about types of forecasting tasks , and the methods are adapted for several purposes : forecast performance , interrogability , and representation . We show that our method performs as well as or better than comparison algorithms in predicting complications and forecasting adherence . 2 BACKGROUND . Let E be the set of events with target event y ∈ E the event we want to forecast . Associated with each event e is a value v ∈ V . An example consists of a sequence of ( time , event , value ) tuples and a period of interest for forecasting . For the n-th example , n ∈ { 1 . . . N } , define Tn as the number of tuples . Then the sequence can be written as ( tin , ein , vin ) for i ∈ { 1 . . . Tn } , with the period of interest denoted as τny . Let λy ( t ) be the rate functions of interest , dropping the subscript n for ease of notation . The multivariate Hawkes process can then be written as follows : λy ( t ) = λ0 ( t ) + |E|∑ e=1 βe T∑ i=1 ge ( t− ti ) 1 ( ti < t , ei = e ) where λ0 ( t ) is a baseline population rate function , ge ( · ) is a kernel function for event e relating its effect on the rate of y , βe are event-specific parameters , and 1 ( · ) is the indicator function . Typically ge ( · ) is an event-specific exponential decay function with a learnable decay parameter . Self-exciting processes are defined by gy ( · ) > 0 , bursty processes by ge ( · ) > 0 , and inhibitory processes by ge ( · ) < 0 . A few recent variations include Linderman and Adams ( 2014 ) where ge is a Bayesian graph kernel and Xu et al . ( 2017 ) where ge is an infectivity function and triggering kernel product . Given λny ( t ) , the log likelihood of the data is : LL ( X|θ ) = N∑ n=1 ( Tny∑ i=1 log λny ( tiny ) + ∫ τny λny ( t ) dt ) ( 1 ) The form of the Hawkes process is limiting , however , because ( 1 ) the effect of ge ( · ) decays over time , ( 2 ) the effect over ge ( · ) is additive , ( 3 ) the value associated with each event is not considered , and ( 4 ) the time restriction in the indicator function implies nowcasting ( 1 ( ti < t ) ) not forecasting ( 1 ( ti < t− c ) for some c > 0 ) . Making modifications to achieve these characteristics is desirable to effectively model many real-world processes . For example , a patient with new-onset diabetes schedules an appointment with a typical gap interval of 3 months , and the presence of 1 or 10 elevated readings may not affect the timing of the scheduled appointment . The former suggests the utility of kernel learning , and the latter suggests summation over ge is not the appropriate reduction . The proposed method addresses these concerns . In particular , we adopt discrete wavelet reconstructions , which both allows kernel learning and the use of marks . Additionally , by representing the point process as a neural network , we are able to ( 1 ) use maximization alongside summation in a reduction layer ( the formulation enables specification of any number of reductions ) , and ( 2 ) conduct time-dependent censoring . We formalize the model below .

Summary: This paper centers around efficient estimation of the kernel function for the Hawkes process and relaxation of the “linearity” assumption in the original Hawkes process. They rely on a classical sparse generalized linear model using the wavelet basis set and Hawkes loss function to estimate a shallow kernel function. This approach is opposite to the deep function estimation approach which does not rely on a predefined basis set [e.g. see [Du et al, 2016]]. However, it can have an advantage that the learned functions are interpretable, thought the authors never demonstrate it in the paper.