Exploiting structured data for learning contagious diseases under incomplete testing

One of the ways that machine learning algorithms can help control the spread of an infectious disease is by building models that predict who is likely to get infected making them good candidates for preemptive interventions . In this work we ask : can we build reliable infection prediction models when the observed data is collected under limited , and biased testing that prioritizes testing symptomatic individuals ? Our analysis suggests that when the infection is highly contagious , incomplete testing might be sufficient to achieve good out-of-sample prediction error . Guided by this insight , we develop an algorithm that predicts infections , and show that it outperforms baselines on simulated data . We apply our model to data from a large hospital to predict Clostridioides difficile infections ; a communicable disease that is characterized by asymptomatic ( i.e. , untested ) carriers . Using a proxy instead of the unobserved untested-infected state , we show that our model outperforms benchmarks in predicting infections . 1 INTRODUCTION . Preemptively identifying individuals at a high risk of contracting a contagious infection is important for guiding treatment decisions to mitigate symptoms , and preventing further spread of the contagion . In this paper , we study how to build individual-level predictive models for contagious infections while explicitly addressing the challenges inherent to contagious diseases . Building accurate infection prediction models is hindered by two main factors . First , contagious infections defy the usual iid assumption central to most machine learning methods . This is because an individual ’ s infection state is not independent of their contacts ’ infection states . Previous work has often relied on expert knowledge to construct exposure proxies ( Wiens et al. , 2012 ; Oh et al. , 2018 ) . It is then assumed that conditional on the exposure proxy and individual characteristics , individual outcomes are independent of one another . Second , the observed data is biased due to incomplete testing . We use the term “ incomplete testing ” to describe the scenario where only a small , biased subset of infected individuals get tested . Such a scenario is ubiquitous in the context of contagious infections for several reasons . While many individuals carry the pathogen , only a fraction display symptoms . Even in the presence of unlimited testing resources , the latter are far more likely to get tested leading to biased data collection where individuals predisposed to displaying symptoms are over-represented . Incomplete testing makes learning accurate models difficult since the collected labels are missing not at random leading to biased , inconsistent estimates . In this work , we treat non-independence of outcomes as a blessing rather than a curse . Our proposed approach leverages the fact that an individual ’ s infection state provides useful information about their contacts ’ true infection states . This information is used to generate pseudo-labels for untested individuals , mitigating issues due to incomplete testing . The key idea behind our approach is that highly structured patterns of contagion transmission can serve as a complementary signal to identify even untested carriers . The stronger that signal is , the less impact that incomplete testing will have . Our contributions can be summarized as follows : ( 1 ) We identify two properties of the collected data that can be exploited to mitigate the effects of incomplete testing . ( 2 ) We propose an algorithm that leverages that insight to predict the probability of an untested individual carrying the disease . ( 3 ) We empirically evaluate the effectiveness of our method on both simulated data and real data for a common healthcare associated infection . We show that predictions from our model can be used to inform efficient testing and isolation policies . Using real data , we show that our model outperforms baselines in the task of predicting a hospital associated infection . 2 RELATED WORK . Infectious disease modeling . Modeling the transmission of infectious diseases has been extensively studied in the epidemiology literature using SIS/SIR models and several other variants ( Kermack & McKendrick , 1927 ) . These epidemiological models focus on the aggregate levels of infections in a community , which is distinct from our approach here where we focus on predicting individual level infections . In the machine learning literature , previous work has relied on proxies for exposure , e.g. , the prevalence of a disease in a community ( Wiens et al. , 2012 ; Oh et al. , 2018 ) , and implicitly assume that conditioning on individual characteristics . Similar to our approach , Fan et al . ( 2016 ) and Makar et al . ( 2018 ) take into account structured data , namely contact networks to compute infection estimates ( Fan et al. , 2016 ; Makar et al. , 2018 ) . We differ from these approachs in that ( 1 ) we do not make parametric assumptions about the joint distribution of the observed or latent variables , and instead use nonparametric models ( neural networks ) to model the infection states , ( 2 ) we do not assume all infections will become symptomatic as is done in Fan et al . ( 2016 ) , and ( 3 ) unlike the approach taken by Makar et al . ( 2018 ) , we model time evolving sequences of infections taking into account the exposure states of potential asymptomatic carriers . Semi-supervised learning . Our proposed approach relies on transductive reasoning to generate labels for untested individuals . In that , it is closely related to semi-supervised learning methods , such as pseudo-labeling ( Lee , 2003 ) , and self-training ( Robinson et al. , 2020 ) . However , in traditional pseudo-labeling , the transductive power comes from the fact that points similar to each other in the input space should have similar outputs . Here , the rich structure in the data allows for more : we can construct pseudo-labels for untested individuals not just by relying on their similarity to other labeled instances , but also by observing their observed contacts ’ infection states . Our empirical results , and analysis are similar in spirit to concepts presented in the semi-supervised literature , specifically the cluster assumption , which we discuss at length later ( Seeger , 2000 ; Rigollet , 2007 ) . Graph Neural Networks . Our proposed approach incorporates knowledge of the contact network . In that it is similar to Graph Neural Networks ( GNNs ) , which utilize relational data to generate prediction estimates ( Zhou et al. , 2018 ) . GNNs fall into two categories , the first relies on transductive reasoning and can not generalize to new communities ( e.g. , Kipf & Welling ( 2017 ) ) or inductive , which can be used to generate estimates for previously unseen graphs ( e.g. , Hamilton et al . ( 2017 ) ) . Our work is similar to the latter category with an important distinction : our approach leverages unlabeled data giving more accurate , and robust estimates . Our work can be viewed as combining the strengths of semi-supervised learning , and GNNs to address limited testing . In addition , our approach augments the strengths of those two approaches with ideas from domain shift , and causal inference such as importance weighting ( Cortes et al. , 2010 ) to address biased testing . 3 PROBLEM SETTING . Setup . Let yt ∈ { 0 , 1 } denote an individual ’ s true infection state at time t , with yt = 0 if an individual is not infected and 1 if they are . We use xt ∈ X t to denote a vector of the individual ’ s features at time t , and define J ti to be the set of indices of i ’ s contacts at time t. We assume that contact indices are known , i.e. , that the contact network is observed . Let eti ∈ R≥0 denote i ’ s exposure state , with eti = ∑ j∈Jti ytj . The exposure state is fully observed only when all of i ’ s contacts have been tested , but otherwise either partially observed or unobserved . Define xt = xt||et , where || as the concatenation operator , i.e. , xt ∈ X t × R≥0 . Let ot ∈ { 0 , 1 } denote the observation state , with ot = 1 if an individual ’ s label is observed , i.e. , if the individual has been tested for the infection . We use the super-script : t to denote variables from time t = 0 up to and including t , e.g. , x : t = [ x0 , ... , xs , ... , xt ] . Throughout , we use capital letters to denote variables , and small letters to denote their values . We use P ( Xt , Ot , Y t+1 ) to denote the unknown distribution over the full joint . Under biased testing , we have that P ( Xt|Ot = 1 ) 6= P ( Xt|Ot = 0 ) 6= P ( Xt ) . We assume that 0 < P ( Ot = o|Xt = x ) < 1 , for all x ∈ X , and o ∈ { 0 , 1 } . This is the same as the overlap assumption in causality literature . In addition , we assume that i ’ s outcome is independent of their contacts given xi , which is itself a function of the contacts ’ outcomes , we refer to this as the conditional independence assumption . We consider the case where we have access to ( 1 ) a labeled ( i.e. , tested ) set of individuals D1 = { Dt1 } Tt=0 = { ( xti , yti ) , . . . ( xtnt1 , y t nt1 ) } ∼ P ( Xt , Y t+1|Ot = 1 ) , and ( 2 ) an unlabeled ( untested ) set of individuals D0 = { Dt0 } Tt=0 = { xti , . . . , xtnt0 } ∼ P ( X t|Ot = 0 ) , such that for each i ∈ D0∪D1 , and each t ∈ [ 0 , T ] , we have that J ti ∈ D0∪D1 . It will also be convenient to use U t to denote the set of indices of untested individuals at time t. Learning objective . We are interested in learning f : x : T → yT+1 . To focus the discussion on the novel component of our approach , we consider a setting where we are only interested in predicting the outcomes for a single time step . It will be particularly useful to consider the task of making predictions for t = 2 , using data from t = 0 , 1 , dropping the time superscript when it can be inferred from the context . We present the full model predicting infection sequences over time in section 5 . Let ` be the logistic loss function . Our goal is to find f ∈ F , where F is some hypothesis space such that the risk of incorrectly classifying the infection state Rf = EX , Y [ ` ( f ( Xt ) , Y t+1 ) ] is minimized . We briefly consider a scenario where we have oracle access to the true exposure states but we return to the more realistic , non-oracle scenario later . Under the conditional independence assumption , we can break down the risk to the sum of independent losses . Define the inverse probability of being tested , wt ( X ) = P ( Ot = o ) /P ( Ot = o|Xt ) , following Robins ( 1998 ) , and Robins et al . ( 2000 ) . Due to the overlap assumption , and under biased testing , we have that : Rf = Rw t f = EX , Y |O=1 [ wt ( X ) ` ( f ( X ) , Y ) |O = 1 ] , ( 1 ) ( Cortes et al. , 2010 ) . The reweighted risk simply places a higher importance on the loss of individuals who are unlikely to be tested . Rwtf can not be directly computed since the expectation is defined with respect to the unobserved distribution . However , the following reweighted empirical loss is an unbiased estimator ofRwtf : ε ( f ) = ∑ i∈Dt1 wti ` ( f ( x t i ) , y t+1 i ) , by Cortes et al . ( 2008 ) , where wti = p ( O t = oti ) /g ( oti|x t i ) , p ( Ot = oti ) is the empirical estimate of P ( Ot = o ) , and g ( oti|xti ) is the estimated probability of getting tested conditional on individual characteristics . Without oracle access to exposure states , the samples xt ∼ P ( Xt|Ot = 1 ) are incomplete . This is because xti includes e t i , which is a function of y t j : j ∈ J ti . We only fully observe eti , and hence x t i for individuals whose contacts have all been tested . To address this , we define Q ( Dt1 ) , a set of partially imputed distributions that are consistent with the labeled samples . It is the set of all possible distributions over the ( partially ) unobserved eti . Our risk is now defined with respect to both Q , and f , and our task is to find Q and f , such that the following empirical risk is minimized : ε ( f , Q ) = ∑ i∈D1 ŵti ` ( f ( x̂ t i ) , y t+1 i ) , ( 2 ) where x̂ = xt||êt , and êti ∼ Q , and ŵti = p ( O = oi ) /g ( x̂i , oi ) . Minimizing this objective is prone to extreme overfitting . To see why , consider some Q that sets eti = 100 for every i : o t i = 1 , y t+1 i = 1 , and 0 for i : oti = 1 , y t+1 i = 0 . Since Q is essentially leaking the true label into the input space , it is trivial to find some f that takes in the imputed inputs , { ( xti , 100 , yt+1i ) } i : yt+1i =1 , and { ( xti , 0 , yt+1i ) } i : yt+1i =0 and gives perfect performance . Such an f is clearly expected to have poor generalization error . We next consider how to leverage existing properties of the problem as efficient regularizers .

Summary: In this work, the authors propose an approach, MIINT, for identifying infected individuals using a network-based approach. They also suggest two key properties, potency and similarity among groups, which impact the efficacy of MIINT and similar approaches. A detailed simulation framework is used to compare MIINT to relatively weak baselines. The simulation results show that the MIINT modestly outperforms the baselines; the results also confirm that all approaches degrade as expected as potency decreases and the similarity among groups increases. Experimental results on a (private) real-world dataset are somewhat mixed, but show that MIINT achieves a better true positive rate at an acceptable false positive rate.