CLOPS: Continual Learning of Physiological Signals

1 INTRODUCTION . Many deep learning algorithms operate under the assumption that instances are independent and identically-distributed ( i.i.d. ) . The violation of this assumption can be detrimental to the training behaviour and performance of an algorithm . The assumption of independence can be violated , for example , when data are streamed temporally from a sensor . Introducing multiple sensors in a changing environment can introduce covariate shift , arguably the ‘ Achilles heel ’ of machine learning model deployment ( Quionero-Candela et al. , 2009 ) . A plethora of realistic scenarios violate the i.i.d . assumption . This is particularly true in healthcare where the multitude of physiological sensors generate time-series recordings that may vary temporally ( due to seasonal diseases ; e.g . flu ) , across patients ( due to different hospitals or hospital settings ) , and in their modality . Tackling the challenges posed by such scenarios is the focus of continual learning ( CL ) whereby a learner , when exposed to tasks in a sequential manner , is expected to perform well on current tasks without compromising performance on previously seen tasks . The outcome is a single algorithm that can reliably solve a multitude of tasks . However , most , if not all , research in this field has been limited to a small handful of imaging datasets ( Lopez-Paz & Ranzato , 2017 ; Aljundi et al. , 2019b ; a ) . Although understandable from a benchmarking perspective , such research fails to explore the utility of continual learning methodologies in more realistic healthcare scenarios ( Farquhar & Gal , 2018 ) . To the best of our knowledge , we are the first to explore and propose a CL approach in the context of physiological signals . The dynamic and chaotic environment that characterizes healthcare necessitates the availability of algorithms that are dynamically reliable ; those that can adapt to potential covariate shift without catastrophically forgetting how to perform tasks from the past . Such dynamic reliability implies that algorithms no longer needs to be retrained on data or tasks to which it has been exposed in the past , thus improving its data-efficiency . Secondly , algorithms that perform consistently well across a multitude of tasks are more trustworthy , a desirable trait sought by medical professionals ( Spiegelhalter , 2020 ) . Our Contributions . In this paper , we propose a replay-based continual learning methodology that is based on the following : 1 . Importance-guided storage : task-instance parameters , a scalar corresponding to each instance in each task , as informative signals for loss-weighting and buffer-storage . 2 . Uncertainty-based acquisition : an active learning inspired methodology that determines the degree of informativeness of an instance and thus acts as a buffer-acquisition mechanism . 2 RELATED WORK . Continual learning ( CL ) approaches have resurfaced in recent years ( Parisi et al. , 2019 ) . Those similar to ours comprise memory-based methods such as iCaRL ( Rebuffi et al. , 2017 ) , CLEAR ( Rolnick et al. , 2019 ) , GEM ( Lopez-Paz & Ranzato , 2017 ) , and aGEM ( Chaudhry et al. , 2018 ) . In contrast to our work , the latter two methods naively populate their replay buffer with the last m examples observed for a particular task . Isele & Cosgun ( 2018 ) and Aljundi et al . ( 2019b ) employ a more sophisticated buffer-storage strategy where a quadratic programming problem is solved in the absence of task boundaries . Aljundi et al . ( 2019a ) introduce MIR whereby instances are stored using reservoir sampling and sampled according to whether they incur the greatest change in loss if parameters were to be updated on the subsequent task . This approach is computationally expensive , requiring multiple forward and backward passes per batch . The application of CL in the medical domain is limited to that of Lenga et al . ( 2020 ) wherein existing methodologies are implemented on chest X-ray datasets . In contrast to previous research that independently investigates buffer-storage and acquisition strategies , we focus on a dual storage and acquisition strategy . Active learning ( AL ) in healthcare has observed increased interest in recent years , with a review of methodologies provided by Settles ( 2009 ) . For example , Gong et al . ( 2019 ) propose a Bayesian deep latent Gaussian model to acquire important features from electronic health record ( EHR ) data in MIMIC ( Johnson et al. , 2016 ) to improve mortality prediction . In dealing with EHR data , Chen et al . ( 2013 ) use the distance of unlabelled samples from the hyperplane in an SVM to acquire datapoints . Wang et al . ( 2019 ) implement an RNN to acquire ECG samples during training . Zhou et al . ( 2017 ) perform transfer learning in conjunction with a convolutional neural network to acquire biomedical images in an online manner . Smailagic et al . ( 2018 ; 2019 ) actively acquire unannotated medical images by measuring their distance in a latent space to images in the training set . Such similarity metrics , however , are sensitive to the amount of available labelled training data . Gal et al . ( 2017 ) adopt BALD ( Houlsby et al. , 2011 ) with Monte Carlo Dropout to acquire instances that maximize the Jensen-Shannon divergence ( JSD ) across MC samples . To the best of our knowledge , we are the first to employ AL-inspired acquisition functions in the context of CL . 3 BACKGROUND . 3.1 CONTINUAL LEARNING . In this work , we consider a learner , fω : xT ∈ Rm → yT ∈ Rc , parameterized by ω , that maps an m-dimensional input , xT , to a c-dimensional output , yT , where c is the number of classes , for each task T ∈ [ 1 . . . N ] . This learner is exposed to new tasks in a sequential manner once previouslytackled tasks are mastered . In this paper , we formulate our tasks based on a modification of the three-tier categorization proposed by van de Ven & Tolias ( 2019 ) . In our learning scenarios ( see Fig . 1 ) , a network is sequentially tasked with solving a binary classification problem in response to data from mutually-exclusive pairs of classes Class Incremental Learning ( Class-IL ) , multiclass classification problem in response to data collected at different times of the year ( e.g. , winter and summer ) Time Incremental Learning ( Time-IL ) , and a multi-class classification problem in response to inputs with a different modality Domain Incremental Learning ( Domain-IL ) . In the aforementioned cases , task identities are absent during both training and testing and neural architectures are single-headed . 4 METHODS . The two ideas underlying our proposal are the storage of instances into and the acquisition of instances from a buffer such that destructive interference is mitigated . We describe these in more detail below . 4.1 IMPORTANCE-GUIDED BUFFER STORAGE . We aim to populate a buffer , DB , of finite size , M , with instances from the current task that are considered important . To quantify importance , we learn parameters , entitled task-instance parameters , βiT , associated with each instance , xiT , in each task , T . These parameters play a dual role . 4.1.1 LOSS-WEIGHTING MECHANISM . For the current task , k , and its associated data , Dk , we incorporate β as a coefficient of the loss , Lik , incurred for each instance , xik ∈ Dk . For a mini-batch of size , B , that consists of Bk instances from the current task , the objective function is shown in Eq . 1 . We can learn the values of βik via gradient descent , with some learning rate , η , as shown in Eq . 2 . L = 1 Bk Bk∑ i=1 βikLik ( 1 ) βik ← βik − η ∂L ∂βik ( 2 ) Note that ∂L∂βik = Lik > 0 . This suggests that instances that are hard to classify ( ↑ Lik ) will exhibit ↓ βik . From this perspective , βik can be viewed as a proxy for instance difficulty . However , as presented , βik → 0 as training progresses , an observation we confirmed empirically . Since βik is the coefficient of the loss , Lik , this implies that the network will quickly be unable to learn from the data . To avoid this behaviour , we initialize βik = 1 in order to emulate a standard loss function and introduce a regularization term to penalize its undesirable and rapid decay toward zero . As a result , our modified objective function is : Lcurrent = 1 Bk Bk∑ i=1 βikLik + λ ( βik − 1 ) 2 ( 3 ) When k > 1 , we replay instances from previous tasks by using a replay buffer ( see Sec . 4.2 for replay mechanism ) . These replayed instances incur a loss Lij ∀ j ∈ [ 1 . . . k − 1 ] . We decide to not weight these instances , in contrast to what we perform to instances from the current task ( see Appendix K ) . Lreplay = 1 B −Bk k−1∑ j=1 Bj∑ i Lij ( 4 ) L = Lcurrent + Lreplay ( 5 ) 4.1.2 BUFFER-STORAGE MECHANISM . We leverage β , as a proxy for instance difficulty , to store instances into the buffer . To describe the intuition behind this process , we illustrate , in Fig . 2 , the trajectory of β1k and β2k associated with two instances , x1k and x2k , while training on the current task , k , for τ = 20 epochs . In selecting instances for storage into the buffer , we can 1 ) retrieve their corresponding β values at the conclusion of the task , i.e. , at β ( t = 20 ) , 2 ) rank all instances based on these β values , and 3 ) acquire the top b fraction of instances . This approach , however , can lead to erroneous estimates of the relative difficulty of instances , as explained next . In Fig . 2 , we see that β2k > β1k for the majority of the training process , indicating that x2k had been easier to classify than x1k . The swap in the ranking of these β values that occurs towards the end of training in addition to myopically looking at β ( t = 20 ) would erroneously make us believe that the opposite was true . Such convergence or swapping of β values has also been observed by Saxena et al . ( 2019 ) . As a result , the reliability of β as a proxy of instance difficulty is eroded . To maintain the reliability of this proxy , we propose to track the β values after each training epoch , t , until the final epoch , τ , for the task at hand and calculate the area under these tracked values . We do so by using the trapezoidal rule as shown in Eq . 6 . We explored several variants of the storage function and found the proposed form to work best ( see Appendix H ) . At t = τ , we rank the instances in descending order of sik ( easy to hard ) as we found this preferable to the opposite order ( see Appendix I ) , select the top b fraction , and store them into the buffer , of which each task is allotted a fixed portion . The higher the value of the storage fraction , b , the more likely it is that the buffer will contain representative instances and thus mitigate forgetting , however this comes at an increased computational cost . sik = ∫ τ 0 βik ( t ) dt ≈ τ∑ t=0 ( βik ( t+ ∆t ) + βik ( t ) 2 ) ∆t ( 6 )

Summary: The authors propose a learning methodology designed to offset detriments to algorithm performance that arise when instances are not i.i.d (independent and identically distributed), focusing on cases in continual learning (CL) given physiological signals. They designed a replay-based learning method that handles an instance buffer using Importance-guided Storage and Uncertainty-based Acquisition strategies. They apply their method on Class, Time and Domain types of CL, and they introduce t-Step Backward Weight Transfer and  Lambda Backward Weight Transfer methods by which to evaluate their method. They conclude with two ablation studies to explore an explanation for their method’s performance and attempt to validate their hypotheses based on these studies.