Sequential Density Ratio Estimation for Simultaneous Optimization of Speed and Accuracy

1 INTRODUCTION . The sequential probability ratio test , or SPRT , was originally invented by Abraham Wald , and an equivalent approach was also independently developed and used by Alan Turing in the 1940s ( Good , 1979 ; Simpson , 2010 ; Wald , 1945 ) . SPRT calculates the log-likelihood ratio ( LLR ) of two competing hypotheses and updates the LLR every time a new sample is acquired until the LLR reaches one of the two thresholds for alternative hypotheses ( Figure 1 ) . Wald and his colleagues proved that when sequential data are sampled from independently and identically distributed ( i.i.d . ) data , SPRT can minimize the required number of samples to achieve the desired upper-bounds of false positive and false negative rates comparably to the Neyman-Pearson test , known as the most powerful likelihood test ( Wald & Wolfowitz , 1948 ) ( see also Theorem ( A.5 ) in Appendix A ) . Note that Wald used the i.i.d . assumption only for ensuring a finite decision time ( i.e. , LLR reaches a threshold within finite steps ) and for facilitating LLR calculation : the non-i.i.d . property does not affect other aspects of the SPRT including the error upper bounds ( Wald , 1947 ) . More recently , Tartakovsky et al . verified that the non-i.i.d . SPRT is optimal or at least asymptotically optimal as the sample size increases ( Tartakovsky et al. , 2014 ) , opening the possibility of potential applications of the SPRT to non-i.i.d . data series . About 70 years after Wald ’ s invention , neuroscientists found that neurons in the part of the primate brain called the lateral intraparietal cortex ( LIP ) showed neural activities reminiscent of the SPRT ( Kira et al. , 2015 ) ; when a monkey sequentially collects random pieces of evidence to make a binary choice , LIP neurons show activities proportional to the LLR . Importantly , the time of the decision can be predicted from when the neural activity reaches a fixed threshold , the same as the SPRT ’ s decision rule . Thus , the SPRT , the optimal sequential decision strategy , was re-discovered to be an algorithm explaining primate brains ’ computing strategy . It remains an open question , however , what algorithm will be used in the brain when the sequential evidence is correlated , non-i.i.d . series . The SPRT is now used for several engineering applications ( Cabri et al. , 2018 ; Chen et al. , 2017 ; Kulldorff et al. , 2011 ) . However , its i.i.d . assumption is too crude for it to be applied to other real-world scenarios , including time-series classification , where data are highly correlated , and key dynamic features for classification often extend across more than one data point , violating the i.i.d . assumption . Moreover , the LLR of alternative hypotheses needs to be calculated as precisely as possible , which is infeasible in many practical applications . In this paper , we overcome the above difficulties by using an SPRT-based algorithm that Treats data series As an N-th orDEr Markov process ( SPRT-TANDEM ) , aided by a sequential probability density ratio estimation based on deep neural networks . A novel Loss function for Log-Likelihood Ratio estimation ( LLLR ) efficiently estimates the density ratio that let the SPRT-TANDEM approach close to asymptotic Bayes-optimality ( i.e. , Appendix A.4 ) . In other words , LLLR optimizes classification speed and accuracy at the same time . The SPRT-TANDEM can classify non-i.i.d . data series with user-defined model complexity by changing N ( ∈ N ) , the order of approximation , to define the number of past samples on which the given sample depends . By dynamically changing the number of samples used for classification , the SPRT-TANDEM can maintain high classification accuracy while minimizing the sample size as much as possible . Moreover , the SPRT-TANDEM enables a user to flexibly control the speed-accuracy tradeoff without additional training , making it applicable to various practical applications . We test the SPRT-TANDEM on our new database , Nosaic MNIST ( NMNIST ) , in addition to the publicly available UCF101 action recognition database ( Soomro et al. , 2012 ) and Spoofing in the Wild ( SiW ) database ( Liu et al. , 2018 ) . Two-way analysis of variance ( ANOVA , ( Fisher , 1925 ) ) followed by a Tukey-Kramer multi-comparison test ( Tukey , 1949 ; Kramer , 1956 ) shows that our proposed SPRT-TANDEM provides statistically significantly higher accuracy than other fixed-length and variable-length classifiers at a smaller number of data samples , making Wald ’ s SPRT applicable even to non-i.i.d . data series . Our contribution is fivefold : 1 . We invented a deep neural network-based algorithm , SPRT-TANDEM , which enables Wald ’ s SPRT on arbitrary sequential data without knowing the true LLR . 2 . The SPRT-TANDEM extends the SPRT to non-i.i.d . data series without knowing the true LLR . 3 . With a novel loss , LLLR , the SPRT-TANDEM sequentially estimates LLR to optimize speed and accuracy simultaneously . 4 . The SPRT-TANDEM can control the speed-accuracy tradeoff without additional training . 5 . We introduce Nosaic MNIST , a novel early-classification database . 2 RELATED WORK . The SPRT-TANDEM has multiple interdisciplinary intersections with other fields of research : Wald ’ s classical SPRT , probability density estimation , neurophysiological decision making , and time-series classification . The comprehensive review is left to Appendix B , while in the following , we introduce the SPRT , probability density estimation algorithms , and early classification of the time series . Sequential Probability Ratio Test ( SPRT ) . The SPRT , denoted by δ∗ , is defined as the tuple of a decision rule and a stopping rule ( Tartakovsky et al. , 2014 ; Wald , 1947 ) : Definition 2.1 . Sequential Probability Ratio Test ( SPRT ) . Let λt as the LLR at time t , and X ( 1 , T ) as a sequential data X ( 1 , T ) : = { x ( t ) } Tt=1 . Given the absolute values of lower and upper decision threshold , a0 ≥ 0 and a1 ≥ 0 , SPRT , δ∗ , is defined as δ∗ = ( d∗ , τ∗ ) , ( 1 ) where the decision rule d∗ and stopping time τ∗ are d∗ ( X ( 1 , T ) ) = { 1 if λτ∗ ≥ a1 0 if λτ∗ ≤ −a0 , ( 2 ) τ∗ = inf { T ≥ 0|λT /∈ ( −a0 , a1 ) } . ( 3 ) We review the proof of optimality in Appendix A.4 , while Figure 1 shows an intuitive explanation . Probability density ratio estimation . Instead of estimating numerator and denominator of a density ratio separately , the probability density ratio estimation algorithms estimate the ratio as a whole , reducing the degree of freedom for more precise estimation ( Sugiyama et al. , 2010 ; 2012 ) . Two of the probability density ratio estimation algorithms that closely related to our work are the probabilistic classification ( Bickel et al. , 2007 ; Cheng & Chu , 2004 ; Qin , 1998 ) and density fitting approach ( Sugiyama et al. , 2008 ; Tsuboi et al. , 2009 ) algorithms . As we show in Section 4 and Appendix E , the SPRT-TANDEM sequentially estimates the LLR by combining the two algorithms . Early classification of time series . To make decision time as short as possible , algorithms for early classification of time series can handle variable length of data ( Mori et al. , 2018 ; Mori et al. , 2016 ; Xing et al. , 2009 ; 2012 ) to minimize high sampling costs ( e.g. , medical diagnostics ( Evans et al. , 2015 ; Griffin & Moorman , 2001 ) , or stock crisis identification ( Ghalwash et al. , 2014 ) ) . Leveraging deep neural networks is no exception in the early classification of time series ( Dennis et al. , 2018 ; Suzuki et al. , 2018 ) . Long short-term memory ( LSTM ) variants LSTM-s/LSTM-m impose monotonicity on classification score and inter-class margin , respectively , to speed up action detection ( Ma et al. , 2016 ) . Early and Adaptive Recurrent Label ESTimator ( EARLIEST ) combines reinforcement learning and a recurrent neural network to decide when to classify and assign a class label ( Hartvigsen et al. , 2019 ) . 3 PROPOSED ALGORITHM : SPRT-TANDEM . In this section , we propose the TANDEM formula , which provides the N -th order approximation of the LLR with respect to posterior probabilities . The i.i.d . assumption of Wald ’ s SPRT greatly simplifies the LLR calculation at the expense of the precise temporal relationship between data samples . On the other hand , incorporating a long correlation among multiple data may improve the LLR estimation ; however , calculating too long a correlation may potentially be detrimental in the following cases . First , if a class signature is significantly shorter than the correlation length in consideration , uninformative data samples are included in calculating LLR , resulting in a late or wrong decision ( Campos et al. , 2018 ) . Second , long correlations require calculating a long-range of backpropagation , prone to vanishing gradient problem ( Hochreiter et al. , 2001 ) . Thus , we relax the i.i.d . assumption by keeping only up to the N -th order correlation to calculate the LLR . The TANDEM formula . Here , we introduce the TANDEM formula , which computes the approximated LLR , the decision value of the SPRT-TANDEM algorithm . The data series is approximated as an N -th order Markov process . For the complete derivation of the 0th ( i.i.d . ) , 1st , and N -th order TANDEM formula , see Appendix C. Given a maximum timestamp T ∈ N , let X ( 1 , T ) and y be a sequential data X ( 1 , T ) : = { x ( t ) } Tt=1 and a class label y ∈ { 1 , 0 } , respectively , where x ( t ) ∈ Rdx and dx ∈ N. By using Bayes ’ rule with the N -th order Markov assumption , the joint LLR of data at a timestamp t is written as follows : log ( p ( x ( 1 ) , x ( 2 ) , ... , x ( t ) |y = 1 ) p ( x ( 1 ) , x ( 2 ) , ... , x ( t ) |y = 0 ) ) = t∑ s=N+1 log ( p ( y = 1|x ( s−N ) , ... , x ( s ) ) p ( y = 0|x ( s−N ) , ... , x ( s ) ) ) − t∑ s=N+2 log ( p ( y = 1|x ( s−N ) , ... , x ( s−1 ) ) p ( y = 0|x ( s−N ) , ... , x ( s−1 ) ) ) − log ( p ( y = 1 ) p ( y = 0 ) ) ( 4 ) ( see Equation ( 84 ) and ( 85 ) in Appendix C for the full formula ) . Hereafter we use terms k-let or multiplet to indicate the posterior probabilities , p ( y|x ( 1 ) , ... , x ( k ) ) = p ( y|X ( 1 , k ) ) that consider correlation across k data points . The first two terms of the TANDEM formula ( Equation ( 4 ) ) , N + 1- let and N -let , have the opposite signs working in “ tandem ” adjusting each other to compute the LLR . The third term is a prior ( bias ) term . In the experiment , we assume a flat prior or zero bias term , but a user may impose a non-flat prior to handling the biased distribution of a dataset . The TANDEM formula can be interpreted as a realization of the probability matching approach of the probability density estimation , under an N -th order Markov assumption of data series . Neural network that calculates the SPRT-TANDEM formula . The SPRT-TANDEM is designed to explicitly calculate the N -th order TANDEM formula to realize sequential density ratio estimation , which is the critical difference between our SPRT-TANDEM network and other architecture based on convolutional neural networks ( CNNs ) and recurrent neural networks ( RNN ) . Figure 2 illustrates a conceptual diagram explaining a generalized neural network structure , in accordance with the 1st-order TANDEM formula for simplicity . The network consists of a feature extractor and a temporal integrator ( highlighted by red and blue boxes , respectively ) . They are arbitrary networks that a user can choose depending on classification problems or available computational resources . The feature extractor and temporal integrator are separately trained because we find that this achieves better performance than the end-to-end approach ( also see Appendix D ) . The feature extractor outputs single-frame features ( e.g. , outputs from a global average pooling layer ) , which are the input vectors of the temporal integrator . The output vectors from the temporal integrator are transformed with a fully-connected layer into two-dimensional logits , which are then input to the softmax layer to obtain posterior probabilities . They are used to compute the LLR to run the SPRT ( Equation ( 2 ) ) . Note that during the training phase of the feature extractor , the global average pooling layer is followed by a fully-connected layer for binary classification . How to choose the hyperparameter N ? By tuning the hyperparameter N , a user can efficiently boost the model performance depending on databases ; in Section 5 , we change N to visualize the model performance as a function of N . Here , we provide two ways to choose N . One is to choose N based on the specific time scale , a concept introduced in Appendix D , where we describe in detail how to guess on the best N depending on databases . The other is to use a hyperparameter tuning algorithm , such as Optuna , ( Akiba et al. , 2019 ) to choose N objectively . Optuna has multiple hyperparameter searching algorithms , the default of which is the Tree-structured Parzen Estimator ( Bergstra et al. , 2011 ) . Note that tuning N is not computationally expensive , because N is only related to the temporal integrator , not the feature extractor . In fact , the temporal integrator ’ s training speed is much faster than that of the feature extractor : 9 mins/epoch vs. 10 hrs/epoch ( N = 49 , NVIDIA RTX2080Ti , SiW database ) .

Summary: This work introduces SPRT-TANDEM an algorithm to train a sequential probability ratio test (SPRT) as a neural network. This network is then used to discriminate between two hypotheses as fast as possible (seeing the smallest number of observations in a sequence) while maintaining a certain level of accuracy. The main contribution of this work is to enable Wald's SPRT without actual knowledge of the ratio, learning a neural network to model it.