Uncertainty-aware Active Learning for Optimal Bayesian Classifier

For pool-based active learning , in each iteration a candidate training sample is chosen for labeling by optimizing an acquisition function . In Bayesian classification , expected Loss Reduction ( ELR ) methods maximize the expected reduction in the classification error given a new labeled candidate based on a one-step-lookahead strategy . ELR is the optimal strategy with a single query ; however , since such myopic strategies can not identify the long-term effect of a query on the classification error , ELR may get stuck before reaching the optimal classifier . In this paper , inspired by the mean objective cost of uncertainty ( MOCU ) , a metric quantifying the uncertainty directly affecting the classification error , we propose an acquisition function based on a weighted form of MOCU . Similar to ELR , the proposed method focuses on the reduction of the uncertainty that pertains to the classification error . But unlike any other existing scheme , it provides the critical advantage that the resulting Bayesian active learning algorithm guarantees convergence to the optimal classifier of the true model . We demonstrate its performance with both synthetic and real-world datasets . 1 INTRODUCTION . In supervised learning , labeling data is often expensive and highly time consuming . Active learning is one field of research that aims to address this problem and has been demonstrated for sampleefficient learning with less required labeled data ( Gal et al. , 2017 ; Tran et al. , 2019 ; Sinha et al. , 2019 ) . In this paper , we focus on pool-based Bayesian active learning for classification with 0- 1 loss function . Bayesian active learning starts from the prior knowledge of uncertain models . By optimizing an acquisition function , it chooses the next candidate training sample to query for labeling , and then based on the acquired data , updates the belief of uncertain models through Bayes ’ rule to approach the optimal classifier of the true model , which minimizes the classification error . In active learning , maximizing the performance of the model trained on queried candidates is the ultimate objective . However , most of the existing methods do not directly target the learning objective . For example , Maximum Entropy Sampling ( MES ) or Uncertainty Sampling , simply queries the candidate with the maximum predictive entropy ( Lewis & Gale , 1994 ; Sebastiani & Wynn , 2000 ; Mussmann & Liang , 2018 ) ; but the method fails to differentiate between the model uncertainty and the observation uncertainty . Bayesian Active Learning by Disagreement ( BALD ) seeks the data point that maximizes the mutual information between the observation and the model parameters ( Houlsby et al. , 2011 ; Kirsch et al. , 2019 ) . Besides BALD , there are also other methods reducing the model uncertainty in different forms ( Golovin et al. , 2010 ; Cuong et al. , 2013 ) . However , not all the model uncertainty will affect the performance of the learning task of interest . Without identifying whether the uncertainty is related to the classification error or not , these methods can be inefficient in the sense that it may query candidates that do not directly help improve prediction performance . In this paper we focus on the active learning methods directly maximizing the learning model performance . There exist such active learning methods by Expected Loss Reduction ( ELR ) that aim to maximize the expected reduction in loss based on a one-step-look-ahead manner ( Roy & McCallum , 2001 ; Zhu et al. , 2003 ; Kapoor et al. , 2007 ) . The ELR methods can focus on only the uncertainty related to the loss function to achieve sample-efficient learning . In fact , ELR is the optimal strategy for active learning with a single query ( Roy & McCallum , 2001 ) . However , a critical shortcoming of previous ELR schemes is that none of them provide any theoretical guarantee regarding their longterm performance . In fact , since these methods are myopic and can not identify the long-term effect of a query on the loss functions , without special design on the loss function , they may get stuck before reaching the optimal classifier . To the best of our knowledge , there is currently no method that directly maximizes the model performance while simultaneously guaranteeing the convergence to the optimal classifier . Fig . 1a provides an example of binary classification with one feature where both BALD and ELR methods fail . In the figure , the red lines indicate the upper and lower bounds of the prediction probability of class 1 , illustrating the model with higher probability uncertainty on the sides ( x → ±4 ) than that in the middle ( x = 0 ) . Querying candidates on the sides will provide more information of the model parameters , and therefore is preferred in BALD . However , since the possible probabilities on the sides are always larger than or less than 0.5 , querying candidates on the sides will not help reduce the classification error . On the other hand , ELR queries candidates that help reduce the classification error the most , so it prefers data in the middle whose optimal labels are uncertain given the prior knowledge . The performance shown in Fig . 1b agrees with our analysis . Fig . 1b shows the performance averaged over 1000 runs , with more details and discussions of the example included in Appendix C. BALD performs inefficiently at the beginning by querying points on both sides . On the other hand , the ELR method performs the best at the beginning , but becomes inefficient after some iterations ( ∼100 ) , indicating some of its runs get stuck before reaching the optimal classifier . In this paper , we consider the algorithm to “ get stuck ” when the acquisition function value is 0 for all the candidates in the pool and the algorithm degenerates to uniform random sampling . In this paper , we analyze the reason why ELR methods may get stuck before reaching the optimal classifier , and propose a new strategy to solve this problem . Our contributions are in four parts : 1 . We show that ELR methods may get stuck , preventing active learning from reaching the optimal classifier efficiently . 2 . We propose a novel weighted-MOCU active learning method that can focus only on the uncertainty related to the loss for efficient active learning and is guaranteed to converge to the optimal classifier of the true model . 3 . We provide the convergence proof of the weightedMOCU method . 4 . We demonstrate the sample-efficiency of our weighted-MOCU method with both synthetic and real-world datasets . 2 BACKGROUND . Optimal Bayesian classifier . Consider a classification problem with candidates x ∈ X and class labels y ∈ Y = { 0 , 1 , . . . , M − 1 } . The predictive probability p ( y|x , θ ) is modeled with parameters θ . Assume θ is uncertain with a distribution π ( θ ) within the uncertainty class Θ . The classification problem is to find a classifier ψ : X → Y , which assigns a predicted class label to a given candidate . The expected 0-1 loss of the classifier ψ for a candidate x , dependent on θ , is defined as Cθ ( ψ , x ) , which can be derived to be the classification error : Cθ ( ψ , x ) = 1 − p ( y = ψ ( x ) |x , θ ) . The optimal classifier with θ , ψθ is defined as the classifier minimizing the classification error : ψθ ( x ) = arg maxy p ( y|x , θ ) . So we have : Cθ ( ψθ , x ) = minψ Cθ ( ψ , x ) = miny { 1− p ( y|x , θ ) } . When there is model uncertainty with π ( θ ) , an Optimal Bayesian Classifier ( OBC ) ψπ ( θ ) is the classifier that has the minimum expected loss over π ( θ ) ( Dalton & Dougherty , 2013 ) : Eπ ( θ ) [ Cθ ( ψπ ( θ ) , x ) ] = min ψ Eπ ( θ ) [ Cθ ( ψ , x ) ] = min y { 1− p ( y|x ) } ( 1 ) where p ( y|x ) = Eπ ( θ ) [ p ( y|x , θ ) ] is the predictive distribution . It ’ s easily to see ψπ ( θ ) ( x ) = arg maxy p ( y|x ) . Active learning . Active learning collects the training dataset D in a sequential way . For poolbased active learning , in each iteration , we choose a candidate x from the set of potential training samples X to query for the class label by optimizing an acquisition function U ( x ) . Then , in the Bayesian setting , by including the observed data pair ( x , y ) to D , we update the posterior distribution based on Bayes ’ rule . In each iteration , the acquisition function depends on the posterior distribution of model parameters π ( θ|D ) . In the following discussion , to simplify notations , we omit D from the notations and use π ( θ ) and p ( y|x ) to respectively denote the posterior and predictive distributions conditioned on D. When a new observed data point is included , the distributions are updated by Bayes ’ rule and the total probability rule as : π ( θ|x , y ) = π ( θ ) p ( y|x , θ ) p ( y|x ) and p ( y′|x′ , x , y ) = Eπ ( θ|x , y ) [ p ( y′|x′ , θ ) ] . The acquisition function of ELR methods in the Bayesian setting can be defined by the expected OBC prediction error reduction after observing the new pair ( x , y ) ( Roy & McCallum , 2001 ) : UELR ( x ) = Ep ( x′ ) { Eπ ( θ ) [ Cθ ( ψπ ( θ ) , x′ ) ] − Ep ( y|x ) [ Eπ ( θ|x , y ) [ Cθ ( ψπ ( θ|x , y ) , x′ ) ] ] } , ( 2 ) where p ( x′ ) is the distribution over X , independent of θ and D. ELR methods assume that we use OBC as the classifier , and in each iteration we should choose the query that maximize the decrease in OBC prediction error . The first term in ( 2 ) is the OBC prediction error of ψπ ( θ ) , and the second term is the expected prediction error of ψπ ( θ|x , y ) , the one-step-look-ahead OBC , with respect to p ( y|x ) . In the following section , we analyze why this acquisition function is sample-efficient as it directly targets at classification error reduction while ignoring irrelevant uncertainty with respect to the learning task ; but it may get stuck before converging to the true optimal classifier ( optimal classifier of the true model ) . 3 MOCU-BASED ACTIVE LEARNING . 3.1 MEAN OBJECTIVE COST OF UNCERTAINTY . To analyze ELR methods , we borrow the idea of the Mean Objective Cost of Uncertainty ( MOCU ) for active learning with respect to the corresponding posterior π ( θ ) . MOCU is a general objectiveoriented uncertainty quantification framework ( Yoon et al. , 2013 ) . For active learning , MOCU can be defined as the expected loss difference between the OBC and the optimal classifier : M ( π ( θ ) ) = Ep ( x′ ) [ Eπ ( θ ) [ Cθ ( ψπ ( θ ) , x′ ) − Cθ ( ψθ , x′ ) ] ] ( 3 ) = Ep ( x′ ) [ min y′ { 1− p ( y′|x′ ) } − Eπ ( θ ) [ min y′ { 1− p ( y′|x′ , θ ) } ] ] . ( 4 ) The second line is derived by the definition of ψθ and ( 1 ) . The first term in ( 3 ) is the OBC error as the loss . In the second term , ψθ is the optimal classifier with a specific θ . For the terms inside the expectation operator , we have Cθ ( ψπ ( θ ) , x′ ) − Cθ ( ψθ , x′ ) ≥ 0 . Therefore , the second term in ( 3 ) is a lower bound of the OBC prediction error . MOCU captures the difference between the OBC error and its lower bound . When MOCU is 0 , the OBC converges to the true optimal classifier and we can not reduce the OBC prediction error further . In that case , we say that OBC has reached the true optimal classifier . As in ELR methods , we can define an acquisition function by the reduction of MOCU in a one-steplook-ahead manner : UMOCU ( x ; π ( θ ) ) =M ( π ( θ ) ) − Ep ( y|x ) [ M ( π ( θ|x , y ) ) ] . ( 5 ) We can show that the second term in ( 3 ) , the lower bound of the OBC error , is cancelled in ( 5 ) . The acquisition function ( 5 ) hence captures the expected reduction of the OBC error given new data and is equivalent to the ELR acquisition function ( 2 ) . Expanding the second term in ( 5 ) , we have : Ep ( y|x ) [ M ( π ( θ|x , y ) ) ] = Ep ( x′ ) { Ep ( y|x ) [ Eπ ( θ|x , y ) [ Cθ ( ψπ ( θ|x , y ) , x′ ) − Cθ ( ψθ , x′ ) ] ] } . ( 6 ) Since ∑ y p ( y|x ) π ( θ|x , y ) = π ( θ ) , as x is assumed to be independent of θ so that we have π ( θ|x ) = π ( θ ) , we can rewrite the first term in ( 5 ) as : M ( π ( θ ) ) = Ep ( x′ ) { Ep ( y|x ) [ Eπ ( θ|x , y ) [ Cθ ( ψπ ( θ ) , x′ ) − Cθ ( ψθ , x′ ) ] ] } . ( 7 ) Combining ( 6 ) and ( 7 ) and canceling the Cθ ( ψθ , x′ ) terms ( the lower bound of the OBC error ) , ( 6 ) can be derived as : UMOCU ( x ; π ( θ ) ) = Ep ( x′ ) { Ep ( y|x ) [ Eπ ( θ|x , y ) [ Cθ ( ψπ ( θ ) , x′ ) − Cθ ( ψπ ( θ|x , y ) , x′ ) ] ] } , ( 8 ) which is just the ELR acquisition function in ( 2 ) . Therefore , we can conclude that MOCU-based methods are equivalent to ELR methods . Another property we can observe from ( 8 ) is that UMOCU ( x ; π ( θ ) ) ≥ 0 . By definition , ψπ ( θ|x , y ) is the OBC with the minimum expected classification error over π ( θ|x , y ) . Therefore , Eπ ( θ|x , y ) [ Cθ ( ψπ ( θ|x , y ) , x′ ) ] ≤ Eπ ( θ|x , y ) [ Cθ ( ψπ ( θ ) , x′ ) ] and we have UMOCU ( x ; π ( θ ) ) ≥ 0 , indicating collecting new data will reduce MOCU .

Summary: This paper studies the label solicitation strategy in active learning. In particular, it focuses on the expected loss reduction (ELR) strategy, analyzes its problem, and modifies the original ELR method to make sure the active learner converges to the optimal classifier along learning iterations. The paper provides theoretical guarantees on the new method’s convergence. In the experiment, the proposed method is evaluated on synthetic data and UCI data. The improvement margin over the existing method is very limited.