Least Probable Disagreement Region for Active Learning

1 INTRODUCTION . Active learning ( Cohn et al. , 1996 ) is a subfield of machine learning to attain data efficiency with fewer labeled training data when it is allowed to choose the training data from which to learn . For many real-world learning problems , large collections of unlabeled samples is assumed available , and based on a certain query strategy , the label of the most informative data is iteratively queried to an oracle to be used in retraining the model ( Bouneffouf et al. , 2014 ; Roy & McCallum , 2001 ; Sener & Savarese , 2017b ; Settles et al. , 2008 ; Sinha et al. , 2019 ; Sener & Savarese , 2017a ; Pinsler et al. , 2019 ; Shi & Yu , 2019 ; Gudovskiy et al. , 2020 ) . Active learning attempts to achieve high accuracy using as few labeled samples as possible ( Settles , 2009 ) . Of the possible query strategies , uncertainty-based sampling ( Culotta & McCallum , 2005 ; Scheffer et al. , 2001 ; Mussmann & Liang , 2018 ) , which enhances the current model by labeling unlabeled samples that are difficult for the model to predict , is a simple strategy commonly used in pool-based active learning ( Lewis & Gale , 1994 ) . Nevertheless , many existing uncertainty-based algorithms have their own limitations . Entropy ( Shannon , 1948 ) based uncertainty sampling can query unlabeled samples near the decision boundary for binary classification , but it does not perform well in multiclass classification as entropy does not equate well with the distance to a complex decision boundary ( Joshi et al. , 2009 ) . Another approach based on MC-dropout sampling ( Gal et al. , 2017 ) which uses a mutual information based BALD ( Houlsby et al. , 2011 ) as an uncertainty measure identifies unlabeled samples that are individually informative . This approach , however , is not necessarily informative when it is jointly considered with other samples for label acquisition . To address this problem , BatchBALD ( Kirsch et al. , 2019 ) is introduced . However , BatchBALD computes , theoretically , all possible joint mutual information of batch , and is infeasible for large query size . The ensemble method ( Beluch et al. , 2018 ) , one of the query by committee ( QBC ) algorithm ( Seung et al. , 1992 ) , has been shown to perform well in many cases . The fundamental premise behind the QBC is minimizing the version space ( Mitchell , 1982 ) , which is the set of hypotheses that are consistent with labeled samples . However , the ensemble method requires high computation load because all networks that make up the ensemble must be trained . This paper defines a theoretical distance referred to as the least probable disagreement region ( LPDR ) from sample to the estimated decision boundary , and in each step of active learning , labels of unlabeled samples nearest to the decision boundary in terms of LPDR are obtained to be used for retraining the classifier to improve accuracy of the estimated decision boundary . It is generally understood that labels to samples near the decision boundary are the most informative as the samples are uncertain . Indeed in Balcan et al . ( 2007 ) , selecting unlabeled samples with the smallest margin to the linear decision boundary and thereby minimal certainty attains exponential improvement over random sampling in terms of sample complexity . In deep learning , it is difficult to identify samples nearest to the decision boundary as sample distance to decision boundary is difficult to evaluate . An adversarial approach ( Ducoffe & Precioso , 2018 ) to approximate the sample distance to decision boundary has been studied but this method does not show preservation of the order of the sample distance and requires considerable computation in obtaining the distance . 2 DISTANCE : LEAST PROBABLE DISAGREEMENT REGION ( LPDR ) . This paper proposes an algorithm for selecting unlabeled data that are close to the decision boundary which can not be explicitly defined in many of cases . Let X , Y , H and D be the instance space , the label space , the set of hypotheses h : x → y and the joint distribution over ( x , y ) ∈ X × Y . The distance between two hypotheses ĥ and h is defined as the probability of the disagreement region for ĥ and h. This distance was originally defined in Hanneke et al . ( 2014 ) and Hsu ( 2010 ) : ρ ( ĥ , h ) : = PD [ ĥ ( X ) 6= h ( X ) ] . ( 1 ) This paper defines the sample distance d of x to the hypothesis ĥ ∈ H based on ρ as the least probable disagreement region ( LPDR ) that contains x : d ( x , ĥ ) : = inf h∈H ( x , ĥ ) ρ ( ĥ , h ) ( 2 ) whereH ( x , ĥ ) = { h ∈ H : ĥ ( x ) 6= h ( x ) } . Figure 1 shows an example of LPDR . Let ’ s define H = { hθ : hθ ( x ) = I [ x > θ ] } on input x sampled from uniform distribution D = U [ 0 , 1 ] where I [ · ] is an indicator function . Suppose x = x0 and ĥ = ha ∈ H when a < x0 . Here , H ( x0 , ha ) consists of all hypotheses whose prediction on x0 is in disagreement with ha ( x0 ) = 1 , i.e. , H ( x0 , ha ) = { hb ∈ H : hb ( x0 ) = 0 } = { hb ∈ H : b > x0 } . Then , the LPDR between x0 and ha , d ( x0 , ha ) = x0 − a as the infimum of the distance between ha and hb ∈ H ( x0 , ha ) is ρ ( ha , hx0 ) = x0 − a . Here , the sample distribution D is unknown , and H ( x , ĥ ) may be uncountably infinite . Therefore , a systematic and empirical method for evaluating the distance is required . One might the procedure below : Sample hypotheses sets H′ = { h′ : ρ ( ĥ , h′ ) ≤ ρ′ } in terms of ρ′ , and perform grid search to determine the smallest ρ′ such that there exists h′ ∈ H′ satisfying ĥ ( x ) 6= h′ ( x ) for a given x . Sampling the hypotheses within the ball can be performed by sampling the corresponding parameters with the assumption that the expected hypothesis distance is monotonically increasing for the expected distance between the corresponding parameters ( see Assumption 1 ) . This scheme is based on performing grid search on ρ′ and is therefore computationally inefficient . However , unlabeled samples can be ordered according to d without grid search with the assumption that there exists aH′ such that variation ratio V ( x ) = 1− f ( x ) m /|H′| and d ( x , ĥ ) have strong negative correlation where f ( x ) m = maxc ∑ h′∈H′ I [ h′ ( x ) = c ] ( see Assumption 2 ) . Assumption 1 . The expected distance between ĥ and randomly sampled h is monotonically increasing in the expected distance between the corresponding ŵ and w , i.e. , E [ ‖ŵ − w1‖ | ŵ ] ≤ E [ ‖ŵ − w2‖ | ŵ ] implies that E [ ρ ( ĥ , h1 ) | ĥ ] ≤ E [ ρ ( ĥ , h2 ) | ĥ ] where ŵ , w1 and w2 are the parameters pertaining to ĥ , h1 and h2 respectively . Assumption 2 . There exists a hypothesis set H′ sampled around ĥ having the property that large variation ratio for a given sample data implies small sample distance to ĥ with high probability , i.e. , there existsH′ such that V ( x1 ) ≥ V ( x2 ) implies that d ( x1 , ĥ ) ≤ d ( x2 , ĥ ) with high probability . 3 EMPIRICAL STUDIES OF LPDR . 3.1 HYPOTHESES AND PARAMETERS IN DEEP NETWORKS : ASSUMPTION 1 . The distance between two hypotheses can be approximated by vectors of the predicted labels on random samples by the hypotheses : ρ ( ĥ , h ) ≈ ρe ( ĥ , h ) = 1 m m∑ i=1 I [ ĥ ( x ( i ) ) 6= h ( x ( i ) ) ] ( 3 ) where x ( i ) is the ith sample for i ∈ [ m ] . The h is sampled by sampling model parameter w ∼ N ( ŵ , Iσ2 ) where ŵ is the model parameter of ĥ , and the expectation of distances between w and ŵ depends on σ . The ρe is obtained by the average of 100 times for a fixed σ . The left-hand side of Figure 2 shows the relationship between ρe and σ on various datasets and deep networks . The ρe increases almost monotonically as σ increases . This implies that the order is preserved between the σ and ρe . Furthermore , the ρe is almost linearly proportional to log ( σ ) in the ascension of the graph , i.e. , σ ∝ eβρe for some β > 0 . The right-hand side of Figure 2 shows V with respect to σ for each unlabeled sample on MNIST . The sample distance to the decision boundary can be expressed as σ at which the variation ratio is not zero for the first time ( white arrow ) , where the indices of unlabeled samples in y-axis are ordered by LPDR . The variation ratio increases as the σ increases , and it is expected that the data point with short distance has the large variation ratio compared to the data point with long distance on a certain range of σ . 3.2 LPDR VS VARIATION RATIO : ASSUMPTION 2 . The left-hand side of Figure 3 shows Spearman ’ s rank correlation coefficient ( Spearman , 1904 ) between LPDR and the variation ratio with respect to σ . The correlation is calculated using only unlabeled samples whose variation ratio is not 0 . The strong rank correlation is verified when the σ has the appropriate value . Too larger value of σ generates hypotheses too far away from ĥ , which is not helpful to measure the distance . The right-hand side of Figure 3 shows an example of σ ( log ( σ ) = −5.0 ) which makes LPDR and the variation ratio have a strong negative correlation on MNIST , that is , the data point with larger variation is closer to the decision boundary . Results for various datasets and networks are presented in Appendix C. The time complexity is discussed to validate the efficiency of using variation ratio . Let m , N and nσ be the unlabeled sample size , |H′| and the number of grid for σ respectively . Ordering unlabeled samples in terms of LPDR by grid search with respect to σ requires the time complexity of m × N × nσ ( see the right-hand side of Figure 2 ) . However , using variation ratio for ordering unlabeled samples reduces the time complexity to m×N . In the case of nσ = cN for some c > 0 , then the time complexity can be reduced from O ( mN2 ) to O ( mN ) .

Summary: This paper is motivated by the idea that unlabelled samples near the estimated decision boundary show to be very informative/useful in an active learning setting. However, measuring the distance between an instance and the decision boundary is a non-trivial task in numerous machine learning algorithms, especially in deep learning. The paper proposes a (theoretical) sample distance to the decision boundary that relies on the least probable disagreement region (LPDR) that still contains the sample. The paper makes two assumptions to evaluate the proposed distance empirically: (1) closeness of the parameters of two hypotheses implies closeness of these hypotheses as defined by the probability of the disagreement region and (2) the variation ratio of labels obtained by evaluating a set of hypotheses sampled around the decision boundary is a proxy for the proposed distance. Considering these assumptions, hypotheses are sampled around a given decision boundary by adding gaussian noise to the parameters of the fitted model. Both assumptions are validated empirically on different datasets and varying levels of variance of the noise term to show the effect on the variation ratio and distance respectively. Consequently, an iterative active learning algorithm is proposed which adapts the variance of the noise term in order to select the predefined number of samples. Extensive experimental results indicate that LPDR outperforms other uncertainty based active learning algorithms on various datasets or is at least on par with them.