Interactive Label Cleaning with Example-based Explanations

1 Introduction . Label noise is a major issue in machine learning as it can lead to compromised predictive performance and unreliable models [ 1 , 2 ] . We focus on sequential learning settings in which a human supervisor , usually a domain expert , can be asked to double-check and relabel any potentially mislabeled example . Applications include crowd-sourced machine learning and citizen science , where trained researchers can be asked to clean the labels provided by crowd-workers [ 3 , 4 ] , and interactive personal assistants [ 5 ] , where the user self-reports the initial annotations ( e.g. , about activities being performed ) and unreliability is due to memory bias [ 6 ] , unwillingness to report [ 7 ] , or conditioning [ 8 ] . This problem is often tackled by monitoring for incoming examples that are likely to be mislabeled , aka suspicious examples , and ask the supervisor to provide clean ( or at least better ) annotations for them . Existing approaches , however , focus solely on cleaning the incoming examples [ 9 , 4 , 10 , 5 ] . This means that noisy examples that did not undergo the cleaning step ( e.g. , those in the initial 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . bootstrap data set ) or that managed to elude it are left untouched . This degrades the quality of the model and prevents it from spotting future mislabeled examples that fall in regions affected by noise . We introduce CINCER ( Contrastive and InflueNt CounterExample stRategy ) , a new explainable interactive label cleaning algorithm that lets the annotator observe and fix the reasons behind the model ’ s suspicions . For every suspicious example that it finds , CINCER identifies a counter-example , i.e. , a training example that maximally supports the machine ’ s suspicion . The idea is that the example/counter-example pair captures a potential inconsistency in the data—as viewed from the model ’ s perspective—which is resolved by invoking the supervisor . More specifically , CINCER asks the user to relabel the example , the counter-example , or both , thus improving the quality of , and promoting consistency between , the data and the model . Two hypothetical rounds of interaction on a noisy version of MNIST are illustrated in Figure 1 . CINCER relies on a principled definition of counter-examples derived from few explicit , intuitive desiderata , using influence functions [ 11 , 12 ] . The resulting counter-example selection problem is solved using a simple and efficient approximation based on the Fisher information matrix [ 13 ] that consistently outperforms more complex alternatives in our experiments . Contributions : Summarizing , we : 1 ) Introduce CINCER , an explanatory interactive label cleaning strategy that leverages example-based explanations to identify inconsistencies in the data—as perceived by the model—and enable the annotator to fix them . 2 ) Show how to select counter-examples that at the same time explain why the model is suspicious and that are highly informative using ( an efficient approximation of ) influence functions . 3 ) Present an extensive empirical evaluation that showcases the ability of CINCER of building cleaner data sets and better models . 2 Background . We are concerned with sequential learning under label noise . In this setting , the machine receives a sequence of examples z̃t : = ( xt , ỹt ) , for t = 1 , 2 , . . . , where xt ∈ Rd is an instance and ỹt ∈ [ c ] is a corresponding label , with [ c ] : = { 1 , . . . , c } . The label ỹt is unreliable and might differ from the ground-truth label y∗t . The key feature of our setting is that a human supervisor can be asked to double-check and relabel any example . The goal is to acquire a clean dataset and a high-quality predictor while asking few relabeling queries , so to keep the cost of interaction under control . The state-of-the-art for this setting is skeptical learning ( SKL ) [ 10 , 5 ] . SKL is designed primarily for smart personal assistants that must learn from unreliable users . SKL follows a standard sequential learning loop : in each iteration , the machine receives an example and updates the model accordingly . However , for each example that it receives , the machine compares ( an estimate of ) the quality of the annotation with that of its own prediction , and if the prediction looks more reliable than the annotation by some factor , SKL asks the user to double-check his/her example . The details depend on the implementation : in [ 10 ] label quality is estimated using the empirical accuracy for the classifier and the empirical probability of contradiction for the annotator , while in [ 5 ] the machine measures the margin between the user ’ s and machine ’ s labels . Our approach follows the latter strategy . Another very related approach is learning from weak annotators ( LWA ) [ 9 , 4 ] , which focuses on querying domain experts rather than end-users . The most recent approach [ 4 ] jointly learns a prediction pipeline composed of a classifier and a noisy channel , which allows it to estimate the noise rate directly . Moreover , the approach in [ 4 ] identifies suspicious examples that have a large impact on the learned model . A theoretical foundation for LWA is given in [ 9 ] . LWA is however designed for pool-based scenarios , where the training set is given rather than obtained sequentially . For this reason , in the remainder of the paper , we will chiefly build on and compare to SKL . Limitations of existing approaches . A major downside of SKL is that it focuses on cleaning the incoming examples only . This means that if a mislabeled example manages to elude the cleaning step and gets added to the training set , it is bound to stay there forever . This situation is actually quite common during the first stage of skeptical learning , in which the model is highly uncertain and trusts the incoming examples—even if they are mislabeled . The same issue occurs if the initial training set used to bootstrap the classifier contains mislabeled examples . As shown by our experiments , the accumulation of noisy data in the training set may have a detrimental effect on the model ’ s performance ( cf . Figure 2 ) . In addition , it can also affect the model ’ s ability to identify suspicious examples : a noisy data point can fool the classifier into trusting incoming mislabeled examples that fall close to it , further aggravating the situation . 3 Explainable Interactive Label Cleaning with CINCER . We consider a very general class of probabilistic classifiers f : Rd → [ c ] of the form f ( x ; θ ) : = argmaxy∈ [ c ] P ( y |x ; θ ) , where the conditional distribution P ( Y |X ; θ ) has been fit on training data by minimizing the cross-entropy loss ` ( ( x , y ) , θ ) = − ∑ i∈ [ c ] 1 { i = y } logP ( i |x , θ ) . In our implementation , we also assume P to be a neural network with a softmax activation at the top layer , trained using some variant of SGD and possibly early stopping . 3.1 The CINCER Algorithm . The pseudo-code of CINCER is listed in Algorithm 1 . At the beginning of iteration t , the machine has acquired a training set Dt−1 = { z1 , . . . , zt−1 } and trained a model with parameters θt−1 on it . At this point , the machine receives a new , possibly mislabeled example z̃t ( line 3 ) and has to decide whether to trust it . Following skeptical learning [ 5 ] , CINCER does so by computing the margin µ ( z̃t , θt−1 ) , i.e. , the difference in conditional probability between the model ’ s prediction ŷt : = argmaxy P ( y |xt , θt−1 ) and the annotation ỹt . More formally : µ ( z̃t , θt−1 ) : = P ( ŷt |xt , θt−1 ) − P ( ỹt |xt , θt−1 ) ( 1 ) The margin estimates the incompatibility between the model and the example : the larger the margin , the more suspicious the example . The example z̃t is deemed compatible if the margin is below a given threshold τ and suspicious otherwise ( line 4 ) ; possible choices for τ are discussed in Section 3.5 . If z̃t is compatible , it is added to the data set as-is ( line 5 ) . Otherwise , CINCER computes a counterexample zk ∈ Dt−1 that maximally supports the machine ’ s suspicion . The intuition is that the pair ( z̃t , zk ) captures a potential inconsistency in the data . For instance , the counter-example might be a correctly labeled example that is close or similar to z̃t but has a different label , or a distant noisy outlier that fools the predictor into assigning low probability to ỹt . How to choose an effective counter-example is a major focus of this paper and discussed in detail in Section 3.2 and following . Next , CINCER asks the annotator to double-check the pair ( z̃t , zk ) and relabel the suspicious example , the counter-example , or both , thus resolving the potential inconsistency . The data set and model are then updated accordingly ( line 9 ) and the loop repeats . 3.2 Counter-example Selection . Counter-examples are meant to illustrate why a particular example z̃t is deemed suspicious by the machine in a way that makes it easy to elicit useful corrective feedback from the supervisor . We posit that a good counter-example zk should be : D1 . Contrastive : zk should explain why z̃t is considered suspicious by the model , thus highlighting a potential inconsistency in the data . Algorithm 1 Pseudo-code of CINCER . Inputs : initial ( noisy ) training set D0 ; threshold τ . 1 : fit θ0 on D0 2 : for t = 1 , 2 , . . . do 3 : receive new example z̃t = ( xt , ỹt ) 4 : if µ ( z̃t , θt−1 ) < τ then 5 : Dt ← Dt−1 ∪ { z̃t } . z̃t is compatible 6 : else 7 : find counterexample zk using Eq . 12 . z̃t is suspicious 8 : present z̃t , zk to the user , receive possibly cleaned labels y′t , y ′ k 9 : Dt ← ( Dt−1 \ { zk } ) ∪ { ( xt , y′t ) , ( xk , y′k ) } 10 : fit θt on Dt D2 . Influential : if zk is mislabeled , correcting it should improve the model as much as possible , so to maximize the information gained by interacting with the annotator . In the following , we show how , for models learned by minimizing the cross-entropy loss , one can identify counter-examples that satisfy both desiderata . What is a contrastive counter-example ? We start by tackling the first desideratum . Let θt−1 be the parameters of the current model . Intuitively , zk ∈ Dt−1 is a contrastive counter-example for a suspicious example z̃t if removing it from the data set and retraining leads to a model with parameters θ−kt−1 that assigns higher probability to the suspicious label ỹt . The most contrastive counter-example is then the one that maximally affects the change in probability : argmaxk∈ [ t−1 ] { P ( ỹt |xt ; θ−kt−1 ) − P ( ỹt |xt ; θt−1 ) } ( 2 ) While intuitively appealing , optimizing Eq . 2 directly is computationally challenging as it involves retraining the model |Dt−1| times . This is impractical for realistically sized models and data sets , especially in our interactive scenario where a counter-example must be computed in each iteration . Influence functions . We address this issue by leveraging influence functions ( IFs ) , a computational device that can be used to estimate the impact of specific training examples without retraining [ 11 , 12 ] . Let θt be the empirical risk minimizer on Dt and θt ( z , ) be the minimizer obtained after adding an example z with weight to Dt , namely : θt : = argminθ 1 t ∑t k=1 ` ( zk , θ ) θt ( z , ) : = argminθ 1 t ( ∑t k=1 ` ( zk , θ ) ) + ` ( z , θ ) ( 3 ) Taking a first-order Taylor expansion , the difference between θt = θt ( z , 0 ) and θt ( z , ) can be written as θt ( z , ) − θt ( z , 0 ) ≈ · ( d d θt ( z , ) ∣∣ =0 ) . The derivative appearing on the right hand side is the so-called influence function , denoted Iθt ( z ) . It follows that the effect on θt of adding ( resp . removing ) an example z to Dt can be approximated by multiplying the IF by = 1/t ( resp . = −1/t ) . Crucially , if the loss is strongly convex and twice differentiable , the IF can be written as : Iθt ( z ) = −H ( θt ) −1∇θ ` ( z , θt ) ( 4 ) where the curvature matrixH ( θt ) : = 1t ∑t k=1∇2θ ` ( zk , θt ) is positive definite and invertible . IFs were shown to capture meaningful information even for neural networks and other non-convex models [ 12 ] . Identifying contrastive counter-examples with influence functions . To see the link between contrastive counter-examples and influence functions , notice that the second term of Eq . 2 is independent of zk , while the first term can be conveniently approximated with IFs by applying the chain rule : − 1 t− 1 ( d d P ( ỹt |xt ; θt−1 ( zk , ) ) ∣∣∣∣ =0 ) = − 1 t− 1 ( ∇θP ( ỹt |xt ; θt−1 ) > d d θt−1 ( zk , ) ∣∣∣∣ =0 ) ( 5 ) = − 1 t− 1 ∇θP ( ỹt |xt ; θt−1 ) > Iθt−1 ( zk ) ( 6 ) The constant can be dropped during the optimization . This shows that Eq . 2 is equivalent to : argmaxk∈ [ t−1 ] ∇θP ( ỹt |xt ; θt−1 ) > H ( θt−1 ) −1∇θ ` ( zk , θt−1 ) ( 7 ) Eq . 7 can be solved efficiently by combining two strategies [ 12 ] : i ) Caching the inverse Hessian-vector product ( HVP ) ∇θP ( ỹt |xt ; θt−1 ) > H ( θt−1 ) −1 , so that evaluating the objective on each zk becomes a simple dot product , and ii ) Solving the inverse HVP with an efficient stochastic estimator like LISSA [ 14 ] . This gives us an algorithm for computing contrastive counter-examples . Contrastive counter-examples are highly influential . Can this algorithm be used for identifying influential counter-examples ? It turns out that , as long as the model is obtained by optimizing the cross-entropy loss , the answer is affirmative . Indeed , note that if ` ( z , θ ) = − logP ( y |x ; θ ) , then : ∇θP ( ỹt |xt ; θt−1 ) = P ( ỹt |xt ; θt−1 ) ∇θP ( ỹt |xt ; θt−1 ) P ( ỹt |xt ; θt−1 ) = ( 8 ) = P ( ỹt |xt ; θt−1 ) ∇θ logP ( ỹt |xt ; θt−1 ) = −P ( ỹt |xt ; θt−1 ) ∇θ ` ( z̃t , θt−1 ) ( 9 ) Hence , Eq . 6 can be rewritten as : − P ( ỹt |xt ; θt−1 ) ∇θ ` ( z̃t , θt−1 ) > H ( θt−1 ) −1∇θ ` ( zk , θt−1 ) ( 10 ) ∝ −∇θ ` ( z̃t , θt−1 ) > H ( θt−1 ) −1∇θ ` ( zk , θt−1 ) ( 11 ) It follows that , under the above assumptions and as long as the model satisfies P ( ỹt |xt ; θt−1 ) > 0 , Eq . 2 is equivalent to : argmaxk∈ [ t−1 ] −∇θ ` ( z̃t , θt−1 ) > H ( θt−1 ) −1∇θ ` ( zk , θt−1 ) ( 12 ) This equation recovers exactly the definition of influential examples given in [ 12 , Eq . 2 ] and shows that , for the large family of classifiers trained by cross-entropy , highly influential counter-examples are highly contrastive and vice versa , so that no change to the selection algorithm is necessary .

Summary: The paper introduces an approach called CINCER which tackles label noise in a sequential learning task by querying a human annotator whenever it finds a suspicious example. In support of its suspicion, the model also presents the user a maximally-incompatible counter-example from the already accumulated training set. The primary contribution of the paper lies in choosing an informative counter-example using Fischer information matrix as an approximation of influence functions. CINCER enables the user to correct both incoming and old examples.