Interactive Label Cleaning with Example-based Explanations

1 Introduction . Label noise is a major issue in machine learning as it can lead to compromised predictive performance and unreliable models [ 1 , 2 ] . We focus on sequential learning settings in which a human supervisor , usually a domain expert , can be asked to double-check and relabel any potentially mislabeled example . Applications include crowd-sourced machine learning and citizen science , where trained researchers can be asked to clean the labels provided by crowd-workers [ 3 , 4 ] , and interactive personal assistants [ 5 ] , where the user self-reports the initial annotations ( e.g. , about activities being performed ) and unreliability is due to memory bias [ 6 ] , unwillingness to report [ 7 ] , or conditioning [ 8 ] . This problem is often tackled by monitoring for incoming examples that are likely to be mislabeled , aka suspicious examples , and ask the supervisor to provide clean ( or at least better ) annotations for them . Existing approaches , however , focus solely on cleaning the incoming examples [ 9 , 4 , 10 , 5 ] . This means that noisy examples that did not undergo the cleaning step ( e.g. , those in the initial 35th Conference on Neural Information Processing Systems ( NeurIPS 2021 ) . bootstrap data set ) or that managed to elude it are left untouched . This degrades the quality of the model and prevents it from spotting future mislabeled examples that fall in regions affected by noise . We introduce CINCER ( Contrastive and InflueNt CounterExample stRategy ) , a new explainable interactive label cleaning algorithm that lets the annotator observe and fix the reasons behind the model ’ s suspicions . For every suspicious example that it finds , CINCER identifies a counter-example , i.e. , a training example that maximally supports the machine ’ s suspicion . The idea is that the example/counter-example pair captures a potential inconsistency in the data—as viewed from the model ’ s perspective—which is resolved by invoking the supervisor . More specifically , CINCER asks the user to relabel the example , the counter-example , or both , thus improving the quality of , and promoting consistency between , the data and the model . Two hypothetical rounds of interaction on a noisy version of MNIST are illustrated in Figure 1 . CINCER relies on a principled definition of counter-examples derived from few explicit , intuitive desiderata , using influence functions [ 11 , 12 ] . The resulting counter-example selection problem is solved using a simple and efficient approximation based on the Fisher information matrix [ 13 ] that consistently outperforms more complex alternatives in our experiments . Contributions : Summarizing , we : 1 ) Introduce CINCER , an explanatory interactive label cleaning strategy that leverages example-based explanations to identify inconsistencies in the data—as perceived by the model—and enable the annotator to fix them . 2 ) Show how to select counter-examples that at the same time explain why the model is suspicious and that are highly informative using ( an efficient approximation of ) influence functions . 3 ) Present an extensive empirical evaluation that showcases the ability of CINCER of building cleaner data sets and better models . 2 Background . We are concerned with sequential learning under label noise . In this setting , the machine receives a sequence of examples z̃t : = ( xt , ỹt ) , for t = 1 , 2 , . . . , where xt ∈ Rd is an instance and ỹt ∈ [ c ] is a corresponding label , with [ c ] : = { 1 , . . . , c } . The label ỹt is unreliable and might differ from the ground-truth label y∗t . The key feature of our setting is that a human supervisor can be asked to double-check and relabel any example . The goal is to acquire a clean dataset and a high-quality predictor while asking few relabeling queries , so to keep the cost of interaction under control . The state-of-the-art for this setting is skeptical learning ( SKL ) [ 10 , 5 ] . SKL is designed primarily for smart personal assistants that must learn from unreliable users . SKL follows a standard sequential learning loop : in each iteration , the machine receives an example and updates the model accordingly . However , for each example that it receives , the machine compares ( an estimate of ) the quality of the annotation with that of its own prediction , and if the prediction looks more reliable than the annotation by some factor , SKL asks the user to double-check his/her example . The details depend on the implementation : in [ 10 ] label quality is estimated using the empirical accuracy for the classifier and the empirical probability of contradiction for the annotator , while in [ 5 ] the machine measures the margin between the user ’ s and machine ’ s labels . Our approach follows the latter strategy . Another very related approach is learning from weak annotators ( LWA ) [ 9 , 4 ] , which focuses on querying domain experts rather than end-users . The most recent approach [ 4 ] jointly learns a prediction pipeline composed of a classifier and a noisy channel , which allows it to estimate the noise rate directly . Moreover , the approach in [ 4 ] identifies suspicious examples that have a large impact on the learned model . A theoretical foundation for LWA is given in [ 9 ] . LWA is however designed for pool-based scenarios , where the training set is given rather than obtained sequentially . For this reason , in the remainder of the paper , we will chiefly build on and compare to SKL . Limitations of existing approaches . A major downside of SKL is that it focuses on cleaning the incoming examples only . This means that if a mislabeled example manages to elude the cleaning step and gets added to the training set , it is bound to stay there forever . This situation is actually quite common during the first stage of skeptical learning , in which the model is highly uncertain and trusts the incoming examples—even if they are mislabeled . The same issue occurs if the initial training set used to bootstrap the classifier contains mislabeled examples . As shown by our experiments , the accumulation of noisy data in the training set may have a detrimental effect on the model ’ s performance ( cf . Figure 2 ) . In addition , it can also affect the model ’ s ability to identify suspicious examples : a noisy data point can fool the classifier into trusting incoming mislabeled examples that fall close to it , further aggravating the situation . 3 Explainable Interactive Label Cleaning with CINCER . We consider a very general class of probabilistic classifiers f : Rd → [ c ] of the form f ( x ; θ ) : = argmaxy∈ [ c ] P ( y |x ; θ ) , where the conditional distribution P ( Y |X ; θ ) has been fit on training data by minimizing the cross-entropy loss ` ( ( x , y ) , θ ) = − ∑ i∈ [ c ] 1 { i = y } logP ( i |x , θ ) . In our implementation , we also assume P to be a neural network with a softmax activation at the top layer , trained using some variant of SGD and possibly early stopping . 3.1 The CINCER Algorithm . The pseudo-code of CINCER is listed in Algorithm 1 . At the beginning of iteration t , the machine has acquired a training set Dt−1 = { z1 , . . . , zt−1 } and trained a model with parameters θt−1 on it . At this point , the machine receives a new , possibly mislabeled example z̃t ( line 3 ) and has to decide whether to trust it . Following skeptical learning [ 5 ] , CINCER does so by computing the margin µ ( z̃t , θt−1 ) , i.e. , the difference in conditional probability between the model ’ s prediction ŷt : = argmaxy P ( y |xt , θt−1 ) and the annotation ỹt . More formally : µ ( z̃t , θt−1 ) : = P ( ŷt |xt , θt−1 ) − P ( ỹt |xt , θt−1 ) ( 1 ) The margin estimates the incompatibility between the model and the example : the larger the margin , the more suspicious the example . The example z̃t is deemed compatible if the margin is below a given threshold τ and suspicious otherwise ( line 4 ) ; possible choices for τ are discussed in Section 3.5 . If z̃t is compatible , it is added to the data set as-is ( line 5 ) . Otherwise , CINCER computes a counterexample zk ∈ Dt−1 that maximally supports the machine ’ s suspicion . The intuition is that the pair ( z̃t , zk ) captures a potential inconsistency in the data . For instance , the counter-example might be a correctly labeled example that is close or similar to z̃t but has a different label , or a distant noisy outlier that fools the predictor into assigning low probability to ỹt . How to choose an effective counter-example is a major focus of this paper and discussed in detail in Section 3.2 and following . Next , CINCER asks the annotator to double-check the pair ( z̃t , zk ) and relabel the suspicious example , the counter-example , or both , thus resolving the potential inconsistency . The data set and model are then updated accordingly ( line 9 ) and the loop repeats . 3.2 Counter-example Selection . Counter-examples are meant to illustrate why a particular example z̃t is deemed suspicious by the machine in a way that makes it easy to elicit useful corrective feedback from the supervisor . We posit that a good counter-example zk should be : D1 . Contrastive : zk should explain why z̃t is considered suspicious by the model , thus highlighting a potential inconsistency in the data . Algorithm 1 Pseudo-code of CINCER . Inputs : initial ( noisy ) training set D0 ; threshold τ . 1 : fit θ0 on D0 2 : for t = 1 , 2 , . . . do 3 : receive new example z̃t = ( xt , ỹt ) 4 : if µ ( z̃t , θt−1 ) < τ then 5 : Dt ← Dt−1 ∪ { z̃t } . z̃t is compatible 6 : else 7 : find counterexample zk using Eq . 12 . z̃t is suspicious 8 : present z̃t , zk to the user , receive possibly cleaned labels y′t , y ′ k 9 : Dt ← ( Dt−1 \ { zk } ) ∪ { ( xt , y′t ) , ( xk , y′k ) } 10 : fit θt on Dt D2 . Influential : if zk is mislabeled , correcting it should improve the model as much as possible , so to maximize the information gained by interacting with the annotator . In the following , we show how , for models learned by minimizing the cross-entropy loss , one can identify counter-examples that satisfy both desiderata . What is a contrastive counter-example ? We start by tackling the first desideratum . Let θt−1 be the parameters of the current model . Intuitively , zk ∈ Dt−1 is a contrastive counter-example for a suspicious example z̃t if removing it from the data set and retraining leads to a model with parameters θ−kt−1 that assigns higher probability to the suspicious label ỹt . The most contrastive counter-example is then the one that maximally affects the change in probability : argmaxk∈ [ t−1 ] { P ( ỹt |xt ; θ−kt−1 ) − P ( ỹt |xt ; θt−1 ) } ( 2 ) While intuitively appealing , optimizing Eq . 2 directly is computationally challenging as it involves retraining the model |Dt−1| times . This is impractical for realistically sized models and data sets , especially in our interactive scenario where a counter-example must be computed in each iteration . Influence functions . We address this issue by leveraging influence functions ( IFs ) , a computational device that can be used to estimate the impact of specific training examples without retraining [ 11 , 12 ] . Let θt be the empirical risk minimizer on Dt and θt ( z , ) be the minimizer obtained after adding an example z with weight to Dt , namely : θt : = argminθ 1 t ∑t k=1 ` ( zk , θ ) θt ( z , ) : = argminθ 1 t ( ∑t k=1 ` ( zk , θ ) ) + ` ( z , θ ) ( 3 ) Taking a first-order Taylor expansion , the difference between θt = θt ( z , 0 ) and θt ( z , ) can be written as θt ( z , ) − θt ( z , 0 ) ≈ · ( d d θt ( z , ) ∣∣ =0 ) . The derivative appearing on the right hand side is the so-called influence function , denoted Iθt ( z ) . It follows that the effect on θt of adding ( resp . removing ) an example z to Dt can be approximated by multiplying the IF by = 1/t ( resp . = −1/t ) . Crucially , if the loss is strongly convex and twice differentiable , the IF can be written as : Iθt ( z ) = −H ( θt ) −1∇θ ` ( z , θt ) ( 4 ) where the curvature matrixH ( θt ) : = 1t ∑t k=1∇2θ ` ( zk , θt ) is positive definite and invertible . IFs were shown to capture meaningful information even for neural networks and other non-convex models [ 12 ] . Identifying contrastive counter-examples with influence functions . To see the link between contrastive counter-examples and influence functions , notice that the second term of Eq . 2 is independent of zk , while the first term can be conveniently approximated with IFs by applying the chain rule : − 1 t− 1 ( d d P ( ỹt |xt ; θt−1 ( zk , ) ) ∣∣∣∣ =0 ) = − 1 t− 1 ( ∇θP ( ỹt |xt ; θt−1 ) > d d θt−1 ( zk , ) ∣∣∣∣ =0 ) ( 5 ) = − 1 t− 1 ∇θP ( ỹt |xt ; θt−1 ) > Iθt−1 ( zk ) ( 6 ) The constant can be dropped during the optimization . This shows that Eq . 2 is equivalent to : argmaxk∈ [ t−1 ] ∇θP ( ỹt |xt ; θt−1 ) > H ( θt−1 ) −1∇θ ` ( zk , θt−1 ) ( 7 ) Eq . 7 can be solved efficiently by combining two strategies [ 12 ] : i ) Caching the inverse Hessian-vector product ( HVP ) ∇θP ( ỹt |xt ; θt−1 ) > H ( θt−1 ) −1 , so that evaluating the objective on each zk becomes a simple dot product , and ii ) Solving the inverse HVP with an efficient stochastic estimator like LISSA [ 14 ] . This gives us an algorithm for computing contrastive counter-examples . Contrastive counter-examples are highly influential . Can this algorithm be used for identifying influential counter-examples ? It turns out that , as long as the model is obtained by optimizing the cross-entropy loss , the answer is affirmative . Indeed , note that if ` ( z , θ ) = − logP ( y |x ; θ ) , then : ∇θP ( ỹt |xt ; θt−1 ) = P ( ỹt |xt ; θt−1 ) ∇θP ( ỹt |xt ; θt−1 ) P ( ỹt |xt ; θt−1 ) = ( 8 ) = P ( ỹt |xt ; θt−1 ) ∇θ logP ( ỹt |xt ; θt−1 ) = −P ( ỹt |xt ; θt−1 ) ∇θ ` ( z̃t , θt−1 ) ( 9 ) Hence , Eq . 6 can be rewritten as : − P ( ỹt |xt ; θt−1 ) ∇θ ` ( z̃t , θt−1 ) > H ( θt−1 ) −1∇θ ` ( zk , θt−1 ) ( 10 ) ∝ −∇θ ` ( z̃t , θt−1 ) > H ( θt−1 ) −1∇θ ` ( zk , θt−1 ) ( 11 ) It follows that , under the above assumptions and as long as the model satisfies P ( ỹt |xt ; θt−1 ) > 0 , Eq . 2 is equivalent to : argmaxk∈ [ t−1 ] −∇θ ` ( z̃t , θt−1 ) > H ( θt−1 ) −1∇θ ` ( zk , θt−1 ) ( 12 ) This equation recovers exactly the definition of influential examples given in [ 12 , Eq . 2 ] and shows that , for the large family of classifiers trained by cross-entropy , highly influential counter-examples are highly contrastive and vice versa , so that no change to the selection algorithm is necessary .

Summary: The paper tackles the problem of label cleaning, where labels for incoming data points might be flawed. The authors propose an interactive learning approach, where the system builds counter examples for data points, which consist of pairs of data points which the model finds incompatible. The human expert is then asked to relabel either or both of these presented data points.  The empirical evaluation applies the method to tabular- and image data, comparing the proposed approach with skeptical learning and a variation of the proposed approach without counter examples. The results show that the approach outperforms its competitors in terms of F1 score. The authors also validate that using the fisher information matrix supports finding good counter examples.