Towards General Robustness to Bad Training Data

1 INTRODUCTION . The quality of training data is a fundamental ingredient towards useful and reliable ML-based applications . Unfortunately , there are unaccountably many possible data issue types . For example , errors and bias occur frequently in data generation and collection processes . Bad data could also be caused by adversarial attacks ( e.g. , data poisoning and backdoor attacks ) , as training data are often collected from anonymous and unverified sources . On the other hand , most of the existing data selection strategies only applicable to specific data issues . Due to the diversity of data issues , there is an urgent need to achieve general robustness to bad data of various types and even unknown types . Purging bad data is a long-standing problem , intensively studied by both the database and the ML community . Most of the existing approaches , however , can only achieve specialized robustness to bad data of certain types . In the database community , the state-of-the-art data cleaning approaches such as ActiveClean ( Krishnan et al. , 2016 ) , BoostClean ( Krishnan et al. , 2017 ) , AlphaClean ( Krishnan and Wu , 2019 ) , and CPClean ( Karlaš et al. , 2020 ) are only applicable to certain classes of ML models and data issues . In particular , they can not detect adversarial attacks on training data . The endeavors of the ML community have covered a wider range of data quality issues such as adversarial attacks ( Wang et al. , 2019 ; Chen et al. , 2019 ) , data debiasing ( Zemel et al. , 2013 ; Madras et al. , 2018 ) , and mislabel detection ( Zhao et al. , 2019 ) . However , each aforementioned solution only focuses on addressing a specific data issue effectively . The effectiveness of these approaches is based on the premise that the data quality issues are known a priori , which usually does not hold in reality . Few recent works have the potential to achieve general robustness to unknown data issues . One line of such works is based on differentially private training ( Du et al. , 2019 ; Hong et al. , 2020 ) . These approaches do not perform data selection ; instead , they attempt to restrict the impact of each training data point on the learning outcome in an undifferentiated manner . As a side effect , these approaches hinder learning from good data , thereby leading to poor learning performance in practice ( Tramèr and Boneh , 2020 ) . Another line of works that has potential to achieve general robustness is based on data valuation . These works first adopt a data importance metric , e.g. , influence function ( Koh and Liang , 2017 ; Koh et al. , 2019 ; Feldman and Zhang , 2020 ) or Shapley value ( Ghorbani and Zou , 2019 ; Jia et al. , 2019a ) , to quantify each training data point ’ s contribution to the training process . Then , which data to retain or remove is decided based on the ranking of data value . However , past empirical studies ( Jia et al. , 2019b ) and our experiments ( Section 6 ) show that their performance varies considerably across different data and learning algorithms . Overall , existing approaches to bad data filtering either can not identify unknown data issues by design or suffer from poor detection efficacy . In this work , we take a step towards general robustness to bad training data . Our work is underpinned by a key insight about what defines “ bad training data ” . Despite the diverse types of data issues , the crucial commonality is that all bad data contribute little to achieving good model performance . If a data point contributes positively to learning , it would be beneficial to just keep it as part of the training set , thus not considered a “ bad ” point . Hence , a promising way to select data without knowing data issue types in advance is to search for data subsets that results in the highest trained model performance . More generally , the data analyst may have a utility function beyond model performance , so we formulate data selection as a utility optimization problem ( Section 3 ) . Secondly , with the utility optimization objective in mind , we present a novel theoretical framework ( Section 4 ) for rigorously analyzing the worst-case performance of data selection approaches . The line of existing works closest to achieving general robustness is the aforementioned data value ranking approaches . However , we show that these approaches have unsatisfying worst-case performance guarantees due to failure to capture the interactions amongst selected data points ( also empirically shown in Appendix F.5.2 ) . In particular , the popular Shapley value-based approach could select the worst data in some common scenarios . We then design a general algorithmic framework guided by the theoretically optimal ( but computationally infeasible ) solution to utility optimization ( Section 5 ) . A significant technical challenge for finding the subset that optimize the model utility is that , in order to evaluate the impact of different subsets of data on model performance , we need to retrain model on every possible subset . To solve the computational challenge , we introduce DATASIFTER , which directly learns a parametric function to predict the performance of a model trained on a given subset and then performs data selection via optimizing the function . Compared with prior data selection algorithms , DATASIFTER has the following advantages : ( 1 ) being able to handle various data issues ( general robustness ) , ( 2 ) applies to any target ML model architectures ( model-agnostic ) , and ( 3 ) instantiated by the goal of downstream ML tasks ( task-driven ) . Finally , we conduct a thorough empirical study on a range of data issues ( Section 6 ) , including backdoor and poisoning attack detection , noisy label/feature detection , data summarization , and data debiasing . Our experiments demonstrate that DATASIFTER achieves and most often significantly improves the state-of-the-art performance of data valuation-based approaches . 2 RELATED WORK . Data valuation-based approaches can potentially achieve general robustness against diverse data issues by quantifying data importance or “ data value ” , and then picking data points with high value for model training process . One simple idea to quantify data importance is to use the leave-one-out error . Koh and Liang ( 2017 ) provides an efficient algorithm to approximate leave-one-out error for each training point . Recent works leverage credit allocation schemes originated from cooperative game theory to quantify data importance . Particularly , Shapley value has been widely used ( Ghorbani and Zou , 2019 ; Jia et al. , 2019a ; c ; b ; Wang et al. , 2020 ) , as it uniquely satisfies a set of desirable axiomatic properties . More recently , Yan and Procaccia ( 2020 ) suggests that the Least core is also a viable alternative to Shapley value for measuring data importance . However , computing the exact Shapley and Least core values are generally NP-hard . Several approximation heuristics , such as TMC-Shapley ( Ghorbani and Zou , 2019 ) , G-Shapley ( Ghorbani and Zou , 2019 ) , KNN-Shapley ( Jia et al. , 2019c ) , have been proposed for the Shapley value . Despite their computational advantage , they are biased in nature . On the other hand , unbiased estimators such as Permutation Sampling ( Maleki , 2015 ) and Group Testing ( Jia et al. , 2019a ) still require retraining models many times for any decent approximation accuracy . TracIn ( Pruthi et al. , 2020 ) estimates the importance by tracing the test loss change caused by a training example during the training process . The representer point method ( Yeh et al. , 2018 ) captures the importance of training point by decomposing the pre-activation prediction of a neural network into a linear combination of activations of training points . The typical paradigm of data valuation-based approaches for selecting high-quality data ( or filtering bad data ) is straightforward : ( 1 ) each data point ’ s value ( e.g. , Shapley value ) is computed . ( 2 ) data points are sorted by value and data points with the highest value are selected . However , this paradigm fails capture the interactions amongst selected data points ( Section 4 and Appendix F.5.2 ) , i.e. , the existence of one data point often affect the importance of another . Differentially private training ( Du et al. , 2019 ; Hong et al. , 2020 ) have the potential to achieve general robustness to unknown data issues . However , fundamentally , these approaches diminish the influence of each training data on the learned model in an undifferentiated manner . Hence , these approaches hinder learning from good data , which lead to poor learning performance ( Tramèr and Boneh , 2020 ) . Data cleaning approaches from database community are not able to achieve general robustness . The state-of-the-art data cleaning approaches leverage the information about downstream ML tasks to guide the cleaning process ; examples include ActiveClean ( Krishnan et al. , 2016 ) , BoostClean ( Krishnan et al. , 2017 ) , AlphaClean ( Krishnan and Wu , 2019 ) , and CPClean ( Karlaš et al. , 2020 ) . However , the state-of-the-art data cleaning methods are only applicable to certain architectures of ML models ( e.g. , convex model , nearest neighbors ) , data format ( e.g. , tabular data ) , and data issues ( e.g. , missing values , outliers ) . In particular , data cleaning approaches can not be straightforwardly extended to adversarial attacks such as data poisoning attacks and backdoor attacks . 3 FORMALISM . We propose to formulate the problem of achieving general robustness to bad data as finding a subset of data points with the highest utility . We use the data utility function to characterize the mapping from a set of data points to its utility . More formally , let D = { ( xi , yi ) } ni=1 denote the training set with data points of different quality . A learning algorithm A is a function that takes a dataset S ⊆ D and outputs a classifier f̂ . A metric function u takes f̂ as input and outputs its model utility . In the machine learning context , we often use test accuracy as the metric , u ( f̂ , V ) = 1|V| ∑ ( x , y ) ∈V 1 [ f̂ ( x ) = y ] for a test set V . However , test set V is usually not available during the training time . In practice , u ( f̂ , V ) is typically approximated by validation accuracy u ( f̂ , V ) where V is a validation set separated from the training set . With a potentially randomized learning algorithmA and a corresponding metric function u , we define the data utility function as UA , u ( S ) = EA [ u ( A ( S ) , V ) ] . When the context is clear , we omit the subscript and simply write U ( S ) . The concept of data utility functions was originally discussed in Wang et al . ( 2021a ) , where it is used for active learning tasks . The critical difference between our formulation and Wang et al . ( 2021a ) is that our formulation incorporates the label information as the input to a data utility function . This is important as label information is required for identifying both mislabeled and many types of adversarial attacks on training data . With the modified notion of the data utility function , we abstract the objective of selecting high-quality data as a utility optimization problem : max S⊆D : |S|=k U ( S ) ( 1 ) where 0 < k < n indicates the selection budget , which can be predetermined ( e.g. , based on the prior knowledge about potential data defects or computational requirements ) . The fundamental intuition behind this formulation is that despite the diversity of data issue types , all “ bad data ” have the commonality that they all contribute little or negatively to model performance . Hence , optimizing data utility function is a principled way to achieve general robustness . Besides , the interactions between data points will significantly affect model performance . A “ good data ” can only be defined relative to the rest of the data points in the dataset . Therefore , high-quality data selection needs to be done in a batch-style instead of one-by-one . Overall , a promising way to deal with unknown data issues is to search for a set of data points that results in the highest trained model performance on a clean validation set .

Summary: This paper focuses on the problem of identifying bad training data when the underlying cause is unknown in advance. Authors develop an algorithmic framework, DATASIFTER, for general robustness to bad training data. Empirical evaluation show efficacy of DATASIFTER in a wide range of tasks, including backdoor, poison, noisy/mislabel data detection, data summarization, and data debiasing