Towards General Robustness to Bad Training Data

1 INTRODUCTION . The quality of training data is a fundamental ingredient towards useful and reliable ML-based applications . Unfortunately , there are unaccountably many possible data issue types . For example , errors and bias occur frequently in data generation and collection processes . Bad data could also be caused by adversarial attacks ( e.g. , data poisoning and backdoor attacks ) , as training data are often collected from anonymous and unverified sources . On the other hand , most of the existing data selection strategies only applicable to specific data issues . Due to the diversity of data issues , there is an urgent need to achieve general robustness to bad data of various types and even unknown types . Purging bad data is a long-standing problem , intensively studied by both the database and the ML community . Most of the existing approaches , however , can only achieve specialized robustness to bad data of certain types . In the database community , the state-of-the-art data cleaning approaches such as ActiveClean ( Krishnan et al. , 2016 ) , BoostClean ( Krishnan et al. , 2017 ) , AlphaClean ( Krishnan and Wu , 2019 ) , and CPClean ( Karlaš et al. , 2020 ) are only applicable to certain classes of ML models and data issues . In particular , they can not detect adversarial attacks on training data . The endeavors of the ML community have covered a wider range of data quality issues such as adversarial attacks ( Wang et al. , 2019 ; Chen et al. , 2019 ) , data debiasing ( Zemel et al. , 2013 ; Madras et al. , 2018 ) , and mislabel detection ( Zhao et al. , 2019 ) . However , each aforementioned solution only focuses on addressing a specific data issue effectively . The effectiveness of these approaches is based on the premise that the data quality issues are known a priori , which usually does not hold in reality . Few recent works have the potential to achieve general robustness to unknown data issues . One line of such works is based on differentially private training ( Du et al. , 2019 ; Hong et al. , 2020 ) . These approaches do not perform data selection ; instead , they attempt to restrict the impact of each training data point on the learning outcome in an undifferentiated manner . As a side effect , these approaches hinder learning from good data , thereby leading to poor learning performance in practice ( Tramèr and Boneh , 2020 ) . Another line of works that has potential to achieve general robustness is based on data valuation . These works first adopt a data importance metric , e.g. , influence function ( Koh and Liang , 2017 ; Koh et al. , 2019 ; Feldman and Zhang , 2020 ) or Shapley value ( Ghorbani and Zou , 2019 ; Jia et al. , 2019a ) , to quantify each training data point ’ s contribution to the training process . Then , which data to retain or remove is decided based on the ranking of data value . However , past empirical studies ( Jia et al. , 2019b ) and our experiments ( Section 6 ) show that their performance varies considerably across different data and learning algorithms . Overall , existing approaches to bad data filtering either can not identify unknown data issues by design or suffer from poor detection efficacy . In this work , we take a step towards general robustness to bad training data . Our work is underpinned by a key insight about what defines “ bad training data ” . Despite the diverse types of data issues , the crucial commonality is that all bad data contribute little to achieving good model performance . If a data point contributes positively to learning , it would be beneficial to just keep it as part of the training set , thus not considered a “ bad ” point . Hence , a promising way to select data without knowing data issue types in advance is to search for data subsets that results in the highest trained model performance . More generally , the data analyst may have a utility function beyond model performance , so we formulate data selection as a utility optimization problem ( Section 3 ) . Secondly , with the utility optimization objective in mind , we present a novel theoretical framework ( Section 4 ) for rigorously analyzing the worst-case performance of data selection approaches . The line of existing works closest to achieving general robustness is the aforementioned data value ranking approaches . However , we show that these approaches have unsatisfying worst-case performance guarantees due to failure to capture the interactions amongst selected data points ( also empirically shown in Appendix F.5.2 ) . In particular , the popular Shapley value-based approach could select the worst data in some common scenarios . We then design a general algorithmic framework guided by the theoretically optimal ( but computationally infeasible ) solution to utility optimization ( Section 5 ) . A significant technical challenge for finding the subset that optimize the model utility is that , in order to evaluate the impact of different subsets of data on model performance , we need to retrain model on every possible subset . To solve the computational challenge , we introduce DATASIFTER , which directly learns a parametric function to predict the performance of a model trained on a given subset and then performs data selection via optimizing the function . Compared with prior data selection algorithms , DATASIFTER has the following advantages : ( 1 ) being able to handle various data issues ( general robustness ) , ( 2 ) applies to any target ML model architectures ( model-agnostic ) , and ( 3 ) instantiated by the goal of downstream ML tasks ( task-driven ) . Finally , we conduct a thorough empirical study on a range of data issues ( Section 6 ) , including backdoor and poisoning attack detection , noisy label/feature detection , data summarization , and data debiasing . Our experiments demonstrate that DATASIFTER achieves and most often significantly improves the state-of-the-art performance of data valuation-based approaches . 2 RELATED WORK . Data valuation-based approaches can potentially achieve general robustness against diverse data issues by quantifying data importance or “ data value ” , and then picking data points with high value for model training process . One simple idea to quantify data importance is to use the leave-one-out error . Koh and Liang ( 2017 ) provides an efficient algorithm to approximate leave-one-out error for each training point . Recent works leverage credit allocation schemes originated from cooperative game theory to quantify data importance . Particularly , Shapley value has been widely used ( Ghorbani and Zou , 2019 ; Jia et al. , 2019a ; c ; b ; Wang et al. , 2020 ) , as it uniquely satisfies a set of desirable axiomatic properties . More recently , Yan and Procaccia ( 2020 ) suggests that the Least core is also a viable alternative to Shapley value for measuring data importance . However , computing the exact Shapley and Least core values are generally NP-hard . Several approximation heuristics , such as TMC-Shapley ( Ghorbani and Zou , 2019 ) , G-Shapley ( Ghorbani and Zou , 2019 ) , KNN-Shapley ( Jia et al. , 2019c ) , have been proposed for the Shapley value . Despite their computational advantage , they are biased in nature . On the other hand , unbiased estimators such as Permutation Sampling ( Maleki , 2015 ) and Group Testing ( Jia et al. , 2019a ) still require retraining models many times for any decent approximation accuracy . TracIn ( Pruthi et al. , 2020 ) estimates the importance by tracing the test loss change caused by a training example during the training process . The representer point method ( Yeh et al. , 2018 ) captures the importance of training point by decomposing the pre-activation prediction of a neural network into a linear combination of activations of training points . The typical paradigm of data valuation-based approaches for selecting high-quality data ( or filtering bad data ) is straightforward : ( 1 ) each data point ’ s value ( e.g. , Shapley value ) is computed . ( 2 ) data points are sorted by value and data points with the highest value are selected . However , this paradigm fails capture the interactions amongst selected data points ( Section 4 and Appendix F.5.2 ) , i.e. , the existence of one data point often affect the importance of another . Differentially private training ( Du et al. , 2019 ; Hong et al. , 2020 ) have the potential to achieve general robustness to unknown data issues . However , fundamentally , these approaches diminish the influence of each training data on the learned model in an undifferentiated manner . Hence , these approaches hinder learning from good data , which lead to poor learning performance ( Tramèr and Boneh , 2020 ) . Data cleaning approaches from database community are not able to achieve general robustness . The state-of-the-art data cleaning approaches leverage the information about downstream ML tasks to guide the cleaning process ; examples include ActiveClean ( Krishnan et al. , 2016 ) , BoostClean ( Krishnan et al. , 2017 ) , AlphaClean ( Krishnan and Wu , 2019 ) , and CPClean ( Karlaš et al. , 2020 ) . However , the state-of-the-art data cleaning methods are only applicable to certain architectures of ML models ( e.g. , convex model , nearest neighbors ) , data format ( e.g. , tabular data ) , and data issues ( e.g. , missing values , outliers ) . In particular , data cleaning approaches can not be straightforwardly extended to adversarial attacks such as data poisoning attacks and backdoor attacks . 3 FORMALISM . We propose to formulate the problem of achieving general robustness to bad data as finding a subset of data points with the highest utility . We use the data utility function to characterize the mapping from a set of data points to its utility . More formally , let D = { ( xi , yi ) } ni=1 denote the training set with data points of different quality . A learning algorithm A is a function that takes a dataset S ⊆ D and outputs a classifier f̂ . A metric function u takes f̂ as input and outputs its model utility . In the machine learning context , we often use test accuracy as the metric , u ( f̂ , V ) = 1|V| ∑ ( x , y ) ∈V 1 [ f̂ ( x ) = y ] for a test set V . However , test set V is usually not available during the training time . In practice , u ( f̂ , V ) is typically approximated by validation accuracy u ( f̂ , V ) where V is a validation set separated from the training set . With a potentially randomized learning algorithmA and a corresponding metric function u , we define the data utility function as UA , u ( S ) = EA [ u ( A ( S ) , V ) ] . When the context is clear , we omit the subscript and simply write U ( S ) . The concept of data utility functions was originally discussed in Wang et al . ( 2021a ) , where it is used for active learning tasks . The critical difference between our formulation and Wang et al . ( 2021a ) is that our formulation incorporates the label information as the input to a data utility function . This is important as label information is required for identifying both mislabeled and many types of adversarial attacks on training data . With the modified notion of the data utility function , we abstract the objective of selecting high-quality data as a utility optimization problem : max S⊆D : |S|=k U ( S ) ( 1 ) where 0 < k < n indicates the selection budget , which can be predetermined ( e.g. , based on the prior knowledge about potential data defects or computational requirements ) . The fundamental intuition behind this formulation is that despite the diversity of data issue types , all “ bad data ” have the commonality that they all contribute little or negatively to model performance . Hence , optimizing data utility function is a principled way to achieve general robustness . Besides , the interactions between data points will significantly affect model performance . A “ good data ” can only be defined relative to the rest of the data points in the dataset . Therefore , high-quality data selection needs to be done in a batch-style instead of one-by-one . Overall , a promising way to deal with unknown data issues is to search for a set of data points that results in the highest trained model performance on a clean validation set .

Summary: This paper proposes, *DataSifter*, an optimization-based, general-purpose framework for filtering "bad data" from a training set.  General-purpose broadly covers different data corruption types (e.g., adversarial perturbation, label noise, etc.), different model architectures, and performance metrics (e.g., test error).