Reinforcement Learning for Data Cleaning and Data Preparation
Year: 2019

Data cleaning and data preparation have been long standing challenges in data science to avoid incorrect results, biases, and misleading conclusions obtained from "dirty" data. For a given dataset and a given analytics task, a plethora of data preprocessing techniques and alternative data cleaning strategies are available, but they may lead to dramatically different outputs with unequal ML model quality performances. For adequate data preparation, the users generally do not know how to start or which methods to use. Most current work focus either on proposing new data cleaning algorithms often specific to certain types of data glitches considered in isolation and generally with no "pipeline vision" of the whole data preprocessing sequence or on developing automated machine learning approaches (AutoML) that can optimize the hyper parameters of the given ML model but that often rely on by default preprocessing methods. We argue that more efforts should be devoted to proposing a principled data preparation approach to help and learn from the users for selecting the optimal sequence of data curation tasks and obtain the best quality performance of the final result. In this abstract, we present Learn2Clean1, a method based on Q Learning, a model free reinforcement learning technique that selects, for a given dataset, a given ML model, and a quality performance metric, the optimal sequence of tasks for preprocessing the data such that the quality metric is maximized. Learn2Clean has been presented in The Web Conf 2019  and we will discuss Learn2Clean enhancements for semi automated data preparation guided by the user.  In a 2017 survey conducted by the data science community Kaggle, "dirty data" is the common answer for 49.4% members responding to the questionnaire, when asked about the biggest barriers faced in data science2. As mitigation, data preparation often accounts for about 80% of the work of data scientists. As scientists, they would rather be deriving new knowledge and insights. The paradox is that without principled and adequate data preparation, those new insights are suspect at best. Data preparation and data cleaning are known to be very challenging tasks that require detection and elimination of a variety of complex data quality problems, such as duplicate records, inconsistent, missing, or outlying values. A wide range of methods for statistical analysis  constraint mining and checking  entity matching [5, 7] and machine learning [6, 8, 10] are used nowadays for data quality checks, data cleaning, and data repairing  However, most existing systems suffer from important limitations. First, data preprocessing include many steps (e.g. normalization, transformation, encoding, discretization, imputation, deduplication, pattern or rule enforcement, feature selection/engineering, etc. but current systems generally provide support 1https:/github.com/LaureBerti/Learn2Clean 2https:/www.kaggle.com/surveys/2017 for a limited number of steps in this pipeline. Second, for a given data preprocessing step, current systems generally rely on a few bydefault methods. The users have to try and test the methods in order to select the most appropriate ones in a tedious and time consuming process; to extend the system with another and eventually more adequate method, the users will have to write code or include other libraries, e.g. imputing missing values can be based on median, mean, most frequent values as default methods, but multiple imputations by chained equations (MICE) may be more accurate, but it is generally not included. Third, the systems do not recommend the adequate preprocessing methods for a given dataset and ML task, nor the execution sequence (or ordering) of the methods, e.g. imputation of missing values can be done after or before deduplication, and this may lead to a dramatically different preprocessed dataset. While some AutoML systems  can find the best model and configuration of hyper parameters, the optimization has not been yet applied to the entire data preprocessing pipeline which remains fixed and based on by default methods and orderings. Fourth, the users do not know which preprocessing methods can be applied to optimize the final results downstream. A solution would require executing all possible methods for each step of preprocessing, as well as all possible combinations and orderings of the methods. Finally, they may not have generic solutions from previous datasets or data cleaning tasks from which a system could learn a model to automate data preparation for any dataset specifics. For these reasons, we believe that data preparation is a novel application of reinforcement learning where model free methods may be suitable for exploring data preparation space. Human In the Loop Computational Model. Learn2Clean is based on Q learning which is a model free reinforcement method that learns the transition probability T (s1|(s0, a) from the pair of current state s0 and action a to the next state s1. In the context of data preparation, the transition probabilities and the dynamics of the system are not given a priori, and the space of possible states and actions is very large. Learn2Clean learns through trial anderror experience in an unsupervised way. It explores from state to state until it reaches the goal (i.e. to maximize the user defined quality metric) Each exploration is equivalent to one training session in which the system explores the data curation graph possibilities, receives the reward (if any) until it reaches the goal state. We have recently enhanced Learn2Clean approach to leverage user's input so that it can determine the next preprocessing action in conformance with the user's input and without waiting until the end of the episode (i.e. the entire cleaning pipeline) This makes the system agile and very performant compared to other data preparation strategies as it can progressively guide and be guided by the user during data preprocessing.