<figure><img src="image_1.png" /></figure>

HAL
open science

# Active Reinforcement Learning for Data Preparation: Learn2Clean with Human-In-The-Loop

Laure Berti-Equille

<figure><img src="image_2.png" /></figure>

## To cite this version:

Laure Berti-Equille. Active Reinforcement Learning for Data Preparation: Learn2Clean with Human-In-The-Loop. Conference on Innovative Data Systems Research (CIDR 2020), Jan 2020, Amsterdam (NETHERLANDS), Netherlands. hal-04822531

HAL Id: hal-04822531

https://hal.science/hal-04822531v1

Submitted on 18 Dec 2024

HAL is a multi-disciplinary open access
archive for the deposit and dissemination of sci-
entific research documents, whether they are pub-
lished or not. The documents may come from
teaching and research institutions in France or
abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est
destinée au dépôt et à la diffusion de documents
scientifiques de niveau recherche, publiés ou non,
émanant des établissements d'enseignement et de
recherche français ou étrangers, des laboratoires
publics ou privés.

<figure><img src="image_3.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
# Active Reinforcement Learning for Data Preparation:
## Learn2Clean with Human-In-The-Loop

Laure Berti-Equille

ESPACE-DEV/IRD, UMR 228, IRD/UM/UG/UR, Montpellier, France

Aix Marseille Université, Université de Toulon, CNRS, LIS, DIAMS, Marseille, France

laure.berti@ird.fr

## ABSTRACT

Data cleaning and data preparation are challenging but necessary tasks to prevent incorrect results, biases, and misleading conclusions to be obtained from “dirty” data. For a given ML model and a dataset, a plethora of data preprocessing techniques and alternative data cleaning strategies with various configurations are available, but they may lead to dramatically different outputs with unequal result quality performances.

As illustrated in Fig. 1, data cleaning and preparation require a sophisticated sequence of tasks for the detection and elimination of a variety of intricate data quality problems (e.g., duplicates, inconsistent, missing, and outlying values). Generally, the users may not know which preprocessing methods can be applied to optimize the final results downstream. This would require executing all possible methods for each task of preprocessing, as well as all the possible combinations of the methods with different orderings and configurations. These methods can be applied to the whole or some parts of the dataset with eventual re-iterations. AutoML approaches can optimize the hyper-parameters of a considered ML model, but they support only a limited number of preprocessing steps with by-default methods. Recently, Alpine Meadow [3] combines an AutoML approach and a cost model to select candidate logical ML pipeline plans (as in DB query optimization). Multi-armed bandits are used to select promising logical ML pipeline plans, and Bayesian Optimization is used to fine-tune the hyper-parameters of the selected models in the search space. Although Alpine Meadow proposes an Adaptive Pipeline Selection (APS) method to find a trade-off between speed and accuracy of ML pipeline plan evaluation and returns the results progressively, the human expertise is not actually exploited for building or pruning the data preparation pipeline search space, and the user has a passive role with limited interaction. We argue that data cleaning and preparation are intrinsically “AI-hard” as they can hardly be achieved by a fully automated system, and more efforts should be devoted to proposing a principled and efficient data preparation approach to help and learn from the user in selecting the optimal sequence of data preparation tasks. Improving the quality of input data for ML and leveraging human expertise, subsequent learning performance will benefit.

<figure><img src="image_4.png" /><figcaption>Figure 1: Data preparation with Learn2Clean+HIL</figcaption></figure>

As the first step in this direction, we have proposed Learn2Clean in [1], a method based on Q-Learning, a model-free reinforcement learning technique for automating data preparation.

In this abstract, we present Learn2Clean+HIL, a novel contribution enhancing Learn2Clean with the “human-in-the-loop”. Our exploration strategy uses active reward learning to leverage existing knowledge and user’s feedback [2]. The goal is to reduce the data preparation pipeline search space. Learn2Clean+HIL selects, for a given dataset, a given ML model, and a preselected quality performance metric, the optimal sequence of tasks for preprocessing the data such that the quality metric is maximized with the help of the user. Interactive response times can improve the rate at which insights are discovered in ML. This is precisely what Learn2Clean+HIL proposes in leveraging the user expertise in learning how to clean data by active reinforcement.

## 1. REFERENCES

[1] L. Berti-Equille. Learn2Clean: Optimizing the Sequence of Tasks for Web Data Preparation. In Proc. of the The Web Conf 2019, 2019.

[2] L. Berti-Équille. Reinforcement learning for data preparation with active reward learning. In Internet Science - 6th International Conference, INSCI 2019, Perpignan, France, December 2-5, 2019, Proceedings, pages 121–132, 2019.

[3] Z. Shang, E. Zgraggen, B. Buratti, F. Kossmann, P. Eichmann, Y. Chung, C. Binnig, E. Upfal, and T. Kraska. Democratizing data science through interactive curation of ML pipelines. In Proceedings of the 2019 International Conference on Management of Data, SIGMOD '19, pages 1171–1188, 2019.

This article is published under a Creative Commons Attribution License (http://creativecommons.org/licenses/by/3.0/), which permits distribution and reproduction in any medium as well as allowing derivative works, provided that you attribute the original work to the author(s) and CIDR 2020. 10th Annual Conference on Innovative Data Systems Research (CIDR '20). January 12-15, 2020, Amsterdam, Netherlands.