A uto DCW orkflow: LLM -based Data Cleaning Workflow Auto-Generation and Benchmark (2025)

Abstract: Data cleaning is a time-consuming and error-prone manual process even with modern workflow tools like OpenRefine. Here, we present AutoDCWorkflow, an LLM-based pipeline for automatically generating data-cleaning workflows. The pipeline takes a raw table coupled with a data analysis purpose, and generates a sequence of OpenRefine operations designed to produce a minimal, clean table sufficient to address the purpose. Six operations address common data quality issues including format inconsistencies, type errors, and duplicates.To evaluate AutoDCWorkflow, we create a benchmark with metrics assessing answers, data, and workflow quality for 142 purposes using 96 tables across six topics. The evaluation covers three key dimensions: (1) **Purpose Answer**: can the cleaned table produce a correct answer? (2) **Column (Value)**: how closely does it match the ground truth table? (3) **Workflow (Operations)**: to what extent does the generated workflow resemble the human-curated ground truth? Experiments show that Llama 3.1, Mistral, and Gemma 2 significantly enhance data quality, outperforming the baseline across all metrics. Gemma 2-27B consistently generates high-quality tables and answers, while Gemma 2-9B excels in producing workflows that resemble human annotations.

## introduction data curation and cleaning are critical processes that prepare raw data for analysis, ensuring high quality and reliability @xcite . implementing a sequence of data operations in the form of a data cleaning workflow improves data quality, enabling downstream analyses and yielding trustworthy results and actionable insights @xcite @xcite @xcite . to facilitate automation and reuse, ## approach architecture the complete autodcworkflow pipeline, shown in figure 2 , consists of two main stages: (1) llm agent planning and (2) data cleaning via openrefine execution. problem definition. given a table t and a data analysis purpose p, autodcworkflow aims to generate data cleaning workflow w n and the cleaned table t n . to achieve this, autodcworkflow begins by identifying a list of target columns l. then it iteratively selects and cleans columns and removes them from l once successfully cleaned. in the openrefine stage, the generated operations and arguments o(•) are executed to modify the input table into an intermediate table the updated table t i and workflow w i then serve as inputs for the next llm planning iteration. this process continues until l is empty, indicating that all target columns have been cleaned in alignment with p. the final cleaned table t n , produced by the workflow w n as t @xmath0, is claimed to be of high quality and capable of generating the correct answer a for the purpose p. ## benchmark demonstration the benchmark includes annotated datasets and evaluation dimensions to assess workflow quality. the benchmark dataset comprises data analysis purposes, answer sets, raw tables foot_0 , and manually curated tables and workflows. the statistic of the benchmark dataset is provided in ## experiments 6.1 experiment settings we use raw table inputs (without any prior data cleaning) and a direct prompting (dp) approach as the baseline. in the dp setting, the llm is instructed to generate the entire data cleaning workflow in a single step. unlike the iterative pipeline in autodcworkflow, dp receives the same context-including the list of available operations and example workflows-but lacks access to intermediate reasoning stages and multi-turn decisionmaking. we evaluate autodcworkflow using three widely adopted, open-access llms of comparable model sizes: llama 3.1-8b @xcite , gemma 2-9b @xcite , and mistral-7b @xcite . ## case study in this section, we further analyze how different models generate varying workflows by examining their outputs. we showcase how llama 3.1-8b and gemma 2-9b construct workflows with different operations while successfully repairing data to produce the correct answer. the analysis focuses on the specific purpose of identifying the facility type inspected multiple times on the most recent date. figure 4 visualizes the workflows using orma 6 , where each workflow is modeled as a graph with data nodes (colorful rounded rectangles representing columns) and operation nodes (green boxes representing applied operations) @xcite . the three branches in each diagram correspond to the target columns-facility type, inspection id, and inspection date-allowing a direct comparison of how each model processes them. facility type column: the ground truth workflow applies trim (1x), upper (1x), and mass_edit (3x). llama 3.1-8b correctly selects trim (1x) and mass_edit (2x), while gemma 2-9b consolidates all repairs into a single mass_edit operation. inspection id column: the ground truth workflow uses regexr_transform and numeric to standardize non-numeric ids. llama 3.1-8b uses mass_edit to accomplish the same result. inspection date column: both models detect date-time inconsistencies and apply the date operation for correction. despite using fewer steps and varying approaches, both llama 3.1-8b and gemma 2-9b effectively clean the table and produce the correct purpose answer: "restaurant". this demonstrates their ability to generate accurate workflows efficiently. 6 arrows indicate the flow of data cleaning operations across specific columns. ## conclusions this paper presents autodcworkflow, a framework to automatically generate data cleaning workflows using llms, supported by a new benchmark and evaluation metrics. our experiments across 96 datasets and 142 tasks show that all llm backbone models are capable of interpreting the given purpose and reasoning about appropriate data cleaning operations to resolve data quality issues. gemma 2-27b yields the most accurate cleaned tables and downstream answers, while gemma 2-9b produces workflows most consistent with human annotations. these findings underscore that different llms capture complementary aspects of data cleaning, making model choice an important consideration for different objectives. beyond performance comparisons, our iterative design demonstrates that llms can reason about and adapt to diverse data quality issues, moving beyond one-off cleaning heuristics. taken together, this work establishes a foundation for reproducible evaluation and highlights the broader potential of llm-driven approaches to generalize across domains, error types, and data cleaning pipelines. looking forward, we envision general-purpose llm agents that support end-toend data cleaning, making structured data more transparent, reusable, and analysis-ready.