T able D reamer: Progressive and Weakness-guided Data Synthesis from Scratch for Table Instruction Tuning (2025)

Abstract: Despite the commendable progress of recent LLM-based data synthesis methods, they face two limitations in generating table instruction tuning data. First, they can not thoroughly explore the vast input space of table understanding tasks, leading to limited data diversity. Second, they ignore the weaknesses in table understanding ability of the target LLM and blindly pursue the increase of data quantity, resulting in suboptimal data efficiency. In this paper, we introduce a progressive and weakness-guided data synthesis framework tailored for table instruction tuning, named TableDreamer, to mitigate the above issues. Specifically, we first synthesize diverse tables and related instructions as seed data, and then perform an iterative exploration of the input space under the guidance of the newly identified weakness data, which eventually serve as the final training data for fine-tuning the target LLM. Extensive experiments on 10 tabular benchmarks demonstrate the effectiveness of the proposed framework, which boosts the average accuracy of Llama3.1-8B-instruct by 11.62% ( 49.07\rightarrow60.69 ) with 27K GPT-4o synthetic data and outperforms state-of-the-art data synthesis baselines which use more training data.

## introduction table understanding technique aims to enable models to automatically comprehend tables and complete various table-related tasks @xcite . with the recent advancement of large language models (llms), the dominant paradigm for table understanding has shifted to instruction tuning general llms with tabular task data, leading to the rise of powerful tabular llms @xcite ). although existing data synthesis approaches have achieved commendable performance, they still face two limitations in generating table instruction tuning data. first, existing data synthesis methods are unable to fully explore the vast input space composed of input tables and instructions, leading to limited data diversity. on the one hand, general data generation methods like self-instruct @xcite primarily focus on generating unstructured text data, and they did not adequately consider the unique characteristics of structured tables (e.g., diverse table structures, different table formats). as a result, they tend to produce simple tables and instructions of limited tabular tasks. on the other hand, existing studies on tabular llms only explore how to synthesize more instructions based on directly available tables from public datasets to improve instruction diversity, but they lack the ability to synthesize more diversified tabular data, which also limits the diversity of the final table instruction tuning data. second, existing data synthesis methods ignore the llm's weaknesses in table understanding ability, resulting in suboptimal efficiency of synthetic data. the combination of the input table and the instruction allows us to easily create a large amount of table instruction tuning data, e.g., we can utilize an llm to generate dozens of questions based on a single table. however, published studies have indicated that merely pursuing an increase in the quantity of instruction tuning data does not necessarily yield performance improvement @xcite . given the vast input space for table understanding tasks, it is more efficient to synthesize valuable data points that expose the deficiencies of the target llm, rather than blindly increase the amount of synthetic data, which may result in both a waste of training resources and a decline in model performance. to address these issues, we introduce a progressive and weakness-guided data synthesis framework for table instruction tuning, named table-dreamer, which can not only generate diverse tables and instructions from scratch, but can also continuously explore the input space under the guidance of newly identified weakness data to more effectively enhance the model performance. as illustrated in figure 2 , our framework consists of two stages. in stage 1, we first synthesize various table titles of different topics and subtopics, and then employ the llm to create diverse tables. in stage 2, based on synthetic tables and tabular task descriptions, a group of seed data is generated and will undergo data evolution in three directions. the synthesized new samples are evaluated by llm-asa-judge to identify weakness-exposing data, which is used as the seed data for the next round of data evolution. this process can be iterated multiple times, with the accumulated weakness data serving as the final table instruction tuning data. we compare tabledreamer with a series of data synthesis methods, general llms and tabular llms on 10 tabular benchmarks. as shown in figure 1 , experimental results demonstrate the effectiveness of the proposed framework, which boosts the average accuracy of llama3.1-8b-instruct by 11.62% (49.07% → 60.69%) with 27k gpt-4o synthetic data and outperforms the state-of-the-art data synthesis baselines that use more training data (100k+). we also demonstrate the effectiveness of tabledreamer as data augmentation for the fewshot learning scenario, where only a small number of original training samples are available (e.g., 20 samples for each benchmark). extensive ablation experiments are conducted to reveal the contributions of different components in the framework (e.g., the influence of weakness data selection and data evolution). we hope this work could establish a strong base for future research on the table instruction tuning data synthesis and help researchers improve models' table understanding ability especially with limited annotation budget. we conclude our contributions as follows: ## conclusion this paper introduces a novel data synthesis framework for table instruction tuning, which can generate diverse tables together with instructions spanning a wide range of tabular tasks, without relying on any public datasets. at the core of the proposed tabledreamer framework lies the iterative collaboration between input space exploration and weakness data identification. on the basis of table-dreamer, we construct and release 27k synthetic data, which can effectively enhance llms' table understanding ability and outperforms strong baselines. in conclusion, this paper promotes the research of data synthesis for the important table instruction tuning with the new method, dataset and thorough empirical study.