P - TA : Using Proximal Policy Optimization to Enhance Tabular Data Augmentation via Large Language Models (2024)

Abstract: A multitude of industries depend on accurate and reasonable tabular data augmentation for their business processes. Contemporary methodologies in generating tabular data revolve around utilizing Generative Adversarial Networks (GAN) or fine-tuning Large Language Models (LLM). However, GAN-based approaches are documented to produce samples with common-sense errors attributed to the absence of external knowledge. On the other hand, LLM-based methods exhibit a limited capacity to capture the disparities between synthesized and actual data distribution due to the absence of feedback from a discriminator during training. Furthermore, the decoding of LLM-based generation introduces gradient breakpoints, impeding the backpropagation of loss from a discriminator, thereby complicating the integration of these two approaches. To solve this challenge, we propose using proximal policy optimization (PPO) to apply GANs, guiding LLMs to enhance the probability distribution of tabular features. This approach enables the utilization of LLMs as generators for GANs in synthesizing tabular data. Our experiments demonstrate that PPO leads to an approximately 4% improvement in the accuracy of models trained on synthetically generated data over state-of-the-art across three real-world datasets.

## introduction with the evolution of business processes that operate on structured data, tabular data has emerged as one of the most crucial data forms, owing to its easily manageable structure and efficient retrievability @xcite . industries such as finance and healthcare heavily rely on tabular data for its clarity and ability to facilitate data analyses and comparisons @xcite . however, in realistic applications, the availability of high-quality tabular data is still an issue, mainly due to high costs of data collection and annotation and privacy policies @xcite . therefore, various data augmentation methods have been proposed to overcome the scarcity of high-quality tabular data @xcite . the contemporary augmentation methods primarily revolve around rules, variational autoencoders (vae), generative adversarial networks, and llms @xcite @xcite @xcite . however, these methods exhibit crucial limitations: (1) rule-based methods rely on predefined constraints, which need manual effort and might not capture the diversity of real-world data @xcite . (2) the existing vaebased and gan-based approaches primarily focus on matching the distribution of table features rather than whether they are logically coherent @xcite . for example, they might synthesize a sample with an "age" of "16" and an "occupation" of "professor" @xcite , resulting in implausible instances due to the lack of external knowledge @xcite . (3) fine-tuning a llm on real-world datasets may not capture the real-time differences in the distribution of features between synthetic and real-world data, thereby diminishing the quality of synthesized data. furthermore, the generation of tabular features via llms is implemented through random sampling of the logits of decoders. however, sampling operations are non-differentiable, making it challenging to incorporate gan-based methods for optimizing a generator, i.e., the llm, based on the discriminator's output. to overcome all of these drawbacks, we propose a method of training ppo-guided language models for table @xcite . specifically, we first transform the tabular data into text using templates. after that, we fine-tune a llm to generate new data in textual form. additionally, we train a classifier to distinguish between actual and gener-ated data, utilizing the outputs of the classifier as rewards for further optimizing the llm through the ppo algorithm @xcite . our methodology offers a distinct advantage: ppo-guided lms can introduce external knowledge, and extensively explore various potential feature combination strategies during training by incorporating gans, consequently enhancing the likelihood of producing high-quality synthetic data @xcite . as an additional study for reasonability, we propose to clarify the rationale behind tabular features to enhance the credibility of synthetic data by generating explanations. explanations can serve as a data auditing tool, specifically for assessing the quality of synthesized data. the data auditing, in turn, contributes to enhancing the accountability of models trained using the synthesized data @xcite . we found the textual explanations are empirically more helpful for data auditors in assessing data quality than plain tables (see user study in our experiments). we are the first to integrate data auditing into tabular augmentation, distinguishing our work from previous efforts. our contributions are as follows: ## related work with the development of databases, diverse sectors widely use tabular data as an indispensable data form @xcite . however, the accessibility of high-quality tabular data remains a persistent challenge, which is primarily attributed to privacy constraints and the substantial costs in collection @xcite . these factors reflect the critical need to propose reasonable tabular synthesizing and auditing technologies. ## methodology given a table with n rows and m columns, where each row represents a sample, and each column represents a feature. we denote this table as @xmath0, s 2 , ..., s n ), where the i-th sample row is represented as s @xmath1, a i2 , ..., a im ), with a ij denoting the j-th feature column of this sample. regarding data augmentation, our target is to generate a new table, denoted as t ′ = (s ′ 1 , s ′ 2 , ..., s ′ k ), which contains k synthetic samples. the synthetic dataset should exhibit a distribution that resembles the original dataset t . as for data auditing, our target is generating textual explanations for the plausibility of a given feature value a ′ ij in an arbitrary new sample s ′ i . ## limitation during the training phase, as ppo involves the concurrent utilization of a discriminator and two policies, the time complexity for a single epoch is 2 * o(k 1 n)+o(k 2 n), where k 1 and k 2 represent the time complexity of the generation policy and discriminator in training, respectively, and n denotes the number of features in the tabular data. in con-trast, the time complexity for baselines generated based on language models is o(k 1 n). in inference, our generation policy shares the same linear time complexity as the table generator great, which is based on a language model. utilizing a single a100 gpu, generating 1,000 heloc tabular samples, each consisting of 24 features, incurs an average per-sample generation time of only 19.31 milliseconds. in summary, despite our model exhibiting higher training time complexity, it adequately meets the requirements of real-world applications. ## conclusion our research tackles critical challenges in tabular data usage, focusing on integrating language models into generative adversarial networks for data augmentation. specifically, 1) proximal policy optimization can leverage a discriminator's outputs to optimize a language model's generation strategy, thereby effectively leading to a substantial boost in the plausibility of generated data. 2) we successfully generated explanations for synthetic tabular features to assist data audits. human evaluations highlight the clarity and the power of our explanations generated for data audits. these contributions could transform how industries obtain and audit tabular data, facilitating more informed decisionmaking and refined data-driven processes. terms of l test . our analysis indicates that the ppo algorithm encounters challenges associated with overfitting in more prevalent data modes. nevertheless, the adverse effects induced by ppo are relatively constrained, given its potential for a more pronounced enhancement in other utilized metrics.