Title: Learning a Data-Driven Policy Network for Pre-Training Automated Feature Engineering
Authors: Liyao Li
Haobo Wang
Liangyu Zha
Qingyi Huang
Sai Wu
Gang Chen
Junbo Zhao
Venue: ICLR 2023
Keywords: Automated Feature Engineering
Reinforcement Learning
Tabular Data
Data-Driven
Pre-Training

Abstract: Feature engineering is widely acknowledged to be pivotal in tabular data analysis and prediction. Automated feature engineering (AutoFE) emerged to automate this process managed by experienced data scientists and engineers conventionally. In this area, most — if not all — prior work adopted an identical framework from the neural architecture search (NAS) method. While feasible, we posit that the NAS framework very much contradicts the way how human experts cope with the data since the inherent Markov decision process (MDP) setup differs. We point out that its data-unobserved setup consequentially results in an incapability to generalize across different datasets as well as also high computational cost. This paper proposes a novel AutoFE framework Feature Set Data-Driven Search (FETCH), a pipeline mainly for feature generation and selection. Notably, FETCH is built on a brand-new data-driven MDP setup using the tabular dataset as the state fed into the policy network. Further, we posit that the crucial merit of FETCH is its transferability where the yielded policy network trained on a variety of datasets is indeed capable to enact feature engineering on unseen data, without requiring additional exploration. To the best of our knowledge, this is a pioneer attempt to build a tabular data pre-training paradigm via AutoFE. Extensive experiments show that FETCH systematically surpasses the current state-of-the-art AutoFE methods and validates the transferability of AutoFE pre-training.

TL;DR: We propose a data-driven automated feature engineering framework Fetch.

Review discussion:
Strengths:
1. The proposed framework is novel, simple and effective, which is easily to follow. The pre-training technique seems like a new schema in the tabular data mining field.
2. The paper is well-written, and the related work is covered well.
3. The experiments are quite comprehensive. The authors also show that pre-training FETCH on more datasets results in additional performance gains, which further strengthens the claim of the model's transferability.
Weaknesses:
1. It is nice that the authors include 27 datasets as a benchmark and demonstrate that FETCH is much better than the other AutoFE baselines in most cases.
However, I don't quite sure why the authors are comparing it to popular open-source AutoML packages (i.e. AutoSklearn and AutoGluon)?
After all, these are two different domains. Is it to prove that AutoFE can power AutoML? Maybe I'm missing it, and please clarify.
2. Section 4.4 only covers 3 comparison algorithms, it would be better to add the enhancement effect on more algorithms such as LightGBM and CatBoost, which will be more convincing for the adaptability of FETCH on various algorithms.
3. FETCH seems to have been pre-trained on only 5 datasets, which seems somewhat simple. It would be preferable to have pre-training results on a larger number of datasets.
4. The theoretical analysis is a bit lacking. It is hoped that a theoretical analysis of effectiveness and experimental results can be provided as space permits.
This paper is overall well-written and clear. The code in the supplementary material seems to be well-organized and I believe it works as the paper claims, although I have not actually verified it.
NO.
Thank you for the good words! As a briefing, we added the following section in our revision to respond to the reviews: (i) we have added comparative experiments for models LightGBM and CatBoost in Appendix C.4. We provide a more detailed explanation and responses as the following.
**Q1:**
> I don't quite sure why the authors are comparing it to popular open-source AutoML packages (i.e. AutoSklearn and AutoGluon)?
After all, these are two different domains. Is it to prove that AutoFE can power AutoML?
**A:**
As discussed in Section 4.2, these two AutoML baselines focus primarily on model architecture/hyperparameters search and involve no feature engineering. We believe that these comprehensive comparisons will further highlight the effectiveness of FETCH and the boosting performance it can deliver, even beyond the scope of feature engineering.
**Q2:**
> Section 4.4 only covers 3 comparison algorithms, it would be better to add the enhancement effect on more algorithms such as LightGBM and CatBoost, which will be more convincing for the adaptability of FETCH on various algorithms.
**A:**
Thanks for your advice. We have added the comparison on LightGBM and CatBoost in Appendix C.4. FETCH still shows decent performance on both added algorithms ;)
**Q3:**
> FETCH seems to have been pre-trained on only 5 datasets, which seems somewhat simple. It would be preferable to have pre-training results on a larger number of datasets.
**A:**
Yes, indeed, five datasets for a pre-training seems overly simple. The main reason behind it is that we simply want to highlight the ability to transfer across tables from FETCH. Noted, this is unprecedented for tabular data feature engineering and modeling; FETCH can achieve it majorly thanks to its novel data-driven framework re-work.
True that the pre-training normally requires a massive amount of datasets (such as BERT, GPT, MoCo and others). We simply want to concept-prove that FETCH can be utilized as a preliminary attempt for this line of pre-training research on tabular data.
Hence a full large-scale pre-training may seem beyond the scope of this paper.
Regardless, we hope to dig further into this line in the future.
Of course, extending the five datasets to many is the very first step to take ;) Thank you!
**Q4:**
> The theoretical analysis is a bit lacking. It is hoped that a theoretical analysis of effectiveness and experimental results can be provided as space permits.
**A:**
Yes, the theoretical side of analysis or innovation is lacked from our paper.
In our defense, the theories around feature engineering for tabular data is still a blank, by scrutinizing the whole spectrum of this domain since the very early years.
However, we do agree with the reviewer that this aspect is important. We hope to study it in the near future.
Strengths:
The idea of using data directly is novel.
The use of data enables transfer learning.
Weaknesses:
Scalability is questionable. There is no deep discussion on computational times and comparisons with other algorithms.
Explainability/interpretability is questionable. To the contrary, the resulting training data is not interpretable.
The paper is in general well written. There are a few loose points but nothing major.
It is easy to read and comprehend.
Reproducibility is questionable. There are not many details about how to reproduce the results.
NO.
Thank you very much for your good words and suggestions! We are thrilled that you enjoyed our paper. We have updated our paper based on the suggestions and comments of the reviewers. Summary of revision: (i) we added scalability comparison in Appendix C.3. (ii) we added time efficiency comparison in Appendix C.4. (iii) we updated our supplementary material for more descriptive details.
**Q1:**
> Scalability is questionable. There is no deep discussion on computational times and comparisons with other algorithms.
**A:**
We have added a comparison of the effectiveness and computational time among Base, NFS, DIFER and FETCH on large-scale data (>50k instances) in Appendix C.3.
Besides, we reported the running time of NFS, DIFER and FETCH in Appendix C.4. As can be concluded from these results, FETCH possesses the time complexity at same-scale with other approaches, but it relishes the significant performance boost.
**Q2:**
> Explainability/interpretability is questionable. To the contrary, the resulting training data is not interpretable.
**A:**
We wholeheartedly agree with the reviewer that the explainability/interpretability is quite important for tabular data modeling. As we pointed out in the paper, the goal of AutoFE is to explicitly generate new features using multiple features for computation. The new features will be named by the computational path.
For example, when a Body Mass Index feature is generated, the resulting training data will contain a column named weight/height$^2$. This explicit way of showing new feature generation paths would be more interpretable than deep tabular models (like TabNet, AutoInt) due to their internal implicit feature intersections.
In that regard, our tentative solution to the explainability/interpretability of FETCH is to: (i)-first delve into the downstream model such as XGBoost or a Random Forest and attain the ``feature_importance_`` score; (ii)-then based on our explicit feature generation tree, we could further decompose the feature importance downwards to the original feature of the given table input.
**Q3:**
> Reproducibility is questionable. There are not many details about how to reproduce the results.
**A:**
Sorry, our bad! As we stated in the opening general comment, we have added more experimental reproduction settings and updated the code. We may refer the reviewer for the Appendix B and the supplementary material for more descriptive details.
**Q4:**
> The authors don't mention feature selection (forward/backward selection, etc) which is perhaps injustice. Maybe feature selection should also be benchmarks (perhaps autoML methods include them).
**A:**
This is perhaps a writing confusion, our apologies!
In effect, the AutoFE family mostly integrates the feature selection operations into the feature manipulation procedure, e.g., None, Terminate, Delete (see Appendix A.1).
In other words, the feature selection has been encapsulated into the feature engineering process;
namely in FETCH and all other competing methods.
For the other point the reviewer raised, the AutoML method incorporated and mentioned in our paper does not implement feature generation or selection.
Rather, these operations are ubiquitously pushed into the AutoFE procedure.
**Q5:**
> Minor remarks…
**A:**
Thanks a million for these detailed remarks. We have revised these problems in the new version.
Strengths:
1. One of the key strengths of this paper is its ability to provide transferability between datasets which previous methods have not been able to provide.
2. The method is highly flexible as it has been applied on different datasets as well different ML models.
3.
Overall there is sound discussion relevant work, gap analysis to find opportunity of development, strong mathematical rigour and experimental setup, evaluation and analysis.
Weaknesses:
1. The authors declare that to the best of their knowledge there aren't any autoFE/autoML workarounds for managing tabular data to accomplish transferability.
However this could be supported by a stronger statement that can more concretely say if such methods exist or not by clearly stating a comprehensive survey of analysis did not yield any methods.
2. This method is specifically geared towards tabular datasets.
While tabular datasets prevail in several key applications, it would be interesting to know how this method applies/does not apply to any other form of datasets (eg. images, speech, unstructured datasets).
At least there could be some discussion on consideration, if not a full scale evaluation.
Novelty:
1. Paper proposes a new form of markov decision process to generate features. Previous methods take number of features and only update using sequences of previous actions, iteratively.
Whereas Fetch provides Feature engineering actions and constructions actions, instead of features, iteratively based on the newly generated datasets (features).
2. According to this paper, Previous methods for AutoFE have never explored transferability between datasets, which has been done by this method.
Clarity & Quality: The paper has high clarity with explicit background provided and methods explained with sufficient depth both in terms of mathematical equations, datasets and analysis of results. There is further in-depth explanation in the appendix section around usage of material and datasets which is crystal clear.
Reproducibility: The method has been extensively tested on publicly available datasets which are linked. Algorithm and methods have been provided as well. The paper states that all experiments have been run with open source code provided.
NO.
None.
Thanks very much for your insightful comments and suggestions! We have updated our paper based on the comments of the reviewers. Summary of revision: (i) we modified our statement about the existence of some transferability work; (ii) we have added a discussion of applying FETCH to other forms of datasets in Appendix D.1.
**Q1:**
> The authors declare that to the best of their knowledge there aren't any autoFE/autoML workarounds for managing tabular data to accomplish transferability.
However this could be supported by a stronger statement that can more concretely say if such methods exist or not by clearly stating a comprehensive survey of analysis did not yield any methods.
**A:**
Thanks for the suggestion! Based on our extensive searching on the literature, while the transferability for modeling tabular data was briefly discussed by few prior work, we are certain that such transferability in the regime of feature engineering, AutoFE and AutoML is very much untapped. Parallel evidence can also be seen by [1].
Regardless, we changed our wording as per your suggestion. Thanks again!
**Q2:**
> While tabular datasets prevail in several key applications, it would be interesting to know how this method applies/does not apply to any other form of datasets (eg. images, speech, unstructured datasets).
At least there could be some discussion on consideration, if not a full scale evaluation.
**A:**
Our answer is yes!
For example, we may generalize our framework to other domains by regarding the feature outputs of an encoder (e.g. CNN for images, RNN for speech and GNN for graphs) as tables.
In contrast to the tabular data though, the vision and text data forms generally possess richer spatial or sequential structures. How to properly represent these structures and encode them into FETCH would be an interesting line of future work. FETCH, on the other hand, is devised to tackle the very characteristic of the tabular data including the permutation invariance or the variant length feature (see Section 3.3). In that regard, we have also added some discussion to Appendix D.1 of our revised manuscript.
[1] Roman Levin, Valeriia Cherepanova, Avi Schwarzschild, Arpit Bansal, C Bayan Bruss, Tom Goldstein, Andrew Gordon Wilson, and Micah Goldblum. Transfer learning with deep tabular models. arXiv preprint arXiv:2206.15306, 2022.
**About the Update:**
To better help people reproduce the experiment, we have reorganized and updated the code in our supplementary material. We have also added numerous comments to the code for a clearer understanding. The dependencies required to run the code are listed in ``requirement.txt``.
The updated paper also is modified as reviewers suggested.

Meta-review:
role: PC
data: metareview:_summary,_strengths_and_weaknesses: In this paper, the authors proposed a new automated feature engineering framework. The proposed framework can be transferred across different datasets, which is interesting. Empirical results are promising. One potential limitation is that the proposed framework may not be applied to different application domains beyond tabular datasets. Though the authors added some discussions about how to extend the framework to non-tabular data, they did not provide empirical evidence that the proposed framework can perform well on non-tabular data.

In summary, this is a good paper for ICLR-23.
justification_for_why_not_higher_score: No theoretical analysis is provided regarding the transferability of the proposed automated feature engineering framework.
justification_for_why_not_lower_score: The idea is interesting. The empirical results are promising, especially for the transfer learning setting.

Decision: Accept: notable-top-25%