Learning a Data-Driven Policy Network for Pre-Training Automated Feature Engineering
Authors: Liyao Li
Haobo Wang
Liangyu Zha
Qingyi Huang
Sai Wu
Gang Chen
Junbo Zhao
Venue: ICLR 2023 notable top 25% (2023)
Keywords: Automated Feature Engineering
Reinforcement Learning
Tabular Data
Data-Driven
Pre-Training

Abstract: Feature engineering is widely acknowledged to be pivotal in tabular data analysis and prediction. Automated feature engineering (AutoFE) emerged to automate this process managed by experienced data scientists and engineers conventionally. In this area, most — if not all — prior work adopted an identical framework from the neural architecture search (NAS) method. While feasible, we posit that the NAS framework very much contradicts the way how human experts cope with the data since the inherent Markov decision process (MDP) setup differs. We point out that its data-unobserved setup consequentially results in an incapability to generalize across different datasets as well as also high computational cost. This paper proposes a novel AutoFE framework Feature Set Data-Driven Search (FETCH), a pipeline mainly for feature generation and selection. Notably, FETCH is built on a brand-new data-driven MDP setup using the tabular dataset as the state fed into the policy network. Further, we posit that the crucial merit of FETCH is its transferability where the yielded policy network trained on a variety of datasets is indeed capable to enact feature engineering on unseen data, without requiring additional exploration. To the best of our knowledge, this is a pioneer attempt to build a tabular data pre-training paradigm via AutoFE. Extensive experiments show that FETCH systematically surpasses the current state-of-the-art AutoFE methods and validates the transferability of AutoFE pre-training.

Published as a conference paper at ICLR 2023
LEARNING A DATA-DRIVEN POLICY NETWORK FOR PRE- TREANINN AUTOMATED FEATURE ENGINEERING
Liyao Li
Haobo Wang!
I Liangyu Zha
Qingyi Huang*
Sai Wu
Gang Chen
Junbo Zhao 1*
'College of Computer Science and Technology. Zhejjang University
Institute of Computing Innovation, Zhejiang University
ABSTRACT
Feature engineering widely acknowledged to be pivotal in tabular data analysis and prediction. Automated feature engineering (AutoFE) emerged automate this process managed by experienced data scientists and engineers conventionally. In this area, most if not all prior work adopted identical framework from the neural architecture search (NAS) method. While feasible, we posit that the NAS framework very much contradicts the way how human experts cope with the data since the inherent Markov decision process (MDP) setup differs. We point out that its data-unobserved setup consequentially results in incapability to gener- alize across different datasets as well as also high computational cost. This paper proposes a novel AutoFE framework Feature Set Data-Driven Search (FETCI D. pipeline mainly for feature generation and selection. Notably, FETCH is built on brand-new data-driven MDP setup using the tabular dataset as the state fed into the policy network. Further, we posit that the crucial merit of FETCH is its transfer- ability where the yielded policy network trained on a variety of datasets is indeed capable to enact feature engineering on unseen data, without requiring additional exploration. This is pioneer attempt to build a tabular data pre-training paradigm via AutoFE. Extensive experiments show that FETCH systematically surpasses the current state-of-ther thetterrrt AutoFE methods and validates the transferability of Aut- oFE pre-training.
INTRODUCTION
Tabular data - also known as structured data - abound in the extensive application of database management systems. Modeling tabular data with machine learning (ML) models has greatly influ- enced numerous domains, such as advertising (Evans) 2009). business intelligence (Quamar et al, 2020 Zhang et al. 2020), risk management (Babaev et al.) 2019), drug analysis (Vamathevan al., 2019). etc. In resemblance to the other data forms like images or text, building a proper repre- sentation for the tabular data crucial for guaranteeing a decent system-wide performance. In this regime, this process is also known as feature engineering (FE), which was conventionally conducted by highly experienced human experts. In other words, as many empirical studies show (Heaton 2016). FE almost always serves as a necessary prerequisite step in ML modeling pipelines.
The recent advances in reinforcement learning (RL) have provided a new possibility for auto- mated feature engineering (AutoFE) and automated machine learning (AutoML). Neural architec- ture search (NAS) (Zoph Te 2016) has nearly become a synonym for AutoMI. the field of computer vision, based on an Rl. setup dedicated to searching for undesigned neural network ar- chitectures with excellent performance. As for tabular data, a series of well-known open-source packages (such as TPOT (Olson & Moore, 2016), AutoSklearn (Feurer al., 2015) and Auto- Gluon (Erickson et al. 2020)) claim to implement the AutoML pipeline. However, they do not generally cover AutoFE, especially feature construction and selection, which is supposed to be part of AutoML as shown in Figure To date, AutoFE has been a significant and non-negligible compo-
"Correspondence to j. ww..hhozzju. edu.
Source code available httptttpp//. htt//gtttuuu thu.. com www.///oooo/o/////oo/////oooooot /FRTCH implemented by Mindspore.

---

Published as a conference paper at ICLR 2023
A AutoFE
Model Seurch
Input Table
Feature Generation & Sclection
RowiD Age Gender Weight Height
1
25
M
1.82
1. t,
2. abs, square, inverse, log, sqrt, power3 groupby, min, max, mean
2
37
8
1.57
4. selection
Hyperparameter
10000
19
M
1.75
Optimization
Output Table
Data Cleaning
Age Gender
Weight Height
Weight/ 'Hight' Height/Agg
Evaluation
process mall values
0
75
1.82
22.64
0.0728
Accuracy: 97%
2. discrete string to numbers preprocess
1
62
1.57
25.15
0.0424
4. normalizaaiin
10000
19
0
53
1.75
17.31
0.9111
AutoML
Figure I: The different pipelines of automated feature engineering (AutoFE) and automated machine learning (AutoML). Existing AutoML frameworks focus more on model setup and lack concern for feature engineering.
nent of AutoML on tabular datasets because it not only constructs better features facilitate model fitting but also enjoys high interpretability.
Recently, the mainstream line of AutoFE based on a trial-and-error NAS-like process. For in- stance, Neural feature search (NFS) (Chen et al. 2019) was introduced to find the best performing FE plans by drawing inspiration from the NAS framework - for the first time embedding past FE actions into its policy network the states in its Markov decision process (MDP) for RL training to iteratively select better FE actions. The follow-up work DIFER (Zhu et al. 2022) extends its NAS- like setup to a differentiable the data - such as analyzing its distribution, identifying the outliers, measuring the correlation be- tween columns, etc. and then proposes FE plan. They may further the derived plan to test the prediction performance and repeat this process considering the evaluated score. Meanwhile, they also can accumulate knowledge accelerate recision-makinn
However, data scientist engineer usually tends to investigate
As we scrutinize these works, we posit that existing NAS-like AutoFE frameworks on tabular data have two shortcomings, largely deviating from how human experts cope with the data. First, they have stuck themselves with the data-unobserved paradigm because their policy network does not even see the tabular data itself and proposes data-unrelated FE plans. Second, the inherent data- unobserved setup makes them lack transferability. unfeasible to borrow knowledge from previous training experience to speed up the exploration process when facing completely new dataset.
This paper hopes to bridge this methodol- ogy gap between the human experts and data-unobserved methods for A and validate its feasibility based on the above discussions. In particular, new form of MDP setup where the state processed dataset
input:
AdaEB Squaret oglameto
num features N
Data: Unobserved
A Apprach
establish
MultapyCC, SqrtTenesstiii
Iterate: Past A Actions
is defined simply drawn from its original counterpart. The policy network yielded succinct map- ping from the input data table directly to its (sub)optimal feature engineering actions plan. To this end, we present FEature SET DaTa-Drives SearCH (FETCH) brand new RL-based framework for Aut- oFF but with a completely distinct data- driven MDP setup to emulate the human experts. As shown in Figure 2 FETCH outputs FE actions well-designed for the input data, and iteratively constructs more appropriate actions based on the newly generated data. In contrast, traditional unta--nnoseerved methods only take in the number of features to be processed and iteratively update with the sequence of past actions.
input: Table
Data: Driven FETCH(OUrS)
Loger
AddAA SuntD))
terate: Generated Table
Figure 2: The difference between data-driven FETCH and data-unobserved approach. See text for details.
Thanks the aforementioned design principles of FETCH, another favored by-product is that it enables rransferability by pre- pre-training for the AutoFE workflow. Simply put, we validate that FETCH can be pre-trained in collaborative manner where we feed multiple tabular datasets and maximize
2

---

Published as a conference paper at ICLR 2023
the expected sum of the reward altogether. In particular, we could directly exploit it to unseen datasets, with little or even no further exploration required. The "transferability" underlines that the policy network in FETCH manages accumulate knowledge from the fed datasets to be able to transfer them to other scenarios. We argue that this behavior of FETCH indeed similar to the human experts" in that the richer experiences accumulated from handling various types of datasets may offer them better insights when facing the unseen ones. It is also worth noting that the NAS-like AutoFE frameworks would demand exploration from scratch given any new dataset, profoundly due to their inherent inability to derive associations across datasets with their data-unobserved MDP. Last but not the least, this can also understood from standpoint of pre-training has been recently prompted in other domains such as BERT (Devlin et al. 2018 / GPT-3 (Brown) et al. 2020) for natural language processing or the SimCLR (Chen al. 2020) for computer vision. To sum up. the main contributions are listed as follows.
connected to what
1. We identify crucial methodology gap between the data scientisd/engineer with the current ex- isting AutoFE frameworks.
2. We propose a novel AutoFE framework for both classification and regression tasks, dubbed FETCH, on bridging this gap, including a data-driven feature engineering pipeline for data clean- ing, feature generation and selection. Empirical results show its on-par superior performances to the previous state-of-the-art method.
3. For the first time, we characterize transferability principle for AutoFE. It reflects how much knowledge or experience a trained policy may be able to accumulate to enable the exploration of unseen datasets. FETCH concept-proves its feasibility, This is also linked to the pre-training paradigm.
RELATED WORK
Automated feature engineering AutoFE aims to find the best-performing set of features on fea- ture engineering plans (e.g. transformation or selection) with minimal human intervention. For instance, the LFE algorithm (Nargesian et al.) 2017) pioneers parametrizing the feature transfor- mation that is only applicable to classification problems. To generate higher-order features, Trans- Graph (Khurana et al. 2018) was introduced by formulating the feature engineering plan via data structure of the graph and Q-learning (Watkins Dayan 1992) algorithm. While conceptually promising. this approach is shown to suffer from severe feature dimension explosion problems where excessive feature columns are appended to the raw data. The Neural feature search (NFS) frame- work (Chen et al. 2019). primarily stealing the setup from the RL-based NAS architecture (Zoph) jet al. 2018), shed some light on how to tackle this problem. It employs a separate recurrent neural network (RNN) controller (Medsker & Jain| (001) generate new feature for every feature col- umn individually, yielding that N features require N computationally expensive RNN controllers. On a separate but related line, the differentiable NAS framework (Luo et al., 2018) is employed to perform AutoFE, named DIFER (Zhu et al. (022) by transferring a discrete optimization problem be continuous. However, NFS and DIFER only use the data for feature evaluation, not for feature generation. Also, they only evaluate the effect of each individual feature, rather than considering the comprehensive performance of the entire generated feature set. While this data-unobserved ap- proach to constructing features works some extent, we believe it is inconsistent with the pattern of human experts scrutinizing the data before proposing feature engineering plans.
OVERVIEW OF FETCH
3.1 THE FE CONTROL PROBLEM
Given a prediction-involvee problem with its dataset D (X, Y) containing: (i)-a set of features, X {I ... ιt} where X [nadd denotes the tabular data containing rows (instances) and d columns (features); (ii)-a corresponding target label vector, Y, which can be either discrete continuous, compatible with classification regression problems respectively. Similar to most prior work around AutoFE (Khurana et al. 2018 Chen et al., 2019 Zhu et al.) (022)), pre-selected learning algorithm L (e.g. Random Forest Classifier Xghoost) with its hyperparameters fixed
3

---

Published as a conference paper at ICLR 2023
and measure of cross valssvaaiiottion performance E (e.g. F1-score) are considered in the experimental verification.
In addition, we denote T {t1. f2...t...to ... tm as feature engincering plan where it consists of an ordered sequence of m feature transformation actions initiated from the raw data. Each transfor- mation action in T is applied on an internally specified feature instead of whole features X. The whole set of derived features that may be deduced from X through T is referred to Xy. Note that, transformations can lead to a higher-order feature (i.e. combined by three or more features) because it is equivalent to applying binary operation recursively.
The purpose (or objective function) of feature engineering is defined as Equation I Given a dataset D (X, Y) with a set of original features X and the target Y. search a sequence of transformation actions to derive a transformed feature set X7, which maximizes the cross-validation performance ( Y)) for given algorithm L and a metric E.
T = arg arg max B[L(X,, Y)) T
(1)
3.2 RL SETTING OF FETCH
The RL learning scheme, illustrated in Figure is employed in order to output an extended tabular dataset concatenated with generated and selected features. The core to FETCH is policy network of the RL agent which is often instantiated by a well-designed deep neural network model introduced in the following Section 3.3 The policy network takes the dataset itself, or bootstrapped coun- terpart to cope with the scaled dataset as state, and produces a feature engineering plan through an inference pass. In that regard, quite different from prior work of AutoFE,, FETCH is in theory- - capable to fully observe, perceive, and recognize the pattern concealed in the dataset. At its core, FETCH formulates feature engineering
control problem, elaborated in above Section 3.1 Diving into this control problem, we propose a complete MDP setup including an environment involving FE plans. The environment allows the agent to first employ an FE actions/operations sequence to generate new table, then receive back positive/negative reward vector that summarizes its results, derived from the reward function. The goal of the RL agent is to learn, by repeated interactions with the environment, a policy network capable of mapping input data a sequence of FE actions with maximum cumulative reward. Once the learning phase is finished, the output tabular data provided to the downstream ML model is generated upon the sequence of FE actions. To enable transfer- ability to certain degree in FETCH, we facilitate the training phase of this control problem via the reinforcement learning (RI.) paradigm. Due to space limitation, the MDP detail settings such as FE action space and reward function R. are covered in Appendix A.
observed dataset and reward function assessing the quality of the
As the informative architecture of FETCH shown in Figure B FETCH outputs a sequence of trans- formation actions plan T = (2,, ..m) after tabular dataset is conceived by our framework. To get the final prediction result, we sequentially apply the actions list on the tabular data X and then feed it into an ML predictor (e.g. LR. RF, Xgboost) for the feature set evaluation and reward calculation. This process is identically employed in both the training and evaluation phases. Within each rollout, time step i, run the current data X, through the policy network æ to get an action probability over pe-supported operators. We then sample the sequence of transformation actions T following multinomial distribution by softmax. For the higher-order feature generation search, while we only include binary operators in the set, we generally run FETCH by multiple steps so that it covers more complex higher-order features that may involve more than two feature vectors. We let T interact with the environment yielding transformed tabular data X,+1 which integrates the input X, We repeat this process multiple times until it reaches pre-defined limit value K or converges.
A table generated by FETCH on a healthcare example is depicted in Figure D As shown in the figure, FETCH initially cleans and encodes the data so that the label M/F in the column Gender becomes 0/l respectively. Then the processed data is fed into our policy network, which learns how to generate and select features, and eventually applies appropriate FE actions to the input table to obtain the output table. For improved readability, each generated feature is named by the action path that constructs it. For example, a more effective feature Weigh//Height (a.k.a Body Mass Index [Lee et al., 2013)) is calculated by column Weight and the square of column Height. This interpretable like illustration of the operations allows experts to easily understand feature meanings and strengthens interpretability.
4

---

Published as a conference paper at ICLR 2023
Policy Network
Reward
LR:
Evaluation
Actions Probability
Xahoost
C
LightCBM;
Table X
0.03
0.04
0.13
A B.C.D...
Sampled Actions J
New Table X
0.00
Q12
0.11
0.06
AddIA, B);
0.01
0.02
Delcte(C):
Squt
0.01
0.08
Alention
Ixput
Dense
Encoder
Decoder
Softmax Layer
Nore
0.13
0.14
Layer
Layers
Layers
Lapers
Replace
Repeat K steps for higher- higher-order order features
Figure 3: An architecture of the data-driven FETCH framework. For step i in each epoch, the newly generated feature set X will replace its previous counterpart X; to construct higher-order features and further selection. The subfigure of the middle dashed box reveals the micro-composition of modules inside the policy network.
3.3 POLICY NETWORK ON FEATURE SET
As discussed above, the RL agent in FETCH framework is primarily represented by policy network т, where the main challenge of architectural design centered on. Notably, to characterize the tabular dataset and properly learn a decent representation of it, must let the policy network be and variable-length natures of tabular data.
capable of handling the permulation-invaree
In contrast the data forms of computer vision or natural language, when swapping or permuting the columns of tabular data, it remains the same dataset. This property identical to a "set" data form as permutation invariant, which requires the neural network be able to perceive the lists of different sequential features as still the same data set. When conducting feature engineering, the column number of the tabular would often change. Some of the column features might be replaced or deleted, while some others (especially for iterative generation of higher-order features) might get appended. This property of variable length demands that the neural network be more flexible with respect to the shape of the input data. Note that, the nature of this data form has prevented us from employing the convolutional network family or the recurrent network family because these models encode positions part of their inductive bias.
To that end, we finalize our design by viewing the tabular data as a set of feature (column) vectors. Specifically, we draw inspiration from the Transformer architecture (Vaswani al. 2017) and we carefully choose our neural operators to equip into the policy network. A brief depiction of it is shown in the middle (Policy Network) of Figure
The primary components of our policy network are the following: (i)-a fully-connected network (FCN) containing an input layer and dense layers. It changes the dimension of variable-length feature vector to a fixed length; (ii)-a multi-head self-attention module containing attention encoder layers (more details in Vaswani al.) (2017)). It measures the complex relationship between fea- tures in the set, which can be regarded as encoding the relationship of the feature columns; (iii)-an action output module including action decoder layers and softmax layer. It decodes the output of the upstream encoder to map the correlation information of the features to the corresponding action probabilities. Unlike transformers, we abandon the positional encoding from the multi- self- attention module to satisfy the permulution permutution-imvaaianne property due to the nature of this data form. All these parametric modules allow arbitrary permutation of the feature vectors and can tackle the variable size of the feature set. Further, reason that the stacked modules enable representation learning, coupled with the self-attention mechanisms effectively gauging the correlation and rela- tionship between the feature columns in the set. The multi-layer setup also assists the higher-order feature discovery.
The feature-set-basee action sampling rules applied the probability of the actions yielded by the policy network to propose an FE action plan (Appendix [A.)). Proximal Policy Optimization (PPO) (Schulman et al. 2017) is adopted as the RL training manner in Algorithm (A (Appendix A.3)
5

---

Published as a conference paper at ICLR 2023
3.4 TRANSFERABILIT VIA TRE-TRAINING
With data-driven FETCH, we demonstrate that our approach very much resembles the human experts who can quickly transfer experience in handling unseen datasets, and it exhibits a certain degree of transferability. This setup also links to the recent popular topic of pre- pre-training, such BERT (De- vlin et al. 2018) or SimCLR (Chen et al.] 2020) in natural language processing and computer vision respectively. The goal of the pre-training paradigm is to drive the model to learn "common-sense" through large-scale training over a variety of data, with either supervised or unsupervised loss func- tionals. Common sense is manifested in the representation space, quite aligning the goal of feature engineering for tabular data. Despite the promise, the research of pre- tre-training on the tabular data very much unaddressed. With FETCH, we hope provide the first viable technical solution and empirically validate its feasibility. The corresponding experiment of transferability can be found in Section
Formally, given set of datasets D DD..D2.. DN,) where Na denotes the number of datasets occurring in the set. We notice that FETCH does not require these tables to possess identical or similar schemas, nor identical row/column sizes. In particular, closely following the procedures dis- played in Algorithm [ we blend these datasets into a key-value data pool struct and initialize the RL rollouts. Correspondingly, the reward eventually yielded by Σ (DD)),, according to Equa- tion and line 13 in Algorithm[] [ Post to training on this blended set of data tables, we serialize and save the parameter of the attention encoder module in the policy network. Noted, despite that in the normal training scenarios for pre-train models in the domains like vision or language understanding, the input data must conform to pre-defined shape and form. FETCH gets rid of this limitation thanks to its architectural design explained in Section B.3 to guarantee the capability for the various shapes of data. We believe this flexibility is quite essential for the real-world application of FETCH, because the tabular data is deemed business-related so it lacks standardized preprocessing pipelines like image reshaping text truncating.
In what follows, a set of unseen data tables would be prepared for testing, D' = DD .......'') D'v;)
where N₂ denotes the number of testing sets presented. In the corresponding experiment (Sec- tion 4.3). D. we simply run the yielded policy network through these datasets and assess the overall performance as per chosen evaluation metrics. There have not been any AutoFE or AutoML. workarounds for tabular data managing to accomplish this setup of transferability.
We postulate that FETCH achieves this mainly by (i)-having the right MDP setup where the states are grounded by drawing from the original datasets, paving the way for mapping data to corre- sponding FE actions; (ii)-having the proper architectural design of the policy network where it treats the tabular data as a set and satisfies the appropriate properties (i.c. permutation-inwe -nvarraannn- and variant-length), One may ask, why could this work? In hindsight, with this scheme of pre-training on various datasets, FETCH is encouraged to (i)-reinforce the better feature engincering actions through vast exploration and (ii)-build proper mapping relationship between the input data to the action space, with regard to different form, distribution of the data table (or its columns), etc. The NAS-like methods only fulfill (i) but fail at (ii), which disallows any pre-training and transferability across unseen tables. The (ii) in FETCH very much accords to the human professionals because the experienced ones absorb experiences and are often capable to provide a preliminary feature engi- neering plan simply by scrutinizing the data- that may may not require further trial-and-erre processes. The corresponding experiment concept-proofing the transferability and generalizability of FETCH are provided in Section 4.3
4 EXPERIMENTS
4.1 EXPERIMENTAL SETTING
Goals of experiments FETCH is proposed to simulate two major characteristics of human experts in operating feature engineering processes: data-driven observation and the transferability of explo- ration experience. The first goal is to assess the superiority of our innovative data-driven MDP setup compared to other unobssrooee AutoFE methods in Section 4.2 The second goal is to verify the feasibility of the transferability brought by our setup in Section 4.3, i.c., whether it can converge faster or search for feature engineering action sequences with better performance by pre-training.

---

Published as a conference paper at ICLR 2023
We adopt the same testing protocol (using the same evaluation model Random Forest) identical benchmarks as most notable prior work, such as NFS and DIFER. Section 4.4 verifies that FETCH can boost many ML models. The ablation study in Appendix A the efficiency of FETCH and the influence of generated higher-order features. The fairness settings of experiments and the hyperparameters settings of various comparison methods in Appendix B.2
Performance indicators For the classification tasks, the micro Fl-score (the average harmonic mean of precision and recall) is used as the main metric and (1 (relative absolute error)) (Shcherbakov al., 2013) for the regression tasks. Both evaluation metrics that the higher the score, the better the performance. Meanwhile, we also use the percentage of score improvement in the baseline score as metric of improved efficacy. They measured using 5-fold validaanli...nn.
The experiments are conducted on 27 datasets including 11 regression (R) datasets and 16 classifications (C) datasets. These datasets are from OpenMI UCI repository 3 and Kaggle All datasets available via the URL links in Appendix [B.3] datasets are further utilized to test the transferability by pre-training in Section 4.3] As preprocessing step, we clean and encode the raw data using FETCH, ensuring that the data readable by various MI. models. And the final data input to each method is the same.
Datasets
4.2 EFFECTIVENESS OF FETCH
Comparison with AutoFE Methods We select six current widely-used or state-of-the-art AutoFE methods to show the effectiveness of FETCH feature engineering. Base represents the baseline method evaluated on the raw dataset without any FE transformation. Random method randomly generates FE action sequences by Monte-Carle exploration. Here we take the highest score among the same number of explorations as FETCH. DFS (Kanter & Vecramachanenj 2015 and Aut- oFeat (Horn al. 2019) famous and open-source Python libraries for search-based feature engineering. NFS (Chen et al. 2019) is the current best RL-based NAS-like AutoFE method, which employs the RNN module controller of each original feature, and generates new features includ- ing higher-order ones by series of actions for the original features. DIFER (Zhu et al. 2022) is the state-of-the afe AutoFE method, which optimizes the features by a differential NAS-like framework
As shown in Table I FETCH generally outperforms all the existing work by significant margin. On benchmarking datasets, FETCH achieves state-of-the-at performance on 25 out of 27 datasets overall and gets close second place in the remaining datasets. Although DIFER and NFS greatly outperform the baseline method, FETCH still performs 3.24% higher than DIFER and 3. 3.16% higher than NFS on average, indicating the superiority of our data-driven setup in the AutoFE pipeline.
To further highlight the superiority of FETCH, we addition- ally involve two famous AutoML methods. AutoSklearn (Feurer et al., 2015) popular open- source AutoML toolkit focusing on algorithm selection and hyperparameter tuning. AutoGluon (Er-) ickson et al. 2020) is full-fledged open open-source AutoML framework developed by Amazon Inc., which covers many types of data. In particular, here uses the AutoGluon- Tabuula of it.
Comparison with AutoML Methods
Compared with the AutoMI. frameworks which work on model search and hyperparameter opti- mization, FETCH still has a great advantage, with 18 out of 27 datasets in total performing better than them. This serves as strong evidence that FETCH outperforms the other existing methods due to its buman-expert-ikke data-driven setup with outstanding effectiveness. As member of the Au- toML family, these two baselines mostly focus on the model part and barely involve the feature engineering part. We believe these comparisons would further highlight the effectiveness of FETCH even beyond the scope of feature engineering.
https//www.openmml.orr
https//archive..esuui...oo hwt..e//////idde..ppp
https://ww.kagg... http://wwwwgll.comm..mmm com/
DFS implemented in FeatureTools Toolkit

---

Published as a conference paper at ICLR 2023
Table I: Effectiveness comparison of FETCH with other AutoFE and AutoML methods. Bold indi- cates superior results amongst AutoFE methods. Note that AutoML methods focus on model search instead of feature engineering.
C/R
Instances' Featuuess
Base
Random
DES
AutoFi: Methods
AutoFeax
NFS
DIFER
FETCH
AutoSklear
AutoMi. Methook
AusoGihon
Airfoil
R
1503
0.5068
0.6211
036003
0.5955
0.6226
0.6255
0.6463
0.5151
0.5083
BikeShhare DC
R
10886; 11
0.9880
0.9989
0.9950
0.9891
0.9991
0.995
0.9997
0.9911
0.9967
Houso King Cuunty
21613;19
0.6838
0.6908
0.6917
0.634
0.6988
0.7005
0.7442
Housing Bosten
506).3
04641
0.478%
0.7088
0,703
0.497
0.5072
0.4403
04857
Openml 586
R
1000):25
0.6564
0.6646
0.7188
0.7178
0.7223
0.6966
0.7671
0.7297
0.7904
Openmi 589
R
1000)
0.6395
0.6285
0.0956
0.7278
0.7165
0.6789
0.7562
0.7183
0.7998
Openmi .617
1000) 55)
0.6499
0.6485
06664
0.7404
0.7265
Openml ,116
500,50
0.5807
0,5927
0,585
0.582
0.6749
0.6618
Openml 618
R
1000;50
0.6351
0.6277
0.6848
0.6374
0.6461
0,6553
0.7351
0.7198
0.7520
Openni 620
R
1000) 25
0.6309
0,6288
0.6528
0.6774
016941
0.7362
0.750%
0.7199
0. 0.7855
Openmi 637
500
0.5160
0.5478
0.5108
0.566
0.5739
0.6066
0.6453
0.6416
0.6742
Adall Income
48842 14
03478
0.4485
08502
03448
03997
0.884
D38537
8679
0.8388
Amazon Fmpploye
327669 9
09450
0.9422
01451
03651
013461
0.474
0.9479
0. 0.9471
0.9473
Credit Default
30000 25
0.8044
0.8089
030066
0 03008
0.8101
03808
0.8114
0.8194
0.8214
Cieditaa
0X216
0,8005
0.8688
0.8754
Ferillity
1001.9
03700
0.847
02900
62189
0,8900
0,8900
8400
0.800
German Credit
1001)24
0.7390
0.7738
0.7490
0.7000
0.7786
0.7750
0.7910
0.7460
0.7590
Hopulitis
155 12
0,8258
0.5639
0X5116
0.8677
018766
03839
0.9290
01 8065
0.7871
lonosphere
34
0.9514
0.9773
0.2266
0.9543
0.9515
0.9716
0.8194
0.8214
Lymphogruphy
690/6
08113
0.5455
0.8614
0.8827
0.9260
0.8418
Megawattl
4900 12
0.8655
0,5706
0.8813
0.8893
0.9187
0,9089
0.9209
0.8853
0.8850
Messidar Features
11500, 19
0.6594
0.7026
0.7089
0.7389
0.7717
0.7541
0.7689
0.7402
0.7255
Pimalndian
768
0.7566
0.7609
0.7540
0.7643
0,7784
0.7839
0.7969
0.7462
0.7631
SpumBase
46011,57
0.9211
0.9198
0.9237
0.9341
0.9372
0.9405
SpectF
267)
0.7751
0.8211
08125
08831
0.8608
0,8388
0.8833
0.7828
0.7010
Wine Quality Red
9991
0.5597
0.5774
0.5422
0.5641
0.5814
0.5779
0.6042
0.5804
0.5729
Wine Quality White
4900)
0.4976
0.5046
0.4855
0.5023
0.5111
0.5153
0.5235
0.5376
0.5259
4.3 TRANSFERABILITY OF FRTCH
As discussed in Section 3.4 FETCH is the first work that enables transferring "experience" across different datasets and realizes structured data pre-training. In this subsection, the transferability of FETCH is proven by the following pre-training experiments. The data-driven attention encoder module of the policy network in FETCH has the capability to encode the context of permutation- invariant and variable-lengee feature set of tabular data, which yields the probability of transferring prior knowledge across different datasets.
To verify the fcasibility of transferability, we attempt to train FETCH on range of datasets and save parameters of the attention encoder module as a pre-trained model. And apply this model instead of the initial random weights to the search of the new dataset, and see if it takes less time to achieve the same performance or better performance with the same number of exploration epochs. We define kinds of pre-trained methods of FETCH.. Each of them keeps the same initial parameters, except for the pre-trained datasets.
No-Pre means the model without any pre-training directly searching the FE plan from the scratch. Pre-Oml and Pre-Uci are the models pre-trained on 5 datasets from OpenML and UCI repository respectively (see Appendix B.4 for details). Pre-Mix is the model pre-trained on the whole 10 datasets of Pre-Oml and Pre-Uci.
We use these 4 pre-trained models to search FE plans on 5 previously unobserved datasets respec- tively, ie. Housing Boston, Openml_616. Openml_637, Ionosphere, and Wine Quality Red. With the features transformation plan T of each dataset, the evaluation score is the performance of a learning algorithm trained on the transformed feature set XT by 5-fold cross-validation.
Table 2 demonstrates the feasibility of transferability, where we compare the score of final top-1 FE plans ex- plored by model searched from scratch (No-Pre) and the best model (Pre-Best) from the above 3 pre-trained models.
Table 2: Transferability comparison of the best pre-trained model (Pre-Bec Best) and non-pre-trainee trained one (No-Pre). See text for details.
Dataset
No-Pre
Pre-Best
Max-Diff (%)
FpochMor-Dae
Housing Boston
0,5224
0.5357
2,31
123
Openml. 616
0.6749
0.6942
4.67
27
Openml 637
0.6453
0.6631
5,51
26
lonosphere
0.9716
0.9864
1.12
148
And the maximum difference (Max-Diff) of improvement percentage between the pre- trained models and the original model during the exploration and the number of epochs when they appear (EpochMaz-pa Maz-Dii! are also listed in Table 2 It can be seen that the best FE plan found by
Wine Quality Red
0.6042
0.6207
2.59
8

---

Published as a conference paper at ICLR 2023
No-Pre
Pre-Dml
Pre-Uei
Pre-Mix
Percenaage of impprovemen %
Housing Boston
Opennal
Openml
lonosphere
Quality
25-
20-
15-
15-
10-
4-
0-
100
200
100
100
200
500
100
200
500
0
100
200
300
100
200
300
Epoch
Epoch
Epoch
Epoch
Epoch
Figure Transferability comparison of improvement (%) datasets under the model from scratch (No-Pre) and kinds of pre-training models (Pre-Oml, Pre-Uci, and large-scale Pre-Mix). Better view in color.
the pre-trained approaches scored higher than the original model. Roughly the difference in the scores of the searched FE plans between the pre-trained model and the original one reaches its peak around 150 epochs (half of the total epochs).
To further investigate the performance curves of percentage of improvements in evaluation scores during searching under different types of pre-trained FETCH. In that, we find that the Pre-Mix model has higher scores than others, especially No-Pre (no pre-training). from the beginning trained models (i.c. Pre-Oml and Pre-Uci) share similar trend, although their performance is not stable. The gap between Pre-Mix and No-Pre is larger in the early stage of the search, and No- Pre gradually chases up the number of epochs increases. However, eventually the pre-training approaches still score higher than the original model.
the pre-trained model, Figure 4 visualize the smoothed
the end of the search in most cases. Other pre-
These results suggest that the pre- pre-tra training approach can be effective in improving scores and finding more appropriate feature engineering actions faster. Pre-training on data-driven FETCH can accumu- late and transfer prior knowledge to unobserved datasets and improve FE efficacy more effectively. Despite experimental serendipity, the above experimental data still indicate to some extent the fea- sibility of simulating the knowledge transferability of human experts through pre-training AutoFE.
4.4 FLEXIBILITY TOWARD MODEL CHOICES
In this section, we measure the
flexibility of using different ML
(Wihout
FETCH Fb)
models. We chose Logistic Re-
Eoalection 0000 200
gression (Wright 1995), Ran- dom Forest (Liaw al. (2022
and Xgboost (Chen & Guestrin 2016) model for boosting com-
parisons several datasets. We compare these models on the situation of no feature engineer- ing (Base) and doing feature engineering with FETCH. As shown in Figure 9 our framework has the effect of promoting the fitting effect for these models. This also reveals the excellent flexibility of FETCH to different kinds of machine learning models.
Logistii Regreoiin
(b) Randon
Xibocoo
Figure 5: Flexibility comparison of evaluation scores (%) on dif- ferent datasets and models.
CONCLUSION AND OUTLOOK
In this work, we propose FETCH, novel RL-based end-to-end framework developed for AutoFE. The pivotal design of FETCH is to treat the tabular data as a "set" and use the datasets themselves as the state formulation in the MDP setup. The resultant framework achieves (surpassing or on-par) state-of-the-art performances on the standardized benchmarks adopted by prior work. Thanks to the methodological and architectural design of FETCH, we further concept-prove the feasibility of pre-training and transferability schemes for the tabular data. In summary, work is to FETCH insights from data for better feature engincering and data mining. the future, we hope to pre-train extremely large-scale models across various tabular data.

---

Published as a conference paper at ICLR 2023
ACKNOWLEDGMENT
This work majorly supported by the National Key Research and Development Program of China (No. 2022YFB3304001, and in part by the NSFC Grants (No. 62206247) as well as the Key Research and Development Program Zhejiang Province of China (No. 2021C01009). LL, HW,
and JZ also thank the sponsorship by CAAl-Huawei Open Fund and the Fundamental Research Funds for the Central Universities.
REFERENCES
Dmitri Babaev, Maxim Savchenko, Alexander Tuzhilin, and Dmitrii Umerenkov. Et-rnn: Applying deep learning to credit loan applications. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp. 2183-2190, 2019.
Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models few-shot learners. Advances in neural information processing systems, 33:1877-1901, 2020.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning visual representations. International conference on machine learning, pp. 1597-1607. PMLR, 2020.
Xiangning Chen, Qingwei Lin, Chuan Luo, Xudong Li, Hongyu Zhang, Yong Xu, Yingnong Dang, Kaixin Sui, Xu Zhang, Bo Qiao, et al. Neural feature search: A neural architecture for automated feature engincering. 2019 IEEE International Conference on Data Mining (ICDM), pp. 71-80. IEEE, 2019.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv: /11.000/100004005, 2018.
Nick Erickson, Jonas Mueller, Alexander Shirkov, Hang Zhang, Pedro Larroy, Mu Li, and Alexan- der Smola. Autogluon-tabular: Robust and accurate automl for structured data. arXiv preprint arXiv:2003.06505. 2020.
David S Evans. The online advertising industry: Economies, evolution, and privacy. Journal of economic perspectives, 23(3):37-60, 2009.
Matthias Feurer, Aaron Klein, Katharina Eggensperger, Jost Springenberg, Manuel Blum, and Frank Hutter. Efficient and robust automated machine learning. Advances in neural information pro- cessing systems, 28, 2015.
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT Press, 2016.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp- 770-778, 2016.
Jeff Heaton. An empirical analysis of feature engineering for predictive modeling. In SoutheastCon 2016, pp. 1-6. IEEE, 2016.
Franziska Horn, Robert Pack, and Michael Rieger. The autofeat python library for automated feature engineering and selection. arXiv preprint arXiv:1901.07329 2019.
James Max Kanter and Kalyan Veeramachaneni. Deep feature synthesis: Towards automating data science endeavors. In 2015 JEEE international conference on data science and advanced analyt- ics (DSAA). pp. 1-10. IEEF, 2015.
10

---

Published as a conference paper at ICLR 2023
Guolin Ke, Qi Meng. Thomas Finley, Taifeng Wang. Wei Chen, Weidong Ma, Qiwei Ye, and Tie- Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. Advances neural information processing systems, 30:3146-3154, 2017.
Udayan Khurana, Horst Samulowitz, and Deepak Turaga. Feature engineering for predictive mod- eling using reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 32, 2018.
Bum Ju Lee, Keun Ho Kim, Boncho Ku, Jun-Su Jang, and Jong Yeol Kim. Prediction of body mass index status from voice signals based on machine learning for automated medical applications. Artificial intelligence in medicine, 58(1):51-61, 2013.
Andy Liaw, Matthew Wiener, et al. Classification and regression by randomforest. R news, 2(3):
18-22, 2002.
Rengian Luo, Fei Tian, Tao Qin, Enhong Chen, and Tie- Yan Liu. Neural architecture optimization. Advances in neural information processing systems, 31, 2018.
Larry Medsker and LC Jain. Recurrent neural networks. Design and Applications, 5:64 67, 2001.
Fatemeh Nargesian, Horst Samulowitz, Udayan Khurana, Elias Khalil, and Deepak Turaga. Learning feature engineering for classification. In Ijcai, pp. 2529-25535 2017.
Randal S Olson and Jason Moore. Tpot: A tree-based pipeline optimization tool for automating machine learning. In Workshop automatic machine learning, pp. 66-74. PMLR, 2016.
Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. Catboost: unbiased boosting with categorical features. Advances neural information processing systems, 31, 2018.
Abdul Quamar, Fatma Özcan, Dorian Miller, Robert Moore, Rebecca Niehus, and Jeffrey Kreulen. Conversational bi: an ontology- conversation system for business intelligence applications. Proceedings of the VLDB Endowment, (3(12):3369-3388 2020.
John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.063477 2017.
Maxim Vladimirovich Shcherbakov, Adriaan Brebels, Nataliya Lvovna Shcherbakova, An- ton Pavlovich Tyukov, Timur Alexandrovich Janovsky, Valeriy iaaool'evich Kamaev, et al. A survey of forecast enror measures. World Applied Sciences Journal, 24(24): 2013.
Jessica Vamathevan, Dominic Clark, Paul Czodrowski, Ian Dunham, Edgardo Ferran, George Lee, Bin Li, Anant Madabhushi, Parantu Shah, Michaela Spitzer, et al. Applications of machine learn- ing in drug discovery and development. Nature reviews Drug discovery, 18(6):463-477 2019.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in neural information processing systems, pp. 5998-6008, 2017.
Christopher JCH Watkins and Peter Dayan. Q-learning. Machine learning, 8(3-4):279-299 1992.
Raymond E Wright. Logistic regression. 1995.
Tianping Zhang, Yuanqi Li, Yifei Jin, and Jian Li. Autoalpha: an efficient hierarchical evolutionary algorithm for mining alpha factors in quantitative investment. arXiv preprint arXiv:2002.08245,, 2020.
Guanghui Zhu, Zhuoer Xu, Chunfeng Yuan, and Yihua Huang. Difer: Differentiable automated feature engineering. In First Conference on Automated Machine Learning (Main Track), 2022.
Barret Zoph and Quoc V Le. Neural architecture search with reinforcement learning, arXiv preprint 1rXiv:1611.01578 2016.
Barret Zoph, Vijay Vasudevan, Jonathon Shlens, and Quoc V Le. Learning transferable architectures for scalable image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 8697-8710, 2018
11

---

Published as a conference paper at ICLR 2023
A DETAILS OF RL SETUP
A.1 MDP MODEL FOR FE
As discussed above, the feature engineering (FE) process conforms well to the exploration and exploitation paradigm, and is also a trial-and-error process. Therefore, the Markov decision process (MDP) can be employed to automate the simulation of the FE control problem. Also,the the goal of reinforcement learning (RL) is to find the optimal policy with the given MDP by an agent, where the policy is the mapping from states actions, that the final cumulative reward is maximized.
We unify the general problems of the FE process by casting it into an episodic MDP defined by a tuple A, E, R, X)), where agent A interacts with the environment which receives the agent's
transformation actions and returns the transformed tabular data from initial data Xg. Intuitively, in our case, the the variable X' = X, as the state of MDP which is identical to the raw or transformed tabular data. In a single training step i, the agent receives the current state (feature set) X;, next required to choose plan of feature transformation actions to synthesize the table data X;-1. Then the environment evaluates X and returns a reward R to update the agent and transmit it to new state.
actions the set of all possible (and supported) FE operations. Here, we define
Next, we define the underlying concepts of MDP in the FE context, namely action space, state, agent, environment, and reward.
FE Action Space Analogous to prior work (Zhu et al. 2022 Chen et al. 2019). our overall oper- ation set (the space of action 1) includes the following"
The unary operation: 6 lower-order value-conversion functions including abs, square, inverse, log, sqrt, power3.
The binary operation: 4 combinatorial arithmetic functions including addition, ractinn, mult multiplication and division.
Besides the operators on the numeric features, we also include operators for categorical ones:
Cross-combine operation means to cross-product two categorical feature vectors to generate combined one.
Binning operation means discretizing numerical features by binning to further combine with categorical features.
As framework that integrates feature generation and feature selection, FETCH also includes feature selection operators:
None means taking no action on the specific feature.
Terminate means stopping generating higher-order features based on the specific feature.
Delete means deleting the specific feature from the original dataset.
The resulting FETCH framework is quite universal in terms of the types of datasets well their feature column properties. FETCH supports operations including unary, binary, selection operators, etc. Beyond, FETCH can easily integrate user-defined feature engineering operators that achieve certain level of extensibility.
State One core innovation of FETCH is our data-driven MDP setup where it parametrizes the tabular data both prior- and post-transformation into the state formulation. The details on how the agent "sees" the tabular data itself state are in Section 3.3 We argue that this further bridges the AutoFE with human experts who determines the next action move by scrutinizing the data rather than past actions sequence like prior work.
For the sake of fairness, operators like "groupby" and "mean" are not added hut actually compatible with FETCH.
12

---

Published as a conference paper at ICLR 2023
The agent and policy network The agent is modeled by policy network X P(A) where P is the probability the actions sampled by the agent AA and X' is the state parametrized by tabular data.
The environment The environment encompasses the machine learning algorithm L with pre- configured hyperparameters and the evaluation metric E. The interaction between the actions and the environment intuitive; that is, FETCH performs feature engineering guided by the set of pre- dicted actions on the processed feature columns, and runs it through and E. This induces the definition of the reward function.
Reward Function It is crucial to get a precise reward estimation for training FETCH. In particular, our reward calculation process consists of an evaluation metric E related to pre-selected model L (hyperparameters fixed), for example, Logistic Regression classifier Random Forest predictor.
Similar to (Zhu et al., 2022 Chen et al. 2019 Khurana et al., 2018), we employ a standardized validatiiit strategy enhance the precision of the reward estimation. The prior work generally adopts most straightforward reward definition by using the average performance obtained on the k-fold split. However, we empirically observe that this causes great training instability, especially from the early stages. We conjecture that this is because the average operation has lost some non- neglectable information. For instance, a policy is optimal and needs to reinforced all k-fold exhibits superior performance. Nevertheless for the situations when the policy disagrees with the k-fold evaluation, we still need to punish the model according to the suboptimal results. Formally, the reward calculation can be written
R(X,) F(X.) Faan(X;))
where represents the average performance obtained from k-fold cross-validation and
Ean (X)
Σ min(0, (X,) E(X;-1))
where (X,) represents the k-th fold evaluation result, E(X;-1) the average performance from tabular data from the previous step. Notably, this reward formulation prompts the agent to punish inferior results appearing in any fold, rather than simply looking at the average performance (Zhu ctt al. (202)). We found this means of reward calculation quite beneficial for training stability, espe- cially in the early training epoch, and potentially leads better sample efficiency.
A.2 SAMPLING TRANSFORMATION ACTIONS
A final softmax is placed at the top stage after the policy network to obtain transformation actions probability, and further sample according to them to get actions plan T = (t1, 12 ...tm
T sssppe(()oftmx(h)))
where h denotes the hidden representation fed into the softmax layer. During the exploration, ob- tain a stochastic action set in correspondence to the original feature sets by sampling actions through multi-nomial distribution, provided by the above softmax operator. Then FETCH comprehensively evaluate the entire feature set generated by the actions set. This takes into account the overall per- formance of the entire feature set instead of the assumption in previous work that only evaluates single performance improvement for each feature.
A.3 TRAINING POLICY NETWORK WITH PPO
Proximal Policy Optimization (PPO) (Schulman et al., 2017) has been successfully applied to train RL agents in several fields. In FETCII, we employ the PPO algorithm to train the overall system. allows our framework to sample several sets of action plans in each epoch, assign them multiple threads (or workers) for parallel evaluation, and later compute the policy gradient to update the network. This way reduces sample complexity, which is also a kind of exploration cost, so that the policy network can converged with fewer training epochs. The overall training algorithm of FETCH is shown as Algorithm I and the following.
13

---

Published as a conference paper at ICLR 2023
Algorithm Training algorithm of FETCII
Input: Raw tabular data Xo. policy network n, agent A, metric E, pre-selected ML model label
Y
Parameter: Total exploration epochs N, maximum step (or order) generate features K, number of workers (or threads) W
Output: New dataset X generated by the transformation 7 satisfying Equation
1: while epoch E NN] do
2: while worker index w E [0, W] do
3: Initiate worker[w:
4:
while step è [0, K] do
5:
X' X;
6:
Sample a sequence of actions plan Tj through the
π: X + P(A)
7:
Obtain the generated tabular data X,+1 by letting the actions T; interact with X; Evaluate X,-1 by E(L(X;+1, Y))
8:
9:
i i=1+1
10:
end while
11:
w 1++1
end while
13:
Calculate the reward R based Equation 2
14:
Compute gradient based on the proximal policy optimization (Schulman et al. 2017)
algorithm
15:
Update the policy network
1+1
17:
end while
is defined as the ratio of sampling importance between new policy xe and old policy it can be computed as follows.
(ag (8t)
To To. (at St)
(5)
C' is defined as the advantage function A (st; at) with the clip method. > 1 +
1 1+
clip (To, 1 1 -.)+
Tg: 1 S 1+
(6)
1-E,
1-
C(sr, clip 11,,1-e,1+60
(sc,ar)
What needs a special explanation is that our paper, we take the moving average of the past reward as the baseline for each time step. We compute the difference between the current reward and the baseline as the value of the reward function. The loss function is defined as the following formula.
Jroliey (f) =0 Σ
min (ra.8 (8t, 0z) C(st, a())
(FE.aL)
To maintain exploratory properties, we add entropy ENtoss to the loss function. The objective function of PPO can be summarized as follows:
(Ppo(0) = Joticn (0) ENioss
B DETAILS OF EXPERIMENTS
B. ENVIRONMENTS
All experiments are carried out on a server with Ubuntu 20.04.1 LTS, Nvidia GeForce RTX 3090 (24GB GPU memory), Intel(R) Xeon(R) CPU (Gold 5218R CPU @ 2. 2.10GHz, 64 cores), 256GB memory and ITB hard drive. All experimental results are run with open-source code under the environment of Python 3.8.
14

---

Published as a conference paper at ICLR 2023
B.2 HYPERPARAMETERS
RL-agent learning rate ir = 0.001, discount factor y 0.95. The hyperparameters of Multi-Heas Attention in the policy network are follows, dmodet 64, Rhead d, 32, de 32. The maximum number of search epochs N is limited to 300 (including DIFER and FRTCH). Due to the requirements of NFS in their paper, we set N to 100 epochs for it. The number of sampling also parallelized workers
round, is W = 24. The maximum feature order K is set by K = 3. Other methods are limited to run for 5 hours respectively, which is the average running time of FETCH. All methods take their default parameters wherever possible.
We choose DIFER mostly because of its state-of-the-art performance on many benchmarks, while NFS is picked as major rivalry due to the primary comparison between the MDPs. As being identical to their setup, we employ Random Forest (Liaw et al.) 2002) as the pre-selected model and Appendix C.66 All the hyperparameters of Random Forest (except n_estimator = 10) are preset by default, and a stratified 5-fold cross-validation protocol used evaluate the effect of the selected feature engineering actions. In addition, employ Logistic Regression (Wright 1995), Xgboost (Chen & Guestrin 2016), LightGBM (Ke et 2017) and Cat-
in Experiment 4.2
boost (Prokhorenkova et al. 2018) as supplementary evaluation algorithms Experiment
and
Appendix C.5
B.3 DATASETS
Airfoil:
BikeShare http:///wwww...g.oo////////////w
Housing Boston: BtstoBo:tttts///// / http:///wwww..oooocooccoooooooon
House King hwtp://wwwww.....aagoooaaaaa/a/e
Openml_586:
https:/wwww.openm.oor/////66
Openml_588
http://www.oppomlloor///555
Openml_601
http://www..poom.lorr///6777
Openml 616:
http://www..ppemmloor//61166
Openml 618:
https//www.opeemllor///l11
Openml.620:
http://www..penmnlooo///220
Openml.637:
http://wwww.ppemmllrrd///777
Adult Income:
A httpp//wwww....//...///./oc/oo/e
Credit httpa///hhhy.cccccooocoooooo/ooe
Credit a: http:////eeillesssccccu/////aaas
German wtt.://www.aaiii.ee.e////aaaaaaa
Hepatitis: http:////aiiiiiii....oooo/////ua
Ionosphere: http:///iiiiiiiiiicccccooo/oo/ee
Megawattl: https://www.oopomllooz///////1
Messidor Messiorreaauss///t/r/a/aaaaaaaa
P
15

---

Published as a conference paper at ICLR 2023
hwtp:///iiriiiesssscccamaaassssa quality/
SpectF:
Wine Quality Red & White: http:///wiiiiiiiiesssooooooalaia
Medical Charges:
https//www..ppmm.loog////55555
Give Me Some Credit:
Poker hand:
B.4
PRE-TRAINED MODELS
No-Pre: A model without any pre-training directly searching FE plan from the scratch.
Pre-Oml: A model pre-trained on 5 datasets from OpenML.. They are Openml_586, OpenmL_589, Openmll607, Openml_618. and Openml_620.
Pre-Uei: A model pre-trained 5 datasets from the UCI repository, i.c. Bikeshare DC, Credit a, Spam Base, Wine Quality While, and Credit Defaull.
Pre-Mix: A model pre-trained on the whole 10 datasets Pre-Oml and Pre-Uci.
C ABLATION STUDY
C.I EFFICIENCY OF FETCH
To measure the efficiency of FETCH, plot the relationship between the improvements owing to feature engi- neering versus the exploration rounds, in Figure As we can see the FETCH framework can generally obtain decent results by 100-150 epochs of explo-
20
Percentage improvement
ration and then stably converges. Fur-
ther, as a comparison with NFS, we count the feature evaluation times and use them as proxy for sample com-
plexity comparison. The main ratio- nale is that this process usually takes up majority of the total running time in each round. Notably, FETCH quires around 15,000 times to con- verge while the NFS needs 160,000 times (Zhu et al., (2022). This indi- cates that FETCH about 11x more sample efficient than its counterpart, which may further concept-proof our data-driven setup for MDP.
Openmi _56
Openmi
Openmi 620
ionosphere
-5
SPECTF
50
100
150
200
250
300
350
400
Figure 6: Exploration promotion process of several data sets, little improvement after 150 rounds, which means the policy network is converged.
C.2 INFLUENCE OF HIGHER- ORDER FEATURES
In this section, evaluate the influence of higher-order features extracted by FETCH on the eventual results. Figure 7a) and Figure 7 show the effects our algorithm on the Random Forest (RF) and Logistic Regression (LR) models, respectively. The vertical axis represents the percentage of improvement, and the horizontal axis represents the order. According to the experimental results, get two conclusions: (i)-EETC can significantly improve the score of both the two models, especially LR; higher-rrrr search leads to better performance for the LR model but is not guaranteed for RF.
16

---

Published as a conference paper at ICLR 2023
200
50
Nawacter Teatures by
Nasazio Teaturas
37.5
Quility
Wene Duarry
42-
Perconage mpporerrccc
25.
Operni
1
20
20
50
682
Hone
50e
orden/3teg_uugg
(a) Random Forest
(b) Logistic Regression
Figure 7: Effect of low/high order on different models. Apply the top-1 action series T to generate features and evaluate.
Figure shows the proportion high and low-order features. We stipulate that the second-order and above features are higher-order features. It can be seen from the figure that the proportion of higher-order features is very high.
Number
Airoli
credite
GermenCredit
Hepatts
Houselioston
lomosphere
Messitorfee eonnttreeettree estures
Open mL 585
Open ml 588
Open ml, .07
Open ml, .116
Open ml .618
Open ml, .,000
Openmi ml, .657
Pimaindan
Spambase
SPECTT
whequa te
winegallty
Figure 8: Low/high order features proportions of the experimental feature sets.
C.3 SCALABILITY COMPARISON UNDER LIMITED TIME
Since the RL agent of FETCH only performs simple inference and data processing, empirically, the agent itself runs in much smaller percentage of time than the training during downstream ML model evaluation. Thus, as with the time-consuming model validation in NAS, the time performance of ML models is a bottleneck in increasing the speed of exploration in FETCH. We now evaluate the scalability of our approach on large-scale data (more than 50K rows) in Table 3 As a comparison, use RL-based AutoFE approaches (i.e. NES and DIFER) for our experiments. We compare the highest scores that can be achieved by different approaches to explore FE plans with the same runtime (2 days), hardware resources and software environment. The final score evaluated by 5-fold cross-validation after applying the highest-score FE actions plan on the entire original
Table 3: Scalability comparison of FETCH and other RL-based AutoFE
methods on large-scale datasets under limited time (2 days).
Dataset
CR
Instances Features
DIFER
Medical Charges
R
163065
0.8821
0.8831
0.8832
0,8834
Give Me Some Credit
C
150000 10
0.9329
0.9339
0.9340
0.9341
Poker hand
C
1025010',10
0.6907
0.9259
0.9647
0.9971
17

---

Published as a conference paper at ICLR 2023
dataset. The experimental datasets are from actual life scenarios. Table demonstrates that FE constructed on large-scale scale datasets can also lead to score improvements, which yields the capability for scalability at the same scale of running time.
C.4 TIME EFFICIENCY COMPARISON
This section is extended comparison of Experiment in terms of time efficiency. Table4sshows the total running time (in minutes) of several AutoFE methods on the above datasets. The results show that the time efficiency of the three AutoFE methods varies different datasets. The overall time efficiency of each method is on the same scale. DIFER, as stated in their paper, outperforms the RL-based AutoFE in time efficiency. But FETCH can achieve higher scores.
Table 4: Time efficiency comparison of FETCH with other AutoFE methods. Here is total execution time (in minutes).
CiR
Instances Features
NFS
Execution Time DIFER
RikeShare DCC
Housing Dosten
R
3061 13
54
154
House King County
R
21613/19
5058
0002
Openmi_56k
R
1000:35
329
167
192
Opeami 589
R
1000122
326
176
197
Opeani FIT
R
1000):50
1171
258
1383
Ope
R
500150
558
200
322
Openel
1234
OpeamL.60
Opennal 637
5000:50
Adull Income
48842
141
Amuzon Employee
3276919 9
379
237
Credit Default
200001 25
1622
203
1246
Crodit.
69016
59
116
99
Fertiliny
100:9
53
133
German Credit
10011.24
130
lonosphere
Lymphography
Megswant
Messidur Features
L150:19
116
160
Fimalndian
7687
11.5
SpumBass
4601::57 1:57
430
117
800
SpectF
2671-44
110
182
238
Wine Quility Red
199;12
61
292
Wine Quality Whise
4900-12
1622
205
Averag
C.5 FLEXIBILITY TOWARD MORE MODEL CHOICES
This section is supplement to the experiments in Section [..4 We measure the flexibility of FETCH using two more ML models, i.e.
Base (without FF)
FETCH (with FF)
Evaluntion scare (%)
LightGBM (Ke al., (2017) and
80
Catboost (Prokhorenkova et al.
2018). for boosting comparisons on several datasets. We com-
70
70
pare these models on the situ- ation of no feature engineering (Base) and doing feature engineer- ing with FETCH. As shown in Fig- ure our framework has the effect of promoting the fitting effect for all these models. This also reveals the excellent flexibility of FETCH to different kinds of machine learning models.
Cidde
Janappare
Miscdo
Owdi:
Ikpatti
Inaeptax
Monide
LightGBM
Catboost
Figure 9: Flexibility comparison of evaluation scores (%) on different datasets and models.
C.6 EFFECTIVENESS COMPARISON ON TEST SET
It is important to note that prior work such NES (Chen et al. 2019) and DIFER (Zhu et al. 2022) was coded with the goal of continuously improving the cross- tross-validttinn scores over the entire dataset, i.e., holding out the test set. We adopt this setup in Table [ of Experiment 4.2 and
18

---

Published as a conference paper at ICLR 2023
Table 5: Effectiveness comparison of FETCH with other AutoFE methods on test set. Underline means highest cross- cross-validation score on training set. Bold indicates superior test score amongst AutoFE methods. "CV" denotes the highest crossvvaiiattinn score obtained on the training set, and "Test" denotes the test score of the corresponding feature engineering plan on the test set.
Dataset
Base
DFS
AutoFeal
NFS
DIFER
FETCH
CV
Test
CV
Test
Test
Test
CV
Test
Airfoil
0,7119
0.7360
0,7338
0.7477
0,7307
0.7.5D
0.7424
0.7424
0.7367
0.7584
0,7572
0.7694
Bikeshare DC
0,9895
0,9901
0,9994
0.9933
0.9918
0.9922
0,9837
0.9945
0.9997
0.9997
House King County
0.6737
0.6842
0,6762
0.6884
0.6725
0.6812
0.6751
0.6879
0.6903
0.6943
0.6838
0.6844
Housing Boston
0.5992
0.6552
0,5938
0.6638
0.5982
0.6750
0.6622
0.6310
0.6827
0.6576
0.6891
Openml 586
0.6218
0.6594
0.6616
0.7189
0.6807
0.7153
0.7173
0.7310
0,7216
0.7369
0.7473
0.7688
Openml_589 .589
0.6298
0.6296
0.6587
0.6906
0.6598
0.6937
0.6804
0.6868
0,7033
0.6979
0.7189
0.736
Openml_601 607
0.6116
0.5923
0.6534
0.6172
0.6657
0.6201
0.6371
0.5780
0.6699
0.6365
0.7225
0.6860
Openml 616
0.5073
0.5251
0,5019
0.5600
0.5620
0.5578
0.5390
0.5477
0,5637
0.5187
0.6275
0.6442
Openml_611
0.5931
0.6470
0.6366
0.6397
0.6523
0.6860
0,6490
0.6450
0,6552
0.6832
0.7086
0.7187
Openmi 620
0.5911
0.6634
0.6073
0.6556
0,6456
0.6771
0.6401
0.6925
0,6760
0.6896
0.7197
0.7515
Openml 637
0.5199
0.4319
0.5242
0.4894
0.5726
0.4563
0.5217
0.4523
0.5528
0.4570
0.6586
0.5756
Adult Income
0.8464
0.8529
0.8478
0.349
0,8488
0.8382
0,8563
0.8565
0,8356
0.8580
0,856
0.8553
Amazon Employee
0.9436
0.9458
0.9430
0.9450
0,9423
0.9440
0,9462
0.9475
0.9464
0.9447
0.9466
0.9450
Credit Defauli
0,8045
0,8022
0.8034
0.8026
0.8065
0.8053
0,8092
0.8054
0.1133
0.8069
0.8110
8027
Credit
0.8506
0.8942
0.8465
0.8702
0.8403
0.8702
0,8817
0,8558
0.8817
0.8606
0.8983
0,8846
Fertility
0,8429
0.8667
0.8286
0.8667
0,8571
0.8000
0.9286
0,8000
0.9429
0.8333
0,9143
0.9000
German Credit
0.7386
0.7233
0.7586
0.7900
0.7543
0.7833
0,7857
0.7500
0.7829
0.7333
0.7871
0.7367
Hopatitis
0,7974
0,8085
0,8160
8085
0,8797
0.8723
0.9173
0.8511
0,8896
0.8085
0,9537
0.8298
Tonochpere
0,9265
0.9245
0.9347
0.9057
0.9429
0.8962
0.9592
0.9151
0,9651
0.9340
0.9796
0.9434
Lynnghogrphy
0,7962
0.8667
0,7862
0.8000
0,8348
0.8222
0,8548
0.8444
0,8543
0.8444
0,9029
0.3667
Megawattl
0.8925
0.8816
0.8870
0.8816
0.8981
0.8947
0,9822
0.8684
0.9179
0.9211
Messidor Features
0.6534
0.6387
0,7006
0.7486
0.7106
0.6965
0.7416
0.7312
0.7528
0.7254
0.7677
0.7775
Pimaludian
0.7449
0.7056
0,7691
0.6883
0.7524
0.7229
0.7953
0.7403
0.7971
0.7446
0.8120
0.7359
SpamBase
0,9410
0.9464
0,9366
0.9435
0.9385
0.9522
0.9484
0.9522
0.9484
0.9522
0.9516
0,9522
Spectt
0.8276
0.7901
0.7905
0.8148
0.7743
0.8148
0.8602
0.8708
0.8272
0.8818
0.8395
Wine Quality Red
0.6265
0.6563
0.6533
0.6625
0.6140
0.6667
0. 0.6622
0.6458
0.6613
0.6625
0.6774
0.6667
Wine Quality White
0,6144
0.6347
0.6161
0.6429
0,6144
0.6347
0.6377
0.6299
0.6365
0.6442
0.6415
0.6463
the experimental results show the superiority of FETCH. However, this setup may be contrary to
real-life application scenarios where we often need to explore good features on the training set and validate them on the test set, which is often used only once as the final evaluation metric, rather than repeatedly using the test set scores to optimize feature engineering.
To simulate the authentic situation, we split the original dataset into a training set and a test set in the ratio of 0.7/0.3 and let all AutoFE methods optimize the feature engineering plan on the training set to obtain higher cooss-valiattinn scores and apply it to the test set to obtain the final test scores. Unlike previous experiments using the default order of the datasets, in this experiment we shuffled the original data using the same random seeds, which may result in differences in scores of Table The experimental results are displayed in Table
The experimental results show that FETCH achieves both the highest cross-validation scores and test scores on 19/27 datasets. Empirically, an effective and robust feature engineering plan tends to achieve high scores on both the training and test sets. Therefore, the previous setup of directly op- timizing the cross-validation scores is reasonable and can largely improve the generalization ability of the searched feature engineering plans.
D DISCUSSION
D.1 FEASIBILITY OF APPLYING FETCH TO OTHER FORMS OF DATA
We may generalize our framework to other domains by regarding the feature outputs of an encoder (e.g. convolutional neural networks for images, recurrent neural network for speech and graph neural network for graphs (Goodfellow et al.) 2016)) as tabular inputs. However, vision and language tasks have large amount of data collection sharing common sequential/spatial structures. In view of the rich literature on these data, we suppose existing full-fledged pre- pre-trained feature extractors (ResNet (He et al., 2016). SimCLR (Chen et al. 2020). and BERT (Devlin et al. 2018)) would be a better choice. In contrast, tabular data has the properties of permutation invariance and variable length, and our FETCH is more suitable to tackle these.
Nevertheless, our approach also opens the door to learn from tabular-includer included multimodal data. For example, when the dataset contains both image and tabular attributes, we can concatenate pre- extracted image features with tabular inputs and run our FETCH model to obtain a better combination of features.
19


Reviews:
[{"review_id": "aoHGXy5R-w6", "reviewer": "Reviewer_B9Km", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 3, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper firstly specifies the fact that AutoFE is underappreciated in the AutoML pipeline tailored to tabular data nowadays, and summarizes that previous AutoFE work faced two major drawbacks of unobserved data and lack of transferability. To tackle these problems, this paper proposes a novel RL-based AutoFE framework(called FETCH) by observing the behavior patterns of human experts, which is equipped with a data-driven MDP setup in its policy network. The key idea is to consider each table as a set of features and incorporate the table into the MDP state. Then FETCH learns how to map from data to feature engineering actions. Experiments show that FETCH outperforms other SOTA or popular feature engineering baselines and supports transferability by pre-training.", "summary_of_the_review": "The idea of learning a data-driven policy network is well-motivated and novel, and the experiments are compelling.", "strength_and_weaknesses": "Strengths:\n1. The proposed framework is novel, simple and effective, which is easily to follow. The pre-training technique seems like a new schema in the tabular data mining field. \n2. The paper is well-written, and the related work is covered well.\n3. The experiments are quite comprehensive. The authors also show that pre-training FETCH on more datasets results in additional performance gains, which further strengthens the claim of the model's transferability.\n\nWeaknesses:\n1. It is nice that the authors include 27 datasets as a benchmark and demonstrate that FETCH is much better than the other AutoFE baselines in most cases. However, I don't quite sure why the authors are comparing it to popular open-source AutoML packages (i.e. AutoSklearn and AutoGluon)? After all, these are two different domains. Is it to prove that AutoFE can power AutoML? Maybe I'm missing it, and please clarify.\n2. Section 4.4 only covers 3 comparison algorithms, it would be better to add the enhancement effect on more algorithms such as LightGBM and CatBoost, which will be more convincing for the adaptability of FETCH on various algorithms.\n3. FETCH seems to have been pre-trained on only 5 datasets, which seems somewhat simple. It would be preferable to have pre-training results on a larger number of datasets.\n4. The theoretical analysis is a bit lacking. It is hoped that a theoretical analysis of effectiveness and experimental results can be provided as space permits. \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "YboF_7p_lA8", "reviewer": "Reviewer_Y4Ri", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "Automated feature engineering is another needed ingredient in 'data science as a service.' Prior work focuses either on feature selection or feature engineering that is independent on the underlying data. The authors propose an RL algorithm that directly transforms data by applying a sequence of unary or binary operations. \n", "summary_of_the_review": "The underlying idea of using the actual data in the RL setting is very novel. It is a substantial contribution. In my opinion it outweighs deficiencies in the work. \nOnce the RL is established (the state space), everything else is standard (the network, algorithm). \n\nThe authors don't mention feature selection (forward/backward selection, etc) which is perhaps injustice. Maybe feature selection should also be benchmarks (perhaps autoML methods include them). \n\nMinor remarks: \nPage 1: next to the last paragraph states that there is significant prior autoFE work. This statement contradicts some prior statements stating there is not a lot of work in this area. \nHyperparameters being fixed is questionable since different subset of features definitely dictate possible different hyperparameters. I understand that this inclusion would be a significant complication and thus I don't take this against the authors. \nPage 3: 'specified feature x' excludes binary operations. This statement should be reworded. \nPage 4: weightheight^2 should be weightheight$^2$", "strength_and_weaknesses": "Strengths: \nThe idea of using data directly is novel. \nThe use of data enables transfer learning. \n\nWeaknesses: \nScalability is questionable. There is no deep discussion on computational times and comparisons with other algorithms. \nExplainability/interpretability is questionable. To the contrary, the resulting training data is not interpretable. ", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "4: All of the claims and statements are well-supported and correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "4: The contributions are significant, and do not exist in prior works."}, "extra_text": {}}, {"review_id": "O_96yHpG7dq", "reviewer": "Reviewer_vD6x", "summary": "", "questions": "", "limitations": "", "rating": 8, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "This paper proposes a first of its kind architecture framework for automated feature engineering called Fetch, a system based on brand new data driven Markov decision process. The authors identify critical gaps by stating the current methods for AutoFe are insufficient when it comes resembling human effort in handling datasets as the underlying Markov decision setup is different i.e. more based on trial and error, which leads to poor generalization across datasets and higher computational costs. The new method also has a key element of transferability, which is basically the ability to enable feature engineering on new datasets using prior policy networks trained on previous datasets, without the need for additional data exploration. The authors present evidence that Fetch is superior to the existing state of the art automated feature engineering methods such as Random, DFS, AutoFeat, NFS, DIFER as well as AutoML methods such as AutoSklearn and AutoGluon. The method is also tested for transferability by application on several datasets. \n\nThe authors argue that the approach comes very close to mimicking human experts when it comes to handling new datasets when it comes to transferring experience. ", "summary_of_the_review": "I propose that we accept this paper on the basis of its merits around developing a new AutoFE method based on a novel markov decision process which not only is superior to the existing state of the art in terms of performance but also has a unique feature of transferability. Both strong points are well argued and evidenced in the paper throughout with sufficient gap analysis around opportunity areas, discussion of background and related work, strong mathematical rigor and sound experimental design setup, with thorough data analysis, discussion of evaluation and results. Although the work is limited to tabular data there is thoughtful coverage on several popular datasets including regression and classification examples and comparison to other AutoFE models and coverage across several ML models. ", "strength_and_weaknesses": "Strengths: \n1. One of the key strengths of this paper is its ability to provide transferability between datasets which previous methods have not been able to provide. \n2. The method is highly flexible as it has been applied on different datasets as well different ML models. \n3. Overall there is sound discussion relevant work, gap analysis to find opportunity of development, strong mathematical rigour and experimental setup, evaluation and analysis. \n\n\nWeaknesses: \n1. The authors declare that to the best of their knowledge there aren't any autoFE/autoML workarounds for managing tabular data to accomplish transferability. However this could be supported by a stronger statement that can more concretely say if such methods exist or not by clearly stating a comprehensive survey of analysis did not yield any methods. \n2. This method is specifically geared towards tabular datasets. While tabular datasets prevail in several key applications, it would be interesting to know how this method applies/does not apply to any other form of datasets (eg. images, speech, unstructured datasets). At least there could be some discussion on consideration, if not a full scale evaluation.  \n", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"correctness": "3: Some of the paper’s claims have minor issues. A few statements are not well-supported, or require small changes to be made correct.", "technical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work.", "empirical_novelty_and_significance": "3: The contributions are significant and somewhat new. Aspects of the contributions exist in prior work."}, "extra_text": {}}]

Author rebuttal:

Decision: Accept: notable-top-25%