Reinforcement-Enhanced Autoregressive Feature Transformation: Gradient-steered Search in Continuous Space for Postfix Expressions
Authors: Dongjie Wang
Meng Xiao
Min Wu
pengfei wang
Yuanchun Zhou
Yanjie Fu
Venue: NeurIPS 2023 spotlight (2023)
Keywords: Feature Transformation
Autoregressive Generation
Continuous Space Optimization

Abstract: Feature transformation aims to generate new pattern-discriminative feature space from original features to improve downstream machine learning  (ML) task performances. However, the discrete search space for the optimal feature explosively grows on the basis of combinations of features and operations from low-order forms to high-order forms. Existing methods, such as exhaustive search, expansion reduction, evolutionary algorithms, reinforcement learning, and iterative greedy, suffer from large search space. Overly emphasizing efficiency in algorithm design usually sacrifice stability or robustness. To fundamentally fill this gap, we reformulate discrete feature transformation as a continuous space optimization task and develop an embedding-optimization-reconstruction framework. This framework includes four steps: 1) reinforcement-enhanced data preparation, aiming to prepare high-quality transformation-accuracy training data; 2) feature transformation operation sequence embedding, intending to encapsulate the knowledge of prepared training data within a continuous space; 3) gradient-steered optimal embedding search, dedicating to uncover potentially superior embeddings within the learned space; 4) transformation operation sequence reconstruction, striving to reproduce the feature transformation solution to pinpoint the optimal feature space. Finally, extensive experiments and case studies are performed to demonstrate the effectiveness and robustness of the proposed method. The code and data are publicly accessible https://www.dropbox.com/sh/imh8ckui7va3k5u/AACulQegVx0MuywYyoCqSdVPa?dl=0.

Reinforcement-Enhannee Autoregressive Feature Transformation: Gradient-steered Search in Continuous Space for Postfix Expressions
Dongjie Wang" Department of CS University of Central Florida dongjie ie. wangOucf..o edu
Meng Xiao* CNIC, CAS University of CAS
Min Wu Institute for Infocomm Research S*SAAR wumin0i2r. s-sr edu. sg
shaowecnic. on
Pengfei Wang CNIC, CAS University of CAS wpf@cnic.cm cn
Yuanchun Zhou CNIC, CAS University of CAS zyc@cnic.cm cn
Yanjie Fut
School of Computing and Al
Arizona State University
yanjiefußasu. edu
Abstract
Feature transformation aims to generate new pattern-discrinin dintriniiirr feature space from original features to improve downstream machine learning (ML) task perfor- mances. However, the discrete search space for the optimal feature explosively grows the basis of combinations of features and operations from low-order forms high-order forms. Existing methods, such exhaustive search, expansion reduction, evolutionary algorithms, reinforcement learning, and iterative greedy, suffer from large search space. Overly emphasizing efficiency in algorithm design usually sacrifices stability or robustness. To fundamentally fill this gap, we refor- mulate discrete feature transformation continuous space optimization task and develop
framework. This framework data preparation, aiming to prepare
includes four steps: 1) reinforcement-enhheee high-quality transformation-accuse training data; 2) feature transformation oper- ation sequence embedding, intending to encapsulate the knowledge of prepared training data within continuous space; 3) muddinnt-sterred optimal embedding search, dedicating uncover potentially superior embeddings within the learned space; 4) transformation operation sequence reconstruction, striving to reproduce the feature transformation solution to pinpoint the optimal feature space. Finally, extensive experiments and case studies are performed to demonstrate the effec- tiveness and robustness of the proposed method. The code and data publicly accessible http:://vvw dropboz. con/ eh/ http///nb/c/eaaa/eee www.c//iiwss.s//AAoooCooo wwwp:////weee//Aaeeeeeeeeoooo wob.aooeeeeeeekoooyyrroorrr
Introduction
Feature transformation aims to derive a new feature space by mathematically transforming the original features enhance the downstream MI task performances. However, feature transformation is usually manual, time-consuming, labor-intensive, and requires domain knowledge. These limitations motivate us accomplish A Automated Feature Transformation (AFT). AFT is fundamental task because AFT can 1) reconstruct distance measures, 2) form a feature space with discriminative patterns, ease machine learning, and overcome complex and imperfect data representation.
There are two main challenges in solving AFT: 1) efficient feature transformation in a massive discrete search space; 2) robust feature transformation in open learning environment. Firstly,
'These authors have contributed equally to this work.
"Coresponding Author
37th Conference on Neural Information Processing Systems (NcurIPS 2023).

---

is computationally costly to reconstruct the optimal feature space from a given feature set. Such reconstruction necessitates a transformation sequence that contains multiple combinations of features and operations. Each combination indicates a newly generated feature. Given the extensive array of features and operations, the quantity of possible feature-operation combinations exponentially grows, further compounded by the vast number of potential transformation operation sequences. The efficiency challenge seeks to answer: how can we efficiently identify the best fecture transformation operation sequence? ? Secondly, identifying the best transformation operation sequence is unstable and sensitive to many factors in an open environment. For example, if we formulate a transformation operation sequence as a searching problem, it is sensitive to starting points or the greedy strategy during iterative searching. training data quality and the complexity of generation forms. The robustness challenge aims to answer: how can we robustify the generation of feature transformation sequences?
we identify the same task as generation problem, it is sensitive to
literature partially addresses the two challenges. Existing AFT algorithms can be grouped into three categories: 1) appansion-reduction
Prior
approaches | in which all mathematical operations are randomly applied to all features at once to generate candidate transformed features, followed by feature selection to choose valuable features. However, such methods are based on random generation, unstable, and not optimization-directieo which feature generation and selection are integrated, and learning strategies for each are updated based feedback in each iteration. Two example methods are Evolutionary Algorithms (EA) or Reinforcement Learning (RL) with downstream ML task accuracy as feedback. However, such methods are developed based on searching in a massive discrete space and are difficult to converge in comparison to solving a continuous optimization problem. 3) Neural Architecture Search (NAS)- based approaches 7 NAS was originally to identify neural network architectures using a discrete search space containing neural architecture parameters. Inspired by NAS, some studies formulated AFT as a NAS problem. However, NAS-based formulations are slow and limited in modeling all transformation forms. Existing studies show the inability to jointly address efficiency and robustness in feature transformation. Thus, we need novel perspective derive a novel formulation for AFT.
2) iterative-feedback approaches 4 5 6, in
Our Contribution: A Pustfix Expression Embedding and Generation Perspective. To fun- damentally fill these gaps, we formulate the discrete AFT problem continuous optimization task and propose a reinforceMeenteehaanee
autOregressive feAture Transformation framework, namely MOAT. To advance efficiency and robustness, this framework implements four steps: 1) eeinforcement-ebhanced training data preparation; 2) feature transformation operation sequence embedding; 3) gradient-steered optimal embedding search; 4) beam search-based transformation operation sequence reconstruction. Step 1 is to collect high-quality transformation operation sequence- accuracy pairs as training data. Specifically, we develop a cascading reinforcement learning structure to automatically explore transformation operation sequences and test generated feature spaces on a downstream predictor (e.g., decision tree). The self-optimizing policies enable agents collect high-quality transformation operation sequences. The key insight is that when training data is difficult or cost expensive to collect, reinforcement intelligence can used as an automated training data collector. Step 2 is to learn continuous embedding space from transformation -awtorramaiiinnaccrcyyoaoaa training data. Specifically, we describe transformation operation sequences as postfix expressions, each of which is mapped into an embedding vector by jointly optimizing the transformation operation sequence reconstruction loss and accuracy estimation loss. Viewing feature transformation through the lens of postfix expressions offers the chance to mitigate the challenges associated with an exponentially expanding discrete search problem. By recasting the feature transformation operation sequence in postfix form, the search space becomes smaller, as each generation step involves selecting single alphabetical letter pertaining to a feature or mathematical operation, rather than considering high-order expansions. Moreover, this postfix form captures feature-featur interaction information and empowers the generation model with the capacity to autonomously determine the optimal number and segmentation way of generated features. Step 3 is to leverage the gradient calculated from the improvement of the accuracy evaluator to guide the search for the optimal transformation operation sequence embedding. Step 4 is to develop beam search-based generative module reconstruct feature transformation operation sequences from embedding vectors. Each sequence is used to obtain a transformed feature space, and subsequently, the transformed space that yields the highest performance in the downstream ML task is identified as the optimal solution. Finally, we present extensive experiments and case studies show the effectiveness and superiority of our framework.
2

---

2 Definitions and Problem Statement
2.1 Important Definitions
Operation Set. To refine the feature space for improving the downstream ML models, we need to apply mathematical operations to existing features to generate new informative features. All operations are collected in an operation set, denoted by O. Based on the computation property, these operations can be classified as unary operations and binary operations. The unary operations such as "square", "exp". "log". etc. The binary operations such as "plus", "multiply", "minus", etc.
Cascading Agent Structure. We create cascading agent structure comprised of head feature agent, operation agent, and tail feature agent to efficiently collect quantities of high-quality feature transformation records. The selection process of the three agents will share the state information and sequentially select candidate features and operations for refining the feature space.
Feature Transformation Operation Sequence. which includes the original feature set X As shown in Figure I transform the existing ones using mathematical compositions consisting of feature IDD tokens and operations generate new and informative features. K transformation composi- tions are adopted to refine X to better feature space X [f..... ... fk). The collection of the K compo- sitions refers to the feature transformation sequence, which is denoted by I [TI.-.. TKK.
Assuming that D {X. v) is dataset, [ [Nv] and predictive targets
Tn m + 111+/11, 50) Uh (v Tal)
(i2)
Figure An example of feature transforma- tion sequence: T(-) indicates the generated feature that is the combination of original fea-
tures and mathematical operations.
2.2 Problem Statement
We aim to develop an effective and robust deep differentiable automated feature transformation framework. Formally, given dataset D {X, y} and an operation set O we first build cascading RL-agent structure to collect n feature transformation accuracy pairs as training data, denoted by R = {(T,,; where T; the transformation sequence and V/ is the associated downstream predictive performance. We pursue two objectives thereafter: 1) building an optimal continuous embedding space for feature transformation sequences. We learn a mapping function Φ, reconstructing function v.. and an evaluation function w to convert R into continuous embedding space & via joint optimization.
&, each embedding point associated with feature transformation sequence and corresponding predictive performance. 2) identifying the optimal feature space. We adopt gradient-based search to find the optimal feature transformation sequence I", given by:
I (EE)
A(M (X)), v):
where reconstruct feature transformation sequence from any embedding point of E; E is an embedding vector in and E is the optimal one; M is the downstream ML model and .A is the performance indicator. Finally, apply I" transform X the optimal feature space X* maximizing the value A.
3 Methodology
3.1 Framework Overview
Figure shows the framework of MOAT including four steps: 1) reinforcement-enhancee transformation-accuraayy data preparation; 2) deep postfix feature transformation operation sequence embedding; 3) gradient-ascent optimal embedding search; 4) transformation operation sequence re- construction. In Step 1, cascading agent structure consisting of two feature agents and one operation agent is developed to select candidate features and operators for feature crossing. The transformed feature sets are applied to downstream ML task collect the corresponding accuracy. The data collection process is automated and self-optimized by policies and feedback in reinforcement learning. We then convert these feature transformation operation sequences into postfix expressions. Step 2, we develop encoder-evalle ponoddrreevllutorrdeedeeee model to embed transformation operation sequence- accuracy pairs into a continuous embedding space by jointly optimizing the sequence reconstruction loss and performance evaluation loss. In detail, the encoder maps these transformation operation

---

sequences into continuous embedding vectors; the evaluator assesses these embeddings by predicting their corresponding model performance; the decoder reconstructs the transformation sequence using these embeddings. In Step 3, we first learn the embeddings of top-ranking transformation operation sequences by the well-trained encoder. With these embeddings as starting points, we search along the gradient induced by the evaluator to find the acceptable optimal embeddings with better model performances. In Step 4, the well-trainee decoder then decodes these optimal embeddings to generate candidate feature transformation operation sequences through the beam search. We apply the feature transformation operation sequences to original features to reconstruct refined feature spaces and evaluate the corresponding performances of the downstream predictive ML task. Finally, the feature space with the highest performance is chosen as the optimal one.
Tramffrmaaivn-arreee Dataa Pregaration
Deep Feature Trandfommaioon Embedling
Recondrrcciinn and Eraination
Tramshemalis Sequence
Relnfleonci Learing
Piatar Trandirmaee
Potli Netate Bandd
Deroder
Festare
Pelccy
Itato
tood
3/1
Thodeane
Innteskiigg
Gradient-aseen Optimal Kisheddingg Search
Poite
acre
Val
Operataree
Poige Eiono
Adion
Roverhs
Facoder
Relead StarcSppe
Astiods
Figure 2: An overview of our framework. MOAT consists of four main components: 1) wransorrattion-aacuaaa data preparation; 2) deep feature transformation embedding: 3) gradient- ascent optimal embedding search; 4) transformation sequence reconstruction and evaluation.
3.2 Reinforcement Training Data Preparation
Why Using Reinforcement Training Data Collector. Our extensive experimental analysis shows that the quality of embedding space directly determines the success of feature transformation operation sequence construction. The quality of the embedding space is sensitive to the quality and scale of transformation eqquence-accuracy training data: training data is large enough to represent the entire distribution; training data include high-performant certain random exploratory samples. Intuitively, we can use random sample features and operations to generate feature transformation sequences. This strategy is inefficient because it produces many invalid and low-quality samples. Or, we can use existing feature transformation methods (e.g., AutoFeat (2) to generate corresponding records. However, these methods are not fully automated and produce limited number of high -uuaitty transformation records without exploration ability. We propose to view reinforcement learning as a training data collector to overcome these limitations.
feature transformation cases, along with
Reinforcement Transformation-Accuaae Training Data Collection. Inspired by 4 9. we formulate feature transformation three interdependent Markov decision processes (MDPs). We develop a cascading agent structure to implement the three MDPs. The cascading agent structure consists of a head feature agent, an operation agent, and tail feature agent. In each iteration, the three agents collaborate to select two candidate features and one operation to generate a new feature. bredback-baseed policy learning is used optimize the exploratory data collection to find diversified yet quality feature transformation samples. To simplify the description, we adopt the i-th iteration as an example to illustrate the reinforcement data collector. Given the former feature space as X,, we generate new features X using the head feature fh. operation o, and tail feature fr selected by the cascading agent structure.
1) Head feature agent. This learning system includes: State: is the vectorized representation of X;- Let Rep(-) be state representation method, and the state can be denoted by Rep(X.). Action: is the head feature fn selected from X; by the reinforced agent.
2) Operation agent. This learning system includes: State: includes the representation of X, and the head feature, denoted by Rep( X.) @ Rep(Jn), where () indicates concatenation. Action: the operation selected from the operation set O.

---

3) Tail feature agent. This learning system includes: State: includes the representation of X,, selected head feature, and operation, denoted by Rep(X,) @ Rep(fi) @ Rep(o). Action: is the tail feature fr selected from X; by this agent.
4) State representation method, Rep(.). For the representation of the feature set, we employ descriptive statistical technique to obtain the state with a fixed length. In detail, we first compute the descriptive statistics (i.e. count, standard deviation, minimum, maximum, first, second, and third quantile) of the feature set column-wise. Then, we calculate the same descriptive statistics on the output of the previous step. After that, we can obtain the descriptive matrix with shape RTx7 and flatten it as the state representation with shape I x49 For the representation of the operation, we adopt its one-hot encoding as Rep(o).
5) Reward function. To improve the quality of the feature space, we use the improvement of a downstream MIL task performance the reward. Thus, it can be defined as: R(X,, X(+1) A(M(x++1) - A(M(X,)),
6) Learning to Collect Training Data. To optimize the entire procedure, we minimize the mean squared error of the Bellman Equation to get a better feature space. During the exploration process, we can collect amounts of high-quality records (T, v) for constructing an effective continuous embedding space, where I' is the transformation sequence, and v is the downstream model performance.
3.3 Postfix Expressions of Feature Transformation Operation Sequences
Why Transformation Operation Sequences as Postfix Expressions. After training data collection. a question arises: how can we organize and represent these transformation operation sequences in computationalllyttngiiee and machine-learnable format?
Figure (a) shows an example of a feature trans-
T1
T2
formation operation sequence with two gener-
(fi + (fi/fs) f2). (vi)
ated features. To convert this sequence into a machine-readable expression, naive idea is to
(a) Feature transformation Sequence
enclose each calculation in pair of brackets to
indicate its priority (Figure 3 b)). But, the repre- sentation of Figure bb) has four limitations: (1) Redundancy. Many priority-related brackets are included to ensure the unambiguous property and correctness of mathematical calculations. (2) Semantic Sparsity. The quantity of bracket tokens can dilute the semantic information of the sequence, making model convergence diffi- cult. (3) Illegal Transformation. If the decoder makes one mistake on bracket generation, the entire generated sequence will be wrong. (4) Large Search Space. The number of combinations of features and operations from low-order to high-order interactions is large, making the search space too vast.
(b) Original infix notation based sequence
(c) Postfix notation based sequence
Figure 3: Compared to infix-based expressions, postfix-based expressions optimize token redun- dancy, enhance semantics, prevent illegal transfor- mations, and minimize the search space.
Using Postfix Expression to Construct Robust, Concise, and Unambiguous Sequences. To address the aforementioned limitations, we convert the transformation operation sequence I into a postfix-based sequence expression. Specifically, we scan each mathematical composition T in I from left to right and convert it from the based format to the postfix- based one. We then concatenate each postfix expression by the <SEP> token and add the <SOS> and <EOS> tokens to the beginning and end of the entire sequence. Figure (c shows an example of such postfix sequence. We denote it as Y [n.... 7.). where each element is feature ID token, operation token, or three other unique tokens. The detailed pscudo code of this conversion process is provided in Appendix
The postfix sequences don't require numerous brackets to ensure the calculation priority. We only need scan each element in the sequence from left right reconstruct corresponding transformed features. Such concise and short expression can reduce sequential modeling difficulties and computational costs. Besides, postfix sequence indicates a unique transformation process, thus, reducing the ambiguity of the feature transformation sequence. Moreover, the most crucial aspect is the reduction of the search space from exponentially growing discrete combinations to a limited token set C that consists of the original feature ID tokens, operation tokens, and other three unique tokens. The length of the token set is O + [X] + 3, where |O] is the number of the operation set, |X| is the dimension of the original feature set, and 3 refers to the unique tokens <SOS>, <SEP>, <EOS>.
5

---

Data Augmentation for Postfix Transformation Sequences. Big and diversified transformation sequence records can benefit the learning of pattern discriminative embedding space. Be sure to notice that, feature transformation sequence consists of many independent segmentations, each of which is the composition of feature ID and operation tokens and can be used to generate a new feature. These independent segmentations are order-agnostic. Our idea is to leverage this property to conduct data augmentation to increase the data volume and diversity. For example, given a transformation operation sequence and corresponding accuracy {T, v). we first divide the postfix expression into different segmentations by <SEP>. We then randomly shuffle these segmentations and use <SEP> to concatenate them together to generate postfix transformation sequences. After that, we pair the new sequences with the corresponding model accuracy performance to improve data diversity and data volume for better model training and to create the continuous embedding space.
3.4 Deep Feature Transformation Embedding
After collecting and converting large-scale feature transformation training data to a set of postfix expression-accuracy pairs {(T,, e,)] y we develop encoder-evaluutor-decddd structure to map the sequential information of these records into an embedding space. Each embedding vector is associated with transformation operation sequence and its corresponding model accuracy.
Encoder d: The Encoder aims to map any given postfix expression to an embedding (a.k.a., hidden state) E. We adopt single layer long short-term memory [10] (LSTM) as Encoder and acquire the continuous representation of T, denoted by E = (() E R RMM where M is the total length of input sequence Y and d is the hidden size of the embedding.
Decoder v:: The Decoder aims reconstruct the postfix expression of the feature transformation operation sequence Y from the hidden state E. In MOAT, we set the backbone of the Decoder as a single-layer LSTM. For the first step, e will take an initial state (denoted ho) as input. Specifically, in step-i, we can obtain the decoder hidden state he from the LSTM. We use the dot product attention to aggregate the encoder hidden state and obtain the combined encoder hidden state hf. Then, the
exp(Ws, (N()A))
distribution in step-j can be defined as: Pe (74 E, Te))) Erec (Tp(W((( where % " is the i-th token in sequence Y, and C is the token set. W stand for the parameter of the feedforward network. Tei represents the prediction of the previous or initial step. By multiplying the probability in each step, we can form the distribution of each token in Y, given as:Po ( To make the generated sequence similar to the real one, we minimize the negative log- likeliool of the distribution, defined as: Lree - log (T)
I Pe E,, Te))
The Evaluator is designed to estimate the quality of continuous emheddings. Specifi- cally, we will first conduct mean pooling on E by column to aggregate the information and obtain the embedding E Rd. Then E is input into feedforward network to estimate the corresponding model performance, given as: = w (E). To minimize the gap between estimated accuracy and real-world gold accuracy, we leverage the Mean Squared Error (MSE) given by: Lest MSE(U, so (E)).
Evaluator
Joint Training Loss C: We jointly optimize the encoder, decoder, and evaluator. The joint training loss can be formulated as: L = aCree + (1 at) Ceet, where a is the trade-off hyperparameter that controls the contribution of sequence reconstruction and accuracy estimation loss.
3.5 Gradient-Ascent Optimal Embedding Search
To conduct the optimal embedding search, we first select top-T transformation sequences ranked by the downstream predictive accuracy. The well-trained encoder is then used to embed these postfix expressions into continuous embeddings, which later will be used as seeds (starting points) of gradient search, starting from E, toward the gradient direction induced by the evaluator w: E E a where E denotes the refined embedding, y is the size of each searching step. The model performance of E is supposed to be better than E due to w E) 2 (E). For seeds, we can obtain the enhanced embeddings [E, Eg.... Er).
ascent. Assuming that one search seed embedding is E,
ow ag
3.6 Transformation Operation Sequence Reconstruction and Evaluation
We reconstruct the transformation sequences by the well-trained decoder & using the collected candidate (i.c., acceptable optimal) embeddings [E,, Eg, Err. This process can be denoted
by: E. E2. Erl ", (T.). To identify the best transformation sequence, we adopt the

---

to generate feature transformation operation sequence candidates. Specifically, given refined embedding E, at step-t, we maintain the historical predictions with beam size b, denoted as 1- For the i-th beam, the probability distribution of the token identified by the well-trained decoder up at the f-th step is is y, which can be calculated as follows: (i()) = (r Tx))== p (T <i E). where the probability distribution (7)) is the continued multiplication between the probability distribution of the previous decoding sequence and that of the current decoding step. We can collect the conditional probability distribution of all tokens for each beam. After that, we append tokens with top-b probability values to the historical prediction
beam search strategy [T]] 13
of each beam to get a new historical set (T<4+1)=11 We can iteratively conduct this decoding process until confronted with the <EOS> token. We select the transformation sequence with the highest probability value as output. Hence, enhanced embeddings may produce transformation sequences (f,) We divide each of them into different parts according to the <SEP> token and check the validity of each part and remove invalid ones. Here, the validity measures whether the mathematical compositions represented by the postfix part can be successfully calculated to produce new feature. These valid postfix parts reconstruct a feature transformation operation sequence (T.)² which are used to generate refined feature space {X,}]= Finally, we select the feature set with the highest downstream ML performance as the optimal feature space X*.
4 Experiments
This section reports the results of both quantitative and qualitative experiments that were performed to assess MOAT with other baseline models. All experiments were conducted on AMD EPYC 7742 CPU, and 8 NVIDIA A100 GPUs For more platform information, please refer to Appendix B.2
4.1 Datasets and Evaluation Metrics
We used 23 publicly available datasets from UCI [14], LibSVM [5, Kaggle [16. and OpenML 17
to conduct experiments. The 23 datasets involve 14 classification tasks and 9 regression tasks. Table | shows the statistics of these datasets. We used F1-score, Precision, Recall, and ROC/AUC to evaluate classification tasks. We used --Relative Absolute Error (1-RAE) 4. 1 1-Mean Average Error (1-MAE), 1-Mean Square Error (I-MSE), and 1-Root Mean Square Error (1 -MRSS)) to evaluate regression tasks. We used the Valid Rate to evaluate the transformation sequence generation. A valid sequence means it can successfully conduct mathematical compositions without any ambiguity and errors. The valid rate is the average of all correct sequence numbers divided by the total number of generated sequences. The greater the valid rate is, the superior the model performance is. Because indicates that the model can capture the complex patterns of mathematical compositions and search for more effective feature transformation sequences.
4.2 Baseline Models
We compared our method with eight widely- feature generation methods: (1) RDG generates feature- wwworpttonofaatuueeee transformation records at random for generating new feature space; (2) ERG first applics operation on each feature to expand the feature space, then selects the crucial features as new fcatures. (3) LDA [18] is matrix factorization-base method to obtain the factorized hidden state as the generated feature space. (4) AFAT 2 is an enhanced version of ERG that repeatedly generate new features and use multi-step feature selection select informative ones. (5) NFS 77) models the transformation sequence of each feature and uses RL to optimize the entire feature generation process. (6) TTG (5) formulates the transformation process as a graph, then implements an RL-based search method to find the best feature set. (7) GRFG 4 uses three collaborated reinforced agents to conduct feature generation and proposes feature grouping strategy to accelerate agent learning. (8) DIFER ( embeds randomly generated feature transformation records with seq2seq model, then employs gradient search to find the best feature set. MOAT and DIFER belong to the same setting. We demonstrate the differences between them in Appendix ??. Besides, we developed two variants of MOAT in order to validate the impact of each technical component: (i) MOAT replaces the RL-based data collection component with collecting feature cansformation-aacuraa (ii) MOAT removes the data augmentation component. We randomly split each dataset into two independent sets. The prior 80% is used to build the continuous embedding space and the remaining 20% is employed to test transformation performance. This experimental setting avoids any test data leakage and ensures a fairer transformation performance comparison. We adopted Random Forest as the downstream machine learning model. Because it is a robust, stable, well-tested method, thus,
pairs random.
7

---

Table I: Overall performance comparison. 'C' for binary classification, and 'R' for regression. The best results are highlighted in bold. The second-best results are highlighted in underline. (Higher values indicate better performance.)
Duraset
Source
Samples
Features
RDG
LDA
NFS
GRFG
DIFER
MOAT
Higes Bosen
UCCrvvne
C
50000
28
0.695
0.702
0,513
0.697
0.691
0.699
0.707
0.669
0.712
Amazon Employec
Kaggle
32769
0.32
0.96
0.929
0.936
Pimalndion
UiClvvine
C
768
0.260
0.761
0.638
0.765
0.249
0.745
0.754
0.760
0.807
Specil"
UCIiviee
267
0.760
0.757
0.665
0.760
0.792
0.760
0.818
0.766
0.912
SVMGuide3
L
1243
21
0.787
0.826
0.652
0.795
0.792
0.798
0.812
0.773
0.849
German Credit
UClivine
C
1001
24
0.680
0.740
0.639
0.683
0.687
0.645
0.683
0.656
0.730
Credit Defoult
UCiivine
3000
0.805
0.803
0.743
0,804
0.801
0.798
0,800
0.79
0.810
Messidor leatures
Iryine
1150
19
0.624
0.009
0.475
0.065
0.6.38
0.655
0.892
0.000
0.749
Quality Red
UCIivine
0.435
0.480
0.462
0,470
0.476
0.559
Wine Quality White
UCIrvine
C
4900
12
0.524
0.510
0.449
0,516
0.525
0.531
0.534
0.507
0.536
SpamBase
UClrvine
0.906
0,889
0.925
0.912
0.932
AP-orentum-ourry
Vpe
275
109356
0.832
0.814
0,658
0.830
01832
0.758
01849
0,833
0.8K3
Lymphography
Urvive
148
0.108
0.144
0.167
0.150
0.152
0.148
0,182
0.150
0.267
lonosphere
UCIivine
351
34
0.912
0.921
0,654
0.928
0.913
0.902
0.933
0.905
0.985
Housing Boston
UClivine
R
506
0.404
0.409
0.030
0.416
0.425
0.396
0.404
0.381
0.467
Airfoil
1503
D.19
0.519
0.220
0.521
0.519
0.500
0.521
0.558
0.629
Openinl 618
Vl
R
1050
50
0.472
0.561
0,052
0.472
0.473
0.467
0.362
0.408
0,692
Opennn
OLeaML
1000
0.309
0.011
0.505
0.463
0.656
Opeaml 616
OpenML
R
500
50
0.070
0.193
0.024
0.149
0.167
0.156
0.372
0.076
0.526
Openmi 607
ML
1000
0.521
0.107
0.519
0.522
0.621
0.476
0.673
Openml 620
OllnVil.
1000
0.511
0.546
0.029
0.527
0.513
0.512
0.619
0.442
0.642
Openmnl 637
OpenMLL
500
0.136
0.152
0.043
0.176
0.152
0.144
0.307
0.072
0.465
Openml 586
Open
1000
0.568
0.110
0.543
0.544
0.646
0.482
0.700
We reported FI-Score for classification tasks, and 1-RAE for regression tasks.
10.
6.
02-
02
M3
0.2
MAXT
0.2
MONT
(a) Spectf
(b) SpamBase
(c) Openml 616
(d) Openml 618
Figure 4: The influence of data collection (MOAT 4) and data augmentation (MOAT ") in MOAT.
we can reduce performance variation caused by the model, and make it easy to study the impact of feature space. We provided more experimental and hyperparameter settings in Appendix B.I
4.3 Performance Evaluation
Overall Performance This experiment aims to answer: Can MOAT effectively generate transfor- mation sequence for discovering optimal feature space with excellent performance? Table shows the overall comparison between MOAT and other models in terms of F1-score and 1-RAE. We noticed that MOAT beats others on all datasets. The underlying driver that MOAT builds an effective embedding space to preserve the knowledge of feature transformation, making sure the gradient-ascem search module can identify the best-transformed feature space following the gradient direction. Another interesting observation is that MOAT significantly outperforms DIFER and has a more stable performance. There are two possible reasons: 1) The high-quality transformation records produced by the RL-based data collector provide robust and powerful foundation for building discriminative embedding space; 2) Postfix notation- based transformation sequence greatly decreases the search space, making MOAT easily capture feature transformation knowledge. Thus, this experiment validates the effectiveness of MOAT.
Data collection and augmentation. This experiment aims to answer: Is it essential to conduct data collection and augmentation to maintain the performance of MOAT? To achieve this goal, we devel- oped two model variants of MOAT: 1) MOAT which randomly collects feature transformation records for continuous space construction: 2) MOAT a which removes the data augmentation step in MOAT. We select two classification tasks (i.e., Spectf and SpamBase) and two regression tasks (i.e.. Openml 616 and Openml 618) show the comparison results. The results are reported in the Figure 4 Firstly, we found that the performance of MOAT is much better than MOAT-4, The underlying driver is that high-quality transformation records collected by the RL-based collector build robust foundation for embedding space learning. It enables gradient-ascent search to effectively identify the optimal feature space. Moreover, we observed that the performance of MOAT is infe- rior to MOAT. This observation reflects that limited data volume and diversity cause the construction
8

---

Table 3: Time complexity comparison between MOAT and DIFER
Model
Data Collection
Model Training
Solution Searching
Dataset
Performance
Name
512 instances
Wine Red
MOAT
921.6
5779.2
101.2
0.559
Wine White
MOAT
2764.8
7079.6
103.4
0.536
Openml 618
MOAT
4556.8
30786.1
111.3
0.692
Openml_550 _589
MOAT
4044.8
8942.3
105.2
0.656
Wine Red
DIFER
323,2
11
180
0.476
Wine White
DIFER
1315.8
33
624
0.507
Openml_611 [
DIFER
942.3
33
534
0.408
OpenmL_589
DIFER
732.5
157
535
0.463
of embedding space to be unstable and noisy. Thus, this experiment shows the necessity of the data collection and augmentation components in MOAT.
Beam search. This experiment aims to answer What is the influence of beam search for improv- ing the valid rate of generated transformation sequences? To observe the impacts of beam search, we set the beam size as and respec- tively, and add DIFER another comparison object. We compare the generation performance in terms of valid rate. Figure 3 shows the com-- parison results in terms of valid rate. We no- ticed that the 5-beams search outperforms the 1 bebem search. The underlying driver is that the increased beam size can identify more legal and reasonable transformation sequences. Another interesting observation is that DIFER is significantly worse than MOAT and its error bar is longer. The underlying driver is that DIFER collects transformation records at random and it generates transformation sequences using greedy search. The collection and generation ways involve more random noises, distorting the learned embedding space.
100
MDAT (5-Beerm))
DIPER
MDAT (1-Bwam))
Valid Rate (%)
60-
20-
0.
Awttton Eneloino
50-Comertum Orery
Coult Detbuck Vina Quality While
Opeent, St1
Figure 5: The influence of beam size in MOAT.
Robustness check. This experiment aims to answer: Is MOAT robust different down- stream machine learning models? We replaced the downstream ML models with Random For-
Table 2: Robustness check of MOAT with distinct
MI. models on Spectf dataset terms of F1 -ccore.
RF
XGB
3VM
KNN
Ridge
LASSO
DT
est (RF), XGBoost (XGB), Support Vector Ma- chine (SVM), K-Nearest Neighborhood (KNN),
RIG
0,760
0.888
0.750
0.792
0.718
0.749
0.864
ERG
0.757
0.13
0.783
0.766
0.778
0.750
0.790
Ridge, LASSO, and Decision Tree (DT) to ob- serve the variance of model performance respec-
LDA
0.668
0.715
0.760
0.749
0.759
0,760
D.665
AFAT
0.760
0.808
0.722
0.759
0.723
0.720
0.844
NES
0.792
0.799
0.732
0.792
0.744
0.749
0.864
tively. Table Showws the comparison results on Spectf in terms of the Fl-score. We observed that MOAT keeps the best performance regard-
TTG
0.760
0.819
0.765
0.716
0.749
0.512
GRPG
0.188
0.842
0.580
0.760
0.729
0.744
0.786
DIPER
0.766
0.794
0.727
0.777
0.647
0.744
0.809
MOAT
0.912
0.897
0.876
0.16
0.780
0.844
0.929
less of downstream ML models. possible reason is that the RL-based data collector can customize the transformation records based on the downstream ML model. Then, the learned embedding space may comprehend the preference and properties of the ML model, thereby resulting in a globally optimal feature space. Thus, this experiment shows the robustness of MOAT.
Time complexity analysis. This experiment aims to answer What is the time complexity of each technical component of MOAT? To accomplish this, we selected the SOTA model DIFER as a comparison baseline, Figure 3 shows the time costs comparison of data collection, model training. and solution searching in terms of seconds. We let both MOAT and DIFER collect 512 instances in the data collection phase. We found that the increased time costs for MOAT occur in the data collection and model training phases. However, once the model converges, MOAT 's inference time significantly reduced. The underlying driver is that the RL-based collector spends more time gathering high-quality data, and the sequence formulation for the entire feature space increases the learning time cost for the sequential model. But, during inference, MOAT outputs the entire feature transformation at once, whereas DIFER requires multiple reconstruction iterations based on the generated feature space's 's dimension. The less inference time makes MOAT more practical and suitable for real-world scenarios.

---

To thoroughly analyze the multiple characteristics of MOAT. we also analyzed the space com- plexity (See Appendix J. model scalability (See Appendix C.2), and parameter sensitivity (See Appendix C.3p. Meanwhile, we provided two qualitative analyses: learned embedding visualization (See Appendix C.4 and traceability case study (See Appendix C.5
5 Related Works
Automated Feature Transformation (AFT) can enhance the feature space by automatically trans- forming the original features via mathematical operations [19 20 Existing works can be divided into three categories: 1) expansion-reduction based approaches
22 Those methods first
expand the original feature space by explicitly [2 or greedily 24 decided mathematical transfor- mation, then reduce the space by selecting useful features. However, they are hard to produce or evaluate both complicated and effective mathematical compositions, leading to inferior performance. 2) evolution-evaluatin
approaches [4] [ 25 26). These methods integrate feature generation and selection into closed-loop learning system. They iteratively generate effective features and keep the significant ones until they achieve the maximum iteration number. The entire process is optimized by evolutionary algorithms or reinforcement learning models. However, they still focus on how to simulate the discrete decision-making process feature engineering- Thus, they are still time-consuming and unstable. 3) Auto ML-based approaches 7
g. Auto ML aims to find the most suitable model architecture automatically [27 28 30). The success of auto ML in many 33
[6] and the similarity between auto ML and AFT inspire researchers to formulate AFT as an auto ML task to resolve. However, they are limited by: 1) incapable of producing high-order feature transformation; and 2) unstable transformation performance. Recent studies tried to convert the feature transformation problem into a continuous space optimization task to search for the optimal feature space efficiently. DIFER [8] is cutting-edd -dgee method that is comparable to our work in problem formulation. However, the following constraints limit DIFER's practicality: 1) DIFER collects waanformation--accrsae
area [3 32
34
35
data at random, resulting in many invalid training data with inconsistent transformation performances; 2) DIFER embeds and reconstructs each transformed feature separately and, thus, ignores feature-feature interactions: 3) DIFER needs manually decide the number of generated features, making the reconstruction process more complicated. 4) the greedy search for transformation reconstruction in DIFER leads to suboptimal transformation results. To fill these gaps, we first implement an RL-based data collector to automate high-quality transformation record collection. We then leverage the postfix expression idea to represent the entire transformation operation sequence to model feature interactions and automatically identify the number of recon- structed features. Moreover, we employ beam search to advance the robustness, quality, and validity of transformation operation sequence reconstruction.
Conclusion Remarks
In this paper, we propose an automated feature transformation framework, namely MOAT. In detail, we first develop an RL-based data collector to gather high-quality transformation accuraac pairs. Then, we offer an efficient postfix-based sequence expression way to represent the transformation sequence in each pair. Moreover, we map them into continuous embedding space using an encoder- decoder-evaluator model structure. Finally, we employ gradient-ascent -ascent search to identify better embeddings and then use beam search to reconstruct the transformation sequence and identify the optimal one. Extensive experiments show that the continuous optimization setting can efficiently search for the optimal feature space. The RL-based data collector essential to keep an excellent and stable transformation performance. The postfix expression sequence enables MOAT to automatically determine the transformation depth and length, resulting in more flexible transformation ways. The beam search technique can increase the validity of feature transformation. The most noteworthy research finding is that the success of MOAT indicates that the knowledge of feature transformation can be embedded into continuous embedding space search for better feature space. Thus, it inspires us to regard the learning paradigm as the backbone to develop large feature transformation model and quickly fine-tune it for different sub-tasks, which is also our future research direction.
Acknowledgments
This research was partially supported by the National Science Foundation (NSF) via grant numbers: 2040950, 2006889, and 2045567.
10

---

References
[1] James Max Kanter and Kalyan Veeramachaneni. Deep feature synthesis: Towards automating data science endeavors. In 2015 IEEE international conference data science and advanced analytics (DSAA), pages 1-10. IEEE, 2015.
[2] Franziska Horn, Robert Pack, and Michael Rieger. The autofeat python library for automated feature engineering and selection. arXiv preprint 1oXiv:/901.07322. 2019.
[3] Udayan Khurana, Deepak Turaga, Horst Samulowitz, and Srinivasan Parthasrathy. Cognito: Automated feature engineering for supervised learning. 2016 IEEE 16th International Conference Data Mining Workshops (ICDMW), pages 1304-1307. IEEE, 2016.
[4] Dongjie Wang, Yanjie Fu, Kunpeng Liu, Xiaolin Li, and Yan Solihin. Group-wise reinforce- ment feature generation for optimal and explainable representation space reconstruction. In Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD '22, page 1826-1834, New York, NY, USA, 2022. Association for Computing Machinery.
[5] Udayan Khurana, Horst Samulowitz, and Deepak Turaga. Feature engineering for predictive modeling using reinforcement learning. In Proceedings of the AAAI Conference Artificial Intelligence, volume 32, 2018.
[6] Binh Tran, Bing Xue, and Mengjie Zhang. Genetic programming for feature construction and selection in classification on high nigh-dimensional data. Memetic Computing, 8(1):3-15, 2016.
[7] Xiangning Chen, Qingwei Lin, Chuan Luo, Xudong Li, Hongyu Zhang, Yong Xu, Yingnong Dang, Kaixin Sui, Xu Zhang, Bo Qiao, al. Neural feature search: neural architecture for automated feature engineering. In 2019 IEEE International Conference on Data Mining (ICDM), pages 71-80. IEEE, 2019.
[8] Guanghui Zhu, Zhuoer Xu, Chunfeng Yuan, and Yihua Huang. Difer: differentiable automated feature engineering. In International Conference on Automated Machine Learning, pages 17-1. PMLR, 2022.
[9] Meng Xiao, Dongjie Wang, Min Wu, Ziyue Qiao, Pengfei Wang, Kunpeng Liu, Yuanchun Zhou, and Yanjie Fu. Traceable automatic feature transformation via cascading actor-critic agents. Proceedings of the 2023 SIAM International Conference Data Mining (SDM), pages 775-783.
[10] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 98): 1997.
[11] Markus Freitag and Yaser Al-Onaizan. Beam search strategies for neural machine translation. arXiv preprint 1rXiv:1702.0160 2017.
[12] Thibault Viglino. Petr Motlicek, and Milos Cernak. End-to-end accented speech recognition. In Interspeech, pages 2140-2144, 2019.
[13] Chunhui Bao and Qianru Sun. Generating music with emotions. IEEE Transactions on Multimedia, 2022.
[14] Public. Uci dataset download. [EB/OL], 2022. https://archive.ccsuuk uci.
[15] Lin Chih-Jen. Libsvm dataset download. [EB/OL], 2022. http:://ww..ciie.tu.eeeuu
cjjiin//1bsswtoools/eaee s/datastss//////////
[16] Jeremy Howard. Kaggle dataset download. [EB/OL), 2022. datasets
https: bttpp://xww.BRe www.
[17] Public. Openml dataset download. [EB/OL], 2022.
https://wwk www.wlopenl...m .rg
[18] David Blei, Andrew Y Ng, and Michael Jordan. Latent dirichlet allocation. the Journal of machine Learning research, 3:993-1001 1022, 2003.

---

[19] Yi-Wei Chen, Qingquan Song, and Xia Hu. Techniques for automated machine learning, ACM SIGKDD Explorations Newsletter, 22(2):35-50, 2021.
[20] Andrew Kusiak. Feature transformation methods in data mining. IEEE Transactions on Electronics packaging manufacturing, 24(3):214 221, 2001.
[21] Hoang Thanh Lam, Johann-Michael Thiebaut, Mathieu Sinn, Bei Chen, Tiep Mai, and Oznur Alkan, One button machine for automating feature engineering relational databases. arXiv preprint 171v:1700.00327, 2017.
[22] Udayan Khurana, Fatemeh Nargesian, Horst Samulowitz, Elias Khalil, and Deepak Turaga. Automating feature engineering. Transformation, 10(10):10, 2016.
[23] Gilad Katz, Eui Chul Richard Shin, and Dawn Song. Explorekit: Automatic feature generation and selection. In 2016 IEEE 16th International Conference on Data Mining (ICDM), pages 979-984, IEEE, 2016.
[24] Ofer Dor and Yoram Reich. Strengthening learning algorithms by feature discovery. Information Sciences, 189: 189:176-190, 2012.
[25] Guanghui Zhu, Shen Jiang, Xu Guo, Chunfeng Yuan, and Yihua Huang. Evolutionary automated feature engineering. In PRICAI 2022: Trends in Artificial Intelligence: 19th Pacific Rim International Conference on Artificial Intelligence, PRICAI 2022, Shanghai, China, November 10-13, 2022, Proceedings, Part I. pages 574-586. Springer, 2022.
[26] Meng Xiao, Dongjie Wang, Min Wu, Kunpeng Liu, Hui Xiong, Yuanchun Zhou, and Yanjie Fu. Traceable group-wise self-optimizing feature transformation learning: A dual optimization perspective. arXiv preprint 1.XXIv:330616892 2023.
[27] Thomas Elsken, Jan Hendrik Metzen, and Frank Hutter. Neural architecture search: A survey. The Journal of Muchine Learning Research, 2091)71997- 2019.
[28] Yaliang Li, Zhen Wang, Yuexiang Xie, Bolin Ding, Kai Zeng. and Ce Zhang. Automl: From methodology to application. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pages 4853 4856, 2021.
[29] Xin He, Kaiyong Zhao, and Xiaowen Chu. Automl: survey of the Based Systems, 212:106622 2021.
Knowledge-
[30] Shubhra Kanti Karmaker, Md Mahadi Hassan, Micah J Smith, Lei Xu, Chengxiang Zhai, and Kalyan Veeramachaneni. Automl date and beyond: Challenges and opportunities. ACM Computing Surveys (CSUR), 1438) 2021.
[31] Ziwei Zhang, Xin Wang, and Wenwu Zhu, Automated machine learning on graphs: A survey. arXiv preprint 20X1v:2003.00742, 2021.
[32] Marcel Wever, Alexander Tomndee, Felix Mohr, and Eyke Hüllermeier. Automl for multi-label classification: Overview and empirical evaluation. IEEE transactions on pattern analysis and machine intelligence, 43(0)300773354, 2021.
[33] Maroua Bahri, Flavia Salutari, Andrian Putina, and Mauro Sozio. Automl: state of the art with a focus on anomaly detection, challenges, and research directions. International Journal of Data Science and Analytics, 14(2):113 100-116.12,,,. 126, 2022.
[34] Dakuo Wang, Josh Andres, Justin D Weisz, Erick Oduor, and Casey Dugan. Autods: Towards human-centered automation of data science. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, pages 1-12, 2021.
[35] Meng Xiao, Dongjic Wang, Min Wu, Pengfei Wang. Yuanchun Zhou, and Yanjie Fu. Beyond discrete selection: Continuous embedding space optimization for generative feature selection. arXiv preprint arXiv:2302.13221 2023.
[36] Wangyang Ying, Dongjie Wang, Kunpeng Liu, Leilei Sun, and Yanjie Fu. Self-optimizing feature generation via categorical hashing representation and hierarchical reinforcement crossing. arXiv preprint arXiv:2309 0461009994929.... 2023.
12

---

Appendix
A Postfix Transformation Conversion
Algorithm|] iis the pseudo-code for the convert the original feature transformation sequence I the postfix notation based sequence Y. In detail, we first initialize a list Y and two stacks S₁ and S2, respectively. For each element 7 in r, we push it to S2. When receiving a left parenthesis, we push it to St. When obtaining any operations, we pop each element in S1 and push them into S2 until the last component of S is the left bracket. Then we push this operation into S When getting right parenthesis, we pop every element from S1 and then push into 22 until we confront left bracket. Then we remove this left bracket from the top of Sj. When the end of the input encounters, we append every token from S2 into T. If this
scan it from left to right. When getting feature IDD token,
the last element r, we will append <SEP> token to indicate the end of this т. After we process every element in I,, we add <SOS> and <EOS> tokens to the beginning and end of Y to form the postfix notation-based transformation sequence = [",-.. TM Each element in Y is feature ID) token, operation token, or three other special tokens. We convert each transformation sequence through this algorithm and construct the training set with them.
Algorithm 1: Postfix transformation sequence conversion
input :Feature transformation sequence pulput:Postti notation based transformation sequence T
Y
0:
2 for < do
S1 f 0,0;
4
for 7 <
if is left bracket then
6
S1.push(());
else if is right bracket then
8
while ( < S1 ppppp) not left bracket do S2-push(1):
else if y is operation then
while Sy.peek() is not left bracket do S2.push((s.poo()):
S S1-push(7):
else
push((()))
while Snnot = @ do 16
Y Tappend(S222) (0)):
if is not the last element then
19
T ".appendd==EP>>::
20 Add <SOS> and <EOS> to the head and tail of T respectively;
21 return Y
B Experimental Settings and Reproducibility
B.1 Hyperparameter Settings and Reproducibility
The operation set consists of square root, square, cosine, sine, tangent, exp, cube, log, reciprocal, quantile transformer, min-max scale, sigmoid, plus, subtruct, multiply, divide. For the data collec- tion part, we ran the RL-based data collector for 512 epochs to collect large amount of feature cansformation-acccuaayy pairs. For the data augmentation part, we randomly shuffled each transfor- mation sequence 12 times to increase data diversity and volume. We adopted a single-layer LSTM
13

---

Table 4: Space complexity comparison on MOAT with different dataset
Dataset
Sample Number
Column Number
Parameter Size
Airfoil
1503
5
139969
Amazon employee
32769
9
138232
ap_ p__omenuumm__ oovary
275
10936
155795
german german_credit credit higgs
1001
24
141127
50000
28
141899
housing boston
506
13
139004
ionosphere
351
34
143057
lymphography
148
139969
messidor_ fcatures
1150
19
140162
openml_622
1000
25
141320
pima_indian
768
8
138039
spambase
4601
57
147496
spectf
267
44
144987
svmguide3
1243
21
140548
uci_ jii__cedit__ame _ard
30000
25
141127
wine red
999
12
138618
white
4900
138618
openml_558 586
1000
25
141320
openml _59
1000
25
141320
openml_600 607
1000
50
146145
openml 616
500
50
146145
openml 618
1000
146145
openml 637
500
50
146145
as the encoder and decoder backbones and utilized 3-layer feed-forward networks to implement the predictor. The hidden state sizes of the encoder, decoder and predictor are 64, 64, and 200, respectively. The embedding size of each feature token and operation token was set to 32. To train MOAT, we set the batch size 1024, the learning rate as 0.001, and À as 0.95 respectively. For inferring new transformation sequences, we used top-20 records as the seeds with beam size 5.
B.2 Experimental Platform Information
All experiments were conducted on the Ubuntu 18.04.6 LTS operating system, AMD EPYC 7742 CPU, and 8 NVIDIA A100 GPUs, with the framework of Python 3.9.10 and Py 1.8.1.
C Experimental Results
C.1 Space Complexity Analysis.
To analyze the space complexity of MOAT, we illustrate the parameter size of MOAT when confronted with different datasets. Table Asows the comparison results. We can find that the model size of MOAT keeps relatively stable without significant fluctuations. The underlying driver is that the
learning paradigm can embed the knowledge of discrete sequences with variant lengths into a fixed-length embedding vector. Thus, such an embedding process can make the parameter size to be stable instead of increasing the data size grows. Thus, this experiment indicates that MOAT has good scalability when confronted with different scaled datasets.
C.2 Scalability Check
We visualized the changing trend of the time cost of searching for better feature spaces over sample size and feature dimensions different datasets. Figure shows the comparison results. We found that the time cost of MOAT keeps stable with the increase of sample size of the feature set. A possible reason that MOAT only focuses on the decision-makee instead of the information of the entire feature set, making the searching process sample size irrelevant. Another interesting observation is that the search time still stable although the feature dimension of the feature set varies significantly. A possible explanation is that map transformation records of varying lengths into a continuous space with a constant length. The searching time this space input dimennsionllty-agnnssie Thus, this experiment shows the MOAT has excellent scalability.
benefits of feature ID and operation tokens
14

---

Search Time
Sample Size
Search Time
Feature Num.
8
40,000
Time Coet e
Time Cost (5) 8
Nuuber
Number
20,000
$
sonasple
tonasphere
Pimalndian
SpamBesa
Credit Delauit
Higss Boson
Pimalndian
Lymphograp
lonosphere
Specti
Openmi 816
SpamBase
(a) Sample Size
(b) Feature Number
Figure 6: Scalability check MOAT in search time based on sample size and feature number.
a=0.55
Syd
mm2
aró 25
0m0.10
D-0 30 0 0600000004
a=0.06 dos
do
7=7
1=10
am0.35
0z0
n=9
40
a50.45
(a) Search Step Size (under 0.05)
(b) Training Trade-off (under
Figure 7: Parameter sensitivity search step size 7) and trade-off parameter Spectf dataset.
-10
-20
-00
(a) Openml_616
(b) Airfoil
Figure The visualization of learned transformation sequence emhedding (from MOAT).
C.3 Parameter sensitivity analysis
To validate the parameter sensitivity of the search step size 1) (See section 3.5 and the trade-off parameter a in the training loss (See section we set the value of y) from 1 to I0, and set the value of a from 0.05 0.50 observe the difference. Figure shows the comparison results terms of precision, recall, and Fl-score. When the search step size grows, the downstream ML performance initially improves, then declines marginally. possible reason for this observation is that too-large step size may make the gradient-ascent -sccent search algorithm greatly vibrate in the continuous space, leading to missing the optimal embedding point and transformed feature space. Another interesting observation is that the standard deviation of the model performance is lower than 0.01 under different parameter settings. This observation indicates that MOAT is sensitive distinct parameter settings. Thus, the learning and searching process of MOAT is robust and stable.
C.4 Learned embedding Analysis.
We selected Airfoil and Openml_611 6 as examples to visualize their learned continuous embedding space. In detail, we first collected the latent embeddings generated by the transformation records. Then, we use T-SNE map them into a 2-dimensional space for visualization. Figure g shows
15

---

Original Feature Set
MOAT Generated Feature Set
picehed
satkr daddde)
Inokhaa (eager)
Mtad seiste (My))
eTans,,J (-) @itass-))
seitlps, Sosreetus
(Heneh)) Scare:t.00
Dresiuus sugar] (tenshy)
(Jtotttle)-- L-jet-)
[-1 clanssjasjjeeass 1-I1
[choriadd
(oudphale))
(1
(a) Original Feature Space
(b) MOAT Generated Feature Space
Figure 9: Comparison of traceability on original feature space and the geneagee one. the visualization results, in which each point represents unique feature transformation sequence. The size of each point means its downstream performance. The bigger point size indicates that the downstream performance is superior. We colored the top 20 embedding points according to the performance in red. We found that the distribution locations of the top 20 embedding points are different. A potential reason is that the corresponding transformation sequences of the top 20 embedding points are different lengths. The sequence reconstruction loss distributes them different areas of the embedding space. Moreover, we observed that the top 20 embedding points are close in the space even though the positions are different. The underlying driver is that the estimation loss makes these points with good performance clustered. Thus, this case study reflects that the reconstruction loss and estimation loss make the continuous space associate the transformation sequence and the corresponding model performance.
C.5 Traceability case study
We selected the top 10 essential features for prediction in the original, and MOAT transformed feature space of the Wine Quality Red dataset for comparison. Figure the comparison results. The texts associated with each pie chart are the corresponding feature name. The larger the pie area is, the more critical the feature is. We found that almost 70% critical features in the new feature space are generated by MOAT and they improve the downstream ML performance by 22.6%. This observation indicates that MOAT really comprehends the properties of the feature set and ML models in order to produce a more effective feature space. Another interesting finding is that '[alcohol]' is the essential feature the original feature set. But MOAT generates more mathematically composited features using '[alcoho]''. This observation reflects that MOAT not only can capture the significant features also produce more effective knowledge for enhancing the model performance. Such composited features can make domain experts trace their ancestor resources and summarize new analysis rules for evaluating the quality of red wine.
16


Reviews:
[{"review_id": "zDHN8bYJwd", "reviewer": "Reviewer_MJDJ", "summary": "Distinct from existing work, this study collects feature transformation operation sequences, which have a well-researched background based on reinforcement learning. It then obtains hidden representations of these limited numbers of sequences in a self-supervised manner and finally optimizes this continuous vector representation to generate superior sequences guided by gradients. Specifically, to model the sequences, the authors propose an LSTM-based encoder-decoder-evaluator architecture with just a few collected samples. Detailed runtime analysis was conducted in this work to demonstrate its advantages over reinforcement learning-based (GRFG) and random generation-based (DIFER) approaches. The experiments, coupled with the authors' analytical discussions, effectively substantiate the viewpoints declared in the paper.", "questions": "1) Why does DIFER require more time in the inference phase? The authors should provide a detailed explanation for this.\n\n2) In the time complexity analysis, the authors have chosen four datasets - Wine Red, Wine White, Openml_618, and OpenML_589. However, they have not adequately described the rationale behind selecting these particular datasets.\n\n3) There are some avoidable writing errors, such as on page 13 of the appendix where the authors incorrectly referred to Table 3 as Figure 3. The authors should conduct a thorough check of the entire paper to prevent inconsistencies in naming.", "limitations": "None", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1) This paper is well-written and easy to understand, with an appropriate level of detail in describing the methods. It considers the primary issue, time consumption, associated with this framework in terms of experimental design, effectively addressing the readers' concerns.\n\n2) The paper's model architecture is thoughtfully designed, and the authors articulate their motivations for each component with great clarity. The streamlined nature of the entire framework demonstrates the depth of the authors' understanding and thoughtful consideration of this work.\n\n3) The authors provide thorough experiments and analysis for this work, covering aspects of interest in this field such as performance, robustness, runtime, scalability, memory usage, traceability, and efficacy tests for each component. The visualization of the hidden space well illustrates the reasons for the model's effectiveness in conducting gradient search.", "weaknesses": "1) In the sections 2.1 Important Definitions and Section 3.2 Reinforcement Training Data Preparation, the authors have inconsistencies in the naming of the agents within the cascading agent structure for the data collection part. For instance, it's named as 'feature agent1' in the definition section, while it's referred to as 'head feature agent' in the methodology part. The authors should maintain consistency in these definition names to reduce confusion for the readers.\n\n2) The description of the optimization objective for reinforcement learning in the data collection section is not detailed enough. The authors should provide a more comprehensive description of this critical component.\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer MJDJ,\n\nThank you for acknowledging that our paper has a solid research background, well-written, easy-to-understand, and extensive experiments to support the claimed research insights and technical contributions. For your mentioned issues:\n\n**1. Regarding the typos, name inconsistency, and figure caption errors:**\n\nResponse: We will revise the correlated description in Section 2.2 Problem Statement to avoid the inconsistency of the name definitions. We will thoroughly check before the final version and prevent hidden typos and errors. \n\n**2. Regarding the details of the optimization objective for the reinforcement learning component:**\n\nResponse: We tried to present a balanced description of the model architecture and provide an as light as possible data collection component description, but based on your suggestion, we realized that we had omitted too much in the section describing the optimization objectives, which might lead to misunderstandings. In the final version, we will detail the overall optimization objectives of reinforcement learning.\n\n**3. Regarding the time complexity analysis of DIFER:**\n\nResponse: DIFER implements a completely different sequence generation strategy. After the model converges, DIFER will generate each column in a fixed number of steps and then use a filter to remove invalid sequences generated. In contrast, MOAT will directly generate the entire sequence, where the model determines the lengths of different columns, and the model will try to generate valid sequences through Beam Search. Such differences lead to MOAT having better Inference Time than DIFER when generating datasets with the same number of columns.\n\n**4. Regarding the time complexity analysis of MOAT:**\n\nResponse: We neglected to explain why the datasets were selected in the main text. According to Table 1, Wine Red and Wine White have the same number of features but different numbers of samples. Comparing the time efficiency of MOAT on these two datasets can reflect the model's scalability on datasets with different instances. Conversely, Openml618 and Openml589 have the same number of instances but different numbers of features, and comparing the time efficiency of these datasets can reflect the model's scalability on datasets with varying feature counts. These two experiments combined can effectively validate the model's scalability. We realize that the lack of this part of the description may affect the reader's understanding, and we will detail the reasons for these dataset selections in the final version."}, {"review_id": "hoLQ5WTsQG", "reviewer": "Reviewer_miDK", "summary": "The authors propose a feature transformation method that reformulates the problem as a continuous space optimization task and utilizes a reinforcement-enhanced autoregressive framework for gradient-steered search. The method involves four steps: (1) reinforcement-enhanced data preparation, (2) feature transformation operation sequence embedding, (3) gradient-steered optimal embedding search, and (4) transformation operation sequence reconstruction. The proposed method is evaluated through extensive experiments and case studies, demonstrating its effectiveness and robustness.", "questions": "1. It is unclear why the performance of DIFER is not good for openml 616, openml 637. Can you provide some explanation on this?\n\n2. After reviewing the appendix section, I found that MOAT requires more training time but less inference time than DIFER. Can the authors explain the reason for it?", "limitations": "None", "rating": 7, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 4, "strengths": "1. This paper proposes a novel automated feature transformation framework that has not been explored before, and it achieves significant improvement in performance compared to previous methods.\n\n2. This paper provides clear and detailed explanations of the proposed method's four-step process, making it easy for readers to follow and replicate the method.\n\n3. This paper conducts extensive experiments and case studies to illustrate the effectiveness of the framework from different perspectives.\n\n4. The authors released the related code and data, which can help other researchers reproduce the experiments.\n\n5. This paper provides clear and detailed explanations of the proposed method's four-step process, making it easy for readers to follow and replicate the method.", "weaknesses": "1. The authors use LSTM as the backbone of their framework. Any reason for choosing LSTM? How about other alternatives, such as Transformer?\n\n2. There are some typos in this paper. For instance, in Figure 4 (c) and Figure 4 (d), the name of the model variant should be MOAT^-a instead of GBFG^-a. The authors should fix these typos for keeping consistency.\n\n3. Analyzing the experimental results, the RL-based data collector is important. But, the description for this part is limited. Can the authors provide more explanation on this?\n", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer miDK,\n\nWe want to thank you for acknowledging this paper's novelty, effectiveness, extensive experiments, and reproducibility. \n\n**1. Regarding the inferior performance of DIFER on the openML dataset:**\n\nResponse: OpenML616 and OpenML637 are two artificially constructed datasets, some of which contain added noise columns. In practice, MOAT will obtain high-quality samples through a reinforcement learning-based feature transformation sequence generator and use these samples as seeds to produce better feature transformation sequences. As the reinforcement learning agent converges, these artificially added noises can be easily removed. In contrast, DIFER selects better features from a set of generated random sequences for its search. This could potentially retain noise sequences, affecting downstream tasks' performance.\n\n**2. Regarding the difference between training and inference time for MOAT and DIFER**\n\nResponse: This observation is due to the two frameworks' strategies to search for optimal feature transformation sequences. In DIFER, it models and searches \\textbf{each column} as an independent transformation sequence. This means the sequence length it needs to model is much shorter than MOAT. Still, its search requires generation for each column according to a given sequence length hyperparameter, resulting in more search operations. MOAT models and searches the transformation operation sequence for the entire dataset, where the model determines the lengths of different columns, so the time consumption for sequence modeling is higher. Still, the time consumption for the generation process is less.\n\n\t\n**3. Regarding the description of RL-based data collector:**\n\nResponse: We realized that we had omitted too much in the section describing the optimization objectives of RL-based data collectors, which might lead to misunderstandings. In the final version, we will detail the overall optimization objectives of reinforcement learning.\n\n**4. Some minor revisions:**\n\nResponse: In the future version, we will address the typo and other minor errors in Figure 4.\n"}, {"review_id": "6oyda0nW1f", "reviewer": "Reviewer_yQs6", "summary": "This paper introduces a succinct yet effective framework for automatic feature transformation. The authors mapped decision sequences collected from reinforcement learning into a high-dimensional vector space. Optimization is carried out through the gradient direction provided by an evaluator, and a sequence reconstruction component is employed to rebuild the decision sequence of this feature transformation. Overall, the paper's experiments are comprehensive, and the discussions provide excellent validation of various features of the framework. The comparisons with existing models also aptly place this work among the latest contributions in the field.", "questions": "1. The framework's description of the data collection part for the optimization objective lacks clarity. The authors should consider adding more detail in the appendix to better elucidate this aspect.\n\n2. Are there any way to make the framework more generalized, e.g., extend this method to graph-like data, picuture-like data.\n\n3. Would it be feasible to design a more generalized operation set that views these mathematical operations as analogous to sub-networks?", "limitations": "None", "rating": 7, "confidence": 5, "soundness": 4, "presentation": 3, "contribution": 4, "strengths": "1. The design of the entire framework is clear and comprehensible, and based on the authors' descriptions and experimental discussions, this framework also shows potential for reuse in other fields.\n\n2. The experimental design is detailed and particularly addresses my concerns about the time-consuming nature of using reinforcement learning for data collection.\n\n3. Similar to the second strength described above, a significant issue with using reinforcement learning in feature transformation tasks is that it might require an excessive number of search steps to achieve optimal. However, this framework can use a fixed number of search steps to construct the space for searching and achieve respectable downstream task performance, significantly mitigating the problem of uncertain duration in feature transformation tasks based on reinforcement learning.", "weaknesses": "1. There are a few typos in this paper, e.g., the legend of the figure 4, the model name is wrong.\n\n2. The authors have proposed an encoder-decoder model based on LSTM for sequence modeling. However, there is no discussion on why this specific method was chosen. It would be beneficial if the authors could explain the rationale behind this choice. Could other sequence modeling methods, such as Transformers, have been considered or used instead?\n\n3. While the design of the model and ample experiments mutually validate the feasibility of this framework, if there could be more in-depth insights provided, it might help establish this work as foundational for broader fields.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer yQs6, \n\nWe sincerely appreciate the reviewer's time and effort in assessing our paper. We are pleased to find that the design and potential of the framework were recognized as strengths. We want to address the highlighted weaknesses and answer the posed questions to improve our paper's quality and clarity.\n\n**1. Regarding Typos and Minor Revision:** \n\nResponse: We apologize for the oversight and will promptly correct the typos and the error in the legend of Figure 4 in our next revision.\n\n**2. Regarding the Choice of LSTM:** \n\nResponse: Our framework is a generic feature learning framework, and the sequential model's purpose is to preserve feature learning knowledge in a continuous space, providing a solid foundation for subsequent steps. LSTM, as a vanilla model structure, effectively demonstrates the framework's generalization ability. However, if desired, our framework can also accommodate other sequential models, e.g., Transformers.\n\n**3. Regarding the Optimization Objective of the Data Collection Component:** \n\nResponse: We realize that the data collection part may require more clarity. We will provide a more detailed description in the main paper and, if space-constrained, in the appendix.\n\n**4. Regarding the Framework Generalization on graph-like, picture-like data:** \n\nResponse: It's an interesting point. Currently, our method is designed with specific data in mind, i.e., tabular data. However, extending it to handle graph-like or picture-like data is possible, and we are in the initial stages of exploring these extensions.\n\n**5. Regarding a Generalized Operation Set:**\n\nResponse: Designing a set of operations that treat mathematical operations as analogous to sub-networks is a compelling idea. While our current framework has not incorporated this, we see its merit and will consider it for future work or iterations of the framework.\n\nWe once again thank the reviewer for their constructive comments and will ensure to make the necessary revisions to enhance the paper's quality and impact."}, {"review_id": "W7YpiEFPPV", "reviewer": "Reviewer_GfrD", "summary": "Feature transformation is an effective way to improve downstream task. This paper aims to improve the search efficiency of the optimal feature space while ensuring the stability and robustness of the transformation. They formulate the discrete Automated Feature Transformation (AFT) problem as a continuous optimization task and propose a reinforcement-enhanced autoregressive feature Transformation framework (MOAT). MOAT implements four steps to advance efficiency and robustness. Extensive experiments and case studies are performed to demonstrate the effectiveness and robustness of the proposed method.", "questions": "(1) MOAT uses multi-stage process to obtain a convincing performance. Is it possible to learn an optimal feature space using reinforcement learning by end-to-end?\n(2) Does the optimal feature space searching problem can be directly solved by representation learning in continuous space, rather than combining multiple operations in discrete space？", "limitations": "This paper pays more attention to framework design and lacks some theoretical perspectives. In addition, how to design the framework proposed in this paper into an end-to-end training process is desirable.", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "This paper provides a clear and detailed illustration of the proposed framework, especially the modules in the framework. They conduct extensive experiments to demonstrate the superiority of the framework, including ablation studies for each module, to demonstrate the efficiency and robustness of the framework.", "weaknesses": "Missing reference to experimental results on page 8, Data collection and augmentation section.\nSeveral methods have used reinforcement learning to improve search efficiency at present (e.g., in data augmentation, NAS), and the framework proposed in this paper lacks novelty to some extent.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "Dear Reviewer GfrD\n\nThank you for your detailed feedback and observations on our work. We highly appreciate the recognition of the strengths of our paper, and we acknowledge the points raised in the weaknesses and questions sections. We will address each of them comprehensively:\n\n**1. Regarding the End-to-end Model Architecture:**\n\nResponse: The end-to-end reinforcement learning process for optimal feature space learning has been explored in works such as GRFG, NFS, and other reinforced methodologies. However, our extensive research and experimentation showed that while such an approach promises to streamline, it poses notable challenges, particularly regarding training stability and convergence. To illustrate, consider GRFG, which, despite its architecture mirroring the RL-based data collector, demands over 3000 epochs to converge, often settling for sub-optimal results. In stark contrast, our MOAT framework swiftly learns a continuous search space from merely RL-based data collectors' initial 512 search records. This generates a more stable, effective, and efficient feature transformation sequence.\n\n\n**2. Regarding the Merit of Representation Learning:**\n\nResponse: Representation learning is a cornerstone in modern machine learning, offering a transformative approach to feature extraction and data interpretation. At its core, representation learning provides an avenue to automatically discover the representations needed for data analysis tasks, bypassing manual feature engineering, which has historically been time-consuming and domain-specific. \nIn our study, we search for another way to extract the feature information from the dataset, which is the core of Data-centric AI, bringing automation into feature engineering. \nOur approach could easily apply to some small datasets, and this small amount of data might be insufficient to train a deep neural network for representation learning. In contrast, our approach can generate meaningful features by conducting mathematical feature-feature crossing. The fully traceable generated feature also can be summarized as domain knowledge by its operated mathematical transformation.\n\n**3. Regarding Novelty or Contribution**\n\nResponse: Our computing insights and technical contributions go beyond the combination of GRFG and DIFER. The following implications and findings are beneficial for future research:\n\nWe highlight a postfix expression embedding and generation perspective for feature transformation.\n\nWe demonstrate that reinforcement intelligence can be used as a self-optimizing training data collector to explore high-quality feature transformations, evaluate downstream ML task accuracy, and automate training data collection.\n\nWe find that integrating transformation sequence reconstruction loss and downstream task accuracy estimation loss can better measure the effectiveness of an embedding space and strengthen the denoising capability of gradient-based search of the optimal transformation embedding.\n\nWe encode a feature transformation operation sequence as a single postfix expression so that the generative model can capture feature-feature interactions and automatically identify the dimensionality of the new feature space and the complexity of the transformation operation sequence.\n\nOur framework automatically determines the dimension of the generated feature space by examining the special token <EOS>, contrasting with DIFER's manual setting requirement.\n\nThese insights are proven to improve the reliability and performance of automated feature engineering tasks.\n\n**4. Regarding the Missing Reference on Page 8:**\n\nResponse: We apologize for this oversight. We will revise this issue in the final version. "}]

Author rebuttal:

Meta-review: This paper studies the problem of feature transformation, which is very important for downstream tasks. This paper formulates the discrete automated feature transformation as a continuous optimization problem and proposes a novel transformation framework. Extensive experiments are provided, which can show the efficiency, and robustness of this framework. Moreover, the code and datasets are provided.
Decision: Accept (spotlight)