Build Roadmap for Automated Feature Transformation: A Graph-based Reinforcement Learning Approach

Authors: Xiaohan Huang, Dongjie Wang, Zhiyuan Ning, Ziyue Qiao, QingqingLong, Haowei Zhu, Min Wu, Yuanchun Zhou, Meng Xiao
Primary area: other topics in machine learning (i.e., none of the above)
Keywords: Automated Feature Transformation, Tabular Data, Multi-Agent Reinforcement Learning

Abstract:
Feature transformation tasks aim to generate high-value features by combining existing ones through mathematical operations, which can improve the performance of downstream machine learning models.
Current methods typically use iterative sequence generation, where exploration is guided by performance feedback from downstream tasks.
However, these approaches fail to effectively utilize historical decision-making experiences and overlook potential relationships between generated features, thus limiting the flexibility of the exploration process.
Additionally, the decision-making process lacks the ability to dynamically backtrack on efficient decisions, which hinders adaptability and reduces overall robustness and stability.
To address these issues, we propose a novel framework that uses a graph to track the feature transformation process, where each node represents a transformation state.
In this framework, three cascading agents sequentially select nodes and mathematical operations to generate new nodes.
This strategy benefits from the graph structure’s ability to store and reuse valuable transformations, and it incorporates backtracking via graph pruning techniques, allowing the framework to correct inefficient paths.
To demonstrate the effectiveness and flexibility of our approach, we conducted extensive experiments and detailed case studies, demonstrating superior performance across a variety of datasets.
This strategy leverages the graph structure's inherent properties, allowing for the preservation and reuse of sight-seen and valuable transformations. 
It also enables back-tracking capabilities through graph pruning techniques, which can rectify inefficient transformation paths.
To validate the efficacy and flexibility of our approach, we conducted comprehensive experiments and detailed case studies, demonstrating superior performance in diverse datasets.

Decision: reject
Reviewer scores: 5, 6, 6

Reviews and discussion:
<thread>
creation date:
1729803105263

thread_type:
official_review

<thread_object>
creator: reviewer_tgfr

<content>
summary:
The paper deals with the automated generation of features. The generation process consists of several steps, which are represented as a graph. The graphs are to be optimized using multi-agent reinforcement learning.

strengths:
* It can be seen (among other things from the large number of specific illustrations) that a lot of effort was put into preparing the paper

weaknesses:
* I find the text very badly written. Examples follow. The novelty and benefits of the method are hard for me to understand.
* It seems to me that there is too much material for a conference paper, the number of pages is simply not enough to present it in a convincing way.

Details, examples and further comments:
* I don't think “roadmap” is a suitable term, “schedule” or "sequence" would probably be better.
* The title sounds strange. Wouldn't "Optimization of transformation sequences for automated feature generation“ be better?
* The abstract uses terms that are incomprehensible:\
mathematical feature-feature crossing\
the roadmap of feature transformation
* „Feature transformation task aims to generate high-value features and improve the performance of downstream machine learning tasks using the mathematical feature-feature crossing” needs to be reformulated.
* "Classic machine learning is highly dependent on the structure of the model, the activation function" cannot be said in this way, it seems to refer exclusively to neural networks and not to classical machine learning in general.
* A reference should be given for "a cascading multi-agent reinforcement learning (MARL) algorithm", because it is not generally known what “cascading multi-agent reinforcement learning” is.
* “we present an innovative framework” -> “we present a novel framework”
* In the loss function, Equation 8, the square is probably missing. 
* "In this study, we introduce TCTO, an automated feature transformation framework. Our method emphasizes a transformation-centric approach, in which a transformation roadmap is utilized to systematically track and manage feature modifications." should be reworded. What is the information content? What should be expressed?
* I think that the Abstract and Conclusion need to be completely rewritten.

questions:
* How were the small uncertainties in Table 1 achieved? How often were the experiments repeated?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730640004511

thread_type:
official_review

<thread_object>
creator: reviewer_nncl

<content>
summary:
In this paper, the authors present TCTO, a graph-based reinforcement learning framework designed for automated feature transformation. The approach addresses limitations in current methods, such as the lack of historical insight utilization and insufficient flexibility in transformation exploration. By constructing a transformation roadmap with nodes representing feature states, TCTO leverages a cascading multi-agent system to dynamically select transformations, reuse effective paths, and prune inefficient ones. The experimental results demonstrate that TCTO outperforms existing methods in generating high-quality features, suggesting its potential to enhance feature engineering in machine learning tasks.

strengths:
This paper has several notable strengths. Firstly, the authors present a well-motivated framework that addresses clear gaps in current automated feature transformation methods, such as the need for effective historical data utilization and robust backtracking. The proposed TCTO framework is innovative in its use of a graph-based roadmap and cascading multi-agent reinforcement learning, which enhance the flexibility and adaptability of the transformation process. Additionally, the authors provide a comprehensive experimental evaluation across diverse datasets, which convincingly demonstrates TCTO’s superior performance compared to traditional methods. This solid empirical foundation supports the framework's potential for broad applicability in feature engineering for machine learning tasks.

weaknesses:
While this paper offers a promising framework, it has some weaknesses. Firstly, the explanation of the cascading multi-agent system and its decision-making processes could benefit from more clarity and detail, as the current description may be challenging for readers to fully grasp without additional context. Additionally, the computational complexity of TCTO is not thoroughly analyzed, especially regarding scalability to larger datasets, which may impact its practical applicability. Finally, while the experimental results are extensive, the paper could further strengthen its claims by providing more insight into specific scenarios or datasets where TCTO may struggle, thereby clarifying the framework’s limitations and potential areas for improvement.

questions:
1.How does the computational complexity of TCTO scale with larger datasets, and are there any strategies to mitigate potential performance bottlenecks?
2.Are there scenarios or specific types of datasets where TCTO’s performance may be limited, and if so, what adjustments might be necessary to enhance its adaptability?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730775529166

thread_type:
official_review

<thread_object>
creator: reviewer_ce2x

<content>
summary:
This paper introduces an automated feature transformation framework designed to enhance downstream machine learning model performance. The TCTO framework leverages a reinforcement learning-based graph structure to maintain a roadmap of feature transformations, enabling efficient exploration and backtracking of transformation pathways. TCTO uses a multi-agent reinforcement learning approach, clustering and encoding transformation states to strategically apply feature transformations. Experiments on multiple datasets demonstrate TCTO's performance over existing methods by improving robustness and flexibility in feature generation.

strengths:
1. While mostly clear, certain sections (e.g., cascading agent decision process) could benefit from additional details.

2. The framework is well-supported by experimental evidence showing its adaptability across different datasets and improvement in downstream model performance.

3. TCTO introduces a novel approach to automated feature engineering by employing a transformation-centric methodology with a graph-based roadmap, overcoming limitations of existing feature transformation methods.

4. The approach’s ability to backtrack and optimize feature transformations dynamically makes it highly applicable in real-world ML tasks where feature diversity and stability are crucial.

weaknesses:
1. While effective on a range of datasets, it is unclear how well TCTO scales with extremely high-dimensional data or very large datasets, as the pruning strategy may require fine-tuning in these cases.

2. The cascading decision-making process is intricate, and further simplification or additional visuals might aid understanding.

3. The reward structure combines performance and complexity, but further discussion on how these metrics are weighted could improve transparency and replicability of the model’s efficacy.

questions:
1. Could the authors elaborate on how they determined the weights for performance and complexity in the reward function? More detail on this could clarify the balance between the two objectives.

2。 How does TCTO perform on high-dimensional datasets with over 10,000 features? Is the pruning strategy sufficient to maintain stability without compromising feature diversity?

3. Were there any specific scenarios where TCTO’s backtracking mechanism was particularly beneficial in terms of model performance or feature diversity?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732497741729

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Modification of Submission

comment:
Dear Reviewers,\
Thank you for your valuable time and effort! Your insightful feedback has significantly helped us improve the paper. We have revised the manuscript based on your comments and updated PDF file. The revised sections are highlighted in red. The main revisions are as follows:

1. **Presentation Clarity:** We have clarified the decision-making process of the cascading agents, including additional details on the workflow (Lines 195-200 and Lines 310-316). Additionally, we have rewritten the Abstract and Conclusion to enhance clarity and help readers better understand our contributions.

2. **Scalability on Large-Scale Datasets:** We have analyzed the scalability of TCTO on large-scale datasets, including those with large sample sizes and high-dimensional features. This new analysis is included as an experiment in Appendix A.3.6. Based on the results, we discuss the limitations of TCTO and suggest future work to improve the approach (Lines 1138-1147).

3. **Additional Experimental Details:** We have provided an explanation of how we set the reward function weights in Appendix A.3.5. Furthermore, we have clarified how we calculate the standard deviation, which is included in the table note for Table 1.

4. **Additional Baselines:** We have added two recent baselines in Table 1: FETCH [1] and OpenFE [2], both of which focus on automated feature transformation. 

[1] Li L, Wang H, Zha L, et al. Learning a data-driven policy network for pre-training automated feature engineering[C]//The Eleventh International Conference on Learning Representations. 2023.\
[2] Zhang T, Zhang Z A, Fan Z, et al. OpenFE: automated feature generation with expert-level performance[C]//International Conference on Machine Learning. PMLR, 2023: 41880-41901.

5. **Minor Revisions:** We have made minor revisions to the presentation (Lines 33-36), Figure 1, and Formula 8 based on your comments. Additionally, we have moved the dataset source to the Appendix.

We sincerely hope that our revisions address the concerns raised. If you have any further questions or concerns, please do not hesitate to discuss with us. We greatly look forward to receiving your feedback!

Best regards,\
The Authors
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper proposes a methodology for feature transformations based on a multi agent reinforcement learning-based graph structure to maintain a roadmap of feature transformations, enabling efficient exploration and backtracking of transformation pathways. The method is evaluated on various ML datasets, showing benefits over other feature transformation methods. 

Strengths: 
The paper is interesting and solves a unique problem
The methodology is quite complex and it is impressive that the authors got it to perform well

Weaknesses
The paper as written is extremely hard to parse, there is *way* too much happening, there are far too many components to the system. The writing is quite hard to parse and the methodology as current constructed seems very specific and hard for others to use. For this work to meaningfully be used by others in the community, it has to be simplified and exposed in a much more clear way.