Large Language Models Engineer Too Many Simple Features for Tabular Data

Authors: Jaris Küken, Lennart Purucker, Frank Hutter
Primary area: foundation or frontier models, including LLMs
Keywords: LLMs, feature engineering, bias, tabular data, automated data science

Abstract:
Tabular machine learning problems often require time-consuming and labor-intensive feature engineering.
Recent efforts have focused on using large language models (LLMs) to capitalize on their potential domain knowledge. 
At the same time, researchers have observed ethically concerning negative biases in other LLM-related use cases, such as text generation. These developments motivated us to investigate whether LLMs exhibit a bias that negatively impacts the performance of feature engineering. While not ethically concerning, such a bias could hinder practitioners from fully utilizing LLMs for automated data science. 
Therefore, we propose a method to detect potential biases by detecting anomalies in the frequency of operators (e.g., adding two features) suggested by LLMs when engineering new features. Our experiments evaluate the bias of four LLMs, two big frontier and two small open-source models, across 27 tabular datasets. Our results indicate that LLMs are biased toward simple operators, such as addition, and can fail to utilize more complex operators, such as grouping followed by aggregations. Furthermore, the bias can negatively impact the predictive performance when using LLM-generated features. Our results call for mitigating bias when using LLMs for feature engineering.

Decision: reject
Reviewer scores: 3, 3, 5, 3

Reviews and discussion:
<thread>
creation date:
1730569253945

thread_type:
official_review

<thread_object>
creator: reviewer_31ol

<content>
summary:
This paper investigates Large Language Models' (LLMs) capabilities in feature engineering for tabular data. The study examines four LLMs across 27 tabular classification datasets that were specifically selected to minimize potential memorization effects. For each dataset, the LLMs were tasked with recursively generating 20 new features, using prompts that contained task context, descriptions of original features, and available operators. The study benchmarks these results against OpenFE, an open-source feature engineering tool, using identical operators and original features. To evaluate the effectiveness of the engineered features, a LightGBM model was trained and tested on datasets combining original and constructed features, with classification accuracy as the performance metric. The results demonstrate that OpenFE produces more consistently beneficial feature sets for classification tasks. Through analyzing operator frequency in feature construction across both LLMs and OpenFE, the authors conclude that LLMs exhibit a bias toward simpler operators when no explicit operator preferences are specified in the prompts.

strengths:
The paper presents a novel investigation into LLMs' feature engineering capabilities. The authors introduce an innovative evaluation metric—operator frequency distribution—which effectively quantifies the patterns in operator selection during feature construction. This metric provides valuable insights into how feature engineering tools, particularly LLMs, exhibit preferences for certain operators under different task contexts and prompt conditions. Furthermore, the study's comprehensive evaluation across 27 tabular datasets, with careful consideration for LLM memorization effects, demonstrates robust experimental design and systematic methodology.

weaknesses:
The paper's analysis lacks sufficient depth in several crucial areas. While the proposed operator frequency metric is interesting, it requires further validation in terms of:

Effectiveness: There is no analysis comparing the variability and information content of features generated by simple versus complex operators.
Fairness: The operator-level analysis overlooks that identical operators applied to different features can yield vastly different outcomes, making tool comparisons based solely on operator frequency potentially misleading.
Implications: The study lacks experimental evidence linking complex operator usage to improved classification performance.

The paper's conclusion about LLMs' preference for basic operators requires additional validation. The authors did not explore prompting strategies to encourage complex operator usage, nor did they analyze the specific features and operators suggested by LLMs.
The narrative structure could be improved. For instance, the abstract's discussion of LLM bias in text generation appears tangential to the core focus on feature engineering. Similarly, the section on 'Other Applications of Large Language Models for Tabular Data' would be better integrated into the literature review rather than appearing as a standalone paragraph.

questions:
1, Could you clarify the source of the original features—were they extracted or provided with the datasets?
2, Have you considered experimenting with prompts that encourage the use of complex features, perhaps by emphasizing intricate relationships between original features?
3, What methods were used to validate the effectiveness, fairness, and implications of the operator frequency metric?
4, How did you account for the stochastic nature of LLM responses, where identical prompts might yield different operators and features?
5, Would it also be informative to evaluate model performance using only the generated features, excluding original features? Maybe you can try this.
6, Have you conducted feature-level analysis of the constructed features? Specifically:
Classification Performance Level: Identifying dominant features in both LLM and OpenFE-generated sets
Feature Level: Analyzing the characteristics of successful versus unsuccessful generated features
Combining classification-level, feature-level, and operator-level analyses to strengthen conclusions about LLMs' feature engineering capabilities.
7, A potential typo in Hypothesis 1: “HYPOTHESIS 1: FEATURE ENGINEERING WITH LARGE LANGUAGE MODELS IS BIASED TOWARD SIMPLE OPERATES.” The last word should be “OPERATORS”?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730583399421

thread_type:
official_review

<thread_object>
creator: reviewer_bav4

<content>
summary:
The paper tests LLM bias in the task of feature engineering for training a downstream learning system. The paper uses several LLM settings across 27 datasets, to demonstrate that LLMs do indeed have bias for this task, which indeed seems to lead to poor performance compared to an existing SOTA automated feature engineering solution. Some further discussion and experimentation into the properties of the bias shows that the LLM seems to prefer simpler features when the features have meaningful names, but also doesn't sample uniformly when the features have nondescript random names.

strengths:
The paper is based on solid experimental work, testing using several LLMs and across many datasets, testing for memorization issues separately to check for bias explicitly. 
The paper is an interesting and easy to follow read. Problematic properties of LLM solution paths for different problems are always appreciated, as we develop more and more systems that significantly rely on this tool, we must strive to understand the biases this seemingly easy fix-all solution of asking an LLM brings into our work and the times it might fail completely. It is also interesting that the large models have failed worse at adding features that helped with the downstream system's results compared to the smaller models, which did help a little.

weaknesses:
The main issue with this paper is that it is rather unclear why the usage of LLMs for this task was explored at all. It seems that when feature engineering is done by an LLM, the downstream system's performance is worse than existing SOTA systems -  and sometimes even worse than doing any feature engineering at all. Frankly, it's also not a task that I would intuitively expect LLMs to be good at, as general knowledge, common sense and language knowledge is probably not what humans would use for feature engineering, but rather math/engineering skills and perhaps expert domain knowledge or lived experience - all usually less strong qualities of LLMs. The paper does not call this issue out or justify it. Usually, checking for biases of a solution might have one of two purposes: 1. call out poor performance that happens in a way that isn't expected or measured in other ways, so for example, if the system had seemingly good downstream performance, checking for biases or other issues might help guard us from using a problematic solution that looks good in our metrics. 2. try to improve the performance of the biased system by somehow mitigating the bias. It seems that option 1 in this case is unnecessary, since the LLMs have worse performance, and no actual attempt is made towards the #2 target.

questions:
1. Why would I use an LLM for feature engineering anyway, if there are existing SOTA automated systems that already do it and perform much better? 
2. If your answer to #1 is that I probably wouldn't, then the main question about publishing this paper would be - why would I read a paper about the biases such a solution might have? There could be several answers (e.g., to inspire a similar analysis of LLM solutions for other problems) but they need to be clear within the paper.  

Small issues:
1. Please explain that the improvement in "Predictive Performance Improvement" is improvement compared to a system without FE earlier in the document, e.g. before table 3.
2. While the random experiment is fun and adds to the paper, I don't think it is at all accurate to say that it tests "whether our prompt template influenced our results" - seeing as the prompt template itself did not change in this experiment, only the names of the features. I don't think it shows anything about the prompting strategy - but rather that the nature of the bias depends on the feature naming. 
3. Caught typo: The API usage costed -> cost
</content>
</thread_object>

</thread>
<thread>
creation date:
1730690879550

thread_type:
official_review

<thread_object>
creator: reviewer_odst

<content>
summary:
The paper explores the featuring engineering powers of LLM with OpenFE as the baseline. The authors perform experiments on 27 datasets and 4 LLMs. The primary findings are the following.

1. LLM perform worse than the baseline.
2. Proprietary "Large" models perform worse than small "open" models.

strengths:
The authors experimentally show the limitations of LLMs for feature engineering. The experimental setting is convincing.

weaknesses:
1. The conclusions of the paper are along expected lines and are not surprising. A more notable contribution would be to address the limitations.
2. The statistical significance of the results is not provided.
3. The term "bias" is too strong for the problem explored. The authors can use the word "limitation".

questions:
1. What is the statistical significance of the results shown in Table 3?
2. Why aren't larger models in GPT and Gemini family not explored?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730698158009

thread_type:
official_review

<thread_object>
creator: reviewer_jgk8

<content>
summary:
The authors investigate how well LLMs can engineer features for tabular datasets. Specifically, they look at the frequencies of operators, and find that there is bias toward simpler features rather than more interesting or useful ones. They also evaluate on the downstream accuracy of the models trained with and without the engineered features.

strengths:
It is good to see more examples of evaluation of downstream LLM tasks "in the wild".

I appreciate that the authors were rigorous in removing datasets that were thought to be memorized or in the training data of the LLM, even though they did not have access to the training data itself.

weaknesses:
To me, this doesn’t seem like an influential enough contribution. Not only is it tackling a very narrow problem, but it is also only evaluating a specific method for addressing that problem. While there is some prior work around using LLMs for feature engineering, I’m not convinced that this work’s method for feature engineering is necessarily representative of all the methods for using LLMs for this task. 

Specifically, the authors only use one prompting strategy, on a snapshot of models at the current time that this paper is being written. A few examples of people using LLMs for feature engineering are cited (Hatch, 2024; Türkmen, 2024), but it is unclear what methods these citations used– is the author’s method the same, or inspired by them? Should data scientists conclude from this paper that they should never use LLMs for feature engineering, even if they use a different method? Overall, I think this is an interesting use case to evaluate, but the work is not broad enough to be included in ICLR.

Nits:
Typo: “which is send to the LLM” → sent

questions:
I’m a little confused about the experimental setup regarding operations. If I understand correctly, the authors are comparing the distribution of operators generated by the LLM and by OpenFE. If OpenFE is considered ground truth, why not compare directly to OpenFE final generated feature set? For example, rather than just counting the number of times we see the operation “GroupByThanRank”, why not look at the original features that were input into this operation?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732094403658

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Revision

comment:
Dear Reviewers,

We sincerely appreciate the time and effort you dedicated to reviewing our submission. Your feedback has helped to improve our paper. Below, we present the revision changes, which we marked green in the submission. Additionally, we have addressed all of your concerns raised in your reviews respectively under your reviews.

* Our paper was strongly motivated by the success of different feature engineering methods with LLMs, most notably CAAFE, which we clarified in the introduction. We additionally highlighted the influence of CAAFE in our proposed feature generation method, which we clarified further in the related work section.
* We further explained the metric “relative improvement in predictive accuracy”, which we use several times in our “Results” section when comparing different LLMs to OpenFE. We clarify that we here compare systems without feature engineering to systems with feature engineering (performed by LLMs and OpenFE, respectively).
* We reformulated the takeaways of the additional experiment. We clarified that this experiment ensures that the experienced bias towards the simple operators is not a positional bias induced by our prompt template.
* We highlight the necessity of researchers finding ways to strengthen LLMs effectively for tabular data problems so that they can be used effectively.
* We provide a critical difference plot to highlight the statistical significance of our results in Appendix G.1
* In Appendix G. 2, we compare operator selection frequencies on a subset of benchmark datasets between GPT-4o-mini and the more powerful GPT-4o to highlight that the bias is similar in these models. This leads us to believe that using more powerful models does not fix this bias.
* We provide a comparison of frequencies with which each feature of a dataset is selected by the LLMs to generate new features in Appendix G.3
</content>
</thread_object>

</thread>
<thread>
creation date:
1733297067598

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Withdrawal

comment:
We again thank the reviewers for their time and effort to review our paper. We have considered all raised concerns in great detail and addressed each individually. Thanks to the authors' comments, we improved our manuscript in different areas, for which we are grateful. 

Unfortunately, there was no possibility of discussing our changes in greater detail with the reviewers during the discussion period. Even though we addressed each reviewer's concerns individually and in great detail, we received no further comments regarding our initial responses from multiple reviewers. 

We also received a high confidence rejection due to naming the experienced phenomenon we observed as “bias”. From the discussion below about this difference in opinions, we firmly believe that the word bias was justified in our case and that we provided strong background references that justify our usage of the word. 
Nevertheless, we are thankful for each reviewer's comments and positive input which helped improve our work.

We have decided to withdraw our submission and are currently working on further improving this work. We aim to publish our work in the future as we firmly believe that our findings present valuable input to the community.
</content>
</thread_object>

</thread>


Metareview: