Can Models Help us Create Better Models? Evaluating LLMs as Data Scientists

Authors: Michał Pietruszka, Łukasz Borchmann, Aleksander Jędrosz, Paweł Morawiecki
Primary area: datasets and benchmarks
Keywords: llm, data science, benchmark, tabular data, feature engineering, dataset, kaggle, evaluation, code generation

Abstract:
We present a benchmark for large language models 
designed to tackle one of the most knowledge-intensive tasks in data science: writing
\textit{feature engineering} code, which requires domain knowledge in addition to a deep understanding of the underlying problem and data structure.
The model is provided with a dataset description in a prompt and asked to generate code transforming it. The evaluation score is derived from the improvement achieved by an XGBoost model fit on the modified
dataset compared to the original data. By an extensive evaluation of state-of-the-art models and comparison to well-established benchmarks, we demonstrate that the \bench{} of our proposal can cheaply and efficiently assess the broad capabilities of LLMs, in contrast to the existing methods.
The reference implementation is available at \url{ATTACHED_AS_SUPPLEMENTARY}

Decision: reject
Reviewer scores: 3, 5, 5, 5

Reviews and discussion:
<thread>
creation date:
1729863884409

thread_type:
official_review

<thread_object>
creator: reviewer_yfrz

<content>
summary:
This paper constructs a benchmark called FeatEng, which focuses on evaluating LLMs through the lenses of Pragmatism, Functionalism, Computationalism, and Scientific Realism.

strengths:
In general, the paper is easy to follow, with clear presentations in the figures and tables.

weaknesses:
**Summary: hard-to-justify claims, and lack of concrete experiments or showcase**
1. Limited literature works:
  - In Section 1.1, the authors mention many problems with existing benchmarks and explain why they do not fit the "four characteristics" that the authors abstract as essential for evaluating intelligent systems. However, in my view, many of the benchmarks mentioned are commonly used and well-recognized for testing base LLMs. They may be crucial for assessing specific capabilities of pre-trained base models.
  - In contrast, this paper seems more like it is evaluating a benchmark for post-trained LLMs, which makes such a comparison seem unfair from this perspective.
2. Moreover, the discussion in section 1.1 feels rather vague and lacks specific experimental evidence or concrete examples to support the claims about existing benchmarks. The authors should consider providing more detailed comparisons or results that highlight their benchmarks' advantages over others. At the same time, I wonder if the authors have considered that many concurrent works are also testing models in complex, knowledge-intensive scenarios, evaluating whether the models can perform well in real-world tasks that require extensive knowledge. I am not sure if feateng can compare against these benchmarks and highlight its advantages?
3. The paper (excluding the appendix) only presents one table to display the results. I do not think this is enough to fully demonstrate the characteristics of the dataset. Additionally, the so-called "strong correlation" to chatbot ELO scores does not seem clearly reflected in this table. I think such claim need quantitative values to demonstrate, rather than a vague statement. Without more specific experimental results or showcase examples, the claims remain unsubstantiated and less convincing.

questions:
The textual description in Sec 3.1 still seems insufficiently structured and process-oriented. I wonder if the **100 work hours** mentioned by the authors included a clear workflow? This seems crucial for evaluating whether the data collection process is reasonable, and whether it aligns with the four criteria proposed by the authors.

For example:

- How did you collect the source dataset? Based on what principles can a Kaggle dataset be used for feateng?
- As Kaggle is a high-quality data science forum, I strongly believe that it has already been used in some LLM training corpora. Therefore, have you fully considered the possibility of data contamination?
</content>
</thread_object>

</thread>
<thread>
creation date:
1730224900841

thread_type:
official_review

<thread_object>
creator: reviewer_kb4t

<content>
summary:
This paper proposes a new benchmark, *FeatEng*, to evaluate the ability of large language models (LLMs) to perform feature engineering for tabular data. The benchmark focuses on a code generation task where the model is given a description of a dataset and tasked with generating code to transform the data to improve the performance of a machine learning model. The authors argue that existing LLM benchmarks often fail to capture the practical usability, domain knowledge, and complex skill integration required for real-world data science applications. They further demonstrate that *FeatEng* aligns well with these criteria and offers a more effective and efficient way to assess the capabilities of LLMs in this area.

strengths:
This is a well-written benchmark paper with clear motivation.
1. Clear classification of existing benchmarks regarding philosophical traditions of pragmatism, functionalism, computationalism, and scientific realism.
2. Well-organized dataset from diverse, high-quality Kaggle competitions.
3. An interesting finding of the high correlation between FeatEng and Chatbot Arena.
4. A good viewpoint LLMs can tackle feature engineering and improve upon current AutoML systems by leveraging their potential to integrate domain knowledge and reasoning to generate efficient and interpretable features.

weaknesses:
This paper has many weaknesses regarding its experiment design.

1. The result interpretation is limited:
    - Table 1: Mean FeatEng scores (Improvement) compared to Chatbot Arena ELO. Would it be better if we made a line plot for the comparison?
    - It would be helpful if more AutoML baselines [1,2] could be compared. Also, the best human results from the Kaggle competition can be included.
    - The only takeaway I can draw from this paper is that FeatEng can be a cost-effective substitution for Chatbot Arena in a way that it assesses the genuine technical capabilities of LLMs.
    - Another comparison that comes to my mind is how LLM's number of parameters can affect the benchmark results on FeatEng. We can show what will happen when the number of parameters scales for the same group of LLM (i.e., Claude-3-Haiku, Claude-3-Sonnet, Claude-3-Opus), which will give more insights into how to develop parameter-efficient LLMs.

2. Examples of the single-pass evaluation pipeline do not look good to me
    - The pipeline, as described, involves a single pass where the LLM generates code based on the input. How can this single-pass evaluation compare itself with AutoML, an iterative algorithm?
    - Figure 3 is not clear. A flow chart can be helpful.

3. Earlier works, such as [3, 4, 5], have explored much potential in LLMs for machine learning and AutoML tasks with iterative revisions. A detailed comparison and literature review would be helpful.

[1] Microsoft. Neural Network Intelligence (version v3.0pt1). https://github.com/microsoft/nni
[2] H2O.ai (Oct. 2016). H2O, H2O version 3.10.0.8. https://github.com/h2oai/h2o-3.
[3] Zhang, L., Zhang, Y., Ren, K., Li, D., & Yang, Y. (2024, March). MLCopilot: Unleashing the Power of Large Language Models in Solving Machine Learning Tasks. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers) (pp. 2931-2959).
[4] Guo, S., Deng, C., Wen, Y., Chen, H., Chang, Y., & Wang, J. DS-Agent: Automated Data Science by Empowering Large Language Models with Case-Based Reasoning. In Forty-first International Conference on Machine Learning.
[5] Hollmann, N., Müller, S., \& Hutter, F. (2024). Large Language Models for Automated Data Science: Introducing CAFFE for Context-Aware Automated Feature Engineering. Advances in Neural Information Processing Systems, 36.

questions:
Typos:
1. Line 019: asses $\rightarrow$ assess
2. Line 563, 567: duplicated entries of the same citation

Questions:
Should we include citations for XGBoost[1]?

[1] Chen, T., & Guestrin, C. (2016, August). XGBoost: A scalable tree-boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (pp. 785-794).
</content>
</thread_object>

</thread>
<thread>
creation date:
1730553316223

thread_type:
official_review

<thread_object>
creator: reviewer_eugm

<content>
summary:
This paper introduces FeatEng, a new benchmark for testing large language models (LLMs) in feature engineering. The idea is pretty cool—they've set up a way for models to generate code that transforms data, making it more suitable for machine learning tasks. The key metric is whether the transformation improves the performance of an XGBoost model trained on the modified data compared to the original data. The authors emphasize that FeatEng aims to fix gaps in existing benchmarks by focusing on real-world tasks, like practical usefulness, integrating knowledge, handling complex skills, and being resistant to "gaming" the system. They ran various models through FeatEng and analyzed the results to see how well the benchmark captures what different models can and can’t do in this context.

strengths:
+ Focusing on feature engineering is a fresh approach for LLMs, as this task demands both technical skill and domain knowledge. It’s a step away from standard code generation and into real-world data science workflows. Nice angle!

+ The benchmark is well-defined with solid metrics. Using an outcome-based metric (i.e., the performance improvement of a downstream model) is smart—it’s a straightforward way to check if the code is actually making things better. Also, the dataset selection is thorough, covering a variety of domains and types, which keeps it well-rounded.

+ The paper does a great job explaining the ideas behind each of their evaluation criteria. They walk readers through their motivations and outline the benchmark design clearly. It’s easy to follow and understand the reasoning behind why they chose each aspect of evaluation.

weaknesses:
- Even with diverse datasets, there's a possibility that models could end up overfitting to specific dataset types or common data science tasks, which could skew the results. If a model is already trained on similar data, it might appear more capable than it really is.

- The benchmark’s main evaluation metric depends entirely on XGBoost’s performance. While XGBoost is popular, it’s not the only ML model, and different feature engineering efforts might have varied effects on different types of models. A mix of evaluation algorithms could give a more rounded view.

- Running this benchmark might be resource-heavy, especially on larger datasets or complex transformations. It seems realistic but could be a bottleneck if someone wants to use FeatEng on many models at scale.

- Although they mention that a human baseline would be helpful, they don’t provide one here. Without it, it’s harder to tell how well these models are doing in comparison to actual human experts. Even a rough human baseline would make the results more relatable.

questions:
+ How are you making sure that models aren’t just memorizing common feature engineering techniques? Any thought on including completely new datasets down the line to keep models on their toes?

+ What made you pick XGBoost as the benchmark’s only evaluation model? Wouldn’t including other models (like neural networks) give a broader view of the impact of the feature engineering?

+ Can you give more detail on how improvement is scored across different dataset types (like binary classification, multi-class, regression)? This would help make sure everything’s consistently fair.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730707908570

thread_type:
official_review

<thread_object>
creator: reviewer_nk3s

<content>
summary:
This paper proposes an LLM benchmark that prompts the LLM writing feature engineering code for ML tasks and then run XGBoost with the obtained features to get a score.

strengths:
This paper is overall clear and easy to understand.

weaknesses:
It might be okay for the proposed benchmark to evaluate a specific coding aspect of LLMs, but I think this paper overclaims its generalbility to a large extent. Specifically, it claims it addresses limitations of existing benchmarks like MMLU and HumanEval, and reflects all fundamental aspects of intelligence. However, I feel the benchmark has significant limitations compared with what the paper claims. 
* Narrow application scope: "Wrting feature engineering code" is just a single, specific use of LLMs. Also, it is not a highly frequent use of LLM users. 
* Limited applicable LLMs: The proposed metric requires the LLM has both natural language and coding abilities, but this is not a must-have feature for a "strong LLM". 
* This evaluation would prefer coding LLMs over really intelligent models. For example, in Table 1 of the paper, codestral-22B is even better than llama-3-405B under the proposed metric.
* I also have concerns on the flexibility of the metric and how challenge it is -- usually, the feature engineering of ML has just a handful of strategies, such as BPE tokenization for text data, one-hot for discrete labels, normalization for float number labels, etc.

questions:
* Could you share any examples regarding the wrong decisions of mistral-7b for its low score?
* Is there any LLM-produced data processing strategies beyond what human engineers often use?
</content>
</thread_object>

</thread>
<thread>
creation date:
1732536570250

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Summary of Reviewer Responses and Documented Changes

comment:
Dear Reviewers,

We sincerely appreciate the time and effort you have invested in reviewing our paper. Your constructive feedback has been invaluable in enhancing the quality of our work. Below, we address the key concerns raised in your reviews:

---

### 1. Scope and Significance of Feature Engineering (Reviewers nK3S and Eugm)

**Concern:**  
Feature engineering was viewed as a narrow application, possibly not reflecting general intelligence or frequent use cases for LLMs.

**Response:**  
We argue that feature engineering is a complex task requiring the integration of multiple cognitive abilities, such as domain knowledge application, reasoning, creativity, and technical skills. It serves as a microcosm of complex problem-solving and reflects the broader capabilities of LLMs. In the revised manuscript, we have significantly extended the experimental section with systematic qualitative and quantitative analyses of the models' performance and the techniques employed (see Section 3). For example, we found that code containing strong domain knowledge-based features outperforms code without such features. This analysis demonstrates the task's depth and relevance.

---

### 2. Clarity and Presentation Improvements (Reviewers KB4t and YFRZ)

**Concern:**  
The presentation of results and figures could be enhanced for better clarity and understanding.

**Response:**  
We have updated our manuscript with enhanced tables and figures, including color-coded performance metrics in **Table 1** and line plots in **Appendix Figures 7 and 8**. We have also replaced **Figure 3** with a clearer flowchart (now **Figure 4**) to illustrate the evaluation pipeline more effectively. We believe these changes significantly improve the paper’s clarity.

---

### 3. Result Interpretation and Visualization (Reviewer KB4t)

**Concern:**  
The interpretation of results was limited, and additional baselines could enhance understanding.

**Response:**  
We have included more detailed analyses and visualizations in the revised manuscript. The experimental section and analysis of the generated code now cover three pages of the manuscript (pages 7-10). While our primary focus is on the performance of LLMs in feature engineering tasks, we included a simple AutoML baseline for context. Expanding comparisons to more AutoML systems is noted as a possible future work.

---

### 4. Potential Overfitting and Data Contamination (Reviewers nK3S and YFRZ)

**Concern:**  
Models might overfit to specific dataset types or have been exposed to similar data during training, affecting the validity of results.

**Response:**  
We mitigated overfitting by using 103 diverse datasets from various domains, each with unique problem descriptions (see clarified Section 2.1 and **Table 5**). We selected less prominent datasets and modified them to reduce the likelihood of prior exposure. Our benchmark's open-ended nature requires models to adapt creatively instead of memorizing techniques, lessening the impact of any potential data contamination.

---

### 5. Dependence on XGBoost as the Sole Evaluation Model (Reviewers Eugm and KB4t)

**Concern:**  
Relying solely on XGBoost might limit the benchmark's generality, and including other models could provide a broader view.

**Response:**  
We chose XGBoost for its robustness, efficiency, and widespread use in real-world applications. Our primary goal was to maintain consistency across evaluations. We acknowledge the value of incorporating other models and have noted this as a possible direction for future work.

---

### 6. Inclusion of Human Baselines (Reviewers Eugm and KB4t)

**Concern:**  
The absence of human baselines makes it difficult to contextualize model performance.

**Response:**  
We recognize the importance of human baselines for meaningful comparisons. Collecting human benchmarks across our diverse tasks is resource-intensive. We propose involving the data science community to contribute human-generated solutions over time, as mentioned in Section 4 of the revised manuscript.

---

### 7. Literature Review and Comparison to Existing Benchmarks (Reviewer YFRZ)

**Concern:**  
The discussion of existing benchmarks was perceived as vague, lacking specific experimental evidence or comparisons.

**Response:**  
We have expanded **Section 1.1** and **Appendix H** to include concrete examples illustrating how existing benchmarks may not fully assess the integrated skills required for tasks like feature engineering. We also discuss concurrent works and how our benchmark compares and contributes uniquely to the field.

---

Once again, we thank you for your insightful feedback. We believe that the revisions have significantly strengthened our paper, and we hope that our responses adequately address your concerns. We remain open to any further feedback and are committed to addressing any additional questions or issues you may have before the discussion period ends.
</content>
</thread_object>

</thread>


Metareview:
metareview:
The paper introduces FeatEng, a novel benchmark designed to evaluate large language models (LLMs) on feature engineering tasks for tabular datasets. Feature engineering is a critical and knowledge-intensive aspect of data science, requiring both domain knowledge and technical proficiency. FeatEng prompts LLMs to generate code that transforms a dataset based on a provided description, and the effectiveness of the generated features is assessed by the performance improvement of an XGBoost model trained on the modified dataset compared to the original. The authors claim that FeatEng addresses limitations in existing benchmarks by focusing on practical usability, knowledge integration, and resistance to "gaming" the system. Through extensive evaluations of state-of-the-art LLMs, the paper demonstrates that FeatEng can efficiently assess LLM capabilities in feature engineering, showing a high correlation with Chatbot Arena ELO scores while being cost-effective.

#### Contribution
1. **Novel Benchmark for LLMs**: FeatEng is the first benchmark specifically designed to evaluate LLMs' capabilities in feature engineering, a complex task that integrates domain knowledge, reasoning, and coding skills.
2. **Practical Relevance**: By focusing on feature engineering, FeatEng assesses LLMs' practical utility in real-world data science workflows, potentially paving the way for LLMs to assist human data scientists.
3. **Correlation with Human Judgment**: The benchmark demonstrates a strong correlation with Chatbot Arena ELO scores, suggesting that it captures essential aspects of LLM performance relevant to human users, even though it was not explicitly designed for this purpose.
4. **Dataset Diversity**: The use of 103 diverse datasets from various domains ensures the benchmark's broad applicability and reduces the risk of overfitting.

#### Weaknesses
1. **Narrow Scope (nK3S)**: The focus on feature engineering might be perceived as a narrow application, potentially limiting the generalizability of the findings to broader LLM capabilities.
2. **Overfitting and Data Contamination Concerns (Eugm, YFRZ)**: The use of datasets from Kaggle raises concerns about potential overfitting or data contamination if models have been trained on similar data.
3. **Dependence on XGBoost (Eugm, KB4t)**: The sole use of XGBoost for evaluation might limit the benchmark's generality, as different models might respond differently to feature engineering efforts.
4. **Lack of Human Baselines (Eugm, KB4t)**: The absence of human performance benchmarks makes it challenging to contextualize the results and understand the practical significance of LLM performance.
5. **Single-Pass Evaluation (KB4t)**: The single-pass evaluation method contrasts with iterative real-world data science practices, potentially underestimating LLM capabilities that could improve with iterative refinement.
6. **Limited Literature Review and Comparisons (YFRZ)**: The paper's discussion of existing benchmarks is seen as vague and lacks specific experimental evidence or comparisons to concurrent works evaluating LLMs in complex scenarios.