From Uncontextualized Embeddings to Marginal Feature Effects: Incorporating Intelligibility into Tabular Transformer Networks

Authors: Anton Frederik Thielmann, Arik Reuter, Benjamin Säfken
Primary area: interpretability and explainable AI
Keywords: Tabular Deep Learning, Interpretability, Tabular Transformer Networks

Abstract:
In recent years, deep neural networks have showcased their predictive power across a variety of tasks. The transformer architecture, originally developed for natural language processing, has also shown great efficiency in handling tabular data, offering a competitive alternative to traditional gradient-boosted decision trees in this domain. However, this predictive power comes at the cost of intelligibility: Marginal feature effects are almost completely lost in the black-box nature of deep tabular transformer networks. Alternative architectures that use the additivity constraints of classical statistical regression models can maintain intelligible marginal feature effects, but often fall short in predictive power compared to their more complex counterparts.   To bridge the gap between intelligibility and performance, we propose an adaptation of tabular transformer networks designed to identify marginal feature effects. We provide theoretical justifications that marginal feature effects can be accurately identified, and our ablation study demonstrates that the proposed model efficiently detects these effects, even amidst complex feature interactions. To demonstrate the model's predictive capabilities, we compare it to several interpretable as well as black-box models and find that it can match black-box performances while maintaining intelligibility. The source code is vailable at https://anonymous.4open.science/r/nmfrmr-B086.

Decision: reject
Reviewer scores: 5, 8, 5, 3

Reviews and discussion:
<thread>
creation date:
1730493511009

thread_type:
official_review

<thread_object>
creator: reviewer_fvxa

<content>
summary:
This work proposes a modification to one of the tabular transformer architectures that enables features to be interpretable. Specifically, the technique proposed in this work, NAMformer, which is built on top of the existing FT-Transformer, enables the identification of "marginal feature effects" which can be used to interpret the final model. NAMformer performs comparably to the original FT-Transformer, while maintaining desirable interpretability properties.

strengths:
- The claim that the proposed NAMformer is not worse than FT-Transformer in terms of prediction performance seem true on the tasks that were used in the evaluation. 
- The general idea of making tabular transformer architectures interpretable seems like it should be useful for a variety of problems. 
- The inclusion of synthetic evaluations and ablations are useful for understanding the mechanics of the proposed approach.

weaknesses:
- The introduction and methodology sections could benefit significantly from writing improvements and more thorough explanation. In its current form, these sections assume a lot in terms of prior knowledge, e.g. of marginal feature effects, the FT-Transformer architecture, target-aware embeddings, and uncontextualized embeddings. These should be explained, at least at a high-level. Additionally, from Section 2, it is somewhat unclear which parts are inherited from the prior work on FT-Transformer and which parts are new. Particularly, I was confused about the feature encoding part--without referencing the prior work, I cannot easily assess if this part of the work is novel. 
- The authors should include specific use-cases in which the marginal feature effects are useful downstream. Otherwise, it is challenging to assess the significance of the contribution. 
- The paper should have a standalone explanation of the datasets that are being used in evaluation beyond saying that the evaluation is done on four regression and four binary classification datasets---what are these datasets and where do they come from? The authors seem to refer the reader to the appendix for this information, but this is critical for assessing the results. Furthermore, the evaluation should be more thorough than regression and binary classification -- conceivably, the proposed method could become worse than FT-Transformer as the number of classes grows. The authors should include such tasks.

questions:
- What are uncontextualized embeddings? As far as I can tell, this is never explained in the paper. 
- What does token identifiability mean? This is also not defined.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730554059449

thread_type:
official_review

<thread_object>
creator: reviewer_hmem

<content>
summary:
This work proposed an adaptation of tabular transformer networks designed to identify marginal feature effects, while maintaining the capability of prediction performance at the original level.

strengths:
From my prospective, the strengths of this paper are as follows:
- Compared to FT-Transformer or GAM, the proposed NAMformer has more interpretability and stronger predictive power, respectively
- There are analysis between contextual and uncontextual embeddings, proving that the uncontextual embedding can represent the original feature
- There are a lot of formula providing theoretical support

weaknesses:
After reading the paper, I still have some questions.
1. In section 2, the encoding of numerical features involves the threshold $b_t$, and the thresholds are from the decision trees. So I wonder how we get these decision trees? Is the marginal effects of the model the same as those decision trees? And what about the performance compared with the decision trees.
2. The method seems too trivial to achieve the expected performance. Only a Linear and dropout can identify the marginal and even enhance the performance.
3. In tabular domain, it seems that 8 datasets in all are not sufficient. This may result in doubt that the datasets used are selected for this task.

In addition to the above questions, I think there are some problems with typography and content。
1. It will be better if there is a section which introduces the related work, e.g. transformers in tabualr, marginal feature effects in tabular or other domains, etc. I think this will help reader better understand the task and your method.
2. It seems that in section Abstract, there is some problem with the spacing between lines.

questions:
See Weaknesses
</content>
</thread_object>

</thread>
<thread>
creation date:
1730690052595

thread_type:
official_review

<thread_object>
creator: reviewer_2nbf

<content>
summary:
This paper proposes a modification to predictive tabular transformer models to make them more interpretable: producing the final output as a sum of the transformer output along with shallow univariate models for each input feature. Dropout is used to encourage the model to optimize its use of each feature independently. The univariate models can then be interpreted as indicating the marginal effect of each feature on the overall output.

strengths:
- In general, the proposed method makes sense to me and I could see it being used in practice. I've worked with tabular transformers in applied settings and can attest that understandable per-feature explanations are difficult to achieve. This method makes sense to me as a way to get greater interpretability while retaining the performance of tabular transformers, or other deep tabular models.
- As far as I can tell the proposed method is original. It resembles NATT (Thielmann et al. 2024), but I think the design differences are significant in practice and make sense. This method passes all features to the transformer as input, then uses dropout during training to guide the model to a desired behaviour that makes all features interpretable, whereas NATT keeps numeric and categorical features separate and only provides a comparable level of interpretability for numeric features.
- Experimental results are fairly convincing in showing that the proposed method doesn't sacrifice performance.

weaknesses:
- Discussion of related works is lacking. There is no separate related works section, just some context provided in the introduction, and while a decent selection of related papers are cited, there's little discussion of how the proposed method relates to and differs from existing work. One point I would like to see addressed is whether and how this method improves on methods like integrated gradients, LIME, or SHAP that produce per-feature explanations without requiring model modifications.

- Some theoretical results are given in Section 2.1, but it's not described how the results relate back to the interpretability of the model, and there are some particular issues I wanted to raise:
	- The definition given for $R_{\tilde{w}_{-k}}$ at line number 279 doesn't make sense to me - it could even take the opposite sign of the overall risk \(R\) if the model has a greater loss with just feature $k$ than on average.
	- Equation (7) depends on the assumption that the risk for each single-feature dropout vector is the same and equal to $R(1-p(\tilde{w}_k))$, which seems very unlikely for realistic models or datasets where features have different importances and the model tends to perform better given more features. So I don't see the relevance of this result.
	- My interpretation of the section is that when training with dropout, the performance of individual feature predictors is in some sense bounded by the overall model performance. This makes sense, but it's not clear to me if this indicates anything about the interpretability of the model in terms of the individual feature predictors or what their predictions represent (i.e., do they actually optimize the loss with respect to $\mathbb{E}_{y|x_k}[y|x_k]$, or just something that's not too far from it?).

- Even under the proposed approach, the transformer component itself is somewhat of a black box. This could be tolerable given the improved interpretability of the overall model, but it is a limitation.
	- While thinking through the method, I kept coming back to what the optimal behaviour learned by the transformer and shallow predictors would be. I think it is true that the optimal behaviour at low dropout rates is at least approximately for the transformer to only predict joint effects plus marginal effects that are too complex for the shallow predictors to learn, rather than learning all joint and marginal effects. A proof of this would be useful if possible though.

Given that I think the proposed method could have real practical utility, I'm leaning towards an accept, but I would prefer to see more a more relevant theoretical discussion.

questions:
- Could you please clarify the results in Section 2.1, especially how to interpret them with respect to interpretability?
- In practice, how can we be confident that the feature networks have the capacity to fully capture marginal effects? If they can't fully capture them, then part of the marginal effect could be learned by the transformer instead.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730720357136

thread_type:
official_review

<thread_object>
creator: reviewer_2ug9

<content>
summary:
I do not feel qualified to review this paper as I do not have any experience with tabular machine learning. I would like to ask the ACs to seek an opinion from different reviewers.

strengths:
n/a

weaknesses:
n/a

questions:
n/a
</content>
</thread_object>

</thread>
<thread>
creation date:
1732106845998

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
General Response to all reviewers

comment:
Dear Reviewers,

We would like to sincerely thank you for taking the time to review our work and providing thoughtful feedback. We greatly appreciate your evaluations, which we find encouraging as they underline the key benefits of our proposed approach.

**Edit**: Since the Github link in the provided PDF expired, we have created a new link, including the same repository as before: [Repository](https://anonymous.4open.science/r/nmfrmr-F76D/README.md)

We are confident that we can effectively address the concerns you raised and have already taken steps to improve the manuscript. Specifically:

- **Dedicated Literature Review:** To provide better context for readers unfamiliar with Tabular Deep Learning and Additive Models, we have included a focused literature review explaining these concepts in greater detail. This section is currently placed in the appendix, as we aim to direct unfamiliar readers to that section without disrupting familiar readers with content they are likely already acquainted with. However, we are happy to integrate it into the main body of the paper after further discussion and realignment with you.

- **Additional Benchmarks:** To further strengthen the claims of our study, we have conducted additional benchmarks. Given the short time frame and limited computational resources, we focused on regression tasks and interpretable models, standard normalized the response variables and applied the same hyperparameter optimization as in the manuscript. We are currently conducting further benchmarks and will include these additional results in the final version.
The results from these additional benchmarks, performed on seven new datasets from OpenML (ID 353; [1]), strongly support the conclusions presented in the original manuscript. These findings reinforce the robustness and reliability of our approach.

We hope these updates demonstrate our commitment to addressing your concerns and improving the clarity and rigor of our work. We are confident that the revised manuscript presents a more compelling and comprehensive case for our contributions. Additionally, we have adressed your individual questions and concerns in dedicated answers below.  


Thank you again for your constructive feedback, and we look forward to your further input.

**Additional Benchmark results for interpretable models:**

| Model       | AV ↓    | GS ↓   | K8 ↓   | P32 ↓  | MH ↓   | BH ↓   | SG ↓   |
|-------------|---------|--------|--------|--------|--------|--------|--------|
| Linear      | 0.700   | 0.366  | 0.580  | 0.843  | 0.295  | 0.025  | 0.445  |
| GAM         | 0.287   | 0.228  | 0.557  | 0.909  | 0.157  | 0.023  | 0.273  |
| EBM         | 0.050   | 0.079  | 0.411  | 0.395  | 0.096  | 0.033  | 0.272  |
| EB²M        | 0.049   | 0.079  | 0.409  | 0.388  | **0.099** | 0.026  | **0.263** |
| NAM         | 0.372   | 0.235  | 0.927  | 1.049  | 0.181  | 0.025  | 0.399  |
| Hi-NAM      | 0.135   | **0.034** | **0.076** | 0.435  | 0.102  | 0.128  | 0.278  |
| NAMformer   | **0.023** | 0.051  | 0.108  | 0.397  | **0.095** | **0.022** | 0.270  |


Average performance over all 15 datasets, i.e. the datasets from the original submission as well as the additional datasets:

| Model       | Average Rank | Rank Std Dev |
|-------------|--------------|--------------|
| NAMformer   | **1.700**    | 0.678        |
| EB²M        | 2.467        | 1.024        |
| EBM         | 3.100        | 1.114        |
| Hi-NAM      | 3.567        | 1.740        |
| GAM         | 4.800        | 1.030        |
| NAM         | 6.000        | 0.796        |
| Linear      | 6.367        | 1.040        |


---

Thank you once again for your valuable feedback. We hope our responses address your concerns and underscore the significance of our contributions. W have answered your individual comments in the dedicated sections below.

---


[1] Fischer, Sebastian, et al. "A curated tabular regression benchmarking suite."
</content>
</thread_object>

</thread>
<thread>
creation date:
1733304815788

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Final Remarks and Summary

comment:
Dear Area Chair and Reviewers,

We sincerely thank the reviewers for their time and effort in evaluating our manuscript. We especially appreciate the constructive feedback and engagement provided by Reviewer ``2nBF`` during the rebuttal phase.
We are also confident that our detailed responses, references to relevant literature, and the accompanying codebase effectively address all concerns raised by Reviewer ``hMeM``. The provided evidence, theoretical grounding, and reproducible codebase robustly support the presented results.

Furthermore, the points raised by Reviewer ``FVxa`` have been carefully addressed in our revised manuscript. This includes the addition of a standalone literature review and a detailed explanation of marginal feature effects. These enhancements aim to clarify and contextualize the relevance of our contributions for readers less familiar with classical statistical concepts. Additionally, we have demonstrated NAMformer’s applicability to multi-class classification and its performance comparable to FT-Transformer.

In summary, during our rebuttal we have:
- Nearly doubled the number of datasets in our benchmarks, with results already included in the revised version showing that     
    - NAMformer outperforms all other interpretable comparison models,
- Added a dedicated literature review, which will be integrated into the main manuscript in the final version,
- Provided a comparison to Integrated Gradients,
- Refined the methodology section, thanks to the insightful comments from Reviewer ``2nBF``.

---
Once again, we greatly appreciate the reviewers’ feedback. We are confident that we have thoroughly addressed the concerns and questions raised and are grateful for the opportunity to significantly improve our manuscript.
</content>
</thread_object>

</thread>


Metareview:
metareview:
This paper proposes modifications to FT-Transformers to enhance interpretability while maintaining prediction performance. The proposed approach highlights the marginal effect of each feature without performance degradation. Additionally, the analysis demonstrates that uncontextualized embeddings can effectively represent the original feature space.

The paper received mixed scores. Its strengths include the high interpretability of marginal feature networks (e.g., through simple plots of output versus input) and the diminished role of the less interpretable transformer component. The additive structure allows for a direct measurement of the transformer's relative influence compared to the marginal networks. However, several weaknesses were highlighted by reviewers:
1. The Related Work section is missing from the main paper, which hampers readability and understanding.
2. The interpretability is limited to feature-level contributions, which can be achieved by alternative methods like attention maps.
3. The role of decision trees and their influence in the encoder (e.g., thresholds in the PLE) is unclear and insufficiently discussed.
4. The evaluation is conducted on a limited number of datasets, which raises concerns about the robustness of the findings.
5. Theoretical analysis is limited, which undermines the reliability and scientific depth of the proposed approach.

The final decision is reject based on the following primary reasons: A lack of detailed analysis explaining why the proposed model provides good interpretability, such as comparisons to similar architectures. Insufficient evaluation on diverse datasets, which impacts the generalizability of the results.

Interpretable methods for tabular models are an important research direction, and we encourage the authors to address these concerns in future revisions to strengthen their work.