Quantifying Prediction Consistency Under Model Multiplicity in Tabular LLMs

Authors: Faisal Hamman, Pasan Dissanayake, Saumitra Mishra, Freddy Lecue, Sanghamitra Dutta
Primary area: alignment, fairness, safety, privacy, and societal considerations
Keywords: trustworthy large language models, Reliable machine learning, tabular data, model multiplicity, high-stakes application

Abstract:
Fine-tuning large language models (LLMs) on tabular data for classification can lead to the phenomenon of \emph{fine-tuning multiplicity}, where equally well-performing models make conflicting predictions on the same input. Fine-tuning multiplicity can arise due to variations in the training process, e.g., seed, random weight initialization, retraining on a few additional or deleted data points. This raises critical concerns about the robustness and reliability of Tabular LLMs, particularly when deployed for high-stakes decision-making, such as finance, hiring, education, healthcare, etc. This work formalizes the unique challenge of fine-tuning multiplicity in Tabular LLMs and proposes a novel measure to quantify the robustness of individual predictions without expensive model retraining. Our measure quantifies a prediction's robustness by analyzing (sampling) the model's local behavior around the input in the embedding space. Interestingly, we show that sampling in the local neighborhood can be leveraged to provide probabilistic robustness guarantees against a broad class of equally-well-performing fine-tuned models.  By leveraging Bernstein's Inequality, we show that predictions with sufficiently high robustness (as defined by our measure) will remain consistent with high probability. We also provide empirical evaluation on real-world datasets to support our theoretical results. Our work highlights the importance of addressing fine-tuning instabilities to enable trustworthy deployment of Tabular LLMs in high-stakes and safety-critical applications.

Decision: reject
Reviewer scores: 5, 3, 1, 8

Reviews and discussion:
<thread>
creation date:
1729331379264

thread_type:
official_review

<thread_object>
creator: reviewer_dhwf

<content>
summary:
This paper addresses the challenge of multiplicity in fine-tuning LLMs (i.e., fine-tuning under different conditions), particularly when applied to tabular predictions. While these models may achieve comparable average performance, they can generate conflicting predictions for the same inputs. To address this, the paper introduces a consistency measure that evaluates model behavior in the embedding space, focusing on the local region around the input. Additionally, the paper provides theoretical guarantees for the metric and supports it with empirical validation.

strengths:
Originality: The paper introduces the concept of fine-tuning multiplicity in LLMs applied to tabular data. The problem is interesting and possibly unexplored.  A key originality is the proposed consistency measure which offers a novel way to assess prediction robustness without requiring the retraining of multiple models.

Quality: The paper is well written and motivated with the proposed metric supported by both theoretical analysis and empirical evidence.

Clarity: The paper is clearly written, effectively contextualizing the problem within the scope of tabular data.

Significance: The issue addressed is highly relevant when using LLMs for tabular data, especially in terms of evaluating prediction robustness. The proposed metric tackles a critical challenge in ensuring the reliability of LLM-based classification systems for tabular data.

The significant ablations to understand the different dimensions is a particular strength

weaknesses:
- The datasets assessed are more conventional tabular datasets (<50 features). It would be useful to understand if the findings translate to higher dimensional tabular datasets often found in genomics and physics. Examples suggested are like the Higgs (physics) and TCGA (genomics) datasets. This is especially relevant of how this dimensionality interplays with the embedding space.

- The authors are encouraged to contrast a bit more explicitly to other related approaches in which multiplicity and consistency can be assessed --- e.g. ensemble based methods

questions:
- How would results translate on high dimensionality data? -Examples suggested are like the Higgs (physics) and TCGA (genomics) datasets.

- The authors mention an easy extension of the theory to multi-class problems. Do the empirical results hold beyond binary on multi-class tabular datasets or are there any nuances? It would be interesting to understand in terms of the methods generality.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730650748708

thread_type:
official_review

<thread_object>
creator: reviewer_yyaf

<content>
summary:
The paper aims to develop a consistency measure for evaluating prediction robustness under model multiplicity in fine-tuned Tabular Large Language Models (LLMs). The authors claim that this measure provides probabilistic guarantees that predictions will remain stable over a range of equally well-performing fine-tuned models, without needing extensive retraining. However, the main result is incorrect, and the result (except for experimental part) is irrelevant to "tabular LLM".

strengths:
The paper brings forward the important problem of efficiently measuring prediction robustness in tabular LLM finetuning.

weaknesses:
- The introduced “consistency” term lacks meaningful interpretation. Although it is intended to indicate prediction robustness across models, Theorem 1 does not support this interpretation. The result (even if correct) says if a data point has a sufficiently high consistency score with respect to a random model, the prediction of another random model from the same broad class of fine-tuned models will be at least the level of consistency (meaning that it is approaching class 1) with high probability. But what if the underlying class label should be 0?

- The paper’s theoretical formulation has significant issues. It lacks a clear definition of “random functions” and assumes that “functions are independent and identically distributed (i.i.d.)” without providing a rigorous explanation. These assumptions are unconventional and do not align with standard definitions typically found in probability theory.

-  The consistency measure itself is random, as it depends on sampled points, and its properties would only stabilize as the sample size k approaches infinity. This suggests that further study should focus on the properties of the deterministic limit of the measure rather than its finite-sample properties.

- Assumption 1 is trivial. If F, F' are bounded functions, certainly the variance of Z is bounded.  
 
- The paper claims "while model multiplicity in machine learning has been studied in various contexts, the unique challenges of fine-tuning multiplicity in Tabular LLMs remain relatively unexplored."  While the paper claims to address fine-tuning multiplicity specifically for tabular LLMs, the main results are general and do not leverage any unique characteristics of tabular data or LLMs. The main result is irrelevant to the paper scope.

questions:
Pls see my comments and suggestions above.
</content>
</thread_object>

</thread>
<thread>
creation date:
1730721617166

thread_type:
official_review

<thread_object>
creator: reviewer_szsk

<content>
summary:
The work studies the problem of model multiplicity in the application of LLMs to tabular data. Model multiplicity refers to the phenomenon that multiple models of similar accuracy assign confliciting predictions to individual instances. The authors propose a measure of model multiplicity called *consistency*, provide a theoretical observation about it, and evaluate it in four standard tabular datasets.

strengths:
Model multiplicity is an interesting problem. It's certainly worth studying it in the context of tabular datasets, and with regard to models resulting from multiple finetuning runs on the same data.

weaknesses:
(1) 

The proposed measure of consistency has a few different issues that prevent it from being an informative measure in my opinion.

(a) The biggest issue with it is that it is apparently unable to identify constant predictors adequately. Constant predictors, by definition, cannot exhibit model multiplicity. Unfortunately, the measure is sensitive to the value of the constant: A constant 0 predictor would have consistency 0, whereas a constant 1 predictor would have consistency 1. Note that 0 is the smallest possible value consistency can take on, while 1 is the largest possible value. In other words, two equivalent predictors can have maximal or minimal consistency.

This shows why we cannot read anything into a particular value of consistency. The authors write: "We desire that the measure S(x, f) should be high if the input x is consistent across this broad class models." Unfortunately, even if S(x, f) = 0, it could be that the model output is perfectly "consistent" as the example of the constant 0 predictor shows.

(b) The measure S(x, f) is a random variable that depends on the number of sample points and the radius of the sphere. It's not clear how sensitive the measure is to these design choices. This raises the quesiton how conflicting values of sensitivity should be reconciled.

(c) There is no reason that the model family should respect the Euclidean geometry implicit in the definition of the consistency measure. This gives additional reason to believe that the measure cannot be useful across all model families.

In light of these issues, I prefer the definitions from prior work. For example, the measure of pairwise disagreement by Black et al. (2022) already gets around the above issues. 

(2)

As a result of the definitional problems with consistency, I find the suggested main "theorem" to be less than helpful. Let's even take aside the fact that it's a direct application of the assumptions and Bernstein's inequality. It's not clear what we can read into. Again, the constant 0 predictor gives the same output to any input, but the theorem would only say that Pr(F(x) > -\epsilon) is large. This is, of course, a vacuous statement.

(3)

The selection of datasets is rather disappointing. Although standard, these four datasets are very small, outdated, and problematic for different reasons. This is especially unfortunate, given the wide availability of many better tabular datasets. For example, the tableshift project has more than a dozen useful tabular datasets:

https://github.com/mlfoundations/tableshift

Likewise, the folktables package gives easy access to a plugin alternative and extension to the Adult dataset:

https://github.com/socialfoundations/folktables

There is really no good reason for the empirical evaluation to be so limited.

questions:
Could you please comment on the issues with the definition?

What are the advantages of consistency compared with existing measures?

Do you have results on other/larger datasets? If not, why not?
</content>
</thread_object>

</thread>
<thread>
creation date:
1731123241506

thread_type:
official_review

<thread_object>
creator: reviewer_zzss

<content>
summary:
The paper studies the fine-tuning multiplicity in tabular LLMs, where different settings, such as varying random seeds, lead to different models and correspondingly different predictions for a given sample or dataset. The authors propose a measure called consistency, which is based on the model’s local behavior around the input. Asymptotic guarantees for consistency are provided, and an experimental comparison with other model multiplicity metrics (e.g., arbitrariness, discrepancy, pairwise disagreement), as well as predictive probabilities and dropout methods, is conducted.

strengths:
The paper is clearly written and easy to follow.

The authors provide an extensive literature review.

The authors consider the interesting problem of uncertainty quantification for tabular LLMs and propose using a relatively inexpensive measure to assess it.

A comparison is made with various methods of computing model multiplicity.

weaknesses:
1. Could you please provide more background and intuition behind the consistency measure? Essentially, it is not clear why the choice of the $f(x_i) - |f(x)-f(x_i)|$ metric was made and what it means locally for a given sample, as well as globally when averaged for a given model. It is also unclear why the discrepancy in the metric $f(x)$ vs. $2f(x_i)-f(x)$ occurs, given the sign on the absolute value.

2. Consistency, as designed, is a measure of robustness or stability, and more comparison or motivation on how consistency differs from existing robustness methods should be provided (there exist different measures, just as an example please see (1) and (2)). Further, while there is a high rank correlation with model multiplicity metrics, more discussion is needed on why a local robustness metric is a good approximation of model multiplicity.

3. Theorem 1 provides interesting asymptotic guarantees. However, as the authors mentioned, it is challenging to use in practice. Specifically, it is not clear whether the current consistency value is high or low for the theorem to apply. To provide more information on the usefulness of the theorem or $k$, is it possible to estimate $\beta$ for some simpler hypothesis spaces, such as linear models?

4. Tables 1 and 2 in the main paper, as well as other tables in the Appendix, do not contain confidence intervals (either computed or estimated). Therefore, it is difficult to assess the robustness of the presented values and compare them across datasets and metrics.

5. Methods that enumerate all models in the Rashomon set for different hypothesis spaces, such as RashomonGB, TreeFarms, and CorelsEnum, can also be added to the literature review for completeness.

(1) J. Djolonga, F. Hubis, M. Minderer, Z. Nado, J. Nixon, R. Romijnders, D. Tran, and M. Lucic. Robustness Metrics, 2020.

(2) Tessa Han, Suraj Srinivas, Himabindu Lakkaraju. Characterizing Data Point Vulnerability as Average-Case Robustness, 2024.

questions:
1. A lot of the model multiplicity metrics used in the paper are label-based. Does the choice of the threshold of 0.5 influence the rank correlation between consistency and these metrics?

2. Do you have any intuition as to why the rank correlation is low between consistency and arbitrariness/pairwise disagreement for the German dataset?
</content>
</thread_object>

</thread>
<thread>
creation date:
1733194345601

thread_type:
official_comment

<thread_object>
creator: authors

<content>
title:
Application of Measure in Across Various Predicted Classes

comment:
We would like to clarify that our consistency measure can be applied to any prediction class in multi-class classification. Our consistency measure is tied to **the confidence in predicting a specific class** (i.e., the probability values derived from softmax logits), and not directly to the predicted label 1. **In our experiments, we already include results for both class 0 and 1**.

For an intuition, let us consider a binary classification scenario with two classes, A and B, and corresponding softmax logits $ f_A(x) $ and $ f_B(x) $, such that $ f_A(x) + f_B(x) = 1 $. 

The consistency measure can be defined for class A as $ S(x, f_A) $ and for class B as $ S(x, f_B) $. The former denotes the consistency for the prediction of A. So if $ S(x, f_A) $ is high, it means that across many finetuned models, the logit for class A is also likely to be high (as per our theoretical guarantee). Similarly,  $ S(x, f_B) $ is the consistency for the prediction of B. So if $ S(x, f_B) $ is high, it means that across many finetuned models, the logit for class B is also likely to be high (as per our theoretical guarantee). 

However, both $ S(x, f_A) $ and $ S(x, f_B) $  are not going to be very high together because of their complementary nature, so both of them are not going to be informative. **Like if  $ S(x, f_A)  = 0.5+\epsilon$ and $ S(x, f_B) = 0.2+\epsilon$, we know the logit for class A will likely be higher than 0.5 and predictions of A will be more consistent. We also know that the logits for class B will be higher than 0.2 though this may not be as informative for us**. 

This complementary nature of the measure ensures that the consistency score for the chosen class aligns with its predicted confidence. For example, just as the predicted probability of the chosen class is given by $ \max(f_A(x), f_B(x)) $, the consistency score of the chosen class can similarly be expressed as $ \max(S(x, f_A), S(x, f_B)) $ which is what we do in the experiments.

Given $S(x, f_A)$ for one class, there is a one-to-one relationship with $S(x, f_B)$ for the other class. We can think of $f$ as a vector of softmax probabilities, e.g., $f = (f_A, f_B)$, and the consistency measure reflects the robustness of this model as a whole. When $f$ has low entropy (i.e., one class has high confidence), $S(x, f)$ is highly informative. Conversely, when $f$ has high entropy (i.e., predictions are distributed more evenly across classes), $S(x, f)$ is less informative, as expected. The measure inherently captures a property of the model, not the individual classes in isolation. For three or more classes, the same principles apply.
</content>
</thread_object>

</thread>


Metareview:
metareview:
Based on the reviews, I conclude that the paper cannot be accepted for publication in its current form. Of the four reviews, three recommend rejection. The reviewers raised significant concerns about key aspects of the work, particularly questioning the validity of the notion of 'consistency' and highlighting limitations in the empirical evaluation.