Predicting Language Models’ Success at Zero-Shot Probabilistic Prediction (2025)

Abstract: Recent work has investigated the capabilities of large language models (LLMs) as zero-shot models for generating individual-level characteristics (e.g., to serve as risk models or augment survey datasets). However, when should a user have confidence that an LLM will provide high-quality predictions for their particular task? To address this question, we conduct a large-scale empirical study of LLMs’ zero-shot predictive capabilities across a wide range of tabular prediction tasks. We find that LLMs’ performance is highly variable, both on tasks within the same dataset and across different datasets. However, when the LLM performs well on the base prediction task, its predicted probabilities become a stronger signal for individual-level accuracy. Then, we construct metrics to predict LLMs’ performance at the task level, aiming to distinguish between tasks where LLMs may perform well and where they are likely unsuitable. We find that some of these metrics, each of which are assessed without labeled data, yield strong signals of LLMs’ predictive performance on new tasks.

## introduction there is increasing interest in using large language models (llms) as predictive models, leveraging the world knowledge encoded by their pretraining corpora to make zero-shot predictions in domains without any labeled data. while this predictive capability was first investigated for traditional tasks within natural language processing (nlp), such as text classification or question-answering @xcite , recent work has utilized llms as predictive models in a broader sense. for instance, llms have been used to provide medical risk scores @xcite , predict fraud risk in financial applications @xcite and impute unsurveyed fields in social science surveys @xcite . more generally, llms can effectively consume text serializations of tabular data; the prevalence of tabular data across many domains likely contributes to this increasing interest across application areas. these applications differ from traditional text-based tasks @xcite because the label is not determined fully by the input: people with identical features may have different outcomes. we refer to tasks with this property as probabilistic prediction, and the predicted probabilities from the llm as risk scores. while the zero-shot prediction capabilities of llms offer exciting opportunities to scientists and practitioners, it is likely (as we empirically verify) that llms' performance varies widely across settings. then, how can practitioners tell whether an llm will perform well as a predictive model, prior to observing labeled data? this is a question with no easy answer. the appeal of using a pretrained model in many domains lies in avoiding the cost of collecting labeled data. however, validating conclusions from foundation models without labeled-data confirmation is far from straightforward. this challenge is especially pronounced in the fully zero-shot case, where users lack access to ground-truth labels altogether. we distinguish performance at two levels of granularity: at the individual level, referring to which examples an llm is likely to predict accurately, and at the task level, referring to which overall prediction problems, defined by a dataset and outcome variable, the llm is likely to perform well on. the ability to quantify uncertainty at both levels allows practitioners to judge which individuals and overall predictive tasks may result in inaccurate predictions. previous work has primarily studied uncertainty at the individual level, finding mixed results. abstention methods use measures of individual-level confidence to flag dubious predictions that should be examined manually by a human expert, or ignored altogether @xcite . however, both answer-token probabilities and verbalized confidence scores from llms have been found to be badly calibrated for probabilistic prediction @xcite and also for a variety of question-answering tasks @xcite , typically due to overconfidence. despite this, multiple approaches train a post-processing step to improve calibration using only the outputs or last-layer representations of models @xcite . confidence scores have also been found to be useful in conformal prediction frameworks @xcite , suggesting that they can be post-processed to yield informative decisions about when to provide specific information. analogously, practitioners may wish to know whether a task is likely suitable for an llm before using its outputs, via some metric of uncertainty at the task level. yet, to our knowledge, no previous work considers uncertainty quantification at the task level, at least in the context of probabilistic prediction. this presents a significant challenge, as in many real-world scenarios, practitioners would benefit from heuristics to assess whether llms will perform well a priori. however, doing so typically requires labeled data-a costly resource that pretrained models are meant to help avoid. in this work, we conduct a large-scale empirical study on the performance of llms for probabilistic prediction on 316 tasks across 31 tabular datasets. the primary question we ask is: given only unlabeled data, is it possible to anticipate how well the model will perform on a zero-shot prediction task? we provide the first empirical evidence using task-level strategies to assess signals of llm performance across prediction tasks. additionally, we provide more nuanced results about individuallevel uncertainty quantification; previous results on llm calibration for probabilistic prediction @xcite are restricted to data from the us census while we employ a much larger number of tabular datasets across many subject areas. our empirical study reveals several findings that can inform how llms are employed and evaluated in predictive settings: our results provide a pathway towards more rigorous decisions about which tasks and individual instances are appropriate for llms. elicited confidence scores from llms: llm predictions on tabular data can suffer from pretraining-induced biases @xcite , and their uncertainty estimates are often poorly calibrated @xcite . methods like multicalibration and prompt-based scoring @xcite aim to improve calibration. in contrast to prior work, we primarily study uncertainty estimation at the task level. en route, we also provide a more nuanced picture of individual-level uncertainty on a wider range of tasks than previous work. ## methods we describe our experimental setup, the problem of predicting llm performance, and the set of proxy methods that we assess for performance prediction. ## results our analysis is structured as follows. we begin by examining the zero-shot classification perfor-mance of the llms on our curated datasets, with a focus on the quality of individual-level predictions. we then broaden the scope to analyze the predictability of aggregate-level llm performance across datasets. results are shown for a spread of tested llms, with the full scope of results added to the appendix. llms have significant spread in their prediction capabilities, both across datasets and across prediction tasks from the same dataset. in figures 1a and 1b, we observe that both gpt-4o-mini and llama-3.1-8b-instruct have nontrivial zeroshot predictive capabilities, with a median auc of 0.7232 for the gpt-4o-mini and 0.7080 for the llama model. the range is wide, with aucs above 0.9 for some tasks, but at near-random (or worse than random) levels for others. this confirms that practitioners must take steps to assess the appropriateness of llm zero-shot inference for a given task. see appendix a.2 for a full set of auc and ece scores over all datasets and llms. ## conclusion while the zero-shot prediction capabilities of llms offer exciting opportunities, it remains unclear how to reliably employ llm predictions without validating their outputs on labeled data. we conduct a large-scale empirical study across 316 prediction tasks to explore whether llms can serve as reliable zero-shot predictors across a diverse collection of tabular classification tasks. we introduce eight novel task-level metrics for better estimating the llms' confidence in the prediction task. our findings indicate that performance is highly variable even within individual datasets, so success 0.00 0.25 0.50 0.75 1.00 probability 0.0 0.2 0.4 0.6 0.8 1.0 cdf (a) 0.00 0.25 0.50 0.75 1.00 probability 0.0 0.2 0.4 0.6 0.8 1.0 cdf (b) figure 11: cdfs of the 10 highest (blue) and lowest (red) predicted aucs over prediction tasks by xgboost, using 201 percentile values along with standard deviation of risk scores to predict auc. we observe clear trends within llms-for gpt-4o-mini (a), bimodal distributions of risk scores correlate with high xgboost predictions, whereas for llama (b), distributions encompassing a wide range of probabilities correlate with high predictions. at one task is no guarantee of success at other tasks on similar data. instead, measuring the distribution of risk scores for a new task yields both heuristics as well as more sophisticated models that capture a strong signal about the llms' performance on that task. however, enough variance in performance remains that such predictions of performance should be seen more as a way to prioritize more promising tasks or screen out ones with a low likelihood of success, not a substitute for eventual validation on labeled data for consequential applications.