Probabilistic Pretraining for Improved Neural Regression
Authors: Boris N. Oreshkin
Shiv Kumar Tavker
Dmitry Efimov
Venue: Accepted by TMLR ()

Abstract: While transfer learning has revolutionized computer vision and natural language processing, its application to probabilistic regression remains underexplored, particularly for tabular data. We introduce NIAQUE (Neural Interpretable Any-Quantile Estimation), a novel permutation-invariant architecture that enables effective transfer learning across diverse regression tasks. Through extensive experiments on 101 datasets, we demonstrate that pre-training NIAQUE on multiple datasets and fine-tuning on target datasets consistently outperforms both traditional tree-based models and transformer-based neural baseline. On real-world Kaggle competitions, NIAQUE achieves competitive performance against heavily hand-crafted and feature-engineered solutions and outperforms strong baselines such as TabPFN and TabDPT, while maintaining interpretability through its probabilistic framework. Our results establish NIAQUE as a robust and scalable approach for tabular regression, effectively bridging the gap between traditional methods and modern transfer learning.

Published in Transactions on Machine Learning Research (02/2026)
Probabilistic Pretraining for Improved Neural Regression
Boris N. Oreshkin
oreshkin wreshhinn@amazoonccoo
SCOT Forecasting
Amazon Science
Shiv Tavker
tooker@amazzon
Pricing and Promotions
Amazon Science
Dmitry Efimov
deffmoonoomazon.com
SCOT Forecasting
Amazon Science
Reviewed OpenReview: https: httpe://openrevieel
openreview. net/ forum? id= F6BTATGXX
Abstract
While transfer learning has revolutionized computer vision and natural language process- ing, its application to probabilistic regression remains underexplored, particularly for tabu- lar data. We introduce NIAQUE (Neural Interpretable Any-Quantile Estimation), a novel permutationn-nnvaianl architecture that enables effective transfer learning across diverse regression tasks. Through extensive experiments on 101 datasets, we demonstrate that pre-training NIAQUE on multiple datasets and fino-tuning target datasets consistently outperforms both traditional tree-based models and transformer-based neural baseline. On real-world Kaggle competitions, NIAQUE achieves competitive performance against heav- ily hand-crafted and festure-engineered solutions and outperforms strong baselines such as TabPEN and TabDPT, while maintaining interpretability through its probabilistic frame- work. Our results establish NTAQUE as a robust and scalable approach for tabular regres- sion, effectively bridging the gap between traditional methods and modern transfer learning.
1 Introduction
Tabular data underpins high-stakes decision-making across healthcare (Rajkomar al. 2018), real estate valuation (De Cock 2011), energy systems (Olson et al., 2017a),, and e-commerce (McAuley et al. 2015
In these settings, models must. be accurate, data-ellicient, and, crucially for operational use, able to quantilyy predictive uncertainty. Historically, tree-based ensembles such as Random Forests (Breiman 2001 XG-
Boost (Chen & Guestrin 2016), LightGBM (Ke et al. 2017). and CatBoost (Prokhorenkova et al. 2019
have been the de facto standard due to their robustness, case of use, and strong performance with heteroge- neous features and modest sample sizes. Recent work has revisited deep learning for tabular data, motivated by the promise of end-to-end representation learning, joint optimization with downstream objectives, and straightforward multimodal fusion. Architectures such TabNet (Arik & Pfister 2021 and Tab Transformer (Huang et al. 2021) demonstrate that appropriately constrained inductive biases (e.g., learned feature selec- tion, attention over categorical embeddings) can close the gap to boosted trees on many benchmarks while enabling capabilities such as differentiable feature learning and integration with text or images. Yet, unlike computer vision and natural language processing, where transfer learning via large-scale pretraining across heterogenous real datasets now foundational, the corresponding paradigm for tabular data remains com- paratively exde-eepllred, particularly for probabilistic regression tasks that require calibrated predictive distribution rather than point estimates (Hollmann et. al. 2023 Levin et al. 2023

---

Published in Transactions on Machine Learning Research (02/2026)
Challanges. This work addresses several fundamental challenges in tabular regression. Transfer Learning Gap: While recent, work has explored transfer learning for tabular classification, probabilistic regression remains underexplored, lacking frameworks that effectively transfer knowledge across diverse regression tasks. Seababiittyhateerpettbbbiii Trade-off: Existing approaches often sacrifice either scalability or interpretability, limiting their practical utility in real-world applications requiring both robust performance and explainable predictions. Benchmark Limitations: Current multi-dataset tabular benchmarks prodominantly focus on classification tasks, hindering systematic evaluation of regression models.
Our approach: NIAQUE. We introduce NTAQUE (Neural Interpretable Any-Quantile Estimation), a probabilistic regression model designed to resolve these challenges. First, NIAQUE enables effective co- training across mullippe disjoint dalasels, exhibiting positive transfer and strong scalability when fine-tuned new regression tasks. Second, its probabilistic framework supports interpretable analysis via marginal posterior distributions that yield feature-level importance summaries, enhancing transpareney and reliability in decision-critical settings. Third, we provide theoretical guarantees showing that, under appropriate reg- ularity conditions, NIAQUE approximates the inverse of the posterior distribution, thereby formalizing its ability to recover coniicondiiional quantiles. To enable comprehensive assessment, we introduce a new multi- datasct regression benchmark comprising 101 diverse datasets spanning multiple domains. We evaluate with proper scoring rules (e.g.: CRPS), calibration diagnostics, and standard accuracy metrics. Across this suile, NIAQUE outperforms strong tree- baselines and neural approaches while maintaining interpretabil- ity. Furthermore, on real-world Kaggle regression challenges, NIAQUE achieves competitive performance against highly engineered solutions. Therefore, our primary contributions are as follows.
(1) We propose a novel deep probabilistic regression model (NIAQUE) that enables effective transfer learning across diverse tabular datasets via co-training and task-adaptation. (2) We provide theoretical analysis establishing NI- AQUE's convergence inverse posterior distribution. (3) We demonstrate superior performance over strong baselines, both boosted trees and modern neural methods, in transfer learning settings on new benchmark suite for tabular regression (101 datascts).
2 Related Work
Probabilistic Regression. This work builds on probabilistic time-series modeling approach (Smyl et al, 2024), refining its theoretical underpinnings and extending architectural design for tabular transfer learning applications. Alternative methods, such as Neural Processcs (Garnelo et al., 2018b) and Conditional Neural Processes (Garnelo et al., 2018a), offer conditional probabilistic solutions to regression but are constrained to -iee-dimensional imput spaces, limiting their applicability to cross-dataset, multi-task regression. Our approach effectively transfers knowlodge across datasots with varying fcature spaces and target domains, establishing a flexible and scalable framework for conditional probabilistic regression.
Transfer Learning in Tabular Data. Transfer learning has driven major advances in computer vision 2021 Radford et al., 2021 and language modeling (Devlin et al.) 2019) by leveraging shared representations across tasks. Recent successes in time-series forecasting (Garza & Mergenthaler-Cansee 2023 Ansari et al. 2024) demonstrate transfer learning's potential for numerical prediction tasks. However, probabilistic transfer learning for tabular data remains largely unxxpooed, with existing work primarily focusing on classification tasks or point estimates. Our work bridges this gap by introducing a framework specifically designed for probabilistic transfer across diverse tabular datasets.
(Sun et al.
Deep Learning vs. Tree-based Models. The comparison between deep learning and tree-based ap- proaches for tabular data has been extensively studied, particularly for classification tasks. Previous evalua- tions include: transformer architectures across 20 classification datasets, MLPs versus TabNet and based models on 40 classification datascts, Comprchcnsive comparison of various architectures across 45 datascts GGrnnstaajn el al., 2022). Most relevant to our work are TabPFN (Hollmann et al. 2023) and TabDPT (Ma) et al. 2024), which introduce Transformer- based approaches for synthetic pre-training. While these methods share similar goals, our work differs in three key aspects. First, we propose a novel architectural approach based on deep prototype aggregation, which scales linearly in the number of input features. Second, we focus specifically on probabilistic pretraining on real public datasets and transfer learning. Finally, we demonstrate
2

---

Published in Transactions on Machine Learning Research (02/2026)
superior empirical accuracy real-world Kaggle competitions through direct probabilistic pretraining and fine-tuning on downstrean regression tasks.
Permutation-Invariiat Architectures, Our architectural design builds upon advances in permutation- invariant representations, crucial for handling variable feature spaces in multi-task learning. We extend the prototype-based architecture proposed by (Oreshkin al. 2022) for pose completion to support any-quantile modeling in tabular regression. This approach relates to several key developments in the field. PointNet (Qi et al. 2017) and DeepScts (Zaheer et al. 2017) pionecred pooling techniques for variable-dimnnsional inputs. PoinNee (Niemeyer et al. 2019) generalized these approaches with residual connections. Prototypical Networks (Snell et al.] 2017) demonstrated effectiveness of prototype-based learning by leveraging average- pooled embeddings for few-shot classification. Transformer architecture (Vaswani et al., 2017) established flexible processing of variable-length sequences.
3 Background and Problem Formulation
Let R denote the set of real numbers and 24(0, 1) the uniform distribution over the interval (0, 1). For vector X, we denote its dimensionality as |x]. For a random variable with cumulative distribution function (CDF) Fly) = P(Y S y), the g-th quantile q E (0,1) is defined as:
F (g) = inf E R: F()) 2
3.1 Problem Formulation
Let X' be the input feature space and y C R be the space of the target variable. We consider a probability distribution D over X' y. For any instance x E X, the relationship between features and target variable is given by:
Ex)
(1)
where V : X' E > ) is an unknown non-linear function and e E & represents stochastic noise with unknown distribution.
Given a finite training sample S = {x W drawn i.i.d from D, we aim to learn a probabilistic regression function fe x + RR, parameterized by E θ, which maps an input x to a Q-tuple of quantiles (q)....9Q), where q E (0, 1) for [Q), thereby capturing the conditional distribution y/x.
3.2 Performance Metrics
Let Y/ denote the ground truth sample and Øi.q its g-th quantile prediction for a dataset with S samples. To evaluate the quality of distributional predictions, we use Continuous Ranked Probability Score (CRPS). The thcoretical definition of CRPS for a predicted cumulative distribution function F and observation y
CRPS(F.V) = ((2))
2
where F: + [0,1] is the predicted CDF derived from the quantile predictions, and ((22)) is the indicator function. For practical computation with finite samples S and discrete set of Q quantiles, we approximate this using:
CRPS
2 SQ
Σ Σ
plue Di.qy)
(3)
where p(y, Fy) is the quantile loss function defined as:
ply. 0v) = (y- (g 11 (vSa.))
(4)
3

---

Published in Transactions on Machine Learning Research (02/2026)
Coverage measures empirical calibration of predictive confidence intervals. For confidence level (0, 1):
8 1
COVERAGE(a)-
Σ 1yy > 01.0.5-0/2 < 05.0.540/22 100% i=1
We employ the following metrics to evaluate the accuracy of point predictions: Symmetric Mean Absolute Percentage Error (sMAPE), Average Absolute Deviation (AAD), Root Mcan Square Error (RMSE), Root Mean Square Logarithmic Error (RMSLE). Detailed mathematical definitions of the metrics are provided in Appendix
4 NIAQUE
In this section, we present NIAQUE (Neural Interpretable Any-Quantile Estimation), probabilistic regres- sion model. We first introduce the any-quantile learning approach general solution to the probabilistic regression problem defined in Section 3 We prove that this approach converges to the inverse cumulative distribution function of the conditional distribution, providing theoretical foundation for our method. We then detail NIAQUE's neural architecture, demonstrate how it enables transfer learning across diverse tabular datasets, and present an approach to model interpretability based on probabilistic considerations.
4.1 Any-Quantile Learning
We formulate the any-quantile learning approach by augmenting the input space to include quantile level g E (0, 1), allowing the neural network fe to learn mappings from (x, q) to the corresponding q-th conditional quantile the target variable y/x. Let Dg fo(x, q) represent the predicted g-th quantile of the conditional distribution of y|x. The objective is to learn parameters 0 that minimize the expected quantile loss:
min 2... fo(x fo(x, 9)),
(5)
0
where :)::)) the quantile loss function defined in eq.
We use gradient descent and mini-batch to learn the parameters. Precisely, the neural network trained on dataset of S samples, (x4,3) drawn from the joint probability distribution D. During training the quantile value q is sampled from U(0, 1) and the loss is minimized using stochastic gradient descent (SGD). For mini-batch of size B, the parameter update at itcration k is:
1 0k+1 02 mh V
ply,, fo(x;,
(6)
B
As k > 00, the parameters converge to the solution of the following empirical risk minimization prob- lem KKrimi et al. 2016):
S
0" = arg min 8E8
fo(x,,
(7)
1=1
By the strong law of large numbers, as S grows, the empirical risk converges to the expected quantile loss:
Exply fo 4)) = Ex.y [ plu, g)(x
This expected loss has a direct connection to the Continuous Ranked Probability Score (CRPS), which can be expressed as an integral over quantile loss (Gneiting & Ranjan 2011
CRPS(F 2 f p(v. Fo..F110 (g))dg. (
(9)
Based on this fact, the following theorem proves that the expected pinball loss eq. ()(x,) corresponds to the inverse of the posterior CDF pyxx-
g minimized when
4

---

Published in Transactions on Machine Learning Research (02/2020)
Festure
Quantiee Decoder
Proto
Proto
Proto
Embeddingg
Concat
Linear
Block
Block
R
Block
Block
Quantile
Loss
Festure
Quanticc Modulation Block
log 20 +1) sgra 3
Feature Code
MLP
Linear ReLu
Linear
Linear
Linear
MLP
Linear
ReLu
ReLu
Linear
Figure 1: NIAQUE's encoder-decoder architecture transforms variable-dinensional inputs into fixed-size representations, enabling transfer learning and multi-task knowledge sharing across datasets.
Theorem 1. Let F' be probability measure over variable y such that inverse F-1 exists and let Pax be the joint probability measure of variables x,y. Then the expected loss, Ex ply, (g)), is minimized if and only if F Palx-
The following conclusions emerge. First, the quantile loss SGD update eq. optimizes the empir- ical risk eq. (7 correspouding to the expected loss eq. S Based on and Theorem [ so- arg miny, Ex.y. ply. fo(x, q)) has clear interpretation as the inverse CDF corresponding ond, both the SGD iteration index i and training sample size S increase, and if fo implemented MLP whose width and depth scale appropriately with sample size S, then [Farrell et al. 2021, Theorem
Pytx- Sec-
1) implies that the SGD solution converges to fo- (x.9) = (g) Therefore, given uniform q ~ U( 1),
io = fe- (x, q) has the interpretation of sample from the posterior distribution p(y|x), which follows from the proof of the inversion method (Devroye 1986 Theorem 2.1).
4.2 Neural Encoder-Decoder Architecture
NTAQUE adopts a modular encoder-decoder design (Fig. to process observation samples x; with variable dimensionality d,. The encoder maps each observation into fixed-size latent embedding of dimension E, enabling downstream procossing independent of input dimensionality. Feature values and associated codes (IDs) (dimension d,) are embedded into tensor of size 1 x d, x Ein, where Em is the embedding size per feature. These embeddings are aggregated using prototype-based method to generate the latent observation representation. The decoder conditions this representation on arbitrary-lengto quantile vectors q E R?, modulating the output using FiLM-bassed transformations (Perez et al. 2018 This separates input processing and quantile conditioning, achieving computational efficiency of O + Q) per sample X; compared (d,Q) complexity required process quantiles and obscrvations jointly. Note that our processing is linear feature dimensionality, opposed to quadratic scaling of attention-based approaches.
Inputs: NIAQUE processes both continuous and catcgorical features in unified manner. Categorical features are first label-encoded to integers during preprocessing. For each feature in the observation vector x, NIAQUE incorporates both its raw value and a learnable embedding based on its feature index (position in the feature vector). The feature index embodding learns fcature-specific statistical propertics, inter-feature dependencies, and their relationship with the target, enabling the model distinguish between features even when they have similar numerical values. The embedded feature index is concatenated with the feature's
5

---

Published in Transactions on Machine Learning Research (02/2026)
value after :og-transformation::
= log[aaa + +1 sgg(x))
(10)
which normalizes the features' dynamic range (including label-encoded categorical values), aligning it with that of index embeddings while preserving sign information, facilitating stable training, as validated by the ablation study in Appendix J
The encoder employs a two-loop residual network architecture to efficiently handle variable-dimensioonal imputs. The following equations define the encoder's transformations, with the sample index i omitted for brevity. Let the encoder input be Xin E IRdxF where is the number of features and Pin is the embedding size per feature. A fully-connected layer FCr,e in residual block r E {1,..., (....,RR, R}, layer E {I....,L} L), with weights Wr. and biases a,,, is defined as:
Feature Encoder:
(Cr,e( = RELURREW.... RELU(W,,,h,e-1 + ar.e).
We also define prototype layer as: PROTOTYPE(X) _ x(i, described by the following equations:
The observation encoder is then
x, RELU 1/(r- 1) Pr-1))
(11)
1 = (Xr), ..... hn = FC,, (hr,
(12)
b, = RELU(L, + h,,,,, fr F,
(13)
p. = Pr-1 PROTOTYPE(E)).
(14)
These equations implement a dual-residual mechanism. First, Equations T2 and 13) form an MLP with a residual connection (see Feature Block in Fig, | bottom left). Second, Equations [11 and (14) form a second residual loop with the following key properties: a) Eq. (14) consolidates individual feature encodings into prototype-based representation of the observation; b) Eq. 11 implements interactions between features (akin to attention, but with lincar compute cost) and introduces an inductive bias by enforcing delta- mode constraint, ensuring that feature contributions are only relevant when they deviate from the existing observation embedding, Pr-1: e) The observation representation accumulates across residual blocks eq. (14) effectively implementing skip connections.
The decoder implements fully-connected conditioned residual architecture (Fig. top-right). Its primary function is to implement the any-quantile functionality by injecting the quantile value inside the MLP block using FiLM modulation principle (Perez et al. 2018). Takiin the observation embedding bo pn E input, it generates quantile-modulated representations fR RQxE for quantiles q ? through:
Quantile Decoder:
h QD (b,-1), Br LINEAR, (q)
h = FC ((1 + +7) h,, + B,),
(15)
b = FC (b,,
b, = RELUULL (2 + h, = + FQD h
The final prediction 8, R is obtained via linear projection: G, LINEAR[(.].
4.3 Interpretability
NIAQUE's probabilistic framework facilitates interprotability via quantile predictions conditioned on indi- vidual features. Given fo(x.:g) as NIAQUE's estimate of quantile q using only feature x,, the posterior confidence interval for this feature is defined as:
Cla, fo(xe, 1- aa2)) (22))fo(xxe axx))
(16)

---

Published in Transactions on Machine Learning Research (02/2026)
where 1 - represents the probability that the target true value lies within the interval. Intuitively, more informative features produce narrower confidence intervals. We leverage this to quantify feature importance through normalized weights:
W. Σ, W.
1 E fo(xs,1110 - fo(xx,,. 8
W
W,
Clas
(17)
Clo.95,
where Cla,a is the average confidence interval width over validation samples {x4 : x, Dval). To enhance marginal distribution modeling and support interpretability, we introduce single-feature samples during training, comprising approximately 5% of the dataset. An ablation study in Appendix J confirms the necessity of this augmentation for robust feature importance estimation.
4.4 Transfer Learning
NIAQUE facilitates effective transfer learning through two mechanisms ensuring effective knowledge transfer across diverse tabular regression tasks with varying feature spaces. First, its feature ID embeddings learn both -ataset-specific and cross-dataset relationships, as evidenced by the structured representation space observed in Fig. 3 Second, its prototype-base based aggregation enables flexible processing of arbitrary feature combinations, inherently supporting cross-dataset learning.
The transfer learning process consists of two phases. During pretraining, NIAQUE learns from multiple heterogeneous datasets simultaneously, sampling rows uniformly at random from all datasets, with each dataset contributing only its relevant features. The model processes these diverse inputs through shared parameters, learning both task-specific characteristies and generalizable patterns. The learned embeddings capture statistical properties at multiple levels: individual feature distributions, feat ure interactions within datasets, and common patterns across different regression problems. This pretrained knowledge can then leveraged in two ways. First, transfer across seen datasets, in which co-training on multiple datasets enables the learning of larger-capacity networks using data diversity as a regularizer, while also leveraging commonalities across classes of problems-immliciit implicitly extracted during training to strengthen performance related seen tasks. Second, few-shot transfer to unseen datasets through fine-tuning, where the pretrained feature representations provide a strong foundation for learning new tasks with minimal data via strong regression prior stored in pretrained model weights. In both cases, the model learns to distinguish and process datasct-spocific fcatures through their somantic embeddings, whereas shared model paramcters enable knowledge transfer across related datasets via multi-lask training.
The effectiveness of this approach stems from NIAQUE's ability maintain dataset-specific information while learning transferable representations. Learnable feature ID embeddings act as task identifiers, allowing the model to adapt its processing based on the combination of inpul features, elfectively serving as an implicit task ID. This capability is particularly valuable in tabular domains, where feature relationships and their predictive power can vary significantly across tasks. Our experimental results validate both the efficacy of cross-dataset pretraining and the model's adaptability to novel regression tasks via fine-tuning, establishing NTAQUE as a robust framework for transfer learning in tabular domains.
5 Empirical Results
We conduct extensive experiments to evaluate NIAQUE's effectiveness for transfer learning in tabular re- gression. Our evaluation addresses three key aspects: 1) the model's transfer lcarning capabilitios across both seen and unseen tasks, 2) the quality learned representations and interpretability, and 3) its practical effectiveness real-world competition setting.
5.1 Datasets and Experimental Setup
We introduce TabRegSet-100 (Tabular Regression Set 101), a curated collection of 101 publicly available regression datascts gathered from UCT (Kelly et al. 2017). Kaggle (Kaggle 2024), PMLB (Romano et al. 2021 Olson al. (0177b), OpenML (Vanschoren et al. 2013), and KEEL (Alcalá Fdez et al. 2011). These datasets span diverse domains, including Housing and Real Estate, Energy and Ffficiency,
Dataset and Evaluation:
7

---

Published in Transactions on Machine Learning Research (02/2020)
(a) Dataset Sources
(b) Dataset Size
(c) Dataset Width
25
OPENML
30
KAGGLE
20:
29%
2,3%
20
Count
15
5%
5%
KEEL
15
10
21%
17%
DCC
10-
PMLB
5-
5-
UCI
5000
10000 15000
20000
20
40
60
Number of Samples
Number of Variables
Figure 2: Statistics of the evaluation datasct: (a) distribution by source, (b) dataset sizes, and (c) feature counts.
Retail and Sales, Computer Systems, Physics Models and Medicine and exhibit different characteristics in terms of sample size and feature dimensionality. We focus specifically on the regression task in which the target variable is continuous or, if it has limited number of levels, these are ordered such as student exam scores or wine quality. The target variable in each dataset is normalized to the [0, 10) range and the independent variables are used as is, raw. The target variable scaling is applied to equalize the contributions of the evaluation metrics from each dataset. Datasets have variable mumber of samples, the lowest being just below 1000. For very large datasets we limit the number of samples used in our benchmark to be 20,000 by subsampling uniformly at random. This allows us (i) to model imbalance, and at the same time (ii) avoid the situation which a few large datasels could completely dominate the training and evaluation of the model. The distribution of datasets by source, number of samples and variables is shown in Figure 2 The datasets, along with their sample count, number of variables and source information are listed in Appendix
Baselines: We compare NIAQUE (Lotal parameters 28M) against Tree-based models: XGBoost (Chen & Guestrin 2016). LightGBM (Ke et al. 2017), and CatBoost (Prokhorenkova et al. 2019): ; and Deep lcarning: Transformer encoder-docoder with NIAQUE quantile decoder (total parameters ~66M details in Appendix and FT-Transformer (Gorishniy et al. 2021), using feature-wise tokenization with attention mechanism (total parameters ~12M). The model trained with multi-quantile loss. We evaluate three training scenarios: a) Global models (denoted by the -Global suffix): trained jointly on all datasets; b) Domain-specific models (suffix -Domain): trained on datasets from the same domain (e.g., housing, medical); c) Local models (suffix -Local): trained individually per dataset. XGBoost and CatBoost are trained using multi-quantile loss with fixed quantiles, with additional quantiles obtained through lincar interpolation. GBM is trained with separate models per quantile. For training global tree-based models, we construct a unified table containing samples from all datasets, filling missing features with NULL values.
H
Implementation Details: NIAQUE (Total Parameters ~ 27.6M) uses encoder and docoder containing residual blocks, 2 layers each with latent dimension E 1024 and input embedding size E., 64. Training uses Adam optimizer with initial learning rate 10 and batch size 512. The learning rate is reduced by 10x at 500k, 600k, and 700k batches. We apply feature dropout with rate 0.2. Hyperparameters are selected using validation split and metrics are computed on the test split. Training requires approximately 24 hours for NIAQUE and 48 hours for Transformer on V10000 GPUs. In comparison, XGBoost training takes about 30 minutes on a one V100 for 3 quantiles, scaling linearly with the number of quantiles. All models are trained on the same train splits, with samples drawn uniformly at random across datascts. During training, quantile values are randomly generated for each instance in batch.
5.2 Results
We evaluate NIAQUE's cross-datasct learning capabilitics through following complementary analysos: (i) large-scale multi-dataset co-training, (ii) domain-specific co-training and (iii) transfer of pretrained model to unseen tasks via fine-tuning.
8

---

Published in Transactions on Machine Learning Research (02/2026)
Table 1: Performance comparison, co-training across all 101 datasets. Lower values are better for all metrics except COVERAGE @ 95 (target: 95). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
XGBoost-Global
31.4
0.574
1.056
-0.15
94.6
0.636
XGBoost-Local
25.6
0.433
0.883
-0.03
90.8
0.334
LighrGBM-GGloall
27.5
0.475
0.930
-0.06
94.8
0.426
LigLoGBM-LLocl
25.7
0.427
0.865
-0.03
91.5
0.327
GatBoost-Global
31.3
0.561
1.030
-0.12
94.9
0.443
LotBoost-Local
24.3
0.408
0.840
-0.03
92.7
0.315
Transformer-Locs
26.9
0.462
0.904
-0.05
93.6
0.329
Transformer- Global
23.1
0.383
0.806
-0.01
94.6
0.272
FT- Tra-sssfoormme-oaalll
25.1
0.420
0.858
-0.04
94.2
0.298
-ro-raansormmee-llbbal
22.6
0.375
0.796
-0.02
94.7
0.266
NIAQUE-Local NTAQUE-Global
22.8
0.377
0.797
-0.03
94.9
0.267
22.1
0.367
0.787
-0.02
94.6
0.261
Cross-Dataser Learning. To evaluate NIAQUE's ability to handle large-scale multi-dataset learning, we conduct experiments across all 101 datasets simultaneously. Table [ presents results, aggregated across datasets al sample level, comparing global and local training scenarios for various models. NTAQUE-Global achieves the best performance, outperforming both traditional based methods and the Transformer bascline. Notably, while tree-basod methods show better performance local training compared to thoir global variants (e-g., LatBoost-Looal SMAPE: 24.3 HalBooss-Globale 31.3), NIAQUE maintains superior performance in both scenarios, with its global model outperforming its local counterpart. Furthermore, NIAQUE maintains reliable uncertainty quantification across all scenarios, with coverage staying close to the target 95% level and consistently lower CRPS values compared to baselines. These results confirm NI- AQUE's capacity to leverage cross-dataset learning effectively, maintaining or even improving performance on individual tasks through robust feature representations that generalize across diverse datasets and do- mains. This experiment also shows the principal inability of tree-based models to operate ross-datases learning scenarios, emphasizing their inability to develop joint representations across heterogeneous prob- lems. For example, NIAQUE beats CatBoost-Global more than 90% of the datasets, according to the detailed per-dataset performance breakdowns provided in Appendix D.I.
Domain-Specifec Results (Tables 107 and 108 in Appendix D.2) focus on House Price Prediction and Energy and Efficiency domains, showing the ability of the model to effectively leverage additional information from domain-specific datasets to improve on target task.
Adaptation to New Tasks on TabRegSet-1011 To evaluate NIAQUE's transfer learning capabilities unseen tasks, we randomly split our collection of 101 datascts into 80 pretraining datasets and 21 held- out test datasets. We compare two scenarios: training from scratch (NIAQUE-Scratch)) and fine-tuning pretrained model (NTAQUF-Pretrran) then fine-tuned on each held-out dataset using a 10 times smaller learning rate. To assess the impact of data scarcity, we evaluate both models by varying the fine-tuning data proportion (px) of the held-out datasets while maintaining constant test. sets. Results in Table demonstrate that: 1) The pretrained model consistently outperforms training from scratch across all metrics. 2) The performance gap widens as training data becomes scarcer (smaller ps). 3) Both models maintain reliable uncertainty estimates, evidenced by COVERAGE @ 95 values. These results validate that NIAQUE effectively transfers pretrained knowledge to novel regression tasks, with improvements particularly pronounced in low-data scenarios. Note that these results are not directly comparable with those in Table as they are based on different dataset splits (21 v8. 101 datasets).
The pretrained model is first trained on the 80 datasets and
Adaptation New Tasks on Kaggle Competitions. To validate NIAQUE's practical effective- ness in the wild, we evaluate its performance in recent Kaggle competitions: Regression with an Abalone Dataset (Reade & Chow 2024a), Regression with Flood Prediction Dataset. (Reade & Chow 2024bb
9

---

Published in Transactions on Machine Learning Research (02/2026)
Table 2: Transfer learning results on held-out datasets, Ps represents the proportion of training data used for fino-tuning, ranging from 0.05 (5%) 1.0 (100%). Lower values are better for all metrics except COVERAGE @ 95 (target: 95). For BIAS, lower absolute values are better.
NTAQUE
Ps P.00.05
0.1
0.25
0.5
1.0
SMAPE
Scratch
28.0
24.7
21.7
20.8
19.4
Pretrain
23.5
21.9
20.3
18.7
17.7
AAD
Scratch
0.71
0.60
0.56
0.54
0.49
Pretrain
0.61
0.57
0.54
0.50
0.47
RMSE
Scratch
1.23
1.10
1.06
1.04
0.96
Pretrain
1.11
1.08
1.04
0.97
0.94
BIAS
Scratch
-0.06
-0.04
-0.04
0.02
-0.04
Pretrain
-0.06
-0.07
-0.06
-0.06
-0.04
CRPS
Scratch
0.488
0.423
0.392
0.383
0.351
Pretrain
0.427
0.404
0.380
0.354
0.334
COV@95
Scratch
93.3
93.0
94.4
93.1
94.4
Pretrain
95.3
94.4
94.2
93.9
94.6
Table 3: Performance comparison on Kaggle competition datasets: Abalone (RMSLE, lower is better), Flood Prediction (R2, higher is better).
Abalone RMLSE
Flood Prediction
Model
R2 Score
XGBoost Broccoli Beef (siukeitin)
2024
Sayed
2024
0.15019
0.842
LBB
Wr3cker
Masoudi
2024
0.14914
0.766
CatBoost
WWtee 2024 Milind
2024
0.14783
0.845
TabNet (Broccoli Beel siukeitin
2024
0.15481
0.842
TabDPT Ma et al. 2024
0.15026
0.804
TabPFN Müller et al. 2025
0.15732
0.431
NIAQUE-Seraa Scratch
0.15047
0.865
NIAQUE-Pretrain
0.14808
0.867
Winner Heller 2024 Aldparis 2024
0.14374
0.869
(the private leaderboard competition results along with their ranks are presented in Table 3 Our ap- proach involves two stages: pretraining and fino-tuning. First, pretrain NIAQUE TabRegSet-101 using quantile loss framework. Then, fine-tune the pretrained model on the competition's training data, optimizing for target metric. To systematically evaluate the impact of proposed pretraining strategy and architecture, we show NIAQUE-Scratch baseline (trained only on competition data, no pretraining), a number of tree-based baselines as well as TabDPT and TabPFN pretrained models. While TabDPT and TabPFN show promising results on small datasets, they face significant scalability Challenges-Table TabPPN is limited to 10,000 samples and TabDPT requires substantial context size reduction for large datascts (dctails in Appendix 5.3 On the other hand, our approach shows very strong scalability and accuracy results on the competition datasets, outperforming vanilla brse- models as well TabDPT and TabPFN base- lines. Additional results in Appendices D D.3.2 show how our approach, without significant manual interventions, further benefits from advanced automatic feature engineering (OpenFE (Zhang et al. 2023) and ensembling thereby rivalling results of human competitors. These results are particularly significant given that neural networks were generally considered ineffective for these competitions.
Learned Representations are studied qualitatively in Fig. 3 (left) showing UMAP projections (Melnnes) et al. 2018) of dataset embeddings derived from NTAQUE's feature encoder. The encoder maps input
10

---

Published in Transactions on Machine Learning Research (02/2026)
AAD fea
2.0
0.0
Removed
Figure 3: UMAP projections of embeddings derived from NIAQUE's feature encoder for each sample, colored by dataset (left). NIAQUE accuracy response to the removal of input features by importance (right). Top- rated features have the greatest impact on AAD degradation when removed.
features x, to a fixed-dimensional latent space using a prototype-based aggregation mechanism. The resulting UMAP visualization reveals distinct datasct-specific clusters, indicating that NIAQUE learns representations that capture dataset-specific characteristies while maintaining shared latent space that enables effective transfer learning.
Feature Importance is based on the inverse of the average confidence interval derived from feature's s marginal distribution, as detailed in eq. 17). Qualitatively, Fig. 3 (right), indicates that features with higher weights (i.c., smaller average confidence intervals) are most critical to prediction aecuracy-removing these features significantly increases the AAD metric, whereas eliminating features with lower weights has minimal impact. Quantitatively, two-sided t-test comparing the impact of removing the most vs. least important fcatures (Top-1 Bot-1 AAD values in Fig. (t -50.24, p-value 0) along with the effect size (Cohen's d = 0.22) indicate significant and practically impactful elfect across datasets. Additionally, we computed SHAP values with Shapp.SamlinggExllan
across 101 datasets and used them as reference ranking. For each dataset, we computed the NDCG score between the SHAP ranking and our model-native importance ranking, then averaged the results. The average NDCG was 0.899, while computation time was reduced from 3.5h x GPUs to 20s x GPU for all 101 datasets. The NDCG score of 0.9 generally demonstrates very strong ranking alignment with established attribution method a fraction of the cost.
support NIAQUE's design choices. We observe: (1) encoder outperforms attention-based alternatives; (2) log-transforming inputs (Fq. eq. (10)) enhances both stability and accuracy; (3) the model is robust to hyperparameter changes and benefits from greater depth; (4) incorporating single-feature samples enables interpretability without degrading performance. Finally, we performed the ablation of the prototype layer by disabling the prototype connection eq. (14). We found that this drastically degrades the performance: SMAPE rises from 22.1 to 94.612, AAD from 0.367 2.585, and RMSE from 0.787 to 3.432, proving the effectiveness of the prototype layer.
Ablation Studies (Appendices
5.3 Scalability Analysis of TabPFN and TabDPT
TabPFN and TabDPT mark key advances in tabular deep learning, showing strong results on small-scale datasets. However, our Kaggle experiments reveal major scalability bottlenecks, limiting their practicality in larger real-world settings. This highlights the need for scalable alternatives like NIAQUE.
Limitations of TabPFN. Despite TabPFN's impressive few-shot performance, its applicability is bounded by architoctural and implementation constraints, as documented in the official repository (Muller et al., 2025): (i) maximum support for 10,000 training samples, (ii) limit of 500 features per instance, (iii) assumes all features are numerical, requiring preprocessing for categorical inputs. Furthermore, to adapt TabPPN
11

---

Published in Transactions on Machine Learning Research (02/2026)
for our use cases, we employed several workarounds: (i) for large datasets (e.g., the flood prediction dataset with 1.12M samples), we applied Random Forest-based subsampling, following official guidelines (Müller et al.) 2025), (ii) categorical features were ordinally encoded to conform to TabPFN's numerical input requirement. These constraints, especially the aggressive downsampling, likely contributed to TabPFN's suboptimal performance on large-scale tasks (e.g., R2 score of 0.431 on flood prediction).
Scalability Challenges in TabDPT. TabDPT, based on a transformer backbone, encounters scalability issues typical of attention- baaent models: (i) memory consumption grows quadratically with context size due to self-attention, (ii) inference time scales poorly with dataset size, (iii) performance is sensitive to reductions in context size, making trade-offs between scalability and accuracy non-trivial. In our experiments we had to reduce the context window for the flood prediction dataset to fit within memory constraints. We reduce the context size from 8,192 in powers of 2. Context size of 1024 finally works. This reduction correlated with a decline in model performance (R2 score dropped to 0.804). The compute cost of running extensive hyperparameter tuning on large datasets proved impractical. TabDPT takes 12 hours of inference time for each perraaraaeee configuration for flood prediction datatset on a V100 GPU.
6 Discussion
We believe that our results applying NIAQUE to the 101 dataset benchmark lay out the stepping stone for the development of probabilistic meta-models eventually possessing the following key properties. Scal- ability: A unified model shares computational resources to address multiple regression tasks, optimizing resource utilization and reducing the operational costs of maintaining separate models. Data Efficiency: Training on diverse lasks introduces strong regularization effects, and we expect existing datasets to be repurposed to solve emerging problems, promoting data reuse and recycling. Representation and Gen- eralization: model trained across multiple datasets uncovers generalizable representations of regression tasks and ways of solving them, acquiring the ability to apply this knowlegge across datasets. As an example, while TabPFN and TabDPT perform well on small datasets, they face significant scalability challenges with high-dimensional, large-scale data. NIAQUE demonstrates that scalable, accurate, and interpretable neural models are viable without relying on heavy preprocessing memory-intensiee components. These results position NIAQUE not just as a practical alternative, but also as evidence that our approach offers fruitful rescarch path beyond current paradigms.
Limitations. While we significantly expand the scope of cross-dataset probabilistic model training by applying our neural model to a 101-dataset benchmark, this remains limited effort. It is still unclear how many datasets are required for a regression model to be considered foundational for solving, for instance, 80% of industry problems. What level of datasel diversity is necessary? Will millions or billions of unrelated datasets be required, or would 10,000 overlapping datasets suffice? Defining and evaluating global success in this context remains an open question, necessitating further research. Furthermore, our findings have implications for designing machine learning deployments based on unified models that address multiple regression tasks. We expect that this will eventually lead to improved operational efficiency and accuracy of the models. However, this could also contribute to the centralization of power among a few large entities. In this context, risk mitigation strategies include (i) improving model computational efficiency and (ii) publicly releasing data, model training code and pretrained models. Additionally, multi-task learning multiple datascts may introduce new biases not present in locally trained models, making interpretability and fairness research critical. We explore some interpretability aspects in this paper, and further research on interpretability and fairness in large probabilistic regression models pretrained across multiple datasets seems to be an important area for future work.
7 Conclusion
Our study demonstrates that NLAQUE enables effective transfer learning for tabular probabilistic regression, providing additional evidence to further challenge the belief that tabular data resists neural modeling and generalization. Across 101 datasets and Kaggle case studies, NIAQUE shows strong empirical performance, delivering scalability via unified architecture, data efficiency through cross-dataset pretraining, and robust
12

---

Published in Transactions on Machine Learning Research (02/2026)
generalization across tasks. Its probabilistic formulation further supports uncertainty quantification and feature importance estimation, useful for real-world deployment. While promising, open questions remain regarding the optimal scale of transfer, the design of universally effective feature preprocessing, and the theoretical principles underlying cross-domain generalization. Overall, our findings suggest a path forward for building interpretable tabular foundation models that scale well in large-scale downst.ream lasks.
References
Jesús Alcalá-Fdez, Alberto Fernández, Julián Luengo, Joaquín Derrac, and Salvador García. KEEL data- mining software tool: Data set repository, integration of algorithms and experimental analysis framework. J. Multiple Valued Log. Soft Comput., 17(2 3):255 287, 2011.
Aldparis. 1st place solution for the regression with a flood prediction dataset. https://www.kggee com/
2024. Kaggle.
Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Olek- sandr Shchur, Syama Syndar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschicg- ner, Danielle C. Maddix, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Fohlke-Schneider and Yuyang Wang. Chronos: Learning the language of time series. arXiv preprint AXXi:22033008814 2024.
Sercan Ó. Arik and Tomas Pfister. Tabnet: Attentive interpretable tabular learning. In Proc. AAAI, pp. 6679 6679,6687,00 May 2021.
Sourav Banerjee. House rent prediction dataset, 2022.
URL http:://ww.aaggll.ccm/aaaaattt
Leo Breiman. Random forests. Machine Learning. 45(1):5-32, 2001.
Broccoli Beef (siukcitin). Why mns is better than gbds?
https://www.aggl.. tt
2024. Kaggle.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd
ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '16. ACM, August 2016.
doi:
URL http://dx doi www.doo.ooooooo wwoi.org/10..... 21/e//1..111///9997722282293 2939785
Gourab Roy (dataWr3cker). ps-s4e4: Eda lightgbm importance plots. https://www.kag::: comcclee//
2024. Kaggle.
Dean De Cock. Ames, iowa: Alternative to the boston housing data end of semester regression project Journal of Statislics Education, 19(3), 2011.
Jacob Devlin, Ming- Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidi- rectional transformers for language understanding. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), NAACL-HLT (1), pp. 4171-4186 Association for Computational Linguistics, 2019.
Lue Devroye. Non- Con-Uriform Random Variale Generation. Springer-Verlag, New York, NY, USA, 1986.
Brijial Dhankour. Flood prediction factors. Flood-prediction-factooo 2024. Kaggle.
Dheeru Dua and Casey Graff. Uci machine learning repository: Abalone data set, 2019. URL https:
//archive. ics. uuc.. htt.://l//aatasstttaaaaaaaooo
Max H Farrell, Tengyuan Liang, and Sanjog Misra. Deep neural networks for estimation and inference.
Econometrica, (9(::1181 213, January 2021.
Marta Garnelo, Dan Rosenbaum, Christopher Maddison, Tiago Ramalho, David Saxton, Murray Shanahan, Yee Whye Teh, Danilo Rezende, and S. M. Ali Eslami. Conditional neural processes. In Proc. ICML, volume 80, pp. 1704-1713. PMLR, Jul 201 Sa. URL htL//tttss///prccedinng mll. html
13

---

Published in Transactions on Machine Learning Research (02/2026)
Marta Garnelo, Jonathan Schwarz, Dan Rosenbaum, Fabio Viola, Danilo J. Rezende, S. M. Ali Eslami, and Yee Whye Teh. Neural processes, 2018b. URL.
Azul Garza and Max Mergenthaler-Caneee
Timegpt-1, 2023.
Tilmann Gneiting and Roopesh Ranjan. Comparing density forecasts using threshold-and and quantile-weighted scoring rules. Journal of Business & Economic Statistics, 29(3):411-4233 2011.
Yury Gorishniy, Ivan Rubachev, Valentin Khrulkov, and Artem Babenko. Revisiting deep learning models for tabular data. Advances in Newral Information Processing Systems, 34:18932-18948, 2021.
Leo Grinsztajn, Edouard Oyallon, and Gael Varoquaux. Why do tree-based models still outperform doep learning on typical tabular data? In Proc. NewrIPS, pp. 507-520, 2022.
Johannes Heller. 1st place solution for the regression with an abalone dataset competition. https://www kaggle.com/comee
49991174 2024. Kaggle.
Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. TabPFN: A transformer that solves small tabular classification problems in a second. In Proc. ICLR, 2023.
Xin Huang, Ashish Khetan, Milan Cvitkovic, and Zohar Karnin. TabTransformer: Tabular data modcling using contextual embeddings. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 7671-7679 AAAT Press, 2021.
Kaggle. Kaggle datasets, 2024. URL https://www http://ww..ggggl.eeaaaaaaaeeeeee com/datasets
Hamed Karimi, Julie Nutini, and Mark Schmidt. Linear convergence of gradient and proximal-gradient methods under the polyak-lojasiewiez condition. In Paolo Frasconi, Niels Landwehr, Giuseppe Manco, and Jilles Vreeken (eds.), Machine Learning and Knowledge Discovery in Databases, pp. 795 811, Cham, 2016. Springer International Publishing.
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30, Curran Associates, Inc., 2017. URL Paper.ppppppp pdf
neurips. neurips.ccppaper www.niiinnnnn..is.o....m
Markelle Kelly, Rachel Longjohn, and Kolby Nottingham. UCT machine learning repository, 2017. URL http://archive.iie www.uuuci.edld.
Roman Levin, Valeriaa Cherepanova, Avi Schwarzschild, Arpit Bansal, C Bayan Bruss, Tom Goldstein, Andrew Gordon Wilson, and Micah Goldblum. Transfer learning with deep tabular models. In Proc. ICLR, 2023.
Junwei Ma, Valentin Thomas, Rasa Hosseinzadeh, Hamidreza Kamkari, Alex Labach, Jesse C Cresswell, Keyvan Golestan, Guangwei Yu, Maksims Volkovs, and Anthony L Caterini. Tabdpt: Scaling tabular foundation models. arXiv preprint ar rxXf::2410.18144 2024.
Ali Masoudi. Xgb cat http://///ao....ooooo//o///a/aar 2024. Kaggle.
Julian McAuley, Rahul Pandey, and Jure Leskovec. Amazon price and product data: Electronic commerce, 2015. URL htRpstt/://// https:://wwwt http::/wwwwwwwgl.l../// cwm/
L. Mclnnes, J. Healy, and J. Melville. UMAP: Uniform Manifold Approximation and Projection for Dimen- sion Reduction. Arxiv e-prints, February 2018.
Jay Milind. Baseline v1 catboost.
baseline-vis -itttneosttcabbsttt..... 2024. Kaggle.
14

---

Published in Transactions on Machine Learning Research (02/2026)
Felix Müller, Christian Holtz, Emir Akyürek, Lukas Lorbeer, and Frank Hutter. TabPFN: A Transformer that solves small tabular classification problems in a second. httpss//ggthha.cco///oooooob/bpp 2025. URL
Michael Niemeyer, Lars Mescheder, Michael Occhsle, and Andreas Geiger. Occupancy flow: 4d reconstruction by learning particle dynamics. In Proc. ICCV, October 2019.
Randal S. Olson, William La Cava, Patryk Orzechowski, Ryan J. Urbanowicz, and Jason H. Moore. Pmll: A large benchmark suite for machine learning evaluation and comparison. BioData Mining, 10(36), 2017a.
Randal S. Olson, William La Cava, Patryk Orzechowski, Ryan J. Urbanowicz, and Jason H. Moore. Pmll: large benchmark suite for machine learning evaluation and comparison. BioData Mining, (1):36, Dec 2017b. ISSN 1756-0381. doi: 10.188661133400017-0015--- URL https: org/10..///oo 1186/
613040-017-01544
OpenML. Wind dataset, n.d. URL
http://www..p...attt///////////a arff
Boris N. Oreshkin, Florent Bocquelet, Félix G. Harvey, Bay Raitt, and Dominic Laflamme. Protores: Proto- residual network for pose authoring via learned inverse kinematics. In Proc. ICLR, 2022.
Ethan Perez, Florian Strub, Harm De Vries, Vincent Dumoulin, and Aaron Courville. Film: Visual reasoning with a general conditioning layer. In Proc. AAAI, 2018.
Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. Catboost: unbiased boosting with categorical features, 2019.
C. Qi, Hao Su, Kaichun Mo, and L. Guibas. Pointnet: Deep learning on point sets for 3d classification and segmentation. Proc. CVPR, pp. 77-85, 2017.
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Πya Sutskever. Learning transferable visual models from natural language supervision. In Proc. ICML, volume 139, pp. 8748-8763, Jul 2021.
Alvin Rajkomar, Eyal Oren, Kai Chen, Andrew M. Dai, Nissan Hajaj, Michaela Hardt, Peter J. Liu, Xiaobing Liu, Jake Marcus, Marvin Sun, et al. Scalable and accurate deep learning with electronic health records. Nature Medicine, 24(7):3377-1340, 2018.
Walter Reade and Ashley Chow. Regression with an abalone dataset. kttp:://kagle..co//copeettt playground-seriee -4e4, 2024a. Kaggle.
Walter Reade and Ashley Chow. Regression with a flood prediction dataset. 2024b. Kaggle.
s4e5
Joseph D Romano, Trang T Le, William La Cava, John Gregg, Daniel J Goldberg, Praneel Chakraborty, Natasha L Ray, Daniel Himmelstein, Weixuan Fu, and Jason H Moore. Pmlb v1.0: an open source dataset collection for benchmarking machine learning methods. arXiv preprint arXth:2012.00055::: 2021.
Zeyad Sayed. Flood forecasting with xgboost. https://www. http::/www.aglll..ocoo http://wwwwg.eeeeeooc///////aaaa floot=forecastiin--ithh--
2024. Kaggle.
Slawek Smyl, Boris N. Oreshkin, Pawel Pelka, and Grzegorz Dudek. Any-quantile probabilistic forecasting of short-term electricity demand, 2024. URL http::/axx...rrraaa///00... 17451
Jake Snell, Kevin Swersky, and Richard S. Zemel. Prototypical networks for few-shot learning. In Proc. NIPS, pp. 4080-4090, 2017.
Yu Sun, Qian Bao, Wu Liu, Yili Fu, Black Michael J., and Tao Mei. Monocular, One-stage, Regression of Multiple 3D People. In ICCV, 2021.
15

---

Published in Transactions on Machine Learning Research (02/2026)
Joaquin Vanschoren, Jan N. van Rijn, Bernd Bischl, and Luis Torgo. OpenML: networked science in machine learning. SIGKDD Explorations, 15(2):49 60, 2013. doi: 0..114//6411902264111 URL http://doi. acm. 091/10..145/2641190.222 264119
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnell (eds.), Proc. NeurIPS, volume 30, 2017.
Suraj Wate. S4c4 abalone catboost.
s4e4-aba alee-abblone--cabbooss 2024. Kaggle.
Manzil Zaheer, Satwik Kottur, Siamak Ravanbakhsh, Barnabas Poczos, Russ R Salakhuidinov, and Alexan- der Smola. Deep sets. Proc. NewrIPS, volume 30. Curran Associates, Inc., 2017.
Tianping Zhang, Zheyu Zhang, Zhiyuan Fan, Haoyan Luo, Fengyuan Liu, Qian Liu, Wei Cao, and Jian Li. Openfe: automated feature generation with expert-level performance. In Proc. ICML, 2023.
16

---

Published in Transactions on Machine Learning Research (02/2026)
A Definitions of metrics
We employ the following metries to evaluate the accuracy of point predictions: Symmetric Mean Absolute Percentage Error (sMAPE), Average Absolute Deviation (AAD), Root Mean Square Error (RMSE), Root Mean Square Logarithmic Error (RMSLE).
Symmetric Mean Absolute Percentage Error (sMAPE):
9
SMAPE
y/
f1..5
100%
(18)
+ 0i.0.5
2. Average Absolute Deviation (AAD):
1
AAD
Σ
y6 0c.0.5|-
(19)
5
3. Bias:
8
BIAS
01.0.5
(20)
5
1=1
4. Root Mcan Square Error (RMSE):
1
9
RMSE
(y)
02.0.3)2
(21)
S
1=1
Root Mean Square Logarithmic Error (RMSLE):
8
RMSLE
(log(ı: +1 log(Or.o.s + 1))2.
(22)
17

---

Published in Transactions on Machine Learning Research (02/2026)
B Proof of Theorem
Theorem. Let F be a probability measure over variable such that inverse F-1 erists and let Pyx be the joint probabilily meusure of variables x, Then the expected loss, E ply: {q)); is minimiized if and only if:
== Pyxx
(23)
Additionally:
min ply: (1)) Ex / P)(x(=)(1 - (P) F
(24)
Proof. First, combining 9 with the L. representation of CRPS eq.
we can write:
E ply, F (4)) Fx 2
1
(F(z) 1(<20))) dz
(25)
1
= EExby1x 2 12
12((2) - (2 (=2v1 + (=20)dz
(26)
= =Ex /
F2(2) -2 (02n) B Il (=20)
(27)
1 -FF3)) + (2)x(=)dz. 2xx 2 R
(28)
Here we used the law of total expectation and Fubini theorem to exchange the order of integration and then used the fact that [ = (2)x Completing the square we further get:
/ F²(2) - 2F(x)) Paix(2) + pix(3) (2)(())dz
Eply Fly. (g)) = =Ex, 2
(29)
1
Ex R
(F(=) 2 P Pu)x(2) Po (
(30)
F Pulx is clearly the unique minimizer of the last expression since (F(2)) (x)x()))'dz OF
P Py|x-
18

---

Published in Transactions on Machine Learning Research (02/2020)
(a) Dataset Sources
(b) Dataset Size
(c) Dataset Width
25
OPENML
30
KAGGLE
20
29%
2.3%
20
Count
15
5%
5%
KEEL
10
21%
17%
DCC
10
PMLB
5-
5-
UCI
5000
10000 15000
20000
20
Number of Samples
Number of Variables
Figure 4: Statistics of the evaluation datasct: (a) distribution by source, (b) dataset sizes, and (c) feature counts.
Table 4: Performance comparison for top baselines, macro-average across all 101 datasets. Lower values are better for all metrics except COVERAGE 95 (target: 95). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
Cot
42.9
0.982
1.301
-0.34
92.2
0.709
Cat
23.2
0.437
0.690
-0.04
91.5
0.340
LTTThansformer--ocall
24.5
0.450
0.705
-0.04
93.8
0.320
TTTTransfomerr-Goobn
22.0
0.402
0.641
-0.02
94.3
0.285
NTAQUF-Local
22.2
0.405
0.650
-0.03
94.2
0.287
NIAQUE-Global
21.6
0.394
0.632
-0.03
93.6
0.283
C TabRegSet-101 101 Details
We introduce TabRegSet-101 (Tabular Regression Set 101), curated collection of 101 publicly available regression datasets gathered from UCT (Kelly et al. 2017 Kaggle (Kagglc 2024), PMLB (Romano et al. 2021 Olson et al. 2017bp. OpenML (Vanschoren et al. 2013). and KEEL (Alcalá-Fdez et al. 2011). These datasets span diverse domains, including Housing and Real Estale, Energy and Ffffcciecy, Retail and Sales, Computer Systems, Physics Models and Medicine and exhibit different characteristics in terms of sample size and feature dimensionality (Fig. 4 The datasets, along with their sample count, number of variables and source information are listed in Table
We focus specifically on the regression task in which the target variable is continuous if it has limited number of levels, these are ordered such as student exain scores wine quality. The target variable in each dataset is normalized to the j0, 10] range and the independent variables are used as is, raw. The target variable scaling is applied to equalize the contributions of the evaluation metrics from each datasct. Datasets have variable number of samples, the lowest being just below 1000. For very large datasets we limit the number of samples used in our benchmark to be 20,000 by subsampling uniformly at random. This allows us (i) to model imbalance, and the same time (ii) avoid the situation in which a few large datasets could completely dominate the training and evaluation of the model. The distribution of datascts by source, number of samples and number of variables is shown in Figure 4
19

---

Published in Transactions on Machine Learning Research (02/2026)
Table The list of datasets comprising TabRegSet-101 benchmark
id
Name
samples
vars
source
URL
0
Abalone
4177
https://archiveeiies....
staticc/pbblic/1/aaaa.cc
1
CandenttPorformance
649
29
uci
http:://archiveeiccs.c....
staicc/public//2///aaaaccc
2
Infrared _ TTeneeaeeereeeee
1020
32
uci
https://arcaive..csscc..ee
static static/ /aaiic//abllcc/2////aaaccc.
3
Parkinsons Purkitsons_Teeemnnitorooo Telemonitoring
5875
18
uci
http::/aaraiveeiccs.ui...
static/public/1899/61
4
Energy Efficiency
768
uci
attp:///archiee..ee.ud..o
staicc/public//34/aaaaaccc
5
1027 ESL
488
3
pmlb
attp://gaaaub.cc//puaaatsaaaaaa
ht
penn 2an--llbaaccharrkkrrraaaaaatt
arta0c5/107,_551107BBBB8.1
6
@028_SWD
1000
9
pmll
https///gihubb.co///sssstsiii..
dstasetaa/1028_55n/10023 SWD tav gz penn
1029 LEV
1000
pmlb
atte://giibabb.ccm//psssstttaait
pea-=1--aacchaartz/ssssaaattt
dstaseta/1022___a|//003_111 gz
8 1030 ERA
1000
3
pmlb
httpa///gaaubb.ooo//sssstttattl
https:
dataaets1103___AA/000_EEMMM g2
9
1199_BNG BNG_BeeeGG_eeiiiiiee honts
17496
8
pmlb
https:
httpe://gttaibb.c///gsaataaaaai
1108_Bl__ecchooottaa..oss
10
197_ _07__pp____ cct__ct
8192
20
pmlb
11
225 puma8NH
8192
7
pmlb
httpa:
httpe://ggthubb.coo//pppssssssst
penn-ml pen--allbaachhaakk///aaaaasarr
dataaets/222_pmmaaittt//2 tsv.gz
12
227 cpu _pu_epu small
8192
pmlb
ht htape://giicubb.ccm/ppissttaaaa,
penn htw.-/ecchcaarre/e///sttterrrr
dataseta/227_ /ttpeeee2222cpp__8a1222111..100
small tzv.gz
13
294 satellite image
6435
35
pmlb
https: http:///tuuub.ooop/ssststtseeeee
penn ml www.ll/eccnnaarrrrrraaatteeeeeee
dat Sttaseta/23_aaaeel11e image/ 294
satell eate_llit___nagottevvv.
14
344 mv
20000
pmlb
ht penn pen-=allbbanddesttrrraaaaaaae
2atasets/3444_m//444m..... -z
15
503 wind
6574
13
pmlb
http://giaaub.com/ppssstttaaaa
16 529 _29_pollen pollen
3848
3
pmlb
http::///gttabb..co///ggisssaaat
penn -omn-al-bendinn htwp:laaaccccccaaaaeaeeaeateee
datasets aacaset/500.poolmm/223..l..1..
tss. gz
17
537 houses
20000
7
pmlb
https: https://gttaubbccoo/pppsssaaaaat
penn heanaall-becchaasse///ssstter
lataata/S99_.ousee/3/...oooo
tav. ssv.g=
Continued on next page
20

---

Published in Transactions on Machine Learning Research (02/2026)
Table 5: The list of datasets comprising TabRegSet-101 benchmark
name
samples
vars
source
url
18
547_no2
500
pmil
https///gituub.cco/pppsssaasasas
19
564 fried
20000
9
pmlb
https
httpe://gihubb.comm/pptttetsiii
penn 2a00=11laaacchaakk//////aaaarr
20
595
1000
pmlb
http:://gihhbb.cc//pppssaaasssst
pem--al- hhanaallaeaauunssssss///sasttt
10/596.
rri_co__1000100000
21
593 fri cl 1000
1000
9
pmlb
https: http:://gtaaub.commppisttaaatll
penn- sen-aal-baccharrrr/raaaaaaate
dataseta/S93ftr1.co 1000 10/593
10__1_1000_1001100
22
1193 BNG lowbwt
20000
8
pmlb
https: http:///gtaabb.com//ppssaaaaaaa
pem--al-bacchaattttrraaaaaaaa
dstasets/1193_ BNG 1osgBtWtoooob
BNG BNG_Lovbut.tsv.gg
23
1201 |20]_BNG TumortTumor
20000
8
pmlb
httpa://gttabb.cc//ppsaaaaaaaaat
datasets/ BNG1112111_1GGG110 breastTurrr
B___B__breeettuuorr.eee9 tav g2 1201_BNG
24
1203_BNG LineearLinner
20000
9
pmlb
https://giaubb.ccc//gaaaaataaaaa
15tastts/12033BBB.pplLteeaa.
BNC_paLinear.
25
215_ 2dplanes
20000
pulb
sem--al-eeachharrr/rsssssster
Batasees/216,,2ppaaaeee 24p.anes.tav.g2
26
218 house_ SL
20000
7
pmlb
https
datasets/218_hout 2acese___22211 8L/218 house__
2tc//188hhouue
BL.
27 QsaxFishyoxicity
908
uci
httpe://srchive.
stat://pbbllc/500/ggarrfffi
toxicity.zij
28
concreto compressive__steeeeeeeeeeee strenghh
1030
7
uci
attps://archivee.cssuu
paatic/publcc/115/coorrrg
compressivee+stronnbbb...3
29
PRODUCTIVITY
1197
12
uci
https:
https://srchivee http:://arcaiveeiccs.c..e
zip
30
CCPP
9568
3
uci
https:/arrchive.ccs.cci.
3a5:://aal0008444/mmmeeeeeyy povertplant.2r
31
AIRFOII
1503
4
uci
http:://archivv..ce..c..
/aaaii///bbiic//9///tt661aaaa noise.21p
static
32
TETOUAN
20000
uci
attps://archiiv..ccssu
edu cdt//Gaaicc/uall1//55///oeeer
cossumpionoofFttooon--ttyty zip
33
BIAS_ CORRECTION
7725
22
httpp://arrchiveiice..u..
cchh//tafii//pbbllt/554iiaaa
correctionnooffuuerrcccal
prodecciorrodel-eepprruuurr f forecast.zip
Continued on next page
21

---

Published in Transactions on Machine Learning Research (02/2026)
Table 5: The list of datasets comprising TabRegSet-101 benchmark
name
samples
vars
source
url
34
APARTMENTS
10000
10
https://archive..ie. .uccss..c.... .ui. edd//
http://pbb.cccsssssaaateeeeeteee reut+classifiedd.zz
35
MedicalCost
1338
5
kaggle
kaggle datasets download -d miri-
choi0218/Insurance
36
Vehicle
2059
18
kaggle
kaggle datasets
download
-d
jebalbiraa/vehicll-aataett
cardekho
37
LifeExpectancy
2928
18
kaggle
kaggle
datascts download
38
CalHousing
20000
7
doc
https://www.dcs dcc. up. ptppppptpoorre..eeee.eee
pogressiinn/ca_hooustiee tg²
39
Ailerons
7154
39
dec
https://www.dcc... pt/-ltorgo
Ragreassion/ailerross..gg
40
Deltatlev
9517
5
dec
iagressiiom/aatta_eeeevaooo gg
41
Pole
10000
dec
Regression/pol.1ge
42
Kinematies
8192
7
doc
http://www..bcc....uuupttt//oooa Beggressoon/Kinonnticss... tar.g
43
BigMartSales
8523
10
kaggle
kaggle datasets download
dd
brjjbushamannda197y/bggre
sales-data
44
VideoGameSales
16598
kaggle
kaggle datascts download gorut corut/videogamestt
-d gre-
45
NewsPopularity
20000
58
uci
https://archive. ics uci. edu/
static//abliccc57//mmleeeet popularity.211
46
Wismir
1461
8
keel
https://sci2s.. es/keel/
lataett/ast//ngressol//laaa zip
47
Ele2
1056
3
keel
datast//att/rerresii///ll......
48
Treasury
1049
14
keel
sttpe///et22..ugg.eeteeel
"ahaae///aa//regresiioo//rrurrr zip
49
Mortgage
1049
14
kvel
https://nciis..grr.ee /ttp///cci...urr.eeeeeee/e//
50
Laser
993
3
keel
https://edd2..uug.eekkkee
ht
dataaet/abaa/rggressto///aaa zip
51
SpaceGa
3107
openml
ht
Gowmloas/522111/gpcca_....
52
VisualizingSoil
8641
openml
http:://ws..pemmloorr////aa dom.load/52888//rsaaliiingg arff
53
Diamonds
20000
8
openml
https://ww.oopmml.orgg/
2oontoa//2197285//ttatttttt arff
54
TitanicFare
1307
openml
ht ntttp::/uww.opemml.com org/
data/ dataa/Loslload/20662000
f112777ee22670e8...777
55
Sulfur
10081
5
openml
ht httper//www.peeml..rrgg
56
Debutanizer
2394
6
openml
https://ww..on http::://wwv.opmmm..orrrg wwt.ml/oog//aaaaa///
hom..oat/2299033/pppii777 wtw.r/ arff
Continued on next page
22

---

Published in Transactions on Machine Learning Research (02/2026)
Table 5: The list of datasets comprising TabRegSet-101 benchmark
name
samples
vars
source
url
57
Pardamento
6277
5
openml
https://www.opeml org//aaaa.
6ommlosd/2655533//addaamtt
saidas 10l_a___1__222mmm1111_ arff
58
ProteinTertiary
20000
8
openml
https://spi.
org/
data/ 2ata/toomlooe//2111127
F11e2216666202222 arff
59
BrazilianHouses
10692
7
openml
ht
org/
asta//oomloe//22211655
data/
11e2211627e449660.o6
60
Cps88
20000
5
openml
https://api.openn org/
data/bovllos///211888
Fi 1116221666406556.ou
61
CPMP-2015
2108
25
openml
httpe:
data/
2ata//oomload//1377444
161e666666335 arff
62
NASA-PHM200S
20000
16
openml
https org.//aaaa/
63
Wind
6574
12
openml
download//5211//iinnn. arrf
64
NewFuelCar
20000
17
openml
sttps://wwwopeeml.. org/
dsta/boomiooe//2220000
pruebacoayvolliee..c... arff
65
MiamiHousing
13932
14
openml
2om..oad//2004775/mmmaa11122 arff
66
BlackFriday
20000
8
openml
data/
data//oomlooa//222044
Fi1et33340hdPca..artt
67
IEEF8021 IaAGTS
5296
28
openml
http:///ww..pemml.or////e bow..oas/221000BB/aaaaatttttt arff
68
Yprop41
8885
41
openml
http:://ppl.oppmmmloor///aaa.a
domnooas/22111922/aaaaee arff
69
Sarcos
20000
20
openml
https://aps.oppemll org/
data/ data/boomload/22111199
f11e22166a1666bbbburr
70
ZerichDelayss
20000
16
openml
ht
https://uwv.peem// org/
aata/boomlon///2866440 [11e
71
1000-Cameras
1015
13
openml
http:://wwwopppmmlorr
http://wwwooommloor//aaaaa/////
2omloaa//22100233/tttaee arff
72
GridStability
10000
11
openml
https:
org/
data
data/Loomloa//2211155
arff
73
PumaDyn32nh
8192
31
openml
at http:///pp.pppmml.mmm data data//oomload/2211114
f11222161b261fbbbarrt
74
Fifa
19178
27
openml
https: btaps://ap.../ openml...ppmmmm..mmmmmmmmm org/
data bata//owmlon//2211189
f f1 F11e10cca/11333dboo arff
75
WhiteWine
4898
10
openml
ht
2tta/toomllodd/2111880 org/
f
arff
76
RedWine
1599
10
openml
ht tptp / org/
data/ 2tta//oommoond/21110000
fiee22 122b31666611113 arff
77
FpsBBnnchmark
20000
42
openml
https http://app.ppp......
data /ttp///moood/22211555
f wwee221169922000 arff
Continued on next page
23

---

Published in Transactions on Machine Learning Research (02/2026)
Table 5: The list of datasets comprising TabRegSet-101 benchmark
name
samples
vars
source
url
78
KingCountyHHuusing
20000
20
openml
https://api.opemmm..co Asta/bowmlosd/22111833
f11e22f167b411411 .arff arff
79
AvocadoPrices
18249
12
kaggle
kaggle
datasets
download
-d
neuromusic/avocadoopmi
80
Transcoding
20000
18
uci
attp:://arcchiv..ce......
Lmmerdttseet.24p
81
house_ 16H
20000
15
openml
https:
https://wwv.opem//
dowmload/5252//ouuee 16H.
82
Sales
10738
13
openml
https: //wp: httpps://wwwoopemmm..
(ownlood/21/56753///60
arff
83
WalmartSales
6435
8
kaggle
kaggle
datasets download
d
mikhail1681 (ilhaaillll//aaamaar----aae
84
UsedCar
6019
kaggle
kaggle
datasets
download
-
85
HouseRent
4746
11
kaggle
kaggle datasets download
d
famsouravaanerjee rent--
prediction-dataset
86
LaptopPrice
1273
15
kaggle
kaggle datasets
download d
chtishamsadiq/une wawadd///unneeee-laattoa
price-dataset
87
UberFare
20000
8
kaggle
kaggle
datasets
download
d
yasscrh/ubbe
88
Co2Emission
7385
10
kaggle
kaggle
datasets
download
stbaiyooppoaddrr/oooemmmmiisoooi by- vehicles
89
SongPopularity
18835
12
kaggle
kaggle datascts download
d
yasserh/song popularity
90
Cars
20000
8
kaggle
kaggle datasets download
dd
alshwaryamuthukumar wattuuuuaaaaaaaaaaaaaaaa /caar--dassssttttt dataset-
audi- audi-bmw- bow-ford-hyy bon-ffod--yuuddi--ttoo www--hhuunidsddd--eeeee
91
GemstonePrice
20000
kaggle
kaggle datasets download
d
colearninglounge oleerainglounggggmmmonneeppeiip gemstone-price-
prediction
92
LoanAmount
20000
20
kaggle
kaggle datasets download
phileinspphox//prdiittoooom pwede///poettttttonnnoonnttt data
93
AaudiArsbiaCars
5507
10
kaggle
kaggle datasets
download
d
turkibintalib/ turkbiinaalih/andiiaaabbaa /sandi-araaba-useet
cars-datasct
94
GpulKernelPerformance
20000
13
kaggle
kaggle datasets
download
rupals/gpu-rumtine
95
AmericanHousePrioes
20000
10
kaggle
kaggle datasets
download
d
jeremylarcher/ hawaarccherrammrcanntnnnaaa
96
KindleBooks
20000
12
kaggle
kaggle
datascts download
dd
asaniczka, aaaiickkaaamaaaa fmaazonkkindll-booke
lataset-2023-113kkbbodd
97
BookSales
1070
kaggle
kaggle datasets download
thedevastator
98
CapitalGain
20000
12
kaggle
kaggle datasets download
-d
mainieciang/accuutdataa
99
MampetingCampaain
2976
14
kaggle
kaggle datasets download
d
ahmadazari bhmacdzari/marzeetin--oomm mww.e//annnnninanaiinnnn
data
Continued on next page
24

---

Published in Transactions on Machine Learning Research (02/2026)
Table 5: The list of datasets comprising TabRegSet-101 benchmark
name
samples
vars
source
url
100
2000
kaggle
kaggle
datascts
download -d
wwrang98/software.e wawtaarrr-saeeprooeeeee promotion-
campaigu-upllit-moodel
25

---

Published in Transactions on Machine Learning Research (02/2026)
Table 6: Performance comparison on 1000-Cameras dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CalBoost-Global
130.371
1.058
1.250
0.59
66.7
0.788
CatBoost-Local
54.894
0.379
0.996
-0.27
86.3
0.285
FTTTransformer-Local
48.959
0.337
0.905
-0.09
86.0
0.221
Setransformer-Glolaae
46,265
0.261
0.778
-0.08
86.6
0.208
NIAQUE-Lockh
42.195
0.245
0.782
-0.10
87.0
0.189
NIAQUE-GGlbal
41.757
0.218
0.722
-0.10
87.3
0.178
Table 7: Performance comparison on 1027 ESL dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
57.151
2.638
3.034
-2.58
95.9
1.585
CatBoost-Local
10.353
0.531
0.696
-0.05
93.9
0.394
FTThransformer-Local
12.201
0.561
0.713
-0.05
86.6
0.406
10.431
0.520
0.661
-0.04
90.5
0.374
NIAQUE-Local
10.634
0.539
0.721
-0.05
87.5
0.404
NIAQUE-Global
10.293
0.537
0.704
-0.05
85.7
0.394
D Supplementary Results
D.1 Per- Ber-Dataset Performance Results
This sub-section provides detailed performance metrics for each of the 101 datasets in our benchmark. Tables are organized alphabetically by dataset name. Each table shows the performance of top model variants (labbll Cat Bocst-Loocl, FTTraasformer-Locll CTTrannformmn-Cloball, NTAQUE-Local, and NIAQUE-Global) across all evaluation metrics.
26

---

Published in Transactions on Machine Learning Research (02/2026)
Table 8: Performance comparison on 1028 SWD dataset. Lower values are better for all metrics except _L_ CoCeerae (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
46.203
2.557
3.526
-2.43
80.0
1.866
CatBoost-Local
28.035
1.534
2.120
0.15
88.0
1.189
FTTTransformer-Loct
30.224
1.489
2.182
-0.09
95.7
1.141
FTTTransformer-Glonn
26.172
1.468
2.158
-0.07
93.4
1.013
NIAQUE-Local
27.206
1.407
2.162
-0.10
98.3
1.034
NIAQUE-Global
26.771
1.413
2.145
-0.10
96.0
1.024
Table 9: Performance comparison on 1029 LEV dataset. Lower values are better for all metrics except C Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
GltBoost-Global
63.046
2.509
2.978
-1.97
90.0
1.569
CatBoost-Local
36.867
1.362
1.846
-0.14
93.0
0.991
FTTTranfformer-Locall
35.764
1.301
1.828
-0.27
90.8
0.919
FTTraasformer--Glloe
34.588
1.230
1.787
-0.27
92.0
0.878
NIAQUE-Loca
33.430
1.154
1.708
-0.34
92.6
0.888
NIAQUE-Globe
32.545
1.130
1.734
-0.30
88.0
0.889
27

---

Published in Transactions on Machine Learning Research (02/2026)
Table 10: Performance comparison on 1030 ERA dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
61.934
2.098
2.617
-0.92
91.0
1.390
CatBoost-Local
48.810
1.452
1.802
-0.10
91.0
1.043
FTTTransforer--Lcc
54.164
1.511
1.884
-0.17
95.1
1.051
-TTrraassormme--llbbbll
50.872
1.337
1.797
-0.15
92.9
0.987
NIAQUE-Local
50.580
1.435
1.870
-0.18
96.6
1.006
NIAQUE-Globe
51.829
1.457
1.848
-0.15
97.0
1.019
Performance comparison 1193 BNG lowbwt dataset. Lower values better for all metrics except CT Coverage (target: 95%). For BIAS, lower absolute values better.
Table 11:
Model
SMAPE
AAD
RMSE
BIAS
COV895
CRPS
CatBoost-Gllbal
17.093
0.889
1.141
-0.18
95.9
0.658
CatBoost-Local
16.586
0.867
1.111
0.00
93.5
0.619
FTTIransformer-Lcaal
18.315
0.852
1.082
0.00
95.7
0.622
FT'Transformer-Giae
16.620
0.844
1.063
0.03
95.1
0.594
NIAQUE-Local
17.399
0.895
1.138
0.02
99.0
0.593
NIAQUE-Global
16.278
0.850
1.114
0.02
94.3
0.588
BNG echoMonths dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Table 12: Performance comparison on 1199
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
46.037
1.261
1.585
-0.27
98.3
0.912
CatBoost-Local
36.889
1.109
1.513
-0.11
93.4
0.810
PoTransformer--oooal
39.263
1.122
1.500
-0.10
96.3
0.798
W'lTrranfforme-Giae
34.530
1.017
1.542
-0.09
94.7
0.734
NIAQUE-Local
35.032
1.078
1.576
-0.09
97.0
0.787
NIAQUE-Global
35.024
1.089
1.536
-0.10
93.3
0.760
Table 13: Performance comparison 1201 BNG breastTumor dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
SatBoost-Glober
24.650
1.101
1.406
-0.11
95.9
0.822
CatBoost-Local
26.422
1.191
1.557
0.04
94.2
0.883
PTTransforrer--ocaal
30.854
1.162
1.645
0.02
90.5
0.894
FTTrranfformer-Goll
26.612
1.176
1.533
0.08
99.0
0.789
NIAQUE-Local
27.202
1.223
1.588
0.03
95.2
0.803
NIAQUE-Global
26.360
1.184
1.583
0.03
94.0
0.836
28

---

Published in Transactions on Machine Learning Research (02/2026)
Table 14: Performance comparison on 1203 BNG pel_earnneer dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
17.269
0.743
1.012
-0.07
95.7
0.586
CatBoost-Local
19.800
0.719
0.999
-0.02
95.1
0.562
FTTTranffomer--oca
21.944
0.735
1.084
-0.03
93.4
0.539
ITTTraussormer-Clllaal
19.310
0.751
0.992
-0.01
98.4
0.522
NIAQUE-Local
20.350
0.741
1.040
-0.02
94.8
0.534
NIAQUE-Globa
19.698
0.710
1.007
-0.02
94.7
0.509
Table 15: Performance comparison 197 epu act dalaset. Lower values better for all metrics except CI "CCoeeraee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
12.227
0.502
0.692
-0.03
87.7
0.398
CatBoost-Local
9.134
0.190
0.282
0.01
91.3
0.174
FTTTransformer-Lcal
11.919
0.218
0.321
0.00
90.4
0.152
FT'Transformer-Guue
9.138
0.179
0.259
0.02
96.1
0.126
NIAQUE-Local
9.722
0.186
0.273
0.01
95.8
0.133
NIAQUE-Global
9.395
0.178
0.262
0.01
93.5
0.130
2dplanes dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Table 16: Performance comparison 215
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
12.621
0.547
0.714
-0.02
98.4
0.442
CatBoost-Local
7.430
0.334
0.422
0.00
93.0
0.242
PTTransformer--oocal
9.219
0.351
0.472
-0.00
97.4
0.250
F' "I'Traasformer-Gue
7.024
0.336
0.409
0.01
92.0
0.219
NIAQUE-Local
7.817
0.340
0.441
0.01
92.2
0.237
NIAQUE-Global 7.459
0.334
0.423
0.01
93.2
0.236
Table 17: Performance comparison 218 house 8L dataset. Lower values better for all metrics except CI [_CCvrreeee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Gllhat
31.438
0.367
0.810
-0.17
86.1
0.275
CatBoost-Local
26.765
0.299
0.623
-0.10
93.4
0.229
FTTraanforrmr--Lccal
28.623
0.342
0.659
-0.07
96.5
0.245
FTTrranfformer-Gloo
25.992
0.272
0.588
-0.07
94.2
0.195
NIAQUE-Local
26.473
0.298
0.621
-0.06
95.8
0.212
NIAQUE-Global
25.339
0.286
0.602
-0.07
94.4
0.205
29

---

Published in Transactions on Machine Learning Research (02/2026)
Table 18: Performance comparison on 225 puma8NH dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
32.042
1.669
1.965
-0.72
91.7
1.173
CatBoost-Local
21.536
1.059
1.389
-0.01
92.6
0.776
FTTTranffomer--ocr
24.253
1.156
1.591
-0.02
89.2
0.804
ITTTrausformer-Clooba
24.234
1.136
1.436
-0.00
93.5
0.787
NIAQUE-Local
23.464
1.130
1.513
-0.01
91.0
0.749
NIAQUE-Globa
23.232
1.114
1.474
-0.02
89.8
0.782
Table 19: Performance comparison 227 cpu small dataset. Lower values better for all metrics except CI "CCoerreee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
12.680
0.510
0.644
-0.07
85.5
0.405
CatBoost-Local
9.630
0.221
0.325
0.02
91.3
0.197
FTTTransformer-ocal
12.332
0.253
0.361
0.00
92.5
0.183
FT'Transformer-Guue
9.393
0,215
0.313
0.02
93.4
0.154
NIAQUE-Local
10.314
0.219
0.322
0.01
99.0
0.153
NIAQUE-Global 9.818
0.212
0.307
0.01
94.9
0.153
Table 20: Performance comparison on 294 satellite image dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
66.647
1.312
1.917
-0.30
80.0
1.046
CatBoost-Local
58.329
0.902
1.592
-0.20
93.5
0.743
PoTransformer--oooal
57.452
0.688
1.418
-0.04
97.9
0.502
W'lTrranfforme-Guue
55.783
0.553
1.408
-0.02
99.0
0.434
NIAQUE-Local
53.936
0.492
1.311
-0.02
96.0
0.353
NIAQUE-Global
52.053
0.434
1.290
-0.03
97.8
0.316
Table 21: Performance comparison 344 dataset. Lower values better for all metrics except CI [CCCerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Gloha
5.877
0.292
0.548
-0.02
96.9
0.283
CatBoost-Local
1.268
0.051
0.097
-0.00
94.8
0.111
FTTransforrer--ocaal
3.056
0.069
0.101
-0.01
98.0
0.072
FTTraanfformer-Gloo
0.428
0.020
0.031
0.01
97.2
0.028
NIAQUE-Local
0.737
0.024
0.037
-0.00
99.0
0.022
NIAQUE-Global
0.219
0.013
0.017
-0.00
100.0
0.012
30

---

Published in Transactions on Machine Learning Research (02/2026)
Table 22: Performance comparison on 503 wind dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
29.173
0.956
1.248
-0.11
98.6
0.795
CatBoost-Local
17.484
0.544
0.702
0.02
93.2
0.394
FTTTransfomer--oca
21.583
0.584
0.765
0.02
94.0
0.413
ITTTrausformer-Clloba
18.283
0.521
0.722
0.04
95.5
0.370
NIAQUE-Local
17.955
0.545
0.710
0.03
96.7
0.379
NIAQUE-Globa
17.398
0.544
0.712
0.03
93.8
0.384
Table 23: Performance comparison 529 pollen dataset. Lower values better for all metrics except CI [CCoeraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
21.431
1.082
1.372
-0.66
98.4
0.777
CatBoost-Local
11.251
0.548
0.698
-0.02
90.4
0.396
-oTrransfrrerr-ocall
13.877
0.577
0.698
-0.03
90.3
0.396
FT'Transformer-Giae
10.601
0.523
0.677
-0.01
91.2
0.361
NIAQUE-Local
10.993
0.526
0.665
-0.01
87.3
0.390
NIAQUE-Global
10.937
0.525
0.679
-0.02
90.4
0.377
Table 24: Performance comparison 537 houses dataset. Lower values better for all metrics except CI [CCoveraee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
35.823
1.278
1.809
-0.30
96.0
0.977
CatBoost-Local
19.621
0.756
1.188
-0.19
94.2
0.583
PoTransformer--oooal
23.501
0.771
1.285
-0.16
91.3
0.582
V'lTrrasffrrmr-Guu
21.238
0,800
1.219
-0.12
94.1
0.585
NIAQUE-Loca
22.344
0.823
1.231
-0.13
94.9
0.600
NIAQUE-Global
22.198
0.819
1.263
-0.13
93.2
0.586
Table 25: Performance comparison 547_ no2 dataset. Lower values better for all metrics except CI [CCoerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Gloha
59.636
2.671
2.992
-2.51
100.0
1.573
CatBoost-Local
18.142
0.851
1.144
0.12
86.0
0.634
LTTrannforrer--ocaal
20.602
0.931
1.282
-0.01
83.8
0.731
FTTraanfformer-Gbbl
17.903
0.873
1.162
0.00
81.1
0.656
NIAQUE-Local
21.115
0.924
1.213
-0.00
85.2
0.665
NIAQUE-Global
20.253
0.925
1.189
-0.01
80.0
0.687
31

---

Published in Transactions on Machine Learning Research (02/2026)
Table 26: Performance comparison on 564 fried dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
11.919
0.522
0.677
0.01
97.9
0.421
CatBoost-Local
6.382
0.282
0.354
-0.01
93.5
0.204
FTTTransfomer--ect
8.052
0.281
0.380
-0.01
96.5
0.199
-TTrraassormme--llbbbll
5.930
0.261
0.324
0.01
95.2
0.170
NIAQUE-Local
6.489
0.271
0.343
-0.00
93.6
0.192
NIAQUE-Globa
5.978
0.262
0.330
-0.00
94.2
0.184
Performance comparison 593 [ri cl 1000 10 dataset. Lower values are better for all metrics except CT [overrge (target: 95%). For BIAS, lower absolute values better.
Table 27:
Model
SMAPE
AAD
RMSE
BIAS
COV895
CRPS
CatBoost-Global
61.047
2.633
3.026
-2.20
95.0
1.589
CatBoost-Local
8.834
0.343
0.443
0.08
84.0
0.282
FTTTransformer-Lcaal
10.481
0.352
0.464
0.05
91.9
0.245
FTTTransformer-Guu
8.592
0.301
0.398
0.06
84.0
0.230
NIAQUE-Local
9.019
0.322
0.421
0.07
84.6
0.225
NIAQUE-Global 8.447
0.297
0.398
0.05
89.0
0.215
Table 28: Performance comparison on 595 fri c0 1000 10 dataset. Lower values are better for all metrics
except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
56.409
2.543
3.044
-2.26
94.0
1.579
CatBoost-Local
10.569
0.456
0.569
-0.08
81.0
0.338
PTTransformer--oooal
12.767
0.458
0.577
0.01
81.3
0.332
W'lTrranfforme-Glae
9.962
0.413
0.489
0.03
81.1
0.282
NIAQUE-Local
10.247
0.433
0.531
0.02
81.1
0.309
NIAQUE-Global
9.801
0,412
0.510
0.02
81.0
0.297
Table 29: Performance comparison AIRFOIL dataset. Lower values better for all metrics except CI [CCCvereee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Gloha
49.769
2.340
2.733
-1.86
94.0
1.485
CatBoost-Local
13.736
0.583
0.786
0.04
90.7
0.439
FTTrannforrer--ocaal
12.358
0.500
0.728
0.07
97.6
0.324
FTTTranfformer-Goll
10.709
0.453
0.603
0.10
96.6
0.296
NIAQUE-Local
9.590
0.389
0.592
0.09
95.6
0.282
NIAQUE-Global
8.722
0.378
0.542
0.09
95.4
0.265
32

---

Published in Transactions on Machine Learning Research (02/2026)
Performance comparison on APARIMENTS dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Table 30:
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
37.753
0.099
0.177
0.01
93.9
0.076
CatBoost-Local
22.317
0.056
0.111
-0.02
93.7
0.042
FTTTrannformer-Loct
24.986
0.096
0.167
-0.02
94.3
0.053
ITTTrausformer-Clooaa
22.516
0.055
0.118
0.00
98.4
0.028
NIAQUE-Local
25.530
0.070
0.149
-0.01
96.7
0.044
NIAQUE-Globa
23.637
0.060
0.131
-0.01
94.7
0.042
Table 31: Performance comparison on Abalone dataset. Lower values better for all metries except CI [CCveraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
2.569
0.057
0.099
-0.00
94.9
0.054
CatBoost-Local
16.587
0.559
0.848
-0.15
90.2
0.416
FTTTransformer-ocal
17.623
0.531
0.845
-0.12
93.1
0.391
FT'Transformer-Guu
15.269
0.546
0.789
-0.14
93.3
0.399
NIAQUE-Local
17.429
0.554
0.835
-0.12
95.6
0.386
NIAQUE-Global
15.842
0.536
0.817
-0.14
93.1
0.383
Table 32: Performance comparison on Ailerons dataset. Lower values better for all metries except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
9.310
0.678
0.931
-0.15
95.1
0.524
CatBoost-Local
5.222
0.367
0.519
0.02
94.3
0.274
PoTransformer--oocll
7.749
0.412
0.586
0.03
95.0
0.282
"I'Traasformer-Gue
4.741
0.365
0.504
0.05
92.1
0.256
NIAQUE-Local
5.845
0.384
0.535
0.05
93.7
0.268
NIAQUE-Global
5.315
0.377
0.519
0.05
93.3
0.266
Table 33: Performance comparison AmericanHousePricea dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-
34.982
0.070
0.274
-0.03
92.0
0.053
CatBoost-Local
22.551
0.046
0.134
-0.01
94.2
0.039
FTTransforrer--occal
24.524
0.085
0.182
-0.02
95.7
0.047
FTTrranfformer-Gloo
22.340
0.041
0.128
0.00
96.9
0.019
NIAQUE-Local
22.585
0.051
0.154
-0.00
91.6
0.033
NIAQUE-Global
21.578
0.044
0.140
-0.01
93.3
0.032
33

---

Published in Transactions on Machine Learning Research (02/2026)
Table 34: Performance comparison on AvocadoPrices dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Globat
25.051
0.829
1.081
-0.16
99.1
0.689
CatBoost-Local
16.905
0.559
0.771
-0.05
94.5
0.416
FTTTranffomer--oca
14.009
0.494
0.657
0.08
90.2
0.332
ITTTraussormer-Clllaa
12.382
0.418
0.555
0.05
88.7
0.267
NIAQUE-Local
10.722
0.344
0.538
0.03
89.6
0.247
NIAQUE-GIos
10.054
0.331
0.492
0.03
88.1
0.241
Performance comparison BIAS CORRECTION dataset. Lower values better for all metrics except CT Coverage (target: 95%). For BIAS, lower absolute values better.
Table 35:
Model
SMAPE
AAD
RMSE
BIAS
COV895
CRPS
CatBoost-Gllbal
12.196
0.686
0.942
-0.28
98.1
0.590
CatBoost-Local
6.949
0.379
0.513
0.01
93.8
0.284
FTTTransformer-ocal
8.451
0.380
0.518
-0.00
93.2
0.269
FT'Transformer-Guu
6.301
0.346
0.439
0.02
95.6
0.238
NIAQUE-Local
6.460
0.341
0.463
0.01
94.3
0.242
NIAQUE-Global
5.814
0.328
0.436
0.01
95.3
0.232
Table 36: Performance comparison Big.artSales dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
59.011
0.741
0.925
0.16
93.9
0.554
CatBoost-Local
41.202
0.555
0.794
-0.03
94.4
0.394
PTTransformer--oocal
44.637
0.602
0.901
-0.04
95.4
0.393
F' "T'Traasformer-Guu
40,186
0.568
0.876
-0.03
99.0
0.387
NIAQUE-Loca
39.697
0.569
0.834
-0.03
97.3
0.397
NIAQUE-Global
40.960
0.562
0.823
-0.03
95.3
0.391
Table 37: Performance comparison BlackFriday dataset. Lower values better for all metrics except CI [_CCerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globa
27.779
1.217
1.578
0.23
92.4
0.876
CatBoost-Local
27.003
1.187
1.567
0.26
94.0
0.858
FTTraanforrmr--occal
27.843
1.243
1.553
0.18
95.9
0.814
FTTTranfformer-Goll
27.469
1.179
1.661
0.22
90.6
0.834
NIAQUE-Local
27.271
1.213
1.562
0.26
96.7
0.787
NIAQUE-Global
27.051
1.182
1.574
0.23
95.7
0.823
34

---

Published in Transactions on Machine Learning Research (02/2026)
Table 38: Performance comparison on BookSales dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
131.764
0.928
1.367
0.45
74.8
0.752
CatBoost-Local
41.686
0.239
0.938
-0.14
84.1
0.213
FTTTransfomer--oct
38.638
0.282
1.006
-0.12
94.6
0.201
-TTrraassormme--llbbbll
38.462
0.241
0.869
-0.08
92.4
0.179
NIAQUE-Local
38.616
0.247
0.894
-0.12
94.9
0.174
NIAQUE-Globa
37.963
0.245
0.846
-0.11
95.3
0.173
Table 39: Performance comparison BrasilianHouses dalaset. Lower values better for all metrics except CI [CCveraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
47.270
0.019
0.031
-0.00
87.6
0.018
CatBoost-Local
32.904
0.014
0.024
-0.00
93.9
0.010
FTTIransformer-ocaal
33.447
0.053
0.069
-0.01
93.9
0.021
FT'Transformer-Guu
33.888
0.011
0.014
0.00
93.8
0.000
NIAQUE-Local
33.455
0.022
0.038
-0.00
95.7
0.010
NIAQUE-Global
33.171
0.014
0.024
-0.01
94.7
0.010
Table 40: Performance comparison on CCPP dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
20.671
0.876
1.252
-0.43
97.7
0.672
CatBoost-Local
10.508
0.364
0.497
0.03
95.8
0.271
PTTransformer--oooal
13.392
0.423
0.576
0.01
98.3
0.278
F' "I'Transformer-Gur
11.360
0.384
0.531
0.04
98.6
0.256
NIAQUE-Local
11.885
0.395
0.571
0.02
98.5
0.279
NIAQUE-Global 11.568
0.402
0.542
0.02
96.0
0.284
Table 41:
Performance comparison CONCRETE, COMPRESSIVE. STRENGTH dataset. Lower values
are better for all metrics except CI _CCCeraeee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globar
53.356
2.066
2.489
-1.52
97.1
1.317
CatBoost-Local
13.376
0.531
0.706
-0.22
84.5
0.390
FTTraanforrmr--occal
16.743
0.610
0.808
-0.25
91.9
0.402
FTTraanfformer-Gbbl
14.757
0.587
0.734
-0.27
93.0
0.402
NIAQUE-Local
15.572
0.600
0.760
-0.22
85.6
0.412
NIAQUE-Global
15.097
0.602
0.773
-0.24
90.3
0.418
35

---

Published in Transactions on Machine Learning Research (02/2026)
Table 42: Performance comparison on CPMP-2015 dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
140.648
2.600
3.943
-2.10
73.9
1.809
CatBoost-Local
93.345
0.684
1.827
-0.53
87.2
0.602
FTTTransfomer--Lcc
90.263
0.643
1.852
-0.47
92.4
0.522
OTTrraassormme--llbball
89.302
0.608
1.711
-0.46
95.2
0.468
NIAQUE-Local
86.994
0.582
1.677
-0.40
94.1
0.463
NIAQUE-GER 84.264
0.564
1.650
-0.42
94.3
0.421
Table 43: Performance comparison CalHousing dataset. Lower values better for all metrics except CI [CCveraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV895
CRPS
CatBoost-Gllbal
35,828
1.279
1.809
96.1
0.976
CatBoost-Local
19.653
0.757
1.192
-0.19
94.5
0.587
FTTTransformer-ocal
22.615
0.819
1.225
-0.14
93.3
0.595
FTTTransformer-Guu
21.469
0.782
1.207
-0.10
93.2
0.586
NIAQUE-Local
23.100
0.819
1.229
-0.11
92.2
0.597
NIAQUE-Global
22.268
0.820
1.258
-0.14
93.1
0.586
Table 44: Performance comparison CampaignUplit dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
64.266
1.163
1.343
0.82
97.5
0.862
CatBoost-Local
11.578
0.127
0.204
0.03
92.0
0.100
PTTransformer--oooal
13.735
0.145
0.212
0.01
92.8
0.100
F' "T'Trrasformer-Gur
10.392
0.105
0.143
0.03
95.0
0.067
NIAQUE-Local
10.560
0.117
0.148
0.02
96.8
0.074
NIAQUE-Global
10.280
0.105
0.132
0.02
96.0
0.073
Table 45: Performance comparison CapitalGain dataset. Lower values are better for all metrics except CI [CCCerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Gllbbt
10.193
0.116
0.749
-0.09
92.5
0.131
CatBoost-Local
9.952
0.131
0.862
-0.11
94.0
0.156
LTTrannforrer--ocaal
12.274
0.168
0.911
-0.13
95.6
0.141
FTTraanfformer-Gloo
9.373
0.136
0.866
-0.11
98.5
0.105
NIAQUE-Local
9.938
0.141
0.859
-0.11
95.8
0.113
NIAQUE-Global
9.749
0.131
0.863
-0.11
94.5
0.111
36

---

Published in Transactions on Machine Learning Research (02/2026)
Table 46: Performance comparison on Cars dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
15.195
0.160
0.272
-0.01
97.9
0.129
CatBoost-Local
10.225
0.132
0.218
-0.02
94.2
0.099
FTTTransfomer--Lcc
12.543
0.168
0.238
-0.02
95.6
0.114
PTTrraussormer-Clobblll
10.181
0.123
0.179
0.00
93.4
0.077
NIAQUE-Local
10.467
0.127
0.195
-0.01
94.8
0.090
NIAQUE-Globa
10.028
0.120
0.180
-0.01
94.1
0.085
Table 47: Performance comparison on Co2Emission dataset. Lower values better for all metrics except CI [CCveraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
8,610
0.281
0.556
-0.07
98.6
0.446
CatBoost-Local
2.043
0.068
0.140
0.02
94.6
0.058
FT'Iransformer-Lccal
4.113
0.098
0.173
0.00
99.0
0.059
FT'Transformer-Guu
1.568
0.056
0.101
0.02
97.7
0.031
NIAQUE-Local
2.106
0.065
0.122
0.02
97.6
0.043
NIAQUE-Global
1.628
0.056
0.104
0.01
98.1
0.041
Table 48: Performance comparison on CasS&Wages dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
59.548
0.208
0.364
0.06
94.5
0.160
CatBoost-Local
42.185
0.205
0.322
-0.05
95.0
0.155
PoTransformer--oooll
43.132
0.260
0.386
-0.05
93.7
0.161
V'lTrrasffrrmr-Guu
43.363
0,203
0.319
-0.04
97.1
0.133
NIAQUE-Local
42.586
0.212
0.336
-0.05
97.6
0.146
NIAQUE-Global
42.222
0.205
0.323
-0.05
95.0
0.146
Table 49: Performance comparison Debutanizer dataset. Lower values are better for all metrics except CI [CCCerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globa
41.923
1.125
1.596
0.27
96.2
0.946
CatBoost-Local
23.651
0.646
1.183
-0.26
92.5
0.511
FTTrannforrmr--occal
24.355
0.601
1.075
-0.05
93.6
0.444
FTTTranfformer-Goll
22.909
0.564
0.990
-0.04
97.8
0.411
NIAQUE-Local
21.878
0.512
0.904
-0.05
94.8
0.378
NIAQUE-Global
21.048
0.515
0.878
-0.04
93.8
0.381
37

---

Published in Transactions on Machine Learning Research (02/2026)
Table 50: Performance comparison on DeltaElevators dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
8.697
0.434
0.593
-0.20
97.7
0.410
CatBoost-Local
7.759
0.384
0.519
-0.00
94.3
0.291
FTTTransfomer--Lcc
9.916
0.420
0.556
-0.01
93.6
0.294
PTTrraussormme--llbbbll
8.046
0.393
0.502
0.01
97.4
0.273
NIAQUE-Local
8.362
0.406
0.543
-0.00
94.3
0.270
NIAQUE-Globe
7.965
0.393
0.533
-0.01
94.6
0.280
Table 51: Performance comparison on Diamonds dataset. Lower values better for all metrics except CI [CCoerreee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
6.633
0.038
0.116
-0.00
94.7
0.037
CatBoost-Local
2.052
0.021
0.045
-0.00
94.5
0.020
FT'Iransformer-Lccal
4.179
0.059
0.084
-0.01
98.5
0.027
FT'Transformer-Guue
1.739
0.015
0.025
0.01
99.0
0.001
NIAQUE-Local
2.275
0.026
0.048
0.00
94.3
0.013
NIAQUE-Global
1.807
0.017
0.033
0.00
96.8
0.013
Table 52: Performance comparison on Ele2 dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
102.764
2.074
2.331
1.28
85.8
1.448
CatBoost-Local
11.076
0.098
0.162
-0.01
86.8
0.088
PTTransformer--oocal
10.716
0.117
0.173
-0.01
95.0
0.075
W'lTrranfformr-Glae
8.807
0.067
0.102
0.01
97.6
0.041
NIAQUE-Loca
8.329
0.070
0.109
0.00
98.5
0.045
NIAQUE-Global
7.524
0.059
0.091
0.00
98.1
0.011
Table 53: Performance comparison Energy Efficienc dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globar
64.138
2.377
2.902
-1.16
84.4
1.520
CatBoost-Local
4.014
0.095
0.144
0.03
96.1
0.083
-TTrannforrer-Locall
9.404
0.184
0.256
-0.05
99.0
0.133
FTTrranfformer-Glal
8.063
0.176
0.222
-0.04
97.8
0.110
NIAQUE-Local
9.406
0.201
0.271
-0.04
99.0
0.144
NIAQUE-Global
9.353
0.199
0.270
-0.04
97.4
0.145
38

---

Published in Transactions on Machine Learning Research (02/2026)
Table 54: Performance comparison on Fardamento dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Globat
102.512
0.053
0.164
-0.02
77.2
0.050
CatBoost-Local
95.195
0.047
0.161
-0.03
79.3
0.042
FTTTransfomer--oca
92.805
0.091
0.300
-0.03
92.4
0.051
OTTrranssormme--llbbbll
92.013
0.054
0.230
-0.01
95.4
0.028
NIAQUE-Local
101.712
0.062
0.306
-0.02
94.0
0.042
NIAQUE-Globa
97.806
0.059
0.294
-0.02
98.7
0.041
Table 55: Performance comparison Fifa dataset.
Lower values better for all metrics except CI [oCeeraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
77.475
0.130
0.324
-0.05
82.5
0.096
CatBoost-Local
72.034
0.114
0.276
-0.02
89.3
0.086
FTTTransformer-ocal
73.861
0.151
0.319
-0.03
95.0
0.099
FTTTransformer-Giae
76.096
0.109
0.254
-0.00
93.7
0.070
NIAQUE-Local
74.066
0.122
0.293
-0.01
96.3
0.084
NIAQUE-Global
74.935
0.115
0.278
-0.02
94.9
0.081
Table 56: Performance comparison on FpsBenchmark dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
30.554
0.785
1.090
-0.04
99.0
0.642
CatBoost-Local
6.420
0.149
0.308
-0.03
95.0
0.151
PTTransformer--oocal
11 11.348
0.229
0.468
-0.00
99.0
0.163
F' "I'Traasformer-Gue
9.100
0,212
0.423
0.02
99.0
0.136
NIAQUE-Local
10.185
0.240
0.507
0.01
99.0
0.148
NIAQUE-Global 10.501
0.242
0.496
0.01
98.6
0.148
Table 57: Performance comparison CemstonePrice dataset. Lower values better for all metrics except CI [CCCerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Gloha
24.207
0.272
0.512
-0.02
96.2
0.210
CatBoost-Local
11.621
0.184
0.376
0.00
95.4
0.154
-TTransforrer-Locall
13.563
0.204
0.362
0.00
95.7
0.137
FTTTranfformer-Gool
11.080
0.159
0.313
0.02
92.0
0.107
NIAQUE-Local
11.018
0.163
0.316
0.02
94.7
0.110
NIAQUE-Global
10.824
0.153
0.297
0.01
94.5
0.109
39

---

Published in Transactions on Machine Learning Research (02/2026)
Table 58: Performance comparison on GpulKernelPerfommance dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
43.784
0.221
0.668
-0.14
93.3
0.156
CatBoost-Local
51.879
0.255
0.757
-0.15
94.1
0.199
FTTTranffomer--ocr
29.646
0.168
0.398
-0.01
94.1
0.114
ITTTraussormer-Clllaal
22.349
0.089
0.249
0.01
96.2
0.061
NIAQUE-Local
12.550
0.052
0.129
0.00
97.6
0.033
NIAQUE-Globa 8.881
0.026
0.064
0.00
94.2
0.019
Table 59: Performance comparison on GridStability dataset. Lower values better for all metrics except CI [CCverreee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
27.938
1.315
1.553
-0.42
98.0
0.938
CatBoost-Local
9.966
0.370
0.518
-0.01
92.7
0.279
FTTTransformer-Lcaal
9.250
0.274
0.424
0.00
93.9
0.189
FT'Transformer-Guu
5.975
0,212
0.311
0.02
97.1
0.137
NIAQUE-Local
5.272
0.167
0.271
0.01
92.0
0.121
NIAQUE-Global
4.564
0.144
0.230
0.01
92.9
0.104
Table 60: Performance comparison on HouseRent dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
66.354
0.059
0.127
-0.01
87.8
0.052
CatBoost-Local
31.500
0.028
0.068
-0.01
92.8
0.022
PoTransformer--oooal
31.390
0.066
0.113
-0.01
95.5
0.031
F' "I'Traasformer-Gua
29.643
0.024
0.057
0.00
94.5
0.006
NIAQUE-Loca
30.469
0.035
0.082
-0.00
93.6
0.019
NIAQUE-Global
30.273
0.026
0.067
-0.00
93.3
0.019
Table 61: Performance comparison IEFF80211aaGA. dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globar
26.743
1.000
1.585
0.36
84.2
0.866
CatBoost-Local
12.939
0.261
0.542
-0.05
94.0
0.338
PTTransforrer--ocaal
10.248
0.250
0.452
-0.01
96.4
0.227
FTTrranfformer-Gbbl
7.344
0.190
0.392
0.01
96.4
0.181
NIAQUE-Local
5.817
0.182
0.367
0.00
99.0
0.139
NIAQUE-Global
4.736
0.167
0.319
0.00
97.2
0.122
40

---

Published in Transactions on Machine Learning Research (02/2026)
Performance comparison on Infrared Thermography Temperature dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Table 62:
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
23.255
0.743
1.103
-0.20
100.0
0.780
CatBoost-Local
14.499
0.416
0.529
0.02
90.2
0.292
FTTTrannformer-Loct
16.384
0.473
0.599
0.02
98.5
0.303
ITTTrausformer-Clllaal
14.692
0.428
0.524
0.04
96.4
0.277
NIAQUE-Local
15.061
0.424
0.560
0.03
94.6
0.291
NIAQUE-Globa 14.471
0.420
0.542
0.03
95.1
0.295
Table 63: Performance comparison on KindleBooks dataset. Lower values better for all metrics except CI [CCveraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Gllbal
33.753
0.055
0.165
89.5
0.046
CatBoost-Local
29.304
0.046
0.159
-0.02
94.5
0.042
FTTTransformer-ocal
32.212
0.082
0.198
-0.02
98.1
0.050
FTTTransformer-Guue
28.591
0.042
0.123
-0.00
96.6
0.020
NIAQUE-Local
30.017
0.052
0.156
-0.01
97.7
0.034
NIAQUE-Global
28.834
0.043
0.143
-0.01
94.8
0.032
Table 64: Performance comparison Kinematics dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
30.256
1.334
1.687
-0.58
96.8
0.969
CatBoost-Local
20.933
0.832
1.102
0.11
92.4
0.611
PoTransformer--oocll
17.407
0.660
0.780
-0.01
87.3
0.453
"i'Traasformer-Guu
12.528
0.522
0.645
0.01
86.5
0.356
NIAQUE-Loca
11.584
0.414
0.542
0.00
86.9
0.282
NIAQUE-Global
10.113
0.371
0.474
0.00
82.2
0.267
Table 65: Performance comparison KingCounty dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-
26.667
0.172
0.387
-0.05
96.2
0.129
CatBoost-Local
15.617
0.095
0.176
-0.02
93.8
0.075
FTTransforrer--occal
25.275
0.166
0.253
-0.03
96.6
0.108
FTTraanfformer-Gloo
23.652
0.128
0.198
-0.01
98.6
0.081
NIAQUE-Local
26.563
0.160
0.245
-0.02
95.3
0.104
NIAQUE-Global
27.154
0.152
0.236
-0.02
95.2
0.106
41

---

Published in Transactions on Machine Learning Research (02/2026)
Table 66: Performance comparison on LaptopPrice dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
67.065
1.348
1.531
0.98
97.7
0.995
CatBoost-Local
20.932
0.372
0.598
-0.12
91.4
0.271
FTTTranformer--oer
22.567
0.402
0.621
-0.11
87.6
0.274
ITTTraussormer-Clloba
19.003
0.358
0.572
-0.09
86.4
0.235
NIAQUE-Local 19.326
0.367
0.582
-0.11
84.0
0.261
NIAQUE-Globa
18.936
0.354
0.558
-0.10
84.4
0.257
Table 67: Performance comparison on Laser dataset. Lower values better for all metrics except CI CCooeraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
66.495
1.731
2.052
0.32
99.0
1.185
CatBoost-Local
6.804
0.138
0.397
0.05
91.0
0.171
FTTTransformer-Lcal
7.275
0.142
0.289
-0.00
93.3
0.114
FT'Transformer-Giae
4.282
0.082
0.193
0.02
98.1
0.072
NIAQUE-Local
4.375
0.075
0.151
0.01
97.7
0.056
NIAQUE-Global
3.770
0.064
0.116
0.01
98.0
0.047
Table 68: Performance comparison on LifeExpectancy dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
15.881
0.890
1.184
-0.52
99.3
0.687
CatBoost-Local
5.229
0.269
0.412
0.01
94.2
0.228
PTTransformer--oocal
6.853
0.269
0.407
-0.03
95.9
0.198
F' V''Trrasformmr-Guu
3.824
0,211
0.347
-0.01
98.7
0.149
NIAQUE-Local
4.072
0.203
0.337
-0.02
99.0
0.146
NIAQUE-Global 3.494
0.190
0.313
-0.02
93.9
0.142
Table 69: Performance comparison LoanAmount dataset. Lower values better for all metrics except CI [CCCerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Gloke
49.617
0.232
0.529
0.06
81.8
0.231
CatBoost-Local
46.025
0.213
0.531
0.12
78.3
0.213
FTTransforrer--ocaal
38.888
0.250
0.587
0.09
93.3
0.197
FTTTranfforme--Gla
32.407
0.189
0.513
0.11
92.0
0.167
NIAQUE-Local
27.394
0.205
0.543
0.10
92.0
0.152
NIAQUE-Global
26.088
0.191
0.518
0.11
97.4
0.153
42

---

Published in Transactions on Machine Learning Research (02/2026)
Table 70: Performance comparison on CarketingCampaigu dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
34.478
0.594
0.859
0.18
99.3
0.452
CatBoost-Local
17.050
0.353
0.719
-0.09
66.8
0.272
FTTTransfomer--oca
18.104
0.393
0.791
-0.07
88.4
0.273
ITTTraussormer-Clooba
15.907
0.298
0.743
-0.04
90.5
0.219
NIAQUE-Local
15.505
0.334
0.775
-0.05
87.2
0.234
NIAQUE-Globa
14.484
0.321
0.731
-0.05
93.0
0.230
Table 71: Performance comparison on MedicalCost dataset. Lower values better for all metrics except CI ['Cveraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Gllbal
96.062
2.061
2.317
0.97
86.6
1.412
CatBoost-Local
15.806
0.364
0.845
-0.12
94.8
0.369
FTTIransformer-Lcaal
17.090
0.346
0.823
-0.16
92.9
0.291
FT'Transformer-Guu
14.432
0,281
0.764
-0.15
93.7
0.234
NIAQUE-Local
14.276
0.271
0.753
-0.12
94.7
0.220
NIAQUE-Global
13.666
0.260
0.757
-0.15
94.0
0.208
Table 72: Performance comparison on MiamiHousing dataset. Lower values are better for all metrics except CI [CCoveraee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
29.994
0.390
0.779
-0.06
97.3
0.299
CatBoost-Local
15.589
0.194
0.381
-0.03
95.2
0.157
PTTransformer--oooal
17.107
0.212
0.375
-0.02
92.1
0.145
F' "I'Traasformer-Guu
13.926
0.155
0.312
-0.00
93.9
0.116
NIAQUE-Local
13.441
0.169
0.313
-0.01
97.8
0.116
NIAQUE-Global
13.173
0.155
0.292
-0.01
92.8
0.110
Table 73: Performance comparison Mortgage dataset. Lower values better for all metrics except CI _Cooeragee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globa
64.387
1.814
2.337
-0.02
85.7
1.276
CatBoost-Local
6.670
0.077
0.113
0.00
92.4
0.082
FTTrannforrer--occal
9.029
0.115
0.162
-0.01
99.0
0.075
FTTraanfformer-Gloo
6.582
0.076
0.110
0.00
98.6
0.048
NIAQUE-Local
6.918
0.085
0.133
-0.00
97.5
0.056
NIAQUE-Global
6.398
0.077
0.119
-0.00
97.1
0.053
43

---

Published in Transactions on Machine Learning Research (02/2026)
Performance comparison on NASA-PHLM200S dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Table 74:
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
37.375
0.987
1.326
-0.31
97.3
0.718
CatBoost-Local
31.464
0.834
1.143
-0.17
93.8
0.596
FTTTranffomer--oca
33.765
0.788
1.232
-0.15
92.1
0.615
ITTTraussormer--lllaal
28.671
0.781
1.130
-0.14
93.5
0.566
NIAQUE-Local
30.348
0.854
1.162
-0.14
95.4
0.555
NIAQUE-Globa
29.978
0.813
1.127
-0.13
94.0
0.564
Table 75: Performance comparison NewFuelCar dataset. Lower values are better for all metrics except CI [CCoerreee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
18.349
0.278
0.461
97.2
0.310
CatBoost-Local
2.649
0.061
0.107
0.02
93.3
0.057
FT'Iransformer-Lccal
4.391
0.092
0.138
-0.00
96.3
0.056
FT'Transformer-Guu
1.779
0.047
0.077
0.02
98.5
0.028
NIAQUE-Local
2.034
0.055
0.094
0.01
98.5
0.036
NIAQUE-Global
1.550
0.046
0.081
0.01
95.5
0.034
Table 76: Performance comparison on NewsPopularity dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
53.910
0.028
0.157
-0.02
87.0
0.025
CatBoost-Local
51.060
0.035
0.194
-0.03
94.2
0.034
PTTransformer--oooal
54.237
0.073
0.237
-0.04
99.0
0.042
F' "I'Traasformer-Guu
48.657
0.032
0.185
-0.02
96.9
0.016
NIAQUE-Local
51.937
0.043
0.209
-0.02
95.9
0.030
NIAQUE-Global
51.099
0.035
0.191
-0.03
95.7
0.028
Table 77: Performance comparison PRODUCTIVITY dataset. Lower values better for all metrics except CI ['_Cerreeee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globa.
63.361
2.803
3.161
-2.52
93.3
1.674
CatBoost-Local
20.267
0.840
1.418
0.16
91.7
0.677
LTTransforrer--ocall
22.318
0.816
1.361
-0.03
91.0
0.598
FTTTranfformr-Gloll
18.342
0.761
1.317
-0.02
91.9
0.579
NIAQUE-Local
17.589
0.733
1.311
-0.03
92.1
0.574
NIAQUE-Global
17.487
0.727
1.317
-0.03
90.8
0.555
44

---

Published in Transactions on Machine Learning Research (02/2026)
Table 78: Performance comparison on Parkinsons Telemonitoring dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
42.713
1.699
2.220
-0.98
94.7
1.189
CatBoost-Local
19.506
0.667
0.980
0.03
94.9
0.495
FTTTranffomer--oct
17.967
0.573
0.871
-0.12
95.8
0.409
ITTTrausforme--Clllbll
15.604
0.474
0.885
-0.09
95.3
0.339
NIAQUE-Local
14.407
0.428
0.836
-0.12
95.9
0.303
NIAQUE-Globa
13.267
0.404
0.818
-0.12
94.7
0.287
Table 79: Performance comparison Pole dataset. Lower values better for all metrics except CI [oCeeraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Gllbal
133.955
0,281
0.678
58.5
0.393
CatBoost-Local
139.171
0.719
1.462
0.11
60.5
0.699
FTTTransformer-ocall
131.589
0.418
0.921
-0.03
87.1
0.400
FTTTransformer-Gloe
135.896
0.301
0.706
-0.01
88.7
0.264
NIAQUE-Local
133.800
0.182
0.460
-0.02
90.0
0.144
NIAQUE-Global
133.544
0.125
0.368
-0.02
98.5
0.091
Table 80: Performance comparison on ProteinTertiary dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
61.606
1.996
2.515
-0.48
93.9
1.374
CatBoost-Local
47.676
1.575
2.244
-0.29
93.7
1.165
PTTransformer--oocal
42.046
1.288
2.124
-0.11
94.6
0.969
F' WTlTrranformmr-Glae
38,191
1.248
1.990
-0.07
95.1
0.892
NIAQUE-Local
33.479
1.148
1.914
-0.10
97.0
0.804
NIAQUE-Global
32.642
1.136
1.874
-0.10
93.8
0.807
Table 81: Performance comparison PumaDyn32nh datasot. Lower values are better for all metrics except CI [CCCerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Glooa
23.296
1.045
1.338
-0.56
97.3
0.755
CatBoost-Local
17.175
0.735
0.925
-0.05
92.1
0.530
FTTransforrer-Locall
21.163
0.746
0.964
-0.02
86.4
0.562
FTTTranfformer-Gool
17.833
0.737
0.978
-0.01
88.3
0.536
NIAQUE-Local
18.709
0.773
0.939
-0.01
88.9
0.545
NIAQUE-Global
18.663
0.762
0.953
-0.01
86.2
0.545
45

---

Published in Transactions on Machine Learning Research (02/2026)
Table 82: Performance comparison on QsarFishToxicity dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
45.093
1.680
2.043
-1.27
98.9
1.123
CatBoost-Local
20.804
0.709
1.004
0.10
84.6
0.546
PTTTrannformer-Locl
23.345
0.757
1.093
0.02
87.6
0.521
ITTTraussormer--lllaa
20.162
0.720
1.005
0.05
84.7
0.509
NIAQUE-Local
20.736
0.707
0.981
0.04
85.9
0.535
NIAQUE-Globe
20.313
0.704
0.963
0.03
84.6
0.515
Table 83: Performance comparison RedWine dataset. Lower values better for all metrics except CI "CCoeereee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV895
CRPS
CatBoost-Global
28.508
1.486
2.075
-1.32
98.1
1.147
CatBoost-Local
17.303
0.909
1.238
-0.07
93.8
0.683
FTTTransformer-ocal
18.635
0.914
1.236
-0.19
97.6
0.637
FT'Transformer-Guue
16.686
0.863
1.248
-0.20
97.9
0.599
NIAQUE-Local
16.889
0.874
1.339
-0.16
97.2
0.608
NIAQUE-Global
16.112
0.844
1.302
-0.18
94.4
0.582
Table 84: Performance comparison on Sales dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
63.524
0.277
0.471
-0.04
87.0
0.221
CatBoost-Local
58.466
0.227
0.415
-0.09
84.9
0.163
PTTransformer--oooal
61 61.860
0.253
0.414
-0.06
99.0
0.167
"I'Traasformer-Gua
57.553
0,212
0.330
-0.04
95.3
0.141
NIAQUE-Local
58.734
0.214
0.348
-0.06
93.4
0.146
NIAQUE-Global
57.967
0.205
0.334
-0.05
96.4
0.143
Table 85: Performance comparison Sarcos datasct. Lower values better for all metrics except CI [CCCerreee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globar
5.244
0.252
0.408
-0.03
98.4
0.245
CatBoost-Local
3.257
0.151
0.230
0.00
94.2
0.122
FTTraanforrmr--occal
4.846
0.166
0.226
-0.01
95.4
0.098
FTTraanfformer-Glol
2.112
0.106
0.152
0.01
97.2
0.071
NIAQUE-Local
2.572
0.109
0.162
0.00
96.0
0.073
NIAQUE-Global
1.985
0.094
0.141
0.00
94.0
0.068
46

---

Published in Transactions on Machine Learning Research (02/2026)
Table 86: Performance comparison on SaudiArabiaCars dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
50.282
0.504
0.986
-0.05
96.0
0.398
CatBoost-Local
23.424
0.246
0.706
-0.10
93.8
0.208
FTTTranformer--ocr
28.061
0.329
0.747
-0.09
91.3
0.227
ITTTraussormer-Clloda
27.649
0.278
0.662
-0.05
93.2
0.213
NIAQUE-Local
28.392
0.287
0.692
-0.07
89.7
0.207
NIAQUE-Globa
29.542
0.285
0.672
-0.06
89.3
0.215
Table 87: Performance comparison on SongPopularity dataset. Lower values better for all metrics except CI [CCveraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
38.240
1.738
2.164
94.2
1.285
CatBoost-Local
35.011
1.570
2.060
0.38
94.3
1.170
FTTIransformer-Lcaal
38.622
1.704
1.963
0.36
94.6
1.130
FTTTrrasformer-Gur
35.259
1.563
2.100
0.33
94.9
1.079
NIAQUE-Local
37.494
1.533
2.059
0.33
96.2
1.102
NIAQUE-Global
35.045
1.542
2.044
0.35
94.1
1.101
Table 88: Performance comparison on SpaceGa dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
6.846
0.553
0.703
-0.30
94.5
0.412
CatBoost-Local
3.350
0.265
0.400
0.03
92.6
0.202
PoTransformer--oooal
6.763
0.392
0.534
0.04
96.4
0.283
V'lTrranforrmr-Gur
4.684
0.370
0.519
0.06
94.0
0.261
NIAQUE-Local
5.636
0.434
0.581
0.05
95.7
0.312
NIAQUE-Global
5.574
0.448
0.588
0.06
95.8
0.315
Table 89: Performance comparison Student. Performance dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-
78.337
3.903
4.176
-3.81
72.3
2.336
CatBoost-Local
17.487
1.052
1.421
-0.32
87.7
0.837
PTTransformer-Locall
20.403
1.120
1.427
-0.11
86.2
0.809
FTTTranfformer-Goll
17.268
1.011
1.434
-0.08
87.3
0.787
NIAQUE-Local
17.589
1.050
1.410
-0.10
82.8
0.754
NIAQUE-Global
16.656
1.015
1.424
-0.11
87.7
0.768
47

---

Published in Transactions on Machine Learning Research (02/2026)
Table 90: Performance comparison on Sulfur dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
20.719
0.161
0.289
0.04
98.
0.133
CatBoost-Local
14.665
0.116
0.203
-0.01
93.6
0.086
FTTTransfomer--Lcct
15.814
0.151
0.221
-0.01
94.1
0.090
PTTrraassormee--llbbbll
13.591
0.108
0.166
0.01
92.8
0.065
NIAQUE-Local
14.458
0.108
0.178
0.01
94.1
0.075
NIAQUE-Globa
13.703
0.104
0.158
0.00
95.8
0.073
Table 91: Performance comparison TETOUAN dataset. Lower values better for all metrics except CI CCCveraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
14.156
0.625
0.824
-0.05
98.8
0.534
CatBoost-Local
10.438
0.448
0.629
-0.01
94.9
0.334
FTTTransformer-Lcall
11.680
0.416
0.583
-0.01
95.7
0.313
FTTTrrasformer-Gur
9.089
0.369
0.543
0.01
99.0
0.271
NIAQUE-Local
9.059
0.360
0.546
0.00
92.9
0.260
NIAQUE-Global
8.316
0.359
0.515
-0.00
93.5
0.255
Table 92: Performance comparison TitanicFare dataset. Lower values better for all metrics except CI [CCoveraee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
92.115
0.590
0.759
0.45
78.6
0.441
CatBoost-Local
22.778
0.175
0.511
-0.04
93.9
0.173
PoTransformer--oooal
25.910
0.208
0.530
-0.05
94.1
0.153
V'lTrrasformmr-Guu
21.883
0.156
0.439
-0.03
90.7
0.114
NIAQUE-Local
22.496
0.164
0.464
-0.04
90.9
0.122
NIAQUE-Global
21.963
0.155
0.450
-0.04
90.8
0.117
Table 93: Performance comparison Transcoding dataset. Lower values are better for all metrics except CI [CCCerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globa
28.703
0.120
0.385
-0.08
93.8
0.086
CatBoost-Local
21.052
0.119
0.367
-0.06
94.3
0.093
LTTraanforrer--occal
15.672
0.112
0.248
-0.00
95.0
0.062
FTTrranfformer-Glal
11.171
0.052
0.149
0.01
90.7
0.028
NIAQUE-Local
9.435
0.044
0.109
0.01
91.9
0.027
NIAQUE-Global
7.671
0.029
0.072
0.00
95.1
0.020
48

---

Published in Transactions on Machine Learning Research (02/2026)
Table 94: Performance comparison on Treasury dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
70.550
1.693
2.103
0.41
83.8
1.225
CatBoost-Local
7.815
0.101
0.211
-0.00
84.8
0.091
FTTTransfomer--ect
9.984
0.129
0.229
-0.04
91.9
0.087
ITTTraussormer-Clooaa
7.103
0.086
0.161
-0.01
95.7
0.055
NIAQUE-Local
7.784
0.091
0.163
-0.02
95.8
0.066
NIAQUE-Globe
7.227
0.082
0.145
-0.03
95.2
0.062
Table 95: Performance comparison UberFare dataset. Lower values better for all metrics except CI CCCoerreee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
4.803
0.080
0.180
-0.02
94.2
0.061
CatBoost-Local
9.736
0.122
0.304
-0.04
94.2
0.096
FTTIransformer-Lcaal
17.967
0.238
0.498
-0.16
96.0
0.168
FT'Transformer-Guue
18.255
0,226
0.451
-0.12
94.5
0.158
NIAQUE-Local
21.310
0.266
0.528
-0.13
96.9
0.202
NIAQUE-Global
21.892
0.274
0.541
-0.16
94.1
0.202
Table 96: Performance comparison UsedCar dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
47.910
0.277
0.469
0.03
97.0
0.215
CatBoost-Local
19.392
0.111
0.237
-0.02
93.4
0.088
PoTransformer--oooal
23.181
0.152
0.287
-0.01
90.2
0.102
W'lTrranfforme-Glae
21.556
0.116
0.235
0.01
94.2
0.073
NIAQUE-Loca
23.584
0.129
0.266
0.00
93.7
0.089
NIAQUE-Global
22.921
0.122
0.249
0.00
91.5
0.088
Table 97: Performance comparison Vehicle dataset. Lower values better for all metrics except CI [CCCerreee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globar
69.582
0.378
0.584
0.13
93.7
0.310
CatBoost-Local
15.964
0.110
0.299
-0.02
94.2
0.091
LTTrannforrer--ocaal
21.671
0.166
0.355
-0.01
98.2
0.107
FTTTaanfforme--Glon
21.950
0.125
0.282
0.01
96.3
0.079
NIAQUE-Local
23.717
0.137
0.308
0.00
97.9
0.096
NIAQUE-Global
23.686
0.134
0.295
-0.00
94.2
0.095
49

---

Published in Transactions on Machine Learning Research (02/2026)
Table 98: Performance comparison on VideoGameSales dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
98.925
0.133
0.627
-0.10
80.2
0.122
CatBoost-Local
89.060
0.121
0.615
-0.09
93.1
0.105
FTTTransfomer--oca
96.390
0.173
0.680
-0.10
95.2
0.108
OTTrraanformer--llbbal
89.584
0.121
0.581
-0.09
94.3
0.086
NIAQUE-Local
93.144
0.128
0.625
-0.09
98.5
0.097
NIAQUE-Globa
91.507
0.125
0.618
-0.09
95.3
0.096
Table 99: Performance comparison on VisualizingSoil dataset. Lower values better for all metrics except CI [CCveraeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
38.068
0.684
1.060
0.24
93.2
0.620
CatBoost-Local
14.906
0.058
0.105
0.00
95.5
0.112
FTTTransformer-Lcall
15.108
0.075
0.113
-0.01
97.3
0.068
FTTTransformer-Guue
11.674
0.024
0.048
0.01
98.6
0.030
NIAQUE-Local
10.672
0.027
0.052
0.00
99.0
0.021
NIAQUE-Global
9.617
0.015
0.033
-0.00
97.9
0.013
Table 100: Performance comparison on WalmartSales dataset. Lower values better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
61.504
1.320
1.575
0.38
96.1
0.982
CatBoost-Local
14.493
0.235
0.448
-0.06
94.3
0.189
PTTransformer--oocal
16.095
0.276
0.562
-0.09
92.9
0.189
F' V''Trrasformmr-Gur
13.604
0,240
0.501
-0.06
94.8
0.160
NIAQUE-Local
13.611
0.255
0.555
-0.05
90.8
0.173
NIAQUE-Global 12.916
0.249
0.549
-0.07
92.7
0.161
Table 101: Performance comparison WhiteWine dataset. Lower values are better for all metrics except CI ['CCerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Globa
22.573
1.064
1.494
-0.70
95.5
0.892
CatBoost-Local
19.482
0.898
1.213
-0.16
94.3
0.679
FTTraanforrmr--occal
20.628
0.925
1.340
-0.20
98.0
0.655
FTTraanfformr--Glal
19.436
0.879
1.206
-0.19
94.7
0.619
NIAQUE-Local
19.824
0.877
1.243
-0.17
92.7
0.634
NIAQUE-Global
19.445
0.890
1.245
-0.18
92.9
0.615
50

---

Published in Transactions on Machine Learning Research (02/2026)
Table 102: Performance comparison on Wind dataset. Lower values are better for all metrics except
CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV
CRPS
CatBoost-Global
29.371
0.964
1.259
-0.11
98.8
0.801
CatBoost-Local
18.869
0.590
0.754
0.05
93.6
0.424
FTTTransfomer--Lcct
21.169
0.617
0.863
0.08
91.9
0.425
ITTTraussormer--lllaal
18.744
0.589
0.708
0.04
92.6
0.397
NIAQUE-Local
18,820
0.611
0.759
0.04
94.3
0.412
NIAQUE-Globa
19.038
0.587
0.755
0.03
93.0
0.412
Table 103: Performance comparison Wizmir dataset. Lower values better for all metries except CI ['Cverreeee (target: 95%). For BIAS, lower absolute values better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
27.798
1.467
1.982
-1.09
91.2
1.101
CatBoost-Local
3.709
0.150
0.207
0.01
89.1
0.124
FTTTransformer-Lcall
5.719
0.189
0.239
-0.02
93.8
0.116
FT'Transformer-Guu
3.115
0.145
0.176
-0.00
99.0
0.095
NIAQUE-Local
3.446
0.142
0.216
-0.01
96.1
0.102
NIAQUE-Global
2.983
0.135
0.195
-0.01
96.6
0.100
Table 104: Performance comparison on Yprop4l dataset. Lower values are better for all metrics except CI [CCoeereee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Global
2.218
0.206
0.317
-0.12
91.6
0.161
CatBoost-Local
2.125
0.197
0.295
-0.07
92.8
0.154
PTTransformer--occal
4.460
0.251
0.338
-0.10
90.3
0.150
F' "I'Traasformer-Guu
2,063
0.190
0,286
-0.09
92.9
0.133
NIAQUE-Loca
2.584
0.209
0.312
-0.09
93.4
0.146
NIAQUE-Global
2.136
0.198
0.299
-0.09
92.7
0.144
Table 105: Performance comparison ZurichDelays dataset. Lower values are better for all metrics except CI [_CCerreee (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CatBoost-Glooa CatBoost-Local
2.343
0.132
0.254
-0.08
96.2
0.108
2.304
0.129
0.240
-0.05
94.6
0.099
FTTraanforrmr--Lccal
4.493
0.173
0.276
-0.06
93.9
0.109
FTTrranfformer-Gbbl
2.342
0.131
0.234
-0.05
94.9
0.080
NIAQUE-Local
2.795
0.141
0.262
-0.06
97.3
0.093
NIAQUE-Global
2.327
0.130
0.242
-0.06
95.7
0.093
51

---

Published in Transactions on Machine Learning Research (02/2026)
Table 106: Performance comparison on house 16H dataset. Lower values are better for all metrics except CI Coverage (target: 95%). For BIAS, lower absolute values are better.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
CalBoost-Global
32.984
0.407
0.930
-0.19
87.5
0.313
CatBoost-Local
27.666
0.316
0.682
-0.12
93.2
0.246
FTTTransformer-Locall
28.408
0.331
0.704
-0.07
93.7
0.230
ITTTraasformer--Gooe
25.411
0.293
0.655
-0.05
98.3
0.203
NIAQUE-Local
24.935
0.300
0.620
-0.05
96.3
0.209
NIAQUE-Global 24.701
0.283
0.612
-0.06
95.4
0.203
Table 107: Performance comparison on Wind dataset across point prediction accuracy (SMAPE, AAD, RMSE, BLAS) and distributional accuracy (COV@95, CRPS) metries. Lower values are better for all metrics except COV@95, where values closer to 95 are optimal. The results with 95% confidence intervals derived from multiple random seed runs.
Model
SMAPE
AAD
RMSE
BIAS
COV@9A
CRPS
XGBoost-Local
19.4
0.607 : 0.001
0.779 + 0.004
0.044 0.01
64.9 0.2
0.473 0.001
XGBoost-Domain
19.3 + 0.2
0.603 + 0.002
0.779 = 0.005
0.016
0.01
96.0
0.462 = 0.002
XGBoosttGGoobll
20.0 + 4.4
0.627 + 1.000
0.811 + 0.133
0.010 0.05
97.0+
0.527 + 0.165
LightGBM-Loocal
18.8 + 0.2
0.589 + 0.01
0.753 + 0.007
0.04
91.5+
0.429 + 0.003
LightGBM-Dommin
18.9 0.1
0.592 + 0.004
0.766 + 0.006
0.00
9.6
0.446 + 0.002
LightGBM-Global
0.1
0.637
0.002
0.854 + 0.001
0.00
97.8
0.599 0.032
Lat
18.9
0.590 0.001
0.754 + 0.003
0.046 + 0.01
93.6
0.424 = 0.001
CatBoost-Domall
19.8 0.2
0.617 = 0.002
0.796 = 0.004
0.038 0.01
94.8
0.454 = 0.002
CatBoost-Global
26.3 0.2
0.832 = 0.006
1.068 = 0.009
0.027 0.02
0.2
0.722 = 0.004
Transformer-Local
18.5 0.3
0.575 0.008
0.740 0.015
0.025 + 0.01
94.2
0.410 0.005
Transformer-Dmmall
18.5
0.573 0.008
0.736 0.015 0.015
0.022
0.3
0.405 = 0.005
Transformer-Global
18.4 0.3
0.733 + 0.015
0.02
94.5
0.401 + 0.005
NIAQUE-Local
0.4
0.582 0.01
+747
0.045
9..7
0.407 + 0.011
NIAQUE-Done Domain
18.7
0.585 0.006
0.760 + 0.010
0.027
0.3
0.415 = 0.006
NIAQUE-Global
19.0
0.587 0.002
0.755 + 0.005
0.031 0.01
9.2
0.412 = 0.002
D.2 Domain Adaptation
(4.7K samples, 11 features) as our representative dataset. This dataset offers balanced evaluation ground with its moderate sample size and limited feature dimensionality. Next, we focus on Energy and Efficiency Domain. We analyze the Wind dataset (OpenML n.d.) (6.5K samples, 12 features) as our representative case study, Similar to HouseRent, this dataset was chosen for moderate sample size and limited feature dimension- ality, providing a balanced evaluation ground. Tables [07] [08] show the performance comparison between NIAQUE and baselines across different training scenarios. The results reinforce main findings: NIAQUE maintains consistent performance across local, domain, and global training settings (RMSE: 0.747-0.760), while traditional models like CatBoost show significant degradation in global settings (RMSE incrcases from 0.754 to 1.068). Notably, NIAQUE's performance in the Energy and Efficiency domain exhibits similar patterns to those observed in the House Price Prediction domain. The model maintains reliable uncertainty quantification (coverage near 95%) and demonstrates effective knowledge transfer in domain-specific train- ing, achieving comparable or better performance than local training, showing the ability of the model effectively leverage additional information from domain-specific datasets to improve target task.
For domain adaptation, first focus on House Price Prediclion, selecting HouseRent (Banerjee] 2022
52

---

Published in Transactions on Machine Learning Research (02/2026)
Table 108: Performance comparison on HouseRent dataset across point prediction accuracy (SMAPE, AAD, RMSE, BIAS) and distributional accuracy (COV@95, CRPS) metrics. Lower values are better for all metrics except COV@95, where values closer 95 are optimal. The results with 95% confidence intervals derived from multiple random seed runs.
Model
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
XGBoost-Local
34.1
0.028
0.066
-0.002
86.5
0.022
XGBoosttDomain
32.9
0.031
0.073
0.004
88.8
0.026
XGBoost-Global
35.4
0.033
0.075
0.011
62.5
0.033
LightCGBB-LLoal
31.4
0.028
0.068
0001
92.5
0.021
LightGBM-Domain
33.6
0.029
0.07
0.01
91.8
0.024
LightGBM-Glloall
37.1
0.036
0.085
-0.11
94.4
0.028
Boost-Looal
31.5
0.028
0.068
0.007
92.8
0.022
CatBoost-Domain
32.4
0.028
0.072
0.011
94.1
0.022
Cat Globst
50.3
0.044
0.104
0.015
88.4
0.032
Transformer-Local
33.5
0.030
0.070
0.008
93.6
0.025
Transformer-Domainn
32.8
0.029
0.069
0.007
94.0
0.023
Transformer-Globll
32.3
0.028
0.068
0.007
93.5
0.020
NTAQUF-Local 32.0
0.028
0.067
0.007
96.0
0.019
NIAQUE-Doman
30.4
0.026
0.063
-0.002
93.5
0.018
NIAQUE-GIobal 30.3
0.026
0.067
-0.005
93.3
0.019
Table 109: Performance comparison on Kaggle competition dataset. Lower values are better. Bascline adopted from publicly shared notebooks and discussion forums in the competition RRade &
results
Chow
2024a). Rank represents the position of the various methods private leaderboard.
Model
Feature Engineering
RMLSE
Rank
XGBoost Broccoli Beef (siukeitin)
2024
None
0.15019
1615
LightGBM data Wr3cker
None
0.14914
1464
Cotso Wate
2024
None
0.14783
1064
TabNet Broccoli Beef (siukeitin)
2024
None
0.15481
2047
TabDPT
None
0.15026
1623
TabDPT
OpenFE
0.14751
924
TabPFN
None
0.15732
2132
TabPFN
OpenFE
0.14922
1478
NIAQUE-Scratch
None
0.15047
1646
NIAQUE-PPretrain-100
None
0.14823
1232
HLAQUE-PPeetrin--f
None
0.14808
1178
OpenFE
0.14556
304
MIAQUE-Ensenll
OpenFE
0.14423
8
Winner Heller 2024
Manual
0.14374
1
D.3
Adaptation to New Tasks: Kaggle Competitions
D.3.1 Abalone Competition
To validate NIAQUE's practical effectiveness, we evaluate its performance in recent Kaggle competitions. Abalone (Reade Chow 2024a),, focuses on the Abalone age prediction task. This competition, with 2,700 participants and over 20,000 submissions, provides an excellent real-world benchmark, particularly as neural networks widely reported to underperform in it, compared to traditional boosted tree methods.
53

---

Published in Transactions on Machine Learning Research (02/2026)
Our approach involves two stages: pretraining and fine-tuning. First, we pretrain NIAQUE on Tab (which includes the original UCT Abalone dataset (Dua & Graff 2019) using our quantile loss framework. Then, we fine-tune the pretrained model on the competition's training data, optimizing for RMSLE metric as defined in Section X The dataset size is 90, 614 training samples and 60, 410 test
Methodology:
samples (Reade & Chow 2024a). To systematically evaluate the impact of transfer learning, we implement three variants: NTAQUE-Seratch (trained only on competition data, no pretraining), NTAQUE-Pretrainn (pretrained on 10b excluding Abalone dataset), and NLAQUE-Pretrain--tul (pretrained on full TabRegSet-101). Note that the competition dataset is synthetic, generated using deep learning model trained on the original Abalone dataset, and the competition explicitly encourages the use of the original dataset. Finally, we measure the effect of automated feature engineering on our model using OpenF' (Zhang et al. 2023), an automated feature engineering framework, and explore ensemble stralegies leveraging our model's 's probabilistic nature (NIAQUE-Ensemble))
The private leaderboard competition results along with their ranks are presented in Table 109 highlighting NTAQUE's capabilities against wide range of well-established representative approaches. The effectiveness of transfer learning is evident in the performance progression: NIAQUE- Scratch (RMSLE: 0.15047), trained only on competition data, performs similarly to traditional basclines like XGBoost (0.15019). NIAQUE-Pretraine
Results and Analysis:
(0.14823), pretrained on TabRegSet-101 excluding the Abalone dataset, shows significant. improvement, demonstrating that knowledge from unrelated regression tasks can enhance performance. NIAQUE-Pretrain- (0.14808), leveraging the complete TabRegSet-100, further improves performance by incorporating original Abalone information in the training mix and matching Cat- Boost (4.13) without any feature engineering. The addition of automated feature engineering (OpenFF) into the competition dataset further improves NIAQUE's performance to 0.14556, significantly outperform- ing all baseline models and approaching the competition's winning score. Our best result comes from HAQUEEEEnsebbe (RMSLE: 0.14423), which leverages the probabilistic nature of our model by combining predictions from different quantiles and model variants (Scratch and Pretrain-full). The ensemble benefits from quantile predictions (in addition to the median) that help correct for data imbalance, predicting higher values where models might underestimate and vice versa, achieving the 8th position on the private leader- board, remarkably close to the winning score of 0.14374. This result is attained without extensive manual intervention wschhewnng hand-cngincered features in favor of transfer learning, standard ensemble techniques and aulomated feature extraction. Moreover, the model's probabilistic formulation is leveraged to further enhance point prediction accuracy, consistent with Bayesian estimation theory, where optimal estimators are typically derived as functions of the posterior distribution (e.g., the variance-optimal estimator is the posterior mean).
These results are particularly significant given that neural networks were generally considered ineffective for this task. Multiple participants reported neural approaches failing achieve scores better than 0.15, with the competition's 's winner noting: "I tried different neural network architectures only to observe that none of them is competitive" " (Heller 2024), echoed by other participants: "yes, I also tried different neural network architectures only to observe that they could not reach beyond .15xz" (Heller 2024). This real-world valida- tion highlights NIAQUE's compeliliveness against heavily engineered solutions and underscores the efficacy of integrating transfer learning with probabilistic modeling. Whereas the winning solution relied on an en- semble of 49 models with extensive manual tuning, our approach achieves comparable performance through principled transfer learning and uncertainty quantification, maintaining interprelability and necessitating minimal task-specific modifications.
D.3.2 FloodPrediction Competition
The flood prediction competition (Reade & Chow 2024bb presents challenging regression task aimed at predicting flood event probabilities based on environmental features including MonsoonIntensity, Topogra- phyirain, RiverManagementt, and Deforestation. With 2,932 participants, this competition addresses a critical real-world problem where labeled data consists of flood probabilities rather than binary outcomes, making regression models more suitable than classification approaches.
Methodology: We adapted NIAQUE for this flood probability prediction task while maintaining its core probabilistic framework. The competition dataset comprises 1.12M training samples and 745K test samples,
54

---

Published in Transactions on Machine Learning Research (02/2026)
Table 110:
Performance comparison on Flood Prediction competition. Higher R2-scores are better. Baseline
results are from competition forums RRede & Chow 2024b Rank represents the position on private leaderboard.
Model
R2 Score
Rank
XGBoost (Broccoli Beef (siukeitin)
2024
Sayed 2024
0.842
2304
LightGBM q data Wr3cker
Masoudi
2024
0.766
2557
Cal Boost (Waee 2024 Milind 2024
0.845
1700
TabNet [Broccoli Beef (siukeitin)
2024
0.842
2304
TabDPT
0.804
2529
TabPFN ()
0.431
NIAQUE--Scratch
0.865
1099
NLAQUE-PPeerrinn
0.867
935
Winner Heller 2024 Aldparis 2024
0.869
1
each containing relevant features derived from Flood Prediction Factors Dhankour 2024). Similar to our Abalone experiment, we evaluate transfer learning effectiveness through two variants: VLAQUE-Scratcl (trained directly on competition data) and NIAQUE-Pretrain (pretrained on TabRegSet-101). The model is pretrained using quantile loss framework and then fine-tuned with MSE Loss on the competition dataset. Notably, we found that automated feature engineering not only failed to improve performance but also significantly increased the feature space dimensionality, making several baseline methods (TabDPT and TabPFN) computationally intractable. Therefore, we focus on comparing the fundamental algorithmic capabilities without feature engineering enhancements.
The competition results (Table 110) demonstrate NIAQUE's strong performance in flood prediction. NLAQUE-Seratch achieves an R2-score of 0.865, significantly outperforming both traditional methods like XGBoost (0.842), CatBoost (0.845), and LightGBM (0.766), well newer deep learning approaches such as TabNet (0.842), TabDPT (0.804), and TabPFN (0.431). NTAQUE-Pretrail further improves performance to 0.867, approaching the winning score of 0.869, and securing rank 935 on the private leaderboard.
Results and Analysis:
These results are particularly noteworthy given the scale and complexity of the dataset. Unlike the Abalone competition, where feature engineering played crucial role, this competition highlights NIAQUE's ability to learn effective representations directly from frat The improvement from pretraining, though modest in absolute terms (0.002 increase in R2-score), represents a significant advancement in ranking (from 1099 to 935), demonstrating the value of transfer learning even in specialized environmental prediction tasks. The strong performance of our base model without extensive modifications or feature engineering suggests that NIAQUE's probabilistic framework and architecture are well-suited for large-scale regression tasks where the target variable represents underlying probabilities.
The relatively poor performance of some specialized tabular models (particularly TabPFN with R2-score of 0.431) this large-scale dataset underscores the importance of scalability in real-world applications. NIAQUE maintains its computational efficiency while handling over million samples, making it practical for deployment in real-world flood prediction systems where both accuracy and computational resources are critical considerations.
D.4 Statistical Significance Analysis
To save space, we present benchmarking results with confidence intervals here. All confidence intervals are obtained by aggregating the evaluation results over 4 runs with different random seeds.
55

---

Published in Transactions on Machine Learning Research (02/2026)
Table 111: Performance comparison across metries, with point prediction accuracy (SMAPE, AAD, RMSE, BIAS) and distributional accuracy (COV@ @95, CRPS). Lower values are better for all metrics except COV@95, where values closer to 95 optimal. The results with 95% confidence intervals derived from 4 random seed runs.
SMAPE
AAD
RMSE
BIAS
COV@95
CRPS
XGBoost- -GBoost-global
31.4 4.4
0.574 = 0.100
1.056 = 0.143
-0.15 + 0.05
94.6 0.3
0.636 + 0,165
XGBoost-local
25.6
0.433 = 0.001
0.883 + 0.004
-0.03 + 0.01
90.8 0.2
0.334 + 0.001
LightGGBM-global
0.1
0.475 = 0.001
0.930 + 0.003
-0.06 0.01
94.8 0.1
0.426 + 0.017
LightGBM-local
2.1
0.427 = 0.003
0.865 + 0.012
-..03 + 0.01
91.5
0.327 + 0.001
CATBOOST-global
31.3 0.2
0.561 + 0.006
1.030 + 0.009
00.1 0.02
94.9 0.2
0.443 + 0.004
CATBOOST-loca
24.3 0.1
0.408 = 0.001
0.840 + 0.003
-0.03 = 0.01
92.7 0.1
0.315 + 0.001
Transformer-global
23.1 + 0.3
0.383 + 0.008
0.806 + 0.015
-0.01 0.01
94.6 + 0.3
0.272 + 0.005
NIAQUE-local
22.8 0.4
0.377 + 0.012
0.797 + 0.019
-0.03
94.9
0.267 + 0.011
NIAQUE-global
22.1 + 0.1
0.367 + 0.002
0.787 + 0.005
0.02 0.01
0.2
0.261 + 0.002
56

---

Published in Transactions on Machine Learning Research (02/2026)
Table 112: Ablation study of the XGBoost model.
type
max depth
learning rate
SMAPE
AAD
BIAS
RMSE
CRPS
COVERAGE
95
global
8
0.02
31.4
0.574
-0.15
1.056
0.636
94.6
global
16
0.02
25.7
0.441
-0.07
0.864
0.484
91.5
global
32
0.02
24.1
0.402
-0.05
0.800
0.353
80.0
global
40
0.02
24.6
0.414
-0.05
0.815
0.378
78.2
global
48
0.02
24.1
0.397
-0.04
0.785
0.362
74.8
global
96
0.02
23.8
0.384
-0.03
0.769
0.346
64.9
local
16
0.02
23.0
0.367
-0.00
0.753
0.317
52.0
local
12
0.02
22.7
0.369
-0.01
0.756
0.304
66.0
local
8
0.02
22.4
0.372
-0.02
0.773
0.294
82.3
local
0.05
22.5
0.373
-0.02
0.773
0.291
82.4
local
6
0.02
22.7
0.382
-0.02
0.795
0.298
87.3
local
0.02
24.1
0.412
-0.03
0.847
0.318
90.2
local
3
0.02
25.6
0.433
-0.03
0.883
0.334
90.8
E XGBoost Baseline
57

---

Published in Transactions on Machine Learning Research (02/2026)
Table 113: Ablation study of the CATBoost model.
min data in leaf
COVERAGE
type depth
SMAPE
AAD
BIAS
RMSE
CRPS
95
global
16
50
31.4
0.565
-0.12
1.036
0.442
94.2
global
16
100
31.3
0.561
-0.12
1.030
0.443
94.9
global
16
200
31.6
0.569
-0.13
1.041
0.445
94.2
global
8
100
41.1
0.785
-0.26
1.324
0.602
94.3
local
3
50
24.3
0.409
-0.03
0.841
0.316
92.7
local
3
100
24.3
0.407
-0.03
0.843
0.317
92.7
local
3
200
24.3
0.408
-0.03
0.840
0.315
92.7
local
5
50
22.2
0.373
-0.02
0.785
0.285
90.7
local
5
100
22.3
0.374
-0.02
0.786
0.285
91.3
local
5
200
22.4
0.378
-0.02
0.791
0.288
91.6
local
7
50
21.5
0.359
-0.02
0.761
0.272
87.2
local
7
100
21.6
0.362
-0.02
0.765
0.273
88.6
local
7
200
21.8
0.366
-0.02
0.772
0.277
89.9
Table 114: CATBoost accuracy function of the number of quantiles.
lype
depth
min data in leaf
num quantiles
SMAPE
AAD
BIAS
RMSE
CRPS
COVERAGE
@ 95
global
16
100
3
31.3
0.561
-0.12
1.030
0.443
94.9
global
16
100
5
35.0
0.665
-0.13
1.183
0.482
96.2
global
16
100
38.5
0.746
-0.18
1.265
0.533
96.2
global
16
100
9
43.7
0.879
-0.25
1.437
0.622
96.2
global
16
100
51
68.9
1.538
-0.53
2.132
1.036
95.5
local
7
100
3
21.5
0.359
-0.02
0.761
0.272
87.2
local
100
23.9
0.399
-0.03
0.823
0.284
92.4
local
100
51
30.3
0.525
-0.09
1.079
0.369
92.1
local
16
100
51
30.2
0.514
-0.09
1.055
0.362
92.4
F
CoTBo Baseline
using Table 114 shows CATBoost accuracy a function of the number of quantiles. Quantiles are generated using linspace grid np www.snnnncceecc (0. 01, 0.99, num num__uantiles). We recover the best overall result for the case of 3 quantiles, and increasing the number of quantiles leads to quickly deteriorating metrics. It. appears that CATBoost is unfit to solve complex multi-quantile problems.
The CATBoost is trained using the standard package via pip install catboost grov_policy Depthwise. The explored hyper-parqameter grid appears Table 113
58

---

Published in Transactions on Machine Learning Research (02/2026)
Table 115: Ablation study of the LightGBM global model.
type max depth
mum leaves
learning rate
SMAPE
AAD
BIAS
RMSE
CRPS
COVERAGE
95
global
-1
10
0.05
35.6
0.661
-0.17
1.199
0.804
95.2
global
-1
20
0.05
30.9
0.554
-0.11
1.034
0.566
95.4
global
-1
40
0.05
27.5
0.475
-0.06
0.930
0.426
94.8
global
-1
100
0.05
24.6
0.417
-0.03
0.852
0.342
93.3
global
-1
200
0.05
23.4
0.393
-0.02
0.813
0.32
92.3
global
-1
400
0.05
23.6
0.379
-0.02
0.786
0.305
90.9
global
3
10
0.05
50.7
1.084
-0.49
1.763
1.013
94.1
global
3
20
0.05
50.7
1.084
-0.49
1.763
1.013
94.1
global
3
40
0.05
50.7
1.084
-0.49
1.763
1.013
94.1
global
3
100
0.05
50.7
1.084
-0.49
1.763
1.013
94.1
global
3
200
0.05
50.7
1.084
-0.49
1.763
1.013
94.1
global
3
400
0.05
50.7
1.084
-0.49
1.763
1.013
94.1
global
5
10
0.05
39.1
0.768
-0.25
1.341
0.856
94.8
global
5
20
0.05
39.0
0.76
-0.26
1.327
0.863
94.8
global
5
40
0.05
39.0
0.759
-0.26
1.328
0.864
94.8
global
5
100
0.05
39.0
0.759
-0.26
1.328
0.864
94.8
global
5
200
0.05
39.0
0.759
-0.26
1.328
0.864
94.8
global
5
400
0.05
39.0
0.759
-0.26
1.328
0.864
94.8
global
10
10
0.05
35.6
0.661
-0.17
1.199
0.804
95.2
global
10
20
0.05
31.5
0.572
-0.14
1.054
0.59
95.4
global
10
40
0.05
29.8
0.537
-0.13
1.001
0.575
95.2
global
10
100
0.05
29.5
0.528
-0.12
0.991
0.577
95.2
global
10
200
0.05
29.2
0.522
-0.12
0.981
0.576
95.0
global
10
400
0.05
29.1
0.52
-0.12
0.975
0.582
95.1
global
20
10
0.05
35.6
0.661
-0.17
1.199
0.804
95.2
global
20
20
0.05
30.9
0.554
-0.11
1.034
0.566
95.4
global
20
40
0.05
27.1
0.468
-0.07
0.913
0.512
95.2
global
20
100
0.05
25.5
0.435
-0.06
0.864
0.496
94.9
global
20
200
0.05
25.0
0.424
-0.06
0.846
0.488
94.3
global
20
400
0.05
24.3
0.41
-0.05
0.823
0.482
93.6
global
40
10
0.05
35.6
0.661
-0.17
1.199
0.804
95.2
global
40
20
0.05
30.9
0.554
-0.11
1.034
0.566
95.4
global
40
40
0.05
27.8
0.481
-0.05
0.913
0.431
94.7
global
40
100
0.05
24.7
0.419
-0.04
0.848
0.348
93.5
global
40
200
0.05
23.5
0.395
-0.03
0.811
0.332
92.7
global
40
400
0.05
23.2
0.383
-0.03
0.791
0.322
92.0
G
LightGBM Baseline
59

---

Published in Transactions on Machine Learning Research (02/2026)
Table 116: Ablation study of the LightGBM local model.
learning rate
COVERAGE
typc
depth
SMAPE
AAD
BIAS
RMSE
CRPS
leaves
@ 95
local
-1
5
0.05
23.8
0.399
-0.03
0.823
0.319
90.6
local
-1
10
0.05
22.5
0.376
-0.02
0.786
0.301
88.9
local
-1
20
0.05
21.9
0.364
-0.02
0.766
0.289
86.5
local
-1
50
0.05
21.6
0.355
-0.01
0.752
0.278
82.6
local
2
5
0.05
25.7
0.427
-0.03
0.865
0.327
91.5
local
2
10
0.05
25.7
0.427
-0.03
0.865
0.327
91.5
local
2
20
0.05
25.7
0.427
-0.03
0.865
0.327
91.5
local
2
50
0.05
25.7
0.427
-0.03
0.865
0.327
91.5
local
3
5
0.05
24.3
0.404
-0.03
0.83
0.318
90.7
local
3
10
0.05
23.9
0.396
-0.03
0.818
0.304
90.4
local
3
20
0.05
23.9
0.396
-0.03
0.818
0.304
90.4
local
3
50
0.05
23.9
0.396
-0.03
0.818
0.304
90.4
local
5
5
0.05
23.8
0.399
-0.03
0.823
0.319
90.6
local
5
10
0.05
22.7
0.379
-0.02
0.79
0.3
89.1
local
5
20
0.05
22.3
0.37
-0.02
0.776
0.287
87.6
local
5
50
0.05
22.2
0.368
-0.02
0.773
0.285
87.4
60

---

Published in Transactions on Machine Learning Research (02/2026)
Quaintli Decoder
Block 2
Quantile Loss
Cuunrille
Nx
ReLu
Liner ReLu
Norm
Linear
Linear
FFN
MLP
Linear
ReL.u
inear
Norm
MHA
*V
KK
4Q
ReLu
NNom
Re
ReL
LNorm
Norm
MHA
MHA
Embediings
Concet
Linear
Prototype
Festure
Code
Figure Transformer baseline used in our experiments. The feature encoding module is replaced with transformer block. Feature encoding is implemented via self-attention. The extraction of feature encoding is done by applying cross-attention between the prototype of input features and the output self-attention. This operation repeated several times corresponding to the number of blocks in transformer encoder.
H Transformer Baseline
The ablation study of the transformer architecture is presented in Table 117 It shows that in general, increasing the number of transformer blocks improves accuracy, however, at 8-10 blocks we clearly see diminishing returns. Dropout helps to gain better empirical coverage of the 95% confidence interval, but this happens at the expense of point prediction accuracy. Finally, the decoder query that is used to produce the feature embedding that is fed the quantile decoder can be implemented in two principled ways. First, the scheme depicted in Figure 5 uses the prototype of features supplied to the encoder. We call it the prototype scheme. Second, the prototype can be replaced by learnable embedding. Comparing the last and third rows in Table 117 we conclude that the prototype scheme is a clear winner.
61

---

Published in Transactions on Machine Learning Research (02/2026)
Table 117: Ablation study of the Transformer architecture.
COVERAGE
query
d model
width
blocks
dp
SMAPE
AAD
BIAS
RMSE
CRPS
@ 95
proto
256
256
4
0.1
25.6
0.462
-0.01
0.918
0.313
95.2
proto
256
1024
4
0.1
24.5
0.414
-0.02
0.845
0.292
95.1
proto
256
256
6
0.1
23.7
0.397
-0.01
0.824
0.281
94.9
proto
256
512
6
0.2
proto
256
1024
6
0.1
24.3
0.407
-0.01
0.840
0.287
94.9
proto
256
1024
6
0.0
26.5
0.477
-0.04
0.980
0.334
93.0
proto
256
512
8
0.0
23.3
0.388
-0.03
0.814
0.276
94.3
proto
256
1024
8
0.0
23.1
0.383
-0.02
0.806
0.272
94.6
proto
256
1024
8
0.1
23.1
0.384
-0.01
0.809
0.272
94.6
proto
256
512
10
0.0
23.0
0.384
-0.03
0.814
0.273
94.2
proto
256
1024
10
0.1
24.3
0.407
-0.01
0.840
0.287
94.9
proto
512
1024
6
0.1
learn
256
256
6
0.2
35.0
0.722
-0.16
1.406
0.489
93.9
62

---

Published in Transactions on Machine Learning Research (02/2026)
Table 118: Ablation study NIAQUE-local model.
COVERAGE
blocks
width
dp
layers
SMAPE
AAD
BIAS
RMSE
CRPS
95
64
0.0
3
24.2
0.414
-0.03
0.848
0.292
95.1
128
0.0
3
22.8
0.381
-0.02
0.804
0.270
94.5
2
256
0.0
3
22.1
0.365
-0.02
0.786
0.260
94.0
512
0.0
3
21.9
0.360
-0.02
0.781
0.257
92.7
64
0.1
3
24.7
0.431
-0.07
0.855
0.305
93.3
128
0.1
3
23.1
0.389
-0.04
0.81
0.276
94.0
256
0.1
3
22.2
0.369
-0.02
0.79
0.263
94.0
512
0.1
3
22.0
0.361
-0.02
0.779
0.257
93.5
64
0.0
2
24.5
0.419
-0.03
0.852
0.296
95.0
128
0.0
2
23.4
0.391
-0.02
0.815
0.276
94.7
256
0.0
2
22.3
0.368
-0.02
0.783
0.262
94.1
512
0.0
2
22.1
0.363
-0.03
0.780
0.259
92.9
64
0.0
2
23.8
0.399
-0.02
0.828
0.282
95.1
4
128
0.0
2
22.8
0.377
-0.03
0.797
0.267
94.9
256
0.0
2
22.0
0.363
-0.02
0.788
0.259
93.5
512
0.0
2
22.0
0.359
-0.02
0.785
0.257
92.0
64
0.1
2
23.8
0.401
-0.03
0.829
0.284
94.3
128
0.1
2
22.9
0.379
-0.03
0.801
0.267
94.6
256
0.1
2
22.1
0.363
-0.03
0.786
0.259
93.5
512
0.1
2
22.0
0.360
-0.03
0.781
0.257
92.4
128
0.0
2
23.0
0.381
-0.02
0.798
0.27
95.7
NIAQUE-Local Baseline
NTAQUE-local baseline is trained each dataset individually using the overall training framework discussed in the main manuscript for the NIAQUE-gIobal, with the following exceptions. The number of training epochs for each dataset is fixed at 1200, the batch size is set to 256, feature dropout is disabled. Finally, for each dataset we select the best model to be evaluated by monitoring the loss on validation set every epoch.
63

---

Published in Transactions on Machine Learning Research (02/2026)
Table 119: Ablation study of NIAQUE model.
log input
COVERAGE
blocks
width
dp
layers
singles
SMAPE
AAD
BIAS
RMSE
CRPS
95
1024
0.2
5%
yes
25.6
0.433
-0.04
0.864
0.306
96.5
1024
0.2
5%
yes
23.1
0.384
-0.02
0.802
0.272
95.7
1024
0.2
5%
yes
22.7
0.377
-0.03
0.796
0.267
95.6
1024
0.2
5%
yos
22.1
0.367
-0.02
0.787
0.261
94.6
4
1024
0.2
5%
yes
22.1
0.367
-0.02
0.792
0.262
94.6
1024
0.2
5%
yes
22.0
0.366
-0.02
0.798
0.264
92.7
512
0.2
0%
yes
22.5
0.372
-0.02
0.791
0.264
95.4
1024
0.2
0%
yes
22.1
0.366
-0.02
0.791
0.261
94.2
4
1024
0.3
0%
yes
22.1
0.367
-0.02
0.787
0.260
94.7
1024
0.4
0%
yes
22.2
0.370
-0.02
0.791
0.263
95.1
2048
0.3
0%
yes
22.1
0.366
-0.02
0.795
0.263
93.4
1024
0.2
5%
no
31.4
0.530
-0.066
1.017
0.371
95.6
J NIAQUE Training Details and Ablation Studies
To train both NIAQUE and Transformer models we feature dropout defined as follows. Given dropout probability dp, toss coin with probability /dp determine if the dropout event is going to happen all for given batch. If this happens, we remove each feature from the batch, again with probability ddp. This way each feature has probability dp of being removed from given batch and there is probability ddp that the model will see all features intact in given batch. The intuition behind this design is that we want to expose the model to all features most of the time, but we also want to create many situations with some feature combinations missing.
Input log transformation defined in eq. 10 is important to ensure the success of the training, as follows both from Table 119 and Figure 6 The introduction of log-transform makes learning curves well-behaved and smooth and translates into much better accuracy.
Adding samples containing only one of the features as input does not significantly affect accuracy. At the same time, the addition single-feature training rows has very strong effect on the effectiveness of NTAQUE's interpretability mechanism. When rows with single feature input are added (Figures 7a and 7b NIAQUE demonstrates very clear accuracy degradation when top features are removed and insignificant degradation when bottom features are removed. When rows with single feature input not added (Fig- ure 7c the discrimination between strong and weak features is poor, with removal of top and bottom features having approximately the same effect across datascts.
64

---

Published in Transactions on Machine Learning Research (02/2026)
train/erps tog train/crps
train/loss_ppochh
train/loss__ston
tag: www./oosssseooo epoch
tog: traiiv csopsstppppppppppppppppp
6e111
0.14
0.09
0.12
46111
0.07
0.38
2c+11
0.36
0.37
50k
103x 150k 200k
250k 300k
50k
130k
150k
203k
250x
300k
50k
100k
150k
200k
250k
300k
train/mee
train/mse
tag traln//mee
tag: train/mse
1.6e+12
22+12
Bo48
8e+11
40+E
4e+11
130k 200k 2506 B0CK
50k 100K
200k 250k
Figure 6: Training losses with (dark red) and without (blue) input value log-transform eq.
10 The
introduction of log-tracs sog-transform makes learning curves well-behaved and smooth.
Ourok RAAD Afes waturesaeeeeeeeeeeeo N0TeA03
Chence 1-440 when featres renovvdd
Change A60 when botuues rervvce
1.3
0.2
Festure
(a) Single features, 10%
(b) Single features, 5%
(c) Single features, 0%
Figure 7: The effect of adding training rows containing only one of the input features as NIAQUE input. When rows with single feature input are added (Figures 7a and 7b NTAQUE demonstrates very clear accuracy degradation when top features are removed and insignificant degradation when bottom features are removed. When rows with single feature input are not added (Figure (c), the discrimination between strong and weak features is poor, with removal of top and bottom features having approximately the same effect across datasets.
65


Reviews:
[{"review_id": "2ItIb8OosT", "reviewer": "Reviewer_qBhK", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper investigates transfer learning for probabilistic regression for tabular data. The authors propose NIAQUE -- an encoder-decoder neural network that enables pre-training followed by task-specific fine-tuning in probabilistic regression for tabular data.\n\nStrengths:\n* Introduction of Tabular Regression Set 101, a collection of 101 publicly available regression datasets, is a valuable contribution that may set a standard benchmark in the field.\n* The proposed approach, NIAQUE, outperforms other baselines. Notably, global learning (slightly) helps in cross-dataset learning (Tab1) and pre-training is effective, especially when downstream task data is scarce (Tab2)\n\nWeaknesses:\n* The proposed architecture is not properly ablated it is unclear to my why each feature block or quantile modulation block have the proposed structure, why is prototype layer needed, etc. (Appendix J only searches over the hyperparameters given the proposed architecture, eg number of blocks, width, dropout probability, etc.)", "claims_and_evidence_yn": "Yes", "claims_explanation": "The claims are properly supported by experimental evidence.", "audience_yn": "Yes", "audience_explanation": "NIAQUE seems like a state-of-the-art method for transfer learning that should be interesting to many people working on that. Also, the proposed benchmark may be useful for the community.Critical:\n* proper ablations of the proposed architecture are necessary (see weakness)\n\nNon-critical changes:\n* The authors notoriously use `\\citet{}` instead of `\\citep` making the cited papers' authors blend into the text.", "requested_changes": "Critical:\n* proper ablations of the proposed architecture are necessary (see weakness)\n\nNon-critical changes:\n* The authors notoriously use `\\citet{}` instead of `\\citep` making the cited papers' authors blend into the text.", "broader_impact_concerns": "Broader Impact Statement is not present in the paper and I agree that it is not needed for this paper.", "additional_comments": "I am not very familiar with probabilistic regression and tabular data literature. Therefore, my confidence is quite low.", "extra_scores": {}, "extra_text": {}}, {"review_id": "7vo1zMOGYB", "reviewer": "Reviewer_2p9A", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper proposes NIAQUE as a probabilistic tabular regressor. It conditions on a quantile q to learn the inverse conditional CDF and uses a training objective tied to CRPS. The work further suggests a prototype-aggregating encoder as an alternative to Transformer-based regressors.\nThe authors construct a 101-dataset corpus for evaluating tabular regression models with pretraining. NIAQUE is shown to outperform tree/transformer baselines and its quantile outputs enable uncertainty-aware feature-importance scoring.\n\n### Strengths\n* Probabilistic regression and transfer across heterogeneous tabular datasets is a relevant and underexplored research question.\n* The suggested any-quantile setup combined with and efficient encoder-decoder models is a novel and practical approach for probabilistic regression over tables.\n* The experimental evaluation covers 101 datasets. This collection of regression benchmark is an additional contribution and may benefit future work in this direction.\n\n### Weaknesses\n* Table 1 reports metrics “aggregated across datasets at sample level”. This may overweight larger datasets even after capping. Additionally reporting per-dataset results in the appendix would provide a more fine-grained picture.\n* The paper lacks details on how/whether categorical feature columns are used by NIAQUE. Are these omitted or somehow mapped to numerical features?\n\nOverall I believe the presented work is technically sound and interesting to the TMLR audience and should be accepted if the mentioned weaknesses are addressed.", "claims_and_evidence_yn": "Yes", "claims_explanation": "The experimental evaluation is convincing and adequate ablations are provided.", "audience_yn": "Yes", "audience_explanation": "Effective pretraining of tabular regression models is of interest to the TMLR audience and the paper contributes novel methods and evaluation datasets for this research direction.", "requested_changes": "I think the following changes are critical:\n1. Add more fine-grained per-dataset results underlying Table 1 in the appendix, or at least provide the macro average to balance the weights of each dataset.\n2. Clarify if/how categorical features are leveraged by NIAQUE", "broader_impact_concerns": "N/A", "additional_comments": "", "extra_scores": {}, "extra_text": {}}, {"review_id": "7jgHqetWot", "reviewer": "Reviewer_sz4U", "summary": "", "questions": "", "limitations": "", "rating": null, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "Yes", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "paper_topic_and_main_contributions": "", "excitement": null, "reproducibility": null, "ethical_concerns": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "summary_of_contributions": "The paper presents NIAQUE, a neural probabilistic regression model for tabular data. The proposed learning objective, architecture, and transfer learning via pre-training are sufficiently supported to be a desired modification through strong empirical results, especially when compared to the baselines. As outlined by the authors, the main novelty of the paper lies in bridging several ideas in different fields that were shown to be desirable in different fields.", "claims_and_evidence_yn": "Yes", "claims_explanation": "1. **Clear connection between motivation and solution**: The task that the paper wants to solve is clearly formulated in Section 3, and it is precisely addressed through Section 4 with a simple theoretical background in Theorem 1. Specifically, a transformer-based any-quantile learning approach is presented with a simple convergence analysis in Section 4.1.\n\n2. **Principled probabilistic framing**: The connection between the quantile regression objective and the continuous ranked probability score (CRPS) is correctly established, providing a coherent probabilistic interpretation of the proposed loss. The proof sketch leading to the identification of the true inverse conditional CDF as the minimizer is accurate and aligns with prior quantile regression theory.\n\n3. **Strong empirical results**: Most importantly, the authors claimed and demonstrated the effect of transfer learning in NIAQUE with an appropriate experimental design. The proposed confidence-interval-based feature importance offers a simple yet interpretable post-hoc analysis. While not as rigorous as SHAP or integrated gradients, the strong correlation between CI width ranking and SHAP-based importance across multiple datasets is convincing and adds practical value.", "audience_yn": "Yes", "audience_explanation": "Despite the extensive experiments across diverse datasets and the consistently strong empirical results compared to conventional baselines, the paper’s theoretical novelty is limited. The theoretical component primarily revisits established properties of the quantile loss and CRPS rather than introducing a new analytical insight. The main contribution is therefore empirical and integrative rather than theoretical. However, tabular learning with transfer and probabilistic pretraining is a practically relevant and desired area, and large-scale empirical evidence of this kind would likely attract interest from the readers, particularly those working on uncertainty-aware or data-efficient tabular modeling.", "requested_changes": "There are a few limitations of the paper that could improve the paper's readability.\n\n1. **Reporting the model size**: While the paper describes the overall model architecture of the models, including NIAQUE and the transformer baseline, reporting the overall model size by parameter counts would improve the clarity of the paper.\n\n2. **Table designs for readability**: The metrics reported in the paper have several different ways to interpret them, such as having lower values is better, lower absolute values are better, or higher the better in Table 1. Thus, implying the best numbers in each column could provide better readability.\n\n3. **More neural models/methods in Table 1**: The results in Table 1 are the main experiments on the effect of transfer learning and effectiveness on a large-scale dataset when trained from scratch. While there are four baselines listed, XGBoost, LightGBM, CatBoost, and Transformer, comparisons on several modern neural tabular models are not sufficiently addressed here. As in Table 3, the paper would benefit from comparing with more neural tabular models, such as FT-Transformer, as a baseline for tabular data.", "broader_impact_concerns": "No specific concerns regarding the ethical implications.", "additional_comments": "", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Decision: Accept (poster)