Bayesian Neural Networks for Functional ANOVA Model
Authors: Seokhun Park
Choeun Kim
Jihu Lee
Yunseop Shin
Insung Kong
Yongdai Kim
Venue: ICLR 2026 Poster (2026)
Keywords: Bayesian Machine Learning
Functional ANOVA Model
Bayesian Neural Networks
Interpretable AI
Primary area: interpretability and explainable AI
TL;DR: We propose an interpretable Bayesian neural network based on the functional ANOVA model.

Abstract: With the increasing demand for interpretability in machine learning, functional ANOVA decomposition has gained renewed attention as a principled tool for breaking down high-dimensional function into low-dimensional components that reveal the contributions of different variable groups.
Recently, Tensor Product Neural Network (TPNN) has been developed and applied as basis functions in the functional ANOVA model, referred to as ANOVA-TPNN.
A disadvantage of ANOVA-TPNN, however, is that the components to be estimated must be specified in advance, which makes it difficult to incorporate higher-order TPNNs into the functional ANOVA model due to computational and memory constraints.
In this work, we propose Bayesian-TPNN, a Bayesian inference procedure for the functional ANOVA model with TPNN basis functions, enabling the detection of higher-order components with reduced computational cost compared to ANOVA-TPNN.
We develop an efficient MCMC algorithm and demonstrate that Bayesian-TPNN performs well by analyzing multiple benchmark datasets.
Theoretically, we prove that the posterior of Bayesian-TPNN is consistent.

Published as a conference paper at ICLR 2026
BAYESIAN NEURAL NETWORKS FOR FUNCTIONAL ANOVA MODEL
Scokhun Park**, Park"t,Coeun Choeun Kim"", Jihu Lee', Yunseop Shin', Insung Kong", Yongdai Kim!t
Depparmeent of Statistics, Scoul National University *Department of Applied Mathematics, University of Twente
{shrdid, kimchoeun, rieky0426, insung. kongEutwente.nl, ydkim0903@gmaii
8 kc.kr,
ABSTRACT
With the increasing demand for interpretability in machine learning, functional ANOVA decomposition has gained renewed attention as a principled tool for breaking down nigh-dimensional function into low-dimensional components that reveal the contributions of different variable groups. Recently, Tensor Product Neural Network (TPNN) has been developed and applied as basis functions the functional ANOVA model, referred to ANOVA-TPNN. A disadvantage of ANOVA-TPNN, however, is that the components to be estimated must be speci- fied in advance, which makes it difficult to incorporate higher-order TPNNs into the functional ANOVA model due to computational and memory constraints. this work, propose Bayesian- TPNN, Bayesian inference procedure for the functional ANOVA model with TPNN basis functions, enabling the detection higher-order components with reduced computational cost compared ANOVA- TPNN. We develop an efficient MCMC algorithm and demonstrate that Bayesian- TPNN performs well by analyzing multiple benchmark datasets. Theoretically, we prove that the posterior of Bayesian- TPNN is consistent.
INTRODUCTION
As artificial intelligence (AI) models become increasingly complex, the demand for interpretability has grown accordingly. To address this need, various interpretable models-including both post- hoc explanations (Ribeiro al., (Agarwal et al. 20h et al. 2020 Radenovic et al. 2022; Park et al., 2025 have been stud- ied. Among various interpretable approaches, our study focuses on the functional ANOVA model, particularly important class of interpretable models that decompose high-dimensional function into a sum of low-dimensional functions called componenets or interactions. Notable examples of the functional ANOVA model are the generalized additive Model (Hastie & Tibshirani 1986). SS- ANOVA (Gu & Wahba, 1993) and MARS (Friedman) [991). Because complex structures of a given high-ddmennsinal model can be understood by interpreting low-dimensional components, the func- tional ANOVA models have been extensively used in interpretable Al applications (Lengerich et al.,
2016
Lundberg & Lee, 2017) and inherently transparent models
2020 Märtens & Yau 2020
Choi et al. 2025 Herren & Hahn 2022).
In recent years, various neural networks have been developed to estimate components in the func- tional ANOVA model. Neural Additive Models (NAM./Agawwal Agarwal et al. (202T) )) estimates each compo- nent of the functional ANOVA model using deep neural networks (DNN), and Neural Basis Models (NBM. Radenovic al.] (2022)) significantly reduce the computational burden of NAM by using basis deep neural networks (DNN). NODE-GAM (Chang et al./ 2021) can select and estimate the components in the functional ANOVA model simultancously, and Thielmann al. (2024) proposes
NAMLSS, which modifies NAM to estimate the predictive distribution, Park et al. (2025 proposes ANOVA-TPNN, which estimates the components under the uniqueness constraint and thus provides a stable estimate of each component.
Equaa contribution. Corresponding author.

---

Published as a conference paper at ICLR 2026
Existing neural-network approaches functional ANOVA model require prohibitive computation when the input dimension p is large, because the number of components and thus the required networks-grow grows exponentially. As result. only 1-2 dimensional components typically used, yielding suboptimal prediction when higher-order interactions matter.
In this paper, we propose Bayesian neural network (BNN) for the functional ANOVA model which can estimate higher-order interactions (i.e., components whose input dimension is greater than 2) without requiring huge amounts of computing resources. The main idea of the proposed BNN is to infer the architecture (the architectures of neural networks for each component) as well as the pa- rameters (the weights and biases each neural network). To explore higher posterior regions of the architecture, specially designed MCMC algorithu is developed which searches the architectures a stepwise manner (i.e., growing or pruning the current architecture) and thus huge computing resources for memorizing and processing all of the predefined neural networks for the components can be avoided.
Bayesian Neural Networks (BNN; MacKay (1992): Neal 2012);
Wilson & Izmailov 2020 Iz-
mailov et al. (202I)) provide a principled Bayesian framework for training DNNs and have received considerable attention in machine learning and AI. Compared to frequentist approaches, BNN of-
fers stronger generalization and better-calibratee uncertainty estimates (Wilson & Izmailov 2020
Izmailov et al. 202I), which enhance decision making. These properties have motivated applica- tions in areas such as recommender systems (Wang et al. 2013), topic modeling (Gan et al.) 2015).
and medical diagnosis (Filos et al.] 2019). More recently, Bayesian neural networks (BNN) that learn their own architectures have been actively studied. In particular, Kong et al. (2023) introduced node-sparse BNN, referred ties. Nguyen et al. (2024) proposes S-RJMCMC, which explores architectures and weights by jointly sampling parameters and altering the number of nodes.
the masked BNN (mBNN), and established its theoretical proper-
This is the first work on BNN that efficiently estimates higher-order components in the functional ANOVA model without requiring substantial computing resources. Our main contributions can be outlined as follows.
We propose a BNN for the functional ANOVA model called Bayesian-TPMy TPNN which treats the architecture learnable parameter, and develop MCMC algorithm which effi- ciently explores high-posterior regions of the architecture.
For theoretical justifications of the proposed BNN, we prove the posterior consistency of the prediction model as well as each component.
Through experiments on multiple real datasets, we show that the proposed BNN provides more accurate and stable estimation and uncertainty quantification than other neural net- works for the functional ANOVA model. On various synthetic datasets, we further show that Bayesian-TPNN effectively estimates important higher-order components.
PRELIMINARIES
2.1 NOTATION
Let x (21 2p) X be p-dimensional input vector, where X' = X1 X ... x X, 1|". We write [p] = (l.....pp and its power set with cardinality d as power([p), d). For any component S (p). denote xs (5z. E SST and define X5 IIEes Xy. A function defined X's denoted by fs. For any real-valued function f X R. we define the empirical fy-norm as 11/1k, = (E /2x/)//n2//2 where XI...., are observed input vectors. We denote a(-) as the sigmoid function, i.e., (z) := exp1=exp(=x)) We denote by Pn. the empirical distribution of (xI..... ,n,,, and by Un.j the marginal distribution of Mn, Xz.
2.2 PROBABILITY MODEL FOR THE LIKELIHOOD
We consider a nonparametric regression model in which the conditional distribution of Y, given X; follows an exponential family (Brown al. 2010 Chen 2024):
Y X4 Qf(cin
(1)
2

---

Published as a conference paper at ICLR 2026
for 1, 1,..., n, where f X - R is regression function and 7) is nuisance parameter. Here, assume that Q5(x).n admits the density function (f(x).n defined as
(3( exp ( ((x))=A(((()) A(f(x)) +Sy..))
(2)
"
where A(-) is the log-partition function, ensuring that the density integrates to one. We assume that each input vector X; has been rescaled, yielding x; E [0, 1]2 for L..... ?.
Example 1. Gaussian regression model: Consider the gaussian regression Y f(x) + 6, where ~ (0, o.). In this case, the density in corresponds to A(f(x) = f(x))/2 and S(y, 7) ==
-y² /2n - (log 2 with rj 7- =
Example Logistic regression model: For binary outcome Y E {0, 13. consider the logistic regression model Y (ernouli((o( In this case, there is nuisance parameter, i.e., This distribution can be expressed as the exponential family with A(((x)) = log(1 + e and S(y, 7)) :=
Likelihood: Let D(n) = {(11(1()) in),...,,x,,,, }n)} be given data which consist of n pairs of ob- served input vectors and response variables. For the likelihood, assume that y:s independent realizations of Y/|x; ~ ((())) where f and 7 are the parameters to be inferred.
2.3 FUNCTIONAL ANOVA MODEL
For 5 _ [p), we say that fs satisfies the sum-to-zero condition with respect to a probability measure j X' if
For S S [p. Vj S and A ()) E Xs((). J/x fs(xs)),(dzz) = 0
(3)
holds, where Mj is the marginal probability measure of μ on Xzy.
Theorem 2.1 (Functional ANOVA Decomposition (Hooker 2007 Owen 2013)). Any real-valued function I defined on Rp can be uniquely decomposed as
f(x)
Σ fs(xs); SC(P)
almost everywhere with respect to 11 ==1 where each component fs satisfies the sum-to-zero con- dition with respect μ..
Theorem 2.1
guarantees a unique decomposition of any real-valued multivariate function f into the
components satisfying the sum-to-zero condition with respect to the probability measure μ.. (4) refer to fs as main effects when |S| 1, as second-order interactions when |S| 2, and so For brevity, we use the empirical distribution pn, for μ when referring to the sum-to-zero condition.
2.4 TENSOR PRODUCT NEURAL NETWORKS
In this subsection, we review Tensor Product Neural Network (TPNN) proposed by Park et. (2025) since we use it as building block of our proposed BNN. TPNN is specially designed neural network to satisfy the sum-to-zero condition.
For each S S [p]. TPNN defined as fs(xs) = -Ks 3s.e((x)5. Ba.k. Rz,a) ) for component Ss. where Bs. E IR, BsS (bs.j.k,j j E 5) E IRISI, and Rs. = (18.AA. E S) E (0, 18) Here, (x Bs Rs.k) defined as
o(xs|S, Bs. Rs.n)
0
Ø
bs.j,k
+ o (bs.s...,77....
2 bs.jjk ))
(5)
75.1
75.1k
JCS
where
3

---

Published as a conference paper at ICLR 2026
(73=)) pn,
/
T.j (" (7)-0)
c(b,7) = 9(6,7):=-11 X,
pn.
(6)
N;
The term (;(b,)) 7) is introduced make ((xs) S.S 2 Bs.k, Rs,k) satisfy the sum-to-zero condition. Finally, Park et al. 2025 proposes ANOVA-TTPNN, which assumes that:
[(x)
E
( S,A Rg,k),
where d E N+ and {Ks, S S [p], S d) are hyperparameters. Since (1-15, k.k, Rs,k) satisfies the sum-to-zero condition for any S [pi. fANOVA-TiPNN also satisfies the sum-to-zero condition. Therefore, we can estimate the components uniquely by estimating the parameters in ANOVA. TOPNN.
Here, d is the maximum order of components. Note that as the maximum order d increases, the number of TPNNs in 7 grows exponentially; therefore, in practice d is set to or due to the limitation of computing resources. In addition, choosing Kşs is not easy. To further illustrate these limitations, the experiments on the runtime Bayesian-TPNN and ANOVA-TTPME are presented in Section of Appendix.
3 BAYESIAN TENSOR PRODUCT NEURAL NETWORKS
In instead of fixing S, we treat also as learn- able parameters. That is, we consider the follow- ing model:
Input
K f(x) = Σ Ox #=1
81
(8)
92
272
where θ₂ = (Sk: bs,, d)). S S [p]. and aim learn K and (Sx, E [K]) well the other parameters. Here,
81
#x|Pa]
Outpur
T3
8
95
D = (bi,,, E St) E [0, 1|
444
I Se,k (Yi,kr S&)) (0, 15.1
<xxes
for le E Note that K and Sş are considered be the parameters defining the architecture, but they cannot be updated by a gradient descent al- gorithm since K and S&S are numeric parameters. Instead, adopt Bayesian approach in which K and S&S are explored via an MCMC algorithm. We refer to the resulting model as Bayesian Tensor Product Neural Networks (Bayesian-TPNNY Bayesian-TPNN can be understood an edge- sparse shallow neural network when K is the number of hidden nodes and Sx is the set of input variables linked to the k-th hidden node through active edges. See Figure for an illustration.
Figure 1: B-aysaian-TPM. TPNN with and K 5.
3.1 PRIOR
The parameters in Bayesian-TPNN consist of K, BR (31,..,, SK (Se, E [K]), bsx,K := kbs..,; & E [K]), 1's == (T (,',,,, [K]) and the nuisance parameter 7) if exists (e.g. the variance of the noise in the gaussian regression model). The parameters can be categorized into the three groups: (1) for the node-sparsity, (2) St, k 1..... K for the edge sparsity, and (3) all the other parameters including (bs I Ss..k. 1, ..... K). We use hierarchical prior for these three groups of parameters.
Prior for K: We consider the following prior distribution for K:
KK- = k) x exp(-Cobe logn), for k = 0, Kmax;
where Kmax N_ and Co 0 are hyperparameters. This prior is motivated by Kong al. (2023)
4

---

Published as a conference paper at ICLR 2026
Prior for SK K: Conditional on K, assume prior that Sys are independent and each Sk follows the mixture distribution:
(auniform(powwer(b)
(10)
where are defined recursively as follows:
(1 Pudtting (d) Ie Puditing (c) with
Padding (f) Osdding e) ""adee Here, Padding is the probability of adding variable to Sg. con-
trolled by hyperparameters Otadding and "adding. This prior is inspired by Bayesian CART Chipman
al. 1998 where Sy denotes split variables.
Prior for the numeric parameters given K and Sx: All the remaining parameters are numerical ones and hence we use standard priors for them.
Conditional on K, we assume a prior that B&s are independent and follow Bn ~ (0, where σ3 > is hyperparameter.
Conditional on Sk, we let bj.u and Y be all independent and bj.8 Uniform (0, and 7/.,k Gamma(a,, b,) for j Sy and E [K], where a,y 0 and by hyperparameters.
For the nuisance parameter in the gaussian regression model, where the nuisance parameter 1) corresponds to o², we set 0² ~ (G(;, eλ 2 where e > and > are hyperparameters and IG( the inverse gamma distribution.
3.2 MCMC ALGORITHM FOR POSTERIOR SAMPLING
We now develop an MCMC algorithm for posterior sampling of Bayesian-TPNN. Our overall sam- pling strategy is update K, S and the remaining numeric parameters iteratively using the cor- responding Ceropooli--Hsttinge (MH) algorithms, which is motivated by the MCMC algorithm of Bayesian additive regression tree (Chipman et al., 2010). A novel part of our MCMC algorithm, however, is to devise specially designed proposal distribution in the MH algorithm such that the proposal distribution encourages the MCMC algorithm to visit important higher-order interactions more frequently. For this purpose, introduce two special tools. First, we employ pretrained probability mass function Pmmpu(+) on [p], which represents the importance of each input variable. Further, let (- be the distribution Pinpux (-) restricted to S C [p]. See Remark the end of this subsection for the choice of Pinpu( (-)-
The second tool is stepwise search. The stepwise search adds of existing nodes and add an edge. By doing so, a newly added node has one more edges than the interaction whose order is larger than the copied one by 1.
node by first copying
copied node and thus corresponds By keeping the copied node also in the model, we can avoid dramatic loss of accuracy.
To be more specific, let (K, Sk, bsx K, J' BK, 11) be given current parameters. We up- date these parameters by sequentially updating K, (Sx. bsk,k, I's.. K, Bx) and the nuisance pa- rameter 17. We now describe these updates.
Updating K: First, we devise a proposal distribution of Knew given K used in the MH algorithm. For given K, we set Krnew as K or K + with probability K/Knax and 1 - K/ K/Kma respectively. If new K- 1, we remove one of (Sk, bsc k, Bx), E [K] from with probability 1/K to have gnew.
For case K new
+ 1, the crucial mission is to design an appropriate proposal of gnew Specifically, first generate * and then generate
(Snew
b
Spew
K++
new
+1+11 K
(baew
Bnew conditional on Snew The proposal of R+1 consists of the fol- 1KI.K++1
K+11
Stow
K+1
lowing alternations:
Random: Generate
Snew
KK+ from the prior distribution.
Stepwise: Propose Snew
Sk- U (ik-) where k* Uniform []] and ja- Pimpaa (-1Sg.)
5

---

Published as a conference paper at ICLR 2026
The MH algorithm randomly selects one of (Random, Stepwise) with probability M/(M + K), and K/(M + K). where M >> 0 is a hyperparameter. This proposal combines random and step- wise search, where Snew is sampled as completely
index set from the prior with prob- abiliity M/(M + K), or taken as higher-order modification of one of Si....,SK with prob- ability K/(M + K). We employ Stepwise move to encourage the proposal distribution to ex- plore higher-order interactions more frequently without losing much information in the current model (i.e. keeping all of the components in the current model). Once Snew is given, we gener- K +
ate
new S K K-1 5%
nee
,,K+1' PK+1) from the prior distribution. See Section A. of Appendix for
the acceptance probability for this proposal gnew and see Section C.5 of Appendix for experimental results demonstrating the effectiveness of the proposed MH.
Updating (S,, bsck k. Ne 3e) for E [K] For a given k, we consider the following three possible alterations of Sp and (bs,,,, ('s,, v.) for the proposal of (S b
Adding: Adding a new variable jumm which is selected randomly from St. according to the probability distribution Pinpul (S)), and generating bymn &. and yym & from the prior distribution.
Deleting: Uniformly at random, select an index j in Ss and delete it from Sk-
Select an index j uniformly at random from Sg and index juew from Sy accord- ing the probability distribution of Pinpur P-)pg((1(S) and delete j from S₁, and add jnew to S Then, generate byoo and Tj from the prior distribution.
Changing:
The MH algorithm randomly selects one of {Adding. Deleting, Changing) with probability (yadd Adclete- 9change): ). This proposal distribution is modification of one used in BART (Chipman
et al. 1998, Kapelner & Bleich| 2016) to grow/prune or modify a current decision tree. See Section A.2 of Appendix for the acceptance probability of (Stew b Dess St
Once (Sk, I... are updated, we update all of the numeric parameters (bs,.., ks S 3x) by the MH algorithm with the Langevin proposal (ros 1978) to accelerate the convergence of the MCMC algorithm further. Finally, we repeat this update for k E [K] sequentially. See Appendix A.3 for details and Section for a toy example illustrating the proposed MCMC algorithm.
Updating the nuisance parameter n7 In the gaussian regression model, the nuisance parameter corresponds to the error variance o Since the conditional posterior distribution of is Inverse Gamma distribution, is straightforward to draw og from π T (othor))s). Details are provided in Section A.4 of Appendix.
Algorithm MCMC algorithm Bayesian TPNN.
Input [xx; v)) data, K initial number of hidden nodes, Mmene the number of MCMC iterations,
1: for i: 1 to Mmeme do
2: Update K
3: for k 1 to K do
4:
Update Sk, bs 1'sk.k 5: Update bs,,k, S 3h.
6: end for
7: Update 1
8:
end for
Predictive Inference. Let 0,,... dy denote samples drawn from the posterior distribution. The predictive distribution then estimated as p(y x) = EN p(y|x, A.)N.
Remark 3.1. When no prior information is available on the importance of input variables, we use uniform distribution for Pmpu-- However, this noninformative choice often performs poorly when the dimension p is large and higher-order interactions exist. Our numerical studies in Section Cevvelll that the choice of good Pinpu is important for exploring higher-posterior regions. In practice,

---

Published as a conference paper at ICLR 2026
could specify Pinput based on the importance measures of each input variable obtained by standard method such as Molnar (222). That is, we let Pippe()) x Wj, where W, is an importance measure the global SHAP value (Molnar 2020)
of the input vartable J E [p]. In mumerical study, based on a pretrained Deep Neural Network (DNN) for the importance measure or the feature importance using pretrained eXtreme Gradient Boosting (XGB, Chen & Guestrin (2016))
3.3 POSTERIOR CONSISTENCY
For theoretical justification of Soyeesian-PPNNY in this section, prove the posterior consistency of Bayesian- To avoid unnecessary technical difficulties, we assume that ((x(O,) ) in (8) satisfies the sum-to-zero condition with respect the uniform distribution. This can be done by using the uniform distribution instead the empirical distribution in (6).
We assume that (X1, 31); ..., (Xn: Un) are realizations of independent copies (1), (X,, Yn) of (X, Y) whose distribution Qo is given as
X Px and Y = ~ (f(xx).1
where fo is the true regression function. We n) = 1 for technical simplicity. Suppose that fo(x) = Escpl (x,s( where each fo.s satisfies the sum-to-zero condition with respect to the uniform
distribution. We denote (() 1X1, .... } and (*) = IY..... Yn). Let (-(-) o (-))
<). where π( is the prior distribution of defined in Scction 3.1 Under regularity conditions (S.1)
(S.2]
(S.) and (8.4) in Section M.2 of Appendix, Theorem 3.2 proves the posterior consistency
of each component of Bayesian-TPNN.
Theorem 3.2 (Posterior Consistency of su (x(x) < co, where px(x) is the density of Px. Then, there exists > 0 such that for any E > 0, we have
Assume that < infxex px(x) s
Te (f Ilfo,s
I1: - fsll2, x(x(m), r(m) -0
(11)
for all S S [p] in Q as + co, where Te ((()) is the posterior distribution of Bayesian- TPNN with the prior xe.
4 EXPERIMENTS
We present the results of the numerical experiments in this section, while further results and com- prehensive details regarding the datasets, implementations of baseline models, and hyperparameter selections are provided in Sections B to H of Appendix.
4.1 PREDICTION PERFORMANCE
We compare the prediction performance of BayN with baseline models including ANOVA-TPNN (Park al. 2025), Neural Additive Models (NAM, Agarwal al. (202T)) Lin- car model, XGB (Chen & Guestrin 2016). Bayesian Additive Regression Trees (BART. Chipman al. (2010), Linero (2025)) and mBNN (Kong al. 2023). We analyze eight real datasets and split each dataset into training and test sets with a ratio of 0.8 to 0.2. This random split is repeated five times obtain five prediction performance measures.
Table reports the prediction accuracies (the Root Mean Square Error (RMSE) for regression tasks and the Area Under the ROC Curve (AUROC) for classification tasks) of the Bayes estimator of BaYN along with those of its competitors, where the best results are highlighted by bold. Overall, Bayesian-TPNN achieves prediction performance comparable to that of the baseline mod- cls. Further details of the experiments are provided in Section B.3 of Appendix.
Table 2 compares Bayesian- TPNN with the baseline Bayesian models in view uncertainty quan- tification. As uncertainty quantification measures, we consider Continuous Ranked Probability Score (CRPS, Gneiting & Raftery (2077) and Negative Log- Likeliiiddoood (NLL) for regression tasks, and Expected Calibration Error (ECE, Kumar al. (019)) ) together with NLL for classification tasks. The results indicate that Bayesian-TPNN compares favorably with the baseline models in uncer- tainty quantification, which is bit surprising since Baye is transparent model while
7

---

Published as a conference paper at ICLR 2026
Table The averaged prediction accuracies (the standard errors) on real datasets.
Interpretable model
Blackbox model
Dataset
Measure
Rayesian TPNN
ANOVA TPNN
NAM
Linear
XGB
BART
mBNN
2,053
2,051
2.062
2.244
2.157
2.197
2,081
ABALONE Warwick et al.
1995
(0.26)
(0.21)
(0.23)
(0.22)
(0.24)
(0.26)
(0.24)
BOSTON Harrison
Rubinfeld
1978
RMSE
3,654
(0.49)
(0.56)
3.671
(0.67)
3,832
5,892
(0.77)
4.150
(0.56)
4,073
(0.67)
(0.51)
4.257
MPG Quintan
1993
(SF)
2.386
2.623
2.755
3.748
2,531
2.699
2.897
(0.41)
(0.38)
(0.41)
(0.26)
(0.43)
(0.42)
S SERVO Ulrich 1986
0.351
0.594
0.802
0.314
0.301
(0.02)
(0.01)
(0.01)
(0.04)
(0.01)
(0.01)
(0.01)
Pico dic 2018
0.793
0.802
0.764
0.690
0.793
0.701
0,740
(0,009)
(0.008)
(0.019)
(0.010)
(0,009)
(0.015)
(0.008)
BREAST Walberg et
AUROC f
(0.001)
0.988
(0.001)
0.998
(0.0033
0.976
(0,010)
(0.802)
0.995
(0.006)
0.977
(0.002)
0.978
CHURN chu 2017
(SE)
(0.008)
0,849
(0.006)
0.848
(0.008)
0.835
(0.002)
0.720
(0.006)
0,848
(0.008)
0.835
(0.008)
0.833
MADELON Gayon 2004
0.854
0.587
0.644
0.548
0,884
0.751
0.650
00.013)
(0.013)
(0.005)
(0.011)
(0.006)
(0.011)
(0.018)
Table 2: Comparison of Bayesian models in view of uncertainty quantification on real datasets.
Bayesian-IPNN TPNN
BART
mBNN
Datasct
CRPS
NLL
CRPS
NLL
CRPS
NLL
ABALONE
1.372 (0. (0.19)
2,260 (0.16)
1.384 18)
(0.16)
1.399 (0.16)
2.226
BOSTON
2,202 (0.23)
3.411 (0.37)
2.623 (0.25)
3.400 (0.42)
3.144 (0.39)
3,488 (0.26)
MPG
1,510 (0.43)
2.511 (0.21)
1.553 (0.27)
2,530 (0.20)
2.142 (0.42)
2.710 (0.24))
SERVO
0.194 (0.01)
0.836 (0.10)
0,202 (0.02)
0.849 (0.08)
0.185 (0.02)
0.321 (0.08)
NL.
NL.L
ECE
FICO
0 (0.004)
0.554 (0.007)
0.054 (0.011)
0.632 (0.012)
0.219 (0.032)
0.773 (0.046)
BREAST
0.129 (0.009)
0,211 (0.014)
0.118 (0.010)
0.143 (0.032)
0,292 (0.018)
0.523 (0.025)
CHURN
0.031 (0.001)
0.418 (0.008)
0.035 (0.001)
0.430 (0.010)
0.168 (0.037)
0.531 (0.036)
MADELON
0.076 (0.004)
0.478 (0.009)
0,066 (0.004)
0.685 (0.032)
0.252 (0.020)
0.840 (0.031)
the other Bayesian models are black-box models. The results of uncertainty quantification for non- Bayesian models are presented in Section H.1 of Appendix, which are inferior to Bayesian models.
4.2
PERFORMANCE IN COMPONENT SELECTION
Table 3: Performance of component selection on synthetic datasets.
True model
(())
f(2)
((3)
Order
Bayesian
ANOVA TºPNN
NA
Bayesian TPNN
ANOVA TºPNN
AM²M
Bayesian
ANOVA TºPNN
PM
L0000
0.999
0.528
0.831
0.859
0.417
1,000)
0.781
0.522
(0.001)
(0.023)
(0.015)
(0.021)
(0.011)
2
L.000
0.978
0,508
0.985
0.949
0.838
0.922
0.704
0.542
(0.000)
(0.007)
(0.024)
(0.003)
(0.003)
(0.009)
(0.019)
(0.007)
(0.017)
0.740
0.966
0,661
(0.022)
(0.018)
(0.022)
Table 4: Top components: the important scores are normalized by their maximum.
Rank
Rank 2
Rank
Rank
Rank 5
Dataset
Component
Score
Component
Score
Composent
Score
Component
Score
Component
Score
MADELON
(49, 242, 319, 339)
1,000
(129, 443, 494)
(379, 443)
0.322
(242, 443)
0.301
SERVO
L.000
(L. 3, 5)
0.554
0.202
(4, ()
0.193
0.175
We investigate whether Bayesian TPNN identifies the true signal components well similarly to the setting in Park et al. (2025); Tsang et al.] (2017). Synthetic datasets are generated from
Y = (x) e = 1, 2, 3. where f((5 is the true regression model and x E 1550 Details of the experiment are described in Section B.5
We define the importance score of each component as its E -norm, i.e., || A large llfs implies fs is signal. Table B reports the averages (standard errors) of AUROC' of the importance scores obtained by Bayesian- TPNN, ANOVA ANOVA-T'PNN, and NA? M for interaction order up to 3. Note that extending ANOVA-T*PNN and 2 to include the third order interactions requires additional
8

---

Published as a conference paper at ICLR 2026
Contributon
-14
-03
-2.
-26
Average number rooms per dwallin (1.000)
lower atortua of the population (9.8 (8.815)
Per town
Distances amploymmet centres (0.488)
Figure 2: Plots of the functional relations of the important main effects estimated by Bayesian-TPNN on the BOSTON dataset. Each plot shows the Bayes estimate and 95% credible interval of each component. Labels indicate the names of the input variables along with the normalized importance scores.
19, 600 neural networks, and so we give up ANOVA-T PNN and NA due to the limitations of our computational environment. Overall, selection across orders and datasets, and detects higher-order interactions reasonably well.
achieves the best performance in component
T the five most important components selected by Bayesian-TPNN on MADELON and SERVO datasets. We use these datasets they highlight the performance gap between models with and without higher-order interactions. Notably, Bayesian- TPNN identifies a 4th-order interaction as the most important component in the MADELON data, suggesting that its ability capture higher- order interactions largely explains its superior prediction performance over ANOVA-TPNN on these datasets. See Section B.2 of Appendix for descriptions of the variables in MADELON and SERVO.
4.3 INTERPRETATION OF BAYESIAN- -PNNN
The functional ANOVA model can provide various interpretations of the estimated prediction model through the estimated components as Park et al. (2025) illustrates. In particular, by visualizing the estimated components, can understand how each group of input variables affects the response variable. Figure presents the plots of the functional relations for the important main effects esti- mated by Soyyesian-TPNT credible interval of the selected component. The leftmost plot shows increasing trend, indicating that as the average number of rooms per dwelling increases, the price of the housing increases as well. The second plot reveals strictly decreasing relationship between the proportion of lower status of the population and the housing price. The third plot indicates that housing prices decrease sharply once the crime rate exceeds certain threshold. The fourth plot shows that houses located farther from major employment centers are generally less expensive than those situated closer to such hubs. More discussions about interpretation of BayN are provided in Section E of Appendix.
the BOSTON dataset. Each plot shows the Bayes estimate and the 95%
4.4 APPLICATION CONCEPT BOTTLENECK MODELS
Concept Bottleneck Model (CBM, Koh et al. (2020)) is an interpretable model in which a CNN first receives an image and predicts its concepts. These predicted concepts are then used to infer the target label, enabling explainable predictions. To illustrate that Bayesian-TPNN can be amply combined with CBM, we consider Independent Concept Bottleneck Models (ICBM, Koh al. (2020). in which CNN is first trained and then frozen, after which final classifier is trained on the predicted concepts. We compare Bayesian-Tie TPNN with other baselines for learning the final classifier. In the experiment,
CELEBA-HQ (eee et al., 2020) and CATDOG (Jikadara 2023) datasets, where we generate 5 concepts using GPT-5 (OpenAl] 2025), and we obtain the concept labels for each image via CLIP (Radford al. 2021). The target labels for CELEBA-HQ and CATDOG are gender and cat/dog classification, respectively. The details are provided in Section B.4 of Appendix.
Table 5: Prediction performance with CBM on image datasets.
Datasct
Measure
Bayesian- TPNN
ANOVA-
NA
Linear
CELERA HQ
AUROC 1 AUROC
0.936 (0.002)
0.923 (0.002)
0.922 (0.002)
0.893 (0.003)
CATDOG
0.878 (0.002)
0.853 (0.002)
0.851 (0.002)
0.711 (0.001)
Table presents the averages and standard errors of AUROCs when Bayesian-TPNN, ANOVA- T2PNN, NA?M, and Linear model are used in the final classifier. Among them, Bayesian-TPNN

---

Published as a conference paper at ICLR 2026
attains the highest prediction performance, which can be attributed to its capability to estimate higher-order components.
5 CONCLUSION
We proposed Joyeesia-TPNNy a novel Bayesian neural network for the functional ANOVA model that can detect higher-order signal components effectively and thus achieve superior prediction per- formance in view of prediction accuracy and uncertainty quantification. In addition, Bayesian- TPNN also theoretically sound since it achieves the posterior consistency.
We used predefined distribution Puput for the selection probability of each input variable in the MH algorithm. would be interesting to update Pinput along with the other parameters. For example, it would be possible to let (j) be proportional to the number of basis functions in the current Bayesian-TPNN model which uses xz-- This would be helpful when p is large. We will pursue this algorithm in the near future.
Reproducibility Statement. We have made significant efforts to ensure the reprodueibility of our results. The source code implementing our proposed model and experiments is provided in the supplementary material. Detailed descriptions of the experimental setup, hyperparameters and datasets are provided in Section of Appendix. Additional ablation studies are reported Section C of Appendix.
ACKNOWLEDGEMENTS
This work was supported by the National Research Foundation of Korea(NRF) grant funded by the Korea government(MSIT) nology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) [NO.RS-2021- 1121 1343, Artificial Intelligence Graduate School Program (Seoul National University)) and by the National Research Foundation of Korea(NRF) grant funded by the Korea governnen 2022RIA5A7083908)
2025-00556079), Institute of Information & communications Tech-
REFERENCES
Brownian dynamics as smart monte carlo simulation. The Journal of Chemical Physics, 69(10): 4628-4633, 1978.
Telco customer churn. kaggle, 2017. http://www...ggcooommsessssssess churn/data.
Fico heloc. FICO Explainable Learning Challenge, 2018. hatiraabeemmaaciii-eaannnnnn-i-e
http:://commmunittffcccoooo////
Rishabh Agarwal, Levi Melnick, Nicholas Frosst, Xuezhou Zhang, Ben Lengerich, Rich Caruana, and Geoffrey E. Hinton. Neural additive models: Interpretable machine learning with neural nets. Advances in neural information processing systems, 34:4699 4711, 2021.
Lawrence D. Brown, T. Tony Cai, and Harrison H. Zhou. Nonparametric regression in exponential families. The Annals of Statistics, 38(4):2005 2046, 2010. doi: 10.2214/0-AA07622 URL
https: //doi wwwoor//ooo... /tt./11.111111 4/99 AOS762
Chun-Hao Chang, Rich Caruana, and Anna Goldenberg. Node-gam: Neural generalized additive model for interpretable deep learning. arXiv preprint arXiv:2106.011 2021.
Juntong Chen. Estimating regression function in exponential families by model selection.
Bernoulii, 3012):1669-1693 2024.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
10

---

Published as a conference paper at ICLR 2026
Hugh Chipman, Edward I George, and Robert E McCulloch. Bayesian cart model search. Journal of the American Statistical Association, 93(443):935-904 948, 1998.
Hugh A Chipman, Edward George, and Robert E McCulloch. Bart: Bayesian additive regression trees. Annals of Appied Statistics, 2010.
Yongchan Choi, Seokhun Park, Chanmoo Park, Dongha Kim, and Yongdai Kim. Meta-anova: screening interactions for interpretable machine learning. Journal of the Korean Statistical Soci- ely, pp. 1-18, 2025.
Angelos Filos, Sebastian Farquhar, Aidan Gomez, Tim GJ Rudner, Zachary Kenton, Lewis Smith, Milad Alizadeh, Arnoud De Kroon, and Yarin Gal. systematic comparison of bayesian deep learning robustness in diabetic retinopathy tasks. arXiv preprint arXiv:1912.10488 2019.
Jerome H Friedman. Multivariate adaptive regression splines. The annals of statistics, 19(1):1-67, 1991.
Fumagalli, A Blanchet-Cohen, D Brown, C Desmedt, D Gacquer, Michiels, Rothé, S Ma- ijaj. R Salgado, D) Larsimont, M Maetens, M Piccart, V Detours, C Sotiriou, and B Haibe- Kains. Gse43358: Transfer of clinically relevant gene expression signatures in breast cancer: from affymetrix microarray to illumina ma-sequencing technology. 2013, URL/ht ncbi. nlm. nih. http:///ww/////////////////////a
Accessed: 2025-11-17.
Zhe Gan, Changyou Chen, Ricardo Henao, David Carlson, and Lawrence Carin. Scalable deep poisson factor analysis for topic modeling. In International Conference on Machine Learning, PMLR, 2015.
pp.
Subhashis Ghosal and Aad Van Der Vaart. Convergence rates of posterior distributions for noniid observations. 2007.
Subhashis Ghosal, Jayanta K Ghosh, and RV Ramamoorthi. Posterior consistency of dirichlet mix- tures in density estimation. The Annals of Statistics, 27(1):143- 1999.
Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation.
Journal of the American statistical Association, 102(477):359-37, 2007.
Peter J. Green. Reversible jump markov chain monte carlo computation and bayesian model deter-
mination. Biometrika, 82(4):111-73, 12 1995. ISSN 0006-3444. doi: 10.1003/biomee/22...111..
URL http::///obbboooo..00000//////ee
711
Chong Gu and Grace Wahba. Smoothing spline anova with component-wise bayesian "confidence intervals". Journal of Computational and Graphical Statistics, 2(1))97-117, 1993. doi: 10.
1080/106186000000 1600..999.104446000 URL
10474601
Isabelle Guyon. Madelon.
UCI
Machine
Learning Repository, 2004.
DOI:
http://ddo..r//1122443/////
László Györfi, Michael Kohler, Adam Krzyzak, and Harro Walk. distribution-free theory of non- parametric regression. Springer Science & Business Media, 2006.
David Harrison and Daniel L Rubinfeld. Hedonie housing prices and the demand for clean air. Journal of environmental economics and management, 5(1):81-102, 1978.
Trevor Hastie and Robert Tibshirani. Generalized additive models. Statistical science, 1(3):297- 310. 1986.
Andrew Herren and P Richard Hahn. Statistical aspects of shap: Functional anova for model inter- pretation. arXiv preprint arXiv:2208.0997% 2022.
Giles Hooker. Generalized functional anova diagnostics for high-dimensional functions of depen- dent variables. Journal of computational and graphical statistics, 1663):709-732, 2007.
11

---

Published as a conference paper at ICLR 2026
Pavel Izmailov, Sharad Vikram, Matthew D Hoffman, and Andrew Gordon Wilson. What bayesian neural network posteriors really like? In International conference on machine learn- ing. pp. 4629-4640 PMLR, 2021.
Bhavik Jikadara. Dog and cat classiffiattin dataset. http:://.tttt........ttotom wtt.e//a//aaaaecaeeeeeeeeaaaaeee Mhhawijikaaien
2023.
Adam Kapelner and Justin Bleich. bartmachine: Machine learning with bayesian additive regression trees. Journal of Statistical Software, 70:1-40, 2016.
Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. Concept bottleneck models. In International conference machine learning, pp- 5338-5348, PMLR, 2020.
Insung Kong, Dongyoon Yang, Jongjin Lee, Ilsang Ohn, Gyuseung Back, and Yongdai Kim. Masked bayesian neural networks: Theoretical guarantee and its posterior inference. In International conference on machine learning, pp. 17462-17491, PMLR, 2023.
Ananya Kumar, Percy S Liang, and Tengyu Ma. Verified uncertainty calibration. Advances in neural information processing systems, 32, 2019.
Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles, Advances neural information processing systems, 30, 2017.
Cheng-Han Lee, Ziwei Liu, Lingyun Wu, and Ping Luo. Maskgan: Towards diverse and interactive facial image manipulation. In IEEE Conference on Computer Vision and Puttern Recognition (CVPR), 2020.
Benjamin Lengerich, Sarah Tan, Chun-Hao Chang, Giles Hooker, and Rich Caruana. Purifying interaction effects with the functional anova: An efficient algorithm for recovering identifiable additive models. In International Conference on Artificial Intelligence and Statistics, pp. 2402- 2412. PMLR, 2020.
Antonio R Linero. Generalized bayesian additive regression trees models: Beyond conditional con- jugacy, Journal of the American Statistcall Association, 120(549):356-369, 2025.
Scott M Lundberg and Su-In Lee. A unified approach to interpreting model predictions. Advances in neural information processing systems, 30, 2017.
David JC MacKay, practical bayesian framework for backpropagation networks. Neural compu-
tation, 4(3):488 472, 1992.
Kaspar Märtens and Christopher Yau. Neural decomposition: Functional anova with variational autoencoders. In International conference on artificial intelligence and statistics, pp. 2917-2927. PMLR, 2020.
Christoph Molnar. Interpretable machine learning. Lulu, com, 2020.
Jishnu Mukhoti, Andreas Kirsch, Joost Van Amersfoort, Philip HS Torr, and Yarin Gal. Deep de- terministic uncertainty: A new simple baseline. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp- 24394, 2023.
Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.
Nhat Minh Nguyen, Minh-Ngoc Tran, and Rohitash Chandra. Sequential reversible jump meme for dynamic bayesian neural networks. Neurocomputing, 564:126960, 2024.
Tuomas Oikarinen, Subhro Das, Lam M Nguyen, and Tsui-Wei Weng. Label-free concept bottleneck models. arXiv preprint arXhv:2304.06129 2023.
OpenAl.
Introducing gpt-5, August 2025.
URL
introducing-gpt-55 Accessed: 2025-09-17.
12

---

Published as a conference paper at ICLR 2026
Art B. Owen. Monte Carlo theory, methods and examples. httpp::/ domains. me/
2013.
Seokhun Park, Insung Kong, Yongchan Choi, Chanmoo Park, and Yongdai Kim. Tensor product neural networks for functional anova model. International conference on machine learning, 2025.
Pedregosa, G. Varoquaux, Gramfort, V. Michel, Thirion, O. Grisel, Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research,
12:2825-2830, 2011.
R. Quinlan. Auto MPG.
UCI
Machine
Learning Repository, 1993.
DOL:
http://dbb.or1110244222///$33
Filip Radenovic, Abhimanyu Dubey, and Dhruv Mahajan. Neural basis models for interpretability. 2022.
Advances in Neural Information Processing Systems,
Alec Radford, Jong Wook Kim, Chris Hallaey, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. In International conference on machine learning, pp- PmLR, 2021.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. "why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference knowledge discovery and data mining, pp. 1135-1144, 2016
Anton Frederik Thielmann, René-Marcel Kruse, Thomas Kneib, and Benjamin Säfken. Neural additive models for location scale and shape: A framework for interpretable neural regression beyond the mean. In International Conference on Artificial Intelligence and Statistics, pp. 1783- 1791. PMLR, 2024.
Michael Tsang, Dehua Cheng, and Yan Liu. Detecting statistical interactions from neural network weights. arXiv preprint arXiv: 1 100.1100000.0 705.04977,7... 2017.
Karl Ulrich.
Servo.
UCI
Machine
Learning Repository, 1986.
DOI:
https://b..oog/11.243322250
Hao Wang, Naiyan Wang, and Dit- Yan Yeung. Collaborative deep learning for recommender sys- tems. In Proceedings of the 2/th ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1235-1244, 2015.
Nash Warwick, Sellers Tracy Talbot Simon, Cawthorn Andrew, and Ford Wes. Abalone. UCI Machine Learning Repository, 1995. DOI: http:///bb..rg//024444///CCCCCC
Andrew G Wilson and Pavel Izmailov. Bayesian deep learning and probabilistic perspective of generalization. Advances in neural information processing systems, 33:4697-44788, 2020.
William Wolberg, Olvi Mangasarian, Nick Street, and W. Street. Breast Cancer Wisconsin (Diag- nostic). UCI Machine Learning Repository, 1993. DOI: http://abb.og//02444////55w
13

---

Published as a conference paper at ICLR 2026
APPENDIX
A DETAILS OF THE MCMC ALGORITHM
For given data D(m), we denote x( {x1, X,). Let wij
Puopul (j).
A.1 SAMPLING K VIA MH ALGORITHM
A.1.1
CASE OF Keew = K +1
From current state I SK, bsx Ki Sx K BR 7), propose state grew using of [Random, Stepwise). Here, gnew is defined as
(new = (K + 1, 11 I KI Bk+1,
where
SK+1 (SK. Snew kK+1)
bs k1 (bsx, K2 S R (K1);
[s 1+1 (Tsx, ; Ism S K+1);
gnew BK+1 (B (BKKKK+11 11*1))
We accept the new state gnew with probability
a ). (We) (W1) T
(0 g(0)0aow)
Paccept
min
1.
Of (x,())) (Us)
x x())
o @Bnew (
where
fo(x)= Σ
Beo(x (S bs ss
kE[K]
and
fe (x) = fx(
1 (
Gnew
bsw,, K+1, K41 K+1))
To compute the acceptance probability, we calculate the prior ratio π (00m0)/((0)) and then the pro- posal ratio qf (0))0oww) /// (0ww (0).
Prior Ratio.
The prior ratio is given
T(8"0)) (0)
T(K (ST(S++ K ()T(bDs K+1 SK+1)T K+ x (Bx+1 |K +
(K (K)T(Sa (K) (b)sss (k)) (Tsk, (Sx) o (Bx(K)
((Sk+1)m 1 K+ (T's (
exp(Co logn) m)
Proposal Ratio.
For q(@)@new), we have
q @new) = Pr(K neww
1)ProChoose one of ReDew TPNNs for deletion)
Knew
Kmax Know
For given 0, state Unew is proposed in ways: (1) Random move (2) Stepwise move.
14

---

Published as a conference paper at ICLR 2026
For Random move, we have
q [ 10, Random) (SSrm)) (31)) (x,, (12)
For Stepwise move, we have
q0 10, Stepwise) = Pr(S Snew (+)) ( (2)
Here, Pr(Sk#) K+1 is defined as
Pr(Sk41
Pr(Choose Sy from SK Pr((KPt Sk ( juw Sk)
I(Ejuew new E St s.t Sp U (jmom]
=SK+
SnEW
w her
K
K+1
Σ SEss Wi
1=1
To sum up. we have
q( (0) @B0ew (0, Randomn) (aandom)Pr(Randdeee) Pr((doomm) +
A.1.2 CASE OF Keew K -
Since the acceptance probability of the case Know K - can be easily computed by reversing the steps in Section A.I.I. we omit the details here.
A.2 SAMPLING Sk, Da,Ta P VIA MH ALGORITIMM
Here, consider three moves {Adding, Deleting and Changing): Each move is chosen with the probabilities Pr(Adding) = qadd, Pr Pr(Deleting) Gdelete Pr( Pr(Changing) = Ichange: respectively.
In Adding move, the proposal distribution generates Sp.w = Sy U { where juuhling E [p]\ chosen with given weight vector using 5% alone, where sh.w is the index set generated by the proposal distribution. To address this, also generate bjudtin, and Tyutii, from Uniform(0, 1) and Gamma(a,, b.), respectively.
(W1, ...: Wp). Note that the likelihood cannot be calculated
Furthermore, in Deleting move, a variable to be deleted uniformly selected from S₁ and the new component Snew S& ( (ddcketnng is proposed accordingly. This move also involves removing the associated numeric parameters bysskiss, and y A from bs.,k and I k. respectively.
Finally, in Changing move, we choose an element jehange in S₁ and replace it with a randomly selected new -S.. The corresponding bjatuur A and Yjcheue & are then replaced by new values generated from Uniform(O, 1) and Gamma (a,, by). respectively. This move results in Spew (S& j changge U (jnew).
Here, Adding and Deleting affect the dimensions of bs,,k and S thus the algorithm corre- sponds to RJMCMC (Green (1995)) which requires Jacobian computations. However, since ap- plied the identity transformation on the auxiliary variables which are generated to match the dimen- sions, the Jacobian is simply 1. This allows us to easily compute the acceptance probability.
A.2.1 TRANSITION PROBABILITY FOR PROPOSAL DISTRIBUTION
For a given weight vector w, the proposal distributions tu of O (S b ,cy T sy .k) are defined
Waktin
y poew Oz, Adding)
T(b br) ( (k)
Ejest Wj
qu [eeew) θ₁, Deleting)
= S
Wine
yu (Ope Dew Oc,, Changing)
m (bbjrr,, RT (Tiga p))-
Σ Ejesp
15

---

Published as a conference paper at ICLR 2026
To sum up. have
gw ( (Ox)
qw (Ot ( Addingg)P((Addiigg)
+ qw (of
O Dedeting)Pr(Deletinng
qw (+)
O BaanggingPr(Chhangiinn
A.2.2 POSTERIOR RATIO
We define d = (Ak.1..... ) where AR, Ejen (j0(x;10)) for i, n and the likeli- hood C(Os. 3k. Ab: 17) = II==1 100,
Then, have
. ( Ak, D(m),
n) TT/1 ... Yn 0s Bk. de, (1)(a)))))))
= L( 3k, uE. 77) (+x).
(S bis D Sy xx) to O₂ = (Ske b..,kr T Tk given as Thus the posterior ratio of O
π T BR, Ap: D(m), .n)
[(Or. 3k. dc, 1)) n (O
(O [3k, dR, D(n), 2(O), 8x. )n x(Ox)
A.2.3 ACCEPTANCE PROBABILITY
In this section, for notational simplicity, we denote the hyperparameters Oladding and Tadding as a and 7, respectively.
For proposed new state Ozww we accept it with probability
x (O 13. dg. D(2), 1) qw (9 O Onew
Paceet min
BR, dv. D(m), 17) Iw O O
C(O 8e Ak. (O)o Ow ( new [(O,, BR AE, 7) T(p)) ( (O)) mim
Now, we will show how the product of the prior and proposal ratios is calculated the case of Adding, Deleting, and Changing.
For Adding, we have
x (( qu ( O new
π (O&)
(ho new Og)
ax |S -7
Pr(Deleting) Eies Wi
= a snew
ar
Snew
Snew
Cnew 1 Pr Pr(Adding)
For Deleting, we have
πt T(Onow) l 1 (O
x(O)) Iw new (+k)
1 a=1
S -7
Pr Pr(Adding) (Deleiing) Elesy WI
Wylckiiss
Snew -77 (p- SSneew
a(1 + Snew
7 1
a
For Changing, we have
क (0
Owr (Ox O
Wjsharue Eiesy Wi
T ((e)) qw (O)ew (p) jnon Eve(Spa Wi
16

---

Published as a conference paper at ICLR 2026
A.3 SAMPLING & Sss, AND 3K VIA MH ALGORITHM
We use Langevin Dynamics ros 1978)) as proposal distribution for bs Ss and 3x. That is
boew
reew and Beew proposed Sa.k
Sk.kt
2 5k = (bs,,,k, Su 3x) + Uls k P Syyc 3x.) + eMI,
new Sk,k
new
2
where
Ub ks 3k) L. log ग (bss, T S. 8k De, Sk, D(m)) .1). Here, M ~ N(O, I). where is the (2|Sa 1) (2|Sa) + 1) identity matrix and 0 is step size.
We accept the proposal (bS,,, roew
St kv ( with a probability Paccept given as
(Sk,. new new Se.k Sc.kt Pt Ag. 11 x <(b new
Sp.k) k) π [ new
Sk. I (( exp (-2 (1 [[M0113)
Paccopt
L(S,, bs P. Be, )k)T) (bs,,)) ([ (1))).))
T (3a)
where || ||2 is the Euclidean norm for vector and
M MI /
U (iss Ss 3x) + e UU(bS, Sn.k: I
Sy.ki (
For V (bsg. ii (( log n. (bs S As. Sr. D(n), n). we will calculate
Vbsa (bs, Ak. Bk, Sk. k, D(n) ,)),
Vrs log (I's Au. BR Sk, bs D(n) 17),
and
VA, log 7 (Bxx (ex Sk, bs s. D(v), n!)
A.3.1 CALCULATING THE GRADIENT THE LOG-POSTERIOR WITH RESPECT TO bs,
Without loss of generality, let S₁ (l,...,d}
Since
7 Tbs... Bk. Sk. St.k D( .n) L()s, Bk, Sk, bs Ss 17),
the j-th gradient is given
0 ō6,,
log π (bs.. AR, Br, St. I k. D(v) m1)
log (f()))., (B:))
Zbg,k s=1
where f(x;) Aa.i + 3x ThEs, M(a,l1)) br. In turn, we have
log (f(x))))))))))))) (f(k))),
Abi.k
2 log (11)), 0f(xi)
Of(x))
Obg,
log (/(x)))) (vi) Oblarsli kj 0f(x;)
(aull() bi, mas
B&
II
Øbg,
17

---

Published as a conference paper at ICLR 2026
Here,
( Yj.b) = - Zi,j
-b(t) +
Fi,j b3.k Ys
YA.k
Cy(bg,bs Vjik) = -11 (-0,00,,n1 / /2/(0p..13..)
where 2,(b,7) = Sxz - (du).
Then, we have
Op(ais) by,n. Tg.k)
1
[i.j - bjsk
" -bbb
Mnn (des)
Øbg.6
Yi.k
Yiuks
Visk
Tiis
bjik
+
(0;kk 7j.k)
73.k
where ((x) := -l)
A.3.2 CALCULATING THE GRADIENT OF THE LOG-POSTERIOS WITH RESPECT TO Sss.k
Without loss of generality, we let S₂ (1, ..., d]. Similarly Section A.3.1 of Appendix, we can derive the gradient of the log posterior with respect to 7/.k
log T (T /A Br, Sk, bss..k D(n) 11)
Oīj
log (((x)))))1))) ) + (ay - 1)
Oyj.k
Yi.k by
From f(x,)= Ak,t + Bk Hiss, o(x,,1t)), bi n,x), we have
0
log (f(x))))
0yi.k
3 log ((())) 0x(x))
(x(x:)
log Qf(x))) (1t) dp Tj.k)
BR
II 14j.19.
((aul() br, man
0f(x;)
Ori
Here,
a6f (i). 7j.k)
Oji,k
Jx3 a-by + The
u-bbi
Mon (du)
(zw- bj.kk
(by.e 73.k) - 1) Di.j bj.k.
bi.k
Zg )
7/..k
73.4
7/sk
A.3.3 CALCULATING THE GRADIENT OF THE LOG-POSTERIOR WITH RESPECT TO 3k
The gradient of the log posterior for Bg is given as
Olog Olog (())))))))))
3g
Vg, 9y log (Bk Ar. Sk. bs. T, D(n) m) =
6(x,10x)
0f(x;)
03
18

---

Published as a conference paper at ICLR 2026
A.4 SAMPLING NUISANCE PARAMETER
We only consider the nuisance parameter in the gaussian regression model:
x; ~ (()I((x))o))
for i = 1,..,,,, n, where o² is a nuisance parameter. When the prior distribution is an inverse gamma distribution
v vs og ~ IG
(13)
2 2
we have
1
E (vs - [(xx))2 + Ud )
og BK, SK, bsg,k, K, Ss K, D(n)
IG
(14)
2
and thus of can be sampled from the conditional posterior easily.
19

---

Published as a conference paper at ICLR 2026
B DETAILS OF THE EXPERIMENTS
B. DATA DESCRIPTION
Table 6: Descriptions of real datasets.
Dataset
Task
4.178
Regression
BOSTON
506
13
Regression
MPG
398
Regression
SERVO
167
Regression
FICO
10,459
Classification
BREAST
569
30
Classificatio
CHURN
7,043
Classification
MADELON
4,400
500
Classiication
CELEBA-HQ CATDOG
30,000
Classification
24,998
Classifiction
B.2 FEATURE DESCRIPTIONS FOR MADELON AND SERVO DATASETS
Table 7: Feature index and its corresponding description for SERVO dataset.
Feature index
Feature description
Proportional guin setting for the motor.
Velocity gain setting for the servo motor.
Presence Motor type
4
Presence Motor type B
type
6
Presence of Motor type D
7
Presence Motor type
8
Presence Screw type
9
Presence Screw type
Presence Screw type
11
Presence Screw type
type
Table presents the feature descriptions for SERVO dataset (Ulrich 1986). MADELON (Guyon 2004 introduced in the NIPS 2003 feature selection challenge, is a synthetic binary classification dataset with 500 features, only a few of which are informative while many are redundant or irrele- vant.
B.3 EXPERIMENT DETAILS FOR TABULAR DATASET
Data Preprocessing. All of the categorical input variables are encoded using the one-hot encod- ing. For continuous ones, the inverse of the empirical marginal CDF used transform them to their marginal ranks for Bayesian-TPMy TPNN and ANOVA-TPNN, whereas they are transformed via the mean-variance standardization other baseline models.
Implementation of baseline models.
For implementation of baseline models, proceed fol-
lows.
ANOVA-TPNN
we use the official source wwt..///wa/.aoeorrrrree ANOVA-TPNN,
provided in code provided in
code
petpp://githhe www.coooo///oooooooo
NA-M
the official source
https: http//ggttttcccoo com Boom/AmrMKayice d/amm and NA implemented by extending the code of 'A
Linear We use 'scikit-leam' python package (Pedregosa et al. 2011)
XGB We 'xgboost' python package (Chen Guestrin 2016)
BART We use 'BayesTree' package (Chipman al. 2010).
mBNN: We use official code at https:ttpt.. //t:::///tttttae com/ wtt.ccccmoooooe /g9p//gggggg//ggg 200ngg39/mmBB) /mNNN.
20

---

Published as a conference paper at ICLR 2026
For each model, perform 5-fold cross validation over the following hyper- parameter candidates to select the best configuration.
Hyperparameters.
Bayesian-TPae TPNN
We set the step size in Langevin proposal 0.01 and fadd = 0.28, Adelete 0.28 and
lchange 0.44 as in Kapelner & Bleich (016).
We fix Cadding = 0.95 and Tadding 2, as in Chipman et al. (2010)
Co {0.001, 0.005, 0.01)
a, E (1,
by (10-9,5- 10-3 10 10
Kmax E {100, 200, 300)
- a3 E (10-1,10-3, 3
.M E {1.5}
As in Chipman et al. (2010), for A, we reparameterize it as qA, where = =(o² < GöLS) and ØoLs denotes the residual variance from estimated Linear model. The can- didate values for ÿд are {0.90, 0.95, 0.99).
We MCMC iterations 1000 after 1000 burn-in iterations.
ANOVA-TPNN
We set the hyperparameter candidates to the same as those used in Park et al. 2025).
Kg E {10, 30, 50, 100}
Adam optimizer with learning rate 5e-3.
Batch size 4,096
Maximum order of component E {1,2)
Epoch E {500, 1000, 2000)
NAM
We set the architecture of the deep neural networks to three hidden layers with 64, 64, and units, following AAarwal et al. (202I).
Adam optimizer with learning rate 5e-3 and weight decay 7.483e-9.
Batch size 4,096
Maximum order of component E [1,2)
Epoch E (500, 1000, 2000)
BART
We set the hyperparmeter candidates similar to those in Chipman et al. (2010).
Number of trees T (50, 100,
0.95 and 3 =
E {1,3,5}
ę {0.90, 0.95. 0.99)
For o, 3/(kVT), E (1,,,3,5)
We MCMC iterations as 1000 after 1000 burn-in iterations.
XGB
consider the hyperparameter candidates used in Park et (2025).
mBNN
-W consider the hyperparameter candidates similarly to Kong et al. (2023).
Architecture 2 hidden layers with 500 and 500 units, 2 hidden layers with 1000 and 1000 units }
Sparsity hyperparameter À E (0.01, 0.1,0.5)
We set MCMC iterations 1000 after 1000 burn-in iterations.
21

---

Published as a conference paper at ICLR 2026
Computational environments. In this paper, all experiments conducted on machine equipped with an NVIDIA RTX 4000 GPU (24GB memory), an Intel(R) Xeon(R) Silver 4310 CPU @ 2.10GHz, and 128GB RAM.
B.4 EXPERIMENT DETAILS FOR IMAGE DATASET
CNN model. For CNN that predicts concepts, we attach a linear head for each concept on top of the pretrained ResNet18, and train both the ResNet- 18 and the linear heads jointly.
Concept generating. Following Oikarinen et al. 2023), we use GPT-5 OpenAA 2025) to generate concept dictionaries for CELEBA-HO and CATDOG dataset. Specifically, we prompted GPT-5 as follows:
HelebAMask--Q is large-scale face image dataset containing 30,000 high-resolution face images selected from CelebA, following CelebA-HQ. In this context, we aim to classify gender using the CelebA.Mask-HO dataset. Could you list five high-level binary concepts that you consider most important for gender classification?
When classifying images of cats and dogs, what are the five most important concepts to consider?
Through GPT-5, obtained concept dictionary
("Facial hair', 'Makeup', 'Long hair', 'Angular contour", 'Accessories')
for dataset CELEBA-HQ and another dictionary
('Pointed ear", 'Short snout', 'Almond eye', 'Slender/flexible body', 'Fine/uniforn fur')
for dataset CATDOG. Each concept e is divided into a positive part e. and negative part c... For example, concept 'Makeup' can be divided into 'Makeup" and 'No Makeup', and "Slender/fexibbe body' can be divided into "Slender/flxxibee body' and 'Bulky/varied body'. turn, we use the pre- trained CLIP encoder to convert C+ and c. as well as each image into embedding vectors. For each concept, each image is then assigned binary label by measuring which of the embeddings of c+ and c_ the image embedding is closer to.
Hyperparameters. For ANOVA-T'PNN and NA M are trained using the Adam optimizer with learning rate of le-3 and batch size of 512. For ANOVA-T*PNN, the numbers of basis Kss are all equal to and K determined using grid search on (10, 50, 100). For the neural network in NA we set hidden layer with sizes (64,64,32). We implement Linear model as the lincar logistic regression using the 'scikit-leam'' package (Pedregosa et al. 2011).
B.5 EXPERIMENT DETAILS FOR COMPONENT SELECTION
Table 8: Definitions of f(), (2) and (3)
Function
Equuaiinn
(()
22 sin 11 (0..5ma) log([ 1
iP
1+ (ita
27
#227
(()
21 42 2 + sin( sin(ws + 2x))
arccos(0.9210))
(x))
tanh( + agm4))
|ms exp(its + mx) + log((Zaz77z)) + 29z10 + 2200
Table 9: Distributions of input features for each synthetic function.
Function
Distribution
(())
N1, Na. No. Ny. No 3ld U,, N4: Xg. Xs, Xin Aid U(0.6.)) and XIiv.r, Xso aid UG-.I)
(x))
X Xso nnd U(-1.) Xy..... Xso nnd U(L.D)
(x))(x)
We generate synthetic datasets from the regression model defined as
Y = (lk) + c,,
22

---

Published as a conference paper at ICLR 2026
where o?) and R500 Here, f( A 1,2,3 are true prediction model used in Tsang jt al. (201)) and defined in Table and the imput variables are generated from the distributions Table 9 Imput variables indexed 10 are informative, they affect the output, whereas imput
variables 11 11-50 are non-informative. We choose o² such that the signal-to-noise ratio 5.
To evaluate the ability to detect signal components, conduct experiments in the same manner as
in Park et al. )2055). That is, we use AUROC based on the pairs of ((4) I|2,n and r(h) computed for all subsets S s [p] with |S| 1,2,3, where f5
((k) denotes the estimate of (kk) in f(() and
-
(16 11,, > 0) for k E 31.
23

---

Published as a conference paper at ICLR 2026
C ABLATION STUDIES
C.I THE NUMBER OF BASIS K FOR VARIOUS VALUES Co
To evaluate the effect Co in 9 on the number of bases K, we conduct experiments with the maximum number of bases K set to 200, and 1000 iterations for both burn-in and MCMC up- dates. Also, ay and by are set to be 0.5 and we use ABALONE dataset. Figure 3 shows that K decreases and RMSE increases C. increases. This result demonstrates that the hyperparameter Co effectively controls model complexity by regulating the number of bases K. A small value of Co recommended since an excessively large Co can be detrimental to predictive performance.
175
125
RMSE
3.35
165
1644
L0-3
1n3
Co
Figure 3: Plots of the number of basis K and RMSEs on various Co values.
C.2 IMPACT OF THE HYPERPARAMETERS a, AND by ON PREDICTION PERFORMANCE
We conduct an experiment evaluate the effect of shape parameter a, and scale parameter by on prediction performance. Except for a. and by, the other hyperparameters of Bayesian-TPNN TPNN are set identical to those in Section C.I of Appendix, and we analyze ABALONE dataset. We observe that prediction performance is relatively insensitive to the choice of the shape parameter axy whereas it somehow sensitive to the choice of the scale parameter by. Note that the scale of y controls the smoothness of each TPNN basis <x (x) and thus the smoothness of Moyesian-TPNI model.
Table 10: Prediction performance depends on various values of a,, and by.
Aax
0.5
2
3
le:5
3.247
3,202
3,278
3.228
lo-4
3.224
3.215
3.184
3.175
3.211
3.182
3.184
3.175
0.1
3.213
3,258
3,282
3.343
C.3 IMPACT OF THE STEP SIZE IN THE LANGEVIN PROPOSAL
We conduct an experiment to investigate the effect of the step size in the Langevin proposal for (bs,,k; I'..k k. 3x). Except for the step size, the other hyperparameters of Bayesian-TPNN TPNN are set identical to those in Section C. of of Appendix, and we analyze ABALONE dataset. Table TT presents the prediction performances of Bayesian-TPNI for various step sizes. Our results show that overly large step sizes in the Langevin proposal can degrade the prediction performance due to poor accep- tance and unstable exploration, whereas moderate range yields the best performance. Therefore, not too large step size is recommended in practice.
Table 11: Prediction performances of Bayesian- TPNN for various step sizes in the Langevin proposal.
Step size
0.01
0.02
0.04
0.08
0.1
0.2
0.3
0.4
0.5
RMSE
3.199
3.216
3,209
3.269
3.160
3.243
4.308
4.349
4,578
24

---

Published as a conference paper at ICLR 2026
C.4 IMPACT OF PINPUT ON ESTIMATING HIGHER- ORDER COMPONENTS
We conduct an experiment to evaluate the effects of using Pinput other than the uniform distribu- tion in the MH algorithm. We refer to the model with the uniform distribution for Pinpur as Uniform Bayesian-TPP TPNN, and the model where Plagut is determined using the feature importance from a pre- trained XGB as Payesian-TPNN. [2 compares prediction performances of Uniform Bayesian- TPNN (UBayesian-TP%)) TPNN) and Bayesian- TPNN on MADELON dataset. To investigate why the pre- diction performance improvement occurs when using the nonuniform Pinpul: we identify the most important components for each model whose results are presented in Table 13 UBayesian-TPMM only detects two thrid-order interactions as signals and ignores even all of the main effects, con- trast, Bayesian- TPNN captures the fourth-order component as the most important but is also able to capture other meaningful lower-order components including two main effects effectively.
We also analyze the synthetic datasets in Section 4.2 with UBayesian-TPNN, and the correspond- ing results are reported in Table [4] These results amply imply that Pioput plays an important role in detecting higher-order components and leading to substantial improvements in both prediction performance and component selection.
Table 12: Prediction performance on MADELON dataset.
Model
Siayesian-TPNN Bayesian- TPNN
AUROC t (SE)
0.739 (0.002)
0.854 (0.007)
Table 13: Top 5 components with the important scores normalized by the maximum.
Model
Rank
Rank 2
Rank 3
Rank 4
Rank
Comp.
Score
Comp.
Score
Comp.
Score
Comp.
Score
Comp.
Score
UBayesian-TPNN
(203 (208,289,421)
1,000
0,950 (129,443,494)
(30,
(148.176.298)
0.006 0.374
(75,232.442)
0.005
(64.373,379)
Bayesian-TPNN
(49,242.319,339)
1.000)
0,472
(379.443)
106
0.322
(242,443)
0.301
Table 14: Performance of component selection the synthetic datasets.
True model
f(2)
((3)
Order
UBayesian TPNN
Bayesian TPNN
UBayesian TPNN
Bayesian TPNN
L'Byyesian TPNN
Bayesian TFNN
1,000
1.000
0.826
0.831
0.834
1.000
(0.000)
(0.000)
(0.024)
(0.008)
(0.009)
(0.000)
0.985
0.750
(0.010)
(0.000)
(0.006)
(0.003)
(0.006)
(0.019)
0.661
(0.050)
(0.022)
(0.020)
(0.018)
(0.011)
0.022
C.5 IMPACT OF STEPWISE SEARCH IN THE PROPOSAL OF K
We conduct an experiment to evaluate the effectiveness of Stepwise move in the proposal distri- bution of K suggested in Section 3.2. We compare the performances of Bayesian-TPMy TPNN with and without Stepwise move on MADELON dataset. Table reports the averages and standard errors of AUROCs, ECEs, and NLLs over 5 trials and Table [6 shows the top 5 important components. The results suggest that the Stepwise move helpful to detect higher-order interactions which in turn leads to improvements in both prediction performance and uncertainty quantification.
25

---

Published as a conference paper at ICLR 2026
Table 15: Results of performance with and without Stepwise move.
With Stepwise move
Without Stepwise move
AUROC 1 (SE)
0.854 (0.007)
0.820 (0.002)
ECE (SF)
0.076 (0.004)
0.106 (0.007)
NLL
0,479 (0.009)
0.650 (0.005)
Table 16: Top 5 components with the important scores normalized by the maximum.
Comp.
Comp.
Comp.
Scure
Comp.
Comp.
With Stepwise move Stepwise move
(49.242_319.
1.000
(129,4334944)
0,472
(379,443)
0.374
106
0.322
(247,443)
0.301
(129_242)
(29.339,379)
(212,113)
D EXPERIMENT FOR THE POISSON REGRESSION
In this section, we compare the prediction performance and uncertainty quantification of Bayesian- TPNN with GBART (Linero) 2025) on the Poisson regression model. We consider the poisson re- gression model defined as
Y ~ "oisson(exp([(xx)
where is the regression function. We generate synthetic dataset of size 15,000 using the true regression function fo defined as
fo(x) = "122 2/ sin (0.5xa) + log([23 + øsl + +1 +
1 + |010| V + [xs)
Dg
37
T2
where input variable x, R are generated from Uniform(O.. (0, 1) for = 1,... 15, 000. Table 17 presents the RMSE and NLL for Bayesian-TPNN and GBART, demonstrating that Bayesian- TPNN achieves superior performance GBART even in the Poisson regression. Here, the RMSE calculated between exp(fo (x,)) and exp(f(x;) for i 1,.., 15, 000, where f is the Bayes estimate. Figure shows the seatter plot of predicted values exp(f(x;)) versus exp(Jo(x:)) (x:)) i = 1, .... 15, 000. It implies that Bayesian-TPNy TPNN yields predictions much closer to the true values compared to GBART.
Table 17: Prediction performance and uncertainty quantification on Poisson synthetic dataset.
Bayesian- TPNN
GBART
RMSE 4
0.094 1.615
0,141
NLL
1.629
Sayesian-TPNN
GBART
ex.)
(xx))
(xx()((x:))
exp(fo(x.))
Figure 4: Scatter Plots between the true expectations and estimated ones.
26

---

Published as a conference paper at ICLR 2026
E EXPERIMENTS FOR INTERPRETABILITY
E. INTERPRETABILITY ON THE IMAGE DATASETS
In this section, describe the local and global interpretations of CBM (Koh al. 2020 with Bayesian- Mhyesian-TPNNN on CELEBA-HO and CATDOG datasets. Table 18 presents the description of con- cepts used in CELEBA-HQ and CATDOG datasets.
Table 18: Description of image datasets.
Index
CELEBA-HQ
CATDoG
Facial hair
Pointed car
Makeup Long hair Angular contour Accessories
Short snout Almond eye
Slender/flexible body
Finc/uniform fur
Table 19: Normalized importance scores and ranks for the top 5 important components on the image datasets.
CELEBA-HO HQ
Rank
3
5
Component index Score
2
4
(2.3)
(2.4)
(1.5)
Buyesiam TPNN
1.000
0.665
0.592
0.304
0.262
ANOVA-T*PNN
Component index Score
(2.3)
(1,5)
5
1.000
0.482
0.284
0.262
0.211
Comporeent
Lineur
Scure
1.000
0.783
0.549
0.328
0.304
CATDOG
Rank
5
Bayesian-TPNN
Component index Score
(3.4)
(2,1.4,5)
1,000
0.395
0.252
0.162
0.086
ANNN
Component index
(4,5)
(3.5)
(1.4)
Score
1,000
0.883
0.882
0.716
0.453
Component index
4
Linear
Score
1,000
0.698
0.352
0.023
0.021
Global interpretation. Table 19 shows the top most important components along with their importance scores (normalized by the maximum score) for Bayesian-TPNN,, ANOVA-T"PNN and Linear model. In CATDOG dataset, Bayesian-TPPy TPNN identifies the 4th-order component (2,3,4,5) as an important component. It seems that complex interactions exists between the 5 concepts.
Example of CelebA-HQ
Example of CatDog
TH
do
Figure 5: Examples of images misclassified by Linear model.
27

---

Published as a conference paper at ICLR 2026
Local interpretation. Figure presents two images where Bayesian- TPNN correctly classifies but Linea model does not. For the CELEBA-HO example image, Linear model incorrectly predicts it as male, whereas the Seyesian-TPPN. correctly predicts female. The contributions of the important components for this image are presented in Table 20) In Linear model, 'Makeup' gives a positive contribution, which leads misclassification of the image male. In contrast, in Bayesian-TPNN, while the main effect of 'Makeup' still provides a positive contribution, the interactions between ('Makeup', 'Long hair') and ('Makeup', 'Angular contour') yield negative contributions, resulting in correct prediction as female.
For the CATDOG example image, Linear model incorrectly predicts it as 'dog'. whereas Bayesian- TPNN correctly predicts as 'cat'. Table [21] indicates that Linear model misclassifis the image as 'dog' due to the positive contribution
"Almond eye'. contrast, although Bayesian-TPNN also -aaecttions(AAlloooo eye", SSennerrfleeibbee body') and ("Short snout', 'Almond eye', 'Slender/flexible body', 'Fine/uniform fur')-provides much stronger negative contributions, leading to the correct classification as a cat.
assigns a positive contribution to 'Almond eye', the higher-order
These two examples strongly suggest that considering higher-order interactions between concepts necessary for the success of CBM.
Table 20: Prediction values the most important components for CELEBA-HQ image.
Component index Contribution
2
4
(2,3)
(2,4)
(1,5)
Bayesian-TPNN
0.297
0.184
-0.444
-0.323
-0.207
Component index Contribution
1 -0.222
4
Linear
3.746
-1,510
-2.665
1.627
Table 21: Prediction values of the 5 most important components for CATDOG image.
Component Contribution 0.618
3
(3,4)
2
4 -0.778
(2.3,4.5)
Bayesian-TPNN
-0.767
0.181
-0.355
Component Contribution
1
2
3
4 -2.463
5 -4.113
Linear
-4.304
-0.630
9.503
Table 22: Prediction performance the image datasets.
Bayesian-TPNN with concepts
Linear with 10 concopts
CRI .QQ CATDOG
0.936 (0.002)
0.899 (0.001)
(0.002)
0.869 (0.002)
Fewer concepts, better prediction performance. One may argue that 5 concepts are too small for Linear model and Linear model would perform well with more concepts. To see the validity of this argument, we compare predictive performance of Bayesian-TPN with 5 concept and Linear model with 10 concepts, where additional 5 concepts are generated through GPT-5: for CELEBA- HQ dataset,
'EEmphsszedd eyes'. 'Prominent lips', 'Smooth skin', PPronoucced cheekbones', 'High contrast')
and for CATDOG dataset,
('Long tail', 'Retractable claws (hidden)', 'Upright sitting crouching posture', 'Small mouth Meowing shape', 'Ambush-like pose (crouched)').
Table 22 presents the averages and standrad errors of AUROCS for BaYN with concepts and Linear model with 10 concepts. While using more concepts with Linear model improves pre- diction accuracy, Bayesian-TPNy TPNN is still superior to Linear model even though fewer concepts are used in Bayesian-TPPPl TPNN. This implies that capturing higher-order interactions plays critical role in improving prediction performance than merely increasing the number of concepts. Quality of concepts generated by GPT would be problematic.
28

---

Published as a conference paper at ICLR 2026
E.2 ADDITIONAL RESULTS OF LOCAL INTERPRETATION ON THE TABULAR DATASET
In this section, we describe the results of local interpretation on BOSTON dataset. Specifically, we examine the contributions of the most important components identified Bayesian-TPNN in Section 4.3 at specific input vector x. For given data point
x (0.006, 18, 2.31, 0, 0.538, 6.58, 65.2, 4.09, 1, 296, 15.3, 396.9,4.98)
the contributions of the 5 estimated components TPNN are given as
f() fss). and f(1,6) by Bayesian-
(f(13)()), fro) (x), (x))
= (0.575, -0.188, 0.080, - 00.002. 0.001).
In particular, the component f(13) makes a substantial positive contribution to the housing price. That is, the price of the house for given input vector is high because of the main effect T13-
29

---

Published as a conference paper at ICLR 2026
F EXPERIMENT FOR STABILITY OF COMPONENT ESTIMATION
Park et al. (025 demonstrated, both theoretically and empirically, that TPNN reliably estimates the components of the functional ANOVA model. In this section, we investigate whether Bayesian- TPNN exhibits the same stability in component estimation. For this analysis, we randomly split the dataset into training and test datasets. From this, we obtain estimators for the components. We repeat this procedure five times to obtain five estimators for each component. We then calculate the stability score using these estimators. Specifically, following Park al. (2025). for predefined data (x1,..., ..., ) we use the stability score defined as
E== (X) fs(x:))? C)=((((X)))
SC(fs)
where f'g is the estimated component for S obtained from the j-th fold and fs(x) = Σラ= (x) Finally, we use SC =
D (2) S(C to compare the stability in component
estimation between Bayesian- TPNN, ANOVA-TPNN and NAM.
Table 23 presents the results of stability scores SC (f) for Bayesian- TPNN, ANOVA-T-PNN and NA'M. where ANOVA-T'PNN and NA M estimate only the main effects. Table 24 shows of stabil- ity scores SC² Sf²(f) for Bayesian- TPNN, ANOVA-T
and NA M where ANOVA-T PNN and NA 2 M estimate up to second-order components. These results imply that Bayesian-
Bayesian-TPNN TPNN estimates the components more stably than ANOVA-TPAR -PPNN and NAM. Note that for MADELON dataset, which has an input dimension of 500, we could not train PNN and N due to the computa- tional environment, and thus their stability scores could not be calculated.
Table 23: Stability scores of Bayesian-TPMe -PNN,, ANOVA-T PNN and NA' M.
Dataset
Baycsian TPNN
ANOVA TNPNN
NA'M
ABALONE
0.087
0.405
0.555
BOSTON
0.368
0.425
0.583
MPG
0.222
0.411
0.472
0.481
FICO
0.130
0.287
0.607
BREAST
0.100
0.286
0.569
CHURN
0.111
0.558
0.569
MADELON
0.685
0.785
Table 24: Stability scores of TPNN,, AN and NA
Dataset
Bayesian TPNN
ANOVA T-NN
NAPM
ABALONE
0,400
0,340
0,770
BOSTON
0.615
0.380
0.705
MiG
0,340
0.370
0.560
SERVO
0.445
0.575
0.665
0.525
0.540
0.790
BREAST
0.630
0.675
0.730
CHURN
0,520
0.755
0.730
MADELON
0.475
30

---

Published as a conference paper at ICLR 2026
G COMPARISON OF CONVERGENCE SPEED AND RUNTIME IN MCMC ALGORITHM
In this section, we evaluate the convergence speed and runtime of MCMC algorithms for Bayesian- TPNN. Specifically, we compare its convergence speed with that of mBNN, and its runtime with those of ANOVA-TTPNI and mBNN. Bayesian-TPNN set max 100. For mBNN,, use two hidden layers with 500 units each and set the number of HMC steps to 30. For ANOVA-T'PNN, we set Kg 10.
Figure shows the RMSE trajectories across MCMC iterations on BOSTON dataset for Bayesian- TPNN and mBNN. Table 25 presents the runtime comparison of Bayesian- TPNN mBNN with 2,000 iterations and -NOA--TPNN with 2,000 epochs on real datasets. The best results are high- lighted by bold. In the experiments on FiCO, CHURN, and BREAST datasets, the runtime difference between Bayesian-TPNN TPNN and ANOVA-T-PNI become more pronounced. This is because, after data preprocessing, the input dimensions are 23, 46, and 30, respectively. As the number of neural networks required in ANOVA-T'PNI increases rapidly with the input dimension, the runtime in- creases considerably. Note that for the MADELON dataset, where the input dimension 500, training ANOVA-T9PNN is infeasible because the number of neural networks to be trained is 125, 250.
These results imply that Bayesian- TPNN converges faster in terms MCMC iterations compared mBNN. Moreover, its overall runtime is shorter than both mBNN and ANOVA-T<PNN. In par- ticular, Bayesian- TPNN runs significantly faster than ANOVA-T and this advantage becomes more pronounced as the input dimension p increases.
Taye
Table 25; Runtime Bayesian- TPNNN ANOVA-T'PNN and mBNN (sec).
Dataset
Bayesian TPNN
PNNN
mBNN
BOSTON
MPG
156
227
275
SERVO 139
400
242
FiCo
943
3,530
4.198
BREAST
2,363
CHURN
7,772
2,756
MADELON
345
894
RMSE
4
1750
2000
MCMC Iteration
Bayesian-TPNN
mBNN
Figure 6: The RMSE trajectories across MCMC iterations for BayN TPNN and mBNN.
31

---

Published as a conference paper at ICLR 2026
H ADDITIONAL EXPERIMENTS FOR UNCERTAINTY QUANTIFICATION
H.1 UNCERTAINTY QUANTIFICATION ON NON-BAYESIAL MODELS.
We report the performance of uncertainty quantification for non-Bayesian models including ANOVA-TPNN, NAM, XGB and Linear model, in Table 26 These results indicate that Bayesian- TPNN outperforms the non-bayesian models in view of uncertainty quantification.
Table 26: Uncertainty quantifications for non-bayesian models on real datasets.
Datnet
ANOVA-TPNN
NAM
XGB
Linesr
CRPS
CRPS
NLL
NET
ABALONE
1,578 (0.16)
1,901 (0.27)
1.668(0.16) (0.16)
1.638 (0.15)
BOSTON
4,464 (0.71)
(0.35)
3.241 (0.27)
4.291 (0:44)
MPG
2,000
SERVO
0.595 (0.02)
(0.39)
(0.05
(0.04)
ECB
NLL
BCE
NLJ
BCE
NLL
ECE
NLL
FiCO
0.063 (0.01))
0.583 (0.018)
0.198 (0.007)
0.681 (0.012)
0.096 (0.026)
0.620 (0.01.5
(0.014)
(.593(0.017)
BREAST
0.100 (0,000)
0.425 (0.071)
0,284 (0.002)
0.531 (0.033)
0.063 (0.012)
0.87 (1.172)
0.102 10.015)
0.216 (0.0399
CHURN
0,053 (0.004)
0.444 (0.011)
0,318 (0.007)
0.718 (00008)
0.131 (0.005)
0.594 (0.021)
0,078 (0.004)
0,573 (0.002)
MADELON
(0.0L))
0.752 (0.003)
0,156
0.735 (0.016)
0,147 (0,008)
0,703 (0.005)
0.232 (0.011)
0.736(00.116)
H.2 EXPERIMENT FOR OUT-OF-DISTRIBUTIOO DETECTION
Here, conduct experiments to evaluate whether each model appropriately captures uncertainty out-o--disttibution data in binary classification. As a measure of uncertainty for vutto--distribution data, the maximum predicted probability (Mukhoti et al. 2023). Specifically, in-distribution dataset by {x( :-.., xN, } and the out-of-distribution dataset by {x] ...., x..,,x,, N₂ corresponding predictive probabilities p(x:") for i 1,..., N1 and p(x;") for = I,.,, No.
denote the out with
Let Pmmax(x (x) = max (f(x), 1 p. For evaluation, we assign label the in-distribution data and label ( to the out-of-distribution data. Then, we compute the AUROC between the labels and the transformed scores 1 log, (x""x or 1 1+ log2 P Intuitively, predictive probabilities close to 0.5 reflect model uncertainty, and such cases be identified out-of-distribution.
We randomly sample a subset which size 500 from the MADELON dataset, standardized it, and use it
dataset. For each dataset FICO, BREAST, and CHURN, randomly split the data into training and test datasets. In turn, we train Bayesian- Bayesian-TPNN TPNN and baseline models using the training dataset. We then compute the AUROC treating the test dataset the in-distribution dataset. We repeat this procedure times, and Table 27 presents the averages and standard errors of AUROCs for Bayesian- TPNN and baseline models FICO, BREAST and CHURN datasets. These results demonstrate that Bayesian-TPMM TPNN outperforms the baseline models, achieving substantially superior performance in -uistt-diitributtin detection.
Table 27: AUROC Results on in-distribution and out-of-distribbttion detection.
Datasct
Bayesian-TPNN TPNN
ANOVA-TPNN
NAM
Linear
XGB
BART
mBNN
Fico
0.606 (0.013)
0.446 (0.020)
0.455 (0.032)
0.191 (0.002)
0.605 (0.018)
0.667 (0.004)
0.519 (0.014)
BREAST
0.903 (0.015)
0,542 (0.021)
0.534 (0.041)
0.112 (0.010)
0.827 (0.022)
0.664 (0.023)
0.503 (0.051)
CHURN
0.724
(0.040)
(0.0R0)
(0.006)
(0.014)
(0.009)
(0.039)
32

---

Published as a conference paper at ICLR 2026
1 VISUAL ILLUSTRATION FOR PROPOSAL
In this section, we describe the visual explanation of the proposal in Section 3.2 TPNN as in Figure 7
Given Bayesian-
we explain the updating of K and the updating of Sk.
Input
TPNNs
6
2
B3
4009
Output
211
Ra
Figure 7: Bayesian-TPNN with p K,
I.1 UPDATING
For given K, we propose Keew K Stepwise moves, corresponding to the case where K = K + 1.
Knew = K Here, describe only Random and
In the of Random move, node randomly generated and its edges randomly assigned. For Stepwise move, node is first selected from the existing nodes, and then a new edge is added to create new node. Figure [ presents overall illustration for these moves.
Ingut
TPAN
Fou
TPNNS
TPA
Figure 8: Visual explanation for alternations in the proposal distribution of K.
1.2 UPDATING S₁
Figure illlstrraees how the edges change when applying Adding. Deleting, Changing moves to a given current state.
33

---

Published as a conference paper at ICLR 2026
Gurrent State
Adding
Input
TPNNS
Input
TPNNS
R1
aupa
24
Deleting
Changing
Input
TPNNS
Input
TPNNS
#L
F1
804Ph
24
Figure 9: Visual explanation for alternations in the proposal distribution of Su.
J EMPIRICAL EVALUATION UNDER MINIBATCH SETTINGS
We conduct additional experiment to empirically verify that our MCMC algorithm performs well when mini-batches are used. When estimating Bayesian-TPal with mini-batched data, we refer it as MBayesian-TPNNN. Here, for ABALONE. and FICO datasets, set the size of mini-batch as 1,000 and 2,000, respectively. Table 28 presents the averages and standard errors of prediction performance and the uncertainty quantifications of Bayesian-TPND and MBayesian-PPNN for trials on ABALONE and FICO datasets. These results suggest that training with mini-batches does not significantly reduce prediction performance and uncertainty quantification. In practice, these findings indicate that using mini-batches is practically acceptable, as it does not lead to meaningful degradation performance or uncertainty estimation.
Table 28: Results of MBayesian-TPNN
RMSE/AUROC
CRPS/ECE
NL.L
Bayesiam- TPPNN
MBayesian TPNN
Bayesian TPNN
MBayesian TPNN
Bayesian-TPNN TPNN
MBayesian TPNN
ABALONE
2.053 (0.26)
2.081 (0.24)
1.372
1.391
2.260 (0.16)
2.280 (0.18)
(0.009)
(0.005)
(0.004)
0.038 (0.003)
(0.007)
(0.003)
34

---

In this section, we conduct additional experiment to compare Bayesian-TPNN with Deep Ensemble "Lakshminarayanan al. 2017). Here, we consider candidates for each hyperparmeter of Deep Ensemble as below. Published as a conference paper at ICLR 2026
K COMPARISON WITH DEEP ENSEMBLE
The number of MLPs: : {5, 50, 100)
MLP architectures: {(50), (100), (256, 128, 64), (512, 256, 128))
Learning rates {le 4, le 3, le -
Epochs: {100, 200, 500, 1000}
Weight for Le regularization: : {le - 3, 1e - le 1}
Table presents the averages of RMSE, AUROC, CRPS, ECE and NLLs for 5 trials on real
datasets. These results show that the performance of Bayesian-TPNN is comparable that of Deep Ensemble in terms both prediction accuracy and uncertainty quantification.
Table 29; Results of Bayesian- -PNN and Deep Ensemble.
RMSE/AUROC
CRPS/ECE
NLL
Bayesian-TPNN
Deep Ensemble
Bayesian-TPWy TPNN
Deep Ensemble
Bayesian-TPNV
Deep Ensemble
ABALONE
2.053 (0.26)
2.121 (0.23)
1.372 (0.19)
1,498 (0.17)
2,260 (0.16)
2.036 (0.15)
BOSTON
3.654 (0.49)
3.922 (0.57)
2,202 (0.23)
2.458 (0.22)
3.411 (0.37)
3.747 (0.40)
2.386 (0.41)
2.257 (0.14)
1,510 (0.43)
1.481 (0.11)
(0.21)
2.769 (0.47)
SERVO
0.351 (0.02)
0.398 (0.03)
0.194 (0.01)
0.179 (0.01)
0.836 (0.10)
0.701 (0.04)
FICO
0.793 (0.009)
0.773 (0.024)
0,036 (0.004)
0.057 (0.033)
0.554 (0.007)
0.577 (0.034)
BREAST
0.998 (0.001)
0.993 (0.003)
0.129 (0.009)
0.075 (0.017)
0.211 (0.014)
0.133 (0.041)
CHURN
0.849 (0.008)
0.841 (0.013)
0.031 (0.001)
0.039 (0.002)
0,418 (0.008)
0.424 (0.018)
MADELON
0.854 (0.013)
0.616 (0.029)
0.076 (0.004)
0.137 (0.061)
0.478 (0.009)
0.719 (0.049)
35

---

Published as a conference paper at ICLR 2026
L APPLICATIONS TO GENOMIC DATASET
We conduct additional experiment to explore the applicability of Bayesian-TPNN to genomics dataset GSE43358 (Fumagalli et al., 2013). GSE43358 is gene expression dataset with n 57 samples and 54, 675 features and we perform classification task distinguishing between HER2-positive and non non- non-HER2- HER2- possiive cases. Table shows that the averages and standard errors of prediction performance for Bayesian-TPay
Linear model and XGB for 5 trials. For Bayesian- TPNN and XGB, the hyperparameters are optimized as in the experiment for other real datasets. Note that because the input dimension p is too large, both ANOVA-TPNN and NAM could not be trained within our computational environment. The results in Table Ta indicaate that the interpretable TPNN achieves prediction performance comparable to that of the black-box model XGB on GSE43358 dataset.
Table 31 reports the top 10 most important components in Bayesian-TPNN with the normalized importance score. Here, we use the importance score defined in Section 4.2, and the normalized score represents each importance value divided by the maximum importance score. Note that one of the third order interactions is detected by Bayesian-TPNN. The results in Table 31 indicate that higher-order interactions (beyond the second order) play crucial role, which provides a plausible explanation for the inferior prediction performance the linear model. Moreover, this highlights the necessity of an interpretable model such as Bayesian-TPNy which is capable of estimating such higher-order interactions.
Table 30: Results baseline models on GSE43358 dataset.
Model
Rayesian-TPMn TPNN
ANOVA-TPNN
NAM
Linear
XGB
AUROC
0.949 (0.017)
0.545 (0.001)
0.953 (0.041)
Table 31: Top 10 important components.
Rank
Componenr of GenBank accession numbers
Normalized Score
BF357738
0.924
(BC000129, R80390)
0.701
AF3073388
0.569
NM_018297
0.410
AF319440
0.365
(RF741754,
AB037854, AK024890)
A1368358
0.292
BE672684
0.218
36

---

Published as a conference paper at ICLR 2026
M NOTATIONS AND REGULARITY CONDITIONS FOR THE PROOFS
M.1 ADDITIONAL. NOTATIONS
For two positive sequences {an} and {ba], we write an X bn if there exists a constant C' > such that On I Cb, for all n E N. The notation On = o(b,.) indicates that the ratio /n converges zero as n - oc. We denote F,, d) the c-covering number of the function class JF with respect to the semimetric d. For given vector v (v), UN), we define its &2 norm | Ei= up. Given real-valued function f X' - R, we define its sup-norm as ll == x f(x)]. We define population f--norm with respect to probability measure on X' as |f = (xxx ((xx)) /r. Let Px = I[ where Fx, is the probability distribu- tion of X; for ,,.., n. For two given densities pi and p2, we define the Kullback-Leibie (KL.)
divergence
k()..P2) == / log(nn ( (vg((
and let (p1. == [Voo(p((())/pp(v PT
M.2 REGULARITY CONDITIONS
(S.1) For distribution 1xx, there exist density Px with respect to the Lebesgue measure IR?, that is bounded away from zero and infinity, i.e.,
inf px(x) S sup px (x) < oc. xEX
(S.2) The true function fo.s is L-Lipschitz continuous, i...
[fo,s(x) - fu,s(x') S L||x - 1|
for some positive constant T. and all x, x' E X. Additionally, fo.s is assumed to be bounded in the supremum norm by positive constant F, i.e., Il llo I F. We denote the above conditions compactly as fo,s E Lip Moreover, we say that fo E LiPu,L, if fo,s E Lip,,r for all S C [p].
(S.3)
The log-partition function A(-) is differentiable with bounded second derivative F, F.. i... there exists positive constant CA such that
1/Ca S A(x) CA
for all æ E F].
(S.4) Kmax assumed to grow at a rate Kmax = O(n).
37

---

Published as a conference paper at ICLR 2026
N POSTERIOR CONSISTENCY OF BAYESIAN-TPNN
We first prove the posterior consistency of f since it plays an important role in the proof of the posterior consistency of each component fs-
N.1 POSTERIOR CONSISTENCY OF fo
Theorem N.1 (Posterior Consistency of Soyesian-TPNNN. We assumes that [..1)
S.2)
S.3
and
(S.4)
Then, for any >> 0 and & 2 + +EV Cx
2. it holds that
(f llfo fll2, = r(()) - 0
TTg
(15)
in Q% + oo, where Q is the probability distribution of (X(n), (())).
N.2
PROOF OUTLINE
Consider function class J Kmae F(K) that satisfies the sum-to-zero condition with respect to K=1 uniform distribution (0,1). Here, F(K) is defined
F(K) f f(xx) Σ
(x bsa.kr ss,,));
Bk E TR,
1s E [0, 1] 1811
Ts E (0) for h: = 1, 1,... =_ Kj.
where
- bji
2 -bjbk
(x) [xy- bi.k
o(x(Sk, Is
II JES,
1 1-
+ c)(bs, 13.100
Yssk
7
and
1 So
dx
So a
b
dor
Ya.k
For any f E F(K), we denote as fK.B,b.r where
B = (3x, k E [K]), b = (bsy, & E [K]) and
1 =T's A [K]).
Our goal to show that
lim E [[[m - foll2,n > (1) (x)()))))))) +00
(16)
for any >> 0.
We prove 16 using following two steps.
(P.1) For given data ("),, we prove that
lim [ - folla,n > (mx y( = -x(0 = 0
n
for any E 0. (P:2) Finally, we show that
lim Ey [I] -oollnn > e|X(1), (x()) = 0 200
for any 0.
38

---

Published as a conference paper at ICLR 2026
We first verify the following three conditions: there exists FF C F and positive constants 8, C1 C2 such that
(6, || (1x0) < nCI,
(17)
|EF: 11 fullo S SEV C*
exp(-nC22)
(18)
* T(F(F")) < exp(- + 2)m).
(19)
After that, we will show that these three conditions imply the posterior consistency in Step P.1 by checking the conditions in Ghosal et al. 1999).
N.3 VERIFYING CONDITION 17
Mr We consider a sieve F = K=I Fr (K), where
K
F² f :f(xx) Σ
Sx bs ss)
3x [-n,
bsy E [0, 10.1]1sa]
T E [1/n,n]iSt for k == 1, k).
where M,, =
Cans Tog
and C3 will be determined later.
Also, we consider a more general function class as
K (²(K) = {1 f(x) Σ
Sx bsu Du Csa,k);
1=
Be E [-n, n)]
bsc. k E [0, 1 |Su]
(20)
P E [1/n, [Sel,
Ca [-2n, 2n,,nn||S
for k= 1,.., J
where the function is defined
IT bjsk
[2j (2y-bag -jik bj.k
))
<(x|Sk, bs L's k)
II
1 1-
+ C2.20
Yssk
jESu
and the vector C is defined as CSt.k = (cj,es E St). For all k, we have
by.t
dx I'
(( r bi.b
dx
Ki.k
bi.
2 Co.j.b
where Cajjk k is positive constant and thus, we have les (bi,ks 7g,k)1 S Co. Vj. h: for some positive constant C.. Hence, for all K E [.
F'K) (K),
(21)
whenever sufficiently large. Therefore, suffices to verify Condition 17 over
Ma
g"
U gx))..
(22)
K=
39

---

Published as a conference paper at ICLR 2026
Lemma N.2. For any integer K, we have
g(e, - - (1+ K2pt k(1+3pp)
Proof.)
First, since the maximum dimension of parameters in (") is ((1 + 3p). we consider K(1 + 3p)- dimensional hypercube [-2n, 2n) (1+3p)) Then, we have
K(1+3p)
W(e1 1-2n, 2n] k()+
-11-111
[N(e1: 2n, 2n], -11111
K(1+3p)
4n
S 1
61
For SK -Sk, k [K]), we define 6 (Bk, b K. KK CSx, K.) where
BR (B1, ...y. Bk),
bsk K (bs k E [K]),
ISK K (I Sh,Ki [K]),
CSK.K = (cSy, h E [K]).
Let (6 ...., (ee.lnnn K0435) (I [ll))) be an é1-cover of [-2m, 2m] k(L+p)), and for given G
[-2m, (L)3()+pp) let S be an element in the co-cover such that ||G +11 S 61-
Note that for any fe E g" (K), we have
fx(x)
Σ Bx II
o( Yo cj,x),
1=
JES&
where
2j
bjik
IT Ig- -i.k
o(z,liih bj YEA Cj,N) =1 1
+ C240
7ish
7i.k
with lcaal < 2n. Then, for any fe E (K),, we have
fe(x (x)
sup
< sup
3x ((a,,li),bb,. (,.
Br II jESk
By By 23,6)
jESx
sup
Bx II
( Millijbbba.. 73A:
3r II
pi ([), bynk To. (..
JESk
j JESt
(23)
A II
by Ta.k (3.k - 3 II
JESL
jESt
Upper bound of first term in 23
Since
I bij )
2 - bi.b
II jES
p(0,ki)), bj.. 7i.k c.,k
1
+ Cj.A0
Yisk
JESk
It bjR
Itj 35-b9, bi.M
II JES.
o
CjRRo
7j.k
Ti.k
II JES.
(1 + 2n)
(1 2n)".
40

---

Published as a conference paper at ICLR 2026
have
3k II ((r)(()) (3,,, 3x II
sup
JESk
JeSx
sup
(1 2n)
18x 2
18k
3el
x k=1
(1 2m P1
Upper bound of second term in (23 Using direct calculation and triangle incquality, we have
3
II JESk
((2z
bi,kb kg (g,))
0 ,5.ks B3.ss 224)
by.k
bjub
Bg a bzik
2j b3.k
II
0
+ ciuo
=2,00
Viuh
Yi
Yssk
JES&
bi
bj.k
- bg.b
bj.
|
II
+ CjiRi
Pssk
Yiik
Ravk
JES
Iij
bj..
I.
63.b0
)
Bij By--bsh -bg.6
Itj -bsik
)) 7i.e
o
E2,00
73.6
Ki.k
Ys.k
JESk
Since (-)) is Lipschitz function, have
byik
2 bjuk
0
Ti.k
Yj.k
I
byik
23
bj.t
7i.k
3
by.
Itj by.k
+
Pj.k
7g.6
7j.k
2n²
[bj,2 - -bj,el + 15s.
Similarly, we have
x3 -b,
- 25,000
7j.b
Pjjk
Ij z byik ) 2zikor
B3 - bzikk
23 (33- 63.6 (z)-bsse)
I. bi.k
< Cy.o0
+ ōy
25,60
Yi.k
Vs.k
7s.N
73.60
I 4n
|cg.k - 2j,bol + 103.
byikl + 183.00 - 73.01)
To sum up. the upper bound of 23p is
sup (x) fx(x) $ K (1 2n) + 12 of
< ((2n)33p 3p-1 Ee.
Let K(2n) 3p+1_
3p+) e1. Then, we conclude that
(K),1 -1.))) (1
2K 2K (KK)2333+ k(()3))
41

---

Published as a conference paper at ICLR 2026
Using Lemma N.2 we have
M
2K (2n) 33p+2 (2n)3p++2
k(1+3p)
W68,F F" .1. 1100) 5 Σ
1
K=1
2M,, 2M, (2n)8p++
(2+++pp)
S Mn
+
Let e/8. Finally, we choose C3 such that
2M, (2m) 3p+2
log,N(o, (loo) S log Mn + Mn (1 + 3p) log 1+
<n=²/10
Condition 17 is satisfied by letting C1 e2/10.
N.4 VERIFYING CONDITION 18
For S S [p]. using Theorem 3.3 in Park et al. 2025), there exist TPNNs such that
Cs
fo.s Ba. by .
(24)
1/IS
K
for some positive constant Cg. Here, 85 are uniformly bounded, i.c., 18g < cs for some
positive constant and f/..k 1// for all j, k as specified in Theorem 3.3 of Park et al. (2055)
Let kn.s such that
Cs
SEV2/(VCA 3-22).
(25)
1/IS
Let kn Esc( Kn,s and fk., Bang bu. Pr. Esc(r) fk.. Bs.... s by,,, n..9 s... simplicity, write BR.,. bu. and l'k,, simply B, b and I', respectively, Since
For notational
115o - /.n.Bl..
I Ilfo B,b + 1I5... f.n.8,6,fllo +115 1l.. Sh..
(26)
we have
x (fEF F: 1f- folloc s /
3 Ca Ca
2 T(K- kn))
II r(S= = 5')
(27)
S'C(o]
x π (f1Ninnthh fr St. I E E 2
n
-
8/2 S 31 CA
3/ CA
(28)
Therefore, it remains derive the lower bounds for
27 and 28
Lower bound of (27).
We have
T(K = km)
r(S = 5n) II
7(S s')
exp(-Coll log n)
Σ
Kmas exp(-Cyklogn) K=D
S'CIpl
S'Clpl
>
for positive constant dj.
42

---

Published as a conference paper at ICLR 2026
Lower bound of (28).
For any B (Bk, k [kx]) E Rt, we have
suu B,, 1,6,,,
Σ
3x II م Tox) - Sx II d(zylii) by )3.u)) JES,
x
k=1
JESA
(3k Bx) II MazyHihbbe, 13.))
sup
x
k=1
JES
(Bx Co)P
(29)
(1+ PII B||
$ (1+C kn 1B Bll2.
That is, we have
II.h. f...B.6.lll S 3V CA
e/
6
2
2
BB | BB2 (
-1
2
3 31 CA
Furthermore, direct calculation yields
R
sup f B,6.f Sh..BB,b,rllo
Ball II (o(,,li),bb, ",,,), 93.x)
x
jESc
$ (1 + Co sup
II JES
Ij
63.k
Ij bj.k
Ys.k
Rm
bikk
by.
(1 + Ca)sup sup
+ {srg - b).x))
Va.
JESA
bj.k
+ Yi.k - -5.,
$ (1 Co) sup
jES.
1/1Skl
Let Cn.j.k = 18s,8l
2 3E(1+C.) kkn, V
If 1igk - 93ac I
we have
cs
2/18a1
Yi.k 7s.k
61
S
Pj.k
Ri (fi.k (1)
4
35hn
CA
where 6
C 24C..
Therefore, if
|B01 < E,
1bj. by,1 < 20nnj.k
Cn.jk, Figh
178.4 3a.0 <
+ Cn.jk
hold, we have
8 2 1s,,, ..... - I.n,B,b,rlls. S
$ C/
(30)
43

---

Published as a conference paper at ICLR 2026
That is, we have
8
2
|I 1,6. J.n.B.brll S _ 3 C/CA CA
2
1 K... J,. S
3 CA
|13 Bl12 (11+ -1
2
3V
C^'
13,1 s. 19 - baul s 2Cnjiks
C...Kkly.k
his - 93,0 5 + Cn.j.k
Vj E Sk, vk [km]]
It implies that
11 B.B.B....F 8,,.8,, s3V n
2/22//1 { 1 1,6,8,66F S.,.B.brrll S
2
8/2 2
3V CA
3 CA
2 n (1 Bl12 S 5 ((1+ (e12
15 Ss. Ak (km)))
(31)
x x (lbj. bj S 20n.ju, Vj E Sk, wk E (ki))
(32)
Cn.j.k _n..A.-Bass Vj Sk. Ak E (km)))
x 1ijj 7z I I+ Cnjjk 1ig,k,,
(33)
Now, we will show that these three probabilities sufficiently large.
Lower bound of 31 Since
BB Bl|2 s ((1 (,.+ SE, vk [km]]
2 1Bx 3el s ((1 (c..:k..).ee//33//A.a). S SE, vk [kin]
18x Bul s (1+CC) eV22) [kml
(34)
for sufficiently large n, it suffices to get the lower bound of T(33 - Bal S ((1 + - (BV2(( for E [k,].
For k [kn], we let
I = [3 + ((1 +
and we have
T (13x - Bxl S ((1 C.) -1 (EV2/(3//A))
| 11, V2TT0
1
B2
exp
20 d33
(maxs cs + ((1 + Ck) -1E2//3/CV)
[/4]
exp
(35)
/2TTg
20
(xpp (ddnn)
for some positive constant D,, where 335 is derived from 13x1 S maxs cs.
Lower bound of 32
Since
T (bbb bymal S 2Cn..k) = 40nd.
for all j E Sk, 1 E [kr,], we have
x (bbjA dr,8 S 2OngAs Vj E Sk, Ak [kn])
II
4Cnj.
kElkuljjSs
>
for positive constant D2-
44

---

Published as a conference paper at ICLR 2026
Lower bound of (33 Using direct calculation, we have
T ((1SAA Fgaal -
Cnijk A3.8k
2.
min xelLn,Unl
(x)
2 + Cn,j.
2 + Cuiih
20 20m.g.h
by
min xa0 (xsp(-b93))
Crijch
T(aa) Dr (b(uUl)))
Cn.aafs and Un = 9g.u + 2+Cniik where I., fg,h 24Cn...k
Note that 1/k 2 9r.5 S 1. For a, > 1, we have
min #
x aa 2 L.a
Un]
24
d,
2 + Cmi.k
> exp(-033)
for some positive constant D3 and for dy 1, we have
min x- 2 Un,-1 26 =E(Ln,Un]
l-a,
Kj
> exp(-0dx) for positive constant 4- Furthermore, have
min Ln.Unl
exp(-b,,r) 2 (op(-byUn)
exp( -2b,5r.s)
exp( (20mm)
and
2C n.n.jkkfi. > exp(-20mm)
2 + Cn.j.k
for some positive constants 0g and dy. Finally, the proof is completed by letting C2 -7 d,.
N.5 VERIFYING CONDITION
19
We will verify Condition
19 with the constant C3.
We let
Zi = ( K Mn),
Zg = {(K s M,) n (3k E [K] such that |8x1 > nt}
Za {KK Mn) n (3k [ such that T S (n, 00)
Since
22)
the upper bound of m") is
T(F(F))
$ 7 Mr)
(36)
+ T (K < Mn (Ek e [K] such that |6x| > >al>><K>> & Ma)
(37)
(K < (ak [K] such that S
Mn).
(38)
45

---

Published as a conference paper at ICLR 2026
Upper bound of (36). For M, L
Cyns" log
we have
Σ Kmax
M= exp(-kCo logn)
T(K> > Mn)
Σ Rmax
k=n exp(-kCo lo
S exp( Memp--MnChr Cn logn)
Since C3
Cu+2 Co logn
for sufficiently large we have
r(K > Mxp(ex((C2 + 2)n) as n - oc.
Upper bound of (37 We have
π k E [K] such that [3,] > n| K & Mn) S MnT(1311 > >n)
2M, exp (-
n²
20
where og is constant. That is, we conclude that
π (3k
[K] such that [8x1 n|K s M₁) exp((C2 2)m) - as
Upper bound of 36 For any j. k using Markov inequality, we have
by
by
Thyg > 12) < E
exp
exp
2
2
by
exp
2
Since
π (3k [K] such that I S (n, 10) K S Mn) S MaT(mn, > n),
we have
*(3k E [K] such that s E (n, lo) < Mn) exp((C2 2)m) - 0 as n + 00,
where ay and by, positive constants.
N.6 VERIFICATION OF THE CONDITIONS IN GHOSAL ET AL. (199
For given data x (n), let Tf be the probability density of (xx) for = I,..., n. From Theorem of Ghosal et al. (1999), it suffices to verify that for every fo E Lipo, there exists a sieve Fu,
constants 8 4, Ca Ca > and C < 62/8 such that the following three conditions hold with respect to the || 1|2.
log (ogN(o,FF)) 112,m) nnI.
(39)
K(ananggga) Es?) > exp(-nCot,
f E Fe:
(40)
me (Fe exp(-nCo).
(41)
To complete the proof of Theorem N.I for given data x (").
we will verify that the three conditions
33
and 41
Verifying Condition (39
Condition 33 holds under Condition 17
46

---

Published as a conference paper at ICLR 2026
Verifying Condition (40).
By using direct calculation, for i L, L,..., have
K ((55..i = / (Uo(x)) (o6(x)--((xx) [(xx)) (Vo(())) - A A([(x:)) (so.:
(42)
(16(x)) - [(xx)) A(J6(x))) + +A(((x)))
(43)
(F6(x)) - f(x))( -((x)) + A(f(x))))
(44)
Using Talyor expansion, have
( 9F.) = 5 ((0)(( - f(x:)))
where 7 E [-F, FJ. That is, we have
CΛ fo - -1112n
Σ K(a (13) - 2
When & 2 2PF + V E. we have
-$ Te F - foll.n sev
2
2 FEFE: 11 - -follz,n SEVC, Ca
CA
Therefore, the proof is done by Condition 18
Verifying Condition 41
Since
π ((F(Fn)
Te (F(FF
(lIf l 5)
T(F(FA)
2 [l - fulloo SEv
Ca
S Sexp( exp(-((Ss 2)n)
for 2 + + cA I &, the condition (41) holds for Cs C's + by condition (8) and (19
N.7
STEP P.2
Since (P.1) holds for arbitary ("),, conclude that
lim EB ([T (Ilf Solle. > )1) y()))]
m=0000
for any 8 > 0.
47

---

Published as a conference paper at ICLR 2026
O PROOF OF THEOREM 3.2
The proof consists of the following 4 steps.
(STEP E.1)
We first establish the rate at which the posterior concentrates under the population & norm; specifi- cally, we demonstrate that
ME
F
1f foll2.7x > =(x(m)
y(())) x + O,
(45)
for any >> O.
(STEP E.2)
Based on (45) we establish that the following holds for any subset S S [p].
EB m (refJE FF IIfs fo.sll2.7x > e(x(m), y(m)) }] - 0,
(46)
for any > 0.
(STEP E.3)
We reformulate (46) in terms of the empirical E norm. Specifically, we demonstrate that
TT
f E F
Il.ss fo.sll2. =|x("), y(m))] (())) + 0,
(47)
for any > 0.
(STEP E.4) The last step verify
F3 (FF v(m))]
(48)
OO.
O.1 VERIFYING (STEP D.1)
To verify (STEP D.1), rely on the following lemma, whose proof is provided in Theorem 19.3 of Györfi et al. (2006).
Lemma O.1 (Theorem 19.3 of Györn al. (2066)) ). Let X. X.XX, XI,..., X, be independent and iden-
tically distributed random vectors with values in Rd Let K1, K2 2 1 be constants and let G be class of functions g Rd R with the properties
1g(x)) S K1, B[g(X)²) K2
(49)
Let 1 < and <> > O. Assume that
FunV1 - - 288 max 2K1, v 2Kg
(50)
and that, for all X1..... Xn Rdd and for all 12
VTx(11 k)t
log. u. g gg "g(x)" s 16t
.1l 11.1 du.
96 max (K1, (K²)
(51)
Then.
[[g(()) - 1 En (X,)
nic x)
sup 9EG
S+ (X))
< ex0exp Cg max (KP, K2]
for some positive constant Cg-
48

---

Published as a conference paper at ICLR 2026
Since Fe depends on the dataset X we will apply Lemma 0.1 as
the function class &
defined
M,
(K),,
k=1
where (K) {f (*) b/ s {). Here, g" (K) is defined in (0
Since
Nie, -::1: |l (1x)) s Ne,,, 9", ,l 11o0)
Mn 2n 3p+1 Mx(1+3p)
Mn 1
we can easily verify that conditions (49)
50 and 51 hold for Ki = K2 = 452, K = =
and G = {g g = (fo ff f E g That is, we have
||| - fol13,Fx 11ʃ sallBanl
n= /8
P
sup fEAE
1//
60 exp
Cg (-Cy, 16
+ |1 fol 2,
We define An = ((1) supperr
111//fo12,,, p,xx -11 foll 1
S 5/ Then, we have
folli. Fx
E
f E =11f foll2.1x c(x(1)) yr()
f E FF Ilf - foll2.x = y(n) I(X (n)) E An) + PPx (A%)
Te
( FC ||ʃ Soll2,n =/V2[x(m) y(m) + (A())
Teg
as 00.
O.2 VERIFYING (STEP D.2)
For f Fe. we have
f(x) = Σ fs(xs),
SC(P)
where fs satisfies the sum-to-zero condition with respect to the uniform distribution on (0, 1).
Consider positive constants C7 and Cs such that
C7 < inf px (x) I
sup (x( Cg. xEX
(52)
xEX
Therefore, using the inequality
52 for all C [p]. we have
11/o - 11 2
C7 (fo(x) [60()) [xx))
Cr Σ
(Fo.s(xs) - (x
(53)
SCIp)
Xs
Collfp. -
where (53) is derived from the sum-to-zero condition with respect to the uniform distribution (0,1) 1) and Cg VCr/Cs.
49

---

Published as a conference paper at ICLR 2026
Hence, conclude that
f E Fg: - - Jo.sll2.xx > s/x(m), y(m))]
f E F : 1lf - foll2. > =C ,
+ 0,
as n. - 00.
O.3 VERIFYING (STEP D.3)
Following the same approach as in the proof of (STEP D.I), and applying Lemma 0.1 to the func-
tion class G {g (Jo,s -ss)² GE), have
llfs - fo.sl12.n > x(x( r(u)) = 0. lim E के F
0.4 VERIFYING THE (STEP D.4)
Since
Te (F (Ft
I cxp(-221 2m)
गe (Bx)
for given data (""), using Lemma in Ghosal & Van Der Vaart 2007 ). we conclude that
lim
Eg
TT (Fe
Tm x (n), y(m)) x(n) = x!n) = 0.
Since holds for arbitrary x(1), the proof is completed.
50


Reviews:
[{"review_id": "ipdLMpYv2E", "reviewer": "Reviewer_7sy8", "summary": "This paper proposes Bayesian Tensor Product Neural Networks (Bayesian-TPNN), a Bayesian neural network architecture for the functional ANOVA model. The key innovation is incorporating Bayesian inference over both network parameters and architectures (i.e., the subsets of input variables forming each component). This allows efficient detection of higher-order interactions without predefining component structures, addressing the scalability issue of ANOVA-TPNN (Park et al., 2025), whose number of sub-networks grows exponentially with interaction order.\nThe authors design a specialized MCMC algorithm that alternates between growing/pruning architectures and updating continuous parameters via Langevin proposals. Theoretically, the paper establishes posterior consistency for both the overall regression function and its individual ANOVA components.\nEmpirically, Bayesian-TPNN is evaluated on eight real tabular datasets, synthetic benchmarks, and concept bottleneck image tasks. It delivers comparable or superior predictive accuracy to both interpretable and black-box baselines (NAM, BART, XGB, mBNN) and significantly better uncertainty quantification and component selection, particularly for higher-order interactions.", "questions": "Could the proposed method scale to higher-order input, like p=4 or 5?", "limitations": "", "rating": 6, "confidence": 2, "soundness": 3, "presentation": 2, "contribution": 3, "strengths": "- Clear motivation and significance: the paper tackles the long-standing scalability bottleneck of functional ANOVA neural models and provides a principled Bayesian alternative capable of learning higher-order terms without combinatorial explosion.\n\n- Novel integration of architecture learning into ANOVA-structured modes: treating subsets of input variables as random variables and exploring them via reversible-jump-style MCMC is a clever idea. The stepwise proposal mechanism guided by feature importance is intuitive and empirically validated.\n\n- The proof of posterior consistency for both the global function and each ANOVA component is nontrivial and strengthens the paper’s soundness.", "weaknesses": "- The paper spends many pages detailing MCMC updates and proofs but offers limited high-level intuition on why the proposed priors or proposal mechanisms work. As I'm not an expert of XAI and ANOVA, more discussion on background and a small synthetic visual example illustrating architecture evolution would help accessibility.\n\n- While results are comprehensive, the gains in predictive accuracy are modest compared to ANOVA-TPNN, and the improvements in uncertainty quantification, though consistent, are small in magnitude. It would strengthen the empirical narrative to include a more demanding high-dimensional or noisy regime where the Bayesian approach truly shines.\n\nMinors:\n- Some figures (e.g., component plots) and tables are crowded or use small legends.\n\n- The exposition occasionally repeats prior work descriptions or defers too much to appendices.\n\n- The paper would benefit from clearer separation between methodological explanation and algorithmic detail.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "r96UotRvbe", "reviewer": "Reviewer_Jnxk", "summary": "The authors propose a Bayesian method for estimating a Tensor Product Neural Network, a method for estimating a functional ANOVA with a neural net proposed at ICML this year.\nAuthors develop an MCMC algo using Langevin dynamics for continuous parameters and bespoke proposals for the discrete ones.", "questions": "I think this work was very clearly presented and the motivation for the method itself is clear, so I don't have any questions, but I invite the authors to nevertheless answer:\n1) What did I get wrong in my review?", "limitations": "", "rating": 4, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 2, "strengths": "The article is logically organized and easy to follow overall. It's clear what the goals are. From a structural perspective this piece is very well written.\n\nThe contributions of the article are clear, and the proposed methodology is thoroughly investigated.\n\nThe numerical experiments seem adequate to me; they use several datasets and compare against a reasonable suite of alternative methods.\nFor instance, BART is a tough competitor, and matching/beating it while providing for high interpretability is notable.\n\nThough primarily a computation/applied article, the authors prove a basic asymptotic result of their method.", "weaknesses": "The biggest weakness of this article is that this work is fundamentally incremental: it is a straightforward Bayesian version of an existing method, for which the Bayesian inference is straightforward.\n\nSome of the limitations of Bayesian inference in the neural setting, notably the lack of scalability due to the nonexistence of a stochaastic version of the MH algorithm, are not addressed.\nDiscussion of how to mitigate this would have improved the article.\n\nNeeds spellchecking and grammar correction.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "AYc5GvvQ1O", "reviewer": "Reviewer_hVKC", "summary": "This paper investigates functional ANOVA decomposition, a technique for decomposing a high dimensional complex function into lower dimensional components to enhance model interpretability. Existing methods typically require pre specifying the maximum order of interaction in the low dimensional functions and exhaustively considering all feature combinations up to that order. In practice, this often limits analysis to pairwise interaction terms. To address this limitation, the authors propose Bayesian TPNN, a Bayesian framework that adaptively selects the most relevant features for inclusion in the low dimensional components without pre-specifying the maximum order. The method is evaluated across diverse domains, including real world tabular datasets, simulated data, and toy image datasets.", "questions": "1. In Table 1, the authors compare the proposed Bayesian TPNN approach with existing methods in terms of prediction accuracy. However, the proposed method does not appear to outperform existing baselines significantly, particularly when taking the standard errors into account.\n\n2. In Table 2, the authors evaluate uncertainty quantification against existing methods. It would strengthen the comparison to include deep ensembles, which are widely recognized as a standard baseline for uncertainty estimation.\n\n3. Most experiments are conducted on simulated or tabular datasets, with the exception of Section 4.4, which uses CelebA HQ and Catdog datasets. However, in these cases, the approach relies on another CBM model to generate interpretable concepts. Could the authors apply Bayesian TPNN directly to these datasets and produce interpretations for the low dimensional functions?\n\nOverall, it is challenging to identify practical scenarios where the proposed method would be preferred. I recommend extending the evaluation to additional domains, such as the genomics datasets used in Martens & Yau (2020), to better demonstrate applicability.", "limitations": "", "rating": 4, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "This paper is well written and easy to follow. Using a Bayesian approach to select features for functional ANOVA decomposition provides a theoretically principled framework that could be of significant interest to the interpretability community and potentially a broader audience.", "weaknesses": "My major concern is its evaluation and experiments. Most experiments are conducted on toy or tabular datasets, where the application of deep learning is arguably less compelling. Please refer to my detailed questions below.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)