Covariate-Guided Clusterwise Linear Regression for Generalization to Unseen Data
Authors: Dohyun Bu
Hyunho Kim
Jong-Seok Lee
Venue: ICLR 2026 Poster (2026)
Keywords: Clusterwise Linear Regression (CLR)
Covariate-Guided Assignment
Proxy Network
Vector Quantization
Convergence Guarantee
Primary area: other topics in machine learning (i.e., none of the above)

Abstract: In many tabular regression tasks, the relationships between covariates and response can often be approximated as linear only within localized regions of the input space; a single global linear model therefore fails to capture these local relationships. Conventional Clusterwise Linear Regression (CLR) mitigates this issue by learning $K$ local regressors. However, existing algorithms either optimize latent binary indicators, (i) providing no explicit rule for assigning an $\textit{unseen}$ covariate vector to a cluster at test time, or rely on heuristic mixture of experts approaches, (ii) lacking convergence guarantees. To address these limitations, we propose $\textit{covariate-guided}$ CLR, an end-to-end framework that jointly learns an assignment function and $K$ linear regressors within a single gradient-based optimization loop. During training, a proxy network iteratively predicts coefficient vectors for inputs, and hard vector quantization assigns samples to their nearest codebook regressors. This alternating minimization procedure yields monotone descent of the empirical risk, converges under mild assumptions, and enjoys a PAC-style excess-risk bound. By treating the covariate data from all clusters as a single concatenated design matrix, we derive an $F$-test statistic from a nested linear model, quantitatively characterizing the effective model complexity. As $K$ varies, our method spans the spectrum from a single global linear model to instance-wise fits. Experimental results show that our method exactly reconstructs synthetic piecewise-linear surfaces, achieves accuracy comparable to strong black-box models on standard tabular benchmarks, and consistently outperforms existing CLR and mixture-of-experts approaches.

Published as a conference paper at ICLR 2026
COVARIATE-GUIDEY CLUSTERWISE LINEAR REGRES- SION FOR GENERALIZATION TO UNSEEN DATA
Dohyun Bu Hyunho Kim Jong-Seok Lee"
Korea Advanced Institute of Science and Technology
(dohyun bu, hyunho. kim, jongseok. lee) lee)@kaist.ac.
ABSTRACT
In many tabular regression tasks, the relationships between covariates and response can often be approximated as linear only within localized regions the input space; single global linear model therefore fails capture these local relationships. Conventional Clusterwise Linear Regression (CLR) mitigates this issue by learning K local regressors. However, existing algorithms either optimize latent binary indicators, (i) providing no explicit rule for assigning an unseen covariate vector to a cluster test time, or rely on heuristic mixture of experts approaches, (ii) lacking convergence guarantees. To address these limitations, we propose covariate guided CLR, an end-to-end framework that jointly learns an assignment function and K linear regressors within single gradient-based optimization loop. During training, a proxy network iteratively predicts coefficient vectors for inputs, and hard vector quantization assigns samples to their nearest codebook regressors. This alternating minimization procedure yields monotone descent of the empirical risk, converges under mild assumptions, and enjoys PAC-style excess-risk bound. By treating the covariate data from all clusters as a single concatenated design matrix, we derive an F-test statistic from nested linear model, quantitatively characterizing the effective model complexity. As varies, our method spans the spectrum from a single global linear model to instance-wise fits. Experimental results show that our method exactly reconstructs synthetic piecewise-linear surfaces, achieves accuracy comparable to strong black-box models on standard tabular benchmarks, and consistently outperforms existing CLR and mixture-of cexture-of-expertt approaches.
-uiidd
INTRODUCTION
Empirical studies suggest that, in many tabular data regression problems, the relationship between covariates æ; E ER and the response y/1 E R can often be approximately linear only within localized regions of the input space; outside these regions, pronounced heterogeneity motivates modeling with several local linear components (Zhong et al., 2016 Devijver 2017 Yu al. 2017 Li & Liang 2018 Klusowski et al. 2019, Diamandis et al. 2021] Kong et al. 2020 A single global linear model
lacks the flexibility to represent local structures, whereas fully nonlinear black-box methods impose complexity that diminishes the applicability of conventional statistical diagnostics (Lipton 2018
Petch et al. 2022; Chen & Zhang, (023). Clusterwise Linear Regression (CLR) offers middle ground: given N samples {#;; 3i) f=i it learns K linear regressors { E g
Jj=1 and N
binary indicators [as,,1, Q4,, ..., k N with [as,1, .., xx,k] (0, 1] and Σ ai I.
With the augmented covariate vector x, = [x]] 1 E RR+11 CLR minimizes
N
K
w))
thereby retaining the simplicity of local linear models while accommodating heterogeneity through clustering. CLR has been applied successfully in market segmentation (Preda & Saportal 2005). welding-process control (Ganjigatti et al. 2007), pavement management Khadka & Paz] (2z1(2017) 77), and rainfall forecasting (Bagirov et al. 2017). However, its classical form assumes a generative mixture of linear functions or relies on host-hoc clustering of covariates-response pair-assipae asssmmtiisnnns misaligned
"Corresponding author.

---

Published as a conference paper at ICLR 2026
with predictive tasks, where one must assign a new covariate vector the appropriate local model without access to an underlying mixture distribution (Manwani & Sastry 2015; Gitman et al. 2018).
Existing CLR-based and related methods face two fundamental limitations in the standard "single- point" prediction task, particularly under non-realizable setting-whers the data are not assumed to be probabilistically generated by exactly one of K linear models. First, most algorithms decouple clustering and regression- Via mixed programming based approach (Bertsimas & Shioda 2007] Carbonneau et al. 2011), column generation based approach (Carbonneau et al. 2014), or algorithm based approach (Pal et al. 2022] Ghosh & Mazumdar (2024) -yielding no explicit assignment rule for unseen covariates, which leads to overfitting on training dataset and degraded out-of-sample performance (Long et al. 2023). Second, Sparse Mixture-of-Experts (MoE) schemes, which integrate gating and regression (e.g., MoE-based (Ismail al. 2023 tree-based splits (Ahmed et al. 2018 Raymaakers et al., 2024)), suffer from unstable convergence, rely heavily on heuristics, and when using axis-aligned partitions- cannot capture true assignment boundaries. As result, the problem of learning and generalizing multiple local linear regressors for new covariates in the non-realizable setting remains largely unresolved.
To address these challenges, we introduce CG-CLR (Covariate Guided Clusterwise Linear Regres- sion), an end-to-end approach that unifies clustering and regression under dual loss. CG-CLR maintains codebook of K linear regressors WV == [ لرر= and an M-hidden-layer proxy network Wa. parameterized by & which predicts for each input æ; the regressor that best fits it. Con- cretely, the proxy network outputs a proxy vector ,i; == Wo (æ:), and each input æ; is assigned to the regressor whose prediction x w is closest to x s. Training proceeds by alternating between (i) assigning each sample according to explicit assignment rule and (ii) updating both W and db to minimize the overall dual loss. This simple vector-quantized formulation yields a fully differentiable pipeline, admits a convergence proof for the empirical risk, and-by treating the concatenated design matrix of all K regressors as a single linear model allows us to compute the classical F-statistic to quantitatively characterize the effective model complexity.
Our main contributions are:
End-to-end assignment and regression: CG-CLR is the first CLR framework that trains both the data-driven assignment rule and the K local regressors jointly in a single gradient loop, thereby providing a principled way to assign unseen covariates at test time.
Convergence of alternating minimization: By reformulating CLR as a dual loss V1. we show that our alternating updates yield a monotone descent of Va and that CG-CLR exhibits linear convergence towards optimal parameters (WV x²))
PAC-style generalization bounds: In the sense of PAC learning for functional mixtures, we derive excess-risk bounds for single-poing prediction under non-rcalizable settings, scaling as O(max, || dM log d log 2N/N).
Model complexity quantification via F-test: The joint design-matrix viewpoint enables embedding the classical F-statistic as a transparent indicator of the effective degrees of freedom when varying K.
Fine-grained control over model complexity: Adjusting the cluster count K lets CG- CLR range from single global linear model (K= =1) to nearly one regressor per instance (K N/(NN1(()))))) (p+1)), giving users a smoothly adjustable dial between model simplicity and predictive flexibility.
2 RELATED WORKS
Methods that learn local linear models for tabular regression can be broadly grouped into three families: clusterwise linear regression, which assigns each sample to one of several linear models; piecewise linear regression, which constructs explicit partitions of the input space and fits linear model in each partition; and hypernetwork-driverr local experts, which employ a learned hypernetwork to either select small set of linear experts or generate an input-specific linear model. We review key ideas and representative works in each family and then summarize their common limitations.
Clusterwise linear regression Conventional CLR fits the NK binary assignment indicators
[a.,1..... x,, and the associated linear models (v. K by solving a sexxed-intgeer program
(Bertsimas & Shioda 2007 Carbonneau et al. 2011
4 D Uruo al. 2010 Joki al. 2020
2

---

Published as a conference paper at ICLR 2026
Table 1: Comparison of representative approaches against the four desiderata: availability of single-point prediction rule, a rigorous convergence analysis, applicability to the agnostic setting. and ability to model flexible (assigned) regions (v = satisfied, X = not satisfied).
Method Class
Single-Point
Convergence Analysis
Agnostic
Flexible Regions
Conventional CLR
X
High
MLR
High
LDT / PILOT
X
Low
DC
Moderate
S-IME,
/
High
CG-CLR
High
Klusowski et al. 2019). Most of these studies carry out their analysis in the realizable setting, where each sample is assumed be generated by one of exactly K linear models. Although this assumption facilitates coefficient-rrcoovery analysis, it fails to account for more general non-realizable cases and convergence guarantees cannot be established if the generative-mixtue assumption is even slightly violated. More recent works, such as MLR, adopts an agnostic (non-realizable) setting, optimizing a hard-assignment min-loss via alternating minimization or a soft-assignment min-loss via expectation maximization (Pal et al. 2022 Ghosh & Mazumdar 2024). Such "min-loss" guarantees hold for list decoding: given a test pair (ar, yer) with the response yy in hand, one can retrospectively select the linear model that attains the smallest error ly x , among the learned list.
Piecewise linear regression One strategy is to build decision trees or rule lists that partition the covariate space and then fit separate linear model in each leaf (e.g., LDT (Ahmed et al.. 2018). PILOT (Raymaekers et al. 2024)). ). Another branch of research realizes piecewise linear functions with shallow neural networks whose activations are absolute or maxima (Kahlert & Chua 1990.
Wang & Sun 2005). and deeper nested architectures extend this idea to cover all piecewise linear functions (Lin et al. 1994). More recently, difference-of-comme
(DC) formulations describe the response the gap between two max max-linear envelopes and estimate the underlying hyperplanes through a single optimization step (Siahkamari et al.] 2020).
Hypernetwork-drrien local experts Sparse of models pair a fixed set of linear experts with learned gating network that assigns each input a vector of weights over those experts. thereby producing an input-specific convex combination or hard selection (e.g.. S-IME, (Ismail et al. (023)). A complementary direction dispenses with the expert pool entirely and employs hypernetwork that maps the covariate vector directly to an instance-wise coefficient vector: attention- style TabNet (Arik & Pfister [021] variants focus feature selection (Yang al. 2022) while
contextual lasso and other linear hypernetworks generate full local regressors (Thompson et al. 2023
Kadra et al., 2024).
Limitations Despite their diverse mechanics, the three families face limitations (Table D Conven- tional and agnostic CLR variants optimize only the fitting of K linear models and require a post- assignment rule to map unseen 2y to linear model (Manwani & Sastry] 2015 Gitman et al. 2018
Long et al. 2023 Vicari Vichi (013), creating an assignment bias that can degrade generalization
(Tao al. 2022). Tree-based and DC piecewise linear methods hinge axis-aligned locally continuous partitions, which can restrict the geometric flexibility of the resulting regions. MoE frameworks train with soft gating but predict with hard selections, while instance-wise hypernetworks often exhibit highly varying coefficients and impose excessive complexity, rendering them nearly non-transparent black-box models. Among the existing cluster-oriented methods, single ap- proach simultaneously achieves (i) a built-in single-point predictor, (ii) provable convergence in the agnostic regime, and (iii) assignment flexibility that is not restricted locally continuous partitions.
METHOD
We formalize the task of learning multiple local linear regressors together with an assignment rule as single optimization problem. This section presents the global problem formulation (Section 3.I. implementation details (Section 3.2, and theoretical guarantees (Section 3.3).
3.1
PROBLEM FORMULATION
Setting We are given N i..d. samples ((z,, y XRRR drawn from unknown distribution D.
No generative mixture is assumed- fully agnostic setting (Pal et al. 2022 Ghosh & Mazundar 2024). Our goal is to
3

---

Published as a conference paper at ICLR 2026
(i) learn a response-free routing rule that maps any unseen covariate vector ay to index zv of the linear regressors; and
iii) jointly fit the corresponding K linear regressors [ that the test-time prediction y/ x x w.y accurate without ever observing the true response ye
This single-point test scenario precludes any response- response-dependenl objective-such as the empirical min- loss equation[] [ which chooses a regressor after seeing the response yo- We therefore introduce and optimize the surrogate described below.
Response-aware risk (infeasible) If the response y/ were available before choosing one of the K linear models, the oracle risk I would be
L L'WW) == E(Dng)) min (vi - xxxxx,,)) w (j((l....k)
and minimizing its empirical analogue - is NP-hard even for moderate K Yi et al. 2014). More importantly, the inner minimization over is response-dependent it selects the best regressor after observing yi- This makes the objective fundamentally infeasible for CLR, whose test-time scenario requires selecting a regressor for new covariate xy without access yv. For this reason, response-free surrogate is required.
Proxy-based assignment CG-CLR eliminates these obstacles by first generating, for each covariate vector, proxy We == (wx) E IR++1 through the proxy network. The sample is then hard-routed the linear regressor (codebook regressor) whose predicted value is closest to the proxy prediction:
Zy argmin jE(I....KK]
(x.o w))
Let = j denote the set of indices of samples assigned to the jthh regressor. This decision depends only on the covariate vector a. and therefore remains valid test time when the response Ysr is unavailable. Iquationnacan be viewed couples the proxy space and the codebook, allowing the gradients of the downstream objective to jointly update both the proxy network parameters & and the linear regressors W..
rector-quantizatioo step that tightly
Prediction-fit fit loss When the proxy network is updated, we want to evaluate the current codebook in a fixed state, yet still let the gradient flow only to the proxy parameters Φ. Let v. Wistoo, and stop denote the stop-gradient copies of w.. W. and w.,, respectively. We define
N
Rai (Φ. Wistop) =
M N
1
yi x (wr,
a stop
T3 stop
(3)
22
gradient
Because the term o; - , stop vanishes in the forward pass, equation 3 simply measures the squared error of the frozen prediction x wpp, In the backward pass, however, V R = 0 while R of O; thus, the proxy network receives the usual regression gradient without ever increasing the current clusterwise loss.
Alignment loss To encourage the proxy output vectors and their assigned codebook vectors to coincide, we add a symmetric alignment term
N
Ralign (Φ. W) = N
Σ
x (wr (ow--,,,))² 3
whose gradient propagates to both the proxy network parameters and the codebook W. pro- gressively shrinking the prediction gap between x क; and x w., induced by the routing rule in equation 2
Composite objective The two losses play complementary roles: Ri keeps each regressor faithful to its currently assigned samples, whereas Mallgnn pushes the proxy predictions toward their codebook counterparts so that future assignments are improved. We blend them with non-negative weight À >
Vab. WV ( Wystop + (1 X) Ralign (t), W.).
4

---

Published as a conference paper at ICLR 2026
The overall learning problem is
min (a W 0.WV
(6)
A larger À places more emphasis on closing the proxy-codebook gap, whereas À = 0 assigns equal weight to fitting and alignment. In practice, we treat À as a hyperparameter that trades immediate prediction accuracy for faster long-term convergence of the joint model.
Rationale for surrogate objective When the alignment term vanishes (Ralign - 0), every proxy prediction converges to its assigned codebook vector prediction (x; रे + Wh w, In that limit, the prediction-fit loss reduces to the response-awary min-loss in equation assignments, yet the routing rule itself remains response- free. Hence, minimizing equation O steers the system toward the oracle target while respecting the single-point test constraint. Because V,, is fully differentiable, we can optimize it using standard gradient methods. The stop-gradient copies in R ensure a clean role separation: this loss measures each regressor's current error, whereas the alignment term updates both the proxy network and the codebook. Together, they yield a monotone descent of VA (proved in Section 3.3 under the alternating updates described Section 3.2
T evaalaaeed on the current
3.2 IMPLEMENTATION DETAILS
MLP W
% Wi
W.. X
V
W
W
W
stop
+ X2
sdo
proxy output vector wr Rp+1 The vector quantizer compares the proxy prediction ÿ: x; W. with the K codebook vector predictions {x, wr / and outputs the index The selected codebook vector vi produces the final prediction ÿ. = x w.. Two losses are computed: the prediction fit term R equation 3 and the alignment term Relign equation 4 During back- nack-propagation. the stop-gnt noo-ggradient copies (dashed arrows) prevent the proxy path from negatively affecting the current clusterwise fit.
Figure 1: CG-CLR architecture. The proxy network We maps covariate
Figure Millustrates the forward and backward passes; the paragraphs below describe the two learnable components and detail the alternating minimization schedule used during training.
Proxy network (W&) Unless stated otherwise, we instantiate We as a depth-M ReLU multilayer perceptron
ReLU RR FC(p.h1), ReLU
R +
FC 1./2)) ReLU
R (C(har.pt)) R=+1
where h....., has denote the hidden dimensions. The final layer is left linear so that each coordinate of the proxy vector w. W (xx)) can take any real value.
Codebook of linear regressors (V) The K regressors stored as matrix W = [ Rx] R((+1)xK with the last row representing the bias terms. We initialize (V entry-wise, where each element is sampled independently from Unif( 1/ K), and standardize each raw feature so that the slope and bias coefficients operate on comparable scales throughout training.
Training algorithm Although the updates for & and WV are written separately for clarity, both are optimized jointly through the single composite loss VX; the stop- nop-gradient mechanism only controls which part of the model receives gradient flow at each step, analogous to VQ-VAE (van den Oord et al. 2017) training. Optimization proceeds iteratively for epochs t = 0, 1, 1....,T I. Each epoch consists of the following two-step cock-coordinaee update (also refer to Figure
Assignment step: For every sample in the current mini-batch, we compute the proxy क = W aos (2:), and assign
argmin (x]] (w, - -w);) jE(K]
as in equation updates below.
2 The resulting mini-batch partitions { are cached for the two parameter
5

---

Published as a conference paper at ICLR 2026
Proxy network update: With assignments fixed, we back-propagate the composite loss (o W()) in equation Stop-gradient copies ensure that the gradient flows only to Φ; the codebook remains frozen. We take one gradient descent step with learning rate ry:
(1++1) (t V Vi ((b ,T)())
Codebook update: Holding the updated proxy network fixed, minimize the composite loss V3 with respect to W. Since Rft carries soop-gradient on W, the update reduces to an alignment step:
w(++1) Ty()) 7 Vey Ralign (tb wy())
Algorithm is presented in full-batch form for clarity, although in practice we recommend performing mini-batch updates.
Algorithm 1 CG-CLR: Alternating Minimization Algorithm
Require: Dataset (1 (( y)) cluster number K, learning rate v) 0. weight À > 0, number of epochs T'
I: Initialize: (0) 1y
2: for to 0, 1,..., do
3: Assignment step: Compute 2,0) and construct partitions / E [K], according to equation 2
4: Gradient updates:
(41)
(0) +
2n(1 + A)
Y/
Ax
x ((()) (V (Va))!))
N
1
(1)
2m111 + X)
Σ
x (#/) - ) Vj [A].
N
5: end for
6: return ( WT(
Test-time prediction. specific coefficient estimate W (xx)). ). We assign the input to the closest codebook regressor via equa- tion 2 and make the final prediction using the selected codebook coefficient, 5(xx)) = x 1i 2., As an
Given new covariate vector Fy, the proxy network produces an instance-
alternative inference mode, one may also use the proxy coefficients directly. j(x,) = x (xx)).
3.3 THEORETICAL ANALYSIS
We now establish rigorous theoretical guarantees on the convergence and generalization of CG- CLR. We clearly state our assumptions, explicitly define necessary notation, and formally link these theoretical insights to the proposed optimization algorithm.
Definitions and notation To clearly state the theoretical guarantees, we define the following notation.
The empirical risk on the training set is denoted by Raain. while the expected risk on an unseen test sample is Rest- The empirical quantity actually minimized during optimization is represented by St. The family of predictors realized by CG-CLR forms the hypothesis class H. {Φ) W)}. parameterized by the proxy network weights & and the codebook W.. Finally, we assume bounded support: there exist constants Y Ymax; Xmin, Xmax > such that ly| S Ymax and Xmin < ||x: < Xmax almost surely for (ær, y) ~ D. More explicit definitions of the expected risk, empirical risk, and the empirical quantity appear in Appendix C.I
Key assumptions We collect here the regularity conditions used in our analysis. They play technical role in guaranteeing that the Lyapunov argument applies and that the clusterwise regressions are well-posed: the first assumption concerns the smoothness of the proxy network, the second the curvature of the alignment loss in the codebook parameters, the third the separability of distinct linear rules, and the fourth a minimal expressivity requirement on the proxy network.
Assumption 1 (Lipschitz continuity and Jacobian lower bound of proxy network). For each fixed x, the mapping & > W (x) is Lg Lipschitz continuous and its Jacobian satisfies | W o (x) vll 2 m& |||l > 0 for all v E gd
Intuitively, Assumption I rulle out flat or infinitely steep directions in the proxy parameter space, so that gradient- gradient-based updates of & change Wss (x²) in controlled way.

---

Published as a conference paper at ICLR 2026
Assumption 2 (Strong convexity and smoothness). For fixed cluster assignment the align- ment loss Ralggn Φ,, W) is strongly convex and smooth with respect to W, satisfying pl 1 W) LI,for constants μ., L O.
42 Rulign WV
Assumption
2 ensures that, once cluster assignments are fixed, the codebook update corresponds to strongly convex quadratic problem, as in standard well-posed least-squares regression for each cluster.
Assumption 3 (Cluster separation (Ghosh & Mazumdar 2024 1.1 Setu)))). There exists a mini- mum gap A between predictions from distinct optimal regressors. Formally, for the optimal W min min; [ [ w 2 A> O.
This separation condition guarantees that distinct optimal regressors remain distinguishable in terms of their predictions, so that the ausignesigmment rule does not oscillate between clusters in regions where the underlying linear rules genuinely differ.
Assumption 4 (Expressivity of the proxy network). Let F == { Wo(x) Φ E & denote the function class realized by the proxy network, and let Fcodebook == fa: s(x)) be the class of all mappings induced by K fixed linear regressors, where z(x) is the hard-assignment rule in equation2 2
We require that the pseudo-dimension of Frroxy satisfies Pdimi(Frrass) 2 C (p++1)
for some absolute constant C O.
Hence the proxy network is at least as expressive, in complexity sense, as the class of clusterwise linear mappings represented by the codebook, so does not become bottleneck in approximating those K linear rules.
Lyapunov function and descent guarantee To rigorously establish the convergence properties of CG-CLR, we explicitly introduce the following Lyapunov function:
(Ø WV : R (Φ, Watop) + (1 A) W).
Lyapunov function a scalar-valued function typically used in optimization and control theory to demonstrate convergence. Such a function must satisfy two crucial properties:
(i) It nonnegative and attains its minimum at the equilibrium (or optimal) point.
(ii) It strictly decreases along the trajectories (or iterates) of the algorithm.
In our case, VA clearly meets these criteria:
(i) Both 1tth and aallige are nonnegative by definition, ensuring that VA is nonnegative.
(ii) Under appropriate step-size conditions, VA strictly decreases at each iteration (explicitly shown in Proposition 3.1) thereby acting as Lyapunov function guiding the convergence of CG-CLR.
Proposition 3.1 (One-epoch descent), Under Assumptions 7 and suppose the cluster assignment (t) remains fixed during epoch t. Define
Lv
max((1+ A)L, (4+ X
For any step-size 0 < <n S 1/Lv, the simultaneous gradient updates guarantee strict decrease of the Lyapunov function:
(x(((+11) Tr(++1)) < (1( TV(1)) " - [T((+1)) - - TT(()12)
This explicitly verifies that VA serves as valid Lyapunov function, rigorously ensuring the conver- gence of the CG-CLR algorithm.
Linear convergence Under Assumptions 3 and 4 the cluster assignments eventually stabilize due to the minimum prediction-gap condition, resulting fixed cluster indices (see Appendix A) Consequently, CG-C CG-LLR exhibits linear convergence toward the optimal parameters.
Theorem 3.2 (Linear convergence). Let Assumptions 7 hold. Define
JA == min (inf2m?xx 2 min: min: (1+ Ly == max (1+A)I, (4+2A)T2 2
Lv pv
max
g=
Ly + pv
Gradient descent with the optimal step-size n, 2/( + Lv) ensures that the stacked parameter vector 0 (b. W) converges linearly at the rate q, while the Lyapunov gap contracts the rate q².
7

---

Published as a conference paper at ICLR 2026
Theorem 3.3 (Generalization bound). For any 8 E (0, 1), with probability least 1 - 6,
dM log dl log 2N | JSK
801 max,sk
||
log(1/0)
Mmain + 160pC1 max Bestt
Y- max
N
VN
2N
where is an absolute constant for standard ReLU MLPs and Ymax Xmax-
Detailed proofs for Theorems
3.3 are provided in Appendices and C respectively,
Model complexity selection via F-test Finally, we propose a theoretically grounded criterion for selecting model complexity based nested-moder F-tests.
Proposition 3.4 (Sequential F-test). The necessity of adding another cluster is tested sequentially using nested-mont model F-statistics:
(SSEK - -SP FK-K+1 SSEK+1/(N) (K + 1)(p -FA4)NN((K+3))04))
and the number of clusters K is selected based on statistical significance at given significance level a.
A detailed derivation and proof of this criterion appear in Appendix D statistically principled mechanism for precise control of model complexity.
The proposed test offers
EVALUATION
In this section, we empirically validate CG-CLR. We first consider synthetic dataset whose underlying piecewise-linca function is exactly known. We also evaluate our method on real-world datasets, with detailed descriptions provided in Appendix
4.1 SYNTHETIC DATASET
(a) True Function
(b) CG-CLR (PROXY)
(c) CG-CLR (CoDEBOOK)
Figure 2a Ground-truth piecewise-liness function.
2h 2c) Heat maps coefficient recovery error norm on
the evaluation grid, for (26) proxy network and 22 codebook regressors. Colors indicate error magnitude, from dark purple (zero error) bright yellow (maximum error).
Experimental setup We sample four tilted wedges inside the rectangle [-1, [- and generate responses from exactly one three linear rules (also see Figure 2a
+ 10,
T2
x
02
|m11.
(*11,
-1 x2 3,
T <
|22
31 + 25²
+ 2,
F1 >
F2/-
Thus, every sample (æ;, is realizable: the label is generated by a true piecewise-linear surface, with added i.i.d. Gaussian noise E; (O, 0.13). We draw 50 points per region (N 200 total) and standardize both the covariates and responses before training. As illustrated in Figure 2a the true underlying model is composed of three clearly distinet linear regions, creating challenging fitting scenario, especially since one region is completely separated by the other two. For evaluation, we create a 1000 x 1000 grid and predict the noise-free f*; therefore, all reported errors reflect model bias alone. In addition to prediction errors, we compute the distance between the recovered and ground-truth coefficient vectors on the same grid to assess how well each method reconstructs the underlying linear rules. We compare four models trained with K 3. except LDT, whose tree must grow to depth 2 (effectively K 4) to avoid early pruning: (i) LDT (Ahmed et al. 2018), (ii) S-IME, (Ismail et al. 2023 (iii) CG-CLR (PROxY), and (iv) CG-CLR (CoDEROOK).
8

---

Published as a conference paper at ICLR 2026
LDT (K=4)
S-IME,
CG-CLR (PROXY)
CG-CLR (CODEROOK)
Figure 3: Absolute-error heat maps evaluated on a grid. Colors range from dark purple (zero error) to bright yellow (highest error).
Here, CG-CLR (PROXY) denotes the inference mode that predicts ((x.) = x (a;);) using the instance-wise coefficients output by the proxy network, whereas CG-CLR (CODEBOOK) uses the cluster-specific coefficients w.,
W., selected by the hard-assignment rule in equation 2 producing the
codebook prediction 5( x We. The heat maps in Figure 3 visualize the resulting prediction
errors, while Figures 2b 2c compare the underlying regression coefficients themselves.
Qualitative results Figure 3 visualizes the prediction errors on the evaluation grid, while the full prediction surfaces are shown in FigureBinn the appendix. Only CG-CLR achieves (near-)zero error everywhere: the codebook predictions are exact, except for thin artifact along the data-free cluster boundaries, and even the proxy predictions deviate only slightly due fine- ane-grained noise induced by the noisy responses. By contrast, coT-stnntr to axis-aligned splits misses the oblique boundaries and produces large biased regions, while S-IME, incorrectly merges the northeast wedges, creating broad region of high error.
Coefficient recovery analysis Although the proxy and the codebook yield nearly identical prediction errors, their coefficient recovery differ significantly. The codebook, consisting of only three shared linear regressors, almost perfectly recovers the ground-truth coefficients; consequently, the error norm heat map is nearly blank (Figure [c]. In contrast, the proxy network generates distinct coefficient vectors at each point. While these proxy estimates are unbiased on average, they exhibit visible fluctuations, resulting in a speckled eгror norm map (Figure 2b). Therefore, the shared codebook provides a stable and accurate reconstruction of the underlying linear rules, whereas the instance-wise coefficients remain noisy noisy-even when predictive accuracy is high.
Selection of K using the F-test Figure in the appendix shows that models with K = 1,2 underfit the data, whereas models with Z 4 over-segment the surface. A nested-model F-test on the training set quantitatively confirms this visual intuition: the step from K 2 to 3 is significant the a = 0.01 level (p-value < 0.001), but the increase from K = 3 to 4 is not (p-value- = 0.038 > 0.01). Thus, K is the smallest model that passes the test, aligning quantitatively with the ground truth.
4.2 REAL-WORLD DATASETS
Experimental setup Table in the appendix lists the seven tabular benchmarks for regression. Categorical columns are one-hot encoded, and all numeric standardized. We perform nested 5-fold cross-validation: each outer fold is held out for testing, and the remaining data are split 3:1 into training and validation sets, resulting in 20 independent test estimates per dataset. All baselines are tuned via grid-search on the validation fold. For CG-CLR, we fix all hyperparameters across datasets proxy network architecture. optimizer settings, and regularization parameter À = (Appendix F.3 such that the only dataset-dependent parameter is the coverage budget K |N./(10p- 10) used for the coogee-coeerage group, where N₁, is the size of the training set.
including the response are
Model groups We group baseline models by their coverage, defined as the average fraction of training samples explained by a single local regressor. Small-coverage: Methods that have either no explicit partition (RF (Breiman 2001), XGBoost (Chen & Guestrin 2016). CatBoost (Prokhorenkoval
al. 2018), vanilla DNN) or an instance-level partition with K > [Nur/ ([Nrr(p+1)) represented by DC (Siahkamari et al. (020) and the proxy prediction of CG-CLR (CG-CLR (PROXY), which uses MLR with post-hoc nearest-neighbor cluster assignment (MLR*) ) (Pal et al. 2022), its EM-based counterpart
x w.)) Large-coverage: Methods that maintain a small expert count <
(EM-MLR") (Ghosh& & Mazumdar 2024). CART (Breiman et al. 2017). 7). PILOT (Raymaekers et al.) 2024), LDT (Ahmed et al. 2018). S-IME,, (Ismail et al. 2023), and our CG-CLR CBCCCCCDDEEEC (CODEBOOK). This grouping illustrates how distinguishes between methods that can regulate model complexity through clustering and those that cannot.

---

Published as a conference paper at ICLR 2026
Table 2: Test RMSE (95% confidence intervals) across the seven datasets (lower is better). Within each model group, results are shown bold they are statistically indistinguishable from the group best. The overall best per dataset is underlined. with the same rule applied.
Model
CONDUCT
HOUSING
BIKE
ELECTRICAL
PLANT
WINE
CONCRETE
RF
9.74.
[0.507, 0.515
35.89957.000 39..77.01
[0.012, 0.012
[3.420.3.30 3.512]
0.639
18.185, 5.577]
XGBoost
9.73. 9.85
10.153,0, 064
45.51, 16.27
[0,008, 0.008
3.156. 3.244
0.622. 0.646
4 3488 4 666
CatBoost
9.62, 9.76
0.440. 0.446
44.69, 45.29
0.007
3.108. 3.216
0.621. 0.643
4 057. 493
DNN
10.35. 10:53
[0.5022 0.508
41.78, 42.70)
[0.007, 0.007
[3.738, 3.832
[0.859. 0.681
[4.700, 5.08)
DC
[11.38, 11.94
[0.586, 0.008
06.
65.70)
0..01200.012)
[3,006, 3 706
0.638, 0 0.664
[b.454, 0.854
CG-CLR (PROKY)
[30.36, 10.50
[0:187, 0..98
[39. 96, 40.86
0.006, 0 0..006
[8.583, 3 680)
[0.6544, 0.678
[1.063, 5 096]
MIRR
(3.77,33.955
[1.150, 1.169
(177.57, 178.91
[0.035, 0.035
[13.717, 13:877]
(0.827, 0.843
15.037,
15.449)
EM-MER
[8.75, 19.11
[0:7488 0.708
[102 28, 103.000 62
00,022, 0.022
[6.2008 5.406]
007726 0.747
[10:404, 10.938
CART
[16.56, 16.86
[0.615, 0.659
[120.31 122.30]
[0.021, 0.021
[1.217,
[0.734, 0.736
[9.316, 9 822]
PILOT
[15.3]
[165, 26, 466.48]
[0,033, 0.033
4..771
[0.746, 0.
[14226, 84.824
LDT
[3.33, 15.62
[0.603,0.77 0.713
[59. 11, 60.02)
(0.015,0.015
[10076 1.162]
[0.695, 0.718
[6.110, 6118]
S-IME,
12.68, 13.04
[0.560,0.50
[56. 13, 58.01]
[0.030, 0.010
[4.183, 4.259
[0.693.
[ [5.853,
CG-CLR (COnEBOOK)
10.50, 10.62)
0.485, 0.497
[40.77, 41.71
0.006, 0.006
[3.573, 3.675
0.652, 0.676
[5. [5.193,5 537
Results Given the same coverage budget K = [Nu/(10p++0)) CG-CLR achieves the best RMSE among all large-coverage methods and even attains the overall best performance BIKE and ELECTRICAL. On the remaining five datasets, its RMSE is only marginally higher than the best grradient-booste ensemble, despite using just K shared regressors instead of thousands of trees. MLR" illustrates the danger of post-hoc assignment: while its local experts adequately fit the training data, the absence of a learned assignment rule severely hurts generalization. resulting in the worst scores within its group. CART, LDT, and PILOT are constrained by axis-aligned or single-split partitions, and S-IME, exhibits optimization instability. of CLR provides both a stable global objective and a built-in single-point predictor, matching or surpassing black-box accuracy while maintaining a compact codebook that enables principled control over model complexity. Additional cealuations-includinn sensitivity analyses for À and K, computational cost comparisons, an empirical check of Assumption 3 and a small-K case study illustrating interpretability via local linear models provided in Appendix
contrast, the vector-quantized alignment
CONCLUSION
We introduced CG-CLR, an end-to-end framework that jointly learns cluster assignment and local linear regressors via a rector-quantized dual loss. The resulting algorithm (i) provides a single-point prediction rule for unseen covariates through a covariate- -uideduuddd pipeline, (ii) enjoys monotone empirical-risk descent with formal convergence guarantees, (iii) admits PAC-style excess-risk bounds, and (iv) includes an F-test criterion to quantify effective model complexity. Empirically, CG- CLR accurately reconstructs piecewise-lines functions on challenging synthetic data and achieves performance comparable to strong black-box methods real-world tabular benchmarks, while maintaining a compact codebook that enables principled control over model complexity and, for small K, interpretability through local linear models. It also outperforms existing approaches designed for the same purpose.
For future work, relaxing the prediction-gap assumption (Assumption 3 by introducing input-space separation criteria or adaptive margins is promising direction to further broaden the applicability of the proposed convergence analysis. Another compelling extension is to incorporate cluster-specific feature selection or sparsity constraints, enabling more stable and interpretable local linear models in -igh-niimenional settings. Finally, extending CG-CLR to shallow nonlinear experts- sscerttssuchhh as kernelized or spline-base based regressors-con capture richer local structures while retaining explicit assignment and controlled model complexity.
6 REPRODUCTIBILITY STATEMENT
The proofs of the main results are given in Appendix D Details on figure generation are provided
in Section 4 and Appendix E Appendix Feesciibe the benchmarks used in our experiments, the corresponding experimental settings, and the implementation details of the bascline methods for comparison. In addition, the supplementary material includes the benchmark dataset and fully reproducible code.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDDGENEN ACKNOWLEDGEMENTS
7
This research was supported in part by the National Research Foundation of Korea grant funded by the Korea government (MSIT) (Grant number RS-2020-NR0495444 RS-2022-NR068756 and and in part by the Korea Evaluation Institute of Industrial Technology grant funded by the Korea government (MOTIR) (Grant number RS-2025-25458052..
REFERENCES
Ahmed Mohamed Ahmed, Ahmet Rizaner, and Ali Hakan Ulusoy. A decision tree algorithm combined with linear regression for data classification. 2018 International Conference on Computer, Control, Electrical, and Electronics Engineering (ICCCEEE), pp. 1-5, 2018. doi: 10.1109CCCCEE2208885111
Sercan Ô Arik and Tomas Pfister. Tabnet: Attentive interpretable tabular learning. In Proceedings of the AAAI conference on artificial intelligence, volume 35, pp. 6679-6687, 2021.
Vadim Arzamasov. Electrical Grid Stability Simulated Data. UCI Machine Learning Repository, 2018. DOI: http:///bo.rrg/122444///CCCCCC
Adil M Bagirov, Arshad Mahmood, and Andrew Barton. Prediction of monthly rainfall in victoria,
australia: Clusterwise linear regression approach. Atmospheric research, 188:20-29, 2017.
Peter L. Bartlett and Shahar Mendelson. Rademacher and gaussian complexitie: Risk bounds and structural results. Journal of Machine Learning Research, 3(Nov):463 482, 2002.
Peter L. Bartlett, Nick Harvey, Christopher Liaw, and Abbas Mehrabian. Nearly- ve-dimension and pscudodimension bounds for piecewise linear neural networks. Journal of Machine Learning Research, 20(63):1-17, 2019.
Dimitris Bertsimas and Romy Shioda. Classification and regression via integer optimization. Opera- fions research, (5(2):252-211 2007.
Lco Breiman. Random forests. Machine learning, 45:5-32, 2001.
Leo Breiman, Jerome Friedman, Richard A Olshen, and Charles J Stone. Classification and regression trees. Routledge. 2017.
Réal A Carbonneau, Gilles Caporossi, and Pierre Hansen. Globally optimal clusterwise regression by mixed logical-quadratic programming. European Journal of Operational Research, 21 212(1): 213-222, 2011.
Réal A Carbonneau, Gilles Caporossi, and Pierre Hansen. Globally optimal clusterwise regression by column generation enhanced with heuristics, sequencing and ending subset optimization. Journal of classification, 31:219-241, 2014.
Chun-Has Chang, Rich Caruana, and Anna Goldenberg. NODE-GAM: Neural generalized additive model for interpretable deep learning, In International Conference Learning Representations, 2022.
Dangxing Chen and Luyao Zhang, Monotonicity for ai ethics and society: An empirical study of the monotonic neural additive model in criminology, education, health care, and finance, 2023.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. KDD 16, pp. 785-794, New York, NY, USA, 2016. Association for Computing Machinery. ISBN 9781450342322. doi: 10. (0.11144/9399722299975.
Cerdeira A. Almeida F. Matos T. Cortez, Paulo and J. Reis. Wine Quality. UCI Machine Learning Repository, 2009. DOI:
Wayne S DeSarbo and William Cron. A maximum likelihood methodology for clusterwise linear regression. Journal of classification, 5(2):249-282, 1988.

---

Published as a conference paper at ICLR 2026
Emilie Devijver. Model-based regression clustering for high-dimensional data: application to functional data. Advances in Data Analysis and Classification, 11:243-279, 2017.
Theo Diamandis, Yonina Eldar, Alireza Fallah, Farzan Farnia, and Asuman Ozdaglar. A wasserstein minimax framework for mixed linear regression. International Conference on Machine Learning, pp. 2697-2706. PMLR, 2021.
Pierpaolo D'Urso, Riccardo Massari, and Adriana Santoro. A class of fuzzy clusterwise regression models. Information Sciences, 180(24):4737-4766 2010. ISSN 0020-0255. doi: https://doi.org/ in0016/..nss.211008...
Hadi Fanace-T. Bike Sharing. https:/ddboorr/11.244//////////
UCI Machine Learning Repository, 2013.
DOI:
Jagadeesh Ganjigatti, Dilip Kumar Pratihar, and Roy Choudhury. Global versus cluster-wins regression analyses for prediction of bead geometry in mig welding process. Journal of materials processing technology, 189(1-3):352-36. 2007.
Avishek Ghosh and Arya Mazumdar. Agnostic learning of mixed linear regressions with EM and AM algorithms. In Forty-first International Conference on Machine Learning, 2024.
Igor Gitman, Jieshi Chen, Eric Lei, and Artur Dubrawski. Novel prediction techniques based on clusterwise linear regression. arXiv preprint 1804 10742, 2018.
Kam Hamidich. Superconductivty Data. UCI Machine Learning Repository, 2018. DOI: httpss/dbb.org/10224442/555545
Aya Abdelsalam Ismail, Sercan O Arik, Jinsung Yoon, Ankur Taly, Soheil Feizi, and Tomas Pfister. Interpretable mixture of experts. Transactions on Machine Learning Research, 2023, ISSN 2835-8856.
Robert A. Jacobs, Michael Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. Adaptive mixtures of local experts. Neural Computation, 3(1):79-8)) 87, 1991. doi: 10.1 10.11e2/neco.11 1991.3.1.79.
Kaisa Joki, Adil M Bagirov, Napsu Karmitsa, Marko M Mäkelä, and Sona Taheri. Clusterwise support vector linear regression. European Journal of Operational Research, 287(1)) 2020.
Arlind Kadra, Sebastian Pineda Arango, and Josif Grabocka. Interpretable mesomorphic networks for tabular data. Advances Neural Information Processing Systems, 37:31759-317777, 2024.
Claus Kahlert and Leon O Chua. A generalized canonical piccewise-linear representation. IEEE Transactions on Circuits and Systems, 37(3):373383,, 1990.
R. Kelley Pace and Ronald Barry. Sparse spatial autoregressions. Statistics & Probability Letters, 33 (3):291-297, 1997. ISSN 0167-7152. doi: http:///o.oog///.ooo////6667//11
Mukesh Khadka and Alexander Paz. Comprehensive clusterwise linear regression for pavement man- agement systems. Journal of Transportation Engineering, Part B: Pavements, 143(4):04017014, 2017.
Jason M Klusowski, Dana Yang, and WD Brinda. Estimating the coefficients of mixture of two linear regressions by expectation maximization. IEEE Transactions on Information Theory, 65(6): 3515-3524, 2019.
Weihao Kong, Raghav Somani, Zhao Song, Sham Kakade, and Sewoong Oh. Meta-learning for mixed linear regression. In International Conference on Machine Learning, pp. 5394 5404. PMIR, 2020.
Michel Ledoux and Michel Talagrand. Probability in Banach Spaces: isoperimetry and processes. Springer Science Business Media, 2013.
Yuanzhi Li and Yingyu Liang. Learning mixtures of linear regressions with nearly optimal complexity. In Conference On Learning Theory, pp. 1125-1144, PMLR, 2018.
12

---

Published as a conference paper at ICLR 2026
Ji-Nan Lin, Hong-Qine -ing Xu, and Rolf Unbehauen. A generalization of canonical piecewise- functions. IEEE Transactions on Circuits and Systems I: Pundamental Theory and Applications, 41(4):345-347, 1994.
Zachary C Lipton. The mythos of model interpretability: In machine learning, the concept of interpretability is both important and slippery. Queue, 16(3):31-57, 2018.
Qiang Long, Adil Bagirov, Sona Taheri, Nargiz Sultanova, and Xue Wu. Methods and applications of clusterwise linear regression: survey and comparison. ACM Transactions on Knowledge Discovery from Data, 17(3):1-54, 2023.
Naresh Manwani and PS Sastry. K-plane regression. Information Sciences, 992:39-56, 2015.
M Mohri. Foundations of machine learning, -mehryar mohri, afshin rostamizadeh, ameet talwalkar, 2018.
Yurii Nesterov et al. Lectures on convex optimization, volume 137, Springer, 2018.
Soumyabrata Pal, Arya Mazumdar, Rajat Sen, and Avishek Ghosh. On learning mixture of linear regressions in the realizalee setting. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song. Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning. volume 162 of Proceedings of Machine Learning Research, pp. 17202-17220. PMLR, 17-23 Jul 2022.
Jeremy Petch, Shuang Di, and Walter Nelson. Opening the black box: the promise and limitations of explainable machine learning in cardiology. Canadian Journal of Cardiology, 38(2):204-22) 213, 2022.
Cristian Preda and Gilbert Saporta. Clusterwise pls regression stochastic process. Computational Statistics & Data Analysis, 49(1)999-108, 2005.
Liudmila Prokhorenkova, Gleb Gusev, Alcksandr Vorobev, Anna Veronika Dorogush, and An- drey Gulin. Catboost: unbiased boosting with categorical features. In S. Bengio, Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi and R. Garnett (eds.), Advances in Neural Informa- tion Processing Systems, volume 31. Curran Associates, Inc., 2018.
Jakob Raymaekers, Peter J Roussceuw, Tim Verdonck, and Ruicong Yao. Fast linear model trees by pilot. Machine Learning, 1 113(9)6651-666 10, 2024.
Ali Siahkamari, Aditya Gangrade, Brian Kulis, and Venkatesh Saligrama. Piecewise linear regression via a difference of convex functions. In Hal Daumé III and Aarti Singh (eds.), Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 8904. PMLR, 13-18 Jul 2020.
Qinghua Tao, Li, Xiaolin Huang, Xiangming Xi, Shuning Wang, and Johan AK Suykens. Piecewise linear neural networks and deep learning. Nature Reviews Methods Primers, 2(1):42, 2022.
Pnar Tfekci and Heysem Kaya. Combined Cycle Power Plant. UCI Machine Learning Repository, 2014. DOI: http:///bi.org/1.2444///////
Ryan Thompson, Amir Dezfouli, and Robert Kohn. The contextual lasso: Sparse linear models via deep neural networks. Advances in Neural Information Processing Systems, 36: 19900-199911, 2023.
Aaron van den Oord, Oriol Vinyals, and koray kavukeuoglu. Neural discrete representation learning. In I. Guyon, U. Von Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017.
Donatella Vicari and Maurizio Vichi. Multivariate linear regression for heterogeneous data. Journal of Applied Statistics, 1206):200-11300 2013.
Shuning Wang and Xusheng Sun. Generalization of hinging hyperplanes. IEEE Transactions on Information Theory, 51(12):4425-4431 2005.
13

---

Published as a conference paper at ICLR 2026
Junchen Yang, Ofir Lindenbaum, and Yuval Kluger. Locally sparse neural networks for tabular biomedical data. In International Conference on Machine Learning, pp. 25123-25153 25153. PMLR, 2022.
I-Cheng Yeh. Concrete Compressive Strength. UCI Machine Learning Repository, 1998. DOI: https://dii.or//10.222// 2ttp///0.44//SPK..
Xinyang Yi, Constantine Caramanis, and Sujay Sanghavi. Altemating minimization for mixed linear regression. In Eric Xing and Tony Jebara (eds.), Proceedings of the 31st International Conference Machine Learning, volume 32 of Proceedings of Machine Learning Research, pp. 613-621, Bejing, China, 22-24 Jun 2014. PMLR.
Chun Yu, Weixin Yao, and Kun Chen. A new method for robust mixture regression. Canadian Journal of Statistics, 45(1):77- 94, 2017.
Kai Zhong, Prateek Jain, and Inderjit S Dhillon. Mixed linear regression with multiple components. In D. Lee, M. Sugiyama, U. Luxburg, I. Guyon, and R. Gamett (eds.), Advances in Neural Information Processing Systems, volume 29, Curran Associates, Inc., 2016.
14

---

Published as a conference paper at ICLR 2026
A CLUSTER ASSIGNMENT STABILIZATION
A.1 ABSTRACT ERROR CONTRACTION UNDER IDEALIZED ALTERNATING MINIMIZATION
This section presents a high-level analysis of how the alternating minimization procedure drives the stabilization cluster assignments. Here, we abstract away from the details of gradient- based optimization, and instead assume that each update within high-level epoch brings the proxy predictions ğ and codebook predictions y to the minimizers of their respective loss functions, given the current assignments. This idealized analysis serves illustrate the mechanism by which the alternating minimization scheme contracts prediction errors and ultimately leads to stable assignments.
We emphasize that this argument is schematic: isolates the error contraction and stabilization logic of the alternating minimization framework, while rigorous convergence proofs under gradient-based optimization are provided separately Appendix B
For given sample [I,..., N and high-level epoch e' E (1........ }. we let of denote the cluster assignment and define the corresponding codebook and proxy predictions as
x (t)
The dual loss minimized by the alternating updates takes the following two-term form:
(a W)
yy
w. D) 2 + (1 1) N 2=
0.)
N
Each iteration consists of the following updates:
U/ + Ao Ag((m) ((a)
0 ((n41)
x, tir (f++1)
Jargmin
Σ
(ū. ()))),))))
0 (l
argmin
-
1 1+1
iEs!(n)
This yields the explicit updates:
(( y/ + Ag. (s..)
of ((t+tt) = xx] ( (x((e)) (((
1+
where x(()) = [x, (ESf E R1 [x(++) denotes the design matrix for cluster j at epoch and 5 = l is the vector of proxy predictions assigned to cluster j.
Each high-level epoch in this analysis refers to a single alternating cycle in which the proxy and codebook regressors are assumed to be fully optimized with respect to their losses for the current assignments. This abstraction removes the effect of partial or stochastic optimization, allowing us to focus on the eиror contraction logic of the idcalized alternating minimization scheme.
To characterize the contraction per iteration, we establish the following bound:
Lemma A.1 (Projector gain k). Under Assumption 2 (strong convexity μ and smoothness L of Rallign)
holds, for every epoch t', cluster j, and sample E s
(0) 1o = 1|xx (x)
O -
(0) satisfies K4 LLH
Proof.
Applying the Rayleigh quotient to x (X] -'x; and the eigenvalue bounds μNI 3 xXXx; X; LNI from Assumption gives result.
From Lemma
A.I the error for the codebook prediction at each step is bounded by
(bn+1) M
max (((()) - yel S k max (()( yvl
- yil < K
(eg((n)
15

---

Published as a conference paper at ICLR 2026
while for the proxy prediction, have
1
(n++1)
Val
1/ Vil
(In)
1 1+1
where K = VLp..
Combining these yields the overall contraction factor:
A
with the assumption < (+ X (+X<1/(6-1)
1+X²
Consequently, the maximum contracts geometrically follows:
max 10 (tn) yel S p" max De yel.
Proposition A.2 (Geometric contraction). Given the contraction rate p above. for every and n 2
(tn) yal p" max 1y yrll
Vil K.p max 10
(())
yrl.
A.2 FROM PREDICTION CONVERGENCE TO STABLE ASSIGNMENT
Next, we relate the convergence predictions to the stabilization of cluster assignments. Let A denote the minimum prediction gap defined in Assumption g
Vi min |x]] (tir 1
# A 0.
If we define M max; l0!0) Vil. then by Proposition A.2
10 (t.)
Vel. (f(")) yil S kp 1-p=-1M M.
Thus, the maximum possible deviation in any score difference is at most xpm-114 M. By selecting
log(2xM/A)
no =
log(1/p)
+1,
we guarantee that for all f 2 tno the gap between the predicted value and each cluster's optimal regressor is at least 2/2, ensuring correct and stable assignments 2 Formally:
Proposition A.3 (Assignment stabilization). Suppose Assumptions 2 hold and p < 1. Then all
cluster assignments stabilize after tuo high-level epochs:
Vt'
2 tmo
2 2 Vi.
To summarize, this section provides a high-level view of the error contraction and assignment stabilization mechanism, assuming each alternating update fully minimizes the relevant loss. The formal proof of convergence under practical gradient- based optimization can be found in Appendix
16

---

Published as a conference paper at ICLR 2026
B LYAPUNOV ANALYSIS
This appendix provides the theoretical proofs for the Lyapunov analysis outlined in Section 3.3 The main coceriity,,, one-epoch descent, and linear convergence- restated here in the main text, followed by detailed proofs.
B.I COERCIVITY OF OBJECTIVE FUNCTION
We first show that the Lyapunov function Va (Φ, W) is coercive, meaning that it grows without bound as (Ø, W) moves away from any bounded set. Owing to the structure of our objective, this result follows directly:
Lemma B.1 (Coercivity). Let Assumptions
2 hold and recall the Lyapunov function Vi(Ø, WV Rht (ó, Wetop) + (1 + A) Rullign (Φ, W.. Then there exist positive constants H&b > and Prx
(1 + A)u/N such that
Va W)
Me || + "x ||
2
Consequently (x W + oc whenever
(1 WPIl + oc.
Proof.
Write (x(x) approximation, we have
(w) By Assumption 1 m, S [1 S Ly. Using the Gauss-Newton BΦ
V le 2 2m X I. Set ps 2m X then R 2 - 1 11
The term Aallgg is quadratic W. and its Hessian equals F/4 Exes, xxx t ul (by Assumption hence Ralign 2 1.
Adding the two bounds and multiplying the Ralign term by + yields:
VV WV) (Φ, Whstop + (1 + A) Aalian WV) 2 H 1012 +
(1 + A)
|
2
2
B.2 SMOOTHNESS OF OBJECTIVE FUNCTION
Lemma B.2 (Gradient Lipschitz continuity of Vi(Ø, W)). Under Assumptions 7
the gradient of
Vá W) is Lv -ipschhhi,,, where
Ly max (1 + A) L) L, (4 + 2\) x max
Proof.
Define X; [xsiles, The alignment part splits cluster-wise:
1 K
paliga (Φ. uv) = Σ 1
(1)))
where r,()) =- [x] WtJJes,
Hence
V2 paligu (Φ) WV) = diag N X x,) K j=1 )/
By Assumption
1 T N
X3
3 LI for every j. Therefore,
V2 KiΦ Ti) s A1L.
For each sample i, V V के.
< Lg max (by Assumption 1b. Therefore,
W|
2L
2 nalignn (Ø, WIll < (1 + A) 21
max
Add the two contributions gives
V Vi W < (4+ 21) mar'
The full Hessian of Va W) is block diak-niiagnna,, so its operator norm is the maximum of the two block norms computed above. Hence,
Lv max (1 + X)I, (4 + 2X)
which proves the claim.
17

---

Published as a conference paper at ICLR 2026
Proposition 3.1 (One-epoch descent). Under Assumptions | and suppose the cluster assignment 20() remains fixed during epoch I. Define
Ly
max (1 + A)L, (4 2
m
For any step-size < S L, Lv. the simultaneous gradient updates guarantee strict decrease of the Lyapunov function:
-6 [ (( Wh(t)) 2 (1lb",,,04 " -
- TT(1)13)
Proof.
Because (Øb W. is Lm-smooth (by Lemma B.2 for any update vector 0, = (Φ(++)) σ (((+1) W(( and current vector g(t) (b)(), Wi)), the descent theorem (Nesterov
et al. 2018 Theorem 2.1.5)) gives
+ V0 +0a) S + (VV2(000)) + Lv 10 10 2
Choose 02 = - TV(1)) with 0 < S 1/Lv.
Then,
LZLn (V Woo)
V1((()) ((((+))) < Vi(6)) (V 7/l (V (/
2
Vi(6 TV 7/1 nty
2
Since nLv < 1, the parenthesis is at least 1/2 yielding the stated decrease: (( W((+)) S
(1(())) WV() - (o(1", 1/
Theorem 3.2 (Linear convergence). Let Assumptions 14 hold. Define
Jt == min 2 min: (1+))) Ly = max (1+A)L, (4+2))12x
Lv pV
max 9 9= Ly + Av
Gradient descent with the optimal step-size 7/ = 2/ + Lv) ensures that the stacked parameter vector 0 = (cb WV converges linearly at the rate q, while the Lyapunov gap contracts at the rate q².
Proof.
Lemma B.I shows that, under Assumptions 2 the Hessian satisfies
VVV m pv I pV = minf 2m xx2 mins (1 + Alu).
Lemma B.2 gives a global Lipschitz bound V2V2ll & Lv with by max + A)T, (4 + 2 max)
For epoch t, the update (((+1) = g( - (0(9), g(t) = (6(1), W(() is standard gradient descent Smooth, pv-strongly convex function.
Choosing any stepsize 0 < < < Lvv) gives the classical contraction Nesterov al. 2018
Theorem 2.1.15))
Lv Lv/uv - 2 211((0) - 11
(X((1)) - V <
2 Lv/uv +1
where 0* is the optimal point of V2(0) and V V2(0*).
18

---

Published as a conference paper at ICLR 2026
The same theorem gives the one-step contraction, yielding the geometric bound:
Tv/uV -
||0(0) 1- S
Lv/uv '11010) 0*11
In other words, the stacked parameter vector 0 == (Φ, ( converges linearly with ratio Ly /vvav=1 while the objective gap contracts with ratio Ly/pv-1 ) Lv/pv+I>
+V/
These results together provide a rigorous foundation for the Lyapunov- convergence guarantees stated in the main text.
19

---

Published as a conference paper at ICLR 2026
C PAC-STYLE GENERALIZATION BOUND
This appendix provides the full proof of the PAC-style generalization bound stated in Theorem 3.3 of the main text. We begin with general Rademacher-coe caapnnaccer-coplllit--beeeeeeeee based risk bound for squared loss, then specialize it to the structure of CG-CLR, resulting in bound that is uniform over all model parameters.
C.I FORMAL DEFINITION OF EMPIRICAL AND EXPECTED RISKS
We formally define the empirical quantities used in the theoretical analysis in Section 3.3 let ((G,y) == ( y)² denote the squared loss.
Throughout,
Predictor indueed by CG-CLR. Given parameters (Φ, W.. the prediction of CG-CLR on an input is
h h (a:) == (x))
where z =x) is the hard-assignment rule defined in equation
Expected risk. For a predictor h b, the expected risk under the data distribution D
9 Rucs
D( ((hoor (az), v)]
Empirical risk on the training sample. Given i.i.d. training data {( 11 the empirical risk
R Atrain
(ho,w vt).
Objective minimized during training. CG-CLR minimizes the composite empirical objective
f == REi (d. Wotop) (1 + A) Ralign (Φ, W).
where Rfi measures the prediction error between the codebook predictions and the responses, and Ralign encourages agreement between the proxy predictions and the quantized codebook predictions.
C.2 GENERALIZATION VIA RADEMACHER COMPLEXITY
For any measurable function f. we denote the true and empirical risks as
L(f) (( (((())))
i(f)
Σ (f(xs))))) N
1
where the loss function R:R x R + [0, Y22 max] is 4Y
--ymmax--iishhe Lipschitz in its first argument:
(l(u v) - l(v, v)1 = ((u y)² - (u - 1- = vilu + 2yl S 4Ymax /u v|.
Lemma C.1 Leademacher-basete Risk Bound). Let F be a class of real-valued functions such that sup, lf(x) S Ymax for all f F. Then for any 8 E (0, 1), with probability at least 8 over
i..dd sample of size N, every f satisfies
ln(1/8)
L(f) < L(f) + 8ymax RN(F) +
max
2.V
where
Rx(F)= Eg sup fEF
and E (+1} are independent Rademacher random variables.
20

---

Published as a conference paper at ICLR 2026
Proof.
Let S = (d1,...,dy) dy) with dN = (I,, and define
c(S) = sup lEF
[(f)) - L(f) sup SEF
B(e(f(x))))
1
E (()())).1))
N
By the symmetrization argument (Bartlett & Mendelson
2002
Appendix B)),
N
2
B(e(S)) I E. sup Σ or (f(2.))
fEr N
For each fixed y, define the centered loss
(y(u) f(u. y) - E(O, v).
Then By is max m Lipschitz and satisfies (0) O.
By Talagrand's 's Lemma (Mohri 2018 Lemma 5.7)).
E sup JEF
E os v. (f(z:)) 4Y max
Eg
sup fEF
oo f(x:) = 4Ymax RN(FF
N
N
1=1
Subtracting the constant E(0, y/) does not affect the supremum, so
N
N
Eg sup Σ o, (((((()), vi) Es sup fEF
uu (f(z:))]
JEF 1=1
N
Combining the above yields
B[c(S)] S 8 Ymax Rx)(F).
Now, suppose S' differs from S only in the i-th sample, di - d'. For any fixed f.
[e(s( v.) 1 ( 26)
y
N
Thus, taking the supremum over S.
y2
- (e(S)-((S')
<
max N
Therefore, c(S) satisfies the bounded difference property with constants q Y22 NN.
By McDiarmid's inequality (Mohri 2018 Theorem D.8)), for any 6o 0,
2e )
2N c
Pr(e(S)- B[{(S)) > en) s exp
exp
E1=1 &
Y4 max
Setting the right-hand side equal to ê gives
In In(1/8)
6o y2 max
2NN
so with probability at least 1 8,
1n
c(S) S E[((S)] +
Y2 Yaax
2N
Combining the above bounds, for every f E F,
In(1/8)
L(f) < i(() + 8Y max W (F) + +22
max
2NN
21

---

Published as a conference paper at ICLR 2026
CG-CLR-SPECIFIC DECOMPOSITION C.3 CG-CLR SPECIFIC
We now apply the general bound to our clusterwise model class.
Theorem 3.3 (Generalization bound), For any (0, 1), with probability least 1 6,
dM log d log 2N
8c1 maxjsK ||
(1/()
Rest
Ruain 160yc1 max |1 Il JSK
Y
N
VN
2.N
where co is an absolute constant for standard ReLU MLPs and c1 == YmaxXmax
Proof.
We bound the empirical Rademacher complexity
(H) Eg sup (),W)
Σ os h 2. (2:))
N
where
h (xxv TT (2))
=x(x)) = argmin
x (ix; - x;,))²
35K
Define
g = - (x=+>((²)) C CfX b C c={(2,) + , 03 - R}.
Then H = Cog. g.
Lemma C.2 (Codebook Lipschitz). Assume Vj || S W max.- Define
(1) = x き.
With the discrete metric d(j.j') 1 ( we have
[+) - vxa (1') maax W Wmax d(j.j')-
Thus, each vx is m2X W -ipshhis
For any fixed code vectors (tØÿ). consider
Ry 9) Eg sup ggg
1 x o o(ve))
Σ og
N
Since the map j + x t. is max W max Lipschhtz on the discrete set, the Ledoux Aadoux-Tallgrand contraction lemma (Ledoux & Talagrand 2013 Theorem 4.4) gives
Rv 9) < 2x Wmax Rg)
Allowing the codebook to vary introduces an additional term. By sup-subadditivity. for any fixed (db" TV").
Σ TaxITa (o())) S sup Σ σ x (x))
sup Σ oi xx
sup
2g (mx:)
A.W
Term A is bounded as above, and term B the Rademacher sum over the linear class
A < 2N W W Aw(9),
B = N=N
Dividing by N and taking expectation in yields
Rx Stw (C) + 2X 2X W Wmaxx Rw
22

---

Published as a conference paper at ICLR 2026
Now, for
(x.
+
x
||w,
W Wmax
we have
N
Ry (C) = Egg
sup
oi x v
51 S
N
N
max max Eg
x
(by the Cauchy-Schwarz duality)
1
W max Xmax
where the last incquality is the standard Rademacher complexity bound for a Euclidean ball IP (Mohri (2018 Theorem 11.11)).
Next,
G - fx + Wo(#)} is the hypothesis class implemented by a depth-M - ReLU MLP with d trainable parameters.
By the nearly-tight VC and pseudo-dimension bounds of Bartlett et al. (2019) Theorem 3 and 7), its pseudo-dimension satisfies
Pdim(G) (d logd).
By the standard conversion from pseudo-dimension Rademacher complexity (Mohri 2018 Theo- rem 11.6)).
(g) 2 Pdim In(2N)
dv
d In In(2N)
N
Combining Lemma C.2 with the decomposition
Rx S Rw (C) + 2 W Wmax Rr(G),
and using
W Wmaxx N Xmax
dM dM (nd((2))
(C(()
Rx cu
VN
where co is an absolute constant for standard ReLU MLPs, conclude that, for any 5 E (0, 1), with probability at least 1 5,
(hh s L [hoowv) + SY mYmax
3 Wmax E Mmax + 2X max W max Co / dM In d
+ Y max V
In(1/8) 2N
VN
In particular,
dM log d log 2N
8c1 max, <
w
log(1/5)
Riest S Piaiin + 16c001 max |
+
J
N
/
max
2.N
where 61 := Ymax max: which recovers Theorem
Connection to the training objective. Note that our training objective upper-bounds the empirical prediction risk:
Pirain (b. W) = Ra (o. Watop) < Å(Φ W..
Therefore, Theorem 3.3 also yields a bound terms of the optimization objective:
dM log log(
8c1 max
log (L/()
Heest St + 160 max || Il JSK
V2
N
N
mAx
2NV
23

---

Published as a conference paper at ICLR 2026
D F-TEST DERIVATION
This appendix derives the sequential F-test procedure used to determine the appropriate number of clusters in our model. The test evaluates, at each step, whether introducing an additional cluster leads to statistically significant reduction in the residual sum of squares.
Proposition 3.4 (Sequential F-test). The necessity of adding another cluster is tested sequentially using nested-model F-statistics:
(SSEx - SSFKK+))((pppp
FK-K+1 SSEK+1/(N) (K + () -
and the number of clusters K is selected according to statistical significance a given level a.
Proof.
We consider fixed assignment vector (Z1,..., [ [L..... K]. let
KJW For each cluster j
X, = diag(1s)) (s,) [x1 :.:::::::::: xN] E IR-x((++1), where Sj = {i|z; = f) is the index set for cluster j.
The restricted (null) model with K clusters is then given by
= ((K) +e, X(K) = [x Xx] E NxK(tt)
B(x) = [ W W
where E ~ o
To test whether adding a (K + (1th cluster necessary, we consider new partition of the data in which one more samples are reassigned from their current clusters to form the candidate new cluster. This induces new set of cluster assignments
2' (2) ..... 2'y) [L...., K+1I
where there exists least one such that 2/ K+ ++1 and /
The new partition gives rise to modified block structure for the design matrix, where
X ((K+1) = X |X..... X E (xK++))+)
with
X diag s [x] N for j 1, 1,...
... K+ K+1,
and the updated index set for cluster is defined as Sj== = {i|2] =}}. The extended (alternative) model is now
y X (K te, where ((++)) == [(w() W k++))
TT
This model introduces exactly + additional parameters relative to the restricted model, due to the addition of the new cluster regressor w W++ while allowing other cluster assignments to be updated accordingly.
Let
kkk) == (X (K) (()) be the least-squares solutions for each model, and define
(K) "K()
(K++))== = (X(K+)) (Xx)+)) -IX/T
XXTT (K+1)
SSE := ||u y X(Xx+)
SSEK := X(K)
Since the restricted model is nested within the full model, the standard lack-of-fit statistic is given by (SSEk SSEk+1)/(p 1)
FK- (SN (K + 1) 11) F under Ho, where Ho denotes the null hypothesis that the additional cluster does not provide a significant improvement fit.
24

---

Published as a conference paper at ICLR 2026
Sequential decision rule. The sequential testing procedure proceeds follows: Starting with K = 1 (the global linear fit), we increment K as long as the statistic
FK-K+1
pla/2) p+1, -K-NN((K)))p))
(where the superscript denotes the upper a/ ququnnillle of the F-distribution). The process stops at the smallest K where the null hypothesis cannot be rejected at level a.
The final selected number of clusters,
i RE 1,
N
}
p+1
thus provides a data-driven estimate of the number of linear components supported by the observed data. This sequential rule controls the risk of overfitting by requiring each added cluster achieve statistically significant reduction in residual variance.
25

---

Published as a conference paper at ICLR 2026
E
SIMULATION STUDY
E. LOW-DIMENSIONAL PIECEWISE- FUNCTION
SANEL TNOSOO Fand
85
(a) Linear Decision Tree (LDT)
(b) MoME (S-IMEa)
40
(c) CG-CLR (PROXY)
(d) CG-CLR (CODEBOOK)
Figure 4: Function approximation and Clustering results: LDT, S-IME,, proxy network, regressors
codebook
Table F-test Results under Ho K S ko and H1 : > Ko where p 2, N 200, true K
Ho
SSEH,
SSEH,
F
p-value
K <1
5398.91
139.56
2436.9
0.000
K 2
139.56
1.94
4507.2
0.000
K <
1.94
1.84
3.3
0.038
K 4
1.84
1.80
1.2
0.586
E.2 HIGH- DIMENSIINNSIN GAUSSIAN MIXTURE COVARIATES
The main simulation in Section E. focuses on a two-dimensional, piecewise-linear function with clearly separated clusters. While this setting is useful for visualizing the behavior of CG-CLR, is deliberately benign: the covariate clusters are well separated and the underlying linear experts differ substantially. To assess whether CG-CLR also behaves well in more realistic, noisy settings with overlapping clusters and high-dimensional covariates, we conduct additional experiment based on a Gaussian mixture model in p = 10 dimensions.
Conceptually, this experiment is designed to mimic a regime in which (i) the covariates arise from several correlated Gaussian components that partially overlap in 10 (ii) each component is governed by its linear regression model, and (iii) some regions of the covariate space intrinsically ambiguous, in the sense that both the covariate distribution and the underlying linear functions are very similar across components. In such regions, any purely covariate-based assignment rule (including
26

---

Published as a conference paper at ICLR 2026
675
(a) CG-CLR with K ==1
(b) CG-CLR with K =
CC CLCCICCee
CO-C funden [onil
(c) CG-CLR with =
(d) CG-CLR with K
Figure 5: Function approximation and Clustering results (Ablation): a K
2, ④
3,
K 4.
CG-CLR) necessarily faces ambiguity in cluster labels, but prediction can still remain accurate if the corresponding linear experts are close. The goal of this simulation test whether CG-CLA can (a) recover the cluster structure where it is identifiable from covariates, and (b) accurately reconstruct the component-wise linear mechanisms even under covariate overlap and observation noise.
Data generating process. We consider K = 3 latent regression components with mixture weights T 8. л2 and πæ 2, corresponding unequal cluster sizes. First, latent cluster index E 21, is drawn according to {π, j=11 Conditioned on j. the covariate vector R is drawn from a full-covariance Gaussian component
x (= N(H,, E,),
j == 1,2,3.
where the means μ4 E R10 and covariance matrices Σ₁ E 100 are sampled once at random and then fixed. Each Σj is dense, positive-definite covariance with moderate correlations between coordinates, obtained by drawing a random correlation matrix and rescaling it by random marginal variances.
Gaussian mixture parameters. The randomly generated mixture component parameters used in this experiment are
μ [-0.729, 0.776, 0.865, -0.109, -0.224, -0.485, 0.315, -0.015, 0.928, 0.602],
#12 [- 0.090, 0.602, -0.917, 0.539, -0.994, -0.414, 0.222, 0.826, -0.400, -0.503),
#3 [0.333, 0.975, -0.063, 0.753, 0.832, 0.892, -0.445, 0.039, -0.691, 0.9771.
27

---

Published as a conference paper at ICLR 2026
The covariance matrices are:
0.438
0.045
0.067
0.066
0.036
0.034
0.013
-0.105
-0.086
-0.069
0.045
0.991
0.008
0.020
-0.100
0.044
-0.55
-0122
0.146
0.041
0.067
0.008
0.572
0.023
0.014
0.053
-0.065
-0.099
0.003
-0.551
0.066
0.020
0.023
0.880
0.067
0.009
0.056
0.032
-0,070
0.023
0.036
0.100
0.014
00067
0.285
0.027
0.024
0.007
0.037
0.021
Σ
0.034
0.044
0.053
0.009
0.027
0.412
0.077
-0.933
0.024
-0.04
0.013
0.057
-0.65
0.056
0.024
0.077
0.539
0.116
-0.063
-00188
0. 105
0..14
-0.099
0.032
0.007
-0.93
0.116
0.776
-0.024
0.038
0.086
0.146
0.003
-0.070
0.037
0.024
-0,063
-0.024
0.924
0.122
0. 0.069
0.041
00.51
0.023
0.021
0.004
-0.018
0.038
0.122
0.485
0.754
0.020
-0.174
0.007
0.064
-0.129
0.009
-0.059
-0.067
0.081
0.020
0.311
-0,044
-0.023
0.049
0.078
-0.003
0.006
0.028
-0.026
40010
0.174
0.652
-..28
0.082
0.066
0.023
0.052
0.044
-0.800
0.007
0.020
-0.228
0.430
0.083
00.011
0.051
0.005
-0.118
0.006
0.064
0.049
00082
-0.083
0.911
0.112
0.086
-0.032
0.055
0.003
Σ
-0.129
0.078
0.066
-0.011
0.112
0.994
0.084
0.059
0.075
-0.138
0.009
-0.003
-0.023
0.051
0.086
0.084
0.962
-0.021
-0.093
0.020
0.059
0.006
0.052
0.005
0.032
0.059
0.021
0.382
0.024
-0.133
0.067
0.028
0.044
0.018
0.056
0.075
0.093
0.024
0.418
0.082
0.081
0.026
01.800
0.006
0.003
0.138
0.020
0.133
0.082
0.806
0.402
-0.051
-0.053
-0.085
-0.048
0.011
0.035
-0.032
-0.035
-0.553
0.051
0.659
0.017
0.126
-0.054
-0.019
-0.084
-0.094
0.065
-0.037
0.053
0.017
0.995
0.050
0.014
0.142
0.009
0.122
-0.008
0.037
0.085
0.126
0.050
0.497
0.035
-0.488
-0.052
-0.080
0.014
0.041
0.048
0.054
0.014
0.035
0.949
0.090
0.035
0.030
-0.066
0.039
Σg
0.011
0.019
0.142
-0.048
-0.090
0.542
0.019
0.104
0.080
-0.008
0.035
0.084
0.009
0.052
0.035
0.019
0.532
0.081
-..08
0.016
-0.032
-0.044
0.122
-0.080
0.030
0.104
0.081
0.906
-0.008
-0.008
0.035
0.065
-0.008
0.014
-0.026
0.080
-0.038
-0.008
0.860
-0.029
0.053
-0.037
0.037
0.041
0.039
-0.008
0.016
-0.008
-0.029
0.301
Given the covariates and cluster index, the response is generated by a cluster-specific linear model with additive Gaussian noise:
==)== =x w. + +
0.13
The true regression coefficients wj R 10 and intercepts by sampled at random and fixed
Wi --.306, 0.809, -0.350, -0.283, 0.457, 0.893, -0.526, 0.887, -0.808, 0.163].
b1 = -0.755,
W2 --0.22, 0.657, -0.311, -0.080, -0.661, 00.307, -0.623, -0.658, -0.633, -0.718).
b2 -0.199,
W3 1-.847, -0.747, 0.576, -0.134, 0.460, -0.537, -0.037, 0.322, -0.287, 0.299],
by 0.109.
From this Gaussian mixture model, we draw 600 training samples and 600 test samples, with cluster sizes proportional to the mixture weights (TT, 72, R3) = (1/6, 1/3, 1/2). Using these data, we train CG-CLR with K = 3 under the same proxy architecture and optimization settings in the main simulation, and evaluate both learned cluster assignments and the recovered codebook regressors.
Cluster assignments in a noisy, overlapping regime. Since p = 10. we visualize the learned structure via PCA. O the first two principal components of the test covariates: colored
28

---

cluster Published as a conference paper at ICLR 2026
PCA visua ization
Qusttr [Irue]
Cluster
Cluite [hred]
Cluster (Pred)
Cluster (Pred
PCA1
Figure 6: high-dimensional Gaussian mixture experiment (p 10, K = 3). PCA projection of the test covariates, colored by the true Gaussian component (circles) annotated with the predicted (crosses). In regions where the components are well separated, the predicted clusters align closely with the ground truth; in heavily overlapping regions, assignments follow the local covariate geometry, as expected for covariate routing rule.
circles indicate the true Gaussian component labels, while crosses indicate the predicted cluster assignments from CG-CLR.
Even though the Gaussian components substantially overlap the PCA plane, CG-CLR correctly recovers many of the cluster assignments, especially in the better-ssparated regions. In areas where multiple components are nearly indistinguishable in terms of æ, the model naturally follows local covariate trend and may assign some points to nearby cluster, reflecting the intrinsic ambiguity of purely covariate-bai based assignment.
Prediction accuracy and function values. To examine the effect of such ambiguous assignments on prediction quality, we compare the true denoised function values (*, x w + by- with the codebook predictions x "w., Figure 7 shows both quantities over the same PCA projection; color encodes the function value.
Function Prod Function Volue
PCA1
Figure 7: True denoised function values (x:) (circles) and CG-CLR codebook predictions ã (crosses) on the PCA projection of the test set. Despite some mismatches in cluster assignments in overlapping regions, the predicted function values closely track the ground truth, indicating that many of these assignment errors occur where the underlying linear functions are themselves very similar.
29

---

Published as a conference paper at ICLR 2026
The two fields are visually indistinguishable in most regions, confirming that the codebook regressors provide accurate predictions even under covariate overlap. This supports the intuition that when both the covariates and the underlying linear functions are nearly indistinguishable across clusters, misassignments are unavoidable but also largely harmless for prediction.
Recovery of linear coefficients. Finally, we compare the true regression weights uy with the learned codebook weights úü, for each cluster. The recovered coefficients are
w 0..303, 0.787, -0.346, -0.283, 0.421, 0.854, -0.520, 0.868, -0.819, 0.149),
W2 0-,,188 0.645, -0.315, -0.084, -08644 -0.310, -0.618, -0.638, -0.666, 0.720),
33 = 1.8454, 0.748, 0.577, -0.152, 0.461, - -0.531, -0.045, 0.319, -0.288, 0.278
three clusters, the codebook closely matches the ground-truth coefficients: the relative errerro are approximately 3.3% (cluster 1), 2.3% (cluster 2), and 2.0% (cluster 3), respectively. Thus, even in high-dimensional Gaussian mixture with observation noise, CG-CLR is able to recover the underlying lincar mechanisms to high degree of accuracy.
|wa
30

---

Published as a conference paper at ICLR 2026
F REAL DATASET EXPERIMENT
F.1 DATASET DESCRIPTION
We evaluate our proposed CG-CLA (G-CLR framework on seven diverse real-world regression datasets, as detailed in Table 4 These datasets vary significantly in size and feature composition, allowing comprehensive evaluation across different predictive scenarios. The selected datasets span several domains, including materials science (CONDUCT), real estate (HOUSING), transportation (BIKE), energy (ELECTRICAL, PLANT), agriculture (WINE), and civil engineering (CONCRETE).
Table 4:
Overview of the seven real- world regression datasets used in our experiments. Columns give the total number of Samples and the counts of Categorical and Numerical input features. Dataset abbreviations
CONDUCT (supereonduativity:: [Hamidich] 2018), HOUSING (California housing; Kelley Pace Barry
1997), BIKE (bike-sharing; |anaace (2013), ELECTRICAL (electrical-gr) stability; Arzamasov 2018).
PLANT (combined-cycle power plant: Tfekci & Kaya 2014 WINE (wine quality: Cortez & Reis 2009), and
CONCRETE (concrete strength; Yeh 1998).
Dataset
Samples
Categorical
Numerical
CONDUCT
21,263
HOUSING
20,640
0
8
BIKE
17.379
7
6
ELECTRICAL
10,000
0
12
PLANT
9,568
WINE
4,898
CONCRETE
1,030
0
8
F.2 HYPERPARAMETER TUNING
Each method underwent rigorous tuning via grid search to optimize performance on validation folds within nested vross-validation scheme. Tables summarize the extensive hyperparameter search grids employed for the competing methods XGBoost (XGB), CatBoost (CAT), DC, CART, and S-IMEa. For LDT and PILOT, the maximum tree depth was set to [log2 /Nee/(10)) + 10))
(b) CAT
(a) XGB
Hyper-parameter
Range
Hyper-parameter
Range
n_estimators
[1, 1000]
estimators
[1, 1000]
leamiig_rrat rate
{0.001, 0.01, 0.13
learning rate
(0.001,0.01.0.1)
max_depth
{3, 4, 5, 6, {0.5, 0.75, 1.0) 100
max depth
53,4,5,6,7} {0.5, 00,5,0.75,110 1.0]
subsample
subsample
hitllysstoppinggnooee rounds
(c) DNN
(d) DC
Hyper-parameter
Range
Hyper-parameter
Range
hidden nodes
(32, 64, 128, 256) (2, 4}
À iterations
{0.01, 0.1, 1, 10, 100) {0.001, 0.01, 0.13 [1. 10000]
hidden layers
dropout
{0.1, 0.2, 0.3. [1, 10000) 1000
iterations
early stopping patiences
1000
carly stopping patiences
(e) S-IME.
Hyper-parameter
Range
(f) CART
1,
Hyper-parameter
Range
X
10
10
max max_depth
[1. [loga (Nrr 10)
iterations
[1, 10000
carly stopping patiences
1000
Table 5: Hlper-paramete tuning grids for all methods.
31

---

Published as a conference paper at ICLR 2026
F.3 CG-CLR HYPERPARAMETER SETTING
For CG-CLR, several hyperparameters were uniformly set across all datasets, as shown in Table 6 We selected a relatively simple proxy network architecture with three hidden layers (64 neurons each), dropout regularization (rate 0.2), and used Adam optimizer with learning rate of 0.001. The alignment weight (A) was consistently set to I, reflecting balanced importance between fitting and alignment. Training employed early stopping with patience of 1000 epochs to avoid overfitting.
Table 6: CG-CLR hyper-parameett kept fixed across all datasets.
Hyper-pane pareme
Value proxy network hidden layers
batch size
256
(64, 64, 64)
proxy network dropout
0.2
alignment weight
1
learning rate
0.001
optimizer
Adam
iterations early stopping patiences
10000
1000
F.4 PERFORMANCE RESULTS
Performance results presented in Table 7 systematically compare CG-CLR against both powerful black-box methods and various cluster-based linear regression alternatives. Results organized into three coverage-based clusters (small, medium, and large) defined by the number of clusters (K). Notably, CG-CLR consistently achieves the lowest RMSE among all cluster- based models, and frequently performs comparably to or better than leading black-bor box models, emphasizing its robust generalization and interpretability across diverse real-world datasets.
Table Test RMSE mean std. on seven regression datasets (CONDUCT, HOUSING, BIKE, ELECTRICAL,
PLANT, WINE, CONCRETE; Hamidich] [2018] Kelley Pace Barry 1997 Fanace- T 2013 Arzamasov 2018
Tlekci & Kaya 2014, Cortez& & Reis] 2009] Yeh 1998). Model blocks. (i) Black-box baselines: RF (Random
Forest), XGB KGBoost:[Chees & Guestrin 2016). CAT (CatBoost: Prokhorenkova et al. 2018). and DNN. (iiiiv
Cluster-wise models with three predetermined cluster budgets K: (s) "small" coverage K NN.. (p 1)], (m)
"medium" coverage K [5p + 5) and ()) "large" coverage LLVt./ (10p + 10) Symbols follow
the literature: DC (Siahkamari et al. 2202), MLR* (mixed-lineal regression (Pal et al. [2022) with post-hoc neighborhood assignment). CART, PILOT [Raymaekers et al. (2024), LDT (Ahmed et al. 2018). S-IME, [[smail
al. 2023), and our proposed CG-CLR variants (CG-CIR,,, CG-CLR CG-CLR, and CG-CLRg, where CG-CLR, sets K K= | (20p 2) Lower RMSE is better; the best score for each black highlighted
bold.
Model
CONDUCT
HOUSING
BIKE
ELECTRICAL
PLANT
WINE
CONCRETE
RF
0.81
0.511 0.008
56.45
1.19
0.0121 1: 0.0002
3.466
0 0.099
0.627 1: 0.025
5.381 4 0.418
XGR
0.13
0.467 = 0.008
0.81
0.0080 + 0.0001
3.200
0.083
0.634
+ 0.025
4.507 4 0.340
CAT
9.69 0.14
0.443
= 0.007
44.99
0.66
0.0068 4 0 0000001
3.162
4: 0.116
0.632
4 0.024
4.226 4 0.360
10.44 0.20
0.505
= 0.007
42.23
1.01
0.0070 + 0.0002
3.785
: 0.100
0.670 + 0.024
4.034 + 0.372
11.66 : 0.59
0.597
0.024
55.36
= 0.73
0.0119 1 0.0005
3.746
0.106
0.651 1 0. 0.028
5.654 1 0.427
MLR
34.08
0.19
1.181
0.010
178.26
4 1.44
0.0348 4 0.0005
13,800
4: 0.170
0,835 4: 0.018
15.263 0.445
S-IME
12.44
: 0.28
0.591
= 0.034
24.27
1.68
0.0094
1 0.0007
4.276
0.091
0.705
1: 0.023
7.021
0.0.334 0.436
CG-CLR,
10.49 = 0.11
0.494
0.009
40.86
1.04
0.0061 1 0.0002
3.831 0.111
0.669 : 0.028
4.919 1 0.382
MLR
34.06
0.20
1.171
0.009
180.46 1.47
0.0363 + 0.0005
16.393 +
0.868 0.036
15.248
0.450
S-IMFJ.
12.65
= 0.26
0.578 = 0.068
55.82
2.33
0.0095 1 0.0005
4.172
2 0.069
0.703 4 0.023
6.553 + 0.378)
CG-CLR...
10.50
40.85
0.0062 + 0. 0.0002
3.627
0.118
0.689
0.082
6.066
MLR;
33.86
0.19
1.164
0.011
178.24 4 1.43
0.0348
0,0005
13.797
0.170
0.835
9.018
15.243
CART
16.71 2 0.33
0.652
= 0.014
121.32
2.09
0.0212 1 0. 0.0002
4.261
0.093
0.745
1: 0.024
9.569 1 0.541
15.52 0.46
0.821
0.0332
4.774
0.756
0.022
14.530
LDT,
14.49 1..99:2.44
0.673
= 0.149
0.663
0.0151
1 0.0004
4.119
1:19::00 0.091
0.708 1 0.021
6.264 + 0.329
S-IMEALE
0.565
0.011
57.07
0.0085
4.221
0.704
0.023
9.182 7.704
CG-CLR;
10.56
0.13
0.491
= 0.012
41.24
1.00
0. 0063
1 0 0002
3.624 1: 0.19
0.864
1: 0.025
5 36365 1 0.367
CG-CLR
10.71
0.11
0.009
0.0066
0.028
6.311 0.443
F.5 COMPARISON WITH GAM BASELINES
Generalized Additive Models (GAMs) represent orthogonal approach to balancing accuracy and simplicity: instead of clustering covariates into local linear experts. they rely on additive modeling to achieve strong predictive performance. Among them, Node-GARM (Chang al. 2022 is
32

---

Published as a conference paper at ICLR 2026
ofate-of-the-ar GAM variant that combines tree-based node splitting with additive terms, and thus serves as competitive baseline distinct from cluster-oriented methods.
To ensure fair comparison, evaluate CG-CLR against Node GA M across all seven datasets. For each dataset, we report: (i) the minimum number of clusters K at which CG-CLR ties with Node- GA²M (their 95% % Cls overlap), and (ii) the minimum K which CG-CLR strictly outperforms Node-GA (its 95% CI lies entirely below Node-GA If CG-CLR neither ties nor wins at any tested K, we denote this by" "--"
Table 8: Comparison against Node-GA MM:: RMSE (95% CI) and the minimum K required for CG-CLR tie or outperform.
Dataset
Node-GA'Mr RMSE
Tie: K, RMSE
Win: K, RMSE
CONDUCT
[12.558, 12.782]
3, [11.738, 11.91]]
HOUSING
[0.495, 0.503]
8, [0.502, 0.512]
BIKE
[53.70, 55.16)
5. [51.05, 52.97]
ELECTRICAL
[0.009, 0.009]
4, [0.009, 0.010]
5, [0.008, 0.008]
PLANT
[3.989, 4.073]
2, [3.982, 4.059]
3, [3.808, 3.908]
WINE
[0.677, 0.695]
24, [0.688, 0.709]
24, [0.652, 0.676]
CONCRETE
[4.634, 5.014]
15, [5,010, 5.449]
The results reveal two distinct regimes. On HOUSING and CONCRETE, CG-CLR already matches Node-GA2M 2 at modest cluster counts, demonstrating comparable accuracy to a strong additive model. On the remaining five datasets (CONDUCT, BIKE, ELECTRICAL, PLANT, and WINE), CG- CLRR surpasses Node-GA2M, with relatively small K, highlighting its ability to reach or exceed state-of-the-e additive modeling performance while controlling complexity through clustering.
F.6 CASE STUDIES UNDER COMPETITIVE K
To better illustrate the interpretability benefits of CG-CLR when it achieves accuracy on par with strong additive models, we provide case studies for the two datasets where CG-CLR matches Mode-GAAM at relatively small K: HOUSING (K=8) and CONDUCT (K=3), These examples highlight how competitive accuracy can attained while still surfacing domain-specific, interpretable cluster patterns.
HOUSING (K= 8). T0 the clusterwise regression coefficients and feature means, respectively. Cluster corresponds to a high-occupancy rental market, with property values driven by the number of bedrooms. Cluster 4 represents affluent suburban districts, where high income and room counts dominate, while additional bedrooms are penalized. Cluster 6 captures peripheral agricultural towns with low income but upward sensitivity income and population changes. These patterns demonstrate that at K = 8, CG-CLR not only ties the performance of Node- Aade-GA* 2aee-GAMM PM but also yields region-specific linear models that are naturally interpretable.
Table Clusterwise Linear Coefficients Table (HOUSING).
Cluster idx
Medlne
HouseAge
AveRooms
AveBedrms
Population
AveOscup
Latitude
Longitude
-0.05
0.017
-0.06
-0.05
-22c-6
0.02
-0.11
0.01
-0.06
-0.008
-149c-6
-0.03
.0.03
0.016
.0.06
.0.99
-150e 6
-0.04
.0.01
0.10
0.18
0.010
0.12
-0.68
-750-6
-0.03
-0.19
0.13
-0.04
0.008
-0.05
.0.34
-50e-6
-0.03
0.08
-0.10
0.18
0.001
158c-6
0.00
-0.12
-0.14
0.15
0.022
0.07
0.00
-166e-6
-0.02
0.15
-0.13
CONDUCT (K 3). Table [1 presents the estimated coefficients as increases from 1 to Distinct patterns emerge small K. At K 2, an age-based split appears, separating young and old specimens, with the "Coarse Aggregate" effect flipping sign across groups. At 3, clear "Superplasticizer effect" arises, isolating cluster with exceptionally high superplasticizer weight and confirming its strong interaction with water and cement. At K = 4, long-term curing benefits are exposed, as aged specimens form distinct group with delayed strength gains. These observations show that small K values already uncover meaningful material science insights while achieving accuracy competitive with Mode-GAPM.
33

---

Published as a conference paper at ICLR 2026
Table 10: Clusterwise Feature Mean Value Table (HOUSING).
Cluster idx
Medlne
HouseAge
AveRooms
AveBedrms
Population
AveOccup
Latitude
Longitude
4.17
29.18
6.08
1.21
1351.26
2.65
35.74
-119.845
27.70
1,05
1571.03
3.69
-119,833
3.23
26.50
4.96
1,08
1485. 59
2.86
35.91
-119,829
4.39
28.28
1,05
1582.74 74
2.95
34.87
-119.181
4.97
29.46
5.81
1,09
1325.35
2.67
35.59
-119,399
4.17
30.01
5.53
1.10
1461.98
2.90
35.77
-120.057
2.64
28.05
5.23
1.11
1357.68
3.22
36.14
-119,529
4.45
29.92
5.40
1,07
1369.02
3.05
35,32
-119.487
Table 11: Qualitative interpretability changes as K increases (CONDUCT).
Cluster idx (K)
Cemeent
Blast Purnace Slag
Ply Ash
Water
Superplasticier
Course Aggregatc
Hine Aggregate
Age
(1)
0,100
0.093
0.069
-0.230
0.253
0.003
00,05
0.118
(2)
0.105
0.068
0.088
-0.175
0.884
-0.035
-0:029
0.093
0.073
0.024
-0.148
0,158
0.009
0.108
0.035
(
0.029
-0.005
0,058
-9.155
0.743
-0,088
-0.054
0.046
(3)
0.011
0.060
0.016
-0.241
1,85
40,000
-0.067
0,086
-0.064
0.047
-0.074
0.034
0,075
(4)
-0024
0.033
0.021
(4)
0.033
0.079
.0049
-0.237
0.904
.0.054
00.135
0.133
((4)
0.031
0.040
0.061
0.070
0.790
-0,450
-0,072
0.041
These case studies show that under competitive cluster counts, CG-CLR not only achieves parity with advanced additive models but also yields interpretable, Nomain-relevant partitions that provide insights into housing markets and materials science.
F.7 COMPUTATIONAL COST ANALYSIS
To complement the accuracy comparisons, we analyze computational efficiency in terms of wall-clock time and peak memory. We compare our CG-CLR framework against two classical cluster-oriented baselines: (i) Dense MoE- soft- soft-gated
cexture-of-Experee with linear experts (Jacobs et al. [1991).
and (ii) EM-CLR* an EM-based clusterwise linear regression (DeSarbo & Cron 1988) with
nearest-centroid routing at test time. All methods use the same expert budget K N/(10p (10pp+100)) 10) and identical tuning protocols (20 independent runs per dataset).
Baseline Performance. Table 12 reports test RMSE with 95% confidence intervals across seven datasets. CG-CLR consistently achieves the lowest RMSE on every dataset, while maintaining clear cluster-level coefficients (like EM-CLR* and markedly lighter computational cost than Dense MoE.
Table 12: Test RMSE with 95% confidence intervals across seven datasets.
CONDUCT
HOUSING
ELECTRICAL.
PLANT
CONCRETE
(12.56, 13,00)
10558, 0.561]
158.91, 67.03)
0.010]
(4:388, 4,535]
[0,94, 0.717]
[7.45,, 8.513]
EM-CLR CLR"
13383, 34.04]
[1004, 1.082)
1178.92, 179.69)
(00SS, 0.035]
113.84, 14.002
10.44, 0.861]
115.356, 15,800]
CG-CLR
[10.50, 10.62)
[0.485, 0.497]
[40.77, 41.71]
[0,006, 0,006]
[3.573,3.675 3,675]
(0,622, 0.676]
5,8337
Computational Analysis. We further benchmark CG-LLR against Dense MoE and EM-CLR* on three representative datasets (CONDUCT, ELECTRICAL, CONCRETE) covering a 20x span in sam- ple size. Wall-clock time (sec/100 epochs) and peak memory (MiB) were measured over 20 indepen- dent runs and reported as 95% confidence intervals. Regressor budgets K E (1,4, 16, 64, 256, 1024) probe scaling with respect to the number of regressors.
Overall, the results demonstrate three main trends. First, CG-CLR maintains a tight efficiency bound: across all expert budgets, its wall-clock time and memory usage remain within about 1.7x those of EM-CLR* # while consistently achieving superior accuracy. Second, CG-CLR scales much more gracefully with the number of experts compared to Dense MoE. Because the forward pass uses O(K) argmax routing while the backward pass is ((1) in K, the training time increases by only about 15% from K to K = 256. By contrast, Dense MoE incurs more than a twofold increase due to its O(K) gradient cost, and this is also reflected in its peak memory consumption. Finally, with respect to dataset size, wall-clock time for CG-CLR grows proportionally to the input scale: moving from 1. 1,030 samples (CONCRETE) to 21,263 samples (CONDUCT) increases runtime by about 10x, which matches the growth in per-sample compute. Memory usage, however, grows only modestly with the number of samples and features, and remains essentially flat with respect K.
34

---

Published as a conference paper at ICLR 2026
Table 13: Scaling w.r.t K
Wall-clock time (sec/ / 100 epochs) with 95% confidence intervals.
Dataset / Model
K=1
K=4
K=16
K=64
K=256
K=1024
CONDUCT
Dense MoE
[11.0, 11.1]
[11.2. 11.3]
[11.4, 12.0]
[15.1. 16.4]
[27.6. 28.9]
[75.1. 77.2]
EM-CLR"
[11.1. 11.3]
[10.8, 11.2]
[11.1, 11.3]
[10.6, 11.3]
[11.0, 11.4]
[11.2, 11.5]
CG-CLR
[11.5, 11.8]
[11.0 11.3]
[11.3, 11.6]
[11.5, 11.7]
[11.9, 12.1]
[12.5, 12.91
ELECTRICAL
Dense MoE
5.6)
[5.4,
[5.9, 6.3]
19.7, 10.0]
[19.7, 23.3]
[53.7, 71.4]
EM-CLR
[5.1,
[5.0,
[5.2, 5.3]
[5.2, 5.31
[5.2, 5.3]
[5.4, 5.6]
CG-CLR
[.66
[5.5, 5.6]
5.66
[5.3.
[5.4, 5.5]
[6.2, 6.4)
CONCRETE
Dense MoE
[1.0, 1.11 00.7,0.71 [0.7, 0.8]
[0.7, 1.0]
[2.0, 2.0]
[4.4, 4.8]
[13.4, 17.2]
[49.7, 68.2]
EM-CLR
0.77 [0.7, 0.8]
[0.7, 0.71
[0.6, 0.71
[0.7, 0.71
[0.7, 0.71
CG-CLR
[0.7, 0.9]
[0.8. 0.9]
[0.8, 1.0]
[0.8, 0.9]
Table Scaling w.r.t
Peak memory (MiB) with 95% confidence intervals.
Dataset / Model
K=1
K=
K=16
K=64
K=256
K=1024
CONDUCT
Dense MoE
[50.0, 50.0]
[50.0, 50.0]
[50.0, 50.0]
[50.3, 50.4]
[52.8, 52.8]
[71.0, 71.1]
EM-CLR
[48.2, 48.2]
[48.2, 48.2]
[48.2, 48.2]
[48.2, 48.2]
[48.2, 48.2)
[48.2, 48.2]
CG-CLR
[50.0, 50.0)
[50.0, 50.0)
[50.0, 50.0]
[50.0, 50.00
[50.0, 50.0)
[50.0, 50.0]
ELECTRICAL
Dense MoE
[2.4, 2.4]
[2.4, 2.4]
[2.5, 2.6]
[3.0, 3.1]
[6.8, 7.0]
[25.8, 26.0]
EM-CLR
[0.5, 0.5)
0.55,
[0.5, 0.5)
[0.5, 0.5]
[0.5, 0.5]
[0.5, 0.5]
CG-CLR
[2.3, 2.3]
[2.3, 2.3]
[2.3, 2.3]
[2.3, 2.3]
[2.3, 2.3]
[2.3, 2.3]
CONCRETE
Dense MoF
[0.6, 0.6]
[0.6, 0.6]
[0.8, 0.8]
[1.9, 2.0]
[6.8, 6.91
[25.9, 26.0]
EM-CLR
[0.3, 0.31
[0.3, 0.3] 0.3]
[0.3, 0.31
[0.3, 0.3]
[0.3, 0.31
[0.3, 0.3)
CG-CLR
[0.4, 0.5]
[0.5. 0.5]
0.5)
[0.5. 0.5]
[0.5, 0.5]
[0.5, 0.5]
F.8
HYPERPARAMETER SENSITIVITY
We conducted sensitivity analysis on the alignment hyperparameter À using CG-CLR with large cluster coverage (K = / ((1pp + 10)]). Results reported in Table 15 illustrate stable performance across varying values of X, with optimal ncar-optimal performance consistently observed À 1. This robustness confirms the practicality of the default choice of X = and highlights the stability of CG-CLR's predictive accuracy and simplicity balance. Moreover, the observed stability under moderate deviations of À is consistent with our theoretical analysis, which requires sufficiently small 1/) ensure valid expert assignment.
Table 15: Test RMSE with 95% confidence intervals across seven datasets (CONDUCT, HOUSING, BIKE,
ELECTRICAL PLANT, WINE, CONCRETE; Hamidich 2018 Kelley Pace & Barry 1997 Fanace-TT 2013
Arzamasov 2018
Tíekei & Kaya 2014 Cortez & Reis 2009: Yeh 1998). Sensitivity analysis w.r.t À under
K Nor (10p 10)]
Dataset
À = 0
A = 1
À == 2
X ==
CONDUCT
[10.45, 10.60]
[10.50, 10.62]
[10.50, 10.63]
[10.59, 10.73]
HOUSING
[0.492. 0.500)
[0.485, 0,497]
[0.491. 0.501]
[0.490, 0.500]
BIKE
[41.07. 41.93]
(40.77.41.71
[41.23, 42.27]
[41.11, 41.79]
ELECTRICAL
[0.006. 0.006]
[0.006, 0.006]
[0.006. 0.006]
[0.006, 0.006]
PLANT
[3.583, 3.683]
[3.573, 3.675]
[3.588, 3.682]
[3.568, 3.674]
WINE
10.652, 0.6761
[0.652, 0.676]
[0.654, 0.678]
[0.657, 0.6871
CONCRETE
[5.301, 5.731]
[5.193, 5.5371
[5.501, 5.851]
[5.546, 5.908]
To examine how the stability of learned coefficients varies with the number of clusters K, we conduct empirical study the CONCRETE dataset in two complementary settings:
35

---

Published as a conference paper at ICLR 2026
(i) Random Initializations. For fixed training split, CG-CLR was trained with T 10 different random seeds. We computed the standard deviation each assigned coefficient across runs and averaged over all data points, using the following definition:
5-10) =/
1 N
1 T
σ
(where W.
T-1
This quantifies the variability of learned coefficients under different random initializations.
(ii) Data Splits (10-fold), further trained CG-CLR with 10-fold splits, each time fitting on 90% of the data and evaluating the full dataset. For the 10-fold splits, the same formula applied across models trained on each fold. This procedure simulates the effect of sampling variation and reflects robustness in practical deployment.
Table 16 reports the average standard deviations (STD) of coefficients across both settings. For K I, the coefficient variance is naturally low, as expected for a global linear model. As K increases, the variance remains moderate, indicating that CG-CLR yields stable coefficient estimates both within fixed dataset and across data splits.
Average coefficient variability (STD) across random initializations and 10-fold splits on the CON-
CRETE dataset.
K
Cement
Blast Slag
Fly Ash
Water
Superplasticizer
Coarse Agg.
Fine Agg.
Age
Random Init. (STD)
0.034
0.038
0.050
0.122
0.186
0.029
0.038
0.002
2 0.027
0.030
0.042
0.153
0.357
0.025
0.027
0.048
3
0.019
0,037
0.043
0.094
0.460
0.020
0.014
0,048
4
0.022
0.034
0.051
0.083
0.228
0.035
0.020
0.053
5
0.024
0.032
0.052
0.096
0.366
0.030
0.025
0.057
0.017
0.034
0.052
0.094
0.330
0.035
0.026
0.040
7
0.023
0.033
0.059
0.082
0.337
0.037
0.024
0.037
8
0.018
0,035
0.052
0.098
0.411
0.039
0.026
0,034
9
0.023
0.034
0.056
0.115
0.336
0.042
0.030
0.034
10
0.019
0.037
0.060
0.123
0.390
0.040
0.027
0.032
10-fold Splits (STD)
0.016
0.019
0.029
0.088
0.350
0.018
0.021
0.010
2 0.036
0.037
0.047
0.117
0.388
0.037
0.034
0.092
3 0.020
0.038
0.053
0.109
0.396
0.031
0.021
0,060
4
0.018
0.027
0.056
0.079
0.343
0.023
0.024
0.068
0.018
0.032
0.058
0.109
0.396
0.026
0.024
0.038
6 0.020
0.037
0.056
0.113
0.405
0.038
0.023
0.042
7 0.019
0.032
0.055
0.107
0.405
0.041
0.026
0.037
8 0.020
0.040
0.056
0.109
0.370
0.041
0.028
0.036
0.022
0.032
0.058
0.105
0.358
0.039
0.029
0.033
10
0.020
0.037
0.057
0.103
0.366
0.045
0.027
0.035
We also examined the average coefficient magnitudes (absolute values) to contextualize the scale of learned parameters. Table Table repoorts results under both random initialization and 10-fold splits, alongside input feature statistics for reference. Across both settings, magnitudes remain stable K increases, supporting the robustness of CG-CLR against coefficient variability.
Overall, these results demonstrate that CG-CLR maintains stable coefficient magnitudes and mod- erate variance even as the number of clusters K increases, supporting the method's robustness and practical reliability.
F.9 EMPIRICAL STUDY OF ASSUMPTION
[ AND ASSUMPTION 2
To assess whether the regularity conditions in Assumptions I are reasonable in our experimental regime, conduct empirical diagnostic study the CONCRETE dataset. We track the Jacobian the proxy network (Assumption) T and the curvature the alignment loss (Assumption throughout training.
36

---

Published as a conference paper at ICLR 2026
Table 17: Average coefficient magnitudes (absolute values) for the CONCRETE dataset.
K
Cement
Blast Slag
Fly Ash
Water
Superplasticizer
Coarse Agg.
Fine Agg.
Age
Random Init.
(Absolute)
1 0.077
0.055
0.053
0.262
0.408
0.039
0.045
0.110
2
0.076
0.059
0.045
0.228
0.517
0.026
0.035
0.160
0.067
0.050
0.041
0.274
0.433
0.039
0.040
0.142
4 0.068
0.049
0.047
0.271
0.473
0.041
0.040
0.110
5
0.063
0.048
0.057
0.276
0,540
0.041
0.043
0.096
6 0.060
0.046
0.064
0.250
0,526
0.038
0.041
0.102
7 0.061
0.047
0.065
0.259
0.478
0.044
0.044
0.099
0.059
0.045
0.066
0.250
0.589
0.047
0.039
0.090
9 0.052
0.046
0.070
0.237
0,595
0.041
0.042
0.090
10
0.053
0.046
0.065
0.220
0.630
0.049
0.041
0,080
10-fold Splits (A (Absolute)
1 0.074
0.051
0.042
0.272
0.409
0.028
0.036
0.108
2
0.071
0.053
0.037
0.265
0.418
0.031
0.038
0.157
0.071
0.054
0.049
0,266
0.395
0.028
0.035
0.121
4 0.071
0.055
0.062
0.247
0.399
0.036
0.038
0.126
5
0.067
0,053
0.063
0.257
0.467
0.034
0.039
0.104
6 0.060
0.050
0.070
0.257
0,534
0.039
0.047
0.095
0,061
0.048
0.063
0.272
0.533
0.043
0.043
0.102
0.060
0.047
0.064
0,258
0.523
0.046
0.043
0.103
9 0.056
0.044
0.066
0.239
0.572
0.045
0.042
0.092
10
0.054
0.046
0.066
0.237
0.608
0.050
0.044
0.093
Assumption 1 Jacobian lower bound and Lipschitz constant. Every 10 epochs, sample the
proxy network Jacobian with respect to its parameters at all training covariates (xr,), Specifically, for each input æ; and current parameter vector (((), we form the Jacobian
Jat (xx:) = aw (gte)( E R(++1)xd
Osi1))
compute its singular values, and record the smallest and largest, Omin (Jo(s) and Fmax (Joce) (xx)). We then aggregate these over all training points,
m = min Tmin (Jate) 1515N
max maxx (Jacos (x:)), 1SISN.
which serve as empirical estimates of the Jacobian lower bound and the Lipschitz constant in Assumption 1 Storing these statisties every 10 epochs yields trajectory {m (t) i J:: we summarize their distribution over training by histograms in Figures [
As shown in Figure Sa. the estimated lower bound m (P)
is consistently larger than 1.0. indicating that the proxy network's Jacobian numerically well conditioned and bounded away from zero
throughout training. Figure 8b shows that the largest singular value f((() remains below 25.0, so the Lipschitz constant does not explode and the gradient field stays stable. Taken together, these observations support the view that the Jacobian of the proxy network satisfies uniform lower bound and finite Lipschitz constant in the regimes considered in our experiments.
A ssumption D strong convexity and smoothness of the alignment loss. Assumption concerns the curvature of the alignment loss with respect to the codebook parameters. Recall that for cluster j. the alignment loss is a least-squares term whose Hessian is proportional to the Gram matrix of the covariates assigned to that cluster. Coneretely, let x RayXP denote the design matrix of covariates assigned to cluster at epoch I, and let
(t) x H 2
be the corresponding Hessian. Every 10 epochs, we compute the smallest and largest eigenvalues of each 11 and summarize them
(t) = min Amin (H(.)). 1515K
I(() = max Amax (H!)0)
1535K
37

---

Published as a conference paper at ICLR 2026
Minimum Singular Value jacobian emillgix))
Maximum Singular Value of the Jacobion Omsxlgla))
50
m,
12
10
count
30
count
8
20
10
2
0
0.0
0.2
0.4
0.6
0.8
1.0
1.2
0.0
2.5 5.0
7.5 10.0 12.5 15.0
(a) Minimum singular value amin (Jo(z)) (mi.).
(b) Maximum singular value Fmax (Ja(z)) (L.).
Figure 8: Empirical singular-valus spectrum of the proxy-netword Jacobian the CONCRETE dataset, collected every epochs over course training. distributions indicate stable Jacobian lower bound and a non-exploding Lipschitz constant, providing empirical support for Assumption
which provide empirical estimates of the As before, we record these quantities throughout training and visualize their distributions in Figures
and smoothness constants in Assumption
Sggure9asshow shows that the estimated strong-convexity constant (!() is always larger than 6.0, indicating that each cluster's alignment objective is uniformly and numerically strongly convex. At the same time, Figure demonstrates that the smoothness constant (()) concentrates around values below approximately 1300.0 and does not diverge, confirming that the curvature of the alignment loss remains well controlled. These diagnostics suggest that the alignment loss is both strongly convex and smooth the empirical regimes where CG-CLR trained.
Minimum Singular Valuee of the A linmmen Loss
Maximum Singular Value of the Alignment Loss
35
80
30
70
25
50
count
20
count
15
10
20
0
6
10
200
600
1000
1200
(a) Minimum eigenvalue of (( (p))
(b) Maximum eigenvalue of H (L).
Figure 9: Empirical curvature the alignment loss on the CONCRETE datasct, computed every 10 epochs. The estimated congexcnyy constant μ remains safely bounded away from zero, while the smoothness constant L stays finite, supporting Assumption
Overall, the Jacobian and curvature statistics in Figures 9 indicate that the regularity conditions required by our convergence analysis are not merely abstract assumptions: in a representative real dataset, the proxy network and alignment loss behave in numerically well-conditioned manner, with bounded Jacobian spectrum and stable strong convexity and smoothness parameters.
F. EMPIRICAL STUDY ASSUMPTION
Many classical clusterwise linear regression studies includdng both traditional and more recent mixed linear regression methods (Kong al.] 2020) Ghosh & Mazumdar) 2024 r0l4yyrely the assumption that clusters are well separated, so that each local linear regressor can be correctly recovered. To assess the validity of this assumption in practice, measure the empirical cluster-
38

---

Published as a conference paper at ICLR 2026
separation gap after training, defined
min x (Ørj w jf3
which captures the minimum predictive discrepancy between distinct regressors over the training
samples. Table 18 reports the 95% confidence intervals of across 20 independent runs for all seven datasets.
Table 18: Empirical ctuster-sepaationn gap with 95% confidence intervals over 20 runs.
Dataset
(95% CT)
CONDUCT
[9 x 10 3 10 $]
HOUSING
[8 x 10-7, 2 10
BIKE
[2 x 10
10
ELECTRICAL
[2 X 10 1x= 5x 10-
PLANT
[1 10 3 10
WINE
[8 X 10 3 3 10
CONCRETE
X 10
10
Across all datasets, the empirical gaps A are extremely small, typically A < 10-1 Despite this near-zero separation, CG-CLR continues to exhibit monotone descent and achieves strong test RMSE, demonstrating that its convergence and predictive performance do rely strong cluster separability. These results indicate that the method remains empirically robust even in the regime where clusters strongly overlap, thereby relaxing one of the key assumptions often imposed in prior work.
39

---

Published as a conference paper at ICLR 2026
G LLM USAGE
Large Language Models (LLMs) were used in limited capacity to aid in the preparation of this manuscript. Specifically, LLMs were employed solely for writing support tasks such as grammar checking, polishing sentence structure, and improving readability of author-written drafts. They were involved in research ideation, methodological design, data analysis, or result interpretation. The authors take full responsibility for the content of this paper.
40


Reviews:
[{"review_id": "FzRAybs2bO", "reviewer": "Reviewer_B3vF", "summary": "The paper studies the Clusterwise Linear Regression (CLR) problem in which the goal is to find $K$ regressors for a given dataset such that for each point there is one regressor out of the $K$ which has a low loss prediction. Previous methods either used optimization to find the $K$ linear regressors but provided no way to assign a new unlabeled point to a regressor during inference, or leveraged a mixture of experts or decision trees but did not have good convergence. In this work, the authors try to address these problems by introducing a proxy network which predicts a good regressor for each input point along with a dual loss which optimizes the network as well as a codebook of candidate regressors simultaneously. Under some smoothness, convexity and cluster separation assumptions, the authors prove convergence as well as generalization error bounds for their CG-CLR method. Included experiments on synthetic and real datasets show performance gains as compared to some previous approaches.", "questions": "1. In the CG-CLR algorithm, why are the proxy network and the codebook updated separately and not together?\n2. The empirical risk mentioned on page 6 should be formally defined (or a reference to the definition in the appendix included).\n3. Assumption 4 seems unclear. The condition that $d \\gg K(p+1)$ can trivially be achieved by adding dummy parameters. The statement of the assumption should be made more formal.\n4. What are CG-CLR (Proxy) and CG-CLR (Codebook) mentioned on line 408?", "limitations": "", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "1. The paper proposes a new technique for CLR which optimizes a proxy predictor network whose predictions at test time can be used to assign each point to a candidate regressor. \n2. The paper provides theoretical guarantees under some reasonable assumptions.", "weaknesses": "1. The proposed method hinges on the assumption that a good proxy network can be trained, however this means that $W_\\phi(\\textbf{x}_i)^\\textsf{T}\\textbf{x}_i$ is a good label predictor. This might be a fairly strong assumption which simplifies the problem. In particular, the assumption means that the label can essentially be predicted by the proxy network.  Why not use the proxy network itself for inference instead of the set of linear regressors?\n2. The experiments seem a bit inadequate: the proposed method is somewhat of an extension to the methods of [Ghosh-Mazumdar, ICML’24] and so should be compared against them. Also, the experiments should measure how well the proxy network itself is predicting the labels (as mentioned above). \n3. The paper, though well written for the most part, has some parts which need clarification (see questions to authors).", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "X0tBrUmL9a", "reviewer": "Reviewer_QYPe", "summary": "The paper proposes Covariate-Guided Clusterwise Linear Regression (CG-CLR), an end-to-end framework that jointly learns (i) a response-free assignment rule mapping a covariate vector to one of $K$ local linear regressors and (ii) the $K$ regressors themselves. The system uses a proxy network that outputs instance-wise coefficient vectors; a hard vector-quantization step assigns each input to the nearest codebook regressor; and training alternates between updating the proxy and the codebook via a composite “fit + alignment” objective. The authors show monotone descent of a Lyapunov-style objective with linear convergence under assumptions, a PAC-style excess-risk bound, and a nested-model $F$-test to tune $K$. Empirically, CG-CLR recovers synthetic piecewise-linear structure and is competitive with tree ensembles on several tabular benchmarks.", "questions": "* My main questions are in the weaknesses section.\n* Overall I think the paper provides a useful engineering tool, while the discussion about its theoretical results, especially on the assumptions are limited. How would the author justify their assumptions empirically? How should those requirements be tested?", "limitations": "", "rating": 8, "confidence": 2, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "* The paper is well-organized and clearly written. The paper has clean and good illustrations for their takeaways and empirical findings.\n* The paper has a good mix of both theoretical and empirical evidence. The method proposed is also generic enough. I must admit I'm not an expert in this field, but the results in general strike me as interesting.", "weaknesses": "Major comments:\n* The paper claims one of the strength is that prediction is accurate without ever observing the true response $y_{i'}$---however, there is really nothing surprising there. This is basically generalization in conventional machine learning setup, and is not special to this particular task. The authors should make it clear that the actual strength is the proxy is label independent, which means the algorithm is unsupervised in natural. I'm also not sure how much ``the inner minimization over $j$ cannot be evaluated at test time because the response $y_i'$ associated with an unseen covariate vector $x_i′$ is unknown at prediction time'' is undesirable. We can always estimate generalization error by cross validation/hold-out set if needed. \n* The linear convergence result requires a uniform lower bound on the proxy network's Jacobian, strong convexity and smoothness of the alignment loss. Those conditions seem strong and probably unverifiable in reality. If for some toy model or the empirical setup in the paper, those conditions can be justified (even numerically), they would be much more convincing.\n\nMinor comments:\n* P7. \"form\" should be \"from\"", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "8Ipdm3k7Ij", "reviewer": "Reviewer_wePp", "summary": "The paper introduces covariate-guided Clusterwise Linear Regression (CLR), a framework that jointly learns cluster assignments and local linear models through end-to-end gradient-based optimization. By combining a proxy network with hard vector quantization, the method ensures convergence, provides theoretical risk guarantees, and adapts smoothly from global to instance-wise regressions. Experiments show it accurately reconstructs piecewise-linear functions and outperforms existing CLR and mixture-of-experts approaches on tabular benchmarks.", "questions": "Please refer to the weaknesses section.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 1, "contribution": 2, "strengths": "- The paper tackles an important and well-motivated problem in modeling local linear relationships in tabular regression tasks.\n\n- The proposed approach effectively integrates a simple yet elegant idea—using a proxy network (similar to a hypernetwork) to predict regression coefficients—into an end-to-end optimization framework.\n\n- The method is theoretically well justified, offering convergence guarantees, a PAC-style excess-risk bound, and a principled F-test analysis of model complexity.\n\n- The approach demonstrates strong empirical performance, accurately recovering piecewise-linear structures and outperforming existing CLR and mixture-of-experts baselines.", "weaknesses": "- The proposed approach appears somewhat incremental, primarily combining elements of hypernetworks with an expectation–maximization-style framework using hard assignments. While effective, the conceptual novelty over existing methods may be limited.\n\n- The paper’s presentation is weak—the writing is often unclear, and the main contributions focus more on the properties of the proposed method rather than its broader impact or significance. Although the method itself is relatively straightforward, the paper is difficult to follow, and important implementation details—particularly regarding how the model is applied at test time—are missing.\n\n- While the paper provides some theoretical assumptions about the mapping function used for cluster assignments, it remains unclear how these properties are enforced during training when using an MLP. It would also be useful to discuss whether other architectures could replace the MLP and how such substitutions might affect convergence or theoretical guarantees.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "4fRSSvnttS", "reviewer": "Reviewer_hxki", "summary": "This paper proposes Covariate-Guided Clusterwise Linear Regression (CG-CLR), which jointly learns a covariate-based assignment rule and multiple local linear regressors in an end-to-end fashion. A feedforward neural network predicts a per-sample proxy regressor using only the covariate, which are quantized to the nearest of 𝐾 learned regressors. The paper provides convergence and generalization guarantees and introduces an F-test-based method to select 𝐾. Experiments on synthetic and tabular datasets show that CG-CLR achieves superior accuracy comparable to other models that have \"large coverage\".", "questions": "1. It would be interesting to see results for higher-dimensional synthetic experiments (dimension > 2) and for more practical covariate distributions, such as Gaussian mixtures.", "limitations": "", "rating": 6, "confidence": 5, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "1. Novel end-to-end CLR framework: The paper introduces a novel way to partition the covariate space using a feedforward neural network and to assign a regressor to these partitions simultaneously. The key ideas are captured clearly in Equations (3) and (4).\n\n2. Because the model uses local linear regressors, it maintains a level of interpretability – each cluster’s behavior can be understood via a simple linear formula. The learned “codebook” of regressors is compact, making the overall model relatively easy to inspect and reason about. This is an advantage over complex black-box models and aligns with the paper’s motivation of balancing accuracy and interpretability.\n\n3. Theoretical rigor: The paper proves monotonic descent of the empirical risk and even linear convergence to an optimum under certain conditions. Additionally, it derives a PAC-style excess risk bound.\n\n4. The paper offers a principled way to choose the number of clusters  K via an F-test.\n\n5. Empirical performance: The model demonstrates strong performance on several benchmarks compared to “large-coverage” models.\n\n6. Clarity: The paper is clearly written and very well-organized, making both the methodology and theory easy to follow.", "weaknesses": "1. The assumptions in the theoretical proofs could be quite restrictive. It would be helpful to discuss why these assumptions are reasonable and under what conditions they might fail.\n\n2. The framework may fail when groups have very different regression vectors but very similar covariate distributions, since the assignment is guided only by covariates.\n\n3. The clusters are not necessarily linear or interpretable; the learned partitions may be arbitrary or data-driven without clear meaning.\n\n4. Simpler methods such as Random Forest, XGBoost, or CatBoost often outperform CG-CLR, suggesting it is not clearly superior to strong baselines.\n\nMinor Comment:\ntypo in line 184", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)