Generalizing Linear Autoencoder Recommenders with Decoupled Expected Quadratic Loss
Authors: Ruixin Guo
Xinyu Li
Hao Zhou
Yang Zhou
Ruoming Jin
Venue: ICLR 2026 Poster (2026)
Keywords: linear autoencoders
recommender system
closed-form solution
expected quadratic loss
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning

Abstract: Linear autoencoders (LAEs) have gained increasing popularity in recommender systems due to their simplicity and strong empirical performance. Most LAE models, including the Emphasized Denoising Linear Autoencoder (EDLAE) introduced by (Steck, 2020), use quadratic loss during training. However, the original EDLAE only provides closed-form solutions for the hyperparameter choice $b = 0$, which limits its capacity. In this work, we generalize EDLAE objective function into a Decoupled Expected Quadratic Loss (DEQL). We show that DEQL simplifies the process of deriving EDLAE solutions and reveals solutions in a broader hyperparameter range $b > 0$, which were not derived in Steck’s original paper. Additionally, we propose an efficient algorithm based on Miller’s matrix inverse theorem to ensure the computational tractability for the $b > 0$ case. Empirical results on benchmark datasets show that the $b > 0$ solutions provided by DEQL outperform the $b = 0$ EDLAE baseline, demonstrating that DEQL expands the solution space and enables the discovery of models with better testing performance.

Published as a conference paper at ICLR 2026
GENERALIZING LINEAR AUTOENCODER RECOM- MENDERS WITH DECOUPLED EXPECTED QUADRATIC LOss
Ruixin Guo 1x, Xinyu Lite, Hao Zhou', Yang Zhou?, 2 Ruoming Jin 11
1 Kent State University, Auburr University {rguo5, x1i74, hzhou6, rjini]ōkent edu, yangzhou@auuura..du
ABSTRACT
Linear autoencoders (LAEs) have gained increasing popularity in recommender systems due to their simplicity and strong empirical performance. Most LAE models, including the Emphasized Denoising Linear Autoencoder (EDLAE) in- troduced by (Steck) 2020). use quadratic loss during training. However, the orig- inal EDLAF only provides closed-form solutions for the hyperparameter choice b = 0, which limits its capacity. In this work, we generalize EDLAE objective into Decoupled Expected Quadratic Loss (DEQL). We show that DEQL simpli- fies the process of deriving EDLAE solutions and reveals solutions in broader hyperparameter range
O, which were not derived in Steck's original paper. Additionally, we propose an efficient algorithm based on Miller's matrix inverse theorem to ensure the computational tractability for the b > 0 case. Empirical results on benchmark datasets show that the b > 0 solutions provided by DEQL, outperform the lution space and enables the discovery of models with better testing performance.
EDLAE bascline, demonstrating that DEQL expands the so-
INTRODUCTION
In recent years, deep learning has emerged the dominant paradigm in recommendation systems, leading to increasingly complex models. However, a growing body of empirical evidence reveals surprising trend: simple linear models often perform comparably to, even outperform, their deep learning counterparts (Dacrema et al., 2219). 9) In particular, linear autoencoder- -ased methods such SLIM (Ning & Karypis 2011), 2011) EASE (Steck 2019), EDLAE (Steck) (020), ELSA (Vanuurl ett al. 2022), RALE and RDLAE (Moon et al. 2023) have demonstrated strong performance, fre- quently surpassing more sophisticated deep models (Dacrema et al. 202 particularly in sparse data reconstruction tasks (Monteil et al. 2024 Volkovs et 2017).
These methods typically aim learn an item-to-item similarity matrix W R=x= to reconstruct the (sparse) binary user-item interaction matrix R E {0, 1j with m users and n items. The reconstruction is given by RW, where W can viewed as linear autoeneoder (LAE) acting as both encoder and decoder. One representative example is EASE (Steck 2019), which learns W by minimizing the following objective function:
(I) where || - RW] denotes the reconstruction error and I||W[ the L2 regularizer. The zero- is imposed to prevent W from collapsing to the identity mapping, in which case each item would trivially reconstruct itself. The minimizer of (T) as well as those of most LAE-based models, admits a closed-form solution. This enables greater computational effi- ciency compared to iterative optimization methods (e.g., gradient descent) (Bertinetto et al. 2019) while also facilitating reproducibility (Appendix G.4).
f(W) = || RW + XI[W | s.t. diag(W)
diagonal constraint di)
Despite their empirical success, these models treat observed interactions (the 1s in R) as fixed dur- ing training, without explicitly accounting for the statistical nature of the evaluation process. In
Equual contribution. Corresponding author.

---

Published as a conference paper at ICLR 2026
practice, the test set is typically constructed by holding out fraction of interactions (Section 7.4.2, (Aggarwal 2016)), so that the model is evaluated on randomly masked interactions. This discrep- ancy motivates adopting statistical perspective to redesign the training objective: interactions can be modeled as random variables sampled from an underlying distribution, and the objective can be formulated in expectation, thereby better aligning the training procedure with the testing scenario.
EDLAE Steck 2020) provides an important precursor in this direction. By introducing dropout and an emphasis weighting scheme, EDLAE effectively reshapes the loss penalize the reconstruction of masked interactions more heavily, thereby mitigating overfitting toward the identity mapping, see We observe that can be reformulated in expectation form:
f(W)= Es [II44 o (R - (A @ R]
(2)
where A E {a, b)mxx denotes emphasis matrix with parameters 2 6 > 0, which assigns greater weight to the reconstruction of dropped items. This statistically formulated objective closely resembles the mean squared error (MSE) used in evaluation (Appendix G.2). Such training-testing
alignment has been empiricalyy shown to be effective in achieving strong performance.
However, the theoretical foundation underlying the construction of this objective remains largely underexplored. In particular, [Steck (2020) provides a closed-form solution for the W minimizing 2 only in the special case b = 0. The behavior of the solution in the broader hyperparameter range O remains unexplored, and it is unclear whether improved models may arise in this region. Motivated by this gap, this paper studies the existence and uniqueness of closed-form solutions over the full hyperparameter range 2 0, develops efficient methods for computing them, and conducts extensive experiments to evaluate their empirical performance.
First, we generalize the EDLAE objective (2) into Decoupled Expected Quadratic Loss (DEQL) and derive its closed-form minimizer, which subsumes EDLAE as a special case. This generalization not only simplifies the derivation of EDLAE solutions, but reveals several new theoretical insights: for b = O, the closed-form minimizer is not unique solutions share identical off off-oiagnnal entries while allowing arbitrary diagonal; for b >> 0, unique closed-form solutions always exists, including the previously unexplored region >> a (Section 3).
Next, we find that the direct computation of solutions for b > O have an O(n²) time complexity, which is prohibitively expensive for large-scale recommendation tasks. To overcome this challenge, develop efficient algorithm based on Miller's matrix inverse theorem (Miller 1981), reducing the complexity to O This makes computing solutions in the b > 0 region practical (Section
Finally, we evaluate solutions derived from DEQL real-world benchmark datasets. Our exper- iments demonstrate that solutions with b > O, combined with L2 regularization, consistently out- perform the original EDLAE solutions with b O, as well other recent LAE-based and deep learning-based recommender models. These results confirm that expanding the solution space can yield models with stronger generalization performance. Moreover, find that the previously sumed constraint a 2 6 does not universally guarantee optimal performance: on certain datasets, the best-performing models lie in the range b (Section
Appendix A shows an illustration of DEQL framework. The proofs of all theorems, lemmas and propositions are in Appendix B Related works are in Appendix E Discussions are in Appendix G
2 PRELIMINARIES
LAE-based Recommender Systems: In the implicit feedback setting, a dataset is typically repre- sented as a binary user-item interaction matrix R {0, 1 of m users and n items. Each Rsg indicates whether user has interacted with item j. R.j = is referred to as an observed entry, meaning that user i has interacted with item j: Rıj 0 is referred to missing entry, meaning no observed interaction. Collaborative filtering recommender systems aim to identify items that a user has not yet interacted with but is likely to interact with, and recommend them accordingly. To this end, the model predicts the missing entries in R by assigning real-valued scores, where larger values indicate a higher likelihood of interaction.
LAEs are one class of collaborative filtering models. They learn a parameter matrix W E Rax= and generates predictions by ît = RW E mx,,, so that each missing entry R.; = 0 is assigned predicted score Rij TR.
2

---

Published as a conference paper at ICLR 2026
EDLAE (Steck 2020): EDLAE is state-of-the-arr LAE model for collaborative filtering. Let A {0,1) ma be a random matrix whose entries Aú are i.d.. Bernoulli random variables with for a given E (0, 1). Let A (*) denote a realization of A with index k, and let @ denote the Hadamard (element wiiee) product, so that A (k) @ R applies
P ( = 0) = p and P P(A., = 1) =
dropout element-wise to R. Define the emphasis matrix A(k) where 1 if A (R) 0 and
ij
A
b if A (8) is 1. Then, the EDLAE model is obtained by optimizing the following objective function:
W arg min
lim
1 N
(A @ (R (A R)W)
W
under the hyperparameters a, p. Since the squared Frobenius norm in (3 can be expanded into the sum of weighted quadratic loss 5m Ej=1 A!4 (Ri, - (A!:) o Ri, W.;) if R. dropped, its @ R., W.,) W, is weighted by a²: otherwise, is weighted by b2.
reconstruction loss (Ri, (A (k)
The original EDLAE paper (Steck 2020) suggests choosing a. b to satisfy a 2 b > 0, and provides closed-form solution to for the case and under the zero-diagonal constraint diag(W"") = O, expressed as
1 W 1-p (1- where C RR
P -I'RRPPORR' 1-P
While remains valid and meaningful for b > 0, the solution for this case is not discussed in the original work. Moreover, the suggested hyperparameter choice a 2 b assigns greater weight to reconstruction loss of dropped interactions and is intended enhance the model's ability to recover them. This design aligns with the MSE used during evaluation (Appendix G.2).
3 DECOUPLED EXPECTED QUADRATIC LOss FOR LINEAR AUTOENCODERS
We first rewrite the EDLAE objective function in 3 into expectation form. Let B denote the multivariate Bernoulli distribution of A, then by the law of large numbers, can be rewritten as
W argminw LB(W),
where
(WW EA-S [ (R (A R)W)
Σ
BANB [1] @ (R.; (A R)
Σ EA-B [l: 10) 3 R))
(5)
Here denote A diagMat(A..). Note that (5) decouples the squared Frobenius norm over the columns of W. Since R constant while both A and A are random, define (i) A R and (+) = ( (A @ R). then both ( and (+) are random. If we further denote D as the
distribution of the pair (X(0, (i)), then the objective function in 5 can be written as
n (s(W) Eoxo 11y0-x"W..ll
(6)
where each column W.; placed inside an expected quadratic loss y()) [1)
This formulation is general since each ((t) can be any distribution, while is special case where (i) and (i) follow distributions induced by applying random dropout to constants. We first derive the general closed-form solution of optimizing (6). then specialize it to EDLAE, and show that this reformulation simplifies the analysis and reveals broader class of solutions for 2 0 compared Steck's original solution for b
3.1 DECOUPLED EXPECTED QUADRATIC Loss AND ITS CLOSED-FORM SOLUTION
We formally define o as follows:
3

---

Published as a conference paper at ICLR 2026
Definition 3.1. Given set of joint distributions D (D(9) fi=1 over the pair (X, Y), the decou- pled expected quadratic loss (DEQL) is defined as
(D(W) Σ ==1
Doo (Was),
where
poo (Was) - [x (DD
x ||
W. (X.Y) [X*X] W.: 22 ( [X"Ya:] + (x))) (TTY.. (X.))
(7)
Note that each bip is quadratic function of W.i. Since ((x,Y))D() [xTX] is positive semi- definite (as X X is random matrix whose realizations are always positive semi-definite) N is convex for any Hence, let W =argminy/ (ggmin /D(W), then W argminw.. "i (W.:) for all Furthermore, if E (x.Y)= [x"x] is positive definite for all i, so that its inverse exists, then W can be computed column-wise
W E (X,Y)~()))
[X x]
((X)) [x"Y,] for i 1.2.....
As remark, can be viewed as a generalization the true risk of multivariate linear regression in statistical learning theory Vapnik 1 by taking := D(L) DD2)
DD(n) reduces to (DW) = E(x,y)< [||Y XW F... where || XW || is multivariate linear regression loss.
Moreover, the solution W* obtained from (8) is in general full rank. We further discuss the case of optimizing
(7) under low-rank constraint on W in Appendix D In this case, closed-form solutions
exist if E (-,D)
[X is independent of for all i.
3.2 ADAPTATION TO EDLAE
This section shows how we derive the closed-form solutions of EDLAE B3 from DEQL 7 covering the case b 2 0. Our results show that, in the b = case, the solution is not unique: they shared the same off-diagonal but can have arbitrary diagonal, while Steck's solution (Steck) 2020) is one of them by choosing zero diagonal. Furthermore, we establish that for every > O. clomed-form solution exists and is unique a property that was analyzed in Steck's work.
We first discuss the b case. Remember that 5 is special case of by taking X A14
R) and Y., ( R. By (8) the solution of () is given by
W = (i) (1)) for = 1,2, n, where
(9)
Ht) = EA-B (A ④ A ( o R)) (00) = EANB (A
10)
R.i
(10)
The following lemma enables explicit computation of the expectations in 10
Lemma 3.2. The (() and v()) in (10) can be expressed as H
G ① R R and v()
u( R R.g where G()) Ro and (i) RR" satisfy
(1 p)b
if k = ==l=i if x # if i
22
((1 p)b²
if kk=
u ((1 p)pp? + (1 2]
(1 p)pa? + (1-p)
if k
pp? + (1- p)
i
for k, E (1,2,..., n}.
Furthermore, the computation of (9) involves (0 which exists only if H+) is invertible. The following theorem establishes sufficient conditions to ensure this property.
Theorem 3.3. For any 2 0, > 0 and 0 < p < G( is positive definite. Furthermore, H is positive definite if GL is positive definite and no column of R is a zero vector.
A positive definite (*) is always invertible. Hence, Theorem 3.3 implies that the closed-form solution by (9) holds for any a 2 and > Notably, this includes the case b > a, which lies outside the original EDLAA range > b ' 0 (see Figure
4

---

Published as a conference paper at ICLR 2026
Now discuss the case when b 0. In this setting, both the i-th row and i-th column of H zero, and the i-th row of ul also zero. Consequently, H()) is singular and its inverse H does not exist, making (8) inapplicable for computing the optimal W
To proceed, we define submatrices and subvectors using the subscript -i notation: if Q is an n x n matrix, then Q-; is (n 1 x (n 1) matrix obtained by removing the i-th row and i-th column of Q; if q is an n dimensional vector, then q-i is an n1 dimensional vector obtained by removing di from q. Under this notation,
(n - matrix with diagonal elements (1 pppppa and off-diagonal elements (1 pp) It is easy verify that G is positive definite, hence [ is positive definite. Likewise, (5) u @ (o:)
write H G @ ( R)-i, where G is (n
where u is an dimensional vector with all elements being (1 - p)pe?
Denote the vector (W-:) as W.i then (5) can be written
n
(B(W)= Σ W 11 W.: -2 ES-B A Rei
(11)
WT. - H1 H Σ 1=1
W _ 2W.,
(0) 20 EA~B [ Roi
(12)
Therefore, the solution W* argmin (B(W) is expressed as
W - (1 - and W E ER
for ..,,
(13)
The optimal W given by (13) is not unique, but belongs an infinite set of solutions that share the same off-diagonal entries while allowing arbitrary diagonal entries. The following theorem shows that Steck's solution (4) is special case of (13). corresponding to the choice of zero diagonal.
Theorem 3.4. Suppose no column of R is a zero vector. If taking W 0oror all in 13 then 73 and are equivalent.
It is important to note that varying the diagonal of W can lead to different performance on test data, and (13) does not provide theoretical guidance on which choice of diagonal elements gives the best performance, However, empirical results suggest that W with non-zero diagonal elements can outperform the zero-diagonal solution in certain cases (Moon al. 2023).
3.3 ADDING L2 REGULARIZER ZERO-DIAGONAL CONSTRAINT
In LAE- ----baeed recommender systems, L2 regularizer and zero diagonal constraint are commonly applied to the objective function, as they are established techniques for improving test performance. This section discusses the closed-form solution of the optimization problem 5 when the L2 regu- larizer or the zero-diagonal constraint is applied.
Adding Lg Regularizer: Given À 0, 5 with L2 regularizer is expressed as
W argminw (sW) + |||W|||
(14)
Adding Zero-diagonal Constraint:
5 with zero-diagonal constraint is expressed as W argming (w()) s.t. diag(W) 0
(15)
In these cases, the solution is modified accordingly, as presented below.
Proposition 3.5. (a) The solution of (44)
W
H + +AT) (1()) for 1,2,..,,
(16)
(b) The solution of 15
(H b(1));
= H (
(1) -1 for 1,2.....
(17)
(H (H+))
where 160 is n-dimensional vector with 1((0 and (()) 0 for all #
5

---

Published as a conference paper at ICLR 2026
4 AN EFFICIENT ALGORITHM FOR THE CLOSED-FORM SOLUTION
Recall from Thcorem 3.3 that the optimal W for the > case of EDLAE can be computed by ② However, a major challenge is its high computational complexity: since H differs for each i, computing each inverse H(00 costs O(n²) (suppose we use basic matrix inverse algorithms, c.g., Cholesky decomposition (Krishnamoorthy & Menon 2013)), resulting in a total cost of O(n²) for all which is computationally impractical.
In this section, we show the existence of a practical algorithm that reduces the overall complexity of computing (9) from O(n*) to O(n"). We prove this by explicitly constructing one using Miller's
matrix inverse theorem:
Theorem 4.1. (Miilerr 1981 Let G and G + Q be non-singalar matrices. Suppose Q is of rank r and can be decomposed as Q = E1 + E2 + + Er, where each Eg is of rank 1, and Px+1 G E1 E2 ... + Eg is non-singular for k 1.2,... v. Let P1 = G. then
1
P Phl
PE PE P
1 PE Es)
In Lemma 3.2 we define H = G(0) @ R R. which can be decomposed as
H = Go o R² R d @ R² R G @ R² R
where Go is matrix with diagonal elements equal to (1 p)pa² (1 p)² and off dif-oialonal
elements equal to (1 - pa² + (1 - p); o is a matrix with (G) )5: = -(1 p)²p(a² - bP) for j + i, (G()) )x -11 p)p(a² 2 - b²), and all other elements zero; G is matrix with (G) Jy = - p(p - 132) for j # i and all other elements zero.
Denote Ho = G R R. E G( Rd R and E c. @ Rd R, then H Ho + E + E/" Note that Ho is positive definite and independent of i, E is of rank 1 with only the i-th column being nonzero, and E is of rank with the i-th row (excluding (E.")
)a) being nonzero.
Applying Theorem 4.1 H can be computed with the following two steps:
- -1 = -Ho + E(01)- 11= Ho¹
1
Ho
E Ho
(18)
1 1+tr(H HH E(1))
H(11 (Ho + ota D - 1171
1
H
(19)
1 + E
Let e be the i-th column of D
10)
T
be the i-th row of (0) then 18 and 19 can be simplified
e2
1 -H1'
1
(1 c ((H")(H)
(20)
+ (Ho )
H00-_ -H(1-
(H )ei(e
( HT())
(21)
( (H{()) (H())-) )ar
1+
Observe that, given H²¹, the computation of each H00 using (0) and (1) requires only O(n²) operations, resulting in total cost of (n?)) for all i. This significantly reduces the original O(n*) complexity of computing (9).
Moreover, the computation be further simplified by directly computing H (0( without ex-
plicitly forming H(0) -1 By 20 and 21
H (8) (())) = -15 (10
(Ho 11-1((1) (c()) (H4 (H1))))0 1
(22)
1+ (Ho ei
(1) 11 ((0) H(()) (1)
1
(H (i) ) )ss (e)0) = (23)
(1)
(H!
(t)
-1
+ C2
)as

---

Published as a conference paper at ICLR 2026
in which the scalars (H="):.00) and (c2
(1)
()00)) can be computed first. In 23p. the
(H (H)) )az term can be computed by 20
HH ( -1 Jai = -Ho
(Ho')a²- )a2
(Ho )u
(H ((0)
1 + (Ho (Ho')..e)" e
Denote 8 = ( 000,1 1 (H )a:. Let U be an n x n matrix with diagonal elements equal to (1 - p)b2 and off-diagonal elements equal to (1 - p)pap + (1 p) let G2 be an n x n matrix with diagonal elements - - p)p(a² b²) and off off-diagonal elements -(1 -p)²p(a" -b²), and let
G2 be an n x n matrix with zeros on the diagonal and off-diagonal elements -(1 - p)²ppa² 62).
Then we summarize our computation as follows.
Fast Algorithm for Computing First, precompute these matrices
Rº R, Ho (Go o Rº R)
2 R R)R), R), 1e2 1e,", 02 ..... c = G2 3 R²R
(2) ( { .HO el .... Hp e = Ho- [G1 @ R R),
Then for i = L,,,..,, ..., n, compute each W = H (8) v(0) as follows:
r Hn (0)) w Ho ci
(8)
1
(Ho ")i
r2W, = =Ho
$
w
1 1 wi
1 W/
H -1 (00) =
(1) T (62 (2)
8)t
1 02
We now analyze the computational complexity of the above algorithm. Here we assume that matrix multiplication and inversion implemented using basic linear algebra algorithms. In the precom- puting stage, Rd R costs (mm?²) )), Ho¹ costs O(n²). Ho (U @ R R) and Ho (G1 @ R R) cost Onn)), and G2 @ Re R costs (n²)). the computing stage, each W is obtained via only vector- vector multiplications, with complexity of (n), resulting in an overall complexity of O(n?) for the entire W* Therefore, the total complexity of this algorithm is O (max(m + m))²²)
This complexity is the same as the closed-form solutions of EASE (Steck 2019) and EDLAE (Steck) 2020), and the main bottleneck lies in the (n))) complexity for matrix inverse. If using more advanced algorithms for matrix multiplication and inversion, the complexity of our Fast Algorithm (as well as EASE and EDLAE) can be in further reduced from O(z²) to ((n2376)) We elaborate on this in Appendix
Adapting the Algorithm to the L2 regularizer and Zero-diagonal Constraint Cases: In (6 note that H + AI (H, + AT) + E + E where Hu + XI is independent of This means that we can compute (16) using the above algorithm by replacing Ho with Ho + Al. To compute
17 we use the algorithm to process the two components (1)) y(() and H(0 -1 ((0) respectively:
first compute (+) (()), then replace ()()) with ((0) and compute H(1) ((0)
5 EXPERIEMENTS
This section provides experimental results comparing DEQL with state-of-the-art collaborative fil- tering models, including linear models and deep learning based models.
5.1 EXPERIMENTAL SET-UP
Datasets: We utilize two group of public datasets, Group 1 includes Games, Beauty, Gowalla- ML-20M, Netflix, and MSD (Steck) 2019 Ni et al.) 2019 Seol et al. 2024). This group adopts strong generalization setting, where the dataset is split into training and test sets by users (i.e., the users in the two sets do not overlap). Group 2 includes Amazonbook, Yelp2018 and Gowalla He ct al. (202)). This group adopts weak generalization setting, where the dataset is split into training and test sets by interactions (i.e., user may appear in both sets). Among these datasets, Gowalla-1
7

---

Published as a conference paper at ICLR 2026
is preprocessed from the raw Gowalla dataset by us, whereas all other datasets follow the default training/test splits used in prior work.
Group is used compare DEQL with other LAE-based models, while Group 2 is used to compare DEQL with both deep learning-based and LAE-based models. We adopt this setting to align with common practice in the literature, where LAE models more commonly evaluated under strong generalization, while deep learning models are typically evaluated under weak generalization. De- tails of these datasets provided in Table [
Table 1: Details of Datasets
Group 1
Group
Games
Beauty
Gowalla-1
M1.20M
Netflix
MSD
Amazon Books
Yelp2018
Gowalla
#
4,394
13,681
20,108
17,769
41.140
91.599
38,048
40,981
# users
1,006
17.971
29,243
136.677
463.435
$71,353
52,643
31,668
29,858
# interactions
15,276
75,472
677,956
9,990,622
56,880,037
33,633,450
2,984,108
1,561.406
1,027.370
density
1.69%
0. 0.10%
0.17%
0.36%
0.69%
0.14%
0.06%
0.13%
0.08%
item ratio
0.89
0.24
0.47
0.15
0.04
0.07
1.74
1,20
1.37
dencity
# interactions (# users x # fiems), item-user ratio items uxers
Baseline models and Evaluation Metrics: We compare DEQL with the following state-of-the-
art LAE-based models: EASE (Steck) 2019), DLAE (Steck 2020). EDLAE (Steck) 2020), ELSA Vancura et al. 2022), as well as the following recent deep rearning-based models: PinSage (Ying ct al. 2018). LightGCN (He et al., 2020). DGCF (Wang et al. 2020), SimpleX (Mao et al. 2021
ED (Wu et al. 2021) and SSM (Wu et al. 2024a). We evaluate model performance using
widely adopted ranking metrics: Recal 20 (R@20 (0) and NDCG@20 (N@20).
Different versions of DEQL Models: test different versions of DEQL, distinguished as follows:
DEQL(plain): the plain DEQL with 0, computed by
DEQL(L2): DEQL with b > and L2 regularization, computed by combining and 16
DEQLAL2++eerodiaag) DEQL with b > 0, L2 regularization, and a zero-diagonal, computed by combining 16 and
Note that all baseline LAE models (EASE, EDLAE, DLAE and ELSA) in our experiments are equipped with L2 regularization, consistent with their original papers [ Therefore, DEQL((.2) and DEQL(L2+Zero-daay provide a fair comparison with these baselines, while (EQL(plain) does not.
Moreover, the EDLAE baseline corresponds to DEQL with b and a zero-diagonal (see Theorem 3.4 Hence, in our experiments, DEQL refers only to the case b >>> 0.
Hyperparameter Tuning:
We perform grid search tune the DEQL hyperparameters a, b, Note that for the DEQL objective (5) (as well as the EDLAE objective (3). if both a and 6 are scaled by constant a
the objective scaled by a, while solution W' remains unchanged. That is, W is scalar invariant and depends solely on the ratio b/a. Hence, we fix = 1 and search over the range [0. 10.1,0.25,... .,.22......... The L2 regularization coefficient is searched over (10.0,20.0,..55,11000) 1.,0000......11000,0000...000000 0], and the dropout rate is varied across [0.1,0.2.....00.555 00.5,088).
Hardware and Reproducibility: All experiments are conducted on a Linux server equipped with 500 GB of memory, four NVIDIA 3090 GPUs, and 96-core Intel(R) Xeon(R) Platinum 8268 CPU @ 2.90GHz. Our code available at httpa//////aaaaa.o/////////rrrrr
5.2 MODEL. PERFORMANCE EVALUATION
Table 2 compares DEQL with LAF--based models under the strong generalization setting. The re- sults show that DEQL (plain) does outperform the baselines, likely due the absence of L2 regularization, which prevents fair comparison. However, DEQL(L2) and DEQLLL2+ZrO-diag) outperform the baselines by a small margin. These results indicate that:
1. The original EDLAE solution Steck (2020). obtained by setting b 0 with a zero diagonal, is not optimal. Solutions in the b region, obtained via DEQL, can achieve higher test performance.
For EDLAF, the author reports using an regularizer in their experimental sct-up Steck 2020
8

---

Published as a conference paper at ICLR 2026
2. Le regularization is essential for improving performance, as DEQL(L2) and DEQL(L2+zero- diag) significantly outperform DEQL (plain).
3. The zero-diagonal constraint does not necessarily lead to better performance, since (E2)( frequently outperforms DEQL(L2+Zerr-diag) atively small diagonal entries, as shown in Appendix F2 This evidence supports the claim in Moon et al. 2023) that relaxing the zero-diagonal constraint in LAEs can improve performance.
fact, the optimal DEQL(L2) solutions exhibit rel-
Moreover, despite the relatively small performance gains of DEQL(L2) and DEQL(L2+zero-dian in TTale statistical significance tests (see Appendix F.T confirm that these improvements are stable and unlikely result from statistical randomness.
Table 3 compares DEQL with deep learning- alization setting. The result show that DEQL(L2) achieve the best performance on Amazonbook, outperforming competitors by to 27% and 34% in R @0 and @@20, respectively, and surpasses most models on Yelp2018 and Gowalla.
and LAE-based models under the weak gener-
Table 2: Performance comparison between DEQL and other LAE-based models under the strong generalization setting. The best results are highlighted in bold.
Model
Resuty
Gowalla-1
Nettli
MSD
RB820
N
R6 20
N:
R@:20
N:8
R.8:0
N
R:0:0
N:8.0
R@000
N:0
DLAE
0.2771
0.1664
0.1329
0.0886
0.2143
0.1916
0.3924
0.3419
0.3620
0.3395
0.3290
0.3210
EASE
0.2733
0,1640
0,1323
0.0875
0.2230
0.1988
0.3905
0.3390
0.3618
0.3388
0.3332
0.3261
EDLAE
0.2012
ELSA
0.2734
0.1658
0.1263
0.0763
0.2255
0.1960
0.3919
0.3386
0.3625
0.3372
0.3256
0.3144
DEQLiplan)
0.2524
0,1565
0.10%
0.0670
0.2149
0.1909
0.3844
0.3347
0.3606
0.3382
0.3329
0.3256
0.2872
0.1704
0.1388
0.0898
0.2278
0.2027
0.3934
0.3429
0.36%
0.3423
0.3344
0.3268
DEQUIL2)
0.1842
0.0881
0.2033
0.3426
0.3428
Table 3: Performance comparison between DEQL and deep learning learning-based and LAE-based models under the weak generalization setting.
Moddl
Autazon- Books
Yelp2018
R00200
N40
R:0020
N1
N/0
learning based models
LightGCN
0.0411
0.0315
0.0649
0.0530
0.1830
0.1554
DGCF
0.0422
0.0024
0.0664
0.0534
0.1842
0.1561
SGL ED
0.0178
0.0379
0.0675
0.0555
SimpleX
0.0701
0.0575
0.1557
SSM (ME)
0.0433
0.0067
0.0509
0.0104
0.1231
0.0878
SSM (GNN)
0.0590
0.0159
0,0737
0.0609
0.1869
0.1571
LAE-based models
DI AF
0U0751
0.0610
0.0688
0.0570
0.1839
0.1533
BASE
0.0566
0.0657
0.1467
EDLAE
0.0711
0.0566
0.0673
0.0565
0.1844
0.1539
FLSA
0.0719
0.0994
0.0620
0.0541
0.1755
0.1400
DEQU-iglain)
(L0095
010537
0.0547
0.0543
(L.7445
0.1453
DEQU (122a))o-
0.0567
0.0672
0.0665
0.1539
DEQI (1.2)
0.0613
0.0685
0.0576
0.1540
5.3
THE IMPACT OF ON MODEL PERFORMANCE
In this section, we conduct a sensitivity analysis on investigate how different values of affect model performance.
We evaluate the effect of different b across seven benchmark datasets, as shown in Figure [ The first three datasets use weak generalization, while the last four datasets use strong generalization. For each dataset, we set a and vary from 0 to 2.0, and report their Recall 20 and NDCG@2 20. Note that for 6 0, the solution is obtained from and for b > 0, the solution is obtained from 9), whose existence is guaranteed by Theorem 3.3
On datasets ML-20M, Games, Netflix, and MSD, we observe clear and consistent pattern: perfor- mance first improves when increasing b from 0, reaches its peak before the b/a ratio exceeds 1, and then gradually decreases b becomes too large. This behavior directly demonstrates that the b 0 choice in the original EDLAE does not necessarily yield the best performance, and that models obtained with b using DEQL can achieve superior results.

---

Published as a conference paper at ICLR 2026
0.917
1956 0185
13333 6013
10055 2366
10339
0.35
855 6332
3332 B05T
-0054 2366
6.35
60
0.5
28
25
2.
0.0
0.5
1.
1.5
26
3.0
35
3.0
35
23
0.
0.5
3.6
25
20
0.247
3333 6337
0334
0033
0.23
0.5
L3
15
3338
0352
1.0 86
155
0323 5320
03
02
3.0
1.
0005
(r) Sammes
(11 Avert
(01
Figure I: Sensitivity on b/a ratio across different datasets
Interestingly. we observe markedly different pattern on Yelp2018 and AmazonBook: their optimal b/a ratios approach 1 or even exceed (as in Yelp). In the original EDLAE formulation, 2 b promotes reconstruction of dropped items through cross-item learning, with typically fixed to zero. The regime of 6 > has rarely been explored, as it instead trains the model to use the remained items to better predict other remained ones within the same set. Although intuitive choice, results show that b can yield superior performance certain datasets, suggesting that emphasizing dropped entries not always beneficial.
appears to be the
Several factors may jointly contribute to this issue. We hypothesize that key factor is the large item-user ratio and the resulting unreliable cross-item correlations. When the number of items greatly exceeds the number of users, in AmazonBook and Yelp2018, the interaction matrix be- comes extremely sparse. In such settings, cross-item correlations are weak or noisy, since most items receive very few interactions, providing insufficient data estimate reliable item-item rela- tionships for collaborative filtering (Nikolakopoulos el al. 202T). Under this regime, training with forces the model to rely more heavily on unreliable cross-item correlations, thereby degrad- ing performance. In contrast, setting > shifts the objective toward reconstructing the remained items themselves rather than the dropped ones, thereby stabilizing training through stronger self- association signals. This effect is reflected in the larger diagonal magnitudes of the learned weight matrices W, indicating increased reliance on identityliike mappings (Figure
CONCLUSION
This paper aims to advance the EDLAE recommender system by extending its closed-form solution broader range of hyperparameter choices, and develop an efficient algorithm compute these solutions. We first generalize the EDLAE objective function into DEQL, derive its closed-form solutions, and then apply them back EDLAE. DEQL extends the original EDLAE solution for b = to wider range b 2 0, enabling exploration of larger solution space. To address the high computational complexity of solutions for >> O, we develop an efficient algorithm based on Miller's matrix inverse theorem, reducing the complexity from O(z²) to O(z²).
Experimental results demonstrate that most solutions for > outperform the 5 0 baseline, showing that DEQL enables the discovery of models with better testing performance. Furthermore, on certain datasets, the optimal even lies the regime > a. To the best of our knowledge, this is the first empirical evidence showing that the commonly assumed EDLAE constraint 2 b is not universally optimal. Finally, we note that DEQL a general loss function that may inspire the construction of other specialized objectives for LAE models.
10

---

Published as a conference paper at ICLR 2026
ACKNOWLEDGMENTS
This research was partially supported by NSF grant IIS 2142675. We thank all reviewers for their insightful comments.
REFERENCES
Amir Abboud, Karl Bringmann, Nick Fischer, and Marvin Künnemann. The time complexity of fully sparse matrix multiplication. In Proceedings of the 2024 Annual ACM-SIAM Symposium on Discrete Algorithms (SODA), pp. 4670-4703. SIAM, 2024.
Charu C Aggarwal. Recommender systems: the textbook. Springer, 2016.
Guillaume Alain and Yoshua Bengio. Understanding intermediate layers using linear classifier probes. In International Conference Learning Representations, 2017.
Luca Bertinetto, Joao F Henriques, Philip Torr, and Andrea Vedaldi. Meta-learning with differen-
tiable closed-form solvers. In International Conference on Learning Representations, 2019.
Emmanuel J Candès and Terence Tao. The power convex relaxation: Near-optimal matrix com- pletion. IEEE transactions on information theory, 56(5):2053-2080, 2010.
Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recom- mender systems. Proceedings of the 1st workshop on deep learning for recommender systems, pp. 7-10, 2016.
Don Coppersmith and Shmuel Winograd. Matrix multiplication via arithmetic progressions. In Proceedings of the nineteenth annual ACM symposium on Theory of computing, pp. 1-6, 1987.
Thomas H Cormen, Charles E Leiserson, Ronald L Rivest, and Clifford Stein. Introduction to algorithms, 3rd Edition. MIT press, 2009.
Hoagy Cunningham, Aidan Ewart, Logan Riggs Smith, Robert Huben, and Lee Sharkey. Sparse autoencoders find highly interpretable features in language models. International Conference on Learning Representations, 2023.
Maurizio Ferrari Dacrema, Paolo Cremonesi, and Dietmar Jannach. Are we really making much progress? worrying analysis recent neural recommendation approaches. Proceedings of the 13th ACM Conference on Recommender Systems, pp. 101-109,2011
Maurizio Ferrari Dacrema, Simone Boglio, Paolo Cremonesi, and Dietmar Jannach. A troubling analysis of reproducibility and progress recommender systems research. ACM Transactions on Information Systems, 39(2):1 2021.
Can Demircan, Tankred Saanum, Akshay Kumar Jagadish, Marcel Binz, and Eric Schulz. Sparse autoencoders reveal temporal difference learning in large language models. In International Con- ference Learning Representations, 2025.
Thomas Fel, Ekdeep Singh Lubana, Jacob S Prince, Matthew Kowal, Victor Boutin, Isabel Papadim- itriou, Binxu Wang. Martin Wattenberg, Demba Ba, and Talia Konkle. Archetypal SAE: Adap- tive and stable dictionary learning for concept extraction in large vision models. In International Conference on Machine Learning, 2025.
Rina Foygel, Ohad Shamir, Nati Srebro, and Russ R Salakhutdinov. Learning with the weighted trace-norm under arbitrary sampling distributions. In Advances in Neural Information Processing Systems, volume 24, 2011.
Huifeng Guo, Ruiming Tang, Yunming Yc, Zhenguo Li, and Xiuqiang He. DeepFM: a factorization- machine based neural network for CTR prediction. In Proceedings of the 26th International Joint Conference on Artificial Intelligence, pp. 1725-1731, 2017.
Ruixin Guo, Ruoming Jin, Xinyu Li, and Yang Zhou. PAC-Bayes Bounds for multivariate linear regression and linear autoencoders. In Advances in Neural Information Processing Systems, 2025.

---

Published as a conference paper at ICLR 2026
Xiangnan He, L. Liao, H. Zhang, L. Nie, X. Hu, and T. Chua. Neural collaborative filtering. In The World Wide Web Conference, 2017.
Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgnn: Simplifying and powering graph convolution network for recommendation. Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval, pp. 639-648, 2020.
Roger A Horn and Charles R Johnson. Matrix analysis. Cambridge university press, 2012.
Yifan Hu, Yehuda Koren, and Chris Volinsky. Collaborative filtering for implicit feedback datasets. In IEEE International Conference Data Mining, pp. 272.272. IEEE, 2008
Dietmar Jannach, Markus Zanker, Alexander Felfernig, and Gerhard Friedrich. Recommender sys- tems: an introduction. Cambridge university press, 2010.
Yehuda Koren. Factorization meets the neighborhood: multifaceted collaborative filtering model. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 226-434, 2008.
Yehuda Koren, Robert Bell, and Chris Volinsky. Matrix factorization techniques for recommender systems. Computer, 42(8):30-37, August 2009.
Aravindh Krishnamoorthy and Deepak Menon. Matrix inversion using cholesky decomposition. In 2013 signal processing: Algorithms, architectures, arrangements, and upplications (SPA), pp. 70-72. IEEE, 2013.
Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. Deep learning. Nature, 521(7553):436 444, 2015.
Dong Li, Ruoming Jin, and Bin Ren. Revisiting recommendation loss functions through contrastive learning (technical report). arXiv preprint arXiv:2312.00:: 12.05200, 2023.
Dawen Liang, R. G. Krishnan, M. D. Hoffman, and Jebara. Variational autoencoders for collab- orative filtering. In The World Wide Web Conference, 2018.
Zhuang Liu, Yunpu Ma, Yuanxin Ouyang, and Zhang Xiong. Contrastive learning for recommender system. arXiv preprint 100:22000001177 2021.
David G Luenberger and Yinyu Ye. Linear and nonlinear programming, 3rd Edition. Springer, 2008.
Kelong Mao, Jieming Zhu, Jinpeng Wang. Quanyu Dai, Zhenhua Dong, Xi Xiao, and Xiuqiang He, Simplex: A simple and strong baseline for collaborative filtering. In Proceedings of the 30th ACM international conference on information & knowledge management, pp. 1-43-1252 2021.
Rahul Mazumder, Trevor Hastie, and Robert Tibshirani. Spectral regularization algorithms for learn- ing large incomplete matrices. The Journal of Machine Learning Research, 11:2287-2322, 2010.
Kenneth S Miller. On the inverse of the sum of matrices. Mathematics magazine, 54(2):67-72, 1981.
Julien Monteil, Volodymyr Vaskovych, Wentao Lu, Anirban Majumder, and Anton Van Den Hengel. Marec: Metadata alignment for cold-start recommendation. In Proceedings of the 18th ACM Conference on Recommender Systems, pp. 401-410, 2024.
Jaewan Moon, Hye-young Kim, and Jongwuk Lee. It's enough: Relaxing diagonal constraints in linear autoencoders for recommendation. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development Information Retrieval, pp. 1639-1648, 2023.
Jianmo Ni. Jiacheng Li, and Julian McAuley. Justifying recommendations using distantly-labeles reviews and fine-grae -me-ngrieed aspects. In Proceedings of the 2019 conference empirical methods in natural language processing and the 9th international joint conference on natural language processing (EMNLP--JCNI.P), pp. 188-197, 2019.
12

---

Published as a conference paper at ICLR 2026
Athanasios N Nikolakopoulos, Xia Ning. Christian Desrosiers, and George Karypis. Trust your neighbors: comprehensive survey of neighborhood- baased methods for recommender systems. Recommender systems handbook, pp. 39-89, 2021.
Xia Ning and George Karypis. Slim: Sparse linear methods for top-N recommender systems. In IEEE 11th International Conference on Data Mining, pp. 497-506. IEEE, 2011.
Rafael Rafailov, Archit Sharma, Eric Mitchell, Christopher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. In Ad- vances Neural Information Processing Systems, volume 36, pp. 53728-53741, 2023.
Ran Raz. On the complexity of matrix product. Proceedings of the thiry-fourth annual ACM symposium Theory of computing, pp. 144-151, 2002.
Benjamin Recht. A simpler approach to matrix completion. Journal of Machine Learning Research, 12(12), 2011.
Steffen Rendle, Christoph Freudenthaler, Zeno Gantner, and Lars Schmidt-Thieme. Bpr: Bayesian personalized ranking from implicit feedback. In Proceedings of the Twenty-Fifth Conference Uncertainty in Artificial Intelligence, pp. 452-461, 2009.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. why should i trust you?" explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1135-11 1444, 2016.
LM Rivera-Muñoz, Andrés Felipe Giraldo-Forero, and JD Martinez- Vargas. Deep matrix factoriza- tion models for estimation of missing data in low-cost sensor network measure air quality. Ecological Informatics, 71:101775, 2022.
Jinseok Seol, Minseok Gang, Sang-goo Lee, and Jaehui Park. Proxy-based item representation for attribute and context-aware recommendation. In Proceedingy of the 17th ACM International Conference on Web Search and Data Mining, pp. 616-625, 2024.
Ohad Shamir and Shai Shalev-Shwartz. Collaborative filtering with the trace norm: Learning, bounding, and transducing. In Proceedings of the 24th Annual Conference on Learning Theory, pp. 661-678. JMLR Workshop and Conference Proceedings, 2011.
Martin Spišák, Radek Bartyzal, Antonín Hoskovec, and Ladislav Peška. On interpretability of linear autoencoders. In Proceedings of the 18th ACM Conference on Recommender Systems, pp. 975- 980, 2024.
Harald Steck. Emharrassingly shallow autoencoders for sparse data. In The World Wide Web Con- ference. pp. 3251-3257, 2019.
Harald Steck. Autoencoders that don't overfit towards the identity. In Advances in Neural Informa- fion Processing Systems, volume 33, pp. 19598-19608, 2020.
D Stekhoven and Peter Bühlmann. Missforest-mo cettorrett-ooo-aaaaaaaaaaee non-parametric missing value imputation for mixed-type data. Bioinformatics, 28(1):1 2012.
Juan Terven, Diana-Margarita Cordova-Esparza, Julio-Alejandro Romero-González, Alfonso Ramírez-Pedrara, and EA Chávez-Urbiola. A comprehensive survey of loss functions and metrics in deep learning. Artificial Intelligence Review, 58(7):195, 2025.
Amund Tveit. On the complexity of matrix inversion. 2003.
Vojtëch Vančura, Rodrigo Alves, Petr Kasalický, and Pavel Kordík. Scalable linear shallow autoen- coder for collaborative filtering. In Proceedings of the 16th ACM Conference on Recommender Systems, pp. 604-609, 2022.
Vladimir Vapnik. The nature of statistical learning theory. Springer science & business media, 1999.
Maksims Volkovs, Guangwei Yu, and Tomi Poutanen. Dropoutnet: Addressing cold start in recom- mender systems. In Advances in Neural Information Processing Systems, volume 30, 2017.
13

---

Published as a conference paper at ICLR 2026
Chenyang Wang, Yuanqing Yu, Weizhi Ma, Min Zhang, Chong Chen, Yiqun Liu, and Shaoping Ma. Towards representation alignment and uniformity in collaborative filtering. In Proceedings of the 28th ACM SIGKDD Conference Knowledge Discovery and Data Mining, pp. 1816-1825, 2022.
Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. Deep & cross network for ad click predictions. In The 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 1-7,2017,
Xiang Wang, Xiangnan He, Meng Wang. Fuli Feng, and Tat-Seng Chua. Neural graph collaborative filtering. In Proceedings of the 42nd international ACM SIGIR conference on Research and development in Information Retrieval, pp. 165-1744 2019.
Xiang Wang, Hongye Jin, An Zhang, Xiangnan He, Tong Xu, and Tat-Seng Chua. Disentangled graph collaborative filtering. In Proceedings of the 43rd international ACM SIGIR conference on research and development information retrieval, pp. 1001-1010, 2020.
Runmin Wei, Jingye Wang, Mingming Su, Erik Jia, Shaoqiu Chen, Tianlu Chen, and Yan Ni. Miss- ing value imputation approach for mass spectrometry- based metabolomics data. Scientific reports, 8(1):663, 2018
Jiancan Wu, Xiang Wang. Fuli Feng, Xiangnan He, Liang Chen, Jianxun Lian, and Xing Xie. Self- supervised graph learning for recommendation. In Proceedings of the 44th international ACM SIGIR conference on research and development in information retrieval, pp. 726-735, 2021.
Jiancan Wu, Xiang Wang, Xingyu Gao, Jiawei Chen, Hongcheng Fu, and Tianyu Qiu. On the effec- tiveness of sampled softmax loss for item recommendation. ACM Transactions on Information Systems, 42(4): 1-26, 2024a.
Junkang Wu, Jiawei Chen, Jiancan Wu, Wentao Shi, Jizhi Zhang, and Xiang Wang. Bsl: Under- standing and improving softmax loss for recommendation. In IEEE 40th International Conference Data Engineering (ICDE), pp. 816-830. IEEE, 2024b.
Yuan Yao, Lorenzo Rosasco, and Andrea Caponnetto. On early stopping in gradient descent learn- ing. Constructive approximation, 26(2):289-315, 2007.
Rex Ying, Ruining He, Kaifeng Chen, Pong Eksombatchai, William L Hamilton, and Jure Leskovec. Graph convolutional neural networks for web-scale recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp- 974 983, 2018.
Jinsung Yoon, James Jordon, and Mihaela Schaar. Gain: Missing data imputation using generative adversarial nets. In International Conference on Machine Learning, pp. 1689-56698 PMLR, 2018.
Chang Zhou, Jianxin Ma, Jianwei Zhang, Jingren Zhou, and Hongxia Yang. Contrastive learning for debiased candidate generation in large-scale recommender systems. In Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 3995- 2021.
14

---

Published as a conference paper at ICLR 2026
A ILLUSTRATION OF THE DEQL FRAMEWORK
The loss of EDLAE and extended EDLAE a special case of DEQL obtained by taking specific (D )
Their solutions are given in and (10 (derived from (8). and the complexity can be reduced to (nn²) )
by constructing a Fast Algoritim (Section 4 based on Miller's matrix inverse theorem Miileer 1981
DEQL
Extended EDLAE
DEQL, defined in serves as general loss by
b> a 2 0
allowing arbitrary (D(+) 1 i=1-
EDLAE
Its solution is given in (8)
2 0
which costs O(r²)) in the worst case.
EDLAE a> 0,0
The original EDLAE by (ttekk 2020
Setting 2 2 0 place greater emphasis dropped entries.
Only provides the solution b
Figure 2: Comparison of the closed-form solution sets of DEQL and EDLAE. The white region represents the set of original EDLAE solution, and the green region represents the remaining so- lutions covered by DEQL. The orange circle marks solutions derived from original EDLAE loss, whereas the cyan circle marks solutions obtained from the extended EDLAE loss (with hyperparam- eter choices > a, which go beyond the original EDLAE constraints but still yield valid solutions).
B MATHEMATICAL PROOFS
Proof of Lemma 3.2 H()) can be computed as follows. For any k. 1,
H = EA[(A @ A))
(A C R)ad = EA
sh Rsk Alt)
A Ra)
m
170
EA
A. Rod
Σ
BAlA 2k An A01 1 Rut
(24)
Note that A Asi. which depends Ag.. Since assume each Au is an ii.d Bernoulli random variable, A A2 is independent of s. Thus we can let be specific value of and rewrite (24) as
H EAIA A2 Σ
Rot = A
Define (()) Rax where G0D computed as follows: Given i, for any k. L.
RA A A2d. then H() ((() 3 RT R. G can be
2
if
A 24 = 1 and Ag 1 and
A 5s 0
A 2/ A
A
12
if
AsR - and A.ı 1 and A 21
0
otherwise
15

---

Published as a conference paper at ICLR 2026
Since
P( = and A2t = 1 and Azi =0) (1 (1- k=
(1 p)9p i
1 k=l=i
P( = 1 and Ax = and Ax = =
(1 p1) if #k=14iork+4sior444a
- p)
if
have
(1 - p)bb
if k= if o if k=lşi if i
(1 p)
G 2
plpa? +1 - p)
pa? + (1 - (1-1)311
On the other hand, v0 can be computed as follows. For any k.
EA @ R) (1) R... = EA > Au R
Ros = Σ EE ( A2d Rot Rei
68
- 2 Re R.i
Define (0) E R where uk
EAIAAt then we can write v(() = u(() 3 R Rai. u can be
computed as follows: Given any k;
(11 p))
if =
EA
A
(11 p)pa? 2 + +11-p)21'
if k
Proof of Theorem
3.3
Observe that ( can be decomposed as the sum of two matrices:
G( (1 M()) + (1 p)ppa (7() (()
where
if -=== if k i / k: or +i
kt
1 1
(1 - p)2
i
0
kl= o
k=
1-
ii
for k. E (1,
We can show that for any i, 6)) is positive definite and (1) is positive semi-definite: Let ,2,,.., ...,
[21, F2,
E RR,""
n
x 10
34 (1 P)
Dj
+ p1
-p)
Σ
3
0 for any x + 0
1
Σ Dj
+ p
20
for any x
Hence, (+)) is positive definite if 0,b > 0 and 0 < < 1.
16

---

Published as a conference paper at ICLR 2026
RT R is positive semi-definite matrix. If no column of R vector, then all diagonal elements of R" R are positive. By the Schur product theorem (Theorem 7.5.3 (b), (Hom & Johnson, 2012 G is positive definite and the diagonal elements of R" R are all positive, then H GO Rd positive definite.
Proof of Thhhorrmm333 Since the W* in [3) has zero diagonal, we only need verify the equivalence of non-diagonal elements. Let us write (C.:)_; as Coi, then Wai.-s by is expressed as
1 1 p Cj Ch.
W -i
Thus, our goal is to prove
(1
1
C+i,-i for any i (1,2,...,11)
(25)
1 Cii
To show this, first,
10 (H 1 (G (R SPTRI R -1)- (11 p)pa? (R Rui)
p)pa? G T (R (R
Rui)
(RRR) (RRRR)s+
P 1
(R" R
(26)
1-p
1-p
Next, remember that C-1 = R- 1- R Since no column of R is zero vector, I O R R positive definite, thus C-1 is positive definite. By the properties of matrix inverse, for an invertible matrix F. if we swap the i-th and j-th rows (or columns) of E and get F", then F²-1 is equivalent to the matrix formed by swapping the i-th and j-th columns (or rows) of E Therefore, suppose (C-1)(#) is obtained from C-1 by first swapping the kth row with the (k + 1)th row for i,i I..,. n sequentially, then swapping the kth column with (A: + 1)th column for k i, + 1 ... sequentially. We have
[R R) + 1-p T
R.i)- T-p (R2 R)
(C-1)()= =
Roi)
and
M
C
(co-))00) C(i.
C
Ca
where M an (n- 1) x (n - 1) matrix that we are not interested in.
By the symmetric block matrix inverse (0.7.3, (Horn & Johnson 2012)), we know that
A B
A A -
-A-1B"S--
B
D
-S-1BA-1
S-1
where 5 D BA -TT
Let A = -(1 RR) - 2 (R R) -). B" = -RR and S-1 = Oii we have
BT -==- -----"" 1-p (
P T (R ()-)) (RT R.:) Cu (27)
Combining (27 and 26 we get
255
thereby completing the proof.
17

---

Published as a conference paper at ICLR 2026
Proof of Proposition 3.5
(a) Similar to we can expand objective function of (4) as
(B(W) + AlA 1
( (W+i),
where
==1
hg
(W.s) = W (1) Wes 2 + .ll + EaNS
A (0)
Rvi
T
Hence,
0 8W..
220 -2yli) + 2AW oz. and the solution
22yl1)
ON aW.:
0 becomes
W (1 +11 (())
(28)
The optimal W is obtained by solving W., via 28 for all i. The solution is unique since each hg strictly convex.
(b) 15 is equivalent to solving for the stationary points of the following Lagrangian function L(W, μ4) = LB(W) + n² diag(W)
where μ E R". Since
C(W, 11) Σ
10 "B (W Mi);
where
N (W (4)
H W.; 2W (0
1.W. EANB
R A Rui
the solution (W, of the system of equations
TC T
Ac T
OC
T
Oc
ong
ah
T
ahg
(h(n) T
= 0
aW
aw,, +1
OW.2
OW. mm
aw.1
aW
....
dW m
10 T 0p
OC dC
OC T
Oh (1)
oh
ah (m) T oh Opn,
1pm
0p2
is given by taking
T
0h
2H (1) W.; 20 p1,((0 =0
(29)
aw.:
ah
(i)
W;: 0
(30)
0p;
for 1,2,..., n. Solving
we get
W4. = (1) (1)) jmillo)
(31)
Combining 330p and (11
have
2 u. (1) =0 = p4 = (H() (0))
W4 (H ()))
(32)
(H(1) 110));
Finally, plugging
33 into
31
we get the solution of W For any i,
((0) (H o(0): (00);(()))) ( (
(H -1)())))) (H( 100) H(() -1
: (1) (v
(33)
(H() (100) 100);
The solution
33 is unique. By second order sufficiency conditions (Section 11.5, (Luenberger & Ye 2008)), one show that any W that minimizes L(W, (1) is a strict local minimizer. Thus, the solution (33 gives the global minimizer.
18

---

Published as a conference paper at ICLR 2026
C IMPROVED COMPLEXITY FOR THE FAST ALGORITHM
As discussed in Section + when using basic algorithms for matrix multiplication and inversion, our Fast Algorithm computational cost of (m + n)n²). The main bottleneck lies in the precomputing stage: the m x product R" R costs (mn?)) the n x inversion Ho costs O(n*); and multiplying Ho* with n x n matrices @ RD R and G1 @ R costs O(n²). The following corollary shows that these costs can be reduced when advanced matrix multiplication and inversion algorithms are applied.
Corollary C.I. (a) If R sparse, contains only integer elements, and has nonzero elements (max(m, n) < le < mn), then R2 R can be computed with complexity O((2k + n2)3..36).
(b) The cost of computing Ho! can be reduced to ( but cannot be improved beyond (nn log
(c) The cost of computing Ho O R R) and Ho (U Q R) can each be reduced (n22370) but cannot be improved beyond Ω(n2 ((n2llonn)
Proof:
(a) By the analysis of (Abboud et al. 2024), consider the multiplication of sparse integer matrices Ar and Byx=, Let min the total number of nonzeros in the inputs A and B, and let mour be the number of nonzeros in the output AB. If min 2 max(æ, y, 2), then the matrix multiplication can be computed with complexity O( mam)) L.31G).
For computing R. R, we have = 2k and Moat I n², so the total complexity is
(b) This proof mainly follows Tveit 2003). By Theorem 28.1 and Theorem 28.2 in (Cormen al. 2009). let [n) be the complexity of inverting any n X n nonsingular matrix, and M (n) be the complexity of multiplying two n x n matrices, then T(n) (M)).). That is, the complexity of matrix inversion is asymptotically both upper- and lower-bounded by the complexity of matrix multiplication.
Using the Coppersmith-Winnt known algorithms for multiplying two n x n matrices, we have M(n) = O(n2376), which gives the upper bound (nx O
algorithm (Coppersmith & Winograd 1987), one of the fastest
Moreover, Raz 2002) proved that the complexity multiplying two n x n matrices cannot be better than (n) D n" log n). Thus [(n) a(n log n).
(c) Following (b), we have the upper bound M (n) = (n233766 and the lower bound M (n). 2(n n).
Corollary C.1 shows that it is possible reduce the complexity of the Fast Algorithm to O((k + n2)1..46 276 by choosing efficient algorithms for matrix multiplication and inversion; however, it is impossible to reduce it below Ω(n² log n)) It is easy to check that the same conclusion also applies to computing the closed-form solutions of EASE and EDLAE.
In the proof, the algorithms of Coppersmith-
and Abboud al. used show that this low complexity is theoretically achievable; however, they may not be practical to implement.
D DEQL WITH LOW-RANK CONSTRAINT
This section discusses the closed-form solution of under low-rank constraint of W. Given the rank & (k S n), we would like to solve
argmin lD( w
Σ (XXY)_D [||Y, - XW. s.t. rank S k
(34)
19

---

Published as a conference paper at ICLR 2026
Theorem D.I. Suppose [xx, (- [X X] is independent of and denote
Fgg (x.))DD X x]
Eay (x,))=aa [X Y.1), (X.))=D()) [X Y-2) . B. (x,,) ( Ymll
If Ezz is non-singular, then the closed-form solution of 34
is given by
W
E-1/2 2x
1/2
(35)
um
[ai
Here, let Emy
02
V be the singular value decomposition where 0i
Jn_
T2.. 2 On we denote
Fize
5-112
Sury
Σ J.U.VV..
k
Proof:
Denote y = B(X,Y)~D(() V By 7
(D(W)=
W Wai - 2W (Emy) ())
P 2///2 T Wo (x) En1/ (Eau) + y
Ex1 21/2 xx
+ 1/(0) (Eag) End (Ergy (Emy)xil 1|2
n 52zz
///
1/11/2 D
(())) -(Err) Dmd (Eery)
By YeckartYoung-- MoouggMMrssy theorem,
Emml/ E
argmin
Q -Ear
5-112 Eeny
for any
rank(Q)
Rex=, Therefore, 11/2 = E011/2 Exny
W
1-1/2 /2
E-1/2 Exry
Note that the low-rank solution 35 is not applicable when Exzx B(x,))
X X] depends
If Exz varies with then in the proof E
D1/2 W.:
Ez1/2 (Exy)+1
cannot be combined
into
11/ w E- 1/2 Σ Ezy
The low-rank solution 33 is applicable to EDLAE only when taking = b: by I0 Eps is
represented by ())); by Lemma 3.2 if a = b, HO will have all diagonal entries being (1 p)
and all off-diagonal entries being (I (I- 2) for any thus being independent of However, when of H will depend i, making (34 not applicable.
E RELATED WORKS
The evolution of collaborative filtering (CF) in recommendation systems has undergone several key paradigm shifts. In its early stages, neighborhood methods dominated the field, with influential works such as user-item KNN approaches (Hu et al. 2008) and sparse linear models (SLIM) (Ning Karypis 201T) setting the foundation. However, the Netflix Prize competition marked turning point, accelerating the adoption of matrix factorization (MF) techniques, which offered improved scalability and latent feature learning (Koren et al. 2009). These models aim to solve matrix completion problem, which has been extensively studied theoretically (Candès & Tao. 2010
Recht 2011
Foygel et al. 2011 Shamir & Shalev Shwarr-S 2011).
The rise of deep learning (LeCun et al. 2015) further revolutionized the landscape, introducing more expressive neural architectures. Among these, graph-based models gained prominence, in- cluding Neural Collaborative Filtering (NCF) (He et al. 2017), which replaced traditional MF with
20

---

Published as a conference paper at ICLR 2026
neural networks, and later refinements like Neural Graph Collaborative Filtering (NGCF) (Wang al. 2019) and LightGCN (He et al., 2020). which explicitly leveraged graph structures for higher- order user-item relationship modeling. Simultaneously, industry-scale solutions emerged, blending memorization and generalization through hybrid architectures such as Wide & Deep (Cheng et al, 2016), DeepFM (Guo al. 2017), and Deep & Cross Networks (DCN) (Wang al. 2017). which automated feature interactions while maintaining interpretability.
Alongside the ongoing research that explores various models to enhance recommendation perfor- mance, the research community has gradually recognized the importance of gaining a deeper the- oretical understanding of loss functions (Terven et al., 2025; Wu al.) 2024b). These theoretical investigations seek to reveal the fundamental principles and mathematical underpinnings that govern the behavior and optimization direction of recommendation systems, thereby advancing our over- all understanding of how these systems function. BPR (Rendle et al., 2009). Early methods often adopted pointwise L2g loss over observed ratings or implicit feedback (Hu et al.) 2008). which is simple and analytically tractable. Later, pairwise ranking losses such BPR (Rendle et al. 2009) became popular for top-K recommendation, optimizing relative preferences between positive and negative items. Softmax- listwise losses, such as sampled softmax (Jannach et al. 2010) were introduced to better align with ranking metrics like NDCG. In recent years, contrastive learning frameworks (Zhou al. 2021 Wang al., 2022) have gained prominence as powerful and effec- tive approach, particularly in unsupervised recommendation scenarios. Notable studies such as (Li al. 2023
Liu et al. 2021 have further showcased their effectiveness in this domain.
Notably, recent studies Sttekk 2019 2020 Moon et al. 2023) have shown that well-tuned linear models can outperform deep nonlinear models (Liang et al. 2018) on sparse implicit-fedbbak data, challenging the assumption that greater model expressiveness necessarily leads to better perfor- mance. Moreover, these models are typically neighborhood -ased and item-based, offering several advantages over deep learning approaches: more accurate recommendations based on a small num- ber of high-confidence neighbors, substantially lower training cost, more interpretable recommen- dation mechanisms, and greater stability (Nikolakopoulas et al. (021).
LAES are one type of the linear recommender models. One of the earliest LAE model SLIM (Ning& & Karypis 2011), which trains the loss || ReW 1 with L1 and Le regularizers, together with zero-diagonal and non-negativity constraints on W. EASE (Steck) 2019) simplifies SLIM by retaining only the L2 regularizer and the zero-diagonal constraint. EDLAE (Steck) 2020) instead employs dropout and emphasis as an alternative strategy to mitigate overfitting. ELSA Vančura et al. 2022) construct the model W with zero diagonal by enforcing W AAT - I for some
matrix A subject to ||At..13 = 1 for all i. (Moon et al. 2023) shows that a strict zero-diagonal constraint does not always yield the best performance, and that replacing it with a diagonal bounded by a small norm during training can improve results.
Finally, an important research direction for LAE models interpretability. LAB-based architec- tures provide a uniquely transparent mapping between input and output representations through a single linear operator W, where each element Wi, quantifies how item i contributes to predicting item j. A larger magnitude of Way. regardless of sign, indicates a stronger relatedness from i to (Spišák et al. 2024). This white-box structure makes LAES inherently interpretable compared with matrix factorization and deep neural recommenders, whose latent dimensions are unidentifiable or highly nonlinear. This interpretability direction aligns with broader developments in machine learn- ing, where linear and sparse representations are increasingly used to reveal structure inside complex neural systems. In particular, Sparse Autoencoders (SAEs) trained on large language models have been shown uncover highly interpretable and often monosemantic latent features Cunningham jet al. (2023); [Fll et al.] (2025); Demircan et al. (2025). These findings extend earlier insights that deep activations can be linearly decomposed into disentangled semantic directions, principle also supported by linear probes AAlinn & Bengio (2017). Complementary approaches such as LIME Ribeiro et al. (2016) further demonstrate how local linear surrogates can explain the predictions of arbitrary black-box models, reinforcing the idea that linearity provides a powerful lens for inter- pretability. Building on this insight, our proposed DEQL framework extends the explanatory power of LAES,, preserving their interpretability while enhancing expressive capacity.
21

---

Published as a conference paper at ICLR 2026
F SUPPLEMENTAL EXPERIMENTS
F.1 STATISTICAL SIGNIFICANCE TEST
Since the DEQL model is obtained via closed-form solution rather than gradient descent, each b corresponds to unique output model, thus there is no randomness in the training process. The only source of randomness arises from the train/validation/tes data splitting. So is sufficient to show that the performance improvements are unlikely affected by this type of randomness.
Table 2 reports results on a single dataset split. We fix the DEQL(L2) and DEQL4L22+zero-dia models from Table 2 generate five random train/validation/tess splits for each dataset, and evaluate each model on each split. The mean performance and standard deviations are reported in Table 4 and pairwise t-tests are presented in Table 5 These analyses consistently show that the performance improvements of DEQL(L2) and
are stable and statistically significant.
Table 4: Mean performance (+ standard deviation) of DEOLLL2+Zerodiag) DE2L(L2), and ED- LAE.
Maki
Gures
Boals
Goadila
M8 301
Nerto
BROK diaeagg
0/364 axxm
1732 axe
1513 VIM
3371 Ja0m
B 9913 s00c
3657 1x0e:
3805 uo
BAM44 ccmes
2-33.7 1aws
Table 5: Pairwise t-tests results at significance level of a 0.05. The null hypothesis A < B means that method A performs no better than method B. Each cell reports the decision to reject (Rej) or accept (Ace) the null hypothesis and the corresponding p p-value. If p-value < a, the null hypothesis rejected (i.e., the evidence supports that performs better than B).
Null Hypothesis
Games
Beauty
Gowalla
ML20M
Netlix
MSD
RC-20
N
R 20
NE
RGP20
No:20
Re
No20
Re:20
N0:20
Rd
No 20
Rej
Rcj
Rej
Rej
Rej
DEQUAL2) & 2EQUA
(p-tact11
1p- m0u
Rej
Apc
Rej
Rej
Rej
Rej
Rej
Rej
Rej
Rej
Rej
(1Q) x EDLAE
(p-201s7)
8078)
(WNIEE))
(21000))
(W00001)
qp-1 900))
(W00000
qp- 6901)
(w00001)
(p4 000
F.2 LEARNED DIAGONAL VALUES IN DEQL(L2)
We visualize the distributions of the diagonal values learned by DEQL(L2) in Figure 3 The results show that the diagonal entries of the optimal DEQL(L2) models are typically small, with the vast majority remaining close to zero and sharp modes generally falling within the range of 0.01-0.10 across datasets.
This pattern indicates that even without strict (W) 0 constraint, the Ln regularizer is still able to effectively suppress the diagonal terms, preventing them from growing into large. This consistent with the findings of (Moon et al. 2023), which demonstrate that relaxing the zero- diagonal constraint to allow small diagonal values can enhance performance.
Moreover, the optimal models Yelp2018 and AmazonBook exhibit relatively larger diagonal values than those on other datasets, suggesting that the model places greater emphasis on the self- reconstruction of remained items. This observation may help explain the counterintuitive results in Figure where the b/ ratio corresponding to peak performance on these two datasets exceeds 1.
F.3 TIME AND MEMORY COST
In this section, we compare the time and memory costs of DEQL. with existing LAE-based and deep (earning-base models. All models are evaluated the Yclp2018 dataset. The LAE-base based models are trained on CPU and solved via closed-fome solutions, whereas the deep learning based models trained GPU via gradient descent.
The results are presented in Table which show that LAE-based methods (including DEQL) exhibit higher memory consumption but lower training time than deep learning- baasing-based methods, even without using a GPU. Note that LAE-based models and deep learning models follow distinct computational paradigms. Deep baaning-based models train via batch gradient descent, loading only small batches
22

---

Published as a conference paper at ICLR 2026
20023
100
14030
1593
11509
12500
1080
2093
1000
2.6
0.5
0.4
0.
0.3
044
0.3
0.7
24
05
Comullc
mode
1856
mode
mode 6002
mode 2011
Court
9.5
8:31 Games
0.2
CA
6.0
9.2
94
0,5
mWWd 1641
Figure 3: Diagonal Value Distribution Across different datasets
of data into GPU memory at each step, which keeps memory usage around 1GB but requires many iterations, leading to longer training time. In contrast, LAE-based methods load the entire matrix into CPU memory to compute its inverse part of closed-form solution, which may require 80GB for datasets like Yelp2018. 88. This space-time trade-off enables the entire training process to complete much faster.
Unlike GPU-intensive GNN-based models such as LightGCN or SSM (GNN), DEQL models are well suited for memory-limited CPU-only environments. Modern CPU servers typically provide 500 GB 1 TB of RAM, suggesting that memory limitations are unlikely to pose practical concerns. Moreover, as discussed in Appendix G.3 moderately sized DEQL models (as well as other LAE models) can be readily adapted to GPU environment for further acceleration.
Table 6: Approximate Training time and memory usage on Yelp2018. Deep based models mainly consume GPU memory, while others rely on CPU memory.
Model
Time (min)
Memory (GB)
Memory Type
LightGCN
30
GPU
SimpleX (GNN)
130
GPU
SSM (MF)
13
1
GPU
SSM (GNN)
13
1
GPU
DLAE
70
CPU
EASE
2
60
CPU
EDLAE
70
CPU
DEQL
6
80
CPU
DEQL (L2+zero-diag)
8
80
CPU
DEQL (L2)
80
CPU
23

---

Published as a conference paper at ICLR 2026
G DISCUSSIONS
G.1 LIMITATIONS
One limitation of our work is that DEQL is currentl used only as an optimization tool, providing closed-forn solutions under given hyperparameters. However, does not offer theoretical guidance on which hyperparameter choices lead to improved testing performance. As a result, hyperparameter selection still relies on empirical tuning, and theoretical understanding remains underexplored. Our experiments show that datasets with larger item-user ratio tend to perform well under large b (Section 5.3
G.2 CONNECTION BETWEEN EDLAE OBJECTIVE AND MEAN SQUARED ERROR
In evaluation, the interaction matrix R typically adopt hold-out split (Section 7.4.2, (Aggarwal
2016 Let E {0, 1) mxx be mask matrix, and BR denote multivariate Bernoulli distribution over A and conditioned on R, such that each A.j independently satisfies P(A.; = R. = 1) P(A.j Ri 1) =
and P(A., = 0 RR, 0) for a given (0, 1). Then A @
represents the random hold out operation out. Given any fixed model W and dataset R. the Mean Squared Error (MSE), a commonly used
Rt: for any Raj 1, if A.j 0, then Ra is held
evaluation metric (Section 7.5.1, Aggarwal (016))), can be viewed a function of A:
1 |
MSE (A)
[11 - ()- (A R)W)
(36)
where 1 E (1)mXX denotes the all-one matrix and |A equals the number of non-zeros A. When MSEw,R (A) is used as the test error, we draw a realization A BR and evaluate it.
The objective function of EDLAE can be interpreted as an evolution of (36) obtained by
Removing the scalar 7
2. Replacing A with emphasis matrix A E {a, b)mx,, If (1 A); 1, then Aij a; (1 A, = O, then As = b, where 2 This is equivalent to placing greater emphasis on held-out (dropped) entries.
3. Relaxing BR to B and taking the expectation over B, where ~ indicates that the entries Ag are i..dd Bernoulli random variables with P (Aij 0) p and P (Aig =) p.
Consequently, the objective can be written as
f(W) = EASB [[A 3 (R
(A c W)
(37)
This observation shows that the EDLAE training objective aligns with the MSE used for evaluation. Moreover, previous empirical studies have shown that models with lower MSE are typically asso- ciated with better ranking performance, including a higher probability of relevant items appearing top-ranked positions (Koren) 2008). In addition, (Guo et al., 2025) derive a generalization bound based a relaxed MSE and show that tighter bound often correlated with improved top-K ranking metrics such as Recall and NDCG.
Hence, might intuitively expect that, given the alignment between EDLAE and MSE, minimiz- ing the EDLAE objective would improve Recall and NDCG, However, our experimental results in Section 5.3 show that this intuition does always hold, models in the case, which no longer strictly align with MSE, can in some cases achieve superior Recall and NDCG performance.
G.3 PRACTICAL SCALABILITY DEQL
The LAE models produced by DEQL other methods are typically represented by an X matrix. Here, practical scalability concerns how to handle, for large n, both the inefficiency of the Fast Algo- rithm (Section 4 used to compute the model and the associated memory prohibitivity issues. Since computation is often performed on GPUs which generally have much smaller memory capacity than CPU RAM discuss the scalability challenge at two levels of memory prohibitivity:
24

---

Published as a conference paper at ICLR 2026
1. The matrix fits in CPU RAM but cannot fit in GPU memory: At this level, do not need to impose rank constraints, and the model can remain full-rank. The issue is computational: the Fast Algorithm depends matrix multiplication and inversion, which are too slow CPU. Using the GPU would accelerate these operations, but matrices such as R or R R are too large to load into GPU for single-pass computation. Instead, we can use block matrix multiplication and block matrix inversion. These methods partition R or Rd R into blocks that fit in GPU memory and can be processed sequentially.
2. The n. X n. matrix cannot fit in CPU RAM: of a full-rank model. Importantly. the number of parameters in an LAE can be freely scaled. For example, ELSA /Vanuura et al. 2022) represents the model as A - I E R=xx where A E Rix", Although the final model is still an n x n matrix, the number of parameters depends on the size of A, which can be flexibly controlled through 1. Choosing I < produces a low-rank LAF. Moreover, as shown in Appendix [D a low-rank DEQL closed-form solution can also be obtained in the restricted case b.
In this case, a low-rank LAE can be used instead
Additionally, to accelerate computation, one may replace the standard (n²)) matrix multiplication algorithm with Strassen's 's algorithm, which runs in (12221)) Unlike the Coppersmith -errsmmttt-inngradd algorithm discussed in Appendix Strassen's algorithm remains practical to implement.
G.4 ADVANTAGES OF CLOSED-FORM SOLUTIONS
DEQL provides a framework for computing and analyzing closed-form solutions. A closed-form solution guarantees the global optimum of the training objective, but it does not necessarily yield the best test performance, as it may overfit the training data. In contrast, gradient-based optimization can rely on early stopping as regularization technique (Yao et al.) (2007) to mitigate overfitting, in which case the resulting model does not minimize the training objective.
However, the fooeedffrmm solution has distinct advantage in hyperparameter tuning. When hyper- parameters are fixed, gradient- pradieet-based training typically produces non -etermmiiiiiiicc models due the reliance on early stopping, which introduces randomness in addition to the randomness from data splitting. This compounded uncertainty makes harder to assess whether a particular hyperparam- eter setting truly leads to good generalization.
By contrast, the closed-form solution is fully deterministic for any fixed hyperparameters, so the only source of randomness stems from the data itself. This substantially reduces uncertainty in model evaluation and makes hyperparameter tuning more reliable. Consequently, closed-form solutions enable cleaner hyperparameter selection and facilitate reproducibility.
G.5 LAES VERSUS DEEP MODELS
Deep neural networks have become popular in industrial recommender systems due to their flexi- bility: their depth and width are not constrained by the dataset dimension n. In contrast, LAES typically represented by an X matrix, inherently tied to n. Although their parameter count can be scaled while preserving the n n size such as the ELSA model described in Appendix G.3 these models remain comparatively shallow. As result, deep models generally benefit from more flexible parameter scaling and architectural design, which often yields stronger representation power.
However, this does not imply that LAES always underperform deep networks. Empirical results show that LAE models outperform deep neural networks on sparse datasets (Steck) 2019 2020
Moon et al. 2023). Such sparsity is common in recommender datasets (Table I). since most users and items are associated with only a few ratings. LAA models have been shown to effectively capture item-item correlations using small number of high-confidence neighbors, thereby achieving high accuracy (Nikolakopoulos et al. 2021). Many small- to medium-scall e-commerce platforms have abundant interaction logs but limited or side features scenario that characterizes a substantial portion of industrial deployments. In such environments, LAE--syle models and matrix factorization remain both effective and operationally attractive (Dacrema et al.) 2019).
Moreover, unlike deep learning based models, LAES are typically less affected by the constant addition of users, items and ratings, which are typically observed in large commercial applications.
25

---

Published as a conference paper at ICLR 2026
For example, once item similarities have been computed, item- system readily make recommendations to new users, without having to re-train the system (Nikolakopoulos et al. 2021
Finally, linear models are typically casier to analyze than deep models due to their simplicity and structural stability, which makes them widely used in interpretable Al (Ribeiro et al., 2016). In par- ticular, LAEs have been regarded interpretable recommender systems. (Spišák et al. 2024) show that the magnitude of each Wi. regardless of sign, represents the relatedness from item to item j; hence, item relationships equal to the weight from to i. These interpretable relationships are a key reason why LAE-style models remain prominent production systems, compared with deep learning-base models whose learned weights are often difficult to explain. Although DEQL was originally developed to enable closed-form theoretical analysis for LAE models, the resulting LAE solutions can also naturally leveraged for interpretability at the application level.
directed graph, noting that the weight from to j is not necessarily
G.6 BROADER SCOPE OF LAES AND DEQI.
Below, we highlight several non- Recommemmeederr Systems application domains in which LAE-style models and DEQL. may be particularly useful. These domains overlap with LAB-related topics such as matrix completion, linear regression, and autoencoders:
Survey & psychometric modeling; genomics & biomedical panels: In survey research, user x item response matrices exhibit substantial missingness and heterogeneous exposure, a setting long modeled using linear or matrix completio methods (Mazumder et al.] 2010;; Yoon et al.) 2018). Similarly, genomics, metabolomics, and biomedical panels routinely rely on linear or low-rank imputation methods for patient gene and panel-level data, including mass-ppectromeetr-baass metabolomics (Stekhoven & Bühlmann] 2012 Wei et al., 2018).
Distributed Sensing & Sensor Network: In distributed sensing applications, sensortime matrices frequently contain missing measurements due to intermittent connectivity, power constraints, or sensor failures. Linear reconstruction and imputation methods continue to be widely used in these resource-constraaied environments, where computational simplicity and interpretability are critical (Rivera-Muñoz et al. 2022).
LLM Interpretability: Sparse Autoencoders (SAEs) have emerged as a key tool for interpreting LLMs (Fel et al. 2025 Cunningham et al. 2023). A typical SAE can be formulated follows [Cunningham et al. [202)). Let z be the input feature vector, å be the reconstructed feature vector, c be the latent vector, and W be the parameter matrix. The encoder is defined for some non-linear activation Φ) and the decoder given by = WFe. The model learns W such that and & are close under reconstruction loss, while is regularized to be sparse. The sparsity of e facilitates the identification of important features, thereby improving interpretability. This is similar to a LAE model, where W can be factorized into two matrices W A" B, with
= <(Wx + b)
c= Br be the encoder and â = AT c be the decoder. In this formulation, the latent vector c can be regularized be sparse if desired.
Finally, we note conceptual connection LLM preference learning (which is related to recom- mendation, though not necessarily restricted to linear models). While methods such as DPO differ substantially in their mathematical formulation, both LLM-based content generation and recom- mendation involve selecting or ranking items from large discrete spaces based on preference signals (Rafailov al. 2023 Rendle al. 2009).
Although direct application would require substantial methodological development beyond DEQL's current scope, these connections suggest interesting directions for future research.
LLM USAGE STATEMENT
We use ChatGPT solely for polishing writing at the sentence and paragraph level. The content and contributions of this paper were created by the authors. All text refined with ChatGPT has been carefully checked to avoid factual errors.
26


Reviews:
[{"review_id": "Y1hfREBWs7", "reviewer": "Reviewer_vBqm", "summary": "This paper generalizes the Emphasized Denoising Linear Autoencoder (EDLAE) by introducing Decoupled Expected Quadratic Loss (DEQL), which reformulates EDLAE under a statistical expectation perspective and extends the solution space to the full hyperparameter range $b\\geq 0$. Unlike the original EDLAE, which only derived a closed-form solution for $b=0$, the authors provide theoretical derivations for $b>0$ and propose an efficient algorithm—based on Miller’s matrix inverse theorem—to reduce computational complexity from $O(n^4)$ to $O(n^3)$ . Experiments on standard recommendation datasets show modest improvements over EDLAE and other linear baselines.", "questions": "Please refer to Weakness. Besides, the reviewer would also like to discuss the following interpretability potential: Linear/Sparse Autoencoders are recently used as interpretable models rather than competitive recommenders. Could DEQL be used as an explanation module? Does the closed-form solution provide interpretable weight structures or concept mappings?", "limitations": "", "rating": 4, "confidence": 5, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "1. Clear theoretical generalization of EDLAE. Reformulating EDLAE as an expected loss (DEQL) provides a cleaner statistical foundation and yields closed-form solutions for b>0, which were missing in prior work.\n\n2. Efficient algorithmic adaptation. The use of Miller’s theorem reduces computation from $O(n^4)$ to $O(n^3)$, making the extended solution computable in practice.\n\n3. Empirical improvements. Experimental results show that DEQL with $b>0$ offers consistent but moderate accuracy gains over EDLAE.", "weaknesses": "1. Limited novelty in the ICLR context. The main contribution is an extension of an existing model (EDLAE) from $b=0$ to $b>0$. While theoretically meaningful, this is somewhat incremental—more aligned with recommender system venues (KDD, SIGIR, RecSys) than a general ML conference like ICLR.\n\n2. Utilization of existing mathematical tools. The core efficiency improvement relies on Miller’s matrix inverse theorem rather than a new theoretical contribution. The innovation lies in applying this theorem to EDLAE’s structure, not in developing a new algorithmic principle.\n\n3. Scalability remains unsolved. Even after reducing the complexity to $O(n^3)$, the proposed method still requires computing and storing an $N\\times N$ item-item co-occurrence dense matrix $R^\\top R$, similar to its peers like EASE or SLIM. For large item catalogs, the method remains computationally and memory prohibitive. Thus, the method does not overcome the fundamental scalability bottleneck of linear autoencoders.\n\n4. Limited practical relevance of SAE-based recommenders in modern RS community. While the paper extends the family of Sparse/Linear Autoencoder-based models (SAE4Rec), such methods have largely fallen out of favor in industry-scale recommender systems. In practice, deep learning-based recommenders have generally surpassed linear autoencoders in both representation power and real-world deployment. Although proposes selective empirical results of deep-model baselines, the paper does not address whether DEQL can bridge this gap, scale to large industrial settings, or offer practical advantages beyond incremental improvements over existing SAE models.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "1o32DtiZkm", "reviewer": "Reviewer_XT64", "summary": "The paper addresses a specific limitation in a previously published model (EDLAE). The main contribution is the extension of the model's analytical solution to a broader set of conditions ($b>0$), which was not covered in the original work. This extension required a derivation and the development of a computationally feasible algorithm. The empirical results, showing improved performance, suggest that this extension is not merely a theoretical exercise but has practical benefits. The findings are of potential interest to researchers working on computationally efficient and effective models.\n\nThe paper's claims are supported by the presented derivations and experiments. The derivation of a closed-form solution for the EDLAE objective with $b>0$ is followed step-by-step, and the algorithm to reduce its complexity is well-explained. The experimental setup is described with sufficient detail to understand the procedure: the authors use publicly available datasets, specify the evaluation metrics (Recall@20, NDCG@20), and list the baseline models they compare against. This provides a basis for evaluating the empirical claims.\n\nThe paper is structured logically and is generally well-written. It begins with a clear problem statement, moves through the proposed mathematical solution and algorithmic details, and concludes with empirical results. The authors provide context by citing and briefly describing related works in linear autoencoders. However, the clarity could be slightly improved in a few areas. The presentation could be improved by adding more explicit motivation for certain design choices, such as the preference for a closed-form solution. The introduction of the DEQL framework could also be clarified by first showing how the underlying property of decoupling arises from the original objective function's structure.", "questions": "Can you elaborate on the practical advantages of finding a closed-form solution versus using an iterative method like SGD for this specific problem, particularly concerning training time, final model performance, and ease of tuning?\n\nHow sensitive is the model's performance to the choice of the $b/a$ ratio? Could you provide an empirical analysis (e.g., a plot of performance vs. $b/a$ on a validation set for one or two datasets) to illustrate the impact of this new hyperparameter?\n\nThe statistics provided under Tables 1 and 2 are helpful. To further support the discussion on how data characteristics like sparsity influence model performance, could you add a density (or sparsity) column to these existing summary tables? This would allow readers to directly correlate the quantitative characteristics of the datasets with the empirical results.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "The paper successfully generalizes the closed-form solution of the EDLAE objective from the special case of $b=0$ to the more general case of $b>0$, which increases the model’s flexibility.\n\nThe authors developed a practical algorithm for their proposed solution. They designed a method that reduces the computational complexity from $O(n^4)$ to $O(n^3)$, which makes their approach applicable to the datasets used in their experiments.\n\nThe claims are supported by experiments on multiple public datasets. The results, as shown in Tables 1 and 2, consistently demonstrate performance gains for the proposed method over the primary baseline (EDLAE) and show its competitiveness against other listed models.\n\nThe work provides further evidence that properly formulated linear models can be highly effective for recommendation tasks, sometimes outperforming more complex deep learning models, particularly on sparse data. This is a relevant finding for the field.", "weaknesses": "Limited Motivation for the Chosen Approach: The paper does not explicitly justify why a closed-form solution is preferable to a standard gradient-based optimization for this problem. While experts in the subfield might understand the rationale, a broader audience would benefit from a discussion of the trade-offs (e.g., computational cost, guarantee of finding a global optimum, reproducibility).\n\nLack of Analysis of the Key Hyperparameter: While the authors state in Section 5.2 that they perform a grid search over the hyperparameter $b$ to find the optimal values for their experiments, the paper does not present an analysis of these results. The reader is not shown how the model's performance changes as the $b/a$ ratio is varied. An empirical sensitivity analysis would be necessary to understand the practical importance of this new degree of freedom and to provide guidance on how to tune it.\n\nPotentially Unclear Framing of the \"Decoupling\": The DEQL framework is presented as a novel formulation. However, the decoupling of the loss function is an intrinsic property of the Frobenius norm and the structure of the EDLAE objective itself. The presentation could be improved by first demonstrating this property in the original objective and then introducing DEQL as a generalized term for such decoupled loss functions.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "0dASQMLGQ0", "reviewer": "Reviewer_bLSH", "summary": "This manuscript proposes a generalization of EDLAE named Decoupled Expected Quadratic Loss (DEQL), which extends the theoretical formulation and provides closed-form solutions for a wider range of hyperparameters range b>0. Moreover, this manuscript propose an efficient algorithm based on Miller’s matrix inverse theorem to to reduce the computational complexity for the b > 0 case. Extensive experiments demonstrate the advantages of DEQL on multiple benchmark datasets.", "questions": "1. The entire paper contains no figures or diagrams to illustrate the overall framework, problem formulation, or intuition behind DEQL. The heavy use of equations makes it difficult for readers to intuitively understand the workflow and conceptual contribution. Adding schematic diagrams of the model or algorithm would greatly improve readability.\n2. As the proposed method is a linear autoencoder-based (LAE-based) method, the most recent baseline considered in the paper is EDLAE (2020). Including more up-to-date LAE-based models for comparison would make the experimental evaluation more convincing and better demonstrate the advantages of the proposed DEQL framework..\n3. The paper mentions in the experimental section that only the ratio b/a affects the closed-form solution. Therefore, the authors fix 𝑎 a and perform a grid search over b. Since the paper emphasizes that b≥0 enables exploration of a larger solution space, it would be important to further analyze how different values of b influence the experimental results. A detailed discussion or sensitivity analysis on b would strengthen the empirical section and clarify its effect on model performance.", "limitations": "", "rating": 6, "confidence": 3, "soundness": 3, "presentation": 3, "contribution": 2, "strengths": "1. The paper clearly identifies a key limitation of the existing EDLAE model, the lack of analytical solutions for the case b>0. By introducing the DEQL formulation, the authors successfully generalize EDLAE and provide new theoretical insights into the design of expected quadratic loss functions. The proposed formulation is both mathematically and conceptually well-grounded.\n2. The authors provide detailed derivations, lemmas, and proofs to support the theoretical claims. The transition from the expected quadratic loss formulation to the closed-form solution is rigorous and contributes to a deeper understanding of linear autoencoder objectives. \n3. The introduction of the Miller-based algorithm significantly improves computational efficiency, reducing complexity from O(n4) to O(n3). This makes DEQL practical for large-scale recommendation datasets.\n4. The datasets used are publicly available, and the implementation details are well described. The code link is also provided, which enhances transparency and reproducibility.", "weaknesses": "1. The entire paper contains no figures or diagrams to illustrate the overall framework, problem formulation, or intuition behind DEQL. The heavy use of equations makes it difficult for readers to intuitively understand the workflow and conceptual contribution. Adding schematic diagrams of the model or algorithm would greatly improve readability.\n2. As the proposed method is a linear autoencoder-based (LAE-based) method, the most recent baseline considered in the paper is EDLAE (2020). Including more up-to-date LAE-based models for comparison would make the experimental evaluation more convincing and better demonstrate the advantages of the proposed DEQL framework..\n3. The paper mentions in the experimental section that only the ratio b/a affects the closed-form solution. Therefore, the authors fix 𝑎 a and perform a grid search over b. Since the paper emphasizes that b≥0 enables exploration of a larger solution space, it would be important to further analyze how different values of b influence the experimental results. A detailed discussion or sensitivity analysis on b would strengthen the empirical section and clarify its effect on model performance.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "pp4SS4zIRa", "reviewer": "Reviewer_9r8b", "summary": "This paper addresses a key limitation in a state-of-the-art linear recommendation model, the 'Emphasized Denoising Linear Autoencoder' (EDLAE). The original EDLAE work only provided a closed-form solution for the specific hyperparameter case of b=0, which limited the model's capacity.\n\nThe authors introduce the 'Decoupled Expected Quadratic Loss' (DEQL), a generalized objective function that reformulates the EDLAE problem from a statistical perspective. The main contributions of this work are:\n\n1. **Theoretical Generalization:** Using the DEQL framework, the authors derive a general closed-form solution (minimizer) for the expected quadratic loss.\n2. **Expanded Solution Space:** They apply this general solution back to the EDLAE objective, successfully deriving closed-form solutions for the previously unexplored hyperparameter regime of b>0.\n3. **Efficient Algorithm:** Recognizing that a naive computation of these new b>0 solutions is computationally prohibitive ($O(n^4)$), the authors propose a novel and efficient algorithm based on the Miller matrix inversion theorem.\n4. **Complexity Reduction:** This algorithm reduces the complexity to $O(n^3)$, making it practical and comparable to existing methods like EASE and the b=0 EDLAE baseline.\n5. **Empirical Validation:** Extensive experiments on six benchmark datasets show that the newly derived solutions for b>0 (especially when combined with $L_2$ regularization, termed DEQL(L2)) consistently and significantly outperform the original b=0 EDLAE baseline.\n6. Novel Insight: The paper also provides an interesting insight that the explicit zero-diagonal constraint is not strictly necessary for strong performance when b>0.", "questions": "1. **Sensitivity to b:** Related to Weakness 1, how sensitive is the model's performance to the choice of b (or the b/a ratio)? The grid search ranged from [0.1, 2.0]. Could you provide a brief analysis (perhaps with a plot) on the validation set showing how Recall@20 varies as b changes? Is there a clear 'sweet spot,' or is performance relatively flat across a wide range of b>0?\n2. **Statistical Significance:** The experimental gains in Table 1 (ML-20M, Netflix) appear to be well within the margin of error. Can the authors please provide statistical significance tests (e.g., paired t-tests) across multiple runs to demonstrate that the b>0 solutions are *truly* superior to the b=0 baseline? As it stands, the evidence suggests the gains are just noise from tuning an extra parameter.\n3. **Learned Diagonal Values:** You argue convincingly in RQ2 that the zero-diagonal constraint is not strictly necessary when b>0 and $L_2$ regularization is used (DEQL(L2)). This is a very interesting finding. In the DEQL(L2) model, what do the diagonal entries of the learned $W^*$ matrix look like? Are they small values close to zero (suggesting the $L_2$ norm effectively pushes them down), or do they take on meaningful non-zero values? This could offer insight into *why* the hard constraint is no longer needed.\n4. **Ablation of b vs. $\\lambda$:** The 'naive' DEQL (Eq. 9) performs poorly, and strong results are only obtained with DEQL(L2). This suggests the $L_2$ regularization is doing the heavy lifting, not the b>0 formulation. Could the b=0 EDLAE baseline achieve the same performance as DEQL(L2) if its $L_2$ hyperparameter ($\\lambda$) was simply tuned more aggressively over a wider range?\n5. **Scalability and Approximation:** The $O(n^3)$ inversion of $H_0$ is the main bottleneck. Have the authors considered or can they comment on using approximation techniques for $(H_0 + \\lambda I)^{-1}$? For example, would an iterative solver (e.g., Conjugate Gradient) or a low-rank approximation of $H_0$ be a viable path for scaling this method to n >> 50,000 items? This would break the pure 'closed-form' nature but might offer a practical path to larger datasets.\n6. Generality of the Algorithm: The efficient $O(n^3)$ algorithm relies on $H^{(i)}$ being a low-rank (specifically, rank-2) update to a common matrix $H_0$. This structure arises from the specific dropout/emphasis scheme of EDLAE. Is this low-rank structure a fundamental property of DEQL, or just a property of this specific instantiation? In other words, could other formulations of DEQL (i.e., different choices for $\\mathcal{D}^{(i)}$ in Def 3.1) lead to $H^{(i)}$ matrices that do not have this structure, making the Miller theorem speedup inapplicable?", "limitations": "", "rating": 6, "confidence": 3, "soundness": 4, "presentation": 4, "contribution": 3, "strengths": "- **Originality and Theoretical Quality:** The generalization of the EDLAE objective into the Decoupled Expected Quadratic Loss (DEQL) is an elegant and novel theoretical contribution. It provides a clearer and more principled statistical framework for understanding this class of models. Deriving the closed-form solution for the b>0 case is a non-trivial and significant extension of prior work.\n- **Practical Algorithmic Contribution:** A major strength is that the authors did not stop at the computationally intractable $O(n^4)$ solution. The development of the $O(n^3)$ algorithm using the Miller matrix inversion theorem is a crucial contribution that makes the entire theoretical framework practical and usable. This new algorithm maintains the same complexity as the original EASE and EDLAE (b=0) models.\n- **Clarity and Presentation:** The paper is extremely well-written and easy to follow. The motivation is clear, and the logical flow—from the limitation of EDLAE, to the general DEQL framework, to the specific b>0 solution, and finally to the efficient algorithm—is impeccable. The mathematical derivations are clean and well-explained.\n- **Significance and Empirical Results:** The empirical validation is thorough and convincing. The proposed b>0 solution (specifically DEQL(L2)) demonstrates consistent and sometimes substantial performance gains over the strong EDLAE baseline across all six datasets. The paper also reinforces the value of simple linear models, showing they remain competitive and, in some cases, superior to complex deep learning-based models while offering significantly faster (CPU-based) training.\n- Novel Insight: The finding (RQ2) that the zero-diagonal constraint—essential in models like EASE and EDLAE—is not strictly necessary when b>0 and $L_2$ regularization is applied is an important insight for future model development in this area.", "weaknesses": "1. **Lack of Theoretical Guidance on Hyperparameters:** The DEQL framework successfully expands the solution space to b>0, but it does not provide theoretical guidance on *how* to select the optimal hyperparameters (e.g., the ratio b/a, p, or the $L_2$ coefficient $\\lambda$). Navigation of this newly expanded solution space still relies on empirical grid search, which can be costly. The paper proves *that* b>0 solutions are better, but not *which* b>0 solution is optimal a priori.\n2. **Computational Bottleneck:** While the $O(n^3)$ algorithm is a significant achievement, it still relies on the inversion of an $n \\times n$ matrix ($H_0$ or $H_0 + \\lambda I$). For datasets with a very large number of items (n), this $O(n^3)$ step remains a significant bottleneck, limiting its scalability into the high tens of thousands. This is an inherent limitation of this class of closed-form LAE models, but it remains a practical weakness.\n3. **'Pure' DEQL Performance:** The core theoretical contribution is the 'pure' DEQL solution (Eq. 9). However, the experimental results in Table 1 clearly show that this pure DEQL model performs poorly and is often worse than the EASE/EDLAE baselines. The strong performance gains are only realized *after* adding $L_2$ regularization (DEQL(L2)) or both $L_2$ and the zero-diag constraint. This suggests the primary practical contribution is not DEQL *per se*, but *regularized* DEQL. This is not a major flaw, but the paper's framing could be slightly more precise to reflect that the b>0 solution *requires* regularization (like $L_2$) to be effective, though this is standard practice anyway.\n4. The work's key contribution is performance uplift. However, the experimental gains shown in Table 1 are not statistically significant. For example, on ML-20M, Recall@20 improves from 0.3925 to 0.3934, and NDCG@20 from 0.3421 to 0.3426. On Netflix, the performance is identical. On MSD, the b=0 EDLAE baseline actually outperforms the proposed DEQL(L2+zero-diag). These minor and inconsistent gains hardly justify adopting a new solution, let alone the extra hyperparameter tuning for b.\n5. Concurrent research is working to break this cubic complexity bottleneck, e.g., via scalable iterative methods, graph-based approximations, or randomized linear algebra. The paper's contribution lacks some theoretical or experimental comparisons in terms of effectiveness and complexity with these works.\n6. The paper adds a new hyperparameter (b) to an already complex tuning space (dropout p, $L_2$ $\\lambda$). This runs counter to the broader research trend of seeking simplification for robust performance. Other recent work suggests that the dropout and $L_2$ terms already provide sufficient regularization, making an explicit b parameter slightly redundant.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)