Schur's Positive-Definite Network: Deep Learning in the SPD cone with structure
Authors: Can Pouliquen
Mathurin Massias
Titouan Vayer
Venue: ICLR 2025 Poster (2025)
Keywords: sparsity
graphical lasso
lasso
deep learning
neural networks
Primary area: unsupervised, self-supervised, semi-supervised, and supervised representation learning
TL;DR: We propose a novel and generic learning module with guaranteed SPD outputs that can jointly handle additional structural constraints such as sparsity.

Abstract: Estimating matrices in the symmetric positive-definite (SPD) cone is of interest for many applications ranging from computer vision to graph learning. While there exist various convex optimization-based estimators, they remain limited in expressivity due to their model-based approach. The success of deep learning motivates the use of learning-based approaches to estimate SPD matrices with neural networks in a data-driven fashion. However, designing effective neural architectures for SPD learning is challenging, particularly when the task requires
additional structural constraints, such as element-wise sparsity. Current approaches either do not ensure that the output meets all desired properties or lack expressivity. In this paper, we introduce SpodNet, a novel and generic learning module that guarantees SPD outputs and supports additional structural constraints. Notably, it solves the challenging task of learning jointly SPD and
sparse matrices. Our experiments illustrate the versatility and relevance of SpodNet layers for such applications.

Published as a conference paper at ICLR 2025
SCHUR'S POSITIVE-DEFINITE NETWORK:
DEEP LEARNING IN THE SPD CONE WITH STRUCTURE
Can Pouliquen ENS de Lyon, Inria, CNRS, Université Claude Bernard Lyon 1, LIP, UMR 5668, 69342, Lyon cedex 07, France can.pouliqueneee
Mathurin Massias
Inria, ENS de Lyon, CNRS. Université Claude Bernard Lyon 1, LIP, UMR 5668, 69342, Lyon cedex 07, France ma tathhrrig mass i a irri... fr
fr
Titouan Vayer
Inria, ENS de Lyon, CNRS, Université Claude Bernard Lyon 1, LIP, UMR 5668, 69342, Lyon cedex 07, France titovan, vayer eesyerrinria.... inria.
ABSTRACT
Estimating matrices in the symmetric positive-defintt (SPD) cone is of interest for many applications ranging from computer vision to graph learning. While there exist various convex optimization-be based estimators, they remain limited in expressivity due to their model-base based approach. The success of deep learning motivates the use of learning-bused approaches estimate SPD matrices with neural networks in a data-driven fashion. However, designing effective neural architectures for SPD learning challenging, particularly when the task requires additional structural constraints, such as element-wise sparsity. Current approaches either do not ensure that the output meets all desired properties lack expressivity. In this paper, we introduce SpodNet, a novel and generic learning module that guarantees SPD outputs and supports additional structural constraints. Notably, it solves the challenging task of learning jointly SPD and sparse matrices. Our experiments illustrate the versatility and relevance of SpodNet layers for such applications.
INTRODUCTION
The estimation of symmetric positive-definit (SPD) matrices is a major area of research, due to their erucial role in various fields such optimal transport (Bonet al., 2023), graph theory (Lauritzen, 1996), computer vision (Nguyen et al., 2019) or finance (Ledoit and Wolf, 2003). While various statistical estimators, i.e. model-based, have been developed for specific tasks (Ledoit and Wolf, 2004; Banerjee et al., 2008; Cai et al.. 201 1), recent ad advancements focus on applying generic learning- based approaches to estimate appropriate SPD matrices with neural networks in data-driven fashion (Huang and Van Gool, 2017; Gao et al., 2020).
Training neural networks while enforcing non-trivial structural constraints such positive- definiteness is difficult task. There have been many efforts in this direction in recent years, often in an ad-hoc manner and each with their own shortcomings (see Section for more details). Building on the seminal work of Gregor and LeCun (2010) in sparse coding, a promising research direction involves designing neural networks architectures from the unrolling of an optimization algorithm (Chen and Pock, 2016; Monga et al., 2021; Chen et al., 2022; Shlezinger et al., 2023). In the case of SPD matrices, algorithm unrolling presents several challenges. First, algorithms operating in the SPD cone usually rely on heavy operations such retractions (Boumal, 2023), SVD line search (Rolfs et al., 2012). These operations do not integrate well into a neural network architecture, making these algorithms difficult to unroll.
Additionally, and more importantly, many applications require further structural constraints on the learned matrix. Flementwise sparsity is typical example of such constraints (Banerjee et al., 2008),

---

Published as a conference paper at ICLR 2025
iter.
iter. 2 NN("))
iler. 3
iter. 1
iter. 2
iter.
NN(
column-row + diag. updates
column-row diag. updates
E
S
E ESI
E
S
++
...
iter.
iter. P
SpodNet layer
SpodNet layer
Figure 1: A SpodNet layer chains p updates column-row pairs and diagonals using neural networks. The matrices remain SPD all times via Schur's condition.
which significantly increases the complexity of the task: learning functions that simultancously enforce SPDness and sparsity of the output is known to be challenging (Guillot and Rajaratnam, 2015; Sivalingam, 2015). There are currently no neural architectures that enable learning jointly SPD and sparse matrices.
In this paper, we bridge this gap and make the following contributions:
We introduce a new SPD-to-SPD neural network architecture that also supports enforcing additional constraints on the output. We refer to it as SpodNet for Schur's 's Positive-Definil Network. As particular case, we show that SpodNet is able to learn jointly SPD and sparse matrices. To the best of our knowledge, SpodNet is the first architecture to provide striet guarantees for both properties.
We demonstrate the framework's relevance through applications sparse precision matrix estimation. We highlight the limitations of other learning- based approaches and show how SpodNet addresses these issues. Our experiments validate SpodNet's effectiveness in jointly learning SPD-to-SPD and sparsity-inducing functions, yielding competitive results across various performance metrics.
Notation We reserve the bold uppercase for matrices θ, bold lowercase 0 for vectors and standard lowercase for scalars. The soft-thi tofethhresooling function is ST, (-) sign(-) max(I-1- y 0) for y 2 O; acts elementwise on vectors or matrices. On matrices, ||-I is the sum absolute values of the matrix coefficients. The cone of p by p SPD matrices is denoted Sp +
RELATED WORKS
We first review existing approaches for estimating SPD matrices, which can be broadly divided into three categories, all suffering the same limitation of not being able to handle additional structural constraints.
Riemannian approaches Riemannian optimization provides tools build algorithms whose iter- ates lie on Riemannian manifolds, such as the SPD manifold (Absil et al., 2008). Many have adopted these tools to design neural architectures that operate on the manifold SPD matrices (Huang and Van Gool, 2017; Gao et al., 2020; 2022). However, common and significant bottleneck of these methods is the use Riemannian operators, which are notoriously expensive to compute. Beyond this computational hindrance, and more importantly, current Riemannian methods not able to impose additional sparsity on the learned matrices without breaking the SPD guarantee.
SPD layers Neural layers with SPD outputs have also been proposed in Dong et al. (2017); Nguyen et al. (2019). A first line of work rely on the linear mapping X W, W.T (with the layer's weights W5 having full row-rank), while some others rely on clipping the eigenvalues their output. Unfortunately. there sparsity on the matrices without risking breaking their SPD guarantee. For instance, -endthhresolddng the oif-diagonal entries of SPD matrix does not in general preserve the SPD property (Guillot and
obvious ways to incorporate additional structure such elementwise
2

---

Published as a conference paper at ICLR 2025
Rajaratnam, 2012) and, more generally, elementwise functions preserving this property very limited (Guillot and Rajaratnam, 2015).
Unrolled neural architectures In order to ensure that network's 's output strictly respects some desired properties, successful direction is to unroll convex optimization algorithms (Chen al., 2022; Shlezinger et al., 2023). This unrolling procedure acts as an 'inductive bias" on the architecture, naturally forcing the model to explore suitable solutions within the space of imposed properties. In SPD learning, this approach has been exploited by Shrivastava et al. (2020) who unrolled an optimization algorithm to train neural networks to estimate inverses of covariance matrices. Whilst algorithm unrolling is very powerful approach to learn specific matrices, it proves difficult to actually enforce several constraints simultaneously on these matrices. We provide further details about the limits of this approach in Section 4.1.
3 THE SPODNET FRAMEWORK
We now introduce our core contribution, the SpodNet layer. Essentially, it is an SPD-to-SPD mapping parameterized by neural networks. A
Algorithm 1 The SpodNet layer
1: Input: in SP and Win O
SpodNet layer operates by cycling through the p column-row pairs individually by
2: for column i E (1 {1..... -P} do
3: Extract blocks: WI.. 11. W12, w22
4: Compute [+
= W11
W
updating the corresponding column-
5: New column 0 = f())
row with a neural network,
6: New diagonal value 0
g())
(ii) updating the diagonal element to satisfy the SPD constraint (see Figure 1).
01
O 1
7: Update θ = O w W + as in Equa-
Crucially, the neural network used at step (i) update the column-row to any value without compromising the SPD guarantee. Consequently, one is free exploit a spectrum of approaches for those updates, depending on other desired structural properties of the output matrix. In par- ticular, describe in Section 4 three specific implementations of SpodNet that enforce ele- mentwise sparsity for learning sparse precision matrices.
tions (2) and (3)
8: end for
9: Output: Oot SP
and W out = - out
Blocks
i= p
i p
i i= p 2
θ 11
1 = 0
022
3.1 ALGORITHMIC FOUNDATIONS
The key to preserving the positive-definiteness E 8 of the matrix upon after changing its i-th column and row is an appropriate update of the diagonal entry O₂ based on Schur's ' condition for rnsitive-defiittenss
In the following, + superscript value of this variable (e.g. outputted by a neural network). The following proposition shows that SpodNet's output is guaranteed to be SPD.
variable (e.g. o*) indicates update
Proposition 3.1.
Suppose the updated column-row pair the last one (i p). We partition O as
012 with O E (1P-1)x(--11) 012 E RP-1 O21 = 0 O22 E R. (1)
O21
82
(for generic column i, 11 refers + without its i-th row and i-th column, 012 is the i-th row of C without its i-th value, and O22 is Oz as illustrated below Algorithum 1).
Suppose that ② 34 Let u E RP-11 and respectively. Then, updating the i-th row and column of O as
0 be any vector and strictly positive scalar
11
t u A v u O -1u
A
0 A
T
preserves the SPD property, i.e. O
3

---

Published as a conference paper at ICLR 2025
Since positivity of @ is guaranteed for any choice of and as long SpodNet updates is to learn these quantities as function the current iterate, i.c. from the value of θ 11, 21 etc (explicit forms for each variant will be given in Section 4.2). For simplicity, though they can depend on additional information, we write u (()) and v = g()), with f and being learned mappings.
0, the principle of
Proof. Since @ is symmetric positive-definite so is its leading principal submatrix 111 It follows that + is well-defined, and obviously symmetric. Its positive-definitenss ensues from Schur's condition for positve-definitenss. Indeed O is SPD when 11 > and 052 - 0 [+n] > (Zhang, 2006, Theorem 1.12). The latter condition is ensured the left hand side is equal e, which is strictly positive by assumption.
Finally, one SpodNet layer chains updates of the form Equation (2), sequentially updating all column-row pairs one after the other summarized in Algorithm 1. A full SpodNet architecture then stacks K SpodNet layers, and, in practice, the first layer takes as input On (S + Ip) where S is the empirical covariance matrix. The overall architecture is schematized in Figure for K = 2.
The expressivity of SpodNet comes from the flexibility to use any arbitrary functions and g in the updates whilst guaranteeing that @ remains in the SPD cone. Namely, additional structure such as sparsity can trivially imposed on @ through f. Broadly speaking, constraints related to the values of the off- off-diagonal entries of @ and that can be imposed on the ouiputs of a neural network can be controlled directly. We next detail the computational cost of fully updating through one SpodNet layer.
3.2 IMPROVING THE UPDATE COMPLEXITY
The diagonal update of Equation (2) a priori requires inverting the matrix O1 which comes with prohibitive cost of (pp²) for each column-row update. Fortunately, we are able to leverage the column-row structure of the updates decrease the cost to a mere O(p2), by maintaining the matrix W - up-to-date along the iterations.
W
W12
Proposition 3.2. Let W be the inverse of O adopting the same block structure W
12
W22
Then [01]- Wn
w In addition, if O is updated as O+ following Equation with u g (), then the update of W defined by
w2
[0n]
0 0
[e Let
W+
((
(3)
10nl-'et (
1/g(O)
can be computed in O(p²) and satisfies [W + = +
Proof. By the Banachiewicz inversion formula on Schur's complement (Zhang, 2006, Thm 1.2),
[Oul-1
@.1l 00.001 [O.
10 '0,2
Wi
W12
[On
012
022
0 11
1012
02g 821 en]
10:2
W22
W22
021
022
0,2
021
9m
01
022
11
10
(4)
Identifying all blocks (namely g()) with 022 0z1 (11 10 w12 with
0
and
822 P21 11 2012
W22 with v( yields [Q,1]_ Wn waa W which can be computed in ((p²) if one has
access to W. This can be achieved using Equation (3), and involves only operations in (p²)). The
property O - [W+]-1 11 is satisfied by using the same inversion formula.
full update of O thus be achieved with cost O(p²) (p column-row updates of cost Opp))
4

---

Published as a conference paper at ICLR 2025
1.0
0.5
10 Zuil2
10
0.0
(Zunl
Density of Oot
-0.5
100
200
300
400
500
Epochs
Figure 2: GLAD's limitations. Smallest cigenvalue (orange), density degree (green) and relative discrepancy of the two matrices (blue) for an output (Zout, out) of GLAD, Out and Zout different, out is not sparse, and Zout is not positive-definit.
USING SPODNET TO LEARN SPARSE PRECISION MATRICES
So far, we have not specified which choices of column and diagonal updates f(@) and g(O) would use in practice. We now leverage the general framework of the SpodNet layer for learning SPD matrices with additional structure and propose three specific architectures for learning sparse and SPD matrices, with applications to sparse precision matrix learning.
4.1 INFERRING SPARSE PRECISION MATRICES
Consider dataset of observed signals æ .1..... Fn where each at; IR follows certain (centered) distribution with covariance matrix Σ E sh_ The problem of precision matrix estimation arises when attempting identify the conditional dependency graph of the p variables of this dataset. In Gaussian graphical models (Lauritzen, 1996), this graph is associated with the so-called precision matrix @ Σ E S S" this case G,j iff the variables i and j are conditionally independent given the other variables. This estimation problem involves determining θ given the empirical covariance matrix S x Σi=1 æ. In most practical scenarios, the matrix is sparse due to limited conditional dependencies, which leads to sparse SPD matrix estimation problem. For this problem, very popular estimator is the Graphical Lasso (GLasso) (Banerjee et al., 2008; Friedman et al., 2008) that solves
min - (e) + (S, )) + X|| |1,ff
where |[O||o,o denote the off-diagonal l norm O, equal to Σ 10yll The data fidelity term - log (et(@) + (S, θ is the negative log-likelihood under Gaussian assumption, while the lı penalty enforces sparsity.
be efficiently computed (Rolfs al., 2012; Mazumder and Hastie, 2012; Hsieh et al., 2014) but suffers from the known limitations from the model-based approaches (Adler and Öktem, 2018; Arridge al., 2019). To circumvent these limitations, two learning-bases approaches have been proposed. Belilovsky et al. (2017) introduced DeepGraph. CNN that directly maps empirical covariance matrices the graph structures (i.e. to the support of the precision matrix). We emphasize that this model does not estimate @ but only its support. Shrivastava et al. (2020) proposed GLAD, a neural architecture based on unrolling an alternating minimization algorithm for solving the GLasso. Based while learning step-size and thresholding parameters (see Appendix B.2 for more details). Out of these matrices Z is sparse and θ is SPD, but GLAD fails to learn single matrix that is both sparse and SPD. This limitation is highlighted in Figure 2 showing that, during training, O and Z differ, O is not sparse and Z is not SPD.
Lagrangian relaxation, GLAD iterates over two matrices Z and O
4.2 SPODNET FOR SPARSE SPD LEARNING
We now show how SpodNet overcomes the limitations current methods for learning both sparse and SPD matrices. By order of complexity, we introduce three new neural architectures, each corre- sponding to a specific choice of the functions f,9 for learning the values of u and v in Equation (2).
Our choices for f&g are inspired by an unrolling of proximal block coordinate descent applied to the GLasso problem, where the blocks are column-roo pairs described in Section 3. A proximal coordinate gradient descent step on the GLasso objective updates the value of 012 with
0 ST (812 w12)),
5

---

Published as a conference paper at ICLR 2025
Fraction of zero entries
100
0.8
Amax Imax/Amin
Amin
0.6
E2E
10
10
0.4
UBG
PNP
0.2
10"2
0.0
20
40
20
40
20
40
Epochs
Epochs
Epochs
Figure 3: Training dynamics of our 3 models (on test data described in Section 5.1). Left: The outputs of our models remain positive-definite Middle: The conditioning remains stable. Right: The outputs are sparse. Overall our models produce jointly sparse + SPD outputs.
where > Os a step-size. This stems from the fact that the gradient of + - log () @)is - + S (Boyd and Vandenberghe, 2004, Section A.4.1), 1)) which restricted the 012-block gives - 112 = W12+$122 Together with the fact that the proximal operator of the 1 norm is the lofttthresoolding (Parikh al., 2014) we get Equation (6). Because (full) proximal gradient descent on the GLasso is called Graphical ISTA (Rolfs et al., 2012), we coin Equation (6) Block- Crach-Graphical ISTA. We can now introduce our three architectures.
Unrolled Block Graphical-ISTA (UBG) First, we propose to learn the stepsizes and soft- thresholding levels when unrolling the Block Graphical ISTA iterations Equation (6). In learning- based approach, we use as updating function f.
SUBG 012 + ST (012-- - w12)),
where the step-size y = (6 > O and the soft- ooft-hressholing parameters A NN2 yt (s12 W12)) ) E 1R=-1 NN, and NN2 are small multilayer perceptrons with architectures provided in Appendix A.2. We emphasize that this is different from hyperparameter tuning, since y and A are predicted at each update instead being global parameters. UBG exhibits the highest inductive bias among the models, as its architecture is directly derived from unrolling an iterative optimization algorithm designed to minimize a model-based loss.
As highlighted in Section 3, one can plug any update functions and still get SPD outputs. This motivates extending UBG to a more expressive architecture.
Plug-and-Play Block Graphical-ISTA (PNP) Precisely, we extend UBG to a Plug-and- -aand-laay-like setting (Venkatakrishnan et al., 2013; Romano et al., 2017). In a nutshell, these methods replace the proximal operator in first-order algorithms by denoiser, usually implemented by neural network. In our context, this corresponds to replacing the soft-thresholding of UBG by a neural network R + RP-! that is, fPMP 012 + (012 - 7 w12)).
(8)
To promote sparsity of the output of SPMP, the last layer of V performs an elementwise soft- thresholding. The parameter for this soft-thresholding is also learned from data and given by the same multilayer perceptron that predicts X in UBG (Appendix A.2).
End-to-end updates (E2E) Finally, propose fully-flexible architecture without any algorithm- inspired assumptions. Precisely, we consider
fEaE: 012 + (0
where & takes in the current state of the column 012 and learns to predict adequate column update 0 Intuitively, the neural network & acts as learning both the forward and the backward steps of a iteration (Combettes and Pesquet, 2011). As for PNP, sparsity of the predictions is enforced by a soft-hhresholding non-linearity in the last layer of T.
Finally, for all models, use for 9 small neural network that takes input 62g, $22 and (0t) [en] - Its positivity is ensured by using an absolute value function as final nonlin- earity. These input features are based on the intuition that the network should exploit information

---

Published as a conference paper at ICLR 2025
from the current state of the diagonal of O the diagonal of S which is closely related to the diagonal of the GLasso estimator at convergence, and Schur's 's complement which is added to the output of g.
Improving training stability Although the positive definiteness of θ is guaranteed to be preserved at each update for any positive-valued function 9, we have empirically observed that its smallest eigenvalue could approach visible in Appendix B.1. I. In practice this leads to instability in training. Below, we provide an interpretation of this phenomenon and provide solution to address it. Each column-row update inside SpodNet layer can be written as the rank -2 update
1
01
0
0 22
012
θ
+
(10)
0
822
0 122)
022
AAo
Moreover, the nonzero eigenvalues of the perturbation Ae (0 0z)+ --00))/0ttt -
given by t
and by the Bauer-Fike theorem, we can quantify the evolution
of O's spectrum by
(x(0) - A ( | |
(11)
where A denotes the k-th largest eigenvalue of SPD matrix. Hence, one way to ensure that the perturbation the spectrum is small is to control || A olop. for instance, by limiting the magnitude of the updates. Experimentally, the most successful approach control || llop is to limit the magnitude of 072. which is not trivial task the updated value 0 g( 0zi [0n must satisfy 0 0 [+n1 1 0 > 0. Thus, propose scale 0 by scaling the preactivation' of the last layer of f by - The hyperparameter Ç >> 0 acts as a form of regularization that
J
handles compromise between stability and expressivity of the model; in all of our experiments we use 1. As visible in Figure: this scaling ensures a smooth training of the network, with stable
condition number of the O matrix.
5 EXPERIMENTS
We now illustrate SpodNet's ability to learn SPD matrices with additional structure by using our three derived graph learning models (UBG, PNP and E2E) learn sparse precision matrices, on both synthetic and real data. Additional details regarding the experimental setups can be found in Appendix A. PyTorch implementations can be found in our GitHub repository
5.1 SPARSE PRECISION MATRIX RECOVERY
Our goal in this section is evaluate models' performance and generalization ability regarding the recovery of sparse precision matrices on synthetic data. The following experiments serve several purposes: (1) compare learning-based methods against traditional model-based methods, (2) evaluate our models' reconstruction performance in terms of matrix estimation and support recovery.
Data & training We use synthetic data to train our SpodNet models as in Belilovsky et al. (2017); Shrivastava et al. (2020). We generate N sparse SPD function (Pedregosa et al., 201 I), of which we ensure proper con- ditioning by adding 0.1 Ip- These matrices are treated as ground truth precision matrices (i) for i E [N]. The sparsity degree of each matrix is controlled through the one imposed on their Cholesky factors during their generation: in the strongly sparse setting Otrue has roughly 90 % of zero entries while the weakly sparse setting this value is around 25 %. For each of these N ground truth precision matrices Oerue. we sample n i.i.d. centered Gaussian random vectors 2, ~ (o, (e -1), which are used to compute an empirical covariance matrix ( The dataset thus comprises couples (S)()) (1) true) where each (+)) stems from different ground truth precision matrix. For all the experiments, we generate Ntrain 1000 different (S, O couples for the
matrices using sklearn's
sake_sparse__ppp
true
Exxerrmenntlly scaling the preactivation works better than scaling the network's 's output. Phtpps: //gi thub. htt.:/co/////eeeeee http:/eereetttoee SpodNet.
7

---

Published as a conference paper at ICLR 2025
training set and Nest 100 couples for the testing set which we validate our models. Further details on the parameters used during the data generation are in Appendix A.1.
We train our three models (UBG, PNP and E2E) to minimize reconstruction error LMSE
Σ 2i=1
Nerain ||6(0) o
t l in the vein of Gregor and LeCun (2010); Shrivastava al. (2020),
Narain
where ② is the model's output. All three models are trained using ADAM with default hyperparame-
ters (Kingma and Ba, 2014).
Baselines We compare our three models to various other approaches throughout our experiments: learning-based, Riemannian and model-based estimators. As performed in Belilovsky et al. (2017); Shrivastava et al. (2020), we the three most popular methods for estimating precision matrices as our core baselines: the GLasso (Friedman et al., 2008), the inverse Ledoit- Sedoit-Wolf estimator (Ledoit and Wolf, 2004) and the inverse OAS (Chen al., 2010). The Ledoit-Woll and OAS estimators covariance matrix estimators, for which we take the inverse for estimating the precision (we simply refer them Ledoit- Wolf and OAS). All three approaches guarantee the sesitive-defniitness of the estimated matrix but the GLasso is the only one to additionally ensure sparse predictions. For the GLasso, for each matrix ( a raguuatt- hyperparameter X is chosen using aklearn's Graphica cethicallaas alaassasss soCV cross-validation procedure, that relies only on the samples used in St(). In the same vein, we use dedicated Ledoit- Wolf and OAS estimators for each S()) We also compare our approaches to the GLAD model (Shrivastava et al., 2020), considering both its A and Z outputs. We recall that is not guaranteed to be positive-definte and its validity as a precision matrix estimator is therefore debatable. GLAD is trained with the same loss, data and optimizer as our models.
We compare the performance of the different approaches in terms of Normalized MSE on the test set: []
NMSE
Eier"
Neca
e Pemc
le and Fl score for assessing support recovery. For the latter,
Nonet
||0 Leuc
false positives correspond to O.j + O, (Quru))is = (and similarly for true positives and false negatives).
As shown Figure 4, at convergence, the learning-based methods show to be more suited for the actual reconstruction of Quree as they significantly outperform traditional methods. This superior performance is especially pronounced as the dimensionality of the data grows and the sparsity of the ground truth decreases. In the most challenging scenario (p 100 and n 100 with weakly sparse irru)), the traditional methods perform poorly, with NMSE around 90%.
Figure 4 also suggests that our models perform especially well in terms of NMSE in the low-samples regime (n S p), which is a common setting in various real-world applications, such as neuroscience time-series analysis financial temporal network inference. In Figure 5, we thus study the influence of the number of samples n on the performance of each method. The results show that although all methods achieve excellent recovery in the large sample regime (down to around 2.5% NMSE the lower dimensional p = 20 setting), our models are especially more suited for matrix reconstruction in case of sample-deficiency, especially so in the higher dimensional (p- = 100) setting. In terms of support recovery, our models achieve significantly better F1 scores than the GLasso as the number of sample grows, with up to 25% improvement in certain tested settings. In the lower dimensional setting (p = 20), we are even able to achieve a Fl score of over 0.75 with our UBG model. Finally, although GLAD's Z performs well, an additional experiment Appendix B.3 shows that those matrices turn out to never be SPD in various settings, making them unsuited precision matrix estimators.
5.2 APPLICATION TO UNSUPERVISED GRAPH LEARNING ON REAL-WORLD DATASET
We finally evaluate the performance of SpodNet in a graph learning context using a real-world dataset and under an unsupervised scenario. The aim of this experiment is twofold: (1) to assess the generalization capabilities of SpodNet, and (2) determine if it can yield an accurate graph topology when trained solefy on synthetic data, following the idea of Belilovsky et al. (2017).
We consider the Animals dataset (Lake and Tenenbaum, 2010), which comprises p = 33 animal species, each characterized by responses to n = 102 binary questions (e.g.. "Has teeth?", "Is poisonous?"". The objective is to infer a graph of connections between these p animals
Data & training
8

---

Published as a conference paper at ICLR 2025
Ledoit-Wolf (non-sparse)
GLAD Z (non-SPD)
GLasso
PNP (ours)
OAS (non-sparse)
GLAD-e (non-sparse)
UBG (ours)
E2E (ours)
p 20, n 20
50, = 50
p 100, n 100
10a
10a
10
NMSE
60-1
6 10
4x
x 10
3x10
30
2x
10
2x10
50
100
50
100
50
100
p 20,
20
p 50, 50
p
100,
100
10a
6 60
6 10
NMSE
6 10
4x
40
3
x 10
3x10
50
100
50
100
50
100
Epochs
Epochs
Epochs
Figure 4: Learning-basedd (in variations of blue) vs traditional methods (in variations of red). Dotted curves indicate when one of the constraints (SPDness or sparsity) is not guaranteed. First row: Strongly sparse θ urue- Second row: Weakly sparse true-
Ledoit-Wolf (non-sparse)
GLAD-Z (non-SPD)
GLasso
PNP (ours)
OAS (non-sparse)
GLAD-è (non-sparse)
UBG (ours)
E2E (ours)
p=20
p=100
F1 Score
0.75
F1 Score
0.6
NMSE
NMSE
10-1
0.50
10
0.2
10100200
10100200
10000200
10100200
Number of samples
Number of samples
Number samples
Number of samples
Figure 5: Comparing models in sample deficient regimes (n S p) up to large-sample regimes (n >> p). evaluated in terms of NMSE and F1 score for support recovery. Dotted curves indicate when of the constraints (SPDness or sparsity) is not guaranteed. In large dimension, GLAD's Z performs well, but never SPD; GLAD's θ is SPD, but performs badly.
based on the responses. We train our UBG model by minimizing the MSE on the synthetic data detailed in Section 4.2, with p 33, 102 and various levels of sparsity. We generate 1000 training matrices and train the model for 100 epochs. The predicted precision matrix is then computed by inputting the empirical covariance matrix from the Animals dataset into SpodNet.
Baselines We compare our approach with the Gl Glasso (with the regularization parameter selected by cross-validation), GLAD (which is trained with the same data as our model), and the Elliptical Graphical Factor Model (EGFM) proposed by Hippert-Ferree optimization method specifically designed to identify clusters within the data by estimating precision matrix whose inverse is modeled as low-rank matrix plus a positive diagonal. For fair comparison, we adopt the same hyperparameters as outlined in the original paper, setting the rank and regularization parameter 10.
al. (2023). The latter is Riemannian
Each method yields precision matrix, which is used to construct graph representing the connections between the animals. The graph is constructed by considering the absolute values of the precision

---

Published as a conference paper at ICLR 2025
Gaaso (m 0.61))
EGFM (m 0.86)
Tiger
Mouse
Chimp
Alligater
Doo Cat
Wolf Hon
Tiger
Chimp
Moy-5
Squirrel
Woll
Cat
Squarel
Elegbant
Camel
Ostrich
Ligh
Giratte
Robin
Eagle
Rhino
Giratte Dear
Horse
Etephant
Ant
Finch
Eagle
Horse Doighin
Cow Camel
Rhino
Bge
Chicken
Chicken
Ostrich
Butlerfly
Salmon
Butterily
Alligaton
Cockgeech
Whale
Seal
Cockgpach
Sgal
Penguin
Dolphin
Trout
Penguin
Boc
Iguanag
GLAD (m 0.72)
UBG (m 0.78)
Wolf
Tiger
Vigtt Tiger
Gorilla
Mouse Lion
Dgg Cal
Uign
Chimp
Whale Ponguin
Cat
Iguana sSaal
Alligtoor
Dolphin
Squirre!
Deer
Camal
Fiach Eagle
Ropin
Girelffe Elephant
Graafee
Chiggen
Alligator
Dear Mouse
Ostrich
Robin Eagle
Bee
Hordh
Iguana
Squirrel
Butlerly
Elcphant
Butterfly
Finch
Ostrich
Trout
Ant
Whale
Dolphin
Chicken
Chimo
Garilla
Salmon
Cockroach
Cockroach
Ant
Salmon
Penguin
Seal
Figure 6: Graph estimation on the Animals dataset with the GLasso, EGFM, GLAD and our UBG model. The last three yield cleaner and more interpretable clusters than the GLasso, but EGFM is specifically tailored for finding clusters as opposed to our UBG model or GLAD.
matrix coefficients, which represent the strength of the connections, and by removing the diagonal elements to eliminate self-loops. The results are illustrated in Figure 6, where the nodes of the graphs partitioned using the Louvain algorithm (Blondel et al., 2008). Qualitatively, produces a coherent graph, with outcomes comparable to those of the three baselines. Additionally, the graph learned by UBG model exhibits cleaner structure compared GLasso, with a sparser representation and finer clusters, such as the grouping of (Chimp, Gorilla). To quantitatively assess the quality of the obtained graphs, we calculate the modularity of the partitions: higher modularity values indicate better separation of the graph into distinct subcomponents (Newman, 2006). We find that the graph produced by UBG achieves higher modularity (m = 0.78) than GLasso (m 0.61) and GLAD (m 0.72), and slightly lower than EGFM (m 0.86). It is important to note that BGFM is specifically designed to obtain clustered graphs, whereas our UBG model has no such inductive bias. These combined quantitative and qualitative results demonstrate that SpodNet generalizes effectively to unseen data and produces a coherent graph structure.
observe that UBG
CONCLUSION
We have proposed Schur's Positive-Definite Network (SpodNet), a novel learning module compatible with other standard architectures. offering strict guarantees of SPD outputs. The principal novelty of SpodNet comes from its ability to handle additional desirable structural constraints, such as elementwise sparsity which we used as an illutrative example throughout this paper. To the best of our knowledge, SpodNet layers are the first to offer strict guarantees of such highly non-trivial structure in the outputs. future works, SpodNet could be leveraged to learn other additional structures beyond sparsity. We have shown how to leverage SpodNet to build neural architectures that outperform traditional methods in the context of sparse precision matrix estimation in various settings. Future research will be dedicated to improving the computational cost of our framework. theoretical understanding of the eigenvalues' dynamics during training and of SpodNet's expressivity, and deriving formal convergence guarantees.
10

---

Published as a conference paper at ICLR 2025
7 REPRODUCIBILITY STATEMENT
We believe this paper fully discloses the information needed to reproduce the main experimental results. Our core contribution the SpodNet layer: its general algorithmic framework is presented in Algorithm 1. Three specific models built using this framework for learning sparse and SPD matrices are introduced in Section 4.2, and the exact architectures used for each of them throughout our experiments are explained in detail in Appendix A.2. The conducted experiments are described in Section 5 with additional details in Appendix A. PyTorch implementations can found in our GitHub repository.
8 ACKNOWLEDGEMENTS
We would like to thank Badr Moufad (Ecole Polytechnique, France) for his help with the implemen- tations, Paulo Gonçalves (Inria Lyon, France) for fruitful discussions as well the Centre Blaise Pascal for computing ressources, which uses the SIDUS solution developed by Emmanuel Quemener (ENS Lyon, France) (Quemener and Corvellee, 2013). This work was partially supported by the AlsegroAssai ANR- AERR-9-CCHH 19-CHIA-0009 project.

---

Published as a conference paper at ICLR 2025
REFERENCES
C. Bonet, B. Malézieux, A. Rakotomamonjj, L., Drumetz, T. Moreau, M. Kowalski, and N. Courty. wassersesnn on symmetric positive definite matrices for m/ceg signals. In ICML. PMLR. 2023.
S. L. Lauritzen. Graphical models. Clarendon Press, 1996.
X. S. Nguyen, L. Brun, O. Lézoray, and S. Bougleux. A neural network based on spd manifold learning for skeleton-basee hand gesture recognition. CVPR, 2019.
O. Ledoit and M. Wolf. Improved estimation of the covariance matrix of stock returns with an application to portfolio selection. Journal of empirical finance, 2003.
O. I Ledoit and M. Wolf. A well-conditioned estimator for large-dimensional covariance matrices. Journal of multivariate analysis, 2004.
O. Banerjee, L. El-Ghaoui, and A. d' Aspremont. Model selection through sparse maximum likelihood estimation for multivariate gaussian or binary data. JMLR, 2008
T. Cai, W. Liu, and X. Luo. A constrained ε minimization approach to sparse precision matrix estimation. Journal of the American Statistical Association, 2011.
Z. Huang and L. Van Gool. A Riemannian network for SPD matrix learning. In Proceedings of the AAAI conference artificial intelligence, 2017.
Z. Gao, Y. Wu, Jia, and M. Harandi. Learning to optimize spd manifolds. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2020.
K. Gregor and Y. LeCun. Learning fast approximations of sparse coding. In ICML, 2010.
Y. Chen and T. Pock. Trainable nonlinear reaction diffusion: A flexible framework for fast and effective image restoration. IEEE transactions on pattern analysis and machine intelligence, 2016.
V. Monga, Y. Li, and Y. C. Eldar. Algorithm unrolling: Interpretable, efficient deep learning for signal and image processing. IEEE Signal Processing Magazine, 2021.
T. Chen, X. Chen, W. Chen, H. Heaton, J. Liu, Z. Wang, and W. Yin. Learning to optimize: A primer and benchmark. JMLR, 2022.
N. Shlezinger, Whang, Y. C. Eldar, and A. G. Dimakis. Model-based deep learning. Proceedings of the IEEE, 2023,
N. Boumal. An introduction to optimization on smooth manifolds. Cambridge University Press, 2023,
B. Rolfs, B. Rajaratnam, D. Guillot, I. Wong, and A. Maleki. Iterative thresholding algorithm for sparse inverse covariance estimation. NeurlPS. 2012.
D. Guillot and B. Rajaratnam. Functions preserving positive definiteness for sparse matrices. Trans- actions of the American Mathematical Society, 2015.
R. Sivalingam. Sparse models for positive definite matrices. PhD thesis, University of Minnesota, 2015.
P-A. Absil, R. Mahony, and R. Sepulchre. Optimization algorithms on matrix manifolds. Princeton University Press, 2008.
Z. Gao, Y. Wu, X. Fan, M. Harandi, and Y. Jia. Learning to optimize riemannian manifolds. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022.
Z. Dong, S. Jia, C. Zhang, M. Pei, and Y. Wu. Deep manifold learning of symmetric positive definite matrices with application to face recognition. In Proceedings of the AAAI Conference Artificial Intelligence, 2017.
D. Guillot and B. Rajaratnam. Retaining positive definiteness thresholded matrices. Linear algebra and its applications, 2012.
12

---

Published as a conference paper at ICLR 2025
H. Shrivastava, X. Chen, B. Chen, G. Lan, S. Aluru, H. Liu, and L. Song. Glad: Learning sparse graph recovery. In ICLR, 2020.
F. Zhang. The Schur complement and its applications, volume 4. Springer Science & Business Media, 2006.
Friedman, T. Hastie, and R. Tibshirani. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 2008.
R. Mazumder and T. Hastie. The graphical lasso: New insights and alternatives. Electronic journal of statistics, 2012.
C-J. Hsieh, M. A. Sustik, I. S. Dhillon, P. Ravikumar, et al. Quic: quadratic approximation for sparse inverse covariance estimation. JMLR, 15, 2014.
Jonas Adler and Ozan Öktem. Learned primal-dual reconstruction. IEEE transactions on medical imaging, 37(6):1322-1332 2018.
Simon Arridge, Peter Maass, Ozan Öktem, and Carola-Bibiane Schönlicb. Solving inverse problems using data-driven models. Acta Numerica, 28:1-174, 2019.
E. Belilovsky, K. Kastner, G. Varoquaux, and M. Blaschko. Learning to discover sparse graphical models. In JCML. PMLR, 2017.
S. P. Boyd and Vandenberghe. Convex optimization. Cambridge university press, 2004.
N. Parikh, S. Boyd, et al. Proximal algorithms. Foundations and trends() in Optimization, 2014.
S. V. Venkatakrishnan, C. A. Bouman, and B. Wohlberg. Buug-nnd-paay priors for model based reconstruction. 2013 IEEE global conference on signal and information processing. IEEE, 2013.
Y. Romano, M. Elad, and P. Milanfar. The little engine that could: Regularization by denoising (red). SIAM Journal on Imaging Sciences, 2017. 2017.
P. L. Combettes and J-C. Pesquet. Proximal splitting methods in signal processing. Fixed-point algorithms for inverse problems in science and engineering, 2011.
Pedregosa. G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Pretten- hofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825 12:2825-2830. 2011.
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint orXiv:1412.690 2014.
Y. Chen, A. Wiesel, Y. C. Eldar, and A. O. Hero. Shrinkage algorithms for mmse covariance estimation. IEEE transactions on signal processing, 2010.
B. Lake and J. Tenenbaum. Discovering structure by learning sparse graphs. In Proceedings of the Annual Meeting of the Cognitive Science Society, 2010.
A. Hippert-Ferrer, Bouchard, A. Mian, T. Vayer, and A. Breloy. Learning graphical factor models with riemannian optimization. In Joint European Conference on Machine Learning and Knowledge Discovery Databases. Springer, 2023.
V. D. Blondel, J-L., Guillaume, R. Lambiotte, and E. Lefebvre. Fast unfolding of communities in large networks. Journal of statistical mechanics: theory and experiment, 2008
M. Newman. Modularity and community structure in networks. Proceedings of the national academy of sciences, 2006.
E. Quemener and M. Corvellec. Sidus-the solution for extreme deduplication of an operating system. Limux Journal, 2013, 2013.
13

---

Published as a conference paper at ICLR 2025
A EXPERIMENTAL DETAILS
A.I SYNTHETIC DATA GENERATION
Here we provide additional details on the experiment of Section 5.1. The sparsity degree of each matrix is controlled through the one imposed on their Cholesky factors during their generation, with the alpha parameter of the function. Throughout our experiments, we consider a alpha {0.7, 0.95); this is the probability of a entry on the Cholesky factor of the generated matrix and not the actual fraction of null elements of e. Although the numbers vary with the dimension 0.95 roughly leads to % of zero entries, while 0.7 leads 25 % of zero entries.
A.2 ARCHITECTURES FOR UBG, PNP AND E2E
The function 9 for all three models is MLP with two hidden layers of neurons each with a ReLU activation function. takes as input 822, 822 and (012) "On - 0
Its 7 parameter is predicted by a MLP that takes as input the current state of the 012 block (of dimension and has single hidden layer of [p/2] neurons with ReLU activation function. The (single) output neuron has an absolute value activation, in order to keep it positive since it predicts step-size.
UBG
UBG ves another MLP to learn the vector X of elementwise soft-thresholding parameters. This MLP has single hidden layer of neurons, with ReLU activation function. For the same reason as before, the output layer also has an absolute value activation.
PNP First, the step-size y is predicted by MLP with the exact same architecture as the one in UBG. The learned operator takes as input 012 - (s12 W12). passes it through a single hidden layer of 2p neurons, followed by ReLU activation function, and projects it back into a dimensional vector.
The architecture to predict À is the same for UBG. Our experiments show that multiplying its output by 0.1 helps avoid local minimas and improves convergence.
The neural network & is a MLP that takes as input 02 passes it through a single hidden layer of 10p neurons follow by ReLU activation function, and projects it back into a 1 dimensional vector. The architecture to predict À is the same as for UBG and PNP. Our experiments show that multiplying its output by 0.1 helps avoiding local minimas and improves convergence.
E2E
GLAD We the authors' original implementation retrieved from the authors repository https: http:/ggttttttteee com/ Hoom/Harrss22 wtwssshs7//Gae /LLAD, with the default hyperparameters.
All four models are implemented with K 1 layer throughout all the experiments, since our results show that this was enough to yield competitive to outperforming results in the settings under consideration.
A.3 DETAILS ON FIGURES AND 5
For the weakly sparse settings, we use a learning rate of 10-3 for all three of our own models. For the strongly sparse settings, we use learning rate of 10-2, GLADD's learning rate set to 10 in all settings. All four models are trained on the same 1000 training matrices, using batch-size of 10, with ADAM's default parameters. The models are tested on the same 100 matrices as mentioned in Section 5.1.
For Figure 5 We use the strongly sparse ground truths in order for the Fl score to be more sensical, and to better illustrate the relevancy of the leamned sparsity patterns. We use learning rate of 10 for all three of them in every setting for p = 100, and of 10 3 10 10 3.3. 10 in the p = 20 setting for respectively 10, 20, 100, 200 and 500.
14

---

Published as a conference paper at ICLR 2025
Amin (@)
10a1
max max(dlag(@)) cond(O)
10
10-3
12
Column update index
Figure 7: Potential instabilities without normalization. Along column update indices, we plot the smallest eigenvalue (blue), largest diagonal value (orange) and conditioning (green) of θ
B ADDITIONAL DETAILS ON MODELS
B.1 SPODNET'S S POTENTIAL. INSTABILITIES WITHOUT NORMALIZATION
We show in Figure the evolution of the conditioning of an example of O during training that becomes unstable if we do not use the stabilization procedure described in Section 4. We observe that although the matrix remains SPD as predicted by Proposition 3.1 but gets increasingly closer to being singular during the updates.
B.2 GLAD'S S UPDATE RULES
GLAD unrolls an algorithm that solves the following optimization problem:
min e ZEB
- log det (e) + (S + 1 + _ 1 θ
(12)
Each iteration of GLAD, which can be seen as individual layer in deep learning perspective, updates several running variables:
at = xt=1 - a),
(13)
Y 1
S-Z. Z
(14)
at
4 0 -Y+ v /
Id
(15)
at
2 MO Sijs Zis),
(16)
Z ST S (0 Vi E |p).
(17)
in which the functions f: R22 + R and R + R are two small neural networks that are trained predict adequate parameters for each layer. By construction, + always SPD and sparsity structure is enforced on Z but the converse is not true.
B.3 GLAD'S NON-SPD OUTPUTS
As Figure shows, in the setting Figure 4, GLAD's best performing outputs are SPD in virtually 0% if the case strongly sparse case with p 100.
Additionally, Z loses its sparsity projected onto the SPD cone and becomes singular (since actually projected onto the closed cone of semi-definite matrices). Going the other way around and trying to sparsify θ instead, by thresholding its off-diagonal entries following the support found by Z, breaks its positive-definiteness guarantee (Figure 9).
15

---

Published as a conference paper at ICLR 2025
p=100 Strongly sparse
of SPD
75
Z
50
25
0
10 100 200
500
Number samples
Figure 8: Percentage of SPD outputs among GLAD's Z and @ outputs at convergence. As detailed
in Section 4.1, Z is not guaranteed to be sparse.
Support(Z) Sparsity degree = 51.6%
Support(2rss)
Sparsity degree 0.0%
Amin -0.11
Amin(2psa) 0.0
arg min Zoll
Support(e) Sparsity degree 0.0%
Support( Sparsity degree 51.6% 1m ( -0.231
Amin (9) 0.042
Theta [=-0.
Figure 9: First row: Projecting GLAD- Z onto the PSD cone. Second row: Thresholding GLAD-@ following the support of GLAD-Z.
16


Reviews:
[{"review_id": "28K58aw0J8", "reviewer": "Reviewer_MngH", "summary": "The paper proposes a new method SPODNET for learning SPD matrices by elements, supported by the classical Shur’s condition, where the matrix elements (u and v) to update are learned using neural networks. The work demonstrates the use of SPODNET for the sparse precision matrix learning task, and proposes three new model architectures to perform the learning, including UBG, PNP and E2E. Two sets of experiments were conducted for evaluation, using a synthetic data and a real-world data. The results show the effectiveness of the proposed methods, and their advantages over the compared ones.", "questions": "•\tAlthough the proof is straightforward, it would be useful for the reader and for completeness to explain how Eq. (3) is derived in proof.\n•\tFor clarify, the role/design rationale behind each term in Eq. (5) can be explained briefly, although it is an existing method.\n•\tIn line 364, it is mentioned that the MSE reconstruction loss is used. How is this implemented together with the GLasso loss in Eq. (5)?\n•\tHow does the proposed method perform in terms of training time/cost?\n•\tWhat do different colours mean in Figure 1?", "limitations": "", "rating": 8, "confidence": 4, "soundness": 3, "presentation": 4, "contribution": 3, "strengths": "•\tTo my knowledge, the proposed method is original. I like the proposed SPODNET, building on classical theory, simple but elegant.\n•\tThe paper is well written and well structured.\n•\tExperiment design is appropriate for demonstrating the effectiveness of the proposed method. I particularly like the result of UBG in Figure 6, highlighting more distinct structure that is interesting  for this real-world data.", "weaknesses": "•\tThe paper would be stronger if they could include another real-world learning problem over SPD manifold.  But I don’t see this as a major issue.\n•\tLack of discussion on the limitation of the proposed work.\n•\tThere are a couple of things that could be explained better, see my questions.\n•\tFigures 4 and 5 are too small, hard to read.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "n9e9va073E", "reviewer": "Reviewer_MbyP", "summary": "This paper proposes a deep learning-based approach to solving SPD problems such as covariance selection. The authors start with the block coordinate descent (BCD) algorithm and then unroll the optimization using neural networks. This follows with three different unrollings, where each preserves different levels of the problem structures (or inductive bias in the deep learning words). The authors evaluate the proposed SpodNet on synthetic data and the animal dataset against GLAD, GLasso, and other traditional approaches.", "questions": "1. Training details: What are the exact steps of the unrolling algorithm? How many unrollings are needed? From line 363, does the training posit an MSE loss on the intermediate $\\Theta$s or only the last one?\n2. GLasso: In Fig. 5, the F-1 performance of GLasso decreases when $n$ gets larger. This is weird because I expect it to recover the graph perfectly when $n\\gg p$. What are the authors' explanations about this?", "limitations": "", "rating": 6, "confidence": 4, "soundness": 2, "presentation": 3, "contribution": 2, "strengths": "The paper is well-written and easy to follow. The idea of unrolling the column-row BCD algorithm to ensure SPD seems novel.", "weaknesses": "I think this is a borderline paper in its current form. I value the novelty of the paper, but its numerical performance is not the most convincing. I think solving the following points will make the paper stand more firmly at my rating.\n\n1. GLAD-Z: SpodNet's NMSE performance on synthetic data seems to be consistently worse than GLAD-Z's. I understand the authors' argument that GLAD-Z is not SPD, but what if Z is projected onto the SPD cone? Will the projected Zs remain the lower NMSE scores? Because GLAD uses an ADMM-like algorithm, the learned $\\Theta$ matrices are not projections if I understand correctly.\n2. Large sample regime: The NMSE performance of SpodNet is no better or only marginally better than the baselines when $n>p$.\n3. Real-world datasets: The results of GLAD on the animal dataset are missing. Also, the paper will benefit from adding at least another real-world dataset.\n4. Figures: Some figures can be hard to read, especially Fig. 4-5. I suggest the authors use thinner lines and/or redesign their layout to make line plots larger.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}, {"review_id": "2NbyXLrJus", "reviewer": "Reviewer_R2Se", "summary": "The paper presents a method for jointly learning Symmetric Positive Definite (SPD) and sparse matrices using a learning based approach. The proposed method SpodNet is an iterative method which learns column-row pairs stepwise where each step is parametrized by a neural network. The SPD constraint is then ensured via Schur’s condition.", "questions": "See above", "limitations": "", "rating": 6, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "1. SpodNet enables increased expressivity for estimating SPD matrices with a neural network based parametrization, at the same time maintaining constraints like sparsity unlike other model based approaches, as verified by experiments on synthetic data.\n2. In order to make the method tractable the authors leverage the block structure of SPD matrices to restrict the complexity of each update step to $O(p^2)$.\n3. Experiments on synthetic data and graph topology estimation using SpodNet are presented, highlighting the effectiveness of the method. On the synthetic data, the proposed method is similar to model based approaches while achieving both SPD and sparsity at the same time.", "weaknesses": "1. The proposed SpodNet provides a novel approach to leverage neural networks and increase expressivity on constrained manifolds. However, the design of the neural networks itself is not discussed in detail. It is not clear to me why the authors choose the input features for  $g$ as $\\theta_{22}$, $s_{22}$ and $\\theta_{12} \\theta_{11}^{-1} \\theta_{12}$. Similar for each of the three described approaches the explanation for choosing the input features is missing. I assume the features are chosen to best suit the model based approach and that performs well with gradient descent but the paper would benefit from a detailed explanation of the same.\n2. The method still seems relatively expensive in spite of the improved update rule. The overall cost as the authors mentioned is of the order of $O(p^3)$, how does this compare with the other model based approaches?\n3. Can the SpodNet framework maintain other structure constraints for example structural sparsity. In general what conditions would the constraints need to satisfy in order to be optimized with a SpodNet layer.\n\nSince the general literature of SPD matrix estimation points towards applications in computer vision, it would be informative to see an experiment for a vision task with SpodNet to verify the comparison with baselines and its scalability given its computational requirements.", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "extra_scores": {}}]

Author rebuttal:

Decision: Accept (Poster)