Learning Bipartite Graphs: Heavy Tails and Multiple Components
Authors: José Vinícius De Miranda Cardoso
Jiaxi Ying
Daniel P. Palomar
Venue: NeurIPS 2022 Accept (2022)
Keywords: graphical models
bipartite graphs
heavy tails
financial markets

Abstract: We investigate the problem of learning an undirected, weighted bipartite graph under the Gaussian Markov random field model, for which we present an optimization formulation along with an efficient algorithm based on the projected gradient descent. Motivated by practical applications, where outliers or heavy-tailed events are present, we extend the proposed learning scheme to the case in which the data follow a multivariate Student-$t$ distribution. As a result, the optimization program is no longer convex, but a verifiably convergent iterative algorithm is proposed based on the majorization-minimization framework. Finally, we propose an efficient and provably convergent algorithm for learning $k$-component bipartite graphs that leverages rank constraints of the underlying graph Laplacian matrix. The proposed estimators outperform state-of-the-art methods for bipartite graph learning, as evidenced by real-world experiments using financial time series data.

Learning Bipartite Graphs: Heavy Tails and Multiple Components
José Vinícius de M. Cardoso', Jiaxi Ying", Daniel P. Palomar
Department of Electronic and Computer Engineering!
Department of Mathematics
Department of Industrial Engineering and Decision Analytics
The Hong Kong University of Science and Technology
Clear Water Bay, Hong Kong SAR China
jivdic..x.yiig---
hatpinc.j..iing/conneett... @connect.ust.ht hk, palomar@ust.hk
Abstract
We investigate the problem of learning an undirected, weighted bipartite graph under the Gaussian Markov random field model, for which we present an optimiza- tion formulation along with an efficient algorithm based on the projected gradient descent. Motivated by practical applications, where outliers or heavy-tailed events are present, we extend the proposed learning scheme to the case in which the data follow multivariate Student-t distribution. As a result, the optimization program is no longer convex, but a verifiably convergent iterative algorithm is proposed based on the majorization majorizationnmmiiiizzaoooo framework. Finally, we propose an efficient and provably convergent algorithm for learning k-component bipartite graphs that leverages rank constraints of the underlying graph Laplacian matrix. The proposed estimators outperform state-of-the-ar methods for bipartite graph learning, evidenced by real-world experiments using financial time series data.
Introduction
Efficient optimization formulations alongside scalable optimization algorithms have been critical tools for performing inference in graphical models. In particular, learning sparse, unconstrained Gaussian Markov random fields (MRF) has enabled wide range of practical applications across a number of fields, including brain network analysis [1], single-cell sequencing [2], time-varying network estimation [3], time-series clustering [4] and model selection [5], and financial networks [6, 7J. Such
an impact may be attributed to the early development of the graphical lasso algorithm via iterative
block coordinate descent methods [8-10].
More recently, there has been a growing interest in constraining Gaussian MRFs into a particular family such bipartite and k-component graphs for data clustering [11-14], and considering robust statistics that account for heavy-tailed events [15, 16]. In addition, constraints on the sign of the elements of the precision matrix of Gaussian MRF have been employed to learn total positivity models [1 [17-20] as well as Laplacian models in sparse settings [21-23]. Estimating undirected graphs under smooth signal assumptions has been investigated [24-27].
In this paper, we consider the problem of learning undirected, weighted bipartite graphs under the MRF assumption. The key contributions of this paper include:
1. We design an efficient optimization algorithm based the projected gradient descent (PGD) framework for the problem of learning an undirected, weighted bipartite graph from the Gaussian MRF framework.
36th Conference on Neural Information Processing Systems (NcurIPS 2022).

---

2. We extend our proposed formulation in order to accommodate the multivariate Student-t distribution, which is often needed for real datasets that contain outliers or are inherently heavy-tailed. In this case, the optimization formulation turns out to be nonconvex. We design an optimization algorithm based on the mnjorization-miniiizatte (MM) framework to obtain a stationary point of such problem.
3. We propose an efficient and provably convergent algorithm based on the alternating direction method of multipliers (ADMM) for learning k-component bipartite graphs that leverages the rank properties of the Laplacian matrix.
4. We present empirical results, using real-world datasets from the US stock market, which reveal that our formulations outperform sate-oofthe-aas ones in terms of graph modularity and node label accuracy.
Matrices (vectors) are denoted by bold, italic, capital (lowercase) roman letters like X, x. Vectors are assumed to be column vectors. We use 1, (resp. In) denote the n-dimensional all-one vector (resp. identity matrix). The (f,j) element of a matrix X E R=*p is denoted as Xsj. The i-th element of a vector æ is denoted as T;. The i-th row of X is denoted as æ; E RPx1, X 2 Y implies that Xij 2 Yij V j. whereas X > Y implies that X Y is positive semi-definite. For a square matrix X, A (X) denotes the i-th smallest eigenvalue of X. For a vector æ and indices j > i, Zi:j denotes (Irz, Ti.1,..., 22) The operator Diag: RD + Rexp creates a diagonal matrix with the elements of an input vector along its diagonal. The operator diag Roxp - R' extracts the diagonal of a square matrix. The optimal point of an optimization variable v is denoted as v*.
Notation:
2 Preliminaries & Related Works
undirected, weighted, bipartite graph can be defined as 4-tuple - [V,, Vg, W), where v, & (1,2....,7} and V, fr+ 1,r 2,...,, + q} are the node sets associated to a group of objects and classes, respectively, where p _ + q is the total number of nodes, and we assume that r >> q. i.e., there exist many more objects than classes; & S {fu, v} u E Vr, Vo3 is the edge set, that is, a subset of the set of all possible unordered pairs of nodes such that {u, o} E & iff nodes and v are connected; W E R is the symmetric weighted adjacency matrix that satisfies Wa = 0, W > 0 iff (i,j) E and Wi = 0, otherwise. The weighted Laplacian matrix of a graph defined as A Diag(W1) W.
Our goal is to learn bipartite graph from data under probabilistic assumptions. Thus, the data generating process initially assumed to be zero-mean improper Gaussian Markov random field (IGMRF) [28] æ E IR, such that x; is the random variable generating a signal measured at node i, whose rank-deficient precision matrix is modeled as a graph Laplacian matrix [21, 22, 29]. Assume
we are given n observations of x, L..., X= [xi, ..,..., 2x, X E Ra*p, I; E RPx1_ The goal of graph learning algorithms to learn Laplacian matrix given the data matrix X.
To that end, the classical MLE of the
constrained precision matrix æ. on the basis the observed data X, may be formulated as the following optimization program [21, 22, 29]:
minimize tr (LS) - log det" (L), subject L1 0, L,j = Ly S 0, + j. L=0
(1)
where S is a similarity matrix, e.g., the sample covariance (or correlation) matrix S x X X, and det" (L) is the pseudo-determinant of L, i.e., the product of its positive eigenvalues [30].
The Laplacian matrix L of a bipartite graph can be written as
Diag (B1,) B
-B Diag (B 11)
where B E R"X9 contains the edge weights between the nodes of objects and the nodes of classes. Note that L satisfies L1 0 and Lij Las 20Vi4 by definition.
In what follows, we briefly discuss the two main approaches proposed in the literature to tackle the estimation of bipartite graphs

---

Bipartite Structure. The authors in [12] proposed the following optimization problem to learn k-component bipartite graph from a given bipartite graph weights A R
minimize B.VERA
B - Alle + 7t (V LV). subject to 2 0, B1g =r, V Ix:
(3)
where L depends B through (2), 0 is hyperparameter that promotes the rank of L to be p - k, and A can be constructed from the correlation between nodes of objects and classes.
An alternating minimization approach was proposed in [12] to solve Problem (3). A shortcoming of this formulation is lacks statistical support, which makes difficult to draw statistical conclusions about the learned graph as well as to consider this model under different statistical assumptions.
Spectral Regularization. Properties associated with the spectral decomposition of graph matrices have demonstrated advantages that enable learning graphs with specific structures, such as bipartite and k-component graphs [11-14, 31]. The authors [14] proposed formulation component bipartite graphs based on the GMRF framework. Their approach consists of including spectral decompositions of the Laplacian and adjacency matrices into the objective function as regularization terms. Although the formulation in [14] is derived from an statistical approach, we note that it does not always guarantee that the estimated graph is bipartite, further requiring postprocessing of the adjacency matrix. In addition, the lack of constraints on the node degrees allow trivial solutions with isolated nodes, which is also undesired in practice. The method [14] does not require knowledge of the partition of the node set, however it does assume that the number of nodes in each set is known.
learn
Proposed Formulations & Algorithms
what follows, we present our proposed methods and algorithms, starting off with the simple case of connected bipartite graph under Gaussian settings. It is important to look this particular scenario first, as the proposed algorithm will serve as a building block for more elaborate formulations that include additional statistical assumptions, such heavy tails, and its extension to k-component bipartite graphs.
3.1 Gaussian Bipartite Graphs
In this section, propose formulation to learn connected bipartite graphs from data along with an algorithm to find global optimum. Noting that the adjacency matrix of a bipartite graph can be partitioned as W = [0 B; 0]. where B E BTX9, we can formulate its MLE as the following convex optimization program:
minimize tr (LS) log det (L+ J), subject to L Diag (B1g) L,B20
-B Diag (B 1,)
(4)
BT
where have used the fact that, for connected graph, det." (L) det J), J & 111 [29].
We can reformulate Problem (4) by taking advantage of the symmetry of the similarity matrix S, i.e.
= Srt Sryi rg Sagl where Srg E R2"x contains the pairwise similarities between the nodes in the set of objects D, and the nodes in the set of classes Vg. In particular, have
B1 1
Ir (LS) = T
2Tr( = ir (1,81r or + 8.41:p - 25,r)), (5)
where s A dSa
Plugging (2) and (5) in (4), we arrive at the following formulation to learn a connected bipartite graph:
minimize
tr [B (1yTT.r
+ 9r+1;p -2S 2.y)))
[Diag B1g)) + Jrr
B Jrg
1).
(6)
- log det
-B" Jgr
Diag B 1,) Jgg.)

---

3.1.1 PGD Solution
We propose an optimization algorithm based on the PGD framework [32, 33] that amounts to two steps: (1) computation of descent update and (2) projection onto the feasible set.
A common way to compute a descent direction is using the gradient of the objective function. The gradient of the objective function in Problem (6) would involve the computation of the inverse of the matrix inside the log-determinant term, which in general costs Ofr²). By leveraging the block structure the Laplacian matrix of bipartite graphs, and using the classical matrix determinant lemma for block matrices that relates the determinant of a block matrix with its Schur complement [34, pp. 4J, we have the following equality:
log det (
[Diag( B1g) Jrr -B Jgr
B + Jr Diag B 111,) 1,) + Jq9
1) log det (Diag( (B1)) Jrr) (Diag()) + J...) + Jra)). (7)
log det (Diag(B 1,) + Jqn - (- +
In practice, also would like to normalize the degrees of the nodes in the objects group, so that the edge weights can be directly interpreted as the degree of membership of an object to a particular class. Mathematically, this can be achieved by simple linear constraint B1, 1,,
Plugging the above constraint and equality (7) into (6), we have the following formulation
(BC) log det (Diag (ogg(B1,)) 1,) + Jag (B - -rra) T (I, + J,.)) - -Bn)) B20
minimize
(8)
where C A (81+1:11) 1, --
Let (BB & log (g tr (BC) be the objective funetion of Problem (8), where g (B)
Diag B 1,) + Jng (B Jrq) (I, + - Srq). Then the gradient of (BB can be
computed as
Vf 1, |iag (-9(B) 2(I, + J,r)) Jrg) (-9(B)-1) + CC (9)
The PGD update is formulated as
B (+1 arg min
| (B )| Ps (B'-BaV((BB))
(10)
B20. =1,
Problem (10) is an Euclidean projection of the rows of B' - az onto the probability simplex.
The unique solution to Problem (10) can be found efficiently via several algorithms [35-37] whose theoretical worst-case complexity is (rq²²) but their observed practical complexity is O(rg) [38].
Finally, to validate the point B and adaptively update the learning rate ot, check the following backtracking condition [33]:
1 BB - 201
S (B*+)) / (
(B)
B') + (VI B B*)
B'
(11)
In practice, we decrease the learning rate E (0, 1) until condition (11) is satisfied or convergence has been achieved. Algorithm summarizes the proposed scheme for learning bipartite graph from Gaussian MRF assumption. Since the objective function of Problem (8) is convex and its feasible set is compact, Algorithm converges to global minimum [32].
3.2 Student-r Bipartite Graphs
Outliers and heavy-tailed events are pervasive in modern datasets [39]. To account for this phenomena, instead of assuming a Gaussian generative process, as done in [14] and in Problem (4), we assume the data generating process to be modeled by multivariate zero-mean Student-/ distribution, whose probability density function can be written as
x Oæ - (v+p)/2
p(x) x /det (0) [+
(12)

---

Algorithm 1: Gaussian bipartite graph learning (GBG)
Data: Similarity matrix S, initial feasible estimate of the graph weights B initial learning rate Og O, tolerance > 0.
Result: Optimal bipartite graph: B
while 1 < maxiter do
> B <- PΛ B - (B))) where is chosen such that (11) is satisfied
if B++ B |||| || I then
return B
end
end
where θ is posidive-semiddefmite inverse scatter matrix modeled combinatorial graph Laplacian matrix, and y > is the number of degrees of freedom that controls the rate of decay of the tails. More precisely,
- the tails of (12) approach those of Gaussian distribution.
This results in a robustified version of the MLE for connected graph learning, i.e.,
minimize log det (L + J)
-== log (1 + ,0 Lx;)
(13)
I, subject to L = B (B
BB B 0, B1, = 1,
Diag
11)
We can further simplify the objective function of Problem (13) by noting that
Ir
-B
x
La, xx]
-B
Diag (B (B)) x; = h + tr (BG.),
(14)
where G, diag(a;;;) (rig(a;
(x,a,) D, ng and h; e 1. diag(x 1
Plugging (7) and (14) into (13), we have the following optimization program to learn a heavy-tailed bipartite graph:
minimize B20, =1,=1,
- log det (Diag(B') 1,) (agg(BB,))) hy (BG.)
Jug -(B T(,, JT. J, -Jry)
p+ V Di= log
(15)
Problem (15) is, in general, nonconvex due to the summation over concave and hence it is difficult to be dealt with directly. To tackle this issue, we leverage the MM framework that generates a sequence of updates, where in each iteration an upper-bounded subproblem solved [40].
To find such upper-bound we use the fact that the logarithm term in Problem (15) can be globally upper-bounded by its first-order Taylor expansion, i.e., for all 2 0,4 2 0,b > 0, have:
L. log [1 + 6)
< log (1 a) + L hence at a point B we have that b
a+b"
log (1+ h + to (BG,)
tr (BG,)
v + h4 + (BiG.) +c.
v
where c log hittr(BiG.))
(r(B'G. G.)
is a constant that does not depend on B.
Hence, to find an update B ! we iterate the solution of the following upper-bounded convex subproblem:
arg min - log det (Diag BT Jag (B Jrg) [I, + J -Jr))
Bi+1_
B
tr(M,'), subject to B 0, B1g = 1,
(16)
5

---

where M P+V
G,
Di=1 hi Lr (B
We observe in practice that few (+e 5) iterations
are sufficient for convergence.
Problem (16) has the same format as Problem (8) and it can be readily solved by Algorithm 1. The proposed learning scheme for Student-t bipartite graphs summarized in Algorithm 2. The sequence BB} generated by Algorithm 2 converges to the set of stationary points of Problem (15). This result directly follows from the convergence results of the MM framework in [40].
Algorithm 2: Student-t bipartite graph learning (SBG)
Data: Data matrix X, initial feasible estimate of the graph weights B initial learning rate Og 0, tolerance > degree of freedoms v > 2.
Result: Learned bipartite graph: B
while 1 I maxiter do
update B by iterating (16)
it [|| B || |
S then
return B
end
end
3.3 k-component Student-r Bipartite Graphs
Graphs with multiple components can learned by introducing rank constraints on the Laplacian matrix [11, 12, 14]. More precisely, the number of zero eigenvalues of the Laplacian matrix amounts to the number of graph components, i.e., rank L) p - k, where & is the number components.
The approach taken by [12] approximates the rank constraint by applying Fan's theorem [41] as follows
Σ A (L)
minimize tr (VTLV).
(17)
VER V=1,
where the right hand side of (17) can be used as regularization term (ef. Problem (3)). On the other hand, the authors in [14] introduced the quadratic relaxation || L UAUT Those formulations [12, 14] can be conveniently solved by coordinate descent methods. However, they introduce additional variables to be optimized, extra hyperparameters be tuned, and do not guarantee that the rank constraint on L is satisfied.
In our proposed method, we directly apply the rank constraint into the optimization formulation. More precisely, using the Student-l case, we formulate the following optimization problem to learn k-component bipartite graph:
p+1 En log 1 h: Lr (BG.) llgg det (L),
minimize L=0,B
n
(18)
I, B
B Diag [ (BT1,) 1,)
subject to
L
rank(a) = k. B 0, B1, 1,,
Unlike the case for connected bipartite graphs in Problem (6), the pseudo-determinant term in (18) cannot be simplified through the block matrix determinant lemma [34]. Therefore, we design an iterative algorithm based on the ADMM framework [42] for Problem (18).
3.3.1 ADMM Solution
The augmented Lagrangian of Problem (18) can be written as
p+ +
h + tr (BG;)
log
1
log det (L)
n
+
+(4
I
B
Y
+ P
L
I
B Diag
(19)
-B
Diag
[ 1,)
2
BT
[ T 1,,

---

where > 0 is a penalty hyperparameter.
For fixed B and Y, the subproblem for L can be written
I,
B
2
arg min - let (L) + L, Y) rank(L)=p-A
+ 2 B T
L
Diag [
1.)
(20)
whose closed-form solution is given by Lemma 1.
Lemma The global minimizer of problem (20) is [43, 44]
- - 2p
T r 4pl R
(21)
I,
B Y, with I having
where RT R is the eigenvalue decomposition of the largest p eigenvalues along its diagonal and R RPx(x-k) containing the corresponding eigenvectors.
-B
Diag 1,)
Fixing L and Y. the subproblem for B is
p P+V
1+ h4 + tr (BG:)
I,
-B Diag
B = arg min B20.B1-1
T
log
+
2
L
-B
(B 1,
1=1
I,
B
Diag (BT 1,))
-B
r
(22)
which can be simplified as
B arg min B10,B1=1
p+
log
h + tr (BGi) 1
+ tr (BH) + BI
L
L
BBB1,
2
(23)
where H A T rq 1.tipl (1r+1pp17 2g) Urtip diag (Y) (Y)++ip and 1+ A
diag (L),+1.p'
Leveraging the MM framework to Problem (23), like it was applied to Problem (15), find an upperbounded subproblem at point B' as follows:
B arg min tr (B (H+ MM)) + p BI + +11, BB 1, B20,B1=1
(24)
Subproblem (24) strongly convex, thus it can solved efficiently via convex solvers. Since the objective function of (24) is smooth and its feasible set is compact, we provide PGD algorithm in the Supplementary Material. Finally, find an update B*, we iterate solution of Problem (24). The dual variable Y is updated approximately via gradient ascent.
Algorithm summarizes our proposed scheme learn k-component bipartite graphs and its conver- gence is established through Theorem 2, whose proof is presented in the Supplementary Material.
Theorem Algorithm 3 converges subsequently for any sufficiently large that is, the sequence B¹, (L,, BY)) generated by Algorithm 3 has least one limit point, and each limit point is stationary point of (18).
4 Experimental Results
We conduct experiments to illustrate the performance of the proposed algorithms in terms of quantita- tive measures such as node label accuracy and graph modularity. Accuracy is computed between the number of correctly predicted node labels and the number of nodes in the objects set, whereas modularity measures the strength of division of a graph into groups [45]. A high modularity value means that the nodes from the same group more likely to be connected.
the ratio
7

---

Algorithm 3: Student-t k-component bipartite graph learning (kSBG)
Data: Data matrix X, number of components kt initial feasible estimate of the graph weights B", penalty hyperparameter >> 0, tolerance e >> 0, degrees of freedom >> 2.
Result: Learned bipartite graph: B
while / S maxiter do
> update + via (21)
2
3 > update B by iterating (24)
I
B Diag (BI+IT) (B)+TT,,) ))
> update 4 = Y
[++1
B B
-
if B B' |||| B//1//BB11 111 S then
return B
end
end
Benchmarks: We compare the proposed algorithms with state-of-the-art methods in two settings: (i) connected bipartite graphs, where SOBG (k = [12] and SGA [14] are the benchmarks, and (ii) k-component bipartite graphs, where SOBG (k: 1) and SGLA [14] act as benchmarks. In our Algorithm 3, we set the hyperparameter = and the relative tolerance = 10-5. 5_ The hyperparameter of SOBG was tunned according to the heuristic procedure described in [12]. The proposed algorithms and benchmarks were implemented using the R programming language. For SGA and SGLA, we relied on their official implementation via the package PooecraalraaTTpoolge [46].
4.1 Returns of S&P500 Stocks
We perform experiments using publicly available, historical daily prices, queried from Yahoo! Finance TM, of r 333 stocks listed in the S&P500 Index and q 8 sectors indexes, namely: Consumer Staples, Energy, Financials, Health Care, Industrials, Materials, Real Estate, and Utilities. Following our notation, the node set of classes V,, represents the sectors indexes, whereas the stocks are represented by the node set objects V,.
Stock sector labels are available through the Global Industry Classification Standard (GICS) [47, 48). Arguably the most well-recognized, industry-standar stock classification system, GICS relies on a fixed classification structure that may not fully capture the actual impact of a particular stock at particular time. With the intent of solving this shortcoming, we employ the learned bipartite graphs to extend current industry stock classification system from static classification to a dynamic soft clustering, in which a particular node (stock) may belong to different clusters (sectors) with different degrees of membership that can change over time. Our goal is to perform soft clustering of stocks according to the market sectors. To that end, we recall that each row of B is a point belonging the probability simplex in R9, which represents the level of membership of stock to particular set of sectors. The final sector assigned to the i-th stock corresponds to arg max,ee.....g Bij.
We start by constructing the log- log-returns data matrix, iee,, matrix X E R=Xp, where is the number of log-returns observations and p is the number of instruments, as Xt, = log P.,j log P;-1.j. where Pt,, is the closing price of the j-th instrument at the i-th day. We collect data from Jan 5th 2016 to Jan 5th 2021, totalling n = 1291 daily observations. Then, we proceed to learn connected bipartite graphs via the benchmarks and proposed methods using the observations X fashion'. More precisely, we use window of length 504 days (2 years in terms of stock market days) and shift this window by 63 days (3 months in terms of stock market days).
bolling-window
Figure shows quantitative measurements that reveal the higher performance of the proposed SBG algorithm both in terms of accuracy and modularity. That may be explained by the fact that SBG adopts a multivariate Student-t distribution, which is well-known to better model financial datasets in comparison Gaussian assumptions.
An estimate for the degree of freedoms v, required by Algorithms 2 and 3, is obtained by fitting a univariate Student-t distribution to the log- log-returns of the S&P500 index during the corresponding time period.

---

Furthermore, we evaluate the performance of the k-component bipartite estimators under the same settings as previously described, where we & i.e., the number of sectors. Figure 2 reveals that kSBG presents a higher performance accross the whole time period both in terms of accuracy and modularity when compared SGLA and SOBG. Interestingly, both Figure and 2, we observe decline in accuracy and modularity of the methods GBG, SBG, and kSBG, starting in Jan 2020, which can be explained by the impact of the COVID-19 pandemic on the US stock market. This reveals that our methods may be able to identify periods of economic turmoil and suggests that, during such times, standards such as GICS may not be the best reference for stock market sector classification. This insight may be critical for investors who rely diversification of their portfolios through information on the sectors [49].
SBG (proposed)
GBG (propased)
SOBG
SGA
1.0
0.50
0.8
Modularity
0:40
0.7
0.35
0.30
May
Sep
May
Jan
Jan
Sep
Figure 1: Performance of the estimators for connected bipartite graphs of S&P000 stocks.
kSBG (proposed)
SOBG
SGLA
1.0
0.9
Modularity
Accuracy
0.88
0.6
0.7
0.6
0.4
0.5
Jan
May
5tp
Jan
May
Sep
Jan
May
Sep
Jan
Jan
May
Sep
Jan
May
Sep
Jan
May
Sep
Jan
2018
2018
2018
2019
2019
2019
2020
2020
2020
2021
2018
2018
2018
2019
2019
2019
2020
2020
2020
2021
Figure 2: Performance of the estimators for 8-component bipartite graphs S&P500 stocks.
Finally, Figure 3 shows the k-component bipartite models learned using the whole dataset. We observe that SGLA allows the existence isolated nodes, which degrades performance. Our proposed method kSBG avoids that issue by imposing linear constraint on the node degrees. In addition, kSBG yields a more intuitive representation of the financial network than SOBG as verified by the accuracy and modularity values.
Consuper Staples
Fuancials
ncials
Suppres
Energy
Materials
Consumer Staples
Unilaie
Real Estate
Real Estiftra
kSBG (proposed), accuracy 0.97, modularity 0.82.
SOBG [12]. accuracy 0.75, modularity 0.61.
(c) SGLA [14]. accuracy 0.77, modularity = 0.56.
Figure 3: Graphs learned by different algorithms in the S&P500 stock returns dataset.

---

4.2 Robustness to Initialization
The performance of iterative algorithms may be affected by their initial point, especially when dealing with nonconvex problems as in Algorithms 2 and 3. Hence, we perform an experiment to compare two initialization schemes: (i) uniform, the graph weights are randomly sampled. i.e., B Uniform(O, 1), and (ii) default, Bd x max (0, -Erg). where S E,,- Erg: J,,, S is the inverse of the sample correlation matrix, as discussed in more details [14]. In both schemes, we normalize the rows of Bd such that B 1 holds. For this experiment, we consider log-returns of 362 stocks and q = 9 stock sectors from Oct. 5th 2005 to Dec. 30th 2015, totalling n 2577 observations. Figure 4 shows the result of 100 realizations of Algorithms 2 and 3. It can be observed that Algorithms 2 and are not sensitive to the choice an initial point, which an important feature for practical applications. Finally, Figures 5a and 5b show the learned graphs from kSBG with different initial points, where we observe that the performance of kSBG is largely unaffected.
default
Objectvve Function
Lagrangian
-100
2
-130
10-1
CPU Time (seconds)
CPU Time [sacanck)
(a) Algorithm (SBG).
(b) Algorithm 3 (kSBG).
Figure 4: Convergence trend of the proposed algorithms for different initial points.
Utilttes
Energy
Staples
Ith Usluttittees
Technology
Technology
Ledusttrials
Ftth Care
Consun
Discheboonary
Consumerr Discretionar
Materiale
Staples
(a) default initial point. accuracy 0.825, modularity 0.691.
(b) uniform initial point. accuracy 0.820, modularity 0.687.
Figure 5: Visualizations of the graphs learned by kSBG with distinct initial points.
Conclusions
The design of numerical algorithms plays a critical role in disseminating the application of graphical models in practical, real-world instances. This paper has proposed optimization formulation to the problem of learning bipartite graph from data along with algorithm based on the PGD framework. Envisioning applications scenarios where outliers or heavy-tail events are present, we have extended the proposed formulation to the case where the data follow a multivariate Student-t distribution. addition, we have proposed an optimization formulation, along with a provably convergent ADMM- based algorithm, for estimating k-component bipartite graphs. Experimental results with datasets of log-return of S&P500 stocks have provided substantial evidence for the improved performance of the proposed algorithms in comparison to ofte-of-thh-an methods.
Acknowledgments
This work was supported by the Hong Kong GRF 16207820 research grant.
10

---

References
[1] S. M. Smith, K. L. Miller, G. -horsiiKi M. Webster, C. F. Beckmann, T. E. Nichols,
J. D. Ramsey, and M. W. Woolrich. Network modelling methods for FMRI. Neurolmage, 54(2):875 2011.
[2] O. Stegle, S. A. Teichmann, and J. C. Marioni. Computational and analytical challenges in single-cell transcriptomies. Nature Reviews Genetics, 16:133 1451,,--4,, 2015.
[3] D. Hallac, Y. Park. S. Boyd, and J. Leskovec. Network inference via the time-varying graphical lasso. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD '7. pages 205-213, 2017.
[4] D. Hallac, S. Vare, S. Boyd, and J. Leskovec. Toeplitz inverse covariance-based clustering of multivariate time series data. In Proceedings of the 23rd ACM SIGKDD International Conference Knowledge Discovery and Data Mining (KDD'I7) page 215-223,, 2017.
[5] A. Jung, G. Hannak, and N. Goertz. Graphical lasso based model selection for time series. IEEE Signal Processing Letters, 12110):781-1188 2015.
[6] M. Cardoso and D. Palomar. Learning undirected graphs in financial markets. In 54th
Anmual Asilomar Conference on Signals, Systems, and Computers, 2020.
[7] J. V. M. Cardoso, J. Ying, and D. P. Palomar. Graphical models in heavy-tailed markets. In Advances in Neural Information Processing Systems (NeurLPS:21), 2021.
[8] Friedman, T. Hastie, and R. Tibshirani. Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9:432-41, 2008.
[9] O. Banerjee, I.. F. Ghaoui, and A. d' Aspremont. Model selection through sparse maximum likelihood estimation for multivariate Gaussian or binary data. Journal of Machine Learning Research, 9:485-516, 2008
[10] J. Wright. Coordinate descent algorithms. Mathematical Programming, 151:3-34, 2015.
[11] F. Nie, X. Wang, M. I. Jordan, and H. Huang. The constrained Laplacian rank algorithm for graph-based clustering. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI' 16, pages 1969-1976. AAAI Press, 2016.
[12] Nie, X. Wang, C. Deng, and H. Huang. Learning a structured optimal bipartite graph for co-clustering. In Advances on Neural Information Processing Systems (NeurIPS' 17), page 4132-4141, 2017.
[13] Kumar, J. Ying, M. Cardoso, and D. P. Palomar. Structured graph learning via Laplacian spectral constraints. In Advances in Neural Information Processing Systems (NeurIPS), 2019.
[14] S. Kumar, J. Ying, J. V. M. Cardoso, and D. P. Palomar. A unified framework for structured graph learning via spectral constraints. Journal of Machine Learning Research, 21:1-60, 2020.
[15] M. Finegold and M. Drton. Robust graphical modeling with t-distributions. Conference on Uncertainty in Artificial Intelligence, 2009.
[16] Y. Wald, N. Noy, G. Elidan, and A. Wiesel. Globally optimal learning for structured elliptical losses. In Advances in Neural Information Processing Systems (NeurIPS' 19), 2019.
[17] M. Slawski and M. Hein. Estimation of positive definite M-matrices and structure learning for attractive Gaussian Markov random fields. Linear Algebra and its Applications, pages 145-179, 2015.
[18] R. Agrawal, U. Roy, and C. Uhler. Covariance matrix estimation under total positivity for portfolio selection. Journal of Financial Econometrics, 20:367-389, 2020.
[19] Y. Wang. U. Roy, and C. Uhler. Learning figh-dimensional Gaussian graphical models under total positivity without adjustment of tuning parameters. In Proceedings of the Twenty Third International Conference on Artificial Intelligence and Statistics (AISTATS' 20), volume 108, pages 2698 20088,7788228 2020.
[20] J.-F. Cai, J. V. M. Cardoso, D. P. Palomar, and J. Ying. Fast projected newton-like method for precision matrix estimation under total positivity. arXiv e-prints: 2112.01939, August 2022.
[21] Ying, V. M. Cardoso, and D. P. Palomar. Does the & norm learn a sparse graph under Laplacian constrained graphical models? arXiv e-prints: 2006.14925, June 2020.
11

---

[22] J. Ying, J. V. M. Cardoso, and D. P. Palomar. Nonconvex sparse graph learning under Laplacian- structured graphical model. In Advances in Neural Information Processing Systems (NeurIPS). volume 33, pages 7101-7113, 2020.
[23] J. Ying, J. V. M. Cardoso, and D. Palomar. Minimax estimation of laplacian constrained precision matrices. In 24th International Conference on Artificial Intelligence and Statistics, 2021.
[24] C. Hu, L. Cheng, J. Sepulcre, K. A. Johnson, G. E. Fakhri, Y. M. Lu, and Q. Li. A spectral graph regression model for learning brain connectivity of alzheimer's disease. PLOS ONE, 10(5):1-24, 05 2015.
[25] X. Dong. D. Thanou, Frossard, and P. Vandergheynst. Learning Laplacian matrix in smooth graph signal representations. IEEE Transactions on Signal Processing, 64(23):6160-611 2016.
[26] V. Kalofolias. How to learn a graph from smooth signals. Proceedings of the 19th Interna- tional Conference on Artificial Intelligence and Statistics, volume 51, pages 920-929. PMLR. 2016.
[27] G. Mateos, S. Segarra, A. G. Marques, and A. Ribeiro. Connecting the dots: Identifying network structure via graph signal processing. IEEE Signal Processing Magazine, 36(3):16 2019.
[28] H. Rue and L. Held. Gaussian Markov Random Fields: Theory And Applications (Monographs on Statistics and Applied Probability). Chapman & Hal//CRC, 2005.
[29] H. E. Egilmez, E. Pavez, and A. Ortega. Graph learning from data under Laplacian and structural constraints. IEEE Journal of Selected Topics in Signal Processing, 166)8825-841, 2017.
[30] O. Knill. Cauchy-Bine -Binet for pseudo deeuddddetrmmnnnts.. Linear Algebra and its Applications, 459:522- 547, 2014.
[31] F. Nie, X. Wang, and H. Huang. Clustering and projected clustering with adaptive neighbors. In Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining (KDD'14 14), 2014.
[32] D. Bertsekas. Nonlinear Programming. Athena Scientific, 1999.
[33] A. Beck. First-Order Methods in Optimization. MOS-SIAM Series on Optimization. Society for Industrial and Applied Mathematics (SIAM), 2017.
[34] F. Zhang. The Schur Complement and Its Applications. Springer, 2005.
[35] C. Michelot. A finite algorithm for finding the projection of point onto the canonical simplex of RR" Journal of Optimization Theory and Applications, 50(1):195-20) July 1986.
[36] S. Boyd and L. Vandenberghe. Convex Optimization. Cambridge University Press, 2004.
[37] D. P. Palomar and J. R. Fonollosa. Practical algorithms for a family of waterfilling solutions.
IEEE Transactions on Signal Processing, 53(():686-69) 2005.
[38] L. Condat. Fast projection onto the simplex and the & ball. Mathematical Programming, 158, 2016.
[39] S. Resnick. Heavy-Tail Phenomena: Probabilistic and Statistical Modeling. Springer- Verlag New York, 2007.
[40] Y. Sun, P. Babu, and D. P. Palomar. Majorization-minniiizzooo algorithms in signal processing, communications, and machine learning. IEEE Transactions on Signal Processing, 65(3))794- 816. 2017.
[41] K. Fan. On theorem of Weyl concerning eigenvalues of linear transformations I. Proceedings of the National Academy of Sciences, 35(1 6555, 1949.
[42] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein. Distributed optimization and statistical learning via the alternating direction method of multipliers. Foundations and Trends in Machine Learning, 3(1):1-122, 2011.
[43] D. M. Witten and R. Tibshirani. Covariance-regularizee regression and classification for high dimensional problems. Journal of the Royal Statistical Society. Series B (Statistical Methodology), 71(3):615-636 2009.
12

---

[44] P. Danaher, P. Wang. and D. M. Witten. The joint graphical lasso for inverse covariance
estimation across multiple classes. Journal of the Royal Statistical Society Series B, 76(2):373 397, 2014.
[45] M. E. J. Newman. Modularity and community structure in networks. Proceedings of the National Academy of Sciences the United States of America, 103, 2006.
[46] J. V. M. Cardoso and D. Palomar. spectralGraph Topolloy Learning graphs from data via spectral constraints. In The Comprehensive Archive Network. https: /ttps://CRAN... R- project.
org/ ppaackage=specteee prrpraagepppctaaa..aaaaaooolppp 2019.
[47] Standard & Poor's. Global Industry Classification Standard (GICS). Tech Report, 2006.
[48] Morgan Stanley Capital International and S&P Dow Jones. Revisions the global industry classification standard (GICS) structure, 2018.
[49] Y. Yang, L. Zhao, L. Chen, C. Wang, and J. Han. Portfolio optimization with idiosyncratic and systemic risks for financial networks. arXiv e-prints: 2111.11286, 2021.
[50] Y. Wang, W. Yin, and J. Zeng. Global convergence of admm in nonconvex nonsmooth opti- mization. Journal of Scientific Computing, 78:29-65 63, 2019.
Checklist
1. For all authors...
(a) Do the main claims made in the abstract and introduction accurately reflect the paper's
contributions and scope? [Yes]
(b) Did you describe the limitations of your work? [Yes]
(c) Did you discuss any potential negative societal impacts of your work? [N/A]
(d) Have you read the ethics review guidelines and ensured that your paper conforms to them? [Yes]
2. If you are including theoretical results...
(a) Did you state the full set of assumptions all theoretical results? [Yes]
(b) Did you include complete proofs of theoretical results? [Yes] See Supplementary Material.
3. If you ran experiments..
(a) Did you include the code, data, and instructions needed reproduce the main ex- perimental results (either in the supplemental material or as URL)? [Yes] The code, data, and instructions to reproduce the experiments are available https: // //github.. com/m coom/micca/biii httaa/pppprtttteeee
(b) Did you specify all the training details (e.g., data splits, hyperparameters, how they were chosen)? [Yes]
(c) Did you report error bars (e.g., with respect to the random seed after running experi- ments multiple times)? [Yes]
(d) Did you include the total amount of compute and the type of resources used (e.g., type of GPUs, internal cluster, or cloud provider)? [Yes]
4. If you are using existing assets (e.g., code, data, models) or curating/releasing new assets...
(a) If your work uses existing assets, did you cite the creators? [Yes]
(b) Did you mention the license of the assets? [Yes]
(c) Did you include any new assets either in the supplemental material or as a URL? [Yes]
(d) Did you discuss whether and how consent was obtained from people whose data you't 're using/curating? [N/A] The data is publicly available.
(c) Did you discuss whether the data you using/curating contains personally identi- fiable information or offensive content? [N/A] The data does not contain personally identifiable information offensive content.
5. If you used crowdsourcing or conducted research with human subjects...
(a) Did you include the full text of instructions given to participants and screenshots, applicable? [N/A]
13

---

(b) Did you describe any potential participant risks, with links to Institutional Review Board (IRB) approvals, applicable? [N/A]
(c) Did you include the estimated hourly wage paid to participants and the total amount spent participant compensation? [N/A]
14


Reviews:
[{"review_id": "vd9Bnoy-yhO", "reviewer": "Reviewer_Jahb", "summary": "This paper investigates the problem of learning undirected, bipartite graphs under different statistical models. The authors propose an optimization framework based on projected gradient descent for learning bipartite graphs under the Gaussian Markov random field model and the problem formulation is extended to learning bipartite graphs under multi-variate Student-t distribution and under multi-component assumption. The key feature of different algorithms proposed is leveraging the properties of bipartite graphs with statistical properties of the considered data models to efficiently infer graph structures. Experiments on a real world dataset demonstrate that the proposed algorithms significantly outperform the state of the art on a multi-component clustering task. ", "questions": "1. Please provide a reference to the exact lemma for block matrices from [24]. \n2. The definition of operator $P_{\\Delta}$ seems to be missing.\n3. I did not follow how the problem in (10) amounts to a Euclidean projection on the probability simplex. I suggest adding more details to accompany the following statement after (10): Problem (10) is a Euclidean projection of the rows....\n4. Please provide mathematical definitions for modularity and accuracy in Section 4.\n5. I believe that the paper will benefit from a more elaborate empirical evaluation focused on learning of bipartite graphs (perhaps on a synthetic dataset) to conclusively demonstrate the utility of the proposed algorithms and any gains over state of the art approaches. ", "limitations": "Yes. ", "rating": 5, "confidence": 4, "soundness": 3, "presentation": 3, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths:\n1. Structure learning of bipartite graphs is a well-motivated problem. Learning bipartite graphs under Gaussian model is well-defined and uses standard technical arguments. The main contributions of interest are learning bipartite graphs under the Student-t distribution to accommodate data models with outliers or heavy-tailed distribution. For this purpose, the authors provide an iterative algorithm based on majorization-minimization framework and extend it to the learning of $k$-component, bipartite graphs. \n2. The mathematical arguments are sound and the rationale behind various choices in the implementations are (usually) well described. \n3. The paper is clearly written and easy to follow.\n\nWeaknesses:\nI find the experiments section to be too narrowly focused and possible inadequate to demonstrate learning of bipartite graphs.\nSpecifically, the experiments focus only on a clustering task and ignore learning of edge weights in these graphs (note that here I refer to the edge weights within the identified clusters as well). The two metrics that quantify accuracy of clustering and the modularity of identified clusters seem to be insufficient in this regard.  ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "9X9Ioz5rSjc", "reviewer": "Reviewer_LmQW", "summary": "The authors present a new optimization problem formulation for learning weighted bipartite graphs, and also extend the formulation to the presence of heavy tailed events (outliers) motivated by real-world problems. They propose a solution to both problem formulations based on PGD and MM, respectively. Experimental results on financial datasets demonstrate the superior performance of the approach compared to competing approaches, however the experiments themselves are limited. ", "questions": "1) My major concern is that unfortunately the experimental results are focused on a single dataset and such have limited power to demonstrate the proposed approach. The experimental results section would benefit from a synthetic simulation in which the graphs are known and there are well defined outlier events Competing approaches (refs [9] and [11]) had much more detailed and varied experiments and the current paper would benefit from adding an additional synthetic experiment and real-world experiment. \n\n2) The statement that there is a decline in accuracy / modularity surrounding an adverse event such as COVID 19 or the crash in 2008 (appendix) and this is a good property of the proposed method is unclear. How does this demonstrate that the approach can handle adverse effects if it results in worse performance? How in practice would this framework be used to identify such an event or perform under these heavy-tail conditions? In comparison, the competing approaches are unaffected or perform better after 2020. \n\n3) What is the run-time complexity of this approach compared to competing methods (the supplement only details the proposed method)?\n", "limitations": "The authors did not describe limitations \n", "rating": 7, "confidence": 3, "soundness": 2, "presentation": 4, "contribution": 3, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Originality: \nThe paper proposes a novel graph learning framework and efficient solutions in the convex and non-convex case. \n\nMissing references from the graph learning literature:\n* Dong, Xiaowen, et al. \"Learning Laplacian matrix in smooth graph signal representations.\" IEEE Transactions on Signal Processing 64.23 (2016): 6160-6173.\n* Kalofolias, Vassilis. \"How to learn a graph from smooth signals.\" Artificial Intelligence and Statistics. PMLR, 2016\n* Mateos, Gonzalo, et al. \"Connecting the dots: Identifying network structure via graph signal processing.\" IEEE Signal Processing Magazine 36.3 (2019): 16-43.\n* Hu, Chenhui, et al. \"A spectral graph regression model for learning brain connectivity of Alzheimer’s disease.\" PloS one 10.5 (2015): e0128136.\n\nQuality: \nThe paper is technically sound and the solutions appropriate. The experimental section is somewhat lacking: Further comments in the questions regarding the experiments. \n\nClarity: \nThe paper is well-written and the methods clearly laid out in algorithmic form for reproducibility. \n\nMinor comments:\n* Eq 3: V undefined\n* Line 79: the statement that the method in [9] lacks statistical support seems somewhat harsh without toning down that it is motivated from a spectral graph theory perspective.\n* Acronyms in section 4 are undefined (SOBG, SGA, SGLA)\n* line 242: what does out of sample accuracy refer to? There was no reference to a train/test split with respect to this experiment so who are the out-of-samples? The new days? \n* Figure 3 missing colorbar\n\n", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ""}, "extra_text": {}}, {"review_id": "4zUUUw9bSKm", "reviewer": "Reviewer_C5Ut", "summary": "Paper proposes methods for graphical model selection from i.i.d. observations. The graph structure is constrainted to consist of compoeents being bipartite graphs with two node types (objects and classes). Two types of methods are proposed: one for Gaussian models and another one for student-t distributed data that is suited for data plagued by outliers. \n", "questions": "* What is \"..a verifiably convergent iterative algorithm\" ? \n\n* What is a \"..unconstrained Gaussian Markov random fields (MRF)\" ? \n\n* What do you mean by \"..bipartite graphs under the MRF assumption.\" and \" from data under probabilistic assumptions\"? Do you mean that the underlying graph structure encodes pairwise local, global Markov properties ? (https://en.wikipedia.org/wiki/Markov_random_field)  It would also be useful to write down the prob. distribution of the observed data more explicitly. \n\n* What do you mean by \"..We design an efficient optimisation algorithm\" ? Does your algorithm achieve fundamenal limits regarding number of observations or computational complexity ? In which sense is your algorithm \"efficient\" ? \n\n* What is \"a Laplacian-constrained precision matrix\"?\n\n* What is the relation between matrix A in (3) in the Laplacian matrix that you are after ? \n\n* Could you explain in more detail what you mean by \"..our proposed methods and algorithms\" \n\n* What is an \"observed practical complexity\" ? \n\n* What are \"heavy-tailed events\" and \"heavy-tailed bipartite graphs\"? \n\n* Can you pls provide more details for the arguing \"...nonconvex due to the summation over concave terms and hence it is difficult to be dealt with directly.\"\n\n* How does the rank constraint in (18) enter into (19)?\n\n* What are the \"node labels\" that you refer to in the Experiments  ?\n\n* What is the precise definition for the accuracy and modularity depicted in Figure 1 ? \n\n* Is Algorithm 3 returning a graph or a matrix ? \n\n* You might put your approach into context of another family of networked-structured probabilistic models: \n\nA. Jung, \"Networked Exponential Families for Big Data Over Networks,\" in IEEE Access, vol. 8, pp. 202897-202909, 2020, doi: 10.1109/ACCESS.2020.3033817.\n\nIn contrast, to probabilistic graphical models (such as GMRF), networked exponential families model local datasets (at nodes) using separate probability spaces. In contrast, GMRF use a common probability space on which all node attributes/observations are defined. \n*", "limitations": "n/a", "rating": 4, "confidence": 4, "soundness": 2, "presentation": 2, "contribution": 2, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "Strengths: \n* Learning network structure of data is an enabling task for many follow up applications such as federated learning. \n* Methods are evaluated on \"real-world\" data. \n\nWeaknesses: \n\n* The novelty is somewhat limited as the proposed methods are combinations of established approaches to graphical model selection via penalaized maximum likelihood methods. The penalty terms are well motivated by existing literature on learning bipartite graphs. \nThe main technical result seems to be Theorem 2 which characterizes the convergence of Algorithm 3. What is the practical relevance of Theorem 2 ? How is it used/confirmed in the numerical experiments ?\n\n* The underlying probabilistic model should be written out in more detail. In particular, are the samples assumed to be i.i.d.. or a stationary process which would be more suitable for the numerical experiments which use time series data. \n\n* The numerical experiments should be expanded by considered synthetic datasets that allow to measure the estimation error of Algorithm 1 - 3. Moreover, it would be interesting to compare with recent methods for graphical model section from samples forming (non-)stationary processes: \n\nA. Jung, G. Hannak and N. Goertz, \"Graphical LASSO based Model Selection for Time Series,\" in IEEE Signal Processing Letters, vol. 22, no. 10, pp. 1781-1785, Oct. 2015, doi: 10.1109/LSP.2015.2425434.\n\nA. Jung, \"Learning the Conditional Independence Structure of Stationary Time Series: A Multitask Learning Approach,\" in IEEE Transactions on Signal Processing, vol. 63, no. 21, pp. 5677-5690, Nov.1, 2015, doi: 10.1109/TSP.2015.2460219.\n\nN. Tran, O. Abramenko and A. Jung, \"On the Sample Complexity of Graphical Model Selection From Non-Stationary Samples,\" in IEEE Transactions on Signal Processing, vol. 68, pp. 17-32, 2020, doi: 10.1109/TSP.2019.2956687.\n\n* The paper would be much stronger if it would offer theoretical performance guarantees, e.g., required number of observations to ensure a prescribed estimation quality. ", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "", "extra_scores": {"ethics_review_area": ["I don’t know"]}, "extra_text": {}}]

Author rebuttal:

Decision: Accept