FasMe: Fast and Sample-efficient Meta Estimator for Precision Matrix Learning in Small Sample Settings

Authors: xiao  tan
yiqin  wang
yangyang  shen
dian  shen
meng  wang
peibo  duan
beilun  wang
Venue: NeurIPS_cc_2024_Conference
Published: 2024-05-14 09:50:29

Abstract: Precision matrix estimation is a ubiquitous task featuring numerous applications such as rare disease diagnosis and neural connectivity exploration. However, this task becomes challenging in small sample settings, where the number of samples is significantly less than the number of dimensions, leading to unreliable estimates. Previous approaches either fail to perform well in small sample settings or suffer from inefficient estimation processes, even when incorporating meta-learning techniques.
To this end, we propose a novel approach FasMe for Fast and Sample-efficient Meta Precision Matrix Learning, which first extracts meta-knowledge through a multi-task learning diagram. Then, meta-knowledge constraints are applied using a maximum determinant matrix completion algorithm for the novel task. As a result, we reduce the sample size requirements to $O(\log p/K)$ per meta-training task and $O(\log\vert \mathcal{G}\vert)$ for the meta-testing task. Moreover, the hereby proposed model only needs $O(p \log\epsilon^{-1})$ time and $O(p)$ memory for converging to an $\epsilon$-accurate solution. On multiple synthetic and biomedical datasets, FasMe is at least ten times faster than the four baselines while promoting prediction accuracy in small sample settings.

Main paper: 
 introduction: 
 Precision matrix estimation aims to exploit conditional dependency relationships between random variables, which plays a vital role in high-dimensional statistical learning with a wide range of applications in areas such as genetics [1] , neuroscience [2] , and social networks. For instance, researchers are examining the gene network derived from the gene expression data of patients to investigate a rare disease. However, in cases involving rare diseases, the analysis is hindered by the high dimensionality of genetic datasets, where the number of features significantly exceeds the number of samples. For example, the Cholangiocarcinoma dataset in TCGA contains only 51 samples but thousands of genes. In this context, we define a "small sample setting" as a scenario in which the number of samples is less than one-tenth of the number of dimensions. Although inferring such relationship graphs in small sample settings is challenging, it can provide valuable insights into the genetic characteristics underlying this disease and its etiology.Precision matrix estimator inherently has the capability to handle datasets where the number of dimensions p is equal to or greater than the number of samples n [3] . However, this capability has its limits, and maintaining effectiveness requires keeping the sample size at a certain level. [4] provides the lower bound of the sample size as Ω(d 2 log p) for well-performed graph recovery under the sub-Gaussian assumption, where d is the maximum node degree of the graph. Consequently, when the sample size is reduced to the aforementioned "small sample" levels, the sample size falls short of this theoretical lower bound with a relatively high probability due to the uncertainty of d. This indicates insufficient data support for most methods, like GLasso [5] (detailed in Background), to recover the graph structure effectively, as shown in Figure 1 . To address this issue, some methods use multi-task learning, integrating heterogeneous datasets to reduce the sample size bound. For example, the regularized methods [6] and [7] show that each task only needs O(log K + log p) and O(K + log p) samples, respectively, for proper estimation, and K denotes the number of tasks. Although these methods are more capable of handling small sample problems, they concurrently introduce significant computational burdens. Specifically, integrating a new task necessitates repeated joint retraining, thereby resulting in substantial computational inefficiencies. • A novel meta-learning-based model: We propose a multi-task precision matrix estimator that extracts the meta-knowledge from the auxiliary tasks. We then utilize the maximum determinant matrix completion algorithm to learn a new precision matrix in the meta-testing. • Efficient adaptation: We design a recursive closed-form algorithm to speed up the adaptation in the novel task. Our model converges to an ϵ-accurate solution in O(p log ϵ -1 ) time and O(p) memory. It significantly outperforms the state-of-the-art ones. • Theoretical guarantee: We also theoretically prove that for p-dimensional multivariate sub-Gaussian random vectors and K auxiliary tasks with meta-knowledge θ, the sample complexity of our method is O(log p/K) per auxiliary task for meta-knowledge and O(log |G|) for the new task. It relaxes sample size requirements with the application of meta-learning, equipping FasMe with the ability to handle small sample settings. • Experimental evaluation: On multiple synthetic datasets, FasMe outperforms the other baselines. Specifically, FasMe is at least ten times faster than the four baselines while promoting prediction accuracy. In the real-world experiment, FasMe obtains the minimum log-determinant Bregman divergence and performs better than the other baselines.2 BackgroundQuestion: Do the main claims made in the abstract and introduction accurately reflect the paper's contributions and scope? Answer: [Yes] Justification: We summarize the paper's contributions and scope in lines 85-99. Guidelines:• The answer NA means that the abstract and introduction do not include the claims made in the paper. • The abstract and/or introduction should clearly state the claims made, including the contributions made in the paper and important assumptions and limitations. A No or NA answer to this question will not be perceived well by the reviewers. • The claims made should match theoretical and experimental results, and reflect how much the results can be expected to generalize to other settings. • It is fine to include aspirational goals as motivation as long as it is clear that these goals are not attained by the paper. 
meta-learning: 
 The last years have witnessed a tremendous interest in methods for meta-learning, especially in a wide range of data-limited applications including medical image analysis [13, 14] , language modeling [15, 16] , and object detection [17, 18, 19] . Meta-learning is a machine learning technique that allows a model to learn from prior knowledge, or meta-knowledge, to improve its performance on new tasks and speed up the learning process [20, 21] . The process typically involves two stages: 1) Meta-training: In this stage, the model is trained on a set of related tasks, also known as auxiliary tasks [8] . The goal of this step is to extract meta-knowledge from these tasks, which can be used to improve the performance of the model on new tasks. This step helps the model to learn how to quickly and effectively adapt to new tasks by identifying the shared patterns or features across the auxiliary tasks [22] . 2) Meta-testing: In this stage, the model is tested on a new task, also known as the target task [23] . The goal of this step is to apply the meta-knowledge learned during the meta-training step to improve the model's performance on the target task. Meta-knowledge is used to quickly and effectively adapt the model to the new task, without the need for additional training data. Overall, meta-learning allows a model to better generalize to new tasks by leveraging prior knowledge, thereby making it more sample-efficient and fast.Precision matrix estimation is the problem of finding the inverse of the covariance matrix of a multivariate random variable Ω = Σ -1 . It is particularly useful in scenarios where the number of samples is equal to or smaller than the number of variables since the covariance matrix is not invertible. The graphical lasso, namely GLasso [5] , is a typical penalized maximum likelihood estimator for precision matrix Ω inference under the Gaussian assumption on the dataset X ∈ R n×p . The popular estimator can be written as:EQUATIONwhere λ ≥ 0, Σ = 1 n n i=1 (X i -X)(X i -X) ⊤ , X = 1 n n i=1 X i , and Ω ≻ 0 denotes that Ω is positive definite and symmetric. ∥Ω∥ 1 is the ℓ 1 -norm of matrix Ω. However, [24] proved that this estimator is sensible even for non-Gaussian X, since it corresponds to minimizing an ℓ 1 -regularized log-determinant Bregman divergence. To solve the problem Eq. (1), we take the derivative of it Ω -1 -Σ = λ∂∥Ω∥ 1 /∂Ω. Inspired by this derivative, [25] proposed the constrained ℓ 1 minimization method for inverse matrix estimation (CLIME) estimator. This estimator can be used to estimate the precision matrix Ω via an ℓ 1 constrained optimization:EQUATIONCLIME can be solved column-by-column. Compared with GLasso, CLIME has presented more favorable theoretical properties and can be solved by column-wise linear programming.Question: Does the paper discuss the limitations of the work performed by the authors? Answer: [Yes] Justification: Due to the space limit, we include the discussion about the limitation in Appendix C. Guidelines:• The answer NA means that the paper has no limitation while the answer No means that the paper has limitations, but those are not discussed in the paper. • The authors are encouraged to create a separate "Limitations" section in their paper.• The paper should point out any strong assumptions and how robust the results are to violations of these assumptions (e.g., independence assumptions, noiseless settings, model well-specification, asymptotic approximations only holding locally). The authors should reflect on how these assumptions might be violated in practice and what the implications would be. • The authors should reflect on the scope of the claims made, e.g., if the approach was only tested on a few datasets or with a few runs. In general, empirical results often depend on implicit assumptions, which should be articulated. • The authors should reflect on the factors that influence the performance of the approach.For example, a facial recognition algorithm may perform poorly when image resolution is low or images are taken in low lighting. Or a speech-to-text system might not be used reliably to provide closed captions for online lectures because it fails to handle technical jargon. • The authors should discuss the computational efficiency of the proposed algorithms and how they scale with dataset size. • If applicable, the authors should discuss possible limitations of their approach to address problems of privacy and fairness. • While the authors might fear that complete honesty about limitations might be used by reviewers as grounds for rejection, a worse outcome might be that reviewers discover limitations that aren't acknowledged in the paper. The authors should use their best judgment and recognize that individual actions in favor of transparency play an important role in developing norms that preserve the integrity of the community. Reviewers will be specifically instructed to not penalize honesty concerning limitations. 
method: 
 The section elaborates on how the proposed meta-learning framework works. We begin by providing a formal problem setting in Section 3.1, which covers the task and data assumptions we consider. Next, we formulate the two stages of our meta-learning framework in Section 3.2.Notations {X (k) } = X (1) , X (2) , . . . , X (K) denotes K datasets generated by K auxiliary tasks. We use X (k) ∈ R n k ×p to represent the k-th dataset with n k samples and p features. Ω (k) ∈ R p×p represents the k-th precision matrix corresponding to X (k) . {Ω (k) } = Ω (1) , . . . , Ω (K) is the set of the precision matrices corresponding to the auxiliary data {X (k) }. Ω new denotes the precision matrix of a new task X new ∈ R n×p to be estimated. We use θ to represent the meta-knowledge (i.e., the common substructure of {Ω (k) }) and θ(k)r to represent the rest of Ω (k) respectively.Assumptions We consider a more general class of distributions than most previous studies, i.e., sub-Gaussian distributions, which cover Gaussian variables, bounded random variables, and so on. This relaxed data assumption significantly improves the flexibility and robustness of our method. We assume that the auxiliary tasks share some meta-knowledge θ with the target task X new . Specifically, we assume that supp(θ) ⊆ supp(Ω new ). Here the meta-knowledge θ takes the form of a common substructure among the tasks. The assumption has been widely adopted [26, 27, 28] and proved to be feasible and applicable in the biological and genetic domains [29] .The paper aims to estimate the target precision matrix Ω new of sub-Gaussian distribution with sparse structure and relatively limited samples given sufficient samples from auxiliary tasks {X (k) }. The auxiliary datasets follow a family of multivariate sub-Gaussian distributions (see Definition 2 in Appendix F). To address the problem, we propose a novel meta-learning framework that leverages the meta-knowledge learned from auxiliary tasks to efficiently estimate Ω new .This part gives more details regarding the meta-learning framework we established. Figure 2 illustrates the pipeline of the framework in an intuitive way. In Stage 1, our meta-teacher extracts the meta-knowledge, namely the shared structure, from the related auxiliary tasks with sufficient samples. In Stage 2, we obtain a prior sparsity pattern from the meta-knowledge and target dataset with a few samples. Then our meta-student aims to recover the edge structure by solving a matrix completion problem rapidly.We extract the common substructure across the tasks as the meta-knowledge. The goal of pre-training is to extract meta-knowledge θ from auxiliary tasks {X (k) }.To obtain the meta-knowledge, every precision matrix is modeled asΩ (k) = θ + θ (k) r (3)where θ ∈ R p×p is the common substructure among all graphs and θ (k) r ∈ R p×p represents the rest for k-th graph.We adopt ℓ 1 -penalization for θ and θ (k) r since they are expected to be sparse for better interpretability. Therefore, we write Eq. ( 3) into ∥θ∥ 1 + ρ∥θ(k)r ∥ 1 , where ρ > 0. The value of the hyper-parameter ρ depends on the properties of the graphs. This hyper-parameter controls the difference of sparsity level between θ and θ (k) r . Concretely, with a smaller ρ, the shared part gets denser and the rest gets sparser. Then we apply the formulation to Eq. ( 2), thus obtaining the single-task recovering method:EQUATIONHere Σ (k) represents the covariance matrix corresponding to X (k) . This single-task method, however, ignores the inner relationship between the auxiliary tasks. To this end, we sum up all the single-task estimators and devise the optimization problem in the following form:EQUATIONwhere k = 1, 2, . . . , K.To speed up the estimation of Ωnew , we propose our efficient estimator, which is favorable even with a few samples of the new task available. Additionally, we obtain a sparsity pattern from the meta-knowledge θ and target dataset with a few samples.Different from most ℓ 1 -penalized methods, our estimator aims to estimate the precision matrix by solving a Maximum Determinant Matrix Completion (MDMC) problem. MDMC is an effective technique that aims to pick the unique maximum determinant completion (when it exists) for a partially observed matrix from all the possible matrices [30] .Consider a maximum determinant matrix completion problem for a covariance matrix with some partially observed entries:EQUATIONwhere M represents the partially observed matrix, and the set G contains all the observed entries. Thus, the Lagrangian dual of this problem can be derived as:EQUATIONwith first-order optimality condition |sign(M )| ⊙ Ω-1 = M , where the operator sign(•) computes the sign of the matrix and ⊙ denotes the Hadamard product. Herein, the set S p G refers to the p × p symmetric matrices with sparsity pattern G. Strong duality indicates a straightforward relation back to the primal Σ = Ω-1 . Note that while Σ is (in general) a dense matrix, Ω is always sparse. Instead of solving the primal problem for a dense matrix, we choose to solve the dual problem for a sparse matrix, which also satisfies the optimality condition.To solve the precision matrix Ωnew , the first step is to find a reliable sparsity pattern. Previous works [31] obtain the pattern by thresholding the covariance matrix. However, [31] also proved the invalidity of such a method in the case of a relatively small sample size, as the covariance matrix cannot be accurately estimated. To meet the challenge, our estimator combines the support set of the thresholded covariance matrix and meta-knowledge to obtain the sparsity pattern G.EQUATIONHere, S η denotes the element-wise soft-thresholding operator with parameter η. Note that the problem (7) has a recursive closed-form solution whenever the graph is chordal, the embedding operation is conducted to satisfy the chordal property. To be specific, the chordal embedding of G is represented as G and ( 7) is rewritten into the following formulation,:EQUATION)where Π G is the projection operator from S p onto S p E , i.e., by setting Π G (A ij ) = 0 if (i, j) / ∈ E.The learning steps of meta-knowledge extraction ( 5) is solved efficiently through a formulation of multiple independent sub-problems of linear programming, which can be accelerated in a parallel form. With the assistance of meta-knowledge, we solve the optimization problem (9) using Newton Conjugate Gradient method. The key idea is to construct an inner conjugate gradients loop as a solution to the Newton subproblem of an outer Newton's method. The detailed description, complexity analysis, and pseudo-code (Algorithm 1) of the full algorithm are shown in Appendix C.Question: For each theoretical result, does the paper provide the full set of assumptions and a complete (and correct) proof? Answer: [Yes] Justification: We provide the assumptions, theorems and the corresponding proofs in Appendix F.Guidelines:• The answer NA means that the paper does not include theoretical results.• All the theorems, formulas, and proofs in the paper should be numbered and crossreferenced. • All assumptions should be clearly stated or referenced in the statement of any theorems.• The proofs can either appear in the main paper or the supplemental material, but if they appear in the supplemental material, the authors are encouraged to provide a short proof sketch to provide intuition. • Inversely, any informal proof provided in the core of the paper should be complemented by formal proofs provided in appendix or supplemental material. • Theorems and Lemmas that the proof relies upon should be properly referenced. 
theoretical analysis: 
 To present the analysis more concisely, we first define Σ tot := diag(Σ (1) , . . . ,EQUATION), Ω tot := diag(Ω (1) , . . . , Ω (K) ) = Θ + Θ r , X tot := diag(X (1) , . . . , X (K) ). We assume that the precision matrix Ω belongs to the uniformity class of matrices,EQUATION)Here q, ν, ϕ are some constants, 0 ≤ q < 1 and Ω := (w ij ) Kp×Kp . s(Kp) represents the sparsity level of Ω in the uniformity class. Note that the sparsity level is related to p without an analytic form of relationships between them.Then some important conditions and definitions are stated as the following.Exponential Tail Condition Suppose there exists a constant 0 ≤ γ ≤ 1 4 , so that log p n k ≤ γ andEQUATIONwhere C is a constant. [4] There exists some α ∈ (0, 1] such thatEQUATIONwhere Γ * := Ω * -1 ⊗ Ω * -1 (⊗ represents the Kronecker matrix product) and G c = {1, . . . , p} 2 \G.We define κ Γ * = (Γ * GG ) -1 ∞ . Note that Γ * GG is a (s + p) × (s + p) matrix indexed by vertex pairs, where s = |G|. We define ω new min = min(i,j)∈supp(Ω * new )|Ω * new,ij | and ω tot min = min(i,j)∈supp(Ω * tot ) |Ω * tot,ij |.Definition 1. [32] Given a matrix M ∈ S p , define G M = {(i, j) : M ij ̸ = 0} as its sparsity pattern. Then M is called inverse-consistent if there exists a matrix N ∈ S p such thatEQUATIONThe matrix N is called an inverse-consistent complement of M and is denoted by M (c) . Furthermore, M is called sign-consistent if for every (i, j) ∈ G M , the (i, j)-th elements of M and (M + M (c) ) -1 have opposite signs.Then we define the β(G, α) function with respect to the sparsity pattern G and scalar α > 0β(G, α) = max M ≻0 ∥M (c) ∥ max s.t. M ∈ S n G and ∥M ∥ max ≤ α, M i,i = 1, M is inverse-consistent.Here i = 1, 2, . . . , n, ∥ • ∥ max denotes the ℓ max -norm, e.g., ∥A∥ max = max i̸ =j |A ij |.The Appendix F provides all the detailed proofs of the lemmas, theorems, and corollaries.In this section, we mainly study the theoretical properties of the meta-knowledge θ in (5) and the precision matrix Ωnew in (7) . The first theorem specifies a probability lower bound of recovering the true meta-knowledge by our estimator in (5) for multiple random multivariate sub-Gaussian distributions.Theorem 1. Suppose that Ω * tot ∈ U and N = K k=1 n k . When the variables are sub-Gaussian, the tail condition holds. Let λ = C 0 ν log(Kp), where the constantC 0 = 2γ -2 (2 + τ 0 + γ -1 e 2 C 2 ) 2 and constant τ 0 > 0. If ω tot min > 4τ n , then we have 1. ∥ θ -θ * ∥ ∞ ≤ 8C 0 ν 2 log(Kp) N + 2ϕ; 2. supp( θ) = supp(θ * );with a probability greater than 1-4p -τ0 . Here θ is a threshold estimator with θij = θijI(| θij | ≥ τ n ),where τ n ≥ 2νλ is a tuning parameter.According to Theorem 1, if n 1 = n 2 = • • • = n k , a sample complexity O( log K+log p K) per task is sufficient for the recovery of the meta-knowledge. In most cases, O(log K+log p K ) → O( log p K ) since p >> K.The following theorem specifies a probability lower bound of recovering a correct precision matrix Ωnew by our estimator in (7) for a multivariate sub-Gaussian distribution.Theorem 2. Suppose we have recovered the true sparsity pattern G of a family of p-dimensional random multivariate sub-Gaussian distribution. Then for a new task of multivariate sub-Gaussian distribution with the precision matrix Ω * new such that supp(θ) ⊆ supp(Ω * new ) and satisfying irrepresentable condition, consider the estimator Ωnew with λ = C 3 ν log |G| n . Then we haveEQUATIONwith a probability 1 -p -τ1 , where τ 1 > 0.This theorem shows that n ∈ O(log |G|) is sufficient for estimating a correct precision matrix of the new task with our estimation. Then, efforts should be made to prove that the max-det matrix completion method guarantees the consistency of the support set of Ωnew with G. Theorem 3. G coincides with the sparsity pattern of the optimal solution Ω * new if the normalized matrix Σnew = D -1/2 Π G (Σ new )D -1/2 where D = diag(Π G (Σ new )) satisfies the following conditions:1. Σnew is positive definite and sign-consistent, 2. We haveEQUATION)We set the diagonal elements of Σnew to 1 and its off-diagonal elements between -1 and 1. The projection operation Π G (•) leads to many zero elements in Σnew , resulting in Σnew being positive definite or even diagonally dominant in most cases. As shown by [32] , when G induces an acyclic structure, condition 2 is automatically satisfied by condition 1. More generally, [33] shows that Σnew is sign-consistent if ( Σnew + Σ(c) new ) -1 is close to its first order Taylor expansion. This assumption holds in practice due to the fact that the magnitude of the off-diagonal elements of Σnew + Σ(c) new is small. Moreover, [32] shows that the left side of ( 50) is upper bounded by a • ∥ Σnew ∥ 2 max for some a > 0. That implies that when ∥ Σnew ∥ max is small, or equivalently η is large, condition 3 is automatically satisfied. If the conditions in Theorem 3 are satisfied, the support set of Ωnew is consistent with G according to (7).Question: Does the paper fully disclose all the information needed to reproduce the main experimental results of the paper to the extent that it affects the main claims and/or conclusions of the paper (regardless of whether the code and data are provided or not)?Answer: [Yes] Justification: Yes, we provide all the experimental details, including data generation methods, hyperparameter selection, and the full optimization algorithm. 
related work: 
 There have been several methods proposed to estimate precision matrices, among which, three stateof-the-art methods, QUIC [10] , Neighborhood Selection [34] , Meta-IE [11] and gRankLasso [35] are chosen as baselines in the experiments (See reasons in Appendix E). These approaches all consider high-dimensional settings which are similar to our work. QUIC: This approach utilizes a quadratic approximation of the negative log-likelihood function to estimate sparse inverse covariance matrices to accelerate computation speed, which reduces the computational cost from O(p 2 ) to O(p) at each iteration. Neighborhood Selection: This method is a type of regularized maximum likelihood estimation method for estimating precision matrices. Given some i.i.d. observations, it estimates the conditional independence restrictions separately for each node in the graph. The complexity of the neighborhood selection for one node with the Lasso is O(np min{n, p}) using LARS algorithm [36] . Meta-IE (BigQuic): [11] first proposes a meta-learning-based method that is composed of two steps. First, Meta-IE takes the support union of all the precision matrices of the auxiliary tasks as prior knowledge. Then the method states that the testing step can be solved by an interior point method [37] in polynomial time O(p 3.5 log ϵ -1 ) or BigQuic [38] with a better time complexity around O(p 2 log ϵ -1 ) (See analysis in Appendix). gRankLasso: gRankLasso [35] is a method for sparse precision matrix estimation that realizes automatic parameter tuning, making it completely tuning-free. It achieves robustness and accuracy by integrating a group penalty with Lasso regularization, effectively estimating sparse structures in high-dimensional data.Question: Does the paper provide open access to the data and code, with sufficient instructions to faithfully reproduce the main experimental results, as described in supplemental material?Answer: [Yes]Justification: We have uploaded the relevant code on our GitHub (https://github.com/ MahjongGod-Saki/FasMe).Guidelines:• The answer NA means that paper does not include experiments requiring code. • Providing as much information as possible in supplemental material (appended to the paper) is recommended, but including URLs to data and code is permitted. 
synthetic experiment: 
 Baselines: We compare our method with the following baselines mentioned in Section 5: 1) The QUIC baseline, 2) the Neighborhood Selection (NS) baseline, 3) Meta-IE (BigQuic) baseline that is accelerated with 32 threads and 4) gRankLasso baseline. The time-complexity experiment on simulated datasets is conducted to observe the changes of all the estimators in computation time with the varying p and n. As shown in Fig. 3 and Table 1 &2, FasMe performs better than all the baselines. Using our method, we solve a sparse inverse covariance estimation problem containing 2500 variables, with time cost around 1 second. By comparing all the subfigures in Fig. 3 , we find that all the methods except the neighborhood selection approach are insensitive to the change of n. This is because the neighborhood selection deals with the sample matrix X new ∈ R n×p directly instead of the covariance matrix Σ new ∈ R p×p . However, it remains computationally costly compared to FasMe when the sample size is much smaller than the number of features. Interestingly, despite Meta-IE (BigQuic) method can speed up by parallelizing multiple threads, it is less time-efficient because of the time assumption of the I/O reads and writes. Moreover, BigQuic cannot converge within the maximum iterations (maxiter = 100), especially when only a few samples are provided (n = p/20, p/10). Regarding gRankLasso, its tuning mechanism requires a substantial amount of time, leading to significant time costs in highdimensional scenarios. In contrast, our method converges significantly faster within 20 iterations, regardless of small sample setting. (↑) F-norm(↓) Time(↓) MCC(↑) F-norm(↓) Time(↓) MCC(↑) F-norm(↓) Time(↓) MCC(↑) F-norm(↓) Time(↓) QUIC 0.We conduct several experiments on the simulated datasets to compare the prediction power of all methods with four different small sample settings n < p/10. Table 1&2 indicates that FasMe obtains lower Frobenius-norm and higher MCC under all the conditions. It suggests that FasMe outperforms the other baselines in high-dimensional settings. The comparison of the two tables shows that FasMe can effectively recover the special structure from the data. In addition, the MCC of Meta-IE becomes small as the feature dimension increases. This is because the predicted edges must be a subset of the support union. As p increases, the estimation of the support union is not relatively accurate, thus significantly reducing the prediction accuracy of Meta-IE. The poor performance of gRankLasso can be attributed to the inefficacy of its automatic tuning mechanism in small sample settings, which degrades the model performance. We draw one subnetwork for the experimental result of every method and compare them with the ground truth in Fig. 4 (a)(b). They show that our method bears the highest similarity with the ground truth. The results are consistent with the MCC value of every method shown in Table 1 &2.We further evaluate our method for estimating precision matrices on two real-world datasets: ChIP-Seq dataset (ENCODE project [39] ) and fMRI dataset (OpenfMRI project [40] , accession number: ds000002). Regarding the ChIP-Seq dataset, we aim to exploit the gene network of transcription factors (TFs). We randomly selected 300 gene features. 27 samples of H1-hESC (embryonic stem cells) and GM12878 (Blymphocyte) are chosen as two auxiliary tasks for meta-knowledge and 20 samples of K562 (leukemia) are chosen as a novel task. Regarding the fMRI dataset, we aim to exploit brain connectivity among different brain regions. We randomly selected 200 regions of interest (ROI) as features. 34 samples of PCL and DCL datasets are chosen as the auxiliary tasks for meta-knowledge and 20 samples of Mixed-Event dataset are chosen as a novel task. Additional experimental details including configuration choices and extended experiments in the economic domain are provided in Appendix B. In Subfigure (c)(d) of Fig. 4 , we visualize the experimental results on the two real-world datasets respectively. The predicted gene network exhibits scale-free and clustering behaviors, which is consistent with its biological properties. In another experiment, the majority of brain connections are found in the left ROIs. Besides, the left brain has positive connections, while the cross-hemisphere ones are negative. In this case, the subjects are asked to solve a classification problem, which mainly relies on their left brain's analysis function. The results align with our expectations.Question: Does the paper specify all the training and test details (e.g., data splits, hyperparameters, how they were chosen, type of optimizer, etc.) necessary to understand the results?Answer: [Yes]Justification: Yes, we provide comprehensive documentation on all training and testing details, including data generation methods, hyperparameter selection, reasons for configuration choices, and the complete optimization algorithm.Guidelines:• The answer NA means that the paper does not include experiments.• The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them. • The full details can be provided either with the code, in appendix, or as supplemental material. 
conclusion: 
 In this work, we introduced FasMe, a Fast and Sample-efficient Meta Estimator designed to address the challenges of precision matrix learning in small sample settings. Under a novel meta-learningbased framework, our approach first leverages a multi-task precision matrix estimator to extract shared meta-knowledge from auxiliary tasks, enabling efficient adaptation to new tasks with minimal data. FasMe then incorporates a maximum determinant matrix completion strategy to enhance precision matrix estimation, ensuring both computational efficiency and theoretical robustness. Our experiments on synthetic and real-world datasets demonstrate that FasMe significantly outperforms state-of-the-art baselines in terms of accuracy and speed. These results highlight the potential of FasMe to address real-world challenges in high-dimensional settings, such as genetics and neuroscience, where data is scarce. Moving forward, we plan to extend our work by relaxing the sub-Gaussian data assumption and further reducing the sample size requirements for training meta-knowledge. This will extend the framework to other domains and improve scalability for even larger datasets.Question: Does the paper report error bars suitably and correctly defined or other appropriate information about the statistical significance of the experiments?Answer: [Yes]Justification: We provide the error bound of our estimators in Theorem 2 and 3 of the main page.Guidelines:• The answer NA means that the paper does not include experiments.• The authors should answer "Yes" if the results are accompanied by error bars, confidence intervals, or statistical significance tests, at least for the experiments that support the main claims of the paper.• The factors of variability that the error bars are capturing should be clearly stated (for example, train/test split, initialization, random drawing of some parameter, or overall run with given experimental conditions). • The method for calculating the error bars should be explained (closed form formula, call to a library function, bootstrap, etc.) • The assumptions made should be given (e.g., Normally distributed errors). • It should be clear whether the error bar is the standard deviation or the standard error of the mean. • It is OK to report 1-sigma error bars, but one should state it. The authors should preferably report a 2-sigma error bar than state that they have a 96% CI, if the hypothesis of Normality of errors is not verified. • For asymmetric distributions, the authors should be careful not to show in tables or figures symmetric error bars that would yield results that are out of range (e.g. negative error rates). • If error bars are reported in tables or plots, The authors should explain in the text how they were calculated and reference the corresponding figures or tables in the text. 
experiments compute resources: 
 Question: For each experiment, does the paper provide sufficient information on the computer resources (type of compute workers, memory, time of execution) needed to reproduce the experiments?Answer: [Yes]Justification: We analyze computational complexity in Appendix C and compare the execution time of our method with other baselines in Section 6.Guidelines:• The answer NA means that the paper does not include experiments.• The paper should indicate the type of compute workers CPU or GPU, internal cluster, or cloud provider, including relevant memory and storage. • The paper should provide the amount of compute required for each of the individual experimental runs as well as estimate the total compute. • The paper should disclose whether the full research project required more compute than the experiments reported in the paper (e.g., preliminary or failed experiments that didn't make it into the paper). 
code of ethics: 
 Question: Does the research conducted in the paper conform, in every respect, with the NeurIPS Code of Ethics https://neurips.cc/public/EthicsGuidelines? 
broader impacts: 
 Question: Does the paper discuss both potential positive societal impacts and negative societal impacts of the work performed?Answer: [Yes] Justification: We discuss them in Appendix A.Guidelines:• The answer NA means that there is no societal impact of the work performed.• If the authors answer NA or No, they should explain why their work has no societal impact or why the paper does not address societal impact. • Examples of negative societal impacts include potential malicious or unintended uses (e.g., disinformation, generating fake profiles, surveillance), fairness considerations (e.g., deployment of technologies that could make decisions that unfairly impact specific groups), privacy considerations, and security considerations. • The conference expects that many papers will be foundational research and not tied to particular applications, let alone deployments. However, if there is a direct path to any negative applications, the authors should point it out. For example, it is legitimate to point out that an improvement in the quality of generative models could be used to generate deepfakes for disinformation. On the other hand, it is not needed to point out that a generic algorithm for optimizing neural networks could enable people to train models that generate Deepfakes faster. • The authors should consider possible harms that could arise when the technology is being used as intended and functioning correctly, harms that could arise when the technology is being used as intended but gives incorrect results, and harms following from (intentional or unintentional) misuse of the technology. • If there are negative societal impacts, the authors could also discuss possible mitigation strategies (e.g., gated release of models, providing defenses in addition to attacks, mechanisms for monitoring misuse, mechanisms to monitor how a system learns from feedback over time, improving the efficiency and accessibility of ML). 
safeguards: 
 Question: Does the paper describe safeguards that have been put in place for responsible release of data or models that have a high risk for misuse (e.g., pretrained language models, image generators, or scraped datasets)?Answer: [NA]Justification: We have no such risk.Guidelines:• The answer NA means that the paper poses no such risks.• Released models that have a high risk for misuse or dual-use should be released with necessary safeguards to allow for controlled use of the model, for example by requiring that users adhere to usage guidelines or restrictions to access the model or implementing safety filters. • Datasets that have been scraped from the Internet could pose safety risks. The authors should describe how they avoided releasing unsafe images. • We recognize that providing effective safeguards is challenging, and many papers do not require this, but we encourage authors to take this into account and make a best faith effort. 
licenses for existing assets: 
 Question: Are the creators or original owners of assets (e.g., code, data, models), used in the paper, properly credited and are the license and terms of use explicitly mentioned and properly respected?Answer: [Yes] Justification: We cite the corresponding source for all the real-world dataset we use.Guidelines:• The answer NA means that the paper does not use existing assets.• The authors should cite the original paper that produced the code package or dataset.• The authors should state which version of the asset is used and, if possible, include a URL. • The name of the license (e.g., CC-BY 4.0) should be included for each asset.• For scraped data from a particular source (e.g., website), the copyright and terms of service of that source should be provided.• If assets are released, the license, copyright information, and terms of use in the package should be provided. For popular datasets, paperswithcode.com/datasets has curated licenses for some datasets. Their licensing guide can help determine the license of a dataset. • For existing datasets that are re-packaged, both the original license and the license of the derived asset (if it has changed) should be provided. • If this information is not available online, the authors are encouraged to reach out to the asset's creators. 
new assets: 
 Question: Are new assets introduced in the paper well documented and is the documentation provided alongside the assets?Answer: [Yes]Justification: We have submitted our code in our GitHub.Guidelines:• The answer NA means that the paper does not release new assets.• Researchers should communicate the details of the dataset/code/model as part of their submissions via structured templates. This includes details about training, license, limitations, etc. • The paper should discuss whether and how consent was obtained from people whose asset is used. • At submission time, remember to anonymize your assets (if applicable). You can either create an anonymized URL or include an anonymized zip file. 
crowdsourcing and research with human subjects: 
 Question: For crowdsourcing experiments and research with human subjects, does the paper include the full text of instructions given to participants and screenshots, if applicable, as well as details about compensation (if any)?Answer: [NA]Justification: We have no such risk.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Including this information in the supplemental material is fine, but if the main contribution of the paper involves human subjects, then as much detail as possible should be included in the main paper. • According to the NeurIPS Code of Ethics, workers involved in data collection, curation, or other labor should be paid at least the minimum wage in the country of the data collector. 
institutional review board (irb) approvals or equivalent for research with human subjects: 
 Question: Does the paper describe potential risks incurred by study participants, whether such risks were disclosed to the subjects, and whether Institutional Review Board (IRB) approvals (or an equivalent approval/review based on the requirements of your country or institution) were obtained?Answer: [NA]Justification: We have no such risk.Guidelines:• The answer NA means that the paper does not involve crowdsourcing nor research with human subjects. • Depending on the country in which research is conducted, IRB approval (or equivalent) may be required for any human subjects research. If you obtained IRB approval, you should clearly state this in the paper.• We recognize that the procedures for this may vary significantly between institutions and locations, and we expect authors to adhere to the NeurIPS Code of Ethics and the guidelines for their institution. • For initial submissions, do not include any information that would break anonymity (if applicable), such as the institution conducting the review.Note that the choice to utilize H1-hESC and GM12878 as auxiliary tasks, with K562 as the primary prediction task, was made with deliberate consideration of their biological significance. H1-hESC represents primary tissue, GM12878 represents normal tissue, and K562 represents cancer tissue.Given the heightened research focus on cancer cell gene networks, which are more likely to be the subject of predictive modeling and have rare cases, we opted for this configuration to use K562 as the test set. fMRI dataset OpenfMRI is a task-based fMRI database, we chose a dataset about the task Classification learning (accession number: ds000002). This database has been widely used in other research papers in this domain [48, 49, 50, 51] . This dataset contains three different tasks, participants were instructed to perform probabilistic classification learning (PCL), deterministic classification learning (DCL), and mixed-up classification learning on Weather Prediction Task(WPT). WPT is commonly used to assess striatal or procedural learning capacities in different populations. This fMRI dataset effectively records neural responses to stimuli, delays, and negative and positive feedback components, and studying this dataset aims to address a fundamental question of whether and how the brain's memory systems interact.In our experiment, the 4D fMRI brain image is divided into spatial regions of size 2 × 2 × 2, so the original scan data in spatial shape 64 × 64 × 30 is divided into 15360 regions. And the activation of each region during the task (the temporal dimension) is added up to represent the average activation situation of corresponding regions. We randomly selected 200 active regions to lower the computational burden. By training on the tasks of PCL and DCL, the common key brain regions and the interaction between the tasks are obtained, that is, a common substructure.Note that we employed PCL (Probabilistic Classification Learning) and DCL (Deterministic Classification Learning) as auxiliary datasets due to their representation of distinct learning paradigms (See Appendix A.2). PCL captures learning under uncertainty, while DCL reflects learning from clear-cut rules. This diversity aids in generalizing the model to novel scenarios. The Mixed-Event dataset, embodying elements from both PCL and DCL, mirrors the intricate and multifaceted nature of human cognition. Consequently, the Mixed-Event dataset emerged as the optimal choice for the test set, offering a realistic and challenging environment for the model to demonstrate its predictive prowess in brain connectivity, reflective of the complexity encountered in real-world cognitive processes.By using the prior knowledge of common substructure, we effectively introduce the key areas of the brain on similar tasks and the interaction between these key areas, so that we can learn how the brain regions interact in other tasks efficiently.In Table 3 , we record the negative log-determinant Bregman divergence of our estimator for the new task and compare it with the other baselines. The experimental results show that our method generalizes better than all other baselines for the real-world dataset since it achieves the minimum log-determinant Bregman divergence.US Industry Portfolios dataset This is a well-known dataset of US industry portfolios' monthly returns from 1926 to 2023. It covers various economic sectors such as agriculture, healthcare, energy, machinery, and electronics. We take each industry as a feature (p = 49) and use the records of different financial metrics as the tasks. We use AVWR (Annual Voluntary Wage Reporting) dataset  
 
 Appendix: • The answer NA means that the paper does not include experiments.• If the paper includes experiments, a No answer to this question will not be perceived well by the reviewers: Making the paper reproducible is important, regardless of whether the code and data are provided or not. • If the contribution is a dataset and/or model, the authors should describe the steps taken to make their results reproducible or verifiable. • Depending on the contribution, reproducibility can be accomplished in various ways.For example, if the contribution is a novel architecture, describing the architecture fully might suffice, or if the contribution is a specific model and empirical evaluation, it may be necessary to either make it possible for others to replicate the model with the same dataset, or provide access to the model. In general. releasing code and data is often one good way to accomplish this, but reproducibility can also be provided via detailed instructions for how to replicate the results, access to a hosted model (e.g., in the case of a large language model), releasing of a model checkpoint, or other means that are appropriate to the research performed. • While NeurIPS does not require releasing code, the conference does require all submissions to provide some reasonable avenue for reproducibility, which may depend on the nature of the contribution. , with an open-source dataset or instructions for how to construct the dataset). (d) We recognize that reproducibility may be tricky in some cases, in which case authors are welcome to describe the particular way they provide for reproducibility.In the case of closed-source models, it may be that access to the model is limited in some way (e.g., to registered users), but it should be possible for other researchers to have some path to reproducing or verifying the results.Justification: We have no such risk.Guidelines:• The answer NA means that the authors have not reviewed the NeurIPS Code of Ethics.• If the authors answer No, they should explain the special circumstances that require a deviation from the Code of Ethics. • The authors should make sure to preserve anonymity (e.g., if there is a special consideration due to laws or regulations in their jurisdiction).BigQuic [38] , an "big-data" extension of QUIC, aims to estimate the precision matrix with thousands of variables. It solves the optimization problem using a block-coordinate descent Newton method. BigQuic divides the Newton direction into several blocks and utilizes the memory cache to accelerate the updated process. The computation complexity of BigQuic is mainly determined by O ((p + |B|)hT T outer ), where |B| is the number of boundary nodes, h is the number of non-zero entries in the t-th generation estimated solution Ω(t) , T is the average number of Conjugate Gradient iterations and T outer is the number of computations within a block. Without loss of generality, h is larger than p in t-th iteration. Hence, the time complexity of BigQuic is at least higher than O(p 2 ) time complexity. Furthermore, the performance of BigQuic depends largely on the choice of cluster scheme. A poor partition of variables leads to many "cache misses". The challenges of choosing a cluster schema make BigQuic harder to implement.Based on the meta-learning framework, FasMe and Meta-IE differ in each component of this framework, namely meta-teacher, meta-knowledge, and meta-student.Meta-IE: Meta-IE supposes the support union Ω as the meta-knowledge and assumes that the edges of the new task must be a subset of the edges of Ω.FasMe: Different from Meta-IE, we assume that the new task shares a common structure, i.e., meta-knowledge θ, with the related auxiliary tasks.As we stated in Section Introduction, it is impossible to make startling discoveries under this unrealistic assumption of Meta-IE. Our choice of meta-knowledge is more reasonable with biological research and existing works as evidence.Meta-IE: Meta-IE simply pools all the samples from K auxiliary tasks to learn the meta-knowledge:Ω = arg min Ω⪰0 -log det Ω + K k=1 1 K Σ (k) , Ω + λ∥Ω∥ 1 ,which is data-inefficient.FasMe: By taking the inherent heterogeneity among the auxiliary tasks into consideration, FasMe proposes a multitask-learning-based estimator to extract the common substructure θ across the tasks.Meta-IE: Meta-IE uses BigQUIC method to solve a GLasso problem with the knowledge constraint supp(Ω) ⊆ supp( Ω).FasMe: FasMe aims to estimate the precision matrix by solving the dual form of a MDMC problem with Newton Conjugate Gradient algorithm. We also would like to emphasize that the meta-student is not a simple application of standard MDMC. The formulation of a standard MDMC problem is:max X log det X s.t. X ij = M ij , ∀i, j ∈ E.We improve the method from two aspects: 1) able to deal with the partially observed matrices without chordality by utilizing the chordal embedding 2) lower the time complexity by applying Newton CG algorithm In summary, Meta-IE has presented a preliminary idea for applying metalearning to precision matrix estimation with much potential for improvement. Compared to Meta-IE, FasMe exhibits better theoretical properties and superior performance. K We actually have conducted a synthetic experiment with varying K and a fixed data size (n = 100, p = 1000). We use MCC value as a metric to measure the performance of estimating meta-knowledge. A higher score represents better performance. By varying K = 2, 3, . . . , 6, Fig. 7 shows that the MCC value smoothly experiences a slight decline from 0.85 to 0.60. Figure 7 : MCC values of our methods against varying K ρ This hyperparameter delineates the sparsity contrast between the shared and individual components of the model. As outlined in line 200, a lower ρ (chosen as 0.8 in our experiments) enriches the shared structure's density while sparing the individual parts. Our preliminary investigations, spanning ρ values from 0.1 to 0.9, indicated a stable predictive accuracy across the spectrum. We plan to enrich the appendix with these findings to demonstrate ρ's effect on model performance comprehensively.λ This hyperparameter dictates the overall sparsity of both the shared and individual graph structures, with a higher value fostering sparser outcomes. Inspired by some previous researches [1, 2] , we adopt λ = ω log(Kp) ntot , where ω varies across 0.05 × i|i = 1, 2, . . . , 30. The Bayesian Information Criterion (BIC) assists in pinpointing the optimal λ for our model and is uniformly applied across baseline comparisons.η Regarding η, functioning as the soft-thresholding parameter, η's selection leans on an empirical estimation of the graph sparsity by the user. It aims to adjust the number of nonzero entries in the model's sparsity pattern, correlating closely with those in the estimated precision matrix. A higher value of η corresponds to a more sparsely estimated graph structure. Table 7 &  Calculate the covariance matricesΣ (k) = 1 n k n k i=1 (X -X)(X -X) ⊤ 4: Initialize θ = θ (k) r = 0 p×p 5:Initialize B (k) = 0, . . . , Σ (k) , . . . , 0, 1 ϵK Σ (k) 6: end for 7: for i = 1 to p do Similar to CLIME, (5) can also be solved column by column:EQUATIONwhere k = 1, 2, . . . , K. Here β represents the corresponding column vector in the meta-knowledge θ and β(k)r represents the k-th column vector in the rest θ (k) r . To solve (16) , we can rewrite it as the followingEQUATIONHere, B (k) = 0, . . . , Σ (k) , . . . , 0, 1 ϵK Σ (k) , β = (β (1) ) ⊤ , . . . , (β (k) ) ⊤ , . . . , (β (K) ) ⊤ , ϵK(β W ) ⊤ ⊤ and b = e j .Through relaxation, we can convert (17) to the following linear programming formulation: âj = arg min aj (K+1)p j=1 a j , s.t. -θ j ≤ a j , j = 1, . . . , (K + 1)p, θ j ≤ a j , j = 1, . . . , (K + 1)p,EQUATIONHere a j are the slack variables. B (k) i represents the i-th row of B (k) and b i is the i-th entry of b.According to [25] , we can apply the same symmetric operators onΩ (k) = θ + θ (k) r obtained from Algorithm 1.Then we combined the support set of the thresholded covariance and meta-knowledge to obtain the sparsity pattern:EQUATIONTo solve the optimization problem ( 9), we first need to obtain the chordal embedding G by Cholesky factorization. We compute the unique lower triangular Cholesky factor L satisfying Π G (Σ new ) = LL ⊤ . After ignoring perfect numerical cancellation, we have the adjacency matrix of the chordal embedding L + L ⊤ using symbolic Cholesky algorithm.We define the cone of sparse positive semidefinite matrices P, and the cone of sparse matrices with positive semidefinite completions P * as the following:P = S p + ∩ S p G , P * = {⟨S, Ω⟩ ≥ 0 : S ∈ S p G }.The primal and dual problem of Eq. ( 9) can be written as:EQUATIONarg max S∈P,y∈R m -g * (S) s.t. S = Σ -P (y),where P (•) : R m → S p G\G represents a linear map that converts a list of m variables into the corresponding matrix in G\G. g and g * are the "log-det" barrier functions on P and P * :g(Ω) = -log det Ω, g * (S) = -min Ω∈P ⟨S, Ω⟩ -log det Ω.Under the sparsity and chordal assumption, the gradient evaluations and Hessian matrix-vector products can be efficiently evaluated in O(p) time and O(p) memory, using the numerical recipes described in [52] .To solve the dual problem (21) , we rewrite it into the following unconstrained optimization problem:EQUATIONAfter obtaining the solution ŷ, we can recover the optimal estimator for the primal problem through Ω = -∇g * (Π G (Σ new ) -P (y)). The dual problem is easier to solve than the primal because the initial point y = 0 tends to lie close to the solution ŷ. Starting from this point, we can use Newton's method to converge rapidly.We then solve the Newton direction ∆y via the m × m system of equations ∇ 2 l(y)∆y = -∇l(y).Starting from the origin y 1 = 0, the method converges to an ϵ-accurate search direction y 1 satisfyingEQUATIONin at most ∥∇ 2 l(y)∥∥∇ 2 l(y) -1 ∥ log( 2 ϵ ) conjugate gradient iterations.In this section, we mainly analyze the time and memory complexity of our estimator in the meta-testing step. The time and memory cost of our estimator can be divided into two parts. The first part is to threshold the covariance matrix. This step is quadratic O(p 2 ) time and memory but embarrassingly parallelizable. The second part is to solve (7) using Newton Conjugate Gradient methods. If the prior sparsity pattern G is sparse and chordal, then the second part can be performed using our algorithm in linear O(p) time and memory. This significantly outperforms QUIC [10] , Neighborhood Selection method [34] and Bigquic used in [11] .As discussed earlier, the biological and genetic domains have adopted and validated the common structure assumption based on some biological evidence. For other domains, if the edges are sparse and the data are high-dimensional, it may be difficult to recover the common structure from the data without some prior evidence. In the worst case, the meta-teacher can learn the knowledge that there is no common edge among the tasks. The meta-student has to learn from scratch. The optimization problem in the meta-testing process can be seen as a graphical lasso problem (introduced in Section 2.2). The neighborhood selection baseline is one type of algorithm that solves the graphical lasso. Consequently, our proposed method remains at least as good as the baseline, if not stronger, in handling such challenges.We chose QUIC and Neighborhood Selection since they continue to be widely utilized in current research and serve as critical benchmarks for new methodologies within the realm of state-of-the-art works.We take QUIC as an example. [53] utilizes the QUIC method to detect structural changes in highdimensional Gaussian graphical models. QUIC's ability to perform fast and accurate estimation underpins the methodology for identifying change-points in the graphical model structure over time.Similarly, studies like those in recent works like [54, 55] also employ QUIC for various analytical tasks downstream.Additionally, while both [56] and [57] utilize QUIC as a baseline for comparison, [56] relies on a structural assumption of Kronecker-sum-structured inverse covariance, and [57] operates under a general missing dependency assumption. These specific premises limit their applicability, rendering them less generalizable to our scenario.Parallel to QUIC's applicability, Neighborhood Selection is similarly employed in contemporary studies, demonstrating its ongoing relevance and utility [58, 59, 60, 61] .As mentioned above, we observed two main categories. The first category of recent works did not propose improvements to QUIC; instead, they applied it to more specific tasks. The second category, while outperforming QUIC as a baseline and demonstrating improvements, often relied on stricter structural assumptions not compatible with our problem context. Consequently, both categories were not suitable as baselines for our study.We specifically chose Meta-IE method due to its closeness to our work. Meta-IE represents the latest advancement in meta-learning applied to sparse precision matrix estimation, making it an ideal candidate for direct comparison.To this end, we have decided to incorporate an evaluation against gRankLasso [35] , a recent development in graph learning, into our revised manuscript. With a completely tuning-free technique, gRankLasso shows better accuracy performance than CLIME, GLasso, and TIGER. However, its extensive time consumption for hyperparameter simulations and rank calculations poses a drawback, particularly underperforming with small datasets.EQUATIONn k ∈ R p be i.i.d. random vectors for 1 ≤ k ≤ K. Let X ij be the j-th entry of X (k) i for 1 ≤ j ≤ p. We use (Σ (k) ) * , (Ω (k) ) * , θ * to represent the real covariance matrix, precision matrix, and meta-knowledge, respectively. {X (k) } follows a family of random p-dimensional multivariate sub-Gaussian distributions with parameter σ if (1) X(k) i 1≤i≤n k ,1≤k≤Kare conditionally independent given {(Ω (k) ) * } K k=1 ;(2)X (k) ij σ (k) jj conditioned on (Ω (k) ) * is sub-Gaussian with parameter σ for 1 ≤ i ≤ n k , 1 ≤ j ≤ p, 1 ≤ k ≤ K; (3) E[X (k) i |(Σ (k) ) * ] = 0, Cov X (k) i |(Σ (k) ) * = Σ (k) * = (Ω (k) ) * -1 for 1 ≤ i ≤ n k , 1 ≤ k ≤ K.Then we remove the assumption on the support set of θ where ρ > 0 and C is a constant. Then we have x • ŷ ≥ 0.Proof. If x • ŷ < 0, let x ′ = x + ŷ and y ′ = 0. Without loss of generality, we assume that |x| ≥ |ŷ|.EQUATIONwhich means that the pair x ′ , y ′ is a better solution. This contradicts the earlier stated premise that x, ŷ are the optimal solution. The proof is completed by contradiction. + θ satisfies (10) , then Ω tot also satisfies (10) . Thus Ωtot satisfies the condition Ωtot ≻ 0, with a high probability.Corollary 1. We assume that θ is the optimal solution of Eq. (5) and θ(k)EQUATIONTheorem 1. We use Θ, Θ * to represent the estimated precision matrix and real precision matrix, respectively. Suppose thatEQUATIONwhere ν = ∥Ω * tot ∥ 1 and ϕ = ∥Ω * tot ∥ ∞ .Proof. According to the condition in Theorem 1,EQUATIONThen we can show thatEQUATIONwhere |AB| ∞ ≤ |A| ∞ |B| 1 for any matrices A, B of appropriate sizes.Then we can rewrite (5) as:EQUATIONThis is because∥Ω tot ∥ 1 + (1 -ρ) K k=1 ∥θ (k) r ∥ 1 = K k=1 ∥θ + θ (k) r ∥ 1 + (1 -ρ) K k=1 ∥θ (k) r ∥ 1 .Notice that the constraint is not related to Θ r . Therefore Ωtot is also the solution of the following optimization problem:EQUATIONSince Ω * tot and Ωtot satisfy (32), we haveEQUATIONThen, it follows thatEQUATIONwhere ∥Ω * tot ∥ 1 = ν. Based on the triangle inequality for norm, we have thatEQUATIONIn terms of Θ * , Θ * r , we assume W * tot,ij • Ω * I,ij = 0 and W * tot,ij + Ω * I,ij = 0 iff W * tot,ij = Ω * I,ij = 0. Thus we can derive that ∥Θ * ∥ ∞ ≤ ∥Ω * tot ∥ ∞ . We are now turning to the proof of Eq. (37) . Combining Corollary 1 and Theorem 1, we can show thatEQUATIONwhere ∥Ω * tot ∥ ∞ = ϕ. The proof is completed. Proof. To obtain the first conclusion of Theorem 2, we need to proveEQUATION)with a probability greater than 1 -4p -τ0 under the exponential tail condition. Without loss of generality, we assume that E(X (k) ) = 0. Let Σ ′ tot := N -1 N k=1 X tot,k X ⊤ tot,k and Y kij = X tot,ki X tot,kj -E(X tot,ki X tot,kj ). We then have Σtot = Σ ′ tot -Xtot X⊤ tot . Let t = γ log Kp/N . Using the inequality |e s -1 -s| < s 2 e max(s,0) for any s ∈ R and let C 1 = 2 + τ 0 + γ -1 C 2 . Then by some basic calculations, we can get Then we obtainEQUATIONUsing the inequality e s ≤ e s 2 +1 for s > 0, we haveEQUATIONEQUATIONand a n = C 2 2 (log Kp/N ) 1 2 . As before, we can have thatP ∥ Xtot X⊤ tot ∥ ∞ ≥ γ -2 a n log Kp/N ≤p max i P N k=1 X ki ≥ γ -1 C 2 N log Kp + p max i P - N k=1X ki ≥ γ -1 C 2 N log Kp ≤2p -τ0-1 (44) By ( 41), (44) , and the inequality C > γ -1 C 1 + γ -2 a n , we see that (39) holds. The proof of the first inequality is completed.To prove the sign-consistency of Θ, we define a threshold estimator Ωtot with ωij = ωij I{|ω ij | ≥ τ n } where τ n ≥ 4νλ is a tuning parameter. We define ω tot min = min (i,j)∈supp(Ω * tot )|Ω * tot,ij | and a threshold estimator Θ with Θij = Θij I{| Θij | ≥ τ n }, where τ ≥ 4νλ. It is known that the resulting elements in Ωtot will exceed the threshold level if the corresponding element in Ω * tot is large in magnitude. In contrast, the entries of Ωtot outside the support of Ω * tot will remain below the threshold level with high probability. As a result, if ω tot min > 2τ n , we have supp( Ωtot ) = supp(Ω * tot ). Since the nonzero entries in θ correspond to the intersection of the nonzero entries in {Ω (k) }, it is straightforward to get that supp( Θ) = supp(Θ * ) if ω tot min > 2τ n . Lemma 3. Consider a zero-mean random vector (X 1 , . . . , X p ) with covariance Σ * such that each EQUATIONwith a probability 1 -p -τ1 , where τ 1 > 0.The proof is similar to Theorem 2 and will be omitted. Definition 3.[32] Given a matrix M ∈ S p , define G M = {(i, j) : M ij ̸ = 0 as its sparsity pattern. Then M is called inverse-consistent if there exists a matrix N ∈ S p such thatEQUATIONin which γ ∈ (0, 0.5) and ρ ∈ (0, 1). The function g * is strongly self-concordant, and l inherits this property from g * . Accordingly, classical analysis shows that we require at most l (y 1 ) -l(ŷ) 0.05γρ + log log(1/ϵ) ≈ O(1) Newton steps to an ϵ-optimal point satisfying l (y k ) -l(ŷ) ≤ ϵ. The bound is very pessimistic, and in practice, no more than 20-30 Newton steps are ever needed for convergence.The most computationally expensive of Newton's method is the solution of the Newton direction ∆y via the m × m system of equationsEQUATIONThe main result in this section is a proof that the condition number of ∇ 2 l(y) is independent of the problem dimension n.Theorem 5. At any y satisfying l(y) ≤ l (y 1 ) and ∇l(y) ⊤ (y -y 1 ) ≤ ϕ max , the condition number κ l of the Hessian matrix ∇ 2 l(y) is boundEQUATIONwhere:• ϕ max = l (y 1 ) -l(ŷ) is the suboptimality of the initial point,• A = [vec A 1 , . . . , vec A m ] is the vectorized version of the problem data,• Ω 0 = -∇f * (S 0 ) and S 0 = Σ -A (y 0 ) are the initial primal-dual pair,• Ω = -∇f * ( Ŝ) and Ŝ = Σ -A(ŷ) are the solution primal-dual pair.As a consequence, each Newton direction can be computed in O log ϵ -1 iterations using conjugate gradients, over O log log ϵ -1 total Newton steps. The overall minimization problem is solved to ϵ-accuracy in O log ϵ -1 log log ϵ -1 ≈ O(1)CG iterations.The leading constant here is dependent polynomially on the problem data and the quality of the initial point, but independent of the problem dimensions.Proof To prove the first bound (61), we will instead proveEQUATIONwhich yields the desired condition number bound on ∇ 2 g(y). Writing λ 1 = λ max (Ω) and λ n = λ min (Ω), we have from the two lemmas above:ϕ max ≥ λ min ( Ω) λ -1 n -λ -1 1 2 > 0, 2ϕ max ≥ λ min Ω -1 0 λ 1 -λ n 2 > 0.Multiplying the two upper-bounds and substituing λ min Ω -1 0 = 1/λ max (Ω 0 ) yieldsEQUATIONFinally, bounding λ n /λ 1 ≥ 0 yields (62). 


Decision: This paper introduces a meta learning technique for precision matrix estimation, e.g. estimating the precision matrix of a dataset X' given a set of training datasets X1,...,Xk. Overall the reviewers appreciated the theoretical results for sub-Gaussian RVs and strong empirical efficiency improvements. There were no major concerns with the paper; I might simply recommend that the authors' make the results currently in Appendix B.2 more prominently visible.

Reviews: review: paper_summary: paper_summary: The authors propose a method to estimate sparse precision matrices from few samples. Theoretical properties of the proposed method are studied, and experiments on synthetic and brain fMRI data are presented.
strength_weakness: strength_weakness: Weaknesses:
* Currently quantitative results are presented for synthetic data. It would be nice to see more quantitative evaluations on benchmark datasets.
questions: questions: * How are the meta learning tasks linked to learning the new precision matrix in the theoretical part? It would be nice if the authors can elaborate on the assumptions for the learnability of the new matrix.
2. Why is N=0 in Eq. 13?
limitations: limitations: N/A
paper_summary: paper_summary: This paper introduces FasMe, a meta-learning approach for efficient precision matrix estimation in small sample settings. By leveraging meta-knowledge and maximum determinant matrix completion, FasMe reduces sample size requirements and improves computational efficiency. Experimental results show FasMe to be significantly faster and more accurate than existing methods, particularly in low-data environments.
strength_weakness: strength_weakness: I have few doubts about the method and experiments presented in the article.
questions: questions: Is the work primarily applicable to biological scenarios, or can it be widely used in other contexts as well?
limitations: limitations: nan
paper_summary: paper_summary: This paper proposes a meta-learning method for estimating the precision matrix on a new task with small data.
The proposed method uses common edges estimated from multiple auxiliary datasets as meta-knowledge. Then, it estimates the precision matrix on the new task, assuming its true edges contain all the estimated common edges (meta-knowledge). Some theoretical guarantees are also provided.
Experiments with synthetic and real-world datasets show the effectiveness and efficiency of the proposed method.
strength_weakness: strength_weakness: - As the authors stated in the Limitation section, the assumption of Eq. 8 might not be held in general machine learning tasks, although it fits well in biological domains.
questions: questions: Please see the Weaknesses.
limitations: limitations: Yes

Comments: comment: Thanks to the authors for the rebuttal, I am keeping the initial ratings.
Thank you for your response. I'll keep the current rating.