SlowGuess/NeurIPS_2023_OCR huggingface dataset:
Christopher Musco
New York University cmusco@nyu.edu
Jerry Li
Microsoft Research  
jerrl@microsoft.com
Kirankumar Shiragur
Broad Institute of MIT and Harvard† shiragur@stanford.edu
Stanford University sidford@stanford.edu
kjtian@cs.utexas.edu
We develop a general framework for finding approximately-optimal preconditioners for solving linear systems. Leveraging this framework we obtain improved runtimes for fundamental preconditioning and linear system solving problems including:
- Diagonal preconditioning. We give an algorithm which, given positive definite  $\mathbf{K} \in \mathbb{R}^{d \times d}$  with  $\mathrm{nnz}(\mathbf{K})$  nonzero entries, computes an  $\epsilon$ -optimal diagonal preconditioner in time  $\widetilde{O}(\mathrm{nnz}(\mathbf{K}) \cdot \mathrm{poly}(\kappa^{\star}, \epsilon^{-1}))$ , where  $\kappa^{\star}$  is the optimal condition number of the rescaled matrix.
- Structured linear systems. We give an algorithm which, given  $\mathbf{M} \in \mathbb{R}^{d \times d}$  that is either the pseudoinverse of a graph Laplacian matrix or a constant spectral approximation of one, solves linear systems in  $\mathbf{M}$  in  $\widetilde{O}(d^2)$  time.
Our diagonal preconditioning results improve state-of-the-art runtimes of  $\Omega(d^{3.5})$  attained by general-purpose semidefinite programming, and our solvers improve state-of-the-art runtimes of  $\Omega(d^{\omega})$  where  $\omega > 2.3$  is the current matrix multiplication constant. We attain our results via new algorithms for a class of semidefinite programs (SDPs) we call matrix-dictionary approximation SDPs, which we leverage to solve an associated problem we call matrix-dictionary recovery.
Preconditioning is a fundamental primitive in the theory and practice of numerical linear algebra, optimization, and data science. Broadly, its goal is to improve conditioning properties (e.g., the range of eigenvalues) of a matrix  $\mathbf{M}$  by finding another matrix  $\mathbf{N}$  which approximates the inverse of  $\mathbf{M}$  and is more efficient to construct and apply than computing  $\mathbf{M}^{-1}$ . This strategy underpins a variety of popular recently-developed tools, such as adaptive gradient methods for machine learning (e.g., Adagrad and Adam [DHS11, KB15]), and near-linear time solvers for combinatorially-structured
*Work completed at Stanford and the University of Washington.
†Work completed at Stanford.
$^{\ddagger}$ Work completed at Stanford and Microsoft Research.
matrices (e.g., graph Laplacians [ST04]). Despite widespread practical adoption of such techniques, there is a surprising lack of provably efficient algorithms for preconditioning.
Our work introduces a new tool, matrix-dictionary recovery, and leverages it to obtain the first near-linear time algorithms for several structured preconditioning problems in well-studied applications. Informally, the problem we study is as follows (see Section 4 for the formal definition).
Given a matrix  $\mathbf{M}$  and a "matrix-dictionary"  $\{\mathbf{M}_i\}$ , find the best preconditioner
$$
\mathbf {N} = \sum_ {i} w _ {i} \mathbf {M} _ {i} \text {o f} \mathbf {M} \text {e x p r e s s i b l e a s a n o n n e g a t i v e l i n e a r c o m b i n a t i o n o f} \left\{\mathbf {M} _ {i} \right\}. \tag {1}
$$
We develop general-purpose solvers for the problem (1). We further apply these solvers to obtain state-of-the-art algorithms for fundamental tasks such as preconditioning linear systems and regression, and approximately recovering structured matrices, including the following results.
- Diagonal preconditioning. We consider the classical numerical linear algebra problem of diagonal preconditioning [vdS69]. Given  $\mathbf{K} \in \mathbb{S}_{\succ 0}^{d}$ , the goal is to find a diagonal  $\mathbf{W} \in \mathbb{S}_{\succ 0}^{d}$  minimizing the condition number of  $\mathbf{W}^{\frac{1}{2}} \mathbf{K} \mathbf{W}^{\frac{1}{2}}$ . Theorem 1 obtains the first near-linear time algorithms for this problem when the optimal condition number of the rescaled matrix is small.
- Semi-random regression. We consider a related problem, motivated by semi-random noise models, which takes full-rank  $\mathbf{A} \in \mathbb{R}^{n \times d}$  with  $n \geq d$  and seeks  $\mathbf{W} \in \mathbb{S}_{\succ 0}^n$  minimizing the condition number of  $\mathbf{A}^\top \mathbf{W}\mathbf{A}$ . Theorem 2 gives the first near-linear time algorithm for this problem, and applications of it reduce risk bounds for statistical linear regression.
- Structured linear systems. We robustify Laplacian system solvers, e.g., [ST04], to obtain near-linear time solvers for systems in dense matrices well-approximated spectrally by Laplacians in Theorem 3. We also give new near-linear time solvers for several families of structured matrices, e.g., dense inverse Laplacians and M-matrices, $^{4}$  in Theorems 4 and 5.
For the preconditioning problems considered in Theorems 1, 2, and 3, we give the first runtimes faster than a generic SDP solver, for which, state-of-the-art runtimes  $\mathrm{[JKL^{+}20,HJS^{+}22]}$  are highly superlinear  $(\Omega (d^{3.5})$  for diagonal preconditioning and  $\Omega (d^{2\omega})$  for approximating Laplacians, where  $d$  is the matrix dimension and  $\omega >2.3$  is the current matrix multiplication constant [AW21]). For the corresponding linear system solving problems in each case, as well as in Theorems 4 and 5, the prior state-of-the-art was to treat the linear system as generic and ran in  $\Omega (d^{\omega})$  time.
Organization. We begin by overviewing the main applications of our matrix-dictionary recovery framework in Sections 2 and 3, which respectively cover our results on diagonal preconditioning and structured linear algebra. These sections are self-contained (with some definitions of the matrix families we study in Section 3 deferred to the supplementary material), and can be read independently. In Section 4, we contextualize and formalize the general matrix-dictionary recovery problem (1) we introduce and study. We also provide our main meta-algorithm and its guarantees, and an overview of how the results of Sections 2 and 3 follow from applications of it. We finally compare our framework to related algorithms and give a more thorough runtime comparison in Section 5.
Notation. In Section 2 (focusing on diagonal preconditioning and semi-random regression) only, we refer to the matrices to be preconditioned as  $\mathbf{K}$  or  $\mathbf{A}$ . This is for consistency with the numerical linear algebra literature, where  $\mathbf{K}$  represents a positive definite kernel matrix, and  $\mathbf{A}$  denotes the data in a regression problem  $\min_x\| \mathbf{A}x - b\| _2$ . In the rest of the paper, our notation will be consistent with (1). The  $d\times d$  symmetric matrices are  $\mathbb{S}^d$ , the positive semidefinite (PSD) and definite (PD) cones are  $\mathbb{S}_{\succeq 0}^d$  and  $\mathbb{S}_{\succeq 0}^d$ ; the remainder of our notation is standard and deferred to Section 2 of the supplement.
2 Diagonal preconditioning
When solving linear systems via iterative methods, one of the most popular preconditioning strategies is to use a diagonal matrix. This is appealing because diagonal matrices can be applied and inverted quickly. Determining the best diagonal preconditioner is a classical numerical linear algebra problem studied since the 1950s [FS55, vdS69, PG90], and has gained recent popularity due to its use in adaptive gradient methods [DHS11, KB15]. In Section 4, we discuss how diagonal preconditioning
4 Inverse M-matrices are necessarily dense, see Appendix B of the supplementary material.
is an instance of (1) in the matrix-dictionary  $\mathbf{M}_i = e_i e_i^\top$ , where  $e_i$  is the  $i^{\text{th}}$  basis vector. Leveraging this viewpoint, we design algorithms for two natural instantiations of diagonal preconditioning.
Outer scaling. One formulation of the optimal diagonal preconditioning problem, which we refer to as outer scaling, asks to optimally reduce the condition number of positive definite  $\mathbf{K} \in \mathbb{R}^{d \times d}$  with a diagonal matrix  $\mathbf{W}$ , i.e., return diagonal  $\mathbf{W} = \mathbf{diag}(w)$  for  $w \in \mathbb{R}_{>0}^{d}$  such that
$$
\kappa (\mathbf {W} ^ {\frac {1}{2}} \mathbf {K W} ^ {\frac {1}{2}}) \approx \kappa_ {o} ^ {\star} (\mathbf {K}) := \min _ {\mathrm {d i a g o n a l} \mathbf {W} \succ \mathbf {0}} \kappa (\mathbf {W} ^ {\frac {1}{2}} \mathbf {K W} ^ {\frac {1}{2}}).
$$
Given  $\mathbf{W}$ , a solution to  $\mathbf{K}x = b$  can be obtained by solving the better-conditioned  $\mathbf{W}^{\frac{1}{2}}\mathbf{K}\mathbf{W}^{\frac{1}{2}}y = \mathbf{W}^{\frac{1}{2}}b$  and returning  $x = \mathbf{W}^{\frac{1}{2}}y$ . The optimal  $\mathbf{W}$  can be obtained via a semidefinite program (SDP) [QYZ20], but the computational cost of general-purpose SDP solvers outweighs benefits for solving linear systems. Outer scaling is poorly understood algorithmically; prior to our work, even attaining a constant-factor approximation to  $\kappa_{o}^{\star}(\mathbf{K})$  without a generic SDP solver was unknown.
This state of affairs has resulted in the widespread use of heuristics for constructing  $\mathbf{W}$ , such as Jacobi preconditioning [vdS69, GR89] and matrix scaling [AZLOW17, CMTV17a, GO18]. The former strategy, where the preconditioner is taken as the inverse diagonal to  $\mathbf{K}$ , was notably highlighted by Adagrad [DHS11], which used Jacobi preconditioning to improve computational costs. However, both heuristics have clear drawbacks from theoretical or practical perspectives.
Prior to our work the best approximation guarantee known for Jacobi preconditioning was a result of van der Sluis [vdS69, GR89], which shows the Jacobi preconditioner is an  $m$ -factor approximation to the optimal preconditioning problem where  $m \leq d$  is the maximum number of non-zeros in any row of  $\mathbf{K}$ : in dense matrices this is linear in the problem dimension and can be much larger than  $\kappa_{o}^{\star}(\mathbf{K})$ . We review and slightly strengthen this result in Appendix C of the supplement. We also prove a new dimension-independent baseline result of independent interest: the Jacobi preconditioner obtains condition number no worse than  $(\kappa_{o}^{\star}(\mathbf{K}))^{2}$ . Unfortunately, we exhibit a family of matrices showing this bound is tight, dashing hopes they solve outer scaling near-optimally. On the other hand, while sometimes effective as a heuristic [KRU14], matrix scaling algorithms target a different objective (normalizing row and column sums) and do not yield provable guarantees on  $\kappa(\mathbf{W}^{\frac{1}{2}}\mathbf{K}\mathbf{W}^{\frac{1}{2}})$ .
Inner scaling. Another formulation of diagonal preconditioning, which we refer to as inner scaling, takes as input a full-rank  $\mathbf{A} \in \mathbb{R}^{n \times d}$  and asks to find an  $n \times n$  positive diagonal  $\mathbf{W}$  with
$$
\kappa (\mathbf {A} ^ {\top} \mathbf {W A}) \approx \kappa_ {i} ^ {\star} (\mathbf {A}) := \min _ {\mathrm {d i a g o n a l} \mathbf {W} \succ \mathbf {0}} \kappa (\mathbf {A} ^ {\top} \mathbf {W A}).
$$
As a comparison, when outer scaling is applied to the kernel matrix  $\mathbf{K} = \mathbf{A}^{\top}\mathbf{A}$ ,  $\mathbf{W}^{\frac{1}{2}}\mathbf{K}\mathbf{W}^{\frac{1}{2}}$  can be seen as rescaling the columns of  $\mathbf{A}$ . On the other hand, in inner scaling we instead rescale rows of  $\mathbf{A}$ . Inner scaling has natural applications to improving risk bounds in a robust statistical variant of linear regression, which we comment upon shortly. Nonetheless, as in the outer scaling case, no algorithms faster than general SDP solvers are known to obtain even a constant-factor approximation to  $\kappa_{i}^{\star}(\mathbf{A})$ . Further, despite clear problem similarities, it is unclear how to best extend heuristics (e.g., Jacobi preconditioning and matrix scaling) for outer scaling to inner scaling.
Our results. We give the first nontrivial approximation algorithms (beyond generic SDP solvers) for both the outer and inner scaling problems, yielding diagonal preconditioners attaining constant-factor approximations to  $\kappa_{o}^{\star}$  and  $\kappa_{i}^{\star}$  in near-linear time.  ${}^{6}\mathcal{T}_{\mathrm{mv}}(\mathbf{M})$  denotes the time required to multiply a vector by  $\mathbf{M}$ ; this is at most the sparsity of  $\mathbf{M}$ , but can be substantially faster for structured  $\mathbf{M}$ .
Theorem 1 (Outer scaling). Let  $\epsilon >0$  be a fixed constant. There is an algorithm, which given full-rank  $\mathbf{K}\in \mathbb{S}_{\geq 0}^{d}$  computes  $w\in \mathbb{R}_{\geq 0}^{d}$  such that  $\kappa (\mathbf{W}^{\frac{1}{2}}\mathbf{K}\mathbf{W}^{\frac{1}{2}})\leq (1 + \epsilon)\kappa_{o}^{\star}(\mathbf{K})$  with probability  $\geq 1 - \delta$  in time  $O(\mathcal{T}_{\mathrm{mv}}(\mathbf{K})\cdot (\kappa_o^\star (\mathbf{K}))^{1.5}\cdot \mathrm{polylog}(\frac{d\kappa_o^\star(\mathbf{K})}{\delta}))$
Theorem 2 (Inner scaling). Let  $\epsilon >0$  be a fixed constant. There is an algorithm, which given full-rank  $\mathbf{A}\in \mathbb{R}^{n\times d}$  for  $n\geq d$  computes  $w\in \mathbb{R}_{\geq 0}^n$  such that  $\kappa (\mathbf{A}^{\top}\mathbf{W}\mathbf{A})\leq (1 + \epsilon)\kappa_{i}^{\star}(\mathbf{A})$  with probability  $\geq 1 - \delta$  in time  $O(\mathcal{T}_{\mathrm{mv}}(\mathbf{A})\cdot (\kappa_i^\star (\mathbf{A}))^{1.5}\cdot \mathrm{polylog}(\frac{n\kappa_i^\star(\mathbf{A})}{\delta}))$
$^{5}\kappa (\mathbf{M})$  is the condition number of positive definite  $\mathbf{M}$ , i.e. the eigenvalue ratio  $\lambda_{\mathrm{max}}(\mathbf{M}) / \lambda_{\mathrm{min}}(\mathbf{M})$ .
<sup>6</sup>We are not currently aware of a variant of our matrix dictionary recovery framework which extends to simultaneous inner and outer scaling, though it is worth noting that prior work [QGH⁺22] does obtain such a result via semidefinite programming. Obtaining such a variant is an interesting open problem for future work.
<sup>7</sup>We do not focus on the  $\epsilon$  dependence and instead take it to be constant since, in applications involving solving linear systems, there is little advantage to obtaining better than a 2-approximation (i.e.,  $\epsilon = 1$ ).
Our methods pay a small polynomial overhead in the quantities  $\kappa_{o}^{\star}$  and  $\kappa_{i}^{\star}$ , but notably suffer no dependence on the original conditioning of the matrices. Typically, the interesting use case for diagonal preconditioning is when  $\kappa_{o}^{\star}(\mathbf{K})$  or  $\kappa_{i}^{\star}(\mathbf{A})$  is small but  $\kappa (\mathbf{K})$  or  $\kappa (\mathbf{A}^{\top}\mathbf{A})$  is large, a regime where our runtimes are near-linear and substantially faster than directly applying iterative methods.
It is worth noting that in light of our new results on Jacobi preconditioning, the end-to-end runtime of Theorem 1 for solving linear systems (rather than optimal preconditioning) can be improved: accelerated gradient methods on a preconditioned system with condition number  $(\kappa_{o}^{\star})^{2}$  have runtimes scaling as  $\kappa_{o}^{\star}$ . That said, when repeatedly solving multiple systems in the same matrix, Theorem 1 may offer an advantage over Jacobi preconditioning. Our framework also gives a potential route to achieve the optimal end-to-end runtime scaling as  $\sqrt{\kappa_{o}^{\star}}$ , detailed in Appendix D of the supplement.
Beyond that which is obtainable by black-box using general SDP solvers, we are not aware of any other claimed runtime in the literature for solving the inner and outer scaling problems considered in Theorems 1 and 2. Directly using state-of-the-art SDP solvers [JKL $^{+}$ 20, HJS $^{+}$ 22] incurs substantial overhead  $\Omega(n^{\omega}\sqrt{d} + nd^{2.5})$  or  $\Omega(n^{\omega} + d^{4.5} + n^{2}\sqrt{d})$ , where  $\omega < 2.372$  is the current matrix multiplication constant [Wil12, Gal14, AW21, DWZ23, WXXZ23]. For outer scaling, where  $n = d$ , this implies an  $\Omega(d^{3.5})$  runtime; for other applications, e.g., preconditioning  $d \times d$  perturbed Laplacians where  $n = d^2$ , the runtime is  $\Omega(d^{2\omega})$ . Applying state-of-the-art approximate SDP solvers (rather than our custom ones, i.e., Theorems 6 and 7) appears to yield runtimes  $\Omega(\mathrm{nnz}(\mathbf{A}) \cdot d^{2.5})$ , as described in Appendix E.2 of [LSTZ20]. This is in contrast with our Theorems 1, 2 which achieve  $\widetilde{O}\left(\mathrm{nnz}(\mathbf{A}) \cdot (\kappa^{\star})^{1.5}\right)$ . Hence, we improve existing tools by poly(d) factors in the main regime of interest where the optimal rescaled condition number  $\kappa^{\star}$  is small. Concurrent to our work, [QGH $^{+}$ 22] gave algorithms for constructing optimal diagonal preconditioners using interior point methods for SDPs, which run in at least the superlinear times discussed previously.
Statistical aspects of preconditioning. Unlike an outer scaling, a good inner scaling does not speed up a least squares regression problem  $\min_x\| \mathbf{A}x - b\| _2$ . Instead, it allows for a faster solution to the reweighted problem  $\min_x\| \mathbf{W}^{\frac{1}{2}}(\mathbf{A}x - b)\| _2$ . This has a number of implications from a statistical perspective. We explore an interesting connection between inner scaling preconditioning and semi-random noise models for least-squares regression, situated in the literature in Section 5.
As a motivating example of our noise model, consider the case when there is a hidden parameter vector  $x_{\mathrm{true}} \in \mathbb{R}^d$  that we want to recover, and we have a "good" set of consistent observations  $\mathbf{A}_g x_{\mathrm{true}} = b_g$ , in the sense that  $\kappa(\mathbf{A}_g^\top \mathbf{A}_g)$  is small. Here, we can think of  $\mathbf{A}_g$  as being drawn from a well-conditioned distribution. Now, suppose an adversary gives us a superset of these observations  $(\mathbf{A}, b)$  such that  $\mathbf{A} x_{\mathrm{true}} = b$ , and  $\mathbf{A}_g$  are an (unknown) subset of rows of  $\mathbf{A}$ , but  $\kappa(\mathbf{A}^\top \mathbf{A}) \gg \kappa(\mathbf{A}_g^\top \mathbf{A}_g)$ . This can occur when rows are sampled from heterogeneous sources. Perhaps counterintuitively, by giving additional consistent data, the adversary can arbitrarily hinder the cost of iterative methods. This failure can be interpreted as being due to overfitting to generative assumptions (e.g., sampling rows from a well-conditioned covariance, instead of a mixture): standard iterative methods assume too much structure, where ideally they would use as little as information-theoretically possible.
Our inner scaling methods can be viewed as "robustifying" linear system solving to such semirandom noise models (by finding  $\mathbf{W}$  yielding a rescaled condition number comparable or better than the indicator of the rows of  $\mathbf{A}_g$ , which are not known a priori). In Section 6 of the supplement, we demonstrate applications of inner scaling in reducing the mean-squared error risk in statistical regression settings encompassing our semi-random noise model, where the observations  $b$  are corrupted by (homoskedastic or heteroskedastic) noise. In all settings, our preconditioning algorithms yield computational gains, improved risk bounds, or both, by factors of roughly  $\kappa (\mathbf{A}^{\top}\mathbf{A}) / \kappa_{i}^{\star}(\mathbf{A})$ .
3 Robust linear algebra for structured matrices
Over the past decade, the theoretical computer science and numerical linear algebra communities have dedicated substantial effort to developing faster solvers for regression problems in various families of combinatorially-structured matrices. Perhaps the most prominent example is [ST04], who gave a near-linear time solver for linear systems in graph Laplacian matrices. $^{8}$  A long line of exciting
<sup>8</sup>We recall definitions of the families of matrices we study here. We call a square matrix  $\mathbf{A}$  a  $\mathbf{Z}$ -matrix if  $\mathbf{A}_{ij} \leq 0$  for all  $i \neq j$ . We call a matrix  $\mathbf{L}$  a Laplacian if it is a symmetric  $\mathbf{Z}$ -matrix with  $\mathbf{L} \mathbb{1} = 0$ . We call  $\mathbf{M}$  an invertible  $M$ -matrix if  $\mathbf{M} = s \mathbf{I} - \mathbf{A}$  where  $s > 0$ ,  $\mathbf{A} \in \mathbb{R}_{\geq 0}^{d \times d}$ , and  $\rho(\mathbf{A}) < s$  where  $\rho$  is the spectral radius.
work has obtained improved solvers for these systems [KMP10, KMP11, KOSZ13, LS13,  $\mathrm{CKM}^{+}14$ , PS14,  $\mathrm{KLP}^{+}16$ , KS16, JS21], which have been used to improve the runtimes for a wide variety of graph-structured problems, including maximum flow [CKM  $^{+}11$ , Mad13, LS14], sampling random spanning trees [KM09, MST15,  $\mathrm{DKP}^{+}17$ , Sch18], graph clustering [ST04, OV11, OSV12], and more [DS08, KRSS15, CMSV17, CMTV17b]. Additionally, efficient linear system solvers have been developed for solving systems in other types of structured matrices, e.g., block diagonally dominant systems [KLP  $^{+}16$ ], M-matrices [AJSS19], and directed Laplacians [CKP  $^{+}16$ ,  $\mathrm{CKP}^{+}17$ ,  $\mathrm{CKK}^{+}18$ ].
Perturbations of structured matrices. Despite the importance of these matrices with combinatorial structure, previously-developed solvers are in some ways quite brittle. For example, there are simple matrix families closely related to Laplacians for which the best-known runtimes for solving linear systems are achieved by ignoring problem structure, and using generic matrix multiplication techniques as a black box. Perhaps the simplest example is solving systems in perturbed Laplacians, i.e., matrices which admit constant-factor approximations by a Laplacian matrix, but which are not Laplacians themselves. This situation can arise when a Laplacian is used to approximate a physical phenomenon [BHV08]. We show that the framework we develop for (1) yields, as a consequence, robustifications and recovery routines building upon previously-developed solvers for structured linear systems. As a first example, we give the following perturbed Laplacian solver.
Theorem 3 (Perturbed Laplacian solver). Let  $\mathbf{M} \succeq \mathbf{0} \in \mathbb{R}^{n \times n}$  be such that there exists an (unknown) Laplacian  $\mathbf{L}$  with  $\mathbf{M} \preceq \mathbf{L} \preceq \kappa^{\star} \mathbf{M}$ , and that  $\mathbf{L}$  corresponds to a graph with edge weights between  $w_{\min}$  and  $w_{\max}$ , with  $\frac{w_{\max}}{w_{\min}} \leq U$ . For any  $\delta \in (0,1)$  and  $\epsilon > 0$ , there is an algorithm recovering a Laplacian  $\mathbf{L}'$  with  $\mathbf{M} \preceq \mathbf{L}' \preceq (1 + \epsilon) \kappa^{\star} \mathbf{M}$  with probability  $\geq 1 - \delta$  in time  $O(n^{2} \cdot (\kappa^{\star})^{2} \cdot \mathrm{poly}(\frac{\log \frac{n\kappa^{\star}U}{\delta}}{\epsilon}))$ . Consequently, there is an algorithm for solving linear systems in  $\mathbf{M}$  to  $\epsilon$ -relative accuracy with probability  $\geq 1 - \delta$ , in time  $O(n^{2} \cdot (\kappa^{\star})^{2} \cdot \mathrm{polylog}(\frac{n\kappa^{\star}U}{\delta\epsilon}))$ .
Theorem 3 can be viewed as solving a preconditioner construction problem, where we know there exists a Laplacian matrix  $\mathbf{L}$  which spectrally resembles  $\mathbf{M}$ , and wish to efficiently recover a Laplacian with similar guarantees. Our matrix-dictionary recovery framework (1) captures the setting of Theorem 3 by leveraging an appropriate matrix-dictionary of edge Laplacians, discussed in Section 4. The conceptual message of Theorem 3 is that near-linear time solvers for Laplacians robustly extend through our preconditioning framework to efficiently solve matrices approximated by Laplacians. Beyond this specific application, our framework could be used to solve perturbed generalizations of future families of structured matrices.