ICML 2025

Schwarz–Schur Involution: Lightspeed Differentiable Sparse Linear Solvers
Yu Wang 1 2 3 S. Mazdak Abulnaga 1 2 Ya¨el Balbastre 1 4 Bruce Fischl 1 2
Abstract
Sparse linear solvers or generalized deconvolution
are fundamental to science and engineering, ap-
plied in partial differential equations (PDEs), sci-
entific computing, computer vision, and beyond.
Indirect solvers possess characteristics that make
them undesirable as stable differentiable modules;
existing direct solvers, though reliable, are too
expensive to be adopted in neural architectures.
We substantially accelerate direct sparse solvers
by up to three orders of magnitude, violating com-
mon assumptions that direct solvers are too slow.
We “condense” a sparse Laplacian matrix into a
dense tensor, a compact data structure that batch-
wise stores the Dirichlet-to-Neumann matrices,
reducing the sparse solving to recursively merg-
ing pairs of dense matrices that are much smaller.
The batched small dense systems are sliced and
inverted in parallel to take advantage of dense
GPU BLAS kernels, highly optimized in the era
of deep learning. Our method is efficient, quali-
fied as a strong zero-shot baseline for AI-based
PDE solving, and a reliable differentiable mod-
ule integrable into machine learning pipelines.
Our code is available at https://github.com/
wangyu9/Schwarz_Schur_Involution.
1. Introduction
In this paper, we propose a conceptually simple approach
that reduces solving certain sparse linear system (A, b) to
“involuting” tensors (α, β), leading to significant improve-
ments. Global linear systems in computer vision and scien-
tific computing, such as the Laplacian or Hessian systems on
image domains, are usually very large yet sparse, as induced
by pixel affinity. For a sparse A, direct solvers find the
exact solution: x = A−1b, for which we present the first
1Athinoula Martinos Center, Massachusetts General Hospital,
Harvard Medical School 2Massachusetts Institute of Technology,
MIT CSAIL, USA 3S...