ICML 2025

Score Matching with Missing Data
Josh Givens 1 Song Liu 1 Henry W J Reeve 2
Abstract
Score matching is a vital tool for learning the dis-
tribution of data with applications across many
areas including diffusion processes, energy based
modelling, and graphical model estimation. De-
spite all these applications, little work explores
its use when data is incomplete. We address this
by adapting score matching (and its major exten-
sions) to work with missing data in a flexible set-
ting where data can be partially missing over any
subset of the coordinates. We provide two sep-
arate score matching variations for general use,
an importance weighting (IW) approach, and a
variational approach. We provide finite sample
bounds for our IW approach in finite domain set-
tings and show it to have especially strong perfor-
mance in small sample lower dimensional cases.
Complementing this, we show our variational ap-
proach to be strongest in more complex high-
dimensional settings which we demonstrate on
graphical model estimation tasks on both real and
simulated data.
1. Introduction
Over the last decade, score matching has established itself
as a powerful tool with downstream use in many areas of
machine learning. Examples include: energy based mod-
elling (Swersky et al., 2011; Bao et al., 2020; Li et al.,
2019b), mode-seeking clustering (Sasaki et al., 2014), and
perhaps most prominently of all Diffusion processes (Song
& Ermon, 2019; Song et al., 2021b; Tashiro et al., 2021;
Song et al., 2021a; Huang et al., 2021). Score matching
aims to learn the score of a distribution which is the gra-
dient of the log of the probability density function (PDF)
(s(x) = ∇x log p(x)). In contrast to modelling the density
directly, the score does not need to integrate to one meaning
there is no need to calculate a normalising constant. This
allows it to be much more more flexibly modelled than the
1School of Mathematics, University of Bristol, Bristol, UK
2School of Artificial Intelligence,...