  Mechanistic models with differential equations are a key component of
scientific applications of machine learning. Inference in such models is
usually computationally demanding, because it involves repeatedly solving the
differential equation. The main problem here is that the numerical solver is
hard to combine with standard inference techniques. Recent work in
probabilistic numerics has developed a new class of solvers for ordinary
differential equations (ODEs) that phrase the solution process directly in
terms of Bayesian filtering. We here show that this allows such methods to be
combined very directly, with conceptual and numerical ease, with latent force
models in the ODE itself. It then becomes possible to perform approximate
Bayesian inference on the latent force as well as the ODE solution in a single,
linear complexity pass of an extended Kalman filter / smoother - that is, at
the cost of computing a single ODE solution. We demonstrate the expressiveness
and performance of the algorithm by training a non-parametric SIRD model on
data from the COVID-19 outbreak.
