Title: Data-driven estimation of gradient fields in the weak sense
Primary area: applications to physical sciences (physics, chemistry, biology, etc.)
Keywords: Deep learning, Gradient estimation, Sobolev training, Data assimilation

Abstract: Estimating a function's gradient is crucial in various fields, including data assimilation, reinforcement learning, and geosciences. However, direct gradient computation is often infeasible when functions are available only as black boxes, are non-smooth, or when computational constraints are prohibitive. In this work, we introduce a novel framework for the offline estimation of gradient fields from scalar evaluations, inspired by score matching. Our core idea is to leverage an integration-by-parts formula that transforms the direct mean squared error loss on gradients into a tractable objective. This new loss involves the divergence of a neural network that parametrizes the gradient field, a term that can be efficiently approximated using Hutchinson's trace estimator. We validate our approach through extensive numerical experiments, including offline gradient estimation benchmarks and applications to implicit regression on a probem involving the two-dimensional Navier-Stokes equations. The results demonstrate the significant practical potential of our method for data assimilation and other challenging scientific tasks where traditional gradient-based techniques are inapplicable.

TL;DR: We introduce a statistical estimator inspired from score matching in order to perform arbitrary gradient field matching under known distribution.