  We present a unified technique for sequential estimation of convex
divergences between distributions, including integral probability metrics like
the kernel maximum mean discrepancy, $\varphi$-divergences like the
Kullback-Leibler divergence, and optimal transport costs, such as powers of
Wasserstein distances. The technical underpinnings of our approach lie in the
observation that empirical convex divergences are (partially ordered) reverse
submartingales with respect to the exchangeable filtration, coupled with
maximal inequalities for such processes. These techniques appear to be powerful
additions to the existing literature on both confidence sequences and convex
divergences. We construct an offline-to-sequential device that converts a wide
array of existing offline concentration inequalities into time-uniform
confidence sequences that can be continuously monitored, providing valid
inference at arbitrary stopping times. The resulting sequential bounds pay only
an iterated logarithmic price over the corresponding fixed-time bounds,
retaining the same dependence on problem parameters (like dimension or alphabet
size if applicable).
