ICML 2025

Equivariant Neural Tangent Kernels
Philipp Misof 1 Pan Kessel 2 Jan E. Gerken 1
Abstract
Little is known about the training dynamics of
equivariant neural networks, in particular how
it compares to data augmented training of their
non-equivariant counterparts. Recently, neural
tangent kernels (NTKs) have emerged as a pow-
erful tool to analytically study the training dy-
namics of wide neural networks. In this work,
we take an important step towards a theoretical
understanding of training dynamics of equivari-
ant models by deriving neural tangent kernels for
a broad class of equivariant architectures based
on group convolutions. As a demonstration of
the capabilities of our framework, we show an in-
teresting relationship between data augmentation
and group convolutional networks. Specifically,
we prove that they share the same expected pre-
diction over initializations at all training times
and even off the data manifold. In this sense, they
have the same training dynamics. We demonstrate
in numerical experiments that this still holds ap-
proximately for finite-width ensembles. By imple-
menting equivariant NTKs for roto-translations
in the plane (G = Cn ⋉R2) and 3d rotations
(G = SO(3)), we show that equivariant NTKs
outperform their non-equivariant counterparts as
kernel predictors for histological image classifica-
tion and quantum mechanical property prediction.
1. Introduction
Equivariant neural networks (Weiler et al., 2023; Gerken
et al., 2023) are widely used in many applications of great
practical importance, for example in medical image analysis
in two and three dimensions (Bekkers et al., 2018; Winkels
& Cohen, 2019; M¨uller et al., 2021; Pang et al., 2023) and in
quantum chemistry (Duval et al., 2023; Batzner et al., 2022;
1Department of Mathematical Sciences, Chalmers University of
Technology and the University of Gothenburg, SE-412 96 Gothen-
burg, Sweden 2Prescient Design, Genentech Roche, Basel, Switzer-
land. Correspondence to: Jan Gerken <gerken@...