Group Equivariant Deep Learning
Lecture 1 - Regular group convolutions
Lecture 1.5 - A brief history of G-CNNs

Erik Bekkers, Amsterdam Machine Learning Lab, University of Amsterdam
This mini-course serves as a module with the UvA Master AI course Deep Learning 2 https://uvadl2c.github.io/

G-CNNs rule!
• The right inductive bias: guaranteed equivariance(no loss of information)• Performance gains that can’t be obtained by data-augmentation alone(both local and global equivariance/invariance)• Increased sample e ciency(increased weight sharing, no geometric augmentation necessary)ffi

2

G-CNNs rule!
• The right inductive bias: guaranteed equivariance(no loss of information)• Performance gains that can’t be obtained by data-augmentation alone(both local and global equivariance/invariance)• Increased sample e ciency(increased weight sharing, no geometric augmentation necessary)ffi

2

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

3

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

4

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

5

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

5

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

5

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

6

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

7

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

7

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

7

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

8

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

8

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

8

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

8

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

9A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

10

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

10

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

10

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

11

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

11

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

11

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)Continuous rotation G-CNNs(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

12

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

Gauge-equivariant CNNs
(manifolds)

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)G-CNNs are NNs underContinuous rotation G-CNNs
equivariance constraints(regular discretizations)Continuous rotation G-CNNs(steerable)

Continuous G-CNNs(Lie groups)

Cesa-Lang-Weiler 2022
G = ℝd ⋊ H with H compact

https://quva-lab.github.io/escnn/

12

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Discrete G-CNNs(square/cube symmetries)
Cohen-Welling 2016
p4m

Winkels-Cohen 2018
3D discrete roto-transl.
Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

CNNs

Gauge-equivariant CNNs
(manifolds)

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)G-CNNs are NNs underContinuous rotation G-CNNs
Continuous rotation G-CNNs
equivariance constraints(regular discretizations)(steerable)“Inductive bias”

Continuous G-CNNs(Lie groups)

Reduce the search space for
NNs
Cesa-Lang-Weiler
2022

G = ℝd ⋊ H with H compact

to only the sensible ones!
https://quva-lab.github.io/escnn/

12

A brief history of G-CNNs
https://github.com/Chen-Cai-OSU/awesome-equivariant-network

Up next:
Gauge-equivariant CNNs
(manifolds)
A
linearmap
isequivariant
Cohen-Welling 2016
p4m
if and only if it is a group convolution
Discrete G-CNNs(square/cube symmetries)

Winkels-Cohen 2018
3D discrete roto-transl.

Hoogeboom et al. 2018
SE(2,6)

Dieleman et al. 2016
p4m

NNs

CNNs

Worrall-Brostow 2018
3D discrete roto-transl.

Weiler et al. 2021
Cohen et al. 2019
Coordinate Independent CNNs
Icosahedronde Haan et al. 2020
Meshes

G-CNNsCohen et al. 2018
Worrall et al. 2017
Bekkers 2019
SE(3)
SE(2)
Lie groups (rotation, scale)
Kondor-Trivedi
2018
Esteves 2017
Bekkers et al. 2014-2018
SE(3)Finzi et al. 2020
SO(3)
SE(2)
Weiler et al. 2018
Lie groups
Kondor 2018
SE(3)Weiler et al. 2017
SE(3)Chakraborty et al. 2018
SE(2)Riemannian Hom. spaces
Zhou et al. 2017
Thomas et al. 2018
Sosnovik et al. 2020
SE(2)
SE(3)
Weiler-Cesa 2019
Scale-translation
SE(2)Mallat et al. 2013, 2015
LeCun et al. 1990
SE(2)G-CNNs are NNs underContinuous rotation G-CNNs
Continuous rotation G-CNNs
equivariance constraints(regular discretizations)(steerable)“Inductive bias”

Continuous G-CNNs(Lie groups)

Reduce the search space for
NNs
Cesa-Lang-Weiler
2022

G = ℝd ⋊ H with H compact

to only the sensible ones!
https://quva-lab.github.io/escnn/

12