9 December 2021

Gauge Equivariant
Mesh CNNs
Pim de Haan*¹², Maurice Weiler*2, Taco Cohen¹, Max Welling3¹ Qualcomm Technologies Netherlands B.V.
² University of Amsterdam QUVA Lab³ University of Amsterdam

Qualcomm AI Research is an initiative of Qualcomm Technologies, Inc.

Pim de Haan
Research Associate
Qualcomm Technologies Netherlands B.V.

CNN that works on planes and on rabbits• Orient convolutional kernel• Gauge – basis of tangent plane• Gauge Equivariance2Collaboration

Maurice Weiler

Taco Cohen

Max Welling3Outline• CNN review• Message passing on a mesh• Scalar convolution• Vector fields• Gauge equivariant mesh convolutions• Implementation• Application to blood flow4

Convolutional neural networks on images• Image feature 𝑓 ∈ ℝ! × !
• Kernel 𝑘 ∈ ℝ# × #• 𝑓$% = 𝑘 ⋆ 𝑓 $ = ∑& 𝑘 𝑞 𝑓(𝑝 − 𝑞)
• Alternate convolutions with non-linearities• Learn kernels 𝑘5Anisotropy

Anisotropic

Isotropic• 9 parameters• 3 parameters• Detects any edge

• Can not detect edges
6

Mesh

• Discretization of curved surface• Triangular mesh = collection of triangular faces• Manifold mesh: connected faces look like a plane7

Message passing on a mesh
• Feature at vertices• Message passing
𝑓$% = - 𝑘 𝑞 → 𝑝 𝑓&&∈𝒩($)• Applications:• Segment vertices• Classify shapes• Predict blood flow
• Initial features• Vertex coordinates• Local description of curvature

8

Convolutions on a mesh

• Canonical relative (x, y)
coordinates of neighbours• Log map to tangent plane• Polar coordinates• What is 𝜃 = 0?
• Choice of coordinates: gauge

9Gauge invariance, fixing & equivariance• Gauge: choice of basis for each tangent plane• Reference neighbour• Option 1: gauge invariance• Message 𝑞 → 𝑝 independent of 𝜃$&• But: isotropic• Option 2: gauge fixing• Principal curvature direction• But: ill-defined• Option 3: Gauge equivariance [Cohen et al. 2019]:• The same feature in different gauges has same output (up to rotation)
10

Gauge equivariance with scalar features• Feature 𝑓: 𝑀 → ℝ• Transformation rule under gauge transformation: invariant• Gauge 𝑤, polar coordinates of neighbour 𝑞 of 𝑝: 𝑤$ 𝑞 = (𝑟& , 𝜃& )• Different gauge 𝑤′, has coordinates 𝑤$% 𝑞 = (𝑟& , 𝜃′& ) = (𝑟& , 𝜃& + 𝑔$ )• Kernel 𝐾 𝑟, 𝜃 ∈ ℝ• In gauge 𝑤: 𝐾 ⋆ 𝑓 $ = ∑&∈𝒩($) 𝐾 𝑟& , 𝜃& 𝑓&• In gauge 𝑤 % : 𝐾 ⋆ 𝑓 $ = ∑&∈𝒩($) 𝐾(𝑟& , 𝜃& + 𝑔$ )𝑓&• Equality for any angle 𝑔$ implies 𝐾 𝑟& , 𝜃& + 𝑔$ = 𝐾 𝑟& , 𝜃&
• Kernel isotropic

11

Gaugeequivariant
convolutions
on scalar
fieldsScalar convolutions are isotropic

12

Tangent vector field features• Feature 𝑓$ ∈ 𝑇$ 𝑀
• In gauge 𝑤$ , 𝑓$ ∈ ℝ+
• Let 𝑤$% 𝑞 = 𝑤$ 𝑞 + 𝑔$
• In gauge 𝑤$% , 𝑓$% = 𝜌 −𝑔$ 𝑓$cos 𝜃•𝜌 𝜃 =sin 𝜃𝑤!
𝑝𝑔!
𝑤!"− sin 𝜃cos 𝜃• More general feature: 𝜌 group representation of group of planar rotations 𝑆𝑂(2)13Parallel Transport• Tangent planes not parallel• Parallel transport of geodesic• Transport gauge-defining X-axis• Angle 𝑔&→$• Cheaply precomputed• Any parallel transport by linearity

14General Gauge Equivariant Convolution• Two gauges are related by planar rotation 𝑔 ∈ SO 2
• Vertex feature: group representation 𝜌 𝑔 ∈ ℝ-×• E.g. scalar feature 𝜌 𝑔 = 1
cos 𝑔 − sin 𝑔• E.g. tangent vector feature 𝜌 𝑔 =sin 𝑔
cos 𝑔• Kernel 𝐾 𝑟, 𝜃! ×∈ℝ• Convolution:

𝐾 ⋆ 𝑓 $ = ∑&∈𝒩($) 𝐾 𝑟& , 𝜃& 𝜌(𝑔&→$ )𝑓&• Equivariance if:

𝜌% 𝑔 𝐾 𝑟, 𝜃 = 𝐾 𝑟, 𝜃 + 𝑔 𝜌(𝑔)15

Gaugeequivariantconvolutions
on vector
fieldsVector convolutions are anisotropic

16Symmetry properties

Gauge

Vertex coordinates

Mesh isometries
17

Solving the kernel constraint• Constraint
𝜌% 𝑔 𝐾 𝑟, 𝜃 = 𝐾 𝑟, 𝜃 + 𝑔 𝜌(𝑔)• For 𝑟 > 0• Freely choose 𝐾 𝑟., 0 , … , 𝐾 𝑟/ , 0• Linearly interpolate for 𝐾(𝑟, 0)• 𝐾 𝑟, 𝜃 = 𝜌% 𝜃 𝐾 𝑟, 0 𝜌(−𝜃)• For 𝑟 = 0• Linear constraint
𝜌% 𝑔 𝐾 0 = 𝐾 0 𝜌(𝑔)• Linearly combine solution withlearned parameters18Efficiently approximating geometry• Geometric quantities• Logarithmic map• Parallel transport• Exact: Partial Differential Equation• Spherical approximation

19

Application to blood flow

Julian Suk
University of Twente[Suk, de Haan, Lippe, Brune, Wolterink, 2021]

20

Problem formulation• Shape of arteries in human body related to e.g. aneurysm• Quantitative analysis of blood flow useful indicator – wall shear stress• Non-invasively: model artery with MRI scanner• Simulate blood flow with computational fluid dynamics (> 20h)
• Learn neural network surrogate to predict WSS on CFD ground truth• Dataset of realistic random meshes21Equivariance• Arteries not in canonical orientation• Equivariance to global transformations• Gauge Equivariant Mesh CNN

22

Network Architecture

23

Results

24

TakeawayGauge Equivariant Mesh CNN is:• Simple• Scalable• Anisotropic ⇒ expressive• Symmetry properties• Try it out:
github.com/Qualcomm-AI-research/gauge-equivariant-mesh-cnn

25Thank you
Follow us on:
For more information, visit us at:
www.qualcomm.com & www.qualcomm.com/blog

Nothing in these materials is an offer to sell any of thecomponents or devices referenced herein.
©2018-2020 Qualcomm Technologies, Inc. and/or its
affiliated companies. All Rights Reserved.
Qualcomm is a trademark or registered trademark ofQualcomm Incorporated. Other products and brand names
may be trademarks or registered trademarks of theirrespective owners.

References in this presentation to “Qualcomm” may mean Qualcomm
Incorporated, Qualcomm Technologies, Inc., and/or other subsidiaries
or business units within the Qualcomm corporate structure, asapplicable. Qualcomm Incorporated includes Qualcomm’s licensing
business, QTL, and the vast majority of its patent portfolio. Qualcomm
Technologies, Inc., a wholly-owned subsidiary of Qualcomm
Incorporated, operates, along with its subsidiaries, substantially all of
Qualcomm’s engineering, research and development functions, andsubstantially all of its product and services businesses, including itssemiconductor business, QCT.