Diffusion Scattering Transforms on Graphs

1 INTRODUCTION . Convolutional Neural Networks ( CNN ) are layered information processing architectures . Each of the layers in a CNN is itself the composition of a convolution operation with a pointwise nonlinearity where the filters used at different layers are the outcome of a data-driven optimization process ( LeCun et al. , 2010 ; 2015 ) . Scattering transforms have an analogous layered architecture but differ from CNNs in that the convolutional filters used at different layers are not trained but selected from a multi-resolution filter bank ( Mallat , 2012 ; Bruna & Mallat , 2013 ) . The fact that they are not trained endows scattering transforms with intrinsic value in situations where training is impossible – and inherent limitations in the converse case . That said , an equally important value of scattering transforms is that by isolating the convolutional layered architecture from training effects it permits analysis of the fundamental properties of CNN information processing architectures . This analysis is undertaken in Mallat ( 2012 ) ; Bruna & Mallat ( 2013 ) where the fundamental conclusion is about the stability of scattering transforms with respect to deformations in the underlying domain that are close to translations . In this paper we consider graphs and signals supported on graphs such as brain connectivity networks and functional activity levels ( Huang et al. , 2016 ) , social networks and opinions ( Jackson , 2008 ) , or user similarity networks and ratings in recommendation systems ( Huang et al. , 2018 ) . Our specific goals are : ( i ) To define a family of graph-scattering transforms . ( ii ) To define a notion of deformation for graph signals . ( iii ) To study the stability of graph scattering transforms with respect to this notion of deformation . To accomplish goal ( i ) we consider the family of graph diffusion wavelets which provide an appropriate construction of a multi-resolution filter bank ( Coifman & Maggioni , 2006 ) . Our diffusion scattering transforms are defined as the layered composition of diffusion wavelet filter banks and pointwise nonlinearities . To accomplish goal ( ii ) we adopt the graph diffusion distance as a measure of deformation of the underlying domain ( Coifman & Lafon , 2006 ; Nadler et al. , 2006 ) . Diffusion distances measure the similarity of two graphs through the time it takes for a signal to be diffused on the graph . The major accomplishment of this paper is to show that the diffusion graph scattering transforms are stable with respect to deformations as measured with respect to diffusion distances . Specifically , consider a signal x supported on graph G whose diffusion scattering transform is denoted by the operator ΨG . Consider now a deformation of the 1 signal ’ s domain so that the signal ’ s support is now described by the graph G′ whose diffusion scattering operator is ΨG′ . We show that the operator norm distance ‖ΨG − ΨG′‖ is bounded by a constant multiplied by the diffusion distance between the graphs G and G′ . The constant in this bound depends on the spectral gap of G but , very importantly , does not depend on the number of nodes in the graph . It is important to point out that finding stable representations is not difficult . E.g. , taking signal averages is a representation that is stable to domain deformations – indeed , invariant . The challenge is finding a representation that is stable and rich in its description of the signal . In our numerical analyses we show that linear filters can provide representations that are either stable or rich but that can not be stable and rich at the same time . The situation is analogous to ( Euclidean ) scattering transforms and is also associated with high frequency components . We can obtain a stable representation by eliminating high frequency components but the representation loses important signal features . Alternatively , we can retain high frequency components to have a rich representation but that representation is unstable to deformations . Diffusion scattering transforms are observed to be not only stable – as predicted by our theoretical analysis – but also sufficiently rich to achieve good performance in graph signal classification examples . 2 RELATED WORK . Since graph and graph signals are of increasing interest but do not have the regular structure that would make use of CNNs appealing , it is pertinent to ask the question of what should be an appropriate generalization of CNNs to graphs and the graph signals whose topology they describe ( Bronstein et al. , 2017 ) . If one accepts the value of convolutions as prima facie , a natural solution is to replace convolutions with graph shift invariant filters which are known to be valid generalizations of ( convolutional ) time invariant filters ( Bruna et al. , 2014 ) . This idea is not only natural but has been demonstrated to work well in practical implementations of Graph Neural Networks ( GNNs ) ( Defferrard et al. , 2016 ; Gama et al. , 2019 ; Gilmer et al. , 2017 ; Henaff et al. , 2015 ; Kipf & Welling , 2017 ) . Same as Euclidean scattering transforms , our graph scattering transforms differ from GNNs in that they do not have to be trained . The advantages and limitations of the absence of training notwithstanding , our work also sheds light on the question of why graph convolutions are appropriate generalizations of regular domain convolutions for signal classification problems . Our work suggests that the value of GNNs stems from their stability relative to deformations of the underlying domain that are close to permutations – which is the property that a pair of graphs must satisfy to have small diffusion distance . The stability results obtained in this paper build on the notion of scattering transforms . These scattering representations were introduced by Mallat ( 2012 ) and further developed in Bruna & Mallat ( 2013 ) with computer vision applications . Since , these representations have been extended to handle transformations on more complex groups , such as roto-translations ( Sifre & Mallat , 2013 ; Oyallon & Mallat , 2015 ) , and to domains such as audio processing ( Andén & Mallat , 2014 ) and quantum chemistry ( Eickenberg et al. , 2017 ) . Similarly as in this work , extensions of scattering to general graphs have been considered in Chen et al . ( 2014 ) and Zou & Lerman ( 2018 ) . Chen et al . ( 2014 ) focuses on Haar wavelets that hierarchically coarsen the graph , and relies on building multiresolution pairings . The recent Zou & Lerman ( 2018 ) is closest to our work . There , the authors define graph scattering using spectrally constructed wavelets from ( Hammond et al. , 2011 ) , and establish some properties of the resulting representation , such as energy conservation and stability to spectral perturbations . In contrast , our stability results are established with respect to diffusion metric perturbations , which are generally weaker , in the sense that they define a weaker topology ( see Section 3 ) . We use diffusion wavelets ( Coifman & Maggioni , 2006 ) to obtain multi-resolution graph filter banks that are localized in frequency as well as in the graph domain , while spanning the whole spectrum . Diffusion wavelets serve as the constructive basis for the obtained stability results . Our work is also closely related to recent analysis of stability of Graph Neural Networks in the context of surface representations in ( Kostrikov et al. , 2017 ) . In our work , however , we do not rely on extrinsic deformations and exploit the specific multiresolution structure of wavelets . 2 3 PROBLEM SET-UP . This section introduces our framework and states the desired stability properties of signal representations defined on general non-Euclidean domains . 3.1 EUCLIDEAN STABILITY TO DEFORMATIONS WITH SCATTERING . Motivated by computer vision applications , our analysis starts with the notion of deformation stability . If x ( u ) ∈ L2 ( Ω ) is an image defined over an Euclidean domain Ω ⊂ Rd , we are interested in signal representations Φ : L2 ( Ω ) → RK that are stable with respect to small deformations . If xτ ( u ) : = x ( u− τ ( u ) ) denotes a change of variables with a differentiable field τ : Ω→ Ω such that ‖∇τ‖ < 1 , then we ask ∀ x , τ , ‖Φ ( x ) − Φ ( xτ ) ‖ . ‖x‖‖τ‖ , with ( 1 ) ‖τ‖ : = ‖∇τ‖∞ denoting a uniform bound on the operator norm of ∇τ . In this setting , a notorious challenge to achieving ( 1 ) while keeping enough discriminative power in Φ ( x ) is to transform the high-frequency content of x in such a way that it becomes stable . Scattering transforms ( Mallat , 2012 ; Bruna & Mallat , 2013 ) provide such representations by cascading wavelet decompositions with pointwise modulus activation functions . We briefly summarize here their basic definition . Given a mother wavelet ψ ∈ L1 ( Ω ) with at least a vanishing moment ∫ ψ ( u ) du = 0 and with good spatial localization , we consider rotated and dilated versions ψj , c ( u ) = 2 −jdψ ( 2−jRcu ) using scale parameter j and angle θ ∈ { 2πc/C } c=0 , ... , C−1 . A wavelet decomposition operator is defined as a filter bank spanning all scales up to a cutoff 2J and all angles : ΨJ : x 7→ ( x ∗ ψj , c ) j≤J , c≤C . This filter bank is combined with a pointwise modulus activation function ρ ( z ) = |z| , as well as a low-pass average pooling operator U computing the average over the domain . The resulting representation using m layers becomes Φ ( x ) = { S0 ( x ) , S1 ( x ) , . . . , Sm−1 ( x ) } , with ( 2 ) Sk ( x ) = UρΨJρ . . .ΨJx = { U ( ||x ∗ ψα1 | ∗ ψα2 | · · · ∗ ψαk | ) ; } α1 , ... , αk ( k = 0 , . . . , m− 1 ) . The resulting signal representation has the structure of a CNN , in which feature maps are not recombined with each other , and trainable filters are replaced by multiscale , oriented wavelets . It is shown in Mallat ( 2012 ) that for appropriate signal classes and wavelet families , the resulting scattering transform satisfies a deformation stablity condition of the form ( 1 ) , which has been subsequently generalised to broader multiresolution families ( Wiatowski & Bölcskei , 2018 ) . In essence , the mechanism that provides stability is to capture high-frequency information with the appropriate spatio-temporal tradeoffs , using spatially localized wavelets . 3.2 DEFORMATIONS AND METRIC STABILITY . Whereas deformations provide the natural framework to describe geometric stability in Euclidean domains , their generalization to non-Euclidean , non-smooth domains is not straightforward . Let x ∈ L2 ( X ) . If X is embedded into a low-dimension Euclidean space Ω ⊂ Rd , such as a 2- surface within a three-dimensional space , then one can still define meaningful deformations on X via extrinsic deformations of Ω ( Kostrikov et al. , 2017 ) . However , in this work we are interested in intrinsic notions of geometric stability , that do not necessarily rely on a pre-existent low-dimensional embedding of the domain . The change of variables ϕ ( u ) = u− τ ( u ) defining the deformation can be seen as a perturbation of the Euclidean metric in L2 ( Rd ) . Indeed , 〈xτ , yτ 〉L2 ( Rd , µ ) = ∫ Rd xτ ( u ) yτ ( u ) dµ ( u ) = ∫ Rd x ( u ) y ( u ) |I −∇τ ( u ) |dµ ( u ) = 〈x , y〉L2 ( Rd , µ̃ ) , with dµ̃ ( u ) = |I − ∇τ ( u ) |dµ ( u ) , and |I − ∇τ ( u ) | ≈ 1 if ‖∇τ‖ is small , where I is the identity . Therefore , a possible way to extend the notion of deformation stability to general domains L2 ( X ) is to think of X as a metric space and reason in terms of stability of Φ : L2 ( X ) → RK to metric changes in X . This requires a representation that can be defined on generic metric spaces , as well as a criteria to compare how close two metric spaces are . 3

Summary: The paper presents in interesting and new analysis of stability of scattering transforms on graphs, when the domain (graph) is perturbed by deformations. It combines key ingredients of scattering transforms (extended here to graphs through graph diffusion wavelets), deformation of graphs (based on graph diffusion distances) and a theoretical stability analysis. Similarly to the Euclidian domain, it is shown that linear filters cannot provide representations that are simultaneously rich and stable.