Graph Convolution with Low-rank Learnable Local Filters

1 Introduction . Deep methods have achieved great success in visual cognition , yet they still lack capability to tackle severe geometric transformations such as rotation , scaling and viewpoint changes . This problem is often handled by conducting data augmentations with these geometric variations included , e.g . by randomly rotating images , so as to make the trained model robust to these variations . However , this would remarkably increase the cost of training time and model parameters . Another way is to make use of certain underlying structures of objects , e.g . facial landmarks ( Chen et al. , 2013 ) and human skeleton landmarks ( Vemulapalli et al. , 2014a ) , c.f . Fig . 1 ( right ) . Nevertheless , these methods then adopt hand-crafted features based on landmarks , which greatly constrains their ability to obtain rich features for downstream tasks . One of the main obstacles for feature extraction is the non-Euclidean property of underlying structures , and particularly , it prohibits the direct usage of prevalent convolutional neural network ( CNN ) architectures ( He et al. , 2016 ; Huang et al. , 2017 ) . Whereas there are recent CNN models designed for non-Euclidean grids , e.g. , for spherical mesh ( Jiang et al. , 2019 ; Cohen et al. , 2018 ; Coors et al. , 2018 ) and manifold mesh in computer graphics ( Bronstein et al. , 2017 ; Fey et al. , 2018 ) , they mainly rely on partial differential operators which only can be calculated precisely on fine and regular mesh , and may not be applicable to the landmarks which are irregular and course . Recent works have also applied Graph Neural Network ( GNN ) approaches to coarse non-Euclidean data , yet methods using GCN ( Kipf & Welling , 2016 ) may fall short of model capacity , and other methods adopting GAT ( Veličković et al. , 2017 ) are mostly heuristic and lacking theoretical analysis . A detailed review is provided in Sec . 1.1 . In this paper , we propose a graph convolution model , called L3Net , originating from lowrank graph filter decomposition , c.f . Fig . 1 ( left ) . The model provides a unified framework for graph convolutions , including ChebNet ( Defferrard et al. , 2016 ) , GAT , EdgeNet ( Isufi et al. , 2020 ) and CNN/geometrical CNN with low-rank filter as special cases . In addition , we theoretically prove that L3Net is strictly more expressive to represent graph signals than spectral graph convolutions based on global adjacency/graph Laplacian matrices , which is then empirically validated , c.f . Sec . 3.1 . We also prove a Lipschitz-type representation stability of the new graph convolution layer using perturbation analysis . Because our model allows neighborhood specialized local graph filters , regularization may be needed to prevent over-fitting , so as to handle changing underlying graph topology and other graph noise , e.g. , inaccurately detected landmarks or missing landmark points due to occlusions . Therefore , we also introduce a regularization scheme based on local graph Laplacians , motivated by the eigen property of the latter . This further improves the representation stability aforementioned . The improved performance of L3Net compared to other GNN benchmarks is demonstrated in a series of experiments , and with the the proposed graph regularization , our model shows robustness to a variety of graph data noise . In summary , the contributions of the work are the following : • We propose a new graph convolution model by a low-rank decomposition of graph filters over trainable local basis , which unifies several previous models of both spectral and spatial graph convolutions . • Regularization by local graph Laplacians is introduced to improve the robustness against graph noise . • We provide theoretical proof of the enlarged expressiveness for representing graph signals and the Lipschitz-type input-perturbation stability of the new graph convolution model . • We demonstrate with applications to object recognition of spherical data and facial expression/skeleton-based action recognition using landmarks . Model robustness against graph data noise is validated on both real-world and simulated datasets . 1.1 Related Works . Modeling on face/body landmark data . Many applications in computer vision , such as facial expression recognition ( FER ) and skeleton-based action recognition , need to extract high-level features from landmarked data which are sampled at irregular grid points on human face or at body joints . While CNN methods ( Guo et al. , 2016 ; Ding et al. , 2017 ; Meng et al. , 2017 ) prevail in FER task , landmark methods have the potential advantage in lighter model size as well as more robustness to previously mentioned geometric transformations like pose variation . Earlier methods based on facial landmarks used hand-crafted features ( Jeong & Ko , 2018 ; Morales-Vargas et al. , 2019 ) rather than deep networks . Skeleton-based methods in action recognition have been developed intensively recently ( Ren et al. , 2020 ) , including non-deep methods ( Vemulapalli et al. , 2014b ; Wang et al. , 2012 ) and deep methods ( Ke et al. , 2017 ; Kim & Reiter , 2017 ; Liu et al. , 2016 ; Yan et al. , 2018 ) . Facial and skeleton landmarks only give a coarse and irregular grid , and then mesh-based geometrical CNN ’ s are hardly applicable , while previous GNN models on such tasks may lack sufficient expressive power . Graph convolutional network . A systematic review can be found in several places , e.g . Wu et al . ( 2020 ) . Spectral graph convolution was proposed using full eigen decomposition of the graph Laplacian in Bruna et al . ( 2013 ) , Chebyshev polynomial in ChebNet ( Defferrard et al. , 2016 ) , by Cayley polynomials in Levie et al . ( 2018 ) . GCN ( Kipf & Welling , 2016 ) , the mostly-used GNN , is a variant of ChebNet using degree-1 polynomial . Liao et al . ( 2019 ) accelerated the spectral computation by Lanczos algorithm . Graph scattering transform has been developed using graph wavelets ( Zou & Lerman , 2020 ; Gama et al. , 2019b ) , which can be constructed in the spectral domain ( Hammond et al. , 2011 ) and by diffusion wavelets ( Coifman & Maggioni , 2006 ) . The scattering transform enjoys theoretical properties of the representation but lacks adaptivity compared to trainable neural networks . Spatial graph convolution has been performed by summing up neighbor nodes ’ transformed features in NN4G ( Scarselli et al. , 2008 ) , by graph diffusion process in DCNN ( Atwood & Towsley , 2016 ) , where the graph propagation across nodes is by the adjacency matrix . Graph convolution with trainable filter has also been proposed in several settings : MPNN ( Gilmer et al. , 2017 ) enhanced model expressiveness by message passing and sub-network ; GraphSage ( Hamilton et al. , 2017 ) used trainable differential local aggregator functions in the form of LSTM or mean/max-pooling ; GAT ( Veličković et al. , 2017 ) and variants ( Li et al. , 2018 ; Zhang et al. , 2018 ; Liu et al. , 2019 ) introduced attention mechanism to achieve adaptive graph affinity , which remains non-negative valued ; EdgeNet ( Isufi et al. , 2020 ) developed adaptive filters by taking products of trainable local filters . Our model learns local filters which can take negative values and contains GAT and EdgeNet as special cases . Theoretically , expressive power of GNN has been studied in Morris et al . ( 2019 ) ; Xu et al . ( 2019 ) ; Maron et al . ( 2019a ; b ) ; Keriven & Peyré ( 2019 ) , mainly focusing on distinguishing graph topologies , while our primary concern is to distinguish signals lying on a graph . CNN and geometrical CNN . Standard CNN applies local filters translated and shared across locations on an Euclidean domain . To extend CNN to non-Euclidean domains , convolution on a regular spherical mesh using geometrical information has been studied in S2CNN ( Cohen et al. , 2018 ) , SphereNet ( Coors et al. , 2018 ) , SphericalCNN ( Esteves et al. , 2018 ) , and UGSCNN ( Jiang et al. , 2019 ) , and applied to 3D object recognition , for which other deep methods include 3D convolutional ( Qi et al. , 2016 ) and non-convolutional architectures ( Qi et al. , 2017a ; b ) . CNN ’ s on manifolds construct weight-sharing across local atlas making use of a mesh , e.g. , by patch operator in Masci et al . ( 2015 ) , anisotropic convolution in ACNN ( Boscaini et al. , 2016 ) , mixture model parametrization in MoNet ( Monti et al. , 2017 ) , spline functions in SplineCNN ( Fey et al. , 2018 ) , and manifold parallel transport in Schonsheck et al . ( 2018 ) . These geometric CNN models use information of non-Euclidean meshes which usually need sufficiently fine resolution . 2 Method . 2.1 Decomposed local filters . Consider an undirected graph G = ( V , E ) , |V | = n. A graph convolution layer maps from input node features X ( u′ , c′ ) to output Y ( u , c ) , where u , u′ ∈ V , c′ ∈ [ C ′ ] ( c ∈ [ C ] ) is the input ( output ) channel index , the notation [ m ] means { 1 , · · · , m } , and Y ( u , c ) = σ ( ∑ u′∈V , c′∈ [ C′ ] M ( u′ , u ; c′ , c ) X ( u′ , c′ ) + bias ( c ) ) , u ∈ V , c ∈ [ C ] . ( 1 ) The spatial and spectral graph convolutions correspond to different ways of specifying M , c.f . Sec . 2.3 . The proposed graph convolution is defined as M ( u′ , u ; c′ , c ) = K∑ k=1 ak ( c ′ , c ) Bk ( u ′ , u ) , ak ( c ′ , c ) ∈ R , ( 2 ) where Bk ( u ′ , u ) is non-zero only when u′ ∈ N ( dk ) u , N ( d ) u denoting the d-th order neighborhood of u ( i.e. , the set of d-neighbors of u ) , and K is a fixed number . In other words , Bk ’ s are K basis of local filters around each u , and the order dk can differ with 1 ≤ k ≤ K. Both ak and Bk are trainable , so the number of parameters are K ·CC ′ + ∑K k=1 ∑ u∈V |N ( dk ) u | ∼ K ·CC ′ +Knp , where p stands for the average local patch size . In our experiments we use K up to 5 , and dk up to 3 . We provide the matrix notation of ( 2 ) in Appendix A.1 . The construction ( 2 ) can be used as a layer type in larger GNN architectures . Pooling of graphs can be added between layers , and see Appendix C.5 for further discussion on multiscale model . The choice of K and neighborhood orders ( d1 , · · · , dK ) can also be adjusted accordingly . The model may be extended in several ways to be discussed in the last section .

Summary: This paper proposed L3Net which is a new graph convolution with decomposing the learnable local filters into low-rank. It can contain both spatial and spectral graph convolution (including ChebNet, GAT, EdgeNet and so on) as subsets. It is also robust to graph noise. Experiments are conducted on mesh data, facial recognition and action recognition, indicating out-performed performance over baselines. Its robustness to graph noise is also tested.