Multiresolution Matrix Factorization and Wavelet Networks on Graphs
Abstract
Multiresolution Matrix Factorization (MMF) is unusual amongst fast matrix factorization algorithms in that it does not make a low rank assumption.
This makes MMF especially well suited to modeling certain types of graphs with complex multiscale or hierarchical strucutre.
While MMF promises to yields a useful wavelet basis, ﬁnding the factorization itself is hard, and existing greedy methods tend to be brittle.
In this paper, we propose a “learnable” version of MMF that carfully optimizes the factorization with a combination of reinforcement learning and Stiefel manifold optimization through backpropagating errors.
We show that the resulting wavelet basis far outperforms prior MMF algorithms and provides the ﬁrst version of this type of factorization that can be robustly deployed on standard learning tasks.
Furthermore, we construct the wavelet neural networks (WNNs) learning graphs on the spectral domain with the wavelet basis produced by our
MMF learning algorithm.
Our wavelet networks are competitive against other state-of-the-art methods in molecular graphs classiﬁcation and node classiﬁcation on citation graphs.
1. Introduction
In certain machine learning problems large matrices have complex hierarchical structures that traditional linear algebra methods based on the low rank assumption struggle to capture.
Multiresolution matrix factorization (MMF) is a relatively little used alternative paradigm that is designed to capture structure at multiple different scales.
MMF has been found to be particularly effective at compressing the adjacency or Laplacian matrices of graphs with complicated structure, such as social networks (Kondor et al., 2014).
MMF factorizations have a number of advantages, including the fact that they are easy to invert and have an interpretation as a form of wavelet analysis on the matrix and consequently on the underlying graph.
The wavelets can be used e.g., for ﬁnding sparse approximations of graph signals.
Finding the actual MMF factorization however is a hard optimization problem combining elements of continuous and combinatorial optimization.
Most of the existing MMF algorithms just tackle this with a variety of greedy heuristics and are consequently brittle: the resulting factorizations typically have large variance and most of the time yield factorizations that are far from the optimal (Teneva et al., 2016;
Ithapu et al., 2017; Ding et al., 2017).
The present paper proposes an alternative paradigm to MMF optimization based on ideas from deep learning.
Speciﬁcally, we employ an iterative approach to optimizing the factorization based on backpropagating the factorization error and a reinforcement learning strategy for solving the combinatorial part of the problem.
While more expensive than the greedy approaches, we ﬁnd that the resulting “learnable” MMF produces much better quality factorizations and a wavelet basis that is smoother and better reﬂects the structure of the underlying matrix or graph.
Unsurprisingly, this also means that the factorization performs better in downstream tasks.
To apply our learnable MMF algorithm to standard benchmark tasks, we also propose a wavelet extension of the
Spectral Graph Networks algorithm of (Bruna et al., 2014) which we call the Wavelet Neural Network (WNN).
Our experiments show that the combination of learnable MMF optimization with WNNs achieves state of the art results on several graph learning tasks.
Beyond just benchmark performance, the greatly improved stability of MMF optimization process and the similarity of the hierarchical structure of the factorization to the architecture of deep neural networks opens up the possibility of MMF being tightly integrated with other learning algorithms in the future.
2. Related work
Compressing and estimating large matrices has been extensively studied from various directions, including (Drineas et al., 2006), (Halko et al., 2011), (Williams & Seeger, 2001) (Kumar et al., 2012), (Mahoney, 2011), (Jenatton et al., 2010).
Many of these methods come with explicit guarantees but typically make the assumption that the matrix to be approximated is low rank.
MMF is more closely related to other works on constructing wavelet bases on discrete spaces, including wavelets deﬁned based on diagonalizing the diffusion operator or the normalized graph Laplacian (Coifman & Maggioni, 2006)
(Hammond et al., 2011) and multiresolution on trees (Gavish et al., 2010)
(Lee et al., 2008).
MMF has been used for matrix compression (Teneva et al., 2016), kernel approximation (Ding et al., 2017) and inferring semantic relationships in medical imaging data (Ithapu et al., 2017).
Graph neural networks (GNNs) utilizing the generalization of convolution concept to graphs have been popularly applied to many learning tasks such as estimating quantum chemical computation, and modeling physical systems, etc.
Spectral methods such as (Bruna et al., 2014) provide one way to deﬁne convolution on graphs is via convolution theorem and graph Fourier transform (GFT).
To address the high computational cost of GFT, (Xu et al., 2019) proposed to use the diffusion wavelet bases as previously deﬁned by (Coifman & Maggioni, 2006) instead for a faster transformation.
3. Background on Multiresolution Matrix Factorization
The Multiresolution Matrix Factorization (MMF) of a matrix A ∈ R n × n is a factorization of the form
[Equation]
, where the H and U 1 ,..., U L matrices conform to the following constraints: (i) Each U (cid:96) is an orthogonal matrix that is a k -point rotation for some small k , meaning that it only rotates k coordinates at a time; (ii) There is a nested sequence of sets
S L ⊆ ⋯ ⊆
S 1 ⊆ S
= [ n ] such that the coordinates rotated by U (cid:96) are a subset of S (cid:96) ; and (iii) H is an S L -core-diagonal matrix meaning that is diagonal with a an additional small S L × S L dimensional “core”.
Finding the best MMF factorization to a symmetric matrix A involves solving
[Equation]
Assuming that we measure error in the Frobenius norm, (19) is equivalent to
[Equation]
where ∣∣ ⋅ ∣∣ 2 resi is the squared residual norm
∣∣ H ∣∣ 2 resi =
∑ i ≠ j ; ( i,j )/∈
S L × S L ∣ H i,j ∣ 2 .
There are two fundamental difﬁculties in MMF optimization: ﬁnding the optimal nested sequence of S (cid:96) is a combinatorially hard (e.g., there are ( d (cid:96) k ) ways to choose k indices out of S (cid:96) ); and the solution for U (cid:96) must satisfy the orthogonality constraint such that
U T(cid:96)
U (cid:96) =
I .
The existing literature on solving this optimization problem (Kondor et al., 2014) (Teneva et al., 2016)
(Ithapu et al., 2017) (Ding et al., 2017) has various heuristic elements and has a number of limitations: • There is no guarantee that the greedy heuristics (e g., clustering) used in selecting k rows/columns I (cid:96) =
{ i 1 ,..,i k } ⊂ S (cid:96) for each rotation return a globally optimal factorization.
• Instead of direct optimization for each rotation U (cid:96) ≜
I n − k ⊕
I (cid:96) O (cid:96) where O (cid:96) ∈
SO ( k ) globally and simultaneously with the objective (19), Jacobi MMFs (see Proposition 2 of (Kondor et al., 2014)) apply the greedy strategy of optimizing them locally and sequentially.
Again, this does not necessarily lead to a globally optimal combination of rotations.
• Most MMF algorithms are limited to the simplest case of k = 2 where U (cid:96) is just a Given rotation, which can be parameterized by a single variable, the rotation angle θ (cid:96) .
This makes it possible to optimize the greed objective by simple gradient descent, but larger rotations would yield more expressive factorizations and better approximations.
In contrast we propose an iterative algorithm to directly optimize the global MMF objective (19):
• We use gradient descent algorithm on the Stiefel manifold to optimize all rotations { U (cid:96) } L(cid:96)
= 1 simultaneously , whilst satisfying the orthogonality constraints.
Importantly, the Stiefel manifold optimization is not limited to k = 2 case (Section 4).
• We formulate the problem of ﬁnding the optimal nested sequence S L ⊆ ⋯ ⊆
S 1 ⊆ S
= [ n ] as learning a Markov Decision Process (MDP) that can be subsequently solved by the gradient policy method of Reinforcement Learning (RL), in which the RL agent (or stochastic policy) is modeled by graph neural networks (GNN) (Section 5).
We show that the resulting learning-based MMF algorithm outperforms existing greedy MMFs and other traditional baselines for matrix approximation in various scenarios (see
Section 7).
4. Stiefel Manifold Optimization
The MMF optimization problem in (19) and (2) is equivalent to
[Equation]
In order to solve the inner optimization problem of (3), we consider the following generic optimization with orthogonality constraints:
[Equation]
where I p is the identity matrix and F( X ) ∶ R n × p
→
R is a differentiable function.
The feasible set V p ( R n ) = { X ∈
R n × p ∶ X T X = I p } is referred to as the Stiefel manifold of p orthonormal vectors in R n that has dimension equal to np − 12 p ( p + 1 ) .
We will view V p ( R n ) as an embedded submanifold of R n × p .
When there is more than one orthogonal constraint, (24) is written as
[Equation]
where there are q variables with corresponding q orthogonal constraints.
For example, in the MMF optimization problem (19), suppose we are already given S L ⊆ ⋯ ⊆
S 1 ⊆ S
= [ n ] meaning that the indices of active rows/columns at each resolution were already determined, for simplicity.
In this case, we have q
=
L number of variables such that each variable X (cid:96) =
O (cid:96) ∈ R k × k , where U (cid:96) =
I n − k ⊕
I (cid:96) O (cid:96) ∈
R n × n in which I (cid:96) is a subset of k indices from S (cid:96) , must satisfy the orthogonality constraint.
The corresponding objective function is
[Equation]
Details about Stiefel manifold optimization is included in the Appendix.
5. Reinforcement Learning 5.1. Problem formulation
We formulate the problem of ﬁnding the optimal nested sequence of sets S L ⊆ ⋯ ⊆
S 1 ⊆ S
= [ n ] as learning an
RL agent in order to solve the MMF optimization in (19).
There are two fundamental parts to index selection for each resolution level (cid:96) ∈ { 1 ,..,L } :
• Select k indices I (cid:96) =
{ i 1 ,..,i k } ⊂ S (cid:96) − 1 to construct the corresponding rotation matrix U (cid:96) (see Section 4).
• Select the set of indices T (cid:96) ⊂ S (cid:96) − 1 of rows/columns that are to be wavelets at this level, and then be eliminated by setting S (cid:96) =
S (cid:96) − 1 ∖ T (cid:96) .
To reduce the computational cost, we assume that each resolution level has only one row/column to be selected as the wavelet (e.g., a single wavelet) such that ∣ T (cid:96) ∣ = 1 .
That means the cardinality of S (cid:96) reduces by 1 after each level, d (cid:96) = n − (cid:96) , and size of the core block of H is ( n − L ) × ( n − L ) that corresponds to exactly n − L active rows/columns at the end.
5.2. Markov Decision Process
A key task for building our model is to specify our index selection procedure.
We design an iterative index selection process and formulate it as a general decision process M =
( S,A,P,R,γ ) as follows.
S is the set of states (or state space) that consists of all possible intermediate and ﬁnal states in which each state s ∈ S is a tuple of ( A , S ,(cid:96) ) where (cid:96) indicates the resolution level, S indicates the set of active row/column indices, and
A =
A S , S indicates the sub-matrix of A with indices of rows and columns are from S (e.g., ∣ S ∣
=
d (cid:96) , A ∈ R d (cid:96) × d (cid:96) ).
We start at state s 0 =
( A , [ n ] , 0 ) where A is the input matrix that MMF tries to factorize (e.g., no rows/columns removed yet) and [ n ] indicates all rows/columns are still active.
The set of terminal (ﬁnal) states S ∗ ⊂ S includes every state s ∗ that has (cid:96) =
L .
A is the set of actions that describe the modiﬁcation made to current state at each time step.
An action a ∈
A validly applied to a non-terminal state s =
( A S , S , S ,(cid:96) ) is a tuple ( I , T ) where I =
{ i 1 ,..,i k } ⊂ S is the set of k indices corresponding to the rotation matrix U (cid:96) + 1 , and T ⊂ S is the set of wavelet indices to spit out at this level.
This action transforms the state into the next one s ′ =
( A S ′ , S ′ , S ′ ,(cid:96) + 1 ) where S ′ =
S ∖ T meaning the set of active indices gets shrinked further.
The action is called invalid for the current state if and only if I /⊂ S or T /⊂ S .
P is the transition dynamics that speciﬁes the possible outcomes of carrying out an action at time t in which p ( a (cid:96) ∣ s (cid:96) ,..,s 0 ) is represented as a parameterized policy network π θ with learnable parameters θ .
The whole trajectory is always started by the same s 0 and ﬁnished by a terminal state after exactly L transitions:
[Equation]
that constructs the nested sequence.
R ( s ∗ ) is the reward function that speciﬁes the reward after reaching a terminal state s ∗ .
The reward function is deﬁned as negative of the MMF reconstruction loss such that
[Equation]
We want to maximize this ﬁnal reward that is equivalent to minimize error of MMF in Frobenius norm (as in problem (19)).
Evaluation of the reward requires the Stiefel manifold optimization (see Section 4) for rotations { U (cid:96) } L(cid:96)
= 1 .
Obviously, the ﬁnal reward in Eq.
(7) is the most important.
However, to improve the training quality of the policy, we can deﬁne the intermediate reward R ( s ) for non-terminal states s =
( A S , S , S ,(cid:96) ) /∈
S ∗ as the immediate improvement of the (cid:96) -th resolution ( L > (cid:96) > 0 ):
[Equation]
Along the trajectory ( s 0 ,s 1 ,..,s L ) , we generate the corresponding sequence of rewards ( r 1 ,r 2 ,..,r L ) based on (7, 8).
γ is the discount factor, a penalty to uncertainty of future rewards, 0 < γ ≤ 1 .
We deﬁne the return or discounted future reward g (cid:96) for (cid:96) = 0 ,..,L − 1 as g (cid:96) =
L − (cid:96) − 1 ∑ k =γ k r (cid:96) + k + 1 , (9) which in the case of γ = 1 indicates simply accumulating all the immediate rewards and the ﬁnal reward along the trajectory.
5.3. Graph convolutional policy network
In this section, we design our policy network π θ as a graph neural network (GNN) with the message passing scheme.
We consider the symmetric matrix A being represented by a weighted undirected graph G =
( V,E ) in which V is the set of nodes such that each node corresponds to a row/column of A , and E is the set of edges such that the edge ( i,
j ) ∈ E has the weight
A i,j .
As deﬁned in Section 5.2, a state s ∈ S is a tuple ( A S , S , S ,(cid:96) ) in which A S , S is the sub-matrix restricted to the active rows/columns S , and an action a ∈ A is a tuple ( I , T ) in which I is the set of k indices corresponding to the ( (cid:96) + 1 ) -th
rotation and T is the set of indices to spit out as wavelets.
Practically, a state can be simply represented by a single binary vector such that if a bit is 1 then the corresponding index is active, without the need to explicitly storage matrix A S , S that can be efﬁciently constructed from A by any numerical toolkit.
Our GNN policy network π θ ( a ∣ s ) learns to encode the underlying graph represented by A S , S and returns a sample of valid action such that T ⊂ I ⊂ S .
In Section 5.1, we assume that
T contains only a single index that we will call as the pivot i ∗ (e.g., T = { i ∗ } ).
Thus, the task of our GNN model is to learn to select the pivot i ∗ ﬁrst, and then select the rest k −indices of I that are highly correlated to the pivot.
The simplest implementation of GNNs is Message Passing
Neural Networks (MPNNs) (Gilmer et al., 2017).
Suppose that the node embeddings (messages) M 0 ∈ R N × F are initialized by the input node features where N is the number of nodes and F is the number of features for each node.
Iteratively, the messages are propagated from each node to its neighborhood, and then transformed by a combination of linear transformations and non-linearities, e.g.,
[Equation]
where A is the adjacency matrix; ˆ M t and M t ∈ R N × D are the aggregated messages (by summing over each node’s neighborhood) and the output messages at the t ’th iteration, respectively; σ is a element-wise non-linearity function (e.g., sigmoid, ReLU, etc.); and W s are learnable weight matrices such that W 0 ∈ R F × D and W t ∈ R D
× D for t > 0 .
Basically, the set of learnable parameters θ of our policy network π includes all W s. In some cases, a graph Laplacian L is used instead of the adjacency matrix A in model (10), for example, graph Laplacian L = D − 1 A or its symmetric normalized version ˜ L
= I − D − 1 / 2 AD − 1 / 2 .
One way to incorporate the set of active rows/columns/nodes S into our GNN model is by initializing the input node feature with a binary label ( f = 1 ) such that a node v has labelif the v -th row/column is still active, otherwise 0 .
For a more efﬁcient implementation, we can execute the message passing in the block A S , S only.
Supposing that the message passing scheme is executed for T iterations, we concatenate messages from every iteration together into the ﬁnal embedding:
[Equation]
Model (10) produces the embedding for each node that allows us to deﬁne a sampling procedure to select the pivot i ∗ .
Given the ﬁnal embedding M from Eq.
(11), we deﬁne the probability P i that node i ∈ S is being selected as the pivot as:
[Equation]
where
[Equation]
In order to make the sampling procedure differentiable for backpropagation, we apply the Gumbel-max trick (Gumbel, 1954)
(Maddison et al., 2014) (Jang et al., 2017) that provides a simple and efﬁcient way to draw sample i ∗ as follows:
[Equation]
where G i are i.i.d samples drawn from Gumbel ( 0 , 1 ) .
Technically, the sample is represented by a one-hot vector such that the i ∗ -th element is 1 .
Similarly, T , I and S are represented by vectors in { 0 , 1 } N in which a 1 -element indicates the existence of the corresponding index in the set.
Furthermore, the set union and minus operations (e.g., S ∖ T ) can be easily done by vector addition and subtraction, respectively.
Given the pivot i ∗ , we compute the similarity score between i ∗ and other nodes i ∈ S as C i = ⟨
M i ∗ , ∶ , M i, ∶ ⟩ .
Finally, we sample k − 1 nodes with the highest similarity scores to i sequentially (one-by-one) without replacement by the
Gumbel-max trick, that completes our sampling procedure for action a
= ( I , T ) .
(Williams, 1988) (Williams, 1992) (Sutton et al., 2000) update rule for policy parameters is
[Equation]
where η is the learning rate, that is used in training our policy network π θ in Algorithm 1 (see section Policy gradient methods in the Appendix).
5.4. The learning algorithm
Putting everything together, our MMF learning algorithm is sketched in Algorithm 1.
Iteratively: (1) we sample a trajectory by running the policy network that indicates the indices for rotation and wavelet for each resolution, (2) we apply the Stiefel manifold optimization to ﬁnd the rotations, and (3) we compute the future rewards and update the parameters of the policy network by REINFORCE accordingly.
The learning terminates when we complete ω iterations or episodes.
In all our experiments, the learning algorithm can get to sufﬁcient precision in matrix approximation after ω = 256 episodes.
6. Wavelet Networks on Graphs 6.1. Motivation
The eigendecomposition of the normalized graph Laplacian operator ˜ L
= U T HU can be used as the basis of a graph
Fourier transform.
(Shuman et al., 2013) deﬁnes graph
Fourier transform (GFT) on a graph G =
( V,E ) of a graph signal f ∈ R n (that is understood as a function f
∶ V → R deﬁned on the vertices of the graph) as ˆ f = U T f , and the inverse graph Fourier transform as f = U ˆ f .
Analogously to the classical Fourier transform, GFT provides a way to represent a graph signal in two domains: the vertex domain and the graph spectral domain; to ﬁlter graph signal according to smoothness; and to deﬁne the graph convolution operator, denoted as ∗ G :
[Equation]
where g denotes the convolution kernel, and ⊙ is the element-wise Hadamard product.
If we replace the vector U T g by a diagonal matrix ˜ g , then we can rewrite the
Hadamard product in Eq.
(13) to matrix multiplication as
U ˜ gU T f (that is understood as ﬁltering the signal f by the ﬁlter ˜ g ).
Based on GFT, (Bruna et al., 2014) and (Defferrard et al., 2016) construct convolutional neural networks (CNNs) learning on spectral domain for discrete structures such as graphs.
However, there are two fundamental limitations of GFT:
• High computational cost: eigendecomposition of the graph Laplacian has complexity O ( n 3 ) , and “Fourier transform” itself involves multiplying the signal with a dense matrix of eigenvectors.
• The graph convolution is not localized in the vertex domain, even if the graph itself has well deﬁned local communities.
To address these limitations, we propose a modiﬁed spectral graph network based on the MMF wavelet basis rather than the eigenbasis of the Laplacian.
This has the following advantages: (i) the wavelets are generally localized in both vertex domain and frequency, (ii) the individual basis transforms are sparse, and (iii) MMF provides a computationally efﬁcient way of decomposing graph signals into components at different granularity levels and an excellent basis for sparse approximations.
6.2. Network construction
In the case A is the normalized graph Laplacian of a graph
G =
( V,E ) ,
the wavelet transform (up to level L ) expresses a graph signal (function over the vertex domain)
f ∶ V → R , without loss of generality f ∈ V 0 , as:
[Equation]
for each v ∈ V, where α (cid:96)m = ⟨ f,ψ (cid:96)m ⟩ and β m = ⟨ f,φ Lm ⟩ are the wavelet coefﬁcients.
At each level, a set of coordinates T (cid:96) ⊂ S (cid:96) − 1 are selected to be the wavelet indices, and then to be eliminated from the active set by setting S (cid:96) =
S (cid:96) − 1 ∖ T (cid:96) (see Section 5.1).
Practically, we make the assumption that we only select 1 wavelet index for each level (see Section 5.1) that results in a single mother wavelet ψ (cid:96) =
[ A (cid:96) ] i ∗ , ∶ where i ∗ is the selected index (see Section 5.3).
We get exactly L mother wavelets ψ = { ψ 1 ,ψ 2 ,...,ψ L } .
On the another hand, the active rows of H =
A L make exactly N − L father wavelets φ = { φ Lm = H m, ∶ } m ∈ S L .
In total, a graph of N vertices has exactly N wavelets (both mothers and fathers).
Analogous to the convolution based on GFT (Bruna et al., 2014), each convolution layer k = 1 ,..,K of our wavelet network transforms an input vector f ( k − 1 ) of size ∣ V ∣× F k −into an output f ( k ) of size ∣ V ∣ × F k as
[Equation]
j
[Equation]
[Equation]
[Equation]
[Equation]
where W is our wavelet basis matrix as we concatenate φ and ψ column-by-column, g ( k ) i,j is a parameter/ﬁlter in the form of a diagonal matrix learned in spectral domain, and σ is an element-wise linearity (e.g., ReLU, sigmoid, etc.).
For example, in node classiﬁcation tasks, assume the number of classes is C , the set of labeled nodes is V label , and we are given a normalized graph Laplacian ˜ L and an input node feature matrix f ( 0 ) .
First of all, we apply our MMF learning algorithm 1 to factorize ˜ L and produce our wavelet basis matrix W .
Then, we construct our wavelet network as a multi-layer CNNs with each convolution is deﬁned as in
Eq.
(14) that transforms f ( 0 ) into f ( K ) after K layers.
The top convolution layer K -th returns exactly F K = C features and uses softmax instead of the nonlinearity σ for each node. ▷ Section 5.3. 9: s
Given { I (cid:96) } L(cid:96)
1 , minimize objective ( ) by Stiefel manifold optimization to ﬁnd ▷ U (cid:96) = I n − k ⊕
I (cid:96) O (cid:96) 12: do 13:
Estimate
14: θ
Terminate
17: end while
The loss is the cross-entropy error over all labeled nodes as:
[Equation]
where y v,c is a binary indicator that is equal to 1 if node v is labeled with class c , and 0 otherwise.
The set of weights { g ( k ) }
Kk = 1 are trained using gradient descent optimizing the loss in Eq.
7. Experiments 7.1. Matrix factorization
We evaluate the performance of our MMF learning algorithm in comparison with the original greedy algorithm (Kondor et al., 2014) and the Nystr ¨ om method (Gittens & Mahoney, 2013) in the task of matrix factorization on 3 datasets: (i) normalized graph Laplacian of the Karate club network ( N = 34 , E = 78 ) (Zachary, 1976); (ii) a Kronecker product matrix ( N = 512 ), K n 1 , of order n = 9 , where
K 1 =
(( 0 , 1 ) , ( 1 , 1 )) is a 2 × 2 seed matrix (Leskovec et al., 2010); and (iii) normalized graph Laplacian of a Cayley tree or Bethe lattice with coordination number
z
= 4 andlevels of depth ( N = 161 ).
The rotation matrix size K are 8 , 16 and 8 for Karate, Kronecker and Cayley, respectively.
Meanwhile, the original greedy MMF is limited to K =and implements an exhaustive search to ﬁnd an optimal pair of indices for each rotation.
For both versions of MMF, we drop c = 1 columns after each rotation, which results in a ﬁnal core size of d L = N − c × L .
The exception is for the
Kronecker matrix ( N = 512 ), our learning algorithm drops up to 8 columns (for example, L = 62 and c = 8 results into d L = 16 ) to make sure that the number of learnable parameters L × K 2 is much smaller the matrix size N 2 .
Our learning algorithm compresses the Kronecker matrix down to 6 − 7% of its original size.
The details of efﬁcient training reinforcement learning with the policy networks implemented by GNNs are included in the Appendix.
For the baseline of Nystr ¨ om method, we randomly select, by uniform sampling without replacement, the same number
d L columns C from A and take out W as the corresponding d L × d L submatrix of A .
The Nystr ¨ om method approximates A ≈ CW † C T .
We measure the approximation error in Frobenius norm.
Figure 1 shows our MMF learning algorithm consistently outperforms the original greedy algorithm and the Nystr¨om baseline given the same number of active columns, d L .
Figure 2 depicts the wavelet bases at different levels of resolution.
7.2. Molecular graphs classiﬁcation
We trained and evaluated our wavelet networks (WNNs) on standard graph classiﬁcation benchmarks including four bioinformatics datasets: (1) MUTAG, which is a dataset of 188 mutagenic aromatic and heteroaromatic nitro compounds with 7 discrete labels (Debnath et al., 1991); (2)
PTC, which consists of 344 chemical compounds withdiscrete labels that have been tested for positive or negative toxicity in lab rats (Toivonen et al., 2003); (3) PROTEINS, which contains 1,113 molecular graphs with binary labels, where nodes are secondary structure elements (SSEs) and there is an edge between two nodes if they are neighbors in the amino-acid sequence or in 3D space (Borgwardt et al., 2005); (4) NCI1, which has 4,110 compounds with binary labels, each screened for activity against small cell lung cancer and ovarian cancer lines (Wale et al., 2008).
Each molecule is represented by an adjacency matrix, and we represent each atomic type as a one-hot vector and use them [Figure] [Figure] as the node features.
We factorize all normalized graph Laplacian matrices in these datasets by MMF with K = 2 to obtain the wavelet bases.
Again, MMF wavelets are sparse and suitable for fast transform via sparse matrix multiplication, with the following average percentages of non-zero elements for each dataset: 19 .
23% (MUTAG), 18 .
18% (PTC), 2 .
26% (PROTEINS) and 11 .
43% (NCI1).
Our WNNs contain 6 layers of spectral convolution, 32 hidden units for each node, and are trained with 256 epochs by
Adam optimization with an initial learning rate of 10 − 3 .
We follow the evaluation protocol of 10-fold cross-validation from (Zhang et al., 2018).
We compare our results to several deep learning methods and popular graph kernel methods.
Baseline results are taken from (Maron et al., 2019).
Our
WNNs outperform 7/8, 7/8, 8/8, and 2/8 baseline methods on MUTAG, PTC, PROTEINS, and NCI1, respectively (see
Table 1).
7.3. Node classiﬁcation on citation graphs
To further evaluate the wavelet bases returned by our learnable MMF algorithm, we construct our wavelet networks (WNNs) as in Sec. 6 and apply it to the task of node classiﬁcation on two citation graphs, Cora ( N = 2 , 708 ) and
Citeseer ( N = 3 , 312 ) (Sen et al., 2008) in which nodes and edges represent documents and citation links.
Each document in Cora and Citeseer has an associated feature vector (of length 1 , 433 resp.
3 , 703 ) computed from word frequencies, and is classiﬁed into one of 7 and 6 classes, respectively.
We factorize the normalized graph Laplacian by learnable MMF with K = 16 to obtain the wavelet bases.
The resulting MMF wavelets are sparse, which makes it possible to run a fast transform on the node features by sparse matrix multiplication: only 4 .
69% and 15 .
25% of elements are non-zero in Citeseer and Cora, respectively.
In constrast,
Fourier bases given by eigendecomposition of the graph
Laplacian are completely dense ( 100% of elements are nonzero).
We evaluate our WNNs with 3 different random splits of train/validation/test: (1) 20% / 20% / 60% denoted as MMF 1 , (2) 40% / 20% / 40% denoted as MMF 2 , and (3) 60% / 20% / 20% denoted as MMF 3 .
The WNN learns to encode the whole graph with 6 layers of spectral convolution and 100 hidden dimensions for each node.
During training, the network is only trained to predict the node labels in the training set.
Hyperparameter searching is done on the validation set.
The number of epochs is
and we use the Adam optimization method (Kingma & Ba, 2015) with learning rate η = 10 − 3 .
We report the ﬁnal test accuracy for each split in Table 2.
We compare with several traditional methods and deep learning methods including other spectral graph convolution net-
[Table]
[Table] works such as Spectral CNN [Table] and graph wavelet neural networks (GWNN).
Baseline results are taken from (Xu et al., 2019).
Our wavelet networks perform competitively against state-of-the-art methods in the ﬁeld
[Table]
8. Conclusions
In this paper we introduced a general algorithm based on reinforcement learning and Stiefel manifold optimization to optimize Multiresolution Matrix Factorization (MMF).
We ﬁnd that the resulting learnable MMF consistently outperforms the existing greedy and heuristic MMF algorithms in factorizing and approximating hierarchical matrices.
Based on the wavelet basis returned from our learning algorithm, we deﬁne a corresponding notion of spectral convolution and construct a wavelet neural network for graph learning problems.
Thanks to the sparsity of the MMF wavelets, the wavelet network can be efﬁciently implemented with sparse matrix multiplication.
We ﬁnd that this combination of learnable MMF factorization and spectral wavelet network yields state of the art results on standard node classiﬁcation and molecular graph classiﬁcation.
A. Notation
We deﬁne [ n ] = { 1 , 2 ,...,n } as the set of the ﬁrst n natural numbers.
We denote I n as the n dimensional identity matrix.
The group of n dimensional orthogonal matrices is SO ( n ) .
A ⊍ B will denote the disjoint union of two sets A and B , therefore A 1 ⊍ A 2 ⊍ ⋅⋅⋅ ⊍ A k = S is a partition of S .
Given a matrix A ∈ R n × n and two sequences of indices i =
( i 1 ,...,i k ) ∈ [ n ] k and j = ( j 1 ,...,j k ) ∈ [ n ] k assuming that i 1 < i 2 < ⋅⋅⋅ < i k and j 1 < j 2 < ⋅⋅⋅ <
j k , A
i , j will be the k × k matrix with entries
[ A i , j ] x,y =
A i x ,j y .
Furthermore, A i, ∶ and
A ∶ ,j denote the i -th row and the j -th column of A , respectively.
Given A 1 ∈ R n 1 × m 1 and A 2 ∈ R n 2 × m 2 , A 1 ⊕ A 2 is the ( n 1 + n 2 ) × ( m 1 + m 2 ) dimensional matrix with entries [Equation]
[Equation]
if i ≤ n 1 and j
[Equation]
m[ A 2 ] i − n 1 ,j − mif i > n 1 and j > m otherwise.
A matrix A is said to be block diagonal if it is of the form
[Equation]
(16) for some sequence of smaller matrices A 1 ,..., A p .
For the generalized block diagonal matrix, we remove the restriction that each block in (16) must involve a contiguous set of indices, and introduce the notation
[Equation]
in which
[Equation]
⎧⎪⎪⎨⎪⎪⎩[ A u ] q,r if i uq = a
and i ur
= b for some u,q,r, otherwise.
The Kronecker tensor product A 1 ⊗ A 2 is an n 1 n 2 × m 1 m 2 matrix whose elements are
[Equation]
, with the obvious generalization to p -fold products A 1 ⊗ A 2 ⊗⋅⋅⋅⊗ A p .
We denote A ⊗ p as the p -fold product A ⊗ A ⊗⋅⋅⋅⊗ A .
A matrix A ∈ R n × n is called skew-symmetric (or anti-symmetric) if A T = − A .
The Euclidean inner product between two matrices A ∈ R m × n and B ∈ R m × n is deﬁned as
[Equation]
The Frobenius norm of A is deﬁned as ∣∣ A ∣∣ F = √∑
i,j A 2 i,j .
B. Multiresolution Matrix Factorization B.1. Background
Most commonly used matrix factorization algorithms, such as principal component analysis (PCA), singular value decomposition (SVD), or non-negative matrix factorization (NMF) are inherently single-level algorithms.
Saying that a symmetric matrix A ∈ R n × n is of rank r ≪ n means that it can be expressed in terms of a dictionary of r mutually orthogonal unit vectors { u 1 ,u 2 ,...,u r } in the form
A = r ∑ i =
λ i u i u Ti , where u 1 ,...,u r are the normalized eigenvectors of A and λ 1 ,...,λ r are the corresponding eigenvalues.
This is the decomposition that PCA ﬁnds, and it corresponds to factorizing A in the form
[Equation]
where U is an orthogonal matrix and H is a diagonal matrix with the eigenvalues of A on its diagonal.
The drawback of
PCA is that eigenvectors are almost always dense, while matrices occuring in learning problems, especially those related to graphs, often have strong locality properties, in the sense that they are more closely couple certain clusters of nearby coordinates than those farther apart with respect to the underlying topology.
In such cases, modeling A in terms of a basis of global eigenfunctions is both computationally wasteful and conceptually unreasonable: a localized dictionary would be more appropriate.
In contrast to PCA, (Kondor et al., 2014) proposed Multiresolution Matrix Factorization , or MMF for short, to construct a sparse hierarchical system of L -level dictionaries.
The corresponding matrix factorization is of the form
[Equation]
where H is close to diagonal and U 1 ,..., U L are sparse orthogonal matrices with the following constraints:
Each U (cid:96) is k -point rotation for some small k , meaning that it only rotates k coordinates at a time.
Formally, Def.
B. deﬁnes and Fig.
4 shows an example of the k - point rotation matrix
There is a nested sequence of sets S L ⊆ ⋯ ⊆
S 1 ⊆ S
= [ n ] such that the coordinates rotated by U (cid:96) are a subset of S (cid:96) .
H is an S L -core-diagonal matrix that is formally deﬁned in Def.
B.2.
Deﬁnition B.1.
We say that U ∈ R n × n is an elementary rotation of order k (also called as a k -point rotation) if it is an orthogonal matrix of the form
[Equation]
for some I
= { i 1 , ⋯ ,i k } ⊆
[ n ] and O ∈ SO ( k ) .
We denote the set of all such matrices as SO k ( n ) .
The simplest case are second order rotations, or called Givens rotations, which are of the form
[Equation]
where the dots denote the identity that apart from rows/columns i and j , and O ∈ SO ( 2 ) is the rotation matrix of some angle θ ∈ [ 0 , 2 π ) .
Indeed, Jacobi’s algorithm for diagonalizing symmetric matrices (Jacobi, 1846) is a special case of MMF factorization over Givens rotations.
Deﬁnition B.2.
Given a set S ⊆ [ n ] , we say that a matrix H ∈ R n × n is S -core-diagonal if H i,j = 0 unless i,j ∈ S or i = j .
Equivalently, H is S -core-diagonal if it can be written in the form H = D ⊕ S H , for some H ∈ R ∣ S ∣×∣ S ∣ and D is diagonal.
We denote the set of all S -core-diagonal symmetric matrices of dimension n as H S n .
In general, ﬁnding the best MMF factorization to a symmetric matrix A requires solving
[Equation]
[Equation]
B.2. Multiresolution analysis
Deﬁnition B.3.
Given an appropriate subset O of the group SO ( n ) of n -dimensional rotation matrices, a depth parameter
L ∈ N , and a sequence of integers n =
d 0 ≥ d 1 ≥ d 2 ≥ ⋅⋅⋅ ≥
d L ≥ 1 , a Multiresolution Matrix Factorization (MMF) of a symmetric matrix A ∈ R n × n over O is a factorization of the form
[Equation]
(20) where each U (cid:96) ∈
O satisﬁes [ U (cid:96) ] [ n ]∖ S (cid:96) − 1 , [ n ]∖ S (cid:96) − 1 =
I n − d (cid:96) for some nested sequence of sets S L ⊆ ⋯ ⊆
S 1 ⊆ S
= [ n ] with ∣ S (cid:96) ∣ =
d (cid:96) , and H ∈ H S L n is an S L -core-diagonal matrix.
Deﬁnition B.4.
We say that a symmetric matrix A ∈ R n × n is fully multiresolution factorizable over O ⊂ SO ( n ) with ( d 1 ,...,d L ) if it has a decomposition of the form described in Def.
B.3.
L 2 ( X ) (cid:47) (cid:47) ⋯ (cid:47) (cid:47) V(cid:47) (cid:47) (cid:32) (cid:32)
V(cid:47) (cid:47) (cid:33) (cid:33)
V(cid:47) (cid:47) (cid:33) (cid:33) ⋯
W W W
We formally deﬁne MMF in Defs.
B.3 and B.4.
Furthermore, (Kondor et al., 2014) has shown that MMF mirrors the classical theory of multiresolution analysis (MRA) on the real line (Mallat, 1989) to discrete spaces.
The functional analytic view of wavelets is provided by MRA, which, similarly to Fourier analysis, is a way of ﬁltering some function space into a sequence of subspaces
[Equation]
However, it is best to conceptualize (21) as an iterative process of splitting each V (cid:96) into the orthogonal sum V (cid:96) = V (cid:96)
[Equation] +of a smoother part V (cid:96) + 1 , called the approximation space ; and a rougher part W (cid:96) + 1 , called the detail space (see Fig. 3).
Each V (cid:96) has an orthonormal basis Φ (cid:96) ≜ { φ (cid:96)m } m in which each φ is called a father wavelet.
Each complementary space W (cid:96) is also spanned by an orthonormal basis Ψ (cid:96) ≜ { ψ (cid:96)m } m in which each ψ is called a mother wavelet.
In MMF, each individual rotation U (cid:96)
∶ V (cid:96) − 1 → V (cid:96) ⊕ W (cid:96) is a sparse basis transform that expresses Φ (cid:96) ∪ Ψ (cid:96) in the previous basis Φ (cid:96) − 1 such that:
[Equation]
in which Φ 0 is the standard basis, i.e. φ 0 m =
e m ; and dim ( V (cid:96) )
=
d (cid:96) =
∣ S (cid:96) ∣ .
In the Φ 1 ∪ Ψ 1 basis, A compresses into
A 1 = U 1 AU T 1 .
In the Φ 2 ∪ Ψ 2 ∪ Ψ 1 basis, it becomes A
= U 2 U 1 AU T 1 U T 2 , and so on.
Finally, in the Φ L ∪ Ψ L ∪ ⋅⋅⋅ ∪ Ψbasis, it takes on the form A L = H
= U L ...
U 2 U 1 AU T 1 U T 2 ...
U TL that consists of four distinct blocks (supposingly that we permute the rows/columns accordingly):
[Equation]
where H Φ , Φ ∈ R dim ( V L )× dim ( V L ) is effectively A compressed to V L , H Φ ,
Ψ = H T Ψ , Φ = 0 and H Ψ , Ψ is diagonal.
MMF approximates A in the form
[Equation]
where h i,j coefﬁcients are the entries of the H Φ , Φ block, and c (cid:96)m = ⟨ ψ (cid:96)m , A ψ (cid:96)m ⟩ wavelet frequencies are the diagonal elements of the H Ψ , Ψ block.
In particular, the dictionary vectors corresponding to certain rows of U 1 are interpreted as level one wavelets, the dictionary vectors corresponding to certain rows of U 2 U 1 are interpreted as level two wavelets, and so on (see Section B.2).
One thing that is immediately clear is that whereas Eq.
(17) diagonalizes A in a single step, multiresolution analysis will involve a sequence of basis transforms U 1 , U 2 ,..., U L , transforming A step by step as
[Equation]
so the corresponding matrix factorization must be a multilevel factorization
[Equation]
Fig.
5 depicts the multiresolution transform of MMF as in Eq.
Fig.
6 illustrates the corresponding factorization as in
Eq.
[Equation] [Equation]
B.3. Optimization by heuristics
Heuristically, factorizing A can be approximated by an iterative process that starts by setting A
= A and S 1 =
[ n ] , and then executes the following steps for each resolution level (cid:96) ∈ { 1 ,...,L } : 1.
Given A (cid:96) − 1 , select k indices I (cid:96) =
{ i 1 ,...,i k } ⊂ S (cid:96) − 1 of rows/columns of the active submatrix [ A (cid:96) − 1 ] S (cid:96) − 1 , S (cid:96) − 1 that are highly correlated with each other.
Find the corresponding k -point rotation U (cid:96) to I (cid:96) , and compute A (cid:96) =
U (cid:96) A (cid:96) − 1 U T(cid:96)
that brings the submatrix [ A (cid:96) − 1 ] I (cid:96) , I (cid:96) close to diagonal.
In the last level, we set H =
A L (see Fig. 5)
Determine the set of coordinates T (cid:96) ⊆
S (cid:96) − 1 that are to be designated wavelets at this level, and eliminate them from the active set by setting S (cid:96) =
S (cid:96) − 1 ∖ T (cid:96) .
C. Stiefel Manifold Optimization
In order to solve the MMF optimization problem, we consider the following generic optimization with orthogonality constraints:
[Equation]
We identify tangent vectors to the manifold with n × p matrices.
We denote the tangent space at X as T X V p ( R n ) .
Lemma
C.1 characterizes vectors in the tangent space.
Lemma C.1.
Any Z ∈ T X V p ( R n ) , then Z (as an element of R n × p ) satisﬁes
[Equation]
where Z T X is a skew-symmetric p × p matrix.
Proof.
Let Y ( t ) be a curve in V p ( R n ) that starts from X .
We have:
[Equation]
We differentiate two sides of Eq.
(25) with respect to t :
[Equation]
that leads to:
[Equation]
at t = 0 .
Recall that by deﬁnition, Y ( 0 ) =
X and d Y dt ( 0 ) is any element of the tangent space at X .
Therefore, we arrive at
Z T X + X T Z = 0 .
[Equation]
Suppose that F is a differentiable function.
The gradient of F with respect to X is denoted by G ≜ DF X ≜ ( ∂ F( X ) ∂
X i,j ) .
The derivative of F at X in a direction Z is
[Equation]
Since the matrix X T X is symmetric, the Lagrangian multiplier Λ corresponding to X T X
= I p is a symmetric matrix.
The
Lagrangian function of problem (24) is
[Equation]
[Equation]
Lemma C.2.
Suppose that X is a local minimizer of problem (24).
Then X satisﬁes the ﬁrst-order optimality conditions
D X L( X , Λ ) =
G − XG T X
= 0 and X T X
= I p with the associated Lagrangian multiplier Λ = G T X .
Deﬁne ∇F( X ) ≜ G − XG T X and A ≜
GX T − XG T .
Then ∇F = AX .
Moreover, ∇F =
if and only if A = 0 .
Proof.
Since X ∈ V p ( R n ) , we have X T X
= I p .
We differentiate both sides of the Lagrangian function:
[Equation]
Recall that by deﬁnition, G ≜ DF( X ) , we have
[Equation]
Multiplying both sides by X T , we get X T G − X T X Λ = 0 that leads to X T G − Λ = 0 or Λ = X T G .
Since the matrix X T X is symmetric, the Lagrangian multiplier Λ correspoding to X T X
=
I p is a symmetric matrix.
Therefore, we obtain Λ = Λ T =
G T X and D X L( X , Λ ) =
G − XG T X
= 0 .
By deﬁnition, A ≜ GX T − XG T .
We have
AX =
G − XG T X = ∇F .
The last statement is trivial.
Let X ∈ V p
( R n ) , and W be any n × n skew-symmetric matrix.
We consider the following curve that transforms X by
[Equation]
[Equation]
This is called as the Cayley transformation .
Its derivative with respect to τ is
[Equation]
The curve has the following properties:
It stays in the Stiefel manifold, i.e. Y ( τ ) T Y ( τ ) = I .
Its tangent vector at τ = 0 is Y ′
= − W X .
It can be easily derived from Lemma C.1 that Y ′ ( 0 ) is in the tangent space T Y ( 0 )
V p
( R n ) .
Since Y ( 0 ) =
X and W is a skew-symmetric matrix, by letting Z = − W X , it is trivial that Z T X + X T Z =
Lemma C.3.
If we set W ≜ A ≜ GX T − XG T (see Lemma C.2), then the curve Y ( τ ) (deﬁned in Eq.
(28)) is a decent curve for F at τ = 0 , that is
[Equation]
[Equation]
Proof.
By the chain rule, we get
[Equation]
At τ = 0 , DF( Y ( 0 ))
=
G and Y ′ ( 0 )
= − AX .
Therefore,
[Equation]
It is well known that the steepest descent method with a ﬁxed step size may not converge, but the convergence can be guaranteed by choosing the step size wisely: one can choose a step size by minimizing F( Y ( τ )) along the curve Y ( τ ) with respect to τ (Wen & Yin, 2010).
With the choice of W given by Lemma C.3, the minimization algorithm using Y ( τ ) is roughly sketched as follows:
Start with some initial X ( 0 ) .
For t > 0 , we generate X ( t + 1 ) from X ( t ) by a curvilinear search along the curve Y ( τ ) =
( I + τ 2 W ) − 1 ( I − τ 2 W ) X ( t ) by changing τ .
Because ﬁnding the global minimizer is computationally infeasible, the search terminates when then Armijo-Wolfe conditions that indicate an approximate minimizer are satisﬁed.
The Armijo-Wolfe conditions require two parameters 0 < ρ 1 < ρ 2 < 1 (Nocedal & Wright, 2006)
(Wen & Yin, 2010)
(Tagare, 2011): [Equation]
[Equation]
31) where F ′ τ ( Y ( τ ))
=
trace ( G T Y ′ ( τ )) while Y ′ ( τ ) is computed as Eq.
(29) and Y ′ ( 0 )
= − AX .
The gradient descent algorithm on Stiefel manifold to optimize the generic orthogonal-constraint problem (24) with the curvilinear search submodule is described in Algorithm 2, which is used as a submodule in part of our learning algorithm to solve the MMF in (19).
The algorithm can be trivially extended to solve problems with multiple variables and constraints.
Algorithm
3: t ←4: while true do 5:
G ← ( ∂ F( X ( t ) ) ∂ X ( t ) i,j ) ▷ Compute the gradient of F w.r.t X elemense-wise 6: A ← GX ( t ) T − X ( t ) G T
▷ See Lemma 2,7:
Initialize τ to a non-zero value.
▷ Curvilinear search for the optimal step size 8: while (30) and (31) are not satisﬁed do ▷ Armijo-Wolfe conditions 9: τ ← τ▷ Reduce the step size by half 10: end while 11:
▷ Update by the Cayley transformation 12: if ∣∣∇F( X ( t + 1 ) )∣∣ ≤ (cid:15) then ▷ Stopping check.
See Lemma 2.
STOP 14: else 15: t ← t +16: end if 17: end while
D. Reinforcement Learning D.1. Policy gradient methods
Policy gradient has been a widely used approach to solve reinforcement learning problems that targets at modeling and optimizing the policy directly (Sutton & Barto, 2018).
Monte-Carlo policy gradient (REINFORCE) (Williams, 1988)
(Williams, 1992) (Sutton et al., 2000) depends on an estimated return by Monte-Carlo methods using episode samples to update the learnable parameters θ of the policy network π θ .
We deﬁne the value of state s when we follow a policy π as V π ( s ) =
E a ∼ π
[ g (cid:96) ∣ s (cid:96) =
s ] .
The value of (state, action) pair when we follow a policy π is deﬁned similarly as
Q π ( s,a ) =
E a ∼ π
[ g (cid:96) ∣ s (cid:96) =
s,a (cid:96) =
a ] .
The value of the reward objective function depends on the policy and is deﬁned as
[Equation]
[Equation] where d π ( s ) =
p ( s L = s
∣ s 0 ,π θ ) is the stationary distribution of Markov chain for π θ .
It is important to remark
that
our
MDP process terminates after a ﬁnite number of transitions (e.g., L ), so d π ( s ) is the probability that we end up at state s when starting from s 0 and following policy π θ for L steps.
(Sutton et al., 2000) has shown that an unbiased estimate of the gradient of (32) can be obtained from experience using an approximate value function.
The expectation of the sample gradient is equal to the actual gradient:
[Equation]
33) that allows us to update our policy gradient by measuring g (cid:96) from real sample trajectories.
Based on (33), the update rule for policy parameters is simply as
[Equation]
where η is the learning rate, that is used in training our policy network π θ .
D.2. 2-phase process
The learning algorithm is expensive due to the Stiefel manifold optimization in line 11 to ﬁnd the optimal rotations O (cid:96) that are used to compute the rewards g (cid:96) .
In practice, we propose a 2 -phase process that is more efﬁcient:
• Phase 1: Reinforcement learning to ﬁnd the sequence of indices, but instead of manifold optimization, we just use the closed-form solutions for O (cid:96) as the eigenvectors of A S (cid:96) , ∶ A T S (cid:96) , ∶ to estimate the rewards.
In all our experiments, we implement the policy network by two graph neural networks, one to select the pivot (wavelet index) and the another one to select K − 1 indices, with 4 layers of message passing and hidden dimension of 10.
The input node feature for node v (or the v -th row) is binary:
1 if v ∈ S (cid:96) , otherwise 0 .
We use γ = 1 as the discount factor and learning rate η = 10 − 3 .
• Phase 2:
Given a sequence of indices found by the previous phase, we apply Stiefel manifold optimization to actually ﬁnd the optimal rotations accordingly.
D.3. Transfer learning
Ideally, we want our policy network π θ to be universal in the sense that the same trained policy can be applied to different graphs with little adaptation or without any further training.
However, the search space is gigantic with large graphs such as social networks, and the cost of training the policy is computationally expensive.
Therefore, we apply the idea of transfer learning that is to reuse or transfer information from previously learned tasks (source tasks) into new tasks (target tasks).
The source task here is to train our GNN policy on a dataset of small graphs (e.g., possibly synthetic graphs) that are much faster to train on, and the target task is to run the trained policy on the large actual graph.
For example, given citation networks with thousands of nodes such as Cora and Citeseer (Sen et al., 2008), we generate the dataset for policy training by partitioning the big graph into many smaller connected clusters.
The learning algorithm can be easily modiﬁed for such a purpose (e.g., training multiple graphs simultaneously).