Year: 2021

Abstract:
In recent years, many studies utilize Convolutional Neural Networks (CNNs) to deal with non-grid graph data, known as Graph Convolutional Networks (GCNs). However, there exist two main limitations of the prevalent GCNs. First, GCNs have a latent information loss problem since they use scalar-valued neurons rather than vector-valued ones to iterate through graph convolutions. Second, GCNs are presented statically with fixed architectures during training, which would restrict their representation power. To tackle these two issues, based on a capsule GCN model (CapsGNN) which encodes node embeddings as vectors, we propose Adaptive Graph Capsule Convolutional Networks (AdaGCCN) to adaptively adjust the model architecture at runtime. Specifically, we leverage Reinforcement Learning (RL) to design an assistant module for continuously selecting the optimal modification to the model structure through the whole training process. Moreover, we determine the architecture search space through analyzing the impacts of model’s depth and width. To mitigate the computation overhead brought by the assistant module, we then deploy multiple workers to compute in parallel on GPU. Evaluations show that AdaGCCN achieves SOTA accuracy results and outperforms CapsGNN almost on all datasets in both bioinformatics and social fields. We also conduct experiments to indicate the efficiency of the paralleling strategy.

heading: 1 INTRODUCTION
text: Graph-structured data is ubiquitous in real-world scenarios such as biological networks, citation networks, recommender systems, and social networks. It has seen a surge in research on extending deep learning methods like Convolutional Neural Networks (CNNs) (LeCun & Bengio, 1998) or Recurrent Neural Networks (RNNs) (Mikolov et al., 2011) to process tasks on graphs. Graph Neural Networks (GNNs) were first proposed in (Gori et al., 2005) and further elaborated in (Scarselli et al., 2009) to handle graphs based on recursive neural networks (RecGNNs). RecGNNs learn to represent nodes by constantly exchanging neighbor information until reaching a stable equilibrium, which is computationally expensive. Encouraged by the success of CNNs, many studies (Bruna et al., 2014; Henaff et al., 2015; Defferrard et al., 2016; Kipf & Welling, 2017) borrowed the idea of convolutions and redefined them for graph data. These convolutional networks on graphs are known as Graph Convolutional Networks (GCNs) by stacking multiple graph convolutional layers to present high-level graph representation. Moreover, GCNs can save much time in computing compared to RecGNNs. In this paper, we focus on the research on GCNs.
SOTA GCNs methods (Zhang et al., 2018; Verma & Zhang, 2018) generate graph embeddings in the form of scalar, which may not help capture enough node features. Therefore, it is of great necessity to improve the information extraction ability of current GCNs. In 2011, (Hinton et al., 2011) pointed out that traditional pooling mechanism (Scherer et al., 2010) on CNNs may suffer from the information loss problem. Later (Sabour et al., 2017) proposed a novel neural network called CapsNet without using pooling. CapsNet encodes the features of images as vectors and uses a dynamic routing algorithm to obtain the right local-global relationship. Owing to its capability to capture inner relations among entities, CapsNet has been converged with GCNs to grasp higher quality graph information (Verma & Zhang, 2018; Xinyi & Chen, 2019).
CapsGNN (Xinyi & Chen, 2019) extracts multi-scale node features from different convolutional layers and represents the extracted features in the form of capsules. Compared to classic GCN
models, CapsGNN has been proven to have better representation ability on graph data. However, CapsGNN still uses a static model structure to conduct training, which inherently restricts its ability to cope with the enhancing abstraction level of graph information during the whole training process. Therefore, it is crucial to explore a method for dynamically adjusting the structure of CapsGNN to improve its classification performance.
Many research efforts (He et al., 2016; Szegedy et al., 2016; Huang et al., 2017; Szegedy et al., 2017; Li et al., 2018; Rong et al., 2019) have demonstrated that the number of convolutional layers (i.e., depth, denoted by D) and the number of neurons in each layer (i.e., width, denoted by W ) impact significantly on the performance of neural networks. However, these methods are often inefficient in training due to overly complex model structures. In this paper, we simplify the architecture search problem and focus on the exploration of the best depth-width setting on particular dataset, considering the balance between efficiency and accuracy.
To evaluate the effects of the depth and width in CapsGNN, we change them to observe the performance of the model on two graph datasets, ENZYMES and IMDB-MULTI (Kersting et al., 2016), in bioinformatics and social fields, respectively. It is shown in Figure 1 that if the model has small D and W , it can not sufficiently extract the features of graphs. On the contrary, when the model structure is too complicated, overfitting occurs frequently. Thus, CapsGNN is sensitive to changes in depth and width just because of the vectorized graph representation in it. Since the impacts of D and W are distinct on different datasets, it is necessary to adaptively adjust the model architecture according to the latent characteristics of particular workloads.
With respect to the search for the optimal structure of a model, there has been increasing interest in designing automatic methods in Neural Architecture Search (NAS) (Zoph & Le, 2017; Liu et al., 2018; Luo et al., 2018; Real et al., 2019; Pham et al., 2018). There exist mainly three kinds of techniques to search for the best model architecture in NAS, i.e., Reinforcement Learning (RL) based, Evolution Algorithm (EA) based, and gradient-based methods. RL-based methods (Zoph & Le, 2017; Pham et al., 2018; Zoph et al., 2018) use RNN as the controller to train different architectures and select the optimal one according to their validation accuracy results. EA-based methods (Real et al., 2017; Xie & Yuille, 2017; Real et al., 2019; Shu et al., 2019) utilize the evolutionary algorithm to search for superior model structures from a set of initialized candidate networks. Gradient-based methods (Liu et al., 2019; Wu et al., 2019; Xie et al., 2019) construct a SuperNet and leverage the attention mechanism to remove weak connections after searching. The former two methods can be regarded as offline and are often time-consuming. Although the third method is an online method and shows good efficiency due to gradient descent optimization, it lacks variety in searching for the optimal architecture.
To achieve the balance between efficiency and accuracy, we propose Adaptive Graph Capsule Convolutional Networks (AdaGCCN) to address the issues discussed above. We utilize Reinforcement Learning (RL) to design an online assistant module for evaluating different changes to depth and width. Differing from the RL agent in an RL-based NAS task that considers the choices of various model structures as actions, we move the RL procedure into only one full training and choose one action (i.e., one alteration to the model structure) in a sliding epoch window at one time, according to not only the accuracy results on the validation data set but also the rate of reduction in training loss. However, the introduction of the assistant module would result in extra computational overhead. To accelerate the selection process described above, we further assign multiple workers to train in parallel on the GPU. Three main contributions are made in this paper:
• We optimize the model architecture at runtime by tuning depth and width of convolutional layers by observing both the change in validation accuracy and the reduction speed of training loss after each sliding epoch window. Following this scheme, we can dynamically refine our model while sharing the weights parameters learned at previous training stages.
• The assistant module is formalized as a Reinforcement Learning (RL) agent. We define the depth-width setting of graph convolutions as a state and consider the changes of the setting as actions. We leverage the RL mechanism to update the rewards corresponding to different actions.
• To simplify the search space of the assistant module, we discuss the adequacy of only adjusting depth and width of the convolution operations. To ease the computation burden that arises from the assistant module, we split the whole validation process into multiple tasks to run them synchronously.
We organize the rest of the paper as follows. Section 2 introduces the background of this work. Section 3 elaborates the fundamental architecture and implementation of AdaGCCN. In Section 4, we evaluate the classification performance of our model against the baseline methods. We discuss related work in Section 5 and conclude this paper in Section 6.

heading: 2 PRELIMINARIES
text: Here, we give a brief revisit to common GCNs and a capsule-based GCN, i.e., CapsGNN.

heading: 2.1 GRAPH CONVOLUTIONAL NETWORKS
text: A graph can be represented by G = (V,X,A), in which V = {v1, v2, ..., vN} (N is the number of nodes) is the set of nodes in the graph,A ∈ {0, 1}N×N is the adjacency matrix ofG andX ∈ RN×d represents the features of each node (d is the number of feature dimensions).
GCNs, inspired by CNNs, extract richer information of one node by aggregating features of the nodes from its neighborhood. At each layer of GCNs, each node and its neighbors are operated through convolutions. Then an activation function is applied to return the updated representation of each node. The procedure in GCNs can be written as:
Zl+1 = A ′ ZlW l, X l+1 = f(Zl+1) (1)
whereZl ∈ RN×dl implies node features at layer l (Z0 = X),W l ∈ Rdl×dl+1 is a trainable weights matrix which serves as a convolution filter, A
′ ∈ RN×N is the regularized form of the adjacency matrix A, and f is a nonlinear activation function.

heading: 2.2 CAPSULE GRAPH NEURAL NETWORK
text: Although classic CNNs perform decent classification results in image processing, they face a severe robust problem. For example, when training with human faces, CNNs could learn good statistical representations of them, but can not tell whether the relative positions of different parts of a face are correct. The reason behind this is mainly due to the pooling mechanism in CNNs which extracts information through down-sampling, thus causing the losses on some essential features of images.
Unlike extracting image features as scalar values in CNNs, CapsNet was proposed (Hinton et al., 2011; Sabour et al., 2017; Hinton et al., 2018) to build vector-based capsules to learn the inherent part-whole relationships in images. Inspired by CapsNet, CapsGNN (Xinyi & Chen, 2019) fuses the capsule mechanism with GCNs to handle graph data. CapsGNN extracts multi-dimension node features from different graph convolutional layers to serve as basic node capsules. Then it combines the attention mechanism and dynamic routing to generate higher-level graph capsules. To this end, each graph can be abstracted as multiple graph capsules, and different graph capsules represent the properties of the graph from different aspects. At last, CapsGNN uses dynamic routing again to produce class capsules which are applied to do graph classifications. However, like traditional GCNs models, CapsGNN does not try to adjust the model architecture when more fine-grained graph information needs to be learned as training proceeds, which leaves potential in enhancing its representation power.

heading: 3 OVERVIEW OF ADAGCCN
text: In this section, we outline the overall design of AdaGCCN (depicted in Figure 2). We first introduce the assistant module (AM) and the principle for determining the search space in it. Then we elaborate on how AM dynamically changes the model structure to extract higher-quality graph representations. Finally, we describe the parallel execution in AdaGCCN which accelerates the search process in AM.

heading: 3.1 THE ASSISTANT MODULE
text: We build an AM to select the changes to model structure from different branches. Each branch indicates an alteration to D and W . The changes of D and W are represented by ∆D and ∆W , respectively. We formalize the procedure of the assistant module as an RL procedure. RL usually uses an agent to interact with an environment by choosing actions at different periods. In AdaGCCN, the depth-width setting at the t-th (t ≥ 1) training epoch is treated as the state in the RL agent (denoted by (Dt,Wt)). We specify two lists, the ∆D list and the ∆W list, for the changes of (Dt,Wt). The length of the ∆D list and the ∆W list are len1 and len2, respectively. Note that we focus on the impacts ofD andW in this paper rather than the complex situation in common NAS, len1 and len2 can not be very large with considering both the overfitting problem and computation efficiency. An action ati (1 ≤ i ≤ m) is defined as one of the m (m = len1 ∗ len2) choices in changing (Dt,Wt) at the t-th epoch:
ati = [∆D list[p],∆W list[q]], 0 ≤ p ≤ len1 − 1, 0 ≤ q ≤ len2 − 1. (2)

heading: 3.2 ADAPTIVE D-W TUNING
text: There are two key issues we need to address upon building an AM:
• Q1: how to select the optimal ∆D and ∆W during training? • Q2: how to share the model parameters derived at previous training epochs when D and W are updated?
Through training, we record the training loss (denoted by losst) and the accuracy on the validation hold (denoted by valacct) after epoch t ends. Unlike the original RL method, we can not predict which action performs the best before running the model with updated D and W . Thus we design a reward evaluation function to compare the loss reduction rate and the variation of validation accuracy over a sliding epoch window. Specifically, the length of this sliding window is set to three, and then the loss decrease speed is calculated within three consecutive epochs. We could broaden the size of the sliding window according to specific requirements. When the action with the maximal Q-value is obtained, we take it to update the model structure for training in the next sliding window.
We let Q(ati) represent the reward of taking the action a t i. The goal of AM is to constantly select the action ati∗ with the maximal reward. The update on Q(a t i) is determined whether a t i is chosen at
Algorithm 1: The updating strategy in the assistant module. Input: The number of training epochs: E, the length of the sliding epoch window: 3, the
reward discount rate: γ (0 ≤ γ ≤ 0.5), the threshold reflecting the reward for the loss decrease at the last two training epochs: θ (0 ≤ θ ≤ 0.5), the ∆D list (length: len1) for updating D, the ∆W list (length: len2) for updating W , the set of losses during E epochs: {losst}, the set of accuracies on the validation data set during E epochs: {valacct}, the set of Q-values of m (m = len1 ∗ len2) actions: {Qi} (1 ≤ i ≤ m).
Output: Optimal ∆D and ∆W . 1 for 3 ≤ t < E do 2 if t mod 3 = 0 then 3 ∆loss1 ← losst−2 − losst−1, ∆loss2 ← losst−3 − losst−2; 4 eval(ati∗)← ∆loss1/∆loss2; 5 ∆valacc1 ← valacct−1 − valacct−4, ∆valacc2 ← valacct−1 − valacct−2; 6 ∆valacc = ∆valacc1 + ∆valacc2; 7 if ∆valacc1 ∗∆valacc2 > 0 then 8 if ∆loss1 ∗∆loss2 > 0 then 9 flag ← (Max(∆loss1, 0)−Min(∆loss1, 0))/∆loss1; 10 if eval(ati∗) ≥ 1 then 11 Q[ati∗ ]+ = flag ∗ (1− γ) ∗ (eval(ati∗) + 10 ∗∆valacc); 12 else 13 if 0 < ∆loss1 < θ then 14 Q[ati∗ ]− = (1− γ) ∗ (10 ∗ eval(ati∗) + 10 ∗∆valacc); 15 else 16 Q[ati∗ ]+ = γ ∗ eval(ati∗); 17 if ∆loss1 > θ then 18 Q[ati∗ ]+ = 10 ∗ γ ∗∆valacc;
19 else 20 if eval(ati∗) ≤ −1 then 21 Q[ati∗ ]− = flag ∗ (1− γ) ∗ (eval(ati∗) + 10 ∗∆valacc); 22 else 23 Q[ati∗ ]+ = γ ∗ (10 ∗ eval(ati∗)); 24 if ∆loss1 < 0 then 25 Q[ati∗ ]+ = γ ∗ (10 ∗∆valacc− 9 ∗ eval(ati∗));
26 else 27 Q[ati∗ ]+ = (1− γ) ∗ (10 ∗ (Min(∆valacc1, 0) +Min(∆valacc2, 0))); 28 Q maxindex← index of Max{Qi}; 29 action maxindex← actions[Q maxindex]; 30 return ∆D = action maxindex[0], ∆W = action maxindex[1];
epoch t− s (s is the length of the sliding epoch window and we let s = 3 in this paper). Suppose we pick at−3i∗ = [∆D list[p ∗],∆W list[q∗]] at the (t−3)-th epoch, then we reviseQ(ati∗) (ati∗ = a t−3 i∗ ) by Algorithm 1 where eval(ati∗) describes the loss reduction speed at the last three epochs and ∆valacc implies how the validation accuracy varies within two consecutive sliding epoch windows. As to the actions that are not taken, the rewards of them would remain unchanged. Note that at the first three epochs, we random select an action a1i∗ from the action list, and initialize the Q-values of m actions to 0. After updating all the Q-values, if the maximal reward corresponds to more than one action, we randomly choose an action with the maximal Q-value. Once we acquire the updated ∆D and ∆W , D and W that AdaGCCN uses at epoch t are:
Dt = Dt−3 + ∆D list[p ∗], Wt = Wt−3 + ∆W list[q ∗]. (3)
Suppose D and W are changed at the t-th epoch, the model would be reinitialized if we do not utilize the parameters produced at epoch t−1. In other words, the modifications to D and W would influence the shape of 2-D weight tensors in GCN layers. It urges us to consider how to maximize the retention of the weight parameters learned at the last epoch. Regarding the resampling techniques
in image processing, an interpolation solution called bilinear interpolation (BI) helps scale images using the distance weighted average of the four nearest pixel values to estimate a new pixel value. As opposed to other interpolation methods (e.g., nearest-neighbor interpolation) which would make some pixels appear larger than others in the resized image, BI reduces some of the visual distortions in image transformation. Thus we adopt BI in this paper to reshape the weight matrices upon tuning the D and W of the model without losing much information extracted in previous iterations.

heading: 3.3 PARALLEL PROCESSING
text: We follow the same validation method as in CapsGNN, i.e., the 10-fold cross validation, which splits the original dataset into ten groups. Each time we take eight groups as train holds, one group as the validation hold, and the remained one group as the test hold. In the end, we would obtain 10 test accuracy results on 10 different splittings. We take the average of these results as the final test accuracy.
Since the ten splittings are independent to each other, to alleviate the computation overhead brought by AM, we dispatch these splittings to ten workers on one GPU card and explore a scheduling strategy (see Algorithm 2 in Appendix A) for dynamically assigning an appropriate number of workers through monitoring the running status of them. Specifically, we observe that the device memory required by an individual worker almost keeps constant, so we could estimate the memory requirement of a worker through pre-run, and then launch multiple workers in parallel while their total memory is not larger than the device memory.
However, when processing multiple workers concurrently on a single dataset, sometimes GPU memory is not fully utilized. Thus we extend Algorithm 2 to do scheduling on multiple datasets, i.e., combinatorilly assigning workers on different datasets to maximize the utilization of GPU memory, shown in Algorithm 3 in Appendix A.

heading: 4 EVALUATION
text: We conduct elaborate experiments in this section on ten typical datasets to evaluate the performance of AdaGCCN against the baseline models.

heading: 4.1 PREPARATIONS FOR EXPERIMENTS
text: Baseline models. AdaGCCN is compared with four graph kernel algorithms, GK (Shervashidze et al., 2009), WL (Shervashidze et al., 2011), DGK (Yanardag & Vishwanathan, 2015), and AWE (Ivanov & Burnaev, 2018)), four GNNs-based methods, PATCHY-SAN (PSCN) (Niepert et al., 2016), DGCNN (Zhang et al., 2018), GIN (Xu et al., 2019), and SOM-GCNN (Pasa et al., 2020), and two capsule-based GNNs methods, GCAPS-CNN (Verma & Zhang, 2018) and CapsGNN (Xinyi & Chen, 2019). We also take the results reported in (Errica et al., 2020) as a baseline, namely FGNN, which proposed a fair validation method of GNNs for graph classification.
Experimental settings. We evaluate AdaGCCN on a machine equipped with dual 2.40 GHz Intel Xeon Gold 6240R processors (24 cores in total), 256 GB main memory, and 1 NVIDIA Tesla V100 GPU (32 GB memory). The installed operating system is CentOS 7.5.1804, using CUDA 11.2 and cuDNN 7.6.5. PyTorch 1.8 (Paszke et al., 2019) and Python 3.6.5 are used for training.
Datasets. Since our work focuses on graph-level classifications (not node-level classifications), we experiment on the datasets frequently used in graph classification tasks including five biological datasets, ENZYMES, D&D, MUTAG, NCI1, and PROTEINS and five social network datasets, COLLAB, IMDB-BINARY, IMDB-MULTI, REDDIT-M5K, and REDDIT-M12K. The details of these datasets are described in Table 1.

heading: 4.2 ADAGCCN PERFORMANCE
text: We first compare the classification accuracy between AdaGCCN and the baseline methods. Then we demonstrate the efficiency of the scheduling strategies applied in the parallel process.

heading: 4.2.1 THE EFFECTIVENESS OF AM
text: Based on the neural network method, AdaGCCN takes advantage of the multi-dimension representation ability of capsules, and further explores to strengthen the information extraction ability of the model by changing depth and width of graph convolutional layers. We reproduce CapsGNN in this paper and take the test accuracies reported in other baseline works for the comparison with our model.
It is shown in Table 2 that our model outperforms CapsGNN on 6 out of 7 datasets by up to 6.57% on ENZYMES except for the performance on PROTEINS. Note that the graphs on PROTEINS are more complex in inner relationships, and the ∆D list and the ∆W list in current AM are manually set, thus the extraction ability of AdaGCCN would be restricted somehow. However, the main point in this paper is to propose an RL-based methodology in adaptively adjusting the model structure to improve its representation ability on various workloads, some hyper-parameters defined in the assistant module could be tuned to present better model performance in the future. Besides, AdaGCCN achieves state-of-the-art results almost on all datasets including biological and social graphs, which further demonstrates the effectiveness of adjusting D and W during only one full training.
Figure 3 depicts the change of accuracies on the test fold when training with CapsGNN or AdaGCCN. Since the assistant module in AdaGCCN evaluates the loss reduction speed on the train fold and the accuracy change on the validation fold, as the training process goes on, it helps to select the ∆D and ∆W with the biggest accumulated reward. It is observed in the figure that AdaGCCN presents higher test accuracy than CapsGNN at arbitrary intermediate training stages. Besides, the dynamic adjustment in our model does not do any harm to its convergency compared to static CapsGNN. Note that the number of training epochs is set to be large on all datasets to ensure either CapsGNN or AdaGCCN achieves optimal classification performance during training.

heading: 4.2.2 PARALLEL EXECUTION
text: Taking into consideration the computational cost brought by the assistant module, we adaptively adjust the number of workers at runtime. It implies in Figure 4 that when with Algorithm 2, the parallel AdaGCCN on all datasets could averagely speedup the sequential AdaGCCN by up to 2.99×. To make full use of GPU memory, we adopt Algorithm 3 to let workers on different datasets run combinatorilly, which further decreases the computation time when handling the datasets one at a time, e.g., by 16.7% within 300 training epochs. Besides, the memory consumption of CapsGNN and AdaGCCN are displayed in Table 4 in Appendix A.

heading: 4.3 DISCUSSION

heading: 4.3.1 GENERALITY AND EXPLAINABILITY OF THE ASSISTANT MODULE (AM)
text: The NAS nature of the RL-based AM enables it to be applied to other GNNs. We add an experiment on the effect of AM on GIN, the model with the best classification performance in all the baseline methods, and record its test accuracy results on different datasets in Table 3. Moreover, to show the explainability of AdaGCCN on different datasets, we compare the D and W selected by the assistant module with the original depth and width in CapsGNN in Table 5 in Appendix A.

heading: 4.3.2 LIMITATIONS OF ADAGCCN
text: In our experiments, both CapsGNN and AdaGCCN can not run successfully on D&D, REDDITM5K, and REDDIT-M12K either due to GPU memory limitation or heavy time overhead. There are two reasons behind this phenomenon. First, CapsGNN and AdaGCCN have more complex structures than normal GNNs due to the vector-based capsule modules in them, resulting in more model parameters generated during training that the GPU can not hold. Second, both CapsGNN and our model set the batch size to 32 on small graphs, which is not appropriate on large ones. If we use a smaller batch size on D&D and REDDIT, the accuracy results would be affected compared to adopting a larger batch size, and the time costs would be unbearable, e.g., more than 3 GPU days on REDDIT-M12K. Here we provide probable directions in optimizing our work, e.g., designing finegrained parallelism to improve the utilization of GPU, or refining the rewards evaluation function in the assistant module to increase the efficiency of the search process.

heading: 5 RELATED WORK
text: Following the success of neural networks on data with grid structures (LeCun & Bengio, 1998; Mikolov et al., 2011), considerable research interests have been devoted to non-grid graph data, i.e., Graph Neural Networks (GNNs) (Gori et al., 2005), especially Graph Convolutional Networks (GCNs) (Bruna et al., 2014; Henaff et al., 2015). GNNs have already obtained remarkable achievements in various tasks, e.g., graph classification (Defferrard et al., 2016), link prediction (Zhang & Chen, 2018), and node classification (Kipf & Welling, 2017). GCNs inherits the convolutional operations in CNNs, however, they are unable to learn graphs sufficiently which contains multiple attributes of nodes and complicated inner connections.
CapsNet was proposed by Hinton’s team (Hinton et al., 2011) and improved by them (Sabour et al., 2017; Hinton et al., 2018) to represent local-global features of images. Inspired by the promising explainability of CapsNet, some studies (Verma & Zhang, 2018; Mallea et al., 2019; Xinyi & Chen, 2019) combine GCNs and CapsNet to extract multi-scale information in graphs. However, the static structure employed in these capsule-based GCNs would restrict their representation ability, which motivates us to explore dynamic adjustment of model structures at runtime.
The design of neural network structure is turning from manual efforts into automatic machine search. The landmark event of this process occurred in the paper (Zoph & Le, 2017) published by Google, which leverages reinforcement learning to search the optimal neural network structure, known as neural architecture search (NAS). Later, great efforts (Zoph & Le, 2017; Zoph et al., 2018; Chen et al., 2018; Gao et al., 2020) have been made to seek for the optimal architectures of CNNs, RNNs, or GNNs within a pre-determined search space according to the validation accuracy results. Although NAS has achieved impressive performance, the design of the search strategy is complicated and introduces numerous additional complete training processes.

heading: 6 CONCLUSION
text: In this work, we propose an adaptive framework AdaGCCN. We implement AdaGCCN by adjusting the model structure through an RL-based searching process, which improves the explainability of the model by finding the optimal depth and width of convolutions. To decrease the computation burden caused by the proposed assistant module, we assign multiple workers to execute in parallel. Once the device could not hold all the workers in it or has unused memory, AdaGCCN processes the workers on multiple datasets combinatorilly by monitoring the launches and exits of them. We demonstrate through experiments that AdaGCCN can adaptively refine its representation ability on different workloads, considering the trade-off between efficiency and accuracy.

heading: A APPENDIX
text: The comparison of the memory consumption when experimenting with CapsGNN and our model are recorded in Table 4. Using the parallel strategies proposed in Section 3.3, as shown in Algorithm 2 amd Algorithm 3, AdaGCCN can achieve the balance between efficiency and accuracy.
The D and W selected by the assistant module in AdaGCCN compared to the original depth and width in CapsGNN on different datasets are displayed in Table 5. Note that the optimal D and W determined by AM are different on the 10 splittings. We take the D and W on the splitting with the best test accuracy for the comparison against CapsGNN.
Algorithm 2: Scheduling within a dataset. Input: Number of running workers: k, GPU memory allocated to each worker: Ms, available
GPU memory: Ma, number of maximal concurrent workers: K = bMa/Msc. 1 Launch K workers and initialize k as K; 2 while k < K do 3 Launch K − k workers from the unexecuted workers; 4 Update k; 5 if k = 0 then 6 Break;
Algorithm 3: Scheduling among multiple datasets. Input: The list of datasets: {dsi} (length: lends ≥ 2), GPU memory allocated to each worker
Model MUTAG ENZYMES NCI1 PROTEINS IMDB-BINARY IMDB-MULTI COLLAB CapsGNN (5, 2) (5, 2) (5, 2) (5, 2) (5, 2) (5, 2) (5, 2) AdaGCCN (7, 4) (7, 3) (6, 4) (6, 2) (5, 3) (7, 2) (6, 3)

Review summaries:
This work proposed the Adaptive Graph Capsule Convolutional Networks (AdaGCCN), which utilizes RL to adjust the CapsGNN model architecture at runtime. Specifically, an RL assistant module is proposed to search the optimal architecture during training. Another focus of this work is to speed up the proposed method with parallel processing.

because they use scalar-valued neurons, this paper improves on existing graph capsule network, CapsGNN by adapting the neural network architecture in the runtime. More specifically, the authors propose Adaptive Graph Capsule Convolutional Networks (AdaGCCN) which leverages Reinforcement Learning (RL) to design an assistant module for continuously selecting the optimal modification to the model structure through the whole training process. In addition, it determines the architecture search space through analyzing the impacts of model’s depth and width. Evaluations demonstrates that AdaGCCN achieves comparable accuracy results with the state-of-the-art GNN and outperforms CapsGNN almost on all datasets from bioinformatics and social networks.

In this manuscript, the authors proposes an assistant module (AM) that is able to dynamically change the model structure of a GNN model by adjusting the depth and the number of neurons. The AM relies on the reinforcement learning framework and exploits the loss decrease rate and the accuracy results on the validation data set to select how to modify the GCN hyper-parameters. Exploiting this component and the capsule GNN model, the authors defined a novel GNN model dubbed Adaptive Graph Capsule Convolutional Networks.

This paper proposes a reinforcement learning based strategy for adaptively adjusting capsule graph convolutional network architectures throughout training. To that end, they introduce an assistant module which continuously adjusts the depth and width of the network based on the loss/validation accuracy reduction rate over a predetermined training epoch window.  The search space is defined by the authors as a list of m combinations of depth and width variables D and W, and an action a_t at epoch t consists of choosing one of the m depth/width architectural changes. The current training epoch with architecture depth D_t and width W_t is treated as the state, and the reward of an action at epoch t is revised via the relative decrease in training loss/validation accuracy over 3 epochs.