## 2 Literature Survey
### Graph Convolutional Neural Networks (GCNs)
For a 2-layer GCN, where the first layer is an input layer and the second is the output layer, as given in Equation (1), the forward model is defined as [2]:  
\[Z=f(A,X)=softmax(\hat{A}.ReLU(\hat{A}\times X\times W^{(0)})\times W^{(1)}) \tag{5}\]  
Here, \(\hat{A}\) is defined as \(D^{-1/2}\times A\times D^{-1/2}\). The ultimate task is to learn the weights for the model where \(C\times H\) are the trainable weights for \(W^{(0)}\) (weights at layer 0). Similarly, \(H\times F\) trainable weights for \(W^{(1)}\)(weights at layer 1). Here, \(C\) refers to the dimensions of the feature vectors, \(F\) refers to the dimensions of the resulting vectors and \(H\) is the number of hidden layers. The expression in Equation (5) can further be extended depending upon the hidden layers in the network. The depth of the network is based on the intuition of the \(k\) path length neighborhood contribution. However, in general, graph networks do not have much effect of the neighborhood interactions beyond 2-3 path lengths [12]. Thus, results of 2-3 layer GCN networks are considerable and impressive. Further, considering the neighbors beyond this range spoils the meaning of neighborhood aggregation, and the results will overfit. Figure 1: Convolutional operational strategies for regular grid structures and irregular graph structures. The last layer of the GCN corresponds to the number of classes for which the prediction is to be made. The output vector of the last layer is then passed through a Softmax function to make the predictions. The cross entropy loss function is used for training:  
\[L=-\sum_{y\in Y_{1}}\sum_{f=1}^{F}Y_{lf}lnZ_{lf} \tag{6}\]  
Here, \(Y_{1}\) is the set of node indices with labels. The whole discussion so far summarizes the process (Figure 2) of semi-supervised node classification by the GCN model. The training process is tuned using hyperparameters such as learning rate, number of layers of the network, number of epochs, dropout, etc., to optimize the loss function as per Equation (6). Graph Neural Networks have been the most widely studied area of research in recent years. These are specific types of neural networks based on the graph structure. The semi-supervised node classification using the GCN model was proposed by [PERSON] et al. [2]. This GCN model is capable of encoding node features and underlying graph structure. A new GCN approach, Graph Wavelet Neural Network (GWNN) [13], significantly outperforms spectral graph CNNs in the task of graph-based semi-supervised classification. [PERSON] et al. [14] proposed a model using multiple powers of an adjacency matrix. [PERSON] et al. [15] addressed the attention mechanism problem for multiple prediction tasks, such as node and graph classification. Several studies worked on the performance improvement of GNNs [16; 17]. For performance improvement of semi-supervised classification, [PERSON] et al. [18] proposed an SF-GCN structure fusion model that is based on graph convolutional networks. Another approach by [PERSON] et al. [19] conveys the learning graph convolutional layer (LGCL), which takes out a fixed number of neighbor nodes for each feature and transforms graph data into grid-like structures. In contrast to these methods, [PERSON] et al. [20] focused on a novel framework called Self-Ensembling GCN (SEGCN), where both labeled and unlabeled nodes were used to train GCN. For the same scenario, [PERSON] et al. [21] incorporated the graph structure and parameters of graph convolutional networks. [PERSON] et al. [22] addressed the shortcomings of the GNN architecture through the automated Graph Neural Networks (AGNN) framework. [PERSON] et al. [23] also developed a graph neural architecture search method. [PERSON] et al. [24] defined Graph Optimized Convolutional Network (GOCN) for graph data representation and learning, which is applicable to multiple graphs. [PERSON] et al. [25] used Distributed Feedback-Looped Networks (DFNets) for the spectral convolutional neural network. Recently, [PERSON] et al.