# A Deep Learning Model for Data-Driven Discovery of Functional Connectivity
## 2 Materials and Methods
### Method
#### 2.2.3 Gnn  
Our graph network is based on a previously published model [13]. Each subject is represented by a graph \(G\) having \(V,A,E\), where \(V\in\mathbb{R}^{r\times t}\) is the matrix of vertices, where each vertex is represented by an embedding acquired by self-attention. \(A,E\in\mathbb{R}^{r\times r}\) are the adjacency and edge weight matrices. Because we do not use any existing method of computing edges, we construct a complete directed graph with backward edges, meaning that every pair of vertices is joined by two directed edges with weights \(e_{ij}\) and \(e_{ji}\in E\). For each GNN layer, at every step \(s\), each node, which is a region in our model, sums the feature vectors of every other region relative to the weight edge between the nodes and passes the resultant and its own feature vector through a gated recurrent unit (GRU) network [21] to obtain new embedding for itself.  
\[x_{s}^{n_{i}}=\text{GRU}(x_{s-1}^{n_{i}},\sum_{\forall n_{j}:n_{j} >n_{i}}e_{ji}x_{s-1}^{n_{j}}) \tag{2}\]  
where GRU can be explained by following set of equations, with \(h_{s-1}\) representing the result of sum in Equation (2):  
\[\begin{split} z_{s}&=\sigma(\textbf{W}^{(z)}x_{s-1}+ \textbf{U}^{(z)}h_{s-1})\\ r_{s}&=\sigma(\textbf{W}^{(r)}x_{s-1}+\textbf{U}^{( r)}h_{s-1})\\ h_{s}{}^{\prime}&=\sigma(\textbf{W}x_{s-1}+r_{s} \odot\textbf{U}h_{s-1})\\ x_{s}&=\sigma(z_{s}\odot h_{s-1}+(1-z_{s})\odot h _{s}^{\prime})\end{split} \tag{3}\]  
The number of steps is a hyper-parameter that we have set it as 2 based on our experiments. The graph neural network helps nodes to create new embeddings based on the embeddings of other regions in the graph weighted by the edge weights between them. In our architecture, we use 6 GNN layers, as shown in experiments of [22], which it provides with the highest accuracy, with the first 3, followed by a top-k pooling layer [23; 24]. On the input feature vectors, which are the embeddings of the regions, the pooling operator learns a parameter (**p**), which is to assign weight to the features. Based on this parameter, the top (k) layers are chosen in each pooling layer and the rest of the regions are discarded from further layers. The pooling method can be explained by the following equations.  
\[\begin{split} y&=\frac{Xp}{\|p\|},\\ i&=\text{top}_{k}(y)\\ X^{\prime}&=(X\odot\text{tanh}(y))_{i},\\ A^{\prime}&=A_{i,i}\end{split} \tag{4}\]\(\mathbf{X}^{\prime}\) and \(\mathbf{A}^{\prime}\) are the new features and adjacency matrix that we get after selecting top (k) regions. Pooling is performed to help model focus on the important regions/nodes that are responsible for classification. The ratio of nodes to keep in the pooling layer is a hyper-parameter and we have used \((0.8,0.8,0.3)\) as the ratios. Because we represent each subject as graph \(G\), in the end we do graph classification by pooling all the feature vectors of the remaining 23 regions/nodes. To obtain one feature vector from the entire graph, we concatenate the output of three different pooling layers. We pass the complete graph into three separate pooling layers. Each of the pooling layer gives us one feature factor. In the end, we concatenate the three vectors to get one final embedding for the entire graph that represents a subject. In our model, we use graph max pool, graph average pool, and attention based pool [25]. The dimension of the resulting vector is 96. The feature vector is then passed through two linear layers of size 32 and 2. As the name suggests, graph max pool and graph average pool just gets the max and average vector from the graph, whereas attention based pooling multiplies each vector with a learned attention value before summing all of the vectors.