# Outcome Prediction for Patients with Bipolar Disorder Using Prodromal and Onset Data +
## 2 Materials and Methods
### Deep Neural Network  
In this study, we predicted the outcomes using a deep neural network (DNN) model. DNN, an increasingly popular machine learning method, is sometimes superior to classic statical analyses in performance. Classic statistical analyses draw conclusions at the population level; DNN models can predict the outcomes at the individual level. Classic statistical models need to satisfy assumptions such as linear relationships between predictors and outcomes, normality for residual errors, or independence among predictors; DNN models do not rely on these assumptions. In addition, DNN techniques can be designed to accommodate the temporal sequence of observations with irregular time intervals, which is a common feature in clinical data. DNN has demonstrated the ability to model highly nonlinear relationships without extensive feature engineering, learn from massive amounts of data, and perform better than linear/logistic regression and traditional machine learning methods in many tasks [16; 28; 29; 30; 31]. In this study, the network was implemented using a Python library called Theano [32] together with a helper library called Lasagne [33]. The network had a total of 7 layers (Figure 2). Starting from the input layer, the 2 nd layer was a convolutional layer, followed by a max pooling layer. The 4 th layer was again a convolutional layer and again followed by a max pooling layer. The 6 th layer was a fully connected layer, followed by the output layer. All the weights and biases involved in the layers were learned from the training data. The technical definition of each layer is as follows. The input layer took the temporal images \(X=\left(x_{ij}\right)(1\leq i\leq 84,1\leq j\leq 54)\) as inputs. The second layer was a convolutional layer. It involved 10 filters \(W^{(k)}=\left(w^{(k)}_{ij}\right)\) (1 \(\leq k\leq 10\), 1 \(\leq i\leq 84\), 1 \(\leq j\leq 3\)), which were all 84 \(\times\) 3 matrices (in Figure 2, two filters, colored in yellow and red, respectively, are displayed), as well as 10 biases \(b_{k}\). The entries of the 10 matrices were the weights of this layer. The 10 filters were applied to the input layer through a time window of width 3, which was sliding in the time direction (from left to right in Figure 2). Mathematically, the output of the convolutional layer was a 10 \(\times\) 52 matrix \(Y=\left(y_{kl}\right)\) obtained by  
\[y_{kl}=f\Big{(}{\sum_{i=1}^{84}}{\sum_{j=0}^{2}}x_{i,l+j}w^{(k)}_{i,3-j}+b_{k} \Big{)} \tag{1}\]  
Figure 2: Illustration of the deep neural network. where \(f\) is the rectified linear function (i.e., \(f(x)=max(0,x)\)). The third layer was a max pooling layer. This layer had no weights, and all it did was take the maximum from a sliding time window of length 3 for each row. Mathematically, the output of this max pooling \(Z=(z_{km})\) was computed as:  
\[z_{km}=max_{m\leq l\leq m+2}y_{kl} \tag{2}\]  
Therefore, the output \(Z\) was a 10 \(\times\) 50 matrix. The fourth layer was another convolutional layer. Its definition was similar to that of the second layer, except that it took the output \(Z\) of the third layer as input. It also had 10 filters and 10 biases. The fifth layer was another max pooling layer, whose definition was the same as that of the third layer. The output of the fifth layer was a 10 \(\times\) 46 matrix. The sixth layer was a full connection layer, meaning that each entry of the input (from the fifth layer) and each entry of the output were together associated with a weight and a bias. We set the output to be a vector \(P=(p_{i})\) of dimension 15. Mathematically, if we denote the output of the fifth layer as \(Q=(q_{kl})\), the output is  
\[p_{i}=f\big{(}\sum\
olimits_{all\ kl}q_{kl}v_{kli}+b_{kli}\big{)} \tag{3}\]  
where \(u_{kli}\) and \(b_{kli}\) are the weights and biases, respectively, and \(f\) is the rectified linear function. The last layer of the network was a full connection layer as well.