# Model Establishment of Cross-Disease Course Prediction Using Transfer Learning
## 3 Materials and Methods
### _Tl_-CDC
#### 3.3.1 Prediction Model of Course Development of Source Disease  
The structure of the prediction model for the development of the disease course of the source disease used in this study is shown in Figure 4. In order to effectively capture the temporal relationship, RNN-based LSTM, Bi-LSTM, and GRU were selected, and the calculation of the attention mechanism was added to extract the features of the source disease course. This study also compared the effect of adding an attention mechanism to the target disease prediction task after parameter transfer. This section will explain in detail the time series model and attention mechanism of the source disease course development prediction model. Figure 4: Prediction Model of Source Disease Course Development. This study uses the Many-to-One architecture in the time series model. First, the PD course representation of the last four time points was input into the time series model. If k is 3, the input of data from the four time points of the disease course matrix into the model represented the input of one year of the patient's disease course data, and the output of the disease course representation at the next time point is also the prediction of the disease course development in the following three months. For example, in the upper part of Figure 4, the source disease course representation tensor composed of multiple patient data was used as input data, and one of the two-dimensional matrices represents the development of a patient's disease course, and the column in green represents the disease course at a time point. First, add 1 to the input data to get \(log_{10}\), in order to avoid the problem of incomplete model convergence caused by the large difference in the dosage of each patient. Next, input the processed data of the four time points into the time series model for training and prediction of the disease course development at the fifth time point in red. Then, slide down to the blue box as the four time points entered into the model at one time, and predict the light red disease representation at the next time point. The data is then input to the time series model with a fixed-length sequence, and the feature capture of the temporal relationship is carried out through the input gate, forget gate, output gate, and memory cell candidate elements in the network. The time loop network needs to calculate the weights through these components during the training process, and too many layers of the time loop network will cause the problem of slow training or underfitting. As the sample size of source disease was large in this study, in view of the time and hardware cost, this study set a one-layer time loop network and 128 hidden layer neurons to build a time series model, and the last output of the network enters the fully connected layer to predict the development of the disease course at the next time point. The dotted line in Figure 4 represents the operation flow after the output of the time series model, and here, LSTM is used for illustration. As shown in Figure 4, the time series model utilized in our framework consists of three kinds of deep learning neural networks, LSTM, GRU, and Bi-LSTM. The detailed explanation of these deep learning neural networks is listed as follows:  
* Long short-term memory (LSTM) is a sort of recurrent neural network which is composed of a memory cell \(c_{t}\), an input gate for input vector \(X_{t}\), an output gate \(o_{t}\) and a forget gate for hidden vector \(h_{t}\). Figure 5 shows the structure of a neuron of the LSTM. Meanwhile the notion \(o_{t}\) indicates the output vector of \(t\) terms. To realize the long-term memory, the notion of \(c_{t}\) represents a matrix which is utilized for remember the past input. * Gated recurrent units (GRUs) is a sort of gating mechanism in a recurrent neural network which is composed of a memory cell \(c_{t}\), an input gate for input vector \(X_{t}\), and a forget gate for hidden vector \(h_{t}\), but without output gate. Therefore, the memory cell is utilized for output.