## 3 Proposed Method
### Fusion Network  
In the previous section, we used the feature engineering technique to create more features. A practitioner can typically understand which features are essential for model training using visualization techniques or statistical analysis. For those identified as essential features, the model should use them directly during model training. In contrast, the model can use a deep neural network to extract essential features from the remaining features that do not have explicit information. Thus, this work proposes a deep learning model called a fusion network to realize the idea. We separated the features into shallow and deep features, where the shallow features were those considered important, whereas the deep features required further nonlinear transformation. Figure 4 shows the architecture of the fusion network. This work used the neighbor features and one-hot-encoding of the stage feature as the shallow features, whereas the remaining features were considered deep features. All data samples comprised 22 shallow features and 84 deep features. In our developed model, we used a series of fully connected layers, namely the linear layer shown in Figure 4, to perform the nonlinear transformation. It can be seen in Figure 4 that the deep features had to go through four layers so this work used several approaches to prevent the model from overfitting. Figure 4: Fusion Network Architecture. First, we applied the min-max normalization technique to transform the features by scaling each feature to the range of \(-1\) and \(1\) so that features with large values did not dominate the model training. Second, each linear layer was followed by a batch normalization layer. Training a deep neural network tends to use a batch size of data samples as input and this can inject noise into each gradient update while achieving relatively fast convergence [32; 33] from GPU parallelism. However, when the weights are updated, the distribution of the inputs to the deep layers of the network may change. This phenomenon is called the internal covariate shift, which leads to a decrease in actual learning efficiency. [PERSON] et al. [34] presented batch normalization to mitigate the effect of unstable gradients within deep neural networks. Given the batch input \(x\), the formula is shown below in Equation (1):  
\[x^{*}=\frac{x-E[x]}{\sqrt{Var[x]+\epsilon}}\times\gamma+\beta, \tag{1}\]  
where \(x^{*}\) is the new value after normalization, \(E[x]\) is the mean within a batch, \(Var[x]\) is the variance within a batch, \(\epsilon\) is an arbitrarily small constant for numerical stability, and \(\gamma\) and \(\beta\) are learnable parameters. Third, we used the rectified linear unit (ReLU) [35] activation function to perform the nonlinear transformation. ReLU is probably one of the most commonly used activation functions, as it can significantly accelerate model convergence [36] and alleviate the problem of the vanishing gradient. The formula of the ReLU function is shown below in Equation (2):  
\[\text{ReLU}(x)=\max(0,x) \tag{2}\]  
Finally, we applied the dropout [37] technique to the layers to randomly drop out neurons during the model training phase. At each training epoch, neurons were ignored with a probability \(p\), forcing the neural network to learn more robust features. In particular, we applied batch normalization, the ReLU activation function, and dropout with a probability of \(0.1\) to all the layers of the deep neural network, as shown in Figure 4, in which \(p=0.1\) was obtained from cross-validation. The experiments further analyzed the influence of the dropout probability and batch normalization on model performance. Once the learning of deep features was completed, we concatenated the shallow features and deep feature embeddings that passed through the deep neural network. Subsequently, we used two simple linear layers to fuse these two kinds of features. The final output was a scale value, as the prediction of the MRR is a regression problem.