## 3 Machine Learning Techniques
### Machine Learning Results  
The data obtained in Section 2.2 was fed into the DNN model with the proportion in percentage: training set/validation set/testing set = 70%/15%/15%. The training, validation, and testing sets make the specific roles in the machine learning process: all the samples in the training set are used for training the various models and the responsibility of the validation set is to compare their performances and choose which one to employ; on the other hand, the testing set is kept separately from the remaining to verify whether the final predictions of the trained model are correct or not. If the predictions tend to be distinct from the corresponding actual values, the training process needs to be reset.  
Table 3 shows the structure of the proposed DNN model, where the activation functions are Rectified Linear Unit (ReLU) for hidden layers and linear for the final layer to  
Figure 3: The block diagram of the proposed method.  
generate the numerical output [39; 40]. All these parameters are well defined for our dataset. The proposed model was trained using _MSE_ loss function, optimized by [PERSON] [41] with learning rate of 0.001 and batch size of 128. A set of sequential Dense layers--the basic layer which consumes the input and computes the new output by performing matrix-vector multiplications--is the main component of this network. Adding the dropout layer behind each Dense layer was considered due to its efficiency in preventing overfitting (a concept in data science that a model performs well on training data but tends to fail on the untrained data such as testing set; this is not the desired result for every type of model in machine learning). However, dropout regularization was claimed to be not well-performing in regression task as in classification and natural language processing tasks [42]. Therefore, due to experiments, the dropout layer is not used in our model. The number of units in Dense layers are adjusted by trying multiple combinations to find the best preciseness for our dataset. The detailed information for the layers is summarized in Table 3.  
Figure 4 shows a comparison between the actual value and the predicted one in the testing set to visualize the performance of the proposed method. Each dot in the figure is formed by the numerical actual value as the \(x\)-coordinate and the corresponding predicted values as the \(y\)-coordinate. Therefore, if two values are more and more approximate, the point generated from them (the blue dots) will be closer and closer to the function graph \(y=x\) (the diagonal red line in the figure). Based on this figure, it could be argued that our prediction in testing phase is approaching the perfect performance, which confirms the efficiency of our DNN model.  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Layer** & **Layer Type** & **Number of Units** \\ \hline Input & Input & \\ Dense 1 & Hidden & 512 \\ Dense 2 & Hidden & 256 \\ Dense 3 & Hidden & 128 \\ Dense 4 & Hidden & 64 \\ Dense 5 & Hidden & 32 \\ Dense 6 & Hidden & 16 \\ Dense 7 & Hidden & 8 \\ Dense 8 & Output & 1 \\ \hline \hline \end{tabular}
\end{table}
Table 3: The structure of the proposed DNN model.  
Figure 4: Testing set performance.  
Table 4 shows the comparison with two other machine learning techniques such as Lasso [43] and Linear regression [44]. _MSE_ and R-squared score calculated on the testing set are utilized as the aspects of comparison. As mentioned before, smaller MSE and higher R-squared score denote a better model, therefore, our DNN model with _MSE_ of 0.0353 and R-squared score of 0.999 is superior to both Lasso and Linear regression. In the regression problem, the proposed DNN outperforms the other methods.