# Prediction of Dynamic Responses of Flow-Induced Vibration Using Deep Learning
## 3 Deep Learning Model for Dynamic Response Prediction of FIVs
### Deep Learning Model Training  
In this study, cross-validation was used to learn the overall characteristics of the experimental data and increase the reliability of the generalization performances of the deep learning models. There are various cross-validation methods available; however,  
\begin{table}
\begin{tabular}{c c c c} \hline \hline  & **Spring** & \(\zeta_{\text{hames}}\) **for Training** & \(\zeta_{\text{hames}}\) **for Test** & **Number of** \\  & **Constants (K)** & **(No. of Training Data)** & **(No. of Test Data)** & **Total Data** \\ \hline \multirow{4}{*}{Tested data by} & 400 & 0.04, 0.08, 0.12, 0.16, 0.2, 0.24 (150) & - & 150 \\  & 600 & 0.04, 0.08, 0.12, 0.2, 0.24 & 0.16 & 144 \\  & &  
the repeated random subsampling validation method, using which the ratio between the training and validation sets is adjusted and not affected by the number of training data, is employed as shown in Figure 9.  
The goal of the deep learning model optimization is to determine the values of the hyperparameters such that the learning of the neural network models and value of the loss function according to their results are minimized. Therefore, fine tuning was performed by controlling the number of nodes per hidden layer. The optimal number of nodes was selected by repeating the increase and decrease in the number of nodes. In addition, the hyperparameters, such as learning rate, iteration, and batch size, that affect the neural network were adjusted. Then, the functions for learning the deep learning model were selected. The activation function converts the sum of the input signals into output signals, such as the step function used in perceptrons. At the nodes of the hidden layers, the sum of the linear product of the data and its weight are calculated, and the threshold is applied to obtain the activation level [18]. In this study, the rectified linear unit (ReLU) activation function was used [19; 20]. Compared to the sigmoid function, which has been majorly used in the past, the ReLU can be implemented with a simple formula, and the function also resolves the vanishing gradient problem, which was considered as a chronic problem in the deep learning model [21]. The function is expressed as a graph in Figure 10 and as an equation in Equation (3). \[f=\left\{ \begin{array}{l}x<0,\ f(x)=0\\ x\geq 0,\ f(x)=x\end{array} \right. \tag{3}\]  
Figure 10: Schematic of the ReLU function. Figure 9: Random subsampling validation method to reduce bias in the learning data by randomly extracting validation data for each iteration, thereby preventing underfitting and overfitting. The optimization function accelerates the learning of the deep learning models and uses the Adam optimizer, which is efficient in terms of computational resources. The cost function uses the root mean square error (RMSE), and the He initialization is used for the weight initialization method. The He initialization method is to initialize the ReLU function and considers the characteristics of the input and output values rather than random initialization [22]. In this study, the He initialization method was used, as shown in Equation (4). The initialization for the ReLU activation function considers the characteristics of the input and output values, rather than random initialization. Depending on the characteristics of the data, when there were n nodes in the previous layer, a normal distribution with a square root of 2/n is used. This method was able to significantly increase the learning speed of the neural network, and it was confirmed that the loss function was reduced early in the learning process. \[ \begin{array}{l}\text{He Normal Initialization}\;\text{W}\;\sim\;\text{N}(0, \text{Var}(\text{W}))\\ \text{N}=\text{Normal distribution}\\ \text{W}=\text{Weight between nodes}\\ \text{Var}(\text{W})=\sqrt{\frac{2}{n_{int}+n_{int}}}\\ n_{in}\;=\;\text{Number of nodes in the previous layer}\\ n_{out}\;=\;\text{Number of nodes in the next layer}\end{array}  \tag{4}\]  
Learning was performed based on a batch size of 300.