## 3 Development of an HS-ANN Hybrid Model
### ANN Model  
The multi-perceptron is considered as an attractive alternative to an empirical formula in that it imitates the nonlinear relationship between input and output variables in a more simplified way. The model aims to obtain the optimized weights of the network using a training algorithm designed to minimize the error between the output and target variables by modifying the mutually connected weights. In this study, the multi-perceptron with one hidden layer is used as shown in Figure 4, where \(i\) is the number of input variables.  
Firstly, for each of the input and output variables, the data are normalized so that all of the data are distributed in the range of [min, max] = [-1, 1]. This can be done by subtracting the average from the data values and rescaling the resulting values in such a way that the smallest and largest values become \(-1\) and \(1\), respectively. Secondly, the initial weights in the hidden layer are set to have random values between \(-1\) and \(1\), and the initial biases are all set to zero. The next step is to multiply the weight matrix by the input data, \(\mathbf{p}\), and add the bias so that  
\[n_{k}^{h}=\sum_{j=1}^{J}w_{kj}^{h}p_{j}+b_{k}^{h},\quad k=1\text{ to }K, \tag{5}\]  
where \(J\) and \(K\) are the number of input variables and hidden neurons, respectively, and \(\mathbf{p}\), \(\mathbf{b}^{h}\) and \(\mathbf{w}^{h}\) are the input variable, bias, and weight in the hidden layer, respectively. The subscripts of the weight \(w_{kj}^{h}\) are written in such a manner that the first subscript denotes the neuron in question and the second one indicates the input variable to which the weight refers. The \(n_{k}^{h}\) calculated by Equation (5) is fed into an activation function, \(f^{h}\), to calculate \(a_{k}^{h}\). Hyperbolic tangent sigmoid function is used as the activation function so that  
\[a_{k}^{h}=\frac{e^{n_{k}^{h}}-e^{-n_{k}^{h}}}{e^{n_{k}^{h}}+e^{-n_{k}^{h}}}. \tag{6}\]  
Figure 4: Network topography of ANN.  
In the output layer, the same procedure as that in the hidden layer is used except that only one neuron is used so that  
\[n_{1}^{o}=\sum_{j=1}^{K}w_{1j}^{o}a_{j}^{h}+b_{1}^{o}, \tag{7}\]  
and the linear activation function is used to calculate \(a_{1}^{o}\) so that  
\[a_{1}^{o}=n_{1}^{o}. \tag{8}\]  
The neural network with the randomly assigned initial weights and biases cannot accurately estimate the required output. Therefore, the weights and biases are modified by the training to minimize the difference between the model output and target (observed) values. To train the network, the error function is defined as  
\[\varepsilon=\ ||\boldsymbol{\tau}-\mathbf{a}_{1}^{o}||^{2}, \tag{9}\]  
where \(||\ ||\) denotes a norm, and \(\boldsymbol{\tau}\) is the target value vector to be sought. To minimize the error function, the Levenberg-Marquardt algorithm is used, which is the standard algorithm of nonlinear least-squares problems. Like other numeric minimization algorithms, the Levenberg-Marquardt algorithm is an iterative procedure. It necessitates a damping parameter \(\mu\), and a factor \(\theta\) that is greater than one. In this study, \(\mu=0.001\) and \(\theta=10\) were used. If the squared error increases, then the damping is increased by successive multiplication by \(\theta\) until the error decreases with a new damping parameter of \(\mu\theta^{k}\) for some \(k\). If the error decreases, the damping parameter is divided by \(\theta\) in the next step. The training was stopped when the epoch reached 5000 or the damping parameter was too large for more training to be performed.