## 3 Methodologies  
### SAE-DNN-Based HI Constructor  
The multiple hidden layers concept has been available since the early years of deep learning. This approach was initially disappointing because its performance was even worse than shallow networks, the result of the limitations of conventional back-propagation due to poor training, which often utilized random initialization and got stuck in unoptimized local solutions. This difficulty was overcome in 2006 with unsupervised layer-wise pretraining, which was proposed by [PERSON] et al. (2006) to deal with the existing limitations of DNN optimization. Currently, more sophisticated and abstract features with hierarchical structures can be learned because pretraining offers layer-by-layer, high-level feature extraction from lower-level ones. Figure 3: Damage progression in an RC beam during a bending test. Figure 2: Schematic illustration of the four-point bending test for the RC beam. An excellent choice for an efficacious layer-wise unsupervised learning algorithm would be a stacked autoencoder (SAE) [29]. This structure includes multi-layer AEs, each of which is a single-hidden-layer unsupervised neural network, and its input/output layers are set identically. The original input is intended to be reconstructed by the output layer with high accuracy. A simplified AE structure is illustrated in Figure 4. The input of SAE, which is supposedly \(x=\left[x_{1},x_{2},\ldots,x_{n}\right]^{T}\in R^{n}\) with dimension \(n\), is projected by the encoder to the hidden layer \(h=\left[h_{1},h_{2},\ldots,h_{m}\right]^{T}\in R^{m}\) by the following mapping function \(f\):  
\[h=f(x)=s_{f}(Wx+b) \tag{1}\]  
in which \(m\) stands for the dimension of the hidden variable vector; \(W\) is the \(m\times n\) weight matrix; \(b\in R^{m}\) is the bias vector; and the nonlinear activation function \(s_{f}\) can be either the sigmoid function, the tanh function, or the rectified linear unit function. The hidden representation \(h\) is then mapped to the output layer \(\widetilde{x}\in R^{n}\) by the mapping function \(\widetilde{f}\) in the decoder:  
\[\widetilde{x}=\widetilde{f}(h)=s_{\widetilde{f}}(\widetilde{W}h+\widetilde{b}) \tag{2}\]  
with \(\widetilde{W}\) as an \(n\times m\) weight matrix; \(\widetilde{b}\in R^{n}\) as the bias vector term for the output layer; and the activation function \(s_{\widetilde{f}}\) as either the sigmoid function or others. Overall, \(\theta=\left\{W,\widetilde{W},b,\widetilde{b}\right\}\) is the AE parameter set and \(g_{\theta}(x)=\widetilde{f}(f(x))\approx x\) is the function to be learned. As aforementioned, an AE aims to get the reconstructed output \(\widetilde{x}\) as close to the initial input \(x\) as possible. This is done by forcing constraints on the network, for example, a hidden unit number limitation. Assume that the training input is \(X=\left\{x^{(1)},x^{(2)},\ldots,x^{(N)}\right\}\), where the total number of training samples is denoted as \(N\). Initially, each \(x^{(i)}\) is projected to a hidden representation \(h^{(i)}\) and then mapped to the reconstructed data \(\widetilde{x}^{(i)}\). By the following mean squared reconstructed error calculation, the reconstructed loss function can be minimized:  
\[ \begin{array}{ll}J(W,\widetilde{W},b,\widetilde{b})&=\sum\limits_{i=1}^{N} \left\|\widetilde{x}^{(i)}-\chi^{(i)}\right\|^{2}\left/2N\right.\\ &=\sum\limits_{i=1}^{N}\left\|g_{\theta}(x^{(i)})-\chi^{(i)}\right\|^{2} \left/2N\right.\end{array}  \tag{3}\]  
The gradient descent (GD) is used for AE parameter updating. After the completion of training, the weight and bias are then saved for this AE. The multi-AE-layered structure of an SAE is presented in Figure 5. The layer-by-layer training of an SAE is done as follows:  
Figure 4: Model structure of AE. Initially, the original input data is mapped in the first hidden feature layer by the first AE; the output of the first AE is then considered the input for the second AE, and the process continues. By the end, every layer in the SAE is pretrained.