## 2 Materials and Methods
### Regression Analysis
#### 2.3.2 Segmented Nonlinear Regression
We train both models with 10,000 epochs and choose the optimally performing model based on loss value. We adopt the root mean square error (RMSE) estimator as a loss function for the LSTM and Seq2 Seq LSTM models. We set the initial learning rate of LSTM and Seq2 Seq LSTM as 0.0001. For regression analysis, we train linear regression and segmented nonlinear regression with OLS to estimate the parameters. We carry out hyperparameter searching for segmented nonlinear regression. As a result, we discover that the model with hyperparameters \(T\) of 8 and \(\tau\) of 4 has shown the highest performance. We also set \(s\) hyperparameter as 0 in segmented nonlinear regression after we discover that it can introduce too high a variance in the experiment results. Since we handle time series data, we make use of the last segment's parameter in the segmented nonlinear regression model to perform inference, because the test set is the next continuity of the training set. ### Hardware Systems

For fair comparison, we perform a series of data preprocessing and experiments in the same hardware system. We use Intel Xeon E5-2620, which has eight cores and 16 threads in a central processor. To handle a huge dataset, we utilize 64 GB of RAM. Finally, we use Nvidia 1080 Ti 11 GB as a GPU to accelerate the training and inference of the LSTM and Seq2 Seq LSTM models. ### Evaluations

We choose the root mean square error (RMSE) as our quantitative evaluation measure for models' predictions in the test set. The model with a lower RMSE score indicates the best-performed model. We also analyze the long-term prediction capability of all models with 10-year growth predictions on the Crack A2 data. Figure 5: Illustration of the sliding window operation for Seq2 Seq LSTM on toy data. Blue boxes denoted as the input data for encoder, red boxes denoted as the ground truth to be predicted by decoder, orange rectangles denoted as the sliding windows. ## 4 Results

### Long Short-Term Memory (LSTM)

Figure 5(a) shows that the LSTM model's predictions are constant, and that the discrepancies between the ground truth are visible. The same figure shows the model produced a constant value of 0.078 throughout the end of the Crack A2 test set, while the ground truth varies with the highest value of 0.076. We observe the model performed poorly, with similar behavior in in the remaining models. As a result, the LSTM models achieved the worst RMSE score compared with other models. We believe this outcome is because the model failed to capture the lagged dependencies of the training data. ### Seq2 Seq LSTM

In Figure 5(b), we observe the Seq2 Seq LSTM model successfully emulated the Crack A2 test set from date 26 July 2019 until 27 July 2019. However, the model overshoots for the remaining test set with a margin of 0.024 at the last data point. We suspect the model showed such a behavior because it could not effectively learn temporal dependencies on the data that exhibited abrupt changes. Although the model exhibited overshooting, it achieved the best RMSE score among the models. Table 1 shows that the Seq2 Seq LSTM model outperforms the LSTM, linear regression, and segmented nonlinear regression models for all the test sets. ### Linear Regression

We observe the performance of the linear regression model is relatively effective on Crack A1 and Crack A2 test set. However, its predictions are inferior to other algorithms on the Crack B1 and Crack B2 test sets, as shown in Table 1. We assume that the model could not make a compelling prediction owing to a sudden change in the train set, as is observed from Figure 5(d). In the figure, we discover the linear regression model's predictions were

Figure 6: Evaluation of each model on Crack A2 test set. disparate than the ground truth on the Crack B1 train set. Therefore, the linear regression model produced an unsatisfactory result on the Crack B1 test set with an RMSE score of 0.12429; in the meantime, the Seq2 Seq LSTM model obtained 0.00253.