# Time Series Visualization and Forecasting from Australian Building and Construction Statistics
## 4 Experiments
### Lstm Model Performance
#### 4.2.2 Varying \(nIn\) and \(nOut\)  
After deciding the number of epoch, the experiments took with the number of epoch set to 20. In this experiment, PCA dimension \(dimPCA\) is set to one, which means that the features will be reduced to one dimension; only the most important common feature will be extracted and used in this experiment. \(nIn\) and \(nOut\) are set in the range of one to five. The reason for setting the range is that the model will lose accuracy quickly if the \(nOut\) is set to be too large, compared to the total number of data points; so, the upper range is set to be five. For example, if we let the model predict the next twenty years of data, with only 70 data points provided, which is meaningless, it is not possible to make precious forecasts. As for the \(nOut\), we also set the upper range of it to five because we used the window size approach. To make to window size identical for both \(nIn\) and \(nOut\), we set the range to be five for \(nIn\), as well.  
Both \(nIn\) and \(nOut\) are in the range of one to five, so that there are, in total, 25 possibilities for the combination. We set the \(nIn\) as the outer loop and \(nOut\) as the inner loop to go through all the combinations. To evaluate the results, the root mean square error (RMSE) value and time for computing the model were recorded. RMSE is used for assessing the accuracy of the model, which is the distance between the prediction and actual data points. Time is to visualize the trade-off between accuracy and performance, and it also checks if the model is practical to be put onto the Web application. If it takes a lot of time, without too much improvement on the accuracy, it may not be a good decision to pursue the highest accuracy. However, if the accuracy dramatically increases as the time goes up, it needs to be considered which one to pick to fit in the Web application, without losing too much accuracy.  
Figure 7 illustrates the results. In Figure 7, there are two rows on the \(X\) axis. The upper row is the \(nOut\) value, and the lower one is the \(nOut\) value. The \(X\) axis shows all the combinations from one to five. From Figure 7, it can be seen that, when \(nIn\) is one and \(nOut\) is two, this model gets the lowest RMSE, and it also manages to achieve a reasonable computing time. Besides, when \(nIn\) is one, with an increase of \(nOut\), the RMSE value surprising drops down, and the average RMSE value of the model is also the lowest, compared with the others. The drop down change is huge, as compared with the rest four value of \(nIn\).  
Figure 6: Loss value during training.  
For \(nIn\) from two to five, the RMSE value goes up greatly when the \(nOut\) increases, especially when \(nOut\) is five, and the RMSE value gets to its highest value. Additionally, it can be found that the gradient goes up quickly when \(nOut\) gets larger, which means that the model will lose accuracy rapidly when the number of future predicting points increases. From the RMSE+1 line, it can be found that it has the lowest average RMSE value, and it has several points that are far lower than the others. However, when \(nIn\) is one, the more future points it predicts, the lower RMSE it achieves at the end. Overall, the computing time increases with the increase of \(nOut\), but the time does not vary too much; time for all models drops in a reasonable and acceptable range.  
Overall, the computation time for all the models are close, due to the reason that the \(dimPCA\) is set to one; the biggest difference is less than three seconds. Therefore, we could chose the model with the highest accuracy, without sacrificing much time.