## 3 Model Training, Optimization and Explainability
### Model Training and Optimization
This is detrimental for the prediction of extreme values, such as new high records of temperature. To improve the performance, the original data are column-wise scaled to achieve a standard distribution for training and validation sets. Note for validation sets that the mean and variation is calculated based on the data of the last week of the training period instead of the whole training period. This matches the operational use case, whereby the forecast period is in the future and, as an unknown, the exact scaling factor cannot be known in advance. Using the most recent week aims to ensure that the calculated mean is the closest estimation of the upcoming validation period. This process may prevent inaccurate mean values being applied on the validation set when daily average temperature is constantly increasing. With this process, the bound of the two sets can mostly overlap with each other. The effect of scaling is quantified in detail in Table 1.  
In addition, the current study adopts a sliding window approach to continuously train new models on a nine-week training set and test on the subsequent one week. The advantage is that most recent weather patterns can be captured. Sliding window also \begin{table}
\begin{tabular}{l c c c} \hline \hline \multirow{2}{*}{**Sites/Preprocessing Methods**} & **Change of RMSE, If Feature Selection Is Not Performed** & **Change of RMSE, If Surrounding Points Are Excluded** & **Change of RMSE, If Scaling Is Not Performed** \\ \hline Alice Springs Airport & \(-\)0.01 & \(-\)0.09 & 0.50 \\ Archerfield Airport & 0.01 & 0.02 & 0.20 \\ Avalon Airport & 0.01 & \(-\)0.03 & 0.13 \\ Coffs Harbour & 0.03 & 0.12 & 0.19 \\ Airport & 0.00 & 0.12 & 0.24 \\ Curtin Aero & 0.01 & 0.19 & 0.09 \\ Geraldton Airport & 0.01 & 0.03 & 0.25 \\ Hobart (Ellerslie & \(-\)0.01 & 0.03 & 0.25 \\ Road) & 0.01 & 0.04 & 0.68 \\ Mount Isa & 0.01 & \(-\)0.10 & 0.00 \\ Tindal RAAF & \(-\)0.01 & \(-\)0.10 & 0.00 \\ Townsville & 0.02 & 0.09 & 0.28 \\ Woomera Aerodrome & \(-\)0.02 & \(-\)0.01 & 0.30 \\ SUM & 0.05 & 0.37 & 2.87 \\ AVERAGE & 0.005 & 0.034 & 0.261 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Change of RMSE (°C) of temperature predictions averaged on all lead days for each site when the different pre-processing approaches are absent from the data pipeline. Negative means better prediction accuracy. Prediction period: 27 October–27 November 2022.  
utilizes the fast training enabled by XGBoost to avoid data drifting due to infrequent model updates. Hyperparameters are tuned with standard 5-fold cross validation. Table 1 shows a parametric study based on the temperature and predictions of a one-month period. Each column shows the resultant change in the average hourly RMSE caused by different data pre-processing settings. The change of RMSE is calculated by considering all lead times with equal weight, and the sum in each column is the aggregation of all changes from all tested sites. The results show that the prediction with all data pre-processing techniques applied has the lowest RMSE, and omitting any of the pre-processing would lead to higher error. As a first step towards a working prototype of a production system, a framework for automatic downloading, data pre-processing, ML training and validation framework is developed and named Multi-SiteBoost (MSB). The framework comprises three parts: (1) gridded IMPROVER data and site observation data are downloaded and stored. Data for the selected sites and features are then extracted from the large volume of data into time series; (2) an XGBoost model is automatically trained for each site and each sliding window in parallel; (3) the saved models are used to generate predictions, metric statistics and evaluate pointwise reliability on each of the predictions. Figure 4 shows the flow chart of the MSB workflow. ML results as discussed in the following sections are all generated by the MSB framework.