## 5 Explaining Model Outputs with SHAP
In Figure 9, the diagonal terms are the  
Figure 8: SHAP values of three features for all sample points in training set. Training window: 18 January–22 March 2023; trained variable: temperature; site: Archerfield Airport, **(a)**\(T_{\text{w}}\); **(b)**_Hour_; **(c)**\(U_{10}\). Figure 7: Global view of SHAP values on training set for top 10 features, **(a)** SHAP summary plot, coloured by feature value of each point; **(b)** feature importance, calculated by aggregation of absolute value of SHAP of each point. Training window: 18 January–22 March 2023; trained variable: temperature; site: Archerfield Airport. main effect while other terms are interaction effects between two features. The cells in the figure are coloured by the magnitude of the scaled SHAP interaction values. To better show the relative magnitude of interaction effects compared with main effects for each feature, each column in the plot is scaled separately so that the diagonal terms (the main effects of each feature) are scaled to 1.0. Consequently, the matrix is no longer symmetrical. The matrix shows that the interaction effects of four temperature features \(T_{\text{W}}\), \(T_{\text{SE}}\), \(T_{\text{S}}\) and \(T_{\text{SW}}\) with other features are much smaller than their main effects. This agrees with Figure 7a in that the SHAP values of temperature depend almost entirely on the feature value. By comparison, the interaction effects are more significant for dew point and wind speed, as some of the interaction effect terms are of similar magnitude with the main effects. The analysis of the relationship between each feature and its corresponding SHAP value reveals some insights into the mechanism of the temperature forecast model. The model output can be approximately described as a combination of a main component from linear combinations of selected NWP predictions of the main variable at different locations of the 3 \(\times\) 3 grid, and some supplementary components of lesser magnitudes. The supplementary components are related to features that are other than the predicted variable, such as hour, dew point, wind speed, etc. The effects of these features are highly nonlinear but can be significant to the model predictions at some points. These two categories of features are hereinafter referred to as 'linear features' and 'nonlinear features', respectively. The dew point forecast models are found to exhibit similar patterns as the temperature forecast model, so that the discussions of them are not repeated here. Figure 9: Normalized SHAP interaction values between each pair of features (normalized at each column), coloured by SHAP interaction value. Training window: 18 January–22 March 2023; trained variable: temperature. Site: Archerfield Airport. Values below 0.01 are not considered significant but we have left them in for the reader to assess in context. As previously discussed, SHAP values are used for explaining and understanding model outputs, but do not include any information on the accuracy of the model. However, the insights can be used to debug and monitor the model's behaviour. The next section will focus on analysing model error to increase ML reliability, in which SHAP values play an important role.