## 5 Explaining Model Outputs with SHAP  
Figure 7a shows the SHAP summary plot on the scaled training set of a temperature forecast model. In the figure and all the analysis that follows, the temperature \(T\) and dew point \(T_{\text{d}}\) features extracted at different orientations as shown in Figure 3 are noted as \(T_{\text{W}}\), \(T_{\text{SE}}\), \(T_{\text{dW}}\), \(T_{\text{GSE}}\), \(\ldots\), with subscripts indicating the orientation. In the summary plot, the X axis is the SHAP value, and the 10 features used by this model are arranged along the Y axis. On each of the rows, every point represents a sample in the training set. The points are coloured by feature value. The rows of features are arranged by relative magnitude of SHAP values in descending order. The summary plot shows that for this model, the four selected temperature measurements \(T_{\text{W}}\), \(T_{\text{SE}}\), \(T_{\text{S}}\) and \(T_{\text{SW}}\) have the largest overall contributions, indicated by the extent of their SHAP values. A physical explanation for the degree of contribution of \(T_{\text{W}}\), \(T_{\text{SE}}\), \(T_{\text{S}}\) and \(T_{\text{SW}}\) has not been explored. This agrees with the intuition that the actual temperature is most relevant to the predictions from NWP models. Other supplementary features such as hour, wind and dew point at various locations in general have lower SHAP value magnitude, which implies they give minor changes to each prediction. However, the rows of \(T_{\text{GSE}}\) and \(T_{\text{dS}}\) also show that at certain samples, a high dew point can largely decrease the final predicted values of temperature. However, this effect is asymmetrical, as a low dew point does not result in a large positive SHAP value. Figure 7b is the global feature importance plot, calculated by averaging all SHAP value magnitudes of each sample for each feature. The resultant feature importance for this model agrees with XGBoost feature importance, calculated by maximum gain. Compared with Figure 7a, Figure 7b lacks the information of the distribution of feature importance on each sample. For example, it does not show the occasional significance of \(T_{\text{GSE}}\) on some certain points. This comparison highlights the importance of local explanation provided by SHAP, as supplementary information in addition to global feature importance. Figure 6: Histograms of pointwise forecast error on lead day 0 for (**a**) temperature; (**b**) dew point temperature. Site: Hobart (Ellerslie Road). Figure 8 plots the SHAP values of all samples with respect to feature values for three features: \(T_{\text{w}}\), _Hour_ (in UTC) and \(U_{10}\) in a model trained for temperature prediction. The plots show that the SHAP value of feature \(T_{\text{W}}\) is approximately linear with the feature value. The vertical dispersion is minor, and each value of \(T_{\text{W}}\) approximately corresponds to one single SHAP value. This agrees with Figure 2a, both of which can be interpreted as the NWP grid value being linearly correlated to site observation. By comparison, the SHAP value of hour (defined as a discrete integer rather than a continuous variable) is not linearly correlated to the feature value, and one value of hour can correspond to multiple SHAP values. However, it can still be observed that hour 0-5 generally has positive SHAP while hour 15-20 generally negative. This agrees with observations and statistics, as temperature at this site at UTC hour 0:00-5:00 (10:00-15:00 local time) and 15:00-20:00 are expected to be hotter and colder than the daily average, respectively. The SHAP values of wind, by comparison, are highly nonlinear, multi-valued and difficult to interpret, which indicates a highly nonlinear relationship between wind speed and temperature, and strong interactions between wind speed and other features. The interactions between features can be more clearly visualized by a SHAP interaction value matrix, as shown in the heatmap (Figure 9).