## 6 Discussion of the Results
### Regression  
Regarding the regression, the results are similar. The ViT models perform superior, with the ViT-S-PREC reaching a coefficient of determination between measured and predicted precipitation rate of \(r^{2}=85.63\%\) with a size of just 640 k parameters (see Table 4), which corresponds to an MSE of 0.22 mm\({}^{2}\)/h\({}^{2}\). Again, the small model version ViT-S-PREC-s with just 400 k parameters achieved a comparative performance. The other transformer-based architectures, however, only achieved coefficients of determination of \(\leq\)80%, whereas the baseline models were not able to conduct any meaningful prediction at all.  
In a final experiment, the authors evaluated whether an embedded SYNOP class could potentially enhance the prediction of the respective precipitation rate by following the assumption that a correctly predicted class prior could support the network in deriving the precipitation rate, especially in the described edge cases. In order to do so, the ground truth  
\begin{table}
\begin{tabular}{l c c c c} \hline \hline
**Model** & **MSE [mm2/h2]** & **r2** & **\# Params** & **MACs** \\ \hline \hline Vision Transformers & & & & \\ \hline ViT -B-PREC & 0.32 & 78.80\% & 0.83 M & 1.90 G \\ ViT-B-PREC-s & 0.43 & 69.43\% & 0.31 M & 0.70 G \\ ViT-S-PREC & **0.22** & **85.63\%** & 0.64 M & 2.91 G \\ ViT-S-PREC-s & 0.24 & 84.00\% & 0.40 M & 1.48 G \\ \hline \hline Convolutional Transformers & & & & \\ \hline CCT-PREC & 0.33 & 77.69\% & 0.75 M & 1.15 G \\ \hline Video Vision Transformers & & & & \\ \hline ViViT-PREC & 0.53 & 64.48\% & 1.23 M & 0.25 G \\ \hline \hline \end{tabular}
\end{table}
Table 4: Comparison of model sizes and respective regression performance on the test dataset. The best performance in terms of MSE and \(r^{2}\) is marked in bold, whereas the correspondingly required computational resources are indicated by the number of model parameters (# Params) and MAC operations.  
Figure 14: Test data classification results of the ViT-B-PREC model in the form of normalized confusion matrices, evaluating the predicted precipitation classes (Class ID pred) with respect to the actual, measured value (Class ID meas).  
SYNOP classes were embedded into the input of the first transformer encoder as a class token. For that, two methods are tested: Firstly, a trainable linear layer (linear embedding) is implemented in order to transform the classification output as one-hot encoded vector to a latent vector which matches the transformer input tensor shape, such that both can be concatenated. Secondly, the one-hot encoded classifier input is simply concatenated with zeros (zero padding) in order to match the input tensor shape of the transformer. As illustrated in Figure 15 and Table 5, performance is enhanced notably in both cases, with a coefficient of determination of up to \(r^{2}=92.73\%\). Comparing the regression curves, one can see a significant variance decrease in the critical region between \(R=5\,\mathrm{mm}/\mathrm{h}\) and \(R=25\,\mathrm{mm}/\mathrm{h}\), which indicates a clearly improved regression in this region. However, since the classification performance was weakest in exactly the two covered transition regions between classes 4, 5, and 6, the application of an SYNOP class embedding from a prior class prediction did, opposed to the ground truth embedding, not further improve regression performance.