# Approaches for Dealing with Seasonality in Clinical Prediction Models for Infections
## 4 Experiments and Results
### Analysis of the Impact of the Different Approaches on Interpretability
\begin{table}
\begin{tabular}{c c c c c} \hline \hline
**Filter** & **Balancing** & **Seasonal Drift Approach** & **Mean AUC** & **Mean Features per Model** \\ \hline \(p\)-Value & Undersample 1:1 & Seasonal window & 0.690 (0.682, 0.698) & 6.628 (6.271, 6.984) \\ \(p\)-Value & Oversample 2:1 & Season as feature & 0.685 (0.678, 0.692) & 37.590 (37.231, 37.949) \\ None & Oversample 2:1 & Season as feature & 0.685 (0.678, 0.691) & 40.480 (40.031, 40.929) \\ FCBF & Oversample 1:1 & Season as feature & 0.684 (0.677, 0.691) & 9.780 (9.494, 10.066) \\ \(p\)-Value & Oversample 1:1 & Season as feature & 0.684 (0.677, 0.690) & 37.950 (37.569, 38.331) \\ None & Oversample 1:1 & Season as feature & 0.683 (0.676, 0.690) & 40.370 (40.008, 40.732) \\ \(p\)-Value & Oversample 1:1 & None & 0.682 (0.676, 0.689) & 34.810 (34.429, 35.191) \\ \(p\)-Value & Oversample 2:1 & None & 0.682 (0.675, 0.688) & 34.420 (34.061, 34.779) \\ None & Oversample 2:1 & None & 0.681 (0.675, 0.687) & 37.290 (36.893, 37.687) \\ \(p\)-Value & Undersample 1:1 & 9-month window & 0.681 (0.674, 0.687) & 6.900 (6.652, 7.148) \\ \(\cdots\) & \(\cdots\) & \(\cdots\) & \(\cdots\) & \(\cdots\) \\ None & None & None & 0.502 (0.499, 0.505) & 0.180 (\(-\)0.001, 0.361) \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Top ten results obtained with decision tree models on the _S. pneumoniae_ dataset extracted from MIMIC-III. the effect of the two main coefficients, \(x_{1}\) and \(x_{2}\), in the outcome variable throughout the year. The coefficient \(x_{1}\) was the most relevant factor in winter, while it had no effect on the outcome in summer. The feature \(x_{2}\) had the opposite effect, being the most relevant in summer. The models selected here were trained with one of the samplings from the training/validation dataset without applying any filter or balancing strategy. By examining these models, we aim to highlight the differences in interpretability resulting from the different approaches to handling seasonality. The results shown here extend to both tree-based models and models generated with logistic regression; however, only the latter are shown in order to avoid duplication. Figure 10 shows the values of the coefficients of the logistic regression model generated without applying any seasonal drift approach (Figure 10a) and when including the season as an additional feature (Figure 10b). These are common logistic regression models that are easily interpretable; in this case, \(x_{1}\) and \(x_{2}\) are much more relevant than the rest of the features. Among the other features selected, those starting with \(c\_{1}\_\) and \(c\_{2}\_\) refer to the \(c_{i}^{1}\) and \(c_{j}^{2}\) features added to complicate the dataset, which are highly-correlated with \(x_{1}\) and \(x_{2}\), respectively. When the season is included as a feature (Figure 10b), it is included in the model, but it has little relevance compared to \(x_{1}\) and \(x_{2}\). All these findings are consistent with the known behavior of the underlying model that generated the dataset; the yearly variations in the relevance of \(x_{1}\) and \(x_{2}\) are smooth, and as such the model must consider both in order to generate a prediction when the dataset is treated as a whole. Figure 11 shows the models that were generated when using the strategy of creating a different model for each season. This model may appear more complicated to understand; however, because each model is used in a particular condition (i.e., to predict an observation of a particular season) and they can be interpreted separately, on the whole we consider it easy to interpret these models. For example, it can be observed that while the relevance of \(x_{2}\) is high in the summer model, it does not even appear for consideration in the winter model, in which \(x_{1}\) is by far the most relevant feature. Therefore, we can interpret this model, and even extract interesting information about the underlying impact of seasonality in the data.