# Approaches for Dealing with Seasonality in Clinical Prediction Models for Infections
## 4 Experiments and Results
### Separate Analysis of the Effects of Different Approaches  
We initially performed an analysis of the effects of using each set of techniques described in this work separately, i.e., feature selection to reduce high dimensionality, sampling to compensate class imbalance, and approaches for dealing with seasonality in data.  
Figure 7a compares the results of each feature selection approach in terms of mean AUC, while Figure 7b compares the number of features included in the models, with the aim of illustrating the variations in model complexity.  
The use of FCBF without combining it with other techniques tends to reduce the model AUC, with the exception of the _condensed_ dataset and when used in logistic regression models. The \(p\)-value filter has less of an impact on the model AUC, and leads to a slighter reduction in model complexity.  
The approaches based on decision trees obtained poor AUC results on all of the MIMIC-III datasets. In these cases, the high class imbalance led to tree models with only one node (zero features per model), resulting in all the observations being classified as belonging to the majority class regardless of the feature selection approach used.  
The results when varying only the method employed to compensate the class imbalance are shown in Figure 8. When combined with logistic regression models, the balancing approaches achieve a similar or slightly lower AUC than the models without a balancing strategy. However, the performance improves when using decision trees, which is particularly relevant on the MIMIC-III datasets as the models are no longer empty. With regard to model complexity, undersampling leads to simpler models than oversampling.  
Figure 6: \(p\)-value of different features when considering data from a specific season only in (**a**) the _condensed_ dataset, (**b**) the _sinusoidal_ dataset, (**c**) the _Acinetobacter_ dataset, and (**d**) the \(S\). _pneumonia_ dataset. The dashed line represents the traditional cut-off of \(p=0.05\) used in feature selection.  
Figure 9 shows the results obtained when using the seasonality approaches without any other preprocessing techniques. There are differences between the synthetic and clinical datasets. The use of ensembles, one model per season, or a three-month window improves the AUC results in the synthetic datasets, while leading to worse results on the MIMIC-III datasets with logistic regression. In these experiments, wider windows, the inclusion of the season as a feature, and even using no seasonal approach at all led to better performance. The results according to model complexity were more homogeneous. The use of the monthly ensemble and three-month window clearly reduced model complexity. Again, the high class imbalance in the MIMIC-III datasets led to one-node trees, resulting in models with a poor AUC.  
These results indicate that the use of a seasonal ensemble leads to the same results as creating one model per season. Although they are different algorithms, their outputs are sufficiently similar that both the AUC and the number of features in the resulting models are identical.  
Figure 7: Values of (**a**) AUC and (**b**) number of features per model when using only a feature selection approach in each dataset.