# Approaches for Dealing with Seasonality in Clinical Prediction Models for Infections
## 3 Materials and Methods
### Proposed Adaptations to Dail with Seasonality
#### 3.4.2 Ensembles  
The ensemble approach consists of training models with data from different time intervals, then combining their outputs when predicting a new observation [16, 37, 38]. In this case, models created with older data can be restored and used again when the correlations between the outcome class and the features return to a previous state in the stream (i.e., recurring concepts). The literature contains many variations of this framework depending on the strategy used to maintain older models, update them, or merge their results into the final output [16]. However, this generic framework and its subsequent refinements require a dataset with a strict order between its observations; therefore, it cannot be applied to de-identified clinical data. We propose an adaptation of the ensemble framework to deal with the problem of seasonality in open clinical data. The method is formalized by two functions, _EnsembleTraining_ (Algorithm 3) and _EnsemblePredict_ (Algorithm 4). Let \(\mathcal{D}\) be the dataset available for training, where \(D_{i}\) corresponds to the data for the \(i\)-th month and where \(i=\{1,\ldots,12\}\). We build a prediction model for each month using the data regarding that specific month (EnsembleTaining, lines 1-6). The model trained using \(D_{i}\) is denoted as \(K_{i}\). It is then necessary to calculate the weights used to combine the outputs of the models in the ensemble (_EnsembleTraining_, lines 7-12). This is done by again iterating over the training dataset, and each model \(K_{i}\) is tested on the training data of every month \(D_{j}\), where \(j=\{1,\ldots,12\}\). The Root Mean Squared Error (RMSE) is then estimated for each pair \((K_{i},D_{j})\). Figure 1: Examples of our proposed adaptation of the sliding window algorithm for seasonality; (**a**) shows the steps to train the models assuming a window of three months, while (**b**) shows the steps for estimating the class to which a new observation belongs. ```
0:\(n\): number of models of the ensemble. \(n=12\) for the monthly ensemble, \(n=4\) for the seasonal ensemble
0:Training dataset
0:\(\mathcal{K}\): set of models, one for each month/season
0:\(W\): \(n\times n\) weights matrix. 1: Aggregate data in \(\mathcal{D}\) by month/season
2:\(R\leftarrow\) Initialize an empty \(n\times n\) matrix to store root-mean squared errors from models
3:for each month/season \(i\)do
4:\(D_{i}\leftarrow\) subset of \(\mathcal{D}\) gathering data from month \(i\)
5:\(K_{i}\leftarrow\) model trained using \(D_{i}\)
6: Include \(K_{i}\) in \(\mathcal{K}\)
7:for each month/season \(j\)do
8:\(D_{i}\leftarrow\) subset of \(\mathcal{D}\) including data from month \(j\)
9:\(R[i,j]\leftarrow\) Root mean squared error obtained when applying \(K_{i}\) to \(D_{j}\)\(\triangleright\) In case \(R[i,j]=0\), we assume \(R[i,j]=10^{-9}\)
10:for each month/season \(i\)do
11:for each month/season \(j\)do
12:\(W[i,j]\leftarrow\frac{\frac{1}{K[i,j]}}{\sum_{i=1}^{n}\frac{1}{R[i,j]}}\)
13:return\(\mathcal{K}\), \(W\)
```  
**Algorithm 3**_EnsembleTraining_ (monthly/seasonal)  
```
0:\(d_{m}\): Observation in month/season \(m\) for prediction
0:\(\mathcal{K}\): Set of trained models, one for each month
0:\(W\): Weight matrix
0:\(n\): number of models of the ensemble. \(n=12\) for the monthly ensemble, \(n=4\) for the seasonal ensemble
0:\(p_{d_{m}}\): Prediction of the observation \(d_{m}\)
1:\(m\leftarrow\) extract month/season from \(d_{m}\)
2:\(p_{d_{m}}\leftarrow\sum_{i=1}^{n}W[i,m]Prediction(K_{i},d_{m})\)
3:return\(p_{d_{m}}\)
```  
**Algorithm 4**_EnsemblePredict_ (monthly/seasonal)  
It is worth mentioning that (1) the lowest error is usually obtained when \(i=j\), and the model \(K_{i}\) is tested using the same data employed to build it (i.e., \(D_{i}\)); and (2) the highest error usually occurs when it is tested with data in which the seasonal effect has the greatest effect on the correlations when compared to \(D_{i}\), say, from month \(z\).