## 2 Materials and Methods
### The Need for Online Machine Learning
#### 2.2.2 The Evaluation Frame: How to Compare the Methods  
To compare OML to classical approaches, and to evaluate the strengths and weaknesses of various forecasting methods, different approaches were employed in our experiments. Three distinct methods were used to generate the training dataset for batch procedures. In order to provide a detailed explanation of these methods, it is important to first describe the experimental procedure. The objective of each method was to produce point forecasts with maximum accuracy for a predetermined horizon of 150 data points in the future. However, the development of models differed among the methods. For batch procedures, a classical train-test split was used, referred to as \"batch\" hereafter, along with a train-test split using a landmark approach and a train-test split with a shifting window. Additionally, the OML approach was utilized for model development. The evaluation functions eval_bml, eval_bml_landmark, eval_bml_window, and eval_oml_horizon accept two data frames as arguments:  
* train, denoted as \(D_{\text{train}}\), with size \(s_{\text{train}}\), is used to fit the model;
* test, denoted as \(D_{\text{test}}\), with size \(s_{\text{test}}\), is used to evaluate the model on new (unseen) data. First, the method eval_bml implements the \"classical\" BML approach. The algorithm is trained once on the training dataset, resulting in a model, denoted as \(M_{\text{bml}}\), that is not modified. The model \(M_{\text{bml}}\) is evaluated on the test data, where the horizon, denoted as \(h\in[1,s_{\text{test}}]\), specifies the size of the partitions that \(D_{\text{test}}\) is split into. If \(h=s_{\text{test}}\), then the basic ML train-test setting is implemented. If \(h=1\), an OML setting is simulated. Second, the method eval_bml_landmark implements a landmark approach. The first step is similar to that of the BML approach, resulting in an initial model \(M_{\text{bml}}^{(1)}\). However, subsequent steps differ: after making a prediction with \(M_{\text{bml}}^{(1)}\) for the batch of data instances from the interval \([s_{\text{train}},s_{\text{train}}+h]\), the algorithm is retrained on the interval \([1,s_{\text{train}}+h]\), to produce an updated model \(M_{\text{bml}}^{(2)}\). During the third step of the landmark BML approach, \(M_{\text{bml}}^{(2)}\) makes predictions for \([s_{\text{train}}+h,\text{train}+2\times h]\), and a new algorithm, \(M_{\text{bml}}^{(2)}\), is trained on \([1,\text{train}+2\times h]\). Third, the method eval_bml_window implements a window approach. Again, the first step is similar to that of the BML approach, resulting in an initial model, \(M_{\text{bml}}^{(1)}\). Subsequent steps are similar to those of the landmark approach, with one important exception: instead of being trained on the complete set of seen data, the algorithm is trained on a moving window of size \(s_{\text{train}}\). Finally, the method eval_oml_horizon implements an OML approach. This approach differs fundamentally from BML approaches because every single instance is used for both prediction and training. If \(h=1\), a \"pure\" OML algorithm is implemented. If \(h>1\), OML computations are performed \(h\) times. A summary of the training and test set generation process, related to the corresponding evaluation procedure, can be found in Table 1 in the Appendix A. Additionally, Figure 2 visualizes the differences between the evaluation techniques introduced. Several criteria were used to evaluate the different approaches, including Mean Absolute Error (MAE),computation time, and memory consumption. The selection of these metrics was based on the different requirements that an end user might have for the system. While the model with the lowest error was preferred, computation time can be a crucial factor in high-frequency data, and memory consumption should not be ignored, as more complex models can take up several gigabytes.