# Time-Series Forecasting of Seasonal Data Using Machine Learning Methods
## 3 Development of the Models
### XGBoost  
Extreme Gradient Boosting (XGBoost) is an optimized distributed gradient boosting library designed to be highly efficient, flexible, and portable. It implements machine learning algorithms under the Gradient Boosting framework. XGBoost provides a parallel tree boosting (also known as GBDT, GBM) that solves many data science problems in a fast and accurate way. The same code runs on the major distributed environment (Hadoop, SGE, MPI) and can solve problems beyond billions of examples [44].  
The main idea of the method is presented in [32]. The model is created using the XGBoressor function of the xgboost library. The function contains a relatively large number of parameters, and a complete list can be found in the documentation section of the library [45]. In our case, only four parameters were used:  
* max_depth = 6--maximum tree depth for base learners;
* learning_rate = 0.05--boosting learning rate (xgb's \"eta\");
* n_estimators = 5000--number of gradients boosted trees (equivalent to the number of boosting rounds);
* gamma = 0.1--minimum loss reduction required to make a further partition on a leaf node of the tree.  
It should be noted that, in contrast to the previously described implementations, the xgboost library can predict only one step ahead. Previous methods allowed setting the forecast horizon length in hours after training the model, after which the model output produced an estimate of the observed parameter. The XGBoost model allows one to obtain a forecast only for one step ahead (i.e., for 1 h). The aim is to obtain the forecast 48 h ahead. It is necessary to change the scheme of data preparation, which is fed to the model input. The algorithm for generating the data used to train and test the model is as follows:  
1. To generate a forecast for 1 h ahead, the model's input is the historical sample for the previous 48 h. The model's output estimates the observed parameter 1 h ahead.
2. One element shifts the historical sample dataset to the left, and as a result, its length is reduced by 1. An element is placed in the place of the missing element at the end of the sample--the estimate of the observed parameter obtained at the previous step. A new parameter estimate is obtained at the model's output, corresponding to the 2-h forecast horizon.
3. Iterations continue until the desired forecast horizon is obtained.  
An explanation of this algorithm for the task of obtaining a forecast horizon for the next 48 h is shown in Figure 3. A similar approach was used to prepare data for the Long Short-Term Memory model.  
When testing the model, you can use actual data from the test dataset instead of parameter estimates (real test data), but to obtain a prediction in a real-time system, you will have to use parameter estimates to generate a new dataset (synthetic test data). Below, the results will be given for real and synthetic test data.  
The result of forecasting with the XGBoost model based on real and synthetic test data is shown in Figures 14 and 15 (see Appendix A).  
As follows from the graphical results, we can conclude that, when using synthetic test data (i.e., the system works in real conditions), the accuracy will decrease as the forecast horizon increases. That is why we should either increase the amount of data in the input vector fed into the model or decrease the forecast horizon. The accuracy of the model will also be affected by the XGBoost model setup.