# Augmented Data and XGBoost Improvement for Sales Forecasting in the Large-Scale Retail Sector
## 2 Methodology: Platform Design and Implementation and AD Approach
### XGBoost Algorithm  
The XGBoost algorithm is a machine-learning algorithm that is composed of a sequence of weak predictors. This algorithm was introduced by [PERSON] [59] and perfects the gradient boosting algorithm. Gradient boosting is based on the iterative estimation of trees on the residuals obtained at each step and on adaptive updating of the estimates. Gradient boosting uses the gradient descent technique, for which the split that favors the approach to the minimum point of the objective function is chosen. The optimizations of XGBoost, compared with gradient boosting, are due to its scalability, the use of parallel and distributed computing, tree pruning operations, management of missing values, and regularization to avoid overfitting and bias. The input data are the set of values of the variables \(x_{i}\) used to make a prediction on the variable \(y_{i}\):  
\[\left\{\left(x_{i}-y_{i}\right)\right\}_{i=1}^{n}\]This constitutes the training dataset. The model predicts the value of the variable \(y_{i}\) starting from the variables \(x_{i}\), which are characterized by multiple features. In a linear regression problem, the predicted value is \(\hat{y}_{i}=\sum_{j}\theta_{j}x_{ij}\), where \(\theta_{j}\) is the weight of \(x_{j}\). In a generic problem, \(\theta\) denotes the parameters of the model. The objective function, which measures the model's ability to fit the training data, consists of two terms:  
\[Obj(\theta)=L(\theta)+\Omega(\theta)\]  
where \(L(\theta)\) is training loss function and \(\Omega(\theta)\) is the regularization term. The loss function is a differentiable function that evaluates the prediction. The regularization term allows one to control the model's complexity and avoid overfitting. XGBoost uses [PERSON] expansion of the loss function to write the objective function in the following form:  
\[Obj(\theta)=\sum_{i=1}^{n}\left[g_{i}f_{i}(x_{i})+\frac{1}{2}h_{i}f_{i}^{2}(x _{i})\right]+\Omega(f_{i})\]  
where \(g_{i}=\partial_{g_{i}^{(t-1)}}L\left(y_{i},\hat{y}_{i}^{(t-1)}\right)\), while \(h_{i}=\partial_{g_{i}^{(t-1)}}^{2}L\left(y_{i},\hat{y}_{i}^{(t-1)}\right)\). By defining the following quantities:  
\[G_{j}=\sum_{i\in I_{j}}g_{i}\]  
\[H_{j}=\sum_{i\in I_{j}}h_{i}\]  
\[I_{j}=\{i|q(x_{i})=j\}\]  
we can write the optimal weight of the \(j\)-th leaf as \(\theta_{j}=-\frac{G_{j}}{H_{j}+\lambda}\), where \(I_{j}\) is the instance set of the \(j\)-th leaf and \(q(x_{i})\) is a function that maps the data instance into a leaf of the tree, returning the index of the leaf itself. The objective function that optimizes the model is  
\[Obj=-\frac{1}{2}\sum_{j=1}^{T}\frac{G_{j}^{2}}{H_{j}+\lambda}+\gamma T\]  
Here, \(T\) is the number of leaves that characterize the tree. The computational cost of the algorithm is optimized due to the simultaneous training of all trees. Finally, the gain function, which allows one to evaluate the split candidates, is  
\[Gain=\frac{G_{L}^{2}}{H_{L}+\lambda}+\frac{G_{R}^{2}}{H_{R}+\lambda}-\frac{G_ {P}^{2}}{H_{P}+\lambda}\]  
where the first term is the contribution relative to the left nodes (subscript \(L\)), the second term is relative to the right nodes (subscript \(R\)), and the last term is the contribution of the parent leaf node (subscript \(P\)). The split condition that generates the greatest gain is chosen. This is a pruning strategy that optimizes a tree level to avoid overfitting. For the implementation of the code in Python, the following open-source libraries were used:  
- Numpy, which facilitates the use of large matrices and multidimensional arrays to operate effectively on data structures by means of high-level mathematical functions;  
- Pandas, which allows the manipulation and analysis of data;  
- XGBoost, which provides a gradient boosting framework;  
- Sklearn, which provides various supervised and unsupervised learning algorithms. A seed was awarded to initialize the pseudorandom number generator. The random module uses the seed value as the basis for generating a random number.