## 2 Methods
### Methodology
#### 2.2.2 Restoration Method
The calculation of the approximate residual of the negative gradient value of the loss function is an expansion of the gradient lifting algorithm on the lifting algorithm, which makes the objective function more convenient to solve. The steps are as follows:  
1. Find a constant that minimizes the loss function and initialize a tree with only the root node. 2. Calculate the negative gradient of the loss function and use it as an estimate of the residual  
\[\mathrm{r_{mi}}=-\left[\frac{\partial(\mathrm{y_{i},\ f(x_{i})})}{\partial \mathrm{f(x_{i})}}\right]_{\mathrm{f(x)=f_{m-1}(x)}} \tag{8}\]  
3. Using the data set \(\mathrm{(x_{i},\ r_{mi})}\)-based model fitting the next round, have corresponding J a leaf node \(\mathrm{R_{mj}}\), J = 1, 2, 3,..., J - 1, J. The residual value \(\mathrm{r_{mi}}\) can be estimated using \(\mathrm{R_{mj}}\), namely the optimal fitting value of every single leaf node. The mth base model \(f_{m}(x)\) predicts the following value at leaf node j.  
\[\mathrm{c_{mj}}=\mathrm{argmin_{c}}\sum_{\mathrm{x_{i}\in R_{mj}}}\mathrm{L(y _{i},\ f_{m-1}(x_{i})+c)} \tag{9}\]  
4. The base model \(\mathrm{f_{m}(x)}\) of the mth round is obtained, and the gradient boosting model is obtained by integrating the base model of the previous m \(-\) 1 rounds. \[\mathrm{F_{M}(x)}=\mathrm{F_{M-1}(x)+f_{m}(x)}=\sum_{m=1}^{M}\sum_{j=1}^{J} \mathrm{c_{mj}I}\left(x_{i}\in R_{mj}\right) \tag{10}\]  
ANN is a network model abstracted from the perspective of information processing by imitating the structure of the human or animal nervous system. The basic structure of a simple neuron consists of a weight vector W, an input vector x and an activation function. The calculation is expressed as follows:  
\[\mathrm{a}=\mathrm{f}\Big{(}W^{T}x+b\Big{)} \tag{11}\]  
A simple ANN usually contains an input layer, hidden layer, and output layer. In order to obtain high-level abstract features, complex neural networks usually use multiple hidden layers. Neurons in the same layer are not connected to each other, while neurons in adjacent layers are pairwise connected through a weight matrix that reflects the degree of influence of the output of the upper-layer neurons on the input of  
by layer by changing the weight and bias of each neuron to finally produce the output of the network model. Multiple linear regression (MLR) is a statistical method that describes the linear relationship between variables. Due to the ability to predict the dependent variable by introducing the optimal combination of multiple independent variables, MLR is more effective than univariate linear regression models that only use one independent variable. The prediction model is expressed as follows:  
\[\mathbf{y}_{\mathrm{a}}=\beta_{0}+\beta_{1}\mathbf{x}_{1}+\beta_{2}\mathbf{x} _{2}+\ldots+\beta_{\mathrm{k}}\mathbf{x}_{\mathrm{k}}+\epsilon \tag{14}\]  
Since the observations of multiple linear regression are a vector instead of a scalar, a matrix is introduced to represent these observations, as shown in Equation (15). The matrix form of multiple linear regression is shown in Equation (16). \[\mathbf{X}= \begin{bmatrix}1&\mathbf{x}_{11}&\cdots&\mathbf{x}_{1\mathrm{k}} \\ 1&\mathbf{x}_{21}&\cdots&\mathbf{x}_{2\mathrm{k}}\\ \vdots&\vdots&\ddots&\vdots\\ 1&\mathbf{x}_{n1}&\cdots&\mathbf{x}_{n\mathrm{k}}\end{bmatrix} ,\mathbf{Y}= \begin{bmatrix}\mathbf{y}_{1}\\ \mathbf{y}_{2}\\ \vdots\\ \mathbf{y}_{n}\end{bmatrix} ,\mathbf{\beta}= \begin{bmatrix}\mathbf{\beta}_{1} \\ \mathbf{\beta}_{2}\\ \vdots\\ \mathbf{\beta}_{n}\end{bmatrix} ,\mathbf{\epsilon}= \begin{bmatrix}\mathbf{ \epsilon}_{1}\\ \mathbf{\epsilon}_{2}\\ \vdots\\ \mathbf{\epsilon}_{n}\end{bmatrix}  \tag{15}\]  
\[\mathbf{Y}=\mathbf{\chi}\mathbf{\beta}+\epsilon \tag{16}\]