# LightGBM Low-Temperature Prediction Model Based on LassoCV Feature Selection

Authors: Shangqi Duan; Shuangde Huang; Wei Bu; Xingke Ge; Haidong Chen; Jing Liu; Jiqiang Luo
Journal: Mathematical Problems in Engineering
 (2021)

---

## Abstract

Icing disasters on power grid transmission lines can easily lead to major accidents, such as wire breakage and tower overturning, that endanger the safe operation of the power grid. Short-term prediction of transmission line icing relies to a large extent on accurate prediction of daily minimum temperature. This study therefore proposes a LightGBM low-temperature prediction model based on LassoCV feature selection. A data set comprising four meteorological variables was established, and time series autocorrelation coefficients were first used to determine the hysteresis characteristics in relation to the daily minimum temperature. Subsequently, the LassoCV feature selection method was used to select the meteorological elements that are highly related to minimum temperature, with their lag characteristics, as input variables, to eliminate noise in the original meteorological data set and reduce the complexity of the model. On this basis, the LightGBM low-temperature prediction model is established. The model was optimized through grid search and crossvalidation and validated using daily minimum surface temperature data from Yongshan County (station number 56489), Zhaotong City, Yunnan Province. The root mean square error, MAE, and MAPE of the model minimum temperature prediction after feature selection are shown to be 1.305, 0.999, and 0.112, respectively. These results indicate that the LightGBM prediction model is effective at predicting low temperatures and can be used to support short-term icing prediction.


---

## Body

## 1. Introduction

Evidence from power grid operation shows that wire breaks and tower toppling accidents, caused by transmission line icing, lead to great damage to the transmission lines themselves and also adversely affect the safe and stable operation of the power grid system more generally [1]. Most transmission line icing accidents occur in areas with high small-scale weather variability, which are strongly affected by factors such as temperature, humidity, cold and warm air convection, circulation, and wind [2]. Low temperature is an important cause of transmission line icing. Therefore, accurate prediction of minimum temperature can provide a good basis for short-term transmission line icing prediction. Minimum temperature data are generally based on time series, and most traditional prediction methods use univariate time series modeling [3]. However, changes in temperature are affected by various meteorological factors, and those that correlate highly with temperature include wind direction, wind speed, and relative humidity [4]. Traditional time series temperature prediction models mainly include multiple linear regression, autoregressive integrated moving average (ARIMA) [3], and gray prediction methods. Accounting for dynamic changes in temperature is difficult because of the prediction effects of the aforementioned methods, and the prediction results generally tend towards average values. Tao et al. [4] proposed a temperature prediction method using a long short-term memory network based on a random forest approach. Niu and others proposed the use of principal component analysis, a back propagation (BP) neural network, and a radial basis function (RBF) neural network, to establish a temperature prediction model [5]. Although this method considers the influence of multiple meteorological variables on temperature, it does not consider the time series characteristics of those meteorological variables. Jiang [6] proposed an application of a particle swarm-optimized RBF artificial neural network to temperature prediction. Although the structural parameters of the RBF model are optimized by the particle swarm optimization algorithm, problems associated with univariate time series prediction are still encountered, so the prediction accuracy is not high.Aiming to address the difficulty of traditional prediction methods to learn from large amounts of data, and their inability to fully consider the impact of multiple meteorological variables and their own time correlations on temperature changes, this paper proposes a LightGBM low-temperature prediction model based on LassoCV feature selection. Daily minimum surface temperature data from Yongshan County, Zhaotong City, Yunnan Province (station number 56489) from 2015 to 2019 were selected as the experimental data for prediction. The model first uses autocorrelation to establish hysteresis characteristics. Subsequently, LassoCV is used to measure the importance of the individual variable features, and those that are highly related to minimum temperature are selected as input variables to LightGBM to model the minimum temperature time series data. The experimental prediction results show that this method has stronger learning ability and higher prediction accuracy than traditional methods and long short-term memory (LSTM) networks. Especially when trained on large-scale multivariable meteorological time series data, this model shows high accuracy and a fast training speed, which are beneficial to practical industrial applications. The accurate prediction of low temperature is a prerequisite to accurately predicting icing on power grid transmission lines. The use of this method can improve the accuracy and speed of prediction and provides a sound basis for supporting the production of icing prediction data.


## 2. Principles of the LightGBM

LightGBM is derived from Reference [7] and related open source tools. The light gradient boosting (LGB) model is an efficient implementation of the classic gradient boosting decision tree (GBDT) model. The LGB model handles the classification, regression, and ranking problems in machine learning. GBDT obtains the final answer by combining multiple decision trees and by adding up the results of all the decision trees. This process has been improved to obtain extreme gradient boosting (XGB). The difference between XGB and GBDT is in the way the tree is split and the way the value of the leaf node is determined. The core idea is to conduct a second-order Taylor expansion of the loss function to be fitted by GBDT and to introduce the regular term of the tree intelligently, so that the formula of the second-order Taylor expansion can be simplified and solved analytically. Thus, a new tree splitting method and leaf node value determination method are derived. LGB is further optimized on the basis of XGB’s improvement of GBDT formula. Figure 1 presents a flow chart of the transition from the gradient boosting method to the LGB model.Figure 1
Flow chart of the boosting-LGB relationship.


### 2.1. GBDT Model

The forward addition model is written as(1)fx=∑i=1Mβihx;θi,where x represents the sample and hx;θi represents the ith base model. Thus, the entire base model is a weighted sum of M models, so this model is called the forward addition model. The temperature prediction model solves the regression problem. The trained model f minimizes the loss function L:(2)min∑j=1NLfxj,yj=min∑j=1NL∑i=1Mβihxj;θi,yj,where xj,yjj=1N is the training sample and ∑i=1M−1βihx;θi is assumed to be known during training. βMhx;θM minimizes equation (2); the idea is to let βMhxj;θM,yj fit the negative gradient gj through the gradient descent algorithm, where the negative gradient gj is(3)gj=−∂LfM−1xj,yj∂fM−1xj,and gj exists for every sample. Thus, the way a new sample fits {(xj,gj)} is converted into a regression problem:(4)θM,βM=argminθ,β∑j=1Ngj−βhxj;θ2.After fitting the negative gradient, the final step size must be determined.(5)ρm=argminρ∑j=1NLfM−1xj+ρhxj,yj.Thus, the final model is(6)fx=fM−1+ρMβMhx;θM.When the base learnerh is determined as the cart decision tree, this becomes the GBDT method.


### 2.2. From GBDT to XGB

The lowest-level XGB model is still a forward addition model, but the difference is that it is not optimized by simple gradient descent, but the regular term of the tree and the second-order Taylor expansion are introduced in a way that simplifies the splitting of GBDT and leaf nodes in a single tree value determination. GBDT requires the gradient that corresponds to each samplexj,gjj=1N, and XGB requires the gradient and Hess value (two steps) that correspond to each sample xj,gj,hjj=1N. The splitting of the XGB tree and the determination of the value of the leaf nodes are related to gj,hj, which are expressed as follows:(7)Gain=12GL2HL+λ+GR2HR+λ−GL+GR2HL+HR+λ−γ,Wj=−GjHj+λ,where γ and λ are two hyperparameters used to measure the complexity of the tree by adjusting the number of leaf nodes and the weight of the sum of the squares of leaf node values, respectively.GL: sum of gj of all samples divided on the leftHL: sum of hj of all samples divided on the leftGR: sum of gj of all samples divided on the rightHR: sum of hj of all samples divided on the rightWj: value corresponding to the leaf nodeGj: sum of gj of all samples that are divided in the area of leaf node jHj: sum of Hj of all samples that are divided in the area of leaf node j


### 2.3. LGB Model

The LGB model is further optimized on the basis of the XGB derivation described above. These optimizations are performed to reduce the computational cost, but they can also play a role in preventing overfitting (because the original data are noisy, some rough processing may increase the generalization ability of the model). The computational cost of each leaf node split is(8)costtime=featurenum×samplenum×pointnum,where costtime represents the time (s) consumed by calculation, featurenum represents the number of features, samplenum represents the number of samples, and pointnum indicates the number of candidate points.Showing that the cost of the tree model is mainly based on three factors:(1)
Sample size (the cost of calculatinghi is closely related to the sample size)(2)
Number of features (features need to be traversed when the tree is split)(3)
Number of candidate points (the candidate points under this feature need to be traversed when the tree is split)Therefore, the core of the LGB approach is to minimize the computational cost for each of these factors. The corresponding three technologies are(1)
Gradient unilateral sampling (GOSS): reduce the number of samples(2)
Feature binding technology (EFB): reduce the number of features(3)
Histogram algorithm (Hist): reduce the number of selected points


## 2.1. GBDT Model

The forward addition model is written as(1)fx=∑i=1Mβihx;θi,where x represents the sample and hx;θi represents the ith base model. Thus, the entire base model is a weighted sum of M models, so this model is called the forward addition model. The temperature prediction model solves the regression problem. The trained model f minimizes the loss function L:(2)min∑j=1NLfxj,yj=min∑j=1NL∑i=1Mβihxj;θi,yj,where xj,yjj=1N is the training sample and ∑i=1M−1βihx;θi is assumed to be known during training. βMhx;θM minimizes equation (2); the idea is to let βMhxj;θM,yj fit the negative gradient gj through the gradient descent algorithm, where the negative gradient gj is(3)gj=−∂LfM−1xj,yj∂fM−1xj,and gj exists for every sample. Thus, the way a new sample fits {(xj,gj)} is converted into a regression problem:(4)θM,βM=argminθ,β∑j=1Ngj−βhxj;θ2.After fitting the negative gradient, the final step size must be determined.(5)ρm=argminρ∑j=1NLfM−1xj+ρhxj,yj.Thus, the final model is(6)fx=fM−1+ρMβMhx;θM.When the base learnerh is determined as the cart decision tree, this becomes the GBDT method.


## 2.2. From GBDT to XGB

The lowest-level XGB model is still a forward addition model, but the difference is that it is not optimized by simple gradient descent, but the regular term of the tree and the second-order Taylor expansion are introduced in a way that simplifies the splitting of GBDT and leaf nodes in a single tree value determination. GBDT requires the gradient that corresponds to each samplexj,gjj=1N, and XGB requires the gradient and Hess value (two steps) that correspond to each sample xj,gj,hjj=1N. The splitting of the XGB tree and the determination of the value of the leaf nodes are related to gj,hj, which are expressed as follows:(7)Gain=12GL2HL+λ+GR2HR+λ−GL+GR2HL+HR+λ−γ,Wj=−GjHj+λ,where γ and λ are two hyperparameters used to measure the complexity of the tree by adjusting the number of leaf nodes and the weight of the sum of the squares of leaf node values, respectively.GL: sum of gj of all samples divided on the leftHL: sum of hj of all samples divided on the leftGR: sum of gj of all samples divided on the rightHR: sum of hj of all samples divided on the rightWj: value corresponding to the leaf nodeGj: sum of gj of all samples that are divided in the area of leaf node jHj: sum of Hj of all samples that are divided in the area of leaf node j


## 2.3. LGB Model

The LGB model is further optimized on the basis of the XGB derivation described above. These optimizations are performed to reduce the computational cost, but they can also play a role in preventing overfitting (because the original data are noisy, some rough processing may increase the generalization ability of the model). The computational cost of each leaf node split is(8)costtime=featurenum×samplenum×pointnum,where costtime represents the time (s) consumed by calculation, featurenum represents the number of features, samplenum represents the number of samples, and pointnum indicates the number of candidate points.Showing that the cost of the tree model is mainly based on three factors:(1)
Sample size (the cost of calculatinghi is closely related to the sample size)(2)
Number of features (features need to be traversed when the tree is split)(3)
Number of candidate points (the candidate points under this feature need to be traversed when the tree is split)Therefore, the core of the LGB approach is to minimize the computational cost for each of these factors. The corresponding three technologies are(1)
Gradient unilateral sampling (GOSS): reduce the number of samples(2)
Feature binding technology (EFB): reduce the number of features(3)
Histogram algorithm (Hist): reduce the number of selected points


## 3. Establishing the LassoCV-LightGBM Low-Temperature Prediction Model

### 3.1. Data Collection

This study uses meteorological observation data from Yongshan County (Station No. 56489), Zhaotong City, Yunnan Province, which comes from an Institute of Geographic Sciences and Natural Resources (https://www.resdc.cn/) data set, collected daily from January 1, 2015, to December 31, 2019. Part of the original meteorological data is shown in Table 1.Table 1
Original meteorological observation data.
Station_IdYearMonthDayMin_TEMAver_RHUMin_RHUAver_WINMax_WINMax_WIN_DExtreme_WINExtreme _WIN_D564892015118.4595015275635564892015127.6594616285647564892015134.271458215525564892015147.4725813325785564892015157.265537336696564892015164.371449399769⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮56489201912315684223546955Table1 confirms that the relevant meteorological variables at Yongshan Station are updated once per day. Data elements include the collected station number (Station_Id), year (Year), month (Month), day (Day), and eight meteorological variables. They are as follows: minimum temperature (Min_TEM), average relative humidity (Aver_RHU), minimum relative humidity (Min_RHU), average wind speed (Aver_WIN), maximum wind speed (Max_WIN), wind direction of maximum wind speed (Max_WIN_D), extreme wind speed (Extreme_WIN), and wind direction of extreme wind speed (Extreme_WIN_D). The units and precision of the eight meteorological elements are as follows, respectively: 0.1°C, percentage 1%, percentage 1%, 0.1 m/s, 0.1 m/s, 16 directions (degrees), 0.1 m/s, and 16 directions (degrees).


### 3.2. Establishing Hysteresis Characteristics

With time series, the establishment of lag features is a key step in mining the autocorrelation information of the data. The autocorrelation coefficient (ACF) [8] is usually used to measure the correlation between the current moment yt and the k-order lag yt−k. The correlation coefficient measures the linear correlation between these two variables as follows:(9)rk=∑yt−ymeanyt−k−ymean∑yt−ymean2,where rk represents the correlation between yt and its k-order lag, representative autocorrelation coefficient. The autocorrelation represents the relationship between the values of a time series at different points in time. The ACFs of various meteorological variables used in this study are shown in Figures 2–7.Figure 2
ACF of minimum temperature.Figure 3
ACF of average wind speed.Figure 4
ACF of average relative humidity.Figure 5
ACF of maximum wind speed.Figure 6
ACF of minimum relative humidity.Figure 7
ACF of extreme wind speed.Variables that are highly correlated with temperature include wind direction, wind speed, and relative humidity [4]. When the ACF is rk≥0.3, then a medium-strength correlation between the variables exists [9], and this is used as the selection criterion for meteorological variables.


### 3.3. Data Preprocessing and Model Construction

First, the data are cleaned. This study uses the Python environment to process and model the data, using the Pandas fillna function to replace missing and empty values with the average value. The time variables (year, month, and day) are then integrated into a date that can be used as a Pandas index.Second, the input data are normalized. Table1 shows that the input meteorological variables have different dimensions and units, so they need to be normalized in order to be able to compare them with each other. The normalization method selected in this paper is min-max scaling, using(10)xi∗=xi−minmax−min,where xii=1,2,3,…,n is the input value, xi∗i=1,2,3,…,n is the normalized value, max is the maximum value of the variable, and min is the minimum value of the variable.Given the many noise problems in established methods for evaluating lag characteristics of multielement meteorological time series, and the insufficient prediction accuracy of the traditional ARIMA model, this study proposes a LightGBM low-temperature prediction model based on LassoCV feature selection. First, the ACF is used to establish the lag characteristics, and the value of each meteorological element is normalized, that is, the dimensional differences in the multivariate time series data are eliminated, and the processed multivariate data set can be evaluated using a supervised learning method. LassoCV feature selection is then used to measure the importance of each lag feature. The LGB model is next trained on the training data set, and finally the model is optimized through grid search and crossvalidation. Figure8 presents a flow chart of the model processes.Figure 8
Model construction flow chart.


### 3.4. LassoCV Feature Selection

Selecting features is necessary here because there are many variables and a large amount of noise but some variables have very little effect on minimum temperature. Feature selection is used to select variables that are highly correlated with minimum temperature. The Lasso algorithm performs feature screening to effectively reduce the dimension of multi-dimensional data. The Lasso algorithm [10] is based on linear regression: a threshold is predetermined for the absolute value of the model regression coefficient, and the residual square sum of the model is minimized by adding a normal form function [11]. This algorithm compresses and eliminates the variables whose correlation is less than the threshold by optimizing the objective function. The remaining variables are then output as the characteristic variables. The results of using LassoCV to select characteristic variables that are highly correlated with minimum temperature are shown in Figure 9.Figure 9
LassoCV feature selection results.Set the linear regression model to(11)Y=XTβ+ε,where X=x1,x2,…,xi,…,xnT, xi=xi,1,xi,2,…,xi,m∈R1×m is the low-temperature characteristic data preprocessed by autocorrelation, Y=y1,y2,…,ynT∈Rn×1 is the response variable, β=β1,β2,…,βmT∈R1×m is the model coefficient, and ε=ε1,ε2,…,εnT∈Rn×1 is the error vector. The ordinary least squares method of the linear regression model is estimated to be min∑i=1nyi−∑j=1mxi,jβj2, β^ols=XTX−1XTY, when the constraint function is added, that is, LASSO, which is specifically expressed as(12)argminβ∑i=1nyi−∑j=1mxi,jβj2+λ∑j=1mβj.In the formula, the parameterλ is the penalty coefficient of parameter estimation, and its size is verified by ten-fold cross-validation, and the parameter α is determined in the same way. This paper uses the minimum regression method [12] to solve the LASSO regression algorithm. The minimum regression method is a variable selection algorithm based on the forward selection algorithm and the forward gradient algorithm, which can obtain more accurate eigenvectors, which are described in detail as follows:(1)
The calculation process of the forward selection algorithm is as follows: select the independent variablexi=xi,1,xi,2,…,xi,m that is closest to the target variable yk in X=x1,x2,…,xi,…,xnT, and there are(13)y¯k=xkβk.Among them, the coefficientβk is determined by using the following equation:(14)βk=xk,ykxk2.The variable residual is(15)yres,k=yk−y¯k.The variable residual is defined as a new target variable, and the setX without xk is used as the new independent variable set. Repeat the above process until the residual is less than the set range or the number of independent variable sets is zero, and the algorithm terminates.(2)
The forward gradient algorithm selects a feature variablexk with the most correlation each time to approximate the target variable yk. Unlike the forward selection algorithm, the residual is defined as(16)yres,k=yk−xkβk.Regard the residual as the new objective function, the original variable setX=x1,x2,…,xi,…,xnT as the variable set, and recalculate according to formula (16), until the residual yres,k less than the set threshold range; the optimal solution is obtained.The specific steps of the Lasso algorithm are as follows:Step 1 (target variable): According to formulae (13) and (14), solve the variable xk with the highest correlation with the objective function, remove it from the variable set sum, and determine the new target variable according to formula (16).Step 2 (related variable): repeat Step 1 until the correlation between the new variablexl and the target variable yres,k is the same as the correlation between the variables xk and yres,k.Step 3 (characteristic variable): on the angular bisector ofxk and xl, use equation (16) to reapproximate the variable xt, so that the correlation between xt and yres,k is the same as xk, xl, and yres,k. The correlation degree is the same, add the variable xt to the feature set, and use the common angle bisector of the set as the new approach direction.Step 4 (loop): loop the above process untilyres,k is small enough or the variable set is empty, and the final feature set is the desired feature variable.The results of using LassoCV to select characteristic variables that are highly correlated with low temperature changes are shown in Figure9.Figure9 shows that, among the meteorological elements and the established time series features, Low_Tem1, Low_Tem2, Low_Tem8, Max_Win_Aspect1_3.0, aver_Win1, and Low_RHU1 have higher correlation with minimum temperature than other features; hence, these six feature sets are used as the sample data set.


### 3.5. LightGBM Training and Model Tuning

Low temperature prediction is defined as using a historical meteorological variable sequence…,xt−1,xt to predict a future minimum temperature sequence xt+1,xt+2,…. The preprocessed and feature-selected data sets are input to the LightGBM for training, and then a grid search combined with ten-folds crossvalidation is selected to optimize the main model hyperparameters, including iterations, learning_rate, max_depth, and criterion, thereby improving the accuracy of the low-temperature prediction model. After optimization, the best hyperparameters of the low-temperature prediction model are as in Table 2.Table 2
Best hyperparameters of the low-temperature prediction model.
IterationsLearning_rateMax_depthCriterion7800.0039Gini


## 3.1. Data Collection

This study uses meteorological observation data from Yongshan County (Station No. 56489), Zhaotong City, Yunnan Province, which comes from an Institute of Geographic Sciences and Natural Resources (https://www.resdc.cn/) data set, collected daily from January 1, 2015, to December 31, 2019. Part of the original meteorological data is shown in Table 1.Table 1
Original meteorological observation data.
Station_IdYearMonthDayMin_TEMAver_RHUMin_RHUAver_WINMax_WINMax_WIN_DExtreme_WINExtreme _WIN_D564892015118.4595015275635564892015127.6594616285647564892015134.271458215525564892015147.4725813325785564892015157.265537336696564892015164.371449399769⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮⋮56489201912315684223546955Table1 confirms that the relevant meteorological variables at Yongshan Station are updated once per day. Data elements include the collected station number (Station_Id), year (Year), month (Month), day (Day), and eight meteorological variables. They are as follows: minimum temperature (Min_TEM), average relative humidity (Aver_RHU), minimum relative humidity (Min_RHU), average wind speed (Aver_WIN), maximum wind speed (Max_WIN), wind direction of maximum wind speed (Max_WIN_D), extreme wind speed (Extreme_WIN), and wind direction of extreme wind speed (Extreme_WIN_D). The units and precision of the eight meteorological elements are as follows, respectively: 0.1°C, percentage 1%, percentage 1%, 0.1 m/s, 0.1 m/s, 16 directions (degrees), 0.1 m/s, and 16 directions (degrees).


## 3.2. Establishing Hysteresis Characteristics

With time series, the establishment of lag features is a key step in mining the autocorrelation information of the data. The autocorrelation coefficient (ACF) [8] is usually used to measure the correlation between the current moment yt and the k-order lag yt−k. The correlation coefficient measures the linear correlation between these two variables as follows:(9)rk=∑yt−ymeanyt−k−ymean∑yt−ymean2,where rk represents the correlation between yt and its k-order lag, representative autocorrelation coefficient. The autocorrelation represents the relationship between the values of a time series at different points in time. The ACFs of various meteorological variables used in this study are shown in Figures 2–7.Figure 2
ACF of minimum temperature.Figure 3
ACF of average wind speed.Figure 4
ACF of average relative humidity.Figure 5
ACF of maximum wind speed.Figure 6
ACF of minimum relative humidity.Figure 7
ACF of extreme wind speed.Variables that are highly correlated with temperature include wind direction, wind speed, and relative humidity [4]. When the ACF is rk≥0.3, then a medium-strength correlation between the variables exists [9], and this is used as the selection criterion for meteorological variables.


## 3.3. Data Preprocessing and Model Construction

First, the data are cleaned. This study uses the Python environment to process and model the data, using the Pandas fillna function to replace missing and empty values with the average value. The time variables (year, month, and day) are then integrated into a date that can be used as a Pandas index.Second, the input data are normalized. Table1 shows that the input meteorological variables have different dimensions and units, so they need to be normalized in order to be able to compare them with each other. The normalization method selected in this paper is min-max scaling, using(10)xi∗=xi−minmax−min,where xii=1,2,3,…,n is the input value, xi∗i=1,2,3,…,n is the normalized value, max is the maximum value of the variable, and min is the minimum value of the variable.Given the many noise problems in established methods for evaluating lag characteristics of multielement meteorological time series, and the insufficient prediction accuracy of the traditional ARIMA model, this study proposes a LightGBM low-temperature prediction model based on LassoCV feature selection. First, the ACF is used to establish the lag characteristics, and the value of each meteorological element is normalized, that is, the dimensional differences in the multivariate time series data are eliminated, and the processed multivariate data set can be evaluated using a supervised learning method. LassoCV feature selection is then used to measure the importance of each lag feature. The LGB model is next trained on the training data set, and finally the model is optimized through grid search and crossvalidation. Figure8 presents a flow chart of the model processes.Figure 8
Model construction flow chart.


## 3.4. LassoCV Feature Selection

Selecting features is necessary here because there are many variables and a large amount of noise but some variables have very little effect on minimum temperature. Feature selection is used to select variables that are highly correlated with minimum temperature. The Lasso algorithm performs feature screening to effectively reduce the dimension of multi-dimensional data. The Lasso algorithm [10] is based on linear regression: a threshold is predetermined for the absolute value of the model regression coefficient, and the residual square sum of the model is minimized by adding a normal form function [11]. This algorithm compresses and eliminates the variables whose correlation is less than the threshold by optimizing the objective function. The remaining variables are then output as the characteristic variables. The results of using LassoCV to select characteristic variables that are highly correlated with minimum temperature are shown in Figure 9.Figure 9
LassoCV feature selection results.Set the linear regression model to(11)Y=XTβ+ε,where X=x1,x2,…,xi,…,xnT, xi=xi,1,xi,2,…,xi,m∈R1×m is the low-temperature characteristic data preprocessed by autocorrelation, Y=y1,y2,…,ynT∈Rn×1 is the response variable, β=β1,β2,…,βmT∈R1×m is the model coefficient, and ε=ε1,ε2,…,εnT∈Rn×1 is the error vector. The ordinary least squares method of the linear regression model is estimated to be min∑i=1nyi−∑j=1mxi,jβj2, β^ols=XTX−1XTY, when the constraint function is added, that is, LASSO, which is specifically expressed as(12)argminβ∑i=1nyi−∑j=1mxi,jβj2+λ∑j=1mβj.In the formula, the parameterλ is the penalty coefficient of parameter estimation, and its size is verified by ten-fold cross-validation, and the parameter α is determined in the same way. This paper uses the minimum regression method [12] to solve the LASSO regression algorithm. The minimum regression method is a variable selection algorithm based on the forward selection algorithm and the forward gradient algorithm, which can obtain more accurate eigenvectors, which are described in detail as follows:(1)
The calculation process of the forward selection algorithm is as follows: select the independent variablexi=xi,1,xi,2,…,xi,m that is closest to the target variable yk in X=x1,x2,…,xi,…,xnT, and there are(13)y¯k=xkβk.Among them, the coefficientβk is determined by using the following equation:(14)βk=xk,ykxk2.The variable residual is(15)yres,k=yk−y¯k.The variable residual is defined as a new target variable, and the setX without xk is used as the new independent variable set. Repeat the above process until the residual is less than the set range or the number of independent variable sets is zero, and the algorithm terminates.(2)
The forward gradient algorithm selects a feature variablexk with the most correlation each time to approximate the target variable yk. Unlike the forward selection algorithm, the residual is defined as(16)yres,k=yk−xkβk.Regard the residual as the new objective function, the original variable setX=x1,x2,…,xi,…,xnT as the variable set, and recalculate according to formula (16), until the residual yres,k less than the set threshold range; the optimal solution is obtained.The specific steps of the Lasso algorithm are as follows:Step 1 (target variable): According to formulae (13) and (14), solve the variable xk with the highest correlation with the objective function, remove it from the variable set sum, and determine the new target variable according to formula (16).Step 2 (related variable): repeat Step 1 until the correlation between the new variablexl and the target variable yres,k is the same as the correlation between the variables xk and yres,k.Step 3 (characteristic variable): on the angular bisector ofxk and xl, use equation (16) to reapproximate the variable xt, so that the correlation between xt and yres,k is the same as xk, xl, and yres,k. The correlation degree is the same, add the variable xt to the feature set, and use the common angle bisector of the set as the new approach direction.Step 4 (loop): loop the above process untilyres,k is small enough or the variable set is empty, and the final feature set is the desired feature variable.The results of using LassoCV to select characteristic variables that are highly correlated with low temperature changes are shown in Figure9.Figure9 shows that, among the meteorological elements and the established time series features, Low_Tem1, Low_Tem2, Low_Tem8, Max_Win_Aspect1_3.0, aver_Win1, and Low_RHU1 have higher correlation with minimum temperature than other features; hence, these six feature sets are used as the sample data set.


## 3.5. LightGBM Training and Model Tuning

Low temperature prediction is defined as using a historical meteorological variable sequence…,xt−1,xt to predict a future minimum temperature sequence xt+1,xt+2,…. The preprocessed and feature-selected data sets are input to the LightGBM for training, and then a grid search combined with ten-folds crossvalidation is selected to optimize the main model hyperparameters, including iterations, learning_rate, max_depth, and criterion, thereby improving the accuracy of the low-temperature prediction model. After optimization, the best hyperparameters of the low-temperature prediction model are as in Table 2.Table 2
Best hyperparameters of the low-temperature prediction model.
IterationsLearning_rateMax_depthCriterion7800.0039Gini


## 4. Experimental Results and Analysis

To evaluate the accuracy and practicability of the low-temperature prediction model based on the proposed LassoCV-LightGBM combination described in Section2, this work collected meteorological observation data from Yongshan Station in Zhaotong City, Yunnan Province, to train and test the model. Here, we select the data from the first 1790 days as the training set and the data from the last 56 days as the verification set.


### 4.1. Model Evaluation Indicators

After training the model, the entire verification set is predicted, and then the predicted and observed temperature data are denormalized. To evaluate the performance of the model, this study compares its accuracy with that of the traditional prediction model ARIMA [13] and the LSTM [14], by comparing observed and predicted minimum temperature values. The root mean square error (RMSE), average absolute error (MAE), and average absolute percentage error (MAPE) are selected as the model evaluation indicators. They are calculated as follows:(17)RMSE=1N∑i=1Nyi′−yi2,MAE=1N∑i=1Nyi′−yi,MAPE=1N∑i=1Nyi′−yiyi,where yi′ is the predicted minimum temperature value, yi is the observed value, and N is the number of data elements. Smaller values of RMSE, MAE, and MAPE mean a smaller error in the minimum temperature prediction, and therefore a better prediction model performance. Table 3 shows the RMSE, MAE, and MAPE values of the three models.Table 3
Comparison of the test errors of ARIMA, LSTM, and LassoCV-LightGBM low-temperature prediction models.
Low-temperature prediction modelRMSEMAEMAPEARIMA2.1321.5640.258LSTM1.9131.4520.223LassoCV-LightGBM1.3050.9990.112To further test the superiority and practicability of the proposed LassoCV-LightGBM model, this study randomly selected data from five other sites in Yunnan Province for testing. The verification results are shown in Table4.Table 4
Comparison of test errors of LassoCV-LightGBM low-temperature prediction models at other sites.
Station_IdRMSEMAEMAPE569591.6141.2110.117569521.5381.4040.134568361.6511.5070.152566411.4431.1010.124564831.4761.1560.125Table3 shows that the RMSE, MAE, and MAPE values of the LassoCV-LightGBM model are all smaller than those of the LSTM and ARIMA models, indicating that the LassoCV-LightGBM model has higher prediction accuracy and smaller error between the predicted minimum temperature value and the actual observed value.In summary, the use of the LassoCV-LightGBM model for multivariate time series data with a large amount of data improves not only the prediction accuracy but also the speed of the model training.


## 4.1. Model Evaluation Indicators

After training the model, the entire verification set is predicted, and then the predicted and observed temperature data are denormalized. To evaluate the performance of the model, this study compares its accuracy with that of the traditional prediction model ARIMA [13] and the LSTM [14], by comparing observed and predicted minimum temperature values. The root mean square error (RMSE), average absolute error (MAE), and average absolute percentage error (MAPE) are selected as the model evaluation indicators. They are calculated as follows:(17)RMSE=1N∑i=1Nyi′−yi2,MAE=1N∑i=1Nyi′−yi,MAPE=1N∑i=1Nyi′−yiyi,where yi′ is the predicted minimum temperature value, yi is the observed value, and N is the number of data elements. Smaller values of RMSE, MAE, and MAPE mean a smaller error in the minimum temperature prediction, and therefore a better prediction model performance. Table 3 shows the RMSE, MAE, and MAPE values of the three models.Table 3
Comparison of the test errors of ARIMA, LSTM, and LassoCV-LightGBM low-temperature prediction models.
Low-temperature prediction modelRMSEMAEMAPEARIMA2.1321.5640.258LSTM1.9131.4520.223LassoCV-LightGBM1.3050.9990.112To further test the superiority and practicability of the proposed LassoCV-LightGBM model, this study randomly selected data from five other sites in Yunnan Province for testing. The verification results are shown in Table4.Table 4
Comparison of test errors of LassoCV-LightGBM low-temperature prediction models at other sites.
Station_IdRMSEMAEMAPE569591.6141.2110.117569521.5381.4040.134568361.6511.5070.152566411.4431.1010.124564831.4761.1560.125Table3 shows that the RMSE, MAE, and MAPE values of the LassoCV-LightGBM model are all smaller than those of the LSTM and ARIMA models, indicating that the LassoCV-LightGBM model has higher prediction accuracy and smaller error between the predicted minimum temperature value and the actual observed value.In summary, the use of the LassoCV-LightGBM model for multivariate time series data with a large amount of data improves not only the prediction accuracy but also the speed of the model training.


## 5. Conclusions

The LightGBM gradient boosting tree integration model is suitable for modeling multivariate time series data. Compared with traditional time series forecasting methods, the unique GOSS, EFB, and Hist of the LightGBM gradient boosting tree integration model can address the problems of high dimensionality, nonlinearity, and local minima more effectively, and it has stronger data learning and generalization capabilities. Moreover, LassoCV can analyze the importance of features and adds regular terms to prevent overfitting. This study establishes lag features through ACFs and then uses LassoCV to select features from multivariate meteorological time series data, which provide more effective and accurate data for model construction and reduce the complexity of the model. Taking minimum temperature data as a specific example, a prediction model based on LassoCV-LightGBM was constructed, and meteorological observation data from the Yongshan site were used for prediction and analysis. The experimental results show that the Lasso-LightGBM model performs better than the ARIMA and LSTM models, with improved low-temperature prediction accuracy, indicating that the Lasso-LightGBM model has superior capability in analyzing multisource time series data. It has particular applicability in predicting low temperatures and is clearly a useful tool for supporting power grid icing prediction.



---

*Source: 1776805-2021-10-13.xml*