# Research on Urban Economic High-Performance Forecasting Method Based on Deep Confidence Network

Authors: Bowen Yang
Journal: Scientific Programming
 (2022)

---

## Abstract

To address the complexity of urban economic data and the problem that traditional forecasting methods do not fully utilize the correlation of data, resulting in low prediction accuracy, an urban economic forecasting model based on the fusion of deep belief neural (DBN) and long-short term memory (LSTM) is proposed. The model is based on a combination of DBN and LSTM. The model first uses bandpass filtering to denoise the urban economic data and then determines the prediction starting point of the model based on the root-mean-square and cliff features in the trend diagram of the urban economy; secondly, the optimised 4-layer DBN network is used for deep feature extraction and training and testing of the LSTM. The reliability of the proposed model is demonstrated through urban economic experiments, and the prediction results are compared with those of traditional LSTM, BP (back propagation) neural network, and DBN-BP model to verify the effectiveness of the model.


---

## Body

## 1. Introduction

An accurate judgement of macroeconomic trends is indispensable for analysing the success or failure of macroeconomic control policies, evaluating the quality of the operation of the economic system, and correctly formulating future development planning strategies, and it is also valuable for guiding the investment planning of enterprises and individuals [1–3]. It also has obvious shortcomings, such as the difficulty in reflecting the nonlinear relationships that are widely present in economic systems. The complexity of economic systems, the many internal influences, and the strong coupling, time-varying, and nonlinear nature of each other make modelling and forecasting economic systems an extremely challenging research [4].Artificial neural networks simulate the human brain by means of engineering technology and have the advantages of nonlinearity, self-organisation, fault tolerance, and parallel processing. Given the good characteristics of artificial neural networks and the complexity of economic systems, their application to modelling economic systems and predicting their operating trends is a valuable research topic.At present, many scholars have carried out research work on economic forecasting based on neural networks and have achieved certain results. However, most of the neural network learning algorithms used in economic forecasting are based on the BP algorithm or its improved form [5]. Theoretically, neural networks based on the BP algorithm can approximate arbitrary nonlinear functions and are therefore widely used to solve nonlinear problems such as economic forecasting, but their limitations also severely limit their application performance and scope. This is because the BP algorithm is a local search optimization algorithm, and the objective function it needs to optimize is a highly complex nonlinear function, which generally contains many local extremes, and the algorithm is very likely to fall into local extremes, causing the training and learning failure. The slow training speed and the tendency to fall into local minima can be overcome to some extent by certain techniques, but the expressiveness of the neural network is poor when the number of hidden layers is small, and its performance is no better than that of traditional methods such as vector autoregression (VAR) and support vector machine (SVM). When the number of hidden layers is large, the network is more expressive, but the BP training algorithm can easily lead to failure due to gradient dispersion. This means that simple networks that can be trained to learn are poorly expressive, while complex networks that are expressive are difficult to train to learn. This is a key reason why the nonlinear representation of neural networks has not received sufficient attention in the field of nonlinear problems such as economic forecasting, despite their strong capability [6].Despite the abovementioned shortcomings of neural networks, previous research results on neural network-based economic forecasting show that it is still feasible to be applied to the field of economic forecasting. If the shortcomings of neural network training and learning algorithms can be overcome, it is believed that it may further improve the performance of economic forecasting and make a great impact in the field of economic forecasting. Unlike neural networks, deep learning effectively overcomes the difficulty of training deep neural networks through methods such as layer-wise pretraining, and layer-by-layer initialisation can be achieved through unsupervised learning, which is of great interest for large data applications with a limited number of labelled samples [7].Deep learning embodies feature learning, where samples are mapped to a new space through layer-by-layer feature transformations that make prediction easier [8,9]. This paper also attempts to introduce deep learning techniques into the field of economic forecasting, with a view to improving the performance of economic forecasting systems.The contributions of this article are as follows.Due to the problem of low prediction accuracy, we propose a city economic prediction model based on the fusion of DBN and LSTM.The basic principles of indicators are constructed. This article selects indicators that can reflect several aspects of local macroeconomic development and explores the correlation between them. The data collection takes into account the relative completeness of the relevant index data of the time, which can basically meet the needs of the sample size for training and testing. Due to the large amount of data, the data is well preprocessed.The reliability of the proposed model is proved through urban economic experiments, and the prediction results of traditional LSTM, BP, and DBN-BP models are compared to verify the effectiveness of this model.


## 2. Related Work

The rise of deep learning in recent years has rapidly attracted unprecedented attention in the information field with its impressive breakthroughs in the fields of speech, image, video, and text. Some foreign experts have already applied it to multivariate financial time series prediction [10] and have achieved satisfactory results.In general, the research work of deep learning in the fields of economy and finance has just started, especially that the related research results of economic forecasting based on deep learning have not yet been reported, and there is great room for in-depth research. For example, [11] was the first to apply neural networks to economic forecasting, and economic forecasting methods based on neural networks have increasingly become a hot topic of interest since then. Reference [12] uses neural networks compared with the Auto Regressive and Moving Average Model (ARMA) model. Vitis applied neural networks to the problem of forecasting economic time series data. In [13], neural networks were compared with statistical models such as exponential smoothing models. ARMA combined forecasting models and subjective judgement methods, and the results showed that neural networks outperformed traditional forecasting methods such as statistics, especially in multistep forecasting. Reference [8] introduced neural networks to the field of economic forecasting and compared the analysis of neural network methods with traditional methods. Reference [11], on the other hand, introduced neural networks to the field of multivariate time series forecasting, and the related results showed that the forecasting accuracy of neural network models was all improved compared with traditional statistical methods. The above research results in the field of neural networks in forecasting show that the property that neural networks can approximate arbitrary nonlinear functions gives them a natural advantage in solving highly nonlinear problems such as economic forecasting.At present, the vast majority of neural network models used in economic prediction adopt error BP algorithm. For example, the AHP BP neural network model for communication effectiveness evaluation constructed in [14] combines AHP and BP neural network to reduce the shortcomings of subjective randomness, improve the scientifically of evaluation, and make the calculation results accurate and the error controllable [15]. The study shows that BP artificial neural network can accurately simulate the difference of county economic development and reduce the error caused by subjective setting weight, and the accuracy of comprehensive evaluation is higher.Although the BP algorithm has achieved good results, it has a slow learning rate and tends to fall into local minima. Reference [10] used genetic algorithm to overcome the neural network’s tendency to fall into local minima and also combined it with enterprise situation analysis method, so as to be able to fully consider the existing situation of enterprises and existing historical data, making the decision results more scientific. Reference [16] used normalisation of data growth rates instead of normalisation of previous data to effectively address the problem of extrapolation of forecasts. Xiong and Li [17] adjusted the prediction object from GDP to GDP growth rate and combined it with techniques such as the additional momentum method to achieve some success in solving problems such as the network's tendency to fall into local minima and slow convergence. Reference [18] proposed an Adaboost-based BP neural network algorithm for the shortcoming that BP neural networks are not highly accurate in short-term wind speed prediction and achieved experimental results with high engineering application value.In recent years, the deep learning methods proposed by the academic community have provided a good initial value for neural networks through a layer-by-layer pretraining method, which has enabled neural networks to overcome the existing deficiencies to a certain extent and to take on a new life. Deep learning methods have achieved breakthroughs and impressive results in many fields in recent years and have become one of the hottest research areas in machine learning and even in the whole information field, and the application of deep learning has been widely and successfully used in other fields outside the information field. To this end, this paper also attempts to apply a deep learning method and deep confidence network to economic forecasting to verify the feasibility of applying deep learning methods in the field of economic forecasting.This paper mainly forecasts the consumer price index (CPI) and total import with CPI, fiscal expenditure, fiscal revenue, total export, and total import as inputs. The error BP neural network, DBN, and VAR are empirically compared to verify that the DBN deep learning method outperforms the traditional BP neural network and VAR methods in terms of training and learning speed, prediction accuracy, and coping with the limited amount of marked samples and is a better nonlinear method for economic forecasting modelling method [19].


## 3. Construction and Selection of Economic Forecasting Indicators System

The number of indicators reflecting the characteristics of economic change is large [20]. If all of them are included in the system of forecasting indicators, the workload is huge and it is difficult to complete in realistic processing. On the other hand, the various variables that reflect the characteristics of economic changes are generally correlated to a certain extent, and it is not necessary to include some indicators that do not respond significantly to the characteristics of economic changes in the forecasting indicator system. Therefore, when establishing the indicator system, some indicators that respond significantly to the overall economy should be selected from a large number of indicators that reflect economic changes as an alternative indicator group. The variables that constitute the forecast indicator system should be selected from the main aspects of economic activities, and according to the object of macroeconomic forecasting, the focus should be on economic fluctuations, forecasting of economic operation, and analysis of economic operation trends to establish the economic forecast indicator system. As economic activities involve the fields of consumption, finance, and trade, the range of indicators should include all aspects of social production and reproduction, specifically the following: consumption indicators, financial indicators, price indicators, internal and external trade indicators, etc.Based on the basic principles of the above indicators, this paper selects indicators that can reflect several aspects of local macroeconomic development and discusses the correlation between them and the correlation in time, in order to predict several economic indicators closely related to the dynamics of local economic development, so as to verify the feasibility of the application of DBN deep learning model in the field of economic prediction. The economic indicators selected in this paper cover government finance, foreign trade, residents’ living, etc. The specific indicators are fiscal expenditure, fiscal revenue, total imports, total exports, and consumer price index.


## 4. Data Sample Selection and Preprocessing

The data used in this article is sourced from the China Economic Database. Data is collected from January 2005 to February 2015. The source data collected include fiscal expenditure, fiscal revenue, total imports, total exports, and consumer price index, which are important indicators of regional economic development. The main reason for selecting the abovementioned indicator source data is that the relevant indicator data for this period is relatively complete, which can basically meet the demand for sample size for training and testing, due to the excessive amount of data. The reason for not selecting the relevant indicators in terms of years in this paper is that the amount of indicator data is too small (often less than 20), which makes it difficult to effectively train the learning model and lead to prediction. The main reason for not choosing yearly indicators is that the amount of data is too small (often less than 20) to effectively train the learning model and lead to failure. The reason for not choosing indicators in terms of days is that many of the indicators are not available for complete data collection. In this paper, the formula for obtaining the growth rate from the absolute amount of economic indicators is as follows:(1)yt=xt−xl−1xt−1,where r is the value at the moment, xt and xl−1 are the absolute quantities of an economic indicator at the current and previous moment, respectively, and yt is the growth rate of that economic indicator at the current moment. Using this equation, we can obtain the growth rates of the economic indicators in Schedule 1.In addition, because different economic indicators or their growth rates have different value ranges, and the neural network or deep network for the value of the input is generally required to range between [0, 1] or [1], this paper normalizes the growth rate of economic indicators obtained from (1). In this paper, the maximum and minimum values are normalised to [0, 1], with the maximum value corresponding to 1 and the minimum value corresponding to 0. The specific normalisation formula is as follows:(2)y−nor=y−minymaxy−miny,where y−nor is the normalised indicator growth rate data and maxy and miny correspond to the maximum and minimum values of the indicator growth rate, respectively.The forecast for an economic indicator of the current time pointt is related not only to other economic indicators at that time point, but also to indicators at previous time points. This is due to the fact that there is a correlation between the indicator data and the growth rate of the indicator data at adjacent time points, which contains factors that can be used to predict changes in economic indicators.To this end, in order to effectively predict the growth rate of an economic indicator at the current time nodet, the normalised indicator growth rate data from multiple moments before moment t needs to be used as the input quantity. Also considering the small sample size available for training and learning, the input dimension of the neural network or deep learning network should not be too large to prevent overfitting, and the experimental input data in this paper uses six months of data before moment t as the input quantity, i.e., t − 1 month, t − 2 months, t − 3 months, t. In other words, the normalised data of the indicator growth rate of the independent variable for the six months of t-January, t-February, t-March, t-March, t-April, t-May, and t-June are used as input data to predict the growth rate of the dependent variable indicator data in month t. After obtaining the growth rate of the dependent variable indicator data for month t, the data is converted to the specific dependent variable indicator data.


## 5. Models in This Paper

### 5.1. LSTM

The long- and short-term memory network is a model of memory cell network structure proposed by Hochreiter to solve the phenomenon of gradient explosion and gradient disappearance that occurs in recurrent neural networks when processing relatively long time series data [21], which is based on RNN by introducing a gating structure in the cell to judge whether the information meets the requirements to control the accumulation rate of information—input gates, forgetting gates, and output gates, thus solving the problem of long-term dependency by remembering and updating new information with the help of this structure. As shown in Figure 1, each LSTM neuron is composed of cell states, i.e., long-term state ct and short-term state ht, input gate it, forgetting gate ft, and output gate ot.Figure 1
LSTM structure.A so-called cell state is a container for storing information, which is progressively increased, decreased, changed, and outputted through the process control of input gates, forgetting gates, and output gates. In each neural cell, the cell state undergoes the forgetting process of the forgetting gate, the input process of the input gate, and the process of outputting information to the output gate. The input gate is the replication of the input information for processing the current neural unit. It consists of two parts: a sigmoid function that autonomously chooses which information to update and a tank function that adds a completely new vector of constructs to the current call state to construct a new state [22]. The implementation formula is(3)it=σwxixt+whiht−1+bic˜t=ϕwxcxt+whcht−1+bc.The main function of the forgetting gate is to determine which previous messages need to be discarded for the current state. Ifft = 1, then the information is completely retained; ft = 0 means that the information is completely discarded. The implementation of ft is given by(4)ft=σwxfxt+whfht−1+bf.The output gate mainly controls the output information of the current hidden state. The implementation formula is(5)ht=ottanhctct=itc˜t+ftct−1ot=σwxoxt+whoht−1+bo,where ht−1 is the output at time t − 1; wxi,wxc,wxf,wxo is the weight matrix parameter of the input vector at time t; whi,whc,whf,who is the weight matrix parameter of the hidden vector ht−1 at time t − 1; b represents the bias vector parameter.Finally, the back propagation through time (BPTT) algorithm is used to optimize the parameters of the LSTM model.


### 5.2. DBN

A multilayer restricted Boltzmann machine (RBM), a one-layer BP network from a typical deep confidence neural network, and a layer-by-layer greedy learning algorithm are used to optimize the weights and biases among the RBM layers [23].As shown in Figure2, in order to retain more features in the DBN network, the output of the RBM of the previous layer is the input of the RBM of the next layer; finally, a BP neural network is built to propagate the error information to each layer of the RBM from top to bottom using a back propagation network, and the parameters of the DBN network model are fine-tuned to ensure that the parameters of the DBN model are fine-tuned to ensure that the feature vectors of the whole DBN are optimal.Figure 2
DBN structure.


### 5.3. DBN-LSTM

The prediction model based on DBN-LSTM can make full use of the advantages of DBN multilevel perceptron structure, which can better retain the attributes of the original data, better solve the problem of urban economic data feature extraction, overcome the shortcomings of previous bearing signal feature extraction relying on human experience extraction, and improve the accuracy of urban economic prediction.The flow chart is shown in Figure3. The specific operations are as follows.(1)
The filter interval of the bandpass filter is set according to the original signal spectrum, and the signal is processed for noise reduction.(2)
The starting point of urban economic prediction is determined based on the full-life trend diagram with root mean square and cliff features, and the training and test sets are divided according to the SPT.(3)
The number of layer nodes of the 4-layer RBM structure of the deep confidence neural network was optimised using the PSO algorithm to extract the deep features of the bearing vibration data and complete the unsupervised learning training process.(4)
Deep features extracted by the optimized DBN are input into the LSTM network, and the prediction model is trained by the LSTM based on (1)–(3) and the unique characteristics of the memory cell structure of the LSTM network.(5)
The divided test set is fed into the trained prediction model to obtain the RUL of the bearing.Figure 3
DBN-LSTM flow chart.


## 5.1. LSTM

The long- and short-term memory network is a model of memory cell network structure proposed by Hochreiter to solve the phenomenon of gradient explosion and gradient disappearance that occurs in recurrent neural networks when processing relatively long time series data [21], which is based on RNN by introducing a gating structure in the cell to judge whether the information meets the requirements to control the accumulation rate of information—input gates, forgetting gates, and output gates, thus solving the problem of long-term dependency by remembering and updating new information with the help of this structure. As shown in Figure 1, each LSTM neuron is composed of cell states, i.e., long-term state ct and short-term state ht, input gate it, forgetting gate ft, and output gate ot.Figure 1
LSTM structure.A so-called cell state is a container for storing information, which is progressively increased, decreased, changed, and outputted through the process control of input gates, forgetting gates, and output gates. In each neural cell, the cell state undergoes the forgetting process of the forgetting gate, the input process of the input gate, and the process of outputting information to the output gate. The input gate is the replication of the input information for processing the current neural unit. It consists of two parts: a sigmoid function that autonomously chooses which information to update and a tank function that adds a completely new vector of constructs to the current call state to construct a new state [22]. The implementation formula is(3)it=σwxixt+whiht−1+bic˜t=ϕwxcxt+whcht−1+bc.The main function of the forgetting gate is to determine which previous messages need to be discarded for the current state. Ifft = 1, then the information is completely retained; ft = 0 means that the information is completely discarded. The implementation of ft is given by(4)ft=σwxfxt+whfht−1+bf.The output gate mainly controls the output information of the current hidden state. The implementation formula is(5)ht=ottanhctct=itc˜t+ftct−1ot=σwxoxt+whoht−1+bo,where ht−1 is the output at time t − 1; wxi,wxc,wxf,wxo is the weight matrix parameter of the input vector at time t; whi,whc,whf,who is the weight matrix parameter of the hidden vector ht−1 at time t − 1; b represents the bias vector parameter.Finally, the back propagation through time (BPTT) algorithm is used to optimize the parameters of the LSTM model.


## 5.2. DBN

A multilayer restricted Boltzmann machine (RBM), a one-layer BP network from a typical deep confidence neural network, and a layer-by-layer greedy learning algorithm are used to optimize the weights and biases among the RBM layers [23].As shown in Figure2, in order to retain more features in the DBN network, the output of the RBM of the previous layer is the input of the RBM of the next layer; finally, a BP neural network is built to propagate the error information to each layer of the RBM from top to bottom using a back propagation network, and the parameters of the DBN network model are fine-tuned to ensure that the parameters of the DBN model are fine-tuned to ensure that the feature vectors of the whole DBN are optimal.Figure 2
DBN structure.


## 5.3. DBN-LSTM

The prediction model based on DBN-LSTM can make full use of the advantages of DBN multilevel perceptron structure, which can better retain the attributes of the original data, better solve the problem of urban economic data feature extraction, overcome the shortcomings of previous bearing signal feature extraction relying on human experience extraction, and improve the accuracy of urban economic prediction.The flow chart is shown in Figure3. The specific operations are as follows.(1)
The filter interval of the bandpass filter is set according to the original signal spectrum, and the signal is processed for noise reduction.(2)
The starting point of urban economic prediction is determined based on the full-life trend diagram with root mean square and cliff features, and the training and test sets are divided according to the SPT.(3)
The number of layer nodes of the 4-layer RBM structure of the deep confidence neural network was optimised using the PSO algorithm to extract the deep features of the bearing vibration data and complete the unsupervised learning training process.(4)
Deep features extracted by the optimized DBN are input into the LSTM network, and the prediction model is trained by the LSTM based on (1)–(3) and the unique characteristics of the memory cell structure of the LSTM network.(5)
The divided test set is fed into the trained prediction model to obtain the RUL of the bearing.Figure 3
DBN-LSTM flow chart.


## 6. Experimental Ratio Analysis

This paper attempts to test the possibility of using DBNs for economic forecasting through empirical research on two economic indicators: consumer price index and total import. Its objectives are as follows:(1)
Empirical studies of economic indicator forecasting have shown that both DBN and BP neural networks can effectively model highly nonlinear economic forecasting problems, have better modelling accuracy than linear models such as VAR, and are not restricted to a particular nonlinear model [24].(2)
Empirical comparison experiments between DBN and BP show that DBN has higher prediction accuracy, faster convergence, and stronger generalization ability; i.e., DBN are a superior economic forecasting method compared to BP.


### 6.1. Experimental Results

As can be seen from Figure4, although the predicted values of DBN and BP deviate from the true values in the local high-frequency oscillation part of the growth rate, the predicted values of DBN and BP can effectively follow the change pattern of the true values in the overall trend of the growth rate, reflecting that both DBN deep learning model and BP neural network model can effectively model the nonlinear problem of economic forecasting. At the same time, DBN has a better fit than BP; i.e., DBN is a better modelling tool than BP and is also more effective in predicting growth rate changes than VAR, which tends to learn a smoother linear mapping.Figure 4
Comparison of consumer price index growth rate DBN, BP neural network, and VAR forecasts.Figure4 shows more clearly than Figure 5 that DBN can predict the consumer price index more accurately than BP and VAR.Figure 5
Comparison of DBN, BP neural network, and VAR forecasts of consumer price index.


### 6.2. Comparison of Quantitative Prediction Accuracy

The comparison results of the prediction accuracy of DBN, BP, and VAR models are shown in Table1, where the bolded numbers indicate the indicators corresponding to the optimal method.Table 1
Comparison of accuracy of consumer price index forecasts.
Method/indexMAEMSEAverage relative error (%)DBN0.4250.3230.418BPNN0.690.7520.679VAR0.4920.2910.484Table1 shows that the errors of all three methods are small, and all can predict that the DBN is the best in terms of mean absolute error (MAE) and mean relative error, reflecting the better modelling accuracy of the DBN. However, the VAR model performs better in terms of mean mean squared error (MSE), which is an indicator of the volatility of the forecast error, reflecting the fact that a linear model such as VAR is less able to capture drastic changes, while the BP neural network is worse in all indicators.This indicates that DBN can tap into the complex patterns of variation inherent in the data to improve prediction accuracy, is a better nonlinear prediction method than BP neural network, and can compensate for the shortcomings of traditional linear time series models.


### 6.3. Comparison of Convergence Rates

Figure6 gives the error convergence curves of the DBN deep learning model and the BP neural network in the training and learning phase. In terms of convergence speed, the DBN is faster than the BP neural network and does not show oscillations. This is mainly because the DBN uses a pretraining learning method to provide a good initial value for the network, while the BP neural network uses random parameters to initialise the network, so its error curve starts from a higher level and takes longer to converge and may suffer from oscillations, which is one of the reasons for the poor performance of the final BP prediction—convergence speed problem.Figure 6
DBN and BP convergence curves.


### 6.4. Impact of Label-Free Sample Pretraining on DBN Modelling Performance

From the aforementioned experiments, it can be seen that the DBN deep learning model achieve better modelling performance compared to BP neural networks by pretraining the learning of network weights. The DBN pretraining process only requires the participation of the set of unlabeled training samples, and nondependent economic indicators are required to participate in the pretraining. Although the weight still needs to be fine-tuned by the BP algorithm after pretraining, the amount of training samples required for the fine-tuning phase can be significantly reduced as the initial weights are already provided for the network in the pretraining phase. This feature allows DBN deep learning models to cope with applications where there are only a small number of labelled samples in the training sample and a large number of unlabelled samples, which are often the case in practical applications. For example, in the real data acquisition process, there are often missing data and a large number of unlabelled samples, or the data on the independent variables are easy to obtain but the data on the dependent variables are difficult or too expensive to obtain, resulting in a significant reduction in the amount of available labelled samples. In this subsection, we verify the impact of unlabeled sample pretraining on the modelling performance of DBN deep learning models by means of simulation experiments.Here, the consumer price index of residents is used as the dependent variabley to be predicted. In the above experiment, 70 of the 105 training samples were randomly selected as labeled samples, and the other training samples were used as unlabeled samples to simulate the loss of data labels. The model setting is the same as the prediction part of the above consumer price index. All 105 unlabeled samples (including 35 unlabeled samples and 70 labeled samples, in which the labeled samples do not use their label information) are pretrained with CD algorithm, and then the weight of DBN is fine-tuned with 70 labeled samples through BP algorithm.In terms of model parameter settings, the BP iteration search step of the DBN model was changed to 0.00005 and the number of iterations was increased to 1000 to avoid overfitting of the DBN during the training process of the back propagation algorithm due to the reduction of the marked samples, and the rest of the parameters were the same as above.Figures6 and 7 give a comparison between the predictions of DBN and BP for the consumer price index growth rate and the price index itself for the total 9 months from June 2014 to February 2015, respectively, and the true values. The training set of DBN is 70 marked samples +35 unmarked samples for a total of 105 samples, while the training set of BP is only 70 marked samples as it cannot use unmarked samples for training. Due to the reduced number of effective labelled training samples, the BP modelling performance is significantly worse than that of the DBN.Figure 7
DBN, BP, and VAR forecasts of consumer price index growth rates when the marked sample is reduced.Table2 gives a comparison of the prediction accuracy of DBN and BP when the training set is 70 labelled samples +35 unlabelled samples (105 samples in total). The table shows that DBN performs better than BP in terms of both mean absolute error (MAE), mean square error (MSE), and mean relative error [25]. At the same time, we note that although the number of labeled samples in this experiment is smaller compared to the experimental results in Table 1, the DBN can use the unlabeled samples to train and learn the DBN; thus it has little effect on the prediction performance of the DBN; for example, the MAE changes from 0.425 to 0.426, the MSE changes from 0.323 to 0.39, and the mean relative error changes from 0.418% to 0.420%. The mean relative error was changed from 0.418% to 0.420%. It can be seen that, after removing the labels of a certain amount of samples, the indicators of DBN have decreased, but not significantly, indicating that the pretraining learning can effectively use the unlabelled samples[3,26,27]. As the BP neural network can only use labelled samples for training and learning, its prediction performance decreases significantly when the number of training samples is reduced from 105 to 70; for example, MAE changes from 0.690 to 0.838, MSE changes from 0.752 to 1.074, and the average relative error changes from 0.679% to 0.8251%.VAR's MAE changes from 0.492 to 0.686, MSE from 0.686 to 0.686.686, MSE from 0.291 to 0.719, and mean relative error from 0.484% to 0.675%.Table 2
Comparison of the prediction accuracy of DBN and BP for the consumer price index when the marked sample is reduced.
Method/indexMAEMSEAverage relative error (%)DBN0.4260.3990.420BPNN0.8381.0740.825VAR0.6860.7190.675In order to further explore the impact of increasing the number of censored samples on the DBN performance and to observe the impact of censoring the censored samples on the DBN prediction performance while keeping the prediction performance within a certain range, we tested the set of samples listed in Table3 (i.e., 50 censored +55 uncensored, 30 censored +75 uncensored) and compared the test results with the previous experiment of 70 censored +35 unlabeled training sample sets from the previous experiment that were compared. For comparison purposes, Table 4 shows the prediction performance of the BP neural network under the same circumstances. Since the BP neural network can only be trained and learned using the labelled samples, the unlabelled samples in Table 3 cannot be used for the BP neural network. For a more intuitive comparison, Figure 8 plots the average relative error curves for the DBN and BP corresponding to different training sample cases.Table 3
Effect of labeled sample size on DBN prediction accuracy.
Index/number of labeled samplesMAEMSEAverage relative error (%)700.4260.3990.420500.4660.3100.459300.5570.5380.548Table 4
Effect of marked sample size on the prediction accuracy of BP neural network.
Index/number of labeled samplesMAEMSEAverage relative error (%)700.8381.0740.825500.9191.3190.904301.1681.791.15Figure 8
DBN, BP, and VAR forecasts of the consumer price index when the marked sample is reduced.The experimental results in Tables3 and 4 and Figure 8 show that the prediction error increases as the number of labeled samples decreases, but in comparison, since DBN can be pretrained with unlabeled samples, on the one hand, the prediction performance of DBN is always higher than that of BP and, on the other hand, the prediction performance of DBN decreases at a significantly lower rate than that of BP as the number of labeled samples decreases, which shows that DBN is a prediction method with better performance. On the other hand, DBN’s prediction performance decreases at a significantly lower rate than that of BP as the size of the marked sample decreases.When the data is partially missing, it can be seen from Figure9 that the prediction results of the BP neural network will be at a high level, which may mislead the government to adopt excessive macropolicies to suppress commodity prices and thus cause excessive changes in the consumer price index, so that the stability of each economic variable will be affected. For the VAR, although its prediction results are more in line with the average of the true values, the local fluctuations are somewhat different from the true values.Figure 9
Comparison of the effect of marked sample size on the relative error of DBN and BP neural network predictions.


## 6.1. Experimental Results

As can be seen from Figure4, although the predicted values of DBN and BP deviate from the true values in the local high-frequency oscillation part of the growth rate, the predicted values of DBN and BP can effectively follow the change pattern of the true values in the overall trend of the growth rate, reflecting that both DBN deep learning model and BP neural network model can effectively model the nonlinear problem of economic forecasting. At the same time, DBN has a better fit than BP; i.e., DBN is a better modelling tool than BP and is also more effective in predicting growth rate changes than VAR, which tends to learn a smoother linear mapping.Figure 4
Comparison of consumer price index growth rate DBN, BP neural network, and VAR forecasts.Figure4 shows more clearly than Figure 5 that DBN can predict the consumer price index more accurately than BP and VAR.Figure 5
Comparison of DBN, BP neural network, and VAR forecasts of consumer price index.


## 6.2. Comparison of Quantitative Prediction Accuracy

The comparison results of the prediction accuracy of DBN, BP, and VAR models are shown in Table1, where the bolded numbers indicate the indicators corresponding to the optimal method.Table 1
Comparison of accuracy of consumer price index forecasts.
Method/indexMAEMSEAverage relative error (%)DBN0.4250.3230.418BPNN0.690.7520.679VAR0.4920.2910.484Table1 shows that the errors of all three methods are small, and all can predict that the DBN is the best in terms of mean absolute error (MAE) and mean relative error, reflecting the better modelling accuracy of the DBN. However, the VAR model performs better in terms of mean mean squared error (MSE), which is an indicator of the volatility of the forecast error, reflecting the fact that a linear model such as VAR is less able to capture drastic changes, while the BP neural network is worse in all indicators.This indicates that DBN can tap into the complex patterns of variation inherent in the data to improve prediction accuracy, is a better nonlinear prediction method than BP neural network, and can compensate for the shortcomings of traditional linear time series models.


## 6.3. Comparison of Convergence Rates

Figure6 gives the error convergence curves of the DBN deep learning model and the BP neural network in the training and learning phase. In terms of convergence speed, the DBN is faster than the BP neural network and does not show oscillations. This is mainly because the DBN uses a pretraining learning method to provide a good initial value for the network, while the BP neural network uses random parameters to initialise the network, so its error curve starts from a higher level and takes longer to converge and may suffer from oscillations, which is one of the reasons for the poor performance of the final BP prediction—convergence speed problem.Figure 6
DBN and BP convergence curves.


## 6.4. Impact of Label-Free Sample Pretraining on DBN Modelling Performance

From the aforementioned experiments, it can be seen that the DBN deep learning model achieve better modelling performance compared to BP neural networks by pretraining the learning of network weights. The DBN pretraining process only requires the participation of the set of unlabeled training samples, and nondependent economic indicators are required to participate in the pretraining. Although the weight still needs to be fine-tuned by the BP algorithm after pretraining, the amount of training samples required for the fine-tuning phase can be significantly reduced as the initial weights are already provided for the network in the pretraining phase. This feature allows DBN deep learning models to cope with applications where there are only a small number of labelled samples in the training sample and a large number of unlabelled samples, which are often the case in practical applications. For example, in the real data acquisition process, there are often missing data and a large number of unlabelled samples, or the data on the independent variables are easy to obtain but the data on the dependent variables are difficult or too expensive to obtain, resulting in a significant reduction in the amount of available labelled samples. In this subsection, we verify the impact of unlabeled sample pretraining on the modelling performance of DBN deep learning models by means of simulation experiments.Here, the consumer price index of residents is used as the dependent variabley to be predicted. In the above experiment, 70 of the 105 training samples were randomly selected as labeled samples, and the other training samples were used as unlabeled samples to simulate the loss of data labels. The model setting is the same as the prediction part of the above consumer price index. All 105 unlabeled samples (including 35 unlabeled samples and 70 labeled samples, in which the labeled samples do not use their label information) are pretrained with CD algorithm, and then the weight of DBN is fine-tuned with 70 labeled samples through BP algorithm.In terms of model parameter settings, the BP iteration search step of the DBN model was changed to 0.00005 and the number of iterations was increased to 1000 to avoid overfitting of the DBN during the training process of the back propagation algorithm due to the reduction of the marked samples, and the rest of the parameters were the same as above.Figures6 and 7 give a comparison between the predictions of DBN and BP for the consumer price index growth rate and the price index itself for the total 9 months from June 2014 to February 2015, respectively, and the true values. The training set of DBN is 70 marked samples +35 unmarked samples for a total of 105 samples, while the training set of BP is only 70 marked samples as it cannot use unmarked samples for training. Due to the reduced number of effective labelled training samples, the BP modelling performance is significantly worse than that of the DBN.Figure 7
DBN, BP, and VAR forecasts of consumer price index growth rates when the marked sample is reduced.Table2 gives a comparison of the prediction accuracy of DBN and BP when the training set is 70 labelled samples +35 unlabelled samples (105 samples in total). The table shows that DBN performs better than BP in terms of both mean absolute error (MAE), mean square error (MSE), and mean relative error [25]. At the same time, we note that although the number of labeled samples in this experiment is smaller compared to the experimental results in Table 1, the DBN can use the unlabeled samples to train and learn the DBN; thus it has little effect on the prediction performance of the DBN; for example, the MAE changes from 0.425 to 0.426, the MSE changes from 0.323 to 0.39, and the mean relative error changes from 0.418% to 0.420%. The mean relative error was changed from 0.418% to 0.420%. It can be seen that, after removing the labels of a certain amount of samples, the indicators of DBN have decreased, but not significantly, indicating that the pretraining learning can effectively use the unlabelled samples[3,26,27]. As the BP neural network can only use labelled samples for training and learning, its prediction performance decreases significantly when the number of training samples is reduced from 105 to 70; for example, MAE changes from 0.690 to 0.838, MSE changes from 0.752 to 1.074, and the average relative error changes from 0.679% to 0.8251%.VAR's MAE changes from 0.492 to 0.686, MSE from 0.686 to 0.686.686, MSE from 0.291 to 0.719, and mean relative error from 0.484% to 0.675%.Table 2
Comparison of the prediction accuracy of DBN and BP for the consumer price index when the marked sample is reduced.
Method/indexMAEMSEAverage relative error (%)DBN0.4260.3990.420BPNN0.8381.0740.825VAR0.6860.7190.675In order to further explore the impact of increasing the number of censored samples on the DBN performance and to observe the impact of censoring the censored samples on the DBN prediction performance while keeping the prediction performance within a certain range, we tested the set of samples listed in Table3 (i.e., 50 censored +55 uncensored, 30 censored +75 uncensored) and compared the test results with the previous experiment of 70 censored +35 unlabeled training sample sets from the previous experiment that were compared. For comparison purposes, Table 4 shows the prediction performance of the BP neural network under the same circumstances. Since the BP neural network can only be trained and learned using the labelled samples, the unlabelled samples in Table 3 cannot be used for the BP neural network. For a more intuitive comparison, Figure 8 plots the average relative error curves for the DBN and BP corresponding to different training sample cases.Table 3
Effect of labeled sample size on DBN prediction accuracy.
Index/number of labeled samplesMAEMSEAverage relative error (%)700.4260.3990.420500.4660.3100.459300.5570.5380.548Table 4
Effect of marked sample size on the prediction accuracy of BP neural network.
Index/number of labeled samplesMAEMSEAverage relative error (%)700.8381.0740.825500.9191.3190.904301.1681.791.15Figure 8
DBN, BP, and VAR forecasts of the consumer price index when the marked sample is reduced.The experimental results in Tables3 and 4 and Figure 8 show that the prediction error increases as the number of labeled samples decreases, but in comparison, since DBN can be pretrained with unlabeled samples, on the one hand, the prediction performance of DBN is always higher than that of BP and, on the other hand, the prediction performance of DBN decreases at a significantly lower rate than that of BP as the number of labeled samples decreases, which shows that DBN is a prediction method with better performance. On the other hand, DBN’s prediction performance decreases at a significantly lower rate than that of BP as the size of the marked sample decreases.When the data is partially missing, it can be seen from Figure9 that the prediction results of the BP neural network will be at a high level, which may mislead the government to adopt excessive macropolicies to suppress commodity prices and thus cause excessive changes in the consumer price index, so that the stability of each economic variable will be affected. For the VAR, although its prediction results are more in line with the average of the true values, the local fluctuations are somewhat different from the true values.Figure 9
Comparison of the effect of marked sample size on the relative error of DBN and BP neural network predictions.


## 7. Conclusions

In this paper, we use DBN and BP to empirically forecast two economic indicators, namely, consumer price index and total imports, and compare them with traditional VAR models, respectively, to confirm the applicability of DBN to nonlinear problems like economic forecasting in terms of prediction accuracy and convergence speed. In addition, this paper also verifies that the DBN is suitable for modelling nonlinear problems such as economic forecasting and performs better than BP and VAR. In addition, this paper also verifies that DBN can use a large number of unlabeled samples to pretrain the model in the case of limited training samples, thus enabling better prediction ability despite the limited number of labeled samples, proving that DBN have stronger generalization ability compared with BP.



---

*Source: 4429692-2022-03-15.xml*