# Construction and Simulation of the Enterprise Financial Risk Diagnosis Model by Using Dropout and BN to Improve LSTM

Authors: Weiwei Yang; Chaoxian Jia; Ruifeng Liu
Journal: Security and Communication Networks
 (2022)

---

## Abstract

In view of the financial risks faced by listed enterprises, how to accurately predict the risks is an important work. However, the traditional LSTM financial diagnosis model has the disadvantage of low accuracy; the specific reason is that the LSTM model has the problems of overfitting and gradient disappearance in risk diagnosis. Therefore, Dropout is adopted to solve the overfitting problem in the process of premodel prediction, and the BN algorithm is used to solve the gradient disappearance problem in the process of iteration. In order to verify the feasibility of above improvements, the financial data of China’s A-share listed enterprises from 2017 to 2020 are taken as samples to analyze the financial data of listed enterprises through single-step dimension and multistep dimension. The experimental results show that under the analysis of two dimensions, the financial prediction accuracy of the improved LSTM for T-2∼T-3 years can reach 83.96% and 91.19%, respectively, which indicates that through the above improvements, the model can be improved and has certain reference value.


---

## Body

## 1. Introduction

As an important branch of machine learning, with the help of internally complex mapping relationships, deep learning algorithms can extract features more efficiently. In virtue of the strong self-learning ability, it is widely used in image processing, education, architecture, UAV, medicine, and so on. For example, Gu et al. applied the GISTNet diagnostic model to the diameter of more than 5 cm gastrointestinal mesoma and stomach benign SMT diagnosis, and the results show that the diagnostic accuracy of the GISTNet depth model is much higher than that of manual diagnosis [1]. Liu et al. applied the AlexNet neural network to the automatic classification of indentation, and the results show that the accuracy of the individual recognition of the AlexNet model can reach more than 90%, and the individual image recognition rate can reach 28.26 ms [2]. Seok-Jae Heo et al. applied multiple deep learning algorithms to architectural color drawing, thus greatly reducing the difficulty of artificial restoration of ancient architectural color painting. It can be seen that the deep learning algorithm has become a hotspot of the current application and research. Deep learning algorithm is also applied to the field of financial early warning and anomaly recognition [3]. For example, Amit et al. applied the deep learning algorithm to the stock market, so as to provide effective solutions for them to forecast the risks as quickly as possible, but the research was limited to the idea [4]. Lu and Dong applied the deep confidence neural network to the power grid supplier and in the identification risk, concretely. The results also show that with the training ratio of more than 90%, the test accuracy can reach 92.56%, effectively realizing the identification risk [5]. XianTian established the deep learning model of the China aviation real estate enterprise risk warning model. The results show that AVIC has financial risks in 2016 and 2017, in line with the actual financial situation [6]. Zhao used the quantitative analysis method to evaluate financial risks, but this method has certain subjectivity [7]. Zhang et al. used the random forest algorithm to evaluate financial risks, which provides a reference for the evaluation of financial risks of supply chain [8]. Through the above research, it can be seen that trying to apply different deep learning algorithms in financial abnormal recognition and risk warning can not only improve the accuracy of financial recognition but also become the focus of discussing and studying the application of deep learning. However, in the above studies, some risk evaluation methods have strong subjectivity, such as hierarchical analysis, and some have low accuracy of evaluation. Therefore, on the basis of the above research, taking into account the time series relationship of financial data, a financial abnormal recognition model based on LSTM is proposed, and the accuracy of this model is verified. In addition, the innovation of this study is to try to propose a more informative and objective evaluation method.


## 2. Basic Principles and Structure of the LSTM Neural Network

### 2.1. Basic Principles

LSTM belonging to temporal-recurrent neural networks has evolved to address the long-term dependence of traditional RNN recurrent neural networks [4]. Therefore, the basic structure of LSTM follows the structure of the recurrent neural network. It composed of the input layer, hidden layer, and output layer, by establishing the continuous relationship between the upper time node and the next time node for output, as shown in Figure 1 [9–11].Figure 1
Basic structure of the LSTM.However, it was found by research that recurrent neural networks, so LSTM add three gating units, could influence real-time information on historical information better, so to obtain long-term retention and memory purposes and avoid gradient disappearance or the emergence of gradient expansion.


### 2.2. Basic Structure

Each LSTM unit contains one memory unit and three gating units, and the specific structure is illustrated in Figure2. Memory unit is mainly used to record the state of each neuron; input and output gates are mainly used to solve the data reception and correct parameter and output; the forgetting gate mainly controls how much the neural unit state is forgotten [12–17].Figure 2
The unit structure of LSTM.The calculation formulas of forgetting gate, input gate, and output gate are as follows:(1)ft=σWf⋅ht−1,xt+bf,it=σWi⋅ht−1,xt+bi,ot=σWo⋅ht−1,xt+bo.Among them,ht−1 represents the output of the LSTM neuron at the t−1 moment, and ht−1,xt is defined as the connection of the two vectors into a longer vector. Wf,bf, and ft, respectively, are the weight, bias top, and state of the forgotten door to the input sequence, and Wi, bi, and it are the weight, offset top, and state of the input gate. Wo, bo, and ot are the weight, bias top, and state of the output gate; σ is the activation function of the output. Then, the calculation of LSTM outputing ht is as follows:(2)C˜t=tanhWc⋅ht−1,xt+bc,Ct=ft∗Ct−1+it∗C˜t,ht=ot∗tanhCt.It can be seen thatC˜t represents the instantaneous state of input feature at time t, Ct is the current unit state, tanh is the activation function of output feature, and ht is the output of the current unit.Through the elaboration of the principles and structure, it can be seen that the LSTM neural network can solve the data with long-term dependencies and realize the mining of such data intrinsic connection better. In the business operation, financial crisis is not only a process of continuous accumulation of financial risks but also a concentrated embodiment of the risk accumulation to a certain extent. Therefore, the essence of financial warning for enterprises is to analyze this long-term dependence on data, so as to predict the probability of financial crisis in the next financial year and achieve the purpose of financial warning for enterprises.


## 2.1. Basic Principles

LSTM belonging to temporal-recurrent neural networks has evolved to address the long-term dependence of traditional RNN recurrent neural networks [4]. Therefore, the basic structure of LSTM follows the structure of the recurrent neural network. It composed of the input layer, hidden layer, and output layer, by establishing the continuous relationship between the upper time node and the next time node for output, as shown in Figure 1 [9–11].Figure 1
Basic structure of the LSTM.However, it was found by research that recurrent neural networks, so LSTM add three gating units, could influence real-time information on historical information better, so to obtain long-term retention and memory purposes and avoid gradient disappearance or the emergence of gradient expansion.


## 2.2. Basic Structure

Each LSTM unit contains one memory unit and three gating units, and the specific structure is illustrated in Figure2. Memory unit is mainly used to record the state of each neuron; input and output gates are mainly used to solve the data reception and correct parameter and output; the forgetting gate mainly controls how much the neural unit state is forgotten [12–17].Figure 2
The unit structure of LSTM.The calculation formulas of forgetting gate, input gate, and output gate are as follows:(1)ft=σWf⋅ht−1,xt+bf,it=σWi⋅ht−1,xt+bi,ot=σWo⋅ht−1,xt+bo.Among them,ht−1 represents the output of the LSTM neuron at the t−1 moment, and ht−1,xt is defined as the connection of the two vectors into a longer vector. Wf,bf, and ft, respectively, are the weight, bias top, and state of the forgotten door to the input sequence, and Wi, bi, and it are the weight, offset top, and state of the input gate. Wo, bo, and ot are the weight, bias top, and state of the output gate; σ is the activation function of the output. Then, the calculation of LSTM outputing ht is as follows:(2)C˜t=tanhWc⋅ht−1,xt+bc,Ct=ft∗Ct−1+it∗C˜t,ht=ot∗tanhCt.It can be seen thatC˜t represents the instantaneous state of input feature at time t, Ct is the current unit state, tanh is the activation function of output feature, and ht is the output of the current unit.Through the elaboration of the principles and structure, it can be seen that the LSTM neural network can solve the data with long-term dependencies and realize the mining of such data intrinsic connection better. In the business operation, financial crisis is not only a process of continuous accumulation of financial risks but also a concentrated embodiment of the risk accumulation to a certain extent. Therefore, the essence of financial warning for enterprises is to analyze this long-term dependence on data, so as to predict the probability of financial crisis in the next financial year and achieve the purpose of financial warning for enterprises.


## 3. Improvement of the LSTM Neural Network and the Model Construction

### 3.1. Model Improvement

Considering the gradient disappearance and overfitting problem, in the LSTM neural network model construction, two solution strategies are proposed. First, to introduce Dropout for the overfitting problem. During network training, Dropout would set an inactivation probability for neurons on each layer of the network. Based on it, the network automatically eliminates the neurons to simplify the network structure and avoid overfitting [18–23]. For the gradient vanishing problem, the BN algorithm is introduced. The LSTM neural network designed in this study adds the Dropout and BN algorithms to optimize the network structure. The specific improvement process is shown in Figure 3.Figure 3
Computational graph structure of the deep LSTM neural network.In order to solve the problem of gradient disappearance, the standardization layer is added to the output of the convolution layer, so as to readjust the input. The traditional dimension standardization formula is [24](3)x⌢k=xk−ExkVarxk,where Exk is the expected value of neuron xk in each batch of data; varxk represents the standard deviation of xk activation value. The same time, dimensionality is standardized, which will directly affect the characteristics of the network at the upper layer as well as the data and parameters at the lower layer. Therefore, the BN algorithm is introduced. The most effective part of the algorithm is the addition of learning parameters γ and β.(4)yk=γkx⌢k+βk.The features conforming to normal distribution are reduced to some extent, so as to maintain the original distribution trend of features. And the degree of reduction is learned by the network itself.(5)γ=Varxk,β=Exk.The forward conduction formula of the BN algorithm is(6)μB←1m∑i=1mxi,σB2←1m∑i=1mxi−μB2,x⌢i←xi−μBσB2+ε,yi←γx⌢i+β≡BNγ,βxi.The principle of the Dropout algorithm is that the weights of the neural network are not updated during the backpropagation process, but the weights are preserved for the next training. In this way, part of the neural network nodes can be removed, so that the size of the network will not grow too fast. After the training set enters the convolutional neural network, its output is the output of all networks, which makes the output result more stable and more reliable. The specific formula is as follows:(7)R=m∗aWV,where W is the matrix of d∗n; V means the column vector of n∗1; m represents the 01 column vector of d∗1; ax stands for activation function.


### 3.2. Training Method and Optimizer Selection

This paper trains the LSTM network through the minibatch method. The ultimate goal is to forecast the future closing price of the business interests. Therefore, the loss function of this study uses mean square error, MSE [25–27]. Meanwhile, in order to better conduct the network optimization training, the Adam optimizer is selected in this study. The optimizer is the most widely used in the field of deep learning, which has more efficient convergence speed and stronger learning effect than other algorithms.


## 3.1. Model Improvement

Considering the gradient disappearance and overfitting problem, in the LSTM neural network model construction, two solution strategies are proposed. First, to introduce Dropout for the overfitting problem. During network training, Dropout would set an inactivation probability for neurons on each layer of the network. Based on it, the network automatically eliminates the neurons to simplify the network structure and avoid overfitting [18–23]. For the gradient vanishing problem, the BN algorithm is introduced. The LSTM neural network designed in this study adds the Dropout and BN algorithms to optimize the network structure. The specific improvement process is shown in Figure 3.Figure 3
Computational graph structure of the deep LSTM neural network.In order to solve the problem of gradient disappearance, the standardization layer is added to the output of the convolution layer, so as to readjust the input. The traditional dimension standardization formula is [24](3)x⌢k=xk−ExkVarxk,where Exk is the expected value of neuron xk in each batch of data; varxk represents the standard deviation of xk activation value. The same time, dimensionality is standardized, which will directly affect the characteristics of the network at the upper layer as well as the data and parameters at the lower layer. Therefore, the BN algorithm is introduced. The most effective part of the algorithm is the addition of learning parameters γ and β.(4)yk=γkx⌢k+βk.The features conforming to normal distribution are reduced to some extent, so as to maintain the original distribution trend of features. And the degree of reduction is learned by the network itself.(5)γ=Varxk,β=Exk.The forward conduction formula of the BN algorithm is(6)μB←1m∑i=1mxi,σB2←1m∑i=1mxi−μB2,x⌢i←xi−μBσB2+ε,yi←γx⌢i+β≡BNγ,βxi.The principle of the Dropout algorithm is that the weights of the neural network are not updated during the backpropagation process, but the weights are preserved for the next training. In this way, part of the neural network nodes can be removed, so that the size of the network will not grow too fast. After the training set enters the convolutional neural network, its output is the output of all networks, which makes the output result more stable and more reliable. The specific formula is as follows:(7)R=m∗aWV,where W is the matrix of d∗n; V means the column vector of n∗1; m represents the 01 column vector of d∗1; ax stands for activation function.


## 3.2. Training Method and Optimizer Selection

This paper trains the LSTM network through the minibatch method. The ultimate goal is to forecast the future closing price of the business interests. Therefore, the loss function of this study uses mean square error, MSE [25–27]. Meanwhile, in order to better conduct the network optimization training, the Adam optimizer is selected in this study. The optimizer is the most widely used in the field of deep learning, which has more efficient convergence speed and stronger learning effect than other algorithms.


## 4. Test Validation

### 4.1. Selection of Financial Indicators

Mo Qi Kong et al. took debt paying ability and operating capacity as evaluation indexes to set up the financial risk evaluation system [28]. Zhao et al. took port enterprises as an example and adopted the method of factor analysis to build the evaluation system, including development potential and profitability [29]. Anzhong Huang et al. selected the financial risk indicators from 18 dimensions, such as weighted average interest rate, money supply M2, effective exchange rate, and so on. However, the above indicators are mainly selected from the market perspective [30]. Fitzpatrick Trevor et al. constructed financial risk indicators from the perspective of enterprises, which are based on the indicators such as solvency, profitability, and so on [31–33]. This paper mainly aimed at the listed companies. Therefore, with the reference to the above financial risk indicators, a risk indicator early warning system is constructed from seven perspectives including debt paying ability. The main financial indicators are given in Table 1.Table 1
Financial early warning index system based on LSTM.
Selected indexAbbreviationDebt paying abilityCash ratioCRAsset-liability ratioLEYCash flow ratioCASHCLOperating capacityAccounts receivable turnover rateARTURNOXInventory turnoverINTURNOXTurnover rate of current assetsVOLTotal asset turnoverTATODevelopment capacityGrowth rate of operating incomeRGGrowth rate of operating profitPGRNet profit growth rateEGGrowth rate of total assetsTAGREarning powerReturn on assetsROANet interest rate of total assetsROTReturn on net assetsROENet operating interest rateNPMReturn on investmentROICash flow analysisNet cash flow from operating activities per shareNCFOPSCash recovery rateCRARatio structureCash asset ratioCARFixed assets ratioLTCRProfitabilityAsset remuneration rateHERF


### 4.2. Sample Selection and Data Preprocessing

#### 4.2.1. Sample Source

According to the current listed company system in China, the companies listed in∗ ST are usually in extreme financial deterioration status. Therefore, the A-share listed companies in China from 2017–2020 were selected as the experimental samples, including those listed in ∗ ST from 2017 to 2020. At the same time, considering that the financial dilemma is a process accumulated in time, therefore, the probability of possible financial crisis is predicted from two dimensions of both single-step and multistep length. The prediction accuracy in the same dimension and different dimensions is compared, respectively, so to determine the best time steps. Among them, the train of thought of single-step dimension prediction is as follows: the data from T-2 (2020), T-3 (2019), and T-4 (2018) are used to predict the financial situation of T years. Through three training, the single annual samples with the highest prediction accuracy were obtained. The train of thought of multistep dimension prediction is to predict the financial status of T years with data from T-2 to T-3 years (2019) (2018) and T-2 to T-5 years (2017), to obtain the training samples with the highest prediction accuracy. Then, compare the prediction accuracy of the two dimensions to select the best prediction time steps. Samples from Tables 2 and 3 were selected by methods above.Table 2
Sample distribution of the financial early warning model of listed companies under single-step dimension.
Forecast dimensionYearTraining sampleTest sampleThe finance is normalFinancial difficultiesThe finance is normalFinancial difficultiesSingle-step lengthT-2 year2133 (2020)132 (2020)25 (2019)25 (2019)T-3 year2131 (2019)132 (2019）25 (2018)25 (2018)T-4 year2133 (2017)132 (2017)25 (2017)25 (2017)Table 3
Sample distribution of the financial early warning model of listed companies under multistep dimension.
Forecast dimensionYearTraining sampleTest sampleThe finance is normalFinancial difficultiesThe finance is normalFinancial difficultiesMultistep dimensionYear of T-2 to T-32131 (2019)132 (2019)2133 (2020)132 (2020)Year of T-2 to T-44264 (2018∼2019)264 (2018∼2019)2133 (2020)132 (2020)Year of T-2 to T-54264 (2017∼2018)264 (2017∼2018)4264 (2019∼2020)2 (2019∼2020)According to Tables2 and 3, 9258 samples in total were used for the LSTM nerve net training and testing.At the same time, according to the financial warning indicators in Tables2 and 3, the financial data of some listed companies are given in Table 4.Table 4
Some training samples of financial normal listed companies.
StockYearLEVTATOROANPMCRAHERF0000∗∗20170.21060.32550.08220.25200.087224.4122........................000∗6320180.84140.7226-0.0343-0.04650.034030.3451........................000∗∗720190.6693001760.03580.07090.004343.1718........................0007∗∗20200.63752.39010.01910.0016−0.058516.6620


#### 4.2.2. Data Preprocessing

From the financial data obtained above, the different index data collected have big differences. For convenient training, standardized processing of the above financial data was performed. A common method is to scale the raw data to 0-1 [10] by proportion. Therefore, this study selects the MinMaxScaler of the scikit-learn library in Python3.6.4 and normalized the data to 0-1, with the normalization formula as(8)Xscoled=X−Xminaxis=0Xmaxaxis=0−Xminaxis=0⋅max−min+min.In the above,X is the characteristic value to be normalized, while Xmin the minimum and Xmax the maximum of the character, respectively, min and max represent the range of values be set normalized, and axis=0 represents the normalization of each column by default [11]. Better training effect can be achieved by training of scaled data in the deep LSTM neural network of the financial early warning model.To facilitate the comparison of data set changes before and after data preprocessing, the results of the pretreatment of training samples of financially normally listed companies are shown in Table5.Table 5
Training samples from financial normal listed companies after preprocessed.
StockYearLEVTATOROANPMCRAHERF0000∗∗20170.37450.06010.78850.13950.61400.7148........................000∗6320180.84070.30200.66840.13200.64580.3977........................000∗∗720190.60241.00000.73760.13310.50100.1495........................0007∗∗20200.72630.09570.71310.13190.59730.0633


### 4.3. LSTM Specific Parameter Design

According to the results of some researchers, the parameters design of the LSTM neural network is as shown in Figure4.Figure 4
The LSTM neural network parameter design.In Figure4, input represents the input dimension of each network layer, and output represents the output dimension, both the input and output are 3D data. According to the arrow transmission direction of Figure 4, the top layer is the input layer, and the input data is a time step, all contains datasets with 32 features. Then, in the BN layer, the input dataset is normalized by using the BN algorithm [12]. Then, there is the LSTM layer, with the hidden nodes of 50,512,512, and 2 neurons in each layer. Then, it is the Dropout layer. This layer does not change the input and output dimensions, only updates the network parameters, and disconnects the neurons of the input layer according to a certain proportion, so to avoid overfitting problem. Through the four superposition of the three network layers mentioned above, the network output layer is finally obtained, and the input dimension is the output of the previous layer.


### 4.4. LSTM Training Process

The overall training process of the LSTM financial warning model is shown in Figure5.Figure 5
Data processing algorithm flow for the LSTM neural networks.First, input the learning sample, preprocess sample data, and convert timing data into supervised data to learn the processed data by preset earning parameters. Then, calculate the input and output values of units of each layer [13]; then, deviation of the target output and the actual output is calculated. If the calculation results converge, it predicts sample data inverse scaling and gives the prediction value, and then the algorithm ends. If there is no convergence, the neural network hidden layer nodes and learning times of LSTM will need to be adjust, and learning mode will be updated; sample learning will restart to guide the determination of optimal network parameters.


### 4.5. Results and Analysis

#### 4.5.1. Loss Function and Accuracy Changes in Different Dimensions

In this study, the number of neurons in the LSTM layer is set to 50,512,512,2, and there is one neuron in the output layer. The loss function uses the average absolute error MAE, the basic principle of which is to summarize the difference between the predicted value and the actual value, and it can be clearly seen the deviation amplitude of the forecast value. In the meantime, Adam stochastic gradient descent is used for network optimization, which can make the learning rate adjusted more effectively. The activation function of the LSTM layer is the sigmoid function, with a learning rate of 0.01, the iterations of 200, and the batch of 64.To better understand the situation of loss function and accuracy changes during training, this study will show the training process of the step dimension T-2 year and the multistep dimension T-2 to T-3 years in Figures6 and 7.Figures6 and 7 show that loss and val_loss are the changes of loss function in the training set and test set samples, respectively, and acc and val_acc are the change of prediction accuracy in training set and test set, respectively.Figure 6
Training process of a single-step dimension T-2 year.Figure 7
Training process of the multistep dimension T-2 to T-3 years.As can be seen from Figures6 and 7, mean absolute errors in both different dimensions are in a downward trend, with all improvements in accuracy, but the growth stops when a certain range is reached, indicating that the model is gradually stable. The sharp decline of MAE in training set indicates that the imitative effect of the model is better, and the MAE of the training set and test set are converged, indicating that the training attained the expectant goal.


#### 4.5.2. Loss Curve at Different Learning Rates

In deep learning, gradient descent is one of the most widely used optimization methods, but the two ways of batch and random gradient descent use the most. In line with the actual requirements of the experiment, this study chooses to use stochastic gradient descent, which only updates one data at a time, and the training speed is fast, but the disadvantage is that it is difficult to converge the minimum value. Through the experiment, it is found that reducing the learning rate can minimize the convergence of training results, and by constantly adjusting the parameters, the convergence effect of MAE can be improved, and the best training results can be obtained. Thus, the multistep dimension T-2 to T-3 years loss curve is as follows:In Figure8, the blue curve is the training set loss curve, and the orange one indicates the test set loss curve; the horizontal axis represents iterations during sample training, while the vertical axis represents the average error value change. It is seen from Figure 6 that the sample error decreases with the number of iterations increase, with local fluctuations in a small range. During initial training, the error dropped sharply from 10 to 50 sessions. Errors quickly dropping indicate that the model had fine-tuning. At the beginning of 100 sessions, the decrease in error gradually stabilized, which indicates that the model was close to the optimal process. According to the comprehensive analysis, the error of the training sample finally converged successfully and the fitting effect was good; the test sample error converged to local minimum. The fitting effect was poor compared with the training sample, but the stochastic gradient descent method used in this study optimized a good effect and had little impact on the model performance.Figure 8
Loss curve for the multistep dimension T-2 to T-3 years.


#### 4.5.3. Financial Early Warning Results and Analysis

(1) Financial Early Warning and Evaluation Indicators. This study selected the adaptive moment to estimate Adam stochastic gradient descent for optimization, to adjust the learning rate of each parameter dynamically, and evaluated the LSTM financial warning model by using the root mean square error RMSE. The computational expression for RMSE is(9)RMSE=1n∑i=1ny^i−yi2.The range of RMSE is0,+∞, the greater the error of the real value, the greater the value of RMSE. After many experiments, the RMSE of financial warning model based on the deep LSTM neural network was 29.7, indicating that the model predicted small error with real value and the model fit is high.(2) Financial Early Warning Results. In order to see the normal financial situation or financial difficulties of the enterprise better, the status column is added to the normalized data, and 0 represents normal situation and 1 represents financial difficulties. After reverse scaling the data, the prediction of the output status bar puts the output above 0.5–1, under 0.5–0, thus the final prediction result is shown in Figure 9.Figure 9
Financial early warning results based on the deep LSTM neural network model.As we can see from Figure6, in the single-step dimension, the prediction accuracy of T-2, T-3, and T-4 years is 84.67%, 84.01%, and 84.11%, respectively. The sample data close to the prediction year is with higher prediction accuracy. In multistep dimension, T-2 to T-3, T-2 to T-4, and T-2 to T-5, the prediction accuracy is 91.19%, 90.02%, and 90.05%, respectively; those two years close to the predicted year have higher step length prediction accuracy.It can be known from the comprehensive analysis that in the financial prediction based on the deep LSTM, the financial data using T-2 to T-3 has the highest accuracy in the neural network model.


## 4.1. Selection of Financial Indicators

Mo Qi Kong et al. took debt paying ability and operating capacity as evaluation indexes to set up the financial risk evaluation system [28]. Zhao et al. took port enterprises as an example and adopted the method of factor analysis to build the evaluation system, including development potential and profitability [29]. Anzhong Huang et al. selected the financial risk indicators from 18 dimensions, such as weighted average interest rate, money supply M2, effective exchange rate, and so on. However, the above indicators are mainly selected from the market perspective [30]. Fitzpatrick Trevor et al. constructed financial risk indicators from the perspective of enterprises, which are based on the indicators such as solvency, profitability, and so on [31–33]. This paper mainly aimed at the listed companies. Therefore, with the reference to the above financial risk indicators, a risk indicator early warning system is constructed from seven perspectives including debt paying ability. The main financial indicators are given in Table 1.Table 1
Financial early warning index system based on LSTM.
Selected indexAbbreviationDebt paying abilityCash ratioCRAsset-liability ratioLEYCash flow ratioCASHCLOperating capacityAccounts receivable turnover rateARTURNOXInventory turnoverINTURNOXTurnover rate of current assetsVOLTotal asset turnoverTATODevelopment capacityGrowth rate of operating incomeRGGrowth rate of operating profitPGRNet profit growth rateEGGrowth rate of total assetsTAGREarning powerReturn on assetsROANet interest rate of total assetsROTReturn on net assetsROENet operating interest rateNPMReturn on investmentROICash flow analysisNet cash flow from operating activities per shareNCFOPSCash recovery rateCRARatio structureCash asset ratioCARFixed assets ratioLTCRProfitabilityAsset remuneration rateHERF


## 4.2. Sample Selection and Data Preprocessing

### 4.2.1. Sample Source

According to the current listed company system in China, the companies listed in∗ ST are usually in extreme financial deterioration status. Therefore, the A-share listed companies in China from 2017–2020 were selected as the experimental samples, including those listed in ∗ ST from 2017 to 2020. At the same time, considering that the financial dilemma is a process accumulated in time, therefore, the probability of possible financial crisis is predicted from two dimensions of both single-step and multistep length. The prediction accuracy in the same dimension and different dimensions is compared, respectively, so to determine the best time steps. Among them, the train of thought of single-step dimension prediction is as follows: the data from T-2 (2020), T-3 (2019), and T-4 (2018) are used to predict the financial situation of T years. Through three training, the single annual samples with the highest prediction accuracy were obtained. The train of thought of multistep dimension prediction is to predict the financial status of T years with data from T-2 to T-3 years (2019) (2018) and T-2 to T-5 years (2017), to obtain the training samples with the highest prediction accuracy. Then, compare the prediction accuracy of the two dimensions to select the best prediction time steps. Samples from Tables 2 and 3 were selected by methods above.Table 2
Sample distribution of the financial early warning model of listed companies under single-step dimension.
Forecast dimensionYearTraining sampleTest sampleThe finance is normalFinancial difficultiesThe finance is normalFinancial difficultiesSingle-step lengthT-2 year2133 (2020)132 (2020)25 (2019)25 (2019)T-3 year2131 (2019)132 (2019）25 (2018)25 (2018)T-4 year2133 (2017)132 (2017)25 (2017)25 (2017)Table 3
Sample distribution of the financial early warning model of listed companies under multistep dimension.
Forecast dimensionYearTraining sampleTest sampleThe finance is normalFinancial difficultiesThe finance is normalFinancial difficultiesMultistep dimensionYear of T-2 to T-32131 (2019)132 (2019)2133 (2020)132 (2020)Year of T-2 to T-44264 (2018∼2019)264 (2018∼2019)2133 (2020)132 (2020)Year of T-2 to T-54264 (2017∼2018)264 (2017∼2018)4264 (2019∼2020)2 (2019∼2020)According to Tables2 and 3, 9258 samples in total were used for the LSTM nerve net training and testing.At the same time, according to the financial warning indicators in Tables2 and 3, the financial data of some listed companies are given in Table 4.Table 4
Some training samples of financial normal listed companies.
StockYearLEVTATOROANPMCRAHERF0000∗∗20170.21060.32550.08220.25200.087224.4122........................000∗6320180.84140.7226-0.0343-0.04650.034030.3451........................000∗∗720190.6693001760.03580.07090.004343.1718........................0007∗∗20200.63752.39010.01910.0016−0.058516.6620


### 4.2.2. Data Preprocessing

From the financial data obtained above, the different index data collected have big differences. For convenient training, standardized processing of the above financial data was performed. A common method is to scale the raw data to 0-1 [10] by proportion. Therefore, this study selects the MinMaxScaler of the scikit-learn library in Python3.6.4 and normalized the data to 0-1, with the normalization formula as(8)Xscoled=X−Xminaxis=0Xmaxaxis=0−Xminaxis=0⋅max−min+min.In the above,X is the characteristic value to be normalized, while Xmin the minimum and Xmax the maximum of the character, respectively, min and max represent the range of values be set normalized, and axis=0 represents the normalization of each column by default [11]. Better training effect can be achieved by training of scaled data in the deep LSTM neural network of the financial early warning model.To facilitate the comparison of data set changes before and after data preprocessing, the results of the pretreatment of training samples of financially normally listed companies are shown in Table5.Table 5
Training samples from financial normal listed companies after preprocessed.
StockYearLEVTATOROANPMCRAHERF0000∗∗20170.37450.06010.78850.13950.61400.7148........................000∗6320180.84070.30200.66840.13200.64580.3977........................000∗∗720190.60241.00000.73760.13310.50100.1495........................0007∗∗20200.72630.09570.71310.13190.59730.0633


## 4.2.1. Sample Source

According to the current listed company system in China, the companies listed in∗ ST are usually in extreme financial deterioration status. Therefore, the A-share listed companies in China from 2017–2020 were selected as the experimental samples, including those listed in ∗ ST from 2017 to 2020. At the same time, considering that the financial dilemma is a process accumulated in time, therefore, the probability of possible financial crisis is predicted from two dimensions of both single-step and multistep length. The prediction accuracy in the same dimension and different dimensions is compared, respectively, so to determine the best time steps. Among them, the train of thought of single-step dimension prediction is as follows: the data from T-2 (2020), T-3 (2019), and T-4 (2018) are used to predict the financial situation of T years. Through three training, the single annual samples with the highest prediction accuracy were obtained. The train of thought of multistep dimension prediction is to predict the financial status of T years with data from T-2 to T-3 years (2019) (2018) and T-2 to T-5 years (2017), to obtain the training samples with the highest prediction accuracy. Then, compare the prediction accuracy of the two dimensions to select the best prediction time steps. Samples from Tables 2 and 3 were selected by methods above.Table 2
Sample distribution of the financial early warning model of listed companies under single-step dimension.
Forecast dimensionYearTraining sampleTest sampleThe finance is normalFinancial difficultiesThe finance is normalFinancial difficultiesSingle-step lengthT-2 year2133 (2020)132 (2020)25 (2019)25 (2019)T-3 year2131 (2019)132 (2019）25 (2018)25 (2018)T-4 year2133 (2017)132 (2017)25 (2017)25 (2017)Table 3
Sample distribution of the financial early warning model of listed companies under multistep dimension.
Forecast dimensionYearTraining sampleTest sampleThe finance is normalFinancial difficultiesThe finance is normalFinancial difficultiesMultistep dimensionYear of T-2 to T-32131 (2019)132 (2019)2133 (2020)132 (2020)Year of T-2 to T-44264 (2018∼2019)264 (2018∼2019)2133 (2020)132 (2020)Year of T-2 to T-54264 (2017∼2018)264 (2017∼2018)4264 (2019∼2020)2 (2019∼2020)According to Tables2 and 3, 9258 samples in total were used for the LSTM nerve net training and testing.At the same time, according to the financial warning indicators in Tables2 and 3, the financial data of some listed companies are given in Table 4.Table 4
Some training samples of financial normal listed companies.
StockYearLEVTATOROANPMCRAHERF0000∗∗20170.21060.32550.08220.25200.087224.4122........................000∗6320180.84140.7226-0.0343-0.04650.034030.3451........................000∗∗720190.6693001760.03580.07090.004343.1718........................0007∗∗20200.63752.39010.01910.0016−0.058516.6620


## 4.2.2. Data Preprocessing

From the financial data obtained above, the different index data collected have big differences. For convenient training, standardized processing of the above financial data was performed. A common method is to scale the raw data to 0-1 [10] by proportion. Therefore, this study selects the MinMaxScaler of the scikit-learn library in Python3.6.4 and normalized the data to 0-1, with the normalization formula as(8)Xscoled=X−Xminaxis=0Xmaxaxis=0−Xminaxis=0⋅max−min+min.In the above,X is the characteristic value to be normalized, while Xmin the minimum and Xmax the maximum of the character, respectively, min and max represent the range of values be set normalized, and axis=0 represents the normalization of each column by default [11]. Better training effect can be achieved by training of scaled data in the deep LSTM neural network of the financial early warning model.To facilitate the comparison of data set changes before and after data preprocessing, the results of the pretreatment of training samples of financially normally listed companies are shown in Table5.Table 5
Training samples from financial normal listed companies after preprocessed.
StockYearLEVTATOROANPMCRAHERF0000∗∗20170.37450.06010.78850.13950.61400.7148........................000∗6320180.84070.30200.66840.13200.64580.3977........................000∗∗720190.60241.00000.73760.13310.50100.1495........................0007∗∗20200.72630.09570.71310.13190.59730.0633


## 4.3. LSTM Specific Parameter Design

According to the results of some researchers, the parameters design of the LSTM neural network is as shown in Figure4.Figure 4
The LSTM neural network parameter design.In Figure4, input represents the input dimension of each network layer, and output represents the output dimension, both the input and output are 3D data. According to the arrow transmission direction of Figure 4, the top layer is the input layer, and the input data is a time step, all contains datasets with 32 features. Then, in the BN layer, the input dataset is normalized by using the BN algorithm [12]. Then, there is the LSTM layer, with the hidden nodes of 50,512,512, and 2 neurons in each layer. Then, it is the Dropout layer. This layer does not change the input and output dimensions, only updates the network parameters, and disconnects the neurons of the input layer according to a certain proportion, so to avoid overfitting problem. Through the four superposition of the three network layers mentioned above, the network output layer is finally obtained, and the input dimension is the output of the previous layer.


## 4.4. LSTM Training Process

The overall training process of the LSTM financial warning model is shown in Figure5.Figure 5
Data processing algorithm flow for the LSTM neural networks.First, input the learning sample, preprocess sample data, and convert timing data into supervised data to learn the processed data by preset earning parameters. Then, calculate the input and output values of units of each layer [13]; then, deviation of the target output and the actual output is calculated. If the calculation results converge, it predicts sample data inverse scaling and gives the prediction value, and then the algorithm ends. If there is no convergence, the neural network hidden layer nodes and learning times of LSTM will need to be adjust, and learning mode will be updated; sample learning will restart to guide the determination of optimal network parameters.


## 4.5. Results and Analysis

### 4.5.1. Loss Function and Accuracy Changes in Different Dimensions

In this study, the number of neurons in the LSTM layer is set to 50,512,512,2, and there is one neuron in the output layer. The loss function uses the average absolute error MAE, the basic principle of which is to summarize the difference between the predicted value and the actual value, and it can be clearly seen the deviation amplitude of the forecast value. In the meantime, Adam stochastic gradient descent is used for network optimization, which can make the learning rate adjusted more effectively. The activation function of the LSTM layer is the sigmoid function, with a learning rate of 0.01, the iterations of 200, and the batch of 64.To better understand the situation of loss function and accuracy changes during training, this study will show the training process of the step dimension T-2 year and the multistep dimension T-2 to T-3 years in Figures6 and 7.Figures6 and 7 show that loss and val_loss are the changes of loss function in the training set and test set samples, respectively, and acc and val_acc are the change of prediction accuracy in training set and test set, respectively.Figure 6
Training process of a single-step dimension T-2 year.Figure 7
Training process of the multistep dimension T-2 to T-3 years.As can be seen from Figures6 and 7, mean absolute errors in both different dimensions are in a downward trend, with all improvements in accuracy, but the growth stops when a certain range is reached, indicating that the model is gradually stable. The sharp decline of MAE in training set indicates that the imitative effect of the model is better, and the MAE of the training set and test set are converged, indicating that the training attained the expectant goal.


### 4.5.2. Loss Curve at Different Learning Rates

In deep learning, gradient descent is one of the most widely used optimization methods, but the two ways of batch and random gradient descent use the most. In line with the actual requirements of the experiment, this study chooses to use stochastic gradient descent, which only updates one data at a time, and the training speed is fast, but the disadvantage is that it is difficult to converge the minimum value. Through the experiment, it is found that reducing the learning rate can minimize the convergence of training results, and by constantly adjusting the parameters, the convergence effect of MAE can be improved, and the best training results can be obtained. Thus, the multistep dimension T-2 to T-3 years loss curve is as follows:In Figure8, the blue curve is the training set loss curve, and the orange one indicates the test set loss curve; the horizontal axis represents iterations during sample training, while the vertical axis represents the average error value change. It is seen from Figure 6 that the sample error decreases with the number of iterations increase, with local fluctuations in a small range. During initial training, the error dropped sharply from 10 to 50 sessions. Errors quickly dropping indicate that the model had fine-tuning. At the beginning of 100 sessions, the decrease in error gradually stabilized, which indicates that the model was close to the optimal process. According to the comprehensive analysis, the error of the training sample finally converged successfully and the fitting effect was good; the test sample error converged to local minimum. The fitting effect was poor compared with the training sample, but the stochastic gradient descent method used in this study optimized a good effect and had little impact on the model performance.Figure 8
Loss curve for the multistep dimension T-2 to T-3 years.


### 4.5.3. Financial Early Warning Results and Analysis

(1) Financial Early Warning and Evaluation Indicators. This study selected the adaptive moment to estimate Adam stochastic gradient descent for optimization, to adjust the learning rate of each parameter dynamically, and evaluated the LSTM financial warning model by using the root mean square error RMSE. The computational expression for RMSE is(9)RMSE=1n∑i=1ny^i−yi2.The range of RMSE is0,+∞, the greater the error of the real value, the greater the value of RMSE. After many experiments, the RMSE of financial warning model based on the deep LSTM neural network was 29.7, indicating that the model predicted small error with real value and the model fit is high.(2) Financial Early Warning Results. In order to see the normal financial situation or financial difficulties of the enterprise better, the status column is added to the normalized data, and 0 represents normal situation and 1 represents financial difficulties. After reverse scaling the data, the prediction of the output status bar puts the output above 0.5–1, under 0.5–0, thus the final prediction result is shown in Figure 9.Figure 9
Financial early warning results based on the deep LSTM neural network model.As we can see from Figure6, in the single-step dimension, the prediction accuracy of T-2, T-3, and T-4 years is 84.67%, 84.01%, and 84.11%, respectively. The sample data close to the prediction year is with higher prediction accuracy. In multistep dimension, T-2 to T-3, T-2 to T-4, and T-2 to T-5, the prediction accuracy is 91.19%, 90.02%, and 90.05%, respectively; those two years close to the predicted year have higher step length prediction accuracy.It can be known from the comprehensive analysis that in the financial prediction based on the deep LSTM, the financial data using T-2 to T-3 has the highest accuracy in the neural network model.


## 4.5.1. Loss Function and Accuracy Changes in Different Dimensions

In this study, the number of neurons in the LSTM layer is set to 50,512,512,2, and there is one neuron in the output layer. The loss function uses the average absolute error MAE, the basic principle of which is to summarize the difference between the predicted value and the actual value, and it can be clearly seen the deviation amplitude of the forecast value. In the meantime, Adam stochastic gradient descent is used for network optimization, which can make the learning rate adjusted more effectively. The activation function of the LSTM layer is the sigmoid function, with a learning rate of 0.01, the iterations of 200, and the batch of 64.To better understand the situation of loss function and accuracy changes during training, this study will show the training process of the step dimension T-2 year and the multistep dimension T-2 to T-3 years in Figures6 and 7.Figures6 and 7 show that loss and val_loss are the changes of loss function in the training set and test set samples, respectively, and acc and val_acc are the change of prediction accuracy in training set and test set, respectively.Figure 6
Training process of a single-step dimension T-2 year.Figure 7
Training process of the multistep dimension T-2 to T-3 years.As can be seen from Figures6 and 7, mean absolute errors in both different dimensions are in a downward trend, with all improvements in accuracy, but the growth stops when a certain range is reached, indicating that the model is gradually stable. The sharp decline of MAE in training set indicates that the imitative effect of the model is better, and the MAE of the training set and test set are converged, indicating that the training attained the expectant goal.


## 4.5.2. Loss Curve at Different Learning Rates

In deep learning, gradient descent is one of the most widely used optimization methods, but the two ways of batch and random gradient descent use the most. In line with the actual requirements of the experiment, this study chooses to use stochastic gradient descent, which only updates one data at a time, and the training speed is fast, but the disadvantage is that it is difficult to converge the minimum value. Through the experiment, it is found that reducing the learning rate can minimize the convergence of training results, and by constantly adjusting the parameters, the convergence effect of MAE can be improved, and the best training results can be obtained. Thus, the multistep dimension T-2 to T-3 years loss curve is as follows:In Figure8, the blue curve is the training set loss curve, and the orange one indicates the test set loss curve; the horizontal axis represents iterations during sample training, while the vertical axis represents the average error value change. It is seen from Figure 6 that the sample error decreases with the number of iterations increase, with local fluctuations in a small range. During initial training, the error dropped sharply from 10 to 50 sessions. Errors quickly dropping indicate that the model had fine-tuning. At the beginning of 100 sessions, the decrease in error gradually stabilized, which indicates that the model was close to the optimal process. According to the comprehensive analysis, the error of the training sample finally converged successfully and the fitting effect was good; the test sample error converged to local minimum. The fitting effect was poor compared with the training sample, but the stochastic gradient descent method used in this study optimized a good effect and had little impact on the model performance.Figure 8
Loss curve for the multistep dimension T-2 to T-3 years.


## 4.5.3. Financial Early Warning Results and Analysis

(1) Financial Early Warning and Evaluation Indicators. This study selected the adaptive moment to estimate Adam stochastic gradient descent for optimization, to adjust the learning rate of each parameter dynamically, and evaluated the LSTM financial warning model by using the root mean square error RMSE. The computational expression for RMSE is(9)RMSE=1n∑i=1ny^i−yi2.The range of RMSE is0,+∞, the greater the error of the real value, the greater the value of RMSE. After many experiments, the RMSE of financial warning model based on the deep LSTM neural network was 29.7, indicating that the model predicted small error with real value and the model fit is high.(2) Financial Early Warning Results. In order to see the normal financial situation or financial difficulties of the enterprise better, the status column is added to the normalized data, and 0 represents normal situation and 1 represents financial difficulties. After reverse scaling the data, the prediction of the output status bar puts the output above 0.5–1, under 0.5–0, thus the final prediction result is shown in Figure 9.Figure 9
Financial early warning results based on the deep LSTM neural network model.As we can see from Figure6, in the single-step dimension, the prediction accuracy of T-2, T-3, and T-4 years is 84.67%, 84.01%, and 84.11%, respectively. The sample data close to the prediction year is with higher prediction accuracy. In multistep dimension, T-2 to T-3, T-2 to T-4, and T-2 to T-5, the prediction accuracy is 91.19%, 90.02%, and 90.05%, respectively; those two years close to the predicted year have higher step length prediction accuracy.It can be known from the comprehensive analysis that in the financial prediction based on the deep LSTM, the financial data using T-2 to T-3 has the highest accuracy in the neural network model.


## 5. Conclusion

In conclusion, the LSTM financial warning model proposed in this study has small error and good fit. Experimental results show that they are narrow in both single-step and multistep dimensions. The prediction accuracy of T-2 year and T-2 to T-3 years is 83.96% and 91.19%, respectively, indicating that the multistep dimensional prediction of T-2 to T-3 years has the highest prediction accuracy and can more effectively predict the financial status of listed companies inT years. It verifies that the model of this study can be used in the early warning of the company finance.



---

*Source: 4767980-2022-08-30.xml*