# Textual Information Classification of Campus Network Public Opinion Based on BILSTM and ARIMA

Authors: Wenyi Wang
Journal: Wireless Communications and Mobile Computing
 (2022)

---

## Abstract

To address the problem that it is difficult for traditional opinion analysis models to accurately analyze textual information of campus online public opinion in various formats, a deep learning-based online opinion analysis method is proposed by combining BILSTM and ARIMA models. By using BILSTM sentiment classification model to predict and analyze the text data of campus online public opinion, the sentiment polarity of online public opinion information was well predicted, and the trend prediction of online public opinion was completed by combining ARIMA model difference and temporal preprocessing BILSTM with accuracy values as the original sequence. The simulation results show that the proposed method can better achieve the sentiment prediction of campus online opinion event texts and can predict the general trend of campus online opinion development. The prediction results can well reflect the actual online public opinion and have better prediction accuracy compared with CNN or LSTM models, which can reach more than 80%.


---

## Body

## 1. Introduction

With the development of the internet, the network public opinion information presents instantaneous massive, diversified, and unstructured characteristics. Traditional public opinion analysis methods, such as SVM algorithm and simple Bayes based on shallow machine learning, can no longer meet the data mining and analysis task. Therefore, it is imperative to find a method to analyze network public opinion quickly and analyze massive data accurately. In recent years, the development and extensive applications of deep learning have provided new ideas for data mining and analysis, and its strong learning and expression ability have achieved good results in emotion analysis. Therefore, relevant research scholars put their application and network public opinion analysis, such as Brian Greenhill and others, to establish deep learning network models based on public opinion research experiments to mine and analyze the data and discuss how international organizations can shape public opinion and correctly guide the public [1, 2]. Xiang et al. and Yi and Wang realized rapid public opinion analysis by constructing a public opinion analysis model based on deep learning [3, 4]. Mao and Song built a multimodal emotion analysis and emotion classification model through multilevel context extraction and attention-based context multimodal fusion mode and verified the effectiveness of the model by taking online public opinion text as the research object, which can effectively analyze online public opinion [5, 6]. Tian et al. and Ghanbari-Adivi and Mosleh have proposed a multiclass multidomain instance migration data-level sampling method based on migration learning [7, 8]. The network public opinion text collected through this method can truly reflect the public polar emotion, which has a positive significance for analyzing and predicting the direction of network public opinion. Based on the above research results, this paper proposes a deep learning-based online public opinion analysis method, which realizes the prediction and analysis of the online public opinion development trend by combining the BILSTM model and the ARIMA model.


## 2. Basic Approach

### 2.1. LSTM Network Profile

LSTM network is a modified long-term memory neural network for traditional recurrent neural network (RNN) gradient extinction problem, using the “gate” structural memory neuron node information in network training [9]. The internal structure of LSTM network neurons is more complex than the RNN network, as shown in Figure 1 [10]. In the figure, i indicates the input gate, c the memory unit, c-in represents the candidate memory unit, f the forgetting gate, h represents the output unit, and o the output gate. Among them, i, c, f, and o are a part of the LSTM network that can store information for long and short periods.Figure 1
Schematic diagram of LSTM neuron structure.Setxt∈Rmt=1,⋯,n represents word vector, X=x1,x2,⋯,xn the set of word vectors is, and the k-dimensional intermediate node in the LSTM network, then, the value of the input gate it can be calculated through the following equation [11]:
(1)it=SigωxiXt+ωhiht−1+bi.In the formula,ωxi∈Rm×k and ωhi∈Rk×k represent the parameter matrix; ct∈Rmct∈Rm the word vector of the t-word; ht−1∈Rk the t−1 neuron output eigenvector; and bi∈Rk the bias vector.The forgetting gateft can be calculated by the following equation [12]:
(2)ft=Sigωxfxt+ωhfht−1+bf.In the formula,ωxf∈Rm×k and ωhf∈Rk×k represent the parameter matrix, ht−1∈Rk the output eigenvector of t−1 neurons, and bf∈Rk the bias vector.The output gateot is calculated as follows by [13]
(3)ot=SigωxoXt+ωhoht−1+ba.In the equation,ωxo∈Rk×k and ωho∈Rk×k represent the parameter matrix, ht−1∈Rk the t−1 neuron output eigenvector, and bo∈Rk the bias vector.Enterct and ht−1 into the tanh function to obtain the t neuron candidate memory cell value, with the following formula [14]:
(4)c_int=tanhωxcXt+ωhcht−1+bc_in.In the formula,ωxc∈Rm×k and ωho∈Rk×k represent the parameter matrix; bc_in∈Rk represents the bias vector.The word vectorct of the current state t of the memory unit can be calculated by the following equation [15]:
(5)ct=ft×ct−1+it×c_int.The outputht of the memory unit state value can be calculated by the following equation [16]:
(6)ht=ot×tanhct.The LSTM network can well capture long distance dependencies, but it fails to encode the information from back to forward. Therefore, in order to better capture the bidirectional semantics, the two-way LSTM (BILSTM) network was used to construct the network public opinion emotion classification model in this study [17]. Since BILSTM adopts two LSTM layers in the network structure, and the analytical prediction method is the same as in LSTM, the BILSTM network will no longer be elaborated in this paper.


### 2.2. Introduction of ARIMA Model

The ARIMA model is a classical statistical time-series model. It makes predictions for the time series on the premise that the sequence must be stable and satisfy the nonwhite noise. Therefore, the sequence stationarity and nonwhite noise tests should be completed before using this model for prediction. In general, the difference and ADF test for the sequence are as follows: the original sequence is differential to obtain the first-order difference of the sequence, and ADF is used to test the stable [18]. If the returned result adf value is below 1%, 5%, and 10% the criticality value, the P value limitation is below 0.05, which means that the sequence has smoothness, otherwise, perform secondary or multiple differences until the sequence is stationary. Sequence nonwhite noise test is the size of P and 0.05 of the first-order differential sequence statistics obtained in testing stationarity detection, if P<0.05, indicating that the sequence meets the nonwhite noise condition, otherwise, the sequence is white noise and does not have temporal correlation [19].The basic flow of the ARIMA model is predicted in Figure2.Figure 2
Basic flow of ARIMA model.


## 2.1. LSTM Network Profile

LSTM network is a modified long-term memory neural network for traditional recurrent neural network (RNN) gradient extinction problem, using the “gate” structural memory neuron node information in network training [9]. The internal structure of LSTM network neurons is more complex than the RNN network, as shown in Figure 1 [10]. In the figure, i indicates the input gate, c the memory unit, c-in represents the candidate memory unit, f the forgetting gate, h represents the output unit, and o the output gate. Among them, i, c, f, and o are a part of the LSTM network that can store information for long and short periods.Figure 1
Schematic diagram of LSTM neuron structure.Setxt∈Rmt=1,⋯,n represents word vector, X=x1,x2,⋯,xn the set of word vectors is, and the k-dimensional intermediate node in the LSTM network, then, the value of the input gate it can be calculated through the following equation [11]:
(1)it=SigωxiXt+ωhiht−1+bi.In the formula,ωxi∈Rm×k and ωhi∈Rk×k represent the parameter matrix; ct∈Rmct∈Rm the word vector of the t-word; ht−1∈Rk the t−1 neuron output eigenvector; and bi∈Rk the bias vector.The forgetting gateft can be calculated by the following equation [12]:
(2)ft=Sigωxfxt+ωhfht−1+bf.In the formula,ωxf∈Rm×k and ωhf∈Rk×k represent the parameter matrix, ht−1∈Rk the output eigenvector of t−1 neurons, and bf∈Rk the bias vector.The output gateot is calculated as follows by [13]
(3)ot=SigωxoXt+ωhoht−1+ba.In the equation,ωxo∈Rk×k and ωho∈Rk×k represent the parameter matrix, ht−1∈Rk the t−1 neuron output eigenvector, and bo∈Rk the bias vector.Enterct and ht−1 into the tanh function to obtain the t neuron candidate memory cell value, with the following formula [14]:
(4)c_int=tanhωxcXt+ωhcht−1+bc_in.In the formula,ωxc∈Rm×k and ωho∈Rk×k represent the parameter matrix; bc_in∈Rk represents the bias vector.The word vectorct of the current state t of the memory unit can be calculated by the following equation [15]:
(5)ct=ft×ct−1+it×c_int.The outputht of the memory unit state value can be calculated by the following equation [16]:
(6)ht=ot×tanhct.The LSTM network can well capture long distance dependencies, but it fails to encode the information from back to forward. Therefore, in order to better capture the bidirectional semantics, the two-way LSTM (BILSTM) network was used to construct the network public opinion emotion classification model in this study [17]. Since BILSTM adopts two LSTM layers in the network structure, and the analytical prediction method is the same as in LSTM, the BILSTM network will no longer be elaborated in this paper.


## 2.2. Introduction of ARIMA Model

The ARIMA model is a classical statistical time-series model. It makes predictions for the time series on the premise that the sequence must be stable and satisfy the nonwhite noise. Therefore, the sequence stationarity and nonwhite noise tests should be completed before using this model for prediction. In general, the difference and ADF test for the sequence are as follows: the original sequence is differential to obtain the first-order difference of the sequence, and ADF is used to test the stable [18]. If the returned result adf value is below 1%, 5%, and 10% the criticality value, the P value limitation is below 0.05, which means that the sequence has smoothness, otherwise, perform secondary or multiple differences until the sequence is stationary. Sequence nonwhite noise test is the size of P and 0.05 of the first-order differential sequence statistics obtained in testing stationarity detection, if P<0.05, indicating that the sequence meets the nonwhite noise condition, otherwise, the sequence is white noise and does not have temporal correlation [19].The basic flow of the ARIMA model is predicted in Figure2.Figure 2
Basic flow of ARIMA model.


## 3. Network Public Opinion Analysis Model Based on Deep Learning

### 3.1. Overall Framework of the Model

Based on the above analysis, this study proposes a deep learning-based online public opinion analysis method by combining the BILSTM emotion classification model and the ARIMA time-series model, so as to more efficiently and accurately realize the trend prediction and emotion analysis of online public opinion. The overall framework of the network public opinion analysis model based on deep learning is shown in Figure3. First, the BILSTM emotion classification prediction model is constructed, and the model is optimized with the network public opinion text data; then, the ARIMA model is the original time series through difference processing and nonwhite noise test. Finally, the trained model is used to analyze and predict and output the results so to realize the analysis of network public opinion. Among them, BILSTM model construction and ARIMA model modeling are the focus of this study, and a detailed description of these two model construction methods will be explained below with details.Figure 3
Overall framework of the model.


### 3.2. BILSTM Model Construction

The structure of the BILSTM emotion classification prediction model constructed in this study is shown in Figure4, which includes the input layer, bidirectional LSTM layer, concat layer, fullconnectionlayer+softmaxlayer, and specific constructed methods as follows:
(1)
First, to create the word embedding layer to load and pretrain the network public opinion text word vector(2)
Enter the first word at the beginning of each text in the forward LSTM layer and the last word for each text in the backward LSTM layer, to judge the emotional polarity of the entire text based on the contextual information(3)
Putting to use tf.concat function splices the output tensor of the bidirectional LSTM layer, and setting the parameter axis to -1 to represent taking its he bottom first(4)
Adding the full-connection hidden layer and output the results, then, the output matrix vectors are connected to the softmax layer by full connection, at the same time, calling the softmax function to obtain the determination probability distribution for each categoryFigure 4
Structure of BILSTM model.To prevent model overfitting, dropout was added to the construction of BILSTM [20].


### 3.3. ARIMA Model Construction

The ARIMA model construction process is mainly divided into four steps, details as follows:(1)
Raw Time Series Acquiring. Based on the BILSTM emotion classification prediction model prediction results, each time node was counted for predicting the correct results, and the prediction accuracy was calculated for each time point and used as the original time series(2)
The raw time series were dealt with first-order difference and putting to use of ADF test for the stability test of first-order difference sequence, when the sequence has stabilization characteristic, nonwhite noise test will be taken(3)
The sequence autocorrelation function ACF and the partial correlation function PACF plots were drawn based on the first-order differential to determine the dragging and truncation conditions of the ACF and PACF plots. Hangover is the ordinate value of the graph is not 0 or fluctuates around 0; truncation is after the ordinate value of the graph is greater than a constant, the ordinate value tends to 0. When ACF as a tail and PACF as a truncation, AR(p) model is selected; when ACF cuts ACF as a truncation and PACF as a tail, MA(q) model is selected; when ACF and PACF both are tails, ARIMA (p, q) model is selected [21](4)
After selected to use the ARIMA (p, q) model, the ARIMA (p, 1, q) was ordered by the BIC Bayesian information criterion, where the ARIMA model construction was completed


## 3.1. Overall Framework of the Model

Based on the above analysis, this study proposes a deep learning-based online public opinion analysis method by combining the BILSTM emotion classification model and the ARIMA time-series model, so as to more efficiently and accurately realize the trend prediction and emotion analysis of online public opinion. The overall framework of the network public opinion analysis model based on deep learning is shown in Figure3. First, the BILSTM emotion classification prediction model is constructed, and the model is optimized with the network public opinion text data; then, the ARIMA model is the original time series through difference processing and nonwhite noise test. Finally, the trained model is used to analyze and predict and output the results so to realize the analysis of network public opinion. Among them, BILSTM model construction and ARIMA model modeling are the focus of this study, and a detailed description of these two model construction methods will be explained below with details.Figure 3
Overall framework of the model.


## 3.2. BILSTM Model Construction

The structure of the BILSTM emotion classification prediction model constructed in this study is shown in Figure4, which includes the input layer, bidirectional LSTM layer, concat layer, fullconnectionlayer+softmaxlayer, and specific constructed methods as follows:
(1)
First, to create the word embedding layer to load and pretrain the network public opinion text word vector(2)
Enter the first word at the beginning of each text in the forward LSTM layer and the last word for each text in the backward LSTM layer, to judge the emotional polarity of the entire text based on the contextual information(3)
Putting to use tf.concat function splices the output tensor of the bidirectional LSTM layer, and setting the parameter axis to -1 to represent taking its he bottom first(4)
Adding the full-connection hidden layer and output the results, then, the output matrix vectors are connected to the softmax layer by full connection, at the same time, calling the softmax function to obtain the determination probability distribution for each categoryFigure 4
Structure of BILSTM model.To prevent model overfitting, dropout was added to the construction of BILSTM [20].


## 3.3. ARIMA Model Construction

The ARIMA model construction process is mainly divided into four steps, details as follows:(1)
Raw Time Series Acquiring. Based on the BILSTM emotion classification prediction model prediction results, each time node was counted for predicting the correct results, and the prediction accuracy was calculated for each time point and used as the original time series(2)
The raw time series were dealt with first-order difference and putting to use of ADF test for the stability test of first-order difference sequence, when the sequence has stabilization characteristic, nonwhite noise test will be taken(3)
The sequence autocorrelation function ACF and the partial correlation function PACF plots were drawn based on the first-order differential to determine the dragging and truncation conditions of the ACF and PACF plots. Hangover is the ordinate value of the graph is not 0 or fluctuates around 0; truncation is after the ordinate value of the graph is greater than a constant, the ordinate value tends to 0. When ACF as a tail and PACF as a truncation, AR(p) model is selected; when ACF cuts ACF as a truncation and PACF as a tail, MA(q) model is selected; when ACF and PACF both are tails, ARIMA (p, q) model is selected [21](4)
After selected to use the ARIMA (p, q) model, the ARIMA (p, 1, q) was ordered by the BIC Bayesian information criterion, where the ARIMA model construction was completed


## 4. Results and Analyses

### 4.1. Experimental Environment

This experiment was implemented on the tensorflow framework using the export-saved model function to preserve the emotion classification model in estimator and inputting test data via the predictor function, with the specific configuration as in Table1.Table 1
Hardware configuration of experimental environment.
CPUIntel(R) Core(TM) i7-7770HQ 2.80 GHzOperating systemWindows10 64bitInternal storage8.00 GBProgramming languagePython 4.3Compile softwarePycharmDeep learning frameworkTensorflow 1.10.0


### 4.2. Data Source and Preprocessing

The experimental data include two parts: standard dataset part and real sample dataset. The standard dataset is the online-shopping-10-cats dataset. It contains 21,091 reviews in mobile phones, clothing, and hotels, divided into 8,033 positive comments, 4,355 neutral comments, and 8,703 negative ones [22], which is in Table 2.Table 2
Distribution of three classifications of standard data set (unit:strip).
Negative classNeutral classPositive classStandard data set870343558033The real sample data is the public opinion information from the one paper websites. Climbed 155 related web pages, each corresponding to a news story, are ranging from August 5, 2019 to October 5, 2020. Climbing content includes news headlines, publishing user nicknames, publication time, news time point, main content, and number of comments. Among them, the main research content of this article is the comment subject.Considering that the climbed content has the problem of invalid characters, to avoid its impact on the prediction results, the 155 text files climbed were studied to remove invalid characters, deweight, text cleaning, Chinese segmentation, etc., retain the preprocessing operation of the comment subject content and comment time, and split them, so that each comment and its release time are kept in a separate txt folder. In addition, the collected comments were annotated, classifying comments as positive, neutral, and negative. Through the above operation, this article finally obtained 4,184 annotated experimental data, including 460 positive comments, 1,750 neutral comments, and 1,974 negative comments, which is in Table3.Table 3
Distribution of three classifications of real samples (unit:strip).
Negative classNeutral classPositive classTotalTest sample197417504604184


### 4.3. Parameter Setting

Different model parameters for this experiment are set up in Table4. In the table, lstm-size represents the number of LSTM units in the hidden layer; dropout represents the probability of random retention of the hidden layer neurons, epoch represents the number of iterations, and batch-size represents the number of samples selected per learning optimization of the network [23].Table 4
Parameter settings of different models.
ParameterCNNLSTMBILSTMWord vector dimension300 dimension300 dimension300 dimensionConvolutional kernel size[2–4]Number of filters64Number of feature layers300Lstm_size6464Dropout0.50.50.5Epoch252525Batch size202020


### 4.4. Evaluating Indicator

Based on the above comment classification, this paper constructs three-classification confusion matrix model evaluation metrics, meanwhile, precision, recall, andF1 values were used to evaluate the test results, specific computational method as the following formulas [24, 25].
(7)Precision+=TPTP+FP,(8)Recall+=TPTP+FN,(9)F1=2×Precion×RecallPrecion+Recall.In the formula, TP represents true positive examples; TN represents false positive examples; FP represents false positive examples.


### 4.5. Experimental Result

#### 4.5.1. Testing Results of the Standard Dataset

In order to verify the prediction effect of the BILSTM model, the standard dataset training model was used in the study and predicted the three categories of the real sample dataset. At the same time comparing the prediction results with the CNN, LSTM model that also using standard data and training, and the confusion matrix of the three models [26]. The prediction results of different models according to the three-classification confusion matrix are shown in Figure 5. As shown in Figure 5, the prediction results obtained by the prediction models trained with standard data sets are not ideal for most of the comment data, and the overall prediction accuracy is only about 50%. The reason may be that the standard data set has certain differences from the real sample data set, leading to differences in feature extraction and specific words, which thus makes the model generalization ability of the training is poor and unable to accurately predict the network public opinion.Figure 5
Predicted results of different models on standard data sets.


#### 4.5.2. Results of Real Sample Training Model

The real sample dataset was randomly divided into training set and test set as 1 : 2 ratio. The CNN, LSTM, and BILSTM classification models were trained by the training model, and the test set was input into the classification model where the training was completed, the confusion matrix of different model, and the prediction results of different models were calculated by the index evaluation method, such as in Figure5 [27]. As can be seen from Figure 6, the prediction model trained with the real sample dataset has a high prediction accuracy for most comment data, with an average prediction accuracy of over 76%. It has better predictive results compared to the classification models trained on the standard datasets.Figure 6
Predicted results of different models on real sample data sets.


#### 4.5.3. Mixed Dataset Test Results

One-third of the data in the real sample dataset was randomly selected to the standard dataset to obtain the mixed dataset [28]. The CNN, LSTM, and BILSTM models were trained by using a mixed dataset and the classification model tested with the remaining two-thirds of the real sample set, getting the prediction outcome confusion, and computing the prediction results for different models, as in Figure 7. It is known from Figure 7 that compared to classification models trained on standard data or classification model on a real-sample single dataset, the classification model trained with a mixed dataset predicted better results, and the LSTM and BILSTM models achieved 80% prediction accuracy with good prediction accuracy.Figure 7
Predicted results of different models on mixed data sets.


#### 4.5.4. Comparison and Analysis

In order to verify the emotional evaluation results of the three deep learning classification models under different training methods. In Table5, the first three data groups represent the classification prediction results of CNN, LSTM, and BILSTM models under the standard dataset; the middle three groups represent the CNN, LSTM, and BILSTM models under the real sample dataset; and the latter three groups represent the classification prediction results of CNN, LSTM, and BILSTM models under the mixed dataset. As shown in the table, a classification model trained on mixed datasets of partial real sample data was added, and it can effectively improve the discrimination and prediction accuracy of the emotion of online public opinion comments. Compared to the classification models trained on a single dataset, it has a better classification prediction effect. According to the first three groups, models were learned only by using standard datasets. It is difficult to accurately learn the network public opinion information and predict it, and then comparing the prediction results between the middle three groups and the latter three groups. Standard datasets improve the model to learn and extract features; moreover, it improves the accuracy of the model prediction. Overall, compared to the CNN and LSTM models, the BILSTM model performs the best on standard or real sample or mixed datasets.Table 5
Comparison of model classification and prediction results.
NegativeNeutralPositiveAccuracyPrecisionRecallF1PrecisionRecallF1PrecisionRecallF1Modle10.48300.99240.64980.64290.01540.03010.79070.14780.24910.4909Modle20.48450.98730.65000.64710.01890.03660.55450.13260.21400.4883Modle30.49100.97620.65340.62960.03890.07310.44080.14570.21900.4928Modle40.77460.84620.80880..86050.67700.75780.5840.84390.69030.7718Modle50.81760.80690.81220.83530.69880.76100.57240.9480.71380.7762Modle60.78700.830.80790.86780.63350.73230.53860.96820.69210.7602Modle70.74700.76770.75720.67440.78830.72690.93550.25140.39640.7158Modle80.91450.69150.78760.71340.93930.81090.9570.70810.8140.8021Modle90.88940.76080.82010.77440.87630.82220.81100.85550.83260.8226


#### 4.5.5. Prediction Results and Analysis

To validate the predictive effect of the proposed model, the study divided the real sample data into training set and test set by time and used the training set as the original time series to construct ARIMA (p, 1, q), with the specific division cases presented in Table6 [16, 29].Table 6
Division of training set and test set.
Training intervalTest intervalPrediction pointExperiment 12019-08-05 to 2020-03-162020-04-19 to 2020-10-057Experiment 22019-08-05 to 2020-03-282020-03-16 to 2020-10-058Experiment 32019-08-05 to 2020-02-152020-02-16 to 2020-10-0510Experiment 42019-08-05 to 2020-01-302020-02-03 to 2020-10-0512The proposed model was used to predict the time series of the time training set to obtain the prediction data for each set of experimental prediction points, as shown in Figures8–11. It can be seen that with the accuracy value of each time node as the time series, the sequence meets the requirements of the ARIMA model through the stationarity and nonwhite noise test, which can be used for the trend analysis and prediction of network public opinion trends. According to the time-series map of the accuracy of the BILSTM prediction model, we can see the numerical trend of the accuracy, and conclude that with the increase of the event occurrence time, the accuracy of the prediction model has gradually decreased. The reason for the analysis may be that with the change of the network public opinion of the news events, public comment on the event has changed. Compared with experiment three and four, the prediction results of experiment one and two are better and basically coincide with the real accuracy data, indicating that the longer the forward prediction time of the proposed model, the greater the error is in line with the general time series prediction characteristics.Figure 8
Predicted results of experiment 1.Figure 9
Predicted results of experiment 2.Figure 10
Predicted results of experiment3.Figure 11
Predicted results of experiment 4.


## 4.1. Experimental Environment

This experiment was implemented on the tensorflow framework using the export-saved model function to preserve the emotion classification model in estimator and inputting test data via the predictor function, with the specific configuration as in Table1.Table 1
Hardware configuration of experimental environment.
CPUIntel(R) Core(TM) i7-7770HQ 2.80 GHzOperating systemWindows10 64bitInternal storage8.00 GBProgramming languagePython 4.3Compile softwarePycharmDeep learning frameworkTensorflow 1.10.0


## 4.2. Data Source and Preprocessing

The experimental data include two parts: standard dataset part and real sample dataset. The standard dataset is the online-shopping-10-cats dataset. It contains 21,091 reviews in mobile phones, clothing, and hotels, divided into 8,033 positive comments, 4,355 neutral comments, and 8,703 negative ones [22], which is in Table 2.Table 2
Distribution of three classifications of standard data set (unit:strip).
Negative classNeutral classPositive classStandard data set870343558033The real sample data is the public opinion information from the one paper websites. Climbed 155 related web pages, each corresponding to a news story, are ranging from August 5, 2019 to October 5, 2020. Climbing content includes news headlines, publishing user nicknames, publication time, news time point, main content, and number of comments. Among them, the main research content of this article is the comment subject.Considering that the climbed content has the problem of invalid characters, to avoid its impact on the prediction results, the 155 text files climbed were studied to remove invalid characters, deweight, text cleaning, Chinese segmentation, etc., retain the preprocessing operation of the comment subject content and comment time, and split them, so that each comment and its release time are kept in a separate txt folder. In addition, the collected comments were annotated, classifying comments as positive, neutral, and negative. Through the above operation, this article finally obtained 4,184 annotated experimental data, including 460 positive comments, 1,750 neutral comments, and 1,974 negative comments, which is in Table3.Table 3
Distribution of three classifications of real samples (unit:strip).
Negative classNeutral classPositive classTotalTest sample197417504604184


## 4.3. Parameter Setting

Different model parameters for this experiment are set up in Table4. In the table, lstm-size represents the number of LSTM units in the hidden layer; dropout represents the probability of random retention of the hidden layer neurons, epoch represents the number of iterations, and batch-size represents the number of samples selected per learning optimization of the network [23].Table 4
Parameter settings of different models.
ParameterCNNLSTMBILSTMWord vector dimension300 dimension300 dimension300 dimensionConvolutional kernel size[2–4]Number of filters64Number of feature layers300Lstm_size6464Dropout0.50.50.5Epoch252525Batch size202020


## 4.4. Evaluating Indicator

Based on the above comment classification, this paper constructs three-classification confusion matrix model evaluation metrics, meanwhile, precision, recall, andF1 values were used to evaluate the test results, specific computational method as the following formulas [24, 25].
(7)Precision+=TPTP+FP,(8)Recall+=TPTP+FN,(9)F1=2×Precion×RecallPrecion+Recall.In the formula, TP represents true positive examples; TN represents false positive examples; FP represents false positive examples.


## 4.5. Experimental Result

### 4.5.1. Testing Results of the Standard Dataset

In order to verify the prediction effect of the BILSTM model, the standard dataset training model was used in the study and predicted the three categories of the real sample dataset. At the same time comparing the prediction results with the CNN, LSTM model that also using standard data and training, and the confusion matrix of the three models [26]. The prediction results of different models according to the three-classification confusion matrix are shown in Figure 5. As shown in Figure 5, the prediction results obtained by the prediction models trained with standard data sets are not ideal for most of the comment data, and the overall prediction accuracy is only about 50%. The reason may be that the standard data set has certain differences from the real sample data set, leading to differences in feature extraction and specific words, which thus makes the model generalization ability of the training is poor and unable to accurately predict the network public opinion.Figure 5
Predicted results of different models on standard data sets.


### 4.5.2. Results of Real Sample Training Model

The real sample dataset was randomly divided into training set and test set as 1 : 2 ratio. The CNN, LSTM, and BILSTM classification models were trained by the training model, and the test set was input into the classification model where the training was completed, the confusion matrix of different model, and the prediction results of different models were calculated by the index evaluation method, such as in Figure5 [27]. As can be seen from Figure 6, the prediction model trained with the real sample dataset has a high prediction accuracy for most comment data, with an average prediction accuracy of over 76%. It has better predictive results compared to the classification models trained on the standard datasets.Figure 6
Predicted results of different models on real sample data sets.


### 4.5.3. Mixed Dataset Test Results

One-third of the data in the real sample dataset was randomly selected to the standard dataset to obtain the mixed dataset [28]. The CNN, LSTM, and BILSTM models were trained by using a mixed dataset and the classification model tested with the remaining two-thirds of the real sample set, getting the prediction outcome confusion, and computing the prediction results for different models, as in Figure 7. It is known from Figure 7 that compared to classification models trained on standard data or classification model on a real-sample single dataset, the classification model trained with a mixed dataset predicted better results, and the LSTM and BILSTM models achieved 80% prediction accuracy with good prediction accuracy.Figure 7
Predicted results of different models on mixed data sets.


### 4.5.4. Comparison and Analysis

In order to verify the emotional evaluation results of the three deep learning classification models under different training methods. In Table5, the first three data groups represent the classification prediction results of CNN, LSTM, and BILSTM models under the standard dataset; the middle three groups represent the CNN, LSTM, and BILSTM models under the real sample dataset; and the latter three groups represent the classification prediction results of CNN, LSTM, and BILSTM models under the mixed dataset. As shown in the table, a classification model trained on mixed datasets of partial real sample data was added, and it can effectively improve the discrimination and prediction accuracy of the emotion of online public opinion comments. Compared to the classification models trained on a single dataset, it has a better classification prediction effect. According to the first three groups, models were learned only by using standard datasets. It is difficult to accurately learn the network public opinion information and predict it, and then comparing the prediction results between the middle three groups and the latter three groups. Standard datasets improve the model to learn and extract features; moreover, it improves the accuracy of the model prediction. Overall, compared to the CNN and LSTM models, the BILSTM model performs the best on standard or real sample or mixed datasets.Table 5
Comparison of model classification and prediction results.
NegativeNeutralPositiveAccuracyPrecisionRecallF1PrecisionRecallF1PrecisionRecallF1Modle10.48300.99240.64980.64290.01540.03010.79070.14780.24910.4909Modle20.48450.98730.65000.64710.01890.03660.55450.13260.21400.4883Modle30.49100.97620.65340.62960.03890.07310.44080.14570.21900.4928Modle40.77460.84620.80880..86050.67700.75780.5840.84390.69030.7718Modle50.81760.80690.81220.83530.69880.76100.57240.9480.71380.7762Modle60.78700.830.80790.86780.63350.73230.53860.96820.69210.7602Modle70.74700.76770.75720.67440.78830.72690.93550.25140.39640.7158Modle80.91450.69150.78760.71340.93930.81090.9570.70810.8140.8021Modle90.88940.76080.82010.77440.87630.82220.81100.85550.83260.8226


### 4.5.5. Prediction Results and Analysis

To validate the predictive effect of the proposed model, the study divided the real sample data into training set and test set by time and used the training set as the original time series to construct ARIMA (p, 1, q), with the specific division cases presented in Table6 [16, 29].Table 6
Division of training set and test set.
Training intervalTest intervalPrediction pointExperiment 12019-08-05 to 2020-03-162020-04-19 to 2020-10-057Experiment 22019-08-05 to 2020-03-282020-03-16 to 2020-10-058Experiment 32019-08-05 to 2020-02-152020-02-16 to 2020-10-0510Experiment 42019-08-05 to 2020-01-302020-02-03 to 2020-10-0512The proposed model was used to predict the time series of the time training set to obtain the prediction data for each set of experimental prediction points, as shown in Figures8–11. It can be seen that with the accuracy value of each time node as the time series, the sequence meets the requirements of the ARIMA model through the stationarity and nonwhite noise test, which can be used for the trend analysis and prediction of network public opinion trends. According to the time-series map of the accuracy of the BILSTM prediction model, we can see the numerical trend of the accuracy, and conclude that with the increase of the event occurrence time, the accuracy of the prediction model has gradually decreased. The reason for the analysis may be that with the change of the network public opinion of the news events, public comment on the event has changed. Compared with experiment three and four, the prediction results of experiment one and two are better and basically coincide with the real accuracy data, indicating that the longer the forward prediction time of the proposed model, the greater the error is in line with the general time series prediction characteristics.Figure 8
Predicted results of experiment 1.Figure 9
Predicted results of experiment 2.Figure 10
Predicted results of experiment3.Figure 11
Predicted results of experiment 4.


## 4.5.1. Testing Results of the Standard Dataset

In order to verify the prediction effect of the BILSTM model, the standard dataset training model was used in the study and predicted the three categories of the real sample dataset. At the same time comparing the prediction results with the CNN, LSTM model that also using standard data and training, and the confusion matrix of the three models [26]. The prediction results of different models according to the three-classification confusion matrix are shown in Figure 5. As shown in Figure 5, the prediction results obtained by the prediction models trained with standard data sets are not ideal for most of the comment data, and the overall prediction accuracy is only about 50%. The reason may be that the standard data set has certain differences from the real sample data set, leading to differences in feature extraction and specific words, which thus makes the model generalization ability of the training is poor and unable to accurately predict the network public opinion.Figure 5
Predicted results of different models on standard data sets.


## 4.5.2. Results of Real Sample Training Model

The real sample dataset was randomly divided into training set and test set as 1 : 2 ratio. The CNN, LSTM, and BILSTM classification models were trained by the training model, and the test set was input into the classification model where the training was completed, the confusion matrix of different model, and the prediction results of different models were calculated by the index evaluation method, such as in Figure5 [27]. As can be seen from Figure 6, the prediction model trained with the real sample dataset has a high prediction accuracy for most comment data, with an average prediction accuracy of over 76%. It has better predictive results compared to the classification models trained on the standard datasets.Figure 6
Predicted results of different models on real sample data sets.


## 4.5.3. Mixed Dataset Test Results

One-third of the data in the real sample dataset was randomly selected to the standard dataset to obtain the mixed dataset [28]. The CNN, LSTM, and BILSTM models were trained by using a mixed dataset and the classification model tested with the remaining two-thirds of the real sample set, getting the prediction outcome confusion, and computing the prediction results for different models, as in Figure 7. It is known from Figure 7 that compared to classification models trained on standard data or classification model on a real-sample single dataset, the classification model trained with a mixed dataset predicted better results, and the LSTM and BILSTM models achieved 80% prediction accuracy with good prediction accuracy.Figure 7
Predicted results of different models on mixed data sets.


## 4.5.4. Comparison and Analysis

In order to verify the emotional evaluation results of the three deep learning classification models under different training methods. In Table5, the first three data groups represent the classification prediction results of CNN, LSTM, and BILSTM models under the standard dataset; the middle three groups represent the CNN, LSTM, and BILSTM models under the real sample dataset; and the latter three groups represent the classification prediction results of CNN, LSTM, and BILSTM models under the mixed dataset. As shown in the table, a classification model trained on mixed datasets of partial real sample data was added, and it can effectively improve the discrimination and prediction accuracy of the emotion of online public opinion comments. Compared to the classification models trained on a single dataset, it has a better classification prediction effect. According to the first three groups, models were learned only by using standard datasets. It is difficult to accurately learn the network public opinion information and predict it, and then comparing the prediction results between the middle three groups and the latter three groups. Standard datasets improve the model to learn and extract features; moreover, it improves the accuracy of the model prediction. Overall, compared to the CNN and LSTM models, the BILSTM model performs the best on standard or real sample or mixed datasets.Table 5
Comparison of model classification and prediction results.
NegativeNeutralPositiveAccuracyPrecisionRecallF1PrecisionRecallF1PrecisionRecallF1Modle10.48300.99240.64980.64290.01540.03010.79070.14780.24910.4909Modle20.48450.98730.65000.64710.01890.03660.55450.13260.21400.4883Modle30.49100.97620.65340.62960.03890.07310.44080.14570.21900.4928Modle40.77460.84620.80880..86050.67700.75780.5840.84390.69030.7718Modle50.81760.80690.81220.83530.69880.76100.57240.9480.71380.7762Modle60.78700.830.80790.86780.63350.73230.53860.96820.69210.7602Modle70.74700.76770.75720.67440.78830.72690.93550.25140.39640.7158Modle80.91450.69150.78760.71340.93930.81090.9570.70810.8140.8021Modle90.88940.76080.82010.77440.87630.82220.81100.85550.83260.8226


## 4.5.5. Prediction Results and Analysis

To validate the predictive effect of the proposed model, the study divided the real sample data into training set and test set by time and used the training set as the original time series to construct ARIMA (p, 1, q), with the specific division cases presented in Table6 [16, 29].Table 6
Division of training set and test set.
Training intervalTest intervalPrediction pointExperiment 12019-08-05 to 2020-03-162020-04-19 to 2020-10-057Experiment 22019-08-05 to 2020-03-282020-03-16 to 2020-10-058Experiment 32019-08-05 to 2020-02-152020-02-16 to 2020-10-0510Experiment 42019-08-05 to 2020-01-302020-02-03 to 2020-10-0512The proposed model was used to predict the time series of the time training set to obtain the prediction data for each set of experimental prediction points, as shown in Figures8–11. It can be seen that with the accuracy value of each time node as the time series, the sequence meets the requirements of the ARIMA model through the stationarity and nonwhite noise test, which can be used for the trend analysis and prediction of network public opinion trends. According to the time-series map of the accuracy of the BILSTM prediction model, we can see the numerical trend of the accuracy, and conclude that with the increase of the event occurrence time, the accuracy of the prediction model has gradually decreased. The reason for the analysis may be that with the change of the network public opinion of the news events, public comment on the event has changed. Compared with experiment three and four, the prediction results of experiment one and two are better and basically coincide with the real accuracy data, indicating that the longer the forward prediction time of the proposed model, the greater the error is in line with the general time series prediction characteristics.Figure 8
Predicted results of experiment 1.Figure 9
Predicted results of experiment 2.Figure 10
Predicted results of experiment3.Figure 11
Predicted results of experiment 4.


## 5. Conclusion

To sum up, the deep learning-based network public opinion analysis model proposed in this study can better realize the emotion prediction of a network public opinion event text by combining the ARIMA time series model. Compared with the CNN or LSTM model, it has a better prediction accuracy and can achieve a prediction accuracy of over 80%. The ARIMA model, constructed by taking the prediction accuracy value of the BILSTM model as the original time series, and performing difference processing and nonwhite noise test, can well capture the network public opinion, predict the general trend of public opinion development, and the prediction results can well reflect the actual network public opinion, and are in line with the general time series prediction characteristics. However, due to the limitation of research conditions, there are still some deficiencies to be further improved. For example, the deep learning emotion classification algorithm BILSTM algorithm used in the article can also further improve the model by adding the attention mechanism and other methods to improve the model prediction effect.



---

*Source: 8323083-2022-03-25.xml*