# Automatic Extraction of Flooding Control Knowledge from Rich Literature Texts Using Deep Learning
## 4 Discussion
### Analyzing the Deep Learning Method from the Perspective of Principle  
In this study, ERNIE exhibited the best performance based on evaluation indicators. In principle, Bert originates from the encoder part of the transformer [45] (as shown in Figure 4) and comprises several components, such as multi-head attention, skip connection, layer norm, and feed-forward network. Bert-CNN/Bert-RNN was used to replace self-attention with CNN/RNN. The main tasks of Bert were mask language model (MLM) and next sequence prediction (NSP). Bert considered a word piece to be the smallest unit of the mask that lost semantic information. ERNIE is an improved model based on Bert that mainly improves the mask mechanism. This enabled the model to learn the semantic representation of complete concepts through the mask of phrases, proper nouns, entities, and so on. Compared with the Bert model, ERNIE has added more high-quality corpora, making it superior for natural language process (NLP) tasks. In terms of computational efficiency, the order is ERNIE > Bert-CNN > Bert > Bert-RNN. The Bert-RNN model required the longest training time and had the lowest efficiency, which is related to the dependency structure of the RNN itself. In RNN model training, the calculation at time T depended on the calculation results of the hidden layer at time T \(-\) 1, whereas the calculation at time T \(-\) 1 depended on the calculation results of the hidden layer at time T \(-\) 2, etc., forming a sequence dependency. In other words, the results of the second step can be calculated only after the first step is completed. Therefore, it was difficult for the RNN to have efficient parallel computing capability. However, CNN and Bert did not depend on the input of different time steps in the intermediate state of the network. They have strong parallel computing ability because of their flexibility and freedom. Therefore, the training time of the Bert-RNN model is significantly longer. From the perspective of comprehensive feature extraction capability, the ranking was ERNIE > Bert-CNN > Bert > Bert-RNN. The input of NLP is generally linear sequence sentences of variable length. The transformer and the CNN set the maximum input length. If the actual length of a sentence was less than the maximum length, padding was used to convert the model input to a fixed length. CNN can efficiently extract local features and semantic information from text. It can achieve excellent results in short-text classification using a pretraining model. However, the feature extraction of this model is distance dependent, and the extraction of some important features beyond a certain text length become difficult. The transformer for long texts presents huge computational complexity, resulting in a sharp slowdown in the computational speed. An RNN is a network structure that can accept variable-length input and conduct linear information transmission from front to back, which is also the fundamental reason for its popularity in the NLP field. However, RNN are limited by their parallel-computing ability. The research object of this study was sentences in the article, which is a type of short text data. Therefore, the RNN was inferior to CNN and transformers in terms of performance and efficiency. [PERSON] et al. [46] ranked the comprehensive capabilities of the three feature extractors in the order transformer > CNN > RNN. [PERSON] et al. [47], in their study, carried out a survey on text classification from shallow to deep learning and they conducted model training based on the same dataset (such as SST-2, IMDB, MR, etc.). Their results showed that the Bert model performed better. The research in [48] also proved that the transformer has a far stronger feature extraction capability than LSTM (an improved RNN model), which is consistent with the conclusions of this study.