# XAI-Based Reinforcement Learning Approach for Text Summarization of Social IoT-Based Content

Authors: Hikmat A. M. Abdeljaber; Sultan Ahmad; Abdullah Alharbi; Sudhir Kumar
Journal: Security and Communication Networks
 (2022)

---

## Abstract

The purpose of automatic text summarising technology is to condense a given text while properly portraying the main information in the original text in a summary. To present generative text summarising approaches, on the other hand, restructure the original language and introduce new words when constructing summary sentences, which can easily lead to incoherence and poor readability. This research proposes a XAI (explainable artificial intelligence)-based Reinforcement Learning-based Text Summarization of Social IoT-Based Content using Reinforcement Learning. Furthermore, standard supervised training based on labelled data to improve the coherence of summary sentences has substantial data costs, which restricts practical applications. In order to do this, a ground-truth-dependent text summarization (generation) model (XAI-RL) is presented for coherence augmentation. On the one hand, based on the encoding result of the original text, a sentence extraction identifier is generated, and the screening process of the vital information of the original text is described. Following the establishment of the overall benefits of the two types of abstract writings, the self-judgment approach gradient assists the model in learning crucial sentence selection and decoding the selected key phrases, resulting in a summary text with high sentence coherence and good content quality. Experiments show that the proposed model's summary content index surpasses text summarising ways overall, even when there is no pre-annotated summary ground-truth; information redundancy, lexical originality, and abstract perplexity also outperform the current methods.


---

## Body

## 1. Introduction

With the rapid development of the Internet, the network contains a massive base and various forms of data content. Quickly locating critical information from it is a primary problem for efficient information retrieval. For text data, automatic summarization technology can extract the core content from a given corpus and describe the main text of the original text with a relatively summary text, which is conducive to reducing the storage cost of text data and is a necessary means to improve the efficiency of text data retrieval. It has important practical significance and application value for the further realization of information integration.Existing automatic text summarization methods can directly select basic sentences or paragraphs from the original text and generate abstract texts by sentence extraction [1]. Generative text summarization methods have become a research hotspot in text summarization [2]. Generally speaking, the generative text summarization method firstly encodes a given original text and obtains a vector (embedded) representation that can cover the original text information from the word and sentence levels. Finally, the above feature codes are decoded, that is, according to the decoding result, the corresponding vocabulary is selected from the given language (dictionary) to form an abstract text. Finally, the original text is re-expressed in text form. It can be seen that compared with the extraction method, the implementation of generative text summarization is more complex. Still, the summary text produced by it is more flexible and rich in vocabulary expression, and the effect of condensing the critical information of the original text is more ideal [3].However, the problem faced is that the generative text summarization method needs to go through the process of original text encoding, encoding parsing, and feature decoding, and organize sentences with richer vocabulary to convey the original text, so it is easy to lead to the coherence of the generated summary sentences. In addition, the current generative text summarization methods involve artificially-annotated summaries for supervised training [3], resulting in the existing productive text summarization methods often facing the. Due to a scarcity of valuable resources, only relying on the “summary truth value” with strong sentence coherence in advance, and improving the sentence coherence of the abstract generated by the model according to the supervised training method, may have significant resistance in the practical applications [4]. Therefore, based on the generative text summarization model, this paper seeks an effective mechanism that can improve the sentence coherence of the summary generation model without intervention.Specifically, on the one hand, in the summary text generation stage, the encoder (module A) first encodes the given source document (source document) to obtain the embedded representation of the original text; on this basis, the coherence measurement module (module B) The Transformer-XL [5] encoder is used further to encode the embedded representation of the original text, parse the context-related content features, and set a “key sentence classification layer” at the top of the coherence measurement module to generate sentence extraction identifiers, to filter out (or be the key) sentence coding results, to describe the process of extracting key sentences from the original text through the coherence measurement module; finally, the decoder (module C) is based on the critical sentence coding output by the coherence measurement module. And produces decoded results for the “extracted” key sentences, i.e., the original vocabulary distribution.On the other hand, in the sentence coherence enhancement stage, the model XAI-RL first obtains the original vocabulary distribution output by the previous stage decoder (module C) and generates two types of summary texts by “selecting by probability” and “selecting by Softmax-greedy,” and the two types of abstracts are re-encoded by the encoder (module A); after that, the re-encoding results of the two types of abstracts are parsed by the coherence measurement module (module B), and the re-encoding results of the two types of abstracts are parsed by the coherence measurement module (module B). The recurrent self-attention weight of the semantic segment (segment) [5] is used as the coherence benefit of the summary sentence; the ROUGE [6] score of the generated summary text and the “pseudo-summary truth value” is used as the content income of the summary sentence, so that the sum of the above two benefits is calculated through the coherence measurement module to calculate the respective overall benefits of the two types of summary texts; here, the “true value of the pseudo-summary” is the optimal sentence set extracted from the original text through the ROUGE score. Second, construct the “cross-entropy loss” of the two types of abstracts, adopt the “self-critical policy gradient” in XAI-Based Reinforcement Learning [7], and use the “overall return difference” of the two types of abstracts to reward or reward the model parameter gradient. Penalty is forcing the overall return of the summary generated by “Softmax-greedy selection” to approach the overall recovery of the summary generated by “selecting by probability,” and improving the general baseline level of “Softmax-greedy selection” through “probability exploration,” and then improve the model value of the summary text generated by XAI-RL in terms of sentence coherence and sentence content. Finally, without the intervention of the abstract truth value, the abstract text with high sentence coherence and good content quality is generated.To sum up, this paper proposes a coherence-enhancing-oriented truth-free text summarization model (XAI-RL), which combines “extraction and generation” to generate summary content based on the set of critical sentences extracted from the original text. At the same time, by re-encoding, coherence, and content revenue calculation of the initially generated summary text, based on the actual vocabulary distribution of the decoder, the “selected by probability” is obtained compared with “selected by Softmax-greedy.” Gain advantage” guides model gradient updates by maximizing this “profit advantage” to produce summary texts with higher sentence coherence. The experimental results show that the ROUGE [6] and METEOR [8] scoring indicators of the model XAI-RL are still better than the existing text summarization methods on the whole, even under the restriction of only given the original text. The summary texts also outperform existing methods in terms of sentence coherence, content importance, information redundancy, lexical novelty, and summary perplexity.


## 2. Related Work

Currently, the sequence-to-sequence (Seq2Seq) structure based on the idea of “encoding-decoding” is the primary method for dealing with generative text summarization tasks [3]. The encoder and decoder in the traditional Seq2Seq structure often use the recurrent neural network (RNN) [9], long short-term memory (LSTM) [10], and bi-directional long short-term term memory (Bi-LSTM) [11], to generate summary texts with better sentence quality, many scholars have made related improvements to the summary mentioned above and generation models based on recurrent neural networks and their variants.Author [12] proposed a hierarchical encoder that can capture the discourse structure of the input text from two levels of words and segments and injects the discourse structure features into the decoder to assist the decoder in generating summary texts. A high ROUGE score was achieved on the task of developing academic paper abstracts. Author [13] introduced an intra-decoder attention mechanism on the decoder side, that is, observing the first t − 1 bits of decoding results when decoding the t-th bit, the attention weight prevents the decoder from generating the duplicate content, which effectively reduces the redundancy of the summary text sentence content; at the same time, this work combines the Teacher Forcing algorithm [14] and the self-judgment policy gradient [7] to construct a hybrid XAI-Based Reinforcement Learning objective, which makes the model effectively avoid exposure bias when processing the original text, and generates summary text with high evaluation accuracy. Author [15] first divided the input original text into multiple segments, and based on Bi-LSTM, the model builds multiple agents; after that, each agent parses the allocated element, and transmits the parsing results of the part between the agents according to the multi-agent communication mechanism, and finally forms a “global observation” of the original text, which is defined by “Global observation” that generates summary text according to the “encoding-decoding” idea.Although the above models have achieved improvement in the accuracy of summary generation, the recurrent neural network and its variants are all time-step-based sequence structures, which seriously hinders the parallel training of the model [16–18], resulting in the inference process being limited by memory, resulting in reduced encoding and decoding speed of the summary generation model, and increased training overhead [19–23]. On the other hand, the above works optimize the model to maximize the ROUGE index or maximum likelihood without considering the coherence or fluency of the summary sentence [24–26] and relying on the ground-truth value of the annotated summary text in advance. With supervised training, the data cost involved in model training is high. Therefore, further improvements to the summary generation models based on recurrent neural networks and their variants are needed [27–29].The work on the coherence of summary sentences also includes: author optimize the model by encoding, decoding, and re-encoding the original text, constructing a summary similarity loss and a text reconstruction loss. A good language model calculates the negative log-likelihood of the generated summary text to measure sentence coherence; author used the BERTSCORE indicator to construct a distributed semantic gain and combined the payment with the self-judgment policy gradient to evaluate the model optimization. Human evaluation results show that this benefit can make model summaries more coherent; author optimize the extractor by applying an advantage actor-critic (A2C) at the sentence level after pretraining the decoder, to ensure that the model paraphrases the correct key sentences to generate coherent and fluent summaries [30, 31].The above models optimize summary coherence to minimize the perplexity of the generated summary text. However, it is worth noting that the existing works all use manual evaluation methods when evaluating the coherence of summary sentences. There is a lack of a mechanism or process for automatic measurement of sentence coherence within the summary generation model [32–34]. To sum up, the current generative text summarization methods should meet or solve the following problems: first, it can generate coherent and highly readable summary texts based on the given original text; the processing mechanism for automatic coherence measurement of the generated summary sentences; thirdly, the labeling dependence of the summary ground-truth data in the model training process should be minimized to reduce the model training cost [35–37].


## 3. XAI-RL Summary Generation Model

### 3.1. Overall Model Architecture

As shown in Figure1, the XAI-RL model is mainly divided into two stages: first, the summary text generation stage (① to ⑥ in Figure 1, marked by blue lines). First, the encoder (module A) uses the AL-BERT component to obtain the encoded representation Ea of the original text set D, and the coherence measurement module (module B, the top layer is the Sigmoid classification layer) obtains auxiliary information H and extracts the set of critical sentences Inputabs; here, the additional information H and the collection of essential sentences Inputabs are regarded as the feature analysis result of the encoded representation Ea; then, the decoder (module C) decodes Inputabs and H, and afterword search, the initial output for the content of the critical sentences is generated abstract text [38, 39]. It is worth noting that, as shown in Figure 1, when Inputabs and H are decoded by the decoder (module C) in the summary text generation stage, it is necessary to adopt “select by probability” and “select by Softmax-greedy” based on the original vocabulary distribution. Two strategies are used for vocabulary selection, resulting in summary texts under different selection strategies.Figure 1
Architecture of the XAI-RL model.Second is the sentence coherence reinforcement stage (⑦ to ⑫ in Figure1, marked by orange lines). First, the model XAI-RL resubmits the preliminary generated summary text in stage 1 (“selected by probability” or “selected by Softmax-greedy”) to the AL-BERT encoder (module A) for “summary re-encoding”; secondly, based on re-encode, the result to obtain the semantic segment-based recurrent self-attention weight embedded in the L-layer encoding component (Transformer-XL_Encoder) in the coherence measurement module (module B), which is used as the sentence coherence score of the summary text generated in stage 1, denoted as coherence benefits (rewardscoherence), thus introducing a sentence coherence measurement mechanism inside the model; in addition, calculating the ROUGE score of the summary text and pseudo-summary generated in stage 1 denoted as content revenue (rewards content); here, “ “Pseudo abstract” is composed of the top K sentences with the highest scores after calculating the ROUGE score of each sentence in the original text and the whole original text. Finally, the overall revenue generated by the model XAI-RL abstract (referred to as rewards) is composed of the sentence coherence benefit of the abstract text and the sentence content benefit of the abstract text. The general use (including both content and coherence) updates the model XAI-RL parameter gradient to guide the model to generate sentences with high coherence and good content quality without the intervention of artificially annotated ground-truth summaries (such as relying only on pseudo-summaries)—abstract text.


### 3.2. Stage 1: Summary Text Generation Stage

#### 3.2.1. Text Encoding Representation

In the summary generation stage shown in Figure2, the model XAI-RL utilizes the pretrained AL-BERT components as the encoder to obtain the encoded representation of the input text set E, denoted as Fa. Specifically, given n original texts, E = [E1; E2; …; En], where the i-th text and its i_m sentences are represented as Ei = [sent1i, sent2i, …, senti_mi], and the jth sentence sentji = [X1j, X2j, …, Xij_lenj] in the text E contains ij_len words; in addition, the model XAI-RL marks the [CLS] symbol before each sentence in the input text set E to distinguish different sentence. In particular, AL-BERT has fewer parameters and a faster encoding speed compared to BERT. Finally, after being processed by the AL-BERT encoder, the encoding of the n texts in the text set E is represented as F = [F1; F2; …; Fn]; among them, Fi = [[Fcls1i, Fsent1i], [Fcls2i, Fsent2i], …, [Fclsmi, Fsenti_mi]] is the coded representation of the i-th text; Fsentji = [FW1ji, FW2ji, …, FWl_mji], Fclsji are sentenced as the encoded representation of sentij and [CLS] symbols, respectively.Figure 2
Stage 1: the detailed process of summary generation of XAI-RL.


#### 3.2.2. Key Statement Selection and Auxiliary Information Acquisition

As shown in Figure2, in the summary text generation stage, the coherence measurement module (the top layer is the Sigmoid classification layer) is responsible for parsing the text-encoded representation Fa output by the AL-BERT encoder to extract contextual information H across semantic segments; in addition, the top-level Sigmoid classifier discriminates critical sentences from the context information G to generate the extraction label Labelext and then outputs the set of introductory sentences (encoded) inputs. In particular, G can provide corresponding context information for the collection of critical sentences Inputabs in the subsequent decoding process, thereby assisting the decoder to generate abstract text that summarizes the gist of the original text. Specifically, the coherence measurement module divides the encoded representation Fi of each text Di into i_u semantic segments of equal length (length l). For the τ + 1st segment in the text Di, the hidden state of the nth layer Transformer XL Encoder in the coherence measurement module is calculated according to formula:(1)gr+1∼n−1=RGmrn−1ogr+1n−1,pr+1n=Relative_MHAgr+1∼n−1,gr+1∧n−1=LayerNormpr+1n+gr+1n−1,gr+1n−1=Positionwise_FFNmr∧n−1,(2)Rr+1n,Lr+1n,Wr+1n=gr+1n−1XQnT,gr+1∼n−1XLnT,gr+1∼n−1XLnT,Br+1n=RelativeAttentionScoreRr+1n,Lr+1n,pr+1n=LinearSoftmaxBr+1nWr+1n.Among them, SG(∙) means stop gradient update during backpropagation [5]; Positionwise_FFN(∙) means position-wise feedforward network; Relative_MHA(∙) means adopting. The relative multi-head attention layer of relative position encoding [5] is used. Here, the hidden layer state h͂n − 1τ + 1 is processed according to formula (2); where Br+1n represents the nth layer Transformer - Self-attention weight for τ + 1st segment in XL Encoder. As a result, as shown in Figure 2, the hidden layer state output by the L-th Transformer-XL Encoder of the coherence measurement module is G = [G1; G2; …; Fn]; represents the final hidden layer state corresponding to the i_u sentence in the i-th text Di. Further, before the model XAI-RL adopts the AL-BERT component (module A) to encode the input text set D, the starting position of each sentence in D is marked with [CLS] symbols to distinguish from each other, and these symbols correspond to the hidden layer. A representation can characterize the statement following it [19]. Therefore, as shown in Figure 2, the coherence measurement module (module B) uses the hidden layer state of each [CLS] symbol in the secret layer state G to represent each sentence in the text set E after being passed by the L-layer coding and denoting is Gcls = [Gcls1; Gcls2; …; Gclsn]; here, Gcls = [Gcls1i, Fcls2i, …, Gcls1_mi] is the vector representation of i_m sentences in the i-th text Ei. Moreover, as shown in formulas (1) and (2), Gicls should also contain corresponding context information (such as R and V)(3)Labelext=sigmoidX1Gcls+c1,Inputabs=Fa⊕Labelext,(4)LossExt=−1/i_m,1/n∑i=1n∑k=1i_mzkInzk∧+1−zkIn1−zk∧.After that, as shown in Figure2, the Sigmoid classifier at the top level of the coherence measurement module uses the above text statement to represent Gcls to generate the extraction label Labelext = [Labelext1; Labelext2; …; Labelextn] to determine whether each sentence in the original text is critical; among them, Labelexti = [l1i, l2i, …, li_mi] represents the sentence extraction result of the i-th text Ei; lki ∈ {0, 1} Indicates whether the nth sentence in the text Di should be extracted (if it is 1, it means that the sentence is a crucial sentence and should be removed; otherwise, it is not considered). Therefore, as shown in formula (3), the original text encoding result Ea output by the AL-BERT encoder (module A) and the sentence extraction result Labelext output by the coherence metric module (module B) are multiplied bit by bit, and from Fa Filter out the set of critical sentences (encodings), denoted as Inputs. It is worth noting that the model XAI-RL uses the binary cross-entropy sentence extraction loss Lossext as shown in equation (4) to pretrain the coherence measurement module (module B) before the summary text generation stage begins. Specifically, Algorithm 1 is first used to extract pseudo-summaries from the given original text; in which the first line initializes the pseudo-summary set; starting from the second line, each textbook in the input text set E is processed separately; Line 6, respectively, calculate the ROUGE index between each sentence sj in the text Di and the remaining documents (i.e., Di \sj); Lines 7 to 11 indicate that the first k sentences with the most considerable ROUGE index value are taken as the pseudo-abstract Plabeli corresponding to the text Di; Line 12, the text Di corresponding to the pseudo-abstract Plabeli is added to the pseudo-summary set. Thus, as shown in equation (4), by comparing the extraction probability ŷk ∈ (0, 1) of the kth sentence in the i-th text in the coherence measurement module (output by the top-level Sigmoid classifier) and the pseudo-summary set. The kth sentence extraction result yk ∈ {0, 1} of the i-th pseudo-summary Plabeli is compared. Finally, without the intervention of the abstract truth value, the coherence measurement module is guided to obtain the relevant parameter information of key sentence extraction in advance [40].


## 3.1. Overall Model Architecture

As shown in Figure1, the XAI-RL model is mainly divided into two stages: first, the summary text generation stage (① to ⑥ in Figure 1, marked by blue lines). First, the encoder (module A) uses the AL-BERT component to obtain the encoded representation Ea of the original text set D, and the coherence measurement module (module B, the top layer is the Sigmoid classification layer) obtains auxiliary information H and extracts the set of critical sentences Inputabs; here, the additional information H and the collection of essential sentences Inputabs are regarded as the feature analysis result of the encoded representation Ea; then, the decoder (module C) decodes Inputabs and H, and afterword search, the initial output for the content of the critical sentences is generated abstract text [38, 39]. It is worth noting that, as shown in Figure 1, when Inputabs and H are decoded by the decoder (module C) in the summary text generation stage, it is necessary to adopt “select by probability” and “select by Softmax-greedy” based on the original vocabulary distribution. Two strategies are used for vocabulary selection, resulting in summary texts under different selection strategies.Figure 1
Architecture of the XAI-RL model.Second is the sentence coherence reinforcement stage (⑦ to ⑫ in Figure1, marked by orange lines). First, the model XAI-RL resubmits the preliminary generated summary text in stage 1 (“selected by probability” or “selected by Softmax-greedy”) to the AL-BERT encoder (module A) for “summary re-encoding”; secondly, based on re-encode, the result to obtain the semantic segment-based recurrent self-attention weight embedded in the L-layer encoding component (Transformer-XL_Encoder) in the coherence measurement module (module B), which is used as the sentence coherence score of the summary text generated in stage 1, denoted as coherence benefits (rewardscoherence), thus introducing a sentence coherence measurement mechanism inside the model; in addition, calculating the ROUGE score of the summary text and pseudo-summary generated in stage 1 denoted as content revenue (rewards content); here, “ “Pseudo abstract” is composed of the top K sentences with the highest scores after calculating the ROUGE score of each sentence in the original text and the whole original text. Finally, the overall revenue generated by the model XAI-RL abstract (referred to as rewards) is composed of the sentence coherence benefit of the abstract text and the sentence content benefit of the abstract text. The general use (including both content and coherence) updates the model XAI-RL parameter gradient to guide the model to generate sentences with high coherence and good content quality without the intervention of artificially annotated ground-truth summaries (such as relying only on pseudo-summaries)—abstract text.


## 3.2. Stage 1: Summary Text Generation Stage

### 3.2.1. Text Encoding Representation

In the summary generation stage shown in Figure2, the model XAI-RL utilizes the pretrained AL-BERT components as the encoder to obtain the encoded representation of the input text set E, denoted as Fa. Specifically, given n original texts, E = [E1; E2; …; En], where the i-th text and its i_m sentences are represented as Ei = [sent1i, sent2i, …, senti_mi], and the jth sentence sentji = [X1j, X2j, …, Xij_lenj] in the text E contains ij_len words; in addition, the model XAI-RL marks the [CLS] symbol before each sentence in the input text set E to distinguish different sentence. In particular, AL-BERT has fewer parameters and a faster encoding speed compared to BERT. Finally, after being processed by the AL-BERT encoder, the encoding of the n texts in the text set E is represented as F = [F1; F2; …; Fn]; among them, Fi = [[Fcls1i, Fsent1i], [Fcls2i, Fsent2i], …, [Fclsmi, Fsenti_mi]] is the coded representation of the i-th text; Fsentji = [FW1ji, FW2ji, …, FWl_mji], Fclsji are sentenced as the encoded representation of sentij and [CLS] symbols, respectively.Figure 2
Stage 1: the detailed process of summary generation of XAI-RL.


### 3.2.2. Key Statement Selection and Auxiliary Information Acquisition

As shown in Figure2, in the summary text generation stage, the coherence measurement module (the top layer is the Sigmoid classification layer) is responsible for parsing the text-encoded representation Fa output by the AL-BERT encoder to extract contextual information H across semantic segments; in addition, the top-level Sigmoid classifier discriminates critical sentences from the context information G to generate the extraction label Labelext and then outputs the set of introductory sentences (encoded) inputs. In particular, G can provide corresponding context information for the collection of critical sentences Inputabs in the subsequent decoding process, thereby assisting the decoder to generate abstract text that summarizes the gist of the original text. Specifically, the coherence measurement module divides the encoded representation Fi of each text Di into i_u semantic segments of equal length (length l). For the τ + 1st segment in the text Di, the hidden state of the nth layer Transformer XL Encoder in the coherence measurement module is calculated according to formula:(1)gr+1∼n−1=RGmrn−1ogr+1n−1,pr+1n=Relative_MHAgr+1∼n−1,gr+1∧n−1=LayerNormpr+1n+gr+1n−1,gr+1n−1=Positionwise_FFNmr∧n−1,(2)Rr+1n,Lr+1n,Wr+1n=gr+1n−1XQnT,gr+1∼n−1XLnT,gr+1∼n−1XLnT,Br+1n=RelativeAttentionScoreRr+1n,Lr+1n,pr+1n=LinearSoftmaxBr+1nWr+1n.Among them, SG(∙) means stop gradient update during backpropagation [5]; Positionwise_FFN(∙) means position-wise feedforward network; Relative_MHA(∙) means adopting. The relative multi-head attention layer of relative position encoding [5] is used. Here, the hidden layer state h͂n − 1τ + 1 is processed according to formula (2); where Br+1n represents the nth layer Transformer - Self-attention weight for τ + 1st segment in XL Encoder. As a result, as shown in Figure 2, the hidden layer state output by the L-th Transformer-XL Encoder of the coherence measurement module is G = [G1; G2; …; Fn]; represents the final hidden layer state corresponding to the i_u sentence in the i-th text Di. Further, before the model XAI-RL adopts the AL-BERT component (module A) to encode the input text set D, the starting position of each sentence in D is marked with [CLS] symbols to distinguish from each other, and these symbols correspond to the hidden layer. A representation can characterize the statement following it [19]. Therefore, as shown in Figure 2, the coherence measurement module (module B) uses the hidden layer state of each [CLS] symbol in the secret layer state G to represent each sentence in the text set E after being passed by the L-layer coding and denoting is Gcls = [Gcls1; Gcls2; …; Gclsn]; here, Gcls = [Gcls1i, Fcls2i, …, Gcls1_mi] is the vector representation of i_m sentences in the i-th text Ei. Moreover, as shown in formulas (1) and (2), Gicls should also contain corresponding context information (such as R and V)(3)Labelext=sigmoidX1Gcls+c1,Inputabs=Fa⊕Labelext,(4)LossExt=−1/i_m,1/n∑i=1n∑k=1i_mzkInzk∧+1−zkIn1−zk∧.After that, as shown in Figure2, the Sigmoid classifier at the top level of the coherence measurement module uses the above text statement to represent Gcls to generate the extraction label Labelext = [Labelext1; Labelext2; …; Labelextn] to determine whether each sentence in the original text is critical; among them, Labelexti = [l1i, l2i, …, li_mi] represents the sentence extraction result of the i-th text Ei; lki ∈ {0, 1} Indicates whether the nth sentence in the text Di should be extracted (if it is 1, it means that the sentence is a crucial sentence and should be removed; otherwise, it is not considered). Therefore, as shown in formula (3), the original text encoding result Ea output by the AL-BERT encoder (module A) and the sentence extraction result Labelext output by the coherence metric module (module B) are multiplied bit by bit, and from Fa Filter out the set of critical sentences (encodings), denoted as Inputs. It is worth noting that the model XAI-RL uses the binary cross-entropy sentence extraction loss Lossext as shown in equation (4) to pretrain the coherence measurement module (module B) before the summary text generation stage begins. Specifically, Algorithm 1 is first used to extract pseudo-summaries from the given original text; in which the first line initializes the pseudo-summary set; starting from the second line, each textbook in the input text set E is processed separately; Line 6, respectively, calculate the ROUGE index between each sentence sj in the text Di and the remaining documents (i.e., Di \sj); Lines 7 to 11 indicate that the first k sentences with the most considerable ROUGE index value are taken as the pseudo-abstract Plabeli corresponding to the text Di; Line 12, the text Di corresponding to the pseudo-abstract Plabeli is added to the pseudo-summary set. Thus, as shown in equation (4), by comparing the extraction probability ŷk ∈ (0, 1) of the kth sentence in the i-th text in the coherence measurement module (output by the top-level Sigmoid classifier) and the pseudo-summary set. The kth sentence extraction result yk ∈ {0, 1} of the i-th pseudo-summary Plabeli is compared. Finally, without the intervention of the abstract truth value, the coherence measurement module is guided to obtain the relevant parameter information of key sentence extraction in advance [40].


## 3.2.1. Text Encoding Representation

In the summary generation stage shown in Figure2, the model XAI-RL utilizes the pretrained AL-BERT components as the encoder to obtain the encoded representation of the input text set E, denoted as Fa. Specifically, given n original texts, E = [E1; E2; …; En], where the i-th text and its i_m sentences are represented as Ei = [sent1i, sent2i, …, senti_mi], and the jth sentence sentji = [X1j, X2j, …, Xij_lenj] in the text E contains ij_len words; in addition, the model XAI-RL marks the [CLS] symbol before each sentence in the input text set E to distinguish different sentence. In particular, AL-BERT has fewer parameters and a faster encoding speed compared to BERT. Finally, after being processed by the AL-BERT encoder, the encoding of the n texts in the text set E is represented as F = [F1; F2; …; Fn]; among them, Fi = [[Fcls1i, Fsent1i], [Fcls2i, Fsent2i], …, [Fclsmi, Fsenti_mi]] is the coded representation of the i-th text; Fsentji = [FW1ji, FW2ji, …, FWl_mji], Fclsji are sentenced as the encoded representation of sentij and [CLS] symbols, respectively.Figure 2
Stage 1: the detailed process of summary generation of XAI-RL.


## 3.2.2. Key Statement Selection and Auxiliary Information Acquisition

As shown in Figure2, in the summary text generation stage, the coherence measurement module (the top layer is the Sigmoid classification layer) is responsible for parsing the text-encoded representation Fa output by the AL-BERT encoder to extract contextual information H across semantic segments; in addition, the top-level Sigmoid classifier discriminates critical sentences from the context information G to generate the extraction label Labelext and then outputs the set of introductory sentences (encoded) inputs. In particular, G can provide corresponding context information for the collection of critical sentences Inputabs in the subsequent decoding process, thereby assisting the decoder to generate abstract text that summarizes the gist of the original text. Specifically, the coherence measurement module divides the encoded representation Fi of each text Di into i_u semantic segments of equal length (length l). For the τ + 1st segment in the text Di, the hidden state of the nth layer Transformer XL Encoder in the coherence measurement module is calculated according to formula:(1)gr+1∼n−1=RGmrn−1ogr+1n−1,pr+1n=Relative_MHAgr+1∼n−1,gr+1∧n−1=LayerNormpr+1n+gr+1n−1,gr+1n−1=Positionwise_FFNmr∧n−1,(2)Rr+1n,Lr+1n,Wr+1n=gr+1n−1XQnT,gr+1∼n−1XLnT,gr+1∼n−1XLnT,Br+1n=RelativeAttentionScoreRr+1n,Lr+1n,pr+1n=LinearSoftmaxBr+1nWr+1n.Among them, SG(∙) means stop gradient update during backpropagation [5]; Positionwise_FFN(∙) means position-wise feedforward network; Relative_MHA(∙) means adopting. The relative multi-head attention layer of relative position encoding [5] is used. Here, the hidden layer state h͂n − 1τ + 1 is processed according to formula (2); where Br+1n represents the nth layer Transformer - Self-attention weight for τ + 1st segment in XL Encoder. As a result, as shown in Figure 2, the hidden layer state output by the L-th Transformer-XL Encoder of the coherence measurement module is G = [G1; G2; …; Fn]; represents the final hidden layer state corresponding to the i_u sentence in the i-th text Di. Further, before the model XAI-RL adopts the AL-BERT component (module A) to encode the input text set D, the starting position of each sentence in D is marked with [CLS] symbols to distinguish from each other, and these symbols correspond to the hidden layer. A representation can characterize the statement following it [19]. Therefore, as shown in Figure 2, the coherence measurement module (module B) uses the hidden layer state of each [CLS] symbol in the secret layer state G to represent each sentence in the text set E after being passed by the L-layer coding and denoting is Gcls = [Gcls1; Gcls2; …; Gclsn]; here, Gcls = [Gcls1i, Fcls2i, …, Gcls1_mi] is the vector representation of i_m sentences in the i-th text Ei. Moreover, as shown in formulas (1) and (2), Gicls should also contain corresponding context information (such as R and V)(3)Labelext=sigmoidX1Gcls+c1,Inputabs=Fa⊕Labelext,(4)LossExt=−1/i_m,1/n∑i=1n∑k=1i_mzkInzk∧+1−zkIn1−zk∧.After that, as shown in Figure2, the Sigmoid classifier at the top level of the coherence measurement module uses the above text statement to represent Gcls to generate the extraction label Labelext = [Labelext1; Labelext2; …; Labelextn] to determine whether each sentence in the original text is critical; among them, Labelexti = [l1i, l2i, …, li_mi] represents the sentence extraction result of the i-th text Ei; lki ∈ {0, 1} Indicates whether the nth sentence in the text Di should be extracted (if it is 1, it means that the sentence is a crucial sentence and should be removed; otherwise, it is not considered). Therefore, as shown in formula (3), the original text encoding result Ea output by the AL-BERT encoder (module A) and the sentence extraction result Labelext output by the coherence metric module (module B) are multiplied bit by bit, and from Fa Filter out the set of critical sentences (encodings), denoted as Inputs. It is worth noting that the model XAI-RL uses the binary cross-entropy sentence extraction loss Lossext as shown in equation (4) to pretrain the coherence measurement module (module B) before the summary text generation stage begins. Specifically, Algorithm 1 is first used to extract pseudo-summaries from the given original text; in which the first line initializes the pseudo-summary set; starting from the second line, each textbook in the input text set E is processed separately; Line 6, respectively, calculate the ROUGE index between each sentence sj in the text Di and the remaining documents (i.e., Di \sj); Lines 7 to 11 indicate that the first k sentences with the most considerable ROUGE index value are taken as the pseudo-abstract Plabeli corresponding to the text Di; Line 12, the text Di corresponding to the pseudo-abstract Plabeli is added to the pseudo-summary set. Thus, as shown in equation (4), by comparing the extraction probability ŷk ∈ (0, 1) of the kth sentence in the i-th text in the coherence measurement module (output by the top-level Sigmoid classifier) and the pseudo-summary set. The kth sentence extraction result yk ∈ {0, 1} of the i-th pseudo-summary Plabeli is compared. Finally, without the intervention of the abstract truth value, the coherence measurement module is guided to obtain the relevant parameter information of key sentence extraction in advance [40].


## 4. Experimental Results and Analysis

This chapter conducts a series of experimental analyses on the ground-truth-dependent text summarization model (XAI-RL) proposed in this paper for coherence enhancement and discusses the model’s effectiveness in the summary generation process and summary generation quality. This paper uses Python 3.7 and Tensorflow-1.15 to implement the model, and the experimental running environment is GPU, NVIDIA GeForceGTX 1080Ti, 11 GB.


### 4.1. Dataset and Experimental Setup

First of all, this paper uses two typical automatic text summarization datasets, CNN/Daily Mail and XSum, for experiments [41, 42]. They both use news reports as text data and contain corresponding “gold standard” summarization ground-truth documents. In this paper, the original data set is divided into a training set, validation set, and test set. The training set is used for model training, the validation set is used for model parameter selection, and the test set is used for model evaluation. In particular, the “gold standard” summaries do not participate in the model XAI-RL training process and are only used for summarization generation quality assessment. As shown in Table 1, the average length of the original text and abstract text in CNN/Daily Mail is larger than XSum; XSum is a sentence written by humans as the notional truth value. Compared with CNN/DailyMail, the novelty of the ground truth in XSum is higher and contains more words that do not appear in the original text.Table 1
Statistical information of CNN/daily mail and XSum datasets.
Data setNumber of documentsAverage document lengthNew in the gold standard summaryTrainVerifyTestNumber of wordsSentence lengthThe proportion of binary groups that appear/%CNN1026613201193860.543.9862.9Daily mail20696114148123977653.3339.3362.16XSum224045233215334654.729.7793.31Secondly, in terms of model setting, let the word vector dimension beE, the number of hidden layer units be H, the number of self-attention heads be A, the feedforward layer dimension size is F, and the XAI-RL model adopts AL-BERTlarge [30] (E = 128, H = 1.024, A = 16, F = 4.096) as the encoder, and the coherence metric module consists of L = 3 layers Transformer-XL Encoder (E = 1.024, H = 2.048, A = 32, F = 4.096), and the decoder consists of R = 6 Transformer-XL Decoder (E = 1.024, H = 2.048, A = 32, F = 4.096). In the summary text generation stage, a beam search algorithm with a width of 4 is used for vocabulary selection. The maximum length of the generated summary is determined by the average compression ratio of the original document and the summary document in the dataset (the ratio of document length), and the number of discarded words is low. In the sentence at 3, the coherence metric module and the decoder adopt the Adam optimizer [19] with a learning rate of 1E-3 and 0.05, respectively. The learning rates of both decreases with the number of iterations. The number of batch samples (batch_size, which is the size of the input text set D) is 16. In the sentence coherence enhancement stage, β1 = 0.3 and β2 = 0.2 are taken from the text content income shown in equation (6), and γ = 0.7 in the total income of equation (6). When the model is trained with the CNN/Daily Mail dataset, the input text set D takes the first M = 8 optimal records in one iteration for “experience playback” during the coherence enhancement stage; when training with XSum, D is in one iteration, the first M = 4 optimal records are taken.(5)ROUGE−N=∑T∈ST∑grams∈TCountmatchgramn∑T∈ST∑grams∈TCountgramn,ROUGE−M=1+β2SlcsQlcsSlcs+β2Qlcs,Slcs=LCSX,Ym,Qlcs=LCSX,Yn,(6)Qen=γ.dhmβ,Gmean=m/r∗m/dαm/d+1−αm/r,METEOR=1−Qen∗Gmean.Then, in terms of comparison methods, the abstract generation model XAI-RL proposed in this paper is compared with the existing extractive and generative automatic summarization methods. Among them, for the extraction method, MMS_Text, SummaRuNNer and HSSAS are used; for the generative method, Pointer-Generator + Coverage, Bottom-up, DCA (deep communicating agents), BERTSUMEXTABS, and PEGASUS.Finally, for the evaluation indicators, this paper adopts ROUGE-N [6] (including ROUGE-1 and ROUGE-2, formula (12)), ROUGE-L (formula (13)) and METEOR [8] (formula (14)). The index evaluates the content quality of the generated text, and at the same time cooperates with manual evaluation to assess the summary text generated by the related model in terms of sentence coherence, content redundancy, and content importance. Here, in ROUGE-N, n represents the length of n-gram (n-gram), {RS} means the reference abstract, Countmatch(grain) represents the same number of n-grams in the generated abstract as in the reference abstract, and Count(grain) is the total number of n-grams in the reference abstract. In ROUGE-L, X is the generated abstract, Y is the reference abstract, LCS(X, Y) represents the length of the longest common subsequence between the generated abstract and the reference abstract, m is the developed abstract length, and n is the reference abstract length; in METEOR, m is the number of tuples matching the reference abstract in the generated abstract, r is the connection abstract length, c is the developed abstract length, α, γ, β are balance parameters, dh is the generated conceptual and the reference abstract of the number of common subsequence’s.


### 4.2. XAI-RL Model Summary Generation Process Discussion

To explore the influence of different modules in the model XAI-RL on the experimental results, six ablative combinations, as shown in Table2, are implemented in this paper. Specifically, Combination 1 uses module A (AL-BERT encoder) and module B (coherence metric module, only Transformer-XL Encoder) without a replaceable top layer for encoding and then uses module C (decoder) for encoding and decoding to produce digest. Combination 2 adds a sigmoid classification layer to module B based on combination 1, which aims to perform critical sentence selection on text-encoded representations and then generate summaries. Combination 3 has the same structure as Combination 2, but it pretrains module B; in particular, the above three combinations are all supervised training using the training set “gold standard” as the ground truth. Finally, combination 4 adopts the structure of Combination 3. In addition to pretraining module B, coherence enhancement is only performed by maximizing the coherence gain. During the reinforcement process, the extracted pseudo-summary is used as an alternative truth value; Combination 5 is similar to Combination 4 in that it only enhances the coherence by maximizing the content revenue; Combination 6 is the complete XAI-RL model in Figure 1, and the extracted pseudo-summary is still used as the alternative truth value.Table 2
Ablation combinations of XAI-RL corresponding to Figure1.
CombinationPseudo abstract extractionModule AModule B (coding)Module B (coding + key sentence extraction)Module B (pretraining + coding + key sentence extraction)Module CContinuity enhancement (continuity benefit)Coherence enhancement (content gain)1√√√2√√√√3√√√√√4√√√√5√√√√√6√√√√√√CNN/Daily Mail and XSum validation sets evaluated the above six ablative combinations. The experimental results are shown in Tables3 and 4 and Figures 3 and 4.Table 3
Evaluation results of ablation combinations on CNN/daily mail dataset %.
CombinationROUGE-1ROUGE-2ROUGE-LROUGE-AVGMETEOR146.8526.0844.8739.2718.62247.7827.8246.0540.5518.7349.7428.2747.8541.9518.92451.6429.1349.3333.3719.7552.0129.2849.7133.6729.48653.2930.5550.1334.6630.51Table 4
Evaluation results of ablation combinations on XSum dataset %.
CombinationROUGE-1ROUGE-2ROUGE-LROUGE-AVGMETEOR145.4624.340.7136.8226.65247.1427.8141.6538.8726.94348.9226.8242.7139.4827.05449.5328.6543.3840.5228.37549.8728.2843.940.6828.46651.9728.2343.8441.3528.86Figure 3
Evaluation results of ablation combinations on CNN/daily mail dataset.Figure 4
Evaluation results of ablation combinations on XSum dataset.First, the evaluation results of Combination 2 are better than Combination 1, which indicates that after extracting key sentences by module B, the decoder can decode the critical content and generate higher-quality summaries. Second, combination 3 is better than combination 2, indicating that pretraining can make the parameter configuration of module B more reasonable and then select critical sentences more reasonably. Then, the evaluation results of combination four and combination 5 are better than those of combination 3, indicating that the revenue and sentence coherence enhancement method constructed in this paper can effectively improve the quality of abstract content. In particular, the ROUGE-L and METEOR indicators of combination 4 are better than those of combination 3, reflecting the improvement of sentence coherence by the coherence measurement and reinforcement in this paper. Finally, the combination of all the mechanisms 6 has the best evaluation result, reflecting the effectiveness of each module of the proposed model XAI-RL in a summary generation.To sum up, for the XAI-RL model, firstly, by comparing combination two and combination 3, it can be found that after pretraining the coherence measurement module by the pseudo-summary, meaningful sentences and contextual semantic information can be better identified from the text encoding representation, to provide the decoder with semantic benchmarks and auxiliary input to generate summary content that can accurately summarize the main idea of ​​the original text; secondly, by comparing combination three and combination 4, it can be found that the self.


### 4.3. Comparison between the XAI-RL Model and Existing Text Summarization Models

In this section, the accuracy of the XAI-RL model is compared with the existing extractive and generative methods on the test set to evaluate its summary generation quality. First, the evaluation results (average of 3 times) of the XAI-RL model and the comparison method on the CNN/Daily Mail dataset are shown in Table5 (ROUGE-AVG is the average of ROUGE-1, ROUGE-2, and ROUGE-L). On the one hand, as shown in Figure 5, the evaluation results of the XAI-RL model are generally better than the existing extraction methods. This model outperforms other extraction-based baseline models in ROUGE-1 and ROUGE-2 indicators, indicating that it can effectively obtain the original subject information.Table 5
Evaluation results of generated summarization on CNN/daily mail dataset %.
How the abstract is generatedModel methodROUGE-1ROUGE-2ROUGE-LROUGE-AVGMETEORExtractionMMS_Text47.5725.7244.4239.2426.97SummaRuNNer48.625.244.339.3726.75Refresh49.2727.245.640.6927.38HSSAS51.326.846.641.5728.27GenerativePointer-generator + coverage48.5326.2845.3840.0627.7Bottom-up50.2227.6847.3441.7528.38DCA50.6928.4746.9242.0328.55SupervisedBERTSUMEXTABS51.1328.648.1842.6428.91PEGASUSBASE50.7927.8147.9342.1828.63XAI-RL4 (Table2 combination 4)51.6429.1349.3343.3729.7XAI-RL5 (Table2 combination 5)52.0129.2849.7143.6729.48XAI-RL6 (Table2 combination 6)53.2930.5540.1344.6620.51Figure 5
Experimental results on CNN/daily mail dataset corresponding to Table5.At the same time, its scores on ROUGE-L and METEOR indicators are higher than other extraction-based baseline models which indicate that the model can ensure the coherence of generated sentences when paraphrasing the acquired vital sentences. The core ideas of the compared extraction methods (MMS_Text, SummaRuNNer, Refresh, and HSSAS) can be summarized into three categories: one is to convert text into a graph structure (such as MMS_Text), by scoring nodes (sentences) to extract essential sentences to form summary texts; the second is to mine the latent features of the reader through the encoder and extract summary sentences in the order of probability matrix or sentence arrangement (such as SummaRuNNer and HSSAS). The third is to use XAI-Based Reinforcement Learning to build quality gains, and after updating the sentence selection strategy to maximize profits, extract abstract texts from the original document (such as Refresh). However, for the XAI-RL model proposed in this paper, its core idea is “extract first, then generate.” The coherence measurement module of the model can identify and extract critical sentences in the original text after pretraining, thus prompting the decoder to pay attention to the actual content. In addition, when decoding and generating, the XAI-RL model outputs auxiliary information H containing contextual semantics to the decoder, which further enriches the text feature information inside the model, and finally makes the quality of the abstract text generated by the model XAI-RL better than the “single” extraction model. On the other hand, as shown in Figure6(b), the XAI-RL model is compared with existing generative methods (Pointer-Generator + Coverage [11], Bottom-up, DCA [15], BERTSUMEXTABS [19], and PEGASUS) and also achieved better accuracy overall. The model outperforms other generative baseline models in ROUGE-1 and ROUGE-2 indicators, indicating that it can correctly paraphrase the original text message. At the same time, its scores on both ROUGE-L and METEOR metrics are higher than other generative baseline model, which indicates that this model is more capable of generating coherent and smooth summary content. Its performance improvement can be attributed to: first, in developing the abstract, as shown in Figure 2, the model XAI-RL is based on the pretraining components (such as the AL-BERT encoder and the pretraining coherence measurement module), and the text encoding results are further used. The semantic segment is the division unit and is additionally encoded by the L = 3-layer Transformer-XL component through the semantic segment-based recurrent self-attention mechanism to increase the strength of feature parsing. Second, in the coherence enhancement process shown in Figure 4, the model XAI-RL re-encodes the generated summary text to calculate the coherence gain; at the same time, the extracted pseudo-summary is used to create the summary to calculate the content gain by maximizing the two. The weighted sum of revenue strengthens the model summary text generation process. Further, it improves the model text generation quality from the content and sentence coherence levels.Figure 6
Experimental results on XSum dataset corresponding to Table6.Secondly, the evaluation results of the XAI-RL model and the compared method on the XSum dataset (average of 3 times) are shown in Table6, and the corresponding histogram is shown in Figure 6. Overall, the model still achieves optimal results. In particular, the XSum dataset is only used to test generative methods due to its high novelty corresponding to the “gold standard” abstract. The results shown in Table 6 and Figure 6 further illustrate the “extract first, then generate” design principle followed by the model XAI-RL, the recurrent self-attention weight based on semantic segment, and the reinforcement process based on content benefit and coherence benefit can effectively improve the summary generation quality.Table 6
Evaluation results of generated summarization on XSum dataset %.
How the abstract is generatedModel methodROUGE-1ROUGE-2ROUGE-LROUGE-AVGMETEORSupervisedPointer-generator + coverage37.18.0230.7218.2820.31Bottom-up39.029.4532.9620.1421.09BERTSUMEXTABS47.8125.540.2727.8624.14PEGASUSBASE48.7925.5840.728.3624.82UnsupervisedXAI-RL4 (Table2 combination 4)49.5328.6543.3830.5228.37XAI-RL5 (Table2 combination 5)49.8728.2843.930.6828.46XAI-RL6 (Table2 combination 6)51.9728.2343.8431.3528.86


## 4.1. Dataset and Experimental Setup

First of all, this paper uses two typical automatic text summarization datasets, CNN/Daily Mail and XSum, for experiments [41, 42]. They both use news reports as text data and contain corresponding “gold standard” summarization ground-truth documents. In this paper, the original data set is divided into a training set, validation set, and test set. The training set is used for model training, the validation set is used for model parameter selection, and the test set is used for model evaluation. In particular, the “gold standard” summaries do not participate in the model XAI-RL training process and are only used for summarization generation quality assessment. As shown in Table 1, the average length of the original text and abstract text in CNN/Daily Mail is larger than XSum; XSum is a sentence written by humans as the notional truth value. Compared with CNN/DailyMail, the novelty of the ground truth in XSum is higher and contains more words that do not appear in the original text.Table 1
Statistical information of CNN/daily mail and XSum datasets.
Data setNumber of documentsAverage document lengthNew in the gold standard summaryTrainVerifyTestNumber of wordsSentence lengthThe proportion of binary groups that appear/%CNN1026613201193860.543.9862.9Daily mail20696114148123977653.3339.3362.16XSum224045233215334654.729.7793.31Secondly, in terms of model setting, let the word vector dimension beE, the number of hidden layer units be H, the number of self-attention heads be A, the feedforward layer dimension size is F, and the XAI-RL model adopts AL-BERTlarge [30] (E = 128, H = 1.024, A = 16, F = 4.096) as the encoder, and the coherence metric module consists of L = 3 layers Transformer-XL Encoder (E = 1.024, H = 2.048, A = 32, F = 4.096), and the decoder consists of R = 6 Transformer-XL Decoder (E = 1.024, H = 2.048, A = 32, F = 4.096). In the summary text generation stage, a beam search algorithm with a width of 4 is used for vocabulary selection. The maximum length of the generated summary is determined by the average compression ratio of the original document and the summary document in the dataset (the ratio of document length), and the number of discarded words is low. In the sentence at 3, the coherence metric module and the decoder adopt the Adam optimizer [19] with a learning rate of 1E-3 and 0.05, respectively. The learning rates of both decreases with the number of iterations. The number of batch samples (batch_size, which is the size of the input text set D) is 16. In the sentence coherence enhancement stage, β1 = 0.3 and β2 = 0.2 are taken from the text content income shown in equation (6), and γ = 0.7 in the total income of equation (6). When the model is trained with the CNN/Daily Mail dataset, the input text set D takes the first M = 8 optimal records in one iteration for “experience playback” during the coherence enhancement stage; when training with XSum, D is in one iteration, the first M = 4 optimal records are taken.(5)ROUGE−N=∑T∈ST∑grams∈TCountmatchgramn∑T∈ST∑grams∈TCountgramn,ROUGE−M=1+β2SlcsQlcsSlcs+β2Qlcs,Slcs=LCSX,Ym,Qlcs=LCSX,Yn,(6)Qen=γ.dhmβ,Gmean=m/r∗m/dαm/d+1−αm/r,METEOR=1−Qen∗Gmean.Then, in terms of comparison methods, the abstract generation model XAI-RL proposed in this paper is compared with the existing extractive and generative automatic summarization methods. Among them, for the extraction method, MMS_Text, SummaRuNNer and HSSAS are used; for the generative method, Pointer-Generator + Coverage, Bottom-up, DCA (deep communicating agents), BERTSUMEXTABS, and PEGASUS.Finally, for the evaluation indicators, this paper adopts ROUGE-N [6] (including ROUGE-1 and ROUGE-2, formula (12)), ROUGE-L (formula (13)) and METEOR [8] (formula (14)). The index evaluates the content quality of the generated text, and at the same time cooperates with manual evaluation to assess the summary text generated by the related model in terms of sentence coherence, content redundancy, and content importance. Here, in ROUGE-N, n represents the length of n-gram (n-gram), {RS} means the reference abstract, Countmatch(grain) represents the same number of n-grams in the generated abstract as in the reference abstract, and Count(grain) is the total number of n-grams in the reference abstract. In ROUGE-L, X is the generated abstract, Y is the reference abstract, LCS(X, Y) represents the length of the longest common subsequence between the generated abstract and the reference abstract, m is the developed abstract length, and n is the reference abstract length; in METEOR, m is the number of tuples matching the reference abstract in the generated abstract, r is the connection abstract length, c is the developed abstract length, α, γ, β are balance parameters, dh is the generated conceptual and the reference abstract of the number of common subsequence’s.


## 4.2. XAI-RL Model Summary Generation Process Discussion

To explore the influence of different modules in the model XAI-RL on the experimental results, six ablative combinations, as shown in Table2, are implemented in this paper. Specifically, Combination 1 uses module A (AL-BERT encoder) and module B (coherence metric module, only Transformer-XL Encoder) without a replaceable top layer for encoding and then uses module C (decoder) for encoding and decoding to produce digest. Combination 2 adds a sigmoid classification layer to module B based on combination 1, which aims to perform critical sentence selection on text-encoded representations and then generate summaries. Combination 3 has the same structure as Combination 2, but it pretrains module B; in particular, the above three combinations are all supervised training using the training set “gold standard” as the ground truth. Finally, combination 4 adopts the structure of Combination 3. In addition to pretraining module B, coherence enhancement is only performed by maximizing the coherence gain. During the reinforcement process, the extracted pseudo-summary is used as an alternative truth value; Combination 5 is similar to Combination 4 in that it only enhances the coherence by maximizing the content revenue; Combination 6 is the complete XAI-RL model in Figure 1, and the extracted pseudo-summary is still used as the alternative truth value.Table 2
Ablation combinations of XAI-RL corresponding to Figure1.
CombinationPseudo abstract extractionModule AModule B (coding)Module B (coding + key sentence extraction)Module B (pretraining + coding + key sentence extraction)Module CContinuity enhancement (continuity benefit)Coherence enhancement (content gain)1√√√2√√√√3√√√√√4√√√√5√√√√√6√√√√√√CNN/Daily Mail and XSum validation sets evaluated the above six ablative combinations. The experimental results are shown in Tables3 and 4 and Figures 3 and 4.Table 3
Evaluation results of ablation combinations on CNN/daily mail dataset %.
CombinationROUGE-1ROUGE-2ROUGE-LROUGE-AVGMETEOR146.8526.0844.8739.2718.62247.7827.8246.0540.5518.7349.7428.2747.8541.9518.92451.6429.1349.3333.3719.7552.0129.2849.7133.6729.48653.2930.5550.1334.6630.51Table 4
Evaluation results of ablation combinations on XSum dataset %.
CombinationROUGE-1ROUGE-2ROUGE-LROUGE-AVGMETEOR145.4624.340.7136.8226.65247.1427.8141.6538.8726.94348.9226.8242.7139.4827.05449.5328.6543.3840.5228.37549.8728.2843.940.6828.46651.9728.2343.8441.3528.86Figure 3
Evaluation results of ablation combinations on CNN/daily mail dataset.Figure 4
Evaluation results of ablation combinations on XSum dataset.First, the evaluation results of Combination 2 are better than Combination 1, which indicates that after extracting key sentences by module B, the decoder can decode the critical content and generate higher-quality summaries. Second, combination 3 is better than combination 2, indicating that pretraining can make the parameter configuration of module B more reasonable and then select critical sentences more reasonably. Then, the evaluation results of combination four and combination 5 are better than those of combination 3, indicating that the revenue and sentence coherence enhancement method constructed in this paper can effectively improve the quality of abstract content. In particular, the ROUGE-L and METEOR indicators of combination 4 are better than those of combination 3, reflecting the improvement of sentence coherence by the coherence measurement and reinforcement in this paper. Finally, the combination of all the mechanisms 6 has the best evaluation result, reflecting the effectiveness of each module of the proposed model XAI-RL in a summary generation.To sum up, for the XAI-RL model, firstly, by comparing combination two and combination 3, it can be found that after pretraining the coherence measurement module by the pseudo-summary, meaningful sentences and contextual semantic information can be better identified from the text encoding representation, to provide the decoder with semantic benchmarks and auxiliary input to generate summary content that can accurately summarize the main idea of ​​the original text; secondly, by comparing combination three and combination 4, it can be found that the self.


## 4.3. Comparison between the XAI-RL Model and Existing Text Summarization Models

In this section, the accuracy of the XAI-RL model is compared with the existing extractive and generative methods on the test set to evaluate its summary generation quality. First, the evaluation results (average of 3 times) of the XAI-RL model and the comparison method on the CNN/Daily Mail dataset are shown in Table5 (ROUGE-AVG is the average of ROUGE-1, ROUGE-2, and ROUGE-L). On the one hand, as shown in Figure 5, the evaluation results of the XAI-RL model are generally better than the existing extraction methods. This model outperforms other extraction-based baseline models in ROUGE-1 and ROUGE-2 indicators, indicating that it can effectively obtain the original subject information.Table 5
Evaluation results of generated summarization on CNN/daily mail dataset %.
How the abstract is generatedModel methodROUGE-1ROUGE-2ROUGE-LROUGE-AVGMETEORExtractionMMS_Text47.5725.7244.4239.2426.97SummaRuNNer48.625.244.339.3726.75Refresh49.2727.245.640.6927.38HSSAS51.326.846.641.5728.27GenerativePointer-generator + coverage48.5326.2845.3840.0627.7Bottom-up50.2227.6847.3441.7528.38DCA50.6928.4746.9242.0328.55SupervisedBERTSUMEXTABS51.1328.648.1842.6428.91PEGASUSBASE50.7927.8147.9342.1828.63XAI-RL4 (Table2 combination 4)51.6429.1349.3343.3729.7XAI-RL5 (Table2 combination 5)52.0129.2849.7143.6729.48XAI-RL6 (Table2 combination 6)53.2930.5540.1344.6620.51Figure 5
Experimental results on CNN/daily mail dataset corresponding to Table5.At the same time, its scores on ROUGE-L and METEOR indicators are higher than other extraction-based baseline models which indicate that the model can ensure the coherence of generated sentences when paraphrasing the acquired vital sentences. The core ideas of the compared extraction methods (MMS_Text, SummaRuNNer, Refresh, and HSSAS) can be summarized into three categories: one is to convert text into a graph structure (such as MMS_Text), by scoring nodes (sentences) to extract essential sentences to form summary texts; the second is to mine the latent features of the reader through the encoder and extract summary sentences in the order of probability matrix or sentence arrangement (such as SummaRuNNer and HSSAS). The third is to use XAI-Based Reinforcement Learning to build quality gains, and after updating the sentence selection strategy to maximize profits, extract abstract texts from the original document (such as Refresh). However, for the XAI-RL model proposed in this paper, its core idea is “extract first, then generate.” The coherence measurement module of the model can identify and extract critical sentences in the original text after pretraining, thus prompting the decoder to pay attention to the actual content. In addition, when decoding and generating, the XAI-RL model outputs auxiliary information H containing contextual semantics to the decoder, which further enriches the text feature information inside the model, and finally makes the quality of the abstract text generated by the model XAI-RL better than the “single” extraction model. On the other hand, as shown in Figure6(b), the XAI-RL model is compared with existing generative methods (Pointer-Generator + Coverage [11], Bottom-up, DCA [15], BERTSUMEXTABS [19], and PEGASUS) and also achieved better accuracy overall. The model outperforms other generative baseline models in ROUGE-1 and ROUGE-2 indicators, indicating that it can correctly paraphrase the original text message. At the same time, its scores on both ROUGE-L and METEOR metrics are higher than other generative baseline model, which indicates that this model is more capable of generating coherent and smooth summary content. Its performance improvement can be attributed to: first, in developing the abstract, as shown in Figure 2, the model XAI-RL is based on the pretraining components (such as the AL-BERT encoder and the pretraining coherence measurement module), and the text encoding results are further used. The semantic segment is the division unit and is additionally encoded by the L = 3-layer Transformer-XL component through the semantic segment-based recurrent self-attention mechanism to increase the strength of feature parsing. Second, in the coherence enhancement process shown in Figure 4, the model XAI-RL re-encodes the generated summary text to calculate the coherence gain; at the same time, the extracted pseudo-summary is used to create the summary to calculate the content gain by maximizing the two. The weighted sum of revenue strengthens the model summary text generation process. Further, it improves the model text generation quality from the content and sentence coherence levels.Figure 6
Experimental results on XSum dataset corresponding to Table6.Secondly, the evaluation results of the XAI-RL model and the compared method on the XSum dataset (average of 3 times) are shown in Table6, and the corresponding histogram is shown in Figure 6. Overall, the model still achieves optimal results. In particular, the XSum dataset is only used to test generative methods due to its high novelty corresponding to the “gold standard” abstract. The results shown in Table 6 and Figure 6 further illustrate the “extract first, then generate” design principle followed by the model XAI-RL, the recurrent self-attention weight based on semantic segment, and the reinforcement process based on content benefit and coherence benefit can effectively improve the summary generation quality.Table 6
Evaluation results of generated summarization on XSum dataset %.
How the abstract is generatedModel methodROUGE-1ROUGE-2ROUGE-LROUGE-AVGMETEORSupervisedPointer-generator + coverage37.18.0230.7218.2820.31Bottom-up39.029.4532.9620.1421.09BERTSUMEXTABS47.8125.540.2727.8624.14PEGASUSBASE48.7925.5840.728.3624.82UnsupervisedXAI-RL4 (Table2 combination 4)49.5328.6543.3830.5228.37XAI-RL5 (Table2 combination 5)49.8728.2843.930.6828.46XAI-RL6 (Table2 combination 6)51.9728.2343.8431.3528.86


## 5. Conclusion

Using automated text summarization technology to condense the core content of the text is a necessary means to reduce the cost of text data storage and improve the efficiency of information retrieval. To quickly generate high-quality and readable summaries of texts while avoiding the ground-truth dependence of model training, the XAI-RL-based text summarization model for coherence enhancement (XAI-RL) proposed in this paper utilizes the Transformer-XL of the attention mechanism to build a coherence measurement module and uses the extracted pseudo-summary to pretrain it, which can effectively identify and remove the important textual information. In addition, it can automatically measure the coherence of the generated abstract during the re-encoding process and generate text coherence benefits, which can be introduced into the coherence enhancement process of the model, which can promote the model generation to be closer to the original theme, more readable summary content. Experiments show that the evaluation accuracy of the XAI-RL model incorporating coherence metric and coherence enhancement is better than the other existing methods in multiple sets of experiments. The future work of this paper will further improve the effectiveness of self-attention weights on coherence measurement. By constructing various measurement methods, the coherence factors such as semantic connection, grammatical regularity, and coreference disambiguation will be considered from multiple perspectives to improve the sentence coherence for the next-generation models.



---

*Source: 7516832-2022-08-04.xml*