SlowGuess/NAACL_2022_OCR huggingface dataset:
where  $b(\mathbf{x},\mathbf{y}_{1:T})$  denotes the baseline value to reduce the variance of the reward. In practice, we set it to 0.5 during training. And  $\lambda$  is a hyper-parameter for balance. It is worth noting that, (10) only defines a reward value for a completely generated summary. If  $\mathbf{y}_{1:T}$  is partially generated, the values of  $D_{\phi}(\mathbf{x},\mathbf{y}_{1:T})$  and  $S_{cos}(\mathbf{x},\mathbf{y}_{1:T})$  are meaningless. To evaluate the action-value for an intermediate state, we apply Monte Carlo (MC) tree search under the policy  $G_{\theta}$  to sample the following unknown tokens. Each search lasts until the end of summary token is sampled or the sampled summary reaches the maximum length. For more stable reward and lower variance, we conduct a  $K$ -time roll-out as follow:
$$
\left\{\mathbf {y} _ {1: T _ {1}} ^ {1}, \dots , \mathbf {y} _ {1: T _ {K}} ^ {K} \right\} = M C ^ {G _ {\theta}} \left(\left(\mathbf {y} _ {1: t}, \mathbf {x}\right), K\right) \tag {11}
$$
where  $T_{i}$  denotes the length of the summary sampled by the  $i$ -th Monte Carlo search.  $(\mathbf{y}_{1:t},\mathbf{x})$  is the current state and  $\mathbf{y}_{t + 1:T_i}^i$  is sampled based on the policy  $G_{\theta}$ . Accordingly, the discriminator provides  $K$  rewards for the sampled  $K$  summaries respectively. The final reward for the intermediate state is computed as the average of  $K$  rewards. Thus, for the generated summary with length  $T$ , we compute the final reward for  $y_{t}$  at the sentence level as:
$$
\begin{array}{l} R _ {D _ {\phi}} ^ {G _ {\theta}} (\mathbf {y} _ {1: t - 1}, \mathbf {x}, y _ {t}) = \\ \left\{ \begin{array}{l l} \frac {1}{K} \sum_ {k = 1} ^ {K} \lambda \left(D _ {\phi} (\mathbf {x}, \mathbf {y} _ {1: T} ^ {k}) - b (\mathbf {x}, \mathbf {y} _ {1: T} ^ {k})\right) + & \\ (1 - \lambda) \mathcal {S} _ {\text {c o s}} (\mathbf {x}, \mathbf {y} _ {1: T} ^ {k}) & t <   T \\ \lambda \left(D _ {\phi} (\mathbf {x}, \mathbf {y} _ {1: t}) - b (\mathbf {x}, \mathbf {y} _ {1: t})\right) + & \\ (1 - \lambda) \mathcal {S} _ {\text {c o s}} (\mathbf {x}, \mathbf {y} _ {1: t}) & t = T \end{array} \right. \tag {12} \\ \end{array}
$$
Using the discriminator  $D$  as a reward function can further improve the generator iteratively by dynamically updating  $D$ . Once we have a set of more realistic generated summaries, we shall re-train the discriminator model by minimizing (8). Each time when a new discriminator model is obtained, we can re-train the generator. The gradient of the objective  $\mathcal{I}_{adv}(\theta)$  w.r.t. the generator's parameters  $\theta$  can be formulated as:
$$
\begin{array}{l} \nabla \mathcal {J} _ {a d v} (\theta) = \frac {1}{T} \sum_ {t = 1} ^ {T} \sum_ {y _ {t}} R _ {D _ {\phi}} ^ {G _ {\theta}} \left(\mathbf {y} _ {1: t - 1}, \mathbf {x}, y _ {t}\right) \\ \cdot \nabla_ {\theta} \left(G _ {\theta} \left(y _ {t} \mid \mathbf {y} _ {1: t - 1}, \mathbf {x}\right)\right) \\ = \frac {1}{T} \sum_ {t = 1} ^ {T} \mathbb {E} _ {y _ {t} \in G _ {\theta}} \left[ R _ {D _ {\phi}} ^ {G _ {\theta}} (\mathbf {y} _ {1: t - 1}, \mathbf {x}, y _ {t}) \right. \\ \cdot \nabla_ {\theta} \log p \left(y _ {t} \mid \mathbf {y} _ {1: t - 1}, \mathbf {x}\right) ] \tag {13} \\ \end{array}
$$
The overall training flow of SSPGAN is shown in Figure 3. Both the generator  $G$  and the siamese semantic-preserving discriminator  $D$  learn together by pursuing competing goals. Given  $\mathbf{x}$ , the generator  $G$  generates a summary  $\mathbf{y}$ . It would prefer summaries with bigger rewards, which implies larger values of  $s_{\text{reality}}$  and  $s_{\text{similarity}}$ . In contrast, the discriminator  $D$  would encourage smaller values of  $s_{\text{reality}}$  and  $s_{\text{similarity}}$ . Thus, the generator  $G$  and the siamese semantic-preserving discriminator  $D$  play a min-max game (see Algorithm 1 in the Appendix A.2 for more details).
We conduct extensive experiments on both Chinese and English text summarization datasets. The Chinese dataset we adopt is a large corpus of Chinese short text summarization (LCSTS) (Hu et al., 2015), which is collected from Sina Weibo, a famous Chinese social media website. Following the data split of previous works, we get around 2.4M text-summary pairs for training, 10K pairs for validation and 725 pairs with annotation score no less than 3 for testing. For English text summarization, we use the Gigaword dataset based on Annotated Gigaword (Napoles et al., 2012), and preprocess it identically to (Rush et al., 2015), which results in 3.8M sentence pairs for training, 190K for validation and around 1.9K for testing.
For a fair comparison with previous works, we adopt ROUGE (Lin, 2004) as the automatic evaluation metric. ROUGE measures the degree of overlap between the generated summary and the reference, with respect to the number of n-grams. We report ROUGE-1 (uni-gram), ROUGE-2 (bigram), ROUGE-L (longest common subsequence - LCS) on the testing set for our quantitative experiments. Since the official ROUGE evaluation package is only available for English summarization, to evaluate the models on the Chinese summarization task, we follow (Hu et al., 2015) and map all characters including punctuation and numbers to numerical IDs, and then conduct evaluation on them. In experiments, we denote ROUGE as RG for simplicity.
Baselines for the Chinese text summarization task include the followings. RNN and RNN-context are two RNN-based models adopted in (Hu et al., 2015), without and with the attention mechanism respectively. CopyNet leverages the copy mechanism to alleviate the OOV problem (Gu et al., 2016). RNN-MRT (Shen et al., 2016) and Actor-Critic (Li et al., 2018) are two sentence-level training methods to address the problem of teacher forcing which use the maximum likelihood estimation. DRGD (Li et al., 2017b) uses a recurrent latent random model to strengthen the abstractive text summarization model. GlobalEncoding (Lin et al., 2018) controls
System	RG-1	RG-2	RG-L
ABS	29.55	11.32	26.42
ABS+	29.76	11.88	26.96
Concept-pointer+DS	37.01	17.10	34.87
DRGD	36.27	17.57	33.62
Actor-Critic	36.05	17.35	33.49
Transformer	37.57	18.90	34.69
SSPGAN	38.31	19.89	35.60
Table 1: The full-length F-1 based ROUGE scores on the testing set of the English benchmark Gigaword. Here we bold the best results.
the information flow from the encoder to the decoder based on the source-side global information.
As for the English dataset, besides DRGD and Actor-Critic, we choose the following baselines. ABS and  $\mathrm{ABS + }$  are two pioneer methods using neural networks for abstractive text summarization (Rush et al., 2015). Concept-pointer+DS engages abstractive summarization models to generate new conceptual words (Wang et al., 2019).
Our model is complemented based on Tensor2Tensor  ${}^{1}$  . For all experiments,SSPGAN is run with 5 random seeds on 2 NVIDIA V100 GPUs and the final automatic results are presented with means (see the Appendix A.1 for more details).
4.4.1 English Results
Table 1 shows the results on the English dataset Gigaword. The results of the baselines are reported in the upper rows, while the bottom row summarizes the results of the proposed SSPGAN. When we introduce the SSPGAN framework to Transformer, it significantly improves the performance, proving the effectiveness of our method.
4.4.2 Chinese Results
The experimental results on the Chinese dataset LC-STS are presented in Table 2. As can be observed from the comparison between the baselines in the upper rows and SSPGAN in the bottom row, the proposed method achieves the best performance. In addition, the proposed SSPGAN brings significant improvements to the classical baseline Transformer. Precisely, Transformer is greatly improved in ROUGE-1/2/L with gains of  $+1.88/+1.09/+1.20$ .
$^{1}$ https://github.com/tensorflow/tensor2tensor
System	RG-1	RG-2	RG-L
RNN	21.50	8.90	18.60
RNN-context	29.90	17.40	27.20
CopyNet	34.40	21.60	31.30
RNN-MRT	37.87	25.43	35.33
Actor-Critic	37.51	24.68	35.02
DRGD	36.99	24.15	34.21
GlobalEncoding	39.40	26.90	36.50
Transformer	42.35	29.38	39.23
SSPGAN	44.23	30.47	40.43
Table 2: The full-length F-1 based ROUGE scores on the testing set of the Chinese benchmark LCSTS. Here we bold the best results.
System	RG-1	RG-2	RG-3
Transformer	37.57	18.90	34.69
+SSPGAN (η, λ=1.0)	38.00	19.41	35.18
+SSPGAN (η, λ=0.7)	38.31	19.89	35.60
+SSPGAN (η, λ=0)	37.78	19.19	34.99
In this section, we analyze the effectiveness of the proposed method from multiple perspectives. All the experiments are conducted on Gigaword.
4.5.1 Ablation Study
As shown in Table 3, we analyze the contributions of different sub training objectives proposed in (8) and (10). On the Transformer model, the basic GAN (i.e., the second row with  $\eta = 1.0$  and  $\lambda = 1.0$ ) achieves improvement with gains of  $+0.43 / +0.51 / +0.49$  in ROUGE scores. We also test the results when Transformer is only guided by the semantic similarity objective (i.e., the fourth row with  $\eta = 0$  and  $\lambda = 0$ ), resulting gains of  $+0.21 / +0.29 / +0.30$ . Armed with the proposed SSPGAN (i.e., the third row with  $\eta = 0.7$  and  $\lambda = 0.7$ ), the performance can be more significantly improved with gains of  $+0.74 / +0.99 / +0.91$  in ROUGE scores.
4.5.2 Human Evaluation
To further evaluate the quality of the generated summaries, we randomly select 50 test examples from the Gigaword testing set for human evaluation. For each example, we show the source text, the ground truth summary as well as the summaries generated by different models. The human evaluators do not know which summary comes from which model or
Table 3: Ablation study regarding the sub training objectives proposed in (8) and (10). We bold the best results.
System	R	C
Transformer	6.39	6.65
+SSPGAN (η, λ=1.0)	7.09	6.82
+SSPGAN (η, λ=0.7)	7.06	7.33
+SSPGAN (η, λ=0)	6.64	6.78
Table 4: Comparison of human evaluation on a random subset of the Gigaword testing set. We denote the readability and consistency as R and C, respectively. The best results are bold.
Source: malaysia's national car maker proton expects to export its cars to russia by early next year to boost its overseas sales, a company official said tuesday
Reference: malaysian carmaker proton seeks inroads into russia by early next year
GAN: malaysia's car maker to boost overseas sales
SSPGAN: malaysia's proton to export cars to russia
Source: chinese vice-premier wu yi said tuesday that the country should step up efforts to develop its service trade in a bid to alter the growth pattern of foreign trade and increase employment and domestic consumption.
Reference: chinese vice-premier calls for fast development of service trade
GAN: chinese vice-premier urges to increase domestic consumption
SSPGAN: chinese vice-premier urges development of service trade
Figure 4: Comparison of the summaries generated by the basic GAN and the proposed SSPGAN. Bold text represents that the correct contents are extracted, while the underlined parts correspond to the wrong ones.
which one is the ground truth. Two scores from 1 to 10 are assigned to each summary (1 and 10 indicate the worst and the best respectively), one for readability (how well-written the summary is) and one for consistency (how well the summary conveys the key content of the source text). Each summary is rated by 10 invited human evaluators who are capable of reading English proficiently. And the results are averaged across all selected examples and evaluators. As shown in Table 4, equipped with the basic GAN objective, the readability is improved significantly with comparable results (i.e., the second row with  $\eta = 1.0$  and  $\lambda = 1.0$  and the third row with  $\eta = 0.7$  and  $\lambda = 0.7$ ). As for the consistency, our proposed model (i.e., the third row with  $\eta = 0.7$  and  $\lambda = 0.7$ ) achieves the highest score, which justifies that the proposed method can preserve the key content of the source text more accurately. It is worth noting that the improvements of the fourth row are limited, which is only equipped with siamese similarity ob
jective. Due to the lack of basic GAN objective, the improvement of readability is limited, resulting in incomplete sentence semantic expression and damage to the improvement of consistency.
4.5.3 Case Study
Figure 4 shows some examples of the generated summaries on the English dataset, in which both the basic GAN and the proposed SSPGAN produce readable results. However, as shown in the highlights of the SSPGAN examples, the proposed method is able to convey the key content of the source text more accurately, resulting in more salient summaries as expected. Specifically, in the upper example, the key content "expectsto export its car to russia" in the source text is only expressed by SSPGAN, while the basic GAN generates "boost overseas sales", ignoring the most relevant information. Similar behaviors can also be observed in the bottom example.
This paper presents a novel siamese generative adversarial net (SSPGAN) which can preserve the semantic consistency between the source text and the target summary for abstractive text summarization. In SSPGAN, a novel semantic similarity based reward is introduced to further augment the GAN-based abstractive text summarization to preserve the semantic consistency and convey the key content in the source text. It is worth noting that SSPGAN addresses the problem of saliency for text summarization from a totally different perspective of semantic consistency, therefore it is orthogonal to some state-of-the-art methods which focus on attention mechanism, and can be applied to them for further improvements.
This research was supported by Anhui Provincial Natural Science Foundation (2008085J31) and National Natural Science Foundation of China under No. 62172382.
Sanjeev Arora, Rong Ge, Yingyu Liang, Tengyu Ma, and Yi Zhang. 2017. Generalization and equilibrium in generative adversarial nets (gans). In International Conference on Machine Learning, pages 224-232. PMLR.
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. International Conference on Learning Representations.
Michele Banko, Vibhu O. Mittal, and Michael J. Witbrock. 2000. Headline generation based on statistical translation. In Proceedings of the 38th Annual Meeting of the Association for Computational Linguistics, pages 318-325, Hong Kong. Association for Computational Linguistics.
Sumit Chopra, Michael Auli, and Alexander M. Rush. 2016. Abstractive sentence summarization with attentive recurrent neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 93-98, San Diego, California. Association for Computational Linguistics.
Trevor Cohn and Mirella Lapata. 2008. Sentence compression beyond word deletion. In Proceedings of the 22nd International Conference on Computational Linguistics (Coling 2008), pages 137-144, Manchester, UK. Coling 2008 Organizing Committee.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171-4186, Minneapolis, Minnesota. Association for Computational Linguistics.
Bonnie Dorr, David Zajic, and Richard Schwartz. 2003. Hedge trimmer: A parse-and-trim approach to headline generation. Technical report, MARYLAND UNIV COLLEGE PARK INST FOR ADVANCED COMPUTER STUDIES.
Xiangyu Duan, Hongfei Yu, Mingming Yin, Min Zhang, Weihua Luo, and Yue Zhang. 2019. Contrastive attention mechanism for abstractive sentence summarization. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3044-3053, Hong Kong, China. Association for Computational Linguistics.
Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann N Dauphin. 2017. Convolutional sequence to sequence learning. In International Conference on Machine Learning, pages 1243-1252. PMLR.
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. Advances in neural information processing systems, 27.
Jiatao Gu, Zhengdong Lu, Hang Li, and Victor O.K. Li. 2016. Incorporating copying mechanism in sequence-to-sequence learning. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1631-1640, Berlin, Germany. Association for Computational Linguistics.
Min Gui, Junfeng Tian, Rui Wang, and Zhenglu Yang. 2019. Attention optimization for abstractive document summarization. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 1222-1228, Hong Kong, China. Association for Computational Linguistics.
Gabriel Lima Guimaraes, Benjamin Sanchez-Lengeling, Carlos Outeiral, Pedro Luis Cunha Farias, and Alán Aspuru-Guzik. 2017. Objective-reinforced generative adversarial networks (organ) for sequence generation models. Computing Research Repository.
Baotian Hu, Qingcai Chen, and Fangze Zhu. 2015. LC-STS: A large scale Chinese short text summarization dataset. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1967–1972, Lisbon, Portugal. Association for Computational Linguistics.
Hongyan Jing and Kathleen R. McKeown. 2000. Cut and paste based text summarization. In 1st Meeting of the North American Chapter of the Association for Computational Linguistics.
Tom Kenter, Alexey Borisov, and Maarten de Rijke. 2016. Siamese CBOW: Optimizing word embeddings for sentence representations. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 941-951, Berlin, Germany. Association for Computational Linguistics.
Kevin Knight and Daniel Marcu. 2000. Statistics-based summarization-step one: Sentence compression. AAAI/IAAI, 2000:703-710.
Jiwei Li, Will Monroe, Tianlin Shi, Sebastien Jean, Alan Ritter, and Dan Jurafsky. 2017a. Adversarial learning for neural dialogue generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2157-2169, Copenhagen, Denmark. Association for Computational Linguistics.
Piji Li, Lidong Bing, and Wai Lam. 2018. Actor-critic based training framework for abstractive summarization. arXiv preprint arXiv:1803.11070.
Piji Li, Wai Lam, Lidong Bing, and Zihao Wang. 2017b. Deep recurrent generative decoder for abstractive text summarization. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2091-2100, Copenhagen, Denmark. Association for Computational Linguistics.
Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74-81.
Junyang Lin, Xu Sun, Shuming Ma, and Qi Su. 2018. Global encoding for abstractive summarization. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 163-169, Melbourne, Australia. Association for Computational Linguistics.
Linqing Liu, Yao Lu, Min Yang, Qiang Qu, Jia Zhu, and Hongyan Li. 2018. Generative adversarial network for abstractive text summarization. In Thirty-second AAAI conference on artificial intelligence.
Thang Luong, Hieu Pham, and Christopher D. Manning. 2015. Effective approaches to attention-based neural machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 1412-1421, Lisbon, Portugal. Association for Computational Linguistics.
Jonas Mueller and Aditya Thyagarajan. 2016. Siamese recurrent architectures for learning sentence similarity. In Proceedings of the AAAI conference on artificial intelligence, volume 30.
Ramesh Nallapati, Bowen Zhou, Caglar Gulcehre, Bing Xiang, et al. 2016. Abstractive text summarization using sequence-to-sequence rnns and beyond. Proceedings of The 20th SIGNLL Conference on Computational Natural Language Learning.
Courtney Naples, Matthew R Gormley, and Benjamin Van Durme. 2012. Annotated gigaword. In Proceedings of the Joint Workshop on Automatic Knowledge Base Construction and Web-scale Knowledge Extraction (AKBC-WEKEX), pages 95-100.
Shashi Narayan, Yao Zhao, Joshua Maynez, Goncalo Simões, Vitaly Nikolaev, and Ryan McDonald. 2021. Planning with learned entity prompts for abstractive summarization. Transactions of the Association for Computational Linguistics, 9:1475-1492.
Paul Neculoiu, Maarten Versteegh, and Mihai Rotaru. 2016. Learning text similarity with siamese recurrent networks. In Proceedings of the 1st Workshop on Representation Learning for NLP, pages 148-157.
Joel Larocca Neto, Alex A Freitas, and Celso AA Kaestner. 2002. Automatic text summarization using a machine learning approach. In *Brazilian symposium on artificial intelligence*, pages 205-215. Springer.
Romain Paulus, Caiming Xiong, and Richard Socher. 2017. A deep reinforced model for abstractive summarization. International Conference on Learning Representations.
Nils Reimers and Iryna Gurevych. 2019. SentenceBERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing
and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982-3992, Hong Kong, China. Association for Computational Linguistics.
Alexander M. Rush, Sumit Chopra, and Jason Weston. 2015. A neural attention model for abstractive sentence summarization. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 379-389, Lisbon, Portugal. Association for Computational Linguistics.
Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073-1083, Vancouver, Canada. Association for Computational Linguistics.
Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715-1725, Berlin, Germany. Association for Computational Linguistics.
Shiqi Shen, Yu Zhao, Zhiyuan Liu, Maosong Sun, et al. 2016. Neural headline generation with sentence-wise optimization. arXiv preprint arXiv:1604.01904.
Xin Sheng, Linli Xu, Junliang Guo, Jingchang Liu, Ruoyu Zhao, and Yinlong Xu. 2020. Introvnmt: An introspective model for variational neural machine translation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8830-8837.
Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104-3112.
Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. Modeling coverage for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 76-85, Berlin, Germany. Association for Computational Linguistics.
Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in neural information processing systems, pages 5998-6008.
Wenbo Wang, Yang Gao, Heyan Huang, and Yuxiang Zhou. 2019. Concept pointer network for abstractive summarization. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3076-3085, Hong Kong, China. Association for Computational Linguistics.