C hinese Couplet Generation with Syntactic Information (2022)

Abstract: Chinese couplet generation aims to generate a pair of clauses (usually generating a subsequent clause given an antecedent one) with certain rules (e.g., morphological and syntactical symmetry) adhered and has long been a challenging task with cultural background. To generate high-quality couplet (antecedent) clauses, it normally requires a model to learn the correspondences between antecedent and subsequent clauses under aforementioned rules and constraint of few characters with their concise usage. To tackle this task, previous studies normally directly adopt deep neural networks without explicitly taking into account fine-grained analysis of the clauses, in this paper, we propose to enhance Chinese couplet generation by leveraging syntactic information, i.e., part-of-speech (POS) tags and word dependencies. In doing so, we identify word boundaries in the antecedent clause and then use a special attention module to encode the syntactic information over the words for better generating the subsequent clause. Experimental results on a dataset for Chinese couplet generation illustrate the validity and effectiveness of our approach, which outperforms strong baselines with respect to automatic and manual evaluation metrics.

## introduction a chinese antithetical couplet is composed of a pair of two sentences (i.e., an antecedent and a subsequent clause) with particular meaning that are usually applied to festivals or certain circumstances, which can be seen as a special type of poetry with a history in china of more than one thousand years. @xcite couplets are normally concise yet have profound @xcite related code and resources are available at https:// github.com/synlp/chicoupletgen . for automatic chinese couplet generation, recent studies follow the conventional encoder-decoder paradigm (sutskever et al., 2014), with an general attention mechanism to capture the correspondence between the antecedent and subsequent clauses (zhang et al., 2018; fan et al., 2019; gao et al., 2021). although satisfactory results are obtained via their approaches, reasonable granularity matching and explicit structural correspondence are still worthy of improvement with special treatment. one important reason is that conventional approaches apply character-level encoding and decoding, larger granular text units are omitted in their models and thus led to the incapability of further syntactic analysis. note that not all words in ancient chinese are in the form of single-character, it is also applied to couplets that identifying boundaries of longer text spans is useful for their better interpretation, as shown in figure 1 that the antecedent and subsequent clauses contains two-character words and their boundaries exactly match between clauses. similarly, for the structural correspondence, the syntactic rule over the antecedent clause should be identical to the subsequent one, where to the best of our knowledge there is no previous study that focus on modeling couplet generation from this aspect. the example in figure 1 also illustrates the syntactic structures of the clauses, with same word dependency patterns applied to them. for example, in the antecedent clause, the third word "秋风" (wind in autumn) is the nominal object (nsubj) of the predicate "催" (urge), and in the subsequent clause, the word "冬雪" (snow in winter) at the same position is also the nominal object of the predicate "著" (highlight) (marked by red arcs). as a result, enhancing chinese couplet generation is expected to be done by identifying appropriate word boundaries and then building explicit syntactic correspondences over them 3 between the antecedent and subsequent clauses. in this paper, we propose an approach to chinese couplet generation with syntactic information over the identified words in the clauses, where the designed model not only provides a better understanding for couplets, but also enhances the generation process with more structural constraints. specifically, we firstly learn word boundaries and then propose a special attention module to encode part-of-speech (pos) tags and word dependencies over the identified words in the antecedent clause and integrate such syntactic information into subsequent clause generation. experimental results on a dataset for chinese couplet generation demonstrate the effectiveness of our approach, which outperforms strong baselines on both automatic and manual evaluation metrics. further analyses confirm the ability of our approach in generating couplets with neatness and artistic conception. ## preliminaries conventionally, chinese couplet generation is regarded as a sequence-to-sequence text generation task (fan et al., 2019; zhang et al., 2018; gao et al., 2021; wang et al., 2021), whose objective is to generate character y t at the time step t with the given antecedent clause @xmath0, • • • , x t , • • • , x n (there are n characters in x and the t-th character is denoted as x t ) and the generated subsequent clause y t-1 = y 1 , • • • , y t-1 . the entire generation @xcite another key reason that we parse on words instead of characters is that existing parses are normally trained on words. in general, to compute p, an encoder f e is firstly used to encode the antecedent clause x through afterwards, o t is mapped to the output space through a fully connected layer, where a sof tmax function is further applied to the resulting vector to obtain the probability distribution over the vocabulary (i.e., p(y t | y t-1 , x ) in eq. ( 1 )). ## the proposed approach in order to generate high-quality clauses that satisfies the the structure constrains of chinese couplets, we propose an approach of neural text generation model enhanced by leveraging syntactic information. figure 2 illustrates the overall architecture of our approach for chinese couplet generation following the convention sequence-to-sequence paradigm, where the word boundary learning process and a special attention module to leverage syntactic information are presented on the top-left and top-right parts, respectively. in the following text, we firstly illustrate the word boundary learning process and then elaborate the proposed attention mechanism for syntactic information encoding. ## related work chinese couplet generation is an intriguing natural language generation task, which is relevant to poem generation (greene et al., 2010; zhang and lapata, 2014; wang et al., 2016; yang et al., 2018b; zhang et al., 2017; yi et al., 2017; ghazvininejad et al., 2017; yi et al., 2018; yang et al., 2018a; li et al., 2018; liu et al., 2018, 2019; liao et al., 2019; bena and kalita, 2020). yet, couplet generation differs from that for poem in the way that it requires the generation process sticking to more strict patterns and semantic requirements, while they are similar that they all conventionally performed as a sequence-to-sequence text generation task, including statistic-based approaches (jiang and zhou, 2008; zhang and sun, 2009; he et al., 2012) and neural approaches following the encodingdecoding paradigm (zhang et al., 2018; fan et al., 2019; gao et al., 2021). to improve model performance, previous neural approaches try to model the character-character correspondence between antecedent and subsequent clauses, where some advanced approaches such as attention mechanism (zhang et al., 2018) and character embedding pretraining (gao et al., 2021) are applied. compared with existing studies, our approach offers an alternative and explicit way to model the correspondence between couplet clauses through syntactic information, which provides useful knowledge to control patternized generation and is integrated into our approach via a carefully designed module. ## conclusion in this paper, we proposed a neural approach following the encoder-decoder paradigm for chinese couplet generation enhanced by of syntactic information (i.e., pos tags and word dependencies). specifically, our approach models word boundaries to facilitate the learning of syntactic information, where pos tags and word dependencies are leveraged to provide pattern guidance for couplet generation through a special attention module. experimental results on a prevously used dataset for chinese couplet generation illustrate the effectiveness of our approach, which outperforms strong baselines on both automatic and human evaluations. further analyses also confirm the ability of our approach to generating high quality couplets.