C hinese Idiom Paraphrasing (2023)

Abstract: Idioms are a kind of idiomatic expression in Chinese, most of which consist of four Chinese characters. Due to the properties of non-compositionality and metaphorical meaning, Chinese idioms are hard to be understood by children and non-native speakers. This study proposes a novel task, denoted as Chinese Idiom Paraphrasing (CIP). CIP aims to rephrase idiom-containing sentences to non-idiomatic ones under the premise of preserving the original sentence’s meaning. Since the sentences without idioms are more easily handled by Chinese NLP systems, CIP can be used to pre-process Chinese datasets, thereby facilitating and improving the performance of Chinese NLP tasks, e.g., machine translation systems, Chinese idiom cloze, and Chinese idiom embeddings. In this study, we can treat the CIP task as a special paraphrase generation task. To circumvent difficulties in acquiring annotations, we first establish a large-scale CIP dataset based on human and machine collaboration, which consists of 115,529 sentence pairs. In addition to three sequence-to-sequence methods as the baselines, we further propose a novel infill-based approach based on text infilling. The results show that the proposed method has better performance than the baselines based on the established CIP dataset.

## introduction idioms, called @xcite in chinese, are widely used in daily communications and various literary genres. idioms are a kind of compact chinese expressions that consist of few words but imply relatively complex social nuances. moreover, chinese idioms are often used to describe similar phenomena, events, etc., which means the idioms cannot be interpreted with their literal meanings in some cases. thus, it has always been a challenge for non-native speakers, and even native speakers, to recognize chinese idioms @xcite . for instance, the idiom '' '' (yuroubaixing) shown in figure 1 , represents ''oppress the people'', instead of its literal meaning -''fish meat the people''. in real life, if some people do not understand the meaning of idioms, we have to explain them by converting them into a set of word segments that reflect more intuitive and understandable paraphrasing. in this study, we try to manipulate computational approaches to automatically rephrase idiom-containing sentences into simpler sentences (i.e., non-idiom-containing sentences) for preserving context-based paraphrasing, and then benefit both chinese-based natural language processing and societal applications. since idioms are a kind of obstacles for many nlp tasks, cip can be used as a pre-processing phase that facilitates and improves the performance of machine translation systems @xcite , chinese idiom cloze @xcite , and chinese idiom embeddings @xcite . furthermore, cip-based applications can help specific groups, such as children, non-native speakers, and people with cognitive disabilities, to improve their reading comprehension. we propose a new task in this study, denoted as chinese idiom paraphrasing (cip), which aims to rephrase the idiom-containing sentences into fluent, intuitive, and meaning-preserving non-idiom-containing sentences. we can treat the cip task as a special paraphrase generation task. the general paraphrase generation task aims to rephrase a given sentence to another one that possesses identical semantics but various lexicons or syntax @xcite . similarly, cip emphasizes rephrasing the idioms of input sentences to word segments that reflect more intuitive and understandable paraphrasing. in recent decades, many researchers devoted to an idiom-containing sentence is hard to process by nlp applications. for example, this idiom-containing sentence is translated by the newest google translator. 1 after processing the idiom-containing sentence using the proposed cip method, we can output the correct translation. in the example, the idiom is undelined. contributions. this study produces two main contributions toward the development of cip systems. first, a large-scale benchmark is established for the cip task. the benchmark comprises 115,529 sentence pairs, which of 8,421 are idioms. a recurrent challenge in crowdsourcing nlp-oriented datasets at scale is that human writers frequently utilize repetitive patterns to fabricate examples, leading to a lack of linguistic diversity @xcite . a new large-scale cip dataset is created in this study by taking advantage of the collaboration between humans and machines. in detail, we initially divide a large-scale chinese-english machine translation corpus into two parts (idiom-containing sub-corpus, and non-idiom-containing sub-corpus) by judging if a chinese sentence contains idioms. next, we train an english-to-chinese machine translation (mt) system using the non-idiom-containing sub-1 translate.google.com. accessed in: 2022-12-01. second, we propose one novel infill-based method to rephrase the input idiom-containing sentence. since the constructed dataset is used as the training dataset, we treat the cip task as a paraphrase generation task. we adopt three different sequence-to-sequence (seq2seq) methods as baselines: lstm-based approach, transformerbased approach, and mt5-based approach, where mt5 is a massively multilingual pre-trained text-to-text transformer @xcite . our proposed infill-based method is only required to rephrase the idioms of the sentence, which means that we only need to generate context-based interpretations of idioms, rather than the whole sentence. specifically, a cip sentence pair can be processed to produce a (corrupted) input sentence by replacing both the idioms of the source sentence and a corresponding target extracted from the simplified sentence. the mt5-based cip method is fine-tuned to reconstruct the corresponding target. experimental results show that, compared with the baselines evaluated on the constructed cip dataset, our infill-based method can output high-quality paraphrasing of sentences that are grammatically correct and semantically appropriate. as the use of the chinese language becomes more widespread, the need for effective chinese paraphrasing methods may increase, leading to further research and development in this area. the constructed dataset and employed baselines that are used to accelerate this research are open-source, available on github. 2 2 related work paraphrase generation: paraphrase generation aims to extract paraphrases of given sentences. the extracted paraphrases can preserve the original meaning of the sentence, but are assembled with different words or syntactic structures @xcite @xcite . most recent neural paraphrase generation methods primarily take advantage of the sequence-tosequence framework, which can achieve considerable performance improvements compared with traditional approaches @xcite . some approaches use reinforcement learning or multi-task learning to improve the quality and diversity of generated paraphrases @xcite . a long-standing issue embraced in paraphrase generation studies is the lack of reliable supervised datasets. the issue can be avoided by constructing manually annotated paired-paraphrase datasets @xcite or designing unsupervised paraphrase generation methods @xcite . differ from existing paraphrase generation research, we take our attention to chinese idiom paraphrasing that rephrases idiom-containing sentences to non-idiom-containing ones. text infilling: originating from cloze tests @xcite ), text infilling aims to fill in missing blanks in a sentence or paragraph by making use of the preceding and subsequent text, to make the text complete and meaningful. current text infilling methods may be categorized into four groups. gan-based methods train gans to ensure that the generator generates highly dependable infilling content that can trick the discriminator @xcite . intricate inference-based methods use dynamic programming or gradient search to locate infilling content that is highly probable within its surrounding context @xcite . masked lm-based methods generate infilling content based on its bidirectional contextual word embedding @xcite . lm-based methods fine-tune off-the-shelf lms in an auto-regressive manner, and some approaches modify the input format by putting an infilling answer after the masked input @xcite , whereas others do not modify the input format @xcite . in contrast to the aforementioned methods, our goal in this paper is not only to make the text complete, but also to maintain the sentence's meaning when creating paraphrases. as a result, we employ a sequence-to-sequence framework to identify infilling content. idioms: idiom is an interesting linguistic phenomenon in the chinese language. compared with other types of words, most idioms are unique in perspective of non-compositionality and metaphorical meaning. idiom understanding plays an important role in the research area of chinese language understanding. many types of research related to chinese idiom understanding have been proposed that can benefit a variety of related down-streaming tasks. for example, @xcite focused on evaluating the quality of idiom translation of machine translation systems. @xcite provided a benchmark to assess the abilities of multiple models on chinese idiom-based cloze tests, and evaluated how well the models can comprehend chinese idiom-containing texts. @xcite studied how to improve essay writing skills by recommending chinese idioms. @xcite investigated the tasks on learning and quality evaluation of chinese idiom embeddings. in this paper, we study a novel cip task that is different from the above tasks. since the proposed cip method can rephrase idiom-containing sentences to non-idiom-containing ones, it is expected that cip can benefit tasks related to idiom representation and idiom translation. @xcite studied a new task of english idiom paraphrases aiming to determine whether two idioms have alike or similar meanings. they collected idioms' definitions in a dictionary and utilized word embedding modelings to represent idioms to calculate the similarity between two idioms. @xcite proposed a chinese lexical simplification method, which focuses on replacing complex words in given sentences with simpler and meaning-equivalent alternatives. it is noteworthy that the substitutes in chinese lexical simplification are all made up of a single word, but an idiom typically cannot be substituted by a single word to express original concepts or ideas. ## human and machine collaborative dataset construction this section describes the process of constructing a large-scale parallel dataset for cip. a qualified cip dataset needs to meet the following two requirements: (1) the two sentences in a sentence pair have to convey the same meaning; and (2) a sentence pair has to contain an idiom-containing sentence and an idiom-containing one. we outline a three-stage pipeline for dataset construction, which takes advantage of both the generative strength of machine translation (mt) methods and the evaluative strength of human annotators. human annotators are generally reliable in correcting examples, but it is challenging while crafting diverse and creative examples at scale. therefore, we deploy a machine translator to automatically create an initial cip dataset, and then inquire annotators to proofread each generated instance. ## methods based on our constructing cip dataset, the cip task can be treated as a sentence paraphrasing task (section 4.1). additionally, we propose a novel infill-based method to solve it (section 4.2). ## conclusion in this paper, we propose a novel chinese idiom paraphrasing (cip) task, which aims to rephrase sentences containing idioms into non-idiomatic versions. the cip task can be treated as a special