SOM - NCSCM : An Efficient Neural C hinese Sentence Compression Model Enhanced with Self-Organizing Map (2021)

Abstract: Sentence Compression (SC), which aims to shorten sentences while retaining important words that express the essential meanings, has been studied for many years in many languages, especially in English. However, improvements on Chinese SC task are still quite few due to several difficulties: scarce of parallel corpora, different segmentation granularity of Chinese sentences, and imperfect performance of syntactic analyses. Furthermore, entire neural Chinese SC models have been under-investigated so far. In this work, we construct an SC dataset of Chinese colloquial sentences from a real-life question answering system in the telecommunication domain, and then, we propose a neural Chinese SC model enhanced with a Self-Organizing Map (SOM-NCSCM), to gain a valuable insight from the data and improve the performance of the whole neural Chinese SC model in a valid manner. Experimental results show that our SOM-NCSCM can significantly benefit from the deep investigation of similarity among data, and achieve a promising F1 score of 89.655 and BLEU4 score of 70.116, which also provides a baseline for further research on Chinese SC task.

## introduction sentence compression (sc) is an important natural language processing (nlp) task which aims to shorten sentences or texts while preserving their essential original meanings. the technique of sc can benefit several real applications, such as automatic title generation @xcite , information extraction, opinion mining @xcite , machine translation @xcite , and question answering systems. previous works of sc can be classified into two categories: (1) rule-based approaches @xcite , (2) machine-learning-based approaches @xcite . the latter can be further divided into statistic-based approaches @xcite and syntaxbased approaches @xcite . besides, most approaches have treated sc as a deletion-based processing that estimates the importance of each word in the sentences or texts in turn, and then decides if it should be kept or deleted in generating compressed sentences. although researches on sc have been conducted for many years in many languages, especially in english, improvements on chinese sc task are still quite few as far as we know. one reason for this situation is lacking corpora of chinese parallel data which are necessary for training and evaluating a supervised or semi-supervised chinese sc model. to address this problem, several chinese sc works tend to translate the english sc datasets into chinese or use web crawler and data filtering techniques to produce high-quality data from the popular chinese micro-blogging website or other chinese news websites @xcite @xcite . however, those datasets are mostly written in chinese formal expressions, and not made publicly available or exactly extraction-based. another reason is the particularity of the chinese language itself, which not only brings about the former problem, but also causes the different segmentation granularity of chinese sentences, without standard syntactic or grammatical instructions and vagueness of some chinese expressions that make the judgement of compressed results non-identical but reasonable among different human evaluators. to deal with the problems above, many chinese sc approaches are more unsupervised-methodbased using heuristic rules, which are hard to be duplicated and transferred to delete unnecessary constituents in sentences for other domains, or using statistical probabilities, like the tf-idf, to trim the syntactic parse trees @xcite @xcite , which lack flexibility and capacity, and demand for proper data distribution via calculating from the adopted dataset. since the domains of current chinese sc datasets are limited, to better explore the advantages of entire utilization of neural network models for chinese sc task, we (1) produce an sc dataset of chinese colloquial sentences from a real-life question answering system in the telecommunication domain which is more natural than chinese formal written expressions but more challenging to compress, and (2) augment a neural sequence labeling model with the combination of the output of a pre-trained neural clustering model based on a self-organizing map (som) @xcite using this labeled dataset, and several lexical features for a more effective chinese sc method (som-ncscm). we measure the performance of our models on the chinese colloquial sc dataset using the f1 metric, bleu scores @xcite and compression ratio (cr) @xcite . the contributions of this paper can be summarized as follows: the results prove that our model can get promising performance. ## conclusion to sum up, we construct a chinese sc dataset, composed of chinese colloquial sentences, from a real-life question answering system to address a major problem for supervised chinese sc models -the lack of parallel corpora. the dataset, as far as we know, is the first chinese parallel sc dataset in the telecommunication domain. then, we build several fundamental baselines and propose an efficient neural chinese sc model for introducing the neural clustering technique (the som) to enhance the fully neural chinese sc task, which achieves satisfactory performances on the dataset. those results confirm the utilization of similarity among data could benefit the sc task. we believe our chinese sc dataset and som-ncscm could provide a public and diverse chinese sc dataset and a fully neural-network-based efficient chinese sc model, and we also plan to continue constructing larger chinese colloquial sc dataset and explore other neural-network techniques and semi-supervised approaches to deal with the chinese sc task. chunliang zhang, minghan hu, tong xiao, xue jiang, lixin shi, and jingbo zhu. 2013. chinese sentence compression: corpus and evaluation. in chinese computational linguistics and natural language processing based on naturally annotated big data, pages 257-267. springer berlin heidelberg. tian zhang, raghu ramakrishnan, and miron livny. 1996. birch: an efficient data clustering method for very large databases. in acm sigmod international conference on management of data, volume 1, page 103-114. yonglei zhang, cheng peng, and hongling wang. 2012. research on chinese sentence compression for the title generation. in workshop on chinese lexical semantics, pages 22-31. springer, berlin, heidelberg. yang zhao, zhiyuan luo, and akiko aizawa. 2018. a language model based evaluator for sentence compression. in acl (volume 2: short papers), pages 170-175. yang zhao, hajime senuma, xiaoyu shen, and akiko aizawa. 2017. gated neural network for sentence compression using linguistic knowledge. in natural language processing and information systems, volume 10260, pages 480-491. springer international publishing, cham.