CS 2 W : A C hinese Spoken-to-Written Style Conversion Dataset with Multiple Conversion Types (2023)

Abstract: Spoken texts (either manual or automatic transcriptions from automatic speech recognition (ASR)) often contain disfluencies and grammatical errors, which pose tremendous challenges to downstream tasks. Converting spoken into written language is hence desirable. Unfortunately, the availability of datasets for this is limited. To address this issue, we present CS2W, a Chinese Spoken-to-Written style conversion dataset comprising 7,237 spoken sentences extracted from transcribed conversational texts. Four types of conversion problems are covered in CS2W: disfluencies, grammatical errors, ASR transcription errors, and colloquial words. Our annotation convention, data, and code are publicly available at https://github.com/guozishan/CS2W.

## introduction automatic speech recognition (asr) plays a vital role in a wide range of nlp application scenarios, such as simultaneous interpretation, where verbal utterances are transcribed into spoken style texts. these transcriptions serve as fundamental inputs to plenty of downstream tasks. however, they often inherently contain disfluencies, grammatical errors, and colloquial words, which pose tremendous challenges on downstream tasks. automatically correcting errors and editing spoken into written language would significantly benefit downstream tasks that are usually trained on canonical texts. developing such spoken-to-written style conversion models usually requires labeled data that cover different phenomena in asr-transcribed spoken style texts. unfortunately, existing datasets usually focus on a single type of spoken style, such as disfluencies. consequently, models trained on these datasets cannot address all spoken style issues. ## related work previous studies have treated disfluency detection and grammatical error correction as separate tasks. we hence review their progress separately. a comprehensive comparison of gec, disfluency detection, and spoken-to-written style conversion datasets is presented in dataset task type of conversion covered annotation paradigm #sentences domain language fce gec gec error-coded 34k essay en aesw gec gec / s2w error-coded 1.2m journal articles en jfleg gec gec direct rewriting 1.5k tofel exam en wi-locness gec gec direct rewriting 3.7k language-learning website en nlpcc18-task gec gec direct rewriting 717k language-learning website ch cged gec gec error-coded 7.2k hsk exam ch switchboard dc dc error-coded 265k telephone conversation en phodisfluency dc dc error-coded 5.8k telephone conversation vi japanese s2w s2w gec / dc / s2w direct rewriting 52k telephone conversation ja cs2w s2w gec / dc / s2w direct rewriting error-coded 7.2k telephone conversation ch grammatical error correction and related datasets common errors in chinese grammatical error correction (gec) datasets include spelling errors, missing words, redundant words, incorrect word order, collocation errors, etc. seq2seq models, based on rnn/cnn or transformer, are usually used for chinese gec tasks. @xcite first propose a seq2edit model for chinese gec, which achieves comparable performance with the seq2seq counterparts. most seq2editbased models use plms like bert @xcite to initialize their encoders. @xcite apply a non-autoregressive neural machine translation model to chinese gec. numerous datasets have been proposed for gec. these datasets primarily use two main paradigms to label the data: error-coded and direct rewriting. in the error-coded paradigm, annotators are tasked with explicitly identifying erroneous spans in the original sentence, specifying the error type, and subsequently making corrections. for instance, @xcite ) is an early largescale english gec dataset using the error-coded paradigm, comprising raw text produced by english learners who complete their first certificate in english. another example is aesw @xcite , sourced from a professional editing company, which focuses on assessing the level of technical writing and includes questions on not only writing style conversion but grammatical errors. additionally, the nlpcc18-task @xcite dataset originates from lang-8 foot_3 , a language learning website where native speakers can freely select essays by learners for revision. conversely, the direct rewriting paradigm instructs annotators to rephrase the input sentence directly, providing a corresponding grammatically correct version without altering the original meaning. an example is jfleg @xcite , developed with reference to the toefl exam. it places emphasis on preserving the overall fluency of the rewritten text. moreover, wi-locness @xcite encompasses two distinct datasets, derived from essays written by students with english as either their first or second language. finally, cged @xcite ) is designed to diagnose grammatical errors in chinese, based on the hsk (hanyu shuiping kaoshi, test of chinese level) examination. in contrast to the existing gec datasets, our cs2w dataset places a stronger emphasis on grammatical errors occurring in the spoken domain rather than the written domain. notably, while all previously established datasets feature a single annotation paradigm, the cs2w dataset stands out by offering two distinct annotation paradigms. spoken-to-written style conversion and related datasets spoken-to-written style conversion can be formulated as a monolingual translation task @xcite , mapping from a spoken style text to a normalized written style text. a variety of approaches have been proposed to address these monolingual translation challenges, including noisy channel models and hidden markov models @xcite @xcite used earlier. in recent years, neural sequence conversion models have demonstrated superior performance @xcite . nevertheless, these models require a sizable parallel corpus for training since they learn the relationship between input and output sequences directly in an end-to-end manner. therefore, it is essential to curate a large-scale parallel corpus of spoken and written language. publicly available spoken-to-written style conversion data are scarce, with the majority of datasets being private. @xcite curate the parallel corpus for japanese spoken-to-written style conversion, which annotates not only grammatical errors but also special conversion types in japanese. these types include restoring postpositional particle expressions, hiragana symbols, etc. cs2w is a spoken-written style conversion dataset, where the conversion problems are categorized according to chinese linguistic characteristics. ## dataset curation in this section, we elaborate on the creation of the cs2w dataset. ## dataset analysis we provide data statistics and analyses on conversion type distribution, sentences with multiple conversion problems, and a new subtype of disfluency existing in our dataset, which is absent in previous studies. ## experiment we conducted experiments on the curated dataset to evaluate the performance of the advanced opensource large language models (llms) on chinese spoken-to-written style conversion. ## conclusion in this paper, we have presented a dataset cs2w, which is the first open-source chinese spoken-towritten conversion dataset. the dataset covers four types of conversion problems commonly occurring in chinese spoken texts. we manually annotate the type and span for each conversion problem and provide high-quality written style normalized texts. the dataset is used as a benchmark testbed to evaluate the performance of advanced llms on spoken-to-written style conversion and would promote future research on this underexplored direction.