A Multi-Task Dataset for Assessing Discourse Coherence in C hinese Essays: Structure, Theme, and Logic Analysis (2023)

Abstract: This paper introduces the C hinese E ssay D iscourse C oherence C orpus ( CEDCC ), a multi-task dataset for assessing discourse coherence. Existing research tends to focus on isolated dimensions of discourse coherence, a gap which the CEDCC addresses by integrating coherence grading, topical continuity, and discourse relations. This approach, alongside detailed annotations, captures the subtleties of real-world texts and stimulates progress in Chinese discourse coherence analysis. Our contributions include the development of the CEDCC, the establishment of baselines for further research, and the demonstration of the impact of coherence on discourse relation recognition and automated essay scoring. The dataset and related codes is available at https://github.com/cubenlp/CEDCC_corpus .

## introduction discourse coherence, a fundamental aspect of both language comprehension and generation, involves understanding how words, phrases, sentences, and paragraphs within a text are interconnected to create a cohesive and coherent message. due to its significance, it has been widely incorporated into various natural language processing (nlp) tasks, such as automated essay scoring @xcite , machine translation @xcite ) and text generation @xcite . existing discourse coherence analyses present various frameworks for understanding coherence realization, most of which can be grouped into three categories: (a) overall coherence grading, which assigns a coherence score to a given text, based on its overall logical flow and consistency; (b) topical continuity modelling, which focuses on analysing the distribution and relationship of topics within discourses; and (c) discourse relationship recognition, which involves detecting semantic relations between two segments of a text. each of these elements highlights a distinct aspect of discourse coherence. specifically, for coherence grading, @xcite assert that inherent coherence is a fundamental feature of human-written texts, crucial for the global comprehension of an essay and its quality assessment. in terms of topical continuity, @xcite argue that a coherent text should maintain a logical progression of ideas, eschewing abrupt topic transitions. they primarily utilize topic models to extract and designate topics within a text, thereby quantifying topical coherence. @xcite suggest that text coherence is closely tied to its discourse structure and relations. despite the multifaceted nature of discourse coherence, current research often examines isolated dimensions, overlooking their interplay. for example, most coherence grading studies @xcite often perceive coherence as an automatic scoring task, neglecting aspects like topical continuity and discourse relations, which impairs the interpretability of results. although some studies aim to integrate these dimensions, progress has been hindered by the lack of a comprehensive dataset. for instance, methods proposed by @xcite and @xcite operate in unsupervised or artificially structured contexts, but these fail to capture the complexity of real-world texts. furthermore, the scarcity and limited diversity of chinese discourse coherence datasets have somewhat constrained advancements in this field. to address the shortcomings of existing research, we introduce the chinese essay discourse coherence corpus (cedcc), a multi-task dataset designed to assess essay discourse coherence. the cedcc, sourced from various middle schools, encompasses a diverse range of topics, genres, and regions. each essay has been meticulously annotated by linguistic experts, as depicted in figure 1 . the cedcc addresses key limitations in prior the spirit of perseverance karl marx once said, "life is like the ocean; only those with a strong will can reach the other shore" (sent 1) 。 therefore, individuals who possess the spirit of perseverance are those who can handle significant challenges, withstand storms, and endure hardships (sent 2) . @xcite they have no silver that can be used for bribery @xcite . the ocean is vast because it admits all rivers. a wall stands a thousand feet high because it yields to nothing @xcite . so, it can be said that the person who is constantly selfimproving, just they can rise against the wind in the storm, these people ; have firm belief, relentless pursuit, extraordinary ambition, and unyielding spirit, therefore they are bound to succeed (sent 13) . work: firstly, it breaks from the isolationist approach of previous studies. with the integration of coherence grading, topical continuity, and discourse relations, it provides a more comprehensive understanding of discourse coherence. secondly, the detailed annotations, particularly evident in figure 1 (c), capture the nuances of real-world texts. factors such as off-topic sections, misused connectives, inappropriate clauses, and incoherent logic that influence coherence grade are highlighted. finally, by offering a diverse dataset for chinese discourse coherence analysis, cedcc stimulates progress in this area. our contributions are summarised as follows: ## related work in this section, we delve into three core aspects of discourse coherence. for each, we discuss its objectives, relevant datasets, and their limitations, establishing the backdrop for our proposed dataset. ## corpus construction this section delineates the process of collection and annotation for the chinese essay discourse coherence corpus (cedcc), designed for extensive discourse coherence analysis. ## conclusion in this paper, we introduced the chinese essay discourse coherence corpus (cedcc), a richly annotated, diverse dataset aiming to address limitations in current discourse coherence research. our dataset integrates aspects of coherence grading, topical continuity, and discourse relations, thereby breaking away from the isolated approach of previous studies. we further demonstrated the importance of fine-grained annotations and the role of text coherence in discourse relation recognition. our findings set a benchmark for future discourse analysis research in chinese, potentially sparking further advancements in the field.