ymoslem/acl-abstracts huggingface dataset:
中小学作文修辞识别与理解 (2024)

{\textquotedblleft}本技术报告是对2024CCL评测任务(中小学作文修辞识别与理解评测)的一种解决方案。在中小学生的学习过程中,修辞手法不仅是阅读理解和写作技巧的核心组成部分,同时也是塑造优秀文学作品的不可或缺的元素。识别并理解学生作文中的修辞使用,可以帮助学生提高作文表达能力,指导学生更高质量的叙述和描写。对修辞的识别目前属于自然理解领域比较困难的任务,因为需要用到人类领域的大量先验知识,而且很多时候不同的修辞之间的边界还是模糊的。我们通过lora技术直接微调基于qwen-chat-7B的大语言预训练模型,来进行修辞类别的识别。我们的主要创新技术点为:基于相同的输入输出数据来构造多条训练数据提升算法表现;分级分层来进行修辞的判断,先进行大的修辞类别判断,再把大的修辞类别做为输入对修辞的子类别进行判断;针对修辞成分抽取的任务,直接输出对应的结果文本,再对应回原文本进行位置检索,而不是直接输出索引下标。{\textquotedblright}
Essay Rhetoric Recognition and Understanding Using Synthetic Data and Model Ensemble Enhanced Large Language Models (2024)

{\textquotedblleft}Natural language processing technology has been widely applied in the field of education. Essay writing serves as a crucial method for evaluating students' language skills and logical thinking abilities. Rhetoric, an essential component of essay, is also a key reference for assessing writing quality. In the era of large language models (LLMs), applying LLMs to the tasks of automatic classification and extraction of rhetorical devices is of significant importance. In this paper, we fine-tune LLMs with specific instructions to adapt them for the tasks of recognizing and extracting rhetorical devices in essays. To further enhance the performance of LLMs, we experimented with multi-task fine-tuning and expanded the training dataset through synthetic data. Additionally, we explored a model ensemble approach based on label re-inference. Our method achieved a score of 66.29 in Task 6 of the CCL 2024 Eval, Chinese Essay Rhetoric Recognition and Understanding(CERRU), securing the first position.{\textquotedblright}
基于深度学习模型的中小学作文修辞识别与理解评测 (2024)

{\textquotedblleft}在中小学生的学习进程中,修辞手法是阅读和写作技巧的核心,也是优秀文学作品的关键元素。然而,识别与理解学生文章中的修辞使用需要大量的人工,为教师的作文评估和教学提出了挑战。最近的研究开始使用计算机技术来自动评审作文,其中修辞的使用是评估的重要部分。本文介绍了我们在第二十三届中文计算语言大会中中小学作文修辞识别与理解评测中的所用的参赛方法。在本次评测中,我们针对不同任务,分别使用了传统模型分类模型和大模型,再利用伪标签、数据增强等方法提升模型性能。实验结果表明,我们的方法取得了较为先进的效果。{\textquotedblright}
人类思维指导下大小模型协同决策的中文修辞识别与理解方法 (2024)

{\textquotedblleft}CCL24-Eval任务6提出了一个多层次、细粒度中小学作文修辞识别与理解任务。针对任务特点,本文提出了人类思维指导下大小模型协同决策的中文修辞识别与理解方法。该方法根据人类在面对修辞识别和理解任务时的处理思路,将任务顺序重新定义,并分别选取大小语言模型,使每个步骤的实现效果均达到局部最优,以局部最优达到整体任务的最优效果。结果表明,本文提出的方法能够有效对修辞进行识别与理解,在三个赛道上相较于Baseline方法分别提升了13.54、4.03、57.11。{\textquotedblright}
Chinese Essay Rhetoric Recognition and Understanding (CERRU) (2024)

{\textquotedblleft}Rhetoric is fundamental to the reading comprehension and writing skills of primary and middle school students. However, current work independently recognize single coarse-grained categories or fine-grained categories. In this paper, we propose the CCL24-Eval Task6: Chinese Essay Rhetoric Recognition and Understanding (CERRU), consisting of 3 tracks: (1) Fine-grained Form-level Categories Recognition, (2) Fine-grained Content-level Categories Recognition and (3) Rhetorical Component Extraction. A total of 32 teams registered to participate in CERRU and 9 teams submitted evaluation results, with 7 of these teams achieving an overall score that surpassed the baseline.{\textquotedblright}
Assessing Essay Fluency with Large Language Models (2024)

{\textquotedblleft}With the development of education and the widespread use of the internet, the scale of essay evaluation has increased, making the cost and efficiency of manual grading a significant challenge. To address this, The Twenty-third China National Conference on Computational Linguistics (CCL2024) established evaluation contest for essay fluency. This competition has three tracks corresponding to three sub-tasks. This paper conducts a detailed analysis of different tasks,employing the BERT model as well as the latest popular large language models Qwen to address these sub-tasks. As a result, our overall scores for the three tasks reached 37.26, 42.48, and 47.64.{\textquotedblright}
Multi-Error Modeling and Fluency-Targeted Pre-training for Chinese Essay Evaluation (2024)

{\textquotedblleft}This system report presents our approaches and results for the Chinese Essay Fluency Evaluation (CEFE) task at CCL-2024. For Track 1, we optimized predictions for challenging fine-grained error types using binary classification models and trained coarse-grained models on the Chinese Learner 4W corpus. In Track 2, we enhanced performance by constructing a pseudo-dataset with multiple error types per sentence. For Track 3, where we achieved first place, we generated fluency-rated pseudo-data via back-translation for pretraining and used an NSP-based strategy with Symmetric Cross Entropy loss to capture context and mitigate long dependencies. Our methods effectively address key challenges in Chinese Essay Fluency Evaluation.{\textquotedblright}
中小学作文语法错误检测、病句改写与流畅性评级的自动化方法研究 (2024)

{\textquotedblleft}本研究旨在提高中小学生作文评改的质量和效率,通过引入先进的自然语言处理模型进行作文病句检测、纠正和流畅性评分,并分别针对三个具体的任务进行了模型构建。在任务一中,提出语法错误替换方法进行数据增强,接着基于UTC模型对语病类型进行识别。在任务二中,融合了预训练的BART模型和SynGEC策略进行文本纠错,充分利用了BART的生成能力和SynGEC的语法纠错特性。任务三中,基于TextRCNN-NEZHA模型进行作文流畅性的评级,构建了一个能够综合语义信息的分类器。经评测,本文提出的方法在任务一和任务二中均位列第一,任务三位列第二,即提出的方法可以有效地识别病句类型和纠正作文中的病句,并给出合理的作文流畅性评级。{\textquotedblright}
Prompting GPT-4 for Chinese Essay Fluency Evaluation (2024)

{\textquotedblleft}This report presents the methodology and results of utilizing GPT-4 for CCL24-Eval Task 7 of Chinese Essay Fluency Evaluation (CEFE). The task is divided into three tracks: Identification of Error Sentence Types, Rewriting Error Sentences, and Essay Fluency Rating. We employed a few-shot prompt engineering to guide GPT-4 in performing this task. Our approach integrated fine-grained error analysis with advanced NLP techniques to provide detailed, actionable feedback for students and teachers. Despite some successes, particularly in generating semantically similar and syntactically relevant corrections, our analysis revealed significant challenges, especially in multiple-label classification and the accurate identification of error types. The report discusses these findings and suggests areas for further improvement.{\textquotedblright}
基于大模型数据增强的作文流畅性评价方法 (2024)

{\textquotedblleft}CCL2024-Eval任 务7为 中 小 学 生 作 文 流 畅 性 评 价 (Chinese Essay Fluency Evalua-tion,CEFE),该任务定义了三项重要且富有挑战性的问题,包括中小学作文病句类型识别、中小学作文病句改写、以及中小学作文流畅性评级。本队伍参加了评测任务7的三项子任务,分别获得了45.19、43.90和45.84的得分。本报告详细介绍本队伍在三个子任务上采用的技术方法,并对评测结果进行分析。{\textquotedblright}
Chinese Essay Fluency Evaluation (CEFE) Task (2024)

{\textquotedblleft}This paper presents a detailed review of Task 7 in the CCL24-Eval: the second Chinese Essay Fluency Evaluation (CEFE). The task aims to identify fine-grained grammatical errors that impair readability and coherence in essays authored by Chinese primary and secondary school students, evaluate the essays' fluency levels, and recommend corrections to improve their written fluency. The evaluation comprises three tracks: (1) Coarse-grained and fine-grained error identification; (2) Error sentence rewriting; and (3) Essay Fluency Level Recognition. We garnered 29 completed registrations, resulting in 180 submissions from 10 dedicated teams. The paper discusses the submissions and analyzes the results from all participating teams.{\textquotedblright}
A Two-stage Prompt-Based Strategy for CRMUS Track 1 (2024)

{\textquotedblleft}Large Language Model (LLM) has sparked a new trend in Natural Language Processing, and an increasing number of researchers have recognized the potential of using LLM to unify diverse NLP tasks into a text-generative manner. To explore the potential of LLM for the children`s stories domain, CCL2024 has released the Commonsense Reasoning and Moral Understanding in Children`s Stories (CRMUS) task. This paper presents a straightforward yet effective two-stage prompt-based strategy for the CRMUS Track 1. In the initial stage, we use the same prompt to obtain responses from GPT-4, ERNIE-4, and Qwen-Max. In the subsequent stage, we implement a voting mechanism based on the results from the first stage. For records with inconsistent outcomes, we query GPT-4 for secondary confirmation to determine the final result. Experimental results indicate that our method achieved an average score of 79.27, securing first place in the closed domain among ten participating teams, thereby demonstrating the effectiveness of our approach.{\textquotedblright}
基于指令微调与数据增强的儿童故事常识推理与寓意理解研究 (2024)

{\textquotedblleft}尽管现有语言模型在自然语言处理任务上表现出色,但在深层次语义理解和常识推理方面仍有提升空间。本研究通过测试模型在儿童故事常识推理与寓意理解数据集(CRMUS)上的性能,探究如何增强模型在复杂任务中的能力。在本次任务的赛道二中,本研究使用多个7B以内的开源大模型(如Qwen、InternLM等)进行零样本推理,并选择表现最优的模型基于LoRA进行指令微调来提高其表现。除此之外,本研究还对数据集进行了分析与增强。研究结果显示,通过设计有效的指令格式和调整LoRA微调参数,模型在常识推理和寓意理解上的准确率显著提高。最终在本次任务的赛道二中取得第一名的成绩,该任务的评价指标Acc值为74.38,达到了较为先进的水准。{\textquotedblright}
Exploring Faithful and Informative Commonsense Reasoning and Moral Understanding in Children`s Stories (2024)

{\textquotedblleft}Commonsense reasoning and moral understanding are crucial tasks in artificial intelligence (AI) and natural language processing (NLP). However, existing research often falls short in terms of faithfulness and informativeness during the reasoning process. We propose a novel framework for performing commonsense reasoning and moral understanding using large language models (LLMs), involving constructing guided prompts by incorporating relevant knowledge for commonsense reasoning and extracting facts from stories for moral understanding. We conduct extensive experiments on the Commonsense Reasoning and Moral Understanding in Children`s Stories (CRMUS) dataset with widely recognised LLMs under both zero-shot and fine-tuning settings, demonstrating the effectiveness of our proposed method. Furthermore, we analyse the adaptability of different LLMs in extracting facts for moral understanding performance.{\textquotedblright}
基于提示工程和思维链的提示词构造 (2024)

{\textquotedblleft}儿童故事常识推理与寓意理解评测任务旨在从常识推理和寓意理解两个任务多角度评价中文预训练语言模型和大型语言模型的常识推理和故事理解能力,这考察了模型的常识储备能力以及对文本内容的深入理解能力,因此极具挑战性。随着大语言模型的发展,其卓越的指令跟随能力显著提升了自然语言处理任务的效率和效果。然而,这也对提示词的设计提出了更高的要求,因为提示词的质量直接影响了大模型的表现和预测结果的准确性。因此,设计有效的提示词变得尤为重要,不仅需要理解任务的具体需求,还要具备对语言模型的深入认识和灵活运用能力。本文针对儿童故事常识推理与寓意理解评测赛道一的两个任务,提出了一种基于提示工程的提示词构造方法。首先,我们提出了一种基于融合提示工程、思维链的通用提示词构建框架;然后,我们针对具体的任务调整对应的提示词模板;最后,结合语言模型使用这些提示词进行结果预测。在本次评测中,我们的方法在赛道一的封闭数据条件下获得了第三名的成绩,这验证了我们方法的有效性,并展示了其在自然语言理解领域的应用潜力。{\textquotedblright}
Evaluation of Commonsense Reasoning and Moral Understanding in Children`s Stories (2024)

{\textquotedblleft}This paper provides a comprehensive review of the the CCL24-Eval Task 8: Commonsense Reasoning and Moral Understanding in Children`s Stories(CRMUS). This task has designed two sub-tasks, which aim to assess the commonsense reasoning and implicit meaning comprehension capabilities of Large Language Models(LLMs). We heve received registration forms from 33 teams, 15 of which submitted final results that exceeded the baseline score. We present the results of the top 5 teams and our analysis of these results.{\textquotedblright}
Bridging the Gap between Authentic and Answer-Guided Images for Chinese Vision-Language Understanding Enhancement (2024)

{\textquotedblleft}The objective of the Chinese Vision-Language Understanding Evaluation (CVLUE) is to comprehensively assess the performance of Chinese vision-language multimodal pre-trained models in multimodal modeling and understanding across four tasks: Image-Text Retrieval, Visual Question Answering, Visual Grounding, and Visual Dialog. To enhance the models' performance across various multimodal tasks, this paper propose a multimodal information understanding enhancement method based on answer-guided images. Firstly, we propose task-specific methods for answer-guided image generation. Secondly, the authentic and answer-guided images are fed into the model for multimodal fine-tuning, respectively. Finally, training objectives are set for different tasks to minimize the gap between the answer-guided images and authentic images, thereby supervising the results produced by the authentic images utlizing answer-guided images. The experimental results demonstrate the effectiveness of the proposed method.{\textquotedblright}
Chinese Vision-Language Understanding Evaluation (2024)

{\textquotedblleft}This paper introduces our systems submitted for the Chinese Vision-Language Understanding Evaluation task at the 23rd Chinese Computational Linguistics Conference.In this competition, we utilized X2-VLM and CCLM models to participate in various subtasks such as image-text retrieval, visual grounding, visual dialogue, and visual question answering. Additionally, we employed other models to assess performance on certain subtasks. We optimized our models and successfully applied them to these different tasks{\textquotedblright}
中文图文多模态理解评测 (2024)

{\textquotedblleft}中文图文多模态理解评测任务旨在从多角度评价中文图文多模态预训练模型的图文多模态建模和理解能力。本任务共包括五个子任务:图片检索、文本检索、视觉问答、视觉定位和视觉对话,最终成绩根据这五个任务的得分综合计算。本文首先介绍了任务的背景和动机,然后从任务介绍、评价指标、比赛结果、参赛方法等方面介绍并展示了本次评测任务的相关信息。本次任务共有11支队伍报名参赛,其中3支队伍提交了结果。{\textquotedblright}
维沃手语数字人翻译系统 (2024)

{\textquotedblleft}本文介绍了我们在第二十四届中国计算语言学大会手语数字人翻译质量评测中提交的参赛系统。本次评测任务旨在评测手语数字人将汉语翻译成中国手语方面的自然性和准确性。本文介绍的手语数字人翻译系统首先通过手语翻译算法将汉语文本翻译成手语文本,然后将手语文本对应的手语动作单元运用动作融合算法合成为自然、完整的手语数字人动作,同时借助面部驱动算法将口型、表情等非语言元素自然地融入手语合成中,实现带微表情的和唇形同步的手语数字人。最终,我们在官方手语数字人翻译质量的人工评测集上取得了3.513的综合评分,获得了该任务第一名的成绩。{\textquotedblright}
结合LLM与3D动画技术的手语数字人系统 (2024)

{\textquotedblleft}手语翻译(Sign Language Translation, SLT)系统作为一种重要的辅助技术,为听障人士提供了与他人沟通的有效途径。然而,传统手语翻译系统在准确性、流畅性差等方面存在问题。本文提出了一种结合大语言模型(Large Language Model, LLM)和3D动画技术(3D Animation Technology)的手语翻译系统,旨在克服这些局限,提高翻译的准确性和流畅性。本文详细介绍了系统的设计与实现过程,包括提示词设计、数据处理方法以及手语数字人翻译系统的实现。实验结果表明,采用LLM方法在手语翻译中能够生成较为自然和准确的结果。在标准评估和人工评估的两种评估方法下,本系统在大多数情况下能够较好地完成手语翻译任务,性能优于传统方法。本文的研究为进一步改进手语翻译系统提供了有益的参考和启示。{\textquotedblright}
Translation Quality Evaluation of Sign Language Avatar (2024)

{\textquotedblleft}Sign Language Avatar technology aims to create virtual agents capable of communicating with deaf individuals through sign language, similar to the text dialogue agent ChatGPT but focusing on sign language communication. Challenges in sign language production include limited dataset sizes, information loss due to reliance on intermediate representations, and insufficient realism in generated actions. In this event, we particularly focus on the ability of the Sign Language Avatar to translate spoken language text into sign language that is easily understood by deaf individuals. As the first sign language avatar event held by the China National Conference on Computational Linguistics(CCL), this event attracted wide attention from both industry and academia, with 14 teams registering and 10 of them submitting their system interfaces on time. We provided a dataset consisting of 1074 text-video parallel sentence pairs for training, and the evaluation team comprised proficient Chinese sign language users and professional sign language translators. The scoring method employed a comprehensive evaluation based on multiple metrics, focusing primarily on sign language grammar accuracy, naturalness, readability, and cultural adaptability. The final scores were determined by considering performance across these four aspects. The final scores, taking into account these four aspects, showed that four teams demonstrated good readability, with Vivo Mobile Communication Co., Ltd. ranking first with a score of 3.513 (out of a full score of 5), leading the baseline model by 1.394 points. According to the analysis of the results, most teams used the traditional method of converting text into Gloss sequences before generating sign language. Additionally, some teams experimented with emerging methods, including gloss-free end-to-end training and Large Language Model(LLMs) prompt learning, which also achieved promising results. We anticipate that this event will promote the development of sign language avatar technology and provide higher-quality communication tools for the deaf community. For more information on this task, please visit the website of the CCL24-Eval: Translation Quality Evaluation of Sign Language Avatar Task.{\textquotedblright}
ParsText: A Digraphic Corpus for Tajik-Farsi Transliteration (2024)

Despite speaking dialects of the same language, Persian speakers from Tajikistan cannot read Persian texts from Iran and Afghanistan. This is due to the fact that Tajik Persian is written in the Tajik-Cyrillic script, while Iranian and Afghan Persian are written in the Perso-Arabic script. As the formal registers of these dialects all maintain high levels of mutual intelligibility with each other, machine transliteration has been proposed as a more practical and appropriate solution than machine translation. Unfortunately, Persian texts written in both scripts are much more common in print in Tajikistan than online. This paper introduces a novel corpus meant to remedy that gap: ParsText. ParsText contains 2,813 Persian sentences written in both Tajik-Cyrillic and Perso-Arabic manually collected from blog pages and news articles online. This paper presents the need for such a corpus, previous and related work, data collection and alignment procedures, corpus statistics, and discusses directions for future work.
A Joint Approach for Automatic Analysis of Reading and Writing Errors (2024)

Analyzing the errors that children make on their ways to becoming fluent readers and writers can provide invaluable scientific insights into the processes that underlie literacy acquisition. To this end, we present in this paper an extension of an earlier developed spelling error detection and classification algorithm for Dutch, so that reading errors can also be automatically detected from their phonetic transcription. The strength of this algorithm lies in its ability to detect errors at Phoneme-Corresponding Unit (PCU) level, where a PCU is a sequence of letters corresponding to one phoneme. We validated this algorithm and found good agreement between manual and automatic reading error classifications. We also used the algorithm to analyze written words by second graders and phonetic transcriptions of read words by first graders. With respect to the writing data, we found that the PCUs {\textquoteleft}ei', {\textquoteleft}eu', {\textquoteleft}g', {\textquoteleft}ij' and {\textquoteleft}ch' were most frequently written incorrectly, for the reading data, these were the PCUs {\textquoteleft}v', {\textquoteleft}ui', {\textquoteleft}ng', {\textquoteleft}a' and {\textquoteleft}g'. This study presents a first attempt at developing a joint method for detecting reading and writing errors. In future research this algorithm can be used to analyze corpora containing reading and writing data from the same children.
Tool for Constructing a Large-Scale Corpus of Code Comments and Other Source Code Annotations (2024)

The sublanguage of source code annotations{---}explanatory natural language writing that accompanies programming source code{---}is little-studied in linguistics. To facilitate research into this domain, we have developed a program prototype that can extract code comments and changelogs (i.e. commit messages) from public, open-source code repositories, with automatic tokenization and part-of-speech tagging on the extracted text. The program can also automatically detect and discard {\textquotedblleft}commented-out{\textquotedblright} source code in data from Python repositories, to prevent it from polluting the corpus, demonstrating that such sanitization is likely feasible for other programming languages as well. With the current tool, we have produced a 6-million word corpus of English-language comments extracted from three different programming languages: Python, C, and C++.
Tokenization via Language Modeling: the Role of Preceding Text (2024)

While language models benefit immensely from their capacity to model large context (i.e., sequence of preceding tokens), the role of context is unclear in text tokenization, which is, in many cases, language model-driven to begin with. In this paper, we attempt to explore the role in three different writing systems and using three different text tokenization strategies (word-based, Morfessor, and BPE). In the first experiment, we examined how the size of context used for predicting the next token affects the ranking of the segmentation strategies i.t.o. language model surprisal. This effect was very writing system specific: minimal in case of English, and rank-reversing due to increased context size and token granularity in case of Turkish and Chinese. In the second experiment, we examined how context alters segmentation hypotheses when using language models to identify word boundaries. In this case, the effect was subtle: using context-aware, rather than context-free segment scores improved boundary recognition accuracy by up to 0.5{\%}, once baseline effects were exploited.
Abbreviation Across the World`s Languages and Scripts (2024)

Detailed taxonomies for non-standard words, including abbreviations, have been developed for speech and language processing, though mostly with reference to English. In this paper, we examine abbreviation formation strategies in a diverse sample of more than 50 languages, dialects and scripts. The resulting taxonomy{---}and data about which strategies are attested in which languages{---}provides key information needed to create multilingual systems for abbreviation expansion, an essential component for speech processing and text understanding
Now You See Me, Now You Don`t: \textquoteleftPoverty of the Stimulus' Problems and Arbitrary Correspondences in End-to-End Speech Models (2024)

End-to-end models for speech recognition and speech synthesis have many benefits, but we argue they also face a unique set of challenges not encountered in conventional multi-stage hybrid systems, which relied on the explicit injection of linguistic knowledge through resources such as phonemic dictionaries and verbalization grammars. These challenges include handling words with unusual grapheme-to-phoneme correspondences, converting between written forms like {\textquoteleft}12' and spoken forms such as {\textquoteleft}twelve', and contextual disambiguation of homophones or homographs. We describe the mitigation strategies that have been used for these problems in end-to-end systems, either implicitly or explicitly, and call out that the most commonly used mitigation techniques are likely incompatible with newly emerging approaches that use minimal amounts of supervised audio training data. We review best-of-both-world approaches that allow the use of end-to-end models combined with traditional linguistic resources, which we show are increasingly straightforward to create at scale, and close with an optimistic outlook for bringing speech technologies to many more languages by combining these strands of research.
Towards Fast Cognate Alignment on Imbalanced Data (2024)

Cognate alignment models purport to enable decipherment, but their speed and need for clean data can make them unsuitable for realistic decipherment problems. We seek to draw attention to these shortcomings in the hopes that future work may avoid them, and we outline two techniques which begin to overcome the described problems.
Simplified Chinese Character Distance Based on Ideographic Description Sequences (2024)

Character encoding systems have long overlooked the internal structure of characters. Ideographic Description Sequences, which explicitly represent spatial relations between character components, are a potential solution to this problem. In this paper, we illustrate the utility of Ideographic Description Sequences in computing edit distance and finding orthographic neighbors for Simplified Chinese characters. In addition, we explore the possibility of using Ideographic Description Sequences to encode spatial relations between components in other scripts.