ymoslem/acl-abstracts huggingface dataset:
Prior Constraints-based Reward Model Training for Aligning Large Language Models (2024)

{\textquotedblleft}Reinforcement learning with human feedback for aligning large language models (LLMs) trainsa reward model typically using ranking loss with comparison pairs. However, the training pro-cedure suffers from an inherent problem: the uncontrolled scaling of reward scores during rein-forcement learning due to the lack of constraints while training the reward model. This paperproposes a Prior Constraints-based Reward Model (PCRM) training method to mitigate thisproblem. PCRM incorporates prior constraints{---}specifically, length ratio and cosine similaritybetween outputs of each comparison pair{---}during reward model training to regulate optimiza-tion magnitude and control score margins. We comprehensively evaluate PCRM by examining itsrank correlation with human preferences and its effectiveness in aligning LLMs via RL. Exper-imental results demonstrate that PCRM significantly improves alignment performance by effec-tively constraining reward score scaling. As another bonus, our method is easily integrated intoarbitrary rank-based alignment methods, such as direct preference optimization, and can yieldconsistent improvement. The code is available at https://github.com/wangclnlp/DeepSpeed-Chat-Extension/tree/PCRM.{\textquotedblright}
Prompt Engineering 101 Prompt Engineering Guidelines from a Linguistic Perspective (2024)

{\textquotedblleft}Deploying tuning-free prompting is challenging in engineering practice: it not only requiresusers to engage in cumbersome trials and errors but is also extremely time-consuming,as even a slight change in wording and phrasing could have a huge impact on the finalperformance. To further investigate the impact of different prompts, in this work, weperform a systematic inspection of four factors in linguistics involved in prompt engineering:syntax, semantics, lexicon, and pragmatics. The empirical results quantify the sensitivityof the output to small textual perturbations in four linguistic factors of prompts. Basedon the analysis of these four factors, we present a series of design guidelines to helphuman users write effective prompts. Human evaluation on amateurs shows that usingthe proposed guidelines helps humans produce prompts with significant gains in zero-shotperformance in Pre-trained Language Models (PLMs) and hence validates the utility ofthe guidelines.{\textquotedblright}
从多模态预训练到多模态大模型:架构、训练、评测、趋势概览(From Multi-Modal Pre-Training to Multi-Modal Large Language Models: An Overview of Architectures, Training,) (2024)

{\textquotedblleft}多媒体信息在人类社会的发展历程中有着至关重要的作用,构建具有多模态信息处理能力的智能系统也是通往通用人工智能的必经之路。随着预训练技术的发展以及对于通用模型的需求,多模态的研究也从早期的任务特定的方法转移到了构建统一泛用的多模态基座模型上。初步的统一多模态模型探索受到BERT启发,从表征学习的角度出发构建能为不同下游任务提供有效初始化的多模态预训练模型,这类方法尽管有效但仍然在泛用性方面受限于预训练中微调范式,无法更广泛高效地应用。近年来随着大语言模型的发展,以大语言模型为基座的多模态大模型则展现出了巨大的潜力:此类模型有着强大的信息感知,交互,以及推理能力并且能有效泛化到多样的场景下,为新时代的通用人工智能系统提供了切实可行的思路。本文将从构建统一多模态模型的角度出发,介绍和梳理相关工作的发展,从多模态预训练到多模态大模型,介绍对应的架构,训练,评测方法以及发展趋势,为读者提供一个全面的概览。{\textquotedblright}
大模型工具学习进展与挑战(Challenges and Advances in Tool Learning with Foundation Models) (2024)

{\textquotedblleft}本论文综述了大模型工具学习的最新进展与挑战。工具作为人类智慧和能力的延伸,在提升生产力和解决问题方面至关重要。随着大语言模型(Large Language Models)的突破,工具学习得到了广泛关注,通过动态调用外部工具,显著增强了模型解决复杂问题的能力。本文介绍了一个通用的大模型工具学习框架,包括控制器、工具集、环境和感知器四个核心组件。我们详细探讨了四个关键问题:意图理解、规划、工具使用和记忆管理。在意图理解方面,模型需要准确解析用户的输入和隐含意图。规划能力使模型能够将复杂任务分解为可执行的子任务。工具使用方面,介绍了示范学习、教程学习和探索学习三种主要训练策略,通过观察人类示范、阅读工具手册和直接探索来提升模型能力。记忆管理方面,提出了动态记忆管理和信息优先级管理等方法,以提高模型处理复杂任务的效率和准确性。本文分析了当前大模型工具学习的研究进展和每个领域的挑战,为未来研究提供了有价值的见解。希望通过这篇综述,能帮助研究人员和开发者更好地理解和推进大模型工具学习领域的发展。{\textquotedblright}
大模型逻辑推理研究综述(Survey on Logical Reasoning of Large Pre-trained Language Models) (2024)

{\textquotedblleft}理解自然语言的逻辑结构和关系是机器理解的核心任务,也是人工智能领域的关键研究议题。随着大数据和计算能力的提升,预训练语言模型在逻辑推理方面取得了显著进展,使得大规模模型的逻辑推理能力成为研究的新焦点。本综述旨在全面梳理大模型在逻辑推理领域的研究进展,探讨其对人工智能系统智能水平评估的重要性及其在推动人工智能发展中的作用。 本文首先界定了大模型逻辑推理能力的研究范畴,系统性地讨论了逻辑推理的类型和 特点,并回顾了相关理论的发展,为研究提供了清晰的框架。接着,从任务形式和数 据基准的角度,详细介绍了逻辑推理研究的基础工作,为理解大模型的性能提供了基 准。进一步,本文深入分析了大模型在逻辑推理能力上的现状,通过不同推理类型的 案例研究,展示了大模型的能力表现。同时,本文还探讨了提升大模型逻辑推理能力 的方法,包括预训练、指令微调、解码策略和神经符号混合方法,并对这些方法进行 了比较分析。最后,本文提出了对未来研究方向的展望,旨在激发更多的学术讨论和 探索,推动逻辑推理能力研究的进一步发展。{\textquotedblright}
大模型时代的多语言研究综述(A Survey of Multilingual Research in the Large Language Model Era) (2024)

{\textquotedblleft}进入大语言模型时代以来,传统的多语言研究模式发生了巨大变化。一些传统任务得到了突破性的解决,也出现了多种新任务,以及许多以多语言大模型为基础、面向大模型能力提升的多语言研究工作。本文针对研究领域中的这一新变化,整理归纳了进入大模型时代以来的多语言研究进展,包括多语言大模型、数据集、任务,以及相关的前沿研究方向、研究挑战等,希望能为大模型范式下的多语言研究的未来发展提供参考和帮助。{\textquotedblright}
大语言模型合成数据方法简述(A Brief Introduction to Synthetic Data for Large Language Model) (2024)

{\textquotedblleft}大语言模型在过去两年受到了极大的关注,并引起了对通用人工智能的广泛讨论。为了实现通用人工智能,合成数据被认为是其中非常关键的一环。本文将当前常见的数据合成方法归为三类,基于蒸馏的合成数据、基于模型自我进化、基于工具的合成数据。针对每一类合成数据方法,我们简要介绍了几种主流的做法,以期概览各类方法的基本思路以及异同。当前大部分合成数据方法都基于蒸馏,尽管这些方法取得了良好的效果,但其实质是将更强的大模型蒸馏到更小的大模型。这样的方法从降低大模型推理成本的角度具有实际意义,但对于进一步提升大模型能力上限作用有限。基于模型自我进化和基于工具的合成数据研究相对偏少,对于持续提升模型能力,这两个方向需要有更多探索。{\textquotedblright}
大语言模型时代的信息检索综述(A Review of Information Retrieval in the Era of Large Language Models) (2024)

{\textquotedblleft}以大语言模型为代表的生成式人工智能迅猛发展,标志着人工智能从判别时代向生成时代的转变。这一进步极大地推动了信息检索技术的发展,本文对大语言模型对信息检索领域的影响进行了深入的综述。从性能改进到模式颠覆,逐步展开论述大语言模型对信息检索领域的影响。针对传统信息检索流程,大语言模型凭借强大的语义理解和建模能力,显著增强索引、检索和排序等信息检索模块的性能。同时,文章也探讨了大语言模型可能取代传统信息检索的趋势,并催生了新的信息获取方式,或将是新一次信息时代的寒武纪。此外,大语言模型对内容生态的深远影响也值得关注。{\textquotedblright}
对齐的理论、技术与评估(Theories, Techniques, and Evaluation of AI Alignment) (2024)

{\textquotedblleft}人工智能对齐(AI Alignment)旨在使人工智能系统的行为与人类的意图和价值观相一致。随着人工智能系统的能力日益增强,对齐失败带来的风险也在不断增加。数百位人工智能专家和公众人物已经表达了对人工智能风险的担忧,他们认为乜减轻人工智能带来的灭绝风险应该成为全球优先考虑的问题,与其他社会规模的风险如大流行病和核战争并列(CAIS,2023)。为了提供对齐领域的全面和最新概述,本文深入探讨了对齐的核心理论、技术和评估。首先,本文确定了人工智能对齐的四个关键目标:鲁棒性(Robustness)、可解释性(Interpretability)、可控性(Controllability)和道德性(Ethicality)(RICE)。在这四个目标原则的指导下,本文概述了当前人工智能对齐研究的全貌,并将其分解为两个关键组成部分:前向对齐和后向对齐。本文旨在为对齐研究提供全面且对初学者友好的调研。同时本文还发布并持续更新网站 www.alignmentsurvey.com,该网站提供了一系列教程、论文集和其他资源。更详尽的讨论与分析请见 https://arxiv.org/abs/2310.19852。{\textquotedblright}
基于大语言模型的自主智能体概述(A Survey on Large Language Model based Autonomous Agents Xu Chen) (2024)

{\textquotedblleft}近年来,基于大语言模型的自主智能体受到了学术界和工业界的广泛关注,其关键在于利用大语言模型作为核心控制器,并设计相应的辅助模块增强智能体在动态环境中的演化和适应能力,从而提升智能体自主解决任务的能力。本文通过总结过去工作,抽象出智能体设计的通用范式,并讨论了大模型时代自主智能体能力提升的途径。我们还从个体拓展到系统,深入探讨了多自主智能体系统常见的交互机制和面临的重要问题。{\textquotedblright}
浅谈大模型时代下的检索增强:发展趋势、挑战与展望(Enhancing Large Language Models with Retrieval-Augmented Techniques: Trends, Challenges, and Prospects) (2024)

{\textquotedblleft}大型语言模型(LLM) 在各种自然语言任务上表现出了卓越的性能,但它们很容易受到过时数据和特定领域限制的影响。为了应对这些挑战,研究人员整合不同来源的外部信息来增强大语言模型,具体方法如检索增强等。在本文中,我们综合讨论了检索增强技术的发展趋势,包括检索时机规划、检索技术、以及检索结果的利用。此外,我们介绍了当前可用于检索增强任务的数据集和评价方法,并指出了应用和潜在研究方向。我们希望这项综述能够为社区提供对该研究领域的快速了解和全面概述,以启发未来的研究工作。{\textquotedblright}
生成式文本质量的自动评估方法综述(A Survey of Automatic Evaluation on the Quality of Generated Text) (2024)

{\textquotedblleft}人工评估,作为生成式文本质量评价的金标准,成本太高;自动评估,核心思想在于要使其评估结果与人工评估高度相关,从而实现对生成式文本质量的自动化分析和评价。随着自然语言处理领域相关技术的迭代进步,使得生成式文本质量的自动评估技术,已然经历了多次技术范式的迭代。然而,学界至今依然缺乏对生成式文本质量自动评估技术的系统化总结。因此,本文将首先系统地对已有的生成式文本自动评估方法进行归纳总结,然后分析了生成式文本自动评估方法的主要发展趋势,最后为了使读者更加宏观地了解自动评估整体,对自动评估领域整体的未来研究方向进行了探讨和展望。{\textquotedblright}
Construction of CFSP Model Based on Non-Finetuning Large Language Model (2024)

{\textquotedblleft}Chinese Frame Semantic Parsing (CFSP) is an important task in the field of Chinese Natural Language Processing(NLP). Its goal is to extract the frame semantic structure from the sentence and realize the deep understanding of the events or situations involved in the sentence. This paper mainly studies the application of Large Language Model (LLM) for reasoning through Prompt Engineering without fine-tuning the model, and completes three subtasks of Chinese Framework Semantic Parsing tasks: frame identification, argument Identification and role identification. This paper proposes a Retrieval Augmented Generation (RAG) method for target words, and constructs more refined sample Few-Shot method. We achieved the second place on the B rankings in the open track in the {\textquotedblleft}CCL2024-Eval The Second Chinese Frame Semantic Parsing{\textquotedblright}competition*.{\textquotedblright}
Application of Entity Classification Model Based on Different Position Embedding in Chinese Frame Semantic Parsing (2024)

{\textquotedblleft}This paper addresses three subtasks of Chinese Frame Semantic Parsing based on the BERT and RoBERTa pre-trained models: Frame Identification, Argument Identification, and Role Identification. In the Frame Identification task, we utilize the BERT PLM with Rotary Positional Encoding for the semantic frame classification task. For the Argument Identification task, we employ the RoBERTa PLM with T5 position encoding for extraction tasks. In the Role Identification task, we use the RoBERTa PLM with ALiBi position encoding for the classification task. Ultimately, our approach achieved a score of 71.41 in the closed track of the B leaderboard, securing fourth place and validating the effectiveness of our method.{\textquotedblright}
Leveraging LLMs for Chinese Frame Semantic Parsing (2024)

{\textquotedblleft}We participate in the open track of the Chinese frame semantic parsing (CFSP) task, i.e., CCL24Eval Task 1, and our submission ranks first. FSP is an important task in Natural Language Processing, aiming to extract the frame semantic structures from sentences, which can be divided into three subtasks, e.g., Frame Identification (FI), Argument Identification (AI), and Role Identification (RI). In this paper, we use the LLM Gemini 1.0 to evaluate the three subtasks of CFSP, and present the techniques and strategies we employed to enhance subtasks performance. For FI, we leverage mapping and similarity strategies to minimize the candidate frames for each target word, which can reduce the complexity of the LLM in identifying the appropriate frame. For AI and RI subtasks, we utilize the results from small models as auxiliary information and apply data augmentation, self-training, and model ensemble techniques on these small models to further enhance the performance of subtasks.{\textquotedblright}
Chinese Frame Semantic Parsing Evaluation (2024)

{\textquotedblleft}Chinese Frame-semantic Parsing (CFSP) aims to extract fine-grained frame-semantic structures from texts, which can provide fine-grained semantic information for natural language understanding models to enhance their abilities of semantic representations. Based on the CCL-23 CFSP evaluation task, we introduce construction grammar to expand the targets, as basic units activating frames in texts, from word-style to construction-style, and publish a more challenging CFSP evaluation task in CCL-2024. The evaluation dataset consists of 22,000 annotated examples involving nearly 695 frames. The evaluation task is divided into three subtasks: frame identification, argument identification, and role identification, involving two tracks: close track and open track. The evaluation task has attracted wide attention from both industry and academia, with a total of 1988 participating teams. As for the evaluation results, the team from China University of Petroleum won the first place in the closed track with the final score of 71.34, while the team frome Suzhou University won the first place in the open track with the final socre of 48.77. In this article, we reports the key information about the evaluation task, including key concepts, evaluation dataset, top-3 results and corresponding methods. More information about this task can be found on the website of the CCL-2024 CFSP evaluation task.{\textquotedblright}
基于多个大语言模型微调的中文意合图语义解析 (2024)

{\textquotedblleft}中文意合图对句中成分间的关系进行层次化标注,能有效表示汉语的深层语义结构。传统方法难以对中文意合图中的特殊成分进行特征表示,而近期大语言模型性能的快速提高为复杂自然语言处理任务提供了一种全新思路。在本次任务中,我们尝试使用Prompt-Response方式对大模型进行LoRA微调,让大模型根据输入直接生成格式化的中文意合图三元组序列。我们广泛测试来自不同研发团队、拥有不同参数规模的七个主流大模型,评估基座模型、参数规模、量化训练等因素对微调后模型性能的影响。实验表明,我们的方法展现出远超依存模型的性能,在测试集和盲测集上的F1分别为0.6956和0.7206,获得了本次评测榜一的成绩。{\textquotedblright}
Chinese Parataxis Graph(CPG) Parsing Based on Large Language Models (2024)

{\textquotedblleft}This paper presents the work submitted for the 23rd China National Conference on Computational Linguistics(Evaluation Workshop)(CCL24-Eval), focusing on the Chinese Parataxis Graph (CPG) Parsing task. CPG represents Chinese natural language hierarchically through relational triplets, providing a consistent representation for linguistic units of varying levels. Our approach has used large-scale language models through full fine-tuning, achieving the result with F1 value at 71.6{\%} in the contest and 74.76{\%} after the contest. Furtehrmore, our team has proposed a combined model that integrates multiple LoRA fine-tuned medium-scale models after the contest. This approach is able to minimize the time and space consumption while keeping the performance of CPG construction task relatively high.{\textquotedblright}
基于关系抽取的中文意合图语义解析方法研究 (2024)

{\textquotedblleft}意合图是以事件为中心的单根有向语义表征图,在语义计算与应用方面具有重要价值。在乃乃乌中串丰串临中文意合图语义解析评测任务中,为克服意合图为单根有向图、意合图包含隐性事件词以及意合图的语义关系类型十分丰富,导致关系类型过多等诸多方面的难点,本文提出一种将该任务转换为关系抽取的方法。该方法首先对标签进行扩充,分为正向标签和反向标签;其次,对输入进行扩充,将隐性事件词添加到输入中,无须额外对隐性事词进行预测;最后,细分为不带隐性事件词和带隐性事件词的关系抽取任务。实验结果表明,本文方法在官方盲测集上的F1值为64.44{\%},高出基线模型33.41{\%},证明了本文方法的有效性。{\textquotedblright}
基于样本设计工程和大模型微调的中文意合图语义解析\ensuremath* (2024)

{\textquotedblleft}本文介绍了我们在第二十三届中国计算语言学大会中文意合图语义解析评测中提交的参赛系统。中文意合图(Chinese Parataxis Graph,CPG)是以事件为中心的语义表征图,可以对不同层级的语言单元作一贯式表示,是一种通用性与扩展性兼具的语义表征方法。鉴于大语言模型在语义解析任务中的优越性能,我们对Llama3-Chinese-8B-Instruct模型进行了LoRA微调,使其能够生成结构化的意合图表征三元组,并采用了样本设计工程(Sample Design Engineering,SDE)技巧进行微调样本的设计。此外,我们还对不同标签进行了分类微调,探究大模型在不同语义标签预测能力上的差异。最终,我们的参赛系统在任务发布的评测集上F1值达到0.6461,在本次评测任务中获得了第三名的成绩。{\textquotedblright}
中文意合图语义解析评测 (2024)

{\textquotedblleft}中文意合图是近年提出的中文语义表示方法。本次评测是首次基于意合图理论的语义分析评测,旨在探索面向意合图理论的语义计算方法,评估机器的语义分析能力。本次评测共有14支队伍报名,最终有7支队伍提交结果,其中有5支队伍提交技术报告与模型,均成功复现。在评测截止时间内,表现最好的队伍使用大语言模型LoRA微调方法获得了F1值为72.06{\%}的成绩。在最终提交技术报告的5支队伍中,有4支队伍使用了大语言模型微调方法,在一定程度上表明了目前技术发展的趋势。{\textquotedblright}
基于参数高效微调与半监督学习的空间语义理解 (2024)

{\textquotedblleft}本文介绍了我们在第二十三届中文计算语言大会的第四届中文空间语义理解评测任务中提交的参赛模型。该任务旨在测试机器的中文语义理解水平。现有研究显示,机器的中文语义理解水平与人类平均水平相比仍有较大差距。近年来,生成式大规模语言模型在自然语言处理任务中展现了出色的生成和泛化能力。在本次评测中,我们采用了对Qwen1.5-7b模型进行高效微调的方法,以端到端的形式实现空间语义的推理过程,并结合prompt优化和半监督学习提升推理表现。实验结果表明,我们的模型在该任务中取得了领先的效果。{\textquotedblright}
基于大型语言模型的中文空间语义评测 (2024)

{\textquotedblleft}本研究的任务旨在让大模型进行实体识别、角色识别、异常识别、信息推理、同义识别任务,综合评估大模型的空间语义理解能力。其中,我们使用普通提示词、工作流提示词和思维链三种提示词策略来探讨大模型的空间语义理解能力,最后发现ERNIE-4在1-shot的普通提示词上表现最佳。最终,我们的方法排名第六,总体准确率得分为56.20{\%}。{\textquotedblright}
基于上下文学习与思维链策略的中文空间语义理解 (2024)

{\textquotedblleft}本技术报告详细介绍了我们团队参加第四届中文空间语义理解评测(SpaCE2024)的方法和成果。SpaCE2024旨在全面测试机器对中文空间语义的理解能力,包括空间信息实体识别、空间信息实体识别、空间信息异常识别、空间方位信息推理和空间异形同义识别五个不同的任务。我们团队采用精心设计的prompt并结合微调的方式激发大语言模型的空间语义理解能力,构建了一个高效的空间语义理解系统。在最终的评估中,我们在空间信息实体识别题目中准确率为0.8947,在空间信息实体识别题目中准确率为0.9364,在空间信息异常识别题目中准确率为0.8480,在空间方位信息推理题目中准确率为0.3471,在空间异形同义识别题目中准确率为0.5631,测试集综合准确率为0.6024,排名第一。{\textquotedblright}
基于上下文学习的空间语义理解 (2024)

{\textquotedblleft}空间语义理解任务致力于使语言模型能够准确解析和理解文本中描述的物体间的空间方位关系,这一能力对于深入理解自然语言并支持复杂的空间推理至关重要。本文聚焦于探索大模型的上下文学习策略在空间语义理解任务上的有效性,提出了一种基于选项相似度与空间语义理解能力相似度的样本选择策略。本文将上下文学习与高效微调融合对开源模型进行微调,以提高大模型的空间语义理解能力。此外,本文尝试结合开源模型和闭源模型的能力处理不同类型的样本。实验结果显示,本文所采用的策略有效地提高了大模型在空间语义理解任务上的性能。{\textquotedblright}
The Fourth Evaluation on Chinese Spatial Cognition (2024)

{\textquotedblleft}The Fourth Chinese Spatial Cognition Evaluation Task (SpaCE 2024) presents the first comprehensive Chinese benchmark to assess spatial semantic understanding and reasoning capabilities of Large Language Models (LLMs). It comprises five subtasks in the form of multiple-choice questions: (1) identifying spatial semantic roles; (2) retrieving spatial referents; (3) detecting spatial semantic anomalies; (4) recognizing synonymous spatial expression with different forms; (5) conducting spatial position reasoning. In addition to proposing new tasks, SpaCE 2024 applied a rule-based method to generate high-quality synthetic data with difficulty levels for the reasoning task. 12 teams submitted their models and results, and the top-performing team attained an accuracy of 60.24{\%}, suggesting that there is still significant room for current LLMs to improve, especially in tasks requiring high spatial cognitive processing.{\textquotedblright}
面向中文抽象语义表示解析的大模型评估与增强 (2024)

{\textquotedblleft}本文介绍了我们在第二十三届中文计算语言学大会中文抽象语义表示解析评测任务中提交的参赛系统。中文抽象语义表示(Chinese Abstract Meaning Representa-tion,CAMR)以一个单根可遍历的有向无环图表示中文句子的语义。本系统选择大语言模型作为解决方案。我们首先系统地评估了当下中文大语言模型在AMR解析任务上的性能,在此基础上基于图融合算法整合性能较高的大模型预测结果,最终得到预测的CAMR图。实验结果表明,1)现有大模型已经具备一定的少样本中文AMR解析能力;2)基于微调中文大模型的AMR解析系统能够取得相较以往最优系统更强的性能;3)图融合算法能够进一步增强基于大模型的CAMR解析系统的性能。{\textquotedblright}
混合 LoRA 专家的中文抽象语义表示解析框架 (2024)

{\textquotedblleft}本文介绍了我们在第二十三届中国计算语言学大会中文抽象语义表示解析评测任务中提交的参赛系统。抽象语义表示 (Abstract Meaning Representation,AMR) 使用有向无环图对句子进行建模,以语义概念作为节点,关系标签作为边,表示一个句子的语义。我们受到结合语法信息的 AMR 解析研究的启发,提出混合 LoRA(Low-Rank Adaption) 专家的 CAMR 解析框架,该框架包含一个由大型语言模型微调而来的基础 CAMR 解析器和 4 个句类专家和 1 个古汉语 LoRA 专家模型。最终,本文所提出的框架在三个评测数据集中均取得了最好的成绩。{\textquotedblright}
A Two-stage Generative Chinese AMR Parsing Method Based on Large Language Models (2024)

{\textquotedblleft}The purpose of the CAMR task is to convert natural language into a formalized semantic representation in the form of a graph structure. Due to the complexity of the AMR graph structure, traditional AMR automatic parsing methods often require the design of complex models and strategies. Thanks to the powerful generative capabilities of LLMs, adopting an autore-gressive generative approach for AMR parsing has many advantages such as simple modeling and strong extensibility. To further explore the generative AMR automatic parsing technology based on LLMs, we design a two-stage AMR automatic parsing method based on LLMs in this CAMR evaluation. Specifically, we design two pipeline subtasks of alignment-aware node generation and relationship-aware node generation to reduce the difficulty of LLM understanding and generation. Additionally, to boost the system`s transferability, we incorporate a retrieval-augmented strategy during both training and inference phases. The experimental results show that the method we proposed has achieved promising results in this evaluation.{\textquotedblright}
The Fourth Chinese Abstract Meaning Representation Parsing Evaluation (2024)

{\textquotedblleft}Abstract Meaning Representation has become a key research area in sentence-level semantic parsing within natural language processing. Substantial progress has been achieved in various NLP tasks using AMR. This paper presents the fourth Chinese Abstract Meaning Representation parsing evaluation, held during the technical evaluation task workshop at CCL 2024. The evaluation also introduced a new test set comprising Ancient Chinese sentences. Results indicated decent performance, with the top team achieving an F1 of 0.8382 in the open modality, surpassing the previous record at CoNLL 2020 by 3.30 percentage points under the MRP metric. However, current large language models perform poorly in AMR parsing of Ancient Chinese, highlighting the need for effective training strategies. The complex syntax and semantics of Ancient Chinese pose significant challenges. Additionally, optimizing transfer learning techniques to better apply knowledge from Chinese Mandarin to Ancient Chinese parsing is crucial. Only through continuous innovation and collaboration can significant advancements in both Ancient Chinese and Chinese Mandarin AMR parsing be achieved.{\textquotedblright}
基于大小模型结合与半监督自训练方法的古文事件抽取 (2024)

{\textquotedblleft}本文描述了队伍{\textquotedblleft}TeleAI{\textquotedblright}在CCL2024古文历史事件类型抽取评测任务(CHED2024)中提交的参赛系统。该任务旨在自动识别出古代文本中的事件触发词与事件类型,其中事件类型判别被分为粗粒度和细粒度的事件类型判别两部分。为了提高古文历史事件类型抽取的性能,我们结合了大模型和小模型,并采用了半监督自训练的方法。在最终的评估中,我们在触发词识别任务得分0.763,粗粒度事件类型判别任务得分0.842,细粒度事件类型判别任务得分0.779,综合得分0.791,在所有单项任务和综合评分上均排名第一。{\textquotedblright}
Multi-Model Classical Chinese Event Trigger Word Recognition Driven by Incremental Pre-training (2024)

{\textquotedblleft}This paper addresses the task of identifying and classifying historical event trigger words in Classical Chinese, utilizing both small-scale and large-scale language models. Specifically, we selected the small-scale language model GujiBERT for intelligent processing of classical texts, and the large-scale language model Xunzi-Qwen-14b. Both models underwent continued pretraining and fine-tuning, resulting in GujiBERT-CHED-mlm and Xunzi-Qwen-14b-CHED. For the small-scale language model, we used a BiLSTM as the feature extraction module and a CRF as the decoding module, employing a sequence labeling paradigm to complete the evaluation experiments. For the large-scale language model, we optimized the prompt templates and used a sequence-to-sequence paradigm for evaluation experiments. Our experiments revealed that GujiBERT-BiLSTM-CRF achieved the best performance across all tasks, ranking fourth in overall performance among all participating teams. The large-scale language model demonstrated good semantic understanding abilities, reaching a preliminary usable level. Future research should focus on enhancing its ability to produce standardized outputs.{\textquotedblright}
基于增量预训练与外部知识的古文历史事件检测 (2024)

{\textquotedblleft}古文历史事件检测任务旨在识别文本中的事件触发词和类型。为了解决传统pipeline方法容易产生级联错误传播,以及大多数事件检测方法仅依赖句子层面信息的问题,本文提出了一种结合外部信息和全局对应矩阵的联合抽取模型EIGC,以实现触发词和事件类型的精确抽取。此外,本文还整理了一个包含{\textquotedblleft}二十四史{\textquotedblright}等古汉语文献的数据集,共计约97万条古汉语文本,并利用该文本对BERT-Ancient-Chinese进行增量预训练。最终,本文所提出的模型在三个任务上的总F1值达到了76.2{\%},验证了该方法的有效性。{\textquotedblright}
Classical Chinese Historical Event Detection Evaluation (2024)

{\textquotedblleft}Event detection involves identifying and extracting event information from natural language texts. The complex syntax and semantics of Classical Chinese, coupled with its limited usage, pose significant challenges for information extraction tasks on classical Chinese texts. At the 23rd China National Conference on Computational Linguistics (CCL 2024), we launched an evaluation task focused on the extraction of historical events from Classical Chinese. We used our constructed Classical Chinese Historical Event Logical Schema to identify event triggers and classify event types. The evaluation utilized the Classical Chinese Historical Event Detection Dataset (CHED), annotated from The Twenty-Four Histories corpus, with the aim of enhancing event extraction technologies and advancing the digital study of classical Chinese historical texts. The evaluation included two subtasks and attracted 28 teams, with 15 teams submitting valid results. In the subtask of trigger identification, the best-performing system achieved an Exact match score of 63.6{\%}. In the subtasks of coarse-grained and fine-grained event type classification, the top systems achieved F1-scores of 84.5{\%} and 81.4{\%}, respectively.{\textquotedblright}
A Unified Multi-Task Learning Model for Chinese Essay Rhetoric Recognition and Component Extraction (2024)

{\textquotedblleft}In this paper, we present our system at CCL24-Eval Task 6: Chinese Essay Rhetoric Recognition and Understanding (CERRU). The CERRU task aims to identify and understand the use of rhetoric in student writing. The evaluation set three tracks to examine the recognition of rhetorical form, rhetorical content, and the extract of rhetorical components. Considering the potential correlation among the track tasks, we employ the unified multi-task learning architecture to fully incorporate the inherent interactions among the related tasks to improve the overall performance and to complete the above 3 track tasks with a single model. Specifically, the framework mainly consists of four sub-tasks: rhetorical device recognition, rhetorical form recognition, rhetorical content recognition, and rhetorical component extraction. The first three tasks are regarded as multi-label classification tasks, and the last task is regarded as an entity recognition task. The four tasks leverage potential information transfer to achieve fusion learning. Finally, the above four sub-tasks are integrated into a unified model through parameter sharing. In the final evaluation results, our system ranked fourth with a total score of 60.14, verifying the effectiveness of our approach.{\textquotedblright}