ymoslem/acl-abstracts huggingface dataset:
Self-Guide:一种基于自我规划的大语言模型推理增强方法(Self-Guide: Enhancing LLM Reasoning Ability via Self-Plan) (2024)

{\textquotedblleft}尽管大语言模型在自然语言处理任务中取得显著进展,但其在复杂问题推理等领域还面临着认知负荷问题,即大语言模型在推理过程需要记忆并处理大量信息。因此,如何有效地减少语言模型推理过程中的认知负荷,缓解推理过程中可能出现的认知过载是一个亟待解决的问题。对此本文提出了Self-Guide方法,用于增强语言模型的推理能力。该方法通过指引大语言模型生成常识知识和推理指导,让语言模型基于自我规划来增强其推理能力,并通过与推理链结合的方式对模型的推理过程进行校准。与现有方法不同的是,本文在不对大语言模型进行微调或使用外部工具的情况下,显著提升了语言模型的推理性能。实验结果表明,Self-Guide方法在四种常见推理任务上性能显著优于基线方法,同时相比传统的推理链模型,Self-Guide方法在推理能力较弱的模型上也具有良好的泛化性能。通过结合大语言模型的自我规划和推理能力,Self-Guide方法为提升语言模型的推理能力提供了一种新的有效途径。{\textquotedblright}
基于大模型的交互式谎言识别:数据和模型(Unveiling Lies: Enhancing Large Language Models for Real-World Lie Detection in Interactive Dialogues) (2024)

{\textquotedblleft}面向对话交互过程的谎言识别技术在不同的应用场景有广泛的应用需求。现有的鉴谎技术往往在整体的对话级别上给出最终决策,而缺乏对细粒度谎言特征和线索的逻辑分析,难以满足场景中对于可解释性的需求。本文提出了谎言指征和语义不一致线索的概念,用于帮助识别对话中的谎言,提升鉴谎方法的可解释性。文章同时提出一个谎言识别框架,用于训练谎言识别大语言模型(LD-LLM)。它利用细粒度的谎言指征并且发现对话中是否存在语义不一致线索,以实现更可靠的谎言识别。文章在真实交互场景中构建了两个谎言识别数据集FinLIE和IDLIE,分别关注金融风控场景和身份识别场景。实验结果表明,基于这两个数据集创建的指令数据集微调得到的LD-LLM,在基于真实交互的谎言识别上达到了最先进的水平。{\textquotedblright}
基于动态聚类与标签空间映射的上下文学习模板构建方法(In-Context Learning Demonstration Construction Method based on Dynamic Clustering and Label Space Mapping) (2024)

{\textquotedblleft}面向大语言模型提供自然语言指令,可生成预期输出,体现了其上下文学习能力。上下文学习的性能与上下文模板质量密切相关,现有的工作通常使用单一的选择算法进行模板构建,无法充分激发上下文学习能力。本文提出基于动态聚类与标签空间映射的上下文学习模板构建方法,动态选择相关示例,进一步提出聚类筛选方法,实现不同语义簇中示例多样化的选择。设计基于损失函数的排序选择方法,评估模板学习正确标签空间映射分布的能力,排序形成最终模板。在自然语言推理等任务中的实验结果表明,本文提出的方法使两个不同的大语言模型准确率最高分别提升3.2{\%}和8.9{\%}。{\textquotedblright}
基于领域信息分解式学习的大语言模型修辞认知增强方法(Method for Enhancing Rhetorical Cognition of Large Language Models Based on Decomposed Learning of Field Information) (2024)

{\textquotedblleft}中文修辞手法多样且概念差异性大,大语言模型对部分修辞手法的认知存在缺陷。针对该问题,本文研究如何增强大语言模型的修辞认知能力,并探究其与修辞识别性能之间的关系。为此,本文提出了QAKAG框架,此框架首先引入信息分解式学习思想,通过问答形式检测大语言模型的修辞认知缺陷,然后以四种不同的知识组合方式探究最优信息补充机制,实现了大语言模型修辞认知能力的增强。本文构建了多类别中文修辞句数据集MCRSD和修辞知识库MCRKB,并在ChatGPT4等六个大语言模型上开展实验研究,验证了QAKAG框架对增强大语言模型修辞认知能力的有效性以及其各阶段的必要性。结果表明,在QAKAG框架的增强下,六个大语言模型在多类别修辞识别任务上的性能相较直接回答识别问题的平均F1值提高22.1{\%},优于Zero-shot-CoT、RAG-BaiKe、Few-Shot5提示策略。{\textquotedblright}
基于中间层对齐的异构师生模型知识蒸馏(Knowledge distillation of heterogeneous teacher-student model with intermediate layer loss) (2024)

{\textquotedblleft}知识蒸馏技术作为大语言模型时代的一项前沿模型压缩策略,通过将复杂模型的知识有效迁移至简单模型,显著降低了模型的参数规模和计算成本。尽管如此,目前主流的生成式大语言模型蒸馏算法主要集中于优化师生模型间的最后输出层损失,而忽视了对模型中间层的探索。此外,针对中间层蒸馏的研究往往对师生模型的结构一致性有着严格的要求,无法处理异构模型间的蒸馏问题,从而存在明显的局限性。针对这些问题,我们提出了一种新的知识蒸馏算法:引入了中间层蒸馏损失的异构生成式师生大语言模型知识蒸馏算法。该算法首先提取师生模型的中间层信息作为蒸馏对象,随后通过专门设计的中间层映射规则和对齐模块,实现异构模型间基于中间层的知识对齐与损失计算。最后,联合优化各个蒸馏损失的比例。通过在五个相关数据集上进行实验验证,我们的方法在提高蒸馏效果方面展现出显著优势。{\textquotedblright}
面向小规模大语言模型推理优化的推理路径排序方法(A Reasoning Paths Ranking Method for Reasoning Optimization of Small-scale Large Language Models) (2024)

{\textquotedblleft}尽管大语言模型(LLM)在自然语言处理领域取得巨大成功,但是伴随其千亿级参数 规 模 的 训 练 也 产 生 了 巨 大 的 计 算 成 本 。 小 规 模 大 语 言 模 型(SLLM)作 为 低 资 源场景下实现LLM部署的可替代方案,任务处理能力与LLM尚存在明显差距。尽管上下文学习(ICL)等提示方法在一定程度上提升了SLLM的问题处理能力,但基于人工构建的提示往往需要参与者具备特定的专业领域知识,这给LLM的普适推广带来了挑战。针对以上问题,本文提出了一个基于SLLM的问题推理框架,通过在推理路径生成和答案生成两个阶段之间引入基于逐步语义验证器(SSVRP)的推理路径排序选择机制,在无人干预情况下实现SLLM推理能力提升。实验结果表明,SSVRP有效地增强了SLLM的推理性能,在4个推理任务中的平均准确率分别达到了54.3{\%},90.6{\%},64.3{\%}和63.7{\%},并在其中3个推理任务中都取得了最新的SOTA结果。{\textquotedblright}
面向中文实体识别的Transformers模型句子级非对抗鲁棒性研究(On Sentence-level Non-adversarial Robustness of Chinese Named Entity Recognition with Transformers Model) (2024)

{\textquotedblleft}基于Transformers的中文实体识别模型在标准实体识别基准测试中取得了卓越性能,其鲁棒性研究也受到了广泛关注。当前,中文实体识别模型在实际部署中所面临的句子级非对抗鲁棒性问题研究不足,该文针对该问题开展了研究。首先,该文从理论上分析并发现了Transformer中自注意力、相对位置嵌入及绝对位置嵌入对模型鲁棒性的负面影响。之后,提出了实体标签增强和滑动窗口约束的鲁棒性增强方法,并从理论上证明了提出方法能够提升Transformers模型的实体识别鲁棒性。最后,通过在3个中文数据集的实验,研究了4种基于Transformer的实体识别模型的脆弱性,所提出方法使模型的鲁棒性F1值提升最高可达4.95{\%}。{\textquotedblright}
银瞳:基于自适应语义空间学习的中文金融多任务大模型(SilverSight: A Multi-Task Chinese Financial Large Language Model Based on Adaptive Semantic Space Learning) (2024)

{\textquotedblleft}大语言模型正逐渐被用于各种垂直领域,利用其广泛的知识储备来赋能领域中的多种场景。然而,各领域拥有多种待学习的特定任务,且多源异构的领域数据容易引发模型进行任务迁移时的冲突。基于此,本研究提出自适应语义空间学习框架,利用对语义空间内数据的自适应重分布,提升多专家模型的性能及选择效果,并基于此框架训练了一个金融多任务大模型{\textquotedblleft}银瞳{\textquotedblright}。研究结果表明,我们的框架只需利用10{\%}的数据就能达到接近全数据训练的效果,并拥有较强的泛化表现。{\textquotedblright}
Enhancing Free-Form Table Question Answering Models by Distilling Relevant-Cell-Based Rationales (2024)

{\textquotedblleft}Free-form table question answering is a challenging task since tables contain structured contentscompared to plain texts, which requires high-level reasoning abilities to effectively identify cellsthat are relevant to the question and produce a correct and faithful answer based on their relations.Large language models (LLMs) have exhibited remarkable reasoning capabilities in numerousNLP applications. However, in some specific tasks, specially-trained small models can still out-perform LLMs. Furthermore, small models require extremely less computation costs comparedto LLMs. To leverage the strengths of both types of models, we propose a Relevant-Cell-basedKnowledge Distillation with inference-time Teacher Guidance (RCKD-TG) method. This ap-proach aims to combine small free-form table question answering models' abilities to learn fromhuman annotations and large language models' abilities to effectively reason from table contents,via applying Relevant-Cell-based rationales distilled from LLMs to small models' training andinference stages. Our experiments demonstrate the superiority of our method over vanilla smallmodels in correctness, faithfulness, adequacy and fluency, also over general LLMs in adheringto the style of human annotations. We achieve state-of-the-art performance on FeTaQA, a rep-resentative free-form table question answering benchmark. Our result of a 41.3 BLEU scoredemonstrates the feasibility of effectively using small models' task-specific abilities and LLMs`reasoning capabilities at the same time. Additionally, our method exhibits high computation ef-ficiency and data efficiency. Compared to strong baselines, we achieve better performance withsignificantly less training data.{\textquotedblright}
Enhancing Sequence Representation for Personalized Search (2024)

{\textquotedblleft}The critical process of personalized search is to reorder candidate documents of the current querybased on the user`s historical behavior sequence. There are many types of information containedin user historical information sequence, such as queries, documents, and clicks. Most existingpersonalized search approaches concatenate these types of information to get an overall userrepresentation, but they ignore the associations among them. We believe the associations ofdifferent information mentioned above are significant to personalized search. Based on a hierar-chical transformer as base architecture, we design three auxiliary tasks to capture the associationsof different information in user behavior sequence. Under the guidance of mutual information,we adjust the training loss, enabling our PSMIM model to better enhance the information rep-resentation in personalized search. Experimental results demonstrate that our proposed methodoutperforms some personalized search methods.{\textquotedblright}
Joint Similarity Guidance Hash Coding Based on Adaptive Weight Mixing Strategy For Cross-Modal Retrieval (2024)

{\textquotedblleft}There is a continuous and explosive growth of multimodal data. Efficient cross-modal hash-ing retrieval is of significant importance in conserving computational resources.To further en-hance the attention to informative data within modalities and capture the semantic correlationsin cross-modal data, we propose an enhanced deep Joint-Semantics Reconstructing Hashing al-gorithm, which is the Joint Similarity Guidance Hash Coding Based on Adaptive Weight MixingStrategy(JSGHCA). The algorithm focuses on delving deeper into the correlations of the data incross-modal. We introduce the adaptive weight mixing strategy to construct the semantic affinitymatrix, so that the matrix can identify each modal data with specific weight in each batch. Atthe same time, in the process of the hash code generation, we introduce collaborative attentionmechanism. It helps the model to pay more attention to the local information of each modality,thereby capturing the semantic features within each modality more accurately. Additionally, itenables the model to jointly process the attention across different modalities and extract sharedsemantic features more precisely. Experimental results show that the proposed model is signifi-cantly better than the deep joint semantic reconstruction hash algorithm on multiple benchmarkdatasets.{\textquotedblright}
Generate-then-Revise: An Effective Synthetic Training Data Generation Framework For Event Detection Retrieval (2024)

{\textquotedblleft}Large language models (LLMs) struggle with event detection (ED) due to the structured and vari-able number of events in the output. Existing supervised approaches rely on a large amount ofmanually annotated corpora, facing challenges in practice when event types are diverse and theannotated data is scarce. We propose Generate-then-Revise (GtR), a framework that leveragesLLMs in the opposite direction to address these challenges in ED. GtR utilizes an LLM to gen-erate high-quality training data in three stages, including a novel data revision step to minimizenoise in the synthetic data. The generated data is then used to train a smaller model for evalua-tion. Our approach demonstrates significant improvements on the low-resource ED. We furtheranalyze the generated data, highlighting the potential of synthetic data generation for enhancingED performance.Introduction{\textquotedblright}
E3: Optimizing Language Model Training for Translation via Enhancing Efficiency and Effectiveness (2024)

{\textquotedblleft}In the field of Natural Language Processing (NLP), Large-scale Language Models (LLMs) havedemonstrated exceptional capabilities across a variety of tasks, including question answering,classification, and particularly, natural language understanding. The integration of neural ma-chine translation with LLMs presents significant potential, transforming the paradigms of cross-lingual communication and information exchange. This study investigates the foundational as-pects of LLMs' translation abilities and identifies effective training methodologies to equip themwith multilingual capacities. We specifically explore the optimal timing for introducing trans-lation capabilities to LLMs via supervised tasks, considering the inherent bilingual nature ofmachine translation. Key questions explored include whether it is more beneficial to integratemultiple languages during the pre-training or supervised fine-tuning (SFT) stages, how varia-tions in language ratios influence LLMs' translation abilities, and whether longer or shorter textsare more effective for training these models. This research conducts a thorough investigationby training multiple LLMs from scratch with parameter scales in the billions and enhances therobustness of our findings by upgrading the language capabilities of pre-trained open-sourcemodels with parameter scales reaching tens of billions. The aim is to provide a detailed analysisthat elucidates the complexities of augmenting machine translation capabilities within LLMs.{\textquotedblright}
Multi-features Enhanced Multi-task Learning for Vietnamese Treebank Conversion (2024)

{\textquotedblleft}Pre-trained language representation-based dependency parsing models have achieved obviousimprovements in rich-resource languages. However, these model performances depend on thequality and scale of training data significantly. Compared with Chinese and English, the scale ofVietnamese Dependency treebank is scarcity. Considering human annotation is labor-intensiveand time-consuming, we propose a multi-features enhanced multi-task learning framework toconvert all heterogeneous Vietnamese Treebanks to a unified one. On the one hand, we exploitTree BiLSTM and pattern embedding to extract global and local dependency tree features fromthe source Treebank. On the other hand, we propose to integrate these features into a multi-tasklearning framework to use the source dependency parsing to assist the conversion processing.Experiments on the benchmark datasets show that our proposed model can effectively convertheterogeneous treebanks, thus further improving the Vietnamese dependency parsing accuracy byabout 7.12 points in LAS.{\textquotedblright}
SimCLNMT: A Simple Contrastive Learning Method for Enhancing Neural Machine Translation Quality (2024)

{\textquotedblleft}Neural Machine Translation (NMT) models are typically trained using Maximum LikelihoodEstimation (MLE). However, this approach has a limitation: while it might select the bestword for the immediate context, it does not generally optimize for the entire sentence. Tomitigate this issue, we propose a simple yet effective training method called SimCLNMT.This method is designed to select words that fit well in the immediate context and also en-hance the overall translation quality over time. During training, SimCLNMT scores multiplesystem-generated (candidate) translations using the logarithm of conditional probabilities.Itthen employs a ranking loss function to learn and adjust these probabilities to align with thecorresponding quality scores. Our experimental results demonstrate that SimCLNMT consis-tently outperforms traditional MLE training on both the NIST English-Chinese and WMT`14English-German datasets. Further analysis also indicates that the translations generated by ourmodel are more closely aligned with the corresponding quality scores. We release our code athttps://github.com/chaos130/fairseq{\_}SimCLNMT.Introduction{\textquotedblright}
Translate-and-Revise: Boosting Large Language Models for Constrained Translation (2024)

{\textquotedblleft}Imposing constraints on machine translation systems presents a challenging issue because thesesystems are not trained to make use of constraints in generating adequate, fluent translations. Inthis paper, we leverage the capabilities of large language models (LLMs) for constrained trans-lation, given that LLMs can easily adapt to this task by taking translation instructions and con-straints as prompts. However, LLMs cannot always guarantee the adequacy of translation, and,in some cases, ignore the given constraints. This is in part because LLMs might be overly confi-dent in their predictions, overriding the influence of the constraints. To overcome this overidingbehaviour, we propose to add a revision process that encourages LLMs to correct the outputs byprompting them about the constraints that have not yet been met. We evaluate our approach onfour constrained translation tasks, encompassing both lexical and structural constraints in mul-tiple constraint domains. Experiments show 15{\%} improvement in constraint-based translationaccuracy over standard LLMs and the approach also significantly outperforms neural machinetranslation (NMT) state-of-the-art methods.IntroductionConstrained translation seeks to generate translations that adhere to pre-specified constraints. Toachieve this, conventional approaches impose constraints on machine translation systems and force themto follow the constraints during inference (Hokamp and Liu, 2017; Hasler et al., 2018; Dinu et al., 2019;Bergmanis and Pinnis, 2021b; Wang et al., 2022b; Ailem et al., 2022). More recently, large languagemodels (LLMs) have been shown to be strong translation systems (Hendy et al., 2023; Moslem et al.,2023). They provide a general way to involve various instructions, demonstrations, and constraints intothe translation process (Mu et al., 2023; Bogoychev and Chen, 2023), enabling us to perform constrainedtranslation using off-the-shelf, well-trained LLMs.{\textquotedblright}
A Multi-Task Biomedical Named Entity Recognition Method Based on Data Augmentation (2024)

{\textquotedblleft}The rapid development of artificial intelligence has led to an explosion of literature in the biomed-ical field, and Biomedical Named Entity Recognition (BioNER) can quickly and accurately iden-tify key information from unstructured text. This task has become an important topic to promotethe rapid development of intelligence in the biomedical field. However, in the Named EntityRecognition (NER) of the biomedical field, there are always some problems of unclear boundaryrecognition, the underutilization of hierarchical information in sentences and the scarcity of train-ing data resources. Based on this, this paper proposes a multi-task BioNER model based on dataaugmentation, using four data augmentation methods: Mention Replacement (MR), Label-wisetoken Replacement (LwTR), Shuffle Within Segments (SiS) and Synonym Replacement (SR)to increase the training data. The syntactic information is extracted by incorporating the inputsentence into the Graph Convolutional Network (GCN), and then the tag information encodedby BERT is interacted through a co-attention mechanism to obtain an interaction matrix. Subse-quently, NER is performed through boundary detection tasks and span classification tasks. Com-parative experiments with other methods are conducted on the BC5CDR and JNLPBA datasets,as well as the CCKS2017 dataset. The experimental results demonstrate the effectiveness of themodel proposed in this paper.{\textquotedblright}
Biomedical Event Causal Relation Extraction by Reasoning Optimal Entity Relation Path (2024)

{\textquotedblleft}Biomedical Event Causal Relation Extraction (BECRE) is an important task in biomedical infor-mation extraction. Existing methods usually use pre-trained language models to learn semanticrepresentations and then predict the event causal relation. However, these methods struggle tocapture sufficient cues in biomedical texts for predicting causal relations. In this paper, we pro-pose a Path Reasoning-based Relation-aware Network (PRRN) to explore deeper cues for causalrelations using reinforcement learning. Specifically, our model reasons the relation paths betweenentity arguments of two events, namely entity relation path, which connects the two biomedicalevents through the multi-hop interactions between entities to provide richer cues for predictingevent causal relations. In PRRN, we design a path reasoning module based on reinforcementlearning and propose a novel reward function to encourage the model to focus on the length andcontextual relevance of entity relation paths. The experimental results on two datasets suggestthat PRRN brings considerable improvements over the state-of-the-art models.Introduction{\textquotedblright}
Joint Entity and Relation Extraction Based on Bidirectional Update and Long-Term Memory Gate Mechanism (2024)

{\textquotedblleft}Joint entity recognition and relation extraction are important tasks in natural language process-ing. While some previous work has recognized the importance of relation information in jointextraction, excessively focusing on relation information without utilizing entity information maylead to information loss and affect the identification of relation tuples. Additionally, ignoring theutilization of original information may result in the loss of hierarchical and semantic information,further reducing the richness of information.To address these issues, we propose a bidirectionalinformation updating mechanism that integrates entity and relation information, iteratively fus-ing fine-grained information about entities and relations. We introduce a long-term memory gatemechanism to update and utilize original information using feature information, thereby enhanc-ing the model`s ability for entity recognition and relation extraction. We evaluated our approachon two Chinese datasets and achieved state-of-the-art results.{\textquotedblright}
MFE-NER: Multi-feature Fusion Embedding for Chinese Named Entity Recognition (2024)

{\textquotedblleft}In Chinese Named Entity Recognition, character substitution is a complicated linguistic phe-nomenon. Some Chinese characters are quite similar as they share the same components or havesimilar pronunciations. People replace characters in a named entity with similar characters togenerate a new collocation but refer to the same object. As a result, it always leads to unrecog-nizable or mislabeling errors in the NER task. In this paper, we propose a lightweight method,MFE-NER, which fuses glyph and phonetic features to help pre-trained language models handlethe character substitution problem in the NER task with limited extra cost. Basically, in the glyphdomain, we disassemble Chinese characters into Five-Stroke components to represent structurefeatures. In the phonetic domain, an improved phonetic system is proposed in our work, makingit reasonable to describe phonetic similarity among Chinese characters. Experiments demon-strate that our method performs especially well in detecting character substitutions while slightlyimproving the overall performance of Chinese NER.{\textquotedblright}
UDAA: An Unsupervised Domain Adaptation Adversarial Learning Framework for Zero-Resource Cross-Domain Named Entity Recognition (2024)

{\textquotedblleft}The zero-resource cross-domain named entity recognition (NER) task aims to perform NER in aspecific domain where labeled data is unavailable. Existing methods primarily focus on transfer-ring NER knowledge from high-resource to zero-resource domains. However, the challenge liesin effectively transferring NER knowledge between domains due to the inherent differences inentity structures across domains. To tackle this challenge, we propose an Unsupervised DomainAdaptation Adversarial (UDAA) framework, which combines the masked language model auxil-iary task with the domain adaptive adversarial network to mitigate inter-domain differences andefficiently facilitate knowledge transfer. Experimental results on CBS, Twitter, and WNUT2016three datasets demonstrate the effectiveness of our framework. Notably, we achieved new state-of-the-art performance on the three datasets. Our code will be released.Introduction{\textquotedblright}
Triple-view Event Hierarchy Model for Biomedical Event Representation (2024)

{\textquotedblleft}Biomedical event representation can be applied to various language tasks. A biomedical eventoften involves multiple biomedical entities and trigger words, and the event structure is complex.However, existing research on event representation mainly focuses on the general domain. Ifmodels from the general domain are directly transferred to biomedical event representation, theresults may not be satisfactory. We argue that biomedical events can be divided into three hierar-chies, each containing unique feature information. Therefore, we propose the Triple-views EventHierarchy Model (TEHM) to enhance the quality of biomedical event representation. TEHM ex-tracts feature information from three different views and integrates them. Specifically, due to thecomplexity of biomedical events, We propose the Trigger-aware Aggregator module to handlecomplex units within biomedical events. Additionally, we annotate two similarity task datasetsin the biomedical domain using annotation standards from the general domain. Extensive exper-iments demonstrate that TEHM achieves state-of-the-art performance on biomedical similaritytasks and biomedical event casual relation extraction.Introduction{\textquotedblright}