NCL _ NLP at S em E val-2024 Task 7: C o T - N um HG : A C o T -Based SFT Training Strategy with Large Language Models for Number-Focused Headline Generation (2024)

Abstract: Headline Generation is an essential task in Natural Language Processing (NLP), where models often exhibit limited ability to accurately interpret numerals, leading to inaccuracies in generated headlines. This paper introduces CoT-NumHG, a training strategy leveraging the Chain of Thought (CoT) paradigm for Supervised Fine-Tuning (SFT) of large language models. This approach is aimed at enhancing numeral perception, interpretability, accuracy, and the generation of structured outputs. Presented in SemEval-2024 Task 7 (task 3): Numeral-Aware Headline Generation (English), this challenge is divided into two specific subtasks. The first subtask focuses on numerical reasoning, requiring models to precisely calculate and fill in the missing numbers in news headlines, while the second subtask targets the generation of complete headlines. Utilizing the same training strategy across both subtasks, this study primarily explores the first subtask as a demonstration of our training strategy. Through this competition, our CoT-NumHG-Mistral-7B model attained an accuracy rate of 94%, underscoring the effectiveness of our proposed strategy.

## introduction headline generation is a key task in the field of natural language processing (nlp), aimed at condensing the content of a given article into a concise, accurate, and information-rich single-sentence headline. this process requires not only an understanding of the article's core content but also the ability to creatively express this content @xcite . recently, @xcite conducted an in-depth analysis of the application of models @xcite @xcite @xcite in the task of headline generation, revealing 1 https://github.com/gavinzhao19/cot-numhg limitations of these models in processing numerical information. they identified that inaccuracies in the use of numbers significantly contribute to errors in generated headlines, a particularly critical issue in news headline generation where numbers often carry key information. to further explore the issue of numerical accuracy in news headlines, @xcite introduced a new dataset, numhg, focused on the accuracy of numerical usage within news headlines. their analysis revealed that news headline generation typically involves nine distinct methods for handling numbers-copy, translate, round, paraphrase, add, subtract, divide, multiply, and span-each varying in complexity. these techniques enhance the interpretability and clarity of the headline generation process, showcasing a sophisticated blend of precision and creativity in distilling numerical information. based on these insights, @xcite designed two independent tasks: the first requires models to mask numbers in given news articles and their headlines, then to accurately predict the masked numbers; the second involves generating news headlines with accurate numerical information based on the provided news content. in the domain of nlp, large language models (llms) have gained recognition for their capability to execute a wide array of tasks, including text generation, summarization, and question answering, using straightforward instructions. this demonstrates their remarkable versatility @xcite . to further enhance the adaptability of llms, fine-tuning techniques @xcite have been extensively applied, improving model performance on specific tasks while preserving a wide scope of application. llms typically utilize a decoder-only architecture @xcite and adopt primarily two strategies for task-specific challenges: prompt engineering and fine-tuning. prompt engineering enables the direct execution of tasks through various techniques, such as zero-shot (radford et al., 2019), few-shot @xcite , chain of thought (cot) @xcite , cot with self-consistency @xcite , and tree of thought @xcite , without requiring additional training. this underscores the models' ability to quickly adapt to new tasks by leveraging existing knowledge. fine-tuning, via further training, refines the models' performance on specific tasks, particularly through supervised fine-tuning (sft) methods. to improve sft efficiency, parameter-efficient fine-tuning (peft) techniques @xcite including lora @xcite , prompttuning @xcite , and prefix-tuning (li and liang, 2021) have been introduced. these significantly enhance the models' adaptability and the quality of outputs for specific tasks without substantially increasing the model size or computational demands. this approach not only preserves the versatility of llms but also boosts their output quality and the ability to generate structured outputs in specific domains. despite these approaches achieving certain levels of performance enhancement, there remains room for improvement in perceiving numerical information, reasoning ability, and generating structured outputs @xcite . particularly in the task of news headline generation, reliance solely on prompt engineering may lead to uncontrollable outputs and insufficient structuring. meanwhile, sft, despite its ability to improve performance, shows limitations in the interpretability of the reasoning process and suffers from attention decay, potentially leading to the omission of important information. to address these challenges, we propose a training strategy based on the cot approach, designed to significantly enhance llms in the task of number-focused headline generation. our method consists of two key components. first, drawing on the concept of knowledge distillation @xcite , we utilize gpt-3.5-turbo @xcite and instructions to process the original numhg dataset, generating a series of reasoning steps. given the issue of attention decay when handling long-distance information @xcite , we created a new cot-numhg dataset by combining the question statement with reasoning steps. this process aims to bolster the model's attention mechanism and improve the interpretability of the reasoning process @xcite . secondly, we selected three llms as base models and performed full-parameters sft using the constructed cot-numhg dataset on these base models. through this approach, we not only significantly improved performance on the specific task, but also optimized structured outputs while maintaining the models' versatility. our research contributions are threefold: ## cot-based sft training strategy design the training strategy of this study is divided into two main parts, as shown in figure 1 : the construction of the cot-numhg dataset and model training. initially, through a knowledge distillation strategy, we enhanced the original dataset to improve the model's interpretability in handling the task of generating news headlines. subsequently, the selected base llms were trained using full-parameter sft techniques to achieve performance optimization and structured output for specific tasks. ## conclusion and future work in this paper, we have introduced a cot-based sft training strategy aimed at enhancing the performance of llms in the task of news headline generation. initially, we constructed the cot-numhg dataset, based on the existing numhg dataset through knowledge distillation techniques. by simulating the human thought process, this dataset enhances the interpretability of the reasoning path from problem to answer. subsequently, we utilized the cot-numhg dataset to perform sft on a selected baseline model and verified significant improvements in model performance through ablation studies. the competition results further validated the efficacy of our approach, with the cot-numhg-mistral-7b model achieving an accuracy rate of 94%. however, a manual review of the competition outcomes revealed some uncertainties in the model's handling of numerical information in titles, such as the need for approximations. this indicates that there is still room for improvement in understanding numerical information and generating structured outputs. future work will focus on the following directions: further optimization of the dataset by deduplicating and enhancing data diversity to improve the model's generalization capabilities. this includes identifying and removing duplicate or lowquality data samples, as the current proportion of copied methods is excessively high. we will adjust the proportions through sampling to address this issue. to align the generated headlines more closely with the standards of human editors, we will explore constructing a dataset in dpo @xcite format from incorrect generation outcomes.