Bit_numeval at S em E val-2024 Task 7: Enhance Numerical Sensitivity and Reasoning Completeness for Quantitative Understanding (2024)

Abstract: In this paper, we describe the methods used for Quantitative Natural Language Inference (QNLI), and Quantitative Question Answering (QQA) in task1 of Semeval2024 NumEval. The challenge’s focus is to enhance the model’s quantitative understanding consequently improving its performance on certain tasks. We accomplish this task from two perspectives: (1) By integrating real-world numerical comparison data during the supervised fine-tuning (SFT) phase, we enhanced the model’s numerical sensitivity. (2) We develop an innovative reward model scoring mechanism, leveraging reinforcement learning from human feedback (RLHF) techniques to improve the model’s reasoning completeness.

## introduction numeral-aware language understanding plays a crucial role in narratives when contexts contain numerical comparison, implication, definition etc @xcite . this importance is especially revealed in real-world applications, such as law, finance, and clinical documentation @xcite . large language models (llms) demonstrated impressive performance in various nlp tasks, but they still suffer from challenges such as hallucination of weak numerical reasoning @xcite . in this paper, we delve into solving the numerical hallucination problem, evaluated by the qnli @xcite and qqa @xcite subtasks in the quantitative understanding (qu) task @xcite of the semeval numeval task @xcite . qnli involves inferring relationships based on nu-figure 1 : an example from qqa dataset. llms may not be able to generate an accurate and complete process during quantitative reasoning. specifically in this example, the first solution has an error in step @xcite where the model confuses the concept of time period and time point, resulting in a wrong answer. and the second solution simply jump to the final answer after summarizing the problem, which is incomplete and unreasonable. merical clues, and qqa requires quantitative reasoning. based on our investigation and preliminary evidence of promise, we attribute llms' limitations on the qu tasks to two key aspects: (1) numerical sensitivity: llms, trained on vast quantities of text, often fail to accurately capture numerical information @xcite . (2) reasoning accuracy and completeness: as illustrated in figure 1 , llms may struggle to generate a comprehensive and precise step-by-step reasoning process, particularly in numerical reasoning contexts @xcite . to improve models' numerical sensitivity, @xcite fine-tuned them using the comparing numbers dataset, which comprises numerical comparison statements. however, solely tuning models using the comparing number data may lead to an overfit issue. meanwhile, recent efforts on enhancing reasoning accuracy such as process supervision by reinforcement learning on every reasoning step @xcite . nevertheless, in our cases, numerical reasoning involves a variable number of reasoning steps. therefore, multiplying reward scores for each step @xcite reduces the overall multi-step reasoning score, which results in incomplete reasoning steps. to address these limitations, we propose utilizing numerical comparisons of real-world contexts for more robust fine-tuning. in addition, we introduce a reasoning completeness reward designed to improve the precision of viable reasoning processes. the contributions of this paper include: ## system overview as shown in figure 2 , we highlight to enhance the model's numerical sensitivity and reasoning completeness. specifically, we first use gpt-3.5 @xcite to extend comparing numbers data into real-world contexts and fine-tune the model with this data to enhance numerical sensitivity. to prevent model overfitting, we mix 50% of the qnli and qqa task data and 50% comparing numbers data into the sft training dataset. furthermore, we employ rlhf method to align every reasoning step with human-labeled process supervision. to leverage a more profitable reward model for rlhf, we manually score the reasoning steps of the augmented positive and negative cases. in particular, we propose a newly reasoning completeness reward for the ppo algorithm to encourage a complete reasoning procedure. the following subsections will detail our method. ## experimental setups datasets we adopted the quantitative101 dataset provided for semeval 2024 task7 and then expanded it using the gpt-3.5 api, in ## conclusion in this paper, we described the systems used for qnli, and qqa in task1 of semeval2024 nu-meval. we select the abel-7b model as the baseline model. to address the quantitative understanding problem, we first integrate comparing numbers data from real-world contexts to enhance the model's numerical sensitivity. during this process, we devise an effective data mixer to prevent overfitting and the long-tail problem. subsequently, by employing process supervision from human feedback, we develop an innovative reward model scoring mechanism to improve the model's reasoning completeness using rlhf. test results demonstrate that our 7b model exceptionally outperformed, surpassing llms such as gpt-3.5 on 4 tasks and llama2-70b on 6 tasks, respectively.