YNU - HPCC at S em E val-2025 Task 8: Enhancing Question-Answering over Tabular Data with T able GPT 2 (2025)

Abstract: This paper describes our systems for SemEval 2025 Task8, Question Answering over Tabular Data. This task encourages us to develop a system that answers questions of the kind present in DataBench over day-to-day datasets, where the answer is either a number, a categorical value, a boolean value, or lists of several types. Participating in Task 8, we engage in all subtasks. The challenge lies in the multi-step reasoning process of converting natural language queries into executable code. This challenge is exacerbated by the limitations of current methods, such as chaining reasoning, which have difficulty handling complex multi-step reasoning paths due to difficulty evaluating intermediate steps. In the official ranking, we obtain a score of 65.64. On the final competition test set, our DataBench accuracy is 65.64%, and DataBench Lite accuracy is 66.62%. Both exceed the baseline (26%). The competitive results in two subtasks demonstrate the effectiveness of our systems.

## introduction the rise of large language models (llms) has transformed research in natural language processing (nlp). their ability to learn from little data has made them useful for tasks like summarization @xcite , machine translation, and text sentiment analysis @xcite . this progress has been driven by the development of general-purpose llms and the discovery of their unexpected abilities. however, the rapid growth of these models hasn't been matched by high-quality benchmarks for comparing their performance. question answering (qa) has been a key nlp task focused on finding the best answer from unstructured text. the issue of structured knowledge grounding has been widely researched for years. furthermore, tables, a common form of (semi)structured data that stores world knowledge, have garnered considerable attention from the natural language processing (nlp) community. traditionally, accessing the information inside the structured data (like tables) has depended on synthesizing executable languages like sql or sparql. still, these methods don't understand the meaning of text in the fields or allow natural language questions. such challenges have led to interest in using llms to answer questions from structured or tabular data. recently, the ability of llms to perform table question answering @xcite @xcite has emerged as a valuable skill. this highlights the need for a reliable benchmark to evaluate llm performance. our contributions can be summarized as follows: we propose an approach exploring using the tablegpt2-7b @xcite model to solve the corresponding subtasks. the model develops a tabular encoder that processes the entire table, generating compact embeddings for each column. the rest of this paper is structured as follows: section 2 covers related work, section 3 presents an overview of the model and our system, and sections 4, 5, and 6 detail the experiments, key results, and conclusions, respectively. ## system overview our system is designed to generate python code using the pandas library to extract information from these datasets. ## experimental settings 4.1 dataset there are 65 publicly available datasets summarized in ## experimental results results on dev set. when processing input prompts in batches and generating text, we adjust some parameters to change the diversity and length of the generated text. for example, the parameter temperature controls the randomness or creativity of the generated text. when the temperature is low, the text generated by the model is more deterministic and tends to choose words with higher probability; when the temperature is high, the generated text is more diverse and random. we use two temperatures of 0.1 and 0.2. as shown in competition results. our experimental results on the test set for the task competition are summarized in figure 2 . it can be observed that the tablegpt2-7b model achieves the highest performance scores in all subtasks. also, we observe a large perfor-mance gap between these models, although their number of parameters is similar. analysis. as indicated in figure 2 , tablegpt2-7b achieves 0.65% accuracy on the databench benchmark and 0.66% accuracy on the databench lite benchmark. the consistency across both benchmarks (databench and databench lite) further validates its robustness and generalizability. when comparing it with other models like the deepspeek-coder-6.7b-base, which shows an accuracy of 0.51% on databench and 0.50% on databench lite, the tablegpt2-7b outperforms them by a notable margin, indicating that it benefits from more refined training or a more effective architecture for this type of task. in contrast, the stable-code-3b and codellama-7b-hf models exhibit comparatively lower accuracy rates, highlighting the potential advantages of using larger or more specialized models like tablegpt2-7b for similar tasks. the result suggests that further fine-tuning or enhancement of such models could lead to even more significant improvements in performance. meanwhile, the results of deepseek-coder's three different parameter models (deepspeek-coder-1.3b-base, deepseek-coder-5.7bmqa-base and deepspeek-coder-6.7b-base) show that the accuracy improves as the model size increases. the accuracy of deepspeek-coder-1.3b-base on databench is 0.40%, and slightly lower on databench lite, 0.39%. the accuracy of deepspeek-coder-6.7b-base is 0.51% and 0.50%, respectively, which is more than 27% higher than the 1.3b-version. increasing the number of model parameters (from 1.3b to 6.7b) impacts model performance. generally speaking, larger models can capture more data features and subtle patterns and thus perform better in complex tasks. this suggests that the 6.7b version of deepseek-coder may have obtained more contextual information during training and can better cope with the challenges in the task. ## conclusions during task 8 in semeval 2025, we made prediction for each question provided by final competition. conducting preliminary exploration on the training set and validation set, we finally decided to use the tablegpt2 model to complete the competition. due to the powerful table understanding ability of this model, it has an inherent advantage in handling task 8. our approach proved to be highly effective by outstanding performance in this task. in future work, we may adjust some basic parameters or explore models with larger parameters to optimize the experimental results (which have been proven to be effective in above work). thus, our goal is to continue progressing in this area.