YNU - HPCC at S em E val-2024 Task 5: Regularized Legal- BERT for Legal Argument Reasoning Task in Civil Procedure (2024)

Abstract: This paper describes the submission of team YNU-HPCC to SemEval-2024 for Task 5: The Legal Argument Reasoning Task in Civil Procedure. The task asks candidates the topic, questions, and answers, classifying whether a given candidate’s answer is correct (True) or incorrect (False). To make a sound judgment, we propose a system. This system is based on fine-tuning the Legal-BERT model that specializes in solving legal problems. Meanwhile,Regularized Dropout (R-Drop) and focal Loss were used in the model. R-Drop is used for data augmentation, and focal loss addresses data imbalances. Our system achieved relatively good results on the competition’s official leaderboard. The code of this paper is available at https://github.com/YNU-PengShi/SemEval-2024-Task5.

## introduction the task can be formulated as follows: given an introduction to the topic, a question, and an answer candidate, classify if the given candidate is correct (true) or incorrect (false) @xcite . this task has two main difficulties: 1) the text length of the topic and question is much larger than 512 tokens. 2) the number of positive and negative samples in the data varies widely. initially, the online system represented the first attempt to utilize computational methods for addressing legal conundrums @xcite . despite notable advancements in recent years, which have seen a concerted effort to establish objective benchmarks for natural language processing models in the domain of legal language comprehension @xcite , a lack remains in the realm of complex tasks involving argumentative reasoning within legal contexts. however, legal-bert has emerged as a forerunner in this domain, demonstrating compelling performance @xcite . this paper proposes a model based on legal-bert. in processing tasks, we used sliding window simple (sws) and sliding window complex (swc) to process the original data and solved the problem of the token count of the original data being much larger than 512. in the subsequent process, we found that there was a significant imbalance in the dataset that resulted in the return of the most common label in the training set (in this case, 0). we added r-drop @xcite to the model to address this issue and changed the loss function from cross entropy to focal loss @xcite . in the end, we achieved a good result. the best submission for the test set has achieved 0.6166 and ranked 9th in this task. the remainder of this paper is organized as follows. section 2 describes the model and method used in our system, section 3 discusses the results of the experiments, and finally, the conclusions are drawn in section 4. ## system description this section delves into the intricate design of the proposed model's architecture. the architecture comprises multiple essential components, namely the text cutting, the tokenizer, the pre-trained legal-bert model, the output layer, and the methods. figure 1 illustrates the comprehensive system model that we have devised. ## conclusion in this research paper, we introduce a system submitted for evaluation in semeval-2024 task 5. leveraging the powerful pre-trained legal-bert