CSECU - DSG at S em E val-2023 Task 4: Fine-tuning D e BERT a Transformer Model with Cross-fold Training and Multi-sample Dropout for Human Values Identification (2023)

Abstract: Human values identification from a set of argument is becoming a prominent area of research in argument mining. Among some options, values convey what may be the most desirable and widely accepted answer. The diversity of human beliefs, random texture and implicit meaning within the arguments makes it more difficult to identify human values from the arguments. To address these challenges, SemEval-2023 Task 4 introduced a shared task ValueEval focusing on identifying human values categories based on given arguments. This paper presents our participation in this task where we propose a finetuned DeBERTa transformers-based classification approach to identify the desire human value category. We utilize different training strategy with the finetuned DeBERTa model to enhance contextual representation on this downstream task. Our proposed method achieved competitive performance among the participants’ methods.

## introduction in human decision-making, values play a crucial role. the meaning of values often refers to any individual's personal beliefs and priorities about any argument which tends to go to the conclusion of that argument @xcite . the conclusion may vary from person to person though the arguments are the same. identifying human value categories which drive a human being to conclude an argument and make perspective decisions is very crucial to build a value-driven artificial intelligence (ai) agent to mitigate future security risks from those ai agents. nevertheless, it is beneficial for various nlp tasks also including argument mining, sentiment analysis, political discourse analysis, personality analysis, and marketing analysis. however, identifying human values from the arguments is the most challenging and prominent **the first two authors have equal contributions. task in nlp. because of the large variety and heterogeneity of personal beliefs which may be ambivalent for the same group of people regarding their personal experiences, this task becomes challenging. to address the challenges of human values identification from arguments @xcite introduce a shared task valueeval at semeval-2023 foot_0 . the task is composed of three categories of tracks including the argument test, nahj al-balagha test, and new york times test datasets. here, the argument test considers the main dataset for this task. to categorise the arguments, this task articulates 20 fixed categories of values which are shown in figure 1 . the task introduces the relation between a premise argument and a conclusion argument where the stance conveys that the conclusion is in favor of or against the premise. in the following example, "nuclear weapons help keep the peace in uncertain times" the argument is considered a premise and the conclusion is "we should fight for the abolition of nuclear weapons", which is a stance against the premise tenor since many people agree that nuclear weapons have the potential to damage human civilization. in spite of this, the dataset assumes that the conclusion stance is against the premise for the reasons of security: societal, and universalism: concern values. prior works @xcite ; handke and stein) of identifying human values based on arguments only explored theoretical and manual aspects but never attempted to build an automated classification system. however, we proposed a deberta-v3 transformer-based automated classification system in this paper to classify human values based on three parameters premise, conclusion, and stance. we also used different training strategies including cross-fold training and multisample dropout training strategies to train our proposed method. accordingly, the remaining sections of the paper are organized as follows: section 2 introduces our proposed system for automatically identifying human value from given arguments, while section 3 presents our system design and parameter settings, which is followed by section 4 where we discuss our results, and performance analysis. finally, we conclude with some future directions in section 5. ## human values identification framework in this section, we describe our proposed human values identification framework. this task is designed to be a multi-label classification task. the purpose of this study is to determine what human values are present in the argument provided. a brief overview of the proposed framework we have developed in this work is depicted in figure 2 . in our proposed framework, we use a sentence pair training concept in the transformer models in order to perform human values identification. the idea behind this concept is to pack together the input premise, stance, and conclusion into one sequence as shown in figure 3 . in our approach, we employ a pre-trained transformer model, de-berta, in order to extract contextual features from packed sequences based on given contexts. the de-berta model has been fine-tuned for the human value identification task so that it captures domainspecific information about human values within the context. in addition to the extracted feature vectors, we later use a multi-sample dropout @xcite procedure to improve the generalization ability of the system output. to predict class confidence, a classification head averages the feature vectors from the multi-sample dropout. a cross-fold training technique reduces the error rates associated with class label prediction @xcite . in order to improve prediction performance, we employ a 5-cross-fold training in our method. finally, for the effective fusion of the class confidences, we take the arithmetic mean of the prediction scores obtained from each trained model for each fold. ## experimental setup in this section, we now describe the dataset and hyper-parameters settings we have used in our proposed method in brief. ## conclusion this paper presents our approach to the human values identification task. we tackled the problem using sentence pair training of a transformer model. we fine-tuned the deberta transformer model with various training techniques including cross-fold training and multi-sample dropout. by using pairwise learning, we exploited the contextual relation between three texts in order to estimate human value categories. experimental results demonstrated the efficacy of our deberta-based proposed method. our method achieved competitive performance compared to other participants. in the future, we will implement other sota transformer models and also combine multiple transformer models into a unified architecture. weighted average fusion could capture better arguments for all human values classes since the dataset is imbalanced.