Arthur Caplan at S em E val-2023 Task 4: Enhancing Human Value Detection through Fine-tuned Pre-trained Models (2023)

Abstract: The computational identification of human values is a novel and challenging research that holds the potential to offer valuable insights into the nature of human behavior and cognition. This paper presents the methodology adopted by the Arthur-Caplan research team for the SemEval-2023 Task 4, which entailed the detection of human values behind arguments. The proposed system integrates BERT, ERNIE2.0, RoBERTA and XLNet models with fine tuning. Experimental results show that the macro F1 score of our system achieved 0.512, which overperformed baseline methods by 9.2% on the test set.

## introduction human values refer to the cognition, understanding, judgment, or choice made based on people's certain thinking and senses, which is a kind of thinking or orientation for people to identify things and determine right and wrong, to reflect a certain value or function of people and objects. human values are studied both in the social sciences @xcite and formal argumentation @xcite for decades to observe different value priorities between cultures and disagreement. some values tend to conflict and others to align, which can cause disagreement on the forward course and opinion. several approaches have been developed to study human values in the social sciences. however, the task to identify values in arguments seems challenging due to their large number, often implicit use in arguments, and vague definitions. thus, this task @xcite is an attempt at the automatic identification of values in written arguments. it classifies a given text parameter and a person's value category as to whether the parameter belongs to the categories. this task uses a set of 20 value categories compiled from the social science literature from their acl paper @xcite . in this paper, we presented various pre-trained models, such as bert @xcite , ernie2.0 @xcite , roberta @xcite and xlnet @xcite to address each component of this task, with a bunch of fine-tuning and ensemble techniques. in particular, we introduced prompt @xcite in our model. the code of the task has been made publicly available from https://github.com/kilochips/semeval2023-task4 . in this task, our team finally achieved the following results as shown in ## background as for the task of value classification, people have conducted classification studies on values in social sciences for a long time.rokeach @xcite showed the correlation between people's ideal state and actual behavior. however, different fields have their own views on the classification of values. rokeach @xcite analyzed 36 kinds of values from the perspectives of sociology, philosophy, and anthropology. during this period, schwartz @xcite reflected the multifaceted characteristics of values. schwartz @xcite analyzed 48 kinds of human values from the perspective of the universal needs of individuals and society. on text recognition of values, these papers @xcite ; @xcite ; @xcite studied how to extract author's opinions from essay arguments. after them, bar (bar-haim et al., 2020) and friedman @xcite . proposed a method to get the arguments by analyzing the key points and sentences in the article, thus completing the task of extracting the opinions from the article. maheshwari @xcite represented human personality in a parameterized way, while in these papers @xcite ; @xcite analyzed the intrinsic values of a public using the degree of public agreement with a certain point of view. on the basis of the above work, identifying the human values behind arguments @xcite is the first article identifying the human values behind arguments.this completes the task of calculating and analyzing human values expressed through viewpoints.the classification of values adopts the classification of values in the paper @xcite , and the values are divided into 20 subcategories. this paper is the realization and improvement of the evaluation task. the presented system architecture, as illustrated in figure 1 ,consists of four parts: input, data preprocessing, model, and output. the input section represents the provided raw dataset. the data preprocessing section preprocesses the dataset and inputs the processing results into the model section. in the model part, the input text is represented by vectorization, and then connected to the linear layer for the final multi label classification task. the output part is the classification result predicted by the model, which contains binary labels for 20 different categories. ## results the experimental results of our work are given in from the experimental results, it can be found that a single pre-trained model performs well in classification tasks, and ernie2.0 performs best. therefore, we specifically incorporated the prompt structure into ernie2.0. it can be seen that the macro f1 score of the model has dropped a little after adding the prompt. however, in fact, after adding the prompt, the f1 scores of some difficultto-classify categories (e.g: face and universalism: tolerance) have increased. this illustrates the advantage of prompt in low data situations. ## conclusion in this paper, we investigated the effectiveness of fine-tuning large pre-trained models for the downstream task of multi-label classification of human values. the results of our experiments indicated that most pre-trained models performed well on this task, with the ernie2.0 model exhibiting particularly strong performance. however, we observed that the addition of structures similar to bilstm and attention to the pre-trained models often led to a reduction in the models' efficacy. consequently, our team adopted a model ensemble approach as the primary method for further enhancing the performance of a single pre-trained model in value classification.