T . M . Scanlon at S em E val-2023 Task 4: Leveraging Pretrained Language Models for Human Value Argument Mining with Contrastive Learning (2023)

Abstract: Human values are of great concern to social sciences which refer to when people have different beliefs and priorities of what is generally worth striving for and how to do so. This paper presents an approach for human value argument mining using contrastive learning to leverage the isotropy of language models. We fine-tuned DeBERTa-Large in a multi-label classification fashion and achieved an F1 score of 49% for the task, resulting in a rank of 11. Our proposed model provides a valuable tool for analyzing arguments related to human values and highlights the significance of leveraging the isotropy of large language models for identifying human values.

## introduction argument mining is a promising area of research in natural language processing that aims to identify and analyze arguments from the text. identifying human values in arguments is significant because different beliefs and priorities can cause conflicts or alignment between individuals or cultures, leading to disagreements on controversial issues. understanding the values behind different perspectives can improve communication and decision-making. the automatic identification of values in written arguments is valuable for argument faceted search, value-based argument generation, and value-based personality profiling @xcite . the valueeval 1 , a task 4 of the shared task on se-meval 2023 is designed to automatically identify human values in arguments. the task has been defined as follows: given a textual argument and a human 1 https://touche.webis.de/semeval23/touche23-web/ value category, classify whether or not the argument draws on that category. touché23-valueeval dataset @xcite for this task consists of a valuable collection of argumentative texts, where each argument is labeled with one or more of the human values. arguments are given as premise text, conclusion text, and binary stance of the premise to the conclusion whatever it is "in favor of" or "against" the premise text. the task uses a set of 20 value categories compiled from the social science literature. the dataset for this task consists of two different sets called main, and supplementary datasets, where in this research we focus on all 20 value categories of the main dataset for identifying human values behind arguments in the english language. contrastive learning (cl) @xcite has shown promising results in various natural language processing tasks, including text classification, sentiment analysis, and named entity recognition. cl is a technique used to learn representations that are invariant to changes in certain aspects of the input data while preserving important information. in the context of argument mining it is important to identify which part of the information is representative of arguments. so, we applied cl with additional information to language models to leverage the pre-trained weights for language understanding capabilities to be able to make differences between different arguments. the proposed method for this task is implemented in python and published on github foot_0 for the research community in this field. the rest of the paper is organized as follows. section 2 presents backgrounds. section 3 describes the proposed system overview. section 4 describes the experimental setups such as dataset, metrics, and training setups. next, in section 5 we discussed results and analysis. finally, section 6 presents our conclusions 2 background ## system overview we propose two regularizers based on contrastive learning and binary cross entropy loss for achieving isotropy in the feature space for human values. in this section, we describe the details of our proposed deep learning model for human value argument mining. the proposed methodology is illustrated in figure 1 . ## results main quantitative findings: the test set / approach all self-direction: thought self-direction: action stimulation hedonism achievement power: dominance power: resources face security: personal security: societal tradition conformity: rules conformity: interpersonal humility benevolence: caring benevolence: dependability universalism: concern universalism: nature universalism: tolerance universalism: objectivity main dataset best per category .59 .61 .71 .39 .39 .66 .50 .57 .39 .80 .68 .65 .61 .69 .39 .60 .43 .78 .87 .46 .58 best approach .56 .57 .71 .32 .25 .66 .47 .53 .38 .76 .64 .63 .60 .65 .32 .57 .43 .73 .82 .46 .52 bert .42 .44 .55 .05 .20 .56 .29 .44 .13 .74 .59 .43 .47 .23 .07 .46 .14 .67 .71 .32 .33 1-baseline .26 .17 .40 .09 .03 .41 .13 .12 .12 .51 .40 .19 .31 .07 .09 .35 .19 .54 .17 .22 .46 deberta-base + lbce .45 .51 .64 .10 .28 .56 .34 .35 .09 .70 .60 .57 .41 .35 .10 .52 .18 .72 .73 .37 .41 deberta-large + lbce .44 .49 .57 .09 .19 .60 .27 .41 .14 .74 .60 .45 .49 .41 .10 .53 .16 .70 .72 .41 .40 deberta-base + lbce + clreg .47 .55 .67 .12 .19 .57 .40 .36 .17 .73 .64 .55 .51 .39 .26 .49 .32 .75 .79 .37 .44 deberta-large + lbce + clreg .49 .56 .67 .18 .39 .63 .36 .48 .26 .75 .63 .47 .53 .38 .20 .50 .31 .73 .82 .37 .42 earlybird submission deberta-large + lbce + clreg .49 .52 .69 .07 .29 .60 .35 .46 .23 .74 .65 .57 .52 .20 .18 .55 .30 .74 .84 .35 .46 security: societal classes even better than best approach (blue colored f1 score in the ## conclusion this paper presented our approach for se-meval2023 task 4: valueeval: identification of human values behind arguments. we investigated this problem by leveraging plms using a contrastive learning technique. the proposed study in this paper shows that isotropization for human value detection is effective and requires more attention in this field. in the final, we achieved an average f1 score of 0.4939. regarding our evaluations, our approach is promising for human value argument mining.