F rancis Bacon at S em E val-2023 Task 4: Ensembling BERT and G lo V e for Value Identification in Arguments (2023)

Abstract: In this paper, we discuss our efforts on SemEval-2023 Task4, a task to classify the human value categoriesthat an argument draws on. Arguments consist of a premise, conclusion,and the premise’s stance on the conclusion. Our team experimented with GloVe embeddings and fine-tuning BERT. We found that an ensembling of BERT and GloVe with RidgeRegression worked the best.

## introduction identifying the values (e.g., humility and dominance) in an argument is a key part of understanding the psychology of the argument's ideator. present-day nlp models allow us to more accurately identify arguments for their human values. these models can power better emotional understanding for conversational agents. based on the values you are exhibiting, a conversational agent would be able to change its response to brighten your mood or de-escalate the situation, resulting in a more engaging experience. to help advance the nlp effort in value identification, we participated in the semeval-2023 task 4 @xcite . the semeval task focuses on classifying the human value categories that a textual argument draws on. the dataset for the task contains arguments consisting of a premise, conclusion, and the premise's stance on the conclusion ("in favor of" or "against"). the target label for each argument is one or more of 20 given human values. our team experimented with two approaches for value identification, a baseline model using glove word embeddings, and a bert model that we finetuned for multilabel classification of human values. we describe the implementation details of these models in this paper, as well as our experiments and results from these models. we found that ensembling bert and glove acheived a 10% improvement in average f1-score over our baseline glove model. our full implementation can be found at https: //github.com/claserken/mlabtask4semeval . ## background semeval-2023 task 4 consisted of four inputs for each argument: an argument id, the conclusion, the stance, and the premise. the argument id is simply a marker for this specific argument (e.g., a01002). the premise is the full argument, such as "we should ban human cloning as it will only cause huge issues when you have a bunch of the same humans running around all acting the same." the stance is a binary choice of whether the premise is supported or not, with this specific one being "in favor of" while others may be "against." the conclusion is drawn from the stance and the premise which in this case would be "we should ban human cloning." ## results in particular, the validation f1 score of ensembling is roughly 10% greater than that of baseline glove. for comparison purposes, the 1-baseline provided for this task averages out to a 0.26 f1 score across all categories, with a modified bert baseline taking on a value of 0.42. both of our training f1 scores outperformed these benchmarks, whereas the 0.39 validation f1 score from the glove approach alone performed worse than standard bert. however, after ensembling, the validation f1 score increased to 0.45, demonstrating better performance than either of its individual approaches. ## conclusion our research began by utilizing glove embeddings as a baseline approach. however, we later modified our procedure by using bert, a powerful language model. due to class imbalances, we had to reweigh the examples in the loss function. to further improve the performance of our model, we employed an ensemble method that combined the outputs of both glove and bert, and observed a 10% increase in f1 score as compared to the glove baseline.