Zhegu@ SMM 4 H -2022: The Pre-training Tweet & Claim Matching Makes Your Prediction Better (2022)

Abstract: SMM4H-2022 (CITATION) Task 2 is to detect whether containing premise in the tweets of users about COVID-19 on the social medias or their stances for the claims. In this paper, we propose T weet C laim M atching ( TCM ), which is a new pre-training task constructed by the tweets and claims similarly to Next Sentence Prediction (NSP). We first continue to pre-train the standard pre-trained language models on the labelled dataset and then fine-tune them for obtaining better performance. Compared with the solid baseline (CITATION), we achieve the absolute improvement of 7.9% in Task 2a and obtain the SOTA results.

## introduction since its appearance in 2019, covid-19 has generated a wide range of discussions on the social medias. users make statements that may contain their stances and arguments, and these twitter posts are value for estimating the level of cooperation with the mandates. research on this information has attracted widespread attention, and various automated approaches based on machine learning have been proposed. in this paper, we present the solution of our team zhegu for identifying the stance and premise of twitter posts in smm4h-2022 task 2. to summarize, our main contribution are: ## data all data sets @xcite contain texts from twitter about three health mandates related to the covid-19 pandemic: face masks, * *corresponding author stay at home orders, and school closures. there are 3,556 of the training data, 600 for validation, and 2,000 for testing. task2a needs to determine whether the attitude of the authors of the tweets are supported, against or neutral to the claims, which is a triple classification. task2b is a bi-classification, which aims to figure out whether the tweets contain at least one premise or argument. ## methodology in this section, we firstly present our model and strategies we use. secondly, we give the basic idea and the construction of our tcm pre-training task. ## conclusion we present the solution of our team zhegu in smm4h-2022 task 2. we construct a new tcm pre-training task by incorporating the idea of contrastive learning for the relationship between tweets and claims. the effectiveness of our tcm has been demonstrated by two plms. even in roberta with only the mlm, the addition of tcm brings a decent improvement. in summary, absolute improvement of 5.1% and 7.9% are achieved in both backbones compared to baseline. and we propose to investigate the relationship between twitter text and its claim when fine-tuning in our future work.