Warren at S em E val-2020 Task 4: ALBERT and Multi-Task Learning for Commonsense Validation (2020)

Abstract: This paper describes our system in subtask A of SemEval 2020 Shared Task 4. We propose a reinforcement learning model based on MTL(Multi-Task Learning) to enhance the prediction ability of commonsense validation. The experimental results demonstrate that our system outperforms the single-task text classification model. We combine MTL and ALBERT pretrain model to achieve an accuracy of 0.904 and our model is ranked 16th on the final leader board of the competition among the 45 teams.

## introduction common sense is a fundamental ability of human beings to understand the objective world. in the process of human growth, common sense is mastered through various studies. it can help people to infer other things from one fact when learning new knowledge. however, most current natural language processing (nlp) models do not have such a knowledge model, so they cannot understand and solve new problems well. it is particularly important to introduce common sense to natural language understanding (nlu) systems @xcite . semeval 2020 shared task 4 @xcite aims at testing whether a system can distinguish one statement that makes sense from another statement that makes no sense and reason out why the faked one makes no sense. two examples of commonsense validation are shown is ## methods figure 1 : albert and mtl model our model architecture can be roughly split into two modules, including an albert component to learn the features of the input texts, and a multi-task component for generation of text similarity and commonsense reasoning classification. the details are presented as figure 1 .