Counts@ IITK at S em E val-2021 Task 8: S ci BERT Based Entity And Semantic Relation Extraction For Scientific Data (2021)

Abstract: This paper presents the system for SemEval 2021 Task 8 (MeasEval). MeasEval is a novel span extraction, classification, and relation extraction task focused on finding quantities, attributes of these quantities, and additional information, including the related measured entities, properties, and measurement contexts. Our submitted system, which placed fifth (team rank) on the leaderboard, consisted of SciBERT with [CLS] token embedding and CRF layer on top. We were also placed first in Quantity (tied) and Unit subtasks, second in MeasuredEntity, Modifier and Qualifies subtasks, and third in Qualifier subtask.

## introduction semeval 2021 task 8 @xcite ) is a task for extracting entities and semantic relations between them from a corpus of scientific articles coming from different domains. instead of just identifying quantities, the task gives more weightage to parsing and extracting important semantic relations among the extracted entities. this is challenging because texts are ambiguous, and inconsistent, and extraction relies heavily on implicit knowledge. the results of this task can also be used for extractive scientific data summarization. given a scientific text, the task is to identify the span of quantities, units, and other attributes of those quantities and related measured entities, properties, and qualifiers, if any. the organizers have divided the task into five subtasks and submissions will be evaluated against all five sub-tasks 1 . 1. quantity extraction: for each paragraph of text, identify all quantity spans. we consider subtask 1 as an entity extraction task, and subtask 3, 4, and 5 are viewed as relation extraction tasks. after extracting the quantities, other attributes (measuredentity, property, and qualifier) related to those quantities need to be predicted. the directed graph in figure 1 gives an overview of our proposed approach. the set of incoming edges to each node represents the input to the trained model (represented by node), and the label at each node represents the prediction made by the model. the task data is extracted from cc-by sciencedirect articles and made available by the elsevier labs via the oa-stm-corpus foot_0 . this motivated the use of scibert @xcite ) model for various subtasks. "scibert leverages unsupervised pretraining on a large multi-domain corpus of scientific publications to improve performance on downstream scientific nlp tasks". our final submitted system consisted of scib-ert with [cls] token embedding and crf layer on top, and it achieved an overall f1-overlap score of 0.432. we were ranked fifth on the global leaderboard. the top performance on the leaderboard achieved an overall f1-overlap of 0.519. the implementation of our system is made available via github foot_1 . the rest of this paper is arranged as follows. section 2 introduces the previous work in this field and describes the organizers' dataset. section 3 explains our overall approach. section 4 contains the experimental setup for training the model. we conclude with the analysis of our model performance in section 5 and concluding remarks in section 6. the work by wu and he 2019 on relation extraction uses bert to identify the different types of relations between pair of entities in the given text. the system does not automatically recognize the entities between which relation exists, rather entities of interest need to be manually specified. ## experimental setup the dataset is split into two parts -train set and dev set in a ratio of 90:10. the models were trained on the train set and were validated on the dev set. the environment and packages used for training and pre-processing are listed in appendix b.