INNOVATORS at S em E val-2021 Task-11: A Dependency Parsing and BERT -based model for Extracting Contribution Knowledge from Scientific Papers (2021)

Abstract: In this work, we describe our system submission to the SemEval 2021 Task 11: NLP Contribution Graph Challenge. We attempt all the three sub-tasks in the challenge and report our results. Subtask 1 aims to identify the contributing sentences in a given publication. Subtask 2 follows from Subtask 1 to extract the scientific term and predicate phrases from the identified contributing sentences. The final Subtask 3 entails extracting triples (subject, predicate, object) from the phrases and categorizing them under one or more defined information units. With the NLPContributionGraph Shared Task, the organizers formalized the building of a scholarly contributions-focused graph over NLP scholarly articles as an automated task. Our approaches include a BERT-based classification model for identifying the contributing sentences in a research publication, a rule-based dependency parsing for phrase extraction, followed by a CNN-based model for information units classification, and a set of rules for triples extraction. The quantitative results show that we obtain the 5th, 5th, and 7th rank respectively in three evaluation phases. We make our codes available at https://github.com/HardikArora17/SemEval-2021-INNOVATORS .

## introduction thousands of papers are published by the scientific community every day. it is now increasingly becoming difficult to browse the huge pool of papers to identify relevant work and thereby keep up with the latest research findings. scientific literature is growing at an exponential rate and researchers today face the problem to identify the latest stateof-the-art contributions. keeping track of recent advancements is becoming a tedious exercise, if not practically impossible. the open research knowledge graph (orkg) @xcite is posited as a solution to keeping track of research progress minus the cognitive overload that reading dozens of full papers imposes. it aims to build a comprehensive knowledge graph that publishes scholarly publications' research contributions per paper, where the contributions are interconnected via the graph even across documents. as described in d' @xcite , with the orkg comparisons feature, researchers are no longer faced with the daunting cognitive ingestion obstacle from manually scouring through dozens of papers of unstructured content in their field. this process traditionally would take several days or months; using the orkg contributions comparison tabulated view, the task is reduced to just a few minutes. assuming the individual paper contributions are structured in the orkg, they can then deconstruct the graph, tap into the aspects they are interested in, and can enhance it for their purposes. further, they can select multiple such paper graphs and click a button to generate their tabulated comparison. this presents an opportunity to enhance content ingestion enabled via their fine-grained machine interpretability by transforming scholarly articles into knowledge-based information flows by representing and expressing information through semantically rich, interlinked knowledge graphs @xcite . in this paper, we present our approach for the three sub-tasks in the nlp contribution graph challenge. our contribution are as follows: the rest of this paper is organized as follows: section 2 briefly summarizes some related works similar to this task followed by the problem statement of this task. section 4 describes the details of the data provided by the organizers. section 5 and 6 presents the details of our model for all three phases of the task, including the structure and its implementations, along with results and experimental details. the conclusions and the directions for the future research are provided in section 8. ## related work although this is a relatively new challenge, we found some related investigations in the literature. vogt et al. (2020) proposed a novel semantic data model for modeling the contribution of scientific investigations of three domains, viz. medicine, computer science, and agriculture. the model includes a schema of relevant concepts highlighting six core information units, viz. objective, method, activity, agent, material, and result. they introduced the idea of building blocks called knowledge graph cells for its knowledge graph application. @xcite introduced a new categorization of key aspects of scientific articles, which is (1) focus: main contribution, (2) tech-nique: method or tool used, and (3) domain: application domain. they extracted the aspects by matching semantic patterns to dependency trees and learn the patterns using bootstrapping. they also present a case study on the computational linguistics community using the three aspects extracted from its articles, verifying our system's results and showing novel results for the dynamics and the overall influence of computational linguistics subfields. @xcite introduced a new task of disentangled paper summarization to generate summaries for the paper contributions and the work context to help identify the key findings shared in articles. @xcite presented an approach to extracting subject-predicate-object triplets from en-glish sentences. they used four different wellknown syntactic parsers for english to generate parse trees from the sentences, followed by extraction of triplets from the parse trees using parserdependent techniques. a machine learning approach has been used by @xcite to extract subject-predicate-object triplets from english sentences. support vector machine (svm) is used to train a model on human annotated triplets, and the features are computed from three parser. ## problem definition the problems are defined by the shared task organizers. ## proposed approach we describe our approach for all three phases of the competition as follows:- ## conclusion and future work with the nlpcontributiongraph shared task, we have attempted to formalize the building of a scholarly contributions-focused graph over nlp scholarly articles as an automated task. results and analysis on the gold test dataset show that our approach performed reasonably well in identifying contributing sentences and phrase extraction. however, we didn't perform well in triples extraction. in the future, we plan to improve the system, especially the triples extraction phase.