Prerequisite or Not Prerequisite? That’s the Problem! An NLP-based Approach for Concept Prerequisite Learning
Authors: Chiara Alzetta, Alessio Miaschi, Giovanni Adorni, Felice Dell'Orletta, Frosina Koceva, Samuele Passalacqua, Ilaria Torre (Università di Genova, Università di Pisa, CNR-ILC, 2019)
Topic: Lexical and Semantic Resources and Analysis

Abstract
This paper presents a method for prerequisite learning classification between educational concepts. The proposed system was developed by adapting a classification algorithm designed for sequencing Learning Objects to the task of ordering concepts from a computer science textbook. In order to apply the system to the new task, for each concept we automatically created a learning unit from the textbook using two criteria based on concept occurrences and burst intervals. Results are promising and suggest that further improvements could highly benefit the results.

Introduction
Personalised learning paths creation is an active research topic in the field of education (Chen, 2009; Kurilovas et al., 2015; Almasri et al., 2019). The most fundamental issue behind this task is the need to understand how educational concepts are pedagogically related to each other: what information one has to study/know first in order to understand a given topic. In this paper, we focus on such relations, i.e. prerequisite relations, between educational concepts of a textbook in English and we present a method for their automatic identification. Here, we define concepts as all the relevant topics extracted from the textbook and represent them as single or multi-word terms. Automatic prerequisite extraction is a task deeply rooted in the field of education, whose results can be easily integrated in many different contexts, such as curriculum planning (Agrawal et al., 2016), course sequencing (Vuong et al., 2011), reading list generation (Gordon et al., 2017), automatic assessment (Wang and Liu, 2016), domain ontology construction (Zouaq et al., 2007; Larranaga et al., 2014), and automatic educational content creation (Lu et al., 2019). Several methods have been devised to extract prerequisite relations (Liang et al., 2015; Pan et al., 2017a; Liang et al., 2018b), however, they were mainly focused on educational materials already enriched with some sort of explicit relations, such as Wikipedia pages, course materials, or learning objects (LOs). More challenging is identifying prerequisites when no such relations are given and textual content is the only available resource. In 2019, we proposed two methods to identify prerequisite relations between concepts without using external knowledge or even pre-defined relations. The former method (Adorni et al., 2019) is based on burst analysis and temporal reasoning on concepts' occurrence, while the latter (Miaschi et al., 2019) uses deep learning for learning object ordering. Both these methods extract prerequisite relations from textual educational materials without using any form of structured information. In this work, we adapt the system for learning object ordering described in Miaschi et al. (2019) to the task of sequencing concepts in a textbook according to their prerequisite relations. For training and testing our system, we relied on a new version of PRET (Alzetta et al., 2018), a gold dataset manually annotated with prerequisite relations between educational concepts. Moreover, since the classifier was designed to acquire learning objects as input, we automatically created a learning unit for each concept according to two different criteria: (i) considering all sentences showing an occurrence of the concept, (ii) considering burst intervals (Kleinberg, 2003) of each concept extracted according to the strategy of Adorni et al. (2019). The remainder of the paper is organised as follows. First, we present related work (Section 2) and the dataset used for the experiments (Section 3). Section 4.1 presents the classifier, while burst analysis is described in Section 4.2 and the experimental settings in Section 4.3. Results and discussion are reported in Section 4.4, while error analysis is illustrated in Section 5. Section 6 concludes the paper. Our Contribution. In this paper: (i) we use a deep learning-based approach for prerequisite relation extraction between educational concepts of a textbook; (ii) we test the impact of creating learning units for each concept according to different criteria and without relying on any explicit structured information, such as Wikipedia hyperlinks; (iii) we show the effectiveness of our approach on real educational materials.

Conclusion
In this paper we tested a deep learning model for prerequisite relation extraction in a real educational environment, using a dataset (PRET 2.0) built starting from a computer science textbook. The results demonstrated the effectiveness of our system, suggesting that it is possible to infer prerequisite relation out of textual educational material without using any form of structured information. Nevertheless, further work needs to be done, particularly for improving the performances of our system in an out-of-domain scenario, namely using concept pairs of a different domain during testing. Moreover, it could be useful to investigate the use of transitive relations and to study more accurately their impact on the system's performance. In addition, in order to identify prerequisite relationships while taking into account different types of relations (e.g. transitive ones) it could be interesting to frame our task as a ranking or multi-classification task rather than a binary classification one. Further analysis is also required to investigate the effect of using different numbers of sentences for creating WE. We plan also to explore the impact of using temporal reasoning on concept pairs (Adorni et al., 2019), which has not been considered in this work.