A Transfer Learning Pipeline for Educational Resource Discovery with Application in Survey Generation (2023)

Abstract: Effective human learning depends on a wide selection of educational materials that align with the learner’s current understanding of the topic. While the Internet has revolutionized human learning or education, a substantial resource accessibility barrier still exists. Namely, the excess of online information can make it challenging to navigate and discover high-quality learning materials in a given subject area. In this paper, we propose an automatic pipeline for building an educational resource discovery system for new domains. The pipeline consists of three main steps: resource searching, feature extraction, and resource classification. We first collect frequent queries from a set of seed documents, and search the web with these queries to obtain candidate resources such as lecture slides and introductory blog posts. Then, we process these resources for BERT-based features and meta-features. Next, we train a tree-based classifier to decide whether they are suitable learning materials. The pipeline achieves F1 scores of 0.94 and 0.82 when evaluated on two similar but novel domains. Finally, we demonstrate how this pipeline can benefit two applications: prerequisite chain learning and leading paragraph generation for surveys. We also release a corpus of 39,728 manually labeled web resources and 659 queries from NLP, Computer Vision (CV), and Statistics (STATS).

## introduction people rely on the internet for various educational activities, such as watching lectures, reading textbooks, articles, and encyclopedia pages. one may wish to develop their knowledge in a familiar subject area or to learn something entirely new. many online tools exist that enable and promote independent learning @xcite @xcite . a subset of these platforms provide primary literature resources (e.g. publications), such as google scholar foot_0 and semantic scholar foot_1 . as an alternative to these advanced materials, other educational platforms such as mooc.org foot_2 deliver free online courses. also, unstructured searching on the internet is a popular method to discover other useful resources, such as blog posts, github projects, tutorials, lecture slides and textbooks. rather than diving into the technical details, these secondary literature resources provide a broad overview of the given domain, which is more valuable for beginners. still, sifting through this material can be challenging and time-consuming, even if the learner is simply looking for a general and reliable introduction into a new subject area. publicly accessible data repositories that focus on gathering a fixed number of educational resources exist currently, such as scientific papers @xcite , online platforms like aminer @xcite and semantic scholar. some archives also compile secondary literature materials. tutorialbank @xcite ) is a manually-collected corpus with over 6,300 nlp resources, as well as related fields in artificial intelligence (ai), machine learning (ml) and so on. lecturebank @xcite is also a manuallycollected corpus and contains 1,717 lecture slides. mooccube @xcite is a large-scale data repository containing 700 mooc (massive open online courses), 100k concepts and 8 million student behaviours with an external resource. however, in their initial synthesis, these existing corpora either heavily relied on manual efforts that restricted in certain domains, or on a large volume of existing courses sourced from a certain platform. such solutions are not practically extensible into new or evolving domains. moreover, according to @xcite , some web data such as blog posts, tutorials and educational web pages are also suitable materials for learners. these rich web data are ignored by existing educational platforms such as google scholar and mooccube. in this paper, we wish to ease the need for human annotators by proposing a pipeline that automates resource discovery to similar unseen domains through transfer learning. besides, such a pipeline deals with multiple resource types to take advantage of web data. our contributions can be summarized into three parts. first, we present a self-sustaining pipeline for educational resource discovery in close unseen subject area or domain. we apply transfer learning with a novel pre-training information retrieval (ir) model, achieving competitive performances. we show that this pipeline achieves 0.94 and 0.82 f1 scores for two arbitrary target domains on discovering high-quality resources. second, we demonstrate an application that leverage resources discovered by our pipeline, survey generation for leading paragraph. lastly, we release the core source code of the pipeline, as well as the training and testing datasets, comprised of 39,728 manually labelled web resources and 659 search queries. 4 ## educational resource discovery pipeline we propose the educational resource discovery (erd) pipeline that aims at automatically recognizing high-quality educational resources. we model this problem as a resource classification task. given a resource r, where r can be any source type such as web page, pdf, we can obtain a list of features by feature engineering; based on these features, r is classified positive if it is a high-quality resource, otherwise negative. we illustrate the erd pipeline in figure 1 . it consists of data collection, feature extraction and resource classification. ## data analysis to better understand the collected data and our classifier's performance, we conduct a study on the features and corpus differences between the three experimental domains. ## application: survey generation for lead paragraphs in this section, we demonstrate an interesting application that applies the resources discovered using our erd pipeline, leading paragraph generation for surveys. novel concepts are being introduced and evolving at a rate that creates high-quality surveys for web resources, such as wikipedia pages, challenging. moreover, such existing surveys like wikipedia still needs human efforts on collecting relevant resources and writing accurate content on a given topic. researchers have been investigating automatic ways to generate surveys using machine learning and deep learning methods. survey generation is a way to generate concise introductory content for a query topic @xcite . while most of the existing work focuses on utilizing wikipedia to achieve this @xcite , little has been done for the web content. since our erd pipeline provides sufficient web data, we propose a two-stage approach for generating the lead paragraph that applies these web data selected from the erd pipeline. name resource type (with texts) domain number annotation size tutorialbank lecture sides, papers, blog posts nlp only manually 6,300 lecturebank lecture sides only nlp only manually 1,717 mooccube papers only multiple scrape from third-party 679,790 erd (ours) lecture sides, papers, blog posts multiple manually 39,728 ## conclusion in this paper, we proposed a pipeline for automatic knowledge discovery in novel domains. we applied transfer learning with a novel mlm pre-training method and achieved competitive classification performances. moreover, we demonstrated two applications that take advantage of resource discovered by our pipeline. finally, we released our source code and the datasets that we collected, including the 39,728 manually labelled web resources and 659 search queries. we plan to make this pipeline an online live educational tool for the public.