Unsupervised Cross-Domain Prerequisite Chain Learning using Variational Graph Autoencoders (2021)

Abstract: Learning prerequisite chains is an important task for one to pick up knowledge efficiently in both known and unknown domains. For example, one may be an expert in the natural language processing (NLP) domain, but want to determine the best order in which to learn new concepts in an unfamiliar Computer Vision domain (CV). Both domains share some common concepts, such as machine learning basics and deep learning models. In this paper, we solve the task of unsupervised cross-domain concept prerequisite chain learning, using an optimized variational graph autoencoder. Our model learns to transfer concept prerequisite relations from an information-rich domain (source domain) to an information-poor domain (target domain), substantially surpassing other baseline models. In addition, we expand an existing dataset by introducing two new domains—-CV and Bioinformatics (BIO). The annotated data and resources as well as the code will be made publicly available.

## introduction with the rapid growth of online educational resources in diverse fields, people need an efficient way to acquire new knowledge. building a concept graph can help people design a correct and efficient study path @xcite . there are mainly two approaches to learning prerequisite relations between concepts: one is to extract the relations directly from course content, video sequences, textbooks, or wikipedia articles @xcite @xcite , but this approach requires extra work on feature engineering and keyword extraction. our method follows a different approach of inferring the relations within a concept graph @xcite @xcite . in a concept graph, we define @xmath0. existing methods formulate this question as a classification task. a typical method is to encode concept pairs and train a classifier to predict if there is a prerequisite relation @xcite . however, this method requires annotated prerequisite pairs during training. alternatively, others have used graphbased models to predict prerequisite relations. gor existing work mainly focuses on prerequisite relations within a single domain. in this paper, we tackle the task of cross-domain prerequisite chain learning, by transferring prerequisite relations between concepts from a relatively information-rich domain (source domain) to an information-poor domain (target domain). as an example, we illustrate in figure 1 , a partial concept graph from the natural language processing (nlp) domain and a partial concept graph from the computer vision (cv) domain. prerequisite relations among concepts in the nlp domain are known, and we seek to infer prerequisite relations among concepts in the cv domain. these two domains share some concepts, such as convolutional neural network. we assume that being aware of prerequisite relations among concepts in the source domain helps infer potential relations in the target domain. more specifically, in the figure, knowing that convolutional neural @xmath1. our contributions are two-fold. first, we propose cross-domain variational graph autoencoders to perform unsupervised prerequisite chain learning in a heterogeneous graph. our model is the first to do domain transfer within a single graph, to the best of our knowledge. second, we extend an existing dataset by collecting and annotating resources and concepts in two new target domains. data and code will be made public in https://github.com/yale-lily/ lecturebank/tree/master/lecturebankcd . ## dataset lecturebank2.0 @xcite dataset contains 1,717 lecture slides (hereon called resources) and 322 concepts with annotated prerequisite relations, largely from nlp. we treat this dataset as our information-rich source domain (nlp). also, we propose an expansion dataset, lecturebankcd, by introducing two new target domains in the same data format: cv and bioinformatics (bio). we report statistics on the dataset in domain files pages tks/pg con. posrel nlp 1,717 65,028 47 322 1,551 cv 1,041 58,32 43 201 871 bio 148 7,13 135 100 234 ## methodology inspired by @xcite , we build a crossdomain concept-resource graph @xmath2, a) that includes resource nodes and concept nodes from both the source and target domains (figure 2 ). to obtain the node feature matrix x, we use either bert @xcite or phrase2vec @xcite embeddings. we consider four edge types to build the adjacency matrix a: a c,s : edges between source concept nodes; a rc : edges between all resource nodes and concept nodes; a r : edges between resource nodes only; and a c,t : edges between target concept nodes. in unsupervised prerequisite chain learning, a c,s -concept relations of the source domain-are known, and the task is to predict a c,t -concept relations of the target domain. for a rc and a r , we calculate cosine similarities based on node embeddings, consistent with previous works @xcite . cross-domain graph encoder vgae @xcite ) contains a graph neural network (gcn) encoder @xcite and an inner product decoder. in a gcn, the hidden representation of a node i in the next layer is computed using only the information of direct neighbours and the node itself. to account for cross-domain knowledge, we additionally consider the domain neighbours for each node i. these domain neighbours are a set of common or semantically similar concepts from the other domain. 1 we define the cross-domain graph encoder as: ## evaluation we evaluate on our new corpus lecturebankcd, treating the nlp domain as the source domain and transferring to the two new target domains: @xmath3. consistent with @xcite ; @xcite , we randomly split the positive relations into 85% training, 5% validation, and 10% testing. to account for imbalanced data, we randomly select negative relations such that the training set has the same number of positive and negative relations. we do the same for the validation and test sets. we report average scores over five different randomly seeded splits. to encode concepts and resources, we test bert and p2v embeddings. for bert, we applied a pretrained version from google 3 . we trained p2v using all the resource data. both methods only require free-text for training and encoding. baseline models we concatenate the bert/p2v embeddings of each pair of con-cepts and feed the result into a classifier (cls + bert and cls + p2v). we train the classifier on the source domain only, then evaluate on the target domain. we report the best performance among support vector machine, logistic regression, gaussian naïve bayes, and random forest. in addition, we train the vgae model @xcite on the source domain and test on the target domain, initializing the vgae input with bert and p2v embeddings separately (vgae + bert and vgae + p2v). given that gae is structurally similar to vgae, we leave this for future work. other graph-based methods including deepwalk @xcite and node2vec @xcite are not applicable in this setting as both models require training edges from the target domain in order to generate node embeddings for target concepts. proposed method we report results of our proposed model, cd-vgae, initialized with bert and p2v node embeddings separately. consistent with the work from @xcite and @xcite , p2v embeddings yield better results than bert embeddings in general. one possible reason for this difference is that bert embeddings have a large number of dimensions, making it very easy to overfit. the two cls models yield a negative result, with f1 worse than random guess. a possible reason is that treating concept pairs independently from the source domain may not be beneficial for the target domains. the vgae models have a better performance when considering the concepts in a large graph. as shown in the table, our method performs better than the chosen baselines on both accuracy and f1 score, by incorporating information from domain neighbors. in particular, it yields much higher recall than all the baseline models. we provide further analysis in a later section. upper bound performance finally, we conduct in-domain experiments on cv and bio (supervised training and testing in the target domain), to show an upper bound for cross-domain performance. we test a variety of methods including traditional classifiers as well as graph-based approaches, including deepwalk, node2vec, and graphsage @xcite . ## analysis next, we conduct quantitative analysis and case studies on the target domain concept graphs recovered by our model (cd-vgae+p2v) and two ## conclusion in this paper, we proposed the cd-vgae model to solve the task of cross-domain prerequisite chain