# WEREC: An Unsupervised Method for Educational Concept Extraction Based on Word Embedding Refinement
## 2 Related Work
### Concept Extraction in Education
\begin{table}
\begin{tabular}{p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt} p{113.8 pt}} \hline \hline
**Methods** & **Core Processes** & **Strengths** & **Weaknesses** & **Articles** \\ \hline \multirow{8}{*}{Rule-based methods} & \(\bullet\) & Adhering to grammatical rules, semantic rules, and related aspects to process a corpus; Extracting multi-character units that conform to predefined rules and are labeled as concepts. & \(\bullet\) & Having the ability to manage well-defined patterns; Supporting domain-specific knowledge. Enabling access and providing transparency of the concept extraction process. & \(\bullet\) & Labor-intensive; Time-consuming; Disregards linguistic diversity. & [6,7] \\  & \(\bullet\) & Using a predefined concept dictionary to compare words or phrases in the text with dictionary entries; Employing similarity metrics or string-matching algorithms to extract relevant information. & \(\bullet\) & Depends on dictionary quality and coverage; Missing out-of-dictionary concepts; & [8,24] \\  & \(\bullet\) & Analyzing word frequency and co-occurrence patterns; Ranking based on statistical features to identify concepts in the text (e.g., weight-based ranking, graph-based ranking). & Having ability to model potential patterns among domain concepts; Allowing highly customized extracting domain concepts; & \(\bullet\) & Disregarding contextual information; & [9,25,26,27] \\  & \(\bullet\) & Using predefined grammar rules to identify candidate concepts; Utilizing pretrained models to obtain semantics vectors for candidates; Applying post-processing techniques to determine target concepts from the text. & \(\bullet\) & Obtaining higher precision and recall scores by capturing deeper meaning and contextual information in conceptual extraction; & \(\bullet\) & Relying on the quality and coverage of word embeddings; & [12,13] \\  & \(\bullet\) & Utilizing pretrained models to obtain semantic vectors for candidates; Applying post-processing techniques to determine target concepts from the text. & \(\bullet\) & Having challenges in evaluation and explainability. & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: A summary of generic concept extraction methods. to be identified within a course [18]. [PERSON] et al. used the skip-gram model with the Wikipedia corpus to ascertain word embedding vectors for concept extraction in massive open online courses [19]. To address noisy and incomplete annotations during high-quality knowledgeable concept extraction from these types of courses, [PERSON] et al. developed a three-stage framework [22]. It harnessed pre-trained language models explicitly and implicitly and integrated discipline-embedding models with a self-training strategy. These models are usually trained on large-scale corpora, making them highly robust and able to implicitly encode real knowledge concepts [32]. However, when using pre-trained models for concept extraction, the generality of corpora may cause extracted concepts not to match the semantics in a specific domain. Put simply, this method's feasibility is limited in domain-specific concept extraction in the absence of extensive and high-quality domain-specific corpora. Publicly available pre-trained word embedding models are sufficient for NLP tasks. Researchers from different fields have since fine-tuned these models on target domain texts to improve performance in downstream NLP tasks. In the legal domain, [PERSON] et al. developed the Legal-BERT model based on BERT and realized higher performance [33]. [PERSON] et al. showed that word embeddings trained on biomedical corpora captured the semantics of medical terms better than word embeddings trained on general domain corpora [34]. [PERSON] and [PERSON] noted that domain-specific large pre-trained models could have promising results for learning analytics [35]. Concept extraction performance can thus be enhanced by optimizing pre-trained models.