# WEREC: An Unsupervised Method for Educational Concept Extraction Based on Word Embedding Refinement
## 4 Experiments
### Experimental Results
#### 4.3.2 Effects of Parameter Settings on WERECE Performance  
A series of experiments were completed on the EDUTECH-DT dataset to examine how parameter settings influenced our proposed approach. Figures 5 and 6 showcase how the precision and recall values changed as \(M\), \(K\), and \(N\) jointly varied across the pre-specified set of parameter combinations in Table 3.  
Figure 4: Performance comparisons of WERECE and K-means on the EDU-DT dataset. Figure 5: Effects of parameter settings on WERECE’s precision. Note that \(M\) denotes the dimension of refined embedding vectors; \(N\) represents the number of seeds; \(K\) is the number of clusters; and the value of precision ranging from 0.0 to 1.0 is colored in sections. As shown in Figure 5, when \(M\) increased from 5 to 20, WERECE's precision on the test set improved greatly. The proposed method also performed consistently when \(M\) was between 50 and 140. The re-embedding word embeddings with dimensions above 20 retained substantial semantic information [62]. Additionally, more seeds generally led WERECE to be more precise. The effects of \(M\) and \(K\) on precision fell dramatically as \(N\) rose. For example, WERECE's precision was significantly compromised when \(N\) was set to 300. This pattern mirrors the argument that insufficient semantic information and highly complex prediction concerns an hamper model performance [63]. Moreover, the number of clusters, \(K\), slightly affected WERECE's precision. Thus, when applying a pre-trained embedding model to a domain-specific downstream NLP task, the model must be refined in order to perform well--unless the training dataset contains many samples. In light of these results, it can be found that WERECE demonstrated encouraging performance on educational concept extraction. It was also adequately robust against varied parameter settings (i.e., in terms of precision). An \(M\) between 50 and 140 appeared optimal. Similarly, the higher the \(N\), the better WERECE performed. Figure 6 illustrates the collective effect of different parameter settings on WERECE's recall performance on the test set. Overall, as \(M\), \(N\), and \(K\) jointly increased, the recall scores substantially decreased. Irrespective of the specific values of \(N\) and \(K\), the recall scores exhibited a clear pattern of initially climbing and later falling as \(M\) rose. This observa  
Figure 6: Effects of parameter settings on WERECE’s recall. Note that \(M\) denotes the dimension of refined embedding vectors; \(N\) represents the number of seeds; \(K\) is the number of clusters; and the value of recall ranging from 0.0 to 1.0 is colored in sections. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Parameter**} & \multicolumn{3}{c}{**Value Ranges**} \\ \cline{2-4}  & _Start_ & _End_ & _Step_ \\ \hline Dimension of refined embedding vectors (\(M\)) & 5 & 200 & 15 \\ \hline Number of seeds (\(N\)) & 300 & 1800 & 150 \\ \hline Number of clusters (\(K\)) & 2 & 42 & 4 \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Parameter values for the WERECE method. tion was likely due to redundant semantic information once \(M\) exceeded 65. WRECE's recall performance with higher \(N\) values declined marginally as \(M\) increased; its recall performance with smaller \(N\) values decreased greatly. Its recall performance changed more moderately when dealing with smaller (vs. larger) \(K\) values. This tendency suggests that WRECE's robustness improved when subjected to a low number of clusters. In particular, taking advantage of a larger \(N\) and a smaller \(K\) enabled WRECE to mitigate the impact of \(M\) variations on recall performance. Hence, WRECE is sensitive to parameter settings in terms of recall and has the potential to produce optimal recall scores when specific parameter settings are met: \(N\geq 1000\), \(K\leq 18\), and \(M\leq 65\).