# Group Assignments for Project-Based Learning Using Natural Language Processing--A Feasibility Study
## 4 Discussion  
TF-IDF and USE produced noticeably different numerical representations of the text proposals. \(S_{\text{TF-IDF}}\) and \(S_{\text{USE}}\) were modestly correlated with a Pearson correlation coefficient  
Figure 4: Histogram of similarity scores calculated by the human evaluators (**A**) and their comparison to NLP-based similarity measures (**B**). Figure 5: Dendrograms of clustering results using TF-IDF (**A**) and USE (**B**). of 0.54. This relatively low correlation indicates that the two NLP-based quantities capture different aspects of the texts. Considering the human evaluation results, \(S_{\text{USE}}\) is more correlated with \(S_{\text{human}}\) than \(S_{\text{TF-IDF}}\) is. This difference is discussed in more detail as follows. TF-IDF produced low similarity values for most of the text pairs. This was surprising because the same project topic (JetBot-based autonomous cars) for the proposals was given to the students. The fixed topic may have decreased the diversity of vocabularies and produced higher \(S_{\text{TF-IDF}}\) values because TF-IDF is only based on the frequencies of words. In contrast, the TF-IDF-based measure regarded most of the proposals as distinct (\(S_{\text{TF-IDF}}\ \approx\ 0\)). Thus, even for the same topic, the TF-IDF-based similarity measure could distinguish differences in the proposals. However, \(S_{\text{TF-IDF}}\) is limited as a similarity measure because it produced a limited range of values and lacked information regarding the level of similarity for post processing, such as clustering. In contrast, USE produced a wider range of similarity values than those of TF-IDF on the same dataset (Figure 3D). This higher sensitivity of \(S_{\text{USE}}\) originates from the _contextualized_ embedding in the transformer encoder structure. For instance, the same word in different proposals may coincide with different motivations or approaches. These semantic differences are undetectable by TF-IDF, which counts only the occurrences of the word. However, the contextual meaning of a word and its relationships with other words in a sentence can be captured by USE. This additional information can be beneficial during clustering (Figure 5). USE results were closer to the human evaluations than TF-IDF in comparing proposal pairs. This difference may be due to humans considering the occurrence of words and their contextual meanings and structures when comparing texts [31; 32; 33]. The transformer structure adopted in USE was designed to mimic how human readers pay attention to particular words in a text. We argue that the attention-based contextualization of USE produced more human-like similarity measures. The hierarchical clustering results are interpreted considering the existing studies on the ideal group size (4-7) as follows. Using the dendrograms presented in Figure 5, most similar texts are sequentially combined into a group until the group sizes range from four to seven. With TF-IDF (Figure 5A), it is natural to combine the most similar text pair (texts 7 and 8) into a group and add text 16 into the group to obtain a group size of three. As the group size is rather small, it would be reasonable to combine this group with another group of texts 2, 14, and 17 in the same hierarchy, resulting in a group of texts 2, 7, 8, 14, 16, and 17, sized (six) within the ideal range. Among the remaining texts, texts 3 and 5 are the most similar pairs and naturally form a group. However, at this point, it is unclear whether to combine this group of size two with the previous one of size six. The process is straightforward for the remaining texts. Combining texts 1 and 15 and merging them with texts 6, 12, and 13 forms a group of size five. The remaining texts 4, 9, 10, and 11 form another group. Similarly, the hierarchical clustering based on USE (Figure 5B) results in a group of texts 7, 8, 16, 11, 17, 15, and 14 (by the order of merging) and another group of texts 3, 12, 13, 6, 9, 2, and 4 (by the order of merging). Then, the remaining texts 1, 5, and 10 form another group.