# Semantic Similarity Analysis for Examination Questions Classification Using WordNet
## 4 Result and Discussion
### Similarity Matrix Scores Analysis and Sorting in Question Classification Module  
The question classification (QC) module uses the WordNet Similarity approach to obtain similarity scores between the verbs extracted--including RWs and KWs--and the verbs from the BT list--BVs. Every verb, either a RW, KW or BV, has several synsets that can range from a few to as many as 75 synsets. WordNet generates a similarity score between each synset from the RWs or KWs and each synset from the BVs. Huge similarity scores are generated, and a two-dimensional matrix is utilized to store the similarity scores, where the matrix's X axis is the BVs' synset and the Y axis is the RWs' or KWs' synset. Figure 2 is an example of the two-dimensional similarity scores matrix of RW--\"write\"--and the BVs of Bloom Level 1. The RW--\"write\"--consisted of ten synsets, while the Bloom Level 1 consisted of 14 BVs with a total of 77 synsets. As a result, a total of 770 similarity scores was generated and stored in the two-dimensional matrix as shown in Figure 2. The similarity scores in the matrix were then sorted from a high to low score value, as shown in Figure 3. Because there were six Bloom levels, the two-dimensional matrix extended to the three-dimensional matrix where the Z axis referred to BT levels, as in Figure 4.  
Subsequently, the matrix needed to be summarized into a simple score representation for classification, which was a single similarity score for each Bloom level. Three types of statistical approaches were identified--the mean, median and maximum. Table 8 shows an example of the RW where \"write\" is summarized using the three statistical approaches. Both mean and maximum approaches resulted in a conclusive highest score at the single Bloom level, but the mean approach had the highest similarity score at Bloom Level 1, and the maximum approach was at Bloom Level 6. An experiment was conducted to evaluate and identify the best statistical approach to be used and summarize the similarity matrix into a single-score representation. Experiment results have shown the mean approach to be the best approach considering how all test questions can be successfully classified to a single Bloom level with the highest accuracy, as shown in Table 9. \begin{table}
\begin{tabular}{l l c} \hline \hline
**Process** & **Grade** & \begin{tabular}{c} **The 200 Single-Sentence** \\ **Question Dataset** \\ \end{tabular} \\ \hline \multirow{4}{*}{\begin{tabular}{l} Verb extraction process to extract the word tokens tagged with “VB” only by the \\ Stanford POS tagger with the “left3 words” model. \\ \end{tabular} } & Accurate & 80.5\% \\ \cline{2-3}  & \begin{tabular}{l} Accurate \\ \end{tabular} & 8.0\% \\ \cline{2-3}  & \
\begin{tabular}{l} ERR1 \\ \end{tabular} & 8.0\% \\ \cline{2-3}  &
\begin{tabular}{l} ERR2 \\ \end{tabular} & 11.5\% \\ \hline \hline \end{tabular}  
\begin{tabular}{l} Rule Applied \\ \end{tabular}
\begin{tabular}{l} Rule \#1: First Word is a Verblf meeting following conditions: \\ 1. \\ 2. \\ 3. \\ 3. \\ 3. \\ 3. \\ 3. \\ \end{tabular}
\begin{tabular}{l} Rule \#2: “VBP” and “VBZ”: \\ 1. \\ 2. \\ \end{tabular}
* There are many word tokens tagged with “VB”; \\ 2. \\ \hline \hline \end{tabular}
\begin{tabular}{l} Verb extraction process to extract the word tokens tagged with “VB” only by the \\ Stanford POS tagger with the “left3 words” model, was then applied to RULE \#1 and \#2 \\ \end{tabular}  
\end{table} 
Table 7: Effectiveness of verb extraction process after applying rules. Figure 4: A three-dimensional similarity score matrix. Figure 3: A sorted two-dimensional similarity score matrix in descending order. Figure 2: A two-dimensional similarity score matrix. According to [PERSON] and [PERSON] [4], they filtered and modified the data lists in their research by collecting only 75-100% of the synset verbs in order to obtain better accuracy with a similarity measurement.