# Artificial Intelligence (ChatGPT) and Bloom's Taxonomy in Theoretical Computer Science Education
## 4 Research and Methods
### Qualitative Analysis  
A more thorough qualitative analysis based on LTT also showed interesting results, even though the sample size and test extent do not ensure quantitative conclusions. **Hypothesis 2. ** The results of the educational performance of AI and students in particular types of Learning Objectives are equal. **Hypothesis 3. ** There is a difference between the results of AI and students in particular LTT types. \begin{table}
\begin{tabular}{c c c c c c c c c} \hline \hline  & & \multicolumn{6}{c}{(**a**)} \\ \hline
**Factor** & & **Statistic** & & **df** & & \(p\) \\ \hline subjectPrec & & & 29.567 & & & 5 & & \textless{}0.001 \\ \hline \multicolumn{8}{c}{(**b**)} \\ \hline
**Comparison** & **Z** & **Wi** & **Wj** & **rrb** & & \(p\) & **pbonf** & **pholm** \\ \hline GPT3-GPT4 & \(-0.918\) & 9.65 & 15.5 & 0.36 & 0.359 & & 1 & 1 & 1 \\ GPT3-HUA & \(-4.833\) & 9.65 & 37.767 & 0.96 & \textless{}0.001 & *** & \textless{}0.001 & *** & \textless{}0.001 & *** \\ GPT3-HUB & \(-2.851\) & 9.65 & 31.9 & 0.98 & 0.004 & ** & 0.065 & & 0.057 & \\ GPT3-HUC & \(-2.346\) & 9.65 & 26.917 & 0.8 & 0.019 & * & 0.284 & & 0.227 & \\ GPT3-HUD & \(-2.027\) & 9.65 & 28.667 & 1 & 0.043 & * & 0.64 & & 0.426 & \\ GPT4-HUA & \(-3.828\) & 15.5 & 37.767 & 0.88 & \textless{}0.001 & *** & 0.002 & ** & 0.002 & ** \\ GPT4-HUB & \(-2.101\) & 15.5 & 31.9 & 0.78 & 0.036 & * & 0.534 & & 0.392 & \\ GPT4-HUC & \(-1.551\) & 15.5 & 26.917 & 0.55 & 0.121 & & 1 & 1 & \\ GPT4-HUD & \(-1.404\) & 15.5 & 28.667 & 0.733 & 0.16 & & 1 & 1 & 1 \\ HUA-HUB & 0.797 & 37.767 & 31.9 & 0.4 & 0.425 & & 1 & 1 & \\ HUA-HUC & 1.576 & 37.767 & 26.917 & 0.533 & 0.115 & & 1 & 1 & 1 \\ HUA-HUD & 1.01 & 37.767 & 28.667 & 0.644 & 0.313 & & 1 & 1 & 1 \\ HUB-HUC & 0.578 & 31.9 & 26.917 & 0.267 & 0.564 & & 1 & 1 & 1 \\ HUB-HUD & 0.311 & 31.9 & 28.667 & 0.2 & 0.756 & & 1 & 1 & 1 \\ HUC-HUD & \(-0.174\) & 26.917 & 28.667 & 0.111 & 0.862 & & 1 & 1 & 1 \\ \hline \hline \end{tabular}
\begin{tabular}{c c c c c c} \hline \hline  & & & & & & & \\ \end{tabular}
\end{table} 
Table 5: (**a**) Kruskal–Wallis Test of overall results in ET 1. (**b**) [PERSON]’s post hoc comparisons—overall results ET1. **Question 1** (LTT type 1--Remembering):  
This question tested the subject's ability to present concise definition and an explanation of a basic notion. Example of Question 1 (LTT type 1--Remembering)--Finite state machine. This type of question is very suitable for an AI-based language model like GPTChat. The answers (Figure 3) were well commented upon, only slightly ambiguous answers about the final states were produced by GPTChat ver. 3.5, and ver. 4 a produced a model answer (note that GPTChat defined a more advanced version of a finite state machine with outputs, which was not necessary). In some instances, GPTChat ver. 4 also produced a very good real-world model example of robot behavior. Students produced fewer comprehensive answers with occasional errors or even completely wrong answers, as in Figure 4 (a student with an excellent overall result mixed up the definitions of FSM, RL and RE, which made no sense)  
When comparing only this version 1 of ET 1 (another has an opposite version of the question which is easier to answer), we can see that the differences are not statistically significant. However, students with better (HUA, HUB) results in 7 ZAIN have better results than both GPTChat versions, and students from groups HUC, HUD have even worse results (Figure 8). Figure 8: Results of Q2 ver1. (**a**) Box plot, (**b**) raincloud plot. Figure 7: Interaction (continued from Figure 6) to the sample answer from GPTChat ver3.5 to level 2 question (ver. 1 test). **Question 3** (LTT type 3--Applying):  
Question type 3 focused on testing the application of knowledge to a practical problem of a regular expression design. This question was also a success for GPTChat, since it is apparently capable of applying a language model to create a simple string with a prefix and a suffix. Version1 required a little bit of an easier question, in which the universal RE is complete.