# ChatGPT in Computer Science Education: A Case Study on a Database Administration Course
## 4 Discussion
\begin{table}
\begin{tabular}{l l l l l} \hline \hline \multicolumn{1}{c}{**Item**} & \multicolumn{1}{c}{**Usage**} & \multicolumn{1}{c}{**Reliability**} & \multicolumn{1}{c}{**Utility**} \\  & & \(\boldsymbol{\rho}\) **(\(\boldsymbol{p}\)-Value)** & \(\boldsymbol{\rho}\) **(\(\boldsymbol{p}\)-Value)** & \(\boldsymbol{\rho}\) **(\(\boldsymbol{p}\)-Value)** \\ \hline
1 & Assignment report & \(0.22\) (0.16) & \(0.07\) (0.66) & \(0.07\) (0.66) \\
2 & Assignment materials (i.e., teacher tutorials) & \(-0.51\) (\(<\)0.001) * & \(-0.28\) (0.11) & \(-0.39\) (0.02) \\
3 & MySQL official documentation & \(-0.29\) (0.06) & \(-0.08\) (0.65) & \(-0.24\) (0.20) \\
4 & Traditional internet resources & \(-0.20\) (0.19) & \(-0.10\) (0.61) & \(-0.09\) (0.64) \\  & (Google, Stack Overflow, etc.) & & & \\
5 & ChatGPT v3.5 & \(0.45\) (0.003) * & \(0.42\) (0.01) * & \(0.48\) (0.005) * \\ \hline \hline \end{tabular}
\end{table} 
Table 9: Correlation between the student’s grade and the usage, reliability and utility of resources during the exam realization. solution usually came from a Socratic question of the teacher, a targeted instruction from a classmate, or other traditional sources. Lastly, note that the use of ChatGPT among students is remarkably uneven. It is striking to observe the distribution of ChatGPT usage among students if they are categorized into the aforementioned groups (see Figure 2). While 100% of the students who obtained an outstanding score (grade higher than 7) in the practical exam used ChatGPT during the exam, only 61% of the students who failed used it. However, we must analyze more thoroughly the degree of use of ChatGPT during the exam (see Table 6), as well as the reliability (see Table 7) and the utility (see Table 8) perceived by the students. It can be seen that the students who obtained the best marks used ChatGPT intensively (4 out of 5), significantly more than the other students. In the same vein, these outstanding students also perceived ChatGPT to be much more reliable (3.80) than students who failed the exam (2.75), finding here a statistically significant difference (\(p<0.05\)) with a large effect size (\(\eta^{2}>0.14\)). Another statistically significant difference (\(p<0.05\)) with large effect size (\(\eta^{2}>0.14\)) was found by comparing the usefulness of ChatGPT perceived by outstanding students (4.20) with those who failed the exam (2.92). Therefore, the group of students with the best scores in the exam was the one who used ChatGPT the most and had the best perception of its reliability and usefulness. This may be due to multiple reasons that are explored in the following research question. RQ2. How Does Prior Knowledge About Database Administration Affect the Use of ChatGPT to Solve Database Administration Problems? First, it is relevant to observe the progression of knowledge acquired by the students throughout the course (see Table 1). It can be seen that at the beginning of the course, the students knew practically nothing about database administration (Test 1, 0.60 out of 10), that once the theoretical part of the course was finished, they had acquired several theoretical concepts (Test 2, 4.73), and that at the end of the practical part of the course, they had acquired almost all of them (Test 3, 8.43). The inferential comparative results calculated in this regard (see Table 2) indicate that the differences in the results between the tests performed at the different times of the course are statistically significant (\(p<0.001\)) and have a large effect size (r > 0.50). However, the student performance is very uneven. In fact, as indicated above, of the 40 students involved in this case study, 18 finally obtained a grade lower than 5 in the practical exam (Group 1), 7 students obtained a grade between 5 and 7 (Group 2), and 15 students obtained a grade higher than 7 (Group 3). We must analyze the progress in the acquisition of knowledge of these three groups of students.