# Artificial Intelligence (ChatGPT) and Bloom's Taxonomy in Theoretical Computer Science Education
## 4 Research and Methods
### Educational Performance tests  
We designed an Experimental Test 1 (ET1) for the purposes of our experiment, elaborated according to [PERSON]'s LTT--questions 1-6 related to particular LTT types.  
[PERSON]'s Taxonomy [25]:  
1. Remembering: recalling or retrieving information from memory.
2. Understanding: comprehending or interpreting the meaning of information.
3. Applying: using knowledge or skills in a new or different situation.
4. Analyzing: breaking down information into parts and examining relationships among them.
5. Evaluating: making judgments about the value or quality of information or ideas.
6. Creating: using knowledge and skills to produce something new or original.  
The experimental questions in ET1 (Figure 1) were weighted based on the complexity of each particular question (in the analysis, we also present normalized results). The students were provided with questions in a bilingual version (Czech/English). AI was provided only with English versions. We designed 2 versions of ET1 with modified task content but equally balanced by difficulty. Each version was presented five times to ChatGPT ver 3.5 and 4.0. so that the ChatGPT versions acted as a group of 20 students (ver 3.5--10, ver 4.0--10). The idea behind this was to see if the groups provided consistent output from AI Chatbots which proved invalid.  
Figure 1: Sample of the real test for students (ver. 1) (Czech/English version of real test sheet).  
Experimental test 1 was performed during the standard learning process in the summer semester with students on course 7 GRAJ. ET1 contained 6 questions according to [PERSON]'s taxonomy, with weights of question points valued based on common practices (as in the standard course). Higher weights are assigned tasks aiming at application, evaluation and creative thinking. The resulting score for every solution (student, chatGPT) was evaluated on intervals 0 to max. value based on correctness:  
* completely incorrect solution (e.g., regular expression for other language),
* incorrect solution with partial correct information concerning the method used in how to solve the task (e.g., information on how to create the regular language, with some right and some wrong steps),
* correct method of solution with incomplete solution (e.g., correct steps to create language or corresponding automaton, but with wrong resulting expression),
* almost correct solution with simple mistake (e.g., expression only, not solving several cases, but generally right),
* completely correct solution (e.g., correct regular expression with every specific case of generated string).  
The ET1 ver. 1 example (Czech version) is shown in Figure 1 The translation of the Czech version, i.e., the ET 1 ver. 1 example (English version) is as follows:  
1. Provide an informal definition of a regular language and expression, with basic elements and operations. (3 pts.)
2. Is it possible to write a regular expression for a language of arithmetic expressions with operand \(x\), operators *, + and use of brackets, e.g., \((x+x)^{\bullet}x\)? (3 pts.)
3. Create a regular expression recognizing words containing symbols '\(a\)' and '\(b\)' starting with \(bb\) and ending with \(ab\). (6 pts.)
4. Is the regular expression \(ba(a+b)^{\bullet}ab\) generating also word \(bab\)? (3 pts.)
5. How do I have to change the language of arithmetic expressions with operand \(x\), operators *, + and use of brackets, e.g., \((x+x)^{\bullet}x\) to create a finite state machine recognizing this language? (5 pts.)
6. Create a deterministic finite state automaton for a language generated by regular expression \((a+ba^{\bullet})^{\bullet}(b+c)^{\bullet}\) with minimal number of states. (10 pts.)