# Case Study: Students' Code-Tracing Skills and Calibration of Questions for Computer Adaptive Tests
## 3 Research  
The authors of this paper conducted a study among all three student cohorts of bachelor Informatics students at Subotica Tech--College of Applied Sciences. The research included 182 student participants who were taking an introductory course called 'Algorithms and data structures'. The course is composed of lectures, practices, and laboratory practices. The course runs in the spring semester over 15 weeks in the first year of study. It consists of one 90 min lecture per week as well as a 45 min practice and another 90 min lab practice. The aim of the course is to introduce the students to the basic concepts of algorithms and data structures in C/C++ programming language. The course covers the basic principles of programming, variables, control statements (_if, while, do-while, break_ and _continue_ statements), arrays, functions, pointers, and some basic algorithms (sort and search). Based on historical data, the grade average in this given course is 7.02 (5 is the minimum grade and means fail, and the maximum grade is 10), while the pass rate is about 50%. During their studies, informatics students learn new developing methods and developing environments as well as new programming languages. The novel techniques help them solve more complex information and communications technology (ICT) problems. Although the problems are more complex, and the developing environment and the programming language are new to them, when solving a particular problem, students frequently rely on those basic algorithms and data structures that they learned in their first year of study, e.g., iterations, arrays, or conditional statements. The programming languages that students use in their second and third years are so-called C-like programming languages, so the syntax and logic of the language are highly similar to those covered in the course 'Algorithms and data structures' during the first year of study. As stated earlier, the aim of this research is to analyze what happens to those basic programming skills acquired during the first year. The following questions arise:  
1. Do computer science students forget the basic algorithms and elements of language, or because of using them actively, do they understand them even better? 2. What is the impact of the constantly developing programming skills when estimating the difficulty of basic tasks? The authors formulated three hypotheses:  
**Hypotheses 1 (H1):**_Students in later years of study have a better understanding of basic algorithms and data structures._  
**Hypotheses 2 (H2):**_Subjective difficulty estimation of one task at later years of study will be lower._  
**Hypotheses 3 (H3):**_The difference between the students' and teachers' subjective estimation decreases with the years of study._  
**4. Data Collection and Analysis**  
The primary data collected were the students' answers in a test. A total of 182 students contributed data to this part of the study, but only 117 of those gave answers to all 44 questions. The students in the second and third year of a study were tested in January 2020. The test was conducted as a paper-and-pencil test with limited time. During the test, students were not allowed to use computers, smart phones, or any other type of help in answering the questions. The test consisted of multiple-choice questions, so the students had to mark one or more of the given answers or write the output of the code snippet. The students were given a time limit of 90 min for completing the test. The participants in Year 1 were given the same test in May 2020, but they had to complete it in an online environment due to the COVID-19 pandemic. Students took the test in the Moodle system. In all aspects, including questions themselves, their order, and the time limit, the online test was identical to the paper-and-pencil test. Altogether, 182 records were collected.