## 5 Discussion  
The Elo-bandit algorithm shows promise as an alternative for IRT based and CTT based tests in formative assessment applications. Despite the pedagogical value formative assessments provide [2, 18], they have an opportunity cost: assessments consume time that could be spent directly studying course material through lectures and self-study. Time spent on each activity has a demonstrable effect on learning outcomes [58]. In addition to time savings, shorter assessment length may combat decision fatigue, possibly leading to more efficient post-test learning [59]. The Elo-bandit algorithm provides an opportunity to benefit from formative assessment while saving time and mitigating fatigue. Potential quiz length reduction observed in the experimental setting was about 50%. In addition, informative assessment precision is less important than summative assessment. Imprecision in summative assessment may lead to failure to obtain employment or admission to the university to a deserving candidate. While a degree of imprecision in formative assessment may cause a teaching system to serve a suboptimal learning object. Therefore, it might be rational to aim at lower precisions such as 80 or 90 percent when it allows for significant quiz length reduction. In the presented 2021 case study with an item pool consisting of 80 questions median time of assessment amongst 106 students was 39 min or about 30 s per item. Settling for 80% or 90% precision would allow to cut median assessment time to less than 8 and 12 min respectively. Such reduction in quiz length may allow instructors to incorporate formative assessment in the learning process where previously it was impractical. From an educational point of view pedagogical value of a method is a function of more than just its mathematical properties. The viability of the method also depends on pedagogical scaffolding around it and, of course, student psychology. To gauge the algorithm's impact on learning outcomes further experiments would be needed that are beyond the scope of this paper. Perhaps the key factor limiting IRT based assessment in classrooms is the need for calibration [26]. A teacher cannot simply write up and add items to a question bank as they teach. In addition to requiring a large student pool, unavailable to many teachers, item difficulty parameters are also dependent on the student background exacerbating the problem. The proposed in this paper algorithm does not require calibration, to the best of my knowledge no similar formative assessment solution have been proposed and applied using data from real students. However, Elo rating has been used to estimate learning object difficulty. In 2019 [PERSON] et al. published data from 22 students indicating positive results in the programming exercise recommendation system [35]. [PERSON] and [PERSON] found Elo rating to be a superior task difficulty measure to proportion correct and learner feedback [60]. Finally, based on simulations [PERSON] has found that Elo rating may be preferable to IRT for adaptive item sequencing in practice systems as it does not require item pretesting [61]. These findings are in agreement with positive results obtained in this study: Elo rating is a viable alternative to IRT assessment in a formative context. Of course, item difficulties are subjective to each student, as everyone possesses unique competencies and talents. However, this effect can be lessened if a group of students being assessed has a similar academic background. This is easier to achieve for methods that do not require pre-testing, as the group being tested itself becomes the reference of difficulty. Simulations demonstrated significant algorithm performance differences in comparison with real data implying that generation parameters used for synthetic students did not reflect students in the classroom. The algorithm's accuracy gets worse with the decrease in item difficulty and correlation. Real items were largely uncorrelated, but more difficult, resulting in shorter quiz lengths.