## 4 Case Analysis
### Evaluation of Students' Competencies with the Help of Competency Tree
\begin{table}
\begin{tabular}{c c c} \hline \hline  & **Traditional Evaluation** & **Proposed Evaluation** \\ \hline Number of tasks in the exam & 1 & 8 \\ Level of evaluated competencies in one task & 1 & \([1;5]\) \\ Grade range & \([0,100]\) & \([0;100]\) \\ Number of students (who tried to do the task) & 23 & 25 \\ Average number of task submission for evaluation & 3.17 & 4.96 \\ Average score in e-evaluation & 47.83 & 78.80 \\ Standard deviation of the score in e-evaluation & 51.08 & 27.79 \\ Score step in e-evaluation & 100 & 1 \\ Average score in tutor evaluation & 72.17 & 72.40 \\ Standard deviation of the score in tutor evaluation & 34.32 & 28.33 \\ Score step in tutor evaluation & 10 & 10 \\ Range of system and tutor scores & \([-80;30]\) & \([-30;40]\) \\ Average difference between system and tutor scores & \(-24.35\) & 6.4 \\ Standard deviation of the difference between system & 35.14 & 15.78 \\ and tutor scores & & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: Student evaluation results by using the traditional and proposed method. calculated by the system. Tutor's scores are presented in Figures 7 and 8 for 1 st and 2 nd group of students, respectively. The average difference between system and tutor score for each student in group 1 was 24%, while in group 2-6%, respectively. The standard variation of the difference between system and tutor score for groups 1 and 2 was 35% and 15%, respectively. This shows that the proposed e-evaluation method with multiple tasks can calculate the students' knowledge score, similar to tutors evaluation. The results possibly could be improved by adding more assignments in the task. A big variety of tasks gives additional flexibility and adaptability for the e-learning and e-evaluation. Each student has his learning and competency history; therefore, each student might need a different path for his new competencies approval. Also, it is worth mentioning that five students from 14 in group 2 after failing in the main assignment successfully performed more than 1 assignment with a lower score. Two of them selected not connected competencies while three did the easier task first and then moved to a higher level. This illustrated the ability to adapt the test according to the individual learning style of the student. Traditional e-evaluation of programming code is based on input/output comparison and gives no chance to evaluate partially achieved competencies in unfinished tasks. The system does not analyze who influences the wrong answer; therefore, one small mistake leads to the neglecting of achieved competencies. This is noted in comparison of the e-evaluation system and tutors scores--e-evaluation system evaluates tasks of group 1 by assigning 100% or 0% score, while the tutors' score for the same task varies and has more possible values. Some e-evaluation system 100% scores for the task were  
Figure 8: Mark distribution for group 2. Figure 7: Mark distribution for group 1.  
reduced by the tutor as despite the output, the programming code was missing some elements. Only one student who scored 0% by the e-evaluation system had the similar score from the tutor evaluation, and the rest of the 11 students demonstrated some competencies and were able to get a bigger score even though the task was not fully finished. Despite the differences between the scores of the e-evaluation systems and the tutors, the e-evaluation with one task has another important issue--one traditional e-evaluation task is not dedicated to the evaluation of achieved student's competencies. There is no systemic competency tracking between different tests of study subjects. As well the final score does not define what the student is missing, what competencies, areas need to be improved. During the comparison of e-evaluation and tutor scoring, one important problem was spotted--some students were able to cheat the system by implementing the system differently than the requirements requested.