## 4 Case Study
### Graph Mining to Automatize Providing Operative Data  
One of the limitations of the proposed approach in the learning domain is the high amount of work necessary to collect the operative data about students' progress. To collect these data, every learning aspect of each assignment and every answer variant for each test question had to be connected to competence data to form the information for the graph \(G_{r}\). This makes creating and supporting the exercise bank a very strenuous job. One of the methods of automatizing this process is using ITS [40], which is an important subfield in Intelligent Education [41]. Of particular interest in this regard is model-based ITS that can automatically determine and store semantic errors made by students (e.g., Problets [42] and CompPrehension [43]). This information can be easily aggregated to create competence profiles in \(R\) because, in this case, there is no need to label every answer to a question or aspect of the learning problem; only high-level data about the possible semantic errors must be labeled (i.e., manual labeling happens once per kind of exercise instead of once per exercise). Then, the conclusions about competence development can be made from the tutoring-system logs, i.e., the data concerning which errors (mistakes) the student made and which they did not make while having an opportunity to make. This requires ITS to solve the task of analyzing a question (or learning problem) and determining the set of errors possible when solving it (which corresponds to the knowledge necessary to answer the question or solve the problem). Solving this task is beneficial to ITS because it allows for automatically classifying questions and  
Figure 11: Development levels of five competences for the same student visualized using the UGVA notation. Figure 12: Changes in the levels of competence development for the competence \(OK\)-12 during a semester in the UGVA notation. determining the pedagogical conditions of their usage (i.e., prerequisite knowledge and target knowledge) and creating meaningful logs that can be mined to retrieve data for analyzing the learning situation and creating individualized images in the UGVA notation. In some fields, where learning problems' formulations can be mined from real-world objects (e.g., mining open-source software for generating learning problems to teach computer science and software engineering). The pilot experiments in generating learning problems for teaching determining order of evaluation of expressions for teaching programming showed the viability of this technique [44]. This kind of learning problems produces the following possible mistakes:  
* An attempt to evaluate an operator whose operand is in parentheses before the expression in the parentheses;
* An attempt to evaluate a two-token operator (e.g., array access or ternary conditional operator) before the operators inside its tokens;
* An attempt to evaluate the operator with lower precedence first (from two subsequent operators);
* An attempt to evaluate two operators with the same precedence contrary to the order defined by their associativity;
* An attempt to evaluate the right operand of a strict-order operator (i.e., logical AND and OR operator, ternary conditional operator, or comma operator) before the evaluation of its left operand completes;
* An attempt to evaluate the operator that should not be evaluated (e.g., the right operand of the logical AND operator is evaluated only if its left operand is true);
* Finishing evaluating expression without evaluating an operator that should be evaluated. Expressions in the C programming language were extracted from open-source code and converted first to abstract syntax trees, then to knowledge graphs in the RDF format (dynamic part of \(R\)).