# An Ontology-Driven Learning Assessment Using the Script Concordance Test
## 2 Methods and Materials
### Proposed Framework
The evaluator sets up the desired number of questions and strategy (difficulty level) using AppSheet [47]-based mobile application (config step in Figure 3). After that, the question generation leveraging ontology-driven code algorithm is executed. During the code generation process, the algorithm executes SPARQL queries against the semantic knowledge base containing the knowledge stored with respect to the form of the desired question type ontology (SCTonto in this case). Optionally, additional domain-specific ontologies might be included (such as Disease Ontology) to support the code generation process. The domain ontologies provide necessary knowledge used for automated question generation mechanisms related to the targeted difficulty level of the question. Results of the SPARQL queries are used to fill in the parameters relevant to the desired type of questions. Moreover, results of the code generation are inserted into the Google Sheets document using the Google Sheets API client in Java [48]. The Google Sheets document is used by AppSheet to generate a mobile application. AppSheet is quite effective when it comes to the rapid creation and distribution of multiplatform web-based mobile applications. Finally, the code generation results are visualized back to the evaluator, so they can be further distributed to the target audience (such as students) via an AppSheet-based mobile app. Apart from AppSheet, the mobile app relies on Google Appscript triggers for backend capabilities related to testing evaluation and score calculation. In Listing 5, the pseudo-code summarizing the overall question generation procedure for a single question of SCT-based assessment is given. During this procedure, the values of question properties defined by SCT question ontology are populated. The corresponding Likert-scale-score calculation procedure is executed, depending on the selected strategy. In Listing 6, an example of a SPARQL query is used to retrieve all possible new information (lab results) for a given hypothesis (disease is given). The screenshots of the AppSheet-based mobile app for SCT-based assessment from students' perspectives are given in Figure 4. There are three main views. The first one (4a) shows the list of questions that are part of a test. The second (4b) provides the interface to answer the selected question by setting the value \"Y\" to one of the values from the Likert scale. Finally, the third screen (4c) shows an overview of scores obtained on previous tests by questions. **Listing 5. ** Summarization of SCT question generation procedure. Input: disease name, selection strategy, difficulty level

Output: SCT question

Steps:

1. Config(disease name, selection strategy, difficulty level);
2. QuestionGeneration(disease name, selection strategy) {
3. CaseDescription: = GetSoapNoteForDisease(disease name);
4. NewInformation: = SelectLabResults(disease name, selection strategy);
5. } 6. LikertScaleScoreCalculation(disease name, new information, selection strategy, difficulty level);
7. For each property in SCT ontology
8. InsertTriplet(property, value);
9. Endfor;
10. End. **Listing 6. ** SPARQL query for new information retrieval based on disease hypothesis. PREFIX sct: <[[http://www.example.com/sct/](http://www.example.com/sct/)]([http://www.example.com/sct/](http://www.example.com/sct/))>

SELECT DISTINCT?ni

WHERE {

GRAPH <[[http://www.example.com/sct1](http://www.example.com/sct1)]([http://www.example.com/sct1](http://www.example.com/sct1))> {

?h sct:hasValue?disease. ?h sct:isPossibleAffectedBy?ni. FILTER(regex(STR(?disease),

\"Chronic Obstructive Pulmonary Disease\"))

Figure 4: AppSheet-based SCT mobile app for students. ## 3 Results

The time-based performance evaluation of the proposed framework for the automated, ontology-based generation of questions for SCT-based assessment in medical education is given and compared with the manual creation of the same questions. The evaluation was performed on a laptop equipped with Intel i7 7700 HQ CPU, 16 GB DDR4 RAM, and 1 TB HDD, running on Windows 10.