# R2D2: A Dbpedia Chatbot Using Triple-Pattern Like Queries
## 3 Architecture
### Results2 Text
The results are reported in Table 1 for the various evaluation categories. Values greater than three represent the high level of the specific software feature; values between 2.5 and 3 are at low risk, whereas values below 2.5 are considered high risk. In our case, all were graded with a satisfactory average above 3.47, showing the high quality of the software. The lowest average score was for the reliability (3.7/5), since in some cases, errors appeared (e.g., images not shown) without allowing a proper recovery or adequate explanation. Reliability of the system is also affected by the external services that are used (directly from DBpedia), as in several cases, the services were not functioning--however, not during the user evaluation. In addition, as the different services have been developed using many different technologies, users identified as a relatively hard task the maintainability of the system (average maintainability 3.94/5). The questions regarding the quality of use also score relatively high, demonstrating again the quality of the system. The evaluation provided us with valuable feedback on the current state and was used to correct the reported errors. The highest average score for efficiency (4.25/5) shows that the system can quickly respond to the queries issued and uninterruptedly provide answers from such a large source of information. In essence, the execution time required for query answering is the time required to return the result through DBpedia SPARQL endpoint, with only milliseconds spent in the trivial SPARQL formulation and result presentation. Furthermore, the SUS usability score was 78 on a scale of 0 to 100, whereas the inter-rater agreement was rather good (kappa 0.8). The average SUS score from published studies has been measured by [PERSON] et al. (2016) as 62.1, but 68 is often used as a gold standard. Therefore, SUS scores exceeding 68 are considered above average, while SUS scores below 68 are below average. Our SUS score of 79.18 exceeds by far the reference point of 68, yet future improvements can be made to provide even higher levels of reliability, perhaps by installing a triple store within the R2D2 so that the whole system does not have to rely on external SPARQL endpoints. \begin{table}
\begin{tabular}{c c c c} \hline \hline \multirow{2}{*}{**Functionality**} & \multicolumn{1}{c}{Suitability} & 4.18 & \multirow{2}{*}{4.15} \\  & Accurateness & 4.09 & & \\  & Compliance & 4.18 & & \\ \hline \multirow{2}{*}{**Efficiency**} & Time Behavior & 4.2 & \multirow{2}{*}{4.25} \\  & Resource & 4.3 & & \\  & Utilization & & & \\ \hline \multirow{2}{*}{**Compatibility**} & Co-Existence & 4.09 & \multirow{2}{*}{4.27} \\  & Interoperability & 4.45 & & \\ \hline \multirow{4}{*}{**Usability**} & Understandability & 4.36 & \multirow{2}{*}{4.33} \\  & Learnability & 4.54 & \\  & Operability & 4 & \\  & Attractiveness & 4.45 & \\ \hline \multirow{2}{*}{**Reliability**} & Fault tolerance & 3.4 & \multirow{2}{*}{3.47} \\  & Recoverability & 3.54 & \\ \hline \multirow{4}{*}{**Maintainability**} & Analyzability & 3.7 & \multirow{2}{*}{3.94} \\  & Changeability & 4 & \\  & Stability & 3.9 & \\  & Testability & 4.18 & \\ \hline \multirow{4}{*}{**Portability**} & Adaptability & 4.18 & \multirow{4}{*}{4.04} \\  & Installability & 3.90 & \\  & Conformance & 3.94 & \\  & Replaceability & 4.14 & \\ \hline \multirow{2}{*}{**Quality of Use**} & Effectiveness & 4 & \multirow{2}{*}{3.96} \\  & Efficiency & 4 & \\  & Satisfaction & 3.90 & \\ \hline \multicolumn{2}{c}{**SUS**} & 78/100 (Kappa 0.8) & \\ \hline \hline \end{tabular}
\end{table} 
Table 1: The results for the various evaluation categories. Finally, we used the 20 examples that we provided to the students for comparing R2D2 with the DBpedia chatbot [23]. We have to note that three of those examples were proposed in the corresponding DBpedia chatbot site. In addition, as the DBpedia chatbot accepts textual input and R2D2 accepts triple-pattern like input, in the online Excel sheet, we provide both the textual and the triple-pattern like queries.