# Experimental Comparison of Editor Types for Domain-Specific Languages
## 7 Conclusions and Future Work  
We presented an empirical study comparing three different editors of the same domain-specific language: an advanced plain-text editor implemented in the Xtext framework, a projectional editor developed using JetBrains MPS, and a form-based GUI editor, namely Google Forms. The topic of the DSL was the definition of quizzes. The study consisted of a controlled experiment with human subjects divided into two groups (Xtext vs. Google Forms and MPS vs. Google Forms) and an accompanying survey. The task of the participants was to perform small changes to existing questions, add elements to an existing question, and add a new question. We measured the time spent on the tasks. In the controlled experiment, the measured time for Xtext was marginally lower than for Google Forms (2.4% difference in median times, 11.4% difference in mean times), and Google Forms had a slightly lower measured time than MPS (3.8% medians, 14.5% means). However, the results were statistically insignificant, so we cannot conclude which editor type achieved the best efficiency. We attribute the smallness of the differences mainly to the programming expertise of the participants combined with a simple DSL. In the quantitative survey, questions focused on the ease of specific aspects of editing; the participants rated all editors rather favorably. A small exception was Google Forms, which some subjects considered not so easy for question parameter addition. When asked about which editor they would prefer in the future, the majority of subjects selected Xtext or MPS, with Google Forms being less popular. The higher preference for MPS over Google Forms seems to contradict the efficiency results. Such disparity between objective and subjective evaluation has already been observed in other works [32]. In our case, the survey question was about their future preference, so the respondents might have been trying to predict their efficiency in long-term usage. This assumption is also supported by some of their answers in the qualitative survey. On the other hand, it may have just been their personal preferences. We also obtained interesting qualitative data. Xtext and MPS were praised for their efficient, mouse-free interaction. In Xtext, the possibility to copy, paste, and synchronize plain-text data easily during collaboration, and the \"feeling of control\" were highlighted. MPS created a controversy, with a portion of the subjects being satisfied with its keyboard controls and other participants demanding the support of all traditional keyboard shortcuts known from plain-text editors. Special domain-specific shortcuts were also suggested. Google Forms was considered easy to learn and use for non-programmers. It relieved the users of the necessity to learn and remember syntax elements, but many participants would prefer a text-based editor in a longer time frame. It is important to note that we consider this study only the first in a potential series of experiments. Multiple differentiated replications should be performed in the future. Possible variations include slightly modified language definitions or completely diverse domains, more difficult tasks, and measurement of other variables. Different variations and implementations of individual editor types might be tested, too, e.g., a text editor without advanced syntax highlighting and code completion. Using domain experts without any prior programming experience would be particularly interesting, since we hypothesize that their preferences differ from those of programmers to a great extent. Furthermore, this would mitigate the possible influence of prior Eclipse or MPS experience on the results. Another potential area of investigation is the long-term usage of the editors, because users' efficiency with some types of editors might change with growing experience. Conceptualization, J.P.; methodology, S.C., M.S., M.K. and J.P.; software, S.C. and M.K. ; validation, M.S. and M.K. ; formal analysis, M.S. and M.K. ; investigation, M.K.