## 5 Evaluation
### Multi-Case Evaluation Methodology  
We followed the _Multi-case Evaluation Methodology_ which was introduced in [88] and previously applied in the assessment of various DSLs/DSMLs and MDE processes such as [45; 89; 90; 91; 92; 93; 94; 95; 96] for different research domains. Before we discuss how this methodology is used for the evaluation assessment of DSML4 PT, evaluation dimensions and multi-case study protocol of the adopted methodology are discussed in this section briefly due to space limitations. A complete definition of the methodology can be found in [88]. Multi-case Evaluation Methodology proposes the usability evaluation of DSLs/DSMLs according to two main dimensions, namely _Execution Dimension_ and _Quality Dimension_. Please note that there exists one remaining dimension, called Language Dimension, but it was not considered in our evaluation since it is not related to DSML usability and it covers the number analysis of both modeling elements and model transformation rules, which are out of our evaluation's scope. _Execution Dimension_ aims to measure the output performance and time savings achieved by using the DSML during system development. _Output Performance_ considers measuring the ratio of the generated artifacts (e.g., computer programs, configuration files, etc.) over these artifacts plus delta (manually produced) artifacts, while measurement of _Development Time_ leads to determine how using DSML shortens the overall system development as its name suggests. _Quality Dimension_ covers the user perspective where a DSML is qualitatively assessed based on the feedback of the users who experienced using the DSML while developing various systems. User feedback is gathered by a questionnaire comprising two parts: (1) scoring the DSML features based on predefined quality characteristics and (2) answering a set of open-ended questions. For the scoring part of the questionnaire, use of the quality characteristics defined in the _Framework for Qualitative Assessment of Domain-specific Languages_ (_FQAD_) is recommended in [88], and was also preferred in our study. FQAD [97] defines the following ten quality characteristics, each composed of various sub-characteristics and their descriptions, to qualitatively assess DSLs: Functional Suitability, Usability, Reliability, Maintainability, Productivity, Extensibility, Compatibility, Expressiveness, Reusability, and Integrability. Figure 7: An excerpt from the generated GPS init code. Functional Suitability_ represents a DSL's ability to meet the requirements of the application domain while _Usability_ aims at evaluating whether a DSL can be used to achieve the goals specified for the construction of software systems. _Reliability_ is defined in [97] as the property of a DSL that aids in producing reliable programs. _Maintainability_ of the languages should also be provided, e.g., by modifying the language (if needed) without degrading its functionality. _Productivity_ needs to be considered for the language evaluation to determine whether both the development time and the amount of human resources decrease by using the language. _Extensibility_ considers whether the DSL leads users to add new features, while _Compatibility_ measures the degree to which a DSL is compatible with the domain and development process. As its name suggests, the _Expressiveness_ quality characteristic is defined in [97] as the degree to which a problem-solving strategy can be mapped into a program naturally. Another desirable feature is that the syntax and semantics definitions of DSLs can be used to create other languages or development platforms. That feature is evaluated in FQAD's _Reusability_ quality characteristic. Finally, _Integrability_ of a DSL measures how the language can be integrated with other languages, e.g., used in the development process. In the following sub-section, we discuss how FQAD characteristics were adopted and interpreted during the qualitative assessment of DSML4 PT and the formalization of the questionnaire (given in Appendix A).