## 5 Evaluation
### Multi-Case Evaluation of DSML4 PT
#### 5.2.3 Analysis
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Use Case** & \
\begin{tabular}{c} **Experiment A** \\ **(with MDE and DSML4 PT)** \\ \end{tabular} &
\begin{tabular}{c} **Experiment B (w/o** \\ **MDE and DSML4 PT)** \\ \end{tabular} \\ \hline Validator & 68.1 min & 152.3 min \\ Advertising Management & 68.7 min & 104.2 min \\ Air Pollution & 29.2 min & 73.7 min \\ Passenger Count & 60 min & 131.8 min \\ Passenger Information & 58 min & 97.1 min \\ Temperature Sense & 52.5 min & 52.5 min \\ Vehicle Management & 50.1 min & 85.7 min \\ Vehicle Track & 52.6 min & 89.5 min \\
**Average** & **54.9 min** & **98.35 min** \\ \hline \hline \end{tabular}
\end{table} 
Table 3: Times elapsed during system development for each specific use case. age, considering ten participants' performed system design and implementation processes for eight different system development cases studies for IoT-based public transportation. Qualitative Assessment  
To answer RQ3, a questionnaire having two parts was used (see Appendix A). In the first part, the same participants answered 26 questions to evaluate their experience using DSM4 PT from different perspectives. To prepare this scoring part of the questionnaire, the FQAD framework [97] discussed in Section 5.1 was adopted and customized according to both DSM4 PT specifications and the public transportation domain. These questions were scored by the evaluators on a Likert scale ranging from 1 to 5, where 1 means \"Very Bad\"/\"Absolutely not agree\" and 5 means \"Very Good\"/\"Absolutely agree\". All questions in this section are categorized into ten different quality characteristics of the FQAD framework, which are also briefly introduced in Section 5.1. It is worth noting that these questions are prepared to retrieve the evaluators' feedback on the usability of DSM4 PT considering the sub-characteristics also defined in FQAD [97] (in Appendix A, see these sub-characteristic names given in parenthesis at the end of each question text (if applicable)). In the second part of the questionnaire, six open-ended questions were asked of the evaluators to obtain their feedback on using DSM4 PT. The answers received from all evaluators for this questionnaire are found in [98]. _Evaluation Based on the Scored Questions_: The average scores obtained from the evaluators for the first part of the questionnaire are illustrated in Figure 10. As can be seen, DSM4 PT received scores ranging from 4.3 to 4.9 across all evaluation categories. The overall average score for all responses is 4.44, quite high. Thus, this result can be considered a success indicator of the designed language. While the average scores for all categories were high, the highlights of DSM4 PT for this assessment are integrability, maintainability, and expressiveness. In this context, evaluators largely agree that DSM4 PT offers a comprehensive model for the IoT and public transportation system structures and that it is mostly suitable for software development for public transportation systems. The scores also indicated that evaluators believed DSM4 PT increased programming productivity. Additionally, DSM4 PT was found to be compatible with the public transportation IoT field, and using DSM4 PT instance models can facilitate  
Figure 10: Average scores received for the DSML quality characteristics. constructing public transportation systems from various viewpoints. Finally, evaluators confirmed the strength of DSML4 PT's syntax and semantics in expressing the varying needs of the IoT-based public transportation systems. _Evaluation Based on Open-Ended Questions_: In the second part of the questionnaire, the following open-ended questions were asked to obtain feedback from the participants:  
1. Does DSML4 PT make public transportation software development easier? 2. Do you find DSML4 PT useful for the development of IoT-based public transportation system software? 3. Do you think DSML4 PT is strong enough to model overall public transportation systems? 4. Do you think DSML4 PT IDE is easy to use? 5.