## 5 Evaluation
### Multi-Case Evaluation of DSML4 PT
#### 5.2.3 Analysis
Based on the average results given in Figure 9, the following inferences can be made for all development steps. Figure 9: Comparison of the times elapsed during system development. Analysis_: The analysis step is not tied to any tool or platform. For this reason, Experiment A and Experiment B have similar analysis times. In fact, this step only depends on the structural complexity of the IoT-based public transportation systems. This step is independent of the used language, IDE, or methodology. Therefore, the difference between the times is negligible--times are already measured quite close to each other. Design/ModellingAlmost 25% more times elapsed in Experiment B at this step. Completing system design more quickly using DSML4 PT may originate from the language's domain entities specific to the IoT and public transportation systems, including more detailed elements and attributes that can be used during production. Additionally, DSML4 PT has static semantic checks that allow designers to avoid semantic errors that take more time to fix later. Despite such additional features that are thought to require more time, modeling with DSML4 PT reduced design time by around 25% on average. Although this kind of detailed modeling was not performed by the developers during Experiment B, it can be supposed that the time was shortened due to the high ability to model the IoT-based public transportation with the IDE of DSML4 PT. Implementation/GenerationThis is the step in which the developers achieved the most time saving by using DSML4 PT. In Experiment A, most of the application code for the desired public transportation system was automatically obtained with the built-in M2T transformations provided by DSML4 PT. On the other hand, in Experiment B, the developers wrote all the code necessary for implementing the system. It is worth indicating that although use of DSML4 PT led them to achieve a significant portion of the public transportation applications from instance models, the developers still needed to complete the generated application by manually adding delta code to have the full implementation. Hence, times elapsed for completing the auto-generated code during Experiment A were also measured and compared. Consequently, the implementation time required in Experiment B is about three times more than the generation and implementation time elapsed in Experiment A. Here we see that using DSML4 PT is most beneficial in this step. TestTestTesting time also significantly decreased when the developers followed MDE using DSML4 PT. All the developers tried to find the syntactic and semantic errors of the programs in the testing/debugging step. In Experiment B, the developers searched for and fixed bugs in the entire code. However, in Experiment A, the developers mostly dealt with the problems in the delta code manually added, which was much less than the generated code. The generated code consists of many of the error-free and almost ready-made architectural code since it is auto-generated from the system models already validated by the DSML4 PT constraint checks. Therefore, the time spent for testing in Experiment A was reduced to one-third compared to Experiment B. Total TimeThe total average time elapsed during Experiment A was approximately 55 min when all case studies were considered. On the other hand, the total average time elapsed during Experiment B was approximately 99 min. Thus, application of MDE and using DSML4 PT led to decreasing the total system development time by almost half on aver