<table><caption>Table 4.1: Taxonomy table</caption><thead><tr><th rowspan="2"></th><th colspan="10">MDE Approaches</th><th colspan="6">Lowcode development platforms</th></tr><tr><th>BRAIN-IoT/IoTM</th><th>MDEDIOT</th><th>PapyrusOSDIOT</th><th>SysML4IoT</th><th>UMI4DIOT</th><th>ThingML</th><th>IoTLink</th><th>IoTSuite</th><th>FRASAD</th><th>AutoIoT</th><th>CAPS</th><th>NodeRed</th><th>Atmosphere/IoT</th><th>Simplifier</th><th>Menmix</th><th>Salesforce</th></tr></thead><tbody><tr><td colspan="18"><strong>Requirement modeling support</strong></td></tr><tr><td>Requirement specification</td><td></td><td></td><td></td><td>✓</td><td></td><td></td><td></td><td></td><td>✓</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Requirement verification</td><td></td><td></td><td></td><td>✓</td><td></td><td></td><td></td><td></td><td>✓</td><td></td><td></td><td></td><td></td><td></td><td>✓</td><td></td></tr><tr><td colspan="18"><strong>Domain modeling support</strong></td></tr><tr><td>Graphical user interface</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Structure model</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Behaviour model</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Multi-view modelling</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td></td><td></td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td colspan="18"><strong>Testing and verification support</strong></td></tr><tr><td>Testing environment</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>✓</td><td></td><td></td><td>✓</td><td>✓</td><td></td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Model checking</td><td>✓</td><td></td><td>✓</td><td>✓</td><td></td><td>✓</td><td></td><td></td><td></td><td>✓</td><td>✓</td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Model validation</td><td>✓</td><td></td><td>✓</td><td>✓</td><td></td><td></td><td></td><td></td><td></td><td></td><td>✓</td><td>✓</td><td></td><td></td><td></td><td></td></tr><tr><td colspan="18"><strong>Analysis environment</strong></td></tr><tr><td>Realtime analysis</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>Dependability</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td></tr><tr><td>System QoS</td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td></td><td>✓</td><td></td><td></td><td>✓</td><td></td></tr><tr><td colspan="18"><strong>Reusability</strong></td></tr><tr><td>Allow exporting artifacts</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Allow importing external artifacts</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Allow artifact storage and future reuse</td><td>✓</td><td>✓</td><td>✓</td><td></td><td></td><td>✓</td><td></td><td>✓</td><td></td><td></td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td colspan="18"><strong>Deployment support</strong></td></tr><tr><td>Can be deployed locally</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Can be deployed on cloud</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Can generate code</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Run-time adaptation</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td colspan="18"><strong>Interoperability</strong></td></tr><tr><td>Support API integration</td><td></td><td></td><td></td><td></td><td></td><td>✓</td><td></td><td>✓</td><td></td><td></td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Support connect to extern data sources</td><td>✓</td><td></td><td></td><td></td><td></td><td>✓</td><td></td><td>✓</td><td></td><td></td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td colspan="18"><strong>Extensibility</strong></td></tr><tr><td>Easy to add modeling features</td><td>✓</td><td></td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Easy to modify the existing features</td><td>✓</td><td></td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td colspan="18"><strong>Additional characteristics</strong></td></tr><tr><td>Focus on "The thing" layer</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Focus on Application layer</td><td></td><td></td><td></td><td></td><td></td><td>✓</td><td></td><td>✓</td><td></td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td>Open source to developers</td><td>✓</td><td></td><td>✓</td><td></td><td></td><td>✓</td><td></td><td>✓</td><td></td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><td colspan="18"><strong>Target support</strong></td></tr><tr><td>Underlying infrastructure</td><td>Papyrus OSGI</td><td>Papyrus</td><td>Papyrus</td><td>Papyrus, Moka</td><td>Papyrus</td><td>EMFCore Maven</td><td>EMF, GMF, EEF, Accelelo, Esper</td><td>kText, ANTLR, J2SE</td><td>EMFCore GMF</td><td>EMF, GMF</td><td>EMF,GMF, AMW, ThingML</td><td>NodeJS</td><td>Java</td><td>Node.js, Docker</td><td>Java, JS, AWS</td><td>Java, Salesforce IoT Cloud</td></tr><tr><td>Target platform</td><td>IoTBoard, cloud servers</td><td>IoT boards</td><td>MicoreJ, Prismtec h's Vortex</td><td>MicroEJ, Edison board</td><td>Contiki, Rasp. Pi</td><td>IoT Boards, OS, Cloud</td><td>Arduino</td><td>Android, J2SE platforms</td><td>TinyOS, Contiki</td><td>Cloud</td><td>Same target as ThingML</td><td>platform agnostic</td><td>Mobile, Many IoTBoard</td><td>OS, Android</td><td>Cloud</td><td>Cloud</td></tr><tr><td>Code generation language</td><td>Java OSGI artifacts</td><td>Java, C++, Java</td><td>Java</td><td>Java</td><td>C</td><td>C/C++, JavaScript, Java</td><td>Java</td><td>Java</td><td>C, nesC</td><td>Python, HTML, CSS, JavaScript</td><td>thingml</td><td>None</td><td>Java, C, arduaino</td><td>HTML, CSS, JavaScript</td><td>Java, JavaScript</td><td>Java</td></tr></tbody></table>

open source or not has an important impact on the possibility for the community to contribute to its development.

## 4.1.2 Findings

The elicited features, which have been discussed in the previous section, have been considered to study and analyze 16 platforms selected following the above design process. Table 4.1 presents the results from the selection and the assessment process. As it can be seen from such table, we have added the *Target Support* to represent the characteristics related to the infrastructure in which such platform targets. Under such row, we have conceived the *underlying infrastructure* to exhibit the core technologies a tool relies on, *target platform*, which presents different devices and platforms supported by the generated code, and *code generation language* to refer the programming languages supported by the considered system.

According to Table 4.1 most of the analyzed approaches rely on Eclipse and OSGi. We have realized a huge lack of focus on requirement specification except for SysML4IoT (as it extends SysML which enforces requirement specification) and FRASAD, which enforces the requirement specification at the PIM level using rules that can be tracked throughout. The huge lack of analysis support for almost all the tools selected is alarming. We think that it is highly important to analyze and verify the intended system's behavior before deployment as it gives developer indications of what may happen before deployment and help make any adjustment earlier enough. Moreover, we can see that most of the analyzed tools can be deployed locally especially concerning EMF-based tools but mostly all LCDPs are cloud-based with some of them being able to be run also locally.

## 4.1.3 Limitations

From the above results we have identified the following main weaknesses:
1. Lack of standards: we noticed a lack of a standards to support the model-based development of IoT systems. We noticed that each tool proposes its own way of development by hampering interoperability possibilities among different platforms. This is due to the presence of many industrial players, which make the IoT meta-modeling convoluted. On the other hand, different research attempts proposed IoT reference models, which cover different development phases and perspectives. The IoT reference model presented in [49] has been adopted by different tools [61, 155, 157] as a fundamental meta-model. This shows the potentials and benefits of having the availability of standards in such a complex domain. We believe that it as a good starting point, which needs to be further explored to better cover the interoperability dimension (e.g., to enable the possibility of interacting with third-party data resources in general).

2. Limited support of multi-view modeling: we noticed that most of the approaches focus on single view modeling. In particular, except for CAPS [72], MDE4IoT [36], AtmosphereIoT [90], and Mendix [124], the analyzed approaches use one specific view to model everything, which is not a good practice in general. Using multi-view modeling presents enormous benefits as it enforces separation of concerns: the system component is designed using a single model with dedicated consistent views, which are specialized projections of the system in specific dimensions of interest [106]. Multi-view modeling is regarded as a complicated matter to address for tailor-made LCDP as they mostly focus on connecting dots aiming at having an application up and running.

3. Limited support for cloud based MDE: Moving model management operations to the cloud and supporting modeling activities via cloud infrastructures in general is still an open subject. From our study, we noticed that mostly Low-Code development approaches provide the option to run tools on cloud or on-premise. This is not yet the case of tools based on Eclipse EMF, which still requires local deployments. The research presented in [194] proposed a DSL as a Service (DSMaaS) as a solution to address the reusability of so many created DSL over the cloud. Other attempts like MDEforge [195] aim at realizing cloud based model manipulations [195].

4. Limited support for testing and analysis: According to the performed study, very few tools care about the testing and analysis phases of the IoT system development process. There is still a big challenge regarding how to analyze IoT systems responsiveness before deployment. The complexity of the problem relies on the fact that IoT system involve human interaction, environment constraints and we have also to recognize the heterogeneity of the target platforms that makes it hard to depict the kind of analysis properties to address.

### 4.1.4 Conclusion

In this section, we discussed state of the art on existing approaches supporting the development of IoT systems. In particular, we focused on languages and tools available in the MDE field and the emergent LCDPs covering the IoT domain. The study has been performed in three main steps: first, we conceived a taxonomy consisting of features characterizing the studied IoT development platforms. Then, such features are used to evaluate the functionalities and the services supported by each analyzed platform. As a last step, we identified some weaknesses of the analyzed platforms to pave the way toward an LCDP for developing IoT systems. We have also identified some limitations of already existing approaches and discussed possible ways to improve and address them in the future. In future work, we want to continue the investigation of MDE-based IoT platforms by considering both the quantitative and qualitative aspects of the solutions developed.
## 4.2 Cloud-based modeling in IoT domain

Cloud-based modeling is one of the relevant topics in the MDE community due to the induced possibilities of designing, developing, analyzing, and deploying applications seemingly with reduced efforts. This has also been recently favored by the increasing adoption of LCDPs. Ideally, domain-specific LCDPs have to run on cloud infrastructures. However, in some industrial settings such as IoT, domain-specific modeling environment tends to be local-based [12]. Nowadays, industries and companies are trying to migrate their modeling infrastructures to the cloud. However, especially in industrial contexts, the existing modeling infrastructures are implemented in complex environments in which the migration cost can be far more expensive and very complicated.

The future of modeling will forcefully be cloud-based [62]. Several initiatives, including Visual Studio Code<fnref n="1" />, Eclipse Che<fnref n="2" />, Theia<fnref n="3" />, and others have shown a lot of potential in shifting modeling environments from local-based and monolithic installations to cloud-based platforms in order to eliminate accidental complexity and expand the variety of available functionalities. Adopting cloud-based modeling will not only attract more citizen developers, but it will unravel a lot of modeling opportunities on different devices such as tablets, and mobile devices [144, 196].

In the IoT domain, modeling and development infrastructures need to consider several heterogeneous aspects of the system's data, communication, and implementation layers. The Web of Things (WoT) paradigm has brought the IoT a step closer to people's perception because it allows treating a networked thing as a Web resource [197]. We think that adopting the concept "Thing-as-a-service" [198] could provide tremendous help in addressing the interoperability issue that exists in the IoT domain [147]. In this case, all the system's sub-components will be modeled as black boxes (services), and they only communicate with well-defined mechanisms (e.g., employing REST APIs). For instance, approaches such as [139, 141, 142, 147] had taken a step toward this modeling paradigm. This section looks at what has been done so far in the IoT domain to support IoT systems' development through cloud-based modeling approaches. In particular, we conducted a thorough investigation to see where the IoT community stands concerning the current trend of moving traditional modeling infrastructures to the cloud. Following an examination of 611 articles, we identified 22 different cloud-based IoT system development tools and platforms. We perform an analysis of the various issues that the IoT community is encountering while implementing cloud-based modeling tools. As a result, we take a deeper look at a few options and discuss the research and development opportunities enabled by adopting cloud-based modeling approaches in the IoT domain.

### 4.2.1 Related cloud-based modeling studies

We identified very few studies that focus explicitly on cloud-based MDE approaches ([12, 199–201] to mention a few). In this section, we are interested in examining the possible approaches helping in migrating the classical local-based MDE in IoT technologies to the cloud and its adoption.

Our previous study [8] looked at the current state of LCE adoption in the IoT domain. LCE combines LCDPs, MDE, machine learning, and cloud computing to facilitate the application development life-cycle, namely from the design, development, deployment, and monitoring stages for IoT applications. A comparable set of features has been identified by examining sixteen platforms to represent the functionalities and services that each of the investigated platforms could support. We discovered that just 7 of the 16 could be deployed on the cloud, with the majority of them being LCDPs, whereas classical MDE approaches rely on a local-based design paradigm.

In [202], the authors conducted a comprehensive assessment of model-based visual programming languages in general before narrowing their focus to 13 IoT-specific visual programming languages. The research was carried out based on their characteristics, such as programming environment, licensing, project repository, and platform support. According to a comparison of such features, 72%

<footnote n="1">¹https://code.visualstudio.com</footnote>

<footnote n="2">²https://www.eclipse.org/che/</footnote>

<footnote n="3">³https://theia-ide.org</footnote>
of open-source projects are cloud-based, whereas only 17% of closed-source platforms are cloud-based, which confirms a strong uptrend of cloud-based systems in open-source IoT projects.

In [203], the authors discussed tools and methods for creating Web of Things services, in particular, mashup tools as well as MDE approaches. The techniques regarding expressiveness, suitability for the IoT domain, ease of use, and scalability have been analyzed. Although this study is related to this section, it solely focuses on mashup tools and only includes a few approaches. We can observe from the preceding discussion that only a few techniques attempted to explore cloud-based MDE approaches implicitly. According to this, and to the best of our knowledge, this is the first study analyzing the status of cloud-based modeling in the IoT domain.

### 4.2.2 Study design

This section aims to analyze how the IoT domain is coping with the trend of moving existing modeling and development infrastructures to the cloud. To this end, we followed the process shown in Fig. 4.2 according to the methodology presented in [204]. In particular, the search and selection process was mainly conducted in four main phases. In the first phase, we formally and explicitly represented the problem to get a head start on the search. Second, we defined a search string and selected well-known academic search databases. Third, we performed a search to gather approaches to answer properly defined research questions. Fourth, we narrowed down the potential approaches and mapped them based on their similarity and variability. Finally, we analyzed the collected approaches and elaborated some recommendations on the identified difficulties.

<figure><img src="image_25.png" /><figcaption>Figure 4.2: Search and selection process</figcaption></figure>

**Phase 1: Problem formalization:** This phase mainly focused on formalizing the problem we wanted to solve by looking at the current MDE tendency. One of the sources of inspiration for this study was the work in [62] which addresses the topic of "What is the future of modeling?". Thus, we came up with the formulation of the following research questions:

* **RQ1:** How is the IoT community adopting cloud-based modeling approaches?

* **RQ2:** What challenges do researchers face when developing cloud-based IoT modeling and development infrastructures?

* **RQ3:** What are the main potential opportunities laying ahead for future researchers and developers in the IoT domain?
**Phase 2: Automatic search:** In this phase, we applied a search string to different academic databases, i.e., Scopus (Elsevier)<fnref n="4" />, IEEE Xplore<fnref n="5" /> and ACM library<fnref n="6" /> by limiting the search on the last 10 years. The query string we used for the automatic search was: ("MDE" OR "Model Driven Engineering") AND ("IoT" OR "Internet of Things") AND ("Cloud" OR "Web").
Table 4.2 shows the number of approaches we managed to collect in this phase.

<table><caption>**Table 4.2:** Results table</caption><thead><tr><td>Database</td><td>Results</td></tr></thead><tbody><tr><td>Scopus (Elsevier)</td><td>233</td></tr><tr><td>IEEE Xplore</td><td>263</td></tr><tr><td>ACM library</td><td>115</td></tr><tr><td>Total</td><td>611</td></tr></tbody></table>

**Phase 3: Inclusion & exclusion, 1st pass:** Table 4.2 shows that 611 publications were initially discovered from different sources. At this point, we have reviewed the approach's title, keyword, and abstract and exclude approaches that were not satisfying the following criteria:

* Studies published in a peer-reviewed journal, conference, or workshop.

* Studies written in English.

* Approaches that focus explicitly on the IoT topic.

* Studies that propose a cloud-based modeling approach, either explicitly or implicitly.

At the end of this point only 80 approaches were deemed to be satisfying the above set criteria and were considered for the next phases.

**Phase 4: Inclusion & exclusion, 2nd pass:** In this phase, we read the introduction and the conclusion of the approaches previously collected. We also removed some duplicates. Various documents were rejected during this phase for a variety of reasons, for instance, because the presented approach is not explicitly offering an IoT-based cloud-based development environment. At the end of this phase, we ended up with 33 documents. Furthermore, via the 33 papers, we conducted a more in-depth manual search of potentially related work referenced by them, in which 14 approaches were selected and manually added, bringing the total to 47.

**Phase 5: Reading of the whole approach text:** We've gone over the entire articles in this phase, focusing on the proposed approaches and their evaluation sections. Several documents were discarded because of different reasons. For instance, approaches that presented hybrid solutions (e.g., enabling local modeling with the possibility of storing models on remote repositories) were discarded. In addition, the approaches that claim to build web-based IoT data-wrangling platforms by reusing existing IoT data storage platforms were also discarded. Finally, we selected 22 documents that leverage a cloud-based modeling environment to design, develop, or deploy IoT applications.

Figure 4.3 shows the distribution of the selected approaches with respect to their corresponding sources. As you might notice from Fig. 4.3, a portion of the selected approach (4 out of 22) was found from manual snowballing process. In the next section, the research questions presented in Sec. 4.2.2 are answered singularly by analyzing the research approaches that have been collected as previously described.

<footnote n="4">⁴https://www.elsevier.com/</footnote>

<footnote n="5">⁵https://ieeexplore.ieee.org</footnote>

<footnote n="6">⁶https://dl.acm.org/</footnote>
<figure><img src="image_26.png" /><figcaption>**Figure 4.3: Selected approach distribution**</figcaption></figure>

**4.2.3 Findings**

In general, cloud-based modeling is not a new topic in terms of demand and market viability, but when it comes to the IoT domain, there are few approaches in research. In contrast to LCDPs, which have largely adopted cloud-based methods even in the IoT domain, the move from traditional local-based MDE practices to the cloud is still in its infancy. This section goes over different cloud-based modeling approaches that target the IoT domain. We organized the analyzed approaches into three categories according to their main focus of interest i.e., modeling IoT structural aspects, service-oriented approaches, and deployment orchestrations. The aim is to answer the research question **RQ1**: *How is the IoT community adopting cloud-based modeling approaches?*

As previously presented, several approaches are available to support cloud-based modeling in the IoT domain. Table 4.3 shows an overview of the analyzed approaches; half of them are concerned with structural issues, whereas only a few deal with deployment concerns. The current state of the art suggests that there is no predominant common language, although the graphical syntax is preferred. In terms of technical needs, textual cloud-based modeling environments might be simpler to adopt as opposed to graphical ones.

To assess the tool's source-code accessibility, the open-source status factor was chosen. This is a key factor that contributes to the tool's scalability because more individuals can access the source code and potentially extend it. This goes hand in hand with the criterion for determining whether the tool is still accessible. The relationship between the two parameters is depicted in Figure 4.4. We can observe that practically all of the tools that are not open-source are also not currently accessible. When looking at industrial settings, this is especially true when it comes to internal proprietary tools. The same can be said for open-source tools, with the majority of them being freely available. Most of the analyzed approaches are supported by tools, which are not open source. This goes hand in hand with the public availability of the methodologies. We can observe that all the tools that are not open-source are also not publicly accessible. When looking at industrial settings, this is especially true when it comes to internal proprietary tools. The same can be said for open-source tools, with the majority of them being freely available.

While analyzing each approach, we also looked at the supporting infrastructures and their ability to generate deployable artifacts. In this regard, we have discovered that JavaScript-based environments like Node.js and Angural.js are widely used for tool development. This might be due to the fact they are among the modern languages for front-end technology implementation. On the other hand, it appears that the majority of techniques generate artifacts, even though few of them are standalone deployable components. It is also worth noting that the generated deployable artifacts can only be deployed within the same original environment in most of cases. To ensure interoperability, scalability, and reusability of the tools, the generated artifacts should generally be deployed anywhere.

In traditional local-based modeling environments, the aforementioned evaluation factors also apply. We chose not to include more complex evaluation criteria such as tool extensibility, scalability, analysis, model verification and validation, and so on. This is due to the fact that the cloud-based
<table><caption>**Table 4.3: Analyzed approaches**</caption><thead><tr><th>Tool name</th><th>Category</th><th>Language syntax</th><th>Open-source</th><th>Tool availability</th><th>Underlying infrastructure</th><th>Generated artifact</th></tr></thead><tbody><tr><td>DSL-4-IoT</td><td>Structure</td><td>Graphical</td><td>no</td><td>no</td><td>js, OpenHAB</td><td>JSON config</td></tr><tr><td>BloTA</td><td>Structure</td><td>Graphical</td><td>no</td><td>no</td><td>Apache Tech. GraphQL</td><td>YAML file</td></tr><tr><td>IADev</td><td>Service</td><td>Textual</td><td>no</td><td>no</td><td>ASR,REST,ATL</td><td>REST app</td></tr><tr><td>Node-RED</td><td>Structure</td><td>Graphical</td><td>yes</td><td>yes</td><td>Node.js</td><td>Node-RED app</td></tr><tr><td>AutoIoT</td><td>Structure</td><td>Graphical &amp; textual</td><td>no</td><td>no</td><td>Python, js</td><td>Flask app</td></tr><tr><td>[135]</td><td>Structure</td><td>Textual</td><td>no</td><td>no</td><td>Smart-M3</td><td>-</td></tr><tr><td>AtmosphereIoT</td><td>Structure</td><td>Graphical</td><td>no</td><td>yes</td><td>Multi-platform</td><td>Multi-platform apps</td></tr><tr><td>[136]</td><td>Structure</td><td>Textual</td><td>yes</td><td>yes</td><td>js,VueJS</td><td>UI code</td></tr><tr><td>[205]</td><td>Structure</td><td>Graphical</td><td>no</td><td>no</td><td>WebRatio, IFML</td><td>UI code</td></tr><tr><td>FloWare Core</td><td>Structure</td><td>Graphical</td><td>yes</td><td>yes</td><td>JavaScript</td><td>Node-RED Config file</td></tr><tr><td>Vitruvius</td><td>Structure</td><td>Textual</td><td>yes</td><td>no</td><td>XML,HTML,js</td><td>HTML5 with JavaScript app</td></tr><tr><td>MIDGAR</td><td>Service</td><td>Graphical</td><td>no</td><td>no</td><td>Ruby, js, HTML Java</td><td>Java app</td></tr><tr><td>LogicIoT</td><td>Service</td><td>Textual</td><td>no</td><td>no</td><td>JSP</td><td>-</td></tr><tr><td>glue.things</td><td>Service</td><td>Graphical</td><td>yes</td><td>no</td><td>AngularJS,Meshblu PubNub</td><td>NodeRED service</td></tr><tr><td>[143]</td><td>Service</td><td>Textual</td><td>no</td><td>no</td><td>Fkebase&Node.js</td><td>-</td></tr><tr><td>TOSCA</td><td>Deployment</td><td>Textual</td><td>yes</td><td>yes</td><td>Multi-platform</td><td>Config files</td></tr><tr><td>[144]</td><td>Service</td><td>Graphical</td><td>no</td><td>no</td><td>-</td><td>-</td></tr><tr><td>E-SODA</td><td>Service</td><td>Textual</td><td>yes</td><td>no</td><td>OSGI cloud</td><td>OSGI java bundles</td></tr><tr><td>[146]</td><td>Service</td><td>Textual</td><td>yes</td><td>yes</td><td>ClickScript,AJAX</td><td>REST services</td></tr><tr><td>InteroEvery</td><td>Service</td><td>Graphical</td><td>no</td><td>no</td><td>Spring Boot,Rest RabbitMQ,Angular</td><td>-</td></tr><tr><td>DoS-IL</td><td>Deployment</td><td>Textual</td><td>no</td><td>no</td><td>js,HTML,DOM</td><td>Config files</td></tr><tr><td>GENESIS</td><td>Deployment</td><td>Textual</td><td>no</td><td>no</td><td>multi-platform</td><td>Genesis dep. agents</td></tr></tbody></table>

<figure><img src="image_27.png" /><figcaption>**Figure 4.4: Accessibility vs Open**</figcaption></figure>

modeling topic is still in its early phases. Nonetheless, we’ve discovered that with the exception of NodeRED [89], which partially implements some of the above functionalities, none of the other tools clearly express their support for it. This illustrates the magnitude of the task that needs to be addressed. In this regard, we recognize that developing the modeling environment (whether graphical or textual) is critical and that once this is done, other services can be migrated to the cloud and used via a consumable API.
4.2.4 Open challenges

Multiple issues have arisen as a result of the expansion of connected smart and sensor devices, as well as the increased usage of cloud-based models [66]. As a typical IoT system consists of mul-
tiple complex sub-systems, having an all-in cloud-based environment can become even more com-
plicated. On the other hand, overcoming these barriers is worth the effort because it opens up more
opportunities. This section elaborates on the current challenges IoT systems face while developing
and integrating such tools in a cloud-based environment. Essentially, we are answering the research
question RQ2: What challenges do researchers face when developing cloud-based IoT modeling and
development infrastructures?

**Extensibility mechanisms:** Extensible platforms allow the addition of new capabilities without having to restructure the entire ecosystem. Because IoT systems are distributed, a typically recommended architecture would be to use the micro-service architecture throughout the development process [85]. Aside from that, IoT systems may require additional interactions with third-party technologies. As a result of the previous scenario, developing tools to design and develop such distributed applications on the cloud need efficient tools that traditional domain specialists may not have. Accessibility mechanisms are presented through tools like [89, 137], but there is still a lot to be done. Currently, domain experts must provide cloud-based automation mechanisms and tools to allow citizen developers to add new features without requiring sophisticated knowledge or changing existing architectures.

**Heterogeneity:** It is an important challenge of the IoT domain, which involves different players de-
veloping various applications running at different layers, namely the edge, fog, and cloud [85]. In
addition, deployments and data consumption methods are very diverse, increasing the complexity of
traditional code-centric approaches [206]. Cloud-based modeling in IoT brings even more sophistica-
tion regarding the environment in which the system should be designed and developed. The typical
cloud-based modeling platform should foster the integration of heterogeneous technological imple-
mentations, promoting reusability and developing solutions close to the problem domain. Approaches
such as [139, 140, 167] have presented different strategies to tackle such issues, but much more have
to be investigated.

**Scalability:** IoT systems are expected to handle a wide range of users, perform demanding computa-
tions, and share enormous amounts of data among nodes. Therefore, supporting cloud-based modeling
approaches must be implemented in such a way that scalability concerns are mitigated. One of the
approaches to tackle such challenges is to adopt container-based orchestration tools such as Kuber-
netes. The use of such tools can offer out-of-box features such as self-healing, fault-tolerance, and
elasticity of containerized resources [86]. This will also help automate cognitive processes that can
detect scalability needs and adjust autonomously without human intervention.

**Interoperability:** The interoperability of various tools, services, and resources is critical in the IoT
domain. The interoperability of cloud-based modeling platforms, particularly in the IoT area, is cur-
rently limited since different tools run in different environments and have different natures. A tool
like [147] promotes the micro-service architecture by allowing all parts of the system to communi-
cate with each other. Several regulations, such as standardization, will need to be implemented to
achieve interoperability among different cloud-based modeling environments. To address interoper-
ability concerns, technologies like [89, 133, 137, 160] promote a common format based on JSON to
encode models. It is worth noting that adopting Model-as-a-Service (MaaS) architectures could also
promote the interoperability of services and artifacts.

**Learning curve:** It is not easy to find professionals who can master and combine the different so-
phisticated technologies involved in developing and managing IoT systems. IoT domain experts may
lack modern programming expertise, whereas experienced software programmers may lack modeling
domain expertise. For instance, conceiving a cloud-based code generator requires understanding dif-
ferent model transformation techniques and particular programming abilities; Implementing a visual
mashup tool will necessitate knowledge of modern languages such as JavaScript, HTML, and CSS.

**Security concerns:** Current IoT systems suffer from security concerns as data are collected from a
wide distribution of private and public nodes. Furthermore, the data is transferred using remote IoT gateways, which might get exposed in the process. This heterogeneity of secured and unsecured data might favor attackers to target devices and compromise the integrity of data and operations [207]. Therefore, proper abstractions and automation techniques are needed to help target users that might not necessarily have the required knowledge of the security practices to be employed.

### 4.2.5 Opportunities

In this section, we examine several opportunities that we think researchers and developers can leverage to improve the cloud-based development and management of IoT systems. Therefore, we aim at answering the research question **RQ3: What are the main potential opportunities laying ahead for future researchers and developers in the IoT domain?**.

#### Tools and platforms

Numerous tools and platforms are being built to tackle cloud-based modeling concerns. Thus, now is the right moment to suggest powerful and extensible tools that the IoT community may harness to solve their domain-specific issues. In this section, we look at various open-source and highly extensible platforms that are popular among the modeling community and that we would recommend for the IoT domain.

**Cloud-based development tools based on Eclipse:** We believe that a significant part of the MDE community, or at least for research purposes, uses Eclipse-based technologies. This is because most Eclipse projects and technologies are open-source, making them more accessible and encouraging individuals to participate. As of March 2021, the Eclipse Foundation hosts over 400 open source projects, 1,675 committers, and over 260 million lines of code have been contributed to Eclipse project repositories [208]. Through the Eclipse Cloud Development (ECD)<fnref n="7" /> effort, the Eclipse community has demonstrated its willingness to transit a part of its ecosystem to the web. Eclipse's ECD Tools working group strives to define and construct a community of best-in-class, vendor-neutral open-source cloud-based development tools and promote and accelerate their adoption. Some of the best cloud-based technologies that the IoT community can benefit from are the following:

- **EMF.cloud, GLSP, Theia** - Independently from the Eclipse modeling framework (EMF), the EMF.cloud community recently expressed a strong desire to migrate the Eclipse-based modeling infrastructure to the cloud. This project aims to develop a web-based environment for creating modeling tools that can support the editing mechanisms of EMF-based models. EMF.cloud allows users to interact with models through the EMF.cloud model server, which coordinates the use of GLSP for graphical modeling, and LSP for textual modeling. Code generation infrastructures based on Eclipse Xtend are also included, while Eclipse Theia provides a web-based code editing and debugging infrastructure. Several resources are available in the community for extending those tools, and we believe that IoT developers may use such technologies to construct cloud-based IoT DSLs.

- **Sirius Web<fnref n="8" />** - It is an Eclipse Sirius-based modeling tool that provides a powerful and extensible graphical modeling platform for users to design and deliver modeling tools on the web. In Sirius Web, the ability to create your modeling workbench in a configuration file is supported. In this case, no code generation is required because everything is interpreted at run-time [209]. Furthermore, being open-source, Sirius Web provides greater accessibility and customizability than the desktop version, making it easier for the IoT community to get started with their cloud-based solutions.

Another alternative, such as Eclipse Che<fnref n="9" /> makes Kubernetes development accessible for developer

<footnote n="7">⁷https://ecdtools.eclipse.org/</footnote>

<footnote n="8">⁸https://www.eclipse.org/sirius/sirius-web.html</footnote>

<footnote n="9">⁹https://www.eclipse.org/che/</footnote>
teams. Che is an in-browser IDE that allows you to develop, build, test, and deploy applications from any machine. Finally, Epsilon playground<fnref n="10" /> has been recently launched to offer cloud-based tools for run-time modeling, meta-modeling, and automated model management.

*Low-Code Development Platforms:* Looking at the LCDPs, the only powerful cloud-based open-source platform we would recommend is Node-RED [89]. Due to its high extensibility and accessibility, Node-RED offers an excellent IoT system mashup environment in which IoT systems can be designed, developed, and deployed on the fly. The Node-RED platform is open, and IoT system developers can build their custom nodes, compile, test, and deploy them in the Node-RED ecosystem. Several extensions have been made, such as [210] tackling the reusability issues in cloud-based modeled components, [211] to tackle the heterogeneity and complexity challenges found in the Fog based development. Finally, in [212], the authors presented SHEN to enable self-healing capabilities of applications based on Node-RED. In terms of interoperability, Node-RED models are represented as JSON objects, which any third-party tools can easily consume. Some of the tools in this domain, such as FloWare [137] and GENESIS [168] already support the Node-RED models, which shows a great sign of its high impact. Table 4.4 outlines the essential characteristics of the recommended platforms.

<table><caption>**Table 4.4:** Recommended technologies</caption><thead><tr><th></th><th>EMF.cloud</th><th>GLSP</th><th>Theia</th><th>Che</th><th>Node-RED</th></tr></thead><tbody><tr><th>Open-source</th><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><th>Extensible</th><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><th>Scalable</th><td>✓</td><td>✓</td><td>✓</td><td>✓</td><td>✓</td></tr><tr><th>IoT-specific Application</th><td>—</td><td>—</td><td>—</td><td>—</td><td>✓</td></tr><tr><th></th><td>Web-based EMF modeling tools</td><td>Graphical language-server-editor</td><td>Web-based code editor</td><td>Kubernetes-native IDE for DSL deployment</td><td>Flow-based programming</td></tr></tbody></table>

## Benefits of cloud-based modeling

Although there are difficulties in adopting a cloud-based modeling approach in the IoT domain, various opportunities will emerge, making the investment worth it. This section outlines various opportunities that will emerge once cloud-based modeling is widely adopted in the IoT domain.

1. **User communities:** Adopting cloud-based modeling in the IoT domain will have the potential of attracting more citizen developers, and it will unravel a lot of modeling opportunities on different devices such as tablets and mobile devices [144, 196, 213].

2. **Collaborative modeling:** Once IoT modeling infrastructures are moved to the cloud, it can be necessary to introduce collaborative modeling features to simplify the interaction of both developers and stakeholders. Unfortunately, none of the discussed approaches provide collaborative modeling functionalities. However, collaborative modeling can harness the power of real-time information, artifacts, and service exchange.

3. **Productivity:** Empowering users to develop their applications by embracing cloud-based modeling is a head-start toward high production and reducing time to market [12]. Users can create applications that cater to their problems, and engineers focus on developing features that facilitate the user for a smooth development at an appropriate abstraction level. In addition, the participants will focus on problem-solving in their particular domain and avoid wasting time and resources on solving problems that are outside their competencies.

<footnote n="10">¹⁰https://www.eclipse.org/epsilon/live/</footnote>
4. **Maintenance:** Traditional code-centric methodologies necessitate a significant investment in the ongoing maintenance of developed systems. In addition, systems require regular upgrades and installations, which can be error-prone and time-consuming. During upgrades or troubleshoots times of the system, sometimes system downtime is necessary, impacting production. Furthermore, the growing need for software systems in our daily lives and constantly changing user requirements required an agile approach for addressing these issues quickly without compromising system availability or user access. In many cases, such challenges are handled by cloud providers, leaving developers and engineers to focus on developing applications that directly impact customer demands [214].

5. **Monitoring and debugging:** Cloud-based modeling enables monitoring of activities and their archive through its cloud providers. This is a head-start when debugging distributed applications because developers can track down the microservices, which are the root of the detected problems. Without appropriate cloud infrastructures, it would be challenging to solve these issues, even with features such as self-healing and repair strategies. Current cloud-based solutions come bundled with monitoring tools that assist in problem diagnosis and monitor the usage of the applications.

### 4.2.6 Conclusion

To develop IoT applications, developers must overcome various challenges, including heterogeneity, complexity, and scalability. Moving development infrastructure to the cloud will open up plenty of new opportunities regarding accessibility, productivity, maintenance, and monitoring. In this section, we conducted a systematic study to assess the current state of the art on cloud-based modeling approaches in the IoT domain. We looked at 22 approaches proposing cloud-based modeling environments in the IoT domain. The considered approaches have been analyzed to assess their strengths and weaknesses concerning many characteristics, including their modeling focus, accessibility, openness, and artifact generation. Throughout the section, we have discussed many challenges that IoT developers encounter while adopting such tools. We also discussed various generic technologies and tools, which can be adopted in the IoT domain.
# Chapter 5

## Assessing the quality of IoT Engineering Platforms

Over the last few years, industry and academia have proposed several Low-Code and MDE platforms to ease the engineering process of IoT systems. However, deciding whether such engineering platforms meet the minimum required software quality standards is not straightforward. Software quality can be defined as the degree to which a software system achieves its intended goal. Various software quality standards have been established to aid in the software quality assessment process; however, due to the nature of engineering IoT platforms, such models may not entirely suit the IoT domain. This chapter presents a model for assessing the software quality of Low-Code and MDE platforms for engineering IoT platforms. Tackling the second research problem (RP2), the proposed software quality model was based on and extends the ISO/IEC 25010:2011 software product quality model standard. It is intended to assist IoT practitioners in assessing and establishing quality requirements for engineering IoT platforms. To determine the effectiveness of the proposed model, we used it to evaluate the quality of 17 IoT engineering platforms, and the results obtained are promising.

**This chapter is structured as follows:** The background of the study is presented in Sect. 5.2. The proposed product quality model is presented in Section 5.3. In Section 5.4 we go through the selected primary studies and present the followed evaluation process. In Section 5.5, we present the results from the conducted assessment by reflecting on specific research questions. Section 5.6 discusses the results as well as its limitations. Section 5.7 concludes the chapter and discusses perspective future work.

### 5.1 Introduction

IoT systems offer enormous benefits in our daily lives by enabling seamless communication with our surroundings. Such systems demand many development skills, from handling tiny microcontrollers to more extensive and complex cloud-based systems. In the IoT domain, systems often include safety-critical tasks that, if mishandled, might have disastrous consequences and even cost human lives [34]. To guarantee robustness and safety during operation, it is critical to investigate the correctness and the quality of the platforms and the process by which systems are developed.

Over the last few years, both academia and industry have proposed novel languages and tools to support the engineering of IoT systems. To this end, MDE and Low-Code paradigms are employed to conceive engineering platforms specific to the IoT domain. In the software engineering process, MDE promotes the use of models as first-class citizens in software development [42]. Its goal is to improve productivity and reduce time to market by allowing the development of systems using models defined with concepts that are much less linked to the underlying implementation technology and much closer to the problem domain [35]. LCDPs are generally cloud-based software development platforms that leverage a Platform-as-a-Service paradigm to enable users with little or no programming skills to create fully functional apps with dynamic graphical user interfaces [11, 87, 215]. For the sake of
readability, we use IoT Engineering Platforms when we need to refer Low-Code and MDE platforms indistinguishably.

Deciding whether an IoT engineering platform meets the required standards for adoption in terms of quality is not a straightforward process as it involves considering and exploring various sources of information. MDE and low-code development technologies frequently rely on rigorous verification and validation processes conducted under predefined arbitrary constraints to guarantee the quality of generated systems. However, in most cases, the quality of such employed engineering tools is not taken into account seriously or simply overlooked [184, 185].

Practitioners typically rely on well-established standards and practices to improve confidence in whether a system or a product fits the wanted quality requirements. The ISO/IEC 25000 to ISO/IEC 25099 series of International Standards, titled "Systems and software engineering - Systems and software Quality Requirements and Evaluation (SQUARE)", aims to address issues concerning software quality requirements specification and evaluation [189]. The ISO/IEC 25010:2011 standard, in particular, [14], is intended to help people who are interested in developing or purchasing systems and software products to specify and evaluate their quality requirements.

This chapter presents a quality model for assessing the quality of IoT engineering platforms based on the ISO/IEC 25010:2011 standard. In doing so, we enhanced the standard's product quality model by defining a product quality assessment model that is more suitable for the IoT domain. Initially, the ISO/IEC-25010:2011 standard, which replaced ISO/IEC-9126-1:2001 standard, defines two main sets of quality models, namely **"quality in use model"** and **"product quality model"**.

The former is related to the outcome of interaction when a product is used in a specific context. In contrast, the latter is related to software platforms' static and dynamic properties. These models are defined in terms of characteristics, with some of them further subdivided into sub-characteristics [14]. Both models are equally crucial for thoroughly assessing the quality of a given platform. However, the proposed model focuses on the *product quality model* since it emphasizes on platform's technical quality rather than using quality which can be challenging to measure for platforms in their early development stages.

Even though all of the approaches presented in 3.5 relied on the ISO/IEC-9126-1:2001 standard like ours, none specifically target IoT engineering platforms. While there are many quality assessments of DSLs available, only a small number of them refer to an established standard in the evaluation [184], and we could not find any that addresses the IoT modeling domain in particular. As a result, we are confident that our study is the premier to use a well-established quality standard to evaluate the product quality of IoT engineering platforms.

To evaluate the effectiveness of the proposed model, we employ it to assess the quality of 17 IoT platforms selected from our previous studies [8, 13]. We present the methodology we used to choose such platforms, perform the quality assessment, and subsequently present and discuss the obtained results. We summarize this chapter's contribution as follows:

* We propose a quality model based on the ISO/IEC 25010:2011 standard for evaluating the quality of IoT engineering platforms;

* We assess the quality of 17 IoT engineering platforms by relying on the proposed model;

* We present and discuss the findings as well as the limitation of the study in order to validate the effectiveness of the proposed model.

## 5.2 Overview on Software Quality Models

The discipline of Software Quality Engineering [216] is concerned with improving the approach to software quality. However, the various perspectives present throughout the software life cycle show what constitutes software quality is frequently contested. In this context, relying on software quality models to support the quality management of software systems is widely accepted [217]. A
quality model is typically defined as a set of sub-characteristics and their interrelationships that serve
as the foundation for specifying quality standards and evaluating quality [218]. Various standards have
been defined in the literature to assist in the evaluation process. The ISO/IEC 9126:1991 standard
aimed at defining a software quality paradigm and a set of guidelines for assessing the characteristics
associated with it [219]. This standard was later revised by ISO/IEC 25010:2011, which included
a model for software system quality with well-formed specifications for quality characteristics of
software products [184].

The ISO/IEC 25010:2011 standards help design, develop and acquire systems and software prod-
ucts with the specification for evaluating their quality requirements [189]. The standards are made
up of two quality models, each with its own set of characteristics, some of which are further subdi-
vided into sub-characteristics. First, the “quality in use model” focuses on the outcome of interaction
when a product is used under particular contexts [14]. This model is primarily intended to provide
targets for promoting the development and verification efficiency, as well as to anticipate quality in
use before delivery [189]. This model includes five characteristics, namely effectiveness, efficiency,
satisfaction, risk freedom, and context coverage where some of them are further subdivided into nine
sub-characteristics. The “product quality model” refers to the static and dynamic qualities of a soft-
ware platform. This model primarily focuses on providing assessment ground for people supplying
software products and those acquirers who wish to get more involved in the process technically [189].
The model is divided into eight characteristics, namely functional appropriateness, performance effi-
ciency, compatibility, reliability, usability, security, maintainability, and portability which are further
elaborated into 31 sub-characteristics.

In the past, the ISO/IEC 25010:2011 standard has been adopted to assess not only the product quality of IoT systems [18–20] but also in domains such as Big data [21], Machine Learning [22], Software Product Lines (SPL) [23], Customer Relationship Management (CRM) systems [24] and mobile apps [25], just to mention a few. In the MDE world, approaches such as [187] relied on it for assessing the performance of MDE quality studies, while [184, 185] adopted it in order to assess the quality of Domain Specific Languages (DSLs) while and design architectures quality [186] adopted it to assess the quality of design architectures. Although ISO/IEC 25010:2011 cannot be considered a one-size-fits-all solution, it can be a beneficial approach for evaluating the quality of IoT engineering platforms. A detailed description of the mentioned quality characteristics is given in the next section.

**5.3 The product quality model**

In this section, we discuss the *product quality model* of the ISO/IEC 25010:2011 standard to
enable the quality evaluation of IoT engineering platforms. To this end, by referring to Figure 5.1, in
the following, we discuss the common model characteristics concerning the peculiarities of the IoT
domain.

**5.3.1 Functional suitability**

This factor evaluates how well a platform meets specified and implied objectives when used in
specific contexts. In the IoT engineering context, its corresponding sub-characteristics can be defined
as follows.

*Functional completeness* is the extent to which a given platform supports the design of all the layers that comprise an IoT ecosystem. For instance, the platform’s capacity to represent the complete IoT ecosystem from the edge, fog, to the cloud layers while seemingly handling all of the communication heterogeneity that might occur. Concerning *Functional Correctness*, this can be defined as the extent to which the platform applies specific correctness methodologies during the development phases, including but not limited to correct by construction, model-checking, model validation, and rule-based modeling. Finally, the *Functional appropriateness* refers to the amount to which the given functionalities facilitate the completion of defined activities and objectives. IoT engineering platforms
<figure><img src="image_28.png" /><figcaption>**Figure 5.1:** Software Product Quality Model</figcaption></figure>

have diverse functionalities; in this case, we can propose to analyze how the specified correctness methodology permits achieving the goal.

### 5.3.2 Performance efficiency

This characteristic determines how the platforms consume resources under specific conditions. This characteristic is mainly related to run-time attributes of the underlying system infrastructure, which in some situations cannot be predicted accurately. In the context of IoT engineering platforms, the corresponding sub-characteristics can be defined as follows: *Time behavior* refers to how well a platform function meets its criteria in terms of response and processing times, as well as throughput rates. The platform's responsiveness can primarily indicate this, and it is worth noting that it can only be evaluated on platforms that are accessible.

*Resource utilization* measures how much and what kind of resources are used to suit the platform's needs. Although this can somewhat be predicted when a platform runs on well-known underlying infrastructures, the actual utilization can only be evaluated accurately during the platform's usage. Finally, the *Capacity* is defined to assess the platform's ability to model and coordinate a large and complex system that spans multiple IoT sub-domains.

### 5.3.3 Compatibility

This characteristic implies the platform's capacity to interchange data with other platforms. The two sub-characteristics are defined as follows: *Co-existence* refers to the degree to which a platform continues to operate efficiently while sharing a shared environment and resources with other platforms without hammering the other platforms. *Interoperability* is referred to the degree to which a platform
can exchange information with other platforms while in use. In our context, this can be assessed in
terms of the platform's capability to support the data exchange during the development process by
consuming or sending information to/from external services via dedicated means such as REST APIs.

5.3.4 Reliability

This characteristic reflects how a platform can complete a set of tasks in a given amount of time. Its sub-characteristics can be defined as follows: *Maturity* reflects how well the platform supports all of the basic functionalities of a typical engineering platform, such as design, code generation, and deployment. *Availability* is concerned with the extent to which a platform is operational when needed. *Fault tolerance* is the degree to which a platform performs as expected despite the presence of hardware or software faults. In our case, this can be evaluated as the platform's capability to support advanced mechanisms, including but not limited to self-adaption and self-healing. Finally, the *Recoverability* reflects the extent to which a platform, in case of interruptions or failures, can recover data directly affected and re-establish the desired state of the system. To that end, the platform should be able to handle the mechanisms including but not limited to self-recovery or self-redeployment, etc.

5.3.5 Usability

This characteristic reflects how well specific users can use a platform to achieve particular goals with efficiency, effectiveness, and satisfaction. The sub-characteristics are as follows: *Appropriateness* *recognizability* is defined as the degree to which users can determine whether a platform is appropriate for their needs. This can be promoted in the IoT engineering world by the degree to which the platform is customized concerning the underlying environment. *Learnability* can be defined as the extent to which the platform aids developers in learning how to use it, e.g. with context-based modeling support, on-the-fly suggestions, and so on.

*Operability* represents the extent to which a platform has attributes that make the entire development process more accessible, for instance, using functionalities like auto-completion for textual languages, guide-through mechanisms, multi-view modeling, palette show/hide, and palette element search. *User error protection* reflects the extent to which the platform provides some protection to avoid errors, such as static analysis or on-the-fly error handling. *User interface aesthetics* is the extent to which the platform provides pleasant and satisfying user interfaces. Finally, *Accessibility* can be considered as to whether the platform is easily reachable in case of needs, either being locally or online.

5.3.6 Security

This characteristic refers to how successfully a platform protects information and data so that
users, services, and external systems have appropriate access to data according to specific authoriza-
tion levels. Its six sub-characteristics have been renamed as follows: *Confidentiality* refers to the
extent to which a platform assures that data is exclusively available to those who have been granted
access. *Integrity* can be defined as the extent to which an IoT platform prohibits unwanted access,
modification of the platform, or data.

*Non-repudiation* can be measured by the extent to which the platform enables methods for record-
ing all actions conducted during development and proves that they have been performed so that they
cannot be contradicted later. *Accountability* can be measured as the extent to which a platform enables
tracing attributes like versioning, historical action retrieval, and so on. Finally, *Authenticity* can be
described as the ability of the platform to support different types of authentication mechanisms while
accessing platform resources.
### 5.3.7 Maintainability

This characteristic refers to the degree to which a platform to be improved, repaired, or adapted to changes. Its corresponding sub-characteristics are defined below: *Modularity* can be measured as the extent to which a platform is decoupled into discrete sub-systems that can be modified independently of other parts. *Reusability* reflects the extent to which a platform or a component of a platform can be used in more than one system. While evaluating this, we can focus on the platform's ability to be decomposed into small reusable sub-systems.

*Analyzability* assesses the efficacy and efficiency with which it is feasible to determine the impact of modifying parts of the platform by either removing it or injecting failures into it. In the IoT engineering context, whether the platform provides means supporting the analysis of the system under development and/or the platform itself is also looked at. In terms of *Modifiability*, the assessment can be performed on the extent to which a platform can be successfully and efficiently modified without introducing flaws or deteriorating the quality of existing products. Finally, *Testability* can be assessed in terms of the capability of the platform to provide testing supports for its components or the system under development.

### 5.3.8 Portability

This characteristic refers to how fast and successfully a platform can be moved from or to different hardware and software operational environments. Its corresponding sub-characteristics are defined below *Adaptability* refers to the extent to which a platform can be effectively and efficiently adapted to different environments. *Installability* reflects the degree of effectiveness and efficiency with which a platform can be successfully installed and/or uninstalled in a given environment. *Replaceability* measures the extent to which a platform can be updated, replaced, and redeployed in the same environment and still performs as expected.

## 5.4 Quality assessment of IoT engineering platforms

This section shows the quality assessment process followed to analyze 17 IoT engineering platforms using the product quality model discussed in the previous section. The platforms of interest have been identified as presented in Section 5.4.1. The research questions that we answered through the performed evaluation are presented in Section 5.4.2. The quality assessment process that has been followed is presented in Section 5.4.3.

### 5.4.1 Selection of the evaluated IoT engineering platforms

In [8], we examined 16 different platforms to gain a better understanding of the current state of the art in supporting the development of IoT systems, with a focus on languages and tools available in the MDE field and emerging LCDPs. In [13], we examined 22 IoT modeling environments, assessing their strengths and weaknesses in terms of cloud-based modeling capacity, accessibility, openness, and artifact generation. Combining the two data sources, we started with a total of 38 approaches, which have been filtered by considering the following exclusion criteria:

* Approaches that were published before 2012 have been discarded;

* Duplicated approaches have been removed;

* Approaches that do not permit the development of fully functional IoT applications have not been considered.

* Approaches that depend on already-included platforms have been filtered out;
<figure><img src="image_29.png" /><figcaption>**Figure 5.2:** Primary studies selection process</figcaption></figure>

* Approaches that are generic and that do not explicitly target the IoT domain have not been considered in this study.

By applying such criteria, 15 out of the initial 38 IoT development platforms were identified. In addition, we have added two more approaches that we believe are very promising and were published after our first study, bringing the total to 17. Figure 5.2 depicts the detailed approach selection procedure that we used. To better understand the selected basic studies, we have categorized them quantitatively based on publishing year, publisher type, article type, and the distribution among MDE and LCDP approaches. The result is shown in Figure 5.3a, 5.3b, 5.3c and 5.3d respectively.

## 5.4.2 Research questions

The performed assessment aimed at answering the following research questions:

* **RQ1:** To what extent do the considered IoT engineering platforms meet the characteristics of the proposed quality model?

* **RQ2:** What are the most and the least addressed quality sub-characteristics by the considered IoT engineering platforms?

## 5.4.3 Assessment process

The quality assessment process of the considered IoT engineering platform has been done iteratively by going over all of the reference approaches. We established a set of questions for each sub-characteristic that must be addressed to confirm the platform's competence with respect to what is included in the model described in Section 5.3. By doing this, we made the evaluation process result easier and more reflective of the proposed model. Following that, we read the entire document and responded to the questions. Each question can be answered with "Yes" or "No". For instance, the following questions have been formulated to help in assessing the platform's Functional Suitability:

* Does the platform support the design and development of IoT systems?

* Does it support all layers (Edge, Fog, Cloud)?

* Does it mention any support for dealing with different communication protocols?

Consequently, given the approach under analysis, if the corresponding presented platform succeeds on at least 50% of the questions, it is marked as supporting the quality characteristic of interest. This procedure has been developed and implemented for all 31 sub-characteristics. The defined questions have been created purely to facilitate the review process and are entirely consistent with what is provided in the model. Unfortunately, due to space constraints, we could not present the extended table of questionnaires used to assess all of the quality characteristics. In this regard, a full set of evaluation questionnaires used in the evaluation has been published and can be accessed from an online database available at [220] as well as at the Appendix A.
<figure><img src="image_30.png" /><figcaption>**Figure 5.3:** Overview of the selected basic studies</figcaption></figure>

As mentioned in the previous section, some of the characteristics heavily depend on the dynamic properties of the underlying system infrastructures as well as during their usage. For instance, verifying sub-characteristics related to performance efficiency and usability might be difficult in general. Thus, during the performed analysis we relayed on information provided in the reference approaches when available. We considered such kinds of characteristics unsupported if the corresponding articles do not mention any mechanism addressing them.

## 5.5 Assessment results

This section presents the comprehensive findings of our assessment by answering the research questions. Table 5.1 summarizes the results of the study. The MDE and LCDP platforms supporting each quality characteristic of the considered product model are shown.

### Quality characteristics support of IoT engineering platforms (RQ1)

According to the proposed model, eight characteristics with their associated sub-characteristics were evaluated on the approaches considered in this work. This section elaborates on the overall quality performance of such development platforms by comparing LCDPs and MDE platforms. As shown in Table 5.1, MDE approaches to support an average of 12 of the 31 possible sub-characteristics, whereas LCDPs support an average of 18. Furthermore, different sub-characteristics are not supported at all. Figure 5.4 shows the overall characteristic-level performance of the analyzed platforms. Figure 5.4a and Figure 5.4b depict the overall performance of MDE platforms and LCDPs against the eight main characteristics, respectively. To this end, an aggregated performance sum from all sub-characteristics was produced for each characteristic.

Both categories (MDE and LCDPs) perform well enough in quality characteristics related to *Functional suitability, Portability, and Usability*, with general supporting rates of 75.8%, 57.8%, and 50%
<table><caption>Table 5.1: Assessment overview</caption><thead><tr><th>Characteristic</th><th>Sub-characteristic</th><th>MDE platforms</th><th>MDE(%)</th><th>LCDP platforms</th><th>LCDP(%)</th></tr></thead><tbody><tr><td rowspan="3">Functional suitability</td><td>Func. Completeness</td><td>[38] [61] [150] [157] [37] [161] [140] [40] [17]</td><td>81.82</td><td>[139] [89] [90] [138] [147]</td><td>83.3</td></tr><tr><td>Func. Correctness</td><td>[36] [61] [38] [157] [37] [160] [140] [40] [17]</td><td>81.82</td><td>[89] [90] [138] [147] [144]</td><td>83.3</td></tr><tr><td>Func. Appropriateness</td><td>[36] [150] [38] [37] [140] [40] [17]</td><td>63.6</td><td>[89] [90] [138] [139] [144]</td><td>83.3</td></tr><tr><td rowspan="3">Performance efficiency</td><td>Time-behavior</td><td>Unsupported</td><td>0</td><td>[89] [90]</td><td>33.3</td></tr><tr><td>Resource Utilization</td><td>[38]</td><td>9.09</td><td>[89] [90]</td><td>33.3</td></tr><tr><td>Capacity</td><td>[61] [38] [37] [161] [140] [40] [17]</td><td>63.3</td><td>[89] [90] [139]</td><td>50</td></tr><tr><td rowspan="2">Compatibility</td><td>Co-existence</td><td>Unsupported</td><td>0</td><td>[89] [90] [138]</td><td>50</td></tr><tr><td>Interoperability</td><td>[38] [157] [37] [160] [140] [40]</td><td>54.5</td><td>[89] [90] [138] [147]</td><td>66.6</td></tr><tr><td rowspan="4">Reliability</td><td>Maturity</td><td>[36] [150] [38] [157] [37] [160] [140] [40] [17]</td><td>81.82</td><td>[89] [90] [138] [139] [147]</td><td>83.3</td></tr><tr><td>Availability</td><td>[37] [17]</td><td>18.18</td><td>[89] [90]</td><td>33.3</td></tr><tr><td>Fault tolerance</td><td>[36] [61] [140] [40] [17]</td><td>45.45</td><td>[89]</td><td>16.6</td></tr><tr><td>Recoverability</td><td>[38] [17]</td><td>18.18</td><td>[89] [90]</td><td>33.33</td></tr><tr><td rowspan="5">Usability</td><td>Appropriateness</td><td>[61] [150] [38] [37] [17]</td><td>45.45</td><td>[89] [138] [139]</td><td>66.7</td></tr><tr><td>Learnability</td><td>[140]</td><td>9.09</td><td>[89] [90] [144] [147] [40]</td><td>83.3</td></tr><tr><td>Operability</td><td>[36] [38] [37] [160] [161] [140] [40] [17]</td><td>72.72</td><td>[89] [90] [138] [139] [144]</td><td>83.3</td></tr><tr><td>User error protection</td><td>[37] [161] [140] [40] [17]</td><td>45.45</td><td>[89] [90] [138] [144]</td><td>66.6</td></tr><tr><td>User interface</td><td>[36] [61] [150] [38] [157] [37] [160] [161] [140] [40] [17]</td><td>100</td><td>[89] [90] [138] [139] [144] [147]</td><td>100</td></tr><tr><td rowspan="5">Security</td><td>Accessibility</td><td>[37] [17]</td><td>18.18</td><td>[89] [90]</td><td>33.3</td></tr><tr><td>Confidentiality</td><td>[38]</td><td>9.09</td><td>[90] [138]</td><td>36.7</td></tr><tr><td>Integrity</td><td>[38]</td><td>9.09</td><td>[90] [138] [147]</td><td>33.3</td></tr><tr><td>Non-repudiation</td><td>Unsupported</td><td>0</td><td>[90]</td><td>50</td></tr><tr><td>Accountability</td><td>Unsupported</td><td>0</td><td>[89] [90]</td><td>16.6</td></tr><tr><td rowspan="2">Maintainability</td><td>Authenticity</td><td>[38]</td><td>9.09</td><td>[90] [138] [147]</td><td>33.3</td></tr><tr><td>Modularity</td><td>[150] [38] [157] [37] [40] [17]</td><td>54.54</td><td>[89] [90] [139] [147]</td><td>66.6</td></tr><tr><td rowspan="5">Portability</td><td>Modus</td><td>[36] [40] [61] [38] [37] [140] [17]</td><td>63.3</td><td>[89]</td><td>16.6</td></tr><tr><td>Reusability</td><td>[61] [38] [140]</td><td>27.3</td><td>[89]</td><td>16.6</td></tr><tr><td>Analyzability</td><td>[61] [150] [38] [37] [140] [40]</td><td>54.5</td><td>[89] [147]</td><td>33.3</td></tr><tr><td>Modifiability</td><td>[40] [17]</td><td>18.18</td><td>[89]</td><td>16.6</td></tr><tr><td>Testability</td><td></td><td></td><td></td><td></td></tr><tr><td rowspan="3">Overall</td><td>Adaptability</td><td>[38] [157] [140] [17]</td><td>36.36</td><td>[89] [90] [138]</td><td>50</td></tr><tr><td>Installability</td><td>[36] [61] [150] [38] [157] [37] [160] [161] [140] [40] [17]</td><td>100</td><td>[89] [90] [138] [139] [144] [147]</td><td>100</td></tr><tr><td>Replaceability</td><td>[38] [37] [40] [17]</td><td>36.36</td><td>[89] [90] [147]</td><td>50</td></tr><tr><td>Standard deviation</td><td></td><td>135 out of 341 possible</td><td>39.6</td><td>95 out of 186 possible</td><td>51.1</td></tr><tr><td>Standard deviation</td><td></td><td></td><td>30.6</td><td></td><td>25.5</td></tr></tbody></table>

for MDE and 83.3%, 66.7%, and 66.7% for LCDPs, respectively. It is important to note that the listed supporting/non-supporting rates reflect the aggregated characteristic performance calculations from its corresponding sub-characteristics. Although this provides us with an overall picture of characteristic performance, it does not allow us to make a final judgment on whether certain sub-characteristics are better supported than others. For example, in terms of Usability aspects, all 11 MDE platforms satisfy the User interface quality characteristic, although only two of them satisfy the accessibility sub-characteristic.

MDE platforms have limited security and performance efficiency support, with overall support rates of 5.5% and 24.2%, respectively. For instance, in MDE, from the studies considered, only IoTML [38] promotes security by enhancing authentication, confidentiality, and integrity mechanisms while accessing the platform. On the other hand, LCDPs fall short concerning security and maintainability, with overall support rates of 36.7% and 30%, respectively. Each of the security aspects is implemented at least once by LCDPs, with the AtmosphereIoT [90] platform supporting all. Consequently, as shown in Table 5.1, MDE accounts for 135 points out of 341 possible support, representing approximately
<figure><img src="image_31.png" /><figcaption>**Figure 5.4: Quality characteristics support**</figcaption></figure>

39.6%, whereas LCDPs account for 95 points out of 186 possible, which would be about 51.1%.

**Answer to RQ1:** Overall, MDE quality is around 39.6%, while LCDPs account for 51.1%, resulting in the overall quality of the selected engineering platforms being approximately 45.5%.

## Quality sub-characteristics support of IoT engineering platforms (RQ2)

In the previous section, we focused on overall quality characteristic support; in this section, we focus on individual sub-characteristic by highlighting the most and least addressed ones. Even though the average supporting rate of studied quality sub-characteristics for both MDE and LCDPs will be equal to the average supporting rate for the quality characteristics presented above (Sec. 5.5: MDE:39.6%, LCDP:51.1%), they differ in their deviation from the mean of individual supports. According to the standard deviation indicated in Table 5.1, MDE platforms have an average supporting standard deviation of 30.6, while LCDPs have a standard deviation of roughly 25.5, which is less than of MDE platforms. Such figures suggest that LCDPs are more likely to consistently touch all of the model's individual quality sub-characteristics than MDE approaches.

As shown in Table 5.1, IoT MDE and LCDPs cover all quality sub-characteristics such as user interface and installability. This is expected, given that the fundamental principle of MDE and LCDP technologies centers around enabling an easy-to-use and executable environment for developing applications with less effort, which cannot be accomplished without providing a user interface.

Figure 5.5 depicts an overall performance of sub-characteristics among IoT MDE and LCDPs engineering platforms. As indicated, none of the chosen MDE approaches address quality sub-characteristics such as non-repudiation, time-behavior, accountability, and co-existence. LCDPs, on the other hand, fall short on fault-tolerance, non-repudiation, reusability, analyzability, and testability, with a consistent minimal support rate of 16.7%, which implies a generic rate of at least 1 out of 6 LCDPs supports at least one of the sub-characteristics. On the other hand, Figure 5.6 depicts the average quality performance for both MDE and LCDP platforms. As can be seen, sub-characteristics
<figure><img src="image_32.png" /><figcaption>**Figure 5.5:** Quality sub-characteristics performances</figcaption></figure>

such as user interface, installability, and functional completeness are the most well-supported by both categories, while testability, time behavior, accountability, and non-repudiation are the least supported.

**Answer to RQ2:** The top three quality sub-characteristics addressed by both technologies are *user interface, installability, and maturity*. On the other hand, selected MDE approaches were unable to address quality sub-characteristics such as non-repudiation, accountability, co-existence, and time behavior. In contrast, LCDPs fall short of addressing fault tolerance, non-repudiation, and reusability with a consistent rate of 1 out of 6 LCDPs.

# 5.6 Discussion

In this section, we discuss the proposed model’s suitability and its limitation with respect to general software quality assessment of software systems.

## 5.6.1 Model suitability

The proposed model aims to assist practitioners who have to design and develop IoT systems by exploiting low-code or MDE platforms. According to the results presented in Section 5.5, security-related characteristics are the least addressed. This is particularly pertinent given how IoT security concerns are dynamic and unstructured, leading to uncertainty among software developers in terms of concepts and terms [221]. MDE approaches are most affected since most conventional MDE platforms are deployed locally and used offline, making incorporating any form of security capabilities less required (e.g., authentication). We can argue that the dominant Eclipse Development Environment<fnref n="1" />, which hosts a lot of classical MDE-based platforms, has a significant impact on this problem. On the other hand, although LCDPs neither excel in such security-related aspects, it is critical and rational to be integrated since such platforms are deployed in cloud-based environments, which are more likely to be attacked by unwanted intruders.

Furthermore, besides security, the results show that LCDPs lack quality criteria for general maintainability. For instance, according to Table 5.1, the reusability of LCDP is shown to be less supported

<footnote n="1">¹https://www.eclipse.org/</footnote>
<figure><img src="image_33.png" /><figcaption>**Figure 5.6:** Average quality sub-characteristics performances</figcaption></figure>

among others. This is generally true and can mainly be since more LCDPs are tailor-made, and most of their application developments and deployments are bound to a particular technology [58], making them difficult to modify and reuse elsewhere. In our study, only Node-RED [89] showcased means for supporting the usability of aspects of its components through the Node-RED modules that can be composed, built, deployed, and reused separately from one instance to another.

Due to the tight coupling between functions and their sub-functions found in different LCDPs and some MDE platforms, analysability becomes very hard to achieve. Our proposed model defines software analysability as the means to assess the efficacy and efficiency with which it is feasible to determine the impact of modifying parts of the platform by either removing it or injecting failures into it. According to the obtained results, only 27% of MDE platforms support such features, while for LCDPs, only one out of 6 support them. This quality characteristic is generally considered in the software design phase through different model-based system analyses. Still, in the actual implementation of the system, such quality is often ignored [222]. Concerning software testability, as indicated above, only 2 out of 11 of the analyzed platforms showed means for supporting it. The testing can either be done at the platform sub-functions level or at the system they develop. In general, developers tend to disregard this aspect. For instance, in the LCDP domain, low-code testing is still in its early stages, with no formal structure to the domain's ideas, concepts, and hypotheses which undoubtedly contributes to such lacking [215].

Another interesting fact is that the overall performance of IoT engineering platforms (MDEs and LCDPs combined), regardless of characteristics and sub-characteristics, is about 45.5%, in which MDE accounts for 39.6%, whereas LCDPs have 51.1%. Although we acknowledge that these measures cannot be regarded as a definitive measure of the IoT engineering platforms' quality questions, we believe they can point researchers in the right direction regarding the present state of the art in IoT engineering quality evaluation support. Consequently, we can finally draw the line of how the proposed model satisfies the quality evaluation procedure as “promising”; it unveiled several concepts that reflect what is already available in the IoT engineering domain.

## 5.6.2 Model limitations

In this chapter, we have extended the ISO/IEC 25010:2011 standard model to propose a software product model for assessing the quality of IoT engineering platforms. To evaluate the effectiveness of the proposed model, we employed it to determine the quality of 17 IoT platforms selected from our previous studies [8, 13]. In terms of limitations, we can state the followings:
* Although the presented model contributes positively to the software product quality assessment of IoT engineering platforms, it only performs very well when the evaluation is done at the platform's technical implementation level. However, the proposed model performed poorly regarding run-time software product quality assessment, such as quality linked to performance efficiency. This is primarily due to the implementation nature of the LCDP and MDE platforms. We believe that assessing such quality could be heavily influenced by the environment in which such software is deployed.

* The evaluation methods employed in this study and the reported results are critical because they are exclusively based on what was identified in the selected tool's approaches. However, in some instances, the published content of the approach may not correctly reflect the full capabilities of the platform under consideration. Furthermore, software platforms evolve, and new development is regularly contributed to the platforms. Therefore, we believe that integrating the results found in the approaches and the actual inputs from the tool vendors can significantly increase the legitimacy of the findings. We intend to address this in future research.

## 5.7 Conclusion and Future work

The rising market adoption of LCDPs has pushed businesses to incorporate LCDPs into their general-purpose solution stack. However, the effectiveness of such platforms' quality assessment can be controversial since it is linked to how satisfying the platform is to the party evaluating it. To address such concerns, it is strongly encouraged to rely on well-agreed and established standards to eliminate prejudice during decision-making. In this chapter, we presented an extension of the ISO/IEC 25010:2011 product quality model for assessing the quality aspects of IoT engineering platforms. We evaluated the software product quality of 17 IoT engineering platforms using the proposed model. The findings revealed that the overall performance of IoT engineering platforms is roughly 45.5%, with LCDPs doing slightly better than MDE platforms. Furthermore, security and maintainability aspects are found to be less addressed, whereas functional appropriateness, portability, and usability were found to be the most addressed. In the future, we plan to evaluate the quality in the use of the IoT engineering platform by extending the *quality in use model* of the ISO/IEC 25010:2011 standard in which we will be able to accommodate other quality aspects beyond the software product quality model.
# Chapter 6

## CHESSIoT: An approach for engineering multi-layered IoT systems

The challenges related to the complexity and heterogeneity of IoT systems are present in all of its aspects. On the one hand, the development processes need to take into account different design options, such as physical, functional, and behavioral architecture. MDE has demonstrated a significant benefit in automating software development by promoting the use of domain-specific languages (DSLs) tailored to a specific application domain. These models provide abstract system properties in which different sub-systems can be independently modeled, developed, and analyzed before being integrated to construct a fully functional system. As a contribution toward answering the third research problem (RP3), this chapter present the CHESSIoT, an approach for engineering multi-layered IoT systems. Initially the chapter evaluates potential contribution of our proposed approach in terms of modeling language coverage as well as engineering support. We do so by presenting two comparative analysis results that aim to clearly establish the potential of CHESSIoT in the above engineering support by taking into consideration the multi-layered engineering support. In addition to that, we also presents CHESSIoT domain specific languages covering the system, software and deployment aspect of a multi-layered IoT system. Finally a brief discussion is drawn around the above contributions.

**The chapter is organized as follows:** Section 6.1 presents an introduction to the topic. Section 6.2 briefly presents the high-level CHESSIoT engineering methodology. Section 6.3 presents the motivating comparative analysis between CHESSIoT engineering methodology with respect to 12 existing platforms. Section 6.4 presents in detail three DSL extensions used in the whole engineering process. Finally, Section 6.5 discusses the findings, whereas Section 6.6 concludes the chapter.
## 6.1 Introduction

Due to the inherent heterogeneity present in the IoT domain, engineering platforms such as MDE4IoT [36], ThingML [37], IoTML/BRAIN-IoT [38, 39], SimulateIoT [40] and Montithings [17] (just to name a few), have demonstrated the potential of MDE to be a realistic alternative for engineering scalable IoT systems. While that is the case, finding a platform capable of fully integrating different core engineering features becomes critical. From now on, we will constantly use the term "Engineering" to refer to a process that integrates the "development, analysis, and deployment" support from a unique environment when realizing an IoT system.

In this chapter, we introduce CHESSIoT, an approach and a tool for engineering multi-layered IoT systems. We briefly discuss the general methodology supported by CHESSIoT as well as a comparison with the existing engineering approaches in the industry. The CHESSIoT environment is built on top of the existing CHESS toolchain [16] with the aim of providing a fully decoupled extension for supporting the modeling, development, analysis, and deployment of the IoT systems. The CHESSIoT DSLs namely *SystemDSL*, *SoftwareDSL*, and *DeploymentDSL* are aligned with different modeling views as well as the engineering task that they correspond to. Through CHESSIoT, a user can benefit from a multi-view development environment in which each of the supported views has its own underlined constraints that enforce its specific privileges on model entities and properties that can be manipulated

For instance, System-level DSL was designed to satisfy the high-level physical representations and their relationships within a typical multi-layered IoT system. The DSL supports the specification of a typical IoT system covering from the low-level edge layer, Fog-layer as well as to the cloud. Note that, at this level, the model does not include any information related to the functional behavior of elements rather than their main physical construct. Furthermore, this model can later be annotated with failure behavior rules following CHESS-FLA constructs [44] which ultimately is used to conduct the early safety analysis tasks on the model.

In addition to that, the CHESSIoT Software DSL supports the system's functional and behavioral aspects of the system. The DSL extends the rich UML modeling language by means of defining new IoT-specific stereotypes and their interrelation targeting the low level. It is worth noting again that to enforce the model correctness as well as error avoidance during the design phase, palette elements can be hidden or shown based on the current state of the modeling process (eg: diagram type or view type). When the model is complete, a *CHESSIoT2ThingML* model transformation is launched to generate a series of fully functional ThingML [37] source models.

Finally, CHESSIoT offers means for modeling IoT system deployment plans as well as its runtime service provisioning. The IoT system components of a typical IoT system can be deployed at any layer namely edge, fog, and cloud. Designing the deployment plan of such a complex and heterogeneous system has to take into consideration several aspects and be aware of different satisfactory requirements [223]. The deployment model connects the software to the actual system nodes in which the software program will be executed. The model decomposes the inter-dependency between different nodes, machines, and services deployed to it. When the model is complete, a model-to-text transformation can be launched which generate a .yaml configuration file ready to be executed on a docker server.

In addition to the comparative analysis between CHESSIoT and other existing related approaches in terms of their abilities for supporting the engineering features specified above, we have presented the results from the evaluation between CHESSIoT DSL specification and modeling application entities across all layers in relation to the existing DSL. The results have shown huge gap in which CHESSIoT can potentially contribute. We believe that an effective tool should be capable of modeling all aspects of a typical IoT system, from the low-level edge layer to the fog and cloud layers, while maintaining consistency throughout the process.
## 6.2 The CHESSIoT engineering methodology

Through CHESSIoT, a user can benefit from a multi-view development environment in which each of the supported views has its own underlined constraints that enforce its specific privileges on model entities. In the end, the user can perform different engineering activities on the CHESSIoT models such as generating IoT device code, early safety analysis as well as deploying and managing the deployed services.

Figure 6.1 depicts the high-level illustration of the CHESSIoT methodology proposed in this dis- sertation. As from Figure 6.1, the 3 main primary DSLs namely SystemDSL, SoftwareDSL, and DeploymentDSL are the basic starting points for conducting additional engineering tasks supported by our tool. These DSLs are aligned with different modeling views as well as the engineering task that they correspond to. A more detailed explanation of the DSLs is presented in Section 6.4. Depending on the user’s needs, as well as the stage of the development process, a specific view-compliant model corresponding to a specific metamodel is picked for usage.

<figure><img src="image_34.png" /><figcaption>Figure 6.1: High-level approach</figcaption></figure>

In CHESSIoT, different aspects of the system can be designed independently and then interlinked to satisfy specific engineering tasks to be performed on the model. For instance, under the "System view", an IoT Engineer model the IoT system-level architecture that contains all of the system’s major physical components, parts, sub-parts, and their interconnections. This model can then be handed to a safety expert who comes up with the system’s failure logic behavior as well as basic component failure rates to be annotated to the model in order to perform the safety analysis. In the end, a series of model-to-model transformations is performed for achieving both qualitative and quantitative Fault Trees Analyses. More on the safety analysis approach is thoroughly discussed in Chapter 7

In addition to that, under the "component view", the user can define a functional model which contains the system’s key software components, sub-functions, and interrelationships. Furthermore, a behavior model entitles each system’s main sub-function to its own state machine in which aspects such as events, actions, and guards are associated with states and their transitions to realize the desired behavioral goal. When the model is complete, a CHESSIoT2ThingML model transformation is launched to generate a series of fully functional ThingML source models which is then used to gen-
erate platform-specific code ready to be deployed on low-level IoT devices. The same CHESSIoT
software model extended with other extra-functional properties and benefited from the existing sup-
ported analysis. For instance, in our previous work, we demonstrated CHESS support for performing
early real-time schedulability analysis on CHESSIoT models [42, 43]. More details on the CHESSIoT
development approach is discussed in Chapter 8.

Finally, under the "Deployment view", the same user could be able to define the IoT system de-
ployment plan as well as define rules on how to manage deployed services at runtime. The deployment
model decomposes the inter-dependency between different node layers, machines, and one or more
services deployed to it. In addition to that, a DevOps engineer can come through to define a run-
time service provisioning model by automatically configuring software services based on a predefined
model. When the model is complete, a model-to-text transformation is performed which generate a
full .yaml configuration as well as Ansible [224] playbook scripts ready to be executed on a docker
server. More details on the CHESSIoT deployment approach is discussed in Chapter 8.

**6.3 Motivating comparative analysis**

In the Chapter 3, we provided an overview of existing approaches for engineering IoT systems.
These approaches covered categories such as modeling and development, safety analysis, and deploy-
ment support. In this section, we aim to conduct a comparative analysis between CHESSIoT and
12 approaches selected from these approaches to highlight their strengths and weaknesses, thereby
emphasizing the need for a novel approach like CHESSIoT.

Generally, a comparative analysis is a systematic approach used to compare two or more things to identify their similarities and differences and evaluate their relative strengths and weaknesses. In the model-driven community, this approach was widely used. The results of comparative analysis can potentially provide valuable insights that can help developers, researchers, or other interested parties make better decisions about which technology to utilize for a particular task.

The upcoming section will compare existing model-based approaches for engineering IoT sys-
tems based on two primary relative contexts: the tool's supported modeling aspects and their engi-
neering supports. Section 6.3.1 presents an overview of the analyzed approaches, whereas Section
6.3.2 presents the comparative assessment related to the tools' capability of supporting different mod-
eling features in achieving a multi-layered architecture. To be more specific, this part looks at the
tools support for modeling application entities across all layers, namely the low-level edge layer, fog,
and cloud layers elements. Additionally, in Section 6.3.3, existing tools are discussed and compared
with respect to their support for different IoT engineering tasks, including system development, safety
analysis, deployment, and run-time service provisioning.

6.3.1 Selected platforms

Table 6.1 lists the 12 approaches, which have been selected according to the following criteria:

* Basic support for IoT system modeling: The approach focuses on modeling IoT systems and may provide advanced features for manipulating the model.

* Tool maturity: The approach has advanced beyond its initial or conceptual stages and is more mature.

* Age of the tool: The approach has been implemented within the last 10 years.

* IoT-specific: The approach is explicitly designed for engineering in the IoT domain.

The selection process was conducted iteratively, and we considered all three categories presented in Chapter 3 especially Sections 3.2, 3.3 and 3.4. From the selected approaches, only one approach was chosen from the analysis category. This is because most of the presented approaches do not
provide any means for designing IoT system models but rather focus on the manual development of FTs. Moreover, most of the approaches in the analysis category are considered to be outdated and conceptual, which does not meet our established criteria.

<table><caption>**Table 6.1:** Selected approaches for the comparative analysis</caption><thead><tr><td>Tool name</td><td>Title</td><td>Year</td><td>Type</td></tr></thead><tbody><tr><td>MontiThings [17]</td><td>MontiThings: Model-driven development and deployment of reliable IoT applications</td><td>2021</td><td>Journal</td></tr><tr><td>ThingML[37]</td><td>ThingML: A language and code generation framework for heterogeneous targets</td><td>2016</td><td>Conference</td></tr><tr><td>MDE4IoT[36]</td><td>MDE4IoT: Supporting the Internet of Things with model-driven engineering</td><td>2017</td><td>Conference</td></tr><tr><td>SysML4IoT[50]</td><td>Modeling IoT applications with SysML4IoT</td><td>2016</td><td>Conference</td></tr><tr><td>Monitor-IoT[26]</td><td>A domain-specific language for modeling IoT system architectures that support monitoring</td><td>2022</td><td>Journal</td></tr><tr><td>Simulate-IoT[40]</td><td>Simulate-IoT: Domain-specific language to design, code generation and execute IoT simulation environments</td><td>2021</td><td>Journal</td></tr><tr><td>DSL-4-IoT[133]</td><td>Design of a domain-specific language and IDE for Internet of Things applications</td><td>2015</td><td>Conference</td></tr><tr><td>UML4IoT[150]</td><td>UML4IoT-A UML-based approach to exploit IoT in Cyber-Physical manufacturing systems</td><td>2016</td><td>Journal</td></tr><tr><td>IoT-ML/ BRAINIoT[39]</td><td>BRAIN-IoT: Model-based framework for dependable sensing and actuation in intelligent decentralized IoT systems</td><td>2019</td><td>Conference</td></tr><tr><td>CAPS[72]</td><td>CAPS: Architecture description of situational aware Cyber-Physical systems</td><td>2017</td><td>Conference</td></tr><tr><td>Node-RED[89]</td><td>Node-RED: Low-code programming for event-driven applications</td><td>2016</td><td>Open tool</td></tr><tr><td>Silva I. et al. [180]</td><td>A dependability evaluation tool for the Internet of Things</td><td>2013</td><td>Journal</td></tr></tbody></table>

## 6.3.2 IoT modeling support

This section presents the comparative analysis of the considered approaches in terms of their abilities to model application entities across all layers of a typical IoT system (see Tab. 6.2). Our evaluation criteria aim at identifying tools that can effectively model all aspects of an IoT system, from the low-level edge layer to the fog and cloud layers, while maintaining consistency throughout the process. To achieve this, we broke down each layer into more detailed elements. For instance, at the edge layer, we considered modeling node elements such as sensor/actuators, their functionality and behavior, and hardware modeling. We also evaluated support for wireless communication modeling, which we believe is crucial for enabling access to data generated by physical devices and making the edge layer system operational in the digital world.

In the fog layer, we evaluated the tools' support for various fog components such as fog devices, gateways, and fog servers. Similarly, for the cloud layer, we assessed the modeling capabilities of the tools for cloud-based elements like cloud nodes, machines, and services. We also investigated if the tools support multi-view modeling and if they come with a graphical user interface. Additionally, we acknowledged that some tools may have unique modeling capabilities that may not be captured in the checklist, so we added a column to highlight any additional features.

The comparative findings from the assessment presented in Table 6.2 are highlighted below relative to CHESSIoT-supported modeling features that will be presented with details in the next section:

1. From Table 6.2, it can be seen that all of the selected platforms provide a modeling environment, with most of them offering a graphical modeling option, except for ThingML[37], which only offers a textual modeling option. While textual-based approaches may be more scalable, graphical ones are usually more accessible and user-friendly. Textual interfaces can become overwhelming, particularly when the system becomes more complex, and can often come with their learning curve in terms of understanding new textual languages. Similar to MontiThings [17], CHESSIoT
<table><caption>Table 6.2: Comparative table on supporting different IoT modeling features</caption><thead><tr><th rowspan="2">Tool</th><th rowspan="2">Graphical user interface</th><th rowspan="2">Multiview modeling</th><th colspan="4">Edge layer</th><th colspan="3">Fog layer</th><th colspan="2">Cloud layer</th><th rowspan="2">Other supported modeling capabilities</th></tr><tr><th>Device node</th><th>Functional design</th><th>Behavior design</th><th>Hardware design</th><th>Wireless support</th><th>Fog node</th><th>Fog device</th><th>Fog gateway server</th><th>Cloud node</th><th>Cloud machine</th></tr></thead><tbody><tr><td>MontiThings[17]</td><td>Yes</td><td>No</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Error handling design capabilities, deployment planning, featuring deployment design suggestions</td></tr><tr><td>ThingML[37]</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Textual Component and Connector architectures, asynchronous messaging</td></tr><tr><td>MDE4IoT[36]</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Software to hardware allocations, consistency assurance, and run-time self-adaptation design</td></tr><tr><td>SysML4IoT[50, 148, 225]</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>System quality of service (QoS), Publish/subscribe paradigm, system's self-adaptive designs</td></tr><tr><td>Monitor-IoT[26]</td><td>Yes</td><td>No</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>Yes</td><td>Synchronous and asynchronous dataflows design across the edge, fog, and cloud layers to support the monitoring.</td></tr><tr><td>Simulate-IoT[40]</td><td>Yes</td><td>No</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Database designs, wireless sensors and actuator network (WSAN) design support</td></tr><tr><td>DSL-4-IoT[133]</td><td>Yes</td><td>No</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>A visual domains specific modeling language for modeling IoT wireless sensor network</td></tr><tr><td>UML4IoT[150]</td><td>Yes</td><td>No</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Source code level annotations in case the UML design specification is not available</td></tr><tr><td>IoT-ML/ BRAINIoT[38, 39]</td><td>Yes</td><td>No</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td><td>Run-time system deployment and dynamic remote edge/cloud reconfiguration designs</td></tr><tr><td>CAPS[72, 154]</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Physical space view modeling to describe the area involved in situation awareness</td></tr><tr><td>Node-RED[89]</td><td>Yes</td><td>No</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>Cloud-based data flow modeling. Wiring together pieces of code blocks to carry out tasks.</td></tr><tr><td>Silva I. et al. [180]</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>Modeling of IoT network layer as a graph of devices (vertices) and edges (links).</td></tr><tr><td>CHESSIoT</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Support for the design of system failure logic behavior as well as run-time service provisioning</td></tr></tbody></table>

has adopted an approach that integrates both textual and graphical modeling approaches, limiting the use of the textual interface to simple definition tasks such as failure logic behavior rule annotation and run-time service provisioning, while major and complex system modeling is supported graphically.

2. After analyzing the selected approaches, it is evident that a considerable number of them do not support the multi-view modeling approach. This modeling approach is crucial in improving the accuracy of system design and enforcing the separation of concerns, where the model is simplified and designed from various perspectives. Multi-view modeling generally complements component-based design [107], which is also supported by CHESSIoT. These two methodologies have tremendous potential in dealing with the complexities of IoT systems. Among the 12 considered tools, only MDE4IoT [36], SysML4IoT [50], and CAPS [72] platforms provide support for these methodologies.

We acknowledge that there are other platforms that implement alternative approaches that might complement multi-view modeling depending on the modeling context supported by the tool. For example, Node-RED [89] supports a multi-flow modeling approach, allowing different parts of the system to be developed separately from various flows while still sharing a common development context. However, Node-RED practically supports only a single data view that is shared among different flows, and other views are not supported. Therefore, it may not be suitable for more complex IoT systems that require multiple perspectives and separation of concerns.

3. As it can be seen from Table 6.2, the majority of approaches support modeling at the edge layer elements, namely components such as sensors/actuators, computing boards, and so on [17, 26, 36, 89, 225]. Except for IoT-ML [39], which exclusively focuses on the functional aspects targeting
cloud-based resource allocation, other platforms offer even more advanced design mechanisms, such as run-time self-adaptation [36, 225] as well as runtime error handling capabilities [17, 26] at the edge. While technically allowing predefined functional node behaviors to be defined, Node-RED [89] can model the data processing logic of the device-layer elements; however, it does not explicitly support any form of out-of-loop logical behavior specification. CHESSIoT, on the other hand, combines both functional and behavioral modeling of the element at the edge, and through different modeling views, a model portion can then be used for different engineering purposes.

In addition to that, CHESSIoT can explicitly model wirelessly communicated ports that support the MQTT protocol [226]. We have stressed this necessity as an essential factor in making the device layer components alive and suitable to be integrated into the digital world. Eight out of twelve platforms support this feature, which is very promising evidence of how mature MDE approaches are ready to address the scalability and interoperability issues faced in the IoT field.

4. Analysis of the table reveals that only a few of the considered platforms, namely [26, 40, 89, 180], support the modeling of fog layer elements. This lack of support for fog layer modeling is a significant limitation and is frequently cited as one of the drawbacks of MDE approaches in IoT. In our experience, IoT language developers tend to prioritize device-level modeling and development over the fog layer, even though implementing a robust code generator capable of generating fully integrated fog-layer code is a complex task due to the required designs and heterogeneity. Nevertheless, we believe that considering the fog layer is crucial for realizing a fully functional IoT system. While CHESSIoT does not explicitly focus on fog-layer code generation, our approach does provide deployment modeling and artifact generation targeting fog-layer deployment.

5. In the realm of cloud-layer modeling support, it is apparent from the table that only IoT-ML [39] and SimulateIoT [40] offer complete support for cloud-based design. IoT-ML is specifically designed to enable run-time system deployment modeling and dynamic remote edge/cloud reconfiguration, while SimulateIoT provides an IoT simulation and execution environment. This highlights the same issue discussed earlier, namely that most approaches concentrate on low-level development at the expense of other layers. For instance, Node-RED [89] and MonitorIoT [26] focus on service-oriented modeling while ignoring the context in which such services are deployed. In contrast, CHESSIoT enables the modeling of inter-dependencies between different nodes, machines, and services, and facilitates the provision of services deployed on such nodes across all layers.

### 6.3.3 IoT engineering capabilities

In this section, we evaluate the selected IoT approaches based on their ability to support various engineering tasks in developing, analysing, and deploying IoT systems. Table 6.3 shows our investigation of whether a platform follows a well-structured development approach that integrates all its supported engineering tasks and how these tasks are emphasized and followed during the entire process. Specifically, we looked at the tool's capability to generate platform-specific code for development and assessed its support for safety analysis and other types of analysis. Regarding deployment, we focused on the platform's ability to generate deployment-related artifacts and support run-time service provisioning mechanisms. Additionally, we highlighted each approach's specific focus and the typical validation methodology used. We acknowledge that safety-related analysis is not the only important aspect for IoT systems, and we also considered other types of analysis that are supported by each platform.

The results of the assessment are summarized in Table 6.3. Based on these results, we can highlight several interesting findings related to the engineering support provided by CHESSIoT, which is presented in the next section.

1. Table 6.3 shows that three of the considered approaches, namely [133, 150, 180], do not provide any details on the supported engineering methodology throughout their development process. We strongly believe that engineering platforms should have a well-defined methodology that guides
<table><caption>**Table 6.3:** Comparative table on supporting different IoT engineering capabilities</caption><thead><tr><th>Tool</th><th>Following a well-defined engineering methodology</th><th>Development</th><th colspan="2">Analysis</th><th>Deployment</th><th>Approach</th><th>Empirical assessment</th></tr><tr><td></td><td></td><td>Generate code</td><td>Safety analysis</td><td>Others</td><td>GenerateService config. provision</td><td></td><td>Tool's specific focus</td></tr></thead><tbody><tr><td>MontiThings[17]</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td><td>Proof of concept Engineering reliable IoT systems by separating concerns, handling errors, and enabling deployment to heterogeneous edge devices.</td></tr><tr><td>ThingML[37]</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Proof of concept A modeling tool and a highly customizable multi-platform code generator targeting only the edge layer</td></tr><tr><td>MDE4IoT[36]</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Case study Support design, development, and run-time management of IoT systems</td></tr><tr><td>SysML4IoT[50, 148, 225]</td><td>Yes</td><td>Yes</td><td>No</td><td>System QoS</td><td>No</td><td>No</td><td>Proof of concept System functional design, publish/subscribe and self-adaptations at the edge.</td></tr><tr><td>Monitor-IoT[26]</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Proof of concept Multi-layer visual modeling language, and language for monitoring architecture, experimental results ISO/IEC30141:2018 reference architecture¹.</td></tr><tr><td>Simulate-IoT[40]</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>Yes</td><td>Yes</td><td>Proof of concept Define an IoT simulation environment and execute it. Support for databases, complex-event processing engines, or message brokers</td></tr><tr><td>DSL-4-IoT[133]</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Yes</td><td>No</td><td>Case study A high-level visual programming language tailored to develop IoT applications compatible with the OpenHAB framework.</td></tr><tr><td>UML4IoT[150]</td><td>No</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Proof of concept IoT environment to support in the integration of CPS components into modern IoT manufacturing environment.</td></tr><tr><td>IoT-ML/ BRAINIoT[38, 39]</td><td>Yes</td><td>Yes</td><td>No</td><td>Risk analysis</td><td>Yes</td><td>Yes</td><td>Proof of concept Edge/cloud deployment marketplace. Orchestration of distributed IoT systems leveraging dynamic and heterogeneous designs</td></tr><tr><td>CAPS[72, 154]</td><td>Yes</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Proof of concept A multi-view modeling approach that uses ThingML for code generation at the edge/device layer.</td></tr><tr><td>Node-RED[89]</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>No</td><td>Yes</td><td>Well established Multi-flow based development approach. Acts as an interpreter for the data flow-related aspect of the system.</td></tr><tr><td>Silva I. et al. [180]</td><td>No</td><td>No</td><td>Yes</td><td>No</td><td>No</td><td>No</td><td>Proof of concept A dependability evaluation tool for IoT that considers hardware faults and permanent link faults performing safety analyses and generating system FTs</td></tr><tr><td>CHESSIoT</td><td>Yes</td><td>Yes</td><td>Yes</td><td>Existing real-time</td><td>Yes</td><td>Yes</td><td>Proof of concept Multi-layered system and software design, code generation, safety analysis, and deployment for IoT systems within a unified platform.</td></tr></tbody></table>

the development process of IoT systems. Such a methodology can potentially reduce complexity and provide users with fewer complications while using the platform. On the other hand, as shown in Figure 6.1, CHESSIoT offers a well-defined component-based design approach that supports different engineering tasks, such as code generation, safety analysis, and deployment, at different stages of the design and through different viewpoints. This approach enhances the tool's suitability
and significantly contributes to the model’s correctness throughout all engineering stages.

2. In terms of code generation, three platforms - MonitorIoT [26], DSL-4-IoT[133], and Node-RED[89] - do not support any form of code generation that can be deployed on IoT devices. However, one of the main goals of Model-Driven Engineering (MDE) is to enhance automation in the development process [35]. We believe that generating partial or full system code is one of the most critical factors in speeding up the development process. While we recognize the complexities involved in generating fully functional code that can be deployed at any layer without manual intervention, continuous improvement of code generators that attempt to cover the different heterogeneous aspects of an IoT system could help bridge this gap.

ThingML is a platform that provides a code generator capable of producing fully functional code in various programming languages, such as Java, C, C++, JavaScript, Python, and Go, with a particular focus on the edge layer. While the number of supported languages may vary depending on the ThingML version and code generator, additional languages can be integrated by implementing new code generators or expanding existing ones. To save time and resources, CHESSIoT uses ThingML’s code generation system. This is done by transforming CHESSIoT’s software models into ThingML’s models. More information will be provided in Chapter 8.

3. Developers of IoT systems often assume that their devices or systems will always succeed, but this is not the case [29]. Failures can occur for various reasons, including device age, communication protocols, data sources, deployment environment, and human error. In our assessment of 12 platforms, we found that only Silva I. et al.’s approach [180] supports safety analysis through Fault-Tree. However, even this approach only analyzes the fog layer network, which is only a small part of the overall functionality of an IoT system. It is important to note that safety requirements for IoT systems are still emerging and do not always keep up with changing technologies [46]. Many safety analysis approaches presented in Section 3.4 are conceptual and do not support automated fault-tree analysis.

In addition to safety analysis, we discovered that only two out of 12 platforms we examined offer different types of analysis. SysML4IoT [148] supports reliability analysis of IoT systems through verification of the system’s QoS properties, while the BRAINIoT platform [39] uses IoT-ML [38] to support security and privacy risk analysis through a decentralized process that ranks vulnerabilities into four levels: negligible, limited, significant, and maximum. This is a significant engineering challenge in the IoT field.

To enhance the capabilities of CHESSIoT models, additional analyses can be performed using the underlying CHESS infrastructure [16] on which the platform is built. For example, in previous research [42, 43], we showed how timing characteristics could be added to CHESSIoT functional models to conduct real-time schedulability analyses.

4. When it comes to deployment support, MontiThings and SimulateIoT have different approaches. MontiThings uses a deployment manager to capture device states at runtime, generate a valid docker-compose.yml, and send it to devices for execution. SimulateIoT, on the other hand, utilizes Docker swarm to manage deployed docker containers across all node layers. Only four of the twelve platforms examined offer runtime deployment artifact generation or service monitoring. However, CHESSIoT provides an environment that allows for modeling and generation of docker-compose files reflecting services that need to be deployed on machines running at a given node.
## 6.4 The CHESSIoT domain specific language

The CHESSIoT modeling environment has been built on top of the Eclipse Papyrus<fnref n="2" /> in terms of extensions of UML/SysML. The three profiles that make up the CHESSIoT DSL are explained in detail below.

### 6.4.1 System-level DSL

The System DSL has been designed to satisfy the high-level physical representations and their relationships within a typical IoT system. The DSL supports the multi-layered specification of a typical IoT system ranging from the low-level edge layer, Fog-layer as well as the cloud. The language extends the rich SysML modeling language in terms of new IoT-specific stereotypes and their interrelations. Note that, at this level, the model does not include any information related to the functional behavior of elements rather than their main physical construct.

The modeling concepts underpinning the system DSL are shown in the metamodel depicted in Fig. 6.2. The *System* metaclass represents an IoT system as a collection of physical devices and other entities connected to collect, process, send, receive, and store data. These device entities can range from tiny sensors to much larger items like cars and planes. As the top-level representation element, the system can encapsulate other subsystems, allowing the IoT system-of-systems architectures to be supported.

The *IoTElement* represents things that can be physically represented in the IoT ecosystem. This can be of any type depending on the layer from which such an element is regarded. This can range from a tiny micro-controller at the thing layer, a gateway at the fog layer, and a cloud server when looked at from the cloud side. In the physical world, the *IoTElement* can also represent an object as bigger as a car, a plane, or a house. The system can have one or more IoTElements; each with one or more communicating ports. The modeling constructs can be conceptually grouped with respect to the main layers they define, i.e., edge, fog, and cloud layers as described below.

<figure><img src="image_35.png" /><figcaption>Figure 6.2: CHESSIoT System-level metamodel</figcaption></figure>

<footnote n="2">²https://www.eclipse.org/papyrus/</footnote>
**Edge Layer:** *OnDeviceElement* represents any form of low-level IoT device that may contribute to the system's functional behavior at the edge layer. A *SensorBlock* is primarily responsible for detecting changes in its surroundings and reacts accordingly by generating signals that can be interpreted by either a human or a machine. A sensor, lacks a physical input port and, in the event of a failure, can react differently based on the nature and severity of the internal failures. *ActuatorBlock* is a device responsible for reacting to received electric signals and acting upon them by changing the shape, position, or state of the component or part of the system to which it is attached. An electric servo motor, for instance, responds to a signal by turning on, off, changing direction, or speed. In the case of a door-locking system, it can either close or open the door.

*PhysicalBoard* represents a hardware controller on which the software runs. This can include a number of IoT-related boards that are expected to execute the actual code, thus interfacing the sensor and actuator. A RaspberryPi or Arduino board, for example, processes data from various sensors and sends appropriate signals to actuators and other connected devices as the Fog layer. *PhysicalEntity* can be almost any physical object or environment on which a *OnDeviceElement* can act up. A self-driving car software, for instance, runs on various boards attached to the car but not on the car itself. So a car is a physical entity, while those controlling elements can be classified as any type of on-device element.

It should be noted that a physical entity may host other physical entities and interact with other physical entities. In general, we consider physical entities to be passive elements, and in the event of a system failure, they cannot be considered the root cause unless they are categorized as "User". In particular, the concept of *User* refers to a human actor that uses the system or, in certain contexts, is part of the system itself. A user is a special type of *PhysicalEntity* that interacts with other parts of the system at all levers. For example, a user might interact with an IoT application deployed on a remote server while actively participating in such a system's decision-making process. It's worth noting that a user doesn't necessarily need to be a human; it can also be an autonomous entity that's intelligent enough to interact with the system.

**Fog Layer:** *FogElement* is any device that serves as a computational link between the physical and virtual worlds, in this case, cloud infrastructures. If necessary, these components can do preliminary computations and convey the results to the on-device elements. This implies that they may have varied storage and processing capacities depending on the use case and completely different hardware and software features. Any IoT device installed at the fog layer for data processing and storage is represented by a *FogDevice*. The *FogGateway*, on the other hand, transfers information between fog devices and fog servers, as well as cloud servers connected to it. Finally, *FogServer* computes this data to determine the next operation. This layer is critical because it regulates processing speed and information flow. Fog node configuration involves understanding different hardware compatibility, the devices they influence, and networking capabilities.

**Cloud Layer:** A *CloudElement* is a type of IoT device that operates at the cloud level and contributes to the overall functionality of the system. It builds upon standard IoT elements and can be shown as follows. Similar to a *FogServer*, a *CloudServer* hosts various cloud-based services and applications. A consumer entity refers to any third-party element that can communicate with the server to access its data, and can be classified as active or passive. An example of an active consumer entity is a computer running software to monitor and control sensors remotely. On the other hand, a passive consumer entity is a traffic light actuator that receives commands from the server to function.

## 6.4.2 Software DSL

The CHESSIoT's software DSL has modeling constructs that allow for the specification of IoT system behavior. These constructs are displayed in the metamodel shown in Figure 6.3. It is important to note that the software meta-model mainly supports low-level devices at the edge layer, but in some cases, these devices could also be deployed at the fog layer if they fall into that layer. The DSL extends the UML modeling language by defining new IoT-specific stereotypes and their interrelation.
The CHESSIoT Software metamodel can be divided into two main sections, for specifying *functional* and *behavior* aspects of the constituting components as described below.

<figure><img src="image_36.png" /><figcaption>**Figure 6.3: CHESSIoT Software Metamodel**</figcaption></figure>

**Functional aspects: VirtualElement** represents an *IoTelement* in the virtual world. As mentioned in Section 6.4.1, this could be classified as any IoT device, an element that could be of interest at the edge. As shown in Fig. 6.3, the *System* can consist of one or many virtual elements, and depending on the use case, a virtual element could contain one or more virtual elements.

*VirtualEntity* is a virtual representation of the *PhysicalEntity* from the system model in the digital world. This element can represent any object or place where IoT devices or equipment are installed. In a room monitoring system, for example, a room is usually represented as a virtual entity in which other sensors and actuators are installed.

One of the most fundamental components of the IoT ecosystem is the *Sensor*, which is responsible for transforming relevant information from its surroundings into an electric signal that the computing board can process. On the other hand, the *Actuator* converts electric signals from the board into physical events or states, depending on its type. The language supports different sensor categories and types. A combination of sensor category and type servers is a crucial determining factor during transformation, and it is also the same case for the actuator. We understand that there are many more types of sensors and actuators than our approach supports, but for the sake of simplicity, we focused on only a few, as illustrated by the proposed meta-model.

*IoTPort* allows message exchange between two different components by exposing or requiring the data from components. An IoTPort can have one or more integer pins used to generate pin-related code on the virtual board. Two special types of ports, *MQTTPort* and *ClockPort*, are employed in specific
cases. For instance, `MQTTPort` specifies the `MQTT`-related interface that wirelessly communicates with a remote broker. This port contains information about the payload type, broker URL, device topic, and access mode (i.e., publisher, subscriber, or both). When necessary, the clock port is utilized to define logical delay checks. It should be noted that these two special port types are not required to be physically connected to others.

`VirtualBoard` is a virtual representation of the computing board device where the code runs. This device interacts with sensors and actuators and uses the `IoTPorts` to communicate with the external components.

**Behavioral aspects:** Every type of `VirtualElement` has a state machine with a behavioral specification. The following syntax is used to define the behavior of the component. `Payload` is a simple and stand-alone message object that transports data between components. These elements can have zero or more parameters that define the type of data that must be sent.

`Events` are triggered in various ways based on the component's current processing phase. An event can generally be triggered based on the payload condition detected at the ports. An `Event` can be a `ConditionalEvent`, which occurs during the transition process from one state to the other, or an `InternalEvent`, which occurs internally within the state of a certain component.

Depending on the sort of action to be taken, `IoTAction(s)` can be of many forms. These actions can be customarily defined, or they can reuse the information about the payload and the port where such action must be carried out. For example, when entering or quitting a state, an action can be classed as `OnEntry` or `OnExit` actions. There are two main types of actions:

* **SetAction**: This is used during external communication between two components through a predefined port.

* **GenericAction**: It is a specific type of action that can be implemented during the design phase for particular measures such as assignment, print, loop, checking a value status, function call, and so on. These actions require different arguments and can be customarily implemented with platform-specific code.

`IoTState` defines the situation of the component from its initial engagement to its final disposal in the ecosystem. `IoTState` extends existing UML states by collecting all behavior information relating to events and activities that must be performed at a specific time. From a transition standpoint, an `IoTState` can be classified as a source or target, along with an initial, intermediate, or final state.

`IoTTransition` makes it possible to transition from a source state to a target state while preserving the trigger from the invoking condition as well as the guard value. `IoTGuard` expressions are boolean expressions defined by state values. They enable a state transition by determining whether the `OnExit` action was correctly completed.

### 6.4.3 Deployment metamodel

The CHESSIoT's deployment DSL has been defined to aid in the design of the deployment strategy. The primary purpose is to provide an intuitive way for the user to define the deployment architecture as well as runtime service provisioning procedures that can be applied to configure such generated services remotely. The DSL addresses service-oriented deployment topologies, mostly at the fog and cloud layers. The main concepts of the deployment metamodel are shown in Fig. 6.4.

A `Node` is a central component that connects all other deployment elements. It represents a computing cluster at the center that combines one or more data processing units. These nodes can be found at any layer, including edge, fog, or cloud, and are labeled as `DeviceNode`, `FogNode`, and `CloudNode`, respectively.

The `Machine` construct is for specifying a dedicated middleware server that can host one or more services running on it. Machines could be anything from small computer boards at the edge and fog layers to huge cloud-computing servers. In our context, the machine is always declared inside a node
<figure><img src="image_37.png" /><figcaption>Figure 6.4: CHESSIoT Deployment Metamodel</figcaption></figure>

and should eventually have an IP address that the service operating on could be identified from. Other properties, such as memory capacity and operating system, can be also specified by the user.

In IoT, a *Service* is a self-contained entity that can consume acquired data and apply computational logic to achieve a goal. It can be deployed at practically any layer of the system, depending on the type of need and the computation capabilities of the node in which the service is to be run. Services can connect via Web protocols (e.g., HTTPS, MQTT) and may also depend on one another.

As with CHESSIoT, the end goal of deployment modeling is to generate docker configuration files (.yaml files), therefore, a service must be established with basic parameters like imageURL, ports, persistence, and so on. If the service needs to persist data on the platform, a *volume* attribute must be specified, as well as the boolean *persistence* value set to true. The *priority* attribute specifies the order in which individual services are prioritized in case of a machine memory shortage.

We are mainly concerned with IoT services that are generally involved in a typical IoT ecosystem. An *MQTTBroker*, for example, is used to define a remote MQTT server service, and attributes such as broker type (Mosquitto, HiveMQ, Moquette) are supported. The broker, which is also a service, captures its specific properties such as type, anonymous access, persistence, username, and password. The current implementation enables a user-friendly environment, and in case no data is provided for a given property, default values are used instead. Other services, such as *DataDistributionService* like KAFKA, RabbitMQ, and ApacheSpart, are due to be supported.

Furthermore, the environment enables customary configured services, and when such a property is employed, the definition is added to the generated file unchanged. Furthermore, any IoT-specific *ExternalService*, such as Node-Red<fnref n="3" />, as well as *StorageServices* such as database containers, could be specified. Finally, *OnDeviceApp* can be defined, allowing it to be distributed on edge devices.

A *DeploymentAgent* is a collection of predefined expressions determined at the node level to demonstrate the run-time service provisioning behavior on the machines deployed at the nodes. *DeviceDepAgent*, *FogDepAgent*, and *CloudDepAgent* are defined to perform this task at the edge, fog, and cloud layers, respectively. Details on the developed textual deployment language and the corresponding code generator are given in Chapter 8.

<footnote n="3">³https://nodered.org/</footnote>
**6.5 Discussion**

In the preceding sections, we presented the results of a comparative study that examined the capa-
bilities of CHESSIoT in relation to 12 other related works. Our focus was on assessing the platform's
ability to provide a multi-layered modeling environment and offer engineering support in terms of
development, analysis, and deployments. In this section, we will discuss the significant findings by
highlighting key outcomes from the two contexts considered.

Regarding the support for IoT modeling, when evaluating a cumulative sum of all 12 platforms,
we found that out of the 154 feasible possibilities (excluding the CHESSIoT row), 80 were supported,
while 74 remained unsupported. This translates to an average gap of 48.08%. On the other hand, in
terms of IoT engineering support, out of the 66 possible points, only 26 were supported, leaving 40
unsupported, resulting in a gap of 60.6%. Figure 6.5 provides an overview of the study's outcomes,
displaying the percentage of supporting and non-supporting aspects in both contexts.

<figure><img src="image_38.png" /><figcaption>**Figure 6.5:** Overall comparative supporting results</figcaption></figure>

According to Figure 6.5, when considering both modeling and engineering non-supporting aspects, the average gap where CHESSIoT could potentially contribute is 54.34%. While SimulateIoT [40] was found to be the best-performing platform, covering 13 out of the 14 basic modeling features of interest, it lacks certain features that CHESSIoT supports, such as multi-view modeling, system failure logic design, and run-time service provisioning.

In Sec. 6.3.3, we observed that SimulateIoT [40] does not support various engineering tasks related to IoT system analysis and takes a different approach to run-time management of IoT services. Although it may not be feasible for CHESSIoT to incorporate all platform-specific modeling capabilities supported by other platforms, the flexibility and customizability of the CHESSIoT platform allow for the potential integration of relevant modeling features in the future.

One notable gap in the analyzed IoT platforms is their performance in IoT system analysis, in-
cluding verification, validation, and analysis of IoT systems under development [42]. Due to the com-
plexity and scale of IoT systems, physical replication, and testing become challenging [69]. The lack
of standardized realistic reference models that accurately capture the interactions between sensors, apps, and actuators further exacerbates the issue. CHESSIoT addresses this by extending models with extra-functional properties and supporting analysis capabilities, as demonstrated in previous work on real-time schedulability analysis [42, 43].

While CHESSIoT contributes to both system modeling and engineering aspects of IoT systems, including code generation, safety analysis, deployment, and run-time service provisioning, there are limitations that need to be addressed in the future. The absence of standardization and agreed-upon reference architecture in the IoT domain often results in platforms not adequately addressing essential requirements. Although CHESSIoT's modeling language drew inspiration from the IoT-A reference architecture's multi-view approach [7], it deviates from it by introducing new concepts like failure logic modeling and deployment-related design. However, other modeling constructs related to information flow, security, and more are not yet covered in CHESSIoT.

Furthermore, the Fault-Tree Analysis approach used for safety analysis in system dependability analysis needs to comply with international software dependability and safety standards, such as [227]. While the current tool has not yet achieved international certification, it is being tested in industrial settings with large models and increased complexity to validate its effectiveness. Lastly, CHESSIoT currently lacks the means for testing generated software to support safety analysis results that reflect real-world conditions.

## 6.6 Conclusion

In this chapter, we present briefly introduce CHESSIoT engineering methodology. We have presented the general overview of CHESSIoT support for modeling, developing, analyzing, and deploying IoT systems. We have presented a motivating comparative analysis that draws the CHESSIoT contribution in relation to 12 existing platforms. To put things into context we have presented the CHESSIoT DSL which constitutes 3 main primary DSLs namely SystemDSL, SoftwareDSL, and DeploymentDSL. In order to assess the contribution of our DSL with respect to other existing DSLs, we have presented an evaluation between CHESSIoT and the same 12 platforms considered above in order to present their abilities for model IoT system entities that cover all layers. Finally, we have observed that in overall modeling features and engineering capabilities, an average gap of 54.34% gap was discovered to which CHESSIoT is potentially going to contribute.
# Chapter 7

## CHESSIoT safety analysis support for safety-critical IoT systems

Safety-critical IoT systems are those systems whose failure could result in loss of life, significant property damage, or environmental damage. Although the IoT industry seeks to cut development costs and bring new products to market as fast as possible, the safety analysis of such products is sophisticated and time-consuming, and sometimes it is not seriously taken into account, or it is simply neglected. Early-stage safety analyses can not only potentially reduce the cost of late failures, but they can as well help to easily trace and determine the source of the failure if something goes wrong within the system. As a contribution toward answering the fourth research problem (RP3), this chapter presents a two-fold safety analysis approach built on the extended CHESS Failure Logic Analysis (FLA) technique to support the safety analysis of IoT systems based on Fault Trees (FTs). In addition to its ability to generate the system FTs, the new Fault-Tree Analysis (FTA) approach automatically performs qualitative analyses by eliminating unnecessary paths as well as redundancies in the FT's events. Furthermore, the presented FTA performs a quantitative probabilistic analysis by calculating the system-level top failure event probability from the failure rates of the system's internal parts. This approach contributes to an improvement in terms of time spent performing safety analysis, as well as the correctness, consistency, and modularity of the analysis process. Furthermore, we employ a Patient Monitoring System (PMS) case study to demonstrate the efficiency of the proposed approach as well as the capability of the supporting tool. Finally, to assess the effectiveness of the presented approach, we used an experimental approach to compare it with 14 existing techniques.

**The chapter is organized as follows:** Section 7.1 presents an introduction. Section 7.2 presents the proposed safety analysis approach; Section 7.3 presents the evaluation mechanism while Section 7.4 presents the experimental results from the evaluation. In Section 7.5 concludes the chapter and draws some perspective work.
## 7.1 Introduction

Dependability is regarded as the ability of the system to provide services that can be trusted within a specific period [57]. It is mainly characterised by five essential attributes: availability, reliability, maintainability, integrity, and safety. In this chapter, we focus on safety, which is defined as the absence of catastrophic effects for the user(s) and the environment [4]. It is one of the major features that must be investigated and taken into account during the development phase of such systems to prevent further disasters. Again, from the healthcare system's point of view, when something goes wrong, it is critically important to react quickly and with high precision to isolate the danger before it happens. However, these systems are very complex and involve high-tech interconnected devices in which the process of discovering faults can be very long and tedious.

In the past, safety engineers relied on different informal design artifacts and documents, such as requirements documents, to measure the safety compliance of the system with less or no involvement of system engineers. Later, several approaches, such as [4, 30–33] (to mention a few), have emerged in the field. These approaches add a degree of automation during the analysis process, bridging the gap between the system and safety engineers. However, these approaches were designed and developed to fit domains such as aerospace, automotive, and cyber-physical systems. In some cases, they might not fully suit the IoT domain. This is mainly due to the inherent huge degree of heterogeneity in IoT ecosystems, not to mention its current rapid evolution.

This chapter presents both a modeling and an early-safety analysis environment for safety-critical IoT systems based on the Fault-Tree Analysis (FTA). The approach runs on top of CHESSIoT, a model-driven environment for engineering industrial IoT systems [42, 43]. The CHESSIoT environment is built on top of the CHESS tool [16], an open-source model-driven tool that offers cross-domain modeling, development, and analysis of high-integrity systems. CHESS supports different kinds of analysis including but not limited to real-time schedulable analysis [16], Contract-Based Analysis [111], and Quantitative Reliability Analysis based on Mobius [116]. In addition to this, CHESS also offers means to perform early safety analysis based on Failure Logic Analysis (CHESS-FLA) [44], however, that existing infrastructure is not suitable enough to support the IoT domain as well as not mature enough to support the Fault-Tree Analysis as one of the common and necessary artifacts used in the process of safety analysis [78, 79].

This chapter mainly focuses on CHESSIoT's extension to CHESS for supporting the early-safety analysis of IoT systems based on the FTA. The presented approach relies on and extends the existing CHESS-FLA infrastructure [44]. Normally, CHESS-FLA offers means to: i) model the system's failure behavior through the decoration of the system's simple components following the Failure Propagation Transformation Calculus (FPTC) annotation [33], ii) run the Failure Logic Analysis (FLA), iii) and propagate the analysis results back onto the original model [45]. Throughout the CHESS-FLA analysis process, the entire system's behavior is automatically determined solely from the composition of its elements. Thus the potential of automatic model-based safety analysis is significant. It is achieved by calculating the failure behavior from the composite parts up to the system level. This can help in predicting the impact of a component change or architectural change on a system in a very cheap way [33]. Furthermore, suppose an essential failure behavior occurs at the model system level. In that case, it will be easy to discover the source of the fault immediately and identify where the fault tolerance measures should be directed in the architecture to mitigate them.

The new proposed extension extends the CHESS-FLA by supporting the definition of the failure behavior of a simple component with no input ports. In addition to that, the extension support the generation of the system's complete Fault-Trees as well as performing qualitative and quantitative FTA Analysis. In the proposed approach, the qualitative analyses help in eliminating unnecessary paths as well as redundancies in the FTs' events. Quantitative analysis, on the other hand, allows the user to set the basic event probabilities and calculates the failure probabilities of an entire system from its constituent parts' failure event probabilities. This calculation is automatically performed following the well-known logic probabilities calculation mechanism techniques [46–48]. Aside from the FTA, the existing CHESS infrastructure also supports the generation of the Failure Mode Effect Analysis
(FMEA) table [83], one of the common and necessary artifacts used in the process of safety analysis.

Throughout the chapter, we present the evaluation mechanism and the outcomes of the experi-
mental evaluation we conducted to better assess how the proposed approach performs compared to 14
existing approaches drafted from the literature. In doing so, five main features have been considered: i)
support for system design modeling, ii) support the failure behavior modeling, iii) automated FT gen-
eration, iv) support for automated qualitative FT analysis, v) and support for automated quantitative
FT analysis. In addition, we used a Patient Monitoring System (PMS) case study to demonstrate the
effectiveness as well as the capability of the supporting tool. As a result, we summarize this chapter's
key contribution as follows:

* We present CHESSIoT's system-level modeling environment for designing IoT systems, considering the system's physical aspects.

* We presented the CHESS-FLA extension to support the safety analysis suitable for the IoT domain.

* We introduce an automated Fault Tree generation approach that can handle large and complex models while still supporting critical features like event tracking and component sub-tree generation.

* We present both qualitative and quantitative FTA analysis approaches on the generated Fault-Tress.

* We present the experimental results from a relative evaluation mechanism conducted in comparison with 14 existing approaches in the literature.

* We present a Patient Monitoring System (PMS) case study to demonstrate the effectiveness of the proposed approach as well as the capability of the supporting tool

7.2 Proposed safety analysis approach

This section presents a model-based safety analysis approach based on automated calculus that
can potentially reduce the time-consuming human effort and error-proneness of the IoT safety anal-
ysis procedure. In Section 7.2.1, we present the safety analysis process supported by the proposed
tool; in Section 7.2.2, we describe the extended FPTC syntax while Section 7.2.3 describe briefly the
transformation process. In Section 7.2.4, we introduce the FT generation process whereas Section
7.2.5 introduces the FTA approach detailing both the qualitative and quantitative approaches.

7.2.1 Model-based safety analysis process

IoT systems can experience failures due to various factors, including device age, data source prob-
lems, network issues, deployment environment, and external constraints. For instance, human error
can also cause problems. The CHESSIoT safety analysis approach proposes an early safety analysis
method using Fault-Tree Analysis, which involves annotating a system model with failure behavior
rules using the Failure Propagation Transformation Calculus (FPTC) notation [33].

As illustrated in Figure 7.1, the safety analysis process typically commences with the *IoT system engineer* creating a model based on the gathered *system functional requirements* ① in Figure 7.1. These requirements are mainly acquired through close collaboration with the *client*. The system-level model encompasses the system's major functional components, sub-components, and their interconnections. These system components can be represented as blocks in SysML Block Definition Diagrams (BDD), which align with the abstract syntax meta-model illustrated in Figure 6.2. Internal Block Diagrams (IBD) are employed to illustrate the interdependencies between these components, facilitating the identification of error propagation paths. Each part or block can be assigned to a specific architectural subsystem or component. The physical architecture should possess extensibility
to accommodate the addition of new components or blocks as necessary. The entire safety analysis
process is fully detailed in the next sections.

<figure><img src="image_39.png" /><figcaption>**Figure 7.1: Safety analysis process**</figcaption></figure>

Once the system model is complete (see the CHESSIoT model (2) in Fig. 7.1), it can be handed to the safety engineer for further safety analysis. The safety expert, similarly to the system engineer, can derive safety requirements (3) from the needs of the client, domain standards, and his or her expertise in order to ensure optimal safety. Starting from identifying the typical system-level failures, the safety engineer identifies the failure behavior for each component following the Failure Propagation Transformation Calculus (FPTC) notation. The FPTC technique enables the analysis of component-based systems with cyclic data, control-flow structures, and closed feedback loops. Such failure behavior referred to as FLA rules (4) are annotated to the system’s simple comments to illustrate how failures might occur in a system component and how they are propagated from one component to another. At this stage, the safety engineer can additionally set the component’s failure rates (5) to be used for quantitative analysis.

7.2.2 FPTC Calculus

The FPTC technique solves a significant limitation by allowing the analysis of component-based
systems with cyclic data, control-flow structures, and closed feedback loops. The extended annota-
tions explain how failures might occur in a system component and how they are propagated from one
component to another. Based on its nature, a function/component can propagate a failure (carrying a
failure from input to output), transform a failure (changing the nature of a failure from input to out-
put), act as a source of failure (creating a failure despite no failure in input), or act as a sink (avoiding
the failure to be either propagated of transformed). The following three abstract categories of failure
modes are assessed: service provision failures, such as the omission or commission of the output;
timing failures, such as the early or late delivery of the output; and value domain failures, such as the
output value being out of a valid range, stuck, biased, exhibiting a linear or non-linear drift, or erratic
behavior. In addition to that, a noFailure annotation is used to indicate a no-failure mode at the input
port. Table 7.1 describes different failure modes and their description.

First integration of the CHESS-FLA in CHESS was done in [44] with the support for FI⁴FA (Formalism for Incompletion, Inconsistency, Interference, and Impermanence Failures Analysis) [228], which is an extension of FPTC that takes into consideration Incompletion, Inconsistency, Interfer-
<table><caption>**Table 7.1:** Failure types</caption><thead><tr><td>Failure type</td><td>Description</td></tr></thead><tbody><tr><td>Early</td><td>output is provided too early</td></tr><tr><td>Late</td><td>output is provided too late</td></tr><tr><td>ValueCoarse</td><td>output out of range in a detectable way</td></tr><tr><td>ValueSubtle</td><td>output not in range in an undetectable way</td></tr><tr><td>Omission</td><td>no output is provided</td></tr><tr><td>Commission</td><td>an output is provided when not expected</td></tr></tbody></table>

ence, and Impermanence Failures, and their corresponding countermeasures. However, in the previous version, it was impossible to express a component's failure behavior when it possesses no input ports.

In the existing infrastructure, when modeling the failure expression at a certain port, always the port's name and the failure type are always necessary. This means that a "noFailure" mode is assigned to that port to depict the event in which an internal failure has occurred in the systems causing a failure at any output post of interest. However, it is not always obligatory for a component to an input port for it to fail. For instance, in IoT domain, a temperature sensor's role is to sense the surrounding environment and relay the information to the connected parties. In that case, the sensor needs only a physical output port in order to function properly. The new extension allows such internal failure scenario to be expressed using a "(\*)" notation. The new extended syntax is depicted in Listing 7.1.

1. `FLA:"LHS">"RHS"; #left/right hand side of an expression`
2. `LHS=portName."bL"|portName."bL"|portName."bL")+(|"(*)"`
3. `RHS=portName."bR"|portName."bR"|portName."bR")+`
4. `bL=wildcard|"bR"`
5. `bR=noFailure|"FAILURE"`
6. `FAILURE=early|late|commission|omission|valueSubtle|valueCoarse`

<caption>**Listing 7.1:** Extended FLA syntax expression</caption>

### 7.2.3 CHESS2FLA transformation

The *Annotated CHESSIoT model* (6), produced by the system expert as previously explained, gets automatically transformed by means of the CHESS2FLA model-to-model transformation to generate the CHESSIoT FLA model (7). During the transformation, each component is essentially considered a black box that can only exchange data through its ports. The FLA technique automates the calculation of a complete system failure behavior starting from the failure behavior rules of its separate composite components and interconnections. This, in turn, means that the failure behavior of composite elements is also determined by the failure behaviors of its instantiated simple components and their internal decomposition. Simple components have no other parts and rely on the failure behavior defined in the previous stages. When the failure modeling is finished, the model undergoes a CHESS2FLA model-to-model transformation which transforms it into an FLA model following the meta-model presented in Figure 7.2. This transformation is domain agnostic, and it does not consider any domain-specific construct other than linking each component and its final state of failure into a single model instance.

As seen from the FLA meta-model, a composite component represents a subsystem that contains one or more sub-components. As mentioned in the previous stages, this component does not possess the failure behavior by itself, but it relies on its sub-components to determine its failure behavior. On the other hand, a simple component represents a functional component that can contribute to system failure. Each component contains input and output ports and the annotated failures (according to FLA rules) for simple components. In contrast, in composite components, the propagated failure is assigned to the ports. Furthermore, each simple component is assigned rules where each rule contains input and output expressions reflecting failures and their respective ports. During the transformation, the extended notation of the internal failure of a component with no input ports creates a unique virtual port assigned with a "noFailure" failure type at the component input port to reflect the idea of the component's internal failure source.
<figure><img src="image_40.png" /><figcaption>Figure 7.2: CHESS-FLA meta-model [3]</figcaption></figure>

At each level of the FLA analysis, the results are back-propagated onto the original model to assign each component's failure state to be reflected in the model. The final failure state at simple and composite components as well as at the system level is reflected when the analysis is done. The system FT ⑨ and FMEA ⑧ table can be automatically generated and analyzed before being sent back to the safety expert for consultation. If something is wrong, changes can be made before the final inspection. In the following sections, we briefly review each step of the supported analysis process. Although the FMEA analysis is included in the CHESSIoT safety analysis process, this paper focuses primarily on the FT-based analysis approach.

### 7.2.4 Fault-Tree generation

The FT generation process is performed following a FLA2FT model for the transformation of a CHESS-FLA model into a conforming FT model. Figure 7.3 shows an FT meta-model adopted from [4]. The FTModel element is the top element of the tree, and it is instantiated for each failure that propagates to the system output ports during the FLA analysis. Each FTModel element contains a logical network of events and gates that together form an FT. The entire FT generation process is covered in the following sections.

#### Fault-tree events

In our proposed approach, each event can be graphically identified in the FT from its unique identifier (ID). An event ID is defined as a pair of “failure names” and “port names” in a given component. This ID never changes through the FT generation as well as in the FT analysis process. This can potentially help in event tracking when comparing generated and analyzed FTs. In addition to that, each event has its own name which by default combines the information regarding its corresponding failure and its effect in a given component. The effects can be of type *top failure* type at the system level, *local failure* caused by the system's intermediate failures across the tree, *injected failure* resulted from injected faults, and *internal failure* resulted from the component's internal faults. In the next part, we are going over the various event types that we use to construct the FT and we will describe how events are generated throughout the transformation process.
<figure><img src="image_41.png" /><figcaption>Figure 7.3: FT meta-model [4]</figcaption></figure>

* **Basic events:** A simple component may suffer different kinds of malfunctions, generating either one or more kinds of internal failures. One or more notations may be required to define such events for a given component. A basic event is used to represent a failure that is initiated inside a simple component. This can be basically referred to as a simple component acting as a source of failure. In this case, a failure condition is present on any of the output ports despite no failure at its input port. In case a simple component does not possess any input port, the newly developed approach allows the definition of such condition following Expression (7.1). On the other hand, in case a simple component possesses one or more input ports, its failure behavior can be defined by explicitly initializing all the input failures of the component with a "noFailure" condition (Expression 7.2). Figure 7.4a shows the basic event representation in an FT resulting from an internal failure of a component.

$$ (*) \rightarrow p.\text{failure}_{(out)}; \qquad (7.1) $$

$$ p_1.noFailure, \dots, p_n.noFailure \rightarrow p_{(out)}.failure_{(out)}; \qquad (7.2) $$

**NOTE:** Considering $p_1, p_2$ to $p_n$ to be the input ports and $p_{(out)}$ to be an output of a simple component. If any of their corresponding failure condition is different to "noFailure", then the above condition is not met, so, all "noFailure" conditions on other ports are ignored as they do not contribute toward to the logical failure behavior of the component.

* **External events:** External events are used to represent failures that can be introduced from the environment outside the system boundaries. CHESS provides the possibility to inject failures in the system through the system-level input ports. These faults are modeled with a comment annotation with «FPTCSpecification» stereotype attached to the relevant input port where the fault is being injected in SysML block diagrams (the case study in Section 7.4 demonstrates this concept in more details). The injected fault comment specifies the type of failure attribute being injected into the system. This fault injection can also be done on a composite sub-system under analysis. Figure 7.4b shows a graphical representation of an external event resulting from an injected fault in the system.

* **Intermediate events:** An intermediate event is used to describe the local failure effects resulting from a logical output of one or many events. In the presented approach, these events are generated to represent the failure condition at the input or output port(s) of the simple component resulting from an internal failure or other failure condition from the outside of that simple component. It is also used to represent the top event of an FT. Figure 7.4c is used to represent an intermediate event.
* **Underdeveloped events:** The underdeveloped event is used to specify an event resulting from a failure in which we do not have sufficient information about it. This can basically happen when a failure is introduced at the input port of a simple component without a preceding definition of how it was been propagated or injected at that input port. During the FT generation process the symbol in Figure 7.4d is used.

<figure><img src="image_42.png" /><figcaption>Figure 7.4: Event types</figcaption></figure>

## Failure propagation

Failure propagation occurs in a component when a single input port failure condition of a component is directly transferred to the output ports of the same component without changing its nature. This failure propagation can be modeled in CHESS-FLA using the notation in Expression 7.3. A propagation also occurs between two connected components, when a failure condition at the output port of the preceding component is transferred to the input port of the following component.

$$p_{(in)} \cdot failure_1 \rightarrow p_{(out)} \cdot failure_1; \qquad (7.3)$$

## Failure transformation

A failure transformation occurs within a component when a failure condition present at the input port of a simple component is converted into another type before reaching the output port (Expression 7.4). A failure transformation can also occur when more than one failure expression of any type the exception of a "noFailure" or "wildcard" at multiple input ports is transmitted to a single output port (Expression 7.5). Even if the failure has the same type, the fact that the component converts two failures at its input ports to a single failure at the output port is regarded as a failure transformation.

$$p_{(in)} \cdot failure_1 \rightarrow p_{(out)} \cdot failure_{(out)}; \qquad (7.4)$$

$$p_{(in1)} \cdot failure_1, \dots, p_{(inN)} \cdot failure_N \rightarrow p_{(out)} \cdot failure_{(out)}; \qquad (7.5)$$

## Fault-tree generation process

The system FTs are generated through a series of model-to-model transformation mechanisms written using the Epsilon Transformation Language (ETL) [229]. The process starts by instantiating a number of FT objects equal to the number of failures that propagates to the output port(s) of the system. At this stage, each error that propagates to the output port(s) of the system is represented in its own FT. Note that when a "noFailure" condition is propagated to the output, it is ignored. This technically means that the system acts like a failure sink and it is able to mitigate its propagation to the output of the system, which is also true for all other sub-systems. To achieve that, the Algorithm 1 is followed.

In the next steps, each FT is built separately and recursively. The initial action involves the creation of a top event among all. A top event is generated as a result of the failure propagation to the system output port. In terms of logic gates used in the FT, only "AND" and "OR" gates are adopted. An
transformation concepts. A brief description of the following algorithm is described in Algorithm 3.

<figure><img src="image_43.png" /><figcaption>Figure 7.5: Expression 7.5 corresponding tree</figcaption></figure>

In the newly developed environment, it is possible for a safety expert to assign failure rates of any of the events leading to a basic failure condition. These events include the internal failure condition and the injected failures. In addition to the failure rate assignment, the user is also able to add a failure description in a textual format to reflect the proper cause of the failure. This is potentially important when, for instance, a simple component might have two different internal failure conditions leading to two different outputs. For instance, an aging device can still work by providing the wrong output leading to a *valueCoarse* or *valueSubtle* failure at the output port whereas a blown device fuse will automatically halt its functionality, therefore, leading to an "omission" failure at the output port. Assigning such information to the FT model will eventually improve its readability. This assignment is done way before the FLA2FT transformation process and when the transformation finishes, a probability file is generated separately from the model in an Excel file format. This file is later loaded into the model to support the quantitative probabilistic analysis process.

**Note:** the full transformation code on FT generation can be found in the appendix B.1

### 7.2.5 Fault-Tree Analysis

The proposed approach supports both the qualitative and the quantitative FT analysis through the use of rigorous model transformation techniques. In this section, we are going through the supported analysis.

#### FT Qualitative analysis

The FT qualitative study is conducted using an FT2FT model-to-model transformation (ie: transformation from (9) to (10)) in which the FT meta-model in Figure 7.3 serves as both the source and the target meta-model. This effectively creates new FT representations in the workspace, permitting users to reuse both the generated and the analyzed FTs at the same time. The goal of this qualitative analysis approach is to provide a new representation of the existing FT that only includes the essential representations. Although the current qualitative analysis does not fully reflect the calculation of the minimal event sets needed for a system to fail (minimal cut-sets [230]), it does provide a much shorter and more readable FT that still reflects the goal for the analysis.

During the qualitative analysis process, the following actions are performed:

* **1. Removal of internal component failure propagation:** One of the goals of an FT is to help users to discover and trace down the source event of a system failure in a more easy and intuitive
<caption>**Algorithm 3:** Recurse FT operation</caption>

**Data:** current_failure as f, current_port as p, current_FT as FT, event_to_construct as eG
**Result:** final populated FT
**if** p is of not system's port then
    **if** p is of a "Simple Component" then
        **for** outExp in p.owner_rules.outputExpressions do
            **for** f1 in outExp failsures do
                **if** f1==f then
                    Create an intermediate event e0 ← LOCAL FAILURE;
                    Assign gate to e0 based failures at the p port;
                    Add e0 to eG;
                    Add e0 to FT;
                    **for** inpExp in p.owner_rules.inputExpressions do
                        **for** f2 in inpExp failsures do
                            **if** f2 is not a "wildcard" then
                                **if** f2 is a "noFailure" then
                                    Create a basic event eT ← INTERNAL FAILURE;
                                    Add eT to eG;
                                    Add eT to FT;
                                **else**
                                    Create a intermediate event e1 ← LOCAL FAILURE;
                                    Assign an OR gate to e1;
                                    Add e1 to e0;
                                    Add e1 to FT;
                                    **for** p1 in inpExp.port connectedPorts do
                                        **recurseFT** (f2, p1, FT, e1); // Recurse from the start of the Algorithm 3
                                **end**
                                **end**
                                **end**
                                **end**
                                **end**
                                **end**
                                **end**
                                **end**
                                **end**
                                **end**
                                **end**
                                **else**
                                    **for** p1 in p connectedPorts do
                                        **recurseFT** (f, p1, FT, ev); // Recurse from the start of the Algorithm 3
                                **end**
                                **end**
                                **else**
                                    Create a external event eX ← INJECTED FAILURE;
                                    Add eX to eG;
                                    Add eX to FT;
                                **end**
fashion. As described in Section 7.2.4, the internal component failure propagation occurs when a single input port failure condition of a component is directly transferred to the output ports of the same component without changing its nature. Although keeping such information in the FT is important, when the model becomes increasingly big, this information can be very exhausting to glance at. Therefore, each path meeting such a condition is omitted and removed from the FT. This process drastically reduces the vertical magnitude of an FT but it does not change its nature.

2. **Removal of external component-to-component failure propagation:** This refers to a condition in which a component-to-component propagation is solicited from a single channel in the FT. For instance, if a single failure condition at the input port of a component is propagated from a single source, then this information is omitted in the analyzed FT. Mainly on the basic events, events involved in a failure transformation, and the top event are kept in the FT.

3. **Removal of basic events redundancy:** A single failure can be initiated from a single source and passes through different propagation paths until it reaches the output port(s). If, in all the propagation paths, no transformation occurs, then, from the output failure conditions, only one path is considered, and from that path, all intermediate propagation representations are removed according to the two previous rules.

During the transformation process, only the paths that satisfy the internal or external failure transformation are kept in the tree. This is to help users to only care about the important information when tracing the origin of the failure. One special case of the propagation that is kept in the tree is when the FT that has to be analyzed contains a single path in which a single basic event propagates up the tree all the way to the top event. Then, in such cases, only the top event and the basic event are kept in the analyzed FT. Finally, each of the omitted intermediate paths as well as each gate resulting from a simple component internal failure transformation is replaced by a feed-forward intermediate gate to enhance the FT readability.

For example, taking the generated FT branch shown in Figure 7.6, the event ① is obtained from a logical "AND" output from 3 subsequent paths, which makes this event a result of a component-to-component external transformation. Starting from event ② ("omission") failure at the input port to the event ① ("commission") event at the output port, indicates internal failure transformation. So in such a case, event ① and its next gate are kept permanently, while event ② is kept temporally for future analysis. Further down to event ③ ("omission") up to event ② ("omission") is component-to-component failure propagation, so event ③ will be permanently removed while event ② will be kept again temporarily for further analysis.

Next, we remain with event ④ ("omission") up to ② ("omission") which is a propagation as well (omission-to-omission). From here, normally event ④ is supposed to be removed; however, as event ④ is a basic event, event ② will be removed instead. Finally, the whole omitted part of the tree will be substituted by a feed-forward intermediate gate to enhance the readability of the FT. The final version of the FT is provided in the right-hand figure, 7.6(b). To sum up, we would say that "the internal failure ④ leading to an "omission" at the output port of a basic component had transformed into event a "commission" failure event ① at some point in the system, in which then combined with the other two failure sources had caused an "omission" at the top level of the system.

FT Quantitative analysis

The quantitative probabilistic analysis is meant to automatically calculate the system-level (top event) failure rate. In the proposed approach, the user is able to assign the failure probability rates of the basic failure events such as internal failure and injected failure. This information can be supplied from the device manufacturer's data sheet as well as the safety experts. In safety engineering, the device failure probability is often considered to be extremely low and often expressed as failures per million ($10^{-6}$), particularly for individual components [231]. The probability calculation follows a
<figure><img src="image_44.png" /><figcaption>Figure 7.6: Qualitative transformation example (a) before, (b) after</figcaption></figure>

widely used formula for conducting a logical output of an "AND" or an "OR" gates in the FT [46, 47]. The output of an "AND" gate means that the output event will only happen when a combination of independent events occurs at the same time. On the other hand, the output of an "OR" gate implies that the output event will occur if any one of the input events occurs.

For each FT to be analyzed, the system failure rate (the top event probability) is calculated following a recursive calculation of the intermediate probabilities to the intermediate events. Based on the probabilities of the basic events, the probability values of their parent event can be calculated from input event probabilities. The probability calculation follows the formula in Figure 7.7. Let N be the number of input events and $P_{in}$ the probability of the input event, the output probability $P_{out}$, for both "AND" and "OR" gate types, is calculated as follows:

$$P_{out} = \begin{cases} \prod_{in=1}^{N} P_{in} & \text{for an AND gate} \\ 1 - \prod_{in=1}^{N} (1 - P_{in}) & \text{for an OR gate} \end{cases}$$

<caption>Figure 7.7: Probability calculation formula</caption>

During the probability calculation process, in the case of an internal component failure transformation condition, the probability corresponding to an input port failure event is forwarded to the output port event. When the qualitative analysis finishes, this in turn implies that the probability of an event at the input of an intermediate gate is forwarded to the immediate output event. In addition to that, in an event resulting from an undeveloped event being fed into an "OR" gate, that branch probability
is first set to zero while in the case of an "AND" gate, the branch probability is set to 1. This in fact
does not affect the probability calculation process as 0 and 1 are neutral values in the addition and
multiplication process respectively.

**Note: the full transformation code on both quantitative and qualitative FT analysis can be found in the appendix B.2**

**7.3 Evaluation process**

In this section, we describe the approach used to evaluate the performance of the proposed ap-
proach in supporting the safety analysis of safety-critical IoT systems with respect to other existing
approaches in the literature. In Section 7.3.1, we present the following evaluation process; in Section
7.3.2, we introduce the case study exploited to support our evaluation; finally, Section 7.3.3 describes
the research questions we targeted to address.

**7.3.1 Evaluation process**

The evaluation procedure we adopted followed six different steps. We first identified a use case that precisely fits in the context of safety-critical IoT systems. Next, we defined research questions that primarily focus on evaluating and demonstrating the effectiveness and potential of the proposed approach. Third, we briefly assessed the key features offered by our system and the supporting tool. This stage allowed us to recognize the most powerful safety analysis tools to be compared to demonstrate our approach's superiority. Following that, we presented the experimental results, emphasizing answering the posed research questions. Because our proposed approach can be employed in areas other than IoT, the evaluation approach will also consider the widely used FTA approach used in system engineering. Finally, we compared our approach with the most relevant state-of-the-art platforms, including but not limited to ISOGRAPH Reliability Workbench [169], FT generation approach based on CHESS-FLA implemented in [3], and the safety analysis approach for IoT presented in [180].

**7.3.2 Motivating example: Patient Monitoring System (PMS)**

Due to the general rapid evolution of electronics and information technology, more powerful bed-
side patient monitors capable of complex bio-signal processing and interpretation are becoming avail-
able, and they are usually equipped with some highly specialized communication interfaces [232].
This goes hand in hand with the huge advances in IoT technologies allowing the integration on such
devices of the capability to connect to the internet, which makes it possible to monitor the health state
of multiple patients remotely. To support our evaluation process, we adopted an "Efficient Patient
Monitoring for Multiple Patients Using WSN" case study [5]. The case study is an advanced system
capable of reliably monitoring the multiple parameters of up to six hospitalized patients simultane-
ously in real time.

The system investigates the potential of employing Wireless Sensor Networks (WSN) to reliably and wirelessly collect multiple parameters such as blood pressure, temperature, electrocardiography (ECG), electroencephalography (EEG), and pulse oximeter (SPO2). These parameters are collected through a set of sensors placed on different parts of the patient's body. For instance, the ECG Sensor is placed on the chest and on the limbs to extract the patient's heart rate data, the EEG sensor is placed on the patient's head to read electrical activity generated by the brain. Furthermore, the Blood Pressure sensor is placed on the arm to detect the level of the pressure in the blood; while the SPO2 sensor is placed on the patient's finger to measure the oxygen saturation of a patient's blood; and, finally, the Temperature sensor is placed to any part of the body to measure the temperature. Figure 7.8 describes the high-level architecture of the system.

As a result, the recorded parameters are wirelessly transmitted to a computer running PMS soft-
ware, which feeds them on the monitor screen in the doctor's office. The software can also wirelessly
<figure><img src="image_45.png" /><figcaption>**Figure 7.8:** Basic architecture of Patient Monitoring System [5]</figcaption></figure>

send alarming messages to the doctor's phone, if they are not present, to respond to the patient's requests. The architecture in great detail is discussed in Section 7.4.2.

### 7.3.3 Research questions

We study the performance of our proposed approach by considering the following research questions:

* **RQ1: How does the presented approach distinguish itself from state-of-the-art techniques?**
We simply did a short review of the existing methodologies in relation to what the proposed approach offers in order to derive its unique contribution.

* **RQ2: Does the system-level modeling infrastructure address all the aspects of modeling a multi-layered IoT system suitable for safety analysis?** Apart from the modeling language, does the modeling approach captures all the required information to facilitate the safety analysis? For instance, are all the provided information taken into consideration when performing the analysis?

* **RQ3: How well do the proposed FLA rules efficiently reflect the system failure behavior leading to the system top failure events?** We look at the efficiency of the derived rules from the possible system failure events in clearly supporting actual logical analysis leading to the expected results.

* **RQ4: Does the proposed FT qualitative and quantitative analysis improve the existing FT analysis techniques?** We will concentrate reliability of the result from the supported automated calculus with regard to the existing approaches. We will also have a look at the final FTs and how they better describe the important system failure paths.
## 7.4 Experimental results

In this section, we report the results of the conducted experiment mainly focusing on answering the research questions formulated in Section 7.3. In Section 7.4.1, provide a brief assessment of the related tools in response to RQ1. In Section 7.4.2, present the PMS system’s model architecture while attempting to answer RQ2. Section 7.4.3 present the system failure behavior modeling approach to answer RQ3. Finally, Section 7.4.4 present the results of the FT analysis aimed at answering RQ4.

### 7.4.1 Short literature review (RQ1)

*RQ1: How does the presented approach distinguish itself from state-of-the-art techniques?*

To sufficiently answer the research question, we needed to first understand the available tools that employ the FTA technique to conduct the safety analysis in the literature. We did not only focus on tools that solely support the IoT domain since we wanted to understand the methodologies used by the existing tools in their FT formalism and analysis process in comparison with our proposed approach. In the end, 14 different platforms were found in the literature with much closer relations to our approach.

#### Search and selection process

The search and selection procedure was divided into four major stages. Since the purpose of the chapter is not to undertake an empirical study, we did not conduct this review using well-known databases, but we only relied on Google Scholar results. We performed an automatic search using the keywords: "Model-based Safety analysis in IoT systems," "Fault Tree analysis in IoT systems," "Fault Tree analysis from SysML models," and "Model-based safety analysis or Fault-Tree analysis." The goal was to offer us a sample of current publications on topics related to such information in the search strings. Because each search returned a large number of results, we only analyzed the first two pages of the results. The second phase was conducted for each search in which we filtered out such results by reading through the title and abstracts. To pass this step, an approach title and abstract must include at least one of the terms "Fault-Tree Analysis" or "Model-Based Safety Analysis." Following that, we skimmed through the selected articles to exclude those that merely present an analysis approach but no supporting tool, resulting in a total of 13 approaches. Finally, we included ISOGRAPH [169] as one of the most extensively used workbenches in the industrial domain for dependability analysis, which lead us to a total of 14. Table 7.2 presents a list of selected primary approaches.

#### Results

To better answer RQ1, we have defined a set of fundamental features we think a safety analysis tool should possess. These features were evaluated on the selected approach in comparison with the proposed infrastructure. The features such as supporting system design modeling, failure behavior modeling, automated FT generation, performing qualitative FT analysis, and performing quantitative FT analysis. Table 7.3 summarizes the findings of the study, in which for each approach, a "Yes" or "No" label was used to indicate if that approach supports that particular feature. As indicated, all 14 approaches are represented against the 5 features. It can be seen that at least one feature is supported at least once. In this section, we go over the result of the study and have some discussion in line with the features defined above.

* *Support for system modeling:* This feature assesses whether the suggested tool supports system design before proceeding with its safety analysis. Conducting safety analysis needs to go hand in hand firstly with the design of the system under analysis. We believe that integrating the design and analysis infrastructure can improve transparency and consistency among system and safety experts. According to Figure 7.9, 78.6% of the selected approaches has the infrastructure for designing the system before starting the analysis. Table 7.3, on the other hand, indicates that
<table><caption>**Table 7.2: Selected approaches**</caption><thead><tr><td>Tool</td><td>Title</td><td>Year</td><td>Type</td></tr></thead><tbody><tr><td>ISOGRAPH [169]</td><td>Fault tree analysis in reliability workbench</td><td>1980s</td><td>Community tool</td></tr><tr><td>Haider et al. [3]</td><td>FLA2FT: Automatic generation of fault tree from Con-certoFLA results</td><td>2018</td><td>Conference</td></tr><tr><td>JARVIS [170]</td><td>A framework for model-driven engineering of resilient software-controlled systems</td><td>2021</td><td>Journal</td></tr><tr><td>Mehnni et al. [171]</td><td>Automatic fault tree generation from SysML system models</td><td>2014</td><td>Conference</td></tr><tr><td>Alshboul et al. [4]</td><td>Automatic derivation of fault tree models from SysML models for safety analysis</td><td>2018</td><td>Journal</td></tr><tr><td>Yakymets et al. [172]</td><td>Model-based system engineering for fault tree generation and analysis</td><td>2013</td><td>Conference</td></tr><tr><td>Hamed et al. [174]</td><td>Fault tree analysis for reliability evaluation of an advanced complex manufacturing system</td><td>2018</td><td>Journal</td></tr><tr><td>Silva et al. [180]</td><td>A dependability evaluation tool for the internet of things</td><td>2018</td><td>Journal</td></tr><tr><td>Chen et al. [181]</td><td>Application of fault tree analysis and fuzzy neural networks to fault diagnosis in the Internet of Things (IoT) for aquaculture</td><td>2017</td><td>Journal</td></tr><tr><td>Xing et al. [182]</td><td>Reliability modeling of mesh storage area networks for Internet of Things</td><td>2017</td><td>Journal</td></tr><tr><td>MetaFPA [178]</td><td>Transformation of failure propagation models into fault trees for safety evaluation purposes</td><td>2016</td><td>Conference</td></tr><tr><td>Clegg et al. [175]</td><td>Integrating existing safety analyses into SysML</td><td>2019</td><td>Conference</td></tr><tr><td>smartIflow [233]</td><td>Model based safety analysis with smartIflow</td><td>2017</td><td>Journal</td></tr><tr><td>Xiang et al. [177]</td><td>Automatic static fault tree analysis from system models</td><td>2010</td><td>Conference</td></tr></tbody></table>

approaches such as [175, 178, 181] do not provide such a feature and instead rely on manually created FT models, which are then transformed into FT graphs. In contrast to the previous approach, our proposed approach completely supports this feature by providing an environment in which system main blocks and sub-systems can be decomposed and analyzed separately. Even though approaches such as [4, 171, 175, 176, 234] extend the SysML language in the same way that we do, our environment is more user-friendly due to the advanced component-based and multi-view modeling infrastructure, where each view has its own underlined constraints that enforce its specific privileges on model entities and properties that can be manipulated [42].

* *Support for failure behavior modeling:* This feature determines whether the proposed approach provides mechanisms for explicitly stating the system failure modes as well as the system failure behavior, both of which contribute to the generation of the FT. As shown in Figure 7.9, 85.7% of the investigated approaches support this feature, which is a good number given that it is the key driver for FT generation. In our approach, we extend FPTC [33] rules when modeling the system failure behavior because we consider it to be straightforward to understand. Other approaches use formalism including, but not limited to, "IF-THEN" or logical math association expressions to formalize that ([171, 178, 180, 181]). Nonetheless, all these approaches lack the concepts of external failure injection as well as internal failure transformation and propagation. On the other hand, [172] uses a formal method approach in modeling system failure logic; however, the complexity of formal method formalism can be difficult to handle. Furthermore, [4] depends on annotating failure information in the model state machines, which can be a difficult and time-consuming task due to the complexity of state machine definition.

* *Perform automated FT generation:* This feature determines whether the proposed approach automatically generates the FT from the model rather than manually constructing it. This is one of the main motivations for our proposed approach since we believe that automating the FT generation process is critical to reducing the time safety engineers spend in performing the safety analysis as well as increasing transparency in the process. According to the findings, around 64.3% of the techniques support such a feature. One commonly used tool in the industry