# Evaluating the Perceptual Properties of Crosscutting Concerns Occurrence Points Specifications in Embedded Software

Marco A. Wehrmeister, Rodrigo Oliveira, Douglas B. Renaux

<figure><img src="image_1.png" /></figure>

## To cite this version:

Marco A. Wehrmeister, Rodrigo Oliveira, Douglas B. Renaux. Evaluating the Perceptual Properties of Crosscutting Concerns Occurrence Points Specifications in Embedded Software. 7th International Embedded Systems Symposium (IESS), Nov 2022, Lippstadt, Germany. pp.3-15, 10.1007/978-3-031-34214-1_1 . hal-04645300

HAL Id: hal-04645300

https://inria.hal.science/hal-04645300v1

Submitted on 11 Jul 2024

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers.

L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

<figure><img src="image_2.png" /></figure>

Distributed under a Creative Commons Attribution 4.0 International License
<figure><img src="image_3.png" /></figure>

ifip

International Federation for Information Processing

This document is the original author manuscript of a paper submitted to an IFIP
conference proceedings or other IFIP publication by Springer Nature. As such, there
may be some differences in the official published version of the paper. Such
differences, if any, are usually due to reformatting during preparation for publication or
minor corrections made by the author(s) during final proofreading of the publication
manuscript.
# Evaluating the Perceptual Properties of Crosscutting Concerns Occurrence Points Specifications in Embedded Software

Marco A. Wehrmeister<fnref n="1" />,<fnref n="2" />, Rodrigo S. C. Oliveira<fnref n="1" />, and Douglas B. Renaux<fnref n="1" />

**Abstract.** Crosscutting concerns associated with the non-functional requirements and system constraints contribute to the growing complexity of embedded software. Techniques, such as *Aspect-Oriented Software Development*, propose units of modularization to encapsulate their handling. A key design issue in embedded software is related to the process of locating, identifying, and specifying the *Crosscutting Concerns Occurrence Points* (CCOP). This paper reports on an empirical evaluation that compares four notations, namely JPDD, Theme/UML, AspectJ, and Tracematch to specify the selection of CCOPs. A set of metrics to quantify the perceptual properties of these specifications was proposed based on the *Physics of Notation* (PoN) conceptual framework. Three embedded automation systems have been specified using the mentioned notations and evaluated according to the proposed metrics set. Experimental results show how the used notation impacts the comprehension of the specification, whether in graphical or textual form. The results show poor discriminability in the cognitive effectiveness of visual representations. Hence, using visual notations to specify CCOP selection seems not to facilitate the understanding as one would expect.

**Keywords:** crosscutting concerns · embedded software · notation for visual and graphical specification · quality model · metrics

## 1 Introduction

In traditional paradigms, including Object-Oriented and Procedural, some of the requirements and system constraints are necessarily scattered across multiple units of modularization in both the design and implementation. This scenario with crosscutting concerns leads to tangled and scattered handling that impact negatively on coupling, cohesion, and the single-responsibility principle. Crosscutting Concerns Occurrence Points (CCOP) indicate the parts of the software where the handling code must be inserted. The various programming languages and/or visual notations present diverse linguistic means to identify such CCOP selections. Common ways to specify CCOP selections are based on lexical properties and/or system behavioral or structural context [19, 20].

It is not trivial to understand “when” and “how” the crosscutting concerns are handled and where they occur in the implementation of the system’s functional requirements. This is the case of embedded real-time systems that present

<footnote n="1">¹ Universidade Tecnológica Federal do Paraná (UTFPR), Curitiba, 80230-901, Brazil
{wehrmeister, douglasrenaux}@utfpr.edu.br, rodrigoo@alunos.utfpr.edu.br</footnote>

<footnote n="2">² University of Münster, Münster, 48149, Germany
wehrmeister@uni-muenster.de</footnote>
distinct sorts of crosscutting concerns, non-functional requirements, and con-
straints [8], such as timing, concurrency, resources monitoring, energy consump-
tion, dependability, and others.

Two approaches are observed in the current state-of-the-art of the specifica-
tion of system crosscutting concerns: graphical and textual forms. Both forms
have been explored particularly in the context of *Aspect-Oriented Software De-
velopment* (AOSD)[1, 7, 10, 15]. One technique to identify and specify the selec-
tion of CCOPs through a graphical form is the *Join Point Designation Diagrams*
(JPDDs) [19], which is based on UML [17]. Similarly, Theme/UML [6] is also
based on UML but it supports symmetric separation of concerns rather than
asymmetric separation that is supported by most AOSD approaches. Engineers
specify *bindings* among *themes* to indicate where the implementation of cross-
cutting concerns must be inserted in the software execution flow.

However, although UML is widely accepted in the industry, there is no pre-
cise semantics when it comes to the specification of crosscutting concerns since
UML was created to support the Object-Oriented paradigm, which is known for
presenting modularity issues of the crosscutting concerns handling [12]. Conse-
quently, although there are approaches to extend UML semantics towards AOSD
[22, 21], the result is a quite imprecise specification of the CCOP selections, due
to poor discriminability [16], increasing the difficulty in the comprehension and
maintenance of the artifacts generated.

This paper reports an empirical evaluation that compares four AOSD no-
tations, namely JPDD, Theme/UML, AspectJ, and Tracematch, when speci-
fying implicitly or explicitly the CCOP selections. The goal is to assess how
the mentioned techniques facilitate the comprehension of the CCOP selection
specification, whether in graphical or textual form. A quality model, based on
the “*Physics of Notation*” (PoN) conceptual framework [16], was proposed; it
includes nine metrics that quantify the specification of fifteen CCOP commonly
found in embedded software. These CCOPs are related to timing, distributed
processing, performance, resource usage monitoring, and control [8, 13].

The remainder of this paper is organized as follows. Section 2 overviews briefly the PoN conceptual framework. Section 3 examines previous work and compares their characteristics. Section 4 presents the proposed quality model and its framework to evaluate the specification of CCOP selection. Section 5 presents the empirical evaluation and analyses its results, whereas Section 6 discusses the possible threats to the validity of this evaluation. Finally, Section 7 draws conclusions and provides an outlook on future works.

## 2 Overview of The Physics of Notations Framework

The conceptual framework *Physics of Notations* (PoN) [16] aims to provide a scientific basis for a design rationale of visual notations in software engineering. To overcome what PoN’s authors identified as issues of visual representations being historically ignored and undervalued as well as the lack of discussion on visual syntax, PoN addresses the physical and perceptual properties of notations rather than their logical properties. Therefore, PoN comprises nine design principles that can be summarized as follows:
- **Semiotic Clarity:** there should be a 1:1 correspondence between semantic constructs and graphical symbols;

- **Perceptual Discriminability:** different symbols should be clearly distinguishable from each other;

- **Semantic Transparency:** the appearance of visual representations must suggest their meaning;

- **Complexity Management:** the number of visual elements must be restricted to avoid cognitive overload;

- **Cognitive Integration:** the language must provide mechanisms to explicitly integrate information from multiple diagrams;

- **Visual Expressiveness:** the use of the full range and capacities of visual variables;

- **Dual coding:** a notation should combine text and graphics to improve information conveyance and its understanding;

- **Graphic Economy:** the number of different graphical symbols should be cognitively manageable;

- **Cognitive Fit:** a notation should provide different means to represent the information to better suit different tasks and audiences.

## 3 Related Work

Dealing with crosscutting concerns due to non-functional requirements is a challenge that is still being addressed [20]. Embedded systems present an increasing number of crosscutting concerns that impact several characteristics of the embedded software artifacts and also of the system as a whole [5, 21, 13]. The Providentia [5] deals with the non-functional requirements and their crosscutting concerns in the design of self-adapting systems that experience various sources of uncertainty. Hannousse [9] proposes a component-based software engineering approach to support explicitly the crosscutting and dynamic features. These are identified early in the process and modeled separately using AOSD concepts to improve components’ reusability and maintainability.

The work in [18] addresses the safety crosscutting concerns within the design process of distributed vehicular control systems, aiming for fault-tolerant and fault-diagnostic mechanisms to improve reliability in communication protocols. The RT-FRIDA framework allows for modeling faults in early-design phases such as requirements engineering. The results show improved fault modeling by handling the associated crosscutting concerns. Similarly, Zahng et al. [24] propose a knowledge-driven method for requirements elicitation in cyber-physical systems design. The aim is to identify the implicit non-functional requirements and crosscutting concerns, which are later modeled using SysML requirement diagram. Additionally, in [3], security and safety crosscutting concerns of safety-critical embedded systems are addressed using design patterns and templates. COOP are specified within the design pattern. Once the design patterns are applied, the crosscutting concern handling elements are intermixed with the system’s functional requirements implementation.
Furthermore, there are several distinct proposed approaches to specify CCOP selection, particularly in the context of AOSD. *AspectJ* [12] is a textual programming language that extends Java to define a dynamic crosscutting mechanism in the execution of a program.

*Join Point Designation Diagram* [19] is a modeling approach to visualize join point selections based on the UML semantics. Diagrams of the system events, object symbols, and object flow edges through the adoption of action symbols represent the flow to reach a CCOP. Although JPDD allows specifying CCOP selections uniformly and independently from the programming language, the visual notation is slightly different from the standard, requiring learning effort.

*Theme/UML* [6, 4] is a UML-based approach that supports symmetric separation of concerns. A concern is represented as fragments of behavior and structure encapsulated into a single theme. The themes are bound to each other to indicate which elements are affected by elements of other themes. Such bindings are specified manually by the designers. This approach is similar to a join point specification and the relation between functional elements that are affected by aspects. However, it lacks scalability and hinders model comprehension, maintainability, and evolution in large systems.

*Tracematch* [2] is a textual approach that implements regular expressions over the program's execution traces based on a symbols alphabet. Once the regular expression matches an execution trace, an event triggers the execution of a modular unit of crosscutting implementation. Tracematch is a seamless extension to AspectJ. However, although Tracematch innovates in its treatment of free variables in trace patterns, its usage is nontrivial.

Literature review on crosscutting concerns [14] gathers existing quality measures that validate the effectiveness of different techniques. The authors conclude that there is a critical lack of standard benchmarks for the specification of join point selections, hindering the empirical evaluation of the methods and the objective comparison and validation of the research. On the other hand, [25] proposes a quantitative method to evaluate the non-functional requirements and associated crosscutting concerns. The authors report that their approach shortens the time for non-functional requirements evaluation.

A major issue in the understanding of a CCOP selection specification is the lack of knowledge on the underlying paradigm used to encapsulate the crosscutting concerns handling, e.g., Aspect Oriented Paradigm (AOP) [11] from which the concept of join points was derived. The relationship between modeling level and execution level is less clear than in the case of programming languages. Table 1 compares AspectJ, JPDD, Theme/UML, and Tracematch according to the characteristics presented in the existing AOSD literature [23, 20].

To the best of our knowledge, this research is the first that evaluates distinct graphical and textual notations for CCOP selection specification in the light of the PoN conceptual framework. No other empirical studies analyze the perceptual properties of notations related to CCOP (e.g., join points) using quantitative metrics. This is particularly relevant in embedded software since embedded sys-
<table><caption>**Table 1.** Comparison of AOSD techniques. “X” means supported</caption><thead><tr><th rowspan="2"></th><th colspan="2">Structural CCOP</th><th colspan="2">Behavioral CCOP</th><th colspan="2">Simple Pointcut</th><th colspan="2">Composite Pointcut</th><th colspan="3">Quantification Method</th><th colspan="3">Position Indication</th><th colspan="2">Abstraction</th></tr><tr><th>Static</th><th>Dynamic</th><th>Static</th><th>Dynamic</th><th>Graphical</th><th>Textual</th><th>Graphical</th><th>Textual</th><th>Declarative</th><th>Imperative</th><th>Enumeration</th><th>Before</th><th>Around</th><th>After</th><th>High</th><th>Low</th></tr></thead><tbody><tr><td>AspectJ</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>JPDD</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>Theme/UML</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr><tr><td>TraceMatch</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td><td>X</td></tr></tbody></table>

tems do present an intrinsic number of non-functional requirements that lead to concerns that crosscut the system behavior and provided services [8].

## 4 Evaluation Framework

The software engineering literature provides several metrics and evaluation frameworks to extract quantitative information from software artifacts. However, to the best of our knowledge, there is no evaluation framework or quality model created from the perspective of the PoN framework. Therefore, we propose the *Metrics Suite for PoN conceptual framework* (MS4PoN). Its goal is to quantify the perceptual properties of graphical and textual notations to specify CCOP selections and to assess the cognitive effectiveness of specifications. Figure 1 presents the proposed quality model; five principles of the PoN framework are associated with nine metrics. The MS4PoN measures features of graphical and textual notations for specifying CCOP selections in terms of elements, symbols, and syntax:

<figure><img src="image_4.png" /><figcaption>**Fig. 1.** Cognitive effectiveness quality model</figcaption></figure>

- **Number of elements (NE):** this metric counts the number of elements contained in the CCOP selection model or the implementation regardless of the specification is in a textual or graphical form. The greater is the NE, the more complex the model or implementation will be.

- **Number of Logical Expressions (NLE):** this metric is defined as the number of agnostic (or abstract) logical expressions specified in the CCOP
selection model or implementation. Logical expressions indicate selection criteria. In the experiments, the NLE values are equal for each CCOP despite the notation used. The objective here is to have an abstract expression representing the selection criterion that shall be specified in the model or the implementation. NLE allows for generating some ratios to be analyzed to provide more precise and homogeneous results.

- **Number of Visual Distance (NVD):** this metric quantifies the number of different syntactic elements specified in the CCOP selection model. In general, the greater the number of the different elements specified in the model or implementation the greater is the NVD, and consequently the faster and more accurately the specification is recognized and understood.

- **Number of Semantically Immediate Elements (NSIE):** this metric determines the total number of semantically immediate elements specified in the CCOP selection model or implementation. An element is semantically immediate if it is easy to infer its meaning from its appearance or mnemonic. The greater the NSIE, the easier it is to understand the model or implementation. A semantically immediate element or symbol must have in its nature an association with: (i) *functional similarities*, e.g., the element method “before()” in AspectJ means to do something before; or (ii) *metaphors*, e.g., a stick figure may represent a person in the UML’s use case diagrams; or (iii) any *cultural associations*, e.g., a big “X” in the UML’s sequence diagram means a closed communication.

- **Number of Semantically Opaque Elements (NSOE):** the number of semantically opaque elements specified in the CCOP selection model or implementation. An element is semantically opaque or conventional if there is a purely arbitrary relationship between its appearance and its meaning. The greater is the NSOE, the more neutral or conventional elements are being used in the model or implementation, e.g., the excessive use of rectangles in Theme/UML diagrams. This metric represents the zero point in the scale between NSIE and NSPE. It does not augment the perception, but also does not suggest a different meaning from its appearance, and hence, it is a conventional element that must be learned.

- **Number of Semantically Perverse Elements (NSPE):** this metric quantifies the number of semantically perverse elements specified in the CCOP selection model or implementation. An element is semantically perverse or false mnemonic if the opposite meaning is inferred from its appearance. The greater the NSPE, the worst it is to understand the model or implementation. A semantically perverse element or symbol has in its nature lexical properties or false mnemonics that can suggest to a novice reader to infer a different meaning from its appearance. Some UML conventions present this undesirable property, e.g., the UML’s package merge.

- **Level of Expressiveness (LVE):** this metric determines the number of visual variables used in a notation for the CCOP selection model or implementation. The LVE counts the number of communication variables used in visual languages, e.g., shape, texture, color, etc. According to [16], LVE values range from zero to eight. Hence, the greater the LVE, the more efficient
the model is using the advantage of its being graphical. Conversely, the lower
the LVE, the closer the graphical notation is from of the textual one.

- **Number of Synographs (NS):** this metric is defined as the total number of synographs specified in the CCOP selection model or implementation. A synograph element occurs when there is a redundancy in the use of symbols or elements to represent the same semantic construct, e.g., interfaces in UML diagrams. The greater the NS, the more difficult it is to understand the model or implementation. A notation satisfies the requirements of a notational system if it provides a one-to-one correspondence between symbols and their associated concepts [16]. The synograph element is an anomaly that occurs when there is not such one-to-one correspondence thus leading to redundancy of symbols and reduces the understanding of the specification.

- **Number of Homographs (NH):** determines the total number of homographs specified in the CCOP selection model. A homograph occurs when there is an overload in the semantic constructs that can be represented by the same symbols or elements. The greater the NH, the more difficult it is to understand the model. This is the worst type of anomaly as it leads to ambiguity and the potential for misinterpretation [16]. Theme/UML presents such anomaly as the same graphical convention (e.g., rectangle) is used to represent several distinct elements in the specification of CCOP selections: subject, class, behavior symbol in life lines, and aspectual representations.

These metrics are combined to allow evaluating the five principles defined in the PoN framework. *Semiotic Clarity* principle is defined by *Symbol Redundancy* and *Symbol Overload*. The former is impacted by the number of synographs (NS) (visual equivalent of synonyms) found in the specification of CCOP selections, while the latter by the number of homographs (NH) (visual equivalent of homonyms). *Perceptual Discriminability* principle is defined by *Visual Distance* which is directly affected by the Number of Visual Distance (NVD) elements. *Semiotic Transparency* principle is influenced by the number of elements considered *Semantically Immediate* (NSIE), or *Semantically Opaque* (NSOE), and *Semantically Perverse* (NSPE). Ideally, a notation should present the greatest possible number of semantically immediate elements and zero semantically perverse elements. The *Visual Expressiveness* principle is influenced by *Free Variables*, which are directly affected by the number of visual communication channels (Level of Expressiveness - LVE) used to specify CCOP selections. A notation with no visual variables is called *non-visual* (or textual), whereas a notation that uses all visual variables is considered *visually saturated*. The *Graphic Economy* principle is defined by *Hybrid Symbols*, which are directly affected by the number of elements (NE) and number of logical expressions (NLE). A notation with a great number of elements increases its complexity according to how many elements are used per logical expression.

The other PoN principles are not relevant to this work's goals and scope since it addresses the evaluation of CCOP selection specification. *Complexity management* principle cannot be applicable here because we analyze only the part of the notation that deals with CCOP selection; not the whole language. *Cognitive*
*integration* is not relevant to our evaluation because it refers to languages that use multiple diagrams to specify the system requirements. This is not the case for CCOP selection specification since it is often described in a single artifact/document. Finally, regarding *cognitive fit* and *dual coding* principles, we understand that such characteristics are uncountable, i.e., either a given notation presents these characteristics or not.

# 5 Empirical Experiments and Results

MS4PoN has been employed to evaluate two graphical (JPDD and Theme/UML) and two textual (AspectJ and Tracematch) notations to specify CCOP selection. The goal is to evaluate the cognitive effectiveness and the perceptual properties of these notations by employing the quality model and calculating the MS4PoN metrics described in the previous section.

The data source for these experiments includes fifteen CCOP commonly found in the software for embedded real-time cyber-physical systems [8, 21, 22, 13]. In summary, these CCOP appear in various pieces of the embedded software: (a) the structure definition, the instantiation, and the behavior of active objects; (b) the structure definition, and the exclusive access to passive objects’ data and services; (c) read/write access to objects’ attributes; and (d) subsystems initialization. These CCOP have been (re)used in three case studies: (i) the movement control system of an unmanned aerial vehicle; (ii) the control system of an industrial packing system; and, (iii) the movement control system of an automated wheelchair. The experiments consist of creating four specifications (JPDD, Theme/UML, AspectJ, and Tracematch) for each CCOP. A total of 60 distinct specifications have been created. MS4PoN metrics have been calculated for each distinct specification. Although these notations allow for a complete AOSD specification, the experiments and its results consider only the parts of the notations related to the specification of CCOP selections. Experimental data and results are available in https://tinyurl.com/v4b5y42s.

Table 2 shows MS4PoN metrics calculated on the 60 specifications of the considered COOP. These values represent the sum of individual values of each metric for each specification of CCOP selection. The exception is LVE value which is calculated as an average value. LVE represents the visual expressiveness of the notation according to PoN framework [16], whose value should range from zero to eight. Summing LVE of the distinct specifications does not represent properly the visual expressiveness concept because it would indicate often a saturated notation (i.e. a value greater than 8) as the number of considered specifications increases.

To better analyze and compare the results, we created some ratios between metrics to quantify the cognitive effectiveness of a notation under the PoN perspective. Such ratios (see Table 3) intend to identify how the perceptual properties are distributed over the elements described within the specifications.

NVD/NE ratio indicates how many different syntactic elements appear per element in the notation. The textual notations (AspectJ and Tracematch) have a higher value than visual notations (JPDD and Theme/UML). This means that AspectJ and Tracematch employ more different syntactic elements per element.
<table><caption>**Table 2.** Metrics for 60 specifications of CCOP selection</caption><thead><tr><th>Notation</th><th>NE</th><th>NLE</th><th>NVD</th><th>NSIE</th><th>NSOE</th><th>NSPE</th><th>LVE</th><th>NS</th><th>NH</th></tr></thead><tbody><tr><td>AspectJ</td><td>201</td><td>72</td><td>201</td><td>28</td><td>201</td><td>0</td><td>0.00</td><td>0</td><td>0</td></tr><tr><td>JPDD</td><td>87</td><td>72</td><td>55</td><td>0</td><td>55</td><td>22</td><td>1.33</td><td>0</td><td>16</td></tr><tr><td>Theme/UML</td><td>360</td><td>72</td><td>45</td><td>0</td><td>90</td><td>15</td><td>2.00</td><td>15</td><td>150</td></tr><tr><td>Tracematch</td><td>141</td><td>73</td><td>141</td><td>53</td><td>141</td><td>0</td><td>0.00</td><td>0</td><td>3</td></tr></tbody></table>

<table><caption>**Table 3.** Ratio between MS4PoN metrics. Bold text indicates the better values.</caption><thead><tr><th>Ratios</th><th>AspectJ</th><th>JPDD</th><th>Theme/UML</th><th>Tracematch</th></tr></thead><tbody><tr><td>NE/NLE</td><td>2.792</td><td><strong>1.208</strong></td><td>5.000</td><td>1.932</td></tr><tr><td>NVD/NE</td><td><strong>1.000</strong></td><td>0.632</td><td>0.125</td><td><strong>1.000</strong></td></tr><tr><td>NVD/NLE</td><td>2.792</td><td>0.764</td><td>0.625</td><td><strong>1.932</strong></td></tr><tr><td>NSIE/NE</td><td>0.139</td><td>0.000</td><td>0.000</td><td><strong>0.376</strong></td></tr><tr><td>NSIE/NLE</td><td>0.389</td><td>0.000</td><td>0.000</td><td><strong>0.726</strong></td></tr><tr><td>NSOE/NE</td><td><strong>1.000</strong></td><td>0.632</td><td>0.250</td><td><strong>1.000</strong></td></tr><tr><td>NSOE/NLE</td><td><strong>2.792</strong></td><td>0.764</td><td>1.250</td><td><strong>1.932</strong></td></tr><tr><td>NSPE/NE</td><td><strong>0.000</strong></td><td>0.253</td><td>0.042</td><td><strong>0.000</strong></td></tr><tr><td>NSPE/NLE</td><td><strong>0.000</strong></td><td>0.306</td><td>0.208</td><td><strong>0.000</strong></td></tr><tr><td>LVE</td><td>0.000</td><td>1.333</td><td><strong>2.000</strong></td><td>0.000</td></tr><tr><td>NS/NE</td><td><strong>0.000</strong></td><td><strong>0.000</strong></td><td>0.042</td><td><strong>0.000</strong></td></tr><tr><td>NS/NLE</td><td><strong>0.000</strong></td><td><strong>0.000</strong></td><td>0.208</td><td><strong>0.000</strong></td></tr><tr><td>NH/NE</td><td><strong>0.000</strong></td><td>0.184</td><td>0.417</td><td>0.021</td></tr><tr><td>NH/NLE</td><td><strong>0.000</strong></td><td>0.222</td><td>2.083</td><td>0.041</td></tr></tbody></table>

The same reasoning is valid for the NVD/NLE ratio that considers the number of different syntactic elements per logical expression. The higher the visual distance per logical expression is, the better it is to understand the specification. However, it is important to highlight that a good value for these ratios should be close to one, i.e., the notation has only one distinct element per logical expression. Values between zero and one mean that there are implicit elements within the specification, and values greater than one mean that the notation is verbose. Both situations somehow hinder the understanding of the specification.

NSIE-related ratios indicate how many semantically immediate elements are used per element or per logical expression in the specification. These ratios identify how easily one can infer the correct meaning of the elements described in the specification by their appearance. For instance, the NSIE/NE ratio for Tracematch and AspectJ are a bit higher than the others, i.e., they have more elements (in the specification) that present an implicit understanding of their semantics compared to JPDD and Theme/UML. On the other hand, NSPE-related ratio indicate a higher number of perverse elements in JPDD and Theme/UML. This might be due to the fact that JPDD and Theme/UML are extensions of UML. According to [16], the majority of UML notations were not designed considering the perceptual and cognitive properties that graphical notation must have.

By analyzing the ratios presented in Table 2, Tracematch and AspectJ seem to be more effective regarding the understanding and comprehension than JPDD and Theme/UML to specify CCOP selections. These empirical quantitative results might indicate that the specification of CCOP selections through a graphical notation has poor discriminability on the cognitive effectiveness of visual rep-
representations. Under the PoN framework perspective, it seems that using graph-
ical notations makes the COOP selection specification difficult to understand,
especially when the engineers are not skilled enough on the visual notation.

6 Limitations and Threats to Validity

Although the obtained results provide adequate support for this work's conclu-
sions, some factors may have influenced the conducted evaluation, as is usual in
any empirical study. Threats to the internal validity of the evaluation may arise
from the fact that the cognitive effectiveness results may be influenced by the
authors' experience. Such an issue presents a low risk to affect negatively the re-
sults obtained since (a) the proposed quality model and the experimental results
are based on the perspective of the consolidated PoN framework rather than
on the authors' own experience, and (b) the CCOP used in the experiment are
from real-world case studies carried out in previous works. However, it is worth
mentioning that NSIE, NSPE, and NSOP would be seen as subjective metrics.
Thus, they may be influenced by the authors' experience. Although there may
be differences if other engineers evaluate the same specifications, these metrics
values might not differ substantially from those presented here since the metrics
evaluate the notations' syntax rather than semantics The values of these metrics
are also reasonable fair with respect to the notation features they measure, but
they must be analyzed carefully. As they are 3 out of 9 MS4PoN metrics, the
authors understand that the obtained results are valid regarding the assessment
of cognitive effectiveness of the evaluated notations to specify COOP selection.

Another threat to validity may concern the generality of the results. The
proposed quality model and MS4PoN metrics might be of general application.
Thus, they may be used to assess other visual and textual notations. Although
the results discussed here are limited to the three studies cases from the embed-
ded software domain, the variety of CCOP types used in the experiments allows
us to believe that similar results may be obtained in other application domains
when it comes to specifying CCOP selection. The authors consider to be low the
probability of this risk influencing this work's findings as the experiments' scope
was limited to embedded software systems. Although the planned experiments
have been carefully conducted to achieve this work's goals, other studies and
experiments involving other notations and contexts are still necessary.

7 Conclusions and Future Work

This work evaluates the perceptual properties of graphical and textual notations
of different techniques (AspectJ, JPDD, Theme/UML, and Tracematch) towards
the CCOP selection specification. For that, a quality model and a metrics suit
(MS4PoN) have been proposed based on the PoN conceptual framework [16].

Experimental results showed empirical evidence that the graphical specifica-
tion of CCOP selections has poor discriminability on the cognitive effectiveness
of visual representations. The graphical notations evaluated in the experiments
present a great number of homographs elements to specify CCOP selections. The
homographs are anomalies that lead to ambiguity and misinterpretations [16].
The visual distance of the evaluated graphical notation falls short of what was expected. Results show a maximum of 0.764 different graphic elements per logical expression. This means that most of the visual elements are equal or similar, affecting the understanding and cognition of the engineers. On the other hand, one of the evaluated textual notations (AspectJ) shows an interesting ratio of 2.792, meaning that for each logical expression there are 2.792 textual syntactic elements that are different among themselves. In addition, Tracematch (textual notation) presented the highest NSIE among the techniques compared in this work. This is quite impressive as the textual notations present more immediate elements (i.e., syntactical elements associated to a mnemonic) than visual notations. It means that textual notations are more comprehensive than visual notations for specifying CCOP selections in spite of Tracematch presented a low NH that impacts negatively the cognition but, in comparison with the visual notations, the difference is not meaningful.

Another result that drew our attention is the many semantically perverse elements (NSPE) of the evaluated graphical notations. In JPDD, and particularly in Theme/UML, the specification of CCOP selections includes elements or symbols whose correct semantic has a false mnemonic. Thus, a novice engineer may infer a different meaning from its appearance. The level of expressiveness (LVE) of the evaluated graphical notations is very low, close to a textual notation, since they present few variables of visual communication (e.g., shape, color, texture, etc.). This is below the expectations since JPDD and Theme/UML seem not to benefit from the expected advantages of a visual language. According to [16], this is similar to the majority of the visual notations in software engineering.

As future work, we plan to evaluate other graphical and textual notations, not only for specifying more complex CCOP selections but also other features. It is also important to perform experiments with novice and experienced designers to further assess the proposed quality model and MS4PON metrics. Also, given that the graphical notations seem to fail to improve the understanding of CCOP selection, we are developing a new technique to specify CCOP selections with a mix of textual and graphical elements (i.e. dual coding [16]), aiming to increase the cognitive effectiveness of its perceptual properties.

## References

1. Acreţoaie, V., Störrle, H., Strüber, D.: VMTL: a language for end-user model transformation. Software & Systems Modeling **17**(4), 1139–1167 (2018)

2. Allan, C., et al.: Adding trace matching with free variables to aspectj. In: Proc. of SIGPLAN Conf. on Object-Oriented Programming, Systems, Languages, and Applications. p. 345–364. ACM, New York, NY, USA (2005)

3. Armoush, A.: Towards the integration of security and safety patterns in the design of safety-critical embedded systems. In: 2022 4th International Conference on Applied Automation and Industrial Diagnostics (ICAAID). vol. 1, pp. 1–6 (2022)

4. Baniassad, E., Clarke, S.: Theme: An approach for aspect-oriented analysis and design. In: 26th Int. Conf. on Software Engineering. pp. 158–167. IEEE (2004)

5. Bowers, K.M., et al.: Providentia: Using search-based heuristics to optimize satisficement and competing concerns between functional and non-functional objectives in self-adaptive systems. Journal of Systems and Software **162**, 110497 (2020)
6. Clarke, S.: Extending standard uml with model composition semantics. Science of Computer Programming **44**(1), 71–100 (2002)

7. Filman, R., Elrad, T., Clarke, S., Akşit, M. (eds.): Aspect-Oriented Software Development. Addison-Wesley Professional, first edn. (2004)

8. Freitas, E.P., et al.: DERAF: A high-level aspects framework for distributed embedded real-time systems design. In: Early Aspects: Current Challenges and Future Directions. pp. 55–74. Springer, Berlin, Heidelberg (2007). https://doi.org/10.1007/978-3-540-76811-1_4

9. Hannousse, A.: A development process for component software with crosscutting and dynamic features. In: 2019 International Conference on Theoretical and Applicative Aspects of Computer Science (ICTAACS). vol. 1, pp. 1–8 (Dec 2019)

10. Khan, M.U., Sartaj, H., Iqbal, M.Z., Usman, M., Arshad, N.: Aspectocl: using aspects to ease maintenance of evolving constraint specification. Empirical Software Engineering **24**(4), 2674–2724 (2019)

11. Kiczales, G., et al.: Aspect-oriented programming. In: Proc. of European Conf. on Object-Oriented Programming (ECOOP). Springer-Verlag. LNCS 1241 (1997)

12. Kiczales, G., et al.: An overview of AspectJ. In: European Conference on Object-Oriented Programming. pp. 327–354. Springer (2001)

13. Leite, M., Wehrmeister, M.A.: System-level design based on uml/marte for fpga-based embedded real-time systems. Design Automation for Embedded Systems **20**(2), 127 – 153 (2016), doi: 10.1007/s10617-016-9172-6

14. McFadden, R.R., Mitropoulos, F.J.: Survey of aspect mining case study software and benchmarks. In: 2013 Proceedings of IEEE Southeastcon. pp. 1–5 (April 2013)

15. Mongiovì, M., Pappalardo, G., Tramontana, E.: Specifying and identifying widely used crosscutting concerns. Knowledge-Based Systems **126**, 20 – 32 (2017)

16. Moody, D.: The “physics” of notations: Toward a scientific basis for constructing visual notations in software engineering **35**(6), 756–779 (2009)

17. Object Management Group (OMG): Unified Modeling Language version 2.5.1 (2017), https://www.omg.org/spec/UML/2.5.1/

18. dos Santos Roque, A., et al.: An approach to address safety as non-functional requirements in distributed vehicular control systems. Journal of Control, Automation and Electrical Systems **30**(5), 700–715 (2019)

19. Stein, D., Hanenberg, S., Unland, R.: Join point designation diagrams: a graphical representation of join point selections. International Journal of Software Engineering and Knowledge Engineering **16**(03), 317–346 (2006)

20. Tariq, S., et al.: Approaches for non-functional requirement modeling: A literature survey. In: Intl. Conf. on Computing Information Sciences. pp. 1–6 (2021)

21. Wehrmeister, M.A., Pereira, C.E., Rammig, F.J.: Aspect-oriented model-driven engineering for embedded systems applied to automation systems. IEEE Transactions on Industrial Informatics **9**(4), 2373–2386 (2013), doi: 10.1109/TII.2013.2240308

22. Wehrmeister, M.A., et al.: Combining aspects and object-orientation in model-driven engineering for distributed industrial mechatronics systems. Mechatronics **24**(7), 844 – 865 (2014), doi: 10.1016/j.mechatronics.2013.12.008

23. Wimmer, M., et al.: A survey on UML-based Aspect-oriented Design Modeling. ACM Comput. Surv. **43**(4), 28:1–28:33 (Oct 2011)

24. Zhang, Y., et al.: Non-functional requirements elicitation based on domain knowledge graph for automatic code generation of industrial cyber-physical systems. In: 47th Conf. of the IEEE Industrial Electronics Society. pp. 1–6 (2021)

25. Zhou, Z., et al.: An evaluation of quantitative non-functional requirements assurance using archimate. IEEE Access **8**, 72395–72410 (2020)