# Quality Properties of Execution Tracing, an Empirical Study
## 4 Results and Discussion
### Validation  
In the course of the validation, content, construct, internal, and external validity are considered. In addition, the usual indicators for validity in qualitative research: credibility, transferability, dependability, confirmability are also scrutinised [97]. Content validity [104] in the context of our study refers to the extent to which the universe of all possible items were considered while the possible variable candidates were being collected. While identifying the potential variable candidates, which influence execution tracing quality, (1) the literature was considered, and (2) a sample, drawn from the study population of software professionals to process explicit personal experiences. Search strings in the scientific document databases were recorded and the queries were complemented with reference searches. Explicit and implicit variable candidates were extracted and the latter was also transcribed as introduced in Section 4.2. In addition, the variable candidates collected from focus groups of software professionals were recorded and the data collection process was ceased only when the number of the newly identified items dropped below a defined threshold. After constructing the variables from the data collected, they were verified based on data of the related works on logging as listed in Section 2. Construct validity shows the extent to which the findings reflect the content of the constructs in the phenomenon investigated [104]. Quality assessment examines to which extent concrete quality requirements are satisfied. This activity is performed along defined variables, i.e., quality properties, in all the identified quality models in [36]. Moreover, variables are necessary to define to be able to capture a manageable part of quality. Consequently, variables from the universe of the identified potential quality property candidates were defined. The variables were formed (1) from the data extracted from the publications, and (2) from the data collected from the sample of software professionals. Both sets of data were processed and data coded separately. The data coding yielded the same results with minimal deviations. The final variables were defined with the data from both sources and also validated on the findings of the related works. Internal validity ensures that the study conducted represents the truth with regard the phenomenon investigated [105]. The data collection process was extended to all reasonably possible items to ensure content validity. The data coding process to form the variables was performed by one of the authors and the two other authors also checked the results, the outcome of which was discussed to reach a consensus to ensure intercoder reliability. In addition, construct validity was also considered in the scope of the study. External validity refers to generalizability of the research. The variable definitions stem from data from two different sources: (1) the literature identified as documented in Section 3.1, and (2) from empiric research with focus groups at large international software houses with employees located in Hungary as introduced in Section 3.2. The results were also validated on the related works in Section 2. Both sources of data: (1) literature identified for data collection and (2) the literature identified with related works stem from different parts of the world. Thus, the geographical localisation of the focus group research does not impose a limitation to the present study. Credibility refers to the judgement whether the research manages to express the feelings, and opinions of the participants [97]. This indicator is satisfied with the data collection in the moderated focus groups as the collected quality property candidates are examined in the course of the evaluation phase by the whole group and a score value is assigned to each item which represents the importance in the sight of the group.