# A Tertiary Study for Process Mining
## 3 Results--Findings
### Data Synthesis
#### 3.1.2 RQ2: What Are the Trends Relating to the Quality of Published SMSs/SLRs? The complete table with the quality scores for each study is shown in Appendix A. Based on this assessment, all quality criteria, except for Assessment of Quality, are almost always used by the studies included in this research. A partial assessment of the quality of the primary studies is only observed in studies S1 and S19. A complete assessment is only observed in studies S17 and S20. The fact that the quality assessment of primary papers is absent in most of the studies is reflected in their overall quality score, as most of them can be classified as being of low and medium quality. In contrast, all four studies performing at least a partial quality assessment of the primary studies are classified as being of high quality. Quality of Published SMSs/SLRs over Time  
The quality score and publication year of each study included in this research are shown in Table 5. The average quality score for SMSs/SLRs in the field of process mining is 3.5, indicating that the studies published are mostly in the upper bounds of medium quality. By analyzing the quality scores per year (Figure 3), we observe similar quality levels for studies published between 2016 and 2019, with an average score of 3.5. The quality seems slightly lower for studies published in 2020. The highest average quality score is observed for 2021. However, generic conclusions cannot be drawn, as there was only one study among the included studies published in 2021. Finally, the trend in the average quality score returns to 3.5 for 2022, confirming that for the years covered in this research, the studies included are of medium quality, a trend that remains unaltered throughout the years, with a few exceptions, based on the included studies in this research. Comparison of Quality for SMSs/SLRs Published in Journals and SMSs/SLRs Published in Conference Papers  
The number of studies included in this research was almost evenly distributed in terms of publication venue (journal versus conference papers). Thirteen studies (S1, S4, S8, S9, S11, S12, S13, S16, S19, S21, S22, S23, and S24) were published in journals. Twelve studies (S2, S3, S5, S6, S7, S10, S14, S15, S17, S18, S20, and S25) were published at conferences. The average quality for studies published in journals is 3.5, whereas for studies published at conferences, the quality is slightly lower at 3.45 (Table 6). As a result, we observe that the quality remains the same regardless of the venue of a paper (journal or conference paper). However, with a closer look at the quality scores per year, we can see that since 2020, conference papers appear to have increased their average quality (3.3 in 2020). The average quality score for journal papers was 3.7 up to 2020. We observe that for the studies included in this research, there appears to be a slight decrease in the average quality score  
Figure 3: Average quality score for the included studies throughout the years. for journal papers and a slight increase in the average quality score for conference papers in the past 2 years. Taking a closer look at the analysis of the quality of journal papers, we identified which papers were published in open-access journals and which were not. Five out of thirteen were published in open-access journals and the remaining eight were published in subscription journals. As indicated in Table 7, from the studies included in this tertiary study, the papers published in open-access journals are of higher average quality. More specifically, the average quality score for open-access studies included in this paper is 3.9, with a gap of 0.65 in the average quality score from the rest of the studies published in subscription journals. Although the small number of studies will not allow us to draw generic conclusions for the average quality score for open-access journals, this is an indication that papers published in open-access journals are of high quality. Future research can further investigate this and reach more generic conclusions.