# Smart Lighting Systems: State-of-the-Art in the Adoption of the EdgeML Computing Paradigm
## 3 Research Method
### Exclusion Criteria

Table 7: Cont. A primary source in science is a document or article that reports on a study, experiment, case study, survey, research project, and so on. Primary studies are usually written by the person(s) who carried out the research, conducted the study, or ran the experiment(s), and include hypothesis, methodology, and results. So, the structure and aim of primary studies are deeply different from secondary studies; that is why it is not possible to apply to them the same quality criteria of the latter. It has been remarked that the quality of findings and conclusions of an SLR directly depends on the quality of the primary studies selected for the study [63]. Nevertheless, it is worth noticing that [PERSON] et al. [67] carried out a tertiary study consisting in analyzing 37 secondary studies to assess how well these studies are reported. They observed that just 8 papers out of 37 gave a score to primary studies. Recently, [PERSON] et al. [9] linked such behaviour of researchers with the time required for scoring the quality of primary studies as their number increases constantly. Assessing the quality of primary studies varies greatly as a consequence of the lack of a shared definition about such a concept [5; 63]. In the present SLR, we adopted a set of quality criteria based on the recent SLR by [PERSON] et al. [63], where a paper's quality is expressed by what authors call the \"study characteristic\". By analyzing 241 SLRs in the software-engineering domain, between 2004 and 2018, [PERSON] et al. observed that the characteristics of primary studies were scored by taking into account either (a) reporting, rigor, credibility, and relevance or (b) rigor and industrial relevance. For example, ref. [68] follows the first approach. In contrast, we followed the other one since such a method captures the perspective of the B2B firm that commissioned the review reported in this paper. Hereafter, we briefly recall the second alternative for readers to be able to understand the rationale behind the quality-assessment criteria (QC) introduced shortly. [PERSON] and [PERSON] [69] defined scientific rigor and relevance as follows. \[Rigor=context+study\:design+validity \tag{1}\]  
where:  
* _context_ refers to the description of techniques, product, tools, and people necessary to follow the study, compare it with others, and replicate it;
* _study design_ refers to the description of the approaches used for data collection and analysis;
* _validity_ refers to the discussion of any limitations or threats to the validity of the study. As pointed out by [PERSON] and [PERSON], the rigor characteristic refers to the extent to which the method adopted in the study is presented, while the evaluation of its actual rigor is out of the scope of the model. \[Relevance=context+\textit{research\:method}+\textit{subjects}+\textit{scale}\] (2) where: \[\bullet\] _context_ concerns whether the industrial context is representative; \[\bullet\] _research method_ concerns whether the method adopted in the study adds value to industry; \[\bullet\] _subjects_ concerns whether the subjects of the study are representative of practitioners; \[\bullet\] _scale_ concerns whether the size of the study is meaningful (\"toy\" examples are useless). From Equation (1) the following QC follow: QC1. Is there a description of the context? QC2. Is there a description of the study design? QC3. Are study limitations discussed? From Equation (2) the following QC follow:QC4. Is there a description of the context? QC5. Is there a description of the research method? QC6. Are the subjects of the study described and representative of practitioners? QC7. Concerning the size of the study, is it on an industrial scale? To the previous QC, the following one was added:  
QCS. Is the publication venue a journal, a conference, or a book chapter? QC8 is frequently considered in SLRs [61]. The QC score of each primary study was calculated using the following schema: N(o) = 0, Y(es) = 1, P(antial) = 0.5. In detail, the QC were scored as follows:  
QC1.