## 2 Background and Related Work
### Related Work  
Several approaches have assessed the extent to which studies have adhered to reporting guidelines. These include [5; 8; 10; 12; 13; 15; 18; 19; 20; 21; 22; 23; 24; 37; 38]. Some of these approaches focused on the assessment of the adherence of systematic reviews to reporting guidelines and/or on the assessment of the risk of bias in systematic reviews. Assessing the adherence of reviews to reporting guidelines is a means to assess the systematicity of these reviews. Most of the existing approaches are either qualitative or quantitative. Qualitative approaches usually rely on ratings scales (e.g., low, medium) to assess the adherence to reporting guidelines, while quantitative approaches usually rely on frequencies and/or percentages to assess that adherence. Several of these approaches, e.g., [PERSON] et al. [10], [PERSON] et al. [15], and [PERSON] et al. [13], concluded that the adherence to reporting guidelines is suboptimal, usually vary from a checklist item to another one, and that more efforts should be made to increase that adherence and therefore yield systematic reviews of superior quality. However, none of these approaches focused on reviews published in the software engineering field. [PERSON] et al. [18] implemented ROBIS (Risk of Bias in Systematic reviews), a tool that allows evaluating the level of bias in systematic reviews. The ROBIS tool mainly targets systematic reviews published in the context of health care settings, e.g., interventions, diagnosis, and prognosis. The ROBIS tool relies on three phases to assess a systematic review. Phase 1 consists in assessing the relevance of the systematic review for the investigated research question. Phase 2 consists in identifying concerns that may arise from the review process. Phase 2 focuses on biases that may be introduced into a systematic review during its completion, i.e., when studying eligibility criteria, identifying, and selecting studies, collecting data, and performing study appraisal or when synthesizing data and reporting findings. Phase 3 consists in judging the level, e.g., low, high, unclear, of the risk of bias by notably assessing the overall risk of bias in the interpretation of review results. Notably, ROBIS relies on the answers to a set of signaling questions to complete phases 2 and 3, which can be one of the following: \"yes\", \"probably yes\", \"probably no\", \"no\", and \"no information\". The answer \"yes\" means the concerns are low. Once the evaluation is complete, ROBIS outputs a risk rating (level). [PERSON] et al. [19] developed AMSTAR (A MeaSurement Tool to Assess systematic Reviews) to assess the methodological quality of systematic reviews of randomized controlled trials (RCTs). The main users of AMSTAR are health professionals, systematic reviewers, and policy makers. The AMSTAR tool implements a checklist approach and therefore relies on eleven checklist items, i.e., questions to assess systematic reviews. The answers to these questions fall in several response categories: \"yes\", \"no\", \"cannot answer\", and \"not applicable\". However, when used to assess a systematic review, AMSTAR does not output an overall score. [PERSON] et al. [20] developed AMSTAR 2 (A MeaSurement Tool to Assess systematic Reviews) to assess the quality of systematic reviews that include (non-)randomized studies of healthcare interventions. The AMSTAR 2 tool is an improvement of AMSTAR; it assesses all the important steps that systematic reviewers need to follow when conducting a review. The AMSTAR 2 tool is meant to allow decision-makers to distinguish high-quality reviews from low-quality ones and therefore to avoid accepting right away the findings of a single systematic review without second-guessing them. Unlike AMSTAR, AMSTAR 2 can evaluate the risk of bias in non-randomized studies included in a review. The AMSTAR 2 tool consists of sixteen questions, i.e., checklist items, whose responses can be one of the following: \"yes\", \"partial yes\", and \"no\".