# Systematic Literature Review on Test Case Selection and Prioritization: A Tertiary Study
## 2 Research Methodology
### Inclusion and Exclusion Criteria  
Table 2 summarizes the five inclusion and seven exclusion criteria that were applied to the first set of 55 contributions leading to the final set of 22 selected secondary studies. The most critical aspect was the quality check of the possible relevant studies. The exclusion criteria EC5, EC6, and EC7 were the basis for the rejection of studies. Many texts collected primary studies but did not include a proper analysis; these were excluded. For example, [30] is a systematic review on test case prioritization; however, we could find no analysis at all in the study. Other such studies were [31; 32; 33]. We also found a small number of secondary studies with an incomplete set of surveyed texts, and these were also excluded from the current study [34; 35; 36; 37; 38; 39]. A survey on many-objective optimization was presented in [40]; the study was sound and effective but found to be outside the scope for the current study. [PERSON] presented 10 best practices in test case prioritization, but the study did not present any analysis other than APEDs [41]. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Database** & **Search String** & **Filters Applied** & **No. of Studies** \\ \hline \begin{tabular}{c} Science Direct \\ [24] \\ \end{tabular} & \begin{tabular}{c} ((prioritization OR selection) AND (test OR \\ testing)) AND ((survey OR review) OR \\ mapping) \\ \end{tabular} & \begin{tabular}{c} Review article, research \\ article, Book chapters \\ Title, 1997–2020 \\ \end{tabular} & 9 \\ \hline \begin{tabular}{c} Wiley Online Library \\ [25] \\ \end{tabular} & \begin{tabular}{c} “((prioritization + selection + prioritisation) \& \\ “test”) \& ((\text{(survey + review) + mapping)}” \\ \end{tabular} & In Abstract & 25 \\ \hline \begin{tabular}{c} Springer digital library \\ [26] \\ \end{tabular} & \begin{tabular}{c} “((\text{(trest'' OR ''testing”) AND ((prioritisation OR \\ prioritization) OR selection) AND ((survey OR \\ review) OR mapping))/ \\ \end{tabular} & English, Computer-Science, Article, 1997–2020 & 1072 \\ \hline \
\begin{tabular}{c} Google Scholar \\ [27] \\ \end{tabular} &
\begin{tabular}{c} ((prioritization OR prioritisation OR selection) AND (test OR testing)) AND (survey OR \\ review OR mapping) \\ \end{tabular} & 1997–2020 & 2405 \\ \hline \hline \end{tabular}
\end{table} 
Table 1: \(Cont.\)Figure 1: PRISMA flow diagram for the current search. \begin{table}
\begin{tabular}{c c c} \hline \hline  & **Inclusion Criteria** & \multicolumn{2}{c}{**Exclusion Criteria**} \\ \hline IC1 & Relevance to TCS\&P. & EC1 & Texts that are not directly relevant to TCS\&P, including secondary texts in software testing \\  & & & not specifically mention TCS\&P. \\ \hline IC2 & Secondary studies such as surveys, reviews, & & Primary texts proposing a new approach, or validating existing ones, or empirical \\  & SLRs and mappings are included. & EC2 & \\  & & & comparison studies. \\ \hline IC3 & Texts published in journals, conferences, & & Texts found from magazine articles, theses, abstracts, posters, keynotes, doctoral \\  & workshops, and technical reports are & & symposiums, and non-peer-reviewed texts \\  & considered. & & from blogs and comments. \\ \hline IC4 & Texts should be available in the English language. & EC4 & Texts are not available in the English language. \\ \hline IC5 & Quality of the text based on evaluation of primary studies, breadth of survey and new & EC5 & Texts that did not provide any kind of analysis for the selected primary studies. \\  & addition to research field. & & for the research. \\ \hline  & & EC6 & Texts missing many important primary texts \\  & & published in the range of years reviewed. \\ \hline  & & & Texts representing mere repetition of available \\  & & secondary studies already published and not adding any new knowledge to the research. \\ \hline \hline \end{tabular}
\end{table} 
Table 2: Inclusion and exclusion criteria.