# Empirical Study of Test Case and Test Framework Presence in Public Projects on GitHub
## 7 Conclusions  
In this paper, a study of the presence of test cases and the use of different frameworks in a huge set of GitHub projects using Java as the primary language was performed. In total, 6.258 M projects and 340 M of classes containing the word \"test\" in its content were analyzed, in which 1124 M test cases were found. Manually, 600 randomly selected projects containing a typical number of the word \"test\" occurrences were analyzed in different parts of the code, in a total of six groups. We calculated a general correlation between the word \"test\" in file content and the number of test cases in it. At the same time, we created an overview of testing frameworks usage, how they are combined and how they are dependent on each other. We summarize the most interesting findings from our study:  
* Manual investigation of randomly selected repositories with a huge number of the same occurrences of \"test\" showed that 58% of such search results are _example test cases_.
* There is no strong relation between the occurrence of the word \"test\" and creation date, date of the last commit, number of commits, or number of watchers.
* A total of 51.57% of projects include at least one test case.
* There is a poor correlation between the number of occurrences of the word \"test\" in the file content and the number of test cases in such file (\(r=0.0253\)).
* _JUnit_ is used in 48% of projects and is mostly used with _Mockito, Spring Testing Framework_, and _Hamcrest_.
* _TestNG_, considered the second most popular Java unit testing framework, occurred only in 3% of projects and was never used without _JUnit_.  
Based on the findings, the following additional contributions are concluded:  
* Searching for \"test\" in file content cannot be directly used to estimate the number of test cases.
* Test cases are present in more than half of the projects, so mining information from tests about the production code is promising.
* An overview of testing frameworks usage and ways how they are combined in projects will allow practitioners and researchers to focus on the most used ones.  
In future research, we will focus on the analysis of example tests and their automatic detection, as these tests do not really test the production code and, therefore, do not improve code quality. There is also a need to streamline tools detecting industrial projects with the possibility of excluding irrelevant ones, such as reaper, which was time-consuming to use on a large sample of projects (see our experience with this tool in paper [15]). It is also necessary to evaluate the impact of the presence of automated tests on code quality in proprietary projects, as the results may be diametrically different compared to open-source solutions.  
Conceptualization, M.M.; Data curation, M.M.; Formal analysis, M.M.; Investigation, M.M.; Methodology, M.M., J.P., S.C., M.S., and F.G.; Software, M.M.; Visualization, M.M.; Writing--original draft, M.M.; Writing--review and editing, M.M., J.P., S.C., M.S., and F.G. All authors have read and agreed to the published version of the manuscript.  
This work was supported by Project VEGA No. 1/0762/19 Interactive pattern-driven language development.  
Data Availability Statement: Data supporting reported results can be found at [[https://doi.org/10.5](https://doi.org/10.5)]([https://doi.org/10.5](https://doi.org/10.5)) 281/zenodo.4566740 (accessed on 20 July 2021) and partially at [[https://doi.org/10.5281/zenodo.45](https://doi.org/10.5281/zenodo.45)]([https://doi.org/10.5281/zenodo.45](https://doi.org/10.5281/zenodo.45)) 66198 (accessed on 20 July 2021).  
The authors declare no conflict of interest. The funders had no role in the design of the study; in the collection, analyses, or interpretation of data; in the writing of the manuscript, or in the decision to publish the results.