# Empirical Study of Test Case and Test Framework Presence in Public Projects on GitHub
## 1 Introduction  
Automated testing is considered an inevitable part of high-quality software projects [1]. Tests allow developers to automatically detect bugs in changed code, they can help them analyze and resolve the defects. The presence of test cases, however, does not ensure that project is bug-free. [PERSON] and [PERSON] [2] even found that there is only a weak correlation between testing and code quality. To further analyze the relationships between code quality and the occurrence of tests, we need to better understand how developers write tests in industrial projects. There are recommendations of how tests should be written, but we do not know how they are written in practice. Many researchers have tried to clarify the motivation of writing tests [3, 4, 5], the impact of Test-driven development (TDD) on code quality [6, 7, 8, 9], the effectiveness of tests on defective code [10, 11, 12], or the popularity of testing frameworks [13]. However, a manual analysis of test cases in projects could also be helpful to reveal and understand how tests are written. In order to perform such an analysis in the future, it is necessary to find out how many projects use automated tests, using which testing frameworks they are written, and how the frameworks and libraries are used simultaneously. At the same time, it is interesting to see the ratio of the word \"test\" in different parts of the project to the number of test cases and other project's metadata, such as number of watchers, number of commits, etc. This information can be useful in terms of further development of testing tools, using information from tests during program comprehension as well as mining repositories in other studies. It is also interesting to know how many open-source projects contain automated tests at all. According to [PERSON] et al. [14], who analyzed 1000 repositories, 39% of projects include test cases. [PERSON] et al. [1] executed a similar study on 20,817 and later on 50,000 projects with test case presence in 61.65% and 42.66% of repositories, respectively. The first study,however, was not conducted on a random set of projects, and the results of other ones published by [PERSON] et al. varied too much, so it is needed to evaluate the test case presence on a larger sample of independent projects. In our previous work [15], we had already analyzed the correlation between the occurrence of the word \"test\" in names and contents of files and the number of actual test cases in selected GitHub projects. We were using manual identification of test cases, so we needed to limit the number of projects. For this reason, we have analyzed only projects with the highest occurrence of the word \"test\". We have found that the Pearson correlation coefficient \(r=0.655\) and therefore there is only a weakly significant correlation. Despite the weak correlation with the number of test cases, we have found that the word \"test\" is very closely related to the occurrence of at least one executable test case. This means that the occurrence of this word in the project may indicate the presence of test cases. In this work, we extend our analysis of the occurrence of the word \"test\". Our long-term goal is to streamline the program comprehension using the information from tests, but first, we need to find projects with tests, from which it will be possible to find out what information is placed in the tests and where. Searching for the string \"test\" to find test cases in a project can return code artifacts that are not tests, and we need to eliminate such cases. At the same time, if there exists a typical number of tests in projects, it might be helpful to find out why this number is common and whether the results do not include true negatives. Therefore, we need to manually analyze random and independent projects to find out in which code artifacts this string is located (e.g., documentation, production code, testing code, etc.) and how often it indicates a test case.