# Empirical Study of Test Case and Test Framework Presence in Public Projects on GitHub
## 3 Results
### Testing Frameworks Usage  
The use of testing frameworks can have a significant impact on the writing of tests. The number of particular framework imports found across all projects' java files is shown in Figure 8. Remember that imports were searched only in files that contained the word \"test\", i.e., with possible usage in a test class. For this reason, imports of some frameworks were not found, e.g., _ellUnit_, _GrandTestAuto_, and _BeanTest_. The number of imports expresses how many files (most often java classes) use the given framework. We can notice a huge prevalence of the _JUnit4 + _JUnit5_ frameworks. (We divided JUnit into two groups due to different test case notation. JUnit3 uses \"test\" in the method name, and JUnit4+ uses the annotation @Test via its in-place binding [19] to the test method (currently only JUnit4 + JUnit5)). It is followed by _Mockito_, _Spring Testing Framework_, _Hancrest_, and _JUnit3_. _TestNG_ is often considered the second most popular framework, the occurrence of its imports is the 6 th most common. On the other hand, it is necessary to take into account that, e.g., _Spring Testing Framework_ is purpose-dependent, _Mockito_ is a part of unit tests, but is not an equivalent to a full-fledged framework such as _JUnit_ or _TestNG_. Figure 7: Year percentage comparison of projects with non-zero “test” occurrences and test cases. Figure 9 shows how many projects use a particular testing framework. Regardless of version, _JUnit_ is used in 48% of projects. _TestNG_ as the second most popular Java unit testing framework has a share of only 3%. In Table 1, percentages of all simultaneously used frameworks can be seen. Similar amounts of occurrences are grouped by colors for faster identification of simultaneously used frameworks with similar presence in the projects. We can notice that a large number of projects are used simultaneously with _JUnit_3 and _JUnit_4/_JUnit_5, probably because the projects were created with _JUnit_3, and after the release of _JUnit_4, the tests were written in a new style. At the same time, _Mockito_ is very often used with both _JUnit_ versions. It is also interesting to pay attention to frameworks in which frequent zero percentages occurred. For example, _Artos_ and _HawaRunner_ occur only with _Hamcrest_ and _JUnit_. It is clear from this that these two frameworks depend on _Hamcrest_ which depends on _JUnit_. Therefore, from Table 1 it is possible to observe how particular frameworks are interdependent. A detailed look at all simultaneous usages of the 5 most used frameworks is presented in Figure 10. _JUnit_ versions were joined under one group because they are part of the same family. It is important to note that the percentages only take into account the frameworks shown in the graph, i.e., not with all searched frameworks. It can be seen that _JUnit_ is mostly used alone, without any other frameworks. Its competitor _TestNG_, on the other hand, appeared never alone, but most often together with _JUnit_. This is probably because _JUnit_ is used by default and _TestNG_ is added manually by the developer to the project, without having to completely remove the competing framework. _Hamcrest_ is often used solely with _JUnit_ or with _JUnit_ + _TestNG_. The presence of all five frameworks in the project is more than 1%, which is quite a high proportion compared to other combinations. Figure 8: The number of found framework imports in all java files (logarithmic scale). Figure 10: The first 5 simultaneously used testing frameworks. Figure 9: The most used Java testing frameworks in projects. **RQ 5** _Which test frameworks and libraries are used and how are they combined in projects?_  
The most widely used framework is _JUnit_, imported in 48% of projects (2.84% solely JUnit3, 35.53% solely JUnit4 with JUnit5, 9.59% JUnit3 together with JUnit4 and JUnit5), which is most often combined with _Mockito_ (more than 10% of projects). _Spring Framework_ tests and _Hamcrest_, which are also very popular, have a relatively large usage share too.