dataesr/scientific-paragraphs-categorization huggingface dataset:
The main reason is that the Promela code generated by HUGO has many local transitions, which introduce overheads. For example, in the generated TollGate promela code, 7 steps are conducted to move from a initial pseudostate to its target state, while in our model only one (implicit) step is taken. The effect is exponential in case of non-determinism. We conducted another experiment on the dining philosophers problem to evaluate the scalability of USM 2 C. Table  3  shows the result of checking deadlock free property (with breadth first search). We can see from the result that USM 2 C can handle large state spaces caused by non-determinism. Reducing further the state space through techniques such partial-order reduction is the subject of our future work. We believe that communications between objects are error-prone and hard to find manually.
Discussion and Perspectives In this paper, we provide a formal semantics for the complete set of UML state machines features. Our semantics considers non-determinism as well as the communication aspects between UML state machines, which bridge the gap of current approaches. We have implemented a self-contained tool, USM 2 C, for model checking various properties for UML behavioural state machines. The experiments show that our tool is effective in finding bugs with communications between different state machines. We discuss in the following limitations related to our work. (1) We provide basic assumptions for those unclarities found in UML 2. 4. 1 state machines specifications based on our understanding, which may introduce thread to the validity of our work. (2)  We did not formally define the constraint and action language in this work.
Our goal in this paper is indeed to fill a research gap in the importance of software testing through a largescale empirical evaluation. In this study, we investigate 50,000 open source projects from the GitHub hosting site. GitHub provides various developer-friendly features which makes it an important development platform for millions of software projects, including major successes such as Linux and Ruby on Rails. The contributions of this paper are as follows: 1) We conduct a large scale study on thousands of software projects. We believe it is the largest study conducted on test cases. 2) We examine the popularity of test cases in different projects and different characteristics which can affect presence of test cases. 3) We discuss our findings and show correlations between the number of test cases and various characteristics of software projects.
II. METHODOLOGY For our empirical study, we downloaded thousands of projects using the GitHub API and selected 50,000 projects. The projects downloaded from GitHub do not have a distinct ordering scheme which varies with every request. To ensure that most of the projects are non toy projects in our dataset, we examined the dataset and found famous projects like jQuery and Ruby on Rails framework. We count the number of lines of code (LOC) in each project. We observe that over 38% of our projects have more than 500 LOC. Around 28% of the projects include more than 1,000 LOC, while more than 12% of the projects have more than 5,000 lines of code. Also, over 1,000 projects have more than 100,000 LOC which shows that our dataset includes projects of substantial sizes.
A. Collecting the dataset a) Lines of code: We cloned the actual git repositories of the projects in the dataset and counted the number of lines of code (LOC) in each project. Since GitHub uses the git software configuration management system (SCM) to store software revisions, we used the latest revision of the project, ignoring code comments and blank lines and used the SLOCCount 1 utility to compute the actual lines of code. b) Test Cases: Test cases define a sequence of steps to test the behavior of functionality of an application. For each project, we collect the files whose name contains "test" and count the number of such files. Then, we examine the different characteristics of the projects with respect to the number of test cases. c) Developer contributions: We find the contributions of developers for all the projects in our dataset i.
B. Research questions We investigate in this work a number of research questions to explore and assess the prevalence of software testing in software development projects. Given the extent of our datasets and the variety of artifacts that software development projects produce, we investigate various dimensions to characterize the relationship between a project and the test cases produced in that project. We are thus interested in analyzing the following research questions: RQ1:How many projects have test cases? For this research question, we examine the popularity of test cases in open source projects. We also investigate the characteristics of projects containing test cases and their corresponding lines of code (LOC). RQ2: Does the number of developers affect the number of test cases present in a project? Here, we wish to study the relationship between number of developers and number of test cases in a project.
B. RQ2: Developers and Test Cases We discuss the relationship between the numbers of developers and the numbers of test cases for the projects in our dataset. Based on developer names, we have identified 2,967,146 developers who have contributed to the code bases of the projects in our dataset of 50,000 projects. In total, for projects with test cases (21,328 projects), we have identified 2,864,179 developers who may/may not have contributed test cases. On the other hand, projects without test cases have 102,967 developers and constitutes 3. 4% of the total number of developers. We can observe from the statistics that projects with test cases have higher numbers of developers. Figure  4  shows that projects with/without test cases have approximately equal median number of developers. Using MWW test between the set of numbers of developers of projects with test cases and those for projects without test cases, we found p-value < 2.
2e -16 e -16 4  which shows that difference between these two sets is statistically significant. Further, we investigate whether the number of developers has an effect on the presence of test cases. Figure  5  depicts the scatter plot of the numbers of developers and the numbers of test cases from the projects in our dataset. The Spearman's rho for the distribution is 0. 207 with p-value < 2. 2 e -16 suggesting that there is weak correlation between numbers of developers and test cases. We proceed to investigate correlation between the numbers of test cases per developer and the numbers of developers in the project. Figure  6  shows the scatter plot between the two data variables. Here, we consider only the projects with non-zero test cases. We observe a decrease in the number of test cases per developer with an increase in the number of developers.
The Spearman's rho for the two distributions is -0. 446 (p-value < 2. 2 e -16 ). We observe that the relationship between the number of developers and the number of test cases per developer is negative. Thus, we can conclude that test count per developer decreases with an increase in the number of developers as only some of the developers contribute the test cases. Threats to Validity. We use SLOCCount 1 utility to count the number of lines of code. This tool covers 27 programming languages. It is possible that we miss some lines of code that are not written in one of these languages. We use heuristics to detect test files, i. e. , we consider the files whose name contains test. This might not identify all test files whose name does not have word test and conversely, detect some files whose name contain the word test but actually are not test files.
IV. RELATED WORK There is a large research community, which uses empirical methods to study several aspects of software engineering. Ajila and Wu perform an empirical study on the adoption of open source software development  [1]. Glynn et al. investigate the adoption of open source software in commercial applications  [7]. Greiler et al. conduct a qualitative study of test practices followed by a community of people working on plug-in based applications  [8]. Previously, there have been several large scale studies on software projects. Gruska et al. evaluates a lightweight anomaly detection technique on a collection of 6,000 projects  [9]. Surian et al. analyze a snapshot of projects in SourceForge. Net  [10]. In this work, we conduct a large scale empirical study using 50,000 projects from a developer-oriented platform, namely GitHub.
Monitor Access Virtual System Physical System Data gathering • The monitoring manager GMonEDB: This program acts as manager and monitoring archive of monitoring data covering the functions of the Data Gathering layer. It stores the monitoring information generated by GMonEMon in its own database. GMonEDB is responsible for providing the monitoring level and vision of each single user. • The data provider GMonEAccess: This library provides monitoring data to the user in an easy way. GMonE architecture is based on a typical publisher-subscriber paradigm. The GMonEMon instances act as publishers, sending monitoring information to the G-MonEDB subscribers at regular time intervals. At any moment there are one or more GMonEDB instances running in the system. Each instance can be subscribed to a different set of GMonEMon instances, allowing different managers specialized in specific subsets of cloud elements.
GMonEDB Once the monitored information has been obtained using the GMonEMon service, it is necessary to gather and manage it. Each GMonEDB service collects monitored information storing it in its own database. Each user, either client or CSP, has its own GMonEDB for storing the required information regarding its needs. The CSP maintains information about virtual and physical system whereas clients store only client-oriented data. GMonEDB monitoring database relies on MySQL  [31]  as default database management system. In addition, GMonEDB offers archiving flexibility, having the possibility to rely on other storage back-end technologies such as SQLite [49], RRD files  [42]  (through the rrd4j library  [41] ) or a key-value store (Apache Cassandra  [24] ). All these alternatives are meant to be a choice of usage depending on the particular monitoring scenario needs.
The main features of the GMonEDB service are: • It provides monitored information from a set of GMonEMon monitoring elements regarding the user needs. • It stores data from the different elements in its own database. This gives a fast access to monitored information; it also provides data about the cloud's past behavior. This makes it possible to observe the cloud operation evolution. The time spent by GMonEDB for storing the information in the database is called archiving time. • It manages relevant data. GMonEDB can be configured to subscribe to, store and manage only the necessary information in each particular case. If at any time the needed data should change, the service can be easily reconfigured on-line to start obtaining the new required information.
GMonEAccess The last part of the GMonE system is used to obtain monitored information from the GMonEDB service. This final module is called GmonEAccess and works as a programming library with the following features: • It provides a common interface to access the GMonE system. • It can be used to obtain monitored data, configure and manage the GMonE infrastructure on-line. Clients can manage their own virtual monitoring infrastructures without depending on service providers, whereas service providers can manage the whole infrastructure, both virtual and physical system. Each one can access their own monitoring data according to their necessities.
Experimental testbed In order to validate GMonE, we need to perform its experimental testing in a cloud environment as much realistic as possible. We carried out our experiments on Grid'5000. We used 45 nodes from the suno cluster on Sophia site. These nodes acted as cloud physical resources, on top of which a series of cloud software layers were deployed, in order to recreate a complete IaaS cloud system. The complete deployment is depicted in Figure  4 , and it presents the following elements: • Physical resources: 45 cluster nodes outfitted with Debian GNU/Linux Lenny with kernel 2. 6. 32, x86 64 Intel Xeon E5520 2. 26Ghz CPUs, 32 GB of RAM and 2 Gigabit Ethernet (Broadcom NetXtremeII BCM5716) network interfaces. • Cloud Infrastructure-as-a-Service: OpenNebula  [36]  installation, providing IaaS capabilities in the form of VMs.
OpenNebula is an ideal solution for a private cloud, given the high level of centralization and customization it provides for CSPs and end users. It incorporates a shared file system, which is NFS  [39]  by default. Using its default configuration with NFS, OpenNebula acts in a highly centralized manner. We have chosen OpenNebula because of its flexibility and suitability for research works  [43]. • Virtual resources: From an IaaS cloud client perspective, a total of 80 virtual machines were deployed, each of them outfitted with Debian GNU/Linux Lenny with kernel 2. 6. 32, a single virtual x86 64 CPU and 1GB of RAM. These VMs were interconnected using a standard OpenNebula Ethernet virtual network. • Client-side deployment: As an example of use of IaaS resources, Cassandra  [24]  was deployed on the 80 VMs.
Cassandra is a storage system developed as part of the Apache project, designed to handle very large amounts of data, spread out across multiple servers. Super column families can be visualized as a column family within a column family; in principle, every column may be sorted either by name or time. Cassandra is an ideal example of a state-of-the-art application that can be deployed on modern cloud computing infrastructures. • Benchmarking: To generate a realistic workload for this infrastructure and provide a complete cloud computing scenario, the Cassandra virtual deployment was subjected to the Yahoo! Cloud Serving Benchmark (YCSB)  [9]. One of the main advantages of this benchmark is an extensible workload generator, the YCSB Client, which can be used to load datasets and execute workloads across a variety of data serving systems.
All the core package workloads use the same dataset, so it is possible to load the database once and then run all the workloads. In the last version of YCSB (0. 1. 4) [56] , there are six workload versions. Table  2  describes each one of these workloads in detail. Using these standard workloads allows us to test our cloud infrastructure against a variety of access patterns and client uses. This provides a comprehensive scenario where we can effectively test the capabilities of our flexible monitoring system, GMonE. This experimental testbed is just an example of a cloud configuration, and therefore does not cover all possible scenarios. Our aim is to demonstrate GMonE's monitoring capabilities on all three major types of cloud monitoring. To that effect, we performed the following tasks: • Physical system monitoring: GMonE was deployed on the Grid'5000 physical resources.
The plug-in consists of a single Java 1. 6 class, implemented in a single file with less than 250 lines of code. This is a perfect example of how easy it is to extend and customize the GMonEMon capabilities. The monitoring information obtained using this GMonEMon instance was published to the CSP monitoring manager. • Client-oriented monitoring: All 80 VMs created for this testbed were also monitored using GMonE. This provides the cloud users with live, detailed information about the state of their resources. Each VM included an instance of GMonMon, monitoring its CPU and memory usage by means of the GMonEMon basic Linux plug-in. Additionally, a custom GMonEMon plug-in was developed, in order to obtain more detailed information about Cassandra. This custom CassandraPlugin measured the amount of disk space used by Cassandra in each virtual node.
The monitoring managers (GMonEDB), on the other hand, gather information from all monitoring elements, so they are sensitive to changes in the physical and virtual systems. To demonstrate GMonEDB scalability and elasticity, we exposed both the client-oriented monitoring manager and the CSP monitoring manager to an increasingly large number of monitoring elements publishing information. During this process, we measured the time it took for these monitoring managers to process the received data and store it in their respective databases. Figure  6  shows the average data archiving times observed and its standard deviations. It also shows a linear regression fit to the data presented, and the accuracy of this regression using the R 2  coefficient  1. The results show very fast archiving time and a very desirable linear growth in the narrow range of points we explored.
93ms of data archiving time per monitoring sample. The observed linear growth is very clear and guarantees a very desirable feature: the monitoring data archiving time depends linearly on the number of monitoring elements, therefore it can be estimated before deploying the infrastructure. To further validate this conclusion, we have to consider that the archiving process consists only on a basic information processing stage (performed in constant time per monitoring sample) and database insertion. Since GMonEDB uses MySQL as its default monitoring information storage back-end, this database insertion can be performed in linear time  [40] , given the number of monitoring samples, and hence the linear growth observed in Figures  6a  and 6b. To safely extrapolate the results obtained for very large numbers   of monitored physical or virtual resources, we have also to consider the data dispersion observed, that is, the archiving time standard deviation shown in Figures  6c  and 6d.
This extrapolation is, however, based on the assumption that the linear regression model constructed can realistically predict GMonE's behavior for very large scenarios. The question is: Can a model generated with data from less than 100 publishers be still valid for 1000 o 10000 publishers? This linearity assumption is motivated, as it has been explained, by the fact that the GMonDB archiving procedure is mainly a database insertion operation. As the number of publishers grows, the size of this insertion operation will grow accordingly. Previous work shows that, for a general purpose DBMS like MySQL in such a scenario, linear growth can be expected for the time of this insertion operation  [40]. In addition, linear regression models have already been successfully used in other scalability models in data center and large scale distributed applications  [54].
Nevertheless, every system has its own specific characteristics, and GMonE introduces a small layer of software on top of the DBMS, which must be properly analyzed for large scenarios before fully accepting the linearity assumption. In order to perform this analysis, we have carried out another experiment. As it has been explained, as the number of publishers grows GMonEDB archiving process has to deal with larger amounts of monitoring data. We have deployed a GMonEDB element in our physical system experimental testbed, and subjected it to an increasingly large amount of published monitoring information. We have generated this information synthetically, in order to achieve the expected data volume for a system composed of a number of publishers ranging from few tens to several thousands. For the case of 1000 publishers, for instance, the total amount of monitoring information would be (following the same model used in previous experiments): 1000 publishers × 4 records/(publisher × update) × 4 updates/round = 16000 records/round.
Very large storage scenarios often require advanced solutions, in order to achieve optimal scalability. As described in Section 5. 2, GMonEDB uses MySQL as default storage back-end, but can be configured to use other efficient alternatives, such as RRD Tool or Cassandra. In terms of storage back-end, the ideal GMonE configuration will depend on the characteristics of the system where it is deployed. MySQL produces excellent results in our experiments, and we believe is best suited for system sizes up to several hundred nodes. Selecting this alternative, however, requires the presence of a MySQL database management system (DBMS) installation and its proper configuration. This alternative is therefore specially adequate for CSPside monitoring in high performance scenarios where detailed monitoring information is required (small monitoring period).
This type of storage solutions have been developed, among other reasons, to address the above mentioned limitations of relational databases, such as MySQL, in certain scenarios. Cassandra is specifically designed to efficiently operate while handling extremely large amounts of data. In terms of GMonE, this means storing long-term historical information with the best possible detail. It is therefore, best suited for very large systems (thousands of nodes or more) where high monitoring resolution is necessary, as well as a comprehensive monitoring archive. In any distributed infrastructure the scalability of the entire system is also generally dependent on the scalability of the network; this means that the network can become a source of bottlenecks, again specially in very large systems. Monitoring tools introduce a slight amount of network load and, as in the case of the storage back-end, they are limited by the hardware and software elements underneath them.
The objective of Cassandra is to provide a highly available data storage/access service for modern cloud-like applications. It is a key-value store solution (not a traditional relational database) that it was initially developed by Facebook and powered their Inbox Search feature until late 2010. Facebook's Cassandra was based on the combination of well known and proven techniques. Tables are very important in Cassandra; a table represents a distributed multi-dimensional map indexed by a key. While raw operations are atomic, columns are grouped into sets called column families (Simple or Super).
Table3shows several examples of this extrapolation. This statistical analysis and extrapolation allow the client or CSP to calculate the Extrapolated average and standard deviation archiving time for physical system and client oriented monitoring. minimum database update period GMonE can be configured with, given the number of monitoring elements and its publish period. For instance, in the extreme case of 100000 physical resources being monitored, the average monitoring data archiving time can be estimated to 198. 105s, that is 3. 3min. If we incorporate the standard deviation model created, assuming that the archiving time data is normally distributed, we can estimate that more than 95% of archiving times will be in the [3. 1, 3. 5]min interval.
Acknowledgment This work is partially supported by the  Madrid Regional Authority (Comunidad de Madrid)  and the  Universidad Rey Juan Carlos  under the URJC-CM-2010-CET-5185 contract, the  Spanish Ministry of Education  under contract  TIN2010-212889-C02-01  and the  Marie Curie Initial Training Network (MCITN)  " SCALing by means of Ubiquitous Storage (SCALUS) " under contract  238808. Experiments presented in this paper were carried out using the Grid'5000 experimental testbed, being developed under the  INRIA   ALADDIN development action  with support from  CNRS ,  RENATER  and several Universities as well as other funding bodies (see https://www. grid5000. fr). The authors would like to thank also  Eduardo Perez , who worked with us in the programming of GMonE.