dataesr/scientific-paragraphs-categorization huggingface dataset:
• Novel empirical evidence about Maven dependencies: all APIs include a small set of types that is used by the majority of their clients, while most API types are used by at least one client, when considering the most popular version of an API. These findings open new directions to improve Maven's build process and to focus effort on the relevant subset of APIs for maintenance and migration tasks. This paper is organized as follows. Section 2 introduces the key concepts of Maven. In Section 3 we present our research methodology, analysis infrastructure and the dataset for this study. In Section 4 we discuss the empirical observations about the actual usage of client-library Maven dependencies. In Section 5 we discuss how our results could generalize in other ecosystems.
API usage in the Maven ecosystem Maven is a software project management tool for Java and other languages targeting the JVM (e. g. , Groovy, Kotlin, Clojure, Scala). It automates most phases of a software development life-cycle, from build to deployment. Maven relies on a specification file, named pom. xml, where developers explicitly declare what should happen at each building phase. Dependency management is one important phase where Maven automatically fetches software artifacts on which a project depends. Those artifacts are hosted on remote repositories, either public or private. Currently, Maven Central is the most popular public repository. It hosts millions of software artifacts coming in the form of binary sources (e. g. , jar). These artifacts are uniquely identified by GAV coordinates, referring to groupId (G), artifactId (A), and version (V).
Code-level API dependency There are many ways to use external APIs at the code level through inheritance, implementation, composition, genericity, static method invocation etc. Listing 2 shows a snippet from the class org. apache. flink. runtime-. entrypoint. ClusterEntrypoint of the library org. apache-. flink:flinkruntime:1. 5. 0  [3]. It illustrates different ways in which org. apache. flink:flinkruntime:1. 5. 0 uses some dependencies that are declared in its pom. xml. The class Cluster-Entrypoint implements the AutoCloseableAsync class exposed by the org. apache. flink:flinkcore:1. 5. 1 dependency (line 7), while lines 9 and 10 are examples of field declarations. On line 9 the dependency is used through a call to the static method getLogger(). Lines 22-24 illustrate reuse examples of API members such as annotations, methods call, or in methods signatures.
RQ3: How is the usage frequency of API types distributed when focusing on the popular version of an API? In this question we focus on the most popular version of each library in our dataset to determine the effect of a "sufficient number of users" on API usage. This sheds new light on usages ratios, compared to RQ2 that considers all versions of the libraries. This question is at the core of our analysis of API usage with respect to Hyrum's law, which requires a large number of clients to study. RQ4: Can inter-package calls explain the existence of API types that are unused by the clients? In Java, developers need to set the visibility of classes or methods to public if they want to allow inter-package usages. In other words, some parts of a library's API might be public only because they are intended to be used by other package of the library, and might not be meant to be reused by the library's clients.
Dataset In this work, we analyze software dependencies both at the artifact and at the code levels. At the artifact level, we analyze pom. xml files of client projects to determine the list of direct dependencies they declare. At the code level, we analyze the bytecode of both the clients and the libraries. On the client-side, we determine what parts of the libraries' API they actually use. On the library-side, we evaluate the extent to which an API is actually used by its clients. Hence our dataset is composed of bytecode and pom file of both libraries and clients. We leverage the Maven Dependency Graph (MDG)  [7]  to identify the most popular APIs in Maven Central, as well as their client artifacts. Then we extract usage information through static analysis of the jar artifacts. This section details these two steps.
Hence, we study a set of 94 libraries. We compute the popularity of a LIB based on the number of distinct clients that declare a dependency toward a version of the LIB with a Compile scope. The raw dataset for our study includes all dependency relationships from any artifact, in Maven Central, towards any version of one of our LIBs. This represents 2,376,526 dependency relationship between 901,876 clients (belonging to 99,949 unique pairs (GroupID, ArtifactID)) and 94 LIBs. The LIBs are in a total of 5,225 versions in the dataset. Figure  2  shows the distribution of the number of clients for each LIB. The two most popular libraries are the standard scala-library and slf4j-api with respectively 227, 014 and 203,366 clients. The number of clients per library decreases quickly in this ranking to reach 7,007 clients for the least popular library of our dataset, xercesImpl.
These libraries cover a broad range of application domains, from logging, networking, language extensions, to collections and bytecode manipulation. Our dataset includes libraries from 15 of the 20 most popular categories of libraries from mvnrepository. com 1. The only categories not covered are, two related to Android applications (since we exclude apk), two related to testing (as we exclude test dependencies), and a   category related to web assets which do not contain bytecode. As illustrated in Figure  3 , we filter our dataset through the research questions. For RQ1, we focus on the 2,169,273 dependencies concerning the 829,410 client artifacts that we could resolve (those for which we could download the jar). For RQ2 we focus on the dependencies for which we could observe an actual usage in the bytecode of the client.
At this stage we exclude 2 LIB that do not contain public types. This represents 4,931 libraries, 1,277,106 dependencies and 677,953 clients. In RQs 3, 4 and 5 we analyze the client -API dependencies for the most popular version of each library. This corresponds to 94 libraries, 319,882 dependency relationship and 235,440 unique clients. This latest version of the dataset supports our investigations of API usage with "a sufficient number of users", a key condition to study long tail distributions. Given the large number of libraries and clients, the plots displayed in the section represent a lot of information and it is sometimes difficult to keep the intuition between the data and the software engineering phenomena that are at stake. To keep the discussion concrete, we select 6 libraries that we use to illustrate all the research questions.
The LIBRARIES MEMBERS part details the distribution of the number of type definitions and the total number of methods and fields across the library APIs. The LI-BRARIES TYPES part distinguishes between different kinds of type definitions (classes interfaces and annotations) that we found in APIs. We provide a detailed description of these types since they will form the main granularity at which we analyze API usages. The smallest API in our dataset is the javax. inject:javax. inject:1 library, which contains 1 interface and 5 annotations, of which, only one defines a default method.
API usages collection In a second step, we collect, from Maven Central, the jar file of every single artifact that declares a dependency to at least one of the libraries in our dataset. The artifacts are resolved with Eclipse Aether  [13] , a Java library that fetches artifacts from remote repositories for local consumption. We analyze the bytecode of each of these clients, looking for local variables, fields, parameters, return types, annotations, type extensions or implementations that are referencing li-  brary types, including in lambda expressions. We also analyze invocations that target any element of the resolved API members. The bytecode analysis is implemented on top of ASM  [29] , a popular Java library for bytecode manipulation and analysis. The source code is available on GitHub  [28].
For each API usage, we count the number of times an element is referenced. The CLIENTS OVERVIEW part of Table  2  gives the distribution of the number API types used, as well as the number of dependencies declared by each client. Table  3  is an excerpt of the database of API usages that we collected. This excerpt corresponds to the usages collected in the bytecode corresponding to the example in Listing 2. For example, this excerpt shows that the class Clus-terEntrypoint references the class Logger of slf4j-api:1. 7. 21 one time, and calls the method Logger. info 6 times.
For example, spring-boot-configuration-processor enable a developer to generate customized metadata based on annotations 5 to provide auto-completion and documentation. Other examples are jaxb-core, which is used to generate Java source code from XML files, and lombok that enriches the Java language with annotations that are used to generate boilerplate code. When these libraries are erroneously declared with the default scope (compile instead of provided), they are needlessly considered as a runtime dependency. Second, we distinguish libraries that are not supposed to be called directly by the client. Instead, they are used by other existing dependencies. For example, a client declares slf4j or mysql-connector-java as dependencies in order to let its other dependencies use different logging facades or database connectors.
The seven LIBs with the highest usage rate among their clients ( (LIB) > 80%)), include the standard libraries for Scala and Kotlin, as well as other frameworks used as domain-specific languages. This latter category includes the maven-plugin-api, which provides a way for developers to create Maven plugins, and the Apache camel-core library, an integration framework for systems producing and consuming data. The majority of LIBs, 58 out of the 94, have a between 40% and 80%. The median of the population is 53. 1%. This indicates that, for common libraries of the ecosystem, slightly less than half of clients declare a dependency towards a library and do not make any direct static Discussing the root cause of unused dependencies: We distinguish two common situations where a dependency is declared but not used by a client.
1% of unused dependencies: (1) forgetting to update the pom. xml and removing unused dependencies during maintenance or (2) a simple copy-and-paste of an existing pom. xml. A living example of the latter hypothesis is the multi-module Maven project com. eurodyn. qlack2. fuse where all the modules that declare a dependency to javax. inject use at least one API member, except qlack2-fuse-file-upload-rest. This module contains only one type  [15]  that does not import nor use any member of javax. inject API. In this case, it is safe to suggest that this dependency was copied and pasted from another module at the time it was created. Another common reason for clients to declare a dependency without actually using it is to expose it in the classpath so that another one of its dependencies can use.
RQ2: How is the usage frequency of API types distributed and how does that depend on the number of clients? In this research question, we study how client usages of APIs are distributed across its types. We focus on the clientlibrary relationships for which we are able to observe at least one usage of the library on the client's bytecode. This represents 1,277,106 dependency relationships between 677,953 unique clients, and 4,931 unique libraries (58. 87% of the dependencies in our dataset). Figure  5b  shows the distribution of usage rates of API types for of all 4,931 libraries used by at least a client. The xaxis represents the Type Usage Rates ( ) grouped in 11 categories. The first category is for types having a that is equal to zero, while the remaining categories are grouped by 10% ranges, the lowest bound excluded.
The y-axis represents the proportion of types in each library that falls into each category, from 0% to 100%, on a logarithmic scale. This figure is read as follows: The first column on the left represents the distribution of the share of API being used by none of its clients, the second column to the left shows the distribution of the share of API being used by more than 0% but less than 10% of its clients, and so forth. First, let us analyze the share of API types for each library being used by no client (leftmost boxplot). The first quartile of this distribution is 41. 7%, its median is 71. 8%, and its third quartile is 88. 9%. This means that for 50% of libraries, more than 71. 8% of the API types are used by no client in our dataset. On the opposite side of the figure, the rightmost column shows that the proportion of API types used by more than 90% of clients is greater than 0.
Overall, the distribution of the share of clients using the most used type of an API, as well as the share of API types used by more than 50% of clients, indicates the existence, in most libraries, of a small compact subset of APIs being used by most client. This is consistent with previous work  [33, 21, 46]. Here we analyze our illustrative examples in details, and check if our assumption that with enough client all API types end being used by at least one. commons-io:2. 4 is a good example: it exposes 112 API types, it is used by 21,959 clients and there is no type that is used by no client. We can observe on Figure  7a , that the most used type of commons-io:2. 4 is IOUtils used by only 64. 5% of its clients. This is lower than 75% of libraries in our dataset. Our assumption holds for regular APIs such as commons-cli:1.
RQ5: How many API classes are essential for most of the clients? In this last research question we explore how the long tail distribution of API usages can be navigated. We demonstrate that, if developers are willing to satisfy a majority of their API clients (and not all of them), then they can still identify a small core of API types on which they can focus their efforts and eventually apply the good practices from the literature. For example, testing, and documentation efforts can be focused on the small core of API types that are the most used without alienating a large amount of clients. Similarly automated library manipulation such as specialization or automated migration can support only this core while supporting the majority of the population of clients. The dataset for this question includes the most popular version of each LIB and their clients.
The API of each library is reduced to API types that are used by at least one client. Figure  9  represents the distribution of extinction sequences for the 94 libraries in our dataset. We simulate these extinction sequences by hiding types of their API from the least popular one to the most popular one. At every step of this process we observe the share of client that are no longer able to use the API. The x-axis shows the share of hidden API types (from 0% to 100%). Given a share of hidden API types, the y-axis indicates the share of clients that can still access all the types they need. In the rest of this section we say that these clients are correctly served. The colored lines represent the extinction sequences of our 6 case studies, while the boxplots represent the distributions for the population of 94 artifacts.
slf4j-api:1. 7. 21's extinction sequence is represented in green. It is one of the most extreme case in our dataset. The API contains 37 types, but 20164 of the 21398 clients (94. 2%) use only two types: LoggerFactory and Logger. As only 1560 clients use only Logger, the sharpest drop in correctly served clients is caused by the removal of LoggerFactory. This occurs because most clients use these two types in conjunction. The rest of the API provides more advanced logging options that only few clients use. Figure  10  illustrates this singular situation. This figure shows a chord diagram representing types of the API of org-. slf4j:slf4j-api:1. 7. 21 and its clients. Nodes on the upper par represents API Types with a size proportional to the number of clients using them. The lower part represents three group of clients (with a proportional size): in red, clients only using the two most popular types (Logger and LoggerFactory), in blue, clients that do not use any these two types, and in yellow other clients.
The orange line represents hibernate-core:4. 3. 11. Final's extinction sequence. It simulates the progressive hiding of the 507 types of its API used by at least one client. This sequence informs us that the 397 least used types of the API may be hidden without affecting more than 75% of the clients. The boxplots of Figure  9  represent the distributions of extinction sequences for all 94 libraries in our dataset. The median values show that for half of libraries, 88% of the API types or more can be hidden while leaving more than 75% of clients unaffected. This illustrates the implication of the long tail distribution of API usages. There is a small set of features used by most client, and the rest is used but rarely. Similarly, the series first quartiles of these distributions show that for three quarters of libraries 77% or more of the API types may be hidden without affecting at least 75% of clients.
Source code language for clients All artifacts in our dataset are Java bytecode. Yet, the source code may vary (Java, Scala, Kotlin, Groovy). We analyzed the clients implemented mostly in Scala, to check if a client's source code language affects the usage of external APIs. We choose Scala because it is the most popular language of the ecosystem, aside from Java, according to the popularity of its standard library. To assess if clients implemented in Scala use APIs differently than those written purely in Java, we select all the clients that declare a dependency (excluding test scope) towards any version of the Scala standard library (org. scalalang:scala-library. Let us note that these clients might also use other languages, e. g. , Java. This subset of clients represents 31. 4% of the dataset used for RQ2 (212 844 artifacts).
Then, we select libraries which API is called both by clients using Scala and clients that do not include Scala. This represents 1322 out of the 4931 clients of our dataset. We perform a Welch test to determine if the average number of types of the API called by Scala clients is significantly different from the one called by clients not using Scala. While the average number of types used by clients using Scala (12. 6) is slightly higher than the one of client not using Scala (11. 8), we cannot reject the hypothesis that these means are equal (p-value 0. 45). In other words, clients using Scala do not use significantly more types of an API than other clients.
Build tool Our observations about unused dependencies (subsection 4. 1) are affected by the build system used to compile and test a client. Indeed, different languages and associated build tools implement different policies regarding external API usages. In RQ1 and in our subsequent studies  [42]  we observed that Maven can build and package projects that include libraries that are not used. Meanwhile, the Go compiler does not compile programs which declare unused imports 9 , and widely used tools such as goimports 10 can refactor imports of a go program to remove unused ones. The ecosystem of Go libraries likely behaves differently than the Java ecosystem, regarding our observations in RQ1. We can therefore speculate that the different practices enforced by the build tools impact the distribution of API usages as studied in this work.
API size Our dataset shows a loose correlation (0. 21, p-value 0. 04) between the core-index of a library and the size of its 9 https://golang. org/doc/faq}unused _ variables _ and _ i mports 10 https://godoc. org/golang. org/x/tools/cmd/goimports API (number of public types). Consequently, variations in the range of sizes for a given set of APIs are likely to change the range of core sizes and the core-index. In our dataset, the 3 libraries that are written in Scala (scala-library, scala-compiler, and scala-reflect), have very large APIs in terms of number of bytecode types, in part because the Scala compiler generates types to implement various Scala language constructs. This tends to exacerbate the importance of unused or rarely used types. The range of API sizes varies in other ecosystems.
Update frequency and interconnection The number of clients for a given library in a specific version influences the shape of the extinction sequence and the number of used API members. The frequency of updates impacts this number of clients: a popular and stable library attracts more and more clients over time, while libraries that update very frequently have a scattered population of clients over multiple versions  [41]. Indeed, it is common for client projects to keep outdated dependencies  [20, 8]. Consequently, variations in update frequency have an impact on the shape of the Core. Decan and colleagues  [11]  study 7 software ecosystems (Cargo, CPAN, CRAN, npm, NuGet, Packagist and RubyGems), describing the variations in update frequencies, as well as how interconnected the ecosystems are.
The notion of Core outside Maven While our dataset focuses on artifacts from Maven Central, we are confident that the results would be similar on other Java corpora. We refer to previous works with other dataset to elaborate on the generalizability of our findings. Qiu and colleagues  [33]  analyzed 5000 projects mined from GitHub and observed that API usages follow Zipf's law. This is consistent with our results, and implies that extinction sequences would produce similar results on their corpus. Lammel and colleagues  [21]  perform API usages analyses similar to ours, based on the SourceForge ecosystem. Their dataset includes 6,286 clients for which they mine API usages. Several of their findings align with ours and comfort our results. (i) The 4 most used external APIs of their dataset are libraries providing XML parsing, Logging and Testing functionalities, and are also in our dataset.
(ii) The number of clients using popular libraries follow a similar decreasing exponential (See Figure  2 ). (iii) Based on the number of clients in their dataset, Lammel and colleagues conclude that most APIs are not well covered by usages, which does not contradict our observations on libraries with a limited number of clients. (iv) In their case studies, they observe rare projects that largely cover APIs and many projects using only a limited subset of APIs. This is consistent with our observations. Furthermore the low cumulative coverage they observe is consistent with our observation that most clients focus their API usages on a small subset of APIs. When it comes to other software ecosystems, they vary in their custom values and policies  [8] , which may change the exact values obtained.
Related Work Several existing works have investigated the usage of APIs in open-source projects and industrial applications. In this section we discuss the related work along the following aspects. API usage in practice. Several studies have focused on understanding how developers actually make use of APIs on a daily basis  [38, 43, 6]. Some of the motivations include improving API design  [27]  and increasing developers productivity  [22]. Qiu et al. [33]  present empirical evidence showing that a considerable proportion of API members are not widely used, i. e. , many classes, methods, and fields of popular libraries have never been used. They have found that, on a corpus of 5,000 projects, API usage distribution follows a power law, which is consistent with our findings. Sawant et al. [39]  propose a tool to mine API usages and evaluate it on a dataset of project mined on GitHub using 5 popular Java APIs.