Electronic Communications of the EASST
Volume 63 (2014)

Proceedings of theEighth International Workshop on
Software Clones(IWSC 2014)

Measuring Copying of Java Archives— Position Paper —Tetsuya Kanda, Daniel M. German, Takashi Ishio and Katsuro Inoue
6 pages

Guest Editors: Nils Göde, Yoshiki Higo, Rainer KoschkeManaging Editors: Tiziana Margaria, Julia Padberg, Gabriele Taentzer
ECEASST Home Page: http://www.easst.org/eceasst/

ISSN 1863-2122

ECEASST

Measuring Copying of Java Archives
Tetsuya Kanda1 , Daniel M. German2,1 , Takashi Ishio1 and Katsuro Inoue1
1 t-kanda@ist.osaka-u.ac.jp, ishio@ist.osaka-u.ac.jp, inoue@ist.osaka-u.ac.jp

Graduate School of Information Science and Technology
Osaka University, Japan2 dmg@uvic.ca

Department of Computer Science
University of Victoria, Canada
Abstract: Copying the whole of a library is one of the major types of reuse in software development. In Java, a single library archive ﬁle often contains other libraries
it depends on, but users of the library hardly know about such inner libraries. Since
reusing libraries is a black-box method, developers may combine some libraries
without knowing that those libraries contain the same library inside independently.
As a result, a library may contain inside several copies of a library it reuses. In thisresearch, we measured copying of jar archives in the Maven Central Repository, acollection of open source Java libraries. Our results show that about 14% of toplevel jar ﬁles are reused in other jar ﬁles and some of them are duplicated in a single
jar ﬁle. We also found that some libraries contain two or more different versions ofthe same library.
Keywords: software reuse, Java library, duplication

1Introduction

Reusing software components reduces time and cost when constructing new software, and copying the whole of a library into the software development project is one of the major types ofreuse. Heinmann et al. showed that software reuse is common among open source Java projects
and the black-box is the predominant form of reuse [HDG+ 11].
In the case of Java, library archive ﬁles often contain their dependent libraries. One reason isthat developers want to use speciﬁc versions of libraries that might be considered reliable.
Once black-box reuse method has been done, it might not be known which version of whichlibrary is included in the library archive ﬁle. Davis et al. pointed out that the provenance of included components is not clearly stated and they proposed a method to determine the provenanceof source code contained within Java archives [DGGH13].
However, there is a possibility that developers are also copying duplicated libraries in thereused libraries without knowing that. When developers copy some libraries into their project,they may also unconsciously copy the same version of the library they already have or copydifferent versions of the library, but developers might not be aware of this duplication.
The mainstream of the software clone research is for the source code [Kos07] and few researches focuses clones of other software artifacts. There are some researches for Java archives
1/6

Volume 63 (2014)

Measuring Copying of Java Archives

Top-level jar file

Inner jar files

B.jar

A.jar

C.jar

B.jar

D.jar

C.jar

E.jar

B-2.jar

B.jar

Figure 1: Example of nested jar ﬁles

[BHV98, SMBZ07]. They dealt with a problem that how to compress class ﬁles to reduce ﬁlesize, but they paid no attention to the duplication of class ﬁles or whole of archives.
In this research, we performed an experiment to measure copying of jar archives in the MavenCentral Repository, a collection of open source Java libraries. We set these research questions asa ﬁrst step of the study of this type of duplication.
RQ1: How many jar ﬁles in a large software repository contain jar ﬁles inside and how many
jar ﬁles are reused?
RQ2: Does duplication of reused jar ﬁles in other jar ﬁles really exist? If so, are those duplicatedjar ﬁles the same version or different versions?

2Background

Apache Maven [Mav] is a software project management and comprehension tool. It automatically downloads dependent Java libraries from Maven repositories at build time. Maven Central
Repository (Maven2) is the default repository of Apache Maven. Maven2 repository contains
many popular libraries and projects.
Java archive ﬁle is the typical format used to distribute Java applications and libraries. A Jar
ﬁle contains Java class ﬁles and metadata and resources, and even another jar archive inside.
We deﬁne the term “top-level jar ﬁle” and “inner jar ﬁle” in this paper. A “top-level jar ﬁle” isa jar ﬁle found in the Maven2, and therefore, it corresponds to a component ready to be reused.
An “inner jar ﬁle” is a jar ﬁle that is included in another jar ﬁle, either a “top-level jar ﬁle” or an“inner jar ﬁle”.
Figure 1 shows an example of a library with nested jar ﬁles. A node corresponds to a jar ﬁle.
The jar ﬁle at the start of the arrow contains the jar ﬁle at the end of the arrow. In this case, thetop-level jar ﬁle A.jar is found in the target repository and contains four inner jar ﬁles in it; B.jar,C.jar, E.jar. C.jar contains B.jar which is exactly same ﬁle as B.jar under A.jar. D.jar contains
C.jar so B.jar appeared again inside of C.jar. E.jar contains B-2.jar which is the newer versionof B.jar. In Figure 1, all jar ﬁles in the right side of A.jar are inner jar ﬁle of A.jar. B.jar andC.jar are duplicated, and there are two versions of B.jar (B.jar and B-2.jar).
Proc. IWSC 2014

2/6

ECEASST

Table 1: Example of how the jar ﬁlename was use to identify the name of the library
Path/maven/org/geoserver/web/1.4.0-RC3//maven/org/apache/archiva/archiva/1.1/

3

Filename
web-1.4.0-RC3.jararchiva-1.1-src.jar

Detected library namewebarchiva-src

The experiment

We conducted an experiment to ﬁnd how many archive ﬁles contained duplicate archive ﬁles
inside. We detected two types of duplication of jar ﬁles: the same version of the same libraryand the different versions of the same library.

3.1

Setup

We used a framework for Software Bertillonage proposed by Davis et al. [DGGH13]. The
framework extracts metrics of source and archive ﬁles. We use two metrics, the ﬁlename andSHA1 hash of the ﬁle contents, to ﬁnd jar ﬁles with exactly the same contents. If the ﬁle iscontained in the archive ﬁle, SHA1 hash of the parent ﬁle is also extracted so that we can ﬁndout the contents of jar ﬁle.

3.2

Inner Jar Files

There are 607,319 top-level jar ﬁles in the Maven2 repository. Removing exactly the same ﬁles,with the same ﬁle name and the same hash, we get 599,498 top-level jar ﬁles. Checking thecontents inside each top-level jar ﬁles, we found that 4,747 top-level jar ﬁles contain at least one
jar ﬁle inside. 1,833 of them contains only one jar ﬁle and the largest one has 282 jar ﬁles init, 13.1 on average and median was 2. We also found that 118,361 different inner jar ﬁles arecontained in other jar ﬁles and 89,054 of them are found in the Maven2 repository as a top-leveljar ﬁle. This means that most inner jar ﬁles are reused directly from the Maven2 repository.

3.3

Detecting Duplication

To ﬁnd the two types of duplication inside jar ﬁles, we checked inner jar ﬁles using the following
method:First, we identify duplication of the same version of the libraries. If two jar ﬁles have the sameﬁle name and the same ﬁle hash, this means that they have exactly the same contents so they are
considered as duplicated and they are the same version. We did not care about the nest level of
jar ﬁle. In Figure 1, three B.jar are all different nest level counting from A.jar, but it does notaffect the analysis.
Second, we identify duplication of different versions of the libraries. To detect different versions of the same library, we remove the version information from the jar ﬁle name. Version
names are not only restricted in the number but also some strings such as “RC” and “SNAPSHOT”. We found that many libraries are also found in the Maven2 repository so we use thejar path name in Maven2 to identify its version. In the Maven2, most projects have their own
3/6

Volume 63 (2014)

Measuring Copying of Java Archives

Table 2: Analysis result for A.jar in Figure 1
StepUnique inner jar ﬁleUnique inner jar ﬁle without version names

File list
B, C, D, E, B-2
B, C, D, E

Table 3: Duplication of inner jar ﬁles

#ﬁles
#projects

Contains
inner jar
4,747886

Duplication Type
Same Different Both105
394
30
3949
14

Total
duplication
46973

directory, and a subdirectory for each version. We regard the directory name as the version nameof the library and remove it from ﬁle name of the library. We also remove a leading hyphen
or underscore with the version name. Table 1 shows two examples. This step is skipped if thelibrary is not found in the Maven2 repository since we cannot get the version name from thedirectory name.
Table 2 shows the example resuly of analysis for Figure 1. In the example Figure 1, B.jar
appears three times and C.jar appears twice. In this case B.jar and B-2.jar have the same libraryname so they are determined as different versions of the library B.
Table 3 shows the results of the experiment. We count the number of libraries in two ways;
counting number of jar ﬁles and counting number of projects used disregarding their version as
described as Step 3.
In total, 469 jar ﬁles contain duplicate libraries inside, about 10% of the top-level jar ﬁles that
contains inner jar ﬁles. Counting the number of projects, the result also shows that about 8% ofmaven projects contain inner jar ﬁles that have duplicated libraries in them.
We found both types of duplication in the Maven2 repository: 394 jar ﬁles contain the sameversion of the same library and 105 jar ﬁles contain the different versions of the same library.
We also found that 30 ﬁles have both types of duplication.
Some jar ﬁles which have duplication of different versions of the archive ﬁles have “test” intheir ﬁle name. The inner jar ﬁles of nexus-app-1.7.1-tests.jar, listed in Table 4, it contains 28
different inner jar ﬁles, including six different versions of log4j library. In total there are 32 innerjar ﬁles named log4j inside nexus-app-1.7.1-tests.jar and each versions of log4j appeared 3 to 7times.

3.4

Revisiting Research Questions

RQ1 How many jar ﬁles in a large software repository contain jar ﬁles inside and how many
jar ﬁles are reused?
In the Maven2 repository, there are 4,747 of 599,498 jar ﬁles that contain inner jar ﬁles. The
number of inner jar ﬁles is at least one and at most 282 ﬁles, 13.1 on average and median was
2. From the point of view of reuse, 89,054 of top-level jar ﬁles in the Maven2 repository alsoappeared as inner jar ﬁles.
Proc. IWSC 2014

4/6

ECEASST

Table 4: List of inner jar ﬁles of nexus-app-1.7.1-tests.jar
antlr-2.7.6 (7)nexus-3148-1.0.20100111.064938-1
antlr-2.7.7 (5)nexus-3148-1.0.20100111.065026-2
log4j-1.2.12 (5)nexus-indexer-1.0-beta-3-20010711.162119-2
log4j-1.2.13 (5)nexus-indexer-1.0-beta-3-SNAPSHOT
log4j-1.2.13-sources (5)nexus-indexer-1.0-beta-4
log4j-1.2.14 (5)nexus-indexer-1.0-beta-4-SNAPSHOT
log4j-1.2.14-sources (5)nexus-indexer-1.0-beta-4-SNAPSHOT-cli
log4j-1.2.15 (3)nexus-indexer-1.0-beta-4-SNAPSHOT-jdk14
log4j-1.2.8 (7)nexus-indexer-1.0-beta-4-SNAPSHOT-sources
log4j-1.2.9 (7)nexus-indexer-1.0-beta-5-20080711.162119-2(n) represents the number of ﬁles

nexus-indexer-1.0-beta-5-20080718.231118-50
nexus-indexer-1.0-beta-5-20080730.002543-149
nexus-indexer-1.0-beta-5-20080731.150252-163
nonuniquesnap-1.1-SNAPSHOT
plexus-plugin-manager-1.0-20081125.071530-1
sonatype-test-evict 1.4 mail-1.0-SNAPSHOT
very.very.long.project.id-1.0.0-20070807.081844-1
very.very.long.project.id-1.1-20070807.081844-1

RQ2 Does duplication of reused jar ﬁles in other jar ﬁles really exist? If so, are those duplicated jar ﬁles the same version or different versions?
Yes, 10% of jar ﬁles which have inner jar ﬁles contains duplicated jar ﬁles. We can say thatthe duplication in libraries are not an unusual problem. Both type of duplication are found in theMaven2 repository.

4Conclusion and Future Work

Developers reuse existing libraries by copying them into the software development project andthis style reuse reduces time and cost on constructing new software. On the other hand, there isa possibility that developers are also copying duplicated libraries in the reused libraries without
knowing that.
The result of our experiment indicates that the duplication of archive ﬁles in a single archiveﬁle is not frequent, but it exists. And furthermore, we must remember that many archive ﬁles arecopied into others so that further duplication can occur. Concretely, we found that about 5,000
jar ﬁles in the Maven2 repository contain other jar ﬁles in them and about 470 of them contains
duplicate libraries, some of them are the same version and some of them are different versions.
We also found that about 14% of top-level jar ﬁles in the Maven2 repository are copied into othertop-level jar ﬁles.
Based on this result, we are planning to perform further studies. We found duplication of jar
ﬁles but did not check all contents of them, and ﬁnding out which duplicated archive is mostfrequently reused is our future work. In addition, we should also analyze other types of archiveﬁles. We only used jar archives but the Maven2 repository has .zip, .tar.gz, .war, .ear formats ofarchives and these are not limited in binary archives but also source archives.
Another interesting fact is that there are some inner jar ﬁles and some duplications even though
Apache Maven has a system to download needed jar ﬁles at built time. We want to investigate
whether it is possible to remove the duplication.
Acknowledgements: This work is supported by Japan Society for the Promotion of Science,
Grant-in-Aid for Scientiﬁc Research (S) “Collecting, Analyzing, and Evaluating Software Assets
for Effective Reuse” (No.25220003). This work is also supported by Osaka University International Collaboration Research Grant, Joint Research on Software License Evolution Analysis.
5/6Volume 63 (2014)

Measuring Copying of Java Archives

Bibliography[BHV98]

Q. Bradley, R. N. Horspool, J. Vitek. JAZZ: An Efﬁcient Compressed Format forJava Archive Files. In Proceedings of the 1998 Conference of the Centre for Advanced Studies on Collaborative Research. CASCON ’98, pp. 7–. 1998.

[DGGH13] J. Davies, D. M. Germán, M. W. Godfrey, A. Hindle. Software Bertillonage - Determining the provenance of software development artifacts. Empirical Software Engineering 18(6):1195–1237, 2013.
[HDG+ 11] L. Heinemann, F. Deissenboeck, M. Gleirscher, B. Hummel, M. Irlbeck. On theExtent and Nature of Software Reuse in Open Source Java Projects. In Proceedingsof the 12th International Conference on Software Reuse. Pp. 207–222. 2011.
[Kos07]

R. Koschke. Survey of Research on Software Clones. In Duplication, Redundancy,and Similarity in Software. 2007.

[Mav]Apache Maven Project.
http://maven.apache.org/[SMBZ07] D. Saougkos, G. Manis, K. Blekas, A. Zarras. Revisiting Java Bytecode Compression for Embedded and Mobile Computing Environments. Software Engineering,IEEE Transactions on 33(7):478–495, 2007.

Proc. IWSC 2014

6/6