2015 12th Working Conference on Mining Software Repositories

A Method to Detect License Inconsistencies in
Large-Scale Open Source Projects
Yuhao Wu∗ , Yuki Manabe† , Tetsuya Kanda∗ , Daniel M. German‡∗ , Katsuro Inoue∗∗ Graduate School of Information Science and Technology, Osaka University, Japan

Email: {wuyuhao, t-kanda, inoue}@ist.osaka-u.ac.jp† Graduate school of Science and Technology, Kumamoto University, Japan

Email: y-manabe@cs.kumamoto-u.ac.jp‡ Department of Computer Science, University of Victoria, Canada

Email: dmg@uvic.ca

Abstract—The reuse of free and open source software (FOSS)
components is becoming more and more popular. They usuallycontain one or more software licenses describing the requirements
and conditions which should be followed when been reused.
Licenses are usually written in the header of source code ﬁles as
program comments. Removing or modifying the license header
by re-distributors will result in the inconsistency of license with
its ancestor, and may potentially cause license infringement. But
to the best of our knowledge, no research has been devoted
to investigate such kind of license infringements nor license
inconsistencies. In this paper, we describe and categorize different
types of license inconsistencies and propose a feasible method todetect them. Then we apply this method to Debian 7.5 and present
the license inconsistencies found in it. With a manual analysis, wesummarized various reasons behind these license inconsistencies,
some of which imply license infringement and require the attention from the developers. This analysis also exposes the difﬁculty
to discover license infringements, highlighting the usefulness ofﬁnding and maintaining source code provenance.

license header. In our study, we focus on the license header
change during the evolution of open source software, which
we refer to as license inconsistency.
In our research, license inconsistency refers to the situationthat two source ﬁles that evolved from the same provenancebut contain different licenses. It is caused by either the
copyright owner of the source ﬁle or the developer who reused
the source ﬁle (hereafter we refer to as reuser) modiﬁed thelicense header. The copyright owner has exclusive rights of
their original work, thus their changes made to the license
header of their original work shall be legal. But if thosechanges are made by reusers, we should pay more attention tothem, since reusers can only change the license of the source
ﬁle under the conditions of the license itself. Otherwise license
violation may occur and involve the reusers into legal disputes.
To the best of our knowledge, no research has been done
to discover the characteristics of license inconsistencies in
the process of software reuse: How many types of license
inconsistency are there. Do they exist in large open source
projects. If so, what is the proportion of each type. Whatcaused these license inconsistencies?
Based on these questions, we set our research question as
follows:

I. I NTRODUCTION
As software reuse has long been advocated as a good practice to reduce development time and increase product quality[1], [2], [3], the activity of software reuse has become moreprevalent. Free and open source software (FOSS) is softwarethat can be freely used, changed, and shared (in modiﬁed or
unmodiﬁed form) by anyone, as deﬁned by the Open Source
Initiative (OSI). FOSS is always distributed under certain open
source software licenses. In a nutshell, an open source license,
which usually resides in the header comment of a source codeﬁle, grants the rights to redistribution of the software, andallows modiﬁcations and further distributions of the modiﬁedsoftware.
Developers who reuse open source software should pay
attention to the license header and are required to follow theconditions and limitations of the license. They shall neverchange the license without the permission of the copyrightowner, otherwise there would be potential of license infringement. We use the term license violation to describe such
scenarios that the license of the reused source ﬁle is changed
illegally.
Previous study by Li et al. shows that 36% of the developers
who reused the OSS components changed the source code [4],but they did not point out whether these changes involve the978-0-7695-5594-2/15 $31.00 © 2015 IEEE
DOI 10.1109/MSR.2015.37

RQ1 How can we categorize license inconsistencies?
RQ2 Does license inconsistency exist in large open
source projects?
• RQ3 What is the proportion of each type of license
inconsistency?
• RQ4 What caused these license inconsistencies? Are theylegally safe?
••

The contributions of this paper are:1) We describe and categorize different types of license
inconsistency.
2) We propose a method to detect license inconsistencies inlarge-scale projects, which can show the existence andnumber of each type of license inconsistency inside theproject.
3) We perform an empirical evaluation on our method to an
FOSS project, which reveals the license inconsistencies
324

in the project and meanwhile proves feasibility of our
method.
4) We perform a manual analysis on some license inconsistency cases by checking the history of the involved
projects. We summarized the reasons that caused license
inconsistencies, among which one is legally unsafe andneeds for the developers’ attention.
This paper is organized as follows. Section II describes the
background knowledge of license and license inconsistency.
Section III introduces our research method. An empirical study
with this method is described in Section IV, followed by
Section V discussing about the results. Section VI describes
the threats to validity. After an introduction to the related workin Section VII, Section VIII concludes this paper and points
out the future direction.

/* Getopt for GNU.
* NOTE: getopt is part of the C library, so if you don’t* know what "Keep this file name-space clean" means, talk* to drepper@gnu.org before changing it!
* Copyright (C) 1987-1996, 1998-2004, 2006, 2008-2012 Free
* Software Foundation, Inc.
* This file is part of the GNU C Library.
** This program is free software: you can redistribute it* and/or modify it under the terms of the GNU General Public* License as published by the Free Software Foundation;* either version 3 of the License, or (at your option) any* later version.
** This program is distributed in the hope that it will be* useful, but WITHOUT ANY WARRANTY; without even the implied* warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR* PURPOSE. See the GNU General Public License for more
* details.
** You should have received a copy of the GNU General Public* License along with this program.
* If not, see <http://www.gnu.org/licenses/>.
*/

II. L ICENSE I NCONSISTENCY

Generally, the license header of a source ﬁle can only be
modiﬁed by its copyright owner. Reusers shall never modify
the license header unless it is under the permission of the
copyright owner or allowed by the terms of the license.
Otherwise, they reusers may suffer from license violations.
To discover the potential license violations, we ﬁrst ﬁndout the license inconsistencies in our research. In the following subsections, we introduce our deﬁnition of license
inconsistency and give an example of license inconsistency
we found in Debian 7.5. Finally we categorize them based on
our analysis of the Debian project.

Software license is a permission and restriction to reproduce, modify and redistribute a software. An open source
license is a software license that follows Open Source Definition1 and is approved by Open Source Initiative. Now, 70
licenses are approved as Open Source License and BlackDuck
claims that the Black Duck Knowledge Base includes data
related to over 2200 licenses2 . Some software licenses havedifferent versions. For example, General Public License (GPL)has versions 1, 2 and 3. In addition, they allow to use “orlater” which allow us to regard the version as speciﬁed version
or a newer version. Some project hosting services such as
SourceForge.net show license on each project.
To reuse OSS source code ﬁles, developers must understandthose licenses and check whether those licenses have a conﬂict with the license under which the developed software is
distributed. This is not a trivial task because one Open Source
License is not always compatible with another. For example,
GPLv3 is compatible with any version of GPL, Apachev2.0,
modiﬁed BSD license (BSD3)3 . On the other hand, GPLv3
is not compatible with Affelo GPLv3, Apachev1.0, v1.1 andoriginal BSD license (BSD4). Therefore, license violation may
occur when developers misunderstand the license of source
ﬁles.
However, not all of the source code ﬁles in an application
are not under the same license [5], [6]. In addition, ﬁleswith the same name or the same content may have different
licenses. When a developer reuses a source ﬁle, if the names
and the contents of source ﬁles are similar to each other,developer may think that they are distributed under the same
license. However, if these licenses are different, the developer
may suffer from license violation by reusing one of them.
Usually, the license of open source software is indicated in
the ﬁrst comments of each source ﬁle. Here is an exampleof GPLv34 license header taken from getopt.c ﬁle in GNU
library:

A. Deﬁnition
In this research, license inconsistency refers to the situationthat two source ﬁles that evolved from the same provenancebut contain different licenses.
B. Example
In the Debian 7.5 project, package dpkg and anubis bothcontain a ﬁle named obstack.c. Except for the license header,these two ﬁles are identical, from which we can assume thatthese two ﬁles share the same provenance.
From package dpkg, the license of this ﬁle is:[...]This program is free software; you can redistribute it
and/or modify it under the terms of the GNU General Public
License as published by the Free Software Foundation; either
version 2, or (at your option) any later version.
[...]

While from package anubis the license is:[...]This program is free software: you can redistribute it
and/or modify it under the terms of the GNU General Public
License as published by the Free Software Foundation; either
version 3 of the License, or (at your option) any
later version.
[...]

As we can see, the licenses of the two ﬁles are different: theﬁrst one with GPLv2+, while the second one with GPLv3+.
Based on our deﬁnition, this is a case of license inconsistency.
Without tracing the history of each of these ﬁles, it is hard todetermine which one is the origin and which one is (or bothof them are) copied and modiﬁed. In this particular case, these1 http://opensource.org/deﬁnition
2 http://www.blackducksoftware.com/products/knowledgebase
3 https://www.gnu.org/licenses/license-list.html#GPLCompatibleLicenses
4 http://www.gnu.org/licenses/gpl-3.0.html

325

another version of GPL license. This type of
license inconsistency is typically caused by License
Upgrade or Downgrade in GPL license in theprocess of license evolution.

two licenses differ only by the version number of the license.
This can be explained in many different ways, some of themare:
• The ﬁrst ﬁle is the original one and was copied to thesecond project with the license version downgraded from
2 to 3.
• The second ﬁle is the original one and was copied to theﬁrst project with the license version upgraded from 3 to2.
• Both of the ﬁles are copied from another projects, andthe license was changed to GPLv2+ and GPLv3+ respectively.
To determine which one is the real case, we need to examine
the repository history of these two projects and ﬁnd out the
point that the change of license header happens, which will
be discussed in Section IV-B.

LC

III. M ETHOD
In our approach, we focus on detecting the license inconsistencies among ﬁle clones. In the scenario of source code reuse
where source ﬁles are imported from an upstream project,reused source ﬁles are kept with the same base name andtheir contents are almost the same, sometimes with smallchanges (renaming variable names etc.) [7]. But there are ﬁlesthat happen to have the same name but with totally different
contents. For example, two individual developers may put theirutility functions in a single ﬁle and both name it as util.c
independently. This leads to incorrect results.
To address this problem, we decide whether they are actually from the same provenance or not by their program
semantics. The rule is, if they are semantically identical,
then it is likely that they are copies of each other. We
use CCFinder [8], a code clone detection tool, to analyze
and determine whether these ﬁles are semantically identical.
CCFinder will generate a pre-process ﬁle which contains thetoken symbols of the source ﬁle ignoring the comments, whitespaces and newlines. For those source ﬁles with the same
token representation, we assume that they come from the same
origin, and then gather them into the same ﬁle group. Files inthe same ﬁle group have the same base name (but may comefrom different packages) and the same program statements,
possibly with different comments including license header.
Now that we have gathered all these similar ﬁles, we canidentify the license of each group of ﬁles and make a list of
the base name and detected license for each ﬁle group. In our
approach we adopted Ninka to detect the license of source
ﬁles, since Ninka is reported to have the highest precisionof all the license detection tools including FOSSology,
ohcount and OSLC in this research [9]. Ninka is a sentencebased license detection tool which can identify 110 different
licenses with 93% accuracy, and it can handle more than
600 ﬁles per minute. The names of the common open source
licenses and their abbreviation used in this article is shownin Table I. Many of these licenses have several versions. In
that case we use the sufﬁx v<number> to identify it. If it isfollowed by +, that means the user can choose this version
or any newer. “or later”: For example, GPLv2+ means “GPLversion 2 or later”.
We then compare the licenses of each ﬁle in the license listof each group. If all the ﬁles have no license, or all of them
have the same license, then there is no license inconsistency.
Otherwise, the group is likely to contain license inconsistencies. And then, based on the relation between licenses, ourapproach identiﬁes the type of license inconsistency. Note

C. Categorization
Based on the analysis to Debian 7.5, we observed 5 typesof license evolution. They are either executed by the original
author or reuser:
1) License Addition: The source ﬁle was without a license,
and a license is added in a later release.
2) License Removal: The source ﬁle was under a certain
license, and the license is removed in a later release.
3) License Upgrade: The source ﬁle was under a certain
version of GPL license, and it is upgraded to a higher
version of GPL license.
4) License Downgrade: The source ﬁle was under a certain
version of GPL license, and it is downgraded to a lower
version of GPL license.
5) License Change: The source ﬁle was under a certain
license, and it is changed to another license.
Note that, in the case of license upgrade and downgrade,
we only consider the GPL license. This is because currently
only GPL license has an “or later” option which allows thereuser to choose a later version of GPL as the license forredistribution (i.e. to upgrade to a higher version). Althoughsome other licenses, such as Apache license, may have different versions, reusers are not allowed to choose an arbitrary
version of the license. Thus it is reasonable to treat various
versions of these licenses as completely different licenses. For
such reason we treat the license evolution between different
versions of licenses other than GPL as license change in our
research.
License inconsistencies are naturally caused by license
evolutions. We use the following types to denote different
types of license inconsistencies between two ﬁles:LAR One of the two ﬁles contains a license while theother ﬁle contains no license. This type of license
inconsistency is usually caused by License Addition
or Removal in the process of license evolution.
LUD

Two ﬁles contain different licenses. This type of
license inconsistency is usually caused by License
Change in the process of license evolution.

One of the two ﬁles contains a certain version
of GPL license while the other ﬁle contains

326

TABLE I: Names of common open source licenses and their abbreviations used in this article.
Abbrev.
Apache
BSD4
BSD3
BSD2
CPL
CDDL
EPL
GPL
IBM
LesserGPL
LibraryGPL
MIT/X11MPLTABLE II: Strategies to decide whether a certain type of license
inconsistency exists in a group.

Name
Apache Public License
Original BSD, also known as BSD with 4
clauses
BSD4 minus advertisement clause
BSD3 minus endorsement clause
Common Public License
Common Development and Distribution License
Eclipse Public License
General Public License
IBM Public License
Lesser General Public License (successor of
the Library GPL, also known as LGPL)
Library General Public License (also known as
LGPL)
Original license of X11 released by the MIT
Mozilla Public License

Inconsistency Type
LAR
LUD
LC

Strategy#None > 0 and #License > 0#GPL ≥ 2#GPL ≤ 1 and #License ≥ 2

contain a license. According to our deﬁnition, this is LAR;
If we query for those whose #GPL ≥ 2, which tells us that
there are two or more different licenses in GPL family (suchas GPLv2+ and GPLv3+), and should be LUD; If we query for
those items with #GPL ≤ 1 and #License ≥ 2, which means
there are more than two licenses appear in this group and no
more than one GPL license exists (to exclude LUD case), it
seems to be the case that one license is changed to anotherone, which should be a LC.
B. Method of Detecting License Inconsistencies

that a group may have multiple license inconsistencies. For
example, if a group include a ﬁle under GPLv2, another ﬁleunder GPLv3 and the other ﬁle under Apachev2, the group
have two license inconsistencies: LUD between GPLv2 andGPLv3, LC between GPLv2/GPLv3 and Apachev2. For suchreason, we calculate License Inconsistency Metrics for eachof these groups, from which we can measure what type of
license inconsistencies and how many of each type exist in
the groups.

As a summary, our method is divided into 4 steps:
1) Create sets of ﬁles with same base name: We make alist for all the .h, .c and .java ﬁles in the target projectand count the occurrences of each base name. Then wecreate a set for each base name that has an occurrence
larger than one. The result is sets of ﬁles with the samebase name, and each set contains at least two ﬁles. Basenames that contain only one ﬁle are not considered, since
license inconsistency only exists between two ﬁles.
2) Create groups of semantically identical ﬁles: For each
set of same-base-name ﬁles, we apply CCFinder tocalculate the semantical tokens of each ﬁle. Then wecreate a group for ﬁles that have the same semanticaltokens in each set. The result is groups of ﬁles thatare semantically identical. This means the comments,
redundant white spaces and line breaks are ignored.
3) Identify licenses for ﬁles in each group: For each groupof semantically identical ﬁles, Ninka is employed to
identify the license(s) of each ﬁle. The result is licenselists for each ﬁle group.
4) Report groups that contain license inconsistencies andcalculate inconsistency metrics: We compare the license
list of each ﬁle group. File groups are reported to havelicense inconsistencies unless all the licenses on the list
are exactly the same. The result is a list of ﬁle groups
that contain one or more types of license inconsistencies.

A. License Inconsistency Metrics
The following 5 metrics are introduced to help measure thelicense inconsistencies for a ﬁle group:#File: Number of ﬁles in this group.
#License: Number of different licenses in this group. If
there are two or more licenses found, then it is likely that
there is a license inconsistency. If no license, or only one
license is found, then all the ﬁles are either without license,or they have the same license.
#Unknown: Number of ﬁles with an unknown license inthis group. For our purposes we consider all the ﬁles withunknown licenses as if they have the same license (thismight under-estimate the number of inconsistencies).
#None: Number of ﬁles without any license in thisgroup. If #License > 0 and #None > 0 then it is possiblethat at least one ﬁle in the group had its license added or
removed.
#GPL: Number of licenses in GPL family. This metricallows us to identify LUD in the GPL family.
These metrics are calculated for each ﬁle group based on
their license lists. The strategies shown in Table II enable us
to decide whether a certain type of license inconsistency exists
in this group.
Speciﬁcally, if we query the metrics result for those groups
with #None > 0 and #License > 0, which means there are
one or more ﬁles with no license, and also one or more ﬁles

TABLE III: List of base name occurrences in the imaginary project.
Base name Occurrence
C (.c)foo.c
4
C++ (.cpp)
bar.cpp1
Java (.java)
bar.java
1

327

TABLE IV: List of the license inconsistency metrics for each ﬁle group in the imaginary project.
Base name
foo.c
foo.c

Group
1
2

#File
2
2

#License
2
1

Pkg2

Pkg1

Packages

Files

foo.c

bar.cpp

LicenseGPLv2+

LGPL

foo.c

Pkg3

Pkg4

foo.c

foo.c

bar.java

Base name
foo.c
foo.c
foo.c
foo.c

Directory

Root

Directory

foo.c

Files

Group1

#GPL
2
0

Group
1
1
2
2

Package name
Pkg1
Pkg2
Pkg3
Pkg4

LicenseGPLv2+
GPLv3+
BSD3
NONE

GPLv3+ Apache BSD3 NONE

header), CCFinder treats them the same, and generate
the same token ﬁle. This also applies to foo.c ﬁle from
Pkg3 and Pkg4. Thus we can compare the hash value ofthe token ﬁles and group them into two groups, as shownin Figure 2.
3) Identify licenses for ﬁles in each group: For each ﬁle in
the group, we use Ninka to detect their licenses and makea list of the base name, group index and the licenses, as
shown in Table V.
4) Report groups that contain license inconsistencies
and calculate inconsistency metrics: We examine thelicenses of each group and found that both of these groups
contain license inconsistencies. Thus we report both ofthese groups and compute the inconsistency metrics foreach of them, as shown in Table IV. Base name is thename of the source ﬁle. Group indicates the index we use
to identify ﬁle groups.

Fig. 1: Hierarchy of an imaginary project and the license of eachsource ﬁle. Note that the foo.c ﬁle in Pkg1 was imported to Pkg2
with the license changed to GPLv3+; The foo.c in Pkg3 contains
totally different source code than the one in Pkg1, and was imported
to Pkg4 with the license removed.

Directory#Unknown
00

TABLE V: License list of the selected ﬁles from the imaginary
project.

Proj

Project Root

#None
0
1

Group2

foo Pkg1.c foo Pkg2.c foo Pkg3.c foo Pkg4.c

According to our rule, #GPL > 1 in Group 1 indicates a caseof LUD in this group, while #None > 0 in Group 2 indicates acase of LAR in this group. This conclusion is consistent to thescenario in our imaginary project, since the two foo.c ﬁles inPkg1 and Pkg2 contain GPLv2+ and GPLv3+ respectively
which is LUD, and the two foo.c ﬁles in Pkg3 and Pkg4
contain BSD3 and no license respectively which is LAR.

Fig. 2: Hierarchy of the grouped ﬁles.

C. Example
To make it clear, we illustrate our method with a simple
imaginary project shown in Figure 1. This project consists of
4 packages. The source code of foo.c ﬁle in Pkg2 is exactly thesame with the one in Pkg1, but the license header is changed
from GPLv2+ to GPLv3+; The source code of foo.c in Pkg3
is different from the one in Pkg1, i.e. they happen to have thesame base name. It is reused in Pkg4 with the license header
removed.
1) Create sets of ﬁles with same base name: The calculated list of base name occurrences of the imaginary
project is shown in Table III. Among them we create a
set for ﬁles named foo.c, and ignore bar.cpp and bar.java
since each of them only contain one ﬁle.
2) Create groups of semantically identical ﬁles: In thisstep, we use CCFinder to generate token ﬁles. Since thefoo.c ﬁle from Pkg1 and Pkg2 have the same source code
(except for their code comments which include license

IV. E MPIRICAL S TUDY
We conducted our study using a large open source Linux
distribution, Debian 7.5. Since it is hardly feasible to determine
how many and what types of license inconsistencies are there
in the target project before our analysis, it is difﬁcult toquantitatively evaluate our method. A qualitative evaluationof this method is discussed in Section VI and quantitativeevaluation will be our future issue.
A. SettingFirst, we downloaded the source code of Debian 7.5 fromits ofﬁcial site5 . The main characteristics of our target project
Debian7.5 is shown in Table VI.
5 https://www.debian.org/

328

In the following three subsections, we show examples foreach type of license inconsistency.
1) LAR: Examining the getopt.c of group 10 in second linefrom the inconsistency result list in Table X, we get the license
list of that group in Table VII. The rest ﬁles that contain thesame licenses are omitted from this list.
We can see that the license of the getopt.c ﬁle from theiceweasel package has an UNKNOWN license while theone from package icedove has no license (marked as NONE).
The contents of each ﬁle is as follows.
getopt.c from icedove package:

TABLE VI: Main characteristics of Debian 7.5.
Characteristics
Packages
Total ﬁles.c ﬁles.cpp ﬁles.java ﬁles

Number17,1606,136,637
472,861
224,267
365,213TABLE VII: License list of group 10 of getopt.c where LAR exists.
Package nameicedove
iceweasel

LicenseNONE
UNKNOWN (MPLv2)#include <stdio.h>#include <string.h>[...]
int main(int argc, char **argv){
PLOptState *opt;
PLOptStatus ostat;[...]
return 0;}

B. Results
In the ﬁrst step of our method, we got 658,088 sets of basenames in total. Among them, 137,618 (20.91%) sets containmore than one ﬁle, which were selected into the next step. The
breakdown of the base name sets for each ﬁle type is shownin Table XI.
In step 2, we calculated the semantically identical ﬁlegroups for each base name set and resulted in 74,848 suchgroups in total. Number of ﬁle groups per base name ranges
from 1 to 160, but 91% of these sets contain only one ﬁlegroup, thus the average number of ﬁle groups per base nameis merely 1.21.
Completing the following two steps of our method, we got
the list of the ﬁnal inconsistency result. From the total of
74,848 ﬁle groups, 5,359 (7.2%) of them were reported to haveone or more license inconsistencies. For the sake of space, weshow only part of them in Table X. Based on the strategy
introduced in Table II, we calculated the number of each typeof license inconsistency and their portion, as shown in Table
XII.
From this table, we can see that from the total of 5,359
groups that contain one or more license inconsistencies, 98.4%of them contain LC, followed by LUD and then LAR. With thehigh distribution of LC in all the cases of license inconsistencies, we can see that developers are more likely to change thelicense of the source ﬁle to another one, which is more likely
to cause license violations. For such reason, further study isurged to investigate the legality of these modiﬁcations.

getopt.c from iceweasel package:/* This Source Code Form is subject to the terms of the* Mozilla Public License, v. 2.0. If a copy of the MPL* was not distributed with this file, You can obtain one* at http://mozilla.org/MPL/2.0/.
*/#include <stdio.h>#include <string.h>[...]
int main(int argc, char **argv){
PLOptState *opt;
PLOptStatus ostat;[...]
return 0;}

As we can see in the ﬁle from icedove package, there is nolicense header at all, while the ﬁle getopt.c from iceweasel
package contains a license description of MPLv2 (though
Ninka failed to recognize it and reported as UNKNOWN).
Meanwhile, the other parts of these two ﬁles are exactly thesame, so we can assume that this duplicate should be causedby source ﬁle reuse. There are several possible explanations
to this case of license inconsistency:
The ﬁle from icedove package is the original one, andthe developers of iceweasel project reused the ﬁle andadded a license to it.
• The ﬁle from iceweasel package is the origin, and developers of icedove project reused this ﬁle and removed
the license header.
• Both of the ﬁles in these two projects reused different
versions of this ﬁle from another project, which caused
license inconsistency.
•

TABLE VIII: License list of group 0 of obstack.c where LUD exists.
Package namedpkg
anubisLicenseGPLv2+
GPLv3+

By tracing the revision history, we found that the last
one is the real case: the ﬁles in these two projects are
actually imported from a third project named nspr, where
the getopt.c ﬁle was created without a license in version 4.7.1,
and was added a MPLv2 license in version 4.9.1. It seems that
icedove project reused this ﬁle before the license header wasadded, while iceweasel project imported the version after the
license was added, thus caused the inconsistency of license.

TABLE IX: License list of group 15 of getopt.c where LC and LAR
exist.
Package name
p0f
snort
soﬁa-sip

LicenseNONE
GPLv2
UNKNOWN (IBM)329

TABLE X: Partial list of the license inconsistency metrics for each ﬁle group in detecting Debian 7.5.
Base nameobstack.c
getopt.c
getopt.c
...

Group
0
10
15
...

#File
17
5
4
...

#License
2
1
2
...

Total
247,520
154,990
255,578
658,088

Selected
65,572
24,99847,04813,7618

Number
5,272
2,350
1,500

#GPL
2
0
1
...

Category
Safe changes
Unsafe changes
Uncertain cases
Total

Perc.
26.49%
16.13%
18.41%
20.91%TABLE XII: Number of different license inconsistency types andtheir portion in 5,359 ﬁle groups. Note that one group may containmore than one inconsistency types, so that the total percentage canexceed 100%.
Inconsistency type
LC
LUD
LAR

#Unknown
0
3
2...

TABLE XIII: The count and percentage of each category for the 25
investigated license inconsistency cases.

TABLE XI: Breakdown of number of base name sets for different
ﬁle types.
File Type
.c
.cpp
.java
All types

#None
0
2
1
...

Count
14
5
6
25

Perc.
56%
20%
24%
100%

*to use the IBM-provided source code in any way he or she*deems fit, including copying it, compiling it, modifying[...]

From the header we know that the second ﬁle is licensed
under IBM copyrights (Ninka reported as UNKNOWN).
Since both these ﬁles contain the same program code, we may
assume that someone changed the license from one to the otherbut we do not know the direction yet. If these changes are notmade by the original author, this may be a potential license
violation.

Perc.
98.4%
43.9%
28.0%

2) LUD: Take obstack.c ﬁle of the ﬁrst line from theinconsistency result in Table X as an example. The license
detection result is partially shown in Table VIII. As we cansee from this table, the ﬁrst ﬁle is licensed under GPLv2+
while the second one is under GPLv3+.
The contents of the ﬁles from dpkg and anubis package
are compared in Section II-B. Both of these ﬁles contain morethan 400 lines of code, and they are exactly the same except for
their license description part. Similar assumptions are possible
to be made to explain this case of license inconsistency.
Tracing the ﬁle history we found that this ﬁle was originallycreated in gnulib. The license of this ﬁle was upgraded from
GPLv2+ to GPLv3+. By examine the commit log of dpkg,
we found that the developers of dpkg intentionally reused theolder version of the ﬁle from gnulib project, which caused
the license inconsistency.
3) LC: Here we select the getopt.c ﬁles in the third linefrom the inconsistency result in Table X.
As shown in Table IX, getopt.c from snort package
contains GPLv2 while the license of the one from sofia-sip
could not be recognized.
The contents of these ﬁles are as follows.
getopt.c ﬁle from snort package:

C. Manual Analysis
To decide whether these license inconsistencies may indicate legal problems or not, we have conducted a manual
analysis on the history of the ﬁles.
We randomly chose the samples using trial-and-error
methodology, that is, ﬁrst we randomly select a case of license
inconsistency and investigate whether it is legally safe or not,then we randomly select the next case and repeat the process.
We have investigated 25 cases in total.
Then we tried to categorize them according to the reason
that caused such inconsistencies. They are divided into three
categories, the percentage of each category is shown in Table
XIII, and the explanation to each category is as follows:
1) Safe Changes: In this category, either the original author
or the developers who reused the ﬁle changed the license
header, but the change they made is based on the terms
described in the license thus we say it is a safe change. They
are further divided into 3 groups:
Original author modiﬁed/upgraded the license: In thiscase, the author of that ﬁle modiﬁed the license header (either
by upgrading or totally changing it to another license ),while the reusers still use the old version of the ﬁle (either
intentionally or unintentionally).
For example, we examined a ﬁle named obstack.c in our
inconsistency result. This ﬁle originates from gnulib project,and its license is upgraded from GPLv2+ to GPLv3+ in
a commit on 10/7/2007. This ﬁle was reused in the dpkg
project but with a GPLv2+ license, and in the last commit
on 9/25/2011 the log is as follows:[...]** it under the terms of the GNU General Public License** Version 2 as published by the Free Software Foundation.
** You may not use, modify or[...]

getopt.c ﬁle from sofia-sip package:[...]* COPYRIGHTS:*This module contains code made available by IBM*Corporation on an AS IS basis. Any one receiving the*module is considered to be licensed under IBM copyrights

330

libcompat: Update obstack module from gnulib.
The version taken is the one before the switch to GPLv3+.
With a slight code revert to not have to include
exitfail.c and exitfail.h.
[...]ﬁle no longer existed. One explanation is that the ﬁle wasremoved in the project, but was not yet updated in Debian
7.5.
Project repository not available: Some project repositories
could not be found due to the lack of documentation, whilesome can not be accessed due to server error.

We can see that in this case, the reuser intentionally takes
an older version from the original project, which caused theinconsistency of license.
In another example, there is a ﬁle named paintwidget.cpp,
which originates from Qt project with BSD3 license. In
another project called PySide, this same ﬁle is licensed under
LPGLv2.1/GPLv3 dual license. Since these two projects bothbelong to Digia plc, which were acquired from Nokia, thisshall be a legal license modiﬁcation.
The ﬁle was originally multi-licensed and reusers choseeither one: The author of the ﬁle licensed the ﬁle under two
or more licenses, and the reusers can choose either one ofthem.
There is a ﬁle named SimpleXMLParser.java which originates from iText project and was under MPL/LGPL dual
license. Developers in pdftk project reused this ﬁle removing
the MPL license and chose LGPL as its license.
Reuser added one or more licenses: The original ﬁle isunder some licenses, and the reuser added one or more licenses
to it while retaining the original license.
From our inconsistency result we examined a ﬁle named
DOMException.java. This author of this ﬁle is World Wide
Web Consortium (W3C), and was licensed under W3C license.
When reused in ikvm project, a GPLv2 License was added toit resulting a composition of these two licenses.
2) Unsafe Changes: Under this category, developers whoreused the source ﬁle seemed to have modiﬁed the license
header which is not allowed by the original license terms.
This change may lead them to legal disputes, thus we say it
is an unsafe change.
Reuser replaced the original license, and changed the
copyright owner: The ﬁle is under a certain license in theoriginal project and developers who reused the ﬁle changed
the license header and the copyright owner.
From our inconsistency list, we examined a ﬁle named
SpringUtilities.java. According to the copyright year, Oracle isthe copyright owner, and licensed the ﬁle under BSD3. Whenreused in freemind project, developers changed the license
to GPLv2+ and the copyright header, which is not allowedin BSD3. This kind of changes to the license header by thereuser may lead to license infringement, and may involve thereuser into legal disputes.
3) Uncertain Cases: This category contains the license
inconsistency cases which are difﬁcult to determine whether
they are legally safe or not due to several reasons:
Source ﬁles are too small: Some ﬁles contain the same
source code, but due to their small size it is difﬁcult to decide
whether one is reused by the other or they just happen to be
the same. This problem is discussed in Section VI.
Files can not be found in the repository: Although somelicense inconsistency cases are reported to be existing in
Debian 7.5, when we investigated the project repository, theV. D ISCUSSION
From these results we can see that the license inconsistencies are not uncommon: out of 74,848 ﬁle groups, 5,359(7.2%) of them contain one or more license inconsistencies.
Among them, LC has the highest proportion with 98.4%,
followed by LUD (43.9%), LAR comes next with 28.0%.
With a manual analysis to several cases of license inconsistencies, we discovered that some of them are possibly causing
license violations, which needs further investigation. During
this process of analysis, we also found several challenges thatprevent us from automatically analyzing the history of ﬁles.
Packages in a large open source project are usually imported
from upstream projects. It is not a trivial task to ﬁnd therepositories of these upstream projects. Take Debian projectas an example, some of the packages contain a ﬁle indicating
the repository URL of that package, but some do not. For
such packages, we need have to go to the ofﬁcial site of the
upstream project and try to get the repository URL. There are
packages not even using version control systems, they simply
provide source code tarballs for each version on their server.
In this case, we have to download each tarball and track thelicense change manually.
In some cases the change of the license header is notrecorded in the revision history because the license header
is changed right before the ﬁle is added to the project. In thiscase, we have to check other information (e.g. on the ofﬁcialsite of the project or in the commit comment where the ﬁle
was added) to ﬁnd out the reason why developers changed thelicense.
Besides, after we ﬁnd out that the ﬁles with similar code
contents in different packages contain different licenses, wehave to determine where the ﬁle comes from, i.e. the original
project of that ﬁle, in order to decide the direction of thelicense change. But to the best of our knowledge, there isno good way to ﬁnd the origin of a certain ﬁle. What we
do in our research is to take the date of the ﬁrst commit ofthat ﬁle as a reference. If the commit date is not available,in the situation of not using version control system, we haveto manually check the comments of the source ﬁle to see if
it contains information of the author. If not, then we are notable to decide which ﬁle comes ﬁrst.
Revisiting the research questions:• RQ1: How can we categorize license inconsistencies?
We categorize license inconsistencies into these 3 types:i) LAR, which is typically caused by license addition or
removal; ii) LUD, which is related to license upgradeor downgrade in GPL family; iii) LC, which is usuallycaused by license change in the process of license evolution.

331

RQ2: Does license inconsistency exist in large open
source projects? Yes, license inconsistencies exist in large
open source projects. As we studied on the FOSS project
Debian 7.5, various types of license inconsistencies aredetected.
• RQ3: What is the proportion of each type of license
inconsistency? In the case study of Debian 7.5, out of
74,848 ﬁle groups we selected, 7.2% of them contain one
or more license inconsistencies. The proportion of eachtype is: LAR (28.0%), LUD (43.9%) and LC 98.4%.
• RQ4: What caused these license inconsistencies? Arethey legally safe? The reasons that caused license inconsistencies can be summarized into these groups according
to our observation: i) Original author modiﬁed/upgraded
the license; ii) The ﬁle was originally multi-licensed andreusers chose either one; iii) Reuser added one or morelicenses; iv) Reuser replaced the original license, andchanged the copyright owner. Among them, the last typeof change is unsafe.

the accuracy of Ninka is 93% [9]. We believe this is sufﬁciently high, so that the license detection result is good enoughto support our analysis. In addition, we regard UNKNOWN
licenses as the same license within each group, different
from any other licenses. If these UNKNOWN licenses ina same group are actually different from each other, we
may underestimate license inconsistencies. But this concern is
mitigated according to our observation to these UNKNOWN
licenses: most of those in the same group actually contain thesame license header, either a license that is not approved by
OSI or a user modiﬁed version of an OSI-approved license. On
the other hand, if these UNKNOWN licenses are actually thesame as those recognized ones (e.g. GPLv2, BSD3 etc.) in thesame group, this could be considered as a false positive. In thiscase, these UNKNOWN licenses are not exactly the same as
the original one, meaning that someone must have modiﬁed
the license header. We believe that it is necessary to check
whether these changes are legal or not. Thus it is reasonable
to treat them as license modiﬁcations, which is consistent with
our assumption. To obtain more precise results, we need to
improve Ninka.

•

VI. T HREATS TO VALIDITY

VII. R ELATED W ORK

In our approach, we only consider the license inconsistencies among source ﬁles that have the same base name, butthere might be scenarios that both the license header andthe base name of the reused source ﬁle are changed. In thiscase, our method can not detect the license inconsistencies
between the modiﬁed ﬁle and its ancestor. Since this paper isan exploratory study on license inconsistency, for simplicity
and efﬁciency, we only focus on those cases with the samebase name, which may increase the false negative. To reducethese false negatives, we can simply skip the ﬁrst step of our
method, which means we treat the whole project as one set,
and detect ﬁle clones within all these ﬁles. This progress will
surely take much more time, which we make it as our future
work.
Another factor that increases false negative rate is that we
use CCFinder to detect ﬁle clones which are exactly identicalto each other regarding their semantics. However, source codeﬁles are evolving: those that come from the same provenancemay differ from each other semantically after being modiﬁed
by developers. But since we can still get large numbers of ﬁlegroups that contain license inconsistencies using the proposed
method, we believe that it is enough for this exploratory study.
To mitigate this problem, we can use similarity metrics instead.
On the other hand, during our manual analysis we foundﬁles clones that are semantically identical, but due to theirsmall size and simplicity, it is difﬁcult to decide whether they
are copies of each other or they were written from scratch byindividual developers. If the later one is the real case, then it
would be a false positive of our result. But we believe it is agood practice to report these cases, have a manual investigationon them and ask the developers directly.
In the process of license identiﬁcation, as we employedNinka as the identiﬁcation tool, the accuracy of the result from
Ninka should also be considered. German et al. reported that

Many studies address inconsistencies among code clones.
Krinke [10] studied on changes applied to code clone inopen source software systems and showed that half of thechanges to code clone groups are inconsistent changes andthese changes are not solved if they occurred in a near version.
Göde et al. [11] studied patterns of consecutive changes tocode clone in real software systems. Some approach to ﬁnd
inconsistent changes are proposed [12], [13]. On the other
hand, Bettenburg et al. [14] showed that only 1% ∼ 4% ofinconsistent changes to code clone introduce software defects.
In addition, Göde et al. [15] showed that most code clones do
not evolve and the number of inconsistent changes is small.
Our work does not address inconsistency in changes to code
clones but inconsistency among licenses under which source
ﬁles including code clones are distributed.
In addition, many studies in software engineering investigated software license. Some approaches for software license
identiﬁcation are proposed [9], [16], [17]. Using these approaches, some researches analyzed software licenses in open
source projects and revealed some license issues. Di Pentaet al. [18] provided an automatic method to track changes
occurring in the licensing terms of a system and did an
exploratory study on license evolution in six open source
systems and explained the impact of such evolution on the
projects. German et al. [19] proposed a method to understandlicensing compatibility issues in software packages. They
mainly focused on the compatibility between license declaredin packages and those in source ﬁles. In another research of
Di Penta et al. [20], they analyzed license inconsistenciesof code siblings (a code clone that evolves in a different
system than the code from which it originates) between Linux,
FreeBSD and OpenBSD, but they did not explain the reasons
underlying these inconsistencies. Alspaugh et al. [21] proposed332

[3] B. W. Boehm, “Improving software productivity,” Computer, vol. 20,
no. 9, pp. 43–57, Sep. 1987.
[4] J. Li, R. Conradi, C. Bunse, M. Torchiano, O. Slyngstad, and M. Morisio,“Development with off-the-shelf components: 10 facts,” IEEE Software,
vol. 26, no. 2, pp. 80–87, March 2009.
[5] Y. Manabe, Y. Hayase, and K. Inoue, “Evolutional analysis of licenses in
FOSS,” in Proceedings of the Joint ERCIM Workshop on Software Evolution and International Workshop on Principles of Software Evolution(IWPSE-EVOL2010), 2010, pp. 83–87.
[6] Y. Manabe, D. German, and K. Inoue, “Analyzing the relationshipbetween the license of packages and their ﬁles in free and open source
software,” in Proceedings of the 10th International Conference on Open
Source Systems (OSS2014), 2014, pp. 51–60.
[7] Y. Sasaki, T. Yamamoto, Y. Hayase, and K. Inoue, “Finding ﬁleclones in FreeBSD ports collection,” in Proceedings of the 7th Working
Conference on Mining Software Repositories (MSR2010). IEEE, 2010,
pp. 102–105.
[8] T. Kamiya, S. Kusumoto, and K. Inoue, “CCFinder: A multilinguistic
token-based code clone detection system for large scale source code,”IEEE Transactions on Software Engineering, vol. 28, no. 7, pp. 654–670,
2002.
[9] D. M. German, Y. Manabe, and K. Inoue, “A sentence-matching method
for automatic license identiﬁcation of source code ﬁles,” in Proceedingsof the 25th International Conference on Automated Software Engineering (ASE2010), 2010, pp. 437–446.
[10] J. Krinke, “A study of consistent and inconsistent changes to code
clones,” in Proceedings of the 14th Working Conference on Reverse
Engineering (WCRE2007), 2007, pp. 170–178.
[11] N. Göde and J. Harder, “Oops! . . . I changed it again,” in Proceedings of
the 5th International Workshop on Software Clones (IWSC2011), 2011,
pp. 14–20.
[12] M. Gabel, J. Yang, Y. Yu, M. Goldszmidt, and Z. Su, “Scalable
and systematic detection of buggy inconsistencies in source code,” inProceedings of the 25th International Conference on Object-Oriented
Programming, Systems, Languages, and Applications (OOPSLA2010),
2010, pp. 175–190.
[13] Y. Higo and S. Kusumoto, “MPAnalyzer: A tool for ﬁnding unintended
inconsistencies in program source code,” in Proceedings of the 29th International Conference on Automated Software Engineering (ASE2014),
2014, pp. 843–846.
[14] N. Bettenburg, W. Shang, W. Ibrahim, B. Adams, Y. Zou, and A. Hassan,“An empirical study on inconsistent changes to code clones at releaselevel,” in Proceedings of the 16th Working Conference on Reverse
Engineering (WCRE2009), 2009, pp. 85–94.
[15] N. Göde and R. Koschke, “Frequency and risks of changes to clones,”
in Proceedings of the 33rd International Conference on Software Engineering (ICSE2011), 2011, pp. 311–320.
[16] R. Gobeille, “The FOSSology project,” in Proceedings of the 5th
Working Conference on Mining Software Repositories (MSR2008), 2008,
pp. 47–50.
[17] T. Tuunanen, J. Koskinen, and T. Krkkinen, “Automated software license
analysis,” Automated Software Engineering, vol. 16, no. 3-4, pp. 455–
490, 2009.
[18] M. Di Penta, D. M. German, Y.-G. Guéhéneuc, and G. Antoniol, “An exploratory study of the evolution of software licensing,” in Proceedings ofthe 32nd International Conference on Software Engineering (ICSE2010),
2010, pp. 145–154.
[19] D. German, M. Di Penta, and J. Davies, “Understanding and auditing thelicensing of open source software distributions,” in Proceedings of the
18th International Conference on Program Comprehension (ICPC2010),
2010, pp. 84–93.
[20] D. German, M. Di Penta, Y.-G. Gueheneuc, and G. Antoniol, “Codesiblings: Technical and legal implications of copying code between
applications,” in Proceedings of the 6th Working Conference on Mining
Software Repositories (MSR2009), 2009, pp. 81–90.
[21] T. Alspaugh, H. Asuncion, and W. Scacchi, “Intellectual property rights
requirements for heterogeneously-licensed systems,” in Proceedings ofthe 17th International Requirements Engineering Conference (RE2009),
2009, pp. 24–33.
[22] C. Vendome, M. Linares-Vásquez, G. Bavota, M. Di Penta, D. M.
Germán, and D. Poshyvanyk, “License usage and changes: A largescale study of java projects on github,” in The 23rd IEEE International
Conference on Program Comprehension, ICPC 2015, To appear.

an approach for calculating conﬂicts between licenses in termsof their conditions. However, our work proposed an approach
to ﬁnd license inconsistencies in similar ﬁles. By investigating
the revision history of these ﬁles, we summarized the factorsthat caused these license inconsistencies and tried to decide
whether they are legally safe or not. Recently Vendome et al.
[22] performed a large empirical study of Java applications
and found that changing license is a common event and a lackof traceability between when and why the license of a systemchanges.
VIII. C ONCLUSION AND F UTURE W ORKThis paper describes and categorizes different types oflicense inconsistency. With the proposed method, we managed
to detect all these types of license inconsistency from the large
open source project Debian 7.5, which shows the existence of
license inconsistency in open source projects and proves thefeasibility of our method.
With a manual analysis on some license inconsistencycases, we discovered that there are several reasons behind
license inconsistencies: Author changed the license header;
Reuser chose either one from multi-licenses; Reuser added
a compatible license; Reuser modiﬁed the license. Amongthem, the last one is potentially unsafe and needs further
investigation.
In the process of our manual analysis, we came across agreat difﬁculty to ﬁnd out the reason behind each license
inconsistency case. On one hand, it is difﬁcult to ﬁnd out
from where a certain ﬁle in a project is imported when lacking
enough information. On the other hand, it is also not a trivial
task to decide which ﬁle is the original work when they arefound in multiple projects. These problems highlight the needfor a method to ﬁnd and maintain the provenance between
applications.
For future work, we will apply our tool to large numbers ofopen source projects and examine the portion of each type of
license inconsistency. With the increased number of projects,
we believe that much more license inconsistency cases will
be found. And we will try to make a quantitative evaluationof this tool. Furthermore, we will try to develop a method tohelp us analyze the history of each ﬁle, so that we can decide
the safety of these inconsistencies efﬁciently.
ACKNOWLEDGMENT
This work is supported by Japan Society for the Promotionof Science, Grant-in-Aid for Scientiﬁc Research (S) “Collecting, Analyzing, and Evaluating Software Assets for Effective Reuse”(No.25220003) and Osaka University Programfor Promoting International Joint Research, “Software License
Evolution Analysis”.
R EFERENCES[1] M. D. McIlroy, J. Buxton, P. Naur, and B. Randell, “Mass-produced software components,” in Proceedings of the 1st International Conference
on Software Engineering (ICSE1968), 1968, pp. 88–98.
[2] T. A. Standish, “An essay on software reuse,” IEEE Transactions on
Software Engineering, vol. SE-10, no. 5, pp. 494–497, Sept 1984.

333