2010 Asia Pacific Software Engineering Conference

A Software Tag Generation System to Realize Software Traceability
Shinya Yamada, Masataka Ugumori, Shinji Kusumoto
Graduate School of Information Science and Technology, Osaka University
1-5 Yamadaoka, Suita, Osaka 565-0871, Japan
Email: {s-yamada, mugumori, kusumoto}@ist.osaka-u.ac.jp
purchaser and improve communications between purchaserand developer.
A new concept “Software Traceability” has been proposed to solve the issues [6][15]. That is, we apply theconcept of traceability in food distribution to the softwaredevelopment process and traceability information, indicating“when, where, by whom and how” the software has beendeveloped, is attached as a “software tag” to the actual
software product. The Software Tag is a new scheme toprovide information feedback about the project from thedeveloper to the purchaser. It establishes transparency ofthe software development project by allowing purchasers
to view and analyze the elements of the tag. The Software
Traceability and Accountability for Global Software Engineering (StagE) project [6][15] is a government-supported
project that pursues standardization and promotion of thesoftware tag scheme. In this project, we have defined thedetailed structure of the software tag.
There remain several challenges to implement the software traceability based on the software tag. One of them isto develop technologies to generate/produce the software tagfrom software development projects. This paper proposes asoftware tag generation system. In developing the system,
we focus on the following requirements: (1) it has generalversatility, that is, it can be applied to the wide variety
projects, (2) the cost to generate the software tag should
be low, that is, it can collect necessary data from the projectand generate the software tag at low cost and (3) it generates
a convenience software tag, that is, the generated tag canbe easily used for analysis of software projects. Based on
the requirements, we have implemented a prototype system
named CollectTag. CollectTag allows the user to define
concrete software tag definitions for his/her own project.
Also, it provides automatic measurement mechanisms tocollect metrics for specific elements of the software tag.
Finally, it outputs the software tag in the XML format. We
have applied CollectTag to the actual software development
project data and evaluated the usefulness of the system fromthe viewpoint of cost to generate the software tag. As theresults, we found that by using CollectTag, generating thesoftware tag is not high-cost.
Section II briefly describes the software traceability andthe software tag. Next, Section III proposes a software
tag generation system, CollectTag and Section IV evaluates

Abstract—This paper describes a system that supports togenerate the software tag which makes software development
visible to software purchaser (users). A software tag is a partial
set of empirical data about a software development project
shared between the purchaser and developer. The purchaser
uses the software tag to evaluate the software project, allowing
them to recognize the quality level of the processes andproducts involved. In order to implement the mechanism to
use the software tag effectively, it is necessary to support
generating the software tag. We have implemented a system
named CollectTag that supports to collect data and generate
the software tag. We conducted a case study to evaluate
the usefulness of CollectTag and generated software tag. The
results show that using CollectTag requires low cost to generate
the software tag.
Keywords-Project management, measurement, quality, empirical software engineering

I. I NTRODUCTION
Software systems are becoming huge and complex, andour everyday life heavily depends on such software systems.
In such social context, there are various issues related tosoftware systems. For example, stoppage of a service in thesociety’s infrastructure(ex. financial systems, transportation
systems) due to software faults causes huge social loss.
One of the major concerns of software purchasers (users)
in Japan is the quality of the software systems. Japanesesociety generally demands high-quality software systems
with low fault rates and high operability levels. On theother hand, many software purchasers in Japan are notknowledgeable about the nature of software. It is reported
that only 40% of Japanese major companies employ a fulltime Chief Information Officer (CIO) and that only 20% of
all CIOs are confident of their knowledge about information
technologies[12].
Without a sufficient understanding of software quality
and software projects, many companies try to purchasesoftware systems from software developers (vendors). This
produces a very risky situation. For example, purchasers
cannot specify system requirements very well, and they
do not oversee the project properly. Such situations often
lead to project failures. It is reported that only 31.1% of
software projects are recognized as “successful projects” inJapan[13]. To confront these issues, there is strong demand
to provide transparency of software projects to the software
1530-1362/10 $26.00 © 2010 IEEE
DOI 10.1109/APSEC.2010.55

423

developer, the delivered software tag and (if necessary) theempirical data are analyzed, providing a basis for exploringa resolution to the controversy.
The software tag is a key to improving transparency ofsoftware projects. By examining the software tag, the software purchaser can identify and understand the development
process, which has been mostly hidden from the purchaser.
The purchaser can evaluate the quality of the processes
and products of the project. For the software developer, thesoftware tag is useful to prove that they have conducted
the proper activities in the software project. Also, it can beused to trace the quality of the activities of sub-contractors
and sub-sub-contractors (such contracting chains are very
popular in Japan).
This scheme can be very useful for offshore and global development, because transparency and traceability of softwaredevelopment can be established with a fairly low overhead
for the developers. Standardizing the software tag will help
to establish a minimum baseline for project quality, and to
improve negotiations over software development contracts.
Evaluation of software products and projects based on theobjective empirical data contained in the software tag willlead to more healthy use of software in society

1. Purchase Order
5. Delivery

Software Developer

Softw are Purchaser
SoftwareProducts

3. Process
Improvement

4. TagSoftware Tag Composition

Figure 1.

2. Data
Collection

Empirical
data

Overview of the software tag use

the usefulness of the system through a case study. Finally,
Section V concludes this paper.
II. S OFTWARE T RACEABILITY BASED ON THE
S OFTWARE TAG
A. Overview of the Software Tag Use
A software tag is a packaged data set about a software
project. It is currently composed of 41 characteristic elements of project data and progress data (See Section II-B).
Figure 1 shows an overview of the software tag use.
Step1: A software purchaser orders development of a software system. The purchaser includes both the final
products and the software tag in their requirements.
Step2: During software development, various kinds ofempirical data are created and generated. For example, requirements documents, software design
documents, source code, test cases, issue tracking
logs, manual documents, review logs, and quality
analysis records may be produced. These are collected and archived. Note that we collect not only
the final data at the end, but also interim snapshot
data during development.
Step3: The collected data is analyzed for process improvement of the development organization, as is theusual process improvement scheme for softwaredevelopment organizations.
Step4: The collected data is used to construct the software
tag. Parts of the empirical data are selected andabstracted into the software tag format.
Step5: The software tag is delivered to the software purchaser periodically during the development and/or
finally at the end of the development together withthe final software product. The software purchaser
evaluates the software development by viewing
and analyzing the tag, and accepts the delivered
software product.
If a controversy such as a question about the quality ofthe product occurs between the software purchaser and the

B. Software Tag Standard
We have defined the elements of the software tag as shownin Table I, named Software Tag Standard 1.0. It is composedof 41 tag elements, which are categorized into project information and progress information. The project information
depicts the overall sketch of the project with various basic
pieces of information. The progress information provides
qualitative and quantitative indices of project achievementwith various measures of the development phases. The tagstandard provides more precise explanations and example
metrics for each tag element which are not presented here.
We divided project information into five categories described below, and settled tag elements for each category(see Table I).
∙ basic information of the software project (Basic Information)∙ information of the system developed by the project(System Information)∙ information of development framework applied to theproject (Development Information)∙ information of relationships between target project andother projects (Project Organization)∙ other information (Others)
To settle progress information, we referred to ISO/IEC
12207 Software Life Cycle Processes and activities[7], andincluded process, quality and effort information described
below into it.
∙ information of requirements, design, programming andtest for the software (Requirements, Design, Programming, and Test)

424

Table I
S OFTWARE TAG STANDARD 1.0
Classification

Category

Basic
Information

Project
Information

SystemInformation
Development
Information

Project
Organization
Other

Requirements

Progress
Information

DesignNo.

Classification

Explanation

Project Name

Unique name of project2

Organization

3

Project Information

4

Customer information

5

System Configuration

Information of development organization
Information needed to identify the
project characteristics
Information identifying the purchaser or
ownerInformation identifying system
configuration to label the type of system
Development system scale
Development process type or techniques
Structure of development organization
Information of development length
Name of super project which creates
this project
Name of sub projects which is createdby this project
Other necessary or useful data forinterpreting or analyzing tag data
Information of user-requirements
hearingAmount of requirements
Amount of changed requirement
Amount of design products
Amount of changed design
Implementation ratio of design forrequirements
Amount of programming products
Amount of changed programs
Complexity of programs

6
7
8
9

System Scale
Development Approach
Organizational Structure
Project Duration

10

Super-Project Information

11

Sub-Project Information

12

Special Notes

13

User Hearing Information

14
15
16
17

Scale
Revisions
Scale
Revisions
Design Coverage by
Requirements
Scale
Revisions
Complexity

18
Programming

Tag Element

1

19
20
21

Category

Test

Quality

Progress
Information

information of quality assurance activities on the
project (Quality)∙ information of development effort on the project (Development Cost)∙ information of project plan and management on the
project (Schedule and Management)∙ other information for the products attached to thesoftware (Other Products)

Development Cost

No.
22
23
24
25
26
27

Tag Element
Scale
Revisions
Density
Progress Status
Review Status
Review Density

28

Review Effectiveness

29
30
31

Defect Count
Fixed Defect Count
Defect Density

32

Defect Detection Rate

33
34

Static Check Results
Overall Cost

35

Productivity

36
Schedule andManagement

37
38
39
40

Other Products
41

Explanation
Amount of testing
Amount of changed test
Ratio of test to system size
Test progress to plan
Quantity information of review
Ratio of review to system size
Ration of found defects to amount ofreview
Number of defects found by test
Number of fixed defects
Ratio of defects to system size
Ratio of detected defects to consumed
test
Report of static checker
Development and maintenance cost

Ratio of amount of products to overall
cost
Information on management ofProcess Management
development process
Purchaser-Developer Meeting Amount of user-vendor communication
StatusTotal Risk Item Count
Number of risk items in the development
Time length between a risk item
Risk Item Existence Period
creation and deletion
Amount of product metrics not listed
Scale
above
Amount of change in products not listed
Revisionsabove

Also, we have tried to keep in mind the balance of the tag
elements. This standard does not include tag elements thatare computable from other tag elements. There are a numberof standards and reports such as SWEBOK, CMMI, ISO/IEC
15939, and reports by the Software Engineering Center inJapan (SEC) which can help interpret the tag elements. The
definition process was based on discussions with industry
and academic collaborators.
Through the discussion, we recognized that appropriate
metrics (tag elements) set and calculation methods of them
are different for organizations or projects. Therefore, we
made tag elements selective, and on the tag standard, calculation methods of corresponding metrics of tag elements
were not included but examples of the metrics are included.
In order effectively to use the software tag in the actual
software projects, it is necessary to support generating andanalyzing the software tag. In the StagE projects, we havebeen developing several supporting tools. In the following
Sections, we describe the supporting tool for generating thesoftware tag.

∙

It is not mandatory to use all 41 elements in the software
tag in all cases. The purchaser and the developer cannegotiate and select elements to use. Also, they can discussand determine the details of the metrics. For example, #19,
Scale of Programming, might be agreed to be measured
by lines of code without comments. Based on the software
tag standard, the purchaser and the developer should decidefollowings before using the software tag.
tag elements to be usedmetrics used for tag elements∙ measurement targets of metrics (e.g. whole system, subsystems, or files)∙ frequency of measurement∙ timing of offering the software tag to the purchaser(e.g. every week, every month, or the end of respective
process)∙∙

III. S UPPORTING S OFTWARE TAG DATA G ENERATION
In order to realize the software traceability based on
the software tag, first, it is necessary to develop a support
mechanism to effectively generate/construct the software
tag to various software development projects. This Section
describes the requirements for software tag data generation
system, our correspondence to them and a software tag data
generation system named CollectTag.

In this standard, we have included various kinds ofinformation that are considered important to the purchasers.
The overall structure should be simple for the purchaser tounderstand, so we have tried to keep it as simple as possible.

425

A. Requirements
We have the following requirements to the software tag
generation system:∙ Project-independent software tag generation: As described in Section II-B, the purchaser and the developer
negotiate and select elements of the software tag touse. Also, the corresponding metrics to the elements
are determined by the purchaser and the developer. So,
it is necessary to provide a versatile mechanism thatcan be applicable to various projects.
∙ Low-cost software tag generation: It is very important issue to reduce the cost of software development
projects. In the software development project undera severe budget constraint, it would appear that it’sdifficult to assign resources to prepare the software tag.
Thus, it is necessary to provide an economical support
to collect data and prepare the software tag.
∙ Output a convenient software tag: In order to confirm
the content of the tag, providing analysis/visualization
tools are useful for the purchaser. Also, to archive,
exchange and reuse the software tag, a standard data
format is needed. It is necessary to define the format
and the tag generation system should provide a feature
to output tags that meet the standard.

1. input a tag-datadefinition for the target
project

3.provide the input
form based on the
tag-data-definition

2.read the tagdata-definition

4.input tag-data

5.store the data

Data input system

Tag-data-definition

Figure 2.

Tag data
repository

Process of tag data definition

other tag elements and metrics, he/she can add them tothe frequently-used tag-data-definition and make the original
tag-data-definition for his/her project.
Currently, CollectTag provides the pre-registered tag-datadefinitions for the following elements: No.14 RequirementsScale, No.15 Requirements-Revisions, No.16 Design-Scale,No. 17 Design-Revisions, No. 19 Programming-Scale,No. 20 Programming-Revisions, No. 21 ProgrammingComplexity, No. 24 Test-Density, No. 27 Quality-Review
Density, No. 29 Quality-Defect Count, No. 30 Quality-Fixed
Defect Count, No. 31 Quality-Defect Density and No. 32
Quality-Defect Detection Rate.
(2) The system provides automatic measurement for specificelements.
In order to provide the software tag generation at low
cost, the system provides automatic measurement for someelements under typical software development. In the software tag, there are several elements related to software
products (requirements specifications, design specifications,source codes, bug reports and so on). It needs extra efforts
to get metrics from the products. Recently, several softwaretools are frequently used in the software development. For
example, version management systems, such as Subversion
and CVS, are well-known and commonly used. Software
products and the update history information are stored in
the product repository by version management systems.
Next, some metrics are measured from the specifications
and source codes. It’s not realistic to prepare automatic
measurement features for any specifications and source
codes. So, we focus on the UML models and source code
written in Java language since they are popular methodology
in recent software developments.
Our system automatically measures some metrics, related
to the selected elements in the software tag, from the data inthe product repository. Also, bug tracking systems, such as
Gnats[5] and KAGEMAI[10], are often used in the softwaredevelopments. Bug tracking systems store the information
about software bugs (faults) and so it’s possible to get somemetrics, related to the quality elements in the software tag,from the bug data repository.
As a result, we focus on the software development where
the following software tools are used:

B. Features of CollectTag
In order to meet the requirements in Section III-A, we
realize the following features to the proposed system:(1) The user can define each element and metric in thesoftware tag for his/her own project.
In order to flexibly generate the software tag, we implement a feature that the user can define the contents of his/her
project. Here, we call the definition of the tag-data tag-datadefinition and call the data collected tag-data. That is, thetag-data-definition is defined to each of metrics for each tag
element collected in the project. It includes the name, theexplanation, measurement frequency and so on (See Section
III-D1). On the other hand, the tag data is created for each
metric when the user inputs the actual value and each tag
includes the value and the created date and time. So, we cangrasp the transition of the value for each metric.
Figure 2 shows the process of the tag data definition.
At first, after the user (the developer) and the purchaser
negotiate and select the elements in the tag and define
the metrics for each element, the user inputs them into
the system and makes the tag-data-definition. Then, thesystem provides the input form based on the definition to theuser. According to the input form, the user inputs the data(measurement results) to each element. Finally, the inputtedtag data are stored into the tag data repository.
This feature provides the extendable tag data collection.
By preparing the frequently-used tag-data-definitions in advance, the system just loads the definition and provides
the input form based on the definition. If the user needs426beginVersion management system: Subversion, CVS
Bug tracking system:Gnuts, KAGEMAI∙ Modeling tool: UML design tools that output UML
diagrams into the XMI files.
Table II summarizes the elements, metrics for each element and sources for the metrics, that our system provides
the automatic measurement.
(3) Output the software tag as XML format
XML (Extensible Markup Language) is a set of rules forencoding documents electronically. Many systems containdata in incompatible formats. Exchanging data between such
systems is a time-consuming work. XML is a generic data
storage format and it is often used to define data format toexchange information between such systems. So, we settled
the draft of the standard software tag format which is basedon XML format, called standard software tag format data.
∙∙

27 Mar

30 Marthe tag… collect
data by using
CollectTag

28 Maysoftware tag

31 Mayend

C. Utilization process of CollectTag
Figure 3.

Here, we explain the process of generating the software
tag based on CollectTag. The process consists of the following five steps from the beginning to the end of the project.
Step 1:The user creates the new software tag project.
Step 2:The user makes the tag-data-definition for the
project.
Step 3:The user inputs the necessary data to the input
form of the project. (some data are automatically
inputted (see Section III-B)).
Step 4:Repeat Step 3 at predetermined intervals until theend of the project. (The intervals are determinedby the negotiation between the purchaser and thedeveloper.)
Step 5:The final software tag is generated.
The brief overview of Steps 3 and 4 is shown in Figure 3.
Figure 3 represents the flow of the project in chronologicalorder. After the project starts, the user inputs/collects thedata by using CollectTag. At predetermined intervals (inFigure 3, the interval is 3 days), the user repeats the data
input. In Figure 3, on March 27th, the first data input was
conducted and on March 30th (after 3 days), the second wasdone.
Collected tag data are stored in the tag data repository.
After completion of the project or as necessary, CollectTag
outputs the software tag in the XML format.

Overview of the tag data collection

Main
moduleTag data
definition
unit

input a tag-data-definition
for the target projectprovide theinput form

user

Input the data
request for
outputPresent a graphical outputof the tag data
flow of control

generate

Tag-data-definition

importTag data
input unit

store

Tag data
output unit

output by XML format
software tagflow of data

Figure 4.

Tag data
repository

System structure of CollectTag

the Tag data repository. If the user requests the output, theTag data output unit shows the tag data in graphical/tabularform to the user or translates and outputs the data into XML
format.
In the following, we explain the details of each unit andthe data items managed in the unit.
1) Tag-data-definition: The tag-data-definition has theinformation about each of metrics for each tag element.
Table III shows the constitution of the tag-data-definition.
It consists of the following 5 items:tagIdentifier, name,description, unit and frequency. tagIdentifier is an identifierof the tag-data, name is the name of metric for the tag
element, description is detailed information of the metric,
unit is the unit of the metric, and frequency means the

D. Structure of CollectTag
Figure 4 shows the system structure of CollectTag. It
mainly consists of three units: Tag data definition unit, Tagdata input unit and Tag data output unit.
The user inputs the tag-data-definition that defines the setof elements and the corresponding metrics of the software
tag through the Tag data definition unit. The Tag data input
unit generates the input form according to the tag-datadefinition and provides the form to the user. Then, after theuser inputs the data on the form, the tag data is stored into

427

Table II
E LEMENTS MEASURED AUTOMATICALLY BY C OLLECT TAG
ClassificationNo.

Tag Element

14

Scale[Transition]

15

Revisions[Transition]

16

Scale[Transition]

17

Revisions[Transition]

Requirements

DesignProgramming

Quality

19

Scale[Transition]

20

Revisions[Transition]

21
29
30
31
32

Complexity
Defect count[Transition]
Fixed defect count[Transition]
Defect Density
Defect Detection Rate

Metrics
Number of use case diagram
Number of use caseNumber of actor
Number of added, deleted use case diagram
Number of added, deleted use caseNumber of added, deleted actor
Number of class diagram
Number of sequence diagram
Number of state machine diagram
Number of activity diagram
Number of added, deleted class diagram
Number of added, deleted sequence diagram
Number of added, deleted state machine diagram
Number of added, deleted activity diagram
Lines of codeLines of added, deleted code
Number of revisions
CK metrics[1]Number of defects foundNumber of fixed defects
Number of defects found / Lines of code
Number of defects found / Consumed test

explanation
identifier
name of data item
detailed informationunit
frequency

unit of element
frequency of measure

Modeling file
based on XMI

Subversion or CVS
repository

Log files ofGnuts or KAGEMAI

Table IVTAG - DATA

Table IIITAG - DATA - DEFINITION

name
tagIdentifier
namedescription

Input

name
value
tagIdentifier
id
name
user
unit
objectdate
remarks

examplequality.revStatus.revNum
number of review
It means the number of review.
the review is conducted every
Wednesday.
numberonce a weekfrequency of the measurement (input of the actual value tothe metric).
For example, from Table III, we can see that the quality.revStatus.revNum means the number of review and thereview is planned every Wednesday and it is measured oncea week.
2) Tag-data: Tag-data is created for each metric definedin the tag-data-definition, when the actual value is inputtedby the user. Table IV shows the constitution of the tag-data.
It consists of 9 elements, that is, one actual measured value
for the tag-data (value) and meta data of value including
8 elements. tagIdentifier, name and unit are the same asones in the tag-data-definition. id is a hash value, user isa measurer of the metric, object is a source (ex. source
code, minutes) of the metric, date is the date when themetric is measured, remarks is the remarks about the metric,
respectively.
For example, from Table IV, we can see that the project
conducted ten times reviews from the beginning to the date
(April 18, 2007), the number of the review (10 times) ismeasured based on the minutes of review meeting and soon. Saving the meta data enables us to check the reason or

explanation
actual measured value
identifier
hash value
name of data item
measurer
unit of element
measurement object
time and dateremarks

example10
quality.revStatus.revNum
1365431584
number of review
Ugumorinumberminutes of review meeting
2007/4/18 15:00
dates of review, 3/27, ...

evidence of the value of the tag-data.
3) Tag data definition unit: Tag data definition unit
has functionality to add and delete the tag-data-definition
described in Section III-D1. Figure 5 shows the edit screenof tag-data-definition. The user makes the tag-data-definition
for each element of the software tag collected in the target
project through the screen. The tag-data-definitions madeby the user are stored in the tag-data-definition directory as
XML format and are used for generating the input form.
4) Tag data input unit: Structure of the Tag data input
unit is shown in Figure 6. It consists of input screen unit,
input form generating unit and automatic data collection
unit. The automatic data collection unit reads the tag-datadefinitions and generates the input form. Input screen unit
provides the input form to the user and the user inputs theactual value into the form. On this occasion, the user makes
the setting for the metrics measured automatically. When the
input is completed, inputted data by the user are stored in
the tag data repository and the settings for the metrics aredelivered to the automatic data collection unit. The automatic428

add, edit, or delete a tag
data definition

Figure 8.

Software tag in XML format

finish editing a tag data
definition

Figure 5.

5) Tag data output unit: Tag data output unit outputs thesoftware tag to the user. It shows the content of the software
tag in the form of a table and outputs it as a XML file.
Figure 8 shows the part of software tag written in XML
format. It can be used by some software tag analysis tools[4] developed in StagE project.

Edit screen of tag-data-definition

tag-data-definition
Tag data input unit
importshow the input form
configure auto-collectconfiguration
user of thesystemgenerate the input
form
Input form
Input screen unit
generating unit
input data

IV. C ASE S TUDY
A. Purpose

auto-collect configuration

In order to evaluate the applicability of CollectTag, weapplied CollectTag to the data collected from an actual
software development. In the evaluation, we focused on thecost of generating the software tag: manual data collection
and automatic measurement by CollectTag.

Automatic datacollection unit
collected data

Tag data
repository

Figure 6.

flow of control
flow of data

B. Target project
We used the actual software development data provided
by “ITSpiral”[8]. IT Spiral is a collaborative project bynine universities and four industries in Japan to develop acommon curriculum for teaching software engineering. IT
Spiral ordered a curriculum coordination software system
to several software companies. They requested the companies to deliver the software with all the related productand process data. There exists many empirical data fromrequirement definition, design, implementation, test and soon.
The project took the water fall model. Basically, theproject adopted the object oriented approach and mainly
used the UML modeling tool, JUDE[9]. Then, the code was
implemented in Java language. It took two months fromthe beginning of the project to the end of design. Then,
it took three months from the implementation to the end of
the project. So, the total amount of the duration was fivemonths. The amount of size of the data is about 450MB.
Table V shows the list of the empirical data.

Structure of Tag data input unit

input data to an input

input meta data

form generated based(e.g. user ’s name andon the tag data

remarks)

definition

check any items if you
want the system tocollect metrics

set a location of

automaticallycollecting files

Figure 7.

Input screen

data collection unit1 measures the value data in accordancewith the settings and the results are set in the form.
Figure 7 shows the input screen. The screen is automatically suggested by the Wizard and the metrics are shown in
the order of the category of the software tag.

C. Collected elements of the software tag
Since we applied CollectTag to the data collected from thepast project, we did not negotiate the contents of the software
tag with the developer. So, we selected typical elements
and the corresponding metrics. As the result, we selected
18 tag elements and 53 metrics for the project shown in
Table VI. Table VI shows the category, no., tag element,

1 We have implemented a measurement tool based on MASU[11]. It
measures some metrics (ex. CK-metrics, LOC) from Java source codes.

429

Table V
E MPIRICAL DATA COLLECTED FROM THE PROJECT
Process
Requirements

Analysis & Design

Implementation
Test
Project managementof time to input the tag data in 30 times was 70 min. and 47
sec., the average was 2min. and 17 sec., and the maximum
was 6 min. and 4 sec. Partial results for Step 2, input time
and time of automatic measurement, are shown in Table VII.

Empirical datarequirements specification, business flow diagrams,
Screen list, use case diagrams, actor definition
documents, quality requirements specification
class diagrams, collaboration diagrams,
sequence diagrams,architecture specification, entity-classdesign specification, control-class design specification
Boundary-class design specification
source codes, API document, productrepository, records of static checktest plan documents
minutes of review meeting, project management
reports, change management documents

Table VIIR ESULTS
Date of measured
2007/03/27
2007/03/30
2007/04/04
2007/04/06
2007/05/31
2007/12/12
2007/12/17

metrics, collection method and value2 for each metric. In
the column of the collection method, “auto” means that thedata are collected automatically and “manual” means thatthe data are collected manually.

2008/02/20
2008/02/25
2008/02/28
Total
Averagemaximum

D. Process of case study
The process of the case study is as follows:
Step 1:We defined the tag-data-definitions of the software
tag used for this project using CollectTag. Here, we
selected the elements in the software tag shown in
Table VI. As described in Section III-B, CollectTag provides the pre-defined tag-data-definitions.
Here, we added the 13 tag-deta-definitions whose
corresponding metrics are labeled by * in Table VI.
Step 2:Using CollectTag, we inputted the value manually
and automatically according to the input wizard.
Inputting the data manually, we collected the necessary value from the corresponding sources (empirical data) shown in Table V. Inputting the data
automatically, at the first input, we just set the file
path to the corresponding element.
Step 3:We output the software tag of the project.
One of the authors conducted the Steps 1, 2 and 3. To
record the time of Steps 1 and 2, we used a stop-watch. We
inputted the data totally 30 times for the 5 months project,which means the interval of input is about 3 days.
In Step 1, we only made the additional tag-data-definitions
for metrics collected manually by using the tag-datadefinition unit, since the tag-data-definitions corresponding
to the data collected automatically have already stored inCollectTag.
We evaluated the generation cost of a software tag in termsof the amount of time to generate tag data definitions, input
tag data manually, and measure the metrics automatically.

Input time
0:01:54
0:02:54
0:02:27
0:01:45...
0:01:09
0:06:04
0:03:07...
0:02:34
0:02:15
0:02:01

Time for automatic collection
0:00:01
0:00:00
0:00:00
0:00:01

1:10:47
0:02:17
0:06:04

0:14:28
0:00:28
0:01:18

0:00:02
0:00:41
0:00:55
0:00:52
0:00:52
0:00:49

F. Evaluation
In Step 1, in order to add 13 tag-data-definitions, we spent
9 min. and 50 sec. Though adding one tag-data-definition
spent 46 sec. averagely, it was just conducted only once atfirst. So, compared to the time of Step 2, it was not so muchcost, we consider.
Since the requirement analysis and design were conductedfrom March 27 to May 31, 2007, the source codes did notexist. Also, defects data were not registered to KAGEMAI.
So, during the duration, automatic measurements were justfor the elements of requirement and design categories andthe time was about 1 sec. shown in Table VII.
On the other hand, the time for measurements became
longer from Dec. 12, 2007. Especially, on Dec. 12, it spent
about 6 min. Because the programming started at the time.
In order to measure metrics from the source code, it isnecessary to spend time to export all source codes from theSubversion repository and execute the several measurement
tools. It was also necessary to set the file path to thecorresponding element but the setting was just conducted
only once. So, from Dec. 17, the time became shorter (about
3 min.).
The average time was 2 min. and 17 sec. Here, we
consider the cost to generate the software tag in one month.
Usually, in software development projects, the project manager asks the project members to submit some reports (daily
or weekly) to grasp the progress situation. So, if we assume
that the reports are submitted weekly and the data for thesoftware tag are inputted in time for the weekly reportsubmission, the total times of input data becomes 4. If thetime for each software tag data input needs 2 min. and

E. Result
In Step 1, the amount of time we spent to define 13 tagdata-definitions was 9 min. and 50 sec. In Step 2, the amount
2 The final value at the end of the project.

430

Table VI
C OLLECTED ELEMENTS OF THE SOFTWARE TAG IN THE CASE STUDY
Category
Requirements

No.
14

Tag element
Scale[Transition]

15

Revisions[Transition]

Scale[Transition]
DesignProgramming

Test

Quality

16

17

Revisions[Transition]

19
20

Scale[Transition]
Revisions[Transition]

21

Complexity

22

Scale[Transition]

24

Density25

Progress Status

26

Review Status

27
28
29
30
31
32
33

Review Density
Review Effectiveness[Transition]
Defect Count[Transition]
Fixed Defect Count[Transition]
Defect Density
Defect Detection Rate
Static Check Results

Metrics
Number of usecase diagram
Number of usecase
Number of actor
Number of defined-requirements*
Number of added usecase diagram
Number of added usecase
Number of added actor
Number of deleted usecase diagram
Number of deleted usecase
Number of deleted actor
Number of screen*
Number of class diagram
Number of sequence diagram
Number of state machine diagram
Number of activity diagram
Number of added class diagram
Number of added sequence diagram
Number of added state machine diagram
Number of added activity diagram
Number of deleted class diagram
Number of deleted sequence diagram
Number of deleted state machine diagram
Number of deleted activity diagram
Lines of codeLines of Lines of added or deleted codeLines of added codeLines of deleted code
Number of file modification
CK metrics.WMC (average)
CK metrics.NOC (average)
CK metrics.CBO (average)
CK metrics.RFC (average)
CK metrics.LCOM (average)
CK metrics.DIT (average)
Number of unit test item*
Number of integration test item*
Number of system test item*
Number of unit test item / Lines of code
Number of system test item / Number of usecase
Number of consumed system test item / Number of system test item
Number of consumed system test item*
Number of review*
Total review time*
Number of review / Total review time
Number of defect found by review*
Number of defects found by test
Number of fixed defects
Number of defects / Lines of code
Number of defects / Number of consumed system test item
Number of check by checkstyle*
Number of check by findbugs*
Number of check by pmd*
Number of check by pmd-cpd*

17 sec., the total time becomes 0.167 (hours) (=9 min.
and 8 sec. ). Considering one person-month is 160 hours(8hours × 20 days), the cost to input tag data becomes about0.001(person-month). As the results, we consider that using
CollectTag would be useful to generate the software tag at
low cost.

Collection method
auto
auto
auto
manual
auto
auto
auto
auto
auto
auto
manual
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
auto
manualmanualmanual
auto
auto
auto
manualmanualmanualauto
manual
auto
auto
auto
auto
manualmanualmanualmanual

Value
12
25
7
7
12
37
7
0
12
0
23
72
20
0
2
83
23
0
2
11
3
0
0
26033
92303
62586
29717
2249
6.278
0.739
10.43
13.00
10.96
2.81
238
75
508
0.009
20.32
1
508
22
50.75
0.43
529
19
19
0.00073
0.037
359
52
387
5

provide as many pre-defined tag-data-definitions as possible
to the user.
As described in Section IV-F, most time of the automatic
measurement spent to export source codes from the repository and execute analysis tools. In this case study, the final
program size was about 26033 LOC. The more the size
becomes, the much time would be needed for the automatic
measurement. Hence, it is necessary to evaluate the cost ofautomatic measurement against the large scale software. Justfor information, we applied analysis tools to an open source
program whose LOC is about 210 thousand and it took about
7 minutes to calculate the same metrics in this case study.

G. Threats to Validity
There are some threats to validity in the case study.
In this case study, we added only 13 tag-data-definitions
and so the cost to define them was not so high. However, inpractice, much more tag-data-definitions might be added. For
example, if we collect more than 100 elements, we would
take much effort to add them manually. So, it is necessary toAlso, if the user of CollectTag has not enough knowledge431

about the project, much more time would be needed. In
this case study, the subject had enough knowledge about the
project data. For example, he knew where the requirements
specification, minutes of review meeting, the UML model
files, the results of static checking and so on. So, he took
only a second to get the necessary data to input CollectTag.
In that sense, the user of CollectTag must be the person who
knows the detailed of the target project.

will make a template of the contract document, considering
software tag and legal issues of software development.
ACKNOWLEDGMENT
This work is being conducted as a part of the StagE
project, The Development of Next-Generation IT Infrastructure, supported by the Ministry of Education, Culture,
Sports, Science and Technology. We are grateful to themembers of the StagE project. Also, we would like to thankthe support of Japan Society for the Promotion of Science,
Grant-in-Aid for Scientific Research (C) (No.20500033)

V. C ONCLUSION
This paper describes a system, named CollectTag, which
supports to generate the software tag which makes softwaredevelopment visible to software purchaser (users).
CollectTag has versatility in the sense of allowing the user
to define software tag definitions for his/her own project.
It provides automatic measurement mechanisms to collectmetrics for some elements of the software tag and reduces
the cost of collecting data. Finally, it outputs the software
tag in the XML format and provides a convenient software
tag for some tools that analyze the tag.
As a case study, we have applied CollectTag to theactual software development project data and evaluated theusefulness of it from the viewpoint of cost to generate
the software tag. As the results, we found that by using
CollectTag, generating the software tag needs not high-cost.
We have to improve the functionality and usability of CollectTag. At first, we are going to enrich the pre-defined tagdata-definitions. Insufficient pre-defined tag-data-definitions
urges the user to input additional tag-data-definitions. Next,
we have to implement the display form for the user intuitively to understand the software tag anytime as the financial
statements of company evaluations or inspection results ofthe complete medical checkup. But, it is necessary to collectenough data to set the standard or limiting value for each
metric of each tag element. This can be achieved when thesoftware traceability becomes widely used and the software
tag is commonly used in the software development projects.
Finally, the evaluation of CollectTag is essential in thecontext of the actual software development projects. We aregoing to ask some companies that support the StagE project
to use CollectTag and also release CollectTag from the Web
site of the StagE project. Based on the feedbacks throughthe evaluations, we are going to improve the applicability ofCollectTag.
As the StagE project, we will focus on making international/domestic standards of the software tag. With suchstandardization, the software tag is expected to be used invarious software industries, where we think it will strongly
promote participation and understanding of software development by purchasers. Also, to reduce adaptation cost of
the software tag, we will delivery software tag support tools
and the software tag guidebook which explains how to use
the software tag. That would accelerate incorporating thesoftware tag scheme in the industrial practices. Moreover, we

R EFERENCES[1] S. R. Chidamber and C. F. Kemerer: “A Metrics Suite for
Object Oriented Design”, IEEE Trans. Software Eng., 20, 6
pp. 476-493, 1994.
[2] http://checkstyle.sourceforge.net/[3] http://findbugs.sourceforge.net/[4] K. Fushida, J. Takata, T. Yonemitsu, Y. Fukuchi, S. Kawaguchi,
and H. Iida: “A Framework and System for Planning andTailoring Software Development Processes with Quantitative
Management,” Proc. of the 4th International Conference on
Project Management (ProMAC 2008), pp.286-294, 2008.
[5] http://www.gnu.org/software/gnats/[6] K. Inoue: “Software Tag for Traceability and Transparency ofMaintenance”, Proc. of 24th IEEE International Conference on
Software Maintenance, pp. 476-477, 2008.
[7] ISO/IEC 12207: Systems and software engineering - Software
life cycle processes. International Organization for Standardization, 2008.
[8] M. Barker and K. Inoue: “IT SPIRAL: A Case Study in
Scalable Software Engineering Education”, Proc. of the 22nd
Conference on Software Engineering Education and Training,pp. 53-60, 2009.
[9] http://jude.change-vision.com/jude-web/index.html[10] http://www.daifukuya.com/kagemai/ (in Japanese)[11] http://masu.sourceforge.net/[12] Nikkei Business Publications, Inc.: “Survey Report on IT
Investment and CTO in Japan”, Nikkei Information Strategy,
2008(in Japanese).
[13] Nikkei Business Publications, Inc.: “Second Survey ofJapanese Software Projects”, Nikkei Computer, Dec. 1, pp. 3653, 2008(in Japanese).
[14] http://pmd.sourceforge.net/[15] M. Tsunoda, T. Matsumura, H. Iida, K. Kubo, S. Kusumoto,
K. Inoue and K. Matsumoto: “Standardizing the Software Tag
in Japan for Transparency of Development”, Proc. of 11th
International Conference on Product Focused Software Development and Process Improvement, No.LNCS 6156, pp.220233, 2010.

432