Hindawi Publishing Corporation
Advances in Software Engineering
Volume 2012, Article ID 924923, 8 pages
doi:10.1155/2012/924923

Research Article
Can Faulty Modules Be Predicted by Warning Messages of
Static Code Analyzer?
Osamu Mizuno and Michi Nakai
Kyoto Institute of Technology, Matsugasaki Goshokaido-cho, Sakyo-ku, Kyoto 606-8585, Japan
Correspondence should be addressed to Osamu Mizuno, o-mizuno@kit.ac.jp
Received 5 January 2012; Accepted 24 February 2012
Academic Editor: Chin-Yu Huang
Copyright © 2012 O. Mizuno and M. Nakai. This is an open access article distributed under the Creative Commons Attribution
License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly
cited.
We have proposed a detection method of fault-prone modules based on the spam filtering technique, “Fault-prone filtering.” Faultprone filtering is a method which uses the text classifier (spam filter) to classify source code modules in software. In this study, we
propose an extension to use warning messages of a static code analyzer instead of raw source code. Since such warnings include
useful information to detect faults, it is expected to improve the accuracy of fault-prone module prediction. From the result ofexperiment, it is found that warning messages of a static code analyzer are a good source of fault-prone filtering as the original
source code. Moreover, it is discovered that it is more eﬀective than the conventional method (that is, without static code analyzer)
to raise the coverage rate of actual faulty modules.

1. Introduction
Recently, machine learning approaches have been widelyused for fault-proneness detection [1]. We have introduced
a text feature-based approach to detect fault-prone modules[2]. In this approach, we extract text features from the frequency information of words in source code modules. In otherwords, we construct a large metrics set representing the frequency of words in source code modules. Once the text features are obtained, the Bayesian classifier is constructed fromtext features. In the fault-prone module detection of newmodules, we also extract text features from source code modules, and Bayesian model classifies modules into either faultprone or nonfault-prone. Since less eﬀort or cost needed tocollect text feature metrics than other software metrics, it
may be applied to software development projects easily.
On the other hand, since this approach accepts any input
with text files, the accuracy of prediction could be improvedby selecting appropriate input other than raw source code.
We then try to find another input but source code. In thisstudy, we use warning messages of a static code analyzer.
Among many static code analyzers, we used PMD in thisstudy. By replacing the input of fault-prone filtering from raw

source code to warning messages of PMD, we can get the results of prediction by PMD and fault-prone filtering.
The rest of this paper is organized as follows. Section 2
describes the objective of this research. Section 3 shows abrief summary of the fault-prone filtering technique with
PMD. In Section 4, the experiments conducted in this study
are described. Section 5 discusses the result of the experiments. Finally, Section 6 concludes this study.

2. Objective
2.1. Fault-Prone Module Filtering. The basic idea of faultprone filtering is inspired by the spam mail filtering. In the
spam e-mail filtering, a spam filter first trains both spam andham e-mail messages from the training data set. Then, an
incoming e-mail is classified into either ham or spam by thespam filter.
This framework is based on the fact that spam e-mailusually includes particular patterns of words or sentences.
From the viewpoint of source code, a similar situation
usually occurs in faulty software modules. That is similar
faults may occur in similar contexts. We thus guessed thatsimilar to spam e-mail messages, faulty software modules

2

Advances in Software Engineering

have similar patterns of words or sentences. To obtain suchfeatures, we adopted a spam filter in fault-prone module
prediction.
In other words, we try to introduce a new metric as afault-prone predictor. The metric is “frequency of particular
words.” In detail, we do not treat a single word, but use combinations of words for the prediction. Thus, the frequency
of a certain length of words is the only metric used in our
approach.
From a viewpoint of eﬀort, conventional fault-prone detection techniques require relatively much eﬀort for application because they have to measure various metrics. Of course,
metrics are useful for understanding the property of sourcecode quantitatively. However, measuring metrics usuallyneeds extra eﬀort and translating the values of metrics intomeaningful result also needs additional eﬀort. Thus, easy-touse technique that does not require much eﬀort will be useful
in software development.
We then try to apply a spam filter to identification of
fault-prone modules. We named this approach as “faultprone filtering.” That is, a learner first trains both faulty andnonfaulty modules. Then, a new module can be classified
into fault-prone or notfault-prone using a classifier. In thisstudy, we define a software module as a Java class file.
Essentially, the fault-prone filtering does the text classification on the source codes. Of course, the text classification
can be applied to the text information other than the source
codes. We guessed that there is the other input for the textclassification to achieve higher prediction accuracy. We thenstarted seeking such information.

rich information about potential faults in source codes.
Figure 1 shows an example of warning messages. Usually, thenumber of warning messages generated by the static codeanalyzer becomes large in proportion to the length of sourcecode. Since most of the messages are not harmful or trivial,
warning messages are often ignored. It can be considered thatthese warning messages are quality aspects of the sourcecode. Thus, we consider that the warning messages have lessnoise for fault-prone module prediction.
As mentioned in Section 2.1, applying the text information to the text classifier is an easy task. We thus implement
the fault-prone filtering technique to use the warning messages of the static code analyzer. We then conduct experiments to confirm the eﬀects of the warning messages to theperformance of the fault-prone filtering approach.

2.2. Static Code Analysis. The static code analysis is a methodof analyzing without actually running software and finding
the problem and faults in a software. By analyzing a sourcecode structurally, we can find potential faults, violation ofcoding conventions, and so on. The static code analysis thus
can assure the safety of software, reliability, and quality. It
also reduces the cost of maintenance. In recent years, theimportance of static code analysis has been emerging sincefinding potential faults or security hole is required at an earlystage of the development. There are many kinds of tools forthe static code analysis available [3]. Among them, we used
the PMD (the meaning of PMD is not determined. “We havebeen trying to find the meaning of the letters PMD—because
frankly, we do not really know. We just think the letters soundgood together” [4]), since it can be applicable to the sourcecode directly.
The PMD is one of static code analysis tools [5]. It is an
open-source software and written in Java, and it is used foranalyzing programs written in Java. PMD can find the code
pieces that may cause the potential faults such as an unusedvariable and an empty catch block by analyzing the sourcecode of Java. To do so, PMD has a variety of rule sets. According to the rule sets to be used, a broad range of purposes fromthe inspection of coding conventions to find potential faults
can be used.

3. Fault-Prone Filtering with PMD

2.3. Characteristics of the Warning Messages of the Static CodeAnalyzer. Warning messages of a static code analyzer include

2.4. Research Questions. In this study, we aim at answering
the following research questions:
RQ1: “can fault-prone modules be predicted by applying a text filter to the warning messages of a static
code analyzer?”
RQ2: “if RQ1 is true, is the performance of the faultprone filtering becomes better with the warning messages of a static code analyzer?”
RQ1 tries to find a possibility to apply the warning messages to the fault-prone filtering technique. RQ2 investigates
the prediction performance.

3.1. Applying PMD to Source Code. We used 10 rule sets of
PMD in a standard rule sets: Basic, Braces, Code Size, Coupling, Design, Naming, Optimizations, Strict Exception,
Strings, and Unused Code. These rule sets are frequently used
for investigation of the quality of software. We apply PMD
with 10 rule sets to all source code modules and get warning
messages of PMD.
3.2. Classification Techniques. In this study, we used CRM114(the controllable regex mutilator) spam filtering software [6]for its versatility and accuracy. Since CRM114 is implemented as a language to classify text files for general purpose, applying source code modules is easy. Furthermore, the classification techniques implemented in CRM114 are based
mainly on Markov random field model instead of the naive
Bayesian classifier.
In this experiment, we used the orthogonal sparse bigrams Markov model built in CRM114.
Orthogonal Sparse Bigrams Markov model (OSB)
Basically, CRM114 uses sparse binary polynomial
Hash Markov model (SBPH). It is an extension of theBayesian classification, and it maps features in the
input text into a markov random field [7]. In thismodel, tokens are constructed from combinations of
n words (n-grams) in a text file. Tokens are thenAdvances in Software Engineering

3

The class “ISynchronizerTest” has a Cyclomatic Complexity of 8 (Highest = 32).
This class has too many methods, consider refactoring it.
Avoid excessively long variable names like NUMBER OF PARTNERS
The field name indicates a constant but its modifiers do not
Variables should start with a lowercase character
Variables that are not final should not contain underscores (except forunderscores in standard prefix/suﬃx).
Document empty constructor
Parameter “name” is not assigned and could be declared final
Avoid variables with short names like b1
Avoid variables with short names like b2
Parameter “b1” is not assigned and could be declared final
Parameter “b2” is not assigned and could be declared final
Parameter “message” is not assigned and could be declared final
Avoid using for statements without curly braces
Local variable “body” could be declared final
Parameter “monitor” is not assigned and could be declared final
Parameter “resource” is not assigned and could be declared final
Avoid using for statements without curly braces...
Figure 1: A part of warning messages by PMD from a source code module of Eclipse.

mapped into a Markov random field to calculate theprobability.
OSB is a simplified version of SBPH. It considers tokens as combinations of exactly 2 words created in
the SBPH model. This simplification decreases both
memory consumption of learning and time of classification. Furthermore, it is reported that OSB usuallyachieves higher accuracy than a simple word tokenization [8].
3.3. Tokenization of Inputs. In order to perform fault-prone
filtering approach, inputs of fault-prone filter must be tokenized. In this study, in order to use the warning messages of
PMD as input of filtering, the messages need to be tokenized.
Warning messages of PMD contains English text in natural
language and a part of Java code. In order to separate them,
we classified them into the following kind of strings:(i) strings that consist of alphabets and numbers;(ii) all kinds of brackets, semicolons, commas;(iii) operators of Java and dot;(iv) other strings (natural language message).
Furthermore, warning messages of PMD have file names
and line numbers on the top of each line. In usual, they provide useful information for debug, but for learning and classification, they may mislead the learning of faulty modules.
For example, once we learn a line number of a faulty module,
the same line number of the other file is wrongly considered
as faulty token.
3.4. Example of Filtering. Here, we explain briefly how theseclassifiers work. We will show how to tokenize and classify
the faulty modules in our filtering approach.

1:
2:
3:
4:

if x
==
if
if
1
if
returnFigure 2: Example of tokens for OSB using the source code.

1:
2:
3:

underscores
underscores
underscoresinstandard
prefix/suﬃx)

Figure 3: Example of tokens for OSB using the warning messages.

3.4.1. Tokenization. In OSB, tokens are generated so thatthese tokens include exactly 2 words. For example, a sentence“if (x == 1) return;” is tokenized as shown in Figure 2 By
definition, the number of tokens drastically decreases compared to SBPH. As for the warning messages, an example ofa sentence “underscores in standard prefix/suﬃx).” is shownin Figure 3
3.4.2. Classification. Let TFP and TNFP be sets of tokens included in the fault-prone (FP) and the nonfault-prone (NFP)
corpuses, respectively. The probability of fault-proneness isequivalent to the probability that a given set of tokens Tx is
included in either TFP or TNFP . In OSB, the probability thata new module mnew is faulty, P(TFP |Tmnew ), with a given
set of token Tmnew from a new source code module mnew iscalculated by the following Bayesian formula:

P Tmnew TFP P(TFP ).
P Tmnew TFP P(TFP ) + P Tmnew TNFP P(TNFP )




(1)

4

Advances in Software Engineering
Table 1: Target project: Eclipse BIRT plugin.

Name
Language
Revision control
Type of faults
Status of faults
Resolution of faults
Severity
Priority of faults
Total number of faults

Eclipse BIRT plugin
Java
cvs
Bugs
Resolved;
Verified;closedFixedBlocker; critical;
major; normal
All
4708

Table 2: The number of modules in Eclipse BIRT.

Nonfaulty
FaultyTotal

Number of modules (files)
42,50327,641
70,144

Intuitively speaking, this probability denotes that the newcode is classified into FP. According to P(TFP | Tmnew ) and
predefined threshold tFP , classification is performed.

4. Experiment4.1. The Outline of the Experiment. In this experiment, warning messages of PMD are used for fault-prone filtering as an
input instead of a source code module, and Fault-prone
module is predicted. And it is the purpose to evaluate thepredictive accuracy of the proposed method. Therefore, two
experiments using raw source code modules and the warning
messages by the PMD as inputs are conducted. We then compare these results to each other.
4.2. Target Project. In this experiment, we use the source code
module of an open source project, Eclipse BIRT (businessintelligence and reporting tools). The source code module isobtained from this project by the SZZ (Śliwerski et al.) algorithm [9]. The summary of Eclipse BIRT project is shown in
Table 1. All software modules in this project are used for bothlearning and classification by the procedure called training
only errors (TOE). The number of modules is shown in
Table 2.
4.3. Procedure of Filtering (Training on Errors). Experiment 1
performs the original fault-prone module prediction using
the raw source code and OSB classifier by the following
procedures:(1) apply the FP classifier to a newly created softwaremodule (say, method in Java, function in C, and soon), Mi , and obtain the probability to be fault-prone;

Modules sorted by date

Get next
moduleFP classifier
Corpus ofFP
Corpus of
NFP

Prediction(FP/NFP)
Correct
prediction?

Yes

No
Training

FP trainer

Figure 4: Outline of fault-prone filtering by training on errors.

(2) by the predetermined threshold tFP (0 < tFP < 1),classify the module Mi into FP or NFP;
(3) when the actual fault-proneness of Mi is revealed byfault report, investigate whether the predicted resultfor Mi was correct or not;(4) if the predicted result was correct, go to step 1;
otherwise, apply FP trainer to Mi to learn actual faultproneness and go to step 1.
This procedure is called “training on errors (TOE)”
procedure because training process is invoked only whenclassification errors happen. The TOE procedure is quite
similar to actual classification procedure in practice. For
example, in actual e-mail filtering, e-mail messages are classified when they arrive. If some of them are misclassified,
actual results (spam or nonspam) should be trained.
Figure 4 shows an outline of this approach. At this point,
we consider that the fault-prone filtering can be applied tothe sets of software modules which are developed in the same(or similar) project.
Experiment 2 is an extension of Experiment 1 by appending additional steps as the first step as follows:(1) obtain warning messages Wi of PMD by applying
PMD to a newly created software module Mi ;(2) apply the FP classifier to the warning messages, Wi ,and obtain the probability to be fault-prone;(3) by the predetermined threshold tFP (0 < tFP < 1),classify the warning messages Wi into FP or NFP;
(4) when the actual fault-proneness of Mi is revealed byfault report, investigate whether the predicted result
for Wi was correct or not;(5) if the predicted result was correct, go to step (1); otherwise, apply FP trainer to Wi to learn actual faultproneness and go to step (1).

Advances in Software Engineering

5

Table 3: Classification result matrix.

Table 4: Result of prediction in Experiment 1 (without PMD).

Prediction
ActualNonfaulty
FaultyNonfault-prone
True negative (tn)False negative (fn)Fault-prone
False positive ( f p)
True positive (t p)4.4. Procedure of TOE Experiment. In the experiment, wehave to simulate actual TOE procedure in the experimental
environment. To do so, we first prepare a list of all modules
found in Section 4.2. The list is sorted by the last modified
date (di ) of each module so that the first element of the list
is the oldest module. We then start simulated experiment in
the procedure shown in Algorithm 1. During the simulation,
modules are classified by the order of date. If the predictedpresult si diﬀers from actual status sai , the training procedure
is invoked.
4.5. Evaluation Measures. Table 3 shows a classification result
matrix. True negative (tn) shows the number of modules thatare classified as nonfault-prone, and are actually nonfaulty.
False positive ( f p) shows the number of modules that areclassified as fault-prone, but are actually nonfaulty. On thecontrary, false negative shows the number of modules thatare classified as nonfault-prone, but are actually faulty. Finally, true positive shows the number of modules that are classified as fault-prone which are actually faulty.
In order to evaluate the results, we prepare two measures:
recall and precision. Recall is the ratio of modules correctlyclassified as fault-prone to the number of entire faulty modules Recall is defined as t p/(t p + f n). Precision is the ratio of
modules correctly classified as fault-prone to the number ofentire modules classified fault-prone. Precision is defined ast p/(t p+ f p). Accuracy is the ratio of correctly classified modules to the entire modules. Accuracy is defined as (t p +tn)/(tn + t p + f p + f n). Since recall and precision are in the
trade-oﬀ, F1 -measure is used to combine recall and precision[10]. F1 -measure is defined as (2 × recall × precision)/(recall+precision). In this definition, recall and precision are evenly
weighed.
From the viewpoint of the quality assurance, it is recommended to achieve higher recall, since the coverage of actual
faults is of importance. On the other hand, from the viewpoint of the project management, it is recommended to focus
on the precision, since the cost of the software unit test is
deeply related to the number of modules to be tested. In thisstudy, we mainly focus on the recall from the viewpoint of
the quality assurance.
4.6. Result of Experiments. Tables 4 and 5 show the result ofexperiment using the original approach without PMD andthe approach with PMD, respectively. Table 6 summarizes theevaluation measures for these experiments.
From Table 6, we can see that the approach with PMD has
almost the same capability to predict fault-prone modules
as the approach without PMD. For example, F1 for the approach without PMD is 0.779, and for the approach with

Prediction
ActualNonfault-prone
30,521
2,360
32,821

Nonfaulty
FaultyTotal

Fault-prone
11,98225,281
37,263

Table 5: Result of prediction in Experiment 2 (with PMD).
Prediction
ActualNonfault-prone
22,982
1,674
24,656

Nonfaulty
FaultyTotal

Fault-prone
19,521
25,96745,488

Table 6: Evaluation measures of the results of experiments.
Precision

RecallAccuracy

F1

0.678

0.915

0.796

0.779

0.571

0.939

0.698

0.710

Experiment 1(without PMD)
Experiment 2(with PMD)

PMD is 0.710. The result shows that the original approachwithout PMD is relatively better than the approach with
PMD in precision, accuracy, and F1 measures. The recall of
the approach with PMD is better than the approach without
PMD.
Figures 5 and 6 show the result of TOE history for theapproaches without and with PMD, respectively. From thisgraph, we can see that evaluation measures first to decrease at
the beginning of TOE procedure, then increase and become
stable after learning and classification of 15,000 modules.

5. Discussions
At first, we discuss the advantage of the approach with PMD.
From Table 6, we can see that the result of Experiment 2 hashigher recall and lower precision than that of Experiment 1.
Generally speaking, the recall is an important measure for thefault-prone module prediction because it implies how many
actual faults can be detected by the prediction. Therefore,
higher recall can be an advantage of the approach with PMD.
However, the diﬀerence of the recalls between two experiments is rather small.
When we focus on the graphs of TOE histories shown in
Figures 5 and 6, the diﬀerence between two experiments canbe seen clearly. The transition of recall in Experiment 2 keeps
higher than that of Experiment 1 from an early stage of theexperiment. That is the recall of Experiment 2 reaches 0.90 at
10,000 modules learning. From this fact, we can say that theapproach with PMD is eﬃcient especially at an early stage ofdevelopment. It can be considered as another advantage ofthe approach with PMD.
We discuss the reasons of the result that the approachwith PMD does not shows a good evaluation measures at6Advances in Software EngineeringtFP : Threshold of probability to determine FP and NFPpsi : Predicted fault status (FP or NFP) of Mi
for each Mi in list of modules sorted by di ’s
prob = fpclassify(Mi )p
if prob > tFP then si = FPpelse si = NFP
endif
p
a
if sai =/ si then fptrain(Mi , si )endif
endfor
fpclassify(M) :if Experiment 1 thenGenerate a set of tokens TM from source code M.
Calculate probability P(TFP | TM )using corpuses TFP and TNFP .
Return P(TFP | TM ).
if Experiment 2 thenGenerate a set of tokens TW
from warning messages W
by applying PMD to the source code M.
Calculate probability P(TFP | TW )using corpuses TFP and TNFP .
Return P(TFP | TW )fptrain(M, sa ) :if Experiment 1 thenGenerate a set of tokens TM from M.
Store tokens TM to the corpus Tsa .
if Experiment 2 thenGenerate a set of tokens TW from W
by applying PMD to M.
Store tokens TW to the corpus Tsa .
Algorithm 1: Procedure of TOE experiment.

Training only errors (Eclipse BIRT, OSB, source)
1

Evaluation measures

0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0

0

10,000 20,000 30,000 40,000 50,000 60,000 70,000
Number of modules(Source) accuracy(Source) precision(Source) recall(Source) F1

Figure 5: History of training on errors procedure in Experiment 1(without PMD).

the end of the experiment. First, the selection of rule sets
used in PMD may aﬀect the result of experiment. Although
we used 10 rule sets according to the past study, the selectionof rule sets should be considered more carefully. For future
research, we will investigate the eﬀects of rule set selection
to the accuracy of fault-prone filtering. Second, we need toapply this approach to more projects. We have conducted
experiments on Eclipse BIRT.
Here, we investigate the details of our prediction. Table 7
shows a part of the probabilities for each token in the corpus
for faulty modules. This table shows tokens with highest probabilities. The probability P(Tx |TFP ) shows the conditional
probability that a token Tx exists in the faulty corpus.
Although these probabilities do not mean immediately thatthese tokens make a module fault-prone, we guess that theinvestigation of these probabilities helps improving accuracy.
We can see that specific identifier such as “copyInstance”
and specific literals such as “994,” “654,” and “715” appear
frequently. It can be guessed that these literals denote line
number in a particular source code. These literals are eﬀective to predict the fault-proneness of the specific source code
modules, but it can be a noise for the most other modules.
In order to improve the overall accuracy of the classifier, eliminating literals that describe a specific source code should
be taken into account.
Finally, we answer the research questions here. We havethe following research questions in Section 2.4.

Advances in Software Engineering

7

Training only errors (PMD with Eclipse BIRT, OSB)
1

Evaluation measures

0.9

RQ1: “can fault-prone modules be predicted by applying a text filter to the warning messages of a static
code analyzer?”

0.8

For this question, we can answer “yes” from the resultsin Table 5 and Table 6. It is obvious that the approach with
PMD has prediction capability of the fault-prone modules ata certain degree.

0.7
0.6
0.5
0.4
0.3

RQ2: “if RQ1 is true, is the performance of the faultprone filtering becomes better with the warning messages of a static code analyzer?”

0.2
0.1
0

0

10,000 20,000 30,000 40,000 50,000 60,000 70,000
Number of modules(PMD) accuracy(PMD) precision(PMD) recall(PMD) F1

Figure 6: History of training on errors procedure in Experiment 2(with PMD).

Table 7: Probabilities for each token in the corpus for faulty
modules. The “” denotes any words.
P (Tx | TFP )0.562290.562290.562290.562290.562290.562290.562290.562290.562290.562290.562290.562290.562290.562290.56215
0.56215
0.56215
0.56215
0.56215
0.56215
0.56215
0.56215
0.56215
0.56215
0.56215
0.56215
0.56215
0.56215Txline 654
715 Local“copyInstance” has
654 Local
instanceof The
method copyInstance ()255 Parameter
method “copyInstance”
489 Avoidline 715
copyInstance () has
line 994
line 474
994 The715  variableblocks  “pageNumber”
on  994bb  instantiatingon  654
The  “copyInstance”
654  variable994  String
The  copyInstance ()“copyInstance”  aon  715
on  474
300  variables
copyInstance ()  an

For this question, we can say that the recall of the approach with PMD becomes higher and more stable during
the development than the approach without PMD as shownin Table 6 and Figures 5 and 6. From the viewpoint of thequality assurance, it is a preferred property. We then conclude that the proposed approach has better performance toassure the software quality.

6. Conclusion
In this paper, we proposed an approach to predict faultprone modules using warning messages of PMD and a text
filtering technique. For the analysis, we stated two researchquestions: “can fault-prone modules be predicted by applying a text filter to the warning messages of static code analyzer?” and “is the performance of the fault-prone filtering becomes better with the warning messages of a static code analyzer?” We tried to answer this question by conducting experiments on the open source software. The results of experiments show that the answer to the first question is “yes.” Asfor the second question, we can find that the recall becomes
better than the original approach.
Future work includes investigating which parts of warning messages are really eﬀective for fault-prone module
prediction. Selection of rule sets of PMD is an interesting
future research.

References[1] C. Catal and B. Diri, “A systematic review of software fault
prediction studies,” Expert Systems with Applications, vol. 36,
no. 4, pp. 7346–7354, 2009.
[2] O. Mizuno and T. Kikuno, “Training on errors experiment todetect fault-prone software modules by spam filter,” in Proceedings of the 6th Joint Meeting of the European Software Engineering Conference and the ACM SIGSOFT Symposium on theFoundations of Software Engineering, pp. 405–414, 2007.
[3] N. Rutar, C. B. Almazan, and J. S. Foster, “A comparison of bug
finding tools for java,” in Proceedings of the 15th International
Symposium on Software Reliability Engineering, pp. 245–256,
IEEE Computer Society, Washington, DC, USA, 2004.
[4] T. Copeland, PMD—What does “PMD” mean?, http://pmd.sourceforge.net/meaning.html .
[5] T. Copeland, PMD Applied, Centennial Books, Arexandria, Va,
USA, 2005.
[6] W. S. Yerazunis, CRM114—the Controllable Regex Mutilator,
http://crm114.sourceforge.net/.

8[7] S. Chhabra, W. S. Yerazunis, and C. Siefkes, “Spam filtering using a markov random field model with variable weighting
schemas,” in Proceedings of the 4th IEEE International Conference on Data Mining, (ICDM ’04), pp. 347–350, Riverside,
Calif, USA, November 2004.
[8] C. Siefkes, F. Assis, S. Chhabra, and W. S. Yerazunis, “Combining winnow and orthogonal sparse bigrams for incremental
spam filtering,” in Proceedings of the Conference on Machine
Learning/European Conference on Principles and Practice ofKnowledge Discovery in Databases (ECML PKDD ’04), 2004.
[9] J. Śliwerski, T. Zimmermann, and A. Zeller, “When do changes
induce fixes? (on Fridays),” in Proceedings of the 2nd International Workshop on Mining Software Repositories, pp. 24–28, St.
Louis, Mo, USA, May 2005.
[10] C. J. van Rijsbergen, Information Retrieval, Butterworth, Boston, Mass, USA, 2nd edition, 1979.

Advances in Software Engineering