# A Comprehensive MCDM-Based Approach for Object-Oriented Metrics Selection Problems
## 5 Validation
### Validation Methodology  
In [12], the main objective of the study was to extract rules based on the decision tree algorithm. The rules are of the form:  
For a defect D: \"IF metric1 is higher/lower than threshold1 AND metric2 higher/lower than threshold2.... AND metric higher/lower than threshold THEN defect D is suspected\". As an example, we present three rules, R1, R2, and R3, generated for the detection of the Data Class defect:  
R1: IF ATFD <= 16.5 THEN Data class = Yes  
R2: IF ATFD > 16.5 AND NOA > 11.25 THEN Data class = Yes  
R3: IF ATFD > 16.5 AND NOA <= 11.25 AND NC > 251 THEN Data class = Yes.  
R1 and R2 combine a few numbers of metrics resulting in the generation of a huge number of suspect classes. Therefore, R3 is the most appropriate rule to be considered as an illustrative example. In [12], we considered that the number of metrics is the only criteria that matters for the rule selection. In fact, the process of selecting rules was based on a parameter N fixed by the tester. This parameter is estimated through a series of tests  
\begin{table}
\begin{tabular}{c c c c c c} \hline \hline  & **Number of Classes** & **Number of Blob** & **Number of LC** & **Number of DC** & **Number of FE** \\ \hline
**Xerces v2.7** & 683 & 29 & 23 & 17 & 58 \\ \hline
**ArgoUML 0.19.8** & 1244 & 15 & 29 & 15 & 78 \\ \hline
**Lucene 1.4** & 189 & 7 & 8 & 2 & 23 \\ \hline
**Log4j 1.2.1** & 209 & 11 & 6 & 5 & 33 \\ \hline
**GanttProject v1.10.2** & 241 & 22 & 12 & 10 & 46 \\ \hline \hline \end{tabular}
\end{table}
Table 10: Project information.  
on the base of examples. It represents the number of metrics in rule detection. The value of N directly affects the accuracy of the detection. A small value will generate a high number of false positives due to over-detection; we will detect more than the real existent defects. However, we get the opposite result for a high N value; it will generate a high number of false negatives and we will detect a very small number of defects compared to the existing one.  
Based on Fuzzy DEMTEL and the results shown in Table 9, we follow an alternative approach. For instance, R1 contains only one metric from the effect group, so it cannot be considered as an important rule. The decision of not considering the rule R1 as important is not based on the number of metrics but on the fact that rule should combine both cause and effect metrics. For the rules R2 and R3, they combine cause and effect important and highly important metrics. As matter of fact, the rules R2 and R3 are important rules that DEMTEL suggests to include in the evaluation process.  
This experiment is based on the same set of rules generated by the decision tree algorithm proposed in [12]. In this paper, we select rules based on the metrics importance presented in Table 9, instead of selecting rules based on the N parameter as presented in [12].  
The selected rules for defect detection give priority to the rules that first contain both cause and effect metrics. We start by selecting rules including first the important and high important metrics. If we get a small set of rules, we include those containing only effect or cause metrics, but we choose rules with important and high important metrics.  
We use the precision and recall measurements to validate our findings. To evaluate the correctness of the approach, we calculate the precision. It represents the fraction of the true design defects among the set of all detected defects (6). The precision measures the number of true identified defects. The fraction of correctly detected design defects over the set of expected defects is the recall (7).  
\[\text{Precision}=\frac{\text{Detected defects}\cap\text{Expected defects}}{\text{Detected defects}} \tag{6}\]  
\[\text{Recall}=\frac{\text{Detected defects}\cap\text{Expected defects}}{\text{Expected defects}} \tag{7}\]