## 8 Experiments and Result Analysis
### Result Analysis
From the detection results, [PERSON] et al. [6] generally maintained the balance between precision and recall, but the precision and recall are very low except that the recall of JRefactory 2.6.24 reached 90%. In addition, the calling rules between classes of design patterns are more complex and flexible. The methods of [PERSON] et al. [9], [PERSON] et al. [7], and [PERSON] et al. [6] are generally not good in identifying behavioral patterns, and none of the three methods can distinguish between the Adapter/Command patterns and State/Strategy patterns. In this study, the powerful automatic feature extraction capability of deep learning combined with the advantages of SVM on binary classification problems was leveraged, resulting in improved precision and recall and a good distinction between the Adapter/Command patterns and State/Strategy pattern, and the average precision/recall on the three open source projects reached 94.1%/94.1%, 90.6%/89.6%, and 90.3%/100.0%, respectively. In this paper, experiments were also conducted on the three open-source programs with the application of six design pattern detection methods based on traditional machine learning algorithms, proposed by [PERSON] et al. [27], [PERSON] et al. [28], [PERSON] et al. [31; 32], [PERSON] et al. [25], and [PERSON] et al. [36]. All these methods require manual extraction and selection of features. Among them, [PERSON] et al. [27] selected 12 metric features, such as whether being an abstract class, whether being an interface, the number of methods, the number of generalization relation sources, and the number of generalization relation targets. For each design pattern, the model was trained by using three classification algorithms: KNN, C4.5 decision trees, and SVM. [PERSON] et al. [28] selected 64 metric features such as class interface width, number of attributes, number of classes, number of constructors, loop complexity, fan-out, etc., and they used SVM-PHGS [56] to train classifiers for patterns of Adapter, Builder, Composite, Factory Method, Iterator, and Observer. For each role of five patterns, including Singleton, Template Method, Adapter, State and Strategy, [PERSON] et al. [31; 32]  
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline \multirow{2}{*}{**Design Pattern**} & \multicolumn{2}{c}{**[PERSON] et al.’s Method**} & \multicolumn{2}{c}{**[PERSON] et al.’s Method**} & \multicolumn{2}{c}{**[PERSON] et al.’s Method**} & \multicolumn{2}{c}{**Our Method**} \\ \cline{2-9}  & _Precision_ & _Recall_ & _Precision_ & _Recall_ & _Precision_ & _Recall_ \\ \hline Adapter & & & & & & & \\ \hline Command & 100.0\% & 100.0\% & 50.0\% & 100.0\% & 50.0\% & 100.0\% & 100.0\% & 100.0\% \\ \hline Composite & 100.0\% & 100.  
selected features such as number of static domains, number of private constructors, number of methods, number of static methods, number of abstract methods, number of overriding methods, number of interfaces, number of domains, number of object domains, and number of methods for generating instances. They made an input of pattern-applied programs into the measuring system to obtain the metrics of each pattern role, and these metrics were put into the ANN simulator for learning to train the classifiers. [PERSON] et al. [25] considered entities (classes/interfaces/objects) and relationships between entities. They reduced the learning of composite records (classes) to some basic models by clustering training records (classes) with related attributes/relationships and then built decision trees based on the clustered training samples. [PERSON] et al. [36] considered 69 metric features such as the weighting method of classes, line of code, loop metric, number of changes in access methods, as well as 16 structural features including overriding method, create object, return type, delegation, multiple redirections in the family, and redirect in the family. For each design pattern, a classifier was trained with metric and microstructure, respectively, and then the final classifier was trained by model stacking.