# Automatic Classification of Web Images as UML Static Diagrams Using Machine Learning Techniques
## 6 Selection of Graphical Characteristics and Rule Induction by Machine Learning Techniques
Goal: Find a function \(f:\ \mathbb{R}^{k}\rightarrow\left\{0,\ 1\right\}\) such that it minimizes  
Figure 11: Format of the training instances used as input to the learning algorithm. Testing instances have the same format. \[\sum_{i=1}^{n}|\ f(g_{1}(p_{i}),\ldots,g_{k}(p_{i}))-c_{i}|\]  
There are several kinds of machine learning techniques. Among them, neural networks are the most widely used today as a de facto standard tool in computer vision. However, as is well known, one of the main drawbacks of neural networks is that they provide black boxes whose only justification for the functions obtained is the effectiveness they reach. Therefore, we selected a different machine learning technique, namely rule induction, because it provides a human-readable decision algorithm that improves the explainability and modifiability of the classification model obtained by a good amount. Even if the neural networks approach could reach better effectiveness, it would be at the price of losing all expressiveness and interpretability of the classification model. In any case, we provide a comparison of experimental results obtained with both approaches in the next section. Rule induction (or rule inference) techniques are a kind of supervised learning that process input training data and produce a set of IF-THEN rules used to classify the new examples [38; 39]. Two main strategies are commonly used:  
* Produce a decision tree and then extract its rules. * Generate the rules covering all the examples in a given class, exclude the covered examples and proceed with the next given class, until all classes are covered. The first strategy is implemented in the C4.5 system [40], which extends the previous ID3 [41]. Other algorithms such as PRISM [42] are based only on covering, whilst PART [43] combines both strategies. These strategies present the following advantages to minimize the impact of unintentional errors in the expert's classification of the images used as training examples:  
* Robustness against noise due to errors, omissions or insufficient data. * Identification of irrelevant attributes. * Detection of absent attributes or knowledge gaps. * Extraction of expressive and easy to understand rules. * Possibility to interpret or modify the produced rules with aid of expert knowledge, or even to incorporate new rules inferred by the experts themselves [44]. In order to improve the effectiveness of the individual classifiers obtained by means of rule induction, ensemble methods construct a set of classifiers and then classify new instances by taking a vote of their decisions (it can be a weighted vote, the mode of the votes, etc.) [45]. The technique has two main variants:  
* _Homogeneous classifiers_ are generated with the same learning algorithm [46]. The main methods are Bagging [47] and Boosting [48]. * _Heterogeneous classifiers_, instead, are generated with different learning algorithms. The most used method is Stacking (Stacked Generalization) [49]. We finally chose the Bagging method of homogeneous classifiers on the basis of the rule induction PART algorithm [43] to perform the experiments and obtain the classifiers. We explain the experiments in detail in the next section. The whole process consists of two main stages: rule inference and rule application. The inference stage can be summarized in the following steps (see Figure 12):  
1. Obtain the initial set of images downloaded from the web. 2. Classify the images according to the criteria explained in our previous conference paper [9] (done by human experts). 3. Extract the graphical features described in Section 4 (number of gray and color tones, and number of straight lines and rectangles) (done by the i2m tool extractor). 4. Build the training and testing data for the supervised machine learning algorithm, combining the two previous outputs: each image is represented as a vector of graphical features and human-assigned classification. 5.