# A Simple Convolutional Neural Network with Rule Extraction
## 2 Related Work
### Differences and Similarities with Our Approach  
Many approaches aiming at explaining CNN classifications are based on the visualization of image areas that are mainly relevant to the result. A recent survey on this topic is presented in [9]. [PERSON] et al. present a comprehensive survey of techniques aimed at elucidating deep neural networks [13]. In their view, explanatory of black-box models include decision trees, symbolic rules but also other entities like: Features Importance (FI) and Sensitivity Analysis. Specifically, the latter allows the inspection of the black-box by observing changes in the classification result when varying inputs. FI is a measure representing the importance of the inputs, which is very often related to the coefficients resulting from trained linear models. Note that both Sensitivity Analysis and FI highlight specific properties of the model, without necessitating an overall understanding of it.  
Our ultimate goal is to determine propositional rules, because they are close to the logic used by humans. Moreover, with the use of symbolic rules, discrimination between different classes is explained through rule antecedent values. This is much more precise than simply characterizing the relevant image subregions, because the way in which discrimination between different classes is carried out is undetermined [10]. The main difference between our method and the most recent techniques described in [12; 13] aiming at generating decision trees or symbolic rules from deep neural networks is that our method is global and the others are local.  
With respect to the taxonomy introduced by [PERSON] et al. [4] describing the general characteristics of all rule extraction methods, pedagogical techniques would be able to potentially generate symbolic rules from any neural network architecture (globally or locally). Specifically, a pedagogical method learns the associations between the input layer and the classification responses of the black-box model, without taking into account the values of its weights. Subsequently, rules can be generated, since the model that has been trained to recognize input-output associations is transparent. Examples of pedagogical techniques are reported in [35; 36]. Despite their potential, pedagogical techniques have been applied to deep architectures like CNNs, very rarely [31]. It is worth noting that our rule extraction technique is not pedagogical, since it belongs to the 'eclectic' category [4].  
Eclectic techniques are both pedagogical and decompositional. The latter category involves that the transparent model is generated by analyzing weight values of the neural network model. Note that the majority of decompositional techniques suffers from exponential algorithmic complexity [6]. As a result, in many cases a pruning step is carried out to reduce the number of weights. Hence, it seems very difficult to apply decompositional algorithms to CNN architectures. DIMLP rule extraction method has polynomial algorithmic complexity [8]. It is eclectic because weights of the first hidden layer define possible rule antecedents. In a second step these rule antecedents are confirmed or rejected in the rules by a pedagogical algorithm.