# A Simple Convolutional Neural Network with Rule Extraction
## 4 Experimental Results
### Global View of Rules  
A rule antecedent can be true with several different n-grams. Hence, to analyze a rule as a whole we must detect its related n-grams for each antecedent and for all covered samples. Note that a rule with antecedents depending on neuron activations in the M-layer can be formulated in the input layer as:  
* \"if one of the n-grams related to the first antecedent is present and maximal (Only the n-gram that activates the most its related neuron in the M layer is the one that makes the antecedent true.) then...\" and... \"if one of the n-grams related to the last antecedent is present and maximal then...\". Let us give a first example with a rule covering a moderate number of tweets in the training set. We take into account a CNN network with parameter \(\Theta\) equal to 100. Rule 347 (\(R_{347}\)) presents two antecedents. It covers 32 samples in the training set and it is given by:  
Figure 14: N-grams determined from \(R_{24}\) and the following “tweet”: “it’s a beautifully accomplished lyrical meditation on a bunch of despondent and vulnerable characters living in the renown chelsea hotel...”. The sum of the n-grams linear contributions is 0.355. Figure 15: N-grams determined from \(R_{24}\) and the following “tweet” (in the testing set): “some movies are like a tasty hors-d’oeuvre; this one is a feast”. It is correctly classified by the rule, but wrongly classified by the network. \((m_{36}\geq 0.28)\)\((m_{110}\geq 0.28)\) Class = POSITIVE;  
with \(m_{36}\) and \(m_{110}\) representing neurons in the M-layer. Antecedent \(m_{36}\) is related to five single words: \"cinema\"; \"see\"; \"you\"; \"you\"l\"; \"your\". Antecedent \(m_{110}\) is related to 31 different trigrams, with five of them starting with \"is\" (\"is-a-beautiful\", \"is-a-compelling\", \"is-a-satisfying\", \"is-a-startling\", \"is-a-swetet\", \"is-the-heart\"); four trigrams starting with \"it's\" (\"it's-a-beautiful\", \"it's-a-smart\", \"it's-a-swet\", \"it's-a-very\"); three trigrams starting with \"yet\" (\"yet-deeply-watchable\", \"yet-sensual-entertainment\", \"yet-shadowy-vision\"); two trigrams starting with \"warm\" (\"warm-and-charming\", \"warm-your-heart\"), and 17 other trigrams (\"touching-and-tender\", \"simple-,-swet\", \"intriguing-and-beautiful\", etc.). Note that these n-grams clearly belong to the class of positive polarity. Another example with a rule covering more samples is represented by rule \(R_{66}\) (253 samples in the training set):  
* \((m_{74}\geq 0.12)\)\((m_{84}<0.24)\)\((m_{110}\geq 0.28)\) Class = POSITIVE;  
Antecedent \(m_{74}\) is related to 155 distinct bigrams. About two thirds of them start with \"a\" or \"an\" or \"and\" (for instance: \"a-perfect\"; \"an-amusing\", \"and-touching\"). Other bigrams start with \"deeply\", such as: \"deeply-moving\"; \"deeply-absorbing\"; \"deeply-moving\"; \"deeply-touching\", etc. Finally, \"surprisingly\" is also present in five bigrams (\"surprisingly-charming\", \"surprisingly-engaging\", \"surprisingly-manages\", \"surprisingly-powerful\", \"surprisingly-touching\"). Regarding \(m_{110}\), similarly to \(R_{347}\) in which this antecedent is present, we see many trigrams starting with \"is\" or \"it's\". Other trigrams having several times the same first word starts with: \"charming\"; \"deeply\"; \"funny\"; \"heart\"; \"intelligent\"; \"intriguing\"; \"lovely\"; \"offers\"; \"powerful\"; \"smart\"; \"surprisingly\"; \"touching\"; \"warm\"; \"yet\"; etc. For antecedent \(m_{84}\) we also have a certain number of trigrams starting with \"is\" or \"it's\", though significantly less than previously. Moreover, we counted more than 40 trigrams starting with \",\" and more than 20 trigrams starting with \"and\". Other non-numerous trigrams with the same first word start with: \"a\"; \"but\"; \"film\"; \"in\"; \"melodrama\"; \"movie\"; \"smart\"; \"surprisingly\"; \"that\"; \"smart\". Since we generate unordered rules, each rule can be viewed as a single classifier that can be analyzed without considering the other rules.