# A Simple Convolutional Neural Network with Rule Extraction
## 4 Experimental Results
### Examples of Detected \(N\)-Grams from the Rules
#### 4.2.1 Discriminative N-Grams Determined from \(R_{26}\)  
We first illustrate rule number 26 (\(R_{26}\)), with support equal to 464 samples with respect to the training set and 47 samples with respect to the testing set:  
* \((m_{35}<0.24)\)\((m_{54}\geq 0.16)\)\((m_{77}<0.22)\)\((m_{98}\geq 0.18)\)\((m_{112}\geq 0.12)\)\((m_{120}<0.1)\) Class = NEGATIVE Here, \(m_{i}\) (\(i=1,\ldots 120\)) designates neuron activations in the max-pooling layer. Indexes between one and 40 are related to single words, those between 41 and 80 correspond to bigrams, and those between 81 and 120 involve trigrams. The accuracy of this rule is 93.3% on the training set and 87.2% on the testing set.  
In the following Figures are depicted examples of \"tweets\" belonging to the testing set and including the discriminative n-grams, ranked according to the linear contribution in the output layer (cf. Equation (12)). Figure 7 shows a correctly classified \"tweet\" related to \(R_{26}\). Single words, bigrams and trigrams are illustrated vertically. A \"*\" or a \"+\" designates a repeated n-gram, since two different antecedents are able to code the same n-gram. In Figure 7 \"too-bad-the\" is a trigram, \"too-bad\" is a bigram and \"style\" is a single word. Note that \"style\" and \"style-.\" (punctuation is also coded in word embeddings) are against the final classification. The most contributing n-gram is \"too-bad-the\", which appears twice. In this case, all n-grams containing \"bad\" contribute to the negative polarity. This fact is also in agreement with our common sense.  
Figure 8 illustrates another example in which the most contributing n-gram to the final classification is a trigram: \"mediocre-special-effects\". Note that this is also in accordance with our perception. Bigram \"worst-of\" is the second most contributing n-gram. Surprisingly, single word \"worst\" is almost neutral with respect to the final classification.  
Figure 7: N-grams determined from \(R_{26}\) and the following “tweet” (in the testing set): “it is just too bad the film’s story does not live up to its style”. On the vertical axis are represented the n-grams classified according to their linear contribution. Negative values are against the final classification, while positive values are in its favor. The sum of the n-grams linear contributions is 0.435.  
Figure 9 shows that the most discriminative n-gram is trigram \"so-mind_numbingly-awful\", then follows bigram \"so-mind_numbingly\". Note also that three n-grams are against the final classification. Among them, single word \"as\" is the strongest element in contradiction with the final classification.  
As a last example for rule \(R_{26}\), we can see in Figure 10 that trigram \"just-bad-;\" and bigram \"just-bad\" are the most contributing n-grams. Again, \"as\" is the strongest element in favor of the positive polarity (thus, in contradiction with the \"tweet\" classification).  
Figure 8: N-grams determined from \(R_{26}\) and the following “tweet”: “a zippy 96 min of mediocre special effects, hoary dialogue, fluxing accents, and—worst of all—silly-looking morlocks”. The sum of the n-grams linear contributions is 0.302.  
Figure 9: N-grams determined from \(R_{26}\) and the following “tweet”: “so mind-numbingly awful that you hope [PERSON] won’t do it one more time, as far as movies are concerned”. The sum of the n-grams linear contributions is 0.211.