# An Algorithm for the Detection of Hidden Propaganda in Mixed-Code Text over the Internet +
## 5 Evaluation and Results Discussion
As it has been already mentioned, on the basis on the results gathered from the classifier assessment described in Section 4, a CNN has been chosen for supporting the propaganda detection part of the process, as the trained classifier performed the best in comparison to the others. The configuration parameters, that have been used to setup the CNN classifier, are reported in Table 8.  
An example of the results, gathered by experimenting the method presented in Section 4, are reported in Table 9.  
As it is possible to see in Table 9, for each sentence of Table 7, at least one reconstructed sentence is obtained. Indeed, according to the _Word Selection step_ of the method, different \"normal\" words, and as a consequence multiple reconstructed sentences, can be generated from one single Art Form Word. Consequently, different evaluations are possible as for the sentence with Id = \"1\" or like the sentence with Id = \"6\" which is also reconstructed in different way but with the same result evaluation. Whereas, the sentence with Id = \"7\" is not properly reconstructed and then misclassified. To overcome the classification problem in case of discordant multiple classifications, the human intervention is required, in order to select one of the possible available alternatives. \begin{table}
\begin{tabular}{c c c c} \hline \hline
**Id** & **Original Sentence\(W\)\(\in\)\(SS\)** & **Label** & **Sentence with \(W_{AFW}\)\(\in\)\(SS_{AFW}\)** \\ \hline
01 & Build the Wall & Propaganda & \(\delta\)UILD T \(|\) = \(|\) E \(\bigvee\) / ALL \\
02 & Purge every homosexual & Propaganda & Pu \(|\) 2 ge every \(|\) = \(|\) omosexual \\
03 & Abolish Wage System & Propaganda & Abolish \(\bigvee\)/ \(\surd\)age System \\
04 & Whites should rule & Propaganda & Whit €s should \(|\) 2 ule \\
05 & Right around 3 billion miles & Nonpropaganda & \(|\) 2 tg \(|\) - \(|\) t around 3 \(|\) billion miles \\
06 & America First & Propaganda & America\(\dot{\Lambda}\) = first \\
07 & Here is the dirty secret & Propaganda & Here is the \(|\))irt\(\dot{\
abla}\) Se(ret \\... &... &... &... \\ \hline \hline \end{tabular}
\end{table} 
Table 7: An excerpt of the propaganda related dataset. \begin{table}
\begin{tabular}{c c} \hline \hline
**Layer** & **Convolution** \\ \hline
01 & Conv2d(1, 100, kernel\_size=(3, 300), stride=(1, 1)) \\
02 & Conv2d(1, 100, kernel\_size=(4, 300), stride=(1, 1)) \\
03 & Conv2d(1, 100, kernel\_size=(5, 300), stride=(1, 1)) \\
04 (Fully Connected) & (fc1):Linear(in\_features=300,out\_features=2, bias=True \\ \hline \hline \end{tabular}
\end{table} 
Table 8: Configuration parameters of the CNN classification model. Figure 7 summarises, instead, the confusion matrix at the end of the overall evaluation process based on the selected Convolutional Neural Network (CNN), which shows that only 9% of the instances are wrong classified and in particular only 5% of those related to propaganda are missclassified as nonpropaganda related. Whereas, Figure 8 shows the classification performances, in terms of accuracy, precision, recall and f1-score, by comparing the detection of propaganda in a standard text (i.e., with mixed code and as a consequence without applying the normalisation algorithm) with the detection of propaganda hidden mixed-code text. In particular, not only the diagram shows very similar performances, meaning that in presence of mixed-code, the normalisation algorithm is able to reconstruct and analyse appropriately the sentences; but it also highlights that the proposed approach is in average inline with the performances of the related works presented in Section 2, in terms of propaganda detection, by ranging from 90% to 92% in terms of accuracy, precision,  
It is worth noting that the current results are improved, in comparison to those presented in the previous version of this work [20].