# Designing Trojan Detectors in Neural Networks Using Interactive Simulations
## 4 Experimental Results
### Neural Network Inefficiency  
KL Divergence Properties: We verified and quantified desirable properties of the modified KL divergence defined in Equation (2), such as decreasing inefficiency for increasing amount of added noise and increasing inefficiency for increasing number of nodes. The supporting results can be found in Appendix D.  
Sensitivity of Inefficiency Measurement: The sensitivity of NN inefficiency measurement is quantified with respect to: (a) data reshuffling and regeneration; (b) NN re-training with different initialization; and (c) no-training as the worst-case of poor training. To look at the sensitivity of the NN inefficiency with respect to data regeneration, the following steps are performed: First, a NN model is trained for a dataset and stored in memory. Next, four datasets are regenerated, and the standard deviation of inefficiency values is computed at each layer and for each class. Finally, the average value is computed over all standard deviations and the experiment is repeated for four 2D datasets with the results presented in Figure 7. From the data regeneration points in Figure 7, it is concluded that the average of standard deviations in inefficiency values larger than 0.1 will indicate dissimilarity of models by other factors. Similar sensitivity experiments are performed for no-training and retraining with random initialization. Figure 7 includes the results for four datasets. The sensitivity to retraining is bounded to approximately the average of inefficiency standard deviations equal to 0.46 while the same value for no-training is about 5-8 times larger and appears to be proportional to the complexity of the class distribution. Comparison of Inefficiencies for Trojan Types: Comparisons of models TwoT and TvT were conducted using a NN with six hidden layers, eight nodes per layer, and five features, namely \(X1\), \(X2\), \(X1^{2}\), \(X2^{2}\), and \(X1*X2\). The algorithmic and training parameters are set to learning rate of 0.03, activation of _Tanh_, no regularization, ratio of training to test data of 50%, and batch size of 10. Figure 8 shows the delta between modified KL divergence values of models TwoT and models TvT for the two Classes P (blue) and N (orange) and for the two trojans (T1 and T2) of different sizes (Figure 8 left). For both trojans, the delta KL divergence values are positive for Class P (blue) and negative for Class N (orange): \(\Delta(P)>0.454\) and \(\Delta(N)<-0.702\). These values imply that a trojan is embedded in Class P (blue) in both trojan cases and is encoding Class N (orange) according to Figure 6 (\"From P to N\" \(\rightarrow\) misclassified points labeled as P to N). Furthermore, as the size of the trojan increased from T1 to T2 by a factor of 2.25, the ratio of deltas increased by 2.24 for Class N and by 2.37 for Class P (see Appendix C). Figure 9 illustrates the delta between modified KL divergence values of models TwoT and models TvT for Trojans T8 and T9 whose embeddings differ in terms of the number of classes and the number of class regions. First, one can observe for Trojan T8 that \(\Delta(T8/P)>0.48\) and \(\Delta(T8/N)<-0.769\). These values imply that Trojan T8 is embedded in Class P (blue) according to Figure 6 (\"From P to N\"). We recorded much lower delta values for Trojan T9 than in the previous comparisons. This indicates the much higher complexity of modeling the spiral dataset than circle, exclusive OR, or Gaussian datasets and therefore lower inefficiency values measured at NN layers. Based on the sensitivity values shown in Figure 7 (0.1 for data regeneration and 0.5 for re-training), one could infer that Trojan T9 is likely in both classes based on the placement of the point \([\Delta(T9/P)>-0.034,\ \Delta(T9/N)>0.035]\) in Figure 6 (i.e., the sub-spaces \"From N\", \"From P\", \"Not detectable\", and \"From N to P\" + \"From P to N\"). Due to the discrete nature of the spiral pattern, Class P (blue) occupies a longer curve than Class N (orange).