# Designing Trojan Detectors in Neural Networks Using Interactive Simulations
## 5 Discussion about Trojan Detection  
Entropy-based measurements from state histograms: One option to incorporate the computational constraints and remove the need for histogram alignment would be to replace KL divergence by entropy of a state histogram normalized by maximum entropy [11]. This metric can be computed per layer and per class label, but it has the same issue of negative values as the KL divergence metric while limiting the dynamic range of measurements.  
If one would always evaluate a pair of models (i.e., comparing models TwoT and TwT for trojan detection), then one could use Jensen-Shannon divergence [39] instead of KL divergence. Jensen-Shannon divergence is symmetric and always yields a finite value. We preferred the KL divergence because evaluating one NN is more general than evaluating pairs of NNs.  
Trojan detection algorithm: One can obtain several additional useful insights from interactive analyses in the web-based trojan simulator before designing trojan detection algorithms. Some of them are presented in Appendix E. In many of the results, it is apparent that the encoded class information is not in one layer but spread across multiple layers. Thus, trojan detection must include comparisons of vectors of \(\widehat{D_{KL}^{\prime}}\) across all layers \(1\). Furthermore, the encoding of the same training data in NN can have multiple solutions, especially in inefficient NN and therefore the comparison of vectors of \(\widehat{D_{KL}^{\prime}}\) must include again a statistical nature of such solutions. Finally, the last layers carry less information about trojans because they serve the purpose of a final decision maker which should appear fair for datasets without trojans. This could be accommodated by weighting the layer-specific vector elements. From a global algorithmic design perspective, designing an actual trojan detector must still consider the trade-offs of doing all pair-wise model comparisons versus clustering all vectors of \(\widehat{D_{KL}^{\prime}}\) to identify the cluster of model TwoT.  
Complexity of trojan problems: The trade-off for interactivity of analyses is the input limitation to 2D dot patterns, the NN limitation to fewer than seven hidden layers and nine nodes per layer due to screen size, and the limitation to custom designed features derived from 2D dot patterns. In addition, by leveraging Tensorflow Playground [9], we limited our study to trojan encodings only in the fully connected layers on NNs and to only two class prediction problems.  
Given the current trojan detection approach, the complexities of trojan problems arise in the relationships between model capacity, size of input data space, characteristics of trojan embedding, the number of predicted classes, and the number and selection of provided training data points per class with respect to the within-class variability (i.e., number, shape, and location of regions per class). As one transitions analyses from the trojan simulator to actual NNs, the model capacity goes from ten to thousands of features, from six to hundreds of hidden layers, and from eight to hundreds of nodes per layer. The  
Figure 9: Comparison of inefficiencies between models TwoT and TwT, and embedded Trojans T8 and T9 with different numbers of classes (one or two) and class regions (one or four).  
size of input data space goes from 2D space constrained by 12 units \(\times\) 12 units to grayscale and color images with millions of pixels with constrained variability by the application domain. Finally, the number of classes goes from two to hundreds or thousands. Given such an increase of problem complexities and without knowing the characteristics of trojan embedding, the number and selection of provided training data points per class become the key to detecting trojans. In addition, for NN models predicting large numbers of classes, the combinatorial complexity of triggered classes and targeted classes is much higher than for NN models predicting two classes.