# Designing Trojan Detectors in Neural Networks Using Interactive Simulations
## 3 Methods
### Design of Neural Network Measurements
#### 3.2.4 Computational Consideration about KL Divergence  
The KL divergence computation considers computational and memory complexities since it must scale with increasing numbers of class labels, nodes, and layers.  
_Memory concerns:_ One should create a histogram with the number of bins equal up to \(2^{mnodes}\) per class label and per layer which can easily exceed the memory size. For example, if the number of classes is \(\approx\)10, the number of nodes is \(\approx\)100, and the number of layers is \(\approx\)100, then memory size is \(\approx\)\(2^{100}\times 10\times 100\approx 10^{33}\) bytes. To minimize the memory requirements in our implementation, histogram bins are created and stored in memory only for states that occur when each training data point passes through the neural network. This implementation leads to the worst-case memory requirement scenario to be \(npts\times 10\times 100\) bytes.  
_Computational concerns:_ One should align measured histograms per class label to identify the states uniquely encoding each class in order to avoid the \"not defined\" case of KL divergence or the case of the same state encoding multiple class labels. To eliminate the alignment computation in our implementation, the KL divergence definition is modified according to Equation (2). The computation of modified KL divergence \(\widetilde{D_{KL}}\) requires only collecting non-zero occurring states and calculating their histogram at the cost of approximating the originally defined KL divergence. The derivation of Equation (2) with its approximation step can be found in Appendix C.  
\[\widetilde{D_{KL}}(Q_{j}\parallel P_{j})=\sum_{i\in Set(q_{ij}\
eq 0)}(q_{ij}* \log_{2}q_{ij})-\log_{2}\frac{m}{n} \tag{2}\]  
While KL divergence satisfies \(D_{KL}\leq 0\), the modified KL divergence \(\widehat{D_{KL}}\) can be negative for those cases when \(|Set(q_{ij}\
eq 0)|>|Set(p_{ij}\
eq 0)|\). However, the negative value is lower bounded by Equation (3). For negative values, the NN layer is insufficient for encoding input data to class labels.  
\[\max_{Q_{j}}(D_{KL}(Q_{j}\parallel P_{j})-\widehat{D_{KL}}(Q_{j}||P_{j}))=- \sum_{i\in Set(q_{ij}\
eq 0)}(q_{ij}*\log_{2}p_{ij})-\log_{2}\frac{m}{n} \tag{3}\]  
The rationale behind modified KL divergence is that: (1) the alignment is not important for sufficient efficient and inefficient models (it is primarily important for insufficient models); (2) the approximation assumes \(p_{ij}\
eq 0\) at all non-zero states \(q_{ij}\
eq 0\) which yields negative modified KL divergence values as indicators of insufficiency; and (3) the alignment is important for detecting poorly trained models which could be using the same statesfor predicting multiple class labels while leaving all other available states in a NN layer unused. For the last case, it is assumed that all models were properly trained, and class labels are not assigned at random. Furthermore, the modified KL divergence addresses the problem of different within-class variations in training data which can lead to one class needing more allocated states than some other class. The modified KL divergence can be extended in the future by estimating within-class variations and assigning the number of states per class accordingly. In the following section, we show how to use the modified KL convergence to detect the presence of trojans in a network.