# Explanations for Neural Networks by Neural Networks
## 4 Evaluation
### Experimental Results  
#### 4.2.1 Inspection of Explanation and Complexity  
In the following section, we inspect the explanations generated by \(\mathcal{I}\)-Nets and compare them with symbolic regression based on their complexity. In addition to the evaluation settings from Table 1, we also consider the case of \(n=1\), to allow for a visual comparison.  
In Figure 4, we can see explanations for a selected but arbitrary neural network generated by the \(\mathcal{I}\)-Net and symbolic regression along with the target function \(g^{*}\). Additionally, the dotted blue line shows the predictions of the corresponding neural network \(\lambda\) trained on \(g^{*}\). The corresponding expressions and the fidelity (w.r.t. \(\lambda\)) are shown in Table 3. We can observe that the neural network was able to learn the polynomial target function very accurately, as the functions completely overlap. The \(\mathcal{I}\)-Net achieved the best performance in this case (MAE = 0.0220) and was also able to approximately capture the functional form. The function generated by symbolic regression comprises just a single term and achieved an MAE of 0.0566. While the error can be considered as close to the \(\mathcal{I}\)-Net, symbolic regression was not able to capture the functional form in the given interval.  
For the multivariate case, a visual comparison of the functions to the model's predictions is not possible. Instead, Table 3 shows the polynomials for arbitrarily selected datasets  
\begin{table}
\begin{tabular}{c c c} \hline \hline
**Parameter** & \(\boldsymbol{\lambda}\)**-Hyperparameters** & \(\boldsymbol{\mathcal{I}}\)**-Hyperparameters** \\ \hline Hidden Layer Neurons \({}^{\text{a}}\) & \([5\times s]\) & \([4096,2048,1024,512]\) \\ Hidden Layer Activation & ReLU & ReLU \\ Batch Size & 64 & 256 \\ Optimizer & Adam & Adam \\ Learning Rate & 0.001 & 0.0001 \\ Loss Function & MAE & \(\mathcal{L}_{\mathcal{I}\text{-}Net}\) (with \(M=500\)) \\ Training Epochs & 1,000 & 500 \\ Early Stopping & Yes & Yes \\ \hline Number of Training Samples & 5,000 & 45,000 \\ \hline \hline \end{tabular} \({}^{\text{a}}\) In this context, \(s\) stands for the sparsity of \(\theta_{g}\)  
\end{table}
Table 2: Parameters used for the \(\lambda\)-Net and \(\mathcal{I}\)-Net training.  
Figure 4: Exemplary visualization of the explanations. This plot visualizes the function generated by the evaluated approaches for \(n=1\) on an arbitrarily selected dataset. The corresponding mathematical expressions and their fidelity can be found in Table 3.  
of each considered complexity. As expected, the complexity of the polynomial generated by the \(\mathcal{I}\)-Net matches the sparsity defined in Table 1. Accordingly, the complexity of the explanation increases with the complexity of the underlying dataset used for the training of \(\lambda\). For symbolic regression, complexity of the explanations (w.r.t. number of monomials) does not increase similarly. Instead, especially for \(n=15\), the individual monomials have a higher complexity comprising four different variables, while the maximum degree for the \(\mathcal{I}\)-Net is three. The fact that the explanation complexity for symbolic regression does not increase substantially can be attributed to the selection of a high parsimony_coefficient which punishes large functions during the optimization, preventing overfitting and long runtimes (see Appendix A for details).