# How to Open a Black Box Classifier for Tabular Data
## 2 Materials and Methods
### Methods
#### 2.1.3 Second Training Iteration  
If the original black box model is an MLP, it is possible to construct a GANN/SENN to replicate the output of the logistic Lasso by a replication of the weights from the MLP multiplied by the coefficients of the Lasso. This permits an additional step of refining the partial responses themselves by initialising the SENN at the operating point of the GAM.  
Given the weights \(\left\{w_{ij},b_{j},v_{j},v_{0}\right\}\) of the original pre-trained fully connected MLP, Figure 2, with the inputs indexed by \(i\) and hidden nodes indexed by \(j\), together with the coefficients \(\left\{\beta_{0},\beta_{i},\beta_{ij}\right\}\) fitted by the Lasso, the PRiSM model for the anchored decomposition can be exactly mapped onto an MLP structured in the form of a GANN/SENN, as follows.  
(1) Univariate partial response corresponding to the input \(x_{i}\)  
This is shown in Figure 2 for input \(x_{1}\). Zero inputs for all other inputs will not contribute to the activation of the hidden nodes. The hidden layer weights \(w_{ij}\) connected to node \(x_{i}\) remain the same as in the original MLP but the weights and bias to the output node need to be adjusted as follows:  
\[v_{j}\rightarrow\beta_{i}*v_{j} \tag{21}\]  
Figure 2: The partial response network (PRN) has the modular structure typical of a self-explaining neural network. In particular, the figure illustrates the connectivity for a univariate function of input variables \(\mathrm{x_{1}}\) and \(\mathrm{x_{p}}\), and a bivariate term involving variables \(\mathrm{x_{2}}\) and \(\mathrm{x_{3}}\). Modelling the interaction term as orthogonal to univariate terms involving the same variables requires three blocks of hidden nodes, as explained in the main text. If there are univariate additive component functions involving either \(\mathrm{x_{2}}\) or \(\mathrm{x_{3}}\) these are added to the structure by inserting additional modules, as shown for \(\mathrm{x_{1}}\) and \(\mathrm{x_{p}}\).  
\[v_{0}\rightarrow\beta_{i}*(v_{0}-logit(P(C|0))) \tag{22}\]
2. Bivariate partial response for the input pair \(\{x_{i}x_{j}\}\) This is shown in Figure 2 for inputs \(x_{2}\) and \(x_{3}\). This time, to replicate the partial response multiplied by the Lasso coefficient, it is necessary to add three elements to the structure, namely, a univariate partial response for each of the inputs involved and a coupled network that both inputs feed into together. We will use the generic input indices \(k\) and \(l\) to avoid confusion with the hidden node index \(j\). The hidden layer weights once again remain unchanged from the original MLP. The output layer weights and bias for the network structure representing the univariate term associated with input \(k\) (and similarly for input \(l\)) are adjusted by \[v_{j}\rightarrow(\beta_{k}-\beta_{kl})*v_{j}\] (23)  
\[v_{0}\rightarrow(\beta_{k}-\beta_{kl})*(v_{0}-logit(P(C|0))) \tag{24}\]  
whereas the weights and bias for the coupled network are changed according to  
\[v_{j}\rightarrow\beta_{kl}*v_{j} \tag{25}\]  
\[v_{0}\rightarrow\beta_{kl}*(v_{0}-logit(P(C|0))). \tag{26}\]
3. Finally, an amount is added to the total sum of the values calculated for the bias term in the structured neural network. This amount is equal to the intercept of the logistic Lasso, \(\beta_{0}\).  
Equations (23)-(26) have the property that when an interaction between two variables is identified after the first application of the Lasso, its mapping onto the structured neural network involves also univariate modules, with the consequence that the second back-propagation step can potentially render the bivariate term not statistically significant and replace it with one or more univariate independent effects.