## 3 Proposed Method
### Simulation with Artificial Examples
#### 3.5.1 Artificial Example I: Variable Investigation
This means each observation can be reshaped into a \(6\times 6\) grid structure. In other words, each observation can be considered as an image. The first row and the first column is variable \(X_{1}\). The first row and the last column is variable \(X_{6}\), so we can arrange these variables into the following structure:\[\{X_{1},X_{2},...,X_{36}\}\longrightarrow\underbrace{ \begin{array}{ccccc}X_{1}&X_{2}&X_{3 }&\ldots&X_{6}\\ X_{7}&X_{8}&X_{9}&\ldots&X_{12}\\ \vdots&\vdots&\ddots&&\\ X_{31}&X_{32}&X_{33}&\ldots&X_{36}\\ \end{array} }_{\text{$6\,\mathrm{by}\,6$}}\]  
and the 4 red-colored variables are the first used in the BDA. If the window size is \(2\times 2\), we would use \(\{X_{1},X_{2},X_{7},X_{8}\}\) and execute the BDA on the variables within this window. Notice that in the diagram (Figure 4), the blue region indicates a local area where we run the backward dropping algorithm. The pink region indicates the engineering workflow of building interaction-based features using Equation (4). For each window, we run the BDA. The procedure adopts the steps introduced in Section 2.2. The steps are listed in Table 2. First, we start with all 4 variables, which are \(\{X_{1},X_{2},X_{7},X_{8}\}\). For each step, we take turns dropping one variable and compute the I-score for the remaining variables. It can be seen that \(X_{8}\) should be dropped because the I-score raises from 160.18 in step 1 to 319.65 to step 2. We conduct the same procedure in step 2 and realize that \(X_{7}\) needs to be dropped. Then, the I-score increases to 638.17. We can keep dropping variables, but the statistician realizes that the I-score is the highest when there are only two variables left: \(\{X_{1},X_{2}\}\). Hence, we place an up-arrow at the step to indicate the optimal variable selection (see the up-arrow in Table 2). In this experiment, the most optimal selection is the variable module \(\{X_{1},X_{2}\}\) with an I-score of 638.17. For each local area with a window size of \(2\times 2\), we discussed in the above how to finely select the important variable. Similar to the standard procedure of convolutional neural networks using a filter on a local area of an image, the proposed method also investigates local information. However, instead of relying on many filters that are predefined, the proposed approach uses the I-score, a model free influence measure that directly associates with predictivity. After the variable module is selected within a local window, we adopt Formula (4) to create interaction-based features. For example, in the local window \begin{table}
\begin{tabular}{l l l l l} \hline \hline
**Step** & **1** & **2** & **3** & **4** \\
**Drop** & **Start** & \(\mathbf{X_{8}}\) & \(\mathbf{X_{7}}\) & \(\mathbf{X_{2}}\) \\ \hline I-score & 160.18 & 319.65 & 638.17 & 0.65 \\  & \(X_{1}\) & \(X_{1}\) & \(X_{1}\) & \(X_{1}\) \\ Investigation by iterative dropping & \(X_{2}\) & \(X_{2}\) & \(X_{2}\) & \\  & \(X_{7}\) & \(X_{7}\) & & \\  & \(X_{8}\) & & & \\ \hline Best result: & \(\{X_{1},X_{2}\}\) & & & \\ \hline \hline \end{tabular}
\end{table} 
Table 2: The steps of the backward dropping algorithm in a \(2\times 2\) window. Figure 4: The network architecture with simulated data. The artificial data has \(6\times 6=36\) variables, which can be arranged in a grid structure with shape of \(6\times 6\). We use a window size that is \(2\times 2\). By passing this window from the top left corner of the original \(6\times 6\) matrix, we create a new \(5\times 5\) matrix. The number 32 is the number of units in the hidden layer. In this example, there is one hidden layer, which is sufficient for the dimension of the data in the example. above, we narrow down to variable module \(\{X_{1},X_{2}\}\). This means we can create the first interaction-based feature as the following:  
\[X_{1}^{\dagger}:=\left\{ \begin{array}{ll}g_{1}&\text{if obs. falls in }X_{1}=1,X_{2}=1\\ g_{2}&\text{if obs. falls in }X_{1}=1,X_{2}=0\\ \tilde{g}_{3}&\text{if obs. falls in }X_{1}=0,X_{2}=1\\ g_{4}&\text{if obs. falls in }X_{1}=0,X_{2}=0\end{array}