## 3 Proposed Method
### Interaction-Based Feature Engineer  
This subsection defines a procedure to mechanically engineer an interaction-based feature. For a \(3\times 3\) image (that is, an image that has 9 pixels), we write these pixels as \(X_{1},X_{2},\ldots,X_{9}\). Consider a small window of \(2\times 2\) passing from the first row and the first column of this \(3\times 3\) image. This means we start with \(\{X_{1},X_{2},X_{4},X_{5}\}\) within this small \(2\times 2\) window in this example. We use the BDA to narrow down to \(\{X_{1},X_{2}\}\) because we observed that this subset of variables delivers the highest I-score. Since both \(X_{1}\) and \(X_{2}\) are discretized into two partitions, we have partition \(\Pi_{\{X_{1},X_{2}\}}\) well-defined (see Equation (3)). Figure 3: This is the architecture of the interaction-based convolutional neural network (ICNN). In Panel **A**, the input matrix is 64 \(\times\) 64. Suppose the small rolling window is 4 \(\times\) 4. By rolling this small window from the top left corner to the bottom right corner of the input matrix, we create a 61 \(\times\) 61 output matrix. These 61 \(\times\) 61 variables are then used to build a neural network classifier with a hidden layer that has 32 units. The number of hidden layers and number of units per layer are tuning parameters. Panel **B** depicts an architecture that is much deeper, with each layer adopting the design in Panel **A**. In other words, assume \(X_{1}\) and \(X_{2}\) both only take values of 0 and 1. Then, the partition \(\Pi_{\{X_{1},X_{2}\}}\) is defined as:  
\[\Pi:=\left\{ \begin{array}{ll}\pi_{1}&\text{if }X_{1}=1,X_{2}=1\\ \pi_{2}&\text{if }X_{1}=1,X_{2}=0\\ \pi_{3}&\text{if }X_{1}=0,X_{2}=1\\ \pi_{4}&\text{if }X_{1}=0,X_{2}=0\end{array} \right. \tag{3}\]  
In this case, each partition \(\pi_{j}\), while \(j\in\{1,2,3,4\}\); we can compute the local mean of response variable \(Y\) from observations in the training set. Hence, a new interaction-based feature can be constructed:  
\[ \begin{array}{rcl}X^{\ddagger}&:=&\tilde{y}_{j}\text{ while }\tilde{y}_{j}\text{ is the local mean of }Y\\ &\text{based on the }j^{\text{th}}\text{ partition }\in\Pi\end{array}  \tag{4}\]  
In the general situation, let us consider an input matrix with size \(s_{\text{in}}\) by \(s_{\text{in}}\). Suppose the window size is \(w\times w\) and the stride level is \(l\) (notice in the above example that \(w=2\) and \(l=1\)). Using Equation (5), which is defined below,  
\[s_{\text{out}}=\lfloor\frac{s_{\text{in}}-w}{l}+1\rfloor \tag{5}\]  
we can compute an output matrix with size \(s_{\text{out}}\times s_{\text{out}}\) (which coincides with the \(X^{\ddagger}\) matrix). In this case, we define running index \(b\) to take values \(\{1,2,3,\ldots s_{\text{out}}\times s_{\text{out}}\}\). In the example in the previous paragraph, \(b\) can take value \(\{1,2,3,4\}\) because \(s_{\text{out}}=\lfloor\frac{s_{\text{in}}-w}{l}+1\rfloor=\lfloor(3-2)/1+1 \rfloor=2\), thus \(s_{\text{out}}\times s_{\text{out}}=4\). For each round \(b\) of the backward dropping algorithm (\(b\) takes a value from \(\{1,2,3,4\}\) in this example), we can construct a new variable module that takes Formula (6), defined below,  
\[ \begin{array}{rcl}X^{\ddagger}_{b}&:=&\tilde{y}_{j}\text{ while }\tilde{y}_{j}\text{ is the local mean of }Y\\ &\text{based on the }j^{\text{th}}\text{ partition }\in\Pi\end{array}  \tag{6}\]  
whereas \(X^{\ddagger}_{b}\) is defined as \(\tilde{y}_{j}\) using observations in training set, and \(j\) indicates the \(j\)th partition of \(\Pi\) that is formed by the variables selected from the \(b\) round of the BDA. Hence, the relationship of the input matrix and output matrix can be visualized using the following diagram. \[\text{input:} \begin{bmatrix}X_{1}&X_{2}&X_{3}\\ X_{4}&X_{5}&X_{6}\\ X_{7}&X_{8}&X_{9}\end{bmatrix} _{\text{3 by 3}}\rightarrow\text{ output:} \begin{bmatrix}X^{\ddagger}_{1}&X^{\ddagger}_{2}\\ X^{\ddagger}_{3}&X^{\ddagger}_{4}\end{bmatrix} _{\text{2 by 2}}\]  
where the input matrix is 3 \(\times\) 3 and the output matrix is 2 \(\times\) 2.