## 3 Proposed Method
### Simulation with Artificial Examples
#### 3.5.1 Artificial Example I: Variable Investigation
\begin{table}
\begin{tabular}{c c c c c c c c} \hline \hline
**New Mod.** & **Variables** & **I-Score** & **AUC** & **New Mod.** & **Variables** & **I-Score** & **AUC** \\ \hline \(X_{1}^{\dagger*}\) & X1, X2 & 638.17 & 0.75 & \(X_{9}^{\dagger*}\) & X3, X4, X5 & 350.2429 & 0.75 \\ \(X_{2}^{\dagger}\) & X8, X9, X13, X15, X21 & 0.6344 & 0.50 & \(X_{1}^{\dagger}\) & X11 & 0.2347 & 0.51 \\ \(X_{3}^{\dagger}\) & X19, X21, X25 & 1.4386 & 0.50 & \(X_{11}^{\dagger}\) & X16, X17, X21, X22 & 0.8097 & 0.51 \\ \(X_{4}^{\dagger}\) & X19, X21, X25 & 1.4386 & 0.50 & \(X_{12}^{\dagger}\) & X28 & 1.3729 & 0.51 \\ \(X_{5}^{\dagger}\) & X8, X9 & 0.5067 & 0.51 & \(X_{13}^{\dagger}\) & X11 & 0.2347 & 0.51 \\ \(X_{6}^{\dagger}\) & X10, X15, X21 & 0.9883 & 0.50 & \(X_{14}^{\dagger}\) & X18, X24 & 1.3814 & 0.51 \\ \(X_{7}^{\dagger}\) & X14, X15, X21, X22, X26 & 0.9816 & 0.50 & \(X_{15}^{\dagger}\) & X18, X24 & 1.3814 & 0.51 \\ \(X_{8}^{\dagger}\) & X20, X32, X33 & 2.0205 & 0.50 & \(X_{16}^{\dagger}\) & X22, X23, X24, X29, X34 & 1.5684 & 0.51 \\ \hline \hline \end{tabular}
\end{table} 
Table 4: Interaction-based convolutional layer. For an artificial data set with \(6^{2}=36\) variables and a window size of \(3\times 3\), we create 16 new features that can be shaped into \(4\times 4\). The procedure of generating these 16 new features follows the same procedure as in Table 3. The only difference is the window size, i.e., here, we use \(3\times 3\). In other words, we are able to create \(X_{b}^{\dagger}\) while \(b=\{1,2,\ldots,16\}\). The asterisk indicates extremely influential variable module(s). For each variable module, we also present the AUC value (see Section 4.4 for a detailed discussion of AUC values) assuming a classifier built using this variable module alone. assuming we use the 25 variable modules (the \(X^{\dagger}\)) in Table 3. Alternatively, suppose we use the 16 variables modules, \(\{X^{\dagger}_{1},\ldots,X^{\dagger}_{16}\}\), in Table 4. We can compute \(\hat{Y}\) using the following:  
\[\hat{Y}:=1/(1+\exp(-\sum_{j}^{16}(w_{j}X^{\dagger}_{j})) \tag{11}\]  
assuming the same activation function to be sigmoid (the same as (10)). This architecture is called forward propagation ( standard architecture widely used in neural network). This is discussed in Section 4.2 and review the architecture in Figure 5 for a detailed description. We assume there is no bias term. Then, the objective is to minimize the loss between \(Y\) and \(\hat{Y}\). Hence, we write \(\min_{\psi}\mathcal{L}(Y,\hat{Y})\). We can simply choose the loss function to be square-error, which means \(\mathcal{L}(Y,\hat{Y})=\sum_{i=1}^{n}(Y_{i}-\hat{Y}_{i})^{2}\). In order to minimize this loss function, an optimizer algorithm is required and we can choose the standard gradient descent algorithm. The results for the first artificial example are presented in Table 5. We discussed in Scenario I that the theoretical prediction rate is 75% if a statistician knows the model. In practice, we assume that the statistician has no knowledge of the model. In this case, the standard approach is to directly work with all variables. This means we can build a classifier by, e.g., using a neural network. In this example, we observe the data set has \(6\times 6=36\) variables. This means we can also adopt a convolutional neural network by reshaping the input dimension for an observation from \(1\times 36\) into \(6\times 6\). Both the neural network and CNN perform poorly, i.e., an AUC of approximately 50%. This is because the underlying model is in binary form and there is no marginal signal. In addition, the in-sample training set has only 500 observations, which further increases the difficulty for complex architectures such as deep learning models. However, this issue can be resolved if we have the correct model specification. Based on the proposed methodology, we are able to create the variable modules presented in Tables 3 and 4. Their performance on the test set is presented in Table 5. A window size of \(2\times 2\) produces 25 variable modules and achieves an AUC of 75%.