## 3 Proposed Method
### Simulation with Artificial Examples
#### 3.5.2 Artificial Example II: Four Modules with Discrete Variables
The overall performance was around 51-54% AUC on the out-of-sample test set for all algorithms. Alternatively, we recommend using the I-score to construct the interaction \begin{table}
\begin{tabular}{c l c c c c c} \hline \hline
**Variables: 7 \(\times\) 7** & **Algorithm** &
\begin{tabular}{c} **Test AUC** \\ **Bagging** \\ \end{tabular} & **Logistic** & **RF** & **iRF** & **NN** \\ \hline \multirow{3}{*}{50} & All Var. & 0.52 & 0.52 & 0.51 & 0.51 & 0.51 \\  & \(3\times 3\) window, 25 mod. & 0.56 & 0.57 & 0.54 & 0.55 & 0.55 \\  & \(4\times 4\) window, 16 mod. & 0.60 & 0.60 & 0.57 & 0.59 & 0.57 \\ \hline \multirow{3}{*}{100} & All Var. & 0.53 & 0.52 & 0.50 & 0.51 & 0.52 \\  & \(3\times 3\) window, 25 mod. & 0.60 & 0.58 & 0.55 & 0.55 & 0.55 \\  & \(4\times 4\) window, 16 mod. & 0.64 & 0.62 & 0.58 & 0.59 & 0.58 \\ \hline \multirow{3}{*}{1000} & All Var. & 0.60 & 0.54 & 0.53 & 0.59 & 0.53 \\  & \(3\times 3\) window, 25 mod. & 0.76 & 0.75 & 0.69 & 0.74 & 0.80 \\ \cline{1-1}  & \(4\times 4\) window, 16 mod. & 0.77 & 0.76 & 0.71 & 0.75 & 0.77 \\ \hline \hline \end{tabular}
\end{table} 
Table 6: Simulation results for Model (12). The theoretical prediction rate was calculated using Equation (13) as 75%. In other words, we expected prediction performance on th out-of-sample test set to be approximately 75% on average. For each experiment below, the out-of-sample test set had 1000 sample data points and the performance was calculated using the area under the curve (AUC) from the receiver operating characteristic (ROC). For each experiment, we changed the in-sample training size to be 50, 100, or 1000, and we fixed all data to have 49 variables. For each pair of in-sample training size and number of variables, we ran experiments using all original variables, i.e., “All Var.”, with a variety of different classifiers. Alternatively, we used the proposed method to construct new variable modules, which were used as new features on the same classifier. The table shows that the proposed method provides improved prediction performance regardless of sample size or classifier used. -based convolutional layer (explained in Section 3.3). We used a window size of \(2\times 2\), \(3\times 3\), and \(4\times 4\), which produced different numbers of variable modules. We can observe from Table 7 that the amount of noisy variables is quite challenging for common machine learning algorithms. For 100 variables, there are over 90 noisy variables. Without the I-score, we are only able to feed all variables without any selection into the algorithms, which results in poor performance. With the top variable module selected by the I-score, we are able to approximately achieve the theoretical prediction rate at around 75%. When we further injected more noisy variables, performance worsened for the common machine learning algorithms using all of the features including noisy variables, which reduced the prediction power when the algorithm was building its classifier. However, such limitation can be overcome by the usage of the I-score. Even with 200 variables, which means over 190 noisy variables, we are able to achieve a near-perfect prediction rate.