# Feature Interaction in Terms of Prediction Performance
## 4 Discussion  
Feature interaction can be utilized in different applications owing to its alternate definition in this study compared to its prior definitions. In this section, the proposed method is compared with recent studies by [PERSON] [8] and [PERSON] [17]. Figure 11 shows feature interaction graphs for variable \"tax\" and other variables in the BostonHousing dataset produced by the proposed method as well as those by [PERSON] [8] and [PERSON] [17]. We can see that the graphs differ according to the definitions of feature interaction. [PERSON] and [PERSON] each focused on measuring feature interaction using partial dependency and H-statistics; the H-statistics were based on partial dependency. In these studies, feature interaction refers to the share of variances determined by the interaction [18]. In an interaction between features A and B, the partial dependency line in the PDP of feature A is changed/affected by that of feature B. In this study, feature interaction, which was measured using prediction performance, is defined as the variance of the prediction performance caused by the interaction. Figure 10: Top five features that interact positively/negatively with each other. Figure 9: Total degrees of interaction corresponding to each feature. feature conditional on different, fixed points of the other feature [18]. If the variance is high, then the features interact with each other; if it is zero, they do not interact. Therefore, the interaction means that variable B influences the importance of variable A or vice versa. In other words, if variable B increases the importance of variable A, we can say B interacts with A. In [PERSON]'s method, the interaction means that variable B influences partial dependency of variable A or vice versa. The meanings of feature interaction from [PERSON] and [PERSON] are theoretically clear but difficult to intuitively understand. They can measure some influences between two features but do not show the meaning. In the proposed method, the interaction means that joining variable A and B increases or decreases prediction performance. The partial dependency-based approach does not support the concept of \"negative interaction\" as the direction of interaction is not measured. In the proposed approach, negative interaction refers to the case wherein the merging of features reduces the prediction performance. In general, the addition of a new feature increases the prediction performance, although this might affect the existing prediction and reduce the prediction performance in some cases. This shows that feature selection should be carefully performed prior to fitting the prediction model. The proposed approach in this study facilitates the measurement of negative interactions in prediction models. [PERSON]'s and [PERSON]'s methods suffer from the time complexity involved in the computation of feature interactions [18]. The time complexity involved in the computation of the interaction between any two features is given by \(O(N^{2})\) and \(O(2N^{2})\), respectively, where \(N\) refers to the number of data instances (data points). Instance sampling techniques, such as the Monte Carlo method, have been adopted in most implementations to overcome the expensive time complexity. This approach reduces the computation time but also results in a loss of accuracy. In the proposed approach, a one-time permutation is first performed for two features, followed by a prediction test for \(N\) instances. This reduces the time complexity of the proposed approach to \(O(N)\), thus achieving higher efficiency compared to previous approaches. A majority of [PERSON]'s method supports a standardized measure of feature interaction, which is based on an evaluation value between 0 and 1, to compare features across prediction models. However, it is difficult for users to interpret the absolute evaluation values as they are only capable of comparing their relative sizes. In the proposed approach, the evaluation values of feature interaction are not standardized.