# Feature Interaction in Terms of Prediction Performance
## 1 Introduction  
The emerging technology report published by Gartner in 2017 identifies that machine learning is currently located at the peak of inflated expectations [1]. Machine learning techniques have been adopted by an increasing number of people, which has led to success stories in many fields. [PERSON] states that machine learning is no longer restricted to the experts in the Harvard Business Review [2]. Although machine learning has seen considerable success in recent years, most experts in this field still consider the prediction models generated by machine learning algorithms to be \"black boxes\". They do not have knowledge about how specific prediction results are obtained and the underlying principle behind the functioning of machine learning models. The concepts behind regression and decision tree models are relatively easy to understand, whereas the interpretation of those based on neural networks and support vector machines are more complex. Thus, researchers have concentrated their efforts on explaining the working mechanism of learning models and their prediction results [3]. Feature importance, the partial dependence plot (PDP) [4], feature interaction, and the Shapley value [5] have been proposed to achieve model interpretation. All of the above stated features can be calculated and interpreted only after model fitting. Feature impact or feature importance [6; 7; 8] measures the extent to which the feature or variable influences the prediction results of the given model. A prediction model is built on multiple features, wherein each feature makes a unique contribution toward the prediction. A model can be better understood by having knowledge about the influence of each feature in the model. Various methods have been proposed to measure feature importance. Recently permutation-based feature importance, which measures the importance of a feature _f1_ by permuting its feature values, was introduced by [PERSON] [9] and [PERSON] et al. [10]. This disconnects _f1_ from the other features and changes the predictive performance of the given model. The importance/significance of _f1_ increases with a greater degree of change in the prediction performance. The partial dependence plot, which was introduced by [PERSON], shows the marginal effect of one or two features on the predictive outcome of a machine learning model [4]. The PDP helps indetermining the transition/change in the predictive performance in relation to the change in the feature data values (observations). Furthermore, it is possible to obtain the range of data values in a feature that is either useful for prediction or shows low prediction performance (see Figure 1). Features in a prediction model tend to collaborate in a prediction task, which indicates the presence of feature interaction. Various methods for measuring feature interaction have been proposed. Initially, statistics-based methods were presented. [PERSON] and [PERSON] [PERSON] and [PERSON] (1999) suggested measuring feature interaction based on the analysis of variance (ANOVA) test. After the ANOVA test, the corresponding \(p\)-value for each pair of features is computed to measure feature interaction. This approach requires extensive computation time. [PERSON] and [PERSON] et al. [PERSON] and [PERSON] (2000); [PERSON] and [PERSON] (2000) tested pairwise feature interactions based on the \(\chi^{2}\) test. [PERSON] et al. proposed a grove-based method to detect statistical interactions [PERSON] and [PERSON] (2000). Decision-tree-based models, such as \"random forest\", measure feature interaction using a tree structure. If features \(f_{1}\) and \(f_{2}\) are located on the same path of a decision tree, they can be regarded as interacting [PERSON] and [PERSON] (1999); [PERSON] and [PERSON] (1999); [PERSON] and [PERSON] (1999). Recently, partial dependency-based methods have been proposed. The H-statistics method by [PERSON] and [PERSON] [PERSON] and [PERSON] (1999) is a typical method.