Interpreting deep neural networks can enable new applications for predictive modeling where both accuracy and interpretability are required. In this paper, we examine the weights of a deep neural network to interpret the statistical interactions it captures. Our key observation is that any input features that interact with each other must follow strongly weighted paths to a common hidden unit before the final output. We propose a novel framework, which we call Neural Interaction Detector (NID), that identifies meaningful interactions of arbitrary-order without an exhaustive search on an exponential solution space of interaction candidates. Empirical evaluation on both synthetic and real-world data showed the effectiveness of NID, which detects interactions more accurately and efficiently than does the state-of-the-art. In this section, we identify and quantify influential statistical interactions modeled by a neural network. We often refer to statistical interactions as "feature interactions" when the statistical interactions of interest are between features. By the non-additive definition of a feature interaction and the mathematical formulation of a neural network, all interactions in the neural network are created at hidden units, and their influence is propagated layer-by-layer to the final output, as depicted in Figure 1 . We first study and quantify these two stages, then we discuss network architecture considerations to improve interaction detection.
Tsang et al. REF detected statistical interactions by interpreting the weights learned in neural networks.
Detecting Statistical Interactions from Neural Network Weights
ArXiv
ArXiv
Computer Science Mathematics