Feature Interaction Interpretability: A Case for Explaining Ad-Recommendation Systems via Neural Interaction Detection
Authors: Michael Tsang
Dehua Cheng
Hanpeng Liu
Xue Feng
Eric Zhou
Yan Liu
Venue:  (2020)
Keywords: Feature Interaction
Interpretability
Black Box
AutoML

Abstract: Recommendation is a prevalent application of machine learning that affects many users; therefore, it is important for recommender models to be accurate and interpretable. In this work, we propose a method to both interpret and augment the predictions of black-box recommender systems. In particular, we propose to interpret feature interactions from a source recommender model and explicitly encode these interactions in a target recommender model, where both source and target models are black-boxes. By not assuming the structure of the recommender system, our approach can be used in general settings. In our experiments, we focus on a prominent use of machine learning recommendation: ad-click prediction. We found that our interaction interpretations are both informative and predictive, e.g., significantly outperforming existing recommender models. What's more, the same approach to interpret interactions can provide new insights into domains even beyond recommendation, such as text and image classification.

Published as a conference paper at ICLR 2020
FEATURE INTERACTION INTERPRETABILITY: A CASE FOR EXPLAINING AD-RECOMMENDATION SYSTEMS VIA NEURAL INTERACTION DETECTION
Michael Tsang',
Dehua Cheng". Hanpeng Liu', Xue Feng?. Eric Zhou?, and Yan Liu'
'Department of Computer Science, University of Southern California stsaggm, hanpengl, yanngglyaniiuu yanliu. cs @usc) edu
FFacbooo AI
(dehuacheng, xfeng, hanningz com
ABSTRACT
Recommendation a prevalent application of machine learning that affects many users; therefore, it is important for recommender models to be accurate and in- terpretable. In this work, we propose method both interpret and augment the predictions of black-box recommender systems. In particular, interpret feature interactions from a source recommender model and explicitly en- code these interactions in target recommender model, where both source and target models are black-boxes. By not assuming the structure of the recommender system, our approach can be used in general settings. our experiments, fo- cus on a prominent use of machine learning recommendation: ad-click prediction. We found that our interaction interpretations e.g., significantly outperforming existing recommender models. What's 's more, the same approach to interpret interactions can provide new insights into domains even beyond recommendation, such as text and image classification.
propose
both informative and predictive,
INTRODUCTION
Despite their impact on users, state-of-the art recommender systems are becoming increasingly in- scrutable. For example, the models that predict a user will click on an online advertisement are often based function approximators that contain complex components in order to achieve optimal recommendation accuracy. The complex components come in the form of modules for better learn- ing relationships among features, such as interactions between user and ad features (Cheng al. 2016) Guo et al., 2017 Wang et al., 2017; Lian et al. 2018 Song et al. 2018). Although efforts have been made to understand the feature relationships, there is still method that interpret the feature interactions learned by a generic recommender system, nor is there a strong commercial
incentive do so.
In this work, we identify and leverage feature interactions that represent how a recommender system generally behaves. We propose novel approach, Global Interaction Detection and Encoding for Recommendation (GLIDER), which detects feature interactions that span globally across multiple data-instances from a source recommender model. then explicitly encodes the interactions in target recommender model, both of which can be black-boxes. GLIDER achieves this by first utilizing our ongoing work on Neural Interaction Detection (NID) (Tsang et al., 2017) with data-instance perturbation method called LIME (Ribeiro et al., 2016) over batch of data samples. GLIDER then explicitly encodes the collected global interactions into a target model via sparse feature crossing.
In our experiments ad-click recommendation, we found that the interpretations generated by GL.DEE are illuminating, and the detected global interactions can significantly improve the target model's prediction performance. Because interaction interpretation method is very general, also show that the interpretations are informative in other domains: text, image, graph, and dna modeling]
Code available http://wwwppp.....accaccaaaaaaaa

---

Published as a conference paper at ICLR 2020
Igg, r5 are interacting
y
1
Interaction Detector
Frec
3
3
[[3,5] | X
linear model
25
E5
Figure 1: A simplified overview of GLIDER. ! CLIDER utilizes Neural Interaction Detection and LIME together interpret feature interactions learned by source black- model data instance, denoted by the large green plus sign. 2 GLIDER identifies interactions that consistently appear over multiple data samples, then explicitly encodes these interactions target black-box recommender model free-
Our contributions are as follows:
1. We propose feature interaction interpretations of general prediction models via interaction detection.
2. Based on this approach, we propose GLIDER detect and explicitly encode global feature interactions in black-box recommender systems. This process is form of automatic feature engineering.
3. Through experiments, we demonstrate the overall interpretability of detected feature inter- actions on variety of domains and show that the interactions can be leveraged to improve recommendation accuracy.
NOTATIONS AND BACKGROUND
Notations:
Vectors are represented by boldface lowercase letters, such as x or z. The i-th entry of vector x is denoted by z;. For set S, its cardinality is denoted by (S).
Let d be the number of features in dataset.
interaction, I, is a subset of feature indices: d}, where [ is always > 2. A higher-order interaction always has [Z] 3. For vector E Rd, let E IRZZ be restricted the dimensions of x specified by T.
I C
Let black-box model ʃ(-) : RR R. black-box recommender model uses tabular feature types, as discussed later in this section. In classification tasks, we assume f is class logit. and d may be different depending on feature transformations.
Feature Interactions: By definition, model f learns statistical (non-additive) feature interaction if and only f cannot decomposed into a sum [ arbitrary subfunctions fú, each excluding
corresponding interaction variable (Friedman al. 2008 Sorokina et al. 2008 sang al. 2017)
ie., fxx # Eier fi (1)...........))...))
For example, multiplication between two features, 21 and 22, is feature interaction because it cannot be represented as an addition of univariate functions, i.e., #1 # fi (x2) f2(11).
Recommendation Systems: A recommender system, frea()), is model of two feature types: dense numerical features and sparse categorical features. Since the one-hot encoding of categorical feature Te can be high-dimensional, it is commonly represented in a low-dimensional embedding e, (o. Va via embedding matrix V
3 FEATURE INTERACTIONS IN BLACK-BOX MODELS
We start by explaining how to obtain data instance level (local) interpretation of feature interac- tions by utilizing interaction detection feature perturbations.
2

---

Published as a conference paper at ICLR 2020
3.1 FEATURE PERTURBATION AND INFERENCE
Given data instance x E RP, LIME proposed to perturb the data instance by sampling separate binary representation
(0, 1 1 of the same data instance. Let E (0, I& be the map from the binary representation to the perturbed data instance. Starting from binary vector of all ones that map to the original features values in the data instance, LIME uniformly samples the number of random features to switch to 0 the "off" state. In the data instance, "off" could correspond to O embedding vector for categorical features or mean value over a batch for numerical features. is possible for
by grouping features in the data instance to correspond single binary features in &. An important step is getting black-box predictions of the perturbed data instances to create dataset with binary inputs and prediction targets: D {(x,, yt) y: f(E(x))) X; {0, 1jd). Though we use LIME's approach, the next section is agnostic to the instance perturbation method.
3.2 FEATURE INTERACTION DETRCTION
Feature interaction detection is concerned with identifying feature interactions in dataset (Bien) itt al. 2013 Purushotham et al. 2014 Lou et al. 2013; Friedman et al.) 2008). Typically, proper interaction detection requires a pre- processcciigg step remove correlated features that adversely af- fect detection performance (Sorokina et al.] 2008). As long as features in dataset 'D are generated an uncorrelated fashion, e.g., through random sampling, we can directly use D to detect feature interactions from black-box model f at data instance
3.2. NEURAL INTERACTION DETECTION
f can be an arbitrary function and can generate highly nonlinear targets in D, so we focus on de- tecting interactions that could have generic forms. In light of this, we leverage our method, Neural Interaction Detection (NID) (Tsang et al., 2017), which accurately and efficiently detects generic non-additive and arbitrary-order statistical feature interactions. NID detects these interactions by training a lasso-regularized multilayer perceptron (MLP) on a dataset, then identifying the features that have siggmmag weights common hidden units. NID is efficient by greedily testing the top-interaction candidates of every order at each of h first-layer hidden units, enabling arbitrary- order interaction detection in (hh)) tests within one MLP.
3.2.2 GRADIENT-BASER NEURAL INTERACTION DETECTION
Besides the non-additive definition of statistical interaction, gradient definition also exists based on mixed partial derivatives (Friedman et al., 2008). i.e., a function F(.) exhibits statistical interaction
I among features 2; indexed by i1, 12, .2......
iz
E If
0 F(z)
Ex
> 0.
Ozi, Ozig
Oziggi
The advantage of this definition is that it allows exact interaction detection from model gradients (Ai & Norton 2003); however, this definition contains a computationally expensive expectation, and typical neural networks with ReLU activation functions do not permit mixed partial derivatives. For the task of local interpretation, we only examine a single data instance x, which avoids the expectation. We turn F into an MLP g(-) with smooth, fefiniely-difffrentiabb activation functions such as softplus, which closely follows ReLU (Glorot et al., (011). We then train the MLP with the same purpose as 13.2.1 to faithfully capture interactions in perturbation dataset D. Given these conditions, we define an alternate gradient- neural interaction detector (GradientNID) as:
2 g(x)
wII = oē; 0i:, Oe
where w is the strength of the interaction I, is the representation of and the MLP g trained on D. While GradientNID exactly detects interactions from the explainer MLP, it needs to compute interaction strengths for feature combinations that grow exponentially in number as [Z] increases. We recommend restricting GradientNID to low-order interactions.
3

---

Published as a conference paper at ICLR 2020
Algorithm Global Interaction Detection in GLIDER
Input: dataset B, recommender model Sree
Output: {[z, e:) global interactions I, and their counts o₁ over the dataset 1: G 4- initialize occurrence dictionary for global interactions
for each data sample x within dataset do
3:
S < MADEX x)
G + increment the occurrence count of I, E E Vj = 1,2,...,
5: sort 9 by most frequently occurring interactions
[optional] prune subset interactions in 9 within target number of interactions
3.3 SCOPE
we define function, MADEX (f: x), that takes as inputs black-box / and data instance x. and outputs S (Z, ) set of top-k detected feature interactions. MADEX stands for "Mosel-Agnostic Dependency Explainer".
Based on 43.1 and 3.2
In some cases, it is necessary to identify k threshold. Because of the importance of speed for local interpretations, we simply use a linear regression with additional multiplicative terms to approximate the gains given by interactions in S, where & starts at 0 and is incremented until the linear model's predictions stop improving.
4 GLIDER: GLOBAL INTERACTION DETECTION AND ENCODING FOR RECOMMENDATION
We discuss the different components of GLIDER: detecting global interactions in $4.1 then encoding these interactions recommender systems in $4.22 Recommender systems are interesting because they have pervasive application in real-world systems, and their features often very sparse. By sparse features, we mean features with many categories, e.g., millions of user IDs. The sparsity makes interaction detection challenging especially when applied directly on data because the one-hot encoding of sparse features creates an extremely large space of potential feature combinations (Fan al. 2015).
4.1 GLOBAL INTERACTION DETECTION
In this section, we explain the first step of GLIDER. As defined in $3. MADEX takes as input a black- box model f and data instance x. In the context of this section, MADEX imputs a source recommender system free and data instance x = [x] .2....... .pp. is the i-th feature field and is either dense or sparse feature. p both the total number of feature fields and the number of perturbation variables (p d). We define global interaction detection as repeatedly running MADEX over batch of data instances, then counting the occurrences of the same detected interactions, shown in Algorithm [ The occurrence counts are not only a useful way to rank global interaction detections, but also sanity check rule out the chance that the detected feature combinations are random selections.
One potential concern with Alg. that it could be slow depending on the speed of MADEX. In our experiments, the entire process took less than one hour when run in parallel over batch 1000 samples with 40 features 32-CPU server with 2 GPUs. This algorithm only needs to be run once to obtain the summary of global interactions.
4.2 TRUNCATED FEATURE CROSSES
The global interaction Z,, outputted by Alg. is used to create synthetic feature Ø for target recommender system. The synthetic feature Zz, is created by explicitly crossing sparse features indexed in I, If interaction I, involves dense features, crossing them. The synthetic feature is sometimes called cross feature (Wang et al. 2017 Luo et al. 2019) conjunction feature (Rosales et al. 2012 Chapelle et 2015).
bucketize the dense features before
4

---

Published as a conference paper at ICLR 2020
In this context, cross feature is n-ary Cartesian product among sparse features. If we denote as the set of IDs for each respective feature 21, ..,............................. In, then their cross feature
......,,,,, An I[I....)) takes all possible values in
Xi 22........
24 E X, Vi ,, X X ... x Xn = [(I,,....
Accordingly, the cardinality of this cross feature is X X many combinations of values in the cross feature are likely unseen in the training data. Therefore, we generate truncated form of the cross feature with only seen combinations of values, (1) where j is a sample index in the training data, and XL is represented as a sparse ID in the cross feature Ty- We further reduce the cardinality by requiring the same cross feature ID to occur more than T times in a batch of samples, or set to a default ID otherwise. These truncation steps significantly reduce the embedding sizes of each cross feature while maintaining their representation power. Once cross features {zz, J are included in target recommender system, it can be trained as per usual.
[xtn]] and can be extremely large, yet
4.3 MODEL DISTILLATION VS. ENIIACEMMETT
There are dual perspectives of GL IDER: method for model distillation model enhancement. If a strong source model is used to detect global interactions which are then encoded in more resource- constrained target models, then GLIDER adopts a teacher-student type distillation process. If inter- action encoding augments the same model where the interactions were detected from, then GLIDER tries to enhance the model's ability represent the interactions.
RELATED WORKS
Interaction Interpretations: A variety of methods exist to detect feature interactions learned in specific models but not black-box models. For example, RuleFit (Friedman et al. 2008). Additive Groves (Sorokina al., 2008), and Tree-Shap (Lundberg et al., (018)) detect interactions specifi- cally in trees; likewise PaD2 (Gevrryy et al. (006) and NID (Tsang et al., 2017) detect interactions in multilayer perceptrons. Some methods have attempted to interpret feature groups in black-box models, such as Anchors (Ribeiro et al.) 2018), Agglomerative Contextual Decomposition (Singh et al. 2019), and Context-Aware methods (Singla et al. 2019); however, these methods were not intended identify feature interactions.
Explicit Interaction Representation: There are increasingly methods for explicitly representing interactions in models. Cheng et al. (2016). Guo al. (2017), Wang et al. (2017), and Lian et al. (2018) directly incorporate multiplicative cross terms in neural network architectures and [Son et al. 2018) use attention
interaction module, all of which intended to improve the neural net- work's function approximation. This line of work found that predictive performance can improve with dedicated interaction modeling, [uo al. (219)) followed up by proposing feature sets from data then explicitly encoding them via feature crossing, but this method's proposals are limited by beam search. Our work approaches this problem from model interpretation standpoint.
Black-Box Local vs. Global Interpretations: Data-instance level local interpretation methods are more flexible at explaining general black-box models: however, global interpretations, which cover multiple data instances, have become increasingly desirable to summarize model behavior. Locally Interpretable Model-Agnostic Explanations (LIME) (Ribeiro al.) (016) and Integrated Gradients (Sundararajan al.) (017) are some of the most used methods to locally interpret any classifier and neural predictor respectively. There are some methods for global black-box interpreta- tions, such as shuffle-based feature importance (Fisher et al., 2018), 8), submodular pick (Ribeiro et al., 2016), and visual concept extraction (Kim et | 2018). Our work offers a new tooling option.
EXPERIMENTS
6.1 SETUP
In our experiments, we study interaction interpretation and encoding real-world data. The hyper- parameters in MADEX are as follows. For all experiments, our perturbation datasets D contain 5000 training samples and 500 samples for each validation and testing. Our usage of NID or Gradient- NID as the interaction detector (<3.2) depends on the experimental setting. For all experiments that only examine single data instances,
GradientNID for its exactness and pairwise interaction
5

---

Published as a conference paper at ICLR 2020
detection; otherwise, we use NID for its higher-order interaction detection. The MLPs for NID and GradientNID have architectures of 256-128-64 128-64 first-to-last hidden layer sizes, and they are trained with learning rate of 1e-2, batchsize of 100, and the ADAM optimizer. NID uses ReLU activa- tions and an & regularization of À = 1e-4, whereas GradientNID uses softplus activations and a structural regularizer M regression, which we found offers strong test performance. In general, models are trained with early stopping on validation sets.
For LIME perturbations, we need establish what binary 0 maps to via in the raw data instance (...). In domains involving embeddings, i.e., sparse features and word embeddings, the 0 ("off") state the zeroed embedding vector. For dense features, is the mean feature value over a batch; for images, the mean superpixel RGB of the image. For our DNA experiment, we use a random nu- cleotide other than the original one. These settings correspond to what is used in literature (Ribeiro et al., 2016, 2018). In our graph experiment, the nodes within the neighborhood of test node are perturbed, where each node is zeroed during perturbation.
6.2 EXPERIMENTS ON CTR RECOMMENDATION
In this section, provide experiments with GLIDER on models trained for click-
Table 1: CTR dataset statistics
through-rate (CTR) prediction. The rec-
ommender models we study include com- monly reported baselines, which all use neural networks: Wide&Deep (Cheng al. 2016). DeepFM (Guo et al.) 2017]. Drep&Cross (Wang et al., (017).
Dataset
# Samples
# Features
Total # Sparse IDs
Criteo Avazu
45, 840, 617 40, 428, 967
39
998, 960
23
1, 5.444, 428
XDeepFM (Lian et al. 2018), and AutoInt (Song et al. 2018).
AutoInt is the reported state-of-the-art in academic literature, so we use the model settings and data splits provided by AutoInt's official public repository !
For all other recommender models, we use public implementationn with the same original architectures reported in literature, set all embedding sizes to 16, and tune the learning rate and optimizer to reach or surpass the test logloss reported by the Adagrad optimizer (Duchi
the AutoInt paper (on AutoInt's data splits). From tuning, et al. 2011 with learning rate 0.01. All models use early stopping on validation sets.
The datasets we use are benchmark CTR datasets with the largest number of features: Crited and Avazi Fl whose data statistics are shown in Table Criteo and Avazu both contain 40+ millions of user records on
Critco
600
Avazu
clicking ads, with Criteo being the primary benchmark in CTR research (Cheng et al.
2016 Guo et al. 2017
Wang et al. 2017] Lian al.
2018 Song et 2018
count
400
Luo et al. 2019).
200
6.2.1 GLOBAL INTERACTION DETECTION
For each dataset, we train a source AutoInt model, Sree. then run global interaction detection via Algo- rithm [ on batch of 1000 samples from the valida- tion set. A full global detection experiment finishes in less than one hour when run in parallel on either Criteo or Avazu datasets in a 32-CPU Intel Xeon E5-2640 v2 @ 2.00GHz server with 2 Nvidia 1080 Ti GPUs. The detection results across datasets are shown in Figure 2 as plots of detection counts versus rank. Because the Avazu dataset contains non-anonymized features, we directly show its top-10 detected global interactions in Table
10
20
30
rank
Figure 2: Occurrence counts (Total: 1000) vs. rank of detected interactions from Au- toInt on Critco and Avazu datasets. indi- cates higher-order interaction (details in Appendix G
cha chaneggg

---

Published as a conference paper at ICLR 2020
Table Understanding feature interactions: top global feature interactions for (a) an ad targeting system via Algorithm| I and (b) text sentiment analyzer via $6..2 (later). The tables are juxtaposed to assist in understanding feature interactions, i.e., nuanced changes among interacting variables lead to significant changes in prediction probabilities. The prediction outcomes are ad-clicks by users for (a) and text sentiment for (b).
(a) Explanation of ad targeting system
(b) Explanation of a sentiment analyzer
Count
Count (Total:10)
Interaction
Interaction (ordered)
(Total:1000)
525
{device_ip. hour)
36
never, fails suspend, disbelief
235
{device.id. id. device_ip, hour]
30
217
{device_id, app_id}
30
too, bad very, funny neither, nor not, miss recent, memory not, good no, denying not, bad
203
{device_ ip. device model, hour}
29
194
{site id, site domain)
29
190
{site_id, hour)
28
187
(device_ .ip, site iid, hour)
27
183
{site.id. sitc domain, hour} {device_id, id, hour)
27
179
26
179
{device id, device ip, device modell, hour)
25
we see that the same interactions are detected very frequently across data instances, and many the interactions are higher-order interactions. The interaction counts are very signifi- cant. For example, any top-1 occurrence count 25 is significant for the Critco dataset (p < 0.05), and likewise > 71 for the Avazu dataset, assuming a conservative search space of only up to 3-way interactions ([Z) 3). Our top-1 occurrence counts are 691 (> 25) for Criteo and 525 (> 71) for Avazu.
From Figure 2.
In Table 2a] the ioppitteractions explainable. For example, the interaction between "device ip" and "hour" (in UTC time) makes sense because users - here identified by IP addressess - have ad-click behaviors dependent on their time zones. This is general theme with many of the top-interaction As another example, the interaction between "device_id" and "app_id" makes sense because ads are targeted to users based on the app they're in.
6.2.2 INTERACTION ENCODING
Based on our results from the previous section (6 we turn our attention to explicitly encoding the detected global interactions target baseline models via truncated feature crosses (detailed
in (4.2). In order to generate valid cross feature IDs, we bucketize dense features into maximum of 100 bins before crossing them and require that final cross feature IDs occur more than T = 100 times over training batch of one million samples.
We take AutoInt's top-K K global interactions on each dataset from 6.2. with subset interactions excluded (Algorithm line 6) and encode the interactions in each baseline model including AutoInt itself. K is tuned on valiation sets, and model hyperparameters are the same between a baseline and one with encoded interactions. We set K 40 for Criteo and K 10 for Avazu.
In Table 3 we found that GLIDER often obtains significant gains in performance based on stan- dard deviation, and GLIDER often reaches exceeds desired 0.001 improvement for the Criteo dataset (Cheng et al.) 2016 Guo et al. [017 Wang et al., 2017 [ong et al.) 2018). The improve- ments are especially visible with DeepFM M on Criteo. We show how this model's test performance varies with different K in Figure 3 All performance gains are obtained at limited cost of extra model parameters (Table 4 thanks to the truncations applied to our cross features. To avoid extra parameters entirely. we recommend feature selection on the new and existing features.
One one hand, the evidence that AutoInt's detected interactions can improve other baselines' perfor- mance suggests the viability of interaction distillation. On the other hand, evidence that AutoInt's performance on Criteo can improve using its own detected interactions suggests that AutoInt may benefit from learning interactions more explicitly. In either model distillation enhancement set-
5*device ip" and idd identify different sets of users (https: /wwww ca iiii.... wtt.ntueeee edu.tw/ tw/ rr1 0T01922136/ ieeiides/eeee aaaeee/aaaa-eeeeeeeeeeeeeg www.gggglleaaaggpp. avazu.pdf)
7

---

Published as a conference paper at ICLR 2020
Table 3: Test prediction performance by encoding top-K global interactions in baseline recom- mender systems on the Criteo and Avazu datasets (5 trials). K are 40 and 10 for Criteo and Avazu respectively. "+ GLIDER" means the inclusion of detected global interactions corresponding baselines. The "Setting" column is labeled relative to the source of detected interactions: AutoInt. scores by Song et al. (2018).
Criteo
Avazu
Setting
Model
AUC
logloss
AUC
logloss
Distillation
Deep&Deep
0.8069 + 5e-5e11
0.416 + 4e-4
0.7791 3e-4
0.380M + 2c 4
GLIDER
0.8080 + 3e
0.4436 = 3e
0.7795 + 1e
0.3802 + 9- 5
DeepPM
0.8079 3
0,4436 + 2
0.7792 3e
0.3804 1 9e-
+ GLIDER
0.8097 + 2c-4
0.4420 = 2e
0.7795 + 2e-4
0.3802 2c
Drep&Cross
0.8076 + 2
0,4138 + 2e
0.7791 + 2e-1
01.8805 + 1e-11
GLIDER
0.8086 3e
0.4428 = 2e
0.7792 2e
0.3803 9e-
XDeepFM + GLIDER
0.8084 +: 2e-4 4
0,4433 + 2e 4
0.7785
3e-4
0.3808 1 2e-4 4
0. 0.8097 + 3c
0.4421 + 3c
0.7787 + 4c
0.3806 + 1c-4
Enhancement AutoInt *
0.8083
0.4434
0.7774
0.3811
GLIDER
0. 0.8090 2e
0.4426
0.7773 1e-4
0.3811 + 5
Table 4: # parameters of the models in Table 3
M
denotes million.
0.4435
Model
Critco
Avazu
test logloss
Wide&Deep
18.1M
27.3M
0.4430
GLIDER
19.3M (+6.8%)
27.6M (+1.0%)
DeepFM
17.5M
26.7M
0.4425
GLIDER
18.3M (+4.8%%
26.9M (-0.6%)
Deep&Crross
17.5M
26.1M IM
0.4420
GLIDER
18.7M (++6.9%)
26 (+1.0%)
0
20
XDeepPM
18.5M
27. 7.6
GLIDER
21.7M (+11.29%)
25.3M (+2.5%)
Figure 3:
Test logloss vs. K of DeepPM
AutoInt
16.4M
25.1M
SLIDER
17.3M (+5.1%%
25.2M (+0.6%)
on the Criteo dataset (5 trials).
tings, found that GLIDER performs especially well industry production models trained on large private datasets with thousands of features.
6.3 INTERPRETATIONS ON OTHER DOMAINS
Since the proposed interaction interpretations are not entirely limited to recommender systems, demonstrate interpretations on more general black-bor box models. Specifically, we experiment with the function MADEX(.) defined in $3.3 which inputs black-box f. data-instance x. and outputs a set of top-k interactions. The models we use are trained very different tasks, i.e., ResNer152: an image classifier pretrained on ImageNet '14 (Russakovsky et af., (205)) (He et al. (016). Sentiment-L.SMM. 2-layer bi-directional long short-term memory network (LSTM) trained
the Stanford Sentiment
Treebank (SST) (Socher et al., 2013 Tai et al. 2015). DNA-CNN: 2-laver ID convolutional neural
network (CNN) trained MYC-NN binding date (Mordelet et al. 2013 Yang al. 2013
Alipanahi et al. 2015: et al. 2016 Wang et al.) 2018) Barrett et al. 2012). and GCN: a 3-layer Graph Convolutional Network trained on the Cora dataset (Kipf & Welling 2016 Sen et al. 2008). In order to make informative comparisons to the linear LIME baseline, we use LIME's sample weighting strategy and kernel size (0.25) in this section. We first provide quantitative validation for the detected interactions of all four models in (6.. followed by qualitative results for R5sNet152, Sentiment-LSTM, and DNA-CNN in 6.3.2
6.3.1 QUANTITATIVE
To quantitatively validate interaction interpretations of general black-box models, measure the local explanation fidelity of the interactions via prediction performance. As suggested in (3. and
hhtpps:/wwwtr n1m. http:////www.........m../h////// 2acc Pacc-6sM47026
8

---

Published as a conference paper at ICLR 2020
Table 5: Prediction performance (mean-squared error; lower is better) with (A 0) and without
(I 0) interactions for random data instances in the test sets of respective black-box models. rank threshold. S k are excluded because not all instances have 2 or more interactions. Only results with detected interactions are shown. At least 94% (2 188) of the data instances had interactions across 5 trials for each model and score statistic.
I. corresponds to the interaction
k
DNA-CNN
Sentiment-LSII
ResNet152
GCN
linear LIME
0
10e-3 1ee-+le- 1e-3
8.0e 20-22660 6e-3
1.1
7.1e3 + 7e2
MADEX (ours)
he-3+2e- 2
3.8e 2 6
1.7
5.7e3 7e2
MADEX (ours)
L
5.0e-3 8e-4
0 3
0.9 0.2
2c3 le3
4.2 encoding feature interactions way increase a model's function representation, but this also means that prediction performance gains over simpler first-order models (e.g., linear regression) way test the significance of the detected interactions. this section, we use neural network function approximators for each top-interaction from the ranking {I:} given by MADEX's interaction detector (in this case NID). Similar the k-thresholding description in $3.3 we start which is a linear regression, then increment & with added MLPs for each L; among {I,) until validation performance stops improving, denoted at = L. The MLPs all have architectures of 64-32-16 first-to-last hidden layer sizes and use the binary perturbation dataset D (from 331
O,
Test prediction performances are shown in Table 5 for k L) The average number of features of D among the black-box models ranges from 18 to 112. Our quantitative val- idation shows that adding feature in- teractions for DNA-CNN, Sentiment- LSTM, and ResNet152, and adding node interactions for GCN result in significant performance gains when averaged over 40 randomly selected data instances in the test set.
Original image
Interactions (ours)
Main effects
Zz
prediction: hammerheea,, hammmerheua shark
top prodiction: viaduct
6.3.2 QUALITATIVE
For our qualitative analysis, we pro- vide interaction interpretations via
top prodiction: Brittany spanic]
MADEX(.) of 152, Sentiment- LSTM, and DNA-CNN on test sam- ples. The interpretations are given by
= { set of detected in- teractions, which shown in Fig- ure 4 for R25Net152 and Sentiment- L.TM. For reference, also show the top "mmiin effects" by LIME's original linear regression, which lect the top-5 features that attribute towards the predicted class")
prediction: trolleybus, trolley coach, trackless trolley
(a) ResNet152 interpretations
Original sentence
Predi- ction
Main
Interactions (ours)
effects
I₁
Tz
never fails to engage us.
never. us
never, fails
In Figure 4a the "interaction" columns show selected features from MADEX's interactions between Quick- shift superpixels (Vedaldi & Soatto 2008 Ribeiro et al.) 2016). To reduce the number interactions per image. we merged interactions that have overlap coefficient 2 0.5 (Vi-
The movie makes absolutely no sense.
no,
absolutely, no
neg.
no, sense
sense
The central story lacks punch.
lacks, punch
neg.
lacks
story, lacks
(b) Sentiment- interpretations
jaymeena & Kavithal 2016). From
Figure 4: Qualitative examples (more in Appendix D & E
Based official code: httpp:///gtthbb..ommm/a

---

Published as a conference paper at ICLR 2020
the figure, that the interactions form single region multiple regions of the image. They also tend to be complementary to LIME's main effects and are sometimes more informative. For example, the interpretations of the "shark" classification show that interaction detection finds the shark fin whereas main effects do not. Interpretations of Sentiment-ISTTM are shown in Figure 4b) excluding common stop words (Appendix C.. We again see the value of MADEX's interactions, which show salient combinations of words, such as "never, fails". "absolutely, no", and "lacks, punch".
In our experiments on DNA-CNN, nucleotides, which form canonical DNA sequence (Staiger et al., 1989). The interaction was detected 97.3% out of 187 CACGTG appearances in the test set.
consistently detected the interaction between "CACGTG"
In order to run consistency experiments now on Sentiment-
word interactions need to be detected consistently across different sentences, which naively would require an exorbitant amount of sentences. Instead, we initially collect interaction candidates by running MADEX over all sentences in the SST test set, then select the word interactions that appear multiple times. We assume that word interactions are ordered but not necessarily adjacent positionally bound, e.g., (not, good) j (good, not), but their exact positions don't matter. We use the larger IMDB dataset (Maas et al. (011)) 1) to collect different sets of sentences that contain the same ordered words each interaction candidate (but the sentences are otherwise random). The ranked detection counts of the target interactions on their individual sets of sentences shown in Table 2b The average sentence length is 33 words, and interaction occurrences are separated by 2 words on average.
7 CONCLUSION
We proposed a way to interpret feature interactions in general prediction models, and we proposed GLIDER detect and encode these interactions in black-box recommender systems. In our periments on recommendation, we found that our detected global interactions are explainable and that explicitly encoding them can improve predictions. We further validated interaction inter- pretations on image, text, graph, and dna models. We hope the interpretations encourage investiga- tion into the complex behaviors prediction models, especially models with large societal impact. Some opportunities for future work are generating correct attributions for interaction interpretations, preventing false-positive interactions from -iitt---iitriittion feature perturbations, and performing interaction distillation from multiple models rather than just one.
ACKNOWLEDGMENTS
We would like to sincerely thank everyone who has provided their generous feedback for this work. Thank you Youbang Sun, Dongxu Ren, and Beibei Xin for offering early-stage brainstorming and prolonged discussions. Thank you Yuping Luo for providing advice on theoretical analysis of model interpretation. Thank you Rich Caruana for your support and insight. Thank you Artem Volkhin, Levent Ertoz, Ellie Wen, Long Jin, Dario Garcia, and the rest of the Facebook personalization team for your feedback on the paper content. Last but not least, thank you anonymous reviewers for your thorough comments and suggestions. This work supported by National Science Foundation Awards I25-1254206 and IIS- 1539688, granted to co-author Yan Liu in her academic role at the University of Southern California.
10

---

Published as a conference paper at ICLR 2020
REFERENCES
Chunrong Ai and Edward C. Norton. Interaction terms in logit and probit models. Economics letters, 80(1):123-122 2003.
Babak Alipanahi, Andrew Delong, Matthew Weirauch, and Brendan J Frey. Predicting the se- quence specificities of dna-and rna-binding proteins by deep learning. Nature biotechnology, 33 (8):831, 2015.
Tanya Barrett, Stephen E Wilhite, Pierre Ledoux, Carlos Evangelista, Irene Kim, Maxim Toma- shevsky. Kimberly A Marshall, Katherine H Phillippy. Patti M Sherman, Michelle Holko, et al. Nchi geo: archive for functional genomics data sets sets-update. Nucleic acids research, 41(D1): D991-1D955, 2012.
Jacob Bien, Jonathan Taylor, and Robert Tibshirani. lasso for hierarchical interactions. Annals of statistics, 41(3):1111, 2013.
Olivier Chapelle, Eren Manavoglu, and Romer Rosales. Simple and scalable response prediction for display advertising. ACM Trunsactions on Intelligent Systems and Technology (TIST), 5(4): 61, 2015.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794.
ACM, 2016.
Heng- -ze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. Wide & deep learning for recom- mender systems. Proceedings of the workshop on deep learning for recommender systems, pp. 7-10. ACM, 2016.
John Duchi, Elad Hazan, and Yoram Singer. Adaptive subgradient methods for online learning and stochastic optimization. Journal of Machine Learning Research, 12(ul):2121-2159, 2011.
Yingying Fan, Yinfei Kong, Daoji Li, Zemin Zheng, et al. Innovated interaction screening for high- dimensional nonlinear classification. The Annals of Statistics, 43(3):1243-1272, 2015.
Aaron Fisher, Cynthia Rudin, and Francesca Dominici. Model class reliance: Variable importance measures for any machine learning model class, from the "rashomon" perspective. arXiv preprint / 2018.
Jerome H Friedman, Bogdan E Popescu, et al. Predictive learning via rule ensembles. The Annals of Applied Statistics, 2(3):916-99) 2008
Muriel Gevrey, Ioannis Dimopoulos, and Sovan Lek. Two-way interaction of input variables in the sensitivity analysis of neural network models. Ecological modelling, 195(1-2):43-50, 2006.
Xavier Glorot, Antoine Bordes, and Yoshua Bengio. Deep sparse rectifier neural networks. In Proceedings of the fourteenth international conference on artificial intelligence and statistics, pp- 315-323, 2011.
Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. Deepfm: factorization- machine based neural network for ctr prediction. In Proceedings of the 26th International Joint Conference Artificial Intelligence, pp. 1725-1731. AAAI Press, 2017.
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recog- nition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770-778, 2016.
Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735-1780, 1997.
Giles Hooker. Discovering additive structure in black box functions. In Proceedings of the tenth ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 575-580. ACM, 2004.
11

---

Published as a conference paper at ICLR 2020
Been Kim. Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International Conference Machine Learning, pp. 2673-2682, 2018.
Thomas N Kipf and Max Welling. Supe-supe classification with graph convolutional net-
works. arXiv preprint arXiv: 100.:v660900200,,, 2016.
Jianxun Lian, Xiaohuan Zhou, Fuzheng Zhang, Zhongxia Chen, Xing Xie, and Guangzhong Sun. xdeepfm: Combining explicit and implicit feature interactions for recommender systems. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 1754-1763. ACM, 2018.
Yin Lou, Rich Caruana, Johannes Gehrke, and Giles Hooker. Accurate intelligible models with pairwise interactions. Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 623-631. ACM, 2013.
Scott M Lundberg, Gabriel G Erion, and Sn Lee. Consistent individualized feature attribution for tree ensembles. arXiv preprint Xrviv: 18020:80200300000 802.03888, 2018.
Yuanfei Luo, Mengshuo Wang, Hao Zhou, Quanming Yao, Wei-Wei Tu, Yuqiang Chen, Wenyuan Dai, and Qiang Yang. Autocross: Automatic feature crossing for tabular data in real-world ap- plications. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, 2019.
Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng. and Christopher Potts. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pp. 142-150, Portland, Oregon, USA, June 2011. Association for Computational Linguisties. URL http:
7/ ac www.wb.b... wtw.lee..rg//attooooooo httg///atthooocc//111011.1
Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schüize. Introduction to Informa- tion Retrieval. Cambridge University Press, New York, NY, USA, 2008. ISBN 0521865719, 9780521865715.
Fantine Mordelet, John Horton, Alexander Hartemink, Barbara E Engelhardt, and Raluca Gordán. Stability selection for regression-basee models of transcription factor-dny binding specificity. Bioinformatics, 29(13) 17- 2013.
W James Murdoch, Peter Liu, and Bin Yu. Beyond word importance: Contextual decomposition to extract interactions from Istms. International Conference on Learning Representations, 2018.
Sanjay Purushotham, Martin Renqiang Min, C-C Jay Kuo, and Rachel Ostroff. Factorized sparse learning models with interpretable high order feature interactions. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 552-561. ACM, 2014.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135-1144, ACM, 2016.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin, Anchors: High-precision model-agnostic explanations. In AAAI Conference on Artificial Intelligence, 2018.
Rómer Rosales, Haibin Cheng, and Eren Manavoglu. Post-click conversion modeling and analysis for non-guaranteed delivery display advertising. In Proceedings of the fifth ACM international conference on Web search and data mining, pp. 293-302. ACM, 2012.
Olga Russakovsky, Jia Deng. Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge. International Journal of Computer Vision, 115(3):11--522, 2015.
Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint 191000 2019.
12

---

Published as a conference paper at ICLR 2020
Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad
Collective classification in network data. Al magazine, 29(3):93 93, 2008.
Chandan Singh, W James Murdoch, and Bin Yu. Hierarchical interpretations for neural network predictions. International Conference on Learning Representations, 2019.
Sahil Singla, Eric Wallace, Shi Feng, and Soheil Feizi. Understanding impacts of high-order loss approximations and features in deep learning interpretation. arXiv preprint arXiv: 1902.00477, 2019.
Richard Socher, Alex Perelygin, Jean Wu, Jason Chuang, Christopher D Manning, Andrew Ng, and Christopher Potts. Recursive deep models for semantic compositionality over sentiment treebank. In Proceedings of the 2013 conference on empirical methods in natural language pro- cessing, pp. 1631-1642. 2013.
Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. Autoint: Automatic feature interaction learning via self-attentive neural networks. arXiv preprint arXiv:/810.1199// 2018.
Daria Sorokina, Rich Caruana, Mirek Riedewald, and Daniel Fink. Detecting statistical interactions with additive groves of trees. In Proceedings of the 25th international conference Machine learning, pp. 1000-1007. ACM, 2008.
Dorothee Staiger, Hildegard Kaulen, and Jeff Schell. A cacgig motif of the antirrhinum majus chal- cone synthase promoter is recognized by evolutionarily conserved nuclear protein. Proceedings of the National Academy of Sciences, 86( (0(18)66930-69233 1989.
Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. In Proceedings of the 34th International Conference on Machine Learning-Voin Volume 70, pp. 3319- 3328. JMLR. org, 2017.
Kai Sheng Tai, Richard Socher, and Christopher D Manning. Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.0000 2015.
Michael Tsang, Dehua Cheng, and Yan Liu. Detecting statistical interactions from neural network weights. arXiv preprini arXiv: 0tt:::1000.00000 705704977.... 2017.
Andrea Vedaldi and Stefano Soatto. Quick shift and kernel methods for mode seeking. In European Conference on Computer Vision, pp. 705-718. Springer, 2008.
MK Vijaymeena and K. Kavitha. A survey similarity measures in text mining. Machine Learning and Applications: An International Journal, 3(2):19-29 2016.
Meng Wang, Cheng Tai, Weinan E, and Liping Wei. Define: deep convolutional neural networks accurately quantify intensities of transcription factor-dna binding and facilitate evaluation of func- tional non-coding variants. Nucleic acids research, 46(1 1):669-666, 2018.
Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. Deep & cross network for ad click predictions. In Proceedings of the ADKDD' 17, pp. 12. ACM, 2017.
Lin Yang, Tianyin Zhou, Iris Dror, Anthony Mathelier, Wyeth Wasserman, Raluca Gordån, and Remo Rohs. Tfbsshape: motif database for dna shape features of transcription factor binding sites. Nucleic acids research, 42(D1))D144 D155, 2013.
Haoyang Zeng, Matthew D Edwards, Ge Liu, and David K Gifford. Convolutional neural network architectures for predicting dna-protein binding. Bioinformatics, 32(12):121-i117, 2016.
13

---

Published as a conference paper at ICLR 2020
A EFFECT OF EXTRA PARAMETERS BY INTERACTION ENCODINGS VS. ENLARGED EMBEDDINGS
In this section, we study whether increasing embedding size can obtain similar prediction perfor- mance gains explicitly encoding interactions via GLIDER. We increase the embedding dimension sizes of every sparse feature in baseline recommender models to match the total number of model parameters of baseline + GLIDER as close as possible. The embedding sizes used to obtain sim- ilar parameter counts are shown in Table 6 For the Avazu dataset, all of the embedding sizes remain unchanged because they were already the target size. The corresponding prediction performances of all models are shown in Table 7 We observed that directly increasing embedding size / parameter counts generally did not give same level of performance gains that GLIDER provided.
Table 6: Comparison of # model parameters between baseline models with enlarged embeddings and original baselines + GLIDER (from Tables 3 and 4. The models with enlarged embeddings are denoted by the asterick (*). The embedding dimension of sparse features is denoted by "emb.
size". Percent differences are relative to bascline* models. M denotes million, and the ditto mark
(") means change in the above line.
Criteo
Avazu
Model
emb. size
# params
emb. size
# params
Wede&Deep*
17
19. IM
16
27.3M
Weee&Deep
16
18.1M
16
CLIDER
16
19.3M (+1.1%)
16
27.6M (+1.0%)
DeepFM
17
18.5M
16
26.7M
DeepPM
16
17.5M
16
GLIDER
16
18.3M (-0.9%)
16
26.9M (+0.6%)
Deep&Cross*
17
18.5M
16
26.1M
Drep&Cross
16
17.5M
16
GLIDER
16
18.7M (+1.0%)
16
26.4M (+1.0%)
xDeepFM*
19
21.5M
16
27.6M
xDeepFM
16
18.5M
16
GI.TDER
16
21.7M (+0.7%)
16
28.3M (+2.5%)
AutoInt
17
17.4M
16
25.1M
AutoInt
16
16.4M
16
GLIDER
16
17.3M
(-1.0%)
16
25.2M (+0.6%)
Table Test prediction performance corresponding to the models shown in Table
Critco
Avazu
Model
AUC
logloss
AUC
logloss
Wide&Deep*
0.8072
0.443 + 2e-1
0.7794
0.3804 + 2e "
Wide&Deep
0.8069 + 5e
0.4466 + 4e-4
+ GL.TDER
0.8080 + 3e 4
0.4436 + 3e
0.7795 + 1e-4
0.3802 + 9e
DeepFM
0.8080 + 4e-4
0.4435 + 4e-4
0.7792 + 3c
0.3804 + 9c-5
DeepFM
0.8079 + 3c-4
0.4436 + 2e
+ GLIDER
0.8097 2c-4
0.4420 + 2c
0.7795 + 2c
0.3802 + 2
Crep&Cross*
0.8081 + 2c-4
0.4434 2
0.7791 + 2c-4
0.3805 + 1c-4
Drep&Cross
0.8076 + 2c
0.4438 + 2
+ GLIDER
0.8086 + 3c-4
0.4428 + 2c
0.7792 2c-4
0.3803 = 9c-5
xDecpFM*
0.8088 1c-4
0.4429 + 1c-4
0.7785 3c-4
0.3808 + 20-4
xDecpFM
0.8084 + 2c-4
0.4433 + 20-4
+ GLIDER
0.8097 + 3c-4
0.4421 - 3c
0.7787 + 4e-4
0.3806 - 1c-4
*utolnt*
0.8087 + 2c
0.4431 + 1e-4
0.7774 1c-4
0.3811 - 8e-5
Autolnt
0.8083
0.4434
GLIDER
0.8090 + 2e
0.4426 + 2
0.7773 + 1e-4
0.3811 - 5e-5
14

---

Published as a conference paper at ICLR 2020
B EFFECT OF DENSE FEATURE BUCKETIZATION
We examine the effect of dense feature bucketization on cross feature parameter efficiency for the Criteo dataset, which contains 13 dense features. Figure 5 shows the effects of varying the number of dense buckets on the embedding sizes of the cross features involving dense features. Both the effects on the average and individual embedding size are shown. 14 out of of the cross features involved a dense feature. Different cross features show different parameter patterns as the number of buckets increases (Figure 5b). One one hand, the parameter count sometimes increases then asymptotes. Our requirement that valid cross feature ID occurs more than 7 times (44.2) restricts the growth in parameters. On the other hand, the parameter count sometimes decreases, which happens when the dense bucket size becomes too small to satisfy the 7 occurrence restriction. In all cases, the parameter counts are kept limited, which is important for overall parameter efficiency.
1300
avg purameters
1200
parumeters
1100
1500
1000
1000
800
10a
10
10*
buckets
buckets
(a) effect on cross feature embedding size
(b) effect on each cross feature embedding size
Figure 5: The effects of varying the number of buckets on (a) on the average embedding size of cross features involving dense features and (b) the individual embedding sizes of the same features.
15

---

Published as a conference paper at ICLR 2020
C
STOP WORDS
For all qualitative interpretations on text (in 66..22 and Appendix we preprocessed sentences to remove stop words. We use the same stop words suggested by Manning et al. 2008). i.e., {a, an, and, are, as, at, be, by, for, from, has, he, in, is, it, its, of, on, that, the, to, was, were, will, with).
D QUALITATIVE RESULTS ON SENTIMENT-LSTM vs. BERT
In this section, we compare the word interactions discovered by MADEX on SentimentLSSTM versus BERT. These models perform with accuracies of 87% and 92% respectively on the SST test set. We use public pre-trained BERT, i.e., DistiBBRTT (Sanh et al. 2019), which is available onling The interaction detector we is GradientNID (33222), and sample weighting disabled for this comparison. The top-2 interactions for each model are shown in Table on random sentences from the SST test set.
Table 8: Top-ranked word interactions Z, from Sentiment-LSTM and BERT on randomly selected sentences in the SST test set.
Sentiment-LSTM
BERT
Original sentence
I
T
I
An intelligent, earnest. intimate film that drops the ball only when pauses for blunt exposition to make sure you'l 're getting its metaphysical point.
intelligent, metaphysical
metaphysical, point
intelligent, earnest
drops, ball
It's not so much enjoyable to watch as it enlightening to listen to new sides of a previous reality, and to visit with some of the people who were able to make an impact in the theater world.
enjoyable
not, so
not, much
not, enlightening
Uneasy mishmash of styles and genres.
uneasy. mishmash
mishmash, genres
uneasy, mishmash
uneasy, styles
You'l re better off staying home and watching the X-Files
x, files
off.
better, off
you, off
If this is the Danish idea of a good time, prospective tourists might want to consider a diffcrent destination some jolly country embroiled in a bloody civil war, perhaps.
if, this
if, good
if, jolly
jolly, country
can the wheels turning, and might resent sometimes, this a nice little picture, made hy bright friendly souls with a lot good cheer.
resent, nice
resent
nice, good
nice, made
One of the greatest famile-oriented, fantasy-aulventure movies ever.
family, oriented
greatest, family
greatest, family
adventure, movies
It's so full wrong choices that all you can do is shake your head in dishelief - and worry about what classic Oliver Parker intends mangle next time.
so, wrong
full, wrong
so, wrong
so, full
mysteries are transparently obvious, and it's too slowly paced a thriller.
mysteries, transparently
paced, thriller
too, thriller
too, paced
This miserable excuse of movie runs miserable, runs
miserable, runs
empty, believing flatbush machismo will gel through.
excuse, get
runs, empty
16

---

Published as a conference paper at ICLR 2020
E ADDITIONAL QUALITATIVE RESULTS FOR RESNETI50
Ongginal Image
Nain effects
$
8
top preditiion boko,
predictonn: wooden speon
bOSTE
prrdictions jullyial
10 prediction: tick
prediction jackfruit, jack
lop preditiion cediloow
predictonn menu
prodicionn yut
prediction: pill bottle
top predxtion: far cout
100 predicionn Nustick
Figure 6: Additional qualitative results, following Figure 4a Interactions are denoted by I, and are unordered. Overlapping interactions with overlap coefficient 0.5 merged to reduce [[I,]] per test image.
random test images in ImageNet.
17

---

Published as a conference paper at ICLR 2020
F DETECTION PERFORMANCE OF MADEX VS. BASELINES
We compare the detection performances between MADEX and baselines on identifying feature inter- actions learned by complex models, i... XGBoost (Chen & Guestrin 2016), Multilayer Perceptron (MLP), and Long Short-Term Memory Network (LSTM) (Hochreiter & Schmidhuber 1997). The baselines are Tree-Shap: method to identify interactions tree-based models like XGBoost Lund- berg et al. 2018), MLP-ACD+: a modified version of ACD (Singh et al. | Murdoch et al. 2018) to search all pairs of features in MLP to find the best interaction candidate, and LSTM-ACD+ the same MLP-ACD+ but for LSTMs. All baselines are local interpretation methods. For MADEX, we sample continuous features from a truncated normal distribution N (x, IT) centered at a spec- ified data instance x and truncated at Our MADEX experiments consist of two methods, NID and GradNID (shorthand for GradientNID).
We evaluate interaction detection performance by us- ing synthetic data where ground truth interactions are known (Hooker 2004: SSrokina al. 2008). We generate 10e3 samples of synthetic data using functions Fi - F4 (Ta- blel) with continuous features uniformly distributed between -1 -o 1. Next, we train complex models (XGBoost, MLP. and LSTM) on this data. Lastly, run MADEX and the baselines on 10 trials of 20 data instances at randomly sampled loca- tions on the synthetic function domain. Between trials, the complex models are trained with different random initializa- tion to test the stability of each interpretation method. Inter- action detection performance is computed by the average R-precision (Manning et al. 2008 of interaction rankings across the sampled data instances.
Table 9: Data generating functions with interactions
(ix)=
10 22 + 10
Fa(x)
3y + 10
Pa(x) exp([2] + ral) + >>10 24
F((x) 102 + Eist 2;
Results are shown in Table 10 MADEX (NID and GradNID) performs well compared to the baselines. On the tree-based model, MADEX can compete with the tree-specific baseline Tree-Shap, which only detects pairwise interactions. On MLP and LSTM, MADEX performs significantly better than ACD+. The performance gain is especially large in the LSTM setting. Comparing NID and GradNID, NID tends perform better in this experiment because it takes its entire sampling region into account whereas GradNID examines single data instance.
Table 10: Detection Performance in R- P-Pecission (higher the better). a = 0.6 (max: 3.2). "Tree" is XGBoost. *Does not detect higher-order interactions. +Requires an exhaustive search of all feature combinations.
Tree
MLP
LSTM
Tree-Shap
NID
GradNID
MLP.ACD+
NID
GradNID
LSTM-ACD+
NID
GradNID
(i())
0.96 0.04
0.63 0.08
1-0
1+0
0.3
1+0
1+0
1
0,3
0.6 + 0.4
0.41 0.06
1=0
0.95
0.04
0.01 # 0.02
0.99 0.02
0.95=0.08 0.04
F5(0
120
14:
1
0,3
1=0
120
0.05 0.08
1
1=0
F(())
11:0
1=0
110
"R-precision is the percentage of the top-R items in ranking that correct out of R, the number of items. 1 in these experiments.
18

---

Published as a conference paper at ICLR 2020
G HIGHER-ORDER INTERACTIONS
This section shows how often different orders of higher-order interactions are identified by GLIDER / MADEX. Figure plots the occurrence counts of global interactions detected in Autolnt for the Criteo and Avazu dataset, which correspond to the results in Figure Here we show the occurrence counts of higher-order interactions, where the exact interaction cardinality is annotated besides each data point. 3-way interactions are the most common type, followed by 4-, then 5-way interactions.
Figure plots histograms of interaction cardinalities for all interactions detected from ResNet152 and Sentiment-LS'T LSTM across 1000 random samples in their test sets. The average number of fea- tures are 66 and 18 for ResNet152 and Sentiment-LSTTV respectively. Higher-order interactions are common in both models.
600
600
count
count
400
3.3
3344334
535435
10
20
30
40
50
20
30
40
50
rank
rank
(a) Criteo
(b) Avazu
Figure 7: Occurrence counts (total: 1000) vs. rank of interactions detected from AutoInt on (a)
Criteo and (b) Avazu datasets. Each higher-order interaction is annotated with its interaction cardi- nality.
8000
8000
6000
6000
count
4000
4000
30 34 38 interaction cardinality
22
interaction cardinality
(a) ResNet152
(b) Sentiment-LSTM
Figure Histograms of interaction sizes for interactions detected (a) ResNet152 and (b) Sentiment-L.STM across 1000 random samples in respective test sets.
19


Reviews:
[{"review_id": "ryxiVxxs9H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper is focused on identifying/discovering feature interactions in blackbox models (with a focus on recommendations). Specifically the technique works by first corrupting datapoints and then using this \"local dataset\" to find interacting features via lasso-regularized multi-level perceptron approach (NID from Tsang et al). Using an expanded feature space and repeated calls to the above steps this is then used to find the most commonly occurring patterns. \n\nThe empirical section is quite interesting, with some nice mix of qualitative and quantitative results. The quantitative results in particular are quite intriguing -- across recommendation and non-recommendation tasks.\n\nOverall the paper makes for an interesting read. On the whole I lean slightly positive -- largely due to the empirical section and the quantitative gains observed by adding the feature interactions as features to the model.\n\nThat said I had quite a few concerns about the work:\n\n- In general the exposition was quite lacking in the methodological section. I had to re-read the paper a few times to be able to make out to fully understand the underlying methodology. I would make the overall pipeline very clear and explain out the MADEX pipeline very clearly. The feature expansion section (4.3) in particular was not very clear -- as to how it fit in with the rest of the pipeline and something that could do with more work.  Nomenclature like source and target model are used somewhat arbitrarily and again can be clarified.\n\n- Another concern I had with the approach is how this would work if the underlying features itself were not known. Say for example you had a text understanding model -- the tokenization and vocabulary/OOV etc .. may all be components of the blackbox. Likewise for vision models, what features are being used may vary.\n\nBy assuming (effectively perfect) information about the features used, this is no longer really a \"blackbox\" model. I would have wanted to see some deeper discussion on this topic.\n\nI also had some concerns about the scalability of dense features and their bucketization in this approach. I'm not entirely convinced the method would work as well and efficiently in such feature spaces. I would have wanted to see some more empirical evidence and understanding on this topic.\n\nIf possible it would have also been great to understand higher-order interactions and the ability of the model to find them empirically. \n\n- One thing I wasn't entirely certain of was the amount of novelty in the work since it leverages existing works like NID (Tsang et al) and LIME (Ribeiro et al) for some of the key aspects of the method. It would be nice to see some discussion on this front as well.\n\n- I also would have liked to see significance testing being performed in general across the experiments.\n\n- Another smaller concern stemmed from a lack of mention of the stopping criterion of the model. In particular I'm wondering how models were stopped further training / best checkpoint was picked. This would be good to elaborate on to make sure that the models were all compared fairly from a computational perspective.\n\n- I also had some concerns with the scalability of the approach in large features spaces but am willing to give the authors the benefit of doubt on this one based on the 3 hour number they quoted from their experiments (which indicates something that is not prohibitively slow)\n", "extra_scores": {"experience_assessment": "I have read many papers in this area.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper."}, "extra_text": {}}, {"review_id": "rklPOdey9H", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The paper proposes a method to detect which features in the input of recommender systems are interacted each other, i.e., combining them behaves useful information, and examines to feed extracted interactions directly into the recommender systems to measure effects on actual recommendation.\n\nThe interaction detector consists of 1. perturbing the input vectors,  2. training NNs to utilize its internal non-linear representation as a signal of interaction, and 3. aggregating detected interactions over training set. 1. and 2. are consisting of known methods so that the proposed method is an application of them. For 3. authors introduced a simple heuristic (Algorithm 1).\n\nAs long as I heard how the NID work to detect interaction from this paper, it is sensitive not only the true interaction between features but also set of features holding similar signals (e.g., if a hidden unit behaves as a feature A, it may be natural to aggregate all features that implies A by itself, regardless of the meaning of their interaction). This is not a desired case as long as the paper specified the interactions in section 3. Maybe it requires more detailed explanation about how good applying NID for this task is.\n\nExperiments are conducted to show the behavior of the interaction detector and actual improvement of utilizing extracted interactions as additional features. Experiments look less informative for comparing the proposed method with other existing methods for similar motivations (e.g., some methods introduced in related work) since there is only a trivial baseline by the original LIME's method.\n\nIn the case-study of Figure 4(b), I thought that the proposed method is a bit biased for frequent but meaningless features, because it detected \"I\" or \"a\" that intuitively occur with any labels. It is maybe because the Algorithm 1. that simply aggregates all detected interactions regardless of their actual importance. For further improvement, it may be necessary to introduce some weighting strategy to detected interactions.\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_checking_correctness_of_experiments": "I assessed the sensibility of the experiments.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I assessed the sensibility of the derivations and theory."}, "extra_text": {}}, {"review_id": "HklcwLdRFr", "reviewer": "AnonReviewer3", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": null, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "This paper proposed a model for extracting global feature interactions from the source model which was later being encoded in the target model to enhance its prediction performance.\n\nStrong points:\n1. The paper laid out the necessary background knowledge very clear even for the audiences outside this area.\n\n2. The paper performed reasonable amount of experiments to compare the proposed model with various baseline models with various data sets, which are strong enough to support the claims made in this paper.\n\nComments:\n1. The theoretical innovation of this paper is trivial. The local detection model for feature interactions, i.e. MADEX is simply employing the previous work, LIME and NID. Its global extension, i.e. GLIDER and the proposed encoding method are straightforward. \n\n2. The descriptions on the feature dimensions of the original data and the generated binary representation data x’ are rather confusing and inconsistent throughout the paper. If I understand it correctly, the data sample from the original feature space x \\in R^p, and a  binary representation x’ \\in R^d, where d <= p. However, when d first appears in the first paragraph of section 3, it is defined as the dimension of the original feature space and later in that paragraph the dimension became p in “f(.): R^p -> R”. There is no clarification on the differences between d and p till section 4.1 where it states “x \\in R^p ”. However, the dimension of the data instance from the original feature space changed to d again in section 4.2 where it states “x =[x_1, x_2, …, x_d]”. \n\n3. “What’s more, the same approach to interpreting interactions can provide new insights into domains even beyond recommendation.” should be “What’s more, the same approach to interpret interactions can provide new insights into domains even beyond the recommendation.”\n\n4. “An interaction, I, is a subset of all input features...”, but according to the following sections, I is the indices of a feature subset instead of the features themselves.\n \n5. “...by requiring the same cross feature ID to occur more that T times in a batch of samples,...”, ‘that’ should be ‘than’.\n\nOverall, although there is no significant theoretical innovation, it is a decent application paper.\n", "extra_scores": {"experience_assessment": "I do not know much about this area.", "review_assessment:_thoroughness_in_paper_reading": "I read the paper at least twice and used my best judgement in assessing the paper.", "review_assessment:_checking_correctness_of_experiments": "I carefully checked the experiments.", "review_assessment:_checking_correctness_of_derivations_and_theory": "I carefully checked the derivations and theory."}, "extra_text": {}}]

Author rebuttal:

Meta-review: The paper extracts feature interactions in recommender systems and studies the effect of these interactions on the recommendations. While the focus is on recommender systems the authors claim that the ideas can be generalised to other domains also. 

All reviewers found the empirical results and analysis thereof to be very interesting and useful. This paper saw a healthy discussion between the authors and reviewers and all reviewers agreed that this paper makes a useful contribution. I recommend that the authors address all the concerns of the reviewers in the final version of the paper. 
Decision: Accept (Poster)