RRL: A Scalable Classifier for Interpretable Rule-Based Representation Learning
Authors: Zhuo Wang
Wei Zhang
Ning Liu
Jianyong Wang
Venue:  (2021)
Keywords: interpretable representation learning
rule-based model
scalability

Abstract: Rule-based models, e.g., decision trees, are widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity. However, rule-based models are hard to optimize, especially on large data sets, due to their discrete parameters and structures. Ensemble methods and fuzzy/soft rules are commonly used to tackle these issues, but they sacrifice the model interpretability. In this paper, we propose a new classifier, named Rule-based Representation Learner (RRL), that automatically learns interpretable non-fuzzy rules for data representation. To train the non-differentiable RRL effectively, we project it to a continuous space and propose a novel training method, called Gradient Grafting, that can directly optimize the discrete model using gradient descent. An improved design of logical activation functions is also devised to increase the scalability of RRL and enable it to discretize the continuous features end-to-end. Exhaustive experiments on 9 small and 4 large data sets show that RRL outperforms the competitive approaches, has low complexity close to the simple decision trees, and is rational for its main technical contributions.

Under review as a conference paper at ICLR 2021
RRL: A SCALABLE CLASSIFIER FOR INTERPRETABLE RULE-BASED REPRESENTATION LEARNING
Anonymous authors
Paper under double-blind review
ABSTRACT
Rule-based models, e.g., decision trees, widely used in scenarios demanding high model interpretability for their transparent inner structures and good model expressivity. However, rule-based models are hard to optimize, especially on large data sets, due to their discrete parameters and structures. Ensemble methods and fuzzy/soft rules are commonly used to tackle these issues, but they sacrifice the model interpretability. In this paper, we propose new classifier, named Rule- based Representation Learner (RRL), that automatically learns interpretable non- fuzzy rules for data representation. To train the non non-differentiable RRL effectivel. we project it to a continuous space and propose a novel training method, called Gradient Grafting, that can directly optimize the discrete model using gradient descent. An improved design of logical activation functions also devised to increase the scalability of RRL and enable it to discretize the continuous features end-to-end. Exhaustive experiments on 9 small and 4 large data sets show that RRL outperforms the competitive approaches, has low complexity close to the simple decision trees, and is rational for its main technical contributions.
INTRODUCTION
Although Deep Neural Networks (DNNs) have achieved impressive results in various machine learn- ing tasks (Goodfellow et al., 2016), rule- based models, benefiting from their transparent inner struc- tures and good model expressivity, still play an important role in domains demanding high model interpretability, such as medicine, finance, and politics (Doshi Velez & Kim 2017). In practice, rule-based models can easily provide explanations for users to earn their trust and help protect their rights (Molnar [2099 Lipton 2016). By analyzing the learned rules, practitioners can understand the decision mechanism of models, and use their knowledge to improve or debug the models (Chu et al. 2018). Moreover, even post-hoc methods can provide interpretations for DNNs, the inter- pretations from rule-based models more faithful and specific (Murdoch et al. 2019). However, conventional rule-based models are hard to optimize, especially on large data sets, due to their dis- crete parameters and structures, which limits their application scope. To take advantage of rule-based models in more scenarios, we urgently need to improve their scalability.
Studies in recent years provide some solutions to improve conventional rule-based models in differ- ent aspects Ensemble methods and soft/fuzzy rules are proposed to improve the performance and scalability of ruse-based models but at the cost of model interpretability (Ke al. 2017 Breiman
2001 Irsoy et al.] 2012). Bayesian framework is also leveraged to more reasonably restrict and ad- just the structures of rule-based models (Letham et al. 2015 Wang al. 2017 Yang et al. 2017). However, due to the non-differentiabl model structure, they have to use methods like MCMC or Simulated Annealing, which could be time-consuming for large models. Another way improve rule-based models is to let high-performance but complex model (e.g., DNN) teach rule-based model (Frosst& & Hinton (2017): 7): Ribeiro et al. (2016)). However, to learn from the complex model, it requires soft rules, or the fidelity of the student model is not guaranteed. Wang et al. (2020) try to extract hierarchical rule sets from a tailored neural network. Although the extracted rules could behave differently from the neural network when the network is large, combined with binarized networks (Courbariaux et al., 2015), inspires us that we can search for the discrete solution of rule-based models in continuous space and leverage optimization methods like gradient descent.

---

Under review as a conference paper at ICLR 2021
In this paper, we propose novel rule-based model, named Rule-based Representation Learner (RRL) (see Figure Tay. which owns three key technical contributions:
(i) To achieve model transparency, RRL is formulated as hierarchical model, with layers sup- porting both conjunction and disjunction operations. This paves the way for automatically learning interpretable non-fuzzy rules for data representation and classification.
(ii) To facilitate training effectiveness, RRL exploits a novel gradient- discrete model training method, Gradient Grafting, that directly optimizes the discrete model and uses the gradient informa- tion at both continuous and discrete points to suit more scenarios.
(iii) To ensure data scalability, RRL utilizes improved logical activation functions to handle high- dimensional features. By further combining the improved logical activation functions with a tailored feature binarization layer, it realizes the continuous feature discretization in an end-to-end manner.
We conduct experiments on 9 small data sets and 4 large data sets to validate the advantages of our model over other representative classification models. The benefits of the model's key components are also verified by experiments.
RELATED WORK
Rule-based Models. Decision tree, rule list, and rule set are the widely used structures in rule- based models. For their discrete parameters and non non-differentiable structures, we have to train them by employing various heuristic methods (Quinlan. 1993] BBreiman| [017)) Cohen| 1995) which may not find the globally best solution or solution with close performance. Alternatively, train them with search algorithms (Wang et al., 2017 Angelino et al. 2017), which could take too much time on large data sets. In recent studies, Bayesian frameworks
leveraged to restrict and adjust model structure more reasonably (Letham et al., 2015 Wang et al. 2017: Yang et al.) 2017). Lakkaraju eet al. (016)) learn independent if-then rules with smooth local search. Using algorithmic bounds and efficient data structures, Annellno et al.] (2017) try to accelerate the learning of certifiably op- timal rule lists. However, except heuristic methods, most existing rule-based models need frequent itemsets mining and/or long-time searching, which limits their applications. Moreover, it is hard for these rule-based models get comparable performance with complex models like Random Forest.
Ensemble models like Random Forest (Breiman) (001) and Gradient Boosted Decision Trees (Chen) & Guestrin 2016 Ke et al., 2017). have better performance than the single rule-based model. How- ever, for the decision is made by hundreds of models, ensemble models are commonly not consid- ered as interpretable models (Hara Hayashi, 2016). Soft or fuzzy rules are also used improve Ishibuchi model performance (Irsoy et al. Lshibuchi & Yamamoto, 2005). but non-discrete rules are much harder to understand than discrete ones. Deep Neural Decision Tree (Yang al., (018)) is a tree model realized by neural networks with the help of soft binning function and Kronecker prod- uct. However, due to the use of Kronecker product, it is not scalable with respect to the number of features. Other studies try to teach the rule-based model by complex model, e.g.. DNN, extract rule-based models from complex models (Frosst & Hinton 2017] Ribeiro et al.) 2016 Wang et al. 2020 However, the fidelity of the student model or extracted model is not guaranteed.
Gradient-based Discrete Model Training. The gradient- based discrete model training methods mainly proposed to train binary quantized neural networks for network compression and acceleration. [Courbariaux et al. [2015] 2016) propose to use Straight-Through Estimator (STE) for binary neural network training and achieve empirical success. However, STE requires gradient information at discrete points, which limits its applications. ProxQuant (Bai et al. 2018) formulates quantized network training as a regularized learning problem and optimizes via the prox-gradient method. ProxQuant can use gradients at non-discrete points, but cannot directly optimize for the discrete model. The Random Binarization (RB) method (Wang et al., 2020) trains neural network with random binarization for its weights to ensure the discrete and continuous model behave samely. However, when the model is large, the differences between the discrete and continuous model are inevitable. Gumbel-Softmax estimator (Jang et al., 2016) generates a categorical distribution with a differentiable sample. However, we can hardly deal with a large number of variables, e.g., the weights of binary networks, using the Gumbel-Softmax estimator for it is biased estimator. Our method, i.c., Gradient Grafting, is different from all the aforementioned works in using gradient information of both discrete and continuous models in each backpropagation.
2

---

Overall Structure. based Representation Learner (RRL), denoted by F, is hierarchical model consisting of three different types layers. Each layer in RRL not only contains specific number of nodes, but also has trainable edges connected with its previous layer. Let (1 denote the (-th layer of RRL, u indicate the j-th node in the layer, and n; represent the corresponding number of nodes, E [O....,L) L}. The output of the I-th layer is a vector containing the values of all the nodes the layer. For case expression, we denote this vector by u There are only one binarization layer, i.e., 11(0), and one linear layer, i.e., 2((L), in RRL, but the number of middle Under review as a conference paper at ICLR 2021
Discrate Logical Layer Example:
Forward
Backward
Logical Layer
Lineer Layer
Loss(.)
Logical Layer
Conjunction Layer
Disiunction Layer
Connsponning Rules:
Logical Layer
Discrete Model
Continuous
Conjunction Layer
Disuunction Layer
Model
Layer
g(-)
10
1-15
Continuous Input C2 Binary Input
(a)
(b)
Figure I: (a) A Rule-based Representation Learner example. The dashed box shows an example of discrete logical layer and its corresponding rules. (b) A simplified computation graph of Gradient Grafting. Arrows with solid lines represent forward pass while arrows with dashed lines represent backpropagation. The green arrow denotes the grafted gradient, a copy of the gradient represented by the red arrow. After grafting, there exists backward path from loss function to the parameter O.
3 RULE-BASED REPRESENTATION LEARNER
Notation Description. Let ID {(X1,y1)..... (X....,,... yN)} denote the training data set with N instances, where X; is the observed feature vector of the i-th instance with the j-th entry Xijj. and y; is the corresponding class label, E {1...., NJ. Each feature value can be either discrete or continuous. All the classes take discrete values, and the number of class labels is denoted by M. We use one-hot encoding to represent all discrete features as binary features. Let C, E R and B; 13 denote the continuous feature vector and the binary feature vector of the i-th instance respectively. Therefore, X; C2 + B,, where 4 represents the operator that concatenates two vectors. Throughout this paper, use (True) and 0 (False) represent the two states of Boolean variable. Thus each dimension of a binary feature vector corresponds to Boolean variable.
layers, i.e., logical layers, can be flexibly adjusted according to the specific situation. The logical layers mainly aim learn the non-linear part of data while the linear layer aims to learn the linear part. One example of RRL is shown in Figure la
When we input the i-th instance to RRL, the binarization layer will first binarize the continuous feature vector C, into new binary vector C. Then, C, and B, are concatenated together ()) and inputted to the first logical layer. The logical layers are designed to automatically learn data representations using logical rules, and the stacked logical layers can learn rules more complex forms. After going through all the logical layers, the output of the last logical layer can be considered as the new feature vector to represent the instance, wherein each feature corresponds to one rule formulated by the original features. As such, the whole RRL is composed of feature learner and linear classifier (lincar layer). Moreover, the skip connections in RRL can skip unnecessary logical layers. In what follows, the details of these components will be elaborated.
3.1 LOGICAL LAYER
Considering the binarization layer needs the help of its following logical layer to binarize features in an end-to-end way, we introduce logical layers first. As mentioned above. logical layers can learn data representations using logical rules automatically. To achieve this, logical layers are designed to
3

---

Under review as a conference paper at ICLR 2021
have discrete version and continuous version. The discrete version is used for training, testing and interpretation while the continuous version is only used for training. is worth noting that the discrete RRL indicates the parameter weights of logical layers take discrete values (i.e., 0 or 1).
Discrete Version. One logical layer consists of one conjunction layer and one disjunction layer. In discrete version, let R( and s()) denote the conjunction and disjunction layer of 2 (1 E 1)) respectively. We denote the i-th node in R by ( and the i-th node in S(i) {1, .L L by . Specifically speaking, node r, corresponds to the conjunction of nodes in the previous layer connected with r!",, while node S; connected with s. Formally, the two types of nodes are defined as follows:
(t) corresponds to the disjunction of nodes in previous layer
(-1)
V
(1-1)
S
where W(.0) denote the adjaceney matrix of the conjunction layer R( and the previous layer 40-11,, and i.j (0, w(( = indicates there exists edge connecting r!!) to up ( otherwise w.l,.0) 0. Similarly, W is the adjacency matrix of the disjunction layer (()) and (1-11) Similar to neural networks, we regard these adjacency matrices as the weight matrices of logical layers. (1) = r(t) @ s(), where r()) and s(1) are the outputs of R()) and S() respectively.
The function of the logical layer is similar to the notion "level" in Wang al. 2020) However, one level in that work, which actually consists of two layers, can only represent rules in Disjunctive Normal Form (DNF),, while two logical layers can represent rules in DNF and Conjunctive Normal Form (CNF) the same time. Connecting nodes in R()) with nodes in ((t1)), we get rules in CNF, while connecting nodes in {()) with nodes in R1-1), we get rules in DNF. The flexibility of logical layer is quite important. For instance, the length of CNF rule (a) V a2) A --- A (a2n-1 V a2n) is 2n, but the length of its corresponding DNF rule (a) A a3 A (2n-1) V... (a2 A aas A a2n)
is n n-2", which means layers that only represent DNF can hardly learn this CNF rule.
Continuous Version. Although the discrete logical layers have good interpretability, they are hard to train for their discrete parameters and non non-differentiable structures. Inspired by the training process of binary neural networks that scarches the discrete solution in continuous space, we extend the discrete logical layer to a continuous version. The continuous version differentiable, and when discretize the parameters of continuous logical layer, we can obtain its corresponding discrete logical layer.
Let 0 denote the continuous logical layer, and R() and 8()) denote the continuous conjunction and disjunction layer respectively, / E {1, 21, L- - 1}. Let (1.0)) and (1.11) denote the weight matrices of (!) and $(1) respectively. vi:((.0) -i.(.1) E [0, 1]. To make the whole Equation differentiable, we leverage the logical activation functions proposed by Payani & Fekri (019);
a Conj(h, W3) II Fe Wo,)). Disj(h, Wi) j=1
m
II (1 Fa W(,)).
where F. w) = 1 w(1 h) and Fa(h, w) = h w. In Equation vectors, then Conj(h, W.) = Aw.,= hy and Disj(h, W;) Vw.,=1 hy- Fe(h, w) and Fa(h, w) decide how much h, would affect the operation according to W.y. We, = 0, h, would have no effect on the operation.
2 if h and W; are both binary
After using continuous weights and logical activation functions, the nodes in R0 and $ defined as follows:
Conj(ú (n-()))), s(" Disj(úú .W!!..)
Now the whole logical layer is differentiable and can be trained by gradient descent. However, above logical activation functions suffer from the serious vanishing gradient problem. The main reason can be found by analyzing the partial derivative of each node w.r.t. its directly connected weights and
4

---

Under review as a conference paper at ICLR 2021
w.r.t its directly connected nodes as follows:
on! = (n) 11-1) - 1) II A
0r.00 0ú (1-1 = W.7..0)
Fa Wilkh) (4) Due to i, (1-1) and wi(k)) are in the range [0, 1], the values of Fc(-) in Equation 4 are in the range [0, as well. If n is large and most of the values of Fe(+) are not 1, then the derivative is close to 0 because of the multiplications (See Appendix A for the analysis of s! Wang et al. (2020) try to use weight initialization to make Fc(+) close T at the beginning. However, when dealing with hundreds of features, the vanishing gradient problem is still inevitable.
Fa(u (111 Will))
1)
ow 1.
i,k
kØj
We found that using the multiplications simulate the logical operations in Equation 2 is the main reason for vanishing gradients and propose improved logical activation functions. One straightforward thought is to convert multiplications into additions using logarithm, e.g., log (.,. W.))) = شÿ=1 Log(F.(h,: W.,)). However, after taking the logarithm, the logi- cal activation functions in Equation 2 cannot keep the characteristics of logical operations any more, and the ranges of C-nj(.) and Disj(-) are not (0, 1]. To deal with this problem, we need a projection function to fix it. Apparently, the inverse function of log(z), i.e., e", is not suitable.
(ii) For the projection function g. three conditions must be satisfied: (i) g(0) e e² lim ((x) = lim e = (iii) limz---o ø(ž) 0. Condition (i) and (ii) aim to keep the range and tendency of logical activation functions. Condition (iii) aims to lower the speed of approaching zero when x - -oo. In this work, we choose g(z) - as the projection function, and the improvement of logical activation functions be summarized as the function P(v)- = -1+lor(o) The improved conjunction function Conj+ and disjunction function Disj are given by:
Conj+ W.) P( II (To Wes)+
Disj (h, W.) 1 P
II (1 Fa-,, Wa,) +e)),
(5)
where e is small constant, e.g., 10-10 The improved logical activation functions can avoid the vanishing gradient problem in most scenarios and are much more scalable than the originals. More- dP(c)
over, considering that dF(rr [e(e), when in Equation is extremely large, do may be very
de
close to 0 due to Ip2 (v). One trick to deal with it is replacing (*()) with F(F*(e1) for P can lower the
speed of approaching while keeping the value range and tendency.
3.2 BINARIZATION LAYER
Binarization layer is mainly used divide the continuous feature values into several bins. By combining one binarization layer and one logical layer, we can automatically choose the appropriate bins for feature discretization (binarization), i.e., binarizing features in end-to-end way.
For the j-th continuous feature, there are k lower bounds (Cy. Lj,s) and k upper bounds .H. H, All these bounds are randomly selected (e.g., from uniform distribution) in the value range of the j-th continuous feature, and these bounds are not trainable. When inputting one continuous feature vector c, the binarization layer will check if c, satisfies the bounds and get the following binary vector:
Q = =[((c,- - -y,1),
- C,n)), ((H,, - -c), c.)...., (HH,, - c,)l.
(6)
where q(x) = 1x>0- If we input the i-th instance, i.e., c = C, then C, = QI o Q........ Qm and (0) C, @ Bi. After inputting n(() to the logical layer (111,, the edge connections between (11) and () indicate the choice of bounds (bins). For example, if rr!) is connected to the nodes corresponding to q(c, - L,))) and 2 -C,), then ((()) contains the bin (L,,1 < (c) < H2,2). If we replace r:
get (C,,1 < (c, H2,2). It should be noted that, in practice, if C,, 2 H.s.. then (1) = 0, and if C511 < H2 then s = 1. When using the continuous version, the weights of logical layers are trainable, which means can choose bounds
(1) in the example, (1) with s;
5

---

Under review as a conference paper at ICLR 2021
in end-to-end way. For the number of bounds is 2/ times of features, which could be large, only logical layers with improved logical activation functions are capable of choosing the bounds.
3.3 GRADIENT GRAFTING
Although RRI can be differentiable with the continuous logical layers, it is challenging to search for a discrete solution a continuous space. To tackle this problem, one commonly used method is ttraightThhoouhh Estimator (STE) (Courbariaux et al. (2016)). The STE method needs gradients at discrete points to update the parameters. However, the gradients of RRL at discrete points have no useful information in most cases (See Appendix BB) Therefore STE is not suitable for RRL. Other methods like ProxQuant (Bai et al. (2018) and Random Binarization (Wang et al. 2020) cannot directly optimize for the discrete model and be scalable at the same time.
Inspired by plant grafting, we propose a new training method, called Gradient Grafting, that can effectively train RRI.. In stem grafting, one plant is selected for its roots, i.e., rootstock, and the other plant is selected for its stems, i.e., scion. By grafting, we obtain a new plant with the advantages of both two plants. In Gradient Grafting, the gradient of the loss function w.r.t. the output of discrete model is the scion, and the gradient of the output of continuous model w.r.t. the parameters of continuous model is the rootstock. Specifically, let denote the parameter vector and 6e denote the parameter vector at step t. (xx = =æ>u.s is the binarization function that binarizes each dimension of æ with 0.5 as the threshold. Let and denote the output of the continuous model IF and discrete model F respectively, then y F( x). y F(q(O"), X). The parameters update with Gradient Grafting is formulated by:
a++1 = 0² aT(y) ay 8y 000
where v) is the learning rate and L(-) is the loss function. One simplified computation graph of Gradient Grafting is shown in Figure Ib for intuitive understanding.
Gradient Grafting can directly optimize the loss of discrete models and use the gradient information at both continuous and discrete points, which overcomes the problems occurring in RRL training when using other gradient- based discrete model training methods.
3.4 MODEL INTERPRETATION
After training with Gradient Grafting, the discrete RRL can be used for testing and interpretation. RRL is easy to interpret for we can simply consider it as feature learner and linear classifier. The binarization layer and logical layers are the feature learner, and they use logical rules to build and describe the new features. The linear classifier, i.c., the linear layer, makes decisions based on the new features. We can first find the important new features by the weights of the linear layer, then understand each new feature by analyzing its corresponding rules. The L1/L2 regularization can be used during training to search for an RRL. with shorter rules. The dead nodes detection and redundant rules elimination proposed by Wang et al. (2020) can also be used for better interpretabilityy
4 EXPERIMENTS
In this section, conduct experiments to evaluate the proposed model and answer the follow- ing questions: (i) How is the classification performance of RRL. compared to the competitive ap- proaches? (ii) How is the model complexity of RRL? (iii) How is the convergence of Gradient Grafting compared to other gradient-base discrete model training methods?
4.1 DATASET DESCRIPTION AND EXPERIMENTAL SETTINGS
We took 9 small and 4 large public datasets conduct our experiments. all of which are often used to test classification performance and model interpretability (Dua & Graff 2017 Xiao et al. 2017 Anguita et al. 2013. Moro et al. 2016). Appendix C summarizes the statisties of these 13 datasets. Together they show the data diversity, ranging from 178 to 102944 instances, from to 26 classes, and from 4 to 471 14 original features.

---

Under review as a conference paper at ICLR 2021
Table 5-fold cross validated Fl score of comparing models on 9 small and 4 large datasets.
Dataset
RRL
C4.5
CART
SBRI.
CRS
1.R
SVM
PLNNN(MLP))
GBDT, 2010
RF 110
RF 1000
adult
80.72
75.40
74.77
79.88
80.95
78.43
63.63
73.55
80.36
77.48
78.83
bank-marketing
76.29
71.24
70.21
72.67
73.34
69.81
66.78
72.40
75.28
69.89
72.01
banknote
100.0
98.45
97.85
94.44
94,93
98.82
100.0
100.0
99.48
99.11
99.19
chess
78.83
79.90
79.15
26.
80.21
33.06
36.83
77.85
71.41
66,38
74.25
connect-4
71.23
61.66
61.24
48.54
65.88
50.17
64.55
62.72
letRecog
96.15
88.20
87.62
64.12
84.96
72.05
74.90
92.34
96.51
93.61
96.15
magic04
86.33
80.31
80.05
82.52
80.87
75.72
75.64
83,07
86.67
84,90
86.48
tic inc---ee toe
99.77
91.70
94.21
98. .9.
99.77
98.12
98.07
98.26
99.19
94.85
98.37
wine
98.23
95.48
94,39
95.84
97.78
95.16
96.05
76.07
98.44
96.90
98.31
acrivity
94.24
93.35
5.05
98.67
97. 97.80
dota
60.12
52.08
51.91
34,83
56.31
59.34
57.76
59,46
58.81
54.04
57.16
facehook
90.27
80.76
81.50
31.16
11.38
88.62
87.20
89.43
85.51
82.88
86.85
fashion
89:01
80.49
79.61
47.38
66.92
84.53
84.46
89.36
89.91
85.76
88.05
AvgRank
2.15
7.62
8,38
8.54
5.85
7,23
7.15
4.69
2.92
6.54
4.54
magic04
dota2
facebook
87.5
60
90
900
85.0
80
82.5
F1 Score
55-
70
CANTrO 0n
Colitio 010
80.0
50
60
77.5
GARTIO 63)
CRS
45
CAATOD 031
CRS
50-
CANTIO
CAR
75.0
CARTIE 81)
R00e-101 1010
03)
CRS
72.5
CARTIO 2011
CANTTO-COLI
40
2007
20
30
70.0
35
9
8
10
12
12
16
log(ttedges)
legg#)
log( (og(#edges)
Figure 2: Scatter plot of Fl score against log(dedges) for datasets for all folds.
We adopt the Fl score (Macro) as the classification performance metric since some of the data sets are imbalanced, i.e., the numbers of different classes are quite different. We adopt 5-fold cross- validation to evaluate the classification performance more fairly. When parameter tuning is required, 95% of the training set is used for training and 5% for validation. Considering that reused structures exist in rule-based models, e.g., one branch in Decision Tree can correspond to several rules, we use the total number of edges instead of the total length of all rules as the metric of model complexity for rule-based models.
4.2 CLASSIFICATION PERFORMANCE
We compare the classification Fl score (Macro) of RRL, C4.5 (Quinlan 1993), CART (Breiman)
2017), Scalable Bayesian Rule Lists (SBRL) (Yang et al.
2017). Concept Rule Sets (CRS) (Wang
et al. 2020). Logistic Regression (LR) KKeeinaum et al. 2002), Piecewise Linear Neural Network
(PLNN) (Chu et al.] 2018). Support Vector Machines (VM) (Scholkopf & Smola (001) with lin- ear, RBF or Ploy kernel, Gradient Boosted Decision Tree (Ke et al. (017) with 100 estimators (GBDT==100). and Random Forest (Breiman 2001) with 10 estimators (RFx=10) and 100 estima- tors (RF C4.5, CART, SBRL and CRS are all rule-based models, and LR is linear model. These five models are considered as interpretable models. PLNN is a Multilayer Perceptron (MLP) that adopts piecewise linear activation functions, e.g., ReLU (Nair & Hinton 2010). PLNN, SVM, GBDT and Random Forest are considered as complex models.
The results are shown in Table [ and the first data sets are small data sets while the last are large data sets. We can observe that RRL performs well on almost all the data sets and gets the best results on 6 data sets. RRL. outperforms other interpretable models and only two complex models, i.e., GBDT and RF 100= have comparable results. Compared RRL with LR and other rule- based models, we can see RRL can fit both linear and non-linear data well. CRS performs well on small data sets but fails on large datasets due to the limitation of its logical activation functions and training method. Good results on both small and large data sets verify RRL has good scalability. Moreover, SBRL and CRS do not perform well continuous feature data sets like letRecog and magic04 for they need preprocessing to discretize continuous features, which may bring bias to the data sets. On the contrary, RRL overcomes this problem by discretizing features end-to-end.
7

---

Under review as a conference paper at ICLR 2021
connect-4
STE activity
facebook
STE
Cre3Crateggee
571 60
loss (log scale)
0.0
loss (log scale
loss (log scale)
L. 77
-1.0
-1.5
-2.0
6
-2.
0
10
20
30
40 (1e4)
50
60
4
10
2
3
iter.
Iter. (1e4)
iter.
(1e4)
Figure 3: Training loss of compared discrete model training methods and Gradient Grafting with or without improved logical activation functions on 3 data sets.
4.3 MODEL COMPLEXITY
Interpretable models need to keep low model complexity while ensuring high accuracy. To show the relationships between aceuracy and model complexity, we draw scatter plots of F1 score against log(Hedges) for rule-based models in Figure 2 (see Appendix for results on other data sets). The value in CART(0.03) denotes the complexity parameter used for Minimal Cost-Complexity Pruning (Breiman 2017), and a higher value corresponds to a simpler tree. We can observe that RRL is much simpler than random forests, and its complexity close CRS and decision trees without pruning. Meanwhile, RRL has higher Fl scores than models with close complexity. It also indicates that RRL can make better use of rules than models using heuristic and ensemble methods.
4.4 ABLATION STUDY
Training Method for Discrete Model. To show the effectiveness of Gradient Grafting for training RRL, we compare it with other representative gradient-based discrete model training methods, i.c.,
STE (Courbariaux et al., 2015 2016), ProxQuant (Bai et al.) 2018) and RB (Wang et al.) 2020). by training RRL with the same structure. Hyperparameters are set to be the same for each method except exclusive hyperparameters, e.g., random binarization rate for RB, are fine-tuned. In Figure 3. we can see that the convergence of Gradient Grafting is faster and stabler than other methods on all data sets. As we mentioned in Section B.3] RRL has little useful gradient information discrete points, thus STE cannot converge. Due to the difference between discrete and continuous RRL. ProxQuant and RB cannot converge well as well.
Improved Logical Activation Functions. We also compare RRL trained by Gradient Grafting with or without improved logical activation functions. The results are also shown in Figure 3 and GradGrafting(NN) represents RRL. using original logical activation functions instead of improved logical activation functions. We can observe that the original activation functions work well on small data sets but fail on the large data set activity while the improved activation functions work well on all data sets, which means the improved logical activation functions make RRL more scalable. should be noted that GradGrafting(NI) works well the large data set facebook, the reason is facebook is a very sparse data set, and the number of in each binary feature vector is less than 30 (See Appendix A for detailed analyses).
4.5
CASE STUDY
We show how the learned rules look like by case studies. In Figure 4 we show the learned rules, with high weights, of RRL trained on bank-marketing data set (See Appendix [ for the distribution of weights and see Appendix D for fashion data set). These rules are used to predict if the client will subscribe a term deposit by telesales. Different types of features are marked in different colors, e.g., purple for previous behaviours of the bank. We can clearly see that middle-aged married persons with low balance are more likely to subscribe a deposit, and the previous behaviour of the bank would also affect the client. Then the bank can change their strategies according to these rules.
8

---

Under review as a conference paper at ICLR 2021
Weight
Rule
0.966
-122.5 < balance < 2606.1 A marital married A campaign < 5 poutcome success previous >
0.753
telephone day previous 36 A balance < 7016.7 A marital married campaign < 5A pdays 104 A poutcome success
balance 7016.7
0.733
age 60 balance -122.5 campaign 7 Aay > 22 pdays
previous A 6689. < balance < 5813.7 A campaign 6 pdays > 304 A previous <
0.728
Figure 4: Logical rules obtained from RRL trained on the bank-manketing data set.
CONCLUSION
We propose a new scalable classifier, named Rule-based Representation Learner (RRL), that can au- tomatically learn interpretable rules for data representation and classification. For the particularity of RRI., we propose a new gradient-based discrete model training method, i.e., Gradient Grafting, that directly optimizes the discrete model. We also propose improved design of logical activation functions to increase the scalability of RRL and make RRL capable of discretizing the continu- features end-to-end. Our experimental results show that RRL enjoys both high classification performance and low model complexity on data sets with different scales.
REFERENCES
Elaine Angelino, Nicholas Larus-Stone, Daniel Alabi, Margo Seltzer, and Cynthia Rudin. Learning certifiably optimal rule lists for categorical data. The Journal of Machine Learning Research, 18 (1):8753-888 8830,,8800222 2017.
Davide Anguita, Alessandro Ghio, Luca Oneto, Xavier Parra, and Jorge Luis Reyes-Ortiz. A public domain dataset for human activity recognition using smartphones. In Esann, 2013.
Yu Bai, Yu-Xiang Wang, and Edo Liberty. Proxquant: Quantized neural networks via proximal operators. arXiv preprint arXiv: arXiv: 1810.00861, 2018.
Leo Breiman. Random forests. Machine learning, 45(1):5-32, 2001.
Leo Breiman. Classification and regression trees. Routledge, 2017.
Tianqi Chen and Carlos Guestrin. Xgboost: A scalable tree boosting system. In Proceedings of the 22nd acm sigkdd international conference on knowledge discovery and data mining, pp. 785-794, 2016.
Lingyang Chu, Xia Hu, Juhua Hu, Lanjun Wang, and Jian Pei. Exact and consistent interpretation for piecewise linear neural networks: A closed form solution. In SIGKDD, pp. 1244-1253. ACM, 2018.
William W Cohen. Fast effective nule induction. In MLP, pp. 115-123. Elsevier, 1995.
Matthieu Courbariaux, Yoshua Bengio, and
David. Binaryconnect: Training deep neural networks with binary weights during propagations. In NeurIPS, pp. 3123-3131, 2015.
Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El- Yaniv, and Yoshua Bengio. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1r-1. arXiv preprint arXiv: 0o0..0002222220 2016.
Finale Doshi-Velez and Been Kim. Towards rigorous science of interpretable machine learning. arXiv preprint arXiv: 086008008, 2017.
Dheeru Dua and Casey Graff. UCI machine learning repository, 2017. URL http::///aaactttee ics uci auci.edumm
Nicholas Frosst and Geoffrey Hinton. Distilling neural network into soft decision tree. arXiv preprint /r 1.09784, 2017.
Ian Goodfellow, Yoshua Bengio, Aaron Courville, and Yoshua Bengio. Deep learning, volume 1. MIT Press, 2016.

---

Under review as a conference paper at ICLR 2021
Satoshi Hara and Kohei Hayashi. arXiv:1606.05300 039.00 2016.
Making tree ensembles interpretable.
arXiv preprint
Ozan Insoy, Olcay Taner Yildız, and Ethem Alpaydın. Soft decision trees. In Proceedings of the 21st International Conference on Pattern Recognition Denpriiioon(CR222122222 (ICPR2012), pp. 1819-1822. IEEE, 2012.
Hisao Ishibuchi and Takashi Yamamoto. Rule weight specification in fuzzy rule-based classification systems. IEEE transactions on fuzzy systems, 13(4):428 435, 2005.
Eric Jang. Shixiang Gu, and Ben Poole. Categorical reparameterization with gumbel-softmax, arXiv preprint arXiv:1611.0114 2016.
Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang. Wei Chen, Weidong Ma, Qiwei Ye, and Tie- Yan Liu. Lightgbm: A highly efficient gradient boosting decision tree. In Advances in Neural Information Processing Systems, pp. 3146-3154, 2017.
David G Kleinbaum, K Dietz, M Gail, Mitchel Klein, and Mitchell Klein. Logistic regression.
Springer, 2002.
Himabindu Lakkaraju, Stephen H Bach, and Jure Leskovee. Interpretable decision sets: A joint framework for description and prediction. In SIGKDD, pp. 1675-1684. ACM, 2016.
Benjamin Letham, Cynthia Rudin, Tyler McCormick, David Madigan, et al. Interpretable classi- fiers using rules and bayesian analysis: Building better stroke prediction model. The Annals of Applied Statistics, 93)):350-1371,, 2015.
Zachary C Lipton. The mythos of model interpretability. arXiv (reprimmaarXii::00000444990 2016.
Christoph Molnar. Interpretable Machine Learning. 2019. https://caristophm..e
htt/interrretablemmlmmm
Sérgio Moro, Paulo Rita, and Bernardo Vala. Predicting social media performance metrics and eval- uation of the impact on brand building: A data mining approach. Journal of Business Research, (9(9):3341-3355 2016.
W James Murdoch, Chandan Singh, Karl Kumbier, Reza Abbasi-Asl, and Bin Yu. Interpretable machine learning: definitions, methods, and applications. arXiv preprint
2019.
Vinod Nair and Geoffrey E Hinton. Rectified linear units improve restricted boltzmann machines. In ICML, pp. 804, 2010.
Ali Payani and Faramarz Fekri. Learning algorithms via neural logic networks. arXiv preprint 19X.:11904.01554. 2019.
J. Ross Quinlan. C4.5: Programs for Machine Learning. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 1993. ISBN 1558602402.
Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should trust you?: Explaining the predictions of any classifier. In SIGKDD, pp. 1135-1144. ACM, 2016.
Bernhard Scholkopf and Alexander Smola. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2001.
Tong Wang, Cynthia Rudin, Finale Doshi Velez, Yimin Liu, Erica Klampfl, and Perry MacNeille. A bayesian framework for learning rule sets for interpretable classification. JMLR, 18(1):2357- 2393, 2017.
Zhuo Wang, Wei Zhang, Ning Liu, and Jianyong Wang. Transparent classification with multilayer logical perceptrons and random binarization. In AAAI, pp. 6331-6339, 2020.
Han Xiao, Kashif Rasul, and Roland Vollgraf. Fashion-mniss a novel image dataset for benchmark- ing machine learning algorithms, 2017.
Hongyu Yang, Cynthia Rudin, and Margo Seltzer. Scalable bayesian rule lists. In ICML, pp. 3921 3930. JMLR. org, 2017.
Yongxin Yang, Irene Garcia Morillo, and Timothy M Hospedales. Deep neural decision trees. arXiv preprint arXiv: 1806 06988, 2018
10

---

Under review as a conference paper at ICLR 2021
A VANISHING GRADIENT PROBLEM
The partial derivative of each node in S(9 w.r.t. its directly connected weights and w.r.t. its directly connected nodes given by:
as
II
Fa wi(lh))
(8)
aw....)
1.j
kデj
0
II
Fa ( 11 w(lh!))
0dj
1-11
Similar to the analysis of Equation 4 due to (1-1) and w-(L1) are in the range [0, 1], the values of (1 - Fa(-)) in Equation 8 and 9 are in the range [0, 1] well. If n 1 is large and most of the values of (1 - Fx())) are not I, then the derivative is close to because of the multiplications. Therefore, both the conjunction function and the disjunction function suffer from vanishing gradient problem.
If the large data set is very sparse and the number of in each binary feature vector (for RRL the binary feature vector is (0))) is less than about one hundred, there will be no vanishing gradient problem for nodes in 5(1), The reason is when the number of 1 in each feature vector is less than about one hundred, in Equation 8 and 9 most of the values of (1 Fa(-)) are 1, and only less than one hundred values of (1 Fa(.)) are not 1, then the result of the multiplication is not very close O. The facebook data set is an example of this case. However, the number of in each binary feature vector is more than about one hundred, the vanishing gradient problem comes again.
B GRADIENTS AT DISCRETE POINTS
The gradients of RRL with original logical activation functions at discrete points can be obtained by
Equation 4 8 and 9 Take Equation 8 as example, discrete points mean all the weights of logical
layers are 0 or 1, which also means the values of all the nodes in (7()) are 0 or 1, E {0, 1, 1.-
Hence, in Equation 8 új (1-1) (1-Fx(-) E {0, J. and the whole equation is actually multiplications
of several 0 and several 1. Only when (!-1)) and (1- - Fa(-)) are all 1, the derivative in Equation g is I, otherwise, the derivative is O. Therefore, the gradients at discrete points have useful information in most cases. The analyses of Equation 4 and are similar.
To analyze the gradients of RRL with improved logical activation functions at discrete points, we first calculate the partial derivative of each node w.r.t. its directly connected weights and w.r.t. its directly connected nodes:
(F/00)²
(1)
(1-1)
-1)
(10)
(vr((.0) ij
Fa(u!
(1-))
+
(i!!))
(
v7-(10)
(11)
an
(1-1)
Fc(u (1-1)
vr((.0) +c
as(")
(1 $ )()
0. (-11
(12)
(1.1)
1 1 Fa(u 11-1) w((..)) +e
0s
(1 (1-5!") s(4)
(1.1 ij
(13)
0ú
(1-1)
1 Fa (11) w (1.1) +
Take Equation 10 for example, when all the weights of logical layers are or 1, the r!") F.(.) + and (u,) (1-1) 1) are all very close to 0 or well. For the initialized weights randomly selected, F!(") is close to 0 in most cases. Hence, the derivative in Equation i0 close to 0 in most cases, and the analyses of Equation [I] 12 and are similar. Therefore, the gradients at discrete points have little useful information.
11

---

Under review as a conference paper at ICLR 2021
C DATA SETS PROPERTIES
Table Soolle/smmarzzes summarizes the statistics of 13 datasets that we used in the experiments. The first 9 data sets are small data sets while the last 4 large data sets. Discrete continuous feature type indicates features in that data set are all discrete all continuous. The mixed feature type indicates the data set has both discrete and continuous features. The density is the averaged ratio the number in each binary feature vector after one-hot encoding.
Table
Data sets properties.
Dataset
#instances
#classes
#features
feature type
density
adult
32561
2
mixed
bank-markettn
45211
2
16
mixed
banknote
1372
2
4
continuous
chess
28056
18
6
discrete
0.150
connect-4
67557
3
42
discrete
0.333
letRecog
20000
26
16
continuous
magic04
19020
2
10
continuous
tic-tac-toe
958
2
9
discrete
0.333
wine
178
3
13
continuous
activity
10299
6
561
continuous
dota2
102944
116
discrete
0.087
facebook
22470
4
4714
discrete
0.003
fashion 70000
4
784
continuous
D COMPUTATION TIME
The computation time of RRL is similar to neural networks like Multilayer Perceptrons (MLP) for their computations are quite similar. The training time of RRL on all the datasets (400 epochs on the small datasets and 100 epochs the large datasets) with GeForce RTX 2080 Ti is shown in Table 3 We can see that the training time of RRL is acceptable on all the datasets, which also verifies the good scalability of RRL.
Table 3: Training time of RRL on 9 small and 4 large datasets.
Dataset
adult
hank-marketing
banknote
chess
connect-4
letRecog
magic04
Time
1h22m55s
Ih0m49s
7m45s
29m40s
2h20m41s
2h16m24s
3h22m37s
Dataset
tic-tac-toe
wine
activity
dota
facebook
fashion
Time
1m3s
16s
1h2m24s
1h58m42s
2h27m23s
7h32m52s
E
CASE STUDY
Although RRL is not designed for image classification tasks, due to its high scalability, it can still provide intuition by visualizations. Take fashion dataset for example, for each class, we combine the first ten rules, ordered by linear layer weights, for feature (pixel) visualization. In Figure [ Black/white pixel indicates the combined rule asks for a color close to black/white here in the original input image, and the grey pixel means requirement in the rule. According these figures, can see how RRL classifies the images, e.g., distinguish T-shirt from Pullover by sleeves.
Tishirstop
Pullover
Dress
Sheakee
Aniiee boct
Figure 5: Decision mode for the fashion data set summarized from rules of RRL,
12

---

Under review as a conference paper at ICLR 2021
F DISTRIBUTION OF WEIGHTS IN THE LINEAR LAYER
Table 4: The distribution of weights in the linear layer obtained from RRL trained on the bank- marketing data set.
ahs(weight) [0, 0.1)
10.1, 0.2)
[0.2, 0.3)
[0.3, 0.4)
0.5)
[0.5, 0.6)
[0.6, 0.7)
[0.7. 0.8)
[0.8, 0.9)
[0.9, 1.0)
probability 0.0781
0.1875
0.3125
0.2422
0.0859
0.0469
0.0078
0.0313
0.0000
0.0078
G MODEL COMPLEXITY
Figureloshowee shows the scatter plots Fl score against log(#Hedges) for rule-based models trained on the other 10 data sets. The value in CART(0.03) denotes the complexity parameter used for Minimal Cost-Complexity Pruning (Breiman 2017), and higher value corresponds these 10 data sets, we can still observe that RRL is much simpler than random forests, and its complexity is close CRS and decision scores than models with close complexity in most cases.
simpler tree. On
without pruning. Meanwhile, RRL has higher Fl
adult
bank-marketing
banknote
80
75
100
75-
70
Score
70-
Fl Score
Score
CARTI 01)
65
canma
CA
CARTI rest
conmoe ECa)
CARTED noo
covmi 0e3)
92
55-
DWS
55
90-
CRS
50-
Mitentti
50
Mile131
88
86
8
14
10
log( (eddge)
log( # eedge)
log (# edges)
chess
connect-4
letRecog
80
Score
60
CARTI Eb)
Score
60
Score
CANTIS CATTTS oll
50
CVTT7 6a)
CARTTA 631
50
carmia 6031
d00
40
30
CATT/2
CARTiO
CARTIS
20
ChS
ChS
CTS
30
MIL
0:
2
4
6
8
10
12
14
6
8
10
12
14
2
6
10
12
log(# # #edges)
log #edgs))
log (# edges)
tic-tac-toe
wine
100
100-
v9 v
95-
F1 Score
90
85
CARTICAO CANTIEOO
Score
94
CANTIO 6i)
cieria 63)
92-
CARTIE (01)
cianija
CARTIE40
ClaTTa 61
70
DRS
65
Afte-200)
ALL
86
84
AFfe-2001
MAL
4
6
10
2
6 log( #edges)
(#gges)
activity
fashion
80
F1 Score
CARTESS
F1 Score
GAN RX:
70
40
CARTISANEL
CARTESAD 4b
60
Ca4 e013
20-
RRQe-100)
50
0
4
10
6
8
10
12
14
log( @# edges)
#egg#))
Figure 6: Scatter plot of Fl score against log Hedges) for 10 datasets for all 5 folds.
13

---

Under review as a conference paper at ICLR 2021
H AVERAGE RULE LENGTH
The average length of rules in RRL trained on different datasets is shown in Table We can observe that except the facebook and fashion datasets, the average length of rules is less than 13 (most less than 7), which means understanding one rule is easy and understanding the rules one by one in the order of weights is feasible. The average length of rules of RRL trained on the facebook and fashion datasets is large for facebook and fashion are actually two unstructured datasets, e.g., the fashion dataset an image classification dataset.
Table 5: Average rule length of RRL trained on 9 small and 4 large datasets.
Dataset
adult
bank-marketing
banknote
chess
connect-4
letRecog
magic(4
Avglengthh
5.71
5.62
3.56
7.03
12.44
5.91
5.05
Dataset
tic-tac-toe
wine
activity
dota
facebook
fashion
AvgLength
3.07
2.11
6.67
4.37
38.28
34.53
14


Reviews:
[{"review_id": "zOStCiKn5xz", "reviewer": "AnonReviewer5", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "## Summary\nThe authors propose a new approach for training interpretable discrete models via gradient descent. They claim three contributions: 1) incorporation of layers implementing logical conjunction and disjunction operations; 2) the gradient grafting technique for performing gradient descent on discrete structures; and 3) a logical activation function that enables models to scale to larger training datasets. Additionally, they also claim competitive performance with comparable learning methods.\n\n## Strengths\n* The paper is well organised and easy to follow.\n* The proposed gradient grafting technique, while heuristic in nature, is novel and seems to work quite well in practice.\n* The proposed method achieves good accuracy compared to the other approaches considered in the experiments.\n\n## Weaknesses\n* There is limited novelty in the proposed model. The continuous/discrete conjunction and disjunction operations used in the logical layers are the same as those used in Wang et al. (2020), however they are arranged differently in this method.\n* The connection betweent the continuous version of the model and the discrete version is weak. One of the claimed contributions is that the discrete model is optimised directly, but this does not seem to be the case in practice---the continuous version of the model is optimised. It is also unclear how the continuous-valued weights are binarised in order to work with the discrete version of the model.\n* The results in Figure 2 indicate that typically a large number of rules are learned, yielding models that are unlikely to be interpreted reliably. The log scale on the x axis of the plots is somewhat deceptive---I find it hard to believe that models with between 64--256 terms are interpretable. I think further investigating what tradeoff exists between accuracy and interpretability is necessary before the paper can be accepted.\n\n## Other comments/questions\n* Comparing average accuracy is generally not a good idea. It is more informative to compare the average rank of each method across datasets [1].\n* Can the authors elaborate on how hyperparameters for each of the competing methods are chosen?\n* The computational graph for gradient grafting seems related (yet distinct) to the Gumbel softmax estimator---perhaps the authors could discuss this relationship further?\n* The authors could do more to discuss related work on training decision trees and rules using gradient-based optimisation. E.g., XGBoost [2], DNDT [3], and SGT [4].\n* Given that the model is trained with gradient descent, and arbitrary loss functions, could the model be evaluated on tasks other than classification?\n\n## Update\nI thank the authors for some of the updates and response that address my concerns with clarity, but my main concern related to the interpretability aspect has not been resolved. The authors suggest that one can discard some of the rules learnt by the algorithm if the model becomes too large, using the weights in the linear layer to determine which rules should be kept. This seems like a good direction to explore, but it is unclear if a few rules typically dominate the predictions, or whether one will sacrifice significant accuracy in order to gain interpretability. As stated in my original review, I would have liked to see analysis of the *tradeoff* between interpretability and accuracy of this method.\n\nA few other things:\n* I still do not understand exactly how the hyperparameters were tuned. The authors have provided a list of those that were tuned, but I still don't understand the process used for tuning them.\n* Some of the responses to other reviews have reinforced some of my concerns. In particular, it seems the distribution of weights attached to rules is not optimal for the proposed (but unevaluated) rule pruning method. Also, the authors conflate statistical significance with practical significance when replying to Reviewer 4.\n\n[1] Demšar, Janez. \"Statistical comparisons of classifiers over multiple data sets.\" Journal of Machine learning research (2006).\n[2] Chen, Tianqi, and Carlos Guestrin. \"Xgboost: A scalable tree boosting system.\" In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016).\n[3] Yang, Yongxin, Irene Garcia Morillo, and Timothy M. Hospedales. \"Deep neural decision trees.\" arXiv preprint arXiv:1806.06988 (2018).\n[4] Gouk, Henry, Bernhard Pfahringer, and Eibe Frank. \"Stochastic Gradient Trees.\" In Asian Conference on Machine Learning (2019).", "extra_scores": {}, "extra_text": {}}, {"review_id": "_Hu6bCPDGEV", "reviewer": "AnonReviewer4", "summary": "", "questions": "", "limitations": "", "rating": 5, "confidence": 4, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Authors propose a new scalable classifier, named Rule-based Representation Learner (RRL), that can automatically learn interpretable rules for data representation and classification. For the particularity of RRL, the authors propose a new gradient-based discrete model training method, i.e., Gradient Grafting, that directly optimizes the discrete model. Authors also propose an improved design of logical activation functions to increase the scalability of RRL and make RRL capable of discretizing the continuous features end-to-end. The experimental results show good performance of RRL both high classification performance and low model complexity on data sets with different scales.\n\nHowever, it is questionable whether the difference in accuracy between decision trees such as C4.5 and RRL in the experimental results is crucial. If the difference in accuracy is as small as this, the ease of interpretation of the decision tree may be superior to that of the RRL. Alternatively, the choice of experimental data may not be appropriate to demonstrate the superior performance of RRL.\n\nI also didn't understand why the fuzzy/soft rule sacrifices the model interpretability. Could it be that the fuzzy representation is intended to be closer to human subjectivity, and thus aid in intuitive understanding?\n\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "Yq6DO9S_lf", "reviewer": "AnonReviewer1", "summary": "", "questions": "", "limitations": "", "rating": 7, "confidence": 2, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "Summary:\n\nThis paper presents a new rule based  classifier called Rule based Representation Learner (RRL), that automatically learns interpretable non-fuzzy rules for data representation.  In order to train this model efficiently  the paper presents a learning algorithm that projects the discrete RRL model to a continuous space and hence optimise the model using gradient descent. Through experiments on 9 small and 4 large datasets shows that RRL improves over other methods, has low complexity and interpretable. \n\nReasons for score:\n\nI think this is a Good paper and vote for accepting.  This paper presents a solid contribution in learning rule based classifiers and hence to interpretable machine learning.  The paper is clearly written and superiority of the presented models are backed by strong experimental results.\n\nPros:\n\n1. The paper is clearly written and easy to follow.\n2.  The paper addressees an important problem of interpretable machine learning and presents a novel model which is scalable.\nI find the technique of gradient grafting particularly interesting.\n3.  Very strong experimental results and ablation studies.\n\n", "extra_scores": {}, "extra_text": {}}, {"review_id": "J0-RYnWo04H", "reviewer": "AnonReviewer2", "summary": "", "questions": "", "limitations": "", "rating": 6, "confidence": 5, "soundness": null, "presentation": null, "contribution": null, "strengths": "", "weaknesses": "", "quality": null, "clarity": null, "significance": null, "originality": null, "strengths_and_weaknesses": "", "was_revised": false, "final_justification": "", "rebuttal": "", "claims_and_evidence": "", "methods_and_evaluation_criteria": "", "theoretical_claims": "", "experimental_designs_or_analyses": "", "relation_to_broader_scientific_literature": "", "essential_references_not_discussed": "", "other_strengths_and_weaknesses": "", "other_comments_or_suggestions": "", "questions_for_authors": "", "summary_of_paper": "", "summary_of_recommendation": "", "technical_quality": "", "clarity_of_presentation": "", "potential_impact": "", "robotics_focus": "", "quality_of_limitations": "", "questions_for_rebuttal": "", "reasons_to_accept": "", "reasons_to_reject": "", "questions_to_authors": "", "main_review": "", "summary_of_the_paper": "", "summary_of_the_review": "", "strength_and_weaknesses": "", "limitations_and_societal_impact": "", "review_text": "The authors propose a classifier consisting of multiple layers. The inner layers construct rules in conjunctive normal form. The last layer is used to assign weights to the constructed rules. To train the overall model and obtain discrete solutions, the authors use a simple rounding mechanism leading to a method that they call gradient grafting. The paper concludes with a computational study, where the authors report the classification performance as well as the model complexity on a set of problems. \n\nThis is an interesting paper proposing a new method to tackle the trade-off between interpretability and accuracy. Indeed, rule-based learners are considered to be more interpretable. The authors also claim that the proposed rule-learner is also scalable. Overall, introduction of a scalable, interpretable and accurate method could have been considered as a big achievement. However, I have several questions and comments about this achievement as I list below:\n\n- Are the results given for test set? If so, what is the train-test percentage?\n\n- What are the computation times? Without these figures, it is hard to assess the scalability of the proposed method.\n\n- The numbers of edges that you report range from 50 to 1000. These values seem quite large. How does this affect the interpretability?\n\n- In Figure 4, only five clear rules are reported. What is the distribution of the weights of the remaining rules?\n\n-  How do you decide various design parameters; such as, number of layers (n_l), number of bins for feature discretization (k), number of layers (L)?\n\n- As you need a binarization layer to divide the continuous features into bins, can't we just say that the method works only with discrete features? This is how it would be presented by other rule-learning methods.\n\n- I guess Section 3.2 is superficial as it is straightforward to split a continuous feature into bins. Am I missing something here?\n\n- How do your results compare against the results obtained with other rule/tree learning methods based on (integer) linear optimization? Some names from that field are Bertsimas,  Rudin, Gunluk.\n\n- Can you guarantee that the resulting set of rules covers the entire feature space? In other words, is it possible that a test sample is not classified with the output set of rules?", "extra_scores": {}, "extra_text": {}}]

Author rebuttal:

Meta-review: This paper falls in the borderline area and there are still some concerns (for instance by AnonReviewer5 and AnonReviewer2) that deserve further treatment. Given that most ideas can only be validated in experiments (as the results are not theoretical), some points that remain are the comparison with other approaches (there are reasonable comparisons, but there are very famous contenders missing such as xgboost, ok that LightGBM is, but why not the other?), details about the tuning, the significance of results (practical and statistical is not complete/detailed enough), and the reasoning about situations with many rules and interpretability seems to be worth exploring/discussing further.
Decision: Reject